Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou

Model bake-off on your own codebase

Workflow

Have several models each review the same codebase and suggest areas to improve, then use strong models as judges to score the suggestions. In Theo's test, Grok 4.7 came just behind Astra on viability, and Fable 5.1 made fewer suggestions that still scored well.

Theo's current stance

Theo calls it a Hail Mary test, but says the results were useful.

Try it

  1. Ask each coding agent you use to go through your codebase and identify areas to focus on for improvement.
  2. Have Fable 5.1 and Astra act as judges and score each model's suggestions.
  3. Compare the scores against what each run cost before choosing a default model.

First seen

22:14
that said, I decided to have a bunch of different models go through T3 Code's codebase and identify areas that we should be focusing on for improvement.

Elon promised this one would be good...

Every mention

22:14
that said, I decided to have a bunch of different models go through T3 Code's codebase and identify areas that we should be focusing on for improvement.

Elon promised this one would be good...

23:22
was scored, I set up a set of judges with Fable 5.1 and Astra. So, even though Fable was half the judging work, it actually scored itself below Grock.

Elon promised this one would be good...

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.