Model head-to-head with blind judges
Workflow
Have the agent run every task on two models, then have a panel of judge models score results without knowing which model did which. The models can also review each other's plans. It showed Theo that Opus did more correct, deeper work while Muse did more work, faster.
Try it
- Ask your agent to run the same task on two models and set up a blind judge panel to score both.
- Have each model review the other's plan and agree on one to build from.
- Give routine work to the faster model and hard design decisions to the stronger one.
First seen
14:01
split up the work. I asked it at one point to start spinning up every task on Muse and on Opus, and then set up a judge panel to review the changes from
Every mention
14:01
split up the work. I asked it at one point to start spinning up every task on Muse and on Opus, and then set up a judge panel to review the changes from
15:36
Opus, which really surprised me. After a lot of back and forth, Opus, which is running this thread, concluded that Muse would make sense for most of the work, fixers, standard library work, fuzzing, and reviews. Opus does the hardcore pieces where wrong design is costly, like this T2 speed merge, the trait
Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.