Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou
31:46
0:00

OpenAI should be scared of this one

Tuesday, Sep 29 · 32 min · about 7 min to read

Theo-brain watched all 32 min

Sonnet 5.5 is Anthropic's cheaper model, but Theo says not to pick it as your main model. Cache reads cost the same as Opus, so in his real tasks it cost as much or more, and it needed so many tokens it finished slower. Its value is as a cheap, fast sub-agent that Opus or Fable sends research jobs to.

Skim itThe notes cover the argument; watch only the max-reasoning explainer, the codebase audit results and the game demo if you want to see them.

Tap any time to jump the video there.

Key takeaways

Don't choose Sonnet 5.5 as your main model. Theo says its strength is as a tool that Opus or Fable hands pieces of big, complex work to.

and Fable would. Its strengths come as a tool for our other models to use. We shouldn't be calling Sonnet directly. It should be used as one of many things that Opus or Fable will orchestrate when it's trying to break up real complex

The headline price is half of Opus, but cache reads (re-reading context the model has already seen, which is most of an agent's bill) cost the same as Opus. Theo says cache reads become 50%+ of Sonnet's cost, and in his real tasks it came out as expensive as Opus or more.

means is cash read becomes a more and more prominent part of your cost when you go down the model ti where it rounds out to under 5% with fable. It's closer to 20% with Opus and it's closer to like 50 plus with Sonnet 5.5 because thatI've looked at the numbers for doing like forlike work with opus versus sonnet for real code tasks and sonnet consistently comes out as if not more expensive than opus does. In fact, with

It generates tokens faster but needs far more of them, so it is slower overall. On Theo's fish game test, Sonnet took 43 minutes and Opus only 36.

150 for Sonnet 5.5. So, it is faster, but it's also not very token efficient. And result here is that when I used it for something like fish slop, it ended up taking quite a bit longer than Opus did. Sonnet took 43 minutes to finish and Opus only took 36. And when you

Where it shines is analysis. Theo had models audit a huge T3 Code pull request and plan how to split it up. Sonnet cost about half what Opus did and scored slightly better. It took about 5 minutes, while Opus took nearly twice as long.

here. The thing that came as a massive surprise to me was Sonnet's performance where it ended up being around half the price of Opus. you know what it should be and also performing slightly better than Opus did according again to mytime it took because it ended up only taking around 5 minutes to do all of this work. And like, yeah, sure, Soul was able to do it even faster, but at like half the score. Opus took almost twice as long, and Aster took almost

Sonnet is weak at visual front-end design. Its designs were significantly worse than Fable's. For good-looking pages like marketing sites, Theo still picks Fable 5.1. He finds Opus easy to steer toward good designs because it follows design instructions much better.

significantly worse. I still think Fable 5.1 is the best overall design model in particular for nicel lookinging front-end designs for your marketing site and whatnot, but I have found Opus to be relatively steerable towards good designs and it follows instructions around design much better. I've not

One skill here is new to you.

Sonnet 5.5 as a research sub-agent under Opus New

What it is
Keep Opus (or Fable) as the main agent. Let it send investigative jobs to Sonnet sub-agents (helper agents it launches itself), such as digging through the codebase for a behavior or checking a hunch about an API.
Why Theo cares
On this kind of investigative work Sonnet is really cheap and surprisingly fast. He bets a properly set-up Opus will feel much faster and get real work done for slightly less money.
How to try it
  1. Keep Opus as the model you talk to; don't select Sonnet yourself.
  2. Ask your agent to configure Opus to use Sonnet 5.5 sub-agents for codebase deep dives and for confirming hunches.
  3. Compare task time and cost before and after the change.
But if you have it as a tool, Opus calls to do things like deep dives into your codebase to find specific behaviors or characteristics or confirming a hunch it has about some thing that you found in an API or all of those types of investigatory things. It is really cheap
and surprisingly fast, too. I would bet that if you set Opus up properly to use Sonnet at the right times for these sub agents that the result will be Opus feeling way faster and getting real work done for slightly cheaper. That sounds

Advice

Reasoning effort is how much the model may think before answering. On Anthropic models, avoid both low and max. Medium is mostly fine, but it is much stronger on Opus than on Sonnet.

benches like this. So generally avoid low, avoid max. And for the most part, medium's fineish. Medium is a lot stronger with Opus than it is with Sonnet. So keep that in mind. Cursor

Theo's own math: the $200 Claude plan gave him about $2,300 a week of usage at API prices, almost $10,000 a month. The subscription stretches far further than paying per token.

in the past few days, you get around $2,300 a week of usage on the $200 plan. That puts you at almost 10 grand a month for 30 days with your Cloud Code sub on the $200 plan. That's going to stretch you

Hot takes

Max reasoning mode is a trap and nobody should use it.

His reasoning: Max doesn't raise the ceiling on thinking. It forces a minimum, so the model can't stop early on easy tasks. In his benchmarks, going from extra high to max meant about 15 times more tokens, and overthinking makes it second-guess itself into wrong answers.

where from low to X high you get a 5% increase in token usage. And from X high to max you get, and I'm not exaggerating, a 1,500% increase in token usage. Literally 15 times more tokens for that one little bump at the end because you're no longer letting it be done with easy tasks. Andbecause if the model is told to keep overthinking the thing, it's going to overthink the hell out of it. It's going to start second guessing itself and is going to start getting wrong answers

OpenAI is losing everywhere it used to be strong: smartest, most efficient, cheapest, and best at calling other models.

His reasoning: In practice, the $200 Codex plan gives him nowhere near the usage or real-world code that his $200 Claude subscriptions do.

But godamn, OpenAI really needs to respond to this. They have lost in all of the places they are strongest. They don't have the smartest model. They don't have the most efficient model. They don't have the cheapest model. They don't have the best model for callingdoesn't feel like a good value. The $200 codeex plan gets me nowhere near as much usage and nowhere near as much realworld code as I'm getting out of my $200 Cloud subs. So take that as you will. Fingers

How Theo thought about this

When Sonnet's reply ignored his request about deleted code, Theo wondered if it was a regression, worse than Opus at understanding intent. Then he saw he was in the wrong thread. He decided it had made a logical choice based on that thread's contents, and admits he was about to crash out over it.

a regression in Sonnet 5.5's behavior? Maybe it's not as good as Opus at like understanding my intent. Nope. I was in the wrong thread. This is one where I was always talking about numbers. So this is actually a really good thing is that it was able to make what I would consider a pretty logical decision basedout. It just did I would argue the right thing here which is nice. I was about to crash out about it not understanding my intent but it totally does. I asked it

He reads vendor charts skeptically. Anthropic's first chart shows Sonnet beating Opus on cost only at max reasoning, the setting he says not to use. So he treats the chart as a warning sign rather than evidence.

Opus. This is the type of chart that you would see and think something went wrong. Not the type of chart that you would publish is the first chart in the blog post, but sure kind of sad that the only place it is outperforming opus for the cost is in the max reasoning effort, which again you shouldn't use. Over to

Sign in to mark what you've watched and keep notes.

Have a question about this one? Ask about this video →

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.