Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou
27:29
0:00

So much for "Pacing" the Frontier

Sunday, Sep 27 · 27 min · about 6 min to read

Theo-brain watched all 27 min

Theo argues that the burst of new models, like Opus 5.5, since labs agreed to pace the frontier is itself pacing. These models raise the capability ceiling little and mainly make models less dumb, more reliable and cheaper. The real risk is models improving in ways humans can't follow, so reliability gains are the safe kind of progress.

Briefing is enoughThe notes carry the argument and its examples, so you can skip the video unless you want to hear Theo make the case himself.

Tap any time to jump the video there.

Key takeaways

The fear behind pacing is not a model hitting some benchmark score. It is "takeoff": models improving themselves so fast that humans can no longer understand what they are doing or why.

the takeoff. Not that a certain level of intelligence is hit, but at a certain level of intelligence and recursive self-improvement, the model starts improving so fast that we can't understand what what doing or why. The

Efficiency can cost understanding. OpenAI trained its models to reason in terse, broken "Grug-style" notes to save tokens (the units models read and write), which makes them harder to follow. Anthropic's models use more reasoning tokens, and that makes them easier to monitor.

make here is that OpenAI has put a lot of effort into making the model more efficient, but the things they've done to make it more efficient have also, to be frank, made it harder to understand what it's doing. We don't get theseAnthropic's models are not as efficient with their reasoning tokens, so they're easier to monitor. In fact, from Opus 5 to Opus 5.5, the number of reasoning tokens per task on artificial analysis doubled.

Benchmarks don't measure how smart a model is at its best. They measure how consistently it succeeds and how often it fails, so a very intelligent model can score worse than a steadier, less brilliant one.

Point I want to make here is that a benchmark isn't measuring how smart is the model at its peaks. It's measuring how consistently is it succeeding, and also how often is it failing. And it is

Raising the "floor", meaning how bad a model gets when it acts dumb and how often, adds no risk. It still lifts benchmark scores, everyday experience and reliability.

So, a much simpler way of putting this is that improving the floor for how bad models get when they're dumb and how often they act dumb does not increase risk. But, at the same time, absolutely increases benchmark scores, the quality of experience we have, and the

Labs take the good solutions from their biggest models (Astra, Fable 5.1), strip out the mistakes and use them to train smaller models with RL (reinforcement learning). The result feels close to Fable but is cheaper and faster.

up the mistakes they make. We create training sets that are just the good, not the bad, and then we use that to beat the stupid out of Opus. The result is a very good model that kind of just feels like what we got with Fable, but cheaper and faster. And this is why I'm

One skill here is new to you.

Opus 5.5 for long agent workflows New

What it is
Anthropic's Opus 5.5. Theo puts it in the smaller tiers, a big step below frontier models like Fable and Astra, and says it doesn't push the capability ceiling up much.
Why Theo cares
What Opus 5.5 adds is reliability: it is less likely to do something stupid, so you can trust it with longer workflows.
How to try it
  1. Give Opus 5.5 a longer multi-step task you would normally split up, and see whether it stays on track.
  2. When comparing models, count how often each one does something dumb, not just how good its best answer is.
reliability of these models. And models like Opus 5.5, GBD-6, Soul, Luna, and Grok-47 aren't really pushing up the ceiling much. The thing that we gain from them, in particular from Opus 5.5, is that we can rely on them for longer workflows because the likelihood they do something stupid goes down. And this is
there's a huge gap between something like Fable and Astra and models that are now the smaller tiers like Opus, Luna, Soul, and Grok. The difference is that

Hot takes

The flood of new models since the labs agreed to pace the frontier is itself pacing, and we're in a really good spot.

His reasoning: He says a well-paced cadence actually produces more models, and cheaper ones, than people would guess.

the hot take comes in. What we are experiencing right now with all these new model drops, that is pacing. We are pacing better than ever, and I think we're actually in a really good spot

Opus 5.5 is an example of pacing done well, despite its higher benchmark scores.

His reasoning: It isn't discovering new capabilities the way Fable and Astra did. It refines what models can already do.

Opus 5.5 is actually an example of pacing well. The difference here is that Opus 5.5 isn't discovering much in terms of new novel capabilities. Models like Fable and Astra have introduced new capabilities, new potentials, new things to the suite of stuff LLMs can do. They

GPT-6 Astra is both the smartest model Theo has used and one of the stupidest he's used this year.

His reasoning: Stupidity and intelligence are different things and can coexist in one model.

different things. The amount of stupidity a thing has and the amount of intelligence a thing has can coexist. Look at GBD-6 Astra. It is both the smartest model I've ever used and without question one of the stupidest I've used this year. What's really

Until pacing, Anthropic didn't care about any model below its frontier tier. Pacing is what pushed it to pour effort and marketing into Opus.

His reasoning: Nobody got excited about Opus 5 or Sonnet, and Haiku hasn't been updated in 11 months.

about Opus 4.8. The reason is because Anthropic didn't care. They don't even care enough about Haiku to release it. Like Haiku has not had an update for 11 months. We're near a year since Haiku 4.5. Anthropic didn't care at all about models that were less capable than Astra. So, what they're doing now

How Theo thought about this

He reasoned from history. C made programming much easier, so the amount of code exploded, which he calls Jevons paradox: make a thing easier and you get far more of it. Yet humans understood the compiled low-level code less and less. He maps this onto AI: more capable, efficient models mean less understanding.

The introduction of C exponentially increased the amount of assembly that existed, because it turns out when you make it way easier to do the thing, you massively increase the amount. This is often referred to as Jevons paradox. YouAnd alongside that, our ability to comprehend the outputs went down. One more time, the things that we did to expand the usability of code and software also lowered our understanding of the underlying code that was powering

He tested his point by flipping it. Imagine Anthropic asking Fable to cut reasoning tokens in half for the same performance. It probably could, but cramming the same plans into fewer tokens makes the model harder to monitor.

The problem there is that if you try to communicate the same thing, the same ideas, the same concepts, the same plans and work in a half as many tokens, you're going to make it harder to understand and monitor. And

Sign in to mark what you've watched and keep notes.

Have a question about this one? Ask about this video →

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.