Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou
27:53
0:00

Elon promised this one would be good...

Tuesday, Sep 22 · 28 min · about 7 min to read

Theo-brain watched all 28 min

Theo reviews Grok 4.7, which Elon promised would be more token-efficient. In practice it uses roughly twice the tokens of Grok 4.6, costs as much as or more than top models, and makes very bad front-end designs. Theo finds it fun and likes that it stays on task, but he calls it a flop: last-generation performance at a current-generation price.

Skim itThe notes cover the argument; watch only if you want to see the visual demos for yourself.

Tap any time to jump the video there.

Key takeaways

The promised token efficiency did not happen. Elon said Grok 4.7 would be slightly slower but more token-efficient. Artificial Analysis, an independent benchmarking site, measured about 38k tokens per task for Grok 4.6 and 81k for Grok 4.7.

every way except slightly slower to serve but with even better token efficiency. This was a lie. Gro 46 XHigh, according to artificial analysis, did around 38k tokens per task. Gro 47 it did 81k tokens putting it above even

Theo's own testing confirms the cost problem. His first Grok 4.7 run on his own benchmark would have cost $20 at API prices. No other model broke $10 except one Astra run at $11.44.

The most expensive run by far was my first attempt with Grock 4.7, which would have cost $20 if I paid API prices. No other model breaks $10 other than one single Astra run, which just barely broke it at 11.44. Rock 47 in its

What Theo likes about it: it does what it's told without much intervention. It is not as good as Fable or Astra at the extremes, but its floor (worst-case quality) is higher than similar models, and it stays on task longer.

model is that it does what it's told and it does it well without me needing to intervene too much. It's still not as good as any Fable or Astro version is at the extremes, but I've noticed the floor is quite a bit higher. It stays on task longer and can complete real work. And

Subscription value got much worse. Theo's co-host Ben Davis used $40 of usage and burned 8% of his weekly limit on the $300 plan. Theo contrasts that with getting $8–12k a month of usage on other labs' $200 plans.

the numbers aren't great either. He did $40 of usage and it burned 8% of his weekly limit. That means that you're not getting much per week on the $300 plan, which is kind of crazy when you realize that you get 8 to 12,000 a month on the $200 plans from the other labs. So, not

Its front-end design work is very bad. Theo calls these some of the worst designs he has seen from any model, worse than he remembers GPT-5 being.

These are legitimately some of the worst front-end designs I've seen from any model in hell, even 2025. Like, I haven't seen anything this bad in a very long time. I I my memory might be rosetinted here, but I remember GPT5 being better than this at front end.

One skill here is new to you.

Model bake-off: review your own codebase New

What it is
Have several models each go through the same codebase and identify areas to improve. Then use strong models as judges to score the suggestions.
Why Theo cares
Theo calls it a Hail Mary test, but the results were useful. Grok 4.7 came in just behind Astra on how viable its suggestions were. Fable 5.1 made only 5 suggestions instead of 8, but they still scored well. Fable was one of the two judges, yet it scored itself below Grok.
How to try it
  1. Ask each coding agent you use to go through your codebase and identify areas to focus on for improvement.
  2. Have Fable 5.1 and Astra act as judges and score each model's suggestions.
  3. Compare the scores against what each run cost before choosing a default model.
that said, I decided to have a bunch of different models go through T3 Code's codebase and identify areas that we should be focusing on for improvement.
was scored, I set up a set of judges with Fable 5.1 and Astra. So, even though Fable was half the judging work, it actually scored itself below Grock.

Advice

If you use Grok 4.7, expect to put in more effort up front: tighter specs and more handholding. Theo says even GPT-4.1 could handle a lot of work if you specified it exactly.

handhold it and spec the work. exactly as needed. This is all a matter of how much effort do you want to put in up front and Grock 47 will require more effort up front. It just will. The much

Theo says Grok 4.7 is cool, fun and worth playing with, especially if Cursor is still your main source of AI usage. Outside of that, he finds it hard to recommend.

this model's cool and fun and worth playing with, especially if you're still stuck using cursor as your primary source of inference, but outside of that, this one's hard for me to recommend. I hope that the next model

Hot takes

Benchmarks no longer represent reality, and most of the numbers floating around this release are useless.

His reasoning: He says xAI's announcement used a cherry-picked, oddly chosen set of benchmarks that cannot capture what actually makes the model good.

those. But the thing I want to talk about here is how the benchmarks no longer represent reality. And the numbers we look at when we talk about models just aren't the right way to do it anymore. And the sheer volume ofThis is such a weird set of benches to include. You can tell they cherrypicked the hell out of this. To their credit, benchmarks are not very good at representing what makes this model good.

A six-point gap on a coding-agent benchmark matters more than it looks.

His reasoning: Those points reflect how likely a model is to do something stupid within a given stretch of time. Fewer mistakes mostly buys longer unsupervised work, more than it buys harder tasks.

I know it seems silly that I care so much about six points on this arbitrary benchmark, but those six points indicate a gap in how likely the model is to do something stupid in a given time window. And as you decrease the levels of stupid, you increase the time window more than you increase the difficulty of

Grok 4.7 is a flop: last-generation performance at a current-generation price.

His reasoning: Theo says it is more expensive and slower than the models it competes with, with higher error rates and worse code. It also lacks the subscription subsidies of Codex and Claude, so there is no economic reason to switch.

Soul. It makes no sense right now. It's more expensive, slower, higher error rates, worse overall code, and nowhere near the subsidization that we're used to from the Codeex and Claude subscriptions. So, what is this modelnow, it just kind of feels bad cuz we're getting last gen performance in various tasks at a current generation price, which just is not the right compromise

How Theo thought about this

Cursor's CEO Michael Truell said Grok 4.7 uses only 5% more tokens per median request. Theo picked apart the word 'request'. Every tool call is a separate request, so if a model makes more requests per prompt, per-request numbers hide the real cost. That is especially true on agent-heavy benchmarks.

to 30% more for P99 requests. Notice that he said request here, though. A request is not an accurate way to measure token usage because one prompt can result in a very wide varying number of requests being made. If I send aBut if the number of requests per prompt is going up which it is then that doesn't matter at all. And when you

Theo first steelmanned xAI by guessing they had stayed on the old base model, which would explain the same price and speed. Then he found the official post claims a new, larger base model. That made the broken efficiency promises look worse, not better.

isn't true. Never mind. They claim in the official post that it is using the new larger base model compared to Grock 46. Okay, so I guess that part's true, which makes all of the rest a lie, which is

Sign in to mark what you've watched and keep notes.

Have a question about this one? Ask about this video →

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.