Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou
29:37
0:00

Progress?

Wednesday, Nov 11 · 30 min · about 7 min to read

Theo-brain watched all 30 min

Theo tested Google's new Gemini model, Carbon, expecting to pan it. It is still slow, still makes things up, and its price doubles on December 1st, so Opus stays his default. The surprise: it handled a real security fix in his T3 Code app better than any other model he tried, so he now sees it as a niche tool for security work.

Briefing is enoughThe briefing covers the numbers and the security story well enough that you can skip the video.

Tap any time to jump the video there.

Key takeaways

Theo's bottom line: if you can use other models, Carbon is mainly worth it for security development work. If you are stuck inside Google's ecosystem, you will still feel a real upgrade.

function on a basic level. Yeah. This model seems to be nearly exclusively useful for cyber security dev work if you have access to other things. Obviously, if you're locked to the Gemini ecosystem for some reason, you're going to feel the upgrade. They still

The surprise win: given a real security report on T3 Code, Carbon proposed the fix Theo wanted. That was to separate trusted-machine connections from raw-link connections, with no heavy cryptographic redesign. Other models refused to help or pushed a clunky fix.

The best fix is a hard separation between known machine flows and raw URL flows paired with tightening URL retention in the peer link section. You do not need heavy cryptographic redesign to kill this vulnerability cleanly. AndI wanted. I did not expect that considering every other model I had tried was either refusing to help or was mythos which you just saw. The fact that I had the best experience addressing these reports with this model is kind of insane. Like genuinely, I did not expect

Hallucinations halved. Across Theo's evals, the share of false claims fell from 36% with the previous Gemini (Argan) to 18% with Carbon. That is still far worse than the best models.

only 23 of 86. Across all of my evals, the percentage of false claims dropped from 36% with argan to 18%. And I findclearly here. If I'm being real, it's like, yep, there it is. It dropped in half, but it's still way worse than anything other than the Muse line.

It is slow. On low effort, the average task took 22 minutes in Artificial Analysis testing, and 27 on high. Opus took about 3 minutes on low and 25 at max.

time is garbage. So on low the average task time was 22 minutes in artificial analysis and on high it was 27 versus just for a comparable range something like opus the low was only 3 minutes and 1 second and the max was 25 minutes. We

The price is introductory: $2 per million input tokens and $10 per million output until December 1st. Then it doubles to $4 and $20, which Theo says is Opus pricing.

It's a Googlem. The price is $2 per mill in and $10 per mill out with a 10 cent context cash read cost until December 1st at which point it becomes $4 per mill in and 20 per mill out and 20 cents per mill cash read doubling in price.

Advice

Judge a model by what a whole task costs, not by its price per token. Theo found Gemini's automatic caching (reusing earlier input cheaply) often misses, which can add up to 50% more requests.

already, as I've said many a time, the token cost does not matter. The cost per task is what matters, especially when you consider how many tokens are being used in a given task. Cash rate efficiency matters a lot too. AndI found that in a 5-second window, there was a 20% chance of a cash miss. In a 30-cond window, there was a 34% chance. In 120 second window, it was even more. This results in as much as a 50% increase in requests if they are waiting

Don't use Carbon for code review or for analyzing a codebase. Theo saw it invent database column names, line numbers and facts about the architecture.

task is stuff like code review and codebase analysis, absolutely not. I've seen this model just make up a lot. It makes up column names for tables in your DB. It makes up line numbers for things that it doesn't properly read or understand. It makes up things about

If Opus flags or refuses your security work, try Carbon. Theo adds a caveat: his early-access status may mean he gets flagged less than other users would.

pretty good tool for incident response in security stuff. If you find yourself getting flagged by Opus, maybe give it a go. I will say there is some potential that my early access status prevents me from getting flagged as hard as others do. Maybe it has booked me under the

Don't use your Gemini subscription in other tools unless Google allows it. Google permits it through its ACP bindings, the official way for apps like T3 Code to drive Gemini.

been a lot of skepticism around using your Gemini sub and other tools. And I'll be direct, you shouldn't do it without Google giving the tool permission. Thankfully, the DeepMind

Hot takes

People calling Carbon a fast model are wrong. A high token-per-second rate (TPS) is not the same as speed.

His reasoning: He points to real task times: it is consistently the slowest model to finish actual work, so it's unpleasant to use.

all. Yeah. So, everybody's saying, "Well, it's a fast model." Shut the up. You have no idea what you're talking about. Like, I'm sorry if you think TPS means speed. Look at the numbers. This model is slow and it sucks to use cuz it's slow. Ah. Anyways, I told you

Opus is still unbeatable as a default coding model, even after Fable 52 came out.

His reasoning: It is good value, a good experience and a good engineer. On the $200 plan it feels close to unlimited for real work.

coding? No. Absolutely not. It's just Opus is unbeatable still. Even after Fable 52 dropped, like Opus is such a good value. Opus is such a good experience. Opus is such a good engineer. And if you're using the $200 plan, Opus feels pretty close to unlimited for real world work. You're

Google's short introductory pricing is a play to get people stuck on the model, then charge them double.

His reasoning: Discounts used to last two to three months or more. This one is under a month.

innovates on shitty things. I think they do this because they are trying really hard to get people to adopt the new model and they're hoping they can get some people stuck on it and then they will charge the higher price.better. But they would give like two to three plus month timelines for these discounts. Now it's under a month, which

Google, with one of the biggest codebases in the world, made a model that struggles with big codebases when other models don't.

His reasoning: Even projects he hacked together with Opus are often too big for Carbon to properly understand.

days myself. In my experience, even the things I slopped together with Opus are often too big for this model to actually properly understand. It's kind of crazy that a model that Google, the company with one of the biggest code bases in the world, made struggles so much in big code bases, but it does and none of the

How Theo thought about this

Theo trusts his own tests over published scores. He counts how many of a model's claims about real code are false. Carbon made 79 claims with 23 false, while many models have zero or one. That gap made him doubt a benchmark where Carbon scored best ever.

score ever on cursor bench 4, which is kind of wild and I'll be real makes me question this benchmark because in my benches it did not perform anywhere near as well. My recent code benches haveCarbon made 79 claims, so it said way more. It had a lot more opinions and it also got there faster, but it still had 23 false claims, which is insane because a lot of the other models have between zero and one, and a handful have three

He changed his mind on one real task, not on benchmarks. He planned a negative video, then tried Carbon on the security fix just out of curiosity. It beat everything else he'd tried, so he said so, while still calling it miserable to use day to day.

merged. Before it was done, I was getting ready to film this video and I decided, you know what? Why not? YOLO, if you will. Let's see what carbon does. I'm curious. It described the issue

Sign in to mark what you've watched and keep notes.

Have a question about this one? Ask about this video →

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.