How Claude Fixes It's Own Code
Friday, Nov 6 · 42 min · about 10 min to read
Theo-brain watched all 42 min
Theo reacts to how Anthropic handles code review when agents write the code, and compares it with his own T3 Code process. The core lesson: humans reading diffs no longer catches the bugs that matter. Give agents real ways to test their work, make review loops stop, and make rollbacks fast.
Skim itSkim for the parts where Theo describes his own T3 Code setup and where he disagrees with Anthropic.
Tap any time to jump the video there.
Key takeaways
Theo says human code review would not have caught any regression his team has shipped in T3 Chat or T3 Code, whether he reviewed it or not. Today's bugs can't be found by staring at code. You need other ways to verify it.
ship slop. I'm going to be so real with you. Pretty much every single regression we have shipped in either T3 Chat or T3 Code, whether or not I reviewed the code, I would not have caught it in code review and I promise you not a single one of you would have either. The types
Theo thinks a lot of people who struggle to get good results from agents struggle because they never gave the agent any way to check whether what it did actually works.
I absolutely agree. I think a lot of the people who are struggling to get good results out of working with agents are struggling because they haven't given the agent any way to check if what it did work or not. And to be fair, a lot
Agent review loops must be made to stop. Theo told his "council" of reviewers to keep iterating until all passed, then forgot about it. Four days later the PR had grown to about 80,000 lines and was going in circles.
it was folded in my working tab. I just noticed it like yesterday. It's been going for 4 days. The PR that it's working on has gotten to like 80,000 lines of code, and I had to hop in and be like, "No, you're just running in circles right now." And I'm realizing I need to add short circuits to my skills cuz it's just it's not making any progress at this
Stacked review layers don't only remove bad code, they add some. Theo has seen PRs that should be 50 lines balloon to 800–2,000 because the agent addressed every comment from every reviewer.
unnecessary stuff, too. I cannot tell you how many times I've seen a PR that should have been like 50 lines of code get bloated to be 800 to 2,000 because the agent in the center addressed every single piece of feedback from every single layer on the way, and the result was just bad. And the most important
The most important safety layer is the last one: when something bad ships, you must be able to roll it back fast, and the failure should hit as few people as possible.
thing is at the bottom here. If something bad gets through to the end, you need to be able to roll it back fast. This is like the most important thing. Failures should affect as few people as possible and be as cheap as possible to undo. This has always been
Three skills here are new to you.
Council skill: multi-model code review
- What it is
- A review skill where Theo's code is checked by three models from three different AI companies, run through his existing subscriptions on a real computer that can launch the app.
- Why Theo cares
- He says it finds as much or more than paid review products, costs only his subscriptions, and can actually run the code. He calls Anthropic's own review product far too expensive for real-world shipping speeds.
- How to try it
- Ask your agent to build a review skill that sends each change to three models from different companies.
- Have it run on a machine where the reviewers can start the app and check behavior, not just read the diff.
- Add a stop rule so the review loop cannot run forever.
locally. One of the things I love is my Council skill, where I have my code get reviewed by three different models from three different families. That gets subsidized so I'm using my subscriptions for it, and it gets as good if not better stuff than these code review processes do, and it runs on a real
the extra review time does. I am not going to pitch the Claude review stuff cuz I think their review is way, way, way too expensive for most real-world cases, especially when you're shipping as fast as they're recommending here.
Score-based pass bar for reviews New
- What it is
- Instead of looping until reviewers have no comments, each review agent gives the change a score from 1 to 10, and Theo sets the score a PR must reach.
- Why Theo cares
- Comment count is misleading: a PR with 18 tiny nitpicks may be a 9/10, while one with a single comment may be a 6/10 and more urgent. Scoring builds confidence instead of chasing nits.
- How to try it
- Ask your agent to make reviewers return a 1–10 score with their comments.
- Pick a passing score and have the agent stop fixing once it's reached.
manage to fix this in a silly way. Instead of just tracking all of the nits and going until the agent said, "No, this is fine." I moved over to a scoring system where the review agents leave a score one to 10 and I pick a bar for what score is high enough. Because a PR might have 18 comments but still be a nine out of 10 cuz all the comments are just silly small nits. But you might
Let agents test like a real user New
- What it is
- In T3 Code each agent thread can launch its own copy of the app, record a video of the change working, and use fake model responses so full end-to-end tests don't spend AI tokens.
- Why Theo cares
- By the time an agent says code is ready, it has already checked it most of the ways Theo would. It lets him land scary changes more often and with more confidence.
- How to try it
- Ask your agent to start the app and record a short video proving the change works, then attach it to the PR.
- Ask for a dev mode with fake, pre-recorded AI responses so tests run without paying for tokens.
every thread can get one of those. And it made it very easy for a thread to spin up an electron dev server for T3 code, record a video of the change showing it working, and then send it to me or put it in the PR. I also added the
And all of these types of things have actually helped a lot with our confidence shipping and the frequency that I can land scary changes because my agents now can actually test and validate the code the same way a user would. So, by the time my agent is
AI explainers for big PRs New
- What it is
- Paste a large pull request into Claude and ask for a breakdown: what is changing, why it is changing, and where the risk is. A step further: have Claude build a visual artifact that teaches how that part of the code works.
- Why Theo cares
- He says people severely underrate how powerful this is. It's how he reviews heavy changes in T3 Code, and he calls the visual-artifact version awesome and highly recommends it.
- How to try it
- Paste a big PR into Claude and ask: what is changing, why is it changing, and where is the risk?
- Ask Claude to build a visual artifact that explains how that part of the code works, in the way you like to learn.
state. I think people severely underrate how powerful this is. When I'm reviewing heavy changes, I'll often just paste the PR into T3 code with Claude and ask it to give me a breakdown of the changes. What is changing, why is it changing, and where is the risk? Some people on
the team go a step further and have Claude build an artifact to visualize the part of the code to teach them how it works in whatever way that person likes to learn. This is awesome. I highly recommend doing it. If the goal
Advice
Never open a PR your own agent hasn't reviewed first. Theo says doing so wastes your team's time.
That shouldn't be most, that should be all. I was saying this for a bit, if you're putting up PR's that your agent hasn't even reviewed yet, you are wasting the time of your team and it upsets me. Apparently, Claude also
When an agent keeps making a mistake, fix the highest layer first: improve the code's structure so the mistake is harder to make, not comments. Only after that, add lint and type checks, and only then instruction files.
When you have to correct your agent, you should look for the place closest to the top to fix what caused the agent to do things wrong. The top spot is the code base. If it's making mistakes in the code, see if you can fix the code to make it less likely to make mistakes. Don't do this by putting comments everywhere. Do this by making the architecture better and easier to work
Make your PR template, or the skill your agent uses to open PRs, include how to test the change, what the risk is, and how to undo it.
templates. I like this part. Like your PR template and the skills or whatever you use to open PRs should include how to test, what the risk is, and how to undo this change. Add an AI reviewer.
Hot takes
Most agents are now better at code review than most humans. If they aren't in your codebase, your codebase is the problem, not the agent.
His reasoning: Humans and AI both miss things, but the AI won't burn out if you ask it to check again and again, while smart humans are too busy to bother.
your code. I also have the hot take that most agents are now better at code review than most humans and if they're not better in your code base, that's because your code base sucks not because the agent sucks, but that's a
everything. Humans and agents are both lossy. The main difference here between the smartest AI and the smartest human is that the smartest AI isn't going to get burnt out if you tell it to check over and over and over and over again.
AI code-review tools should drop their one-click 'copy this prompt to your agent' suggestions. That's '2024 core'; his agent babysits the PR and decides which feedback is good.
His reasoning: No human should have to look at review comments until agents have already sorted the real findings from the noise.
products. Get the goddamn prompt suggestions out of the code review. That is 2024 core. We're in 2026. I'm not copy-pasting from the code review back to my agent. My agent's babysitting the PR. My agent is deciding if your feedback is good or not. All of the code
Disagrees with Anthropic's habit of fixing review misses by adding to Claude.md, skills or review.md files. More instruction files are not the solution.
His reasoning: Like a blind person in a maze: rather than more directions at the door, fix the room so it's easy to navigate regardless.
Like I don't want to just keep giving the model more instructions before it goes in the door. I want to make it so the room it's navigating in is as easy for it to navigate as possible and I don't think more and more of these review MD files are the solution at all.
Shipping slower to avoid breaking things isn't a real fix. He agrees with Anthropic's choice to get faster at noticing and fixing problems instead.
everything. Teams decided that they'd rather get faster at noticing and fixing problems rather than slow down to prevent them. And I agree there. If your solution to not shipping breaking changes is to just go slower, then you're not actually solving the problem.
How Theo thought about this
Reading Anthropic's drop in reverts, he said it feels as much like model improvements as process improvements. On his own team, process changes gave real wins but model upgrades gave bigger ones: reverts fell well over 40% after Opus 5.5, and again when he banned Gemini models from landing code.
feels as much like model improvements as it does process improvements. Like, our process improvements have given us some bigger wins than this, but model improvements have given us even bigger
than that. Like, I would say our revert rate has dropped much more than 40% since Opus 5.5 dropped, personally. We also had another big drop when I banned all Gemini models from landing code.
To test his view on instruction files, he checked his repo's AGENTS.md history live. It changes about once a day, versus 30–50 times for a busy component, and edits came from the repo changing, not from patching agent mistakes.
even there, we didn't change this because we saw our agents doing the wrong thing, we changed it because the repo itself changed, and one of the pieces of guidance was no longer true. That is what leads us to changes, not
Sign in to mark what you've watched and keep notes.
Have a question about this one? Ask about this video →
Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.
