Score-based pass bar for reviews
Technique
Instead of looping until reviewers have no comments, each review agent scores the change, and you set the score a PR must reach. Scores can track things you care about, like how likely you are to be okay with it, or several scores at once.
Theo's current stance
Theo says comment count misleads: 18 tiny nitpicks may be a 9/10. He calls custom scoring a lossy but powerful way to steer lots of agent work toward your taste.
Try it
- Write down how you like things built, so reviewers can learn your taste.
- Ask your agent to make reviewers return a 1–10 score with their comments.
- Ask for separate scores, e.g. severity, risk of future problems, and how likely it is to annoy you.
- Pick a passing score and have the agent stop fixing once it's reached.
First seen
29:05
manage to fix this in a silly way. Instead of just tracking all of the nits and going until the agent said, "No, this is fine." I moved over to a scoring system where the review agents leave a score one to 10 and I pick a bar for what score is high enough. Because a PR might have 18 comments but still be a nine out of 10 cuz all the comments are just silly small nits. But you might
Every mention
18:00
it to give two scores or three scores. I had a round where I had to review every PR in three ways. How severe is the problem it fixes? How likely is this to cause problems in the future? And how likely is this to piss Theo off? And the
20:31
what he likes. And third and most importantly, I want you to bring Council back in, but this time Council's job isn't to make sure the changes are good. Council's job is to make sure the PR is in a state where Corey would be happy.
29:05
manage to fix this in a silly way. Instead of just tracking all of the nits and going until the agent said, "No, this is fine." I moved over to a scoring system where the review agents leave a score one to 10 and I pick a bar for what score is high enough. Because a PR might have 18 comments but still be a nine out of 10 cuz all the comments are just silly small nits. But you might
Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.