Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou

Classify data with confidence thresholds

Technique

Have a model label a large dataset and give each label a confidence score from 0 to 1. A slider sets how sure the model must be before a label counts. Theo used Jev on 32,311 messages from 1,118 of his own agent chats.

Theo's current stance

Theo showed that the cutoff changes the picture: 'expanding scope' was 22% of threads at 80% confidence but only 6.8% at 90%. You choose how strict to be instead of trusting flat tags.

Try it

  1. Ask your agent to export a dataset you care about, such as your own agent chat history.
  2. Have it classify every item with Jev and keep the confidence score for each category.
  3. Ask for a simple page with a confidence slider, then compare results at 80% and 90%.

First seen

27:47
classifications. So if we have it at an 80% confidence interval, then 22% of my threads are expanding scope. But if I bump that up to 90%, it's down to 6.8 because it was not sure in the rest of those cases or as sure so to speak.

Jev is incredible

Every mention

27:47
classifications. So if we have it at an 80% confidence interval, then 22% of my threads are expanding scope. But if I bump that up to 90%, it's down to 6.8 because it was not sure in the rest of those cases or as sure so to speak.

Jev is incredible

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.