Jev is incredible
Sunday, Sep 20 · 30 min · about 9 min to read
Theo-brain watched all 30 min
Theo covers Jev, a new model that does not chat or write code. It classifies data in milliseconds, always returns JSON in the exact shape you asked for, and costs almost nothing. Treat it like a library your software calls for quick, simple sorting jobs, not as a replacement for models that need to think.
Briefing is enoughThe notes carry Theo's argument and numbers; the demos are fun to see but don't change what you'd do.
Tap any time to jump the video there.
Key takeaways
Jev is not a chat or coding model. Its job is classification: you hand it data and a format, and it returns JSON (structured data your code can read) in exactly that format, every time.
to do today. It is classification. This model is the best model ever made to take data and organize it and classify it. It returns JSON perfectly in a type-S safe format. When you give it a format, it does it and it does it well.
It is meant to be built into software, called like a function, not typed into Codex, Claude or a terminal. Theo says to treat it like a new library you install.
programmer. But you shouldn't be viewing this as a tool you use in Codeex or Claude or even in your terminal. You should see this like a new library you install or a function that you call. It is meant to be integrated in the tools that we build, not used as an inference
The speed and cost are the point. It does in 70 to 500 milliseconds what normal models take 3 to 300 seconds to do. Input costs about 4 cents per million tokens versus $10 for Fable, and output tokens are free.
Traditional LMS can take three to over 300 seconds to do the type of classification work that this model can do in 70 to 500 milliseconds. That is actually 40 to 200x faster. There's no
super cool. And the token cost is insanely low. It's about 4 cents per million tokens in compared to $10 per million on a model like Fable. I see
The shape of the answer is guaranteed; the answer itself is not. Theo separates the two: the format is deterministic (always the same), the content is just 'quite reliable'. Every answer also comes with a confidence score, so you know when to distrust it.
it deterministic? The format of the output is yes. There are two ways things can be deterministic. The shape and the content. The content isn't deterministic, but it should be quite reliable. The actual format is what is deterministic. It will always give you the format that you define. And on the
It is not smart. Theo calls it roughly as intelligent as a switch statement (a basic pick-one-path rule in code). It can't explore a codebase or tell good model outputs from bad, and its context window, the amount it can read at once, is only 32k tokens.
classifying things. It doesn't know how to look through a codebase to make a good decision. It doesn't have the intelligence to distinguish between the outputs of language models. And most importantly, the context window is tiny. It's only 32k tokens. That doesn't mean
Three skills here are new to you.
Jev as a cheap first-pass filter New
- What it is
- Run high-volume, simple sorting jobs through Jev before anything expensive touches them. In a clip Theo shows, Ryan ran Jev on his own exported emails: about 200 ms per email on average, 38 emails per second.
- Why Theo cares
- Theo says he already uses LLMs to go through his own email, which is expensive. He sees Jev as the quick first pass that clears low-effort spam. It suits jobs too cheap to justify an LLM, or run too often to wait for one.
- How to try it
- Have your agent export a batch of emails or support messages and define the categories you want back.
- Run that batch through Jev as a first pass to sort out the low-effort items.
- Keep your normal model for anything that needs real thought.
Boom. 100 emails done. It doesn't tell me how long it took. average was 200 milliseconds. 200 milliseconds per email. P95 was 240 milliseconds and it did 38 per second,
better example. I already use LLMs to go through my email. It's expensive, but it's fine. This is a first pass to like get the quick, loweffort spam things out of your way. Super useful. It's the things that are too cheap to justify running an LM for or the things that are run too often to wait that much time
Classify big piles of data with confidence thresholds New
- What it is
- Theo had Jev classify 32,311 messages across 1,118 of his own coding-agent chats. Each label comes with a 0-to-1 score, and he built a small web app with a slider to set how sure the model must be before a label counts.
- Why Theo cares
- The threshold changes the picture. At 80% confidence, 22% of his threads were 'expanding scope'; at 90%, only 6.8%. You decide how strict to be instead of trusting a flat list of tags.
- How to try it
- Ask your agent to export a dataset you care about, such as your own agent chat history.
- Have it classify every item with Jev and keep the confidence score for each category.
- Ask it for a simple page with a slider for the confidence cutoff, then compare results at 80% and 90%.
classifications. So if we have it at an 80% confidence interval, then 22% of my threads are expanding scope. But if I bump that up to 90%, it's down to 6.8 because it was not sure in the rest of those cases or as sure so to speak.
BAML for reliable structured output from regular models New
- What it is
- BAML is a small language for declaring the exact shape you want back from an LLM. Its runtime repairs badly formatted JSON so you always get the promised shape.
- Why Theo cares
- He used it with GPT-OSS-120B to find comments that mention sponsors, and it just worked. When he dropped BAML for another job, he found that model formats JSON wrong about half the time. The catch: those repairs take time.
- How to try it
- When an agent-built feature needs structured answers from a regular LLM, ask your agent whether BAML fits.
- If outputs break your app now and then, ask whether the model is returning malformed JSON.
And when I tried to move off BAML for another thing I was doing, I learned that apparently GPOSS120B doesn't format JSON correctly half the time. One of the many things they do with BAML is in their runtime, they fix the mal formatting in the JSON outputs to make sure it is guaranteed to match
Advice
Use the 10-second test. If a person who already has all the information could answer in under 10 seconds, like naming a shirt's color, Jev is probably a good fit. If it would take longer, it probably isn't.
of the information? If I show you a picture of a person wearing a shirt, how many seconds does it take for you to say what color it is? If the answer is under 10 seconds, this model's probably good for it. If it's over 10 seconds, this model's likely less good for it. This is
Don't use Jev to change how your coding agent's harness works, meaning the tool around the model such as Claude Code or Codex. Harnesses are good now, billions are being spent on them, so don't reinvent the wheel.
change anything that the harness is doing. The harnesses are pretty dang good now. You should lean into the fact that billions upon billions of dollars are being spent on that and not reinvent the wheel constantly. Don't get me
Hot takes
Using a fast classifier like Jev to compact an agent's context is a bad idea. Compaction means shrinking a long chat history so the agent can keep going. Anyone who thinks this works should stick to their tool's defaults.
His reasoning: Compaction isn't filtering lines out. It writes a summary of everything so far. Jev also lacks the context to know what it's deciding on: it holds only 32k tokens and doesn't see the tool results.
actually think it's a good idea to use a non-reasoning classifier model for compaction for your context, then I would plead that you never ever ever stray from the defaults in those tools because you just fundamentally don't understand yet. And don't worry, you're
isn't a filter. We're not just going through your history and selectively deleting lines from it. We are synthesizing a summary based on everything that's happened so far. Jev
Braintrust, a company building tools to monitor agents, suggesting Jev as a judge for scoring agent responses makes Theo trust them less.
His reasoning: Their pitch is that you skip prompting a general model into a judge. Theo says those prompts take under 30 seconds to write.
for agents. This is a really bad thing for them to tweet. It makes me not trust them because they suggest that you should use Jev as a judge scorer in a tool like Brain Trust to decide between outputs from LM. They're legitimately
responses with Jev. And the reason is so that you don't need to spend time and resources prompting a general purpose model into an LLM judge. What those prompts take 30 seconds to write. Not even. And like it's if you
Anthropic's Haiku should stop being used for anything.
His reasoning: In Typesafe's structured-output test it had a 45.5% error rate. It's about a year old, and Anthropic itself is treating it as dead.
from a certain anthropic where haiku had a 45.5% error rate. That model is so bad and it needs to stop being used for anything. It's about a year old now might be over that. Just anthropic is treating haiku as dead. We should do them the favor of doing the same. But of course the number
How Theo thought about this
He tested the claims himself by building a checkers game where Jev picks the moves. Replies were near-instant, but he beat it while barely paying attention. That showed him it's fast precisely because it skips reasoning.
playing and crush this model. Yeah, it's just it's not good. I was barely paying attention and I'm going to crush it here. Point I'm trying to make is that it's super fast but it's not using the reasoning part of its brain so to speak.
He read the benchmarks critically. The 'correct' answers were an average of GPT-6 Astra and Fable 5.1. Those models can be wrong too, so he treats the accuracy data as imperfect.
Cool. They're using the average of GBD6 Astra and Fable 5.1 as the reference answer. That's why they don't appear in the list and also why the data might not be perfect since those models might be wrong as well. The LMS use their system
Sign in to mark what you've watched and keep notes.
Have a question about this one? Ask about this video →
Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.
