Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou
25:27
0:00

NVIDIA Just Lost Their Lead

Friday, Aug 28 · 25 min · about 7 min to read

Theo-brain watched all 25 min

Theo argues Nvidia's grip on AI chips is loosening. Nvidia makes buyers pay heavily for memory, and rivals are closing in: Apple's M5 Ultra Mac Studio, Huawei chips serving a popular Chinese model, and OpenAI's own Jalapeno chip. His bottom line is that the real long-term limit on AI will be electricity, not chips.

Briefing is enoughThe notes carry the argument and the buying advice, so you can skip the video unless you want to hear Theo walk through it himself.

Tap any time to jump the video there.

Key takeaways

Nvidia's pricing lever is memory, not raw speed. The RTX 5090 gaming card launched at $2,000 and now sells for about $4,500. The RTX Pro 6000 costs $12,000 to $16,000 and is the same speed or slower. The extra price buys memory: 96 GB versus the 5090's 32 GB.

retail price for the RTX 5090 was originally $2,000. They now consistently go for around 4,500 because the demand is so absurd. There's also a server card they put out called the RTX Pro 6000 that ranges between 12,000 and $16,000.5090? Well, it turns out the chip isn't the only thing that matters. The 96 gigs of GDDR7 are what mattered here. The 5090 only gets 32 gigs of GDDR7 and it's not error

Apple's new M5 Ultra Mac Studio has memory almost six times faster than Nvidia's DGX Spark and only 30 to 40% slower than an RTX 5090. Theo says that kills almost every reason to buy a Spark, a whole category of Nvidia devices.

that you would get on a Spark. Almost six times faster than the Spark and only like 30 to 40% slower than the 5090. That is insane. This kills almost all of the reasons you could ever justify buying the DGX Spark, which is a whole category of Nvidia devices killed. And

An anonymous model, which turned out to be GLM, handled a huge amount of traffic served entirely on Huawei chips, with no Nvidia hardware in the stack. Part of why this was possible is that it is a relatively small model. Theo calls it genuinely impressive.

relatively small model, but also because they served it entirely on Chinese chips. Huawei made the chips that they served all this traffic on, and it's still working great. It's genuinely impressive that without any Nvidia in their stack, they were able to do this

OpenAI's in-house chip, Jalapeno, beat every Nvidia, AMD and Google chip SemiAnalysis tested on several open models, according to that firm's report, which Theo reads through. Theo flags the caveat: only relatively small models have been tested so far.

a look early. In general, first energy and trips are not competitive, but Open AI bucks that trend by being industry leading and beating every Nvidia, AMD, and Google chip we have been able to test on multiple top open source models.also the callout that this is not large model performance. All the models they've tested so far are relatively easy to run in terms of size. So, we don't know how these will perform when

Theo's recurring thesis: the future of AI will be fought over electricity, not chips. Without enough energy, none of the chip race matters.

all of the chips they need. As I've said many times now, the future is going to be fought not on chips, but on electricity. And if we don't have ways to get the energy we need, then none of this ends up mattering in the end. But

One skill here is new to you.

Sizing hardware for running models locally New

What it is
Running an AI model on your own machine requires the whole model to fit in the graphics chip's memory. If a 33 GB model doesn't fit on a 32 GB card, splitting it across two consumer cards doesn't really work.
Why Theo cares
He says the link between two consumer GPUs is far too slow for them to share memory. So the amount of memory, not chip speed, decides what you can run.
How to try it
  1. Before buying hardware to run a model, check the model's size against the memory of a single machine.
  2. Don't plan on linking two consumer graphics cards to fit a bigger model.
  3. Look at machines with large unified memory (shared by the main processor and graphics), like the Mac Studio.
You can't just plug the chips in together. If you have a model that is 40 gigs, hell, if you have a model that's 33 gigs and it doesn't fit on your 5090, you now have to split the data across the two GPUs, which means ideally you

Advice

Don't buy Nvidia's DGX Spark expecting to use it as a regular computer. Theo says Linux on its ARM chip is hell, so use it only for running models.

arm Linux in my life, I promise you you're not going to use this computer as a computer. Linux on arm is hell. Don't bother for this for anything other than inference. So this arm chip in this

If you were weighing an RTX Pro 6000, or skipped the Spark because it was too slow, Theo says the M5 Ultra gives a much better experience for meaningfully less money.

all the people who upgraded from a Spark to a RTX Pro 6000 or skipped the Spark because they wanted speeds that weren't trash, they can now get way, way better experiences for meaningfully cheaper by just getting the M5 Ultra instead.

Hot takes

The DGX Spark is a bad product, and Theo's least favorite computer in his apartment.

His reasoning: Its effective memory speed is, in his word, pathetic.

end to that." And yes, the DGX Spark's effective memory bandwidth is actually this pathetic. It's a [ __ ] chip. It's a [ __ ] system. The DGX Spark is my least favorite computer in this apartment.

Nvidia is scared, whatever Jensen Huang says. Theo mocks Jensen's claim that he is very comfortable with the competition: "Definitely not scared at all, are you, bud?"

His reasoning: AI tools now make it easier than ever for others to catch up, and models are useful enough to help chipmakers in their process.

so, I'm very comfortable with all the competition. >> Okay, and I also >> Yeah. Definitely not scared at all, are you, bud? Well, on the bright side, if theywhere any of this will go. All I know is that Nvidia is scared, and they have good reason to be. The AI world constantly is changing, and these tools and technologies make it easier than ever to catch up. We finally now have models that are useful enough to help

Nvidia's bid for Hugging Face, the main site for sharing open AI models, is a good-faith move to fund open-source AI. It also serves Nvidia's own interest.

His reasoning: Training still happens on CUDA, Nvidia's software platform. So the more businesses train and fine-tune their own models, the more customers Nvidia has for its chips.

and encouraging more and more people to train because let's be real, training is still happening on CUDA. The more they encourage businesses to try and train and fine-tune and customize things themselves, the more customers they have for their chips, the longer-term their

AMD, where Theo owns stock, is still far behind and won't catch up for a while.

buying their stocks, but I have not talked about AMD at any point here cuz I'll be real. I love them to death. They're very behind. It'll be a while before they can catch up to what these other things are that I'm talking about

How Theo thought about this

When he tweeted about the anonymous model, Theo assumed it was served on Nvidia, because that was what everyone expected, and he got called out. He says he privately suspected Xiaomi was doing it on Chinese chips. It turned out to be GLM, served on those same chips.

called out here because people were saying only Frontier Labs have this much compute. When I said that, I assumed that they were still serving Nvidia because we basically expected that. Andfair, when I tweeted that, my secret personal belief was that Xiaomi was doing this on Chinese chips somehow. Then it turned out to be GLM shipping again in like a 2-week window, also on those same chips. All of the traffic was

Sign in to mark what you've watched and keep notes.

Have a question about this one? Ask about this video →

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.