Redefining the Token Efficiency Frontier with Diffusion LLMs

TL;DR

• One thread runs through all of AI: parallelization. Late-90s hardware parallelized matrix multiplication; the Transformer parallelized training (escaping the RNN/LSTM ceiling on how much you could learn per dollar of compute); diffusion displaced GANs in vision; and the current wave brings that idea to text by parallelizing decoding.
• Text is hard for diffusion because diffusion was invented for continuous modalities. Images have a natural notion of adding noise and a sound mathematical theory of principled denoising. Text is discrete, which makes "what is a good noising and denoising process, trainable at scale" fundamentally difficult.
• The commercial story is what speed buys you. On Artificial Analysis's chart, Mercury matches the quality of comparable speed-optimized models (Claude Haiku, GPT-5 mini) while being much faster — and Mercury 2 reasons and is still faster than non-reasoning models like GPT-4.1, which means you can deploy a reasoning model where only a non-reasoning one used to fit.

Key Points

The thread: parallelization as AI's first principle (~00:42–00:44)

Grover opened by saying he'd pick up elements from the preceding talks to define what he believes is the new foundation for generative AI: something extremely fast and efficient built on diffusion. At the heart of every advance he cites is one belief — parallelization is a fundamental concept for computer science, for AI, and for what comes next.

Three inflection points: in the late 1990s, hardware could finally execute a simple but basic operation, matrix multiplication, in parallel, which turned out to be the defining moment for today's generative AI. A few years later, the Transformer parallelized training, taking the field out of the RNN/LSTM world that was fundamentally bottlenecked by how much could be learned per dollar of compute and into an era of massive language models. And on the algorithm side, diffusion replaced GANs as the dominant way to generate images and video efficiently and at high fidelity.

Why text is hard, and how Mercury works (~00:44–00:46)

Diffusion was invented for continuous modalities: for images there's a natural notion of what it means to add noise and a good mathematical theory of principled denoising. Text is discrete, which makes it fundamentally hard to define noising and denoising processes that can be trained at scale.

Inception's approach: instead of generating tokens sequentially, one at a time — the autoregressive paradigm essentially every language model uses today — start from something completely gibberish, pass it through a neural network (any architecture; they use Transformers), and train that network to fix the noise in its input. That's how the model finds structure in text and produces something coherent in far less time, because it predicts all the denoised tokens in parallel.

A timeline of text diffusion (~00:46–00:48)

Denoising diffusion took off for images around 2019, when Midjourney and labs including Google and OpenAI demonstrated large-scale, high-quality image generation. It took years of work by the whole research community to reach a breakthrough in one of his co-founders' labs: a diffusion model for text that reached parity with GPT-2. By 2024, GPT-2 was no longer a meaningful bar for language, which is when they formed Inception to take the idea to scale.

A few months later they launched Mercury, the first commercial-scale diffusion language model — capable of code editing and generation, mathematical problem solving, and common sense at scale, and extremely fast. It caught the attention of notable people in both academia and industry, and within months labs from Google to Nvidia to Alibaba followed with their own diffusion LLM efforts. Earlier in 2026 came Mercury 2, which adds reasoning — and diffusion reasoning looks very different from how it works for autoregressive LLMs.

Reading the Artificial Analysis chart (~00:48–00:49)

X-axis: speed in tokens per second, higher is better. Y-axis: Artificial Analysis's quality index across a wide range of agentic benchmarks. The comparison set is the speed-optimized regime from the frontier labs — the Claude Haikus and GPT-5 minis of the world. Mercury lands at similar quality for models of similar size while being much, much faster, precisely because it denoises tokens in parallel rather than generating them sequentially.

Three case studies for what speed buys (~00:49–00:53)

• Voice and support agents. What matters isn't intelligence alone but intelligence plus response time — and time to first token is critical for customer-support voice applications. Mercury defines a different frontier: strong intelligence at very low time-to-first-token. The sharper point: Mercury 2 reasons and is still faster than GPT-4.1, the non-reasoning model in production at most voice companies. "That's what speed buys you — you can deploy reasoning models at the speed of, or better than, non-reasoning models."
• A hardware footnote. All their deployments run on Nvidia hardware; the speedup comes purely from diffusion and parallel token generation. So complementary advantages from newer hardware like Cerebras and Groq could stack on top of a diffusion model for text.
• Search. Here cost matters alongside speed. Two plots (accuracy vs. speed, accuracy vs. cost) make the same point: a production-deployable model needs a good combination of all three, and Mercury strikes a good balance in that space.
• Coding. "I can't imagine my life now without using coding agents." But the primary agent is often extremely verbose and token-hungry, and that isn't always what you need. To drive token economics toward something sustainable, companies like Augment Code use Mercury alongside heavyweight models like Opus, getting a good balance of latency and cost.

Closing: intelligence per watt (~00:53)

"If you think about the frontier of intelligence, we've reached a point where we have extremely good intelligence for a lot of applications. But now what's really important, as we build value for everyone around us, is to think about the new currency of intelligence per watt — and diffusion LLMs seem like a really good bet to redefining that frontier."

Quotes

"Parallelization is a fundamental concept for computer science, fundamental concept for AI, and a fundamental belief that will also guide towards the future." (~00:42)

The spine of the talk, and how he connects GPUs, Transformers, and diffusion into one line.

"Mercury 2 can do reasoning and still be faster than a GPT [4.1] model. So that's what speed buys you — you can actually deploy reasoning models at the speed of, or even better than, non-reasoning models." (~00:51)

The clearest statement of why speed is a business argument, not a convenience one: it unlocks capability that previously didn't fit.

"The new currency of intelligence per watt — and diffusion LLMs seem like a really good bet to redefining that frontier." (~00:53)