Google's Gemini, OpenAI's GPT, DeepSeek's R1 or Llama models — it doesn't care which of this LLMs you choose, they all suck. YEP! Old tech. They all generate one token at a time... But we have models available that can generate all the output tokens at once. That's right, it doesn't matter how fast you read, you don't have to wait to read the next word.
But... how is this possible? Traditional
LLMs use a transformer based architecture. Think of them like bricklayers, carefully placing one brick after the other. They get a text and they will calculate the probability for the next token, add it to the initial text, and do the same again for the nexty token.
Diffusion models, on the other hand, are like SLA 3D printers, they project an image on the resin to create entire objects. They start with a noisy text that looks random, but after a few interactions it's refined until it becomes Don Quixote.
This coarse-to-fine
approach allows diffusion models to generate text at lightning speed, making them perfect for tasks that require quick responses, like chatbots, code generation, and real-time translation.
Mercury: The Dawn of a New Era
Inception Labs'MErcury is the first commercial-scale diffusion LLM. Some early benchmarks suggest it can rival the performance of leading transformer models while being significantly faster and more cost-effective. Mercury coder can generate over 1,000 tokens per sec on NVIDIA H100s! This incredible speed was demonstrated in head-to-head coding evaluations, where Mercury Coder matched or even outperformed speed-optimized models like OpenAI's GPT-4o Mini and Anthropic's Claude 3.5 Haiku, all while running at a fraction of their latency.
This is a game-changer. Diffusion LLMs have the potential to revolutionize the way we interact with AI, making it more accessible, more powerful, and more useful than ever before.
So keep an eye out for Mercury and the rise of diffusion LLMs. The AI world is about to get a whole lot more interesting!
Conclusion
Diffusion LLMs represent a significant advancement in language modeling, offering a compelling alternative to traditional transformer-based approaches. With their speed, efficiency, and potential for new capabilities like improved reasoning, error correction, and continuous learning, they are poised to revolutionize the way we interact with AI. As diffusion models continue to evolve and overcome existing challenges, they promise to unlock exciting new possibilities for the future of language technology, leading to more powerful, versatile, and accessible AI systems.
You can test yourself and be amazed with the speed of diffusion LLMs using Inception Labs' playground here.