Why Diffusion Will Win AI Inference with Inception Co-Founder and CEO Stefano Ermon

Stefano Ermon has spent his entire career researching generative models, starting at Stanford in 2014 when the field was nascent and difficult to publish in. His work evolved through autoregressive models, VAEs, and GANs before he and PhD student Yang Song developed score-based generative models in 2019, which became the foundation for modern diffusion models. Diffusion models work by training neural networks to denoise images, starting from pure noise and progressively refining to generate clean outputs—an approach that has since dominated image, video, and music generation.

Ermon's recent breakthrough came in 2024 when his lab demonstrated that diffusion models could match the quality of autoregressive models at GPT-2 scale (under 1 billion parameters) while generating text 10x faster. This success led him to co-found Inception to scale the technology commercially. The company, now about 2 years old with 50 people, is actively serving production customers including OpenAI in the voice agent space.

The core argument for diffusion's superiority centers on inference architecture. Autoregressive models generate tokens sequentially left-to-right, creating a memory-bound workload that doesn't map well to GPU parallelization—the computation bottleneck is moving weights through the memory hierarchy rather than arithmetic. Diffusion models, by contrast, process many tokens simultaneously during inference, mirroring the parallelism of training workloads and aligning perfectly with GPU capabilities. This architectural advantage compounds over time: as reasoning models scale test-time compute and RL post-training requires generating rollouts for exploration, inference efficiency becomes increasingly critical.

Ermon emphasizes that diffusion models offer additional advantages beyond speed. The coarse-to-fine iterative generation enables progressive constraint satisfaction, making these models fundamentally easier to control and steer than autoregressive alternatives—an external reward function can guide generation at each step rather than only being applied to final outputs. The academic literature suggests diffusion models may also be more data-efficient due to implicit data augmentation through noisy views during training.

Inception's business strategy combines product deployment with research and engineering. Rather than open-sourcing everything, the company maintains IP around its custom serving engine (necessary because existing frameworks like vLLM cannot run diffusion LLMs), inference acceleration techniques, RL post-training infrastructure, and domain-specific knowledge from customer feedback. The models maintain backward compatibility with OpenAI's API, using standard transformer architecture and public datasets. Inception estimates that 20-30% of AI workloads have latency requirements where speed-optimized models provide the highest quality within a given budget.

When asked about competitive threats from larger labs with greater resources, Ermon argues that building end-to-end production systems creates defensible IP around serving infrastructure, customer insights, and training recipes that are harder to replicate than published research alone. He also points out that custom hardware approaches (like Cerebras) can be displaced by diffusion-based software solutions running on commodity GPUs, improving accessibility and reducing costs while matching performance.

Ermon discusses the evolution from academia to commercialization, noting that his lab's foundational work on diffusion, flash attention, and DPO algorithms all emerged from academic freedom to take contrarian bets. He argues that major AI innovations often have academic roots and that universities provide unique advantages: access to talented students, reduced pressure to follow mainstream approaches, and freedom to explore unconventional directions. While acknowledging that compute resources are always limiting factors, he contends that clever mathematical insights and novel problem-solving approaches remain critical and can't be simply purchased with scale.