The limits of AI scaling laws - NVIDIA CEO explains | Jensen Huang and Lex Fridman

Jensen Huang explains that NVIDIA continues to believe in scaling laws for AI, outlining four dimensions of scaling: pre-training, post-training, test-time (inference), and agentic scaling. He reflects on historical blockers that industry experts predicted would limit AI progress. The first major blocker was data scarcity for pre-training, which some predicted would end AI scaling. However, Huang argues this concern was overcome through synthetic data generation, noting that most human knowledge is already 'synthetic' (created, modified, and regenerated by humans). He explains that training is now limited by compute rather than data, since most training data is synthetically generated.

Regarding test-time scaling and inference, Huang refutes the assumption that inference would be computationally simple compared to pre-training. He argues that inference involves thinking, reasoning, planning, and search—processes far more computationally intensive than the pattern-matching of pre-training. This scaling law, he contends, has proven correct: inference requires massive compute.

The fourth scaling law is agentic scaling, which Huang describes as 'multiplying AI.' Agents spawn sub-agents to accomplish tasks, creating teams of AI systems. He explains that agentic systems will access ground truth (file systems), conduct research, use tools, and communicate externally. These systems create enormous amounts of data and experiences that feed back into pre-training, creating a continuous improvement cycle.

Huang emphasizes the importance of anticipating hardware and system architecture needs 2-3 years in advance, since AI model architectures change every 6 months but hardware architectures take 3 years to develop. NVIDIA's strategy involves internal research, continuous dialogue with industry partners, and maintaining CUDA's balance between specialization and flexibility. He provides concrete examples: when mixture-of-experts emerged, NVIDIA anticipated this and developed NVLink-72 to handle massive models. Similarly, the Vera Rubin rack was designed to support agentic systems with tools, storage, and processing capabilities—a shift from the previous Grace-Blackwell architecture focused on inference.

Huang explains that he reasoned about agentic system requirements first-principles, drawing parallels to how a humanoid robot would use existing tools rather than transforming its hand into different tools. This reasoning led NVIDIA to design systems that support tool access, file systems, research capabilities, and external communication—properties that emerged in Claude's OpenClaw.

On supply chain challenges, Huang emphasizes that he actively works with upstream and downstream partners to shape the future. He describes convincing DRAM manufacturers to invest in HBM memory when its use was marginal, explaining why it would become mainstream. He also worked with manufacturers to adopt LPDDR5 memory (originally for cell phones) in supercomputers. The shift to building supercomputers in the supply chain rather than assembling them in data centers required partners to invest billions in manufacturing capability.

Regarding power consumption, Huang identifies energy efficiency (tokens per second per watt) as critical. NVIDIA has achieved a million-fold improvement in computing over 10 years compared to Moore's Law's 100-fold improvement. However, he emphasizes that absolute power availability is also necessary, proposing solutions involving grid dynamics and flexible power contracts.

Huang argues that power grids are designed for worst-case conditions but operate at 60% capacity 99% of the time. He proposes that data centers could gracefully degrade performance during peak grid demand, shifting workloads or accepting slightly longer latencies. He identifies three barriers: customer contracts demanding 'six nines' uptime, data centers not designed to gracefully degrade, and utilities not offering tiered power delivery guarantees. He suggests that if these three components align, data centers could use excess grid capacity efficiently without requiring massive new grid infrastructure.