How NVIDIA works - explained by NVIDIA CEO | Jensen Huang and Lex Fridman
Jensen Huang explains NVIDIA's evolution from a GPU company to an extreme co-design platform company, discussing the necessity of optimizing across the entire computing stack due to distributed computing challenges. He details how NVIDIA's organizational structure mirrors its product philosophy, with 60 direct reports enabling comprehensive system optimization.
Summary
The conversation explores NVIDIA's transformation from chip-scale to rack-scale design through what Huang calls 'extreme co-design.' This approach became necessary because modern AI problems no longer fit inside single computers and require massive distribution across thousands of systems. When distributing workloads this way, every component becomes a bottleneck - CPU, GPU, networking, switching, memory, power, and cooling - requiring simultaneous optimization across all layers. Huang explains that NVIDIA's organizational structure reflects this philosophy, with his 60 direct reports being specialists in different domains who collaborate on every decision rather than working in silos. This eliminates traditional one-on-one meetings in favor of group problem-solving sessions where experts in cooling, networking, memory, and other areas can immediately identify conflicts or synergies. Huang traces NVIDIA's strategic evolution through key decisions, particularly the costly but crucial choice to put CUDA on GeForce GPUs despite it consuming all company profits and reducing market cap from $8 billion to $1.5 billion. This decision was driven by the understanding that install base is everything for a computing platform - more important than architectural elegance. The move established CUDA's foundation by putting it in millions of gaming PCs, making it accessible to researchers and students who became early adopters. Huang describes his leadership approach as gradually shaping belief systems across the organization, industry partners, and broader ecosystem through consistent messaging over years. Rather than announcing sudden strategic pivots, he plants ideas and builds consensus incrementally, so that when major announcements come, they feel inevitable rather than surprising. He emphasizes that NVIDIA doesn't build end products but creates computing platforms that others integrate, requiring extensive ecosystem coordination and future manifesting.
Key Insights
- Huang argues that extreme co-design became necessary because modern AI problems require distributing workloads across thousands of computers, making every system component from networking to cooling a potential bottleneck that must be optimized simultaneously
- Huang structures NVIDIA with 60 direct reports who are specialists in different domains, eliminating one-on-one meetings in favor of group problem-solving where all experts can immediately identify cross-component conflicts and synergies
- Huang claims that putting CUDA on GeForce GPUs was an existential risk that consumed all company profits and crashed NVIDIA's market cap from $8 billion to $1.5 billion, but was necessary because install base determines platform success more than architectural elegance
- Huang explains his leadership approach involves gradually shaping belief systems across the organization and industry over years through consistent messaging, so major strategic announcements feel inevitable rather than surprising when they come
- Huang asserts that NVIDIA succeeds as a platform company by vertically integrating design and optimization but then opening every layer to partners, requiring extensive ecosystem coordination since customers cannot buy complete solutions directly from NVIDIA
Topics
Full transcript available for MurmurCast members
Sign Up to Access