How NVIDIA works - explained by NVIDIA CEO | Jensen Huang and Lex Fridman
Jensen Huang explains NVIDIA's evolution from a GPU company to an extreme co-design platform company, discussing the necessity of optimizing across the entire computing stack due to distributed computing challenges. He details how NVIDIA's organizational structure mirrors its product philosophy, with 60 direct reports enabling comprehensive system optimization.
Summary
The conversation explores NVIDIA's transformation from chip-scale to rack-scale design through what Huang calls 'extreme co-design.' This approach became necessary because modern AI problems no longer fit inside single computers and require massive distribution across thousands of systems. When distributing workloads this way, every component becomes a bottleneck - CPU, GPU, networking, switching, memory, power, and cooling - requiring simultaneous optimization across all layers. Huang explains that NVIDIA's organizational structure reflects this philosophy, with his 60 direct reports being specialists in different domains who collaborate on every decision rather than working in silos. This eliminates traditional one-on-one meetings in favor of group problem-solving sessions where experts in cooling, networking, memory, and other areas can immediately identify conflicts or synergies. Huang traces NVIDIA's strategic evolution through key decisions, particularly the costly but crucial choice to put CUDA on GeForce GPUs despite it consuming all company profits and reducing market cap from $8 billion to $1.5 billion. This decision was driven by the understanding that install base is everything for a computing platform - more important than architectural elegance. The move established CUDA's foundation by putting it in millions of gaming PCs, making it accessible to researchers and students who became early adopters. Huang describes his leadership approach as gradually shaping belief systems across the organization, industry partners, and broader ecosystem through consistent messaging over years. Rather than announcing sudden strategic pivots, he plants ideas and builds consensus incrementally, so that when major announcements come, they feel inevitable rather than surprising. He emphasizes that NVIDIA doesn't build end products but creates computing platforms that others integrate, requiring extensive ecosystem coordination and future manifesting.
Key Insights
- Huang argues that extreme co-design became necessary because modern AI problems require distributing workloads across thousands of computers, making every system component from networking to cooling a potential bottleneck that must be optimized simultaneously
- Huang structures NVIDIA with 60 direct reports who are specialists in different domains, eliminating one-on-one meetings in favor of group problem-solving where all experts can immediately identify cross-component conflicts and synergies
- Huang claims that putting CUDA on GeForce GPUs was an existential risk that consumed all company profits and crashed NVIDIA's market cap from $8 billion to $1.5 billion, but was necessary because install base determines platform success more than architectural elegance
- Huang explains his leadership approach involves gradually shaping belief systems across the organization and industry over years through consistent messaging, so major strategic announcements feel inevitable rather than surprising when they come
- Huang asserts that NVIDIA succeeds as a platform company by vertically integrating design and optimization but then opening every layer to partners, requiring extensive ecosystem coordination since customers cannot buy complete solutions directly from NVIDIA
Topics
Transcript
You've propelled NVIDIA into a new era in AI, moving beyond its focus on chip-scale design to now rack-scale design. And I think it's fair to say that winning for NVIDIA for a long time used to be about building the best GPU possible. And you still do. But now you've expanded that to extreme co-design of GPU, CPU, memory, networking, storage, power, cooling, software. The rack itself, the pod that you've announced, and even the data center. So let's talk about extreme co-design. What is the hardest part of co-designing a system with that many complex components and design variables? Yeah, thanks for that question. So first of all, the reason why extreme co-design is necessary is because the…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Do antipsychotic drugs work? | Andrew Scull and Lex Fridman
Andrew Scull discusses the efficacy and serious side effects of antipsychotic drugs, explaining that while they help some patients, many are non-responders, and the medications carry substantial risks including tardive dyskinesia, weight gain, metabolic syndrome, and cognitive dulling. The landmark CATIE study revealed that newer, expensive second-generation antipsychotics are no more effective than older first-generation drugs, with 67-82% of patients dropping out due to inefficacy or intolerable side effects.
Surprising origin of antipsychotic drugs | Andrew Scull and Lex Fridman
Andrew Scull explains how chlorpromazine, an antihistamine synthesized in the 1880s, was accidentally discovered to have psychiatric applications in the 1950s through French Navy lieutenant Alain Laborit's experiments. The drug's rapid adoption was driven by pharmaceutical companies' aggressive marketing to politicians and hospital administrators rather than by psychiatrists' initial enthusiasm, transforming it from a 'major tranquilizer' into the first 'antipsychotic' drug.
Cognitive Behavioral Therapy (CBT) vs Psychoanalysis | Andrew Scull and Lex Fridman
Andrew Scull discusses how clinical psychology and cognitive behavioral therapy (CBT) emerged after WWII as alternatives to psychoanalysis, becoming dominant through better empirical evidence, shorter treatment duration, and federal funding advantages. CBT's symptom-focused approach proved more measurable and reproducible than psychoanalytic theory, though its effectiveness remains limited for serious mental disorders.
Do anti-depressant drugs work? | Andrew Scull and Lex Fridman
Andrew Scull discusses the history and efficacy of antidepressants, particularly SSRIs like Prozac, explaining that while they statistically outperform placebo, the clinical improvement is often marginal and comes with significant side effects including sexual dysfunction and withdrawal difficulties. He also addresses 'diagnostic creep' in psychiatry, where conditions become increasingly broadly defined and diagnosed over time.
How drugs and psycho-pharmacology changed mental health | Andrew Scull and Lex Fridman
Andrew Scull traces the accidental discovery of psychopharmacology, beginning with chlorpromazine's unexpected psychiatric effects in the 1950s, and discusses how pharmaceutical companies transformed mental health treatment while also introducing serious side effects like tardive dyskinesia and metabolic complications that remain inadequately addressed.