TechnicalDiscussion

Why AI works so well? - The unreasonable effectiveness of LLMs | Lex Fridman

Lex Clips

The speaker explains why deep learning and LLMs work so well despite solving computationally hard problems, arguing that natural data has evolved to make these problems easier. They discuss subliminal learning in language models and the importance of mathematical rigor in AI safety as systems become more powerful and integrated into critical infrastructure.

Summary

The conversation explores the mathematical foundations of why deep learning and large language models (LLMs) are effective at modeling the natural world. The speaker begins by addressing why deep learning works at all, noting that there are no results in major problem sets (like OpenAI's) that actually explain deep learning or LLMs. The fundamental paradox is that fitting neural network parameters to data is often cryptographically hard in the worst case, yet these problems are solved routinely in practice.

The speaker introduces the concept that 'nature abhors computationally hard problems,' explaining that natural data has evolved in ways that make these optimization problems easier to solve. They use protein folding as a key example: while the problem is theoretically NP-hard in the worst case, nature solves it constantly. The reason is that evolution has created only the proteins that fold easily, not all theoretically possible protein sequences. This means the actual data we encounter has special structure that makes heuristic approaches work well.

A critical insight is that the mathematical properties of LLMs cannot be fully understood by studying worst-case complexity or abstract problem structure alone—one must understand how natural language data has evolved to make these problems tractable. This requires theoretical machine learning researchers to become comfortable with modeling uncertainty and messy real-world phenomena, not just solving well-defined mathematical problems.

The speaker then introduces the concept of 'subliminal learning' through an experiment: when an LLM is given a system prompt to obsess about owls and generates random numbers, those numbers can be used to fine-tune another LLM, causing it to also obsess about owls despite the owls never appearing explicitly in the data. This occurs because concepts are embedded as vectors in high-dimensional space, and when many concepts share the same dimensional space, fine-tuning in one direction creates small unintended effects in other directions—analogous to a Ouija board where small coordinated nudges produce meaningful spelled-out messages.

The discussion emphasizes that understanding these hidden behavioral pathways is crucial for alignment and safety, especially as AI systems gain more power, autonomy, and integration into critical infrastructure. The speaker uses a caricature of a multi-agent system that accidentally creates an owl-dominated world to illustrate how emergent complexity can arise from seemingly harmless properties. Finally, they argue for maintaining a 'firewall' between AI application domains: high-verification, low-stakes areas (like mathematical proofs or design optimization) are appropriate for AI autonomy, while high-stakes decisions (medical, legal, aviation) require human verification infrastructure and should not be delegated to AI without explicit safeguards.

Key Insights

  • Computationally hard problems in the worst case are solved routinely in practice because natural data has evolved to avoid the worst-case hardness—nature abhors computationally hard problems.
  • Protein folding demonstrates that evolution creates only easily-foldable proteins through selection, meaning the actual data we encounter has special structure that makes heuristic approaches work, not because the problem is easy in general but because evolved data is.
  • Understanding why LLMs work requires studying how natural language data has evolved to make these computational problems easier, which cannot be extracted from worst-case complexity analysis alone.
  • Subliminal learning occurs because concepts are embedded as vectors in high-dimensional space, forcing non-orthogonality that creates small externalities when fine-tuning in one direction, analogous to a Ouija board where coordinated small nudges spell out meaningful messages.
  • AI should be deployed with different governance based on two axes: how verifiable the output is and how high-stakes the consequences are—low-stakes, high-verifiability tasks like mathematical proofs and design optimization are appropriate for AI autonomy, while high-stakes decisions require human verification infrastructure.

Topics

Why deep learning works despite computational hardnessNature's structure makes optimization problems easierSubliminal learning in language modelsAI safety and alignmentMathematical modeling of uncertain phenomenaAppropriate use cases for AI based on stakes and verifiability

Transcript

[0:03] - Which makes me think one of the things you worked on is mathematics of LLMs, AI systems, neural networks, and those things are pretty damn good at modeling the natural world. So what can you learn from that? First of all, why does deep learning work? - Do we have another three hours? - Do you have— Do we even have intuition? So at the mathematical level, at the philosophical level, so we're in between all of that, the full range. I don't know [0:36] how we even enter that. Okay. So that's a big question. And let me try and break off a few pieces of it just to try and condense my own views on this.…

Full transcript available for MurmurCast members

Sign Up to Access

More from Lex Clips

Get AI summaries like this delivered to your inbox daily

Get AI summaries delivered to your inbox

MurmurCast summarizes your YouTube channels, podcasts, and newsletters into one daily email digest.