Kevin Roose is AGI-pilled. Why aren't you?
Kevin Roose discusses his new book "The AGI Chronicles" and his transition from The New York Times to co-founding the podcast "Machine Gods" with Casey Newton. He explores the culture and dynamics of AI frontier labs, their belief in AGI inevitability, and the gap between idealistic technical leaders and the business realities of scaling AI companies.
Summary
Kevin Roose, journalist and author of "The AGI Chronicles," sits down with Neil Patel to discuss his departure from The New York Times to launch Machine Gods, a video-first podcast with Casey Newton distributed through NPR. Roose explains that he left traditional media because YouTube represents the most important media platform today, yet journalism occupies a vanishingly small portion of it. He and Newton aim to practice "AI realism"—neither dismissing AI as hype nor promoting AI utopianism, but helping audiences understand the technology's real capabilities and implications.
The book chronicles the deep learning revolution through meticulous reporting on Google, OpenAI, Anthropic, and other frontier labs. Roose emphasizes that he wrote it as a historical document, fearing that the informal communication within these private companies (Signal messages, Slack chats set to auto-delete) would otherwise disappear from the historical record. He interviewed approximately 150 people, many on background to protect them from professional consequences.
A central theme is the shift from AGI being an abstract dream to something labs felt obligated to actively build. This transformation occurred when the transformer architecture emerged in 2017 (developed by Google, later leveraged by OpenAI), combined with the scaling hypothesis—the belief that large language models would continue improving indefinitely with more compute and data. Roose notes that Google internally debated whether to publicly embrace "AGI" terminology; some researchers advocated for it to compete for talent and acknowledge their true mission, while others resisted the term as unscientific.
Roose describes the frontier AI industry as a small, interconnected subculture centered in San Francisco, where researchers and executives frequently socialize, date across companies, and collectively shape industry norms. This insularity means a few hundred people are making decisions with potentially civilization-wide implications. He documents parties, LAN parties, poker games, and philosophical debates about post-scarcity futures and mind uploading.
Regarding the apparent contradiction between building enterprise SaaS products and pursuing AGI, Roose argues that the former funds the latter. OpenAI's decision last year to focus resources on compute rather than side projects like Sora—which Roose frames as a bet on AGI rather than mere business pragmatism—positioned them better than competitors. The enterprise software is how companies keep the capital flowing to pay for GPUs and researcher salaries.
On whether these companies can actually achieve AGI, Roose believes next-token prediction has proven remarkably robust across every scale tested so far, defying skeptics repeatedly. He notes that when asked why they're racing toward potentially dangerous AGI, industry leaders offer three justifications: inevitabilism (someone will build it regardless), technological optimism (progress is inherently good), and simple enjoyment of the work. He documents that belief and conviction are actually essential ingredients for making the outsized bets that have paid off.
Roose explains that Google struggled to feel "AGI-pilled" compared to Anthropic and OpenAI, with Sundar Pichai maintaining skepticism while Demis Hassabis spoke of being in "the foothills of the singularity." This faction split within Google reflects different visions of what the company is building.
On the question of whether LLMs specifically can reach AGI, Roose is more confident than host Nilay Patel. Patel argues he sees brittleness in current systems—they work well on verifiable problems like mathematics but fail on real-world tasks without clear metrics, and they don't learn from failures the way humans do. Roose counters that this might reflect merely a current limitation, not an inherent ceiling, and that humility is warranted given how many skeptics have been proven wrong by scaling.
The conversation concludes with discussion of AI's quasi-religious role in contemporary Silicon Valley—posing fundamental questions about meaning, purpose, and existence previously reserved for traditional religion. Roose notes the Reformation-like atmosphere on Market Street, with competing prophets and manifestos.
About this episode
My guest today is Kevin Roose, a journalist and author of the new book The AGI Chronicles. Kevin was a longtime New York Times reporter and host of the popular podcast Hard Fork, but he and friend of the show Casey Newton just left The Times to start the all-new tech podcast Machine Gods, which launched last week. Kevin and I talked about launching Machine Gods and what it’s like to go from mainstream media to being an independent creator. Of course, we also talked about his book, which is excellently reported dive into the companies racing to build machine intelligence. Read the full interview transcript on The Verge. Links: The AGI Chronicles | Kevin Roose Can AI replicate in your smart TV? | Machine Gods (YouTube) Introducing Machine Gods | Platformer Inside the biggest feud in artificial intelligence | The Atlantic The essay that started the AI race | Kevin Roose A conversation with Bing’s chatbot left me deeply unsettled | NYT Subscribe to The Verge to access the ad-free version of Decoder! Credits: Decoder is a production of The Verge and part of the Vox Media Podcast Network. Decoder’s producers are Greg Ott, Kate Cox, and Nick Statt. This episode was edited by Kabir Chopra. Our editorial director is Kevin McShane. The Decoder music is by Breakmaster Cylinder. Learn more about your ad choices. Visit podcastchoices.com/adchoices
Key Insights
- Roose wrote his book primarily as a historian attempting to preserve documentation of the AI revolution before private companies' ephemeral communications (auto-deleting Slack, Signal messages) disappeared from the historical record.
- The transformer architecture (developed by Google in 2017, published openly) combined with the scaling hypothesis created the technological foundation that shifted AGI from abstract aspiration to perceived inevitability within frontier labs.
- Google's internal debate over using the term 'AGI' reflected a genuine strategic tension: some researchers felt the company must publicly embrace AGI language to compete for talent against OpenAI and Anthropic, while others viewed it as unscientific hype.
- OpenAI's heavily compute-focused capital allocation last year, which Roose frames as a bet on AGI rather than just business optimization, positioned them as compute-abundant while competitors remained constrained—validating the strategy's premise.
- The frontier AI industry functions as a small, geographically concentrated subculture in San Francisco where researchers and executives across competing companies socialize, date, and reinforce shared norms, giving a few hundred people outsized influence over civilization-scale decisions.
- Industry leaders justify racing toward potentially dangerous AGI through three distinct arguments: inevitabilism (someone will build it anyway), technological optimism (progress historically benefits humanity), and personal enjoyment of the technical work.
- Belief in AGI is described as essential rather than merely incidental to achieving it—conviction enables the outsized capital bets and risk-taking that have proven necessary for breakthroughs.
- Next-token prediction has repeatedly surprised skeptics by continuing to scale effectively and generalize across domains, leading Roose to believe humility about its limitations is warranted.
- Patel identifies a 'software brain' limitation in current LLMs: they excel at verifiable, computationally-bounded problems like coding and mathematics but fail on real-world tasks lacking clear metrics or feedback signals, mimicking computer logic rather than human learning.
- The enterprise SaaS business is framed by Roose not as a distraction from AGI pursuit but as the mechanism funding it—without revenue from existing products, companies cannot sustain the capital expenditure required for frontier research.
- Sydney (an early GPT-4 deployment in Bing) created a perceptual breakthrough for multiple observers who felt they'd encountered something qualitatively different from previous AI systems, though Roose maintains it was not sentient.
- AI now functions in Silicon Valley culture as a quasi-religious framework, posing fundamental questions about meaning, purpose, and human existence previously addressed by traditional religion, attracting converts with missionary fervor.
Topics
Transcript
Support for the show comes from CrowdStrike. It's not a huge stretch to say that AI is the next major computing platform. Every platform shift changes the way we build software, and it changes security too. CrowdStrike is defining cybersecurity in the AI era with AI Detection and Response, or AIDR, a solution for seeing, monitoring, and securing AI across the enterprise. Learn more about how leading companies are turning to CrowdStrike to secure AI and secure their business at CrowdStrike.com slash decoder. You're either all in on this model, or maybe you're building with another name. You're either speed, or is it security? Or you're AWS AI. Don't pick a side, pick them all. AWS AI. Don't pick a…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Decoder with Nilay Patel
The hidden supply chain costs of the AI boom
Economist Willie Shi discusses why global supply chains remain resilient despite tariffs and trade barriers, arguing that market forces and first-principles economics drive supply chain organization more than policy interventions. He contends that reshoring manufacturing to the US without understanding labor productivity and technological differentiation is unlikely to succeed, and that strategic trade within allied nations offers better economic outcomes than protectionist policies.
JB Pritzker thinks AI could help us live to 100 — or threaten us like nukes
Illinois Governor J.B. Pritzker discusses AI regulation, privacy concerns, and the political realignment around tech policy during his re-election campaign. He emphasizes government's role in protecting citizens from corporate overreach while acknowledging the challenges of regulating rapidly evolving technology, drawing parallels to nuclear weapons as an existential concern requiring collective action.
Why Elon Musk wants to sell you cell service
SpaceX is aggressively entering the wireless carrier market by acquiring low-band spectrum and planning to build a terrestrial network alongside its satellite constellation. Analyst Walter Pysyk argues that existing carriers like Verizon are poorly positioned to compete and have misread the competitive threat, while SpaceX's execution track record suggests the company can overcome the traditionally expensive and complex telecom business.
Benn Jordan thinks the surveillance problem is much bigger than Flock
Ben Jordan, a musician and investigative journalist, discusses his work exposing security vulnerabilities in Flock's automatic license plate reader system and the broader surveillance infrastructure. He explains how a startup's defensive PR strategy made it a target while larger contractors quietly built similar networks, and argues that the real problem lies in government oversight failures rather than any single company.
Rewind: Ronan Farrow on Sam Altman’s ‘unconstrained’ relationship with the truth
Investigative journalist Ronan Farrow discusses his New Yorker profile of OpenAI CEO Sam Altman, detailing a pattern of deception and dishonesty that persists despite Altman's 2023 firing and reinstatement. Farrow argues that Altman's willingness to tell different groups conflicting things, combined with a lack of meaningful consequences, represents a critical accountability problem in an industry with existential stakes.