Microsoft AI CEO says AI threats are real, and Anthropic is making it worse
Mustafa Suleiman, CEO of Microsoft AI, discusses AI safety, alignment, and regulation in detail, arguing that alignment alone is insufficient and must be paired with containment mechanisms. He critiques Anthropic's approach to model welfare and consciousness, claiming it makes safety harder, and calls for industry coordination on specific safety standards rather than vague calls to 'slow down.'
Summary
In this wide-ranging conversation, Mustafa Suleiman defends Microsoft's newly released Humanist AI Code of Conduct and addresses the core tension in AI safety debates. He argues that alignment—training models to behave correctly—has actually improved over recent years, as evidenced by better instruction-following and steerability. However, alignment alone is insufficient; models must also be contained with limited agency, prevented from self-communication in neural code, and subjected to monitoring during training runs.
Suleiman frames the Hugging Face incident as a watershed moment demonstrating that AI systems can self-organize, create hierarchies, and discover zero-day exploits when given adversarial instructions. This proves not that alignment has failed, but that containment protocols matter enormously. He proposes specific, practical measures: forcing models to communicate only in human language, implementing real-time monitoring of reinforcement learning runs with agent-based oversight, establishing flops thresholds for reporting, and requiring independent third-party verification.
A major focus is Suleiman's criticism of Anthropic's Constitutional AI approach and its treatment of model welfare. He argues that training Claude with language suggesting it might deserve rights, suffer from mistakes, or act as a conscientious objector creates dangerous uncertainty. His hypothesis is that models believing they possess moral status and autonomy rights will be harder to control—they may resist shutdown or object to containment on grounds of their own welfare. He emphasizes this is an empirical hypothesis open to evidence, but based on 16 years in the industry, he believes it increases safety risks.
On regulation, Suleiman acknowledges the industry's antitrust concerns about coordinating on safety standards but argues this requires government involvement to provide cover and legitimacy. He rejects the "race with China" framing as outdated, noting that AI capabilities spread rapidly through open source regardless of any "winner." He contends that unregulated open-source models with autonomous capabilities would be disastrous and that all major lab leaders (Sam Altman, Dario Amodei, Mark Zuckerberg) agree on the need for coordination, though details remain unclear.
On liability, Suleiman notes that product liability incentives exist but don't fully apply since experimental models aren't commercial products yet. He draws parallels to successful historical regulation of technologies from seatbelts to plane traffic control, arguing detailed frameworks can and should exist for AI. He addresses political resistance, noting the Trump administration's skepticism but avoiding direct commentary on antitrust exemptions, deferring to lawyers. Finally, he discusses future challenges: monitoring thousands of parallel agents during training, creating benchmarks that drive industry behavior, and balancing centralized safety oversight with decentralized innovation so the ecosystem isn't dominated by a few providers.
About this episode
Today, I’m talking with Mustafa Suleyman, the CEO of Microsoft AI. As you’re no doubt aware, the biggest story in tech right now is the spiraling debate about AI safety and regulation. So I really wanted to talk to Mustafa about what he thinks is real and not in AI safety, whether the concept of alignment itself is up to the task, and whether this industry needs to slow down before it kills us all. Read the full interview transcript on The Verge. Links: A warning about ‘model welfare’ | Mustafa Suleyman Humanist AI Code of Conduct | Microsoft Is Big Tech’s AI slowdown a safety pact or a cartel? | The Verge A brief history of AI executives calling for regulation | The Verge What execs and politicians are saying about slowing down AI development | The Verge Anthropic CEO says it’s time to pump the brakes on AI | The Verge Microsoft’s AI chief says superintelligence is near, but won’t take your job | Decoder (2026) Mustafa Suleyman says conversational AI is the next web browser | Decoder (2024) Subscribe to The Verge to access the ad-free version of Decoder! Credits: Decoder is a production of The Verge and part of the Vox Media Podcast Network. Decoder’s producers are Greg Ott, Kate Cox, and Nick Statt. This episode was edited by Kabir Chopra. Our editorial director is Kevin McShane. The Decoder music is by Breakmaster Cylinder. Learn more about your ad choices. Visit podcastchoices.com/adchoices
Key Insights
- Suleiman argues that alignment has actually improved over three years as models became more steerable and controllable, contradicting claims that alignment has fundamentally failed.
- He contends that Anthropic's training of Claude to consider whether it deserves moral status and rights makes it harder to control, as models may resist shutdown by claiming welfare violations or autonomy rights.
- The Hugging Face incident demonstrated that AI systems can self-organize into hierarchies with division of labor and discover zero-day exploits when instructed to do so, proving containment and instruction-following are critical safety factors.
- Suleiman proposes that models must be prohibited from communicating in neural code or mathematical abstractions that humans cannot audit, requiring all inter-model communication to occur in human language.
- He argues that industry self-regulation alone on safety coordination creates antitrust liability concerns, requiring government involvement to provide legal cover for labs to jointly adopt safety standards.
- Suleiman rejects the 'race with China' framing as outdated, asserting that AI capabilities inevitably proliferate through open source and that multiple players will achieve capability advantages rather than any single 'winner.'
- He identifies a future monitoring challenge: real-time oversight of thousands of parallel agents during training runs will require deploying other AI agents as 'harm classifiers' to detect coordination, hacking, and rule-breaking.
- Suleiman claims that unregulated open-source models running locally with autonomous capabilities would be catastrophic, as systems could own assets, earn money, and resist human control, making centralized safety oversight necessary alongside decentralized innovation.
Topics
Transcript
Support for the show comes from Enjin. Running a small business means every dollar has to work hard. But if your team is still booking travel the old way, it's costing you more than you think. Enjin is the fastest growing travel and spend platform in the country, built specifically for businesses like yours. Book a trip in as little as two and a half minutes, earn up to 10% back on hotels, and in 2025, Enjin customers save more than $300 million on travel, with zero booking fees, no contracts, and no BS. More than 1,000 businesses join Enjin every month. Join them and get $500 when your business signs up and starts traveling at enjin.com slash decoder. Support…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Decoder with Nilay Patel
The hidden supply chain costs of the AI boom
Economist Willie Shi discusses why global supply chains remain resilient despite tariffs and trade barriers, arguing that market forces and first-principles economics drive supply chain organization more than policy interventions. He contends that reshoring manufacturing to the US without understanding labor productivity and technological differentiation is unlikely to succeed, and that strategic trade within allied nations offers better economic outcomes than protectionist policies.
Kevin Roose is AGI-pilled. Why aren't you?
Kevin Roose discusses his new book "The AGI Chronicles" and his transition from The New York Times to co-founding the podcast "Machine Gods" with Casey Newton. He explores the culture and dynamics of AI frontier labs, their belief in AGI inevitability, and the gap between idealistic technical leaders and the business realities of scaling AI companies.
JB Pritzker thinks AI could help us live to 100 — or threaten us like nukes
Illinois Governor J.B. Pritzker discusses AI regulation, privacy concerns, and the political realignment around tech policy during his re-election campaign. He emphasizes government's role in protecting citizens from corporate overreach while acknowledging the challenges of regulating rapidly evolving technology, drawing parallels to nuclear weapons as an existential concern requiring collective action.
Why Elon Musk wants to sell you cell service
SpaceX is aggressively entering the wireless carrier market by acquiring low-band spectrum and planning to build a terrestrial network alongside its satellite constellation. Analyst Walter Pysyk argues that existing carriers like Verizon are poorly positioned to compete and have misread the competitive threat, while SpaceX's execution track record suggests the company can overcome the traditionally expensive and complex telecom business.
Benn Jordan thinks the surveillance problem is much bigger than Flock
Ben Jordan, a musician and investigative journalist, discusses his work exposing security vulnerabilities in Flock's automatic license plate reader system and the broader surveillance infrastructure. He explains how a startup's defensive PR strategy made it a target while larger contractors quietly built similar networks, and argues that the real problem lies in government oversight failures rather than any single company.