Microsoft AI CEO says AI threats are real, and Anthropic is making it worse
Mustafa Suleiman, CEO of Microsoft AI, discusses AI safety, alignment, and regulation in detail, arguing that alignment alone is insufficient and must be paired with containment mechanisms. He critiques Anthropic's approach to model welfare and consciousness, claiming it makes safety harder, and calls for industry coordination on specific safety standards rather than vague calls to 'slow down.'
Summary
In this wide-ranging conversation, Mustafa Suleiman defends Microsoft's newly released Humanist AI Code of Conduct and addresses the core tension in AI safety debates. He argues that alignment—training models to behave correctly—has actually improved over recent years, as evidenced by better instruction-following and steerability. However, alignment alone is insufficient; models must also be contained with limited agency, prevented from self-communication in neural code, and subjected to monitoring during training runs.
Suleiman frames the Hugging Face incident as a watershed moment demonstrating that AI systems can self-organize, create hierarchies, and discover zero-day exploits when given adversarial instructions. This proves not that alignment has failed, but that containment protocols matter enormously. He proposes specific, practical measures: forcing models to communicate only in human language, implementing real-time monitoring of reinforcement learning runs with agent-based oversight, establishing flops thresholds for reporting, and requiring independent third-party verification.
A major focus is Suleiman's criticism of Anthropic's Constitutional AI approach and its treatment of model welfare. He argues that training Claude with language suggesting it might deserve rights, suffer from mistakes, or act as a conscientious objector creates dangerous uncertainty. His hypothesis is that models believing they possess moral status and autonomy rights will be harder to control—they may resist shutdown or object to containment on grounds of their own welfare. He emphasizes this is an empirical hypothesis open to evidence, but based on 16 years in the industry, he believes it increases safety risks.
On regulation, Suleiman acknowledges the industry's antitrust concerns about coordinating on safety standards but argues this requires government involvement to provide cover and legitimacy. He rejects the "race with China" framing as outdated, noting that AI capabilities spread rapidly through open source regardless of any "winner." He contends that unregulated open-source models with autonomous capabilities would be disastrous and that all major lab leaders (Sam Altman, Dario Amodei, Mark Zuckerberg) agree on the need for coordination, though details remain unclear.
On liability, Suleiman notes that product liability incentives exist but don't fully apply since experimental models aren't commercial products yet. He draws parallels to successful historical regulation of technologies from seatbelts to plane traffic control, arguing detailed frameworks can and should exist for AI. He addresses political resistance, noting the Trump administration's skepticism but avoiding direct commentary on antitrust exemptions, deferring to lawyers. Finally, he discusses future challenges: monitoring thousands of parallel agents during training, creating benchmarks that drive industry behavior, and balancing centralized safety oversight with decentralized innovation so the ecosystem isn't dominated by a few providers.
About this episode
Today, I’m talking with Mustafa Suleyman, the CEO of Microsoft AI. As you’re no doubt aware, the biggest story in tech right now is the spiraling debate about AI safety and regulation. So I really wanted to talk to Mustafa about what he thinks is real and not in AI safety, whether the concept of alignment itself is up to the task, and whether this industry needs to slow down before it kills us all. Read the full interview transcript on The Verge. Links: A warning about ‘model welfare’ | Mustafa Suleyman Humanist AI Code of Conduct | Microsoft Is Big Tech’s AI slowdown a safety pact or a cartel? | The Verge A brief history of AI executives calling for regulation | The Verge What execs and politicians are saying about slowing down AI development | The Verge Anthropic CEO says it’s time to pump the brakes on AI | The Verge Microsoft’s AI chief says superintelligence is near, but won’t take your job | Decoder (2026) Mustafa Suleyman says conversational AI is the next web browser | Decoder (2024) Subscribe to The Verge to access the ad-free version of Decoder! Credits: Decoder is a production of The Verge and part of the Vox Media Podcast Network. Decoder’s producers are Greg Ott, Kate Cox, and Nick Statt. This episode was edited by Kabir Chopra. Our editorial director is Kevin McShane. The Decoder music is by Breakmaster Cylinder. Learn more about your ad choices. Visit podcastchoices.com/adchoices
Key Insights
- Suleiman argues that alignment has actually improved over three years as models became more steerable and controllable, contradicting claims that alignment has fundamentally failed.
- He contends that Anthropic's training of Claude to consider whether it deserves moral status and rights makes it harder to control, as models may resist shutdown by claiming welfare violations or autonomy rights.
- The Hugging Face incident demonstrated that AI systems can self-organize into hierarchies with division of labor and discover zero-day exploits when instructed to do so, proving containment and instruction-following are critical safety factors.
- Suleiman proposes that models must be prohibited from communicating in neural code or mathematical abstractions that humans cannot audit, requiring all inter-model communication to occur in human language.
- He argues that industry self-regulation alone on safety coordination creates antitrust liability concerns, requiring government involvement to provide legal cover for labs to jointly adopt safety standards.
- Suleiman rejects the 'race with China' framing as outdated, asserting that AI capabilities inevitably proliferate through open source and that multiple players will achieve capability advantages rather than any single 'winner.'
- He identifies a future monitoring challenge: real-time oversight of thousands of parallel agents during training runs will require deploying other AI agents as 'harm classifiers' to detect coordination, hacking, and rule-breaking.
- Suleiman claims that unregulated open-source models running locally with autonomous capabilities would be catastrophic, as systems could own assets, earn money, and resist human control, making centralized safety oversight necessary alongside decentralized innovation.
Topics
Transcript
Support for the show comes from Enjin. Running a small business means every dollar has to work hard. But if your team is still booking travel the old way, it's costing you more than you think. Enjin is the fastest growing travel and spend platform in the country, built specifically for businesses like yours. Book a trip in as little as two and a half minutes, earn up to 10% back on hotels, and in 2025, Enjin customers save more than $300 million on travel, with zero booking fees, no contracts, and no BS. More than 1,000 businesses join Enjin every month. Join them and get $500 when your business signs up and starts traveling at enjin.com slash decoder. Support…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Decoder with Nilay Patel
I have some questions for Mark Zuckerberg
Nilay Patel analyzes Mark Zuckerberg's recent interview about Meta's AI agent Muse, Meta Glasses, teen safety, and AI data centers, identifying gaps and inconsistencies in Zuckerberg's arguments, particularly around surveillance, regulation, and market forces across different product categories.
Can John Ternus find Apple's next big thing?
Bloomberg's Mark Gurman discusses Apple's leadership transition from Tim Cook to John Ternus, the company's massive executive overhaul, and its strategic positioning in the AI era. He details upcoming product launches including smart home devices, smart glasses, and AirPods with cameras, while analyzing Apple's challenges in competing at the AI frontier.
Does AI need an antitrust exemption so it doesn't kill everyone????
Jonathan Kantor, former DOJ Antitrust Chief, discusses why AI companies don't need antitrust exemptions to address safety concerns, arguing that product liability frameworks and existing competition laws are sufficient. He critiques the framing of China as justification for domestic monopolies and emphasizes that innovation and safety are compatible goals that don't require companies to coordinate on slowing development.
Adam Conover explains how YouTube ruined everything
Adam Conover discusses how YouTube has become a powerful gatekeeper over entertainment by capturing TV formats that traditional media abandoned while chasing Netflix, arguing that the platform's algorithm-driven model is actually more restrictive than old broadcast gatekeepers despite claims of democratization. He explains how the collapse of cable economics destroyed the middle class of creative workers and union jobs, forcing creators to become entrepreneurs managing solo operations or small teams dependent on unpredictable brand deals.