Web News: Anthropic Released An AI It Doesn't Fully Trust
Two tech commentators discuss the release of Anthropic's new Claude 'Mythos 5' and 'Fable 5' models, focusing on the unprecedented safety guardrails that route certain queries to older models. They explore concerns about model staggering, data retention policies, and the distillation controversy, arguing the pace of AI development is outrunning meaningful oversight.
Summary
The hosts discuss the June 9th release of Anthropic's Claude Fable 5 and Claude Mythos 5 models, with one host having had several days of hands-on experience. He characterizes the new model as a meaningful step up from predecessors — particularly in code review and troubleshooting — but not a foundational leap comparable to earlier generational jumps like GPT-3 to GPT-4.
A central focus is the unprecedented safety approach Anthropic has taken. CEO Dario Amodei reportedly expressed hesitation about releasing the model due to its ability to find security exploits that had eluded both other AI models and professional security researchers for decades. Anthropic first released it under a program called 'Project Glasswing' to large companies including Google and Microsoft, as well as the U.S. government, giving them a month or two of exclusive access under NDA before public release.
The hosts examine the novel guardrail mechanism where certain sensitive queries — specifically around frontier cybersecurity and research biology — are automatically downgraded to the older Opus 4.8 model rather than being handled by Mythos 5. The hosts argue this creates a 'technical debt' problem: as newer and more capable models are released every few months, the staggering of models for safety routing will become increasingly complex and unmanageable, unlike earlier model-switching which was purely a UX or capability decision.
The conversation shifts to a new 30-day mandatory data retention policy, even for enterprise customers who previously could opt out. The hosts speculate this serves multiple purposes: monitoring for misuse and distillation attempts by competitors, gathering behavioral data to refine the model's safety mechanisms, and potentially informing product development. They note a tension since Anthropic is also releasing competing products (like a Figma-like design tool) built on the same models, making data retention concerning for businesses building on their API.
The distillation controversy receives significant discussion — the practice of using API calls to extract a model's capabilities into training data for new open-source models. The hosts acknowledge the irony of Anthropic objecting to this while having themselves scraped internet content (blogs, guides, videos) without explicit permission to train their models. They note Anthropic frames anti-distillation policies around national security and adversarial nations, which the hosts view as a way to sidestep the underlying contradiction.
The episode closes with broader concerns about the pace of AI development outstripping oversight capacity, drawing a parallel to the smartphone era's rapid evolution that eventually stabilized into annual incremental updates. The hosts predict AI model releases will similarly consolidate into annual cycles with simpler naming conventions, but express concern that the current pace — driven by software's lack of physical manufacturing constraints — is producing potentially dangerous progress without adequate governance structures.
Key Insights
- The host argues that Anthropic's decision to route cybersecurity and biology queries to the older Opus 4.8 model — rather than block them outright — creates a compounding technical debt problem that will become unmanageable as newer, more powerful models are released every few months.
- The host contends that Anthropic's initial hesitation to release Mythos 5 was never going to result in a permanent moratorium, arguing it made no business sense when competitors would inevitably catch up, making the delay a temporary stopgap rather than a genuine safety hold.
- The host claims that competitive capitalistic pressure will inevitably force even safety-focused companies like Anthropic to loosen guardrails, because a safer but less capable model will drive users toward competitors with fewer restrictions.
- The host argues that Anthropic's 30-day mandatory data retention policy — even for enterprise users — serves multiple undisclosed purposes beyond stated safety monitoring, likely including detection of distillation attempts and informing product development.
- The host points out a significant irony in Anthropic's anti-distillation policy: the company objects to others extracting capabilities from its model, yet built that model by scraping internet content — blogs, guides, videos — without explicit permission from creators.
- The host argues that Anthropic frames its anti-distillation stance around national security threats from adversarial nations as a way to sidestep the ethically controversial foundation of having trained on human-generated content without consent.
- The host contends that the current model-staggering approach is a data-gathering exercise rather than a permanent solution, predicting Anthropic will develop more elegant safety mechanisms once they observe real-world usage patterns with the new model.
- The host argues that AI development's purely software nature — lacking the physical manufacturing, logistics, and marketing constraints of hardware like smartphones — means it can progress at a pace that fundamentally cannot support meaningful human oversight.
Topics
Full transcript available for MurmurCast members
Sign Up to Access