I Gave ChatGPT 5.5 the Work That Breaks Models. It Finished.
The speaker argues that GPT-5.5 has reset the bar as the strongest model in the world today, not just incrementally better but fundamentally changing what users can reasonably ask a model to do. Through rigorous testing of complex, multi-step tasks, they demonstrate that 5.5 excels at carrying complex work to completion, though it still requires human validation and works best when combined with other tools in the OpenAI ecosystem.
Summary
The speaker presents a comprehensive analysis of GPT-5.5, arguing it represents a significant advancement that 'moved the floor' rather than just incremental improvement. They emphasize that 5.5's importance lies not in being slightly better than 5.4, but in expanding what tasks can be reasonably delegated to AI models. The speaker conducted three rigorous private tests designed to push models to failure: Dingo and Company (executive knowledge work), Splash Brothers (data migration), and Artemis 2 (3D visualization). In the Dingo test, 5.5 scored 87.3 versus 67.0 for Opus 4.7, producing all 23 required deliverables as actual usable files rather than fake formats. For Splash Brothers, 5.5 became the first model to catch planted fake records like 'Mickey Mouse' customers, though it still struggled with backend database hygiene. The Artemis test revealed that while 5.5 excels at information density, Opus 4.7 still maintains an edge in visual composition and taste. The speaker emphasizes that 5.5's strength lies in its ability to 'carry' complex, multi-step work without losing thread, especially when used within Codex rather than just ChatGPT. They argue the future of AI use is routing between different models for different tasks, with 5.5 serving as the strongest default for complex execution, while Opus remains superior for blank canvas visual work. The analysis concludes that 5.5 enables new categories of work that weren't previously feasible, fundamentally changing the question from 'can the model answer this?' to 'what can I now ask it to do?'
Key Insights
- The speaker argues that 5.5 represents a fundamental shift where 'the floor moved' rather than just incremental improvement, changing what users can reasonably ask models to do
- In testing, 5.5 became the first model to successfully catch planted fake records like 'Mickey Mouse' and 'test customer' in data migration tasks that previous frontier models had missed
- The speaker claims that evaluating models on easy tasks is missing the point since previous models are already good enough for simple work, and differences only show up in complex, messy, multi-step tasks
- 5.5 scored 87.3 on the Dingo executive knowledge work test versus 67.0 for Opus 4.7, producing all 23 required deliverables as actual usable files rather than HTML masquerading as proper formats
- The speaker argues that Anthropic services are currently showing 'one nine' (90-something percent) availability compared to OpenAI's 'three nines' (99.9%), making reliability a key differentiator for serious work
Topics
Full transcript available for MurmurCast members
Sign Up to Access