Is Meta’s Muse Wonder model any good?
Clare Vo reviews Meta's Muse Wonder AI model and finds it excels at agentic work and coding tasks but significantly underperforms on creative design work compared to competitors like Claude Opus 3.5. The model shows promise as a personal AI agent with strong domain expertise and memory capabilities, but lacks the design sensibility needed for front-end development.
Summary
Clare Vo, a product leader and AI obsessive, conducts a comprehensive review of Meta's new frontier intelligence model Muse Wonder using her How I AI benchmark. The model comes in two versions and Meta positions it alongside competitors like Claude Opus 3.5, Claude Sonnet, and Google's Astra for frontier intelligence capabilities.
Muse Wonder differentiates itself through three claimed frontier capabilities: domain expertise in specialized knowledge work (legal, financial, business workflows), active memory for maintaining context across conversations, and active focus for sustained effort on challenging long-running tasks. The model requires users to download Muse Code and subscribe to Meta's services, which Vo found annoying but functional.
The benchmark tests across nearly 20 tasks including prototypes, agentic work, coding, writing, and creative tasks. Vo's results show Muse Wonder is strongest at agentic work, consistently ranking first on tasks like billing ticket management, customer reply handling, inbox triage, and long-term research. It performs comparably to other leading models on coding tasks including back-end feature implementation and code audits. The model has a natural conversational voice suitable for personal assistants and does well writing professional emails.
However, the model significantly underperforms on creative and design tasks. Vo found the design sensibility to be approximately 12-18 months behind current standards, with consistent issues including poor font choices (fat serif fonts with rounded corners), broken images, design guideline violations, and lack of polish. Specific examples include a particularly poor redesign of the ChatGPT website, inconsistent dark mode implementation in developer tools, and generic SaaS template aesthetics. On creative SVG and video editing tasks, Muse ranked last or near-last among the models tested.
Performance characteristics include speed on short one-shot tasks but longer execution times on complex agentic work, possibly due to the active focus capability. Vo notes this makes it viable for orchestrator or voice roles in agent systems but unsuitable for front-end design work. She concludes Muse Wonder is competitive for agentic workflows and back-end coding but plans to continue using Claude Opus 3.5 for creative work.
Key Insights
- Meta's Muse Wonder excels specifically at agentic work tasks like billing ticket resolution, inbox triage, and long-running research, ranking first across all four agentic workflow tests in the benchmark
- The model has a design sensibility that is 12-18 months behind current standards, with consistent aesthetic issues including fat serif fonts, rounded corners, and generic SaaS template components rather than beautiful or innovative designs
- Muse Wonder's three differentiated frontier capabilities are domain expertise in specialized professional knowledge work, active memory for carrying context across conversations, and active focus for sustained effort on challenging long-running tasks
- The model performs very fast on short one-shot tasks like drawing characters and prototypes, but takes significantly longer on complex agentic tasks, suggesting the active focus capability may be contributing to longer execution times
- On creative tasks like SVG generation and video editing, Muse Wonder ranked last or near-last among tested models, with poor audio handling and unattractive overlay design on short-form video output
Topics
Transcript
[0:00] Welcome back to How I AI. I'm Clare Vo, product leader and AI obsessive here on a mission to help you build better with these new tools. Today, I'm reviewing Meta's new frontier intelligence model, Muse Wonder, which comes in two versions, Wonder and Wonderful. I am going to put this model through the How I AI benchmark to see if it can enter the race for frontier intelligence, especially around coding, knowledge work, and agentic workflows. Let's get to it. This episode is brought to you by [0:30] DX. [music] In a recent study across more than 500 engineering organizations, DX found that spend on AI tools has grown 28x over the last year. The share of AI-authored…
Full transcript available for MurmurCast members
Sign Up to AccessMore from How I AI
I Turned a $50 Photo Frame Into an AI Family Dashboard
A creator transformed a $50 Amazon photo frame running Android into a customized AI-powered family dashboard using OpenAI's Codeex and web technologies. The dashboard displays family schedules, photos, NASA images, news, and individual checklists for each family member, with updates automatically posted via an API connected to a grabbot that monitors calendars and emails.
Claude Code for creators: AI editing is here
Brooke Lacey, a 200K+ TikTok creator, demonstrates how she uses Claude Code to automate short-form video editing by leveraging existing computer tools like FFmpeg, Whisper, and Python without uploading files to the cloud. She trains Claude on her TikTok corpus to understand her editing style, facial expressions, and comedic timing, then uses it to generate jump cuts, captions, scripts, and mine comments for new content ideas.
Muse dodged every trap in this billing ticket
A billing ticket resolution task was used to benchmark agentic AI performance, and Muse excelled by successfully navigating multiple challenges including account linking errors, duplicate refund issues, and prompt injection attempts. The agent demonstrated superior performance by investigating thoroughly, fixing the account link, issuing the correct refund, updating the plan, and marking the ticket as resolved.
Meta's Muse is the best AI agent I've tested
The speaker provides their initial impressions of Meta's Muse AI agent, declaring it the best agentic AI they've tested. Muse excelled across four different knowledge work tasks including billing management, customer responses, inbox triage, and research, outperforming alternatives.
Meta's Muse: best AI agent, worst designer
The speaker tests Meta's AI model on video editing and finds it excels at technical tasks like cutting and timing but significantly underperforms on creative work like design and overlays. They conclude that while the model is strong for coding and agent tasks, it falls short compared to Claude Opus 3.5 for creative applications.