Claude sucks. And here is why.
This video explores why Claude and other AI models produce inconsistent results, examining how model selection (Haiku vs Sonnet vs Opus vs Claude 3.5 Fable), effort levels, prompt clarity, and system context dramatically affect output quality. The speaker demonstrates these differences through a Venn diagram creation task and advocates for using organized folder systems with documented SOPs and code-based solutions to ensure consistent AI performance.
Summary
The speaker addresses a recurring community complaint: AI assistants like Claude forgetting context, going off-rails, and behaving inconsistently. The core thesis is that inconsistency stems from multiple overlapping factors rather than a single problem.
First, the speaker introduces their PKM (Personal Knowledge Management) system called 'my PKA,' where Claude is configured as 'Larry,' an orchestrator agent that delegates tasks to specialized team members based on documented SOPs, guidelines, and work streams. This folder-based approach helps maintain consistency by providing persistent context.
The video's main contribution is demonstrating how different variables affect AI output quality: (1) Which AI platform you use (Claude, Gemini, etc.), (2) Which model within that platform (Haiku, Sonnet, Opus, Fable), (3) What effort/compute level is selected, and (4) How detailed and clear the instructions are.
Through a practical test—asking different models to create a four-circle Venn diagram with straight lines forming squares—the speaker shows dramatic quality differences. Haiku fails completely at the task. Sonnet creates a basic working diagram. Opus adds unnecessary features like sliders. Fable elegantly solves the problem by rotating the diagram 90 degrees for optimal placement without being asked. This demonstrates that Fable understood the intent better than more-capable-appearing models.
The speaker emphasizes that consistency is nearly impossible with pure prompting because: (1) The same prompt run twice on the same model produces different outputs due to LLM stochasticity, (2) This means validation requires running tests three times to confirm consistency, and (3) Most YouTube comparisons only run prompts once, making their conclusions unreliable.
To address inconsistency, the speaker recommends: using documented SOPs and guidelines within organized folders, converting critical workflows into code (which has deterministic outputs), creating specialized agents with documented expertise, using closed-session commands to create documentation and lessons learned, and asking AI to identify why it's deviating from its instructions.
The speaker notes that folder context only works in specific applications: Claude Desktop with Codex or Claude Code can write back to folders, but browser-based Claude cannot. They demonstrate that even with folder context, the folder doesn't automatically improve outputs unless it contains relevant instructions for that specific task.
Key Insights
- The same prompt run twice through the same model with identical settings produces completely different outputs due to how LLMs interpret and predict language differently each time, making single-test comparisons unreliable—proper validation requires running the same prompt three times to confirm consistency.
- Claude 3.5 Fable demonstrated superior understanding by rotating the Venn diagram 90 degrees without being asked, recognizing that this was the most efficient spatial arrangement, whereas Opus added unnecessary features like interactive sliders that weren't requested.
- Haiku model cannot even create a basic overlapping Venn diagram with squares and circles out of the box, and maxing out its effort level (extended thinking X high) still fails to produce the correct output, indicating fundamental capability limitations below a certain model tier.
- Folder context in Claude only works in specific applications (Claude Desktop with Codex or Claude Code that can write back to folders), not in the browser version—browser uploads are read-only and don't allow the AI to actually update files, defeating the purpose of folder-based systems.
- Converting workflows from pure prompting into code guarantees consistency because code produces deterministic mathematical outputs, whereas prompting always involves LLM prediction that varies—for critical outcomes, teams should identify workflow steps that can be automated as code rather than relying solely on prompts.
Topics
Transcript
[0:00] If you're using Claude and it keeps forgetting or not doing what you want, that is going off rail or behaving differently than it did yesterday. This video is for you because this is a recurring topic that comes up in our community over and over. And that's why I'm making this video to maybe give you some more perspective, why these things happen. So if you can relate to this sentence, AI intelligent and the most stupid assistant in the world, then this video is for you, just for context, every week inside my iCore, we launch a poll here. [0:35] In this case, it was, AI workflow most often break down and slow you down? And there…
Full transcript available for MurmurCast members
Sign Up to AccessMore from ICOR with Tom | AI Productivity
Claude can now prompt its own SESSIONS. I built the boss.
Claude now has the ability to interact with multiple sessions, allowing a streamlined orchestration of tasks through an agent called Larry. This integration can simplify task management and enhance productivity by enabling a single main session to oversee various sub-sessions or tasks.
I switched off Claude's memory. My folder does it better.
Tom explains why he disabled Claude's native memory feature and instead built a local folder-based system (myPKM) to maintain persistent context with AI agents. He shares an audit revealing systemic issues like contradictory documentation, forgotten rules, and parallel session conflicts, then outlines how to systematically fix these problems using the ICOR productivity methodology.
I do not download invoices anymore. My folder and Claude do it.
The speaker demonstrates how he uses AI (Claude) integrated with his local folder system to automate invoice collection for his accountant, downloading 36 invoices in 5 minutes with minimal manual intervention. He explains his AI team methodology, work streams, and how persistent local knowledge bases enable AI to handle recurring tasks independently without constant re-explanation.
Claude finished the work. Then I lost it in the chat.
A creator demonstrates a comprehensive local-folder-based system for managing AI-generated deliverables asynchronously, featuring an interactive browser interface that tracks decisions, changes, and conversations without relying on external databases or web applications. The system enables collaborative editing, visual annotations, and automatic context preservation across multiple parallel AI projects.
Claude writes better weekly reports than I did. Here is proof.
The speaker demonstrates how AI-powered weekly reports can automatically synthesize journal entries, notes, and captured content into personalized insights without manual review. Two different implementations are shown—one dashboard-style and one essay-style—proving the same underlying system can be customized to individual preferences and consumption habits.