How to Build Your Own Agent OS
Julian Goldie presents a seven-layer blueprint for building an 'Agent Operating System' that unifies AI tools like Claude, ChatGPT, and Hermes into a single, persistent, self-improving workflow. The system addresses the core problem of context fragmentation across AI tools by introducing a shared memory layer and feedback loop. The video also promotes his paid community, the AI Profit Boardroom, as the place to access full implementation resources.
Summary
The video opens by framing the central problem: most AI users are context-switching between multiple tools (Claude, ChatGPT, terminals) with no shared memory or unified system, causing 'cold start' friction on every session. Julian Goldie positions himself as someone who spent 100 hours and nine failed attempts to solve this, resulting in a seven-layer 'Agent Operating System' blueprint.
Before presenting the blueprint, Goldie outlines five mistakes he made along the way. These include treating AI as isolated tools rather than parts of a system, paying for subscriptions before validating free alternatives, relying on inadequate built-in chat memory, over-engineering automations in N8N when agents can handle workflows natively, and allowing AI outputs to land in disorganized folders where they lose compounding value.
A significant portion of the video is dedicated to introducing 'Hermes,' described as a fully autonomous agent built by News Research, launched February 2026, that reached 100,000 GitHub stars in under three months. Hermes is highlighted for its persistent memory, skill document creation, integrations with Telegram/Discord/Slack, 40+ built-in tools, and MIT license (free, no lock-in). Goldie presents it as the centerpiece agent within the operating system.
The seven-layer blueprint is then detailed sequentially: Layer 1 is hardware (a modern laptop suffices); Layer 2 is memory, using Obsidian (local markdown vault, 1.5M users, CLI with 100+ commands) combined with Omi for conversation capture; Layer 3 is the brain, a model routing layer using OpenRouter to swap models without rebuilding the system; Layer 4 is agents (Hermes, Claude Code, Open Claude) selected by task type; Layer 5 is a command center, a Next.js dashboard unifying all agents and outputs; Layer 6 is production, covering SEO, content, studio, and research workflows; Layer 7 is the loop, where every output is written back into the Obsidian memory vault so each session starts smarter than the last.
Goldie emphasizes that the loop (Layer 7) is the most commonly skipped yet most critical layer, as it enables the system to compound in capability over time. He closes with practical advice to start with just Layers 2 and 4 (memory and agents), and to build for the architecture rather than specific tools since models and agents change frequently. The video ends with promotion of his AI Success Lab community (75,000 members) and the AI Profit Boardroom, which offers the full dashboard setup, prompts, roadmap, and four weekly coaching calls.
Key Insights
- Goldie argues that the core productivity problem for AI users is not the quality of individual tools but the absence of a system around them — he claims spending 20% of his time re-explaining context across tools was friction, not leverage.
- Goldie claims that Hermes, rather than just being another chatbot, writes reusable 'skill documents' when it solves complex tasks so it never has to figure out the same problem twice, enabling compounding agent capability.
- Goldie asserts that built-in chat memory in most AI tools is insufficiently detailed for real workflows, and that an external memory layer — specifically Obsidian vaults readable by agents via CLI — is the necessary fix.
- Goldie contends that N8N-based automations are brittle because every webhook requires its own logic and breaks on updates, whereas an agent operating system eliminates the need for such plumbing by having agents communicate through a shared workspace.
- Goldie identifies Layer 7 — the feedback loop that writes every agent output back into the Obsidian memory vault — as the most commonly skipped layer and the one that causes the system to compound, arguing that without it the system is identical on day 100 as on day one.
Topics
Full transcript available for MurmurCast members
Sign Up to Access