Stop prompting your AI agents. Start managing them.
The speaker discusses the shift from traditional agent prompting to agent management, highlighting the limitations of local Kanban board approaches and advocating for cloud-based VPS solutions with multiple communication channels to effectively manage autonomous AI agents across projects.
Summary
The transcript opens with an observation that while many people discuss orchestrating multiple agents autonomously, few actually implement it successfully. Most implementations still rely heavily on human-in-the-loop prompting, even when agents are organized in loops or hierarchies with sub-agents.
The speaker then transitions to their personal insight: moving from the role of 'agent prompter' to 'agent manager' represents a fundamental shift in approach. They describe initial attempts using Kanban boards as a management tool, which seemed promising for task initiation but proved problematic in practice. The speaker identifies two specific challenges: difficulty maintaining multi-turn interactions (beyond 2-3 turns) and the limitations of running agents on local infrastructure.
The solution the speaker implemented involved two key changes: migrating from local runtime to cloud-based VPS hosting and establishing multiple communication channels to interact with agents. This infrastructure change enables flexibility in how agents are prompted and managed—through text messages, integration with Linear (a project management tool), or direct shell prompts. This multi-channel approach represents a more sophisticated agent management paradigm compared to single-interface solutions.
Key Insights
- Despite widespread discussion of autonomous multi-agent orchestration, most implementations still rely heavily on human-in-the-loop prompting rather than true autonomy
- Kanban boards are insufficient for managing agent workflows because they struggle to maintain interactions beyond 2-3 turns and lack proper intervention capabilities
- Local runtime environments do not work effectively for agent management, requiring migration to cloud-based infrastructure
- Multi-channel communication interfaces (text, Linear integration, shell prompts) enable more flexible and effective agent management than single-interface approaches
- The conceptual shift from 'agent prompter' to 'agent manager' represents a fundamental change in how practitioners approach autonomous agent systems
Topics
Transcript
[0:00] We have heard a lot of people talk about orchestrating many agents autonomously across a project. We actually haven't seen many people do it. I still see a lot of prompting even if you're prompting into a loop [music] or a goal as far as sub-agents. People are really still human in the loop. >> What really clicked for me was starting to move away from being agent prompter to agent manager. The first thing that everybody tried was a Kanban board. You would put all these things in there and move them back and forth. What I found is that it was hard to get two, three, four turns. Through that, it was easy to [0:31] get to…
Full transcript available for MurmurCast members
Sign Up to AccessMore from How I AI
How to build a custom AI harness with Claude SDK
A custom AI harness is code wrapped around an AI agent to make it more effective for specific workflows. The speaker demonstrates building a Sentry bug-debugging harness using Claude SDK with a terminal UI, showing how structured constraints, custom prompts, and specific tool adapters enable agents to handle complex tasks more efficiently than general-purpose AI tools.
How I run autonomous coding agents from my phone with OpenAI Symphony + Linear
Alessio Finelli demonstrates how he uses OpenAI's Symphony framework combined with Linear and Codex to run autonomous coding agents from his phone, managing both software engineering tasks and a Pokémon card trading business through cloud-based VPS infrastructure instead of local machines.
My taste and the automated benchmark disagreed almost completely
The speaker discusses discrepancies between their subjective evaluation of model performance and automated benchmark results, noting that different judges (Opus, 4A, 5.5) show varying levels of generosity and bias. They conclude that personal judgment matters significantly and plan to incorporate more subjective taste into evaluation metrics while retiring saturated benchmark tasks.
Task-by-task model recommendations
The speaker provides task-specific model recommendations across different use cases, suggesting GPT 5.5 for PRDs, Sonnet 4.6 for prototyping and casual interaction, and Opus 4.8 or Sonnet 5 for codebase work. Model selection varies based on complexity, with Opus 4.8 excelling at dense UI design and Sonnet suitable for simpler implementations.
The How I AI Bench
The speaker introduces 'The How I AI Bench,' a new set of human and AI-graded benchmarks designed to evaluate language models on practical tasks like writing PRDs, solving bugs, and designing systems. They test Claude Sonnet 3.5 against these benchmarks and note that while it scores lower than some specialized benchmarks (69% on Agentic Coding SweetBench Pro, 82% on Terminal Bench 2.1), the difference may not be noticeable in real-world usage.