How I run autonomous coding agents from my phone with OpenAI Symphony + Linear
Alessio Finelli demonstrates how he uses OpenAI's Symphony framework combined with Linear and Codex to run autonomous coding agents from his phone, managing both software engineering tasks and a Pokémon card trading business through cloud-based VPS infrastructure instead of local machines.
Summary
Alessio Finelli, founder of Kernel Labs and co-host of the Latent Space podcast, showcases a practical system for managing autonomous coding agents at scale. Rather than running agents locally, he uses a cloud VPS with 32GB RAM and 4 cores to host multiple pre-logged coding agents. The core workflow uses OpenAI's Symphony framework, which monitors a Linear project board as a source of truth. When tasks are moved to "To Do" status, Symphony automatically creates Codex workpads that include planning, acceptance criteria, and validations. Tasks flow through states: To Do → In Progress → Human Review (where PRs are reviewed and commented on) → Rework (where the agent addresses comments) → Done. This system allows him to manage agents from anywhere—phone, desktop, or web—without direct intervention in traces or long conversations.
Beyond coding orchestration, Finelli demonstrates a real-world application: automating his Pokémon card trading business called "Power Buyer." He uses Codex with browser access to extract PSA certificate numbers from graded cards, search eBay for underpriced inventory, and evaluate cards based on grading equivalencies across different companies. At trade shows, he's building automation to price cards in real time by searching multiple platforms autonomously, saving significant manual labor in what would otherwise be an inefficient human-dependent process.
Key technical insights include: token usage tracking per task helps identify inefficiencies (one task used 221 million tokens due to a failed deployment rewrite); markdown specification files require regular "red diffing" to remove accumulated instructions that confuse models; the hardness of Symphony setup is straightforward, with the real value coming from building better tools and tooling layers to keep agents running longer; and moving from "agent prompter" to "agent manager" represents a fundamental shift in how to work with autonomous systems.
Finelli emphasizes that AI's most positive outcome is enabling small business creation by adding leverage at small scales—his father's fish delivery business could automate inventory tracking with AR glasses; vintage resellers can catalog mismatched inventory; and AI helps intersect the physical world efficiently in ways previously limited by human bandwidth.
Key Insights
- Finelli moved from local runtime to cloud VPS infrastructure specifically to enable multi-channel communication with agents (text, Linear, shell, mobile) and better intervention capabilities throughout task execution, rather than just at the initial kick-off.
- Token usage per task is a key diagnostic metric—when a task unexpectedly costs 221 million tokens instead of expected 15-60 million, it directionally explains how many issues the agent encountered and informs tooling improvements needed.
- Markdown specification files accumulate cruft over time because models tend to add rather than remove instructions, so teams should periodically 'red diff' specs every few months to purge unnecessary or conflicting directions.
- Symphony's value is not in providing new capabilities unavailable in direct Codex use, but in shaping context and providing historical tracking—seeing how a task evolved through specification, workpad, rework cycle, and PR comments helps debug future failures.
- AI enables small business creation by adding leverage at scale that was historically impossible—businesses based on heterogeneous data (trading cards, vintage clothing, inventory) that required manual human labor can now scale autonomously.
Topics
Transcript
[0:00] This is my favorite positive outcome of AI, which is small business creation. Just the ability to like intersect the human world in a way that has been historically very inefficient has been a quality of life improvement for me. >> You know, my dad, their business, they deliver fish to restaurants. They got like this freezer with the frozen stuff and like somebody's going out there with like the pen and paper every morning kind of like writing down what's there. Sometimes they're like, "Oh, my god, we're missing like three tunas or like we're missing a box of shrimp." All of that work now can easily be automated even with just with the magic glasses. [0:31] >>…
Full transcript available for MurmurCast members
Sign Up to AccessMore from How I AI
My taste and the automated benchmark disagreed almost completely
The speaker discusses discrepancies between their subjective evaluation of model performance and automated benchmark results, noting that different judges (Opus, 4A, 5.5) show varying levels of generosity and bias. They conclude that personal judgment matters significantly and plan to incorporate more subjective taste into evaluation metrics while retiring saturated benchmark tasks.
Task-by-task model recommendations
The speaker provides task-specific model recommendations across different use cases, suggesting GPT 5.5 for PRDs, Sonnet 4.6 for prototyping and casual interaction, and Opus 4.8 or Sonnet 5 for codebase work. Model selection varies based on complexity, with Opus 4.8 excelling at dense UI design and Sonnet suitable for simpler implementations.
The How I AI Bench
The speaker introduces 'The How I AI Bench,' a new set of human and AI-graded benchmarks designed to evaluate language models on practical tasks like writing PRDs, solving bugs, and designing systems. They test Claude Sonnet 3.5 against these benchmarks and note that while it scores lower than some specialized benchmarks (69% on Agentic Coding SweetBench Pro, 82% on Terminal Bench 2.1), the difference may not be noticeable in real-world usage.
How a designer became a top engineer
Katie transitioned from designer to top-performing engineer, ranking in the 94th percentile for code throughput across the entire R&D organization. Her success stemmed from technical curiosity combined with supportive engineer mentors who reviewed her code and helped her improve her craft.
No meetings, no Jira, no text threads... and it shipped anyway.
A team successfully shipped a project in 10 weeks by eliminating traditional project management structures entirely—no meetings, Jira, documentation, or text communication. Instead, they relied solely on a 24/7 Zoom room where team members could work synchronously and asynchronously as needed.