Building Realistic Voice Agents Has Never Been Easier
The video demonstrates how to build a voice agent using Claude Code and ElevenLabs without manually configuring APIs or documentation. The creator walks through building a sales voice agent that integrates with Cal.com for booking, iterating through several rounds of debugging. The main argument is that natural language prompting via Claude Code dramatically reduces the complexity of building production-ready voice agents.
Summary
The video opens with the creator showcasing a previously built voice agent trained on 400 YouTube transcripts, demonstrating how users can query it conversationally about tools like Firecrawl and Claude Code workflows. This serves as proof-of-concept that complex voice agents can be built quickly using Claude Code.
The creator then explains the core architecture of any voice agent, identifying four key components: persona (system prompt), voice, knowledge base, and tools. He illustrates these using the ElevenLabs dashboard, showing how each piece is normally configured manually — a process he argues is error-prone and time-consuming.
The live demo centers on building a sales voice agent for a fictional AI consultancy called 'Neural.' The agent's goal is to qualify inbound visitors and book discovery calls via Cal.com. Using Claude Code's 'plan mode,' the creator dictates his requirements in natural language, and Claude asks clarifying questions about ElevenLabs setup, Cal.com configuration, voice persona, widget appearance, and data fields to capture. Claude then generates a complete architectural plan before writing any code.
After providing API keys for both ElevenLabs and Cal.com, Claude autonomously creates the ElevenLabs agent, configures two tools (check availability and book appointment), writes the system prompt, and embeds the widget into the local website. The first test reveals issues: the voice was too enthusiastic, the greeting message wasn't triggering automatically, and the agent was querying Cal.com availability in UTC rather than Central time, causing incorrect slot reporting.
Through iterative natural language feedback to Claude Code, the creator fixes the voice selection, adjusts the system prompt for conciseness and proper email/name spelling confirmation, and resolves the timezone bug in the tool call parameters. A final successful demo shows the agent correctly identifying available slots and booking a 7:00 p.m. Central appointment, with a confirmation email delivered to the correct address.
The creator closes by addressing security and cost concerns: locking the widget to specific domains to prevent credential theft, setting conversation duration caps, implementing rate limits, and considering authentication for public-facing widgets. He also notes the agent could be connected to a phone number via Twilio for the same functionality through a different channel.
Key Insights
- The creator argues that Claude Code's ability to read API documentation and reason about tool integration means users never need to manually inspect endpoints or configure platform dashboards — natural language description of the goal is sufficient to produce working integrations.
- The creator found that the timezone bug causing incorrect availability results was discovered by Claude Code analyzing the conversation transcript, which showed the check-availability tool was constructing its query window in UTC instead of Central time — a subtle error that would be difficult to spot without reading raw API parameters.
- The creator warns that embedding an ElevenLabs widget on a public website means all usage costs fall on the account owner, and that the widget's HTML snippet can be easily stolen via browser inspection and reused on another domain, making domain allowlisting in ElevenLabs security settings a critical safeguard.
- The creator explains that the same configured ElevenLabs agent — with its tools, prompting, and voice — can be surfaced through multiple channels (website widget, phone number via Twilio) without any additional configuration, calling it 'the same engine behind the scenes, just a different door.'
- The creator observes that latency in voice agents is directly tied to model and voice quality choices, and notes that testing on localhost produces worse latency than a live deployed widget, which can cause misleading impressions during development.
Topics
Transcript
[0:00] It has never been so easy to build whatever you want. And so in today's video, I'm going to show you guys how easy it is to build a voice agent with just your natural language using Cloud Code with 11 Labs. So, let me show you guys a quick example. I had this crazy idea where I was thinking, okay, I've got a ton of YouTube videos, exactly 400 actually, and all of these YouTube videos, there's a lot of different tools. There's a lot of different things that I've mentioned, and some people in my audience might be confused and might want to just have a nice way to like chat with those YouTube videos. So my…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Nate Herk | AI Automation
18 Months of Pricing AI Automations in 21 Mins
Nate shares 18 months of experience pricing over 100 AI automation systems, providing a framework for calculating project value based on client savings rather than hourly rates, and structuring payments in 30-day milestones to minimize financial risk.
Fable 5 + Karpathy’s LLM Wiki is Basically Cheating
The video demonstrates how to build a personal LLM-powered knowledge base using Obsidian and Claude, inspired by Andrej Karpathy's approach. By ingesting various data sources (YouTube transcripts, PDFs, URLs) into an interconnected wiki structure, users can create a 'second brain' that helps AI agents understand context and relationships across knowledge domains.
How Claude is Creating a New Generation of Millionaires
Claude, an AI built by Anthropic, is enabling non-technical founders to build million-dollar software companies by automating development work. The transcript outlines why Claude is outpacing competitors, provides real examples like Vulcan, and offers a four-step framework for anyone to start building with Claude today.
How Anthropic Engineers Actually Prompt Fable 5
A guide to effectively prompting Claude Fable 5, covering six key techniques including providing context/intent, using negative prompting, letting the model act once ready, verifying outputs, avoiding reasoning requests, and keeping instructions concise. The speaker emphasizes that Fable 5 is expensive and should only be used 5-15% of the time, with pricing at $10 per million input tokens and $50 per million output tokens.
Stanford's Method Turns Claude Into a PHD Level Research Team
A researcher demonstrates the STORM method from Stanford, which uses five expert perspectives (practitioner, academic, skeptic, economist, historian) to create verified research reports. The method produces 25% more organized articles than competing approaches and is packaged as a reusable Claude skill that generates HTML briefings with peer-reviewed citations.