Gemini 3.1 Flash Live Just Changed Voice Agents Forever
Google released Gemini 3.1 Flash Live, a new speech-to-speech voice AI model with improved latency, accuracy, and multimodal capabilities. The presenter demonstrates building voice agents using the model with Google AI Studio and shows how to integrate it with external tools using Claude for coding assistance.
Summary
The video covers Google's new Gemini 3.1 Flash Live voice model, which represents a significant upgrade from previous text-based processing to direct speech-to-speech communication. The model offers improved precision, lower latency, and more natural interactions, with benchmark improvements of 19% in multi-step function calling over previous Gemini models. Key features include enhanced performance in noisy environments, better accuracy with alphanumeric strings, and contextual awareness for understanding emotions like sarcasm or frustration. The presenter demonstrates the model's capabilities through Google AI Studio, showing how users can create custom voice agents with system instructions, voice options, and tool integrations. Two practical examples are showcased: a customer service agent for a keyboard website and a personal assistant that can access ClickUp tasks and calendar functions. The model supports over 70 languages and offers both free and paid tiers, with the free version allowing experimentation but having usage limits and data sharing with Google. While the technology is impressive, the presenter notes current limitations including synchronous function calling that creates awkward pauses, and the technical complexity of deploying to production environments compared to simpler solutions like 11Labs.
Key Insights
- Gemini 3.1 Flash Live uses direct speech-to-speech processing instead of the traditional speech-to-text-to-speech pipeline, enabling more natural interactions and better contextual understanding of emotions like sarcasm
- The new model outperformed previous Gemini models by 19% in multi-step function calling benchmarks and shows significant improvements in noisy environment performance
- The model currently has a limitation where it stops speaking entirely during function calls and waits for responses, creating awkward silences unlike other voice agents that can talk while processing
- Google offers the model free with data sharing for product improvement, or paid tiers starting around 14 cents for a 10-minute call with enterprise-grade privacy and higher rate limits
- Deploying Gemini Live to production websites requires managing persistent websocket connections and server processes, making it more technically complex than plug-and-play solutions like 11Labs
Topics
Full transcript available for MurmurCast members
Sign Up to Access