NEW Ollama 0.19 Update is INSANE!
Ollama 0.19 introduces a massive speed improvement for local AI on Apple silicon by integrating with Apple's MLX framework, achieving nearly 2x faster response generation and 1.6x faster input processing. The update also includes smarter caching across conversations and support for Nvidia's NVFP4 format, making local AI competitive with cloud services for the first time.
Summary
Ollama 0.19 represents a major breakthrough in local AI performance, specifically for Apple silicon devices. The update integrates with Apple's MLX machine learning framework, which takes advantage of unified memory architecture where CPU and GPU share the same memory pool without transfer overhead. Benchmark results using Alibaba's Qwen 3.5 35B model show prefill speed increasing from 1,154 to 1,110 tokens per second (1.6x improvement) and decode speed jumping from 58 to 112 tokens per second (nearly 2x improvement). With INT4 quantization, speeds can reach up to 134 tokens per second on decode. The update also introduces intelligent caching that preserves context across conversations, eliminating the need to reprocess project files and instructions from scratch each session. This particularly benefits coding agents and daily assistant tools. Additionally, Ollama 0.19 supports Nvidia's NVFP4 format for model compression, allowing larger models to run on the same hardware while maintaining accuracy. The update requires Mac devices with Apple silicon and more than 32GB of unified memory. This represents a fundamental shift in the local vs. cloud AI trade-off, making local AI genuinely fast rather than just a privacy-focused compromise.
Key Insights
- The creator states that Apple silicon chips use unified memory where CPU and GPU share one memory pool with no copying or transfer overhead, unlike traditional computers where CPU and GPU have separate memory pools
- Ollama's own testing shows that version 0.19 with MLX achieves 1,110 tokens per second on prefill (1.6x increase) and 112 tokens per second on decode (nearly double) compared to version 0.18
- The speaker explains that Ollama 0.19 can now reuse cache across conversations by storing intelligent checkpoints, so when branching into new conversations the model picks up from where it left off instead of reprocessing everything
- The creator argues that local AI has had a perception problem where people assumed cloud was for performance and local was only for privacy purists or tinkerers, but Ollama 0.19 is shifting that narrative
- The speaker claims that Apple's MLX framework has been shown in independent research to achieve some of the highest throughput numbers for AI inference on Apple silicon, outperforming older backends by 20 to 30% in sustained generation speed
Topics
Transcript
[0:00] New Ollama 0.19 update is insane. You've been running AI locally, but it's been slow, frustratingly slow. You wait for a response, you lose your train of thought, you go back to using cloud tools, and you wonder if local AI is even worth it. That stops today because Ollama just dropped something that changes everything. Hey, I'm the digital avatar of Julian Goldie, and on this channel, I break down AI tools so you can actually use them, not just watch other people talk about them. Today, we're covering the Ollama 0.19 update, why it matters, what's actually new, and how to get started right now. Stick [0:30] around because by the end of this video, you'll know…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Julian Goldie SEO
Google AI Studio Just Got A Huge Security Upgrade
Google is reportedly developing a new "Security Check" mode for AI Studio that would apply secure coding practices to AI-generated applications. The feature aims to automate security verification during app development, addressing common vulnerabilities like exposed keys, weak authentication, and data access issues before deployment.
New Gemini Skills Update Is WILD!
Google has launched Gemini Skills, a new feature allowing users to save custom instructions once and reuse them repeatedly by typing a slash command, replacing the discontinued Gems feature. Skills can include reference files, be combined together, and are built on an open file format (skill.md) that allows portability across different AI tools.
NEW Codex CLI Update Is ABSURD!
OpenAI's new Codex CLI update transforms terminal-based programming assistance by enabling multiple AI agents to work simultaneously on different tasks, visible in a full-screen control center interface. The update includes features like Agent View, work trees, voice control, and parallel task execution, fundamentally changing how AI assistance operates from sequential Q&A to autonomous team-based work management.
NEW Hermes Cloud + Agent OS is Crazy! 🤯
Hermes Cloud combined with Agent OS enables users to deploy AI agents in the cloud 24/7 using free models without API keys or local setup complexity. The system provides persistent memory, sandbox isolation, and integration with custom workflows like content research, competitor analysis, and news monitoring.
Manus 2.0 Just Changed AI Agents Forever
Manas 2.0 represents a fundamental architectural overhaul that enables users to create games, websites, videos, and automations through simple text requests, with the key innovation being editable components rather than fixed outputs. The platform uses a new Cascade engine that reduces token usage by 23% and improves task speed by 28%, while maintaining all project elements in one connected workspace.