NEW Ollama 0.19 Update is INSANE!
Ollama 0.19 introduces a massive speed improvement for local AI on Apple silicon by integrating with Apple's MLX framework, achieving nearly 2x faster response generation and 1.6x faster input processing. The update also includes smarter caching across conversations and support for Nvidia's NVFP4 format, making local AI competitive with cloud services for the first time.
Summary
Ollama 0.19 represents a major breakthrough in local AI performance, specifically for Apple silicon devices. The update integrates with Apple's MLX machine learning framework, which takes advantage of unified memory architecture where CPU and GPU share the same memory pool without transfer overhead. Benchmark results using Alibaba's Qwen 3.5 35B model show prefill speed increasing from 1,154 to 1,110 tokens per second (1.6x improvement) and decode speed jumping from 58 to 112 tokens per second (nearly 2x improvement). With INT4 quantization, speeds can reach up to 134 tokens per second on decode. The update also introduces intelligent caching that preserves context across conversations, eliminating the need to reprocess project files and instructions from scratch each session. This particularly benefits coding agents and daily assistant tools. Additionally, Ollama 0.19 supports Nvidia's NVFP4 format for model compression, allowing larger models to run on the same hardware while maintaining accuracy. The update requires Mac devices with Apple silicon and more than 32GB of unified memory. This represents a fundamental shift in the local vs. cloud AI trade-off, making local AI genuinely fast rather than just a privacy-focused compromise.
Key Insights
- The creator states that Apple silicon chips use unified memory where CPU and GPU share one memory pool with no copying or transfer overhead, unlike traditional computers where CPU and GPU have separate memory pools
- Ollama's own testing shows that version 0.19 with MLX achieves 1,110 tokens per second on prefill (1.6x increase) and 112 tokens per second on decode (nearly double) compared to version 0.18
- The speaker explains that Ollama 0.19 can now reuse cache across conversations by storing intelligent checkpoints, so when branching into new conversations the model picks up from where it left off instead of reprocessing everything
- The creator argues that local AI has had a perception problem where people assumed cloud was for performance and local was only for privacy purists or tinkerers, but Ollama 0.19 is shifting that narrative
- The speaker claims that Apple's MLX framework has been shown in independent research to achieve some of the highest throughput numbers for AI inference on Apple silicon, outperforming older backends by 20 to 30% in sustained generation speed
Topics
Transcript
[0:00] New Ollama 0.19 update is insane. You've been running AI locally, but it's been slow, frustratingly slow. You wait for a response, you lose your train of thought, you go back to using cloud tools, and you wonder if local AI is even worth it. That stops today because Ollama just dropped something that changes everything. Hey, I'm the digital avatar of Julian Goldie, and on this channel, I break down AI tools so you can actually use them, not just watch other people talk about them. Today, we're covering the Ollama 0.19 update, why it matters, what's actually new, and how to get started right now. Stick [0:30] around because by the end of this video, you'll know…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Julian Goldie SEO
How to Run DeepSeek V4 Flash for FREE!
A tutorial demonstrating how to use DeepSeek V4 Flash for free through Open Code and integration with agent operating systems like Hermes Agent. The speaker showcases building websites and apps using this free AI model and explains how it compares favorably to larger models despite being smaller.
Microsoft Fara1.5 27B NEW Browser Automation Model is WILD!
Microsoft released Phi-3.5, a family of three computer use models (4B, 9B, 27B) that automate browser tasks through vision-based clicking rather than HTML parsing. These open-weight models significantly outperform larger closed-source alternatives like OpenAI's Operator and Google's Gemini 2.0 on web automation benchmarks.
Claude Obsidian 2.0 is INSANE (FREE!)
Claude Obsidian 2.0 is presented as a free AI memory upgrade that allows users to upload files into a folder for permanent retention and linking. The system creates a knowledge graph that learns from business documents, provides sourced answers, and can be shared across teams.
Claude Agent OS is INSANE! 🤯
Julian presents a comprehensive Claude-based agent operating system that integrates multiple AI models, automated workflows, and a persistent memory system to automate daily tasks. The system runs 24/7 and uses free or existing subscriptions, combining tools like voice agents, content creation, competitor monitoring, and real-time news analysis into a single unified dashboard.
NEW ChatGPT Update is INSANE!
OpenAI released a major ChatGPT update featuring a Chrome extension called Side Chat and an improved desktop app that work together to streamline SEO research and content creation. The update allows users to analyze multiple browser tabs simultaneously, highlight text for quick answers, and convert research into finished work without constant tab switching.