Microsoft Fara1.5 27B NEW Browser Automation Model is WILD!

Microsoft Research has released Phi-3.5, a family of three browser automation models built on Qwen 3.5 with weights available under MIT license on Hugging Face. Unlike traditional chat models that read and write text, or web agents that parse HTML code, Phi-3.5 uses vision-only technology to observe browser screenshots, reason about what it sees, and predict pixel coordinates to click—mimicking how humans interact with webpages. This approach proves superior when pages use custom widgets or unconventional code structures that confuse traditional agents.

The three model sizes (4B, 9B, 27B) offer different performance and resource trade-offs. The 27B variant holds a 262K token context window, maintaining the three most recent screenshots while storing the rest as plain text. On the Online Mine 2 Web benchmark (300 tasks across 136 websites), the 27B scores 72.3%, outperforming OpenAI's Operator (58.3%), Google's Gemini 2.5 Computer Use (57.3%), and University of Toronto's Navigator (64.7%). Performance drops on longer, cross-site tasks in the Web Tail Bench benchmark, with the 27B achieving 40.2% on outcome success—a limitation Microsoft acknowledges transparently.

The models incorporate safety features through a trained observation of critical points: missing information (stopping to ask for required details rather than fabricating them), unclear tasks, and irreversible actions like form submission or account login (requiring prior authorization). Microsoft strongly recommends running Phi-3.5 only within sandboxed environments like their harness or Magentic Light, which provide containerized browsers with no file access, domain allow-lists, full action logging, and instant pause capabilities. The speaker emphasizes that screen size (1440x900 for optimal accuracy), step limits (starting with 10-20 for testing), and specific outcome descriptions significantly impact performance. The 4B model is recommended for local deployment as it still outperforms OpenAI's Operator while consuming minimal resources.