HN837: Agentic AI to Reduce MTTR

Edouard Douleroux, an AI startup founder, explains how agentic AI can transform network engineering by elevating engineers from firefighters to system architects rather than replacing them. The core system comprises five specialized agents led by a team leader that processes syslog data to detect incidents and route them to appropriate specialized agents (routing, security, etc.). The architecture relies heavily on a deterministic workflow encoded in YAML files—a "single source of truth" that documents known incidents, their resolutions, troubleshooting procedures, and execution order for MCP tool calls. This prevents the system from making arbitrary decisions and ensures consistency. When an incident is detected, the team leader agent automatically triggers data collection using MCP tools in a predetermined sequence. The collected contextual data feeds into a fine-tuned language model (specialized variants for different domains like BGP, routing, security) that reasons through the data and proposes a solution. Critically, before any change is applied to production, the proposed solution is tested in a digital twin—a one-to-one replica of the network running in containerized environments (Container Lab), automatically synced every 10 minutes from production. If testing succeeds with a confidence score above 0.8, the engineer receives a dashboard report containing the problem, collected evidence, reasoning trace, and validated solution. A compliance officer agent verifies all proposed changes against organizational policies (change windows, freeze periods, etc.) before approval. Douleroux emphasizes that fine-tuning models is the most critical and tedious component of the entire system. The process involves multiple stages: supervised fine-tuning on curated question-answer datasets (requiring tens of thousands of examples), held-out test sets to prevent memorization, evaluation against both the base model and frontier models like GPT-4, and Direct Preference Optimization (DPO) to align model behavior with organizational preferences. He stresses that high-quality, diverse datasets are essential—showing the model many angles and variations of the same concept so it generalizes rather than memorizes. Models cannot perform mathematical operations natively but learn patterns through examples. He uses open-source models (Qwen, CodeLlama, Mistral) fine-tuned on vendor documentation, best practices, and troubleshooting guides to create vendor-agnostic expertise in networking technologies. The entire system is built using Python scripts and local hardware; Douleroux uses a personal server with 128GB RAM and 8 performance cores for approximately $3,250. He argues that while organizations often avoid investing in proper lab environments (viewing them as unnecessary overhead), the cost of not having testable changes far exceeds the infrastructure investment—companies lose millions per hour during major outages. He advocates that the industry needs public domain models trained on foundational networking knowledge applicable across vendors, while private models handle organization-specific context (AS numbers, IP addresses, firewall rules, internal policies). Agents require strict confinement through both ignorance (limited knowledge in YAML) and policy-based RBAC to prevent unintended actions. He positions agentic AI as complementary to, not replacing, deterministic programming—classical scripts handle well-defined scenarios reliably, while models excel at capturing corner cases and handling unpredictable input variations that would require continuous code updates.