NAN126: Fine-Tuning Open Source LLMs for Network Engineering

Edward Tuharu shares his 25+ year career trajectory spanning military radar systems and enterprise networking, culminating in his recent decision to build an AI startup. After failing his second CCIE exam in 2022, he encountered ChatGPT and read the 'Attention is All You Need' paper, which fundamentally changed his perspective on technology's future. He explains that the transformer architecture's attention mechanism—which allows parallel processing of words and identification of meaning-carrying keywords—represented a breakthrough that would eventually surpass human expertise. Rather than pursue a third CCIE attempt, he chose to become deeply familiar with this emerging technology.

Tuharu describes a methodical learning journey through AI technologies, viewing each innovation as solving previous problems while introducing new ones—much like networking protocol evolution from Spanning Tree through VPC, Fabric Path, and VXLAN. He traces his path through prompting (optimizing how to structure interactions with models), RAG/Retrieval-Augmented Generation (grounding responses in updated domain-specific databases to solve hallucination problems), fine-tuning (shaping model behavior through training on proprietary data), and finally agents with model context protocol (MCP) for taking actions.

When discussing model selection between Llama, Mistral, and Qwen, Tuharu emphasizes understanding pre-training data types. He distinguishes reasoning models from general language models, explaining that reasoning models are trained with reward functions incentivizing creative problem-solving. He contrasts the US approach of reinforcement learning from human feedback (RLHF) with recent Chinese research using reinforcement learning from verifiable outcomes, which allowed models to develop reasoning as their optimization strategy without human validation bottlenecks.

Tuharu details the fine-tuning process: defining the problem, collecting high-quality verified data, identifying appropriate GPU resources (describing memory calculations for models with billions of parameters), selecting fine-tuning algorithms with empirically-determined hyperparameters, and testing against predefined benchmarks. He emphasizes that fine-tuning is the practical investment for those with limited budgets, rather than attempting full model retraining, which he nevertheless experienced through participation in a MIT-led collaborative course on building large language models from scratch.

He advocates for learning networking and AI fundamentals deeply—understanding why protocols exist, not just how they work—drawing inspiration from Dr. Russ White's teaching philosophy. He positions his upcoming book 'Gen AI for Network Engineers' as encapsulating hard-learned lessons and failures, progressing from fundamentals through prompting, RAG, agents, and fine-tuning to help network engineers integrate AI tools into their skill sets.