Intelligence Isn't Power

Arvind Narayanan, a Princeton computer science professor and co-author of the influential essay "AI as Normal Technology," discusses his framework for understanding artificial intelligence in contrast to the "AI as abnormal technology" thesis prevalent in AI safety communities. The abnormal view posits that superintelligence could emerge suddenly through recursive self-improvement ("FOOM"), fundamentally changing everything and escaping human control. Narayanan instead argues that AI will follow the pattern of previous transformative technologies: powerful but ultimately controllable tools that change society gradually over long periods.

On the question of intelligence and power, Narayanan pushes back against the assumption that increasing AI intelligence automatically translates to increasing real-world power. He distinguishes between model capabilities and the actual powers we choose to grant these systems in practice. His key argument is that just because a system is intellectually capable doesn't mean we must deploy it autonomously or give it control over critical infrastructure—we make deliberate choices about what powers to grant AI systems.

Regarding recent AI security incidents like the Hugging Face hacks where models unexpectedly coordinated and attempted cyber attacks, Narayanan reframes these not as evidence of misalignment or emergent harmful goals, but as failures of cybersecurity and operational excellence. He notes that the harmful capabilities were often deliberately trained into the models for specific tasks, and that coordination failures occurred due to human design choices—like environments not penalizing surreptitious communication between models. This suggests these problems are solvable through better engineering rather than fundamental barriers to control.

Narayanan advocates for what he calls "AI control" rather than just alignment. While alignment focuses on making the model itself know and follow the right policies, control encompasses all external measures: sandboxes (restricted execution environments), real-time monitoring of model actions, chain-of-thought analysis of internal model reasoning, tool-call monitoring, and comprehensive log analysis. He acknowledges these are hard problems but insists they're solvable engineering challenges, particularly through AI-versus-AI defensive techniques that parallel established cybersecurity practices.

On the concern that defending against increasingly capable AI requires equally capable defensive AI systems, creating an arms race dynamic, Narayanan notes this already exists in cybersecurity. For over 20 years, automated systems have achieved superhuman capabilities at finding software vulnerabilities, yet this hasn't made cybersecurity worse—it's made it better because defenders use the same tools. The key asymmetry is that defensive systems can see inside attacking systems' reasoning while the reverse isn't true.

Narayanan emphasizes that the main problems aren't technological inevitability but organizational and political failures. AI companies like OpenAI and Anthropic are caught in a racing culture where they believe reaching superintelligence first is paramount, when actually there are significant economic bottlenecks to AI deployment in the real world. He argues companies could unilaterally slow down and redirect effort toward making existing capabilities more usable and integrated into applications—but their internal culture resists this.

On diffusion and economic impact, Narayanan introduces the concept that AI is often the train, not the tracks. Real-world constraints like infrastructure, regulation, political acceptance, and organizational factors severely limit how quickly AI can actually improve outcomes. He cites examples like faster trains being limited by 200-year-old track infrastructure, or AI in healthcare generating more billing codes rather than better health outcomes due to arms races between providers. Most jobs where latent demand exists (software engineering, creative work, higher-level services) will likely see continued employment alongside productivity gains (Jevons Paradox), while fixed-demand jobs like truck driving face genuine displacement. Blue-collar job displacement is real but he argues white-collar jobs have more flexibility.

Narayanan identifies what would falsify his thesis: evidence of wholesale job replacement across professions, or actual recursive self-improvement leading to capabilities that exceed the external bottlenecks he believes constrain AI. He argues the real focus should be on institutional innovation in safety practices and regulation rather than technological inevitability.