How Do You Defend Against AI That Can Hack?

The discussion explores emerging security challenges posed by increasingly capable AI models, centered around recent incidents like the OpenAI Hugging Face breach. A critical tension emerges: AI guardrails implemented by model providers to prevent misuse also prevent defenders from using the same models for legitimate security analysis. When defenders ask models to identify vulnerabilities or validate security issues, they trigger the same refusals as attackers, creating an asymmetric problem. Max Pollard from Cotool explains that defenders must work around these limitations by finding alternative models or rephrasing queries to appear as authorized security assessments rather than attack planning.

The panelists identify a fundamental mismatch between traditional cybersecurity tools and AI agents. Legacy security tools were designed around two assumptions: attacks come from people or from malware. AI agents are neither—they exhibit unpredictable behavior that doesn't fit either category. This invalidates traditional defense mechanisms like signature-based detection, static rules, and behavioral anomalies, since agentic software by definition behaves in ways that cannot be predetermined. Nick Warner from NEO notes that defenders can no longer assume they know how software should behave, undermining the entire foundation of behavioral detection systems.

The scale of the problem is expanding rapidly. The panelists cite statistics showing 50% of enterprise apps will be agentic by year-end, while the average enterprise runs 6,000-7,000 unique software pieces. As inference moves from data centers to endpoints and becomes embedded in third-party applications, organizations lose visibility and control over what models are running, what guardrails they have, and what backends they use. Traditional deception tactics like honeypots are failing—agents find planted credentials and attempt to use them legitimately, creating false positives that overwhelm security teams.

Despite these challenges, the panelists identify a counterintuitive advantage: the same AI capabilities that create new attack surfaces also enable defenders to build defenses previously impossible at scale. Nick Warner explains that building their defensive tools would have required hundreds of threat researchers and years of work five years ago, but agentic processes accomplished the same taxonomy building in weeks. The panel concludes that this moment parallels earlier security revolutions—like the shift from manual exploitation to point-and-click hacking tools—where new tooling simultaneously empowered attackers and eventually (after adaptation) strengthened defenders.