OpinionInsightful

The Moral Cost of Automated Decisions | Adam Dunigan | TEDxGeorgeMasonU

TEDx Talks

Adam Dunagan, a former Marine and CIA officer, examines the intersection of AI and national security, exposing how autonomous weapons systems are disconnecting lethal decisions from human operators while simultaneously warning that AI-enabled surveillance and job displacement threaten domestic stability. Despite these dark trends, he argues Americans have the capacity and moral responsibility to implement ethical safeguards and choose human dignity over convenience and profit.

Summary

Adam Dunagan opens by establishing his credentials as a former Marine Corps aviator and CIA human intelligence officer who deployed to Afghanistan and served during the ascendancy of AI in national security. He frames his talk around a central question: what are the implications of ceding the role of human beings to AI, especially when human lives are at stake?

The first major focus is modern autonomous warfare. Dunagan presents concerning statistics—a 600% increase in drone casualties and 81% increase in drone attacks in 2005—and highlights how modern warfare is evolving faster than U.S. defense institutions can adapt. He expresses particular alarm about Russian FPV drones equipped with facial recognition systems and no operator input, meaning autonomous AI systems are making kill decisions entirely disconnected from human operators. He draws a historical comparison to the 19th-century horror at the Maxim gun, noting that while the trigger-pull was then still human, modern warfare is automating the decision to kill itself.

Dunagan cites two concrete examples of AI-enabled tragedy. He references 120 dead schoolgirls in Iran killed by a U.S. AI targeting system, arguing that because of how AI intermediates military decision chains, everyone in the chain can disclaim responsibility—no one feels accountable. He also discusses the February conflict between the U.S., Israel, and Iran, where Iran deployed low-cost Shahed drones (20-30k each) powered by AI targeting packages, forcing the U.S. to spend $4 million per interceptor missile in an economically unsustainable arms race. Iran successfully targeted energy infrastructure including the Strait of Hormuz, which remains closed.

The second major theme is state surveillance technology coming home. Dunagan notes that post-9/11 military tactics and equipment used in Iraq and Afghanistan have been redirected toward American citizens through local law enforcement. He cites the ACLU regarding companies like Palantir and Babel Street using machine learning to identify "suspicious activity" patterns—including criticism of the Trump administration—and automate profiling of labeled dissidents. He expresses discomfort with the concept that whoever holds power determines what "dissident" means.

The third concern is job dislocation and its downstream effects. Dunagan cites figures showing nearly 700,000 layoffs due to AI in the past 18 months, with Boston Consulting predicting 10-15% of American workers (16-25 million people) will lose jobs in the next five years. He warns that a country with no social safety net, no universal healthcare, and no universal basic income is unprepared for this disruption. Drawing on his intelligence work abroad, he argues that unemployed young men with weapons access is a recipe for civil unrest, radicalization, and political violence—particularly dangerous when combined with ubiquitous surveillance and militarized state apparatus.

Dunagan highlights a point of light: Anthropic, the AI company whose Claude model powered the targeting system that killed Iranian schoolgirls, discovered the tragedy and revoked U.S. military sales despite anger from the Pentagon and investors. He praises this as rare ethical behavior prioritizing human dignity over profit, setting a standard that should be the minimum expectation.

He concludes by rejecting fatalism and appealing to American capacity for boldness and moral change. He calls for regulatory fixes, quality-of-life improvements, and individual ethical choices in workplaces and communities. He urges listeners to seek positions where hard moral decisions must be made and commit to doing what's right regardless of cost.

Key Insights

  • Russian FPV drones deployed in Ukraine are equipped with autonomous facial recognition systems and no operator input, meaning the decision to kill is made entirely by an AI system with no understanding of human life, fully abstracting the moral consequence of taking a life from human operators.
  • In the U.S. military's chain of decision-making involving AI, every person in the chain can claim 'I didn't plan that, it's not my fault,' resulting in a situation where 120 Iranian schoolgirls were killed but no one feels responsible because the AI intermediates accountability.
  • Iran deployed Shahed drones costing $20,000-$30,000 each paired with AI targeting systems against U.S. military hardware costing billions, forcing the U.S. to fire $4 million interceptor missiles at cheap targets—an economically unsustainable asymmetry that successfully degraded regional energy infrastructure including the Strait of Hormuz.
  • Companies like Palantir and Babel Street use machine learning algorithms to identify patterns labeled as 'suspicious activity'—including criticism of sitting administrations—and then automate surveillance and profiling of those labeled dissident, concentrating power to define dissent in whoever holds political office.
  • Boston Consulting projects that 10-15% of American workers (16-25 million people) will lose jobs to AI in the next five years, and in a country lacking social safety nets, universal healthcare, or universal basic income, this dislocation will collide with existing weapons proliferation and militarized state surveillance to create conditions for civil unrest and radicalization.

Topics

Autonomous weapons systems and AI-enabled lethal decision-makingState surveillance, data collection, and automated profiling of citizensJob displacement from AI and economic security risksMilitary-to-domestic technology transfer and militarization of law enforcementEthical accountability gaps when AI intermediates human decision chainsCost asymmetry in modern warfare (expensive interceptors vs. cheap autonomous drones)Corporate ethics in AI development and arms restraintCivil unrest and radicalization risks from economic dislocation

Transcript

[0:00] [applause] >> What happens when every aspect of human life is attenuated, numbed by a layer of artificial separation from the very things that make us human? My name is Adam Dunagan. I served in the Marine Corps during the war on terror and I was a case officer at CIA. I worked undercover for a long time. I'm here today to talk about the nexus of AI and national security and I'm about to walk you through some dark, anxiety-inducing trends in modern warfare. [0:32] Not just modern modern warfare, though. Modern warfare, state surveillance, and the potential for significant social fracture. Not exactly light-hearted topics, I know, but I promise we'll get through this. Don't worry. The…

Full transcript available for MurmurCast members

Sign Up to Access

More from TEDx Talks

Get AI summaries like this delivered to your inbox daily

Get AI summaries delivered to your inbox

MurmurCast summarizes your YouTube channels, podcasts, and newsletters into one daily email digest.