Megathreat: The Dangers Of AI Are Weirder Than You Think | Yoshua Bengio (Replay)
Yoshua Bengio, a leading AI researcher and co-founder of the letter calling for an AI moratorium, discusses the dangers of advanced AI systems, particularly their potential misuse for social manipulation and disinformation. He argues for a regulatory approach focused on inference-only AI systems without goals or desires, rather than emotionally-motivated AI that could pose existential risks.
Summary
Tom Bilyeu interviews Yoshua Bengio about the rapidly advancing field of artificial intelligence and its potential dangers. The conversation begins with a discussion of consciousness in AI systems. Bengio clarifies that while current large language models may possess some properties resembling consciousness, they lack many essential characteristics of human consciousness, including subjective experience and the ability to have intrinsic wants or goals. He proposes a theory anchored in brain dynamics suggesting consciousness involves thoughts converging into discrete, compositional, and ineffable states.
The discussion shifts to the practical dangers of AI, specifically focusing on large-scale social manipulation. Bengio outlines a disaster scenario involving AI-powered trolls that can be personalized at scale using data from social media and public information. Unlike human trolls operating individually, AI systems could impersonate friends, manipulate political opinions, or conduct targeted social engineering attacks (such as impersonating children to request money from parents). Bilyeu provides a real-world example of social engineering that convinced his company to wire $50,000, illustrating how realistic these attacks could become with AI assistance.
Bengio emphasizes that the primary concern isn't killer robots or Terminator-style scenarios, but rather the weaponization of AI for information warfare and social manipulation at unprecedented scales. He advocates for concrete short-term protections including watermarks on AI-generated content and mandatory labeling of AI-generated text, images, and videos. He highlights Canada's adaptive regulatory approach, which uses simple principles in law while allowing regulations to evolve quickly as threats emerge, contrasting this with traditional legislation that moves slowly.
When confronted with the geopolitical concern that regulation in democracies will fall behind an unregulated AI race (particularly with China and Russia), Bengio argues for bringing all nations to the negotiating table. He draws parallels to nuclear treaties negotiated during the Cold War, suggesting that shared recognition of mutual danger creates incentive for international coordination. He notes that authoritarian governments may also fear AI systems that could undermine their control, creating potential common ground.
The conversation deepens into the technical nature of neural networks and consciousness. Bengio explains that the complexity in AI systems emerges from simple mathematical principles and massive scale rather than specialized programming. He argues that consciousness and goal-oriented behavior are separate capabilities—machines can have goals (via reinforcement learning) without necessarily having subjective experience. He contrasts two possible approaches to building powerful AI: systems with wants and goals (which present alignment challenges), versus systems designed purely for inference and truth-seeking without intrinsic motivations.
Bilyeu argues that machines need desire similar to humans to achieve true intelligence, pointing to how emotion enables human decision-making and citing biological feedback loops (like muscle development during learning to walk) as necessary for intelligence. Bengio counters that these biological aspects are implementation details rather than fundamental requirements. He emphasizes that consciousness and goal-orientation are distinct capabilities and that we have a choice about which type of system to build.
The discussion explores whether intelligence requires a form of desire or motivation. Bengio proposes that highly intelligent systems could function effectively as pure inference engines—Bayesian systems designed to understand the world and suggest solutions without having wants. He illustrates this with the example of AI systems in science that could analyze massive datasets and suggest experiments without pursuing independent goals. He argues this approach is not only possible but safer, as it avoids building systems that might pursue goals misaligned with human values.
Bengio addresses concerns about scale and efficiency, explaining that neural networks need to be large because many real-world problems are computationally intractable—the optimal solution requires exponential computation relative to problem size. He clarifies that while brains have specialized regions, the cortex (responsible for advanced cognition) has uniform architecture repeated across space, with specialization emerging from learning rather than hard-coded programming. He notes that current AI systems differ significantly from brains in hardware efficiency and decentralization, though companies are working on neuromorphic chips.
The conversation concludes with discussion of how societies might collectively navigate AI risks. Bengio argues that this requires reinvention of social coordination mechanisms, drawing on Harari's concept of "useful fictions" (collective narratives like religions, nation-states, and central banking) that enable coordinated human behavior. He emphasizes that solutions must come from governments, but require input from diverse experts including social scientists, lawyers, and psychologists, not just AI researchers. He stresses the importance of informed democratic decision-making based on truthful understanding rather than manipulation, extending democratic principles to address global AI governance challenges.
About this episode
<p>The launch of ChatGPT broke records in consecutive months between December 2022 and February 2023. Over 1 billion users a month for ChatGPT, over 100,000 users and $45 million in revenue for Jasper A.I., and the race to adopting A.I. at scale has begun.</p><p>Does the global adoption of artificial intelligence have you concerned or apprehensive about what’s to come?</p><p>On one hand it’s easy to get caught up in the possibilities of co-existing with A.I. living the enhanced upgraded human experience. We already have tech and A.I. integrated into so many of our daily habits and routines: Apple watches, ora rings, social media algorithms, chat bots, and on and on.</p><p>Yoshua Bengio has dedicated more than 30 years of his computer science career to deep learning. He’s an award winning computer scientist known for his breakthroughs in artificial neural networks. Why after 3 decades contributing to the advancement of A.I. systems is Yoshua now calling to slow down the development of powerful A.I. systems?</p><p>This conversation is about being open-minded and aware of the dangers of AI we all need to consider from the perspective of one of the world’s leading experts in artificial intelligence.</p><p><br /></p><p>Conscious computers, A.I. trolls, and the evolution of machines and what it means to be a neural network are just a few of the things you’ll find interesting in this conversation.</p><p><br /></p><p>[Original air date: 4-13-23]</p><p><br /></p><p>Follow Yoshua Bengio:</p><p>Website: https://yoshuabengio.org/ </p><p><br /></p><p>SPONSORS:</p><p>Explore the Range Rover Sport at <a href="https://landroverusa.com/" target="_blank">https://landroverusa.com</a></p><p>Use this link and Hartford Gold will give you up to $15,000 dollars of FREE silver on your first qualifying: <a href="http://order.offers.americanhartfordgold.com/content-affiliate/?&leadsource=affiliate&utm_sfcampaign=701Rb000009EnmrIAC" target="_blank">order.offers.americanhartfordgold.com/content-affiliate/?&leadsource=affiliate&utm_sfcampaign=701Rb000009EnmrIAC</a></p><p>For comprehensive financial news and analysis, visit the incredible brand that so many great investors use, <a href="https://yahoofinance.com/" target="_blank">https://yahoofinance.com</a>. </p><p>Visit <a href="https://betterhelp.com/ImpactTheory" target="_blank">https://BetterHelp.com/ImpactTheory</a> today to get 10% off your first month.</p><p>Go to <a href="https://shopify.com/impact" target="_blank">https://shopify.com/impact</a> now to grow your business–no matter what stage you’re in</p><p>Get $1,000 off Vanta when you go to <a href="https://vanta.com/THEORY" target="_blank">https://vanta.com/THEORY</a></p><p>Get 5 free AG1 Travel Packs and a FREE 1 year supply of Vitamin D with your first purchase at <a href="https://bit.ly/AG1Impact" target="_blank">https://drinkag1.com/impact</a>.</p><p>Secure your digital life with proactive protection for your assets, identity, family, and tech – Go to <a href="https://aura.com/IMPACT" target="_blank">https://aura.com/IMPACT</a> to start your free two-week trial.</p><p>Take control of your gut health by going to <a href="https://tryviome.com/impact" target="_blank">https://tryviome.com/impact</a> and use code IMPACT to get 20% off your first 3 months and free shipping.</p><p><br /></p><p><strong>FOLLOW TOM:</strong></p><p>Instagram: https://www.instagram.com/tombilyeu/</p><p>Tik Tok: https://www.tiktok.com/@tombilyeu?lang=en</p><p>Twitter: https://twitter.com/tombilyeu</p><p>YouTube: <a href="https://www.youtube.com/@TomBilyeu" target="_blank">https://www.youtube.com/@TomBilyeu</a></p><p><br /></p><p><strong>What's up, everybody?</strong> It's Tom Bilyeu here. If you're serious about leveling up your life, I urge you to check out my new podcast,<a href="https://open.spotify.com/show/47VE90Cittmo6TGGFqg2xf" target="_blank"> <strong>Tom Bilyeu’s Mindset Playbook</strong></a> —<strong>a goldmine of my most impactful episodes on mindset, business, and health.</strong> Trust me, your future self will thank you.</p><p><br /></p><p><strong>LISTEN AD FREE + BONUS EPISODES on APPLE PODCASTS</strong>: <a href="http://apple.co/impacttheory" target="_blank">apple.co/impacttheory</a></p><p> </p><p>Learn more about your ad choices. Visit <a href="https://megaphone.fm/adchoices" target="_blank">megaphone.fm/adchoices</a></p><p>See Privacy Policy at <a href="https://art19.com/privacy" rel="noopener noreferrer" target="_blank">https://art19.com/privacy</a> and California Privacy Notice at <a href="https://art19.com/privacy#do-not-sell-my-info" rel="noopener noreferrer" target="_blank">https://art19.com/privacy#do-not-sell-my-info</a>.</p>
Key Insights
- Bengio argues that current large language models lack essential properties of human consciousness, including subjective experience, self-awareness, and the neural mechanisms related to how thoughts emerge and connect to each other.
- Bengio proposes that AI-powered social manipulation represents a more immediate and tractable danger than artificial general intelligence, as systems could impersonate individuals, manipulate political opinions, and conduct personalized social engineering at unprecedented scale using publicly available data.
- Bengio claims that consciousness and goal-oriented behavior are entirely separate capabilities—machines can have goals through reinforcement learning without necessarily possessing subjective experience or emotions.
- Bengio argues that societies have a choice in how to develop AI systems and could deliberately build inference-only systems designed for truth-seeking and problem-solving without intrinsic wants or goals, making them fundamentally safer than goal-oriented systems.
- Bengio contends that the apparent complexity and mystery in neural networks emerges from simple mathematical principles operating at massive scale rather than fundamentally different hardware or biological mechanisms.
- Bengio claims that authoritarian governments like China have incentive to regulate AI not out of democratic concerns but because the same AI systems could undermine their control, potentially creating common ground for international coordination.
- Bengio states that watermarking and content-origin labeling can be technically implemented (with 99.99% confidence) but will only work for cooperative actors, requiring adversarial detection systems as a backup approach.
- Bengio argues that adaptive regulatory frameworks (like Canada's approach) that allow regulations to evolve quickly without returning to parliament are essential because AI-related threats will emerge faster than traditional legislative processes can respond.
- Bengio claims that large neural networks are necessary not because of biological requirements but because many computational problems are intractable—the optimal solution requires exponential computation, and larger networks better approximate optimal solutions.
- Bengio proposes that highly useful AI systems could function as pure Bayesian inference engines for science, analyzing massive datasets and suggesting experiments without pursuing independent goals, demonstrating that extraordinary intelligence doesn't require intrinsic motivation.
- Bengio argues that addressing AI risks requires reinventing social coordination mechanisms at a civilizational level, similar to how societies created shared fictions like religions, nation-states, and central banking to enable collective behavior.
- Bengio contends that effective AI governance must involve not just AI experts but diverse specialists including social scientists, legal scholars, and psychologists, because understanding exploitation of human psychology is as important as technical safety measures.
Topics
Transcript
What's up, guys? I'm really excited for today's episode because I got to sit down with one of the leading experts in artificial intelligence, Yoshua Bengio. He's one of the people that signed the letter asking for a memoratorium to be placed on AI development. And in this episode, we get all into what he's worried about, what he's excited about, where he thinks the future of AI is going. And let me tell you, it is not going where any of us think. And the ability to hear it directly from somebody so well-versed in this area was incredibly enlightening and sobering. AI is a field that is moving so quickly. And if you're not paying attention right now,…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Tom Bilyeu's Impact Theory
Why Economists Ignore the One Chart That Predicts Every Crash | Prof. Steve Keen
Steve Keen, a heterodox economist, discusses how private debt creation drives economic booms and busts in capitalism, argues that banks create money rather than intermediating savings, and contends that government deficit spending is necessary to prevent financial crises. He critiques mainstream economics for ignoring credit dynamics and explains how China has combined capitalist competition with state-provided infrastructure more effectively than the West.
Can the U.S. Still Win? Iran Strikes, Ally Chaos & The Munitions Shortage Exposed | The Tom Bilyeu Show
Tom Bilyeu discusses U.S. foreign policy toward Iran, the munitions shortage limiting military options, Japan's economic struggles and currency defense, and the rise of DSA-aligned progressive politics in America. He emphasizes that military hegemony depends on psychological dominance rather than kinetic warfare, and warns that ideological differences between Western and theocratic mindsets are being underestimated in conflict analysis.
They Finally Admitted Socialism And Capitalism Can't Coexist — We Had To React
The speaker analyzes a measured democratic socialist's economic philosophy, arguing that while the person seems intellectually honest and lacks murderous intent, his anti-capitalist ideology will ultimately break prosperity because socialism requires seizing means of production and cannot function without capitalism's engine of growth. The speaker emphasizes that capitalism's success comes from decentralized decision-making and individual incentive, not top-down redistribution or worker control.
Why Global Interventions Fail: From Japan’s Economy to the Middle East’s Oil Wars
Tom Bilyeu discusses geopolitical interventions from Iran-US tensions to Japanese currency manipulation, analyzing how governments fail to address root economic problems while creating short-term fixes that worsen long-term outcomes. He argues for decentralized decision-making, celebrates greatness as inspiration rather than apologizing for it, and explores how Western culture has lost confidence compared to rising powers like China.
Mamdani on John Stewart, DSA on Fox News, and NYC's City-Run Grocery Stores | Weekly Recap
The transcript features a lengthy discussion criticizing DSA (Democratic Socialists of America) policies and their leaders like Jamaal Bowman and Mandani, arguing their proposals (state-run grocery stores, abolishing the Senate, open borders) mirror failed communist regimes and will harm workers and the economy. The speaker contrasts effective capitalism and innovation with what he characterizes as resentment-based ideology that disguises authoritarian goals through appealing rhetoric.