The Warning Label Is The Pitch
Three hosts discuss AI capabilities and hype with physicist Daniel Green, focusing on how AI solved mathematics problems through brute-force computation rather than genuine breakthroughs, and arguing that the real danger of AI lies in human stupidity and misuse rather than autonomous AI becoming sentient or killing humanity.
Summary
The episode features Daniel Green, a physicist at UC San Diego specializing in particle physics and cosmology, discussing recent AI achievements in mathematics and science. The hosts examine OpenAI's claim to have solved a Millennium Prize problem regarding the Navier-Stokes equations by finding a counterexample. Green explains that AI's success in mathematics comes from its ability to exhaustively search for counterexamples across different mathematical domains, rather than generating novel theoretical insights. He notes that humans tend to pattern-match and become demoralized, while AI can systematically pursue possibilities without fatigue. However, Green emphasizes that this differs from genuine scientific progress, which typically emerges from human curiosity about anomalies, not from solving predetermined prize problems.
The hosts critique OpenAI's reported spending of $15 million in compute over three days to beat competitors to these results, which alienated mathematicians who had contributed their research to OpenAI's tools. This raises questions about whether AI companies' stated model of supporting researchers conflicts with their competitive behavior. Green argues that the math and science communities need to reassess what's valuable in their fields now that AI can generate proofs and solve mathematical problems. Unlike physics, which long ago adapted to computers handling calculations, mathematics never confronted automation of its core activities until recently.
The conversation pivots to existential risk narratives around AI. Marco argues that AI companies' warnings about AI causing human extinction are primarily marketing tactics designed to justify astronomical valuations. He draws parallels to regulatory capture, where corporations write the rules they're ostensibly subject to, suggesting AI companies are anchoring investors to a "terminal valuation" where AI becomes godlike—if there's even a small chance of such an outcome, investors will fund it regardless. Jacob notes that the real danger involves humans using AI to create bioweapons or making poor decisions (like the Navy officer who nearly triggered an international incident based on a ChatGPT hallucination about nuclear material on a Chinese ship).
Green pushes back on doomsday scenarios, arguing that creating novel pathogens remains practically difficult because evolution has already been optimizing organisms for billions of years, and simulating complex biological systems remains beyond current computational capability. He uses the paperclip maximizer thought experiment to note that the scenarios people fear (AI pursuing instrumental goals without regard for humans) already describe how corporations operate. The actual problem, according to all three, is not rogue AI but stupid humans in positions of power over-trusting AI without expertise.
The discussion connects this to a broader concern about the decline of expertise and scientific funding in the United States post-COVID, when skepticism of experts increased. Green contrasts this with the 1950s-60s atomic age, when technological progress was led by respected experts. The hosts worry that combining anti-expert sentiment with over-reliance on AI creates dangerous conditions where unqualified people make consequential decisions based on AI outputs they don't understand.
Finally, they discuss AI's limitations in capturing human behavior and preferences. Green notes that human behavior is chaotic and evolves constantly, making it impossible for AI trained on historical data to predict what humans will want tomorrow. Jacob and Marco agree that AI writing tends toward mediocrity (Hallmark-level content) and that while AI may raise the floor for average content production, it cannot match human creativity and originality. The hosts conclude that AI will be useful for basic tasks and average consumers but won't replace genuine expertise or creative work, though most people don't yet understand this distinction.
About this episode
<p>OpenAI burned $15 million in three days to beat Anthropic to a Millennium Prize problem. Nobody's quite sure whose work they trained on to do it. This week the cousins bring in their science cousin - physicist Dan Green, Marko's college roommate - to explain what AI is actually doing to math, and why the whole thing smells less like a real breakthrough and more like an elaborate flex. </p><p>And... when a company tells you that their product might end humanity, is that a warning or a pitch deck? Marko has thoughts. Plus: why the dangerous scenario isn't the robot, it's the idiot who trusts it.</p><p>--</p><p><strong><u>Timestamps:</u></strong></p><p>(00:00) - Welcome Back and Guest Intro</p><p>(00:48) - Meet Dan the Science Cousin</p><p>(04:24) - AI Solves Big Math Problems</p><p>(09:22) - Backlash and Research Ethics</p><p>(16:03) - What AI Means for Science</p><p>(21:47) - Who Pays for Discovery</p><p>(34:23) - Terminator Talk and Real Risks</p><p>(45:18) - Corporations vs Paperclip AI</p><p>(46:58) - Chatbot Intel Near Miss</p><p>(49:10) - Anti Expert Government Risk</p><p>(51:59) - Future Robots and Human Agency</p><p>(58:59) - AI as Godlike Branding</p><p>(01:03:03) - Nuclear Energy Lessons</p><p>(01:11:06) - Why AI Writing Feels Wrong</p><p>--</p><p><strong><u>Referenced in the Show:</u></strong></p><p>Dan Green - <a href="https://drgreen.physics.ucsd.edu/" rel="noopener noreferrer" target="_blank">https://drgreen.physics.ucsd.edu/</a></p><p>--</p><p><strong>Geopolitical Cousins</strong> is produced and edited by Audiographies LLC. More information at <a href="https://audiographies.com" rel="noopener noreferrer" target="_blank">audiographies.com</a></p><p>--</p><p><strong>Jacob Shapiro</strong> is a speaker, consultant, author, and researcher covering global politics and affairs, economics, markets, technology, history, and culture. He speaks to audiences of all sizes around the world, helps global multinationals make strategic decisions about political risks and opportunities, and works directly with investors to grow and protect their assets in today’s volatile global environment. His insights help audiences across industries like finance, agriculture, and energy make sense of the world.</p><p><strong>Jacob Shapiro Site:</strong> <a href="https://jacobshapiro.com" rel="noopener noreferrer" target="_blank">jacobshapiro.com</a></p><p><strong>Jacob Shapiro LinkedIn: </strong><a href="https://www.linkedin.com/in/jacob-l-s-a9337416" rel="noopener noreferrer" target="_blank">linkedin.com/in/jacob-l-s-a9337416</a></p><p><strong>Jacob Twitter:</strong> <a href="https://x.com/JacobShap" rel="noopener noreferrer" target="_blank">x.com/JacobShap</a></p><p><strong>Jacob Shapiro Substack: </strong><a href="https://jashap.substack.com/subscribe" rel="noopener noreferrer" target="_blank">jashap.substack.com/subscribe </a></p><p>--</p><p><strong>Marko Papic</strong> is a macro and geopolitical expert at BCA Research, a global investment research firm. He provides in-depth analysis that combines geopolitics and markets in a framework called GeoMacro. He is also the author of Geopolitical Alpha: An Investment Framework for Predicting the Future.</p><p><strong>Marko’s Book & Newsletter:</strong> <a href="https://www.geopoliticalalpha.com/marko-papic" rel="noopener noreferrer" target="_blank">www.geopoliticalalpha.com/marko-papic </a></p><p><strong>Marko’s Linkedin:</strong> <a href="https://www.linkedin.com/in/marko-papic-geopolitics/" rel="noopener noreferrer" target="_blank">https://www.linkedin.com/in/marko-papic-geopolitics/</a></p><p><strong>Marko’s Twitter:</strong> <a href="https://x.com/Geo_papic" rel="noopener noreferrer" target="_blank">https://x.com/Geo_papic</a></p><p><strong>Marko’s Macro & Geopolitical Research at BCA:</strong> <a href="https://www.bcaresearch.com/marketing/geomacro" rel="noopener noreferrer" target="_blank">https://www.bcaresearch.com/marketing/geomacro</a></p>
Key Insights
- AI excels at finding counterexamples through brute-force computational search across mathematical domains, but this is not the same as generating novel theoretical insights or advancing mathematical understanding
- OpenAI spent $15 million in compute over three days to beat competitors to a mathematical result, then used that success as marketing, despite mathematicians who contributed their research to OpenAI's tools feeling their work was appropriated
- Mathematics as a field never adapted to automation of its core activities the way physics did with calculators and computers, making it uniquely vulnerable to disruption by AI
- AI companies' public warnings about existential AI risk serve as branding to justify trillion-dollar valuations, anchoring investors to a scenario where even a small probability of godlike AI outcomes justifies massive investment
- The real danger of AI is not autonomous superintelligence but humans using AI without expertise, as evidenced by a Navy officer nearly triggering international conflict by trusting ChatGPT's hallucination about nuclear material on a Chinese ship
- Creating novel pathogens computationally faces fundamental obstacles because evolution has optimized organisms for billions of years and simulating complex biological systems remains far beyond current computational capability
- The scenarios people fear about AI pursuing instrumental goals without regard for human welfare already describe how corporations routinely operate under profit maximization incentives
- Government funding for basic science exists because companies cannot recover the value from scientific progress, but AI companies are spending billions on science as an advertising product without understanding the long-term business model
- Post-COVID anti-expert sentiment in the United States creates dangerous conditions when combined with over-reliance on AI, as unqualified people in government positions make consequential decisions based on AI outputs
- AI cannot predict evolving human preferences because human behavior is a chaotic system that constantly changes—what's popular or valued tomorrow differs from today, making historical training data unreliable
- AI-generated writing tends toward mediocrity because it pattern-matches to what it calculates humans want, producing Hallmark-level content rather than capturing what truly engages human interest: novelty, depth, and unexpected connections
- AI will raise the floor for average content and basic tasks but cannot raise the ceiling for expertise and creative work, yet most people misunderstand AI's capabilities and believe it can replace specialized knowledge
Topics
Transcript
Hello listeners, welcome to another episode of Geopolitical Cousins. We apologize for the brief break, but both Marco and I have jobs and there was a lot of travel happening the last two weeks. Joining us on the show today is Daniel Green, Associate Professor at UC San Diego. We wanted to talk about artificial intelligence and physics and all sorts of other things. I know I've been promising this for a while we're in the process of getting that cousins email address set up for the meantime you can email me at jacob jacobshapira.com if you have any questions comments concerns things you want me and marco to see other than that hopefully cadence gets back on our normal…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Geopolitical Cousins
Flies to Poop
In this compressed 50-minute episode of Geopolitical Cousins, the hosts discuss Middle East instability driven by opportunistic actors (Houthis, Iraqi militias) seeking payment rather than ideology, European structural reforms aimed at creating competitive global companies and a single market for services, and concerns about AI regulatory capture by frontier model companies limiting competition and innovation.
The Fourth Reich
Two geopolitical analysts discuss the rise of Germany's AFD party in recent elections, arguing that while it represents a rightward populist shift, it doesn't constitute a return to Nazism but rather reflects legitimate concerns about immigration, energy policy, and restrictions on political discourse. They then pivot to analyzing the Iran-Houthi conflict, arguing that escalations are being managed by market forces and Greek shipping companies rather than leading to catastrophic oil price spikes.
Rabies-infested beavers
In this episode of Geopolitical Cousins, hosts Jacob and Marco analyze the escalating US-Canada trade war, arguing that Mark Carney's retaliatory strategy is stronger than it appears because Canadians have higher pain tolerance, fiscal space, and can wait out Trump's administration. They contend that the US fundamentally misunderstands the deal it has with Canada, which involves Canadian defense spending that actually protects American interests, and that Trump's approach threatens to undermine decades of North American alliance-building with potentially severe long-term geopolitical consequences.
Scotty Doesn't Know
In this episode of Geopolitical Cousins, the hosts discuss rising U.S. Treasury bond yields, Scott Bessent's interventions to stabilize markets, the geopolitical drivers of the bond sell-off (particularly the Iran conflict), and broader economic concerns about AI CapEx sustainability and political populism threatening data center development.
The King of Israel
The podcast episode discusses the ongoing geopolitical dynamics surrounding Iran, the U.S., and the Russia-Ukraine conflict, highlighting how these situations affect global perceptions and alliances. The speakers contemplate the implications of shifting alliances, especially the potential normalization of relations between the U.S. and Iran, and the corresponding impact on regional stability in the Middle East.