DiscussionOpinion

Did AI kill research papers? - mathematicians respond | Lex Fridman

Lex Clips

Mathematicians discuss how AI and changing research practices are transforming academic contributions beyond traditional papers, exploring new models like formal verification with Lean and collaborative GitHub-style approaches, while emphasizing the need to preserve the careful, community-driven verification process essential to mathematical truth.

Summary

The conversation addresses how the traditional research paper model is evolving in light of AI and increased research output. In applied sciences and engineering, academic contributions are already shifting beyond single papers to include reusable code, data, APIs, and tools that support larger collaborative projects like LIGO or cancer research initiatives. The speaker argues that such modular, reusable components are increasingly valuable, especially when leveraged by AI systems.

A critical tension emerges around journal and review systems. While journals were already strained before AI due to reviewer exhaustion and volume, the speakers argue that replacing this system entirely would be worse because mathematics requires a deliberately slow, careful, and precise verification process. The Perelman example illustrates this: his Poincaré conjecture solution required years of community verification through lectures and detailed scrutiny.

The discussion explores new organizational models that could relieve pressure without abandoning rigor. One example is Mathlib's pull-request model on GitHub, where contributions are collaboratively refined in public, with reviewers suggesting improvements and helping identify existing solutions. This contrasts with traditional papers, where contributions are finalized and individual.

The speakers discuss Lean, an interactive theorem prover, and Mathlib, its mathematical library, as enabling formal verification of proofs down to axioms. This addresses a key limitation: natural language AI proofs cannot be trusted because AI systems are stochastic, but formal Lean verification provides certainty. The conversation suggests that formal mathematics could become more like science, with specifications built into objects and crowdsourced improvement.

A notable tension appears between mathematics culture and organizational needs: mathematicians are traditionally independent and solitary, resisting external direction, yet large-scale problems increasingly require collaboration and leadership. The speakers suggest finding new models not to replace the old system but to relieve pressures created by AI-generated content and increased research volume.

Key Insights

  • In engineering and applied sciences, academic contributions have already shifted from single papers to reusable code, data, APIs, and datasets that support larger collaborative projects, making modular contributions more valuable than isolated results
  • Mathematical epistemology requires a deliberately slow and precise community-driven verification process, exemplified by Perelman's Poincaré conjecture solution which took years of worldwide scrutiny through lectures and detailed lemma-by-lemma checking
  • Open source maintainers are becoming psychologically overwhelmed by pull requests and exhaustion, creating a failure mode where the human motivation necessary for the system's success collapses
  • Mathlib's GitHub pull-request model enables crowdsourced mathematics where reviewers publicly suggest improvements and identify reusable ideas, creating a more transparent and collaborative process than traditional anonymous journal review
  • AI-generated natural language proofs cannot be trusted because AI systems are stochastic, but formal Lean verification provides certainty by checking proofs down to axioms, enabling autoformalization as a solution to proof verification

Topics

Academic publishing and the future of research papersAI-generated mathematical proofs and verificationLean and formal verification of mathematicsMathlib and collaborative mathematics via GitHubJournal peer review system strain and alternativesOpen source and community-driven mathematicsMathematician culture and collaboration models

Transcript

[0:02] You know, that would be my guess. Can I ask a almost silly-sounding question? But so a large part for many decades of research has been the paper. If you were to guess in the years to come, how does that change how a human contributes their idea, maybe collaborate, how a team of humans contribute an idea into the public space? - So, I mean, I don't know about in pure math, but in engineering and some of the applied sciences, we already have a pretty [0:34] rapidly shifting view of what an academic contribution is. So if you write code in part of a larger infrastructure that is reusable, that, you know, supports larger projects, so think…

Full transcript available for MurmurCast members

Sign Up to Access

More from Lex Clips

Get AI summaries like this delivered to your inbox daily

Get AI summaries delivered to your inbox

MurmurCast summarizes your YouTube channels, podcasts, and newsletters into one daily email digest.