Can Open Source Keep AI Power From Concentrating?
Lucas Kaiser, co-author of the Transformer paper, discusses how AI power is currently concentrating in large companies due to the resource-intensive nature of current technology, but argues this is not inevitable. He believes research breakthroughs in algorithms and training methods could enable smaller players and distributed models to compete effectively.
Summary
In this episode of the A16Z podcast, host Sophia Du interviews Lucas Kaiser about the concentration of AI power and whether open source can create a more distributed future. Kaiser explains that current AI concentration stems from practical technological requirements rather than fundamental inevitability. Today's dominant architecture, the Transformer, requires enormous amounts of data and compute resources, which only large companies can afford. This has led to a business model where companies train expensive models on massive data centers and charge users for access. However, Kaiser emphasizes that Transformers are less than a decade old and represent just one approach to AI. He argues that humans demonstrate the existence of an alternative model: expertise distributed across specialized domains rather than concentrated in single all-knowing systems. Kaiser points out that current large language models struggle with specialized tasks and domain expertise compared to human experts, suggesting room for algorithmic innovation. He notes that while transformers excel when trained on internet-scale data, they perform poorly on smaller, specialized datasets—a limitation that may be solvable through research breakthroughs in loss functions, training methods, and data approaches rather than just scaling up. Kaiser expresses optimism that increased cost of scaling will redirect research focus toward fundamental breakthroughs. He also highlights democratization of compute: a single RTX 5090 GPU now contains more power than the eight GPUs used to develop Transformers, enabling researchers outside major labs to experiment and innovate. He concludes that the current concentrated state is a temporary phase driven by convenience and technological maturity, not an inherent feature of AI.
About this episode
MTS host Sophia Dew visits the Open Source AI Summit in San Francisco to ask researchers and founders across the AI stack a central question: can open source prevent AI power from concentrating in the hands of a few companies? Lukasz Kaiser, co-author of Attention Is All You Need, argues that today’s concentration may be a feature of the current technological paradigm rather than a permanent feature of AI. Transformers reward enormous amounts of data and compute, but future breakthroughs could make smaller, more specialized models far more capable. Across conversations with researchers and builders working on open models, infrastructure, and applications, Sophia explores why China has taken the lead in open-weight models, whether the U.S. needs more open-model startups, what it means for companies to own their own intelligence, and where openness alone falls short, particularly when access to compute remains concentrated.
Key Insights
- Kaiser argues that current AI concentration is a property of transformer technology and current business practices, not an inevitable feature of AI itself, as evidenced by humans being proof that distributed expertise can exceed generalist performance.
- He claims transformers require internet-scale data to function effectively but perform poorly on specialized datasets, suggesting research breakthroughs in training methods and loss functions—not just scaling—could enable smaller players to build effective specialized models.
- Kaiser observes that while large companies prioritize product development and scaling, increasing costs of training giant models may redirect research focus back toward fundamental algorithmic innovations, creating opportunities for open source projects and academic researchers.
Topics
Transcript
AI is becoming more powerful, but the resources needed to build it are increasingly concentrated. Does it have to stay that way? MTS host Sophia Du heads to the Open Source AI Summit in San Francisco to ask researchers and founders across the AI stack whether open source can create a more distributed future for AI. Lucas Kaiser, co-author of the landmark Attention is All You Need paper, argues that today's concentration may be a property of our current technology, not an inevitable feature of AI. Transformers thrive on enormous amounts of data and compute, but they're less than a decade old, and the next breakthrough could change those economics. From open models and access to compute, to companies owning…
Full transcript available for MurmurCast members
Sign Up to AccessMore from The a16z Show
The Top 100 Consumer AI Apps: Who’s Actually Paying?
A16Z's seventh edition Top 100 Consumer AI Apps Report reveals that while half of Americans use AI, only 4.5% pay for subscriptions, with spending heavily concentrated among developers and power users. The report highlights emerging personal agents as a major trend, significant divergence between ChatGPT, Claude, and Gemini, and substantial untapped opportunities in categories like dating, social, shopping, and entertainment.
David George & Jack Altman on AI, Autonomy, and the Next $25 Trillion
David George from Andreessen Horowitz argues that AI, autonomy, and robotics will create over $25 trillion in market value over the next decade, with success across multiple layers of the stack—frontier models, open source, and applications will all flourish due to massive supply constraints and early-stage demand penetration among knowledge workers.
Beyond the God Model | Alex Atallah & Amjad Masad
Alex Atallah (OpenRouter) and Amjad Masad (Replit) discuss how the future of AI development lies in specialized, diverse models working together rather than relying on single AGI-like models. They argue that enterprises need independence from foundation model providers through model diversity, cost efficiency, and ownership of their own AI capabilities, while also addressing the risks of general-purpose agents and the importance of model specialization.
Why AI Agents Can Beat the Incumbents
Leo, an AI-native startup, is building multi-agent systems to automate end-to-end procurement processes for enterprises by coordinating work across multiple departments, stakeholders, and external systems that traditional procurement software cannot handle. The company differentiates itself from incumbents by owning the entire job workflow rather than being confined to a single system of record, and builds customer trust through human-in-the-loop approaches before scaling to fully autonomous negotiations.
The $1 Trillion AI Buildout | State of Markets
A16Z partners discuss the $1 trillion AI infrastructure buildout, arguing it's driven by real earnings growth rather than inflated valuations, with adoption still extremely early at the enterprise level. They highlight opportunities across consumer agents, robotics, autonomous vehicles, and enterprise diffusion, while noting that the market's 90% gain since ChatGPT reflects fundamental business performance rather than speculative excess.