Future of holograms, brain signal playback, smell, and touch in multimedia software | Lex Fridman
Developers of VLC and FFmpeg discuss the future of multimedia frameworks, predicting expansion into holograms, brain-computer interfaces, smell, touch, and volumetric video. They argue that multimedia should be redefined as any timed data representing human senses. They also critique Dolby for shifting from innovation to patent licensing.
Summary
The conversation centers on the long-term trajectory of FFmpeg and VLC, two foundational open-source multimedia frameworks. The speakers express high confidence that FFmpeg will exist in 100 years, while VLC's survival is considered more uncertain. Looking 5–20 years ahead, they anticipate that multimedia will expand far beyond audio and video to encompass brain-computer interface data (referencing Neuralink), volumetric video, point clouds, RGBD depth data, and XR/VR streaming.
The speakers redefine multimedia as 'a digital representation of several streams for the human senses,' arguing this naturally includes smell and touch. They describe existing work on haptic plugins in VLC used for 4D cinema experiences with hydraulic motion platforms, and they envision future codec tracks for stereo smell and haptic feedback. They frame FFmpeg and VLC not as contributors but as maintainers who shape architecture so that external contributors can add new format modules over time.
The speakers describe a recurring industry pattern in emerging multimedia formats: initial fragmentation into multiple competing standards, followed by a hype decline, consolidation around a community-driven standard, and eventual adoption by market leaders under pressure. They cite 3D audio from six or seven years ago as a cautionary example of premature hype followed by a more mature revival.
Finally, the speakers criticize Dolby, characterizing it as a once-innovative company that defined the standard for sound but has since shifted its focus toward lawyers and licensing revenue rather than engineering innovation, comparing its trajectory unfavorably to HP.
Key Insights
- The speakers argue that multimedia should be defined as 'a digital representation of several streams for the human senses,' which inherently includes smell, touch, and eventually brain wave data — making expansion into those domains logically inevitable for FFmpeg and VLC.
- VLC already has a haptic plugin, not included in the standard release, that transports physical and movement data synchronized with content, used in 4D cinema theme park experiences with hydraulic arms.
- The speakers describe their role as maintainers rather than contributors — their job is to architect the frameworks so that a large community (around 150 VLC contributors and 300 FFmpeg contributors per year) can plug in new modules for emerging formats.
- The speakers identify a consistent pattern in multimedia standardization: new formats start with five competing standards driven by hype, the hype collapses, secondary players converge on a shared standard, it gets implemented, and then the market leader is eventually pressured to adopt it — citing 3D audio from six or seven years ago as an example.
- The speakers characterize Dolby as a company that once defined what sound was with great engineering, but has shifted to being primarily driven by lawyers and licensing, implying it no longer innovates at the level it once did.
Topics
Transcript
[0:02] Do you think VLC and FFmpeg will be here 100 years from now? FFmpeg, yes. Yep, FFmpeg, yes. VLC, maybe. What's the future of what what where is where is FFmpeg going? Where is VLC going? Like in the next if you think about like 5 years, 10 years, 20 years. 5 years, 10 years is easy. The question is after that, right? Um the question is do we arrive at something called holograms, right? Yeah, so will VLC and FFmpeg expand to [0:35] whatever Yes. multimedia. Yes. >> The multimedia might become sorry for the pothead expansion of topic, but you know, um if you look at something like Neuralink with brain computer interfaces, it's very possible that…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
How the Civil War ended | Gary Gallagher and Lex Fridman
Gary Gallagher and Lex Fridman discuss how the Civil War ended, focusing on Lee's surrender at Appomattox and Grant's magnanimous treatment of Confederate forces. They explore how Grant's generous surrender terms, influenced by Lincoln's reconciliation goals, contrasted with calls from Radical Republicans for harsher punishment and how this approach facilitated national reunification rather than perpetuating resentment.
White supremacy after the Civil War | Gary Gallagher and Lex Fridman
Historian Gary Gallagher discusses Reconstruction, white supremacy, and Confederate memory in post-Civil War America, emphasizing that Northern troops were insufficient to enforce equal rights for freed Black people, and explaining how the "Lost Cause" narrative allowed the South to reframe the war's purpose while Jim Crow became the mechanism for maintaining white supremacy.
The truth about the Civil War - Why people volunteered to fight | Gary Gallagher and Lex Fridman
Historian Gary Gallagher discusses the distinction between causation and motivation in Civil War volunteerism, explaining that while slavery caused the war, most Northern soldiers were motivated by belief in preserving the Union as a democratic republic. He emphasizes that understanding citizen-soldiers' sense of civic duty and American exceptionalism is essential to comprehending why ordinary people risked their lives for the Union.
Christianity and religion during the Civil War | Gary Gallagher and Lex Fridman
Both Union and Confederate sides during the American Civil War were overwhelmingly Protestant Christian and used religious justification for their military efforts. Interestingly, Confederate leaders like Lee and Jackson were notably more overtly religious in their official communications than Union commanders like Grant and Sherman.
Racism in America after slavery was abolished | Gary Gallagher and Lex Fridman
Gary Gallagher discusses how abolishing slavery was only the beginning of achieving equal rights for Black Americans, noting that most Northern whites prioritized saving the Union over ensuring racial equality. He explains that legal changes like the Civil Rights Act took over a century to achieve, while the deeper work of changing societal attitudes and prejudices remains ongoing and vulnerable to exploitation.