Future of holograms, brain signal playback, smell, and touch in multimedia software | Lex Fridman
Developers of VLC and FFmpeg discuss the future of multimedia frameworks, predicting expansion into holograms, brain-computer interfaces, smell, touch, and volumetric video. They argue that multimedia should be redefined as any timed data representing human senses. They also critique Dolby for shifting from innovation to patent licensing.
Summary
The conversation centers on the long-term trajectory of FFmpeg and VLC, two foundational open-source multimedia frameworks. The speakers express high confidence that FFmpeg will exist in 100 years, while VLC's survival is considered more uncertain. Looking 5–20 years ahead, they anticipate that multimedia will expand far beyond audio and video to encompass brain-computer interface data (referencing Neuralink), volumetric video, point clouds, RGBD depth data, and XR/VR streaming.
The speakers redefine multimedia as 'a digital representation of several streams for the human senses,' arguing this naturally includes smell and touch. They describe existing work on haptic plugins in VLC used for 4D cinema experiences with hydraulic motion platforms, and they envision future codec tracks for stereo smell and haptic feedback. They frame FFmpeg and VLC not as contributors but as maintainers who shape architecture so that external contributors can add new format modules over time.
The speakers describe a recurring industry pattern in emerging multimedia formats: initial fragmentation into multiple competing standards, followed by a hype decline, consolidation around a community-driven standard, and eventual adoption by market leaders under pressure. They cite 3D audio from six or seven years ago as a cautionary example of premature hype followed by a more mature revival.
Finally, the speakers criticize Dolby, characterizing it as a once-innovative company that defined the standard for sound but has since shifted its focus toward lawyers and licensing revenue rather than engineering innovation, comparing its trajectory unfavorably to HP.
Key Insights
- The speakers argue that multimedia should be defined as 'a digital representation of several streams for the human senses,' which inherently includes smell, touch, and eventually brain wave data — making expansion into those domains logically inevitable for FFmpeg and VLC.
- VLC already has a haptic plugin, not included in the standard release, that transports physical and movement data synchronized with content, used in 4D cinema theme park experiences with hydraulic arms.
- The speakers describe their role as maintainers rather than contributors — their job is to architect the frameworks so that a large community (around 150 VLC contributors and 300 FFmpeg contributors per year) can plug in new modules for emerging formats.
- The speakers identify a consistent pattern in multimedia standardization: new formats start with five competing standards driven by hype, the hype collapses, secondary players converge on a shared standard, it gets implemented, and then the market leader is eventually pressured to adopt it — citing 3D audio from six or seven years ago as an example.
- The speakers characterize Dolby as a company that once defined what sound was with great engineering, but has shifted to being primarily driven by lawyers and licensing, implying it no longer innovates at the level it once did.
Topics
Transcript
[0:02] Do you think VLC and FFmpeg will be here 100 years from now? FFmpeg, yes. Yep, FFmpeg, yes. VLC, maybe. What's the future of what what where is where is FFmpeg going? Where is VLC going? Like in the next if you think about like 5 years, 10 years, 20 years. 5 years, 10 years is easy. The question is after that, right? Um the question is do we arrive at something called holograms, right? Yeah, so will VLC and FFmpeg expand to [0:35] whatever Yes. multimedia. Yes. >> The multimedia might become sorry for the pothead expansion of topic, but you know, um if you look at something like Neuralink with brain computer interfaces, it's very possible that…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Anti-matter & nuclear weapons: Why technology is always a double-edge sword | Don Lincoln
Don Lincoln discusses how advanced energy sources like nuclear fusion, fission, and antimatter represent transformative but double-edged technologies. He argues that science's role is to understand nature, while society must collectively decide how to apply that knowledge. The conversation concludes with a celebration of humanity's innate curiosity as the driver of civilizational progress.
Why antimatter costs $63 trillion dollars to produce | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the extreme difficulty and cost of producing antimatter, noting that Fermilab could only produce about one nanogram per year. They explore the theoretical potential of antimatter as a propulsion system for space travel, while emphasizing that the core challenge is an engineering problem of concentrating energy, not a physics breakthrough.
Is loop quantum gravity wrong? - physicist explains | Don Lincoln and Lex Fridman
Don Lincoln explains the differences between loop quantum gravity and string theory, noting that loop quantum gravity attempts to quantize space itself rather than unify all forces. He discusses how an early prediction of loop quantum gravity — that light speed would vary by frequency — was disproven by gamma ray burst observations, but the theory adapted. He also highlights the landmark gravitational wave observation confirming that gravity travels at the speed of light.
Can antimatter be used as rocket fuel? | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the feasibility of using antimatter as rocket fuel, noting that while it is physically possible, the cost of production (estimated at $62-63 trillion per gram) and containment challenges make it impractical. Lincoln explains that antimatter propulsion is fundamentally an engineering problem rather than a physics mystery, and that breakthroughs would likely come from finding new ways to concentrate energy rather than new physics theory.
Speed of light explained: Was Einstein's theory correct? | Don Lincoln and Lex Fridman
Don Lincoln explains Einstein's special relativity, focusing on the two core premises: the universality of natural laws and the constant speed of light for all observers. He describes modern particle physics experiments that have empirically confirmed Einstein's conjecture. He also reflects on how understanding space-time makes the concept of a universal speed limit intuitive rather than bizarre.