Future of holograms, brain signal playback, smell, and touch in multimedia software | Lex Fridman
Developers of VLC and FFmpeg discuss the future of multimedia frameworks, predicting expansion into holograms, brain-computer interfaces, smell, touch, and volumetric video. They argue that multimedia should be redefined as any timed data representing human senses. They also critique Dolby for shifting from innovation to patent licensing.
Summary
The conversation centers on the long-term trajectory of FFmpeg and VLC, two foundational open-source multimedia frameworks. The speakers express high confidence that FFmpeg will exist in 100 years, while VLC's survival is considered more uncertain. Looking 5–20 years ahead, they anticipate that multimedia will expand far beyond audio and video to encompass brain-computer interface data (referencing Neuralink), volumetric video, point clouds, RGBD depth data, and XR/VR streaming.
The speakers redefine multimedia as 'a digital representation of several streams for the human senses,' arguing this naturally includes smell and touch. They describe existing work on haptic plugins in VLC used for 4D cinema experiences with hydraulic motion platforms, and they envision future codec tracks for stereo smell and haptic feedback. They frame FFmpeg and VLC not as contributors but as maintainers who shape architecture so that external contributors can add new format modules over time.
The speakers describe a recurring industry pattern in emerging multimedia formats: initial fragmentation into multiple competing standards, followed by a hype decline, consolidation around a community-driven standard, and eventual adoption by market leaders under pressure. They cite 3D audio from six or seven years ago as a cautionary example of premature hype followed by a more mature revival.
Finally, the speakers criticize Dolby, characterizing it as a once-innovative company that defined the standard for sound but has since shifted its focus toward lawyers and licensing revenue rather than engineering innovation, comparing its trajectory unfavorably to HP.
Key Insights
- The speakers argue that multimedia should be defined as 'a digital representation of several streams for the human senses,' which inherently includes smell, touch, and eventually brain wave data — making expansion into those domains logically inevitable for FFmpeg and VLC.
- VLC already has a haptic plugin, not included in the standard release, that transports physical and movement data synchronized with content, used in 4D cinema theme park experiences with hydraulic arms.
- The speakers describe their role as maintainers rather than contributors — their job is to architect the frameworks so that a large community (around 150 VLC contributors and 300 FFmpeg contributors per year) can plug in new modules for emerging formats.
- The speakers identify a consistent pattern in multimedia standardization: new formats start with five competing standards driven by hype, the hype collapses, secondary players converge on a shared standard, it gets implemented, and then the market leader is eventually pressured to adopt it — citing 3D audio from six or seven years ago as an example.
- The speakers characterize Dolby as a company that once defined what sound was with great engineering, but has shifted to being primarily driven by lawyers and licensing, implying it no longer innovates at the level it once did.
Topics
Transcript
[0:02] Do you think VLC and FFmpeg will be here 100 years from now? FFmpeg, yes. Yep, FFmpeg, yes. VLC, maybe. What's the future of what what where is where is FFmpeg going? Where is VLC going? Like in the next if you think about like 5 years, 10 years, 20 years. 5 years, 10 years is easy. The question is after that, right? Um the question is do we arrive at something called holograms, right? Yeah, so will VLC and FFmpeg expand to [0:35] whatever Yes. multimedia. Yes. >> The multimedia might become sorry for the pothead expansion of topic, but you know, um if you look at something like Neuralink with brain computer interfaces, it's very possible that…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Sex in the Roman Empire | Anthony Kaldellis and Lex Fridman
Anthony Kaldellis discusses Justinian I's reign, focusing on his meritocratic appointment of talented individuals regardless of social class (including his wife Theodora, a former sex worker), his monumental codification of Roman law through the Corpus Juris Civilis, and the paradox of his harsh stance on sexuality despite his own unconventional personal choices.
Did the plague kill half the Roman Empire? | Anthony Kaldellis and Lex Fridman
Anthony Kaldellis discusses the Plague of Justinian (541 AD), arguing against the maximalist claim that it killed 50% of the Roman population. He contends that modern pathogen identification combined with historical evidence shows the plague had far less impact than commonly assumed, as evidenced by Justinian's continued military campaigns and taxation during the outbreak.
Persian Empire vs Roman Empire: The war that destroyed both empires | Anthony Kaldellis
Anthony Kaldellis discusses the Byzantine-Persian War (602-628 CE) and its catastrophic consequences for both empires, explaining how Heraclius's civil war weakened Roman defenses, the eventual Arab conquests stripped away the empire's richest provinces, and the subsequent Byzantine survival through military innovation like Greek fire and strategic defensive reorganization.
Why the Roman Empire collapsed in the East | Anthony Kaldellis and Lex Fridman
Anthony Kaldellis discusses how the Eastern Roman Empire, despite territorial losses, maintained remarkable internal stability and could have lasted another thousand years without external invasions. He attributes this resilience to two factors: authorities convincingly persuading subjects they ruled on their behalf, and a unified Roman and Orthodox identity that made the alternatives to imperial rule seem worse.
Lessons from the Roman Empire for modern-day - historian explains | Anthony Kaldellis
Historian Anthony Kaldellis discusses lessons from the 2,200-year Roman Empire for modern America, focusing on the importance of building lasting institutions that serve the majority, aligning foreign policy rhetoric with actual military actions, and the persistent elements of human nature across centuries.