Future of holograms, brain signal playback, smell, and touch in multimedia software | Lex Fridman
Developers of VLC and FFmpeg discuss the future of multimedia frameworks, predicting expansion into holograms, brain-computer interfaces, smell, touch, and volumetric video. They argue that multimedia should be redefined as any timed data representing human senses. They also critique Dolby for shifting from innovation to patent licensing.
Summary
The conversation centers on the long-term trajectory of FFmpeg and VLC, two foundational open-source multimedia frameworks. The speakers express high confidence that FFmpeg will exist in 100 years, while VLC's survival is considered more uncertain. Looking 5–20 years ahead, they anticipate that multimedia will expand far beyond audio and video to encompass brain-computer interface data (referencing Neuralink), volumetric video, point clouds, RGBD depth data, and XR/VR streaming.
The speakers redefine multimedia as 'a digital representation of several streams for the human senses,' arguing this naturally includes smell and touch. They describe existing work on haptic plugins in VLC used for 4D cinema experiences with hydraulic motion platforms, and they envision future codec tracks for stereo smell and haptic feedback. They frame FFmpeg and VLC not as contributors but as maintainers who shape architecture so that external contributors can add new format modules over time.
The speakers describe a recurring industry pattern in emerging multimedia formats: initial fragmentation into multiple competing standards, followed by a hype decline, consolidation around a community-driven standard, and eventual adoption by market leaders under pressure. They cite 3D audio from six or seven years ago as a cautionary example of premature hype followed by a more mature revival.
Finally, the speakers criticize Dolby, characterizing it as a once-innovative company that defined the standard for sound but has since shifted its focus toward lawyers and licensing revenue rather than engineering innovation, comparing its trajectory unfavorably to HP.
Key Insights
- The speakers argue that multimedia should be defined as 'a digital representation of several streams for the human senses,' which inherently includes smell, touch, and eventually brain wave data — making expansion into those domains logically inevitable for FFmpeg and VLC.
- VLC already has a haptic plugin, not included in the standard release, that transports physical and movement data synchronized with content, used in 4D cinema theme park experiences with hydraulic arms.
- The speakers describe their role as maintainers rather than contributors — their job is to architect the frameworks so that a large community (around 150 VLC contributors and 300 FFmpeg contributors per year) can plug in new modules for emerging formats.
- The speakers identify a consistent pattern in multimedia standardization: new formats start with five competing standards driven by hype, the hype collapses, secondary players converge on a shared standard, it gets implemented, and then the market leader is eventually pressured to adopt it — citing 3D audio from six or seven years ago as an example.
- The speakers characterize Dolby as a company that once defined what sound was with great engineering, but has shifted to being primarily driven by lawyers and licensing, implying it no longer innovates at the level it once did.
Topics
Transcript
[0:02] Do you think VLC and FFmpeg will be here 100 years from now? FFmpeg, yes. Yep, FFmpeg, yes. VLC, maybe. What's the future of what what where is where is FFmpeg going? Where is VLC going? Like in the next if you think about like 5 years, 10 years, 20 years. 5 years, 10 years is easy. The question is after that, right? Um the question is do we arrive at something called holograms, right? Yeah, so will VLC and FFmpeg expand to [0:35] whatever Yes. multimedia. Yes. >> The multimedia might become sorry for the pothead expansion of topic, but you know, um if you look at something like Neuralink with brain computer interfaces, it's very possible that…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Do antipsychotic drugs work? | Andrew Scull and Lex Fridman
Andrew Scull discusses the efficacy and serious side effects of antipsychotic drugs, explaining that while they help some patients, many are non-responders, and the medications carry substantial risks including tardive dyskinesia, weight gain, metabolic syndrome, and cognitive dulling. The landmark CATIE study revealed that newer, expensive second-generation antipsychotics are no more effective than older first-generation drugs, with 67-82% of patients dropping out due to inefficacy or intolerable side effects.
Surprising origin of antipsychotic drugs | Andrew Scull and Lex Fridman
Andrew Scull explains how chlorpromazine, an antihistamine synthesized in the 1880s, was accidentally discovered to have psychiatric applications in the 1950s through French Navy lieutenant Alain Laborit's experiments. The drug's rapid adoption was driven by pharmaceutical companies' aggressive marketing to politicians and hospital administrators rather than by psychiatrists' initial enthusiasm, transforming it from a 'major tranquilizer' into the first 'antipsychotic' drug.
Cognitive Behavioral Therapy (CBT) vs Psychoanalysis | Andrew Scull and Lex Fridman
Andrew Scull discusses how clinical psychology and cognitive behavioral therapy (CBT) emerged after WWII as alternatives to psychoanalysis, becoming dominant through better empirical evidence, shorter treatment duration, and federal funding advantages. CBT's symptom-focused approach proved more measurable and reproducible than psychoanalytic theory, though its effectiveness remains limited for serious mental disorders.
Do anti-depressant drugs work? | Andrew Scull and Lex Fridman
Andrew Scull discusses the history and efficacy of antidepressants, particularly SSRIs like Prozac, explaining that while they statistically outperform placebo, the clinical improvement is often marginal and comes with significant side effects including sexual dysfunction and withdrawal difficulties. He also addresses 'diagnostic creep' in psychiatry, where conditions become increasingly broadly defined and diagnosed over time.
How drugs and psycho-pharmacology changed mental health | Andrew Scull and Lex Fridman
Andrew Scull traces the accidental discovery of psychopharmacology, beginning with chlorpromazine's unexpected psychiatric effects in the 1950s, and discusses how pharmaceutical companies transformed mental health treatment while also introducing serious side effects like tardive dyskinesia and metabolic complications that remain inadequately addressed.