New video codecs explained: AV2, H.265, H.266 | Lex Fridman Podcast
This transcript discusses next-generation video codecs AV2, H.265, and H.266/VVC, explaining their ~30% efficiency improvements per generation. A key focus is the patent landscape that drove major tech companies like Google, Netflix, and Amazon to create the royalty-free Alliance for Open Media and develop AV1/AV2. The discussion also covers how patent complexity made H.265/HEVC licensing a nightmare, while European software patent law shields projects like VLC.
Summary
The conversation opens with an explanation of AV2, the successor to AV1 developed by the Alliance for Open Media (AOMedia), which includes Google, Netflix, Amazon, Apple, and VideoLAN. AV2 promises approximately 30% better compression efficiency compared to AV1, meaning the same video quality can be delivered with 30% less bandwidth. The speaker notes that AV1 was finalized in 2018 but codec adoption takes years to reach widespread deployment. AV2's reference decoder is humorously named 'Dav2' (spelled with a 2), continuing the tradition of 'Dav1d' — a recursive acronym meaning 'Dav1d is an AV1 Decoder,' with 'D' representing '2' in French.
The discussion clarifies the distinction between codec specifications, encoders, and decoders. The AV2 specification is the formal document defining how the codec works, while encoders (like AVM, or the anticipated SVT-AV2) and decoders (like Dav1d/Dav2) are separate software implementations. Parallels are drawn to the H.264 ecosystem, where x264 is an encoder and FFmpeg's FFH264 is a decoder.
On the H.26x side, the conversation traces the lineage from H.264/AVC through H.265/HEVC to H.266/VVC, each offering roughly 30% compression improvements. The dual naming convention (e.g., H.264/MPEG-4 AVC) is explained as a result of joint standardization between ISO (a private body) and the ITU (a United Nations agency), with the full name being a concatenation of both organizations' designations.
The patent landscape is identified as the central reason the AOMedia codec family exists. Multimedia and wireless communications (3G/4G/5G) are described as the two biggest patent minefields due to their mathematical nature and high commercial value. For H.264, licensing was described as relatively manageable, but H.265/HEVC became chaotic — multiple competing patent pools (MPEG-LA, HEVC Advance, Nokia operating independently) made licensing practically impossible. This complexity led companies like HP to drop HEVC support from Windows laptops, and streaming giants like YouTube and Netflix faced potential licensing costs in the hundreds of millions of dollars annually, which ultimately motivated the creation of AOMedia.
The speakers explain that AV1 and AV2 are designed with patent-freedom as a core goal, with patent review built into the standardization process — unlike the MPEG world where patents are entirely off-topic. Creative workarounds are required, such as inventing 'golden frames' and 'S-frames' as alternatives to patented B-frames. VVC/H.266, by contrast, inherits all of HEVC's patents plus new ones. Finally, VLC's position is explained: based in France, which rejects software patents, VLC is effectively shielded from most codec patent claims, as these are considered mathematical or idea patents and are invalid under European law.
Key Insights
- The speaker explains that each new codec generation — both in the AV and H.26x families — delivers approximately 30% better compression efficiency, though professionals may cite figures ranging from 25–35% depending on the specific use case, with screen recording sometimes yielding gains as high as 70–80%.
- The speaker argues that H.265/HEVC's licensing became practically unusable because features were deliberately pushed into the specification not for technical utility but solely so that patent holders could attach claims to them, resulting in multiple fragmented patent pools and uncapped royalty structures that could cost streaming companies hundreds of millions of dollars per year.
- The speaker states that the Alliance for Open Media was directly created because companies like Google, Meta, and Netflix concluded it was financially rational to build their own codec rather than pay escalating HEVC patent licensing fees, noting 'at 100 million per year you could create your own codec.'
- The speaker notes that AV1 and AV2 require 'double creativity' — engineers must not only improve compression efficiency but also deliberately design around existing patents, leading to constructs like 'golden frames' and 'S-frames' as royalty-free alternatives to patented B-frames.
- The speaker explains that VLC is headquartered in France specifically because France rejects software patents, meaning that most codec-related patents — which are essentially mathematical or idea patents — are legally invalid there, shielding VLC from licensing fees that the speaker calculated would otherwise exceed €200 per user.
Topics
Transcript
[0:02] Let's talk about the future of um multimedia ffmpeg VC but uh some of the codecs we didn't really mention AV2. So can we just lay out what is AV2? What is the hope for it? What is H.265 H.266. So AV1 is this codec that is done by the aliens for open media right where there is uh Google, Netflix, Amazon, Apple, uh Vidolan where we try to make a royaltyfree very good codec right and [0:34] now it's being deployed but actually the codec was finished in 2018 but a codec takes years to be used in wide scenarios right so uh AV2 is the next generation of this codec it's 30% better, right? So, if you…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Surprising origin of antipsychotic drugs | Andrew Scull and Lex Fridman
Andrew Scull explains how chlorpromazine, an antihistamine synthesized in the 1880s, was accidentally discovered to have psychiatric applications in the 1950s through French Navy lieutenant Alain Laborit's experiments. The drug's rapid adoption was driven by pharmaceutical companies' aggressive marketing to politicians and hospital administrators rather than by psychiatrists' initial enthusiasm, transforming it from a 'major tranquilizer' into the first 'antipsychotic' drug.
Cognitive Behavioral Therapy (CBT) vs Psychoanalysis | Andrew Scull and Lex Fridman
Andrew Scull discusses how clinical psychology and cognitive behavioral therapy (CBT) emerged after WWII as alternatives to psychoanalysis, becoming dominant through better empirical evidence, shorter treatment duration, and federal funding advantages. CBT's symptom-focused approach proved more measurable and reproducible than psychoanalytic theory, though its effectiveness remains limited for serious mental disorders.
Do anti-depressant drugs work? | Andrew Scull and Lex Fridman
Andrew Scull discusses the history and efficacy of antidepressants, particularly SSRIs like Prozac, explaining that while they statistically outperform placebo, the clinical improvement is often marginal and comes with significant side effects including sexual dysfunction and withdrawal difficulties. He also addresses 'diagnostic creep' in psychiatry, where conditions become increasingly broadly defined and diagnosed over time.
How drugs and psycho-pharmacology changed mental health | Andrew Scull and Lex Fridman
Andrew Scull traces the accidental discovery of psychopharmacology, beginning with chlorpromazine's unexpected psychiatric effects in the 1950s, and discusses how pharmaceutical companies transformed mental health treatment while also introducing serious side effects like tardive dyskinesia and metabolic complications that remain inadequately addressed.
Freud's radical theory of the unconscious mind | Andrew Scull and Lex Fridman
Andrew Scull discusses Freud's development of psychoanalytic theory, including his structural model of the mind (id, ego, superego) and his theoretical innovations like dream interpretation and Freudian slips. The conversation explores how Freudian ideas gained popularity among intellectuals and artists in the 1920s-30s, particularly following WWI's shell shock crisis, despite remaining largely outside mainstream psychiatric institutions that dealt with severely disturbed patients.