New video codecs explained: AV2, H.265, H.266 | Lex Fridman Podcast
This transcript discusses next-generation video codecs AV2, H.265, and H.266/VVC, explaining their ~30% efficiency improvements per generation. A key focus is the patent landscape that drove major tech companies like Google, Netflix, and Amazon to create the royalty-free Alliance for Open Media and develop AV1/AV2. The discussion also covers how patent complexity made H.265/HEVC licensing a nightmare, while European software patent law shields projects like VLC.
Summary
The conversation opens with an explanation of AV2, the successor to AV1 developed by the Alliance for Open Media (AOMedia), which includes Google, Netflix, Amazon, Apple, and VideoLAN. AV2 promises approximately 30% better compression efficiency compared to AV1, meaning the same video quality can be delivered with 30% less bandwidth. The speaker notes that AV1 was finalized in 2018 but codec adoption takes years to reach widespread deployment. AV2's reference decoder is humorously named 'Dav2' (spelled with a 2), continuing the tradition of 'Dav1d' — a recursive acronym meaning 'Dav1d is an AV1 Decoder,' with 'D' representing '2' in French.
The discussion clarifies the distinction between codec specifications, encoders, and decoders. The AV2 specification is the formal document defining how the codec works, while encoders (like AVM, or the anticipated SVT-AV2) and decoders (like Dav1d/Dav2) are separate software implementations. Parallels are drawn to the H.264 ecosystem, where x264 is an encoder and FFmpeg's FFH264 is a decoder.
On the H.26x side, the conversation traces the lineage from H.264/AVC through H.265/HEVC to H.266/VVC, each offering roughly 30% compression improvements. The dual naming convention (e.g., H.264/MPEG-4 AVC) is explained as a result of joint standardization between ISO (a private body) and the ITU (a United Nations agency), with the full name being a concatenation of both organizations' designations.
The patent landscape is identified as the central reason the AOMedia codec family exists. Multimedia and wireless communications (3G/4G/5G) are described as the two biggest patent minefields due to their mathematical nature and high commercial value. For H.264, licensing was described as relatively manageable, but H.265/HEVC became chaotic — multiple competing patent pools (MPEG-LA, HEVC Advance, Nokia operating independently) made licensing practically impossible. This complexity led companies like HP to drop HEVC support from Windows laptops, and streaming giants like YouTube and Netflix faced potential licensing costs in the hundreds of millions of dollars annually, which ultimately motivated the creation of AOMedia.
The speakers explain that AV1 and AV2 are designed with patent-freedom as a core goal, with patent review built into the standardization process — unlike the MPEG world where patents are entirely off-topic. Creative workarounds are required, such as inventing 'golden frames' and 'S-frames' as alternatives to patented B-frames. VVC/H.266, by contrast, inherits all of HEVC's patents plus new ones. Finally, VLC's position is explained: based in France, which rejects software patents, VLC is effectively shielded from most codec patent claims, as these are considered mathematical or idea patents and are invalid under European law.
Key Insights
- The speaker explains that each new codec generation — both in the AV and H.26x families — delivers approximately 30% better compression efficiency, though professionals may cite figures ranging from 25–35% depending on the specific use case, with screen recording sometimes yielding gains as high as 70–80%.
- The speaker argues that H.265/HEVC's licensing became practically unusable because features were deliberately pushed into the specification not for technical utility but solely so that patent holders could attach claims to them, resulting in multiple fragmented patent pools and uncapped royalty structures that could cost streaming companies hundreds of millions of dollars per year.
- The speaker states that the Alliance for Open Media was directly created because companies like Google, Meta, and Netflix concluded it was financially rational to build their own codec rather than pay escalating HEVC patent licensing fees, noting 'at 100 million per year you could create your own codec.'
- The speaker notes that AV1 and AV2 require 'double creativity' — engineers must not only improve compression efficiency but also deliberately design around existing patents, leading to constructs like 'golden frames' and 'S-frames' as royalty-free alternatives to patented B-frames.
- The speaker explains that VLC is headquartered in France specifically because France rejects software patents, meaning that most codec-related patents — which are essentially mathematical or idea patents — are legally invalid there, shielding VLC from licensing fees that the speaker calculated would otherwise exceed €200 per user.
Topics
Transcript
[0:02] Let's talk about the future of um multimedia ffmpeg VC but uh some of the codecs we didn't really mention AV2. So can we just lay out what is AV2? What is the hope for it? What is H.265 H.266. So AV1 is this codec that is done by the aliens for open media right where there is uh Google, Netflix, Amazon, Apple, uh Vidolan where we try to make a royaltyfree very good codec right and [0:34] now it's being deployed but actually the codec was finished in 2018 but a codec takes years to be used in wide scenarios right so uh AV2 is the next generation of this codec it's 30% better, right? So, if you…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
How the Civil War ended | Gary Gallagher and Lex Fridman
Gary Gallagher and Lex Fridman discuss how the Civil War ended, focusing on Lee's surrender at Appomattox and Grant's magnanimous treatment of Confederate forces. They explore how Grant's generous surrender terms, influenced by Lincoln's reconciliation goals, contrasted with calls from Radical Republicans for harsher punishment and how this approach facilitated national reunification rather than perpetuating resentment.
White supremacy after the Civil War | Gary Gallagher and Lex Fridman
Historian Gary Gallagher discusses Reconstruction, white supremacy, and Confederate memory in post-Civil War America, emphasizing that Northern troops were insufficient to enforce equal rights for freed Black people, and explaining how the "Lost Cause" narrative allowed the South to reframe the war's purpose while Jim Crow became the mechanism for maintaining white supremacy.
The truth about the Civil War - Why people volunteered to fight | Gary Gallagher and Lex Fridman
Historian Gary Gallagher discusses the distinction between causation and motivation in Civil War volunteerism, explaining that while slavery caused the war, most Northern soldiers were motivated by belief in preserving the Union as a democratic republic. He emphasizes that understanding citizen-soldiers' sense of civic duty and American exceptionalism is essential to comprehending why ordinary people risked their lives for the Union.
Christianity and religion during the Civil War | Gary Gallagher and Lex Fridman
Both Union and Confederate sides during the American Civil War were overwhelmingly Protestant Christian and used religious justification for their military efforts. Interestingly, Confederate leaders like Lee and Jackson were notably more overtly religious in their official communications than Union commanders like Grant and Sherman.
Racism in America after slavery was abolished | Gary Gallagher and Lex Fridman
Gary Gallagher discusses how abolishing slavery was only the beginning of achieving equal rights for Black Americans, noting that most Northern whites prioritized saving the Union over ensuring racial equality. He explains that legal changes like the Civil Rights Act took over a century to achieve, while the deeper work of changing societal attitudes and prejudices remains ongoing and vulnerable to exploitation.