New video codecs explained: AV2, H.265, H.266 | Lex Fridman Podcast
This transcript discusses next-generation video codecs AV2, H.265, and H.266/VVC, explaining their ~30% efficiency improvements per generation. A key focus is the patent landscape that drove major tech companies like Google, Netflix, and Amazon to create the royalty-free Alliance for Open Media and develop AV1/AV2. The discussion also covers how patent complexity made H.265/HEVC licensing a nightmare, while European software patent law shields projects like VLC.
Summary
The conversation opens with an explanation of AV2, the successor to AV1 developed by the Alliance for Open Media (AOMedia), which includes Google, Netflix, Amazon, Apple, and VideoLAN. AV2 promises approximately 30% better compression efficiency compared to AV1, meaning the same video quality can be delivered with 30% less bandwidth. The speaker notes that AV1 was finalized in 2018 but codec adoption takes years to reach widespread deployment. AV2's reference decoder is humorously named 'Dav2' (spelled with a 2), continuing the tradition of 'Dav1d' — a recursive acronym meaning 'Dav1d is an AV1 Decoder,' with 'D' representing '2' in French.
The discussion clarifies the distinction between codec specifications, encoders, and decoders. The AV2 specification is the formal document defining how the codec works, while encoders (like AVM, or the anticipated SVT-AV2) and decoders (like Dav1d/Dav2) are separate software implementations. Parallels are drawn to the H.264 ecosystem, where x264 is an encoder and FFmpeg's FFH264 is a decoder.
On the H.26x side, the conversation traces the lineage from H.264/AVC through H.265/HEVC to H.266/VVC, each offering roughly 30% compression improvements. The dual naming convention (e.g., H.264/MPEG-4 AVC) is explained as a result of joint standardization between ISO (a private body) and the ITU (a United Nations agency), with the full name being a concatenation of both organizations' designations.
The patent landscape is identified as the central reason the AOMedia codec family exists. Multimedia and wireless communications (3G/4G/5G) are described as the two biggest patent minefields due to their mathematical nature and high commercial value. For H.264, licensing was described as relatively manageable, but H.265/HEVC became chaotic — multiple competing patent pools (MPEG-LA, HEVC Advance, Nokia operating independently) made licensing practically impossible. This complexity led companies like HP to drop HEVC support from Windows laptops, and streaming giants like YouTube and Netflix faced potential licensing costs in the hundreds of millions of dollars annually, which ultimately motivated the creation of AOMedia.
The speakers explain that AV1 and AV2 are designed with patent-freedom as a core goal, with patent review built into the standardization process — unlike the MPEG world where patents are entirely off-topic. Creative workarounds are required, such as inventing 'golden frames' and 'S-frames' as alternatives to patented B-frames. VVC/H.266, by contrast, inherits all of HEVC's patents plus new ones. Finally, VLC's position is explained: based in France, which rejects software patents, VLC is effectively shielded from most codec patent claims, as these are considered mathematical or idea patents and are invalid under European law.
Key Insights
- The speaker explains that each new codec generation — both in the AV and H.26x families — delivers approximately 30% better compression efficiency, though professionals may cite figures ranging from 25–35% depending on the specific use case, with screen recording sometimes yielding gains as high as 70–80%.
- The speaker argues that H.265/HEVC's licensing became practically unusable because features were deliberately pushed into the specification not for technical utility but solely so that patent holders could attach claims to them, resulting in multiple fragmented patent pools and uncapped royalty structures that could cost streaming companies hundreds of millions of dollars per year.
- The speaker states that the Alliance for Open Media was directly created because companies like Google, Meta, and Netflix concluded it was financially rational to build their own codec rather than pay escalating HEVC patent licensing fees, noting 'at 100 million per year you could create your own codec.'
- The speaker notes that AV1 and AV2 require 'double creativity' — engineers must not only improve compression efficiency but also deliberately design around existing patents, leading to constructs like 'golden frames' and 'S-frames' as royalty-free alternatives to patented B-frames.
- The speaker explains that VLC is headquartered in France specifically because France rejects software patents, meaning that most codec-related patents — which are essentially mathematical or idea patents — are legally invalid there, shielding VLC from licensing fees that the speaker calculated would otherwise exceed €200 per user.
Topics
Transcript
[0:02] Let's talk about the future of um multimedia ffmpeg VC but uh some of the codecs we didn't really mention AV2. So can we just lay out what is AV2? What is the hope for it? What is H.265 H.266. So AV1 is this codec that is done by the aliens for open media right where there is uh Google, Netflix, Amazon, Apple, uh Vidolan where we try to make a royaltyfree very good codec right and [0:34] now it's being deployed but actually the codec was finished in 2018 but a codec takes years to be used in wide scenarios right so uh AV2 is the next generation of this codec it's 30% better, right? So, if you…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Anti-matter & nuclear weapons: Why technology is always a double-edge sword | Don Lincoln
Don Lincoln discusses how advanced energy sources like nuclear fusion, fission, and antimatter represent transformative but double-edged technologies. He argues that science's role is to understand nature, while society must collectively decide how to apply that knowledge. The conversation concludes with a celebration of humanity's innate curiosity as the driver of civilizational progress.
Why antimatter costs $63 trillion dollars to produce | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the extreme difficulty and cost of producing antimatter, noting that Fermilab could only produce about one nanogram per year. They explore the theoretical potential of antimatter as a propulsion system for space travel, while emphasizing that the core challenge is an engineering problem of concentrating energy, not a physics breakthrough.
Is loop quantum gravity wrong? - physicist explains | Don Lincoln and Lex Fridman
Don Lincoln explains the differences between loop quantum gravity and string theory, noting that loop quantum gravity attempts to quantize space itself rather than unify all forces. He discusses how an early prediction of loop quantum gravity — that light speed would vary by frequency — was disproven by gamma ray burst observations, but the theory adapted. He also highlights the landmark gravitational wave observation confirming that gravity travels at the speed of light.
Can antimatter be used as rocket fuel? | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the feasibility of using antimatter as rocket fuel, noting that while it is physically possible, the cost of production (estimated at $62-63 trillion per gram) and containment challenges make it impractical. Lincoln explains that antimatter propulsion is fundamentally an engineering problem rather than a physics mystery, and that breakthroughs would likely come from finding new ways to concentrate energy rather than new physics theory.
Speed of light explained: Was Einstein's theory correct? | Don Lincoln and Lex Fridman
Don Lincoln explains Einstein's special relativity, focusing on the two core premises: the universality of natural laws and the constant speed of light for all observers. He describes modern particle physics experiments that have empirically confirmed Einstein's conjecture. He also reflects on how understanding space-time makes the concept of a universal speed limit intuitive rather than bizarre.