New video codecs explained: AV2, H.265, H.266 | Lex Fridman Podcast
This transcript discusses next-generation video codecs AV2, H.265, and H.266/VVC, explaining their ~30% efficiency improvements per generation. A key focus is the patent landscape that drove major tech companies like Google, Netflix, and Amazon to create the royalty-free Alliance for Open Media and develop AV1/AV2. The discussion also covers how patent complexity made H.265/HEVC licensing a nightmare, while European software patent law shields projects like VLC.
Summary
The conversation opens with an explanation of AV2, the successor to AV1 developed by the Alliance for Open Media (AOMedia), which includes Google, Netflix, Amazon, Apple, and VideoLAN. AV2 promises approximately 30% better compression efficiency compared to AV1, meaning the same video quality can be delivered with 30% less bandwidth. The speaker notes that AV1 was finalized in 2018 but codec adoption takes years to reach widespread deployment. AV2's reference decoder is humorously named 'Dav2' (spelled with a 2), continuing the tradition of 'Dav1d' — a recursive acronym meaning 'Dav1d is an AV1 Decoder,' with 'D' representing '2' in French.
The discussion clarifies the distinction between codec specifications, encoders, and decoders. The AV2 specification is the formal document defining how the codec works, while encoders (like AVM, or the anticipated SVT-AV2) and decoders (like Dav1d/Dav2) are separate software implementations. Parallels are drawn to the H.264 ecosystem, where x264 is an encoder and FFmpeg's FFH264 is a decoder.
On the H.26x side, the conversation traces the lineage from H.264/AVC through H.265/HEVC to H.266/VVC, each offering roughly 30% compression improvements. The dual naming convention (e.g., H.264/MPEG-4 AVC) is explained as a result of joint standardization between ISO (a private body) and the ITU (a United Nations agency), with the full name being a concatenation of both organizations' designations.
The patent landscape is identified as the central reason the AOMedia codec family exists. Multimedia and wireless communications (3G/4G/5G) are described as the two biggest patent minefields due to their mathematical nature and high commercial value. For H.264, licensing was described as relatively manageable, but H.265/HEVC became chaotic — multiple competing patent pools (MPEG-LA, HEVC Advance, Nokia operating independently) made licensing practically impossible. This complexity led companies like HP to drop HEVC support from Windows laptops, and streaming giants like YouTube and Netflix faced potential licensing costs in the hundreds of millions of dollars annually, which ultimately motivated the creation of AOMedia.
The speakers explain that AV1 and AV2 are designed with patent-freedom as a core goal, with patent review built into the standardization process — unlike the MPEG world where patents are entirely off-topic. Creative workarounds are required, such as inventing 'golden frames' and 'S-frames' as alternatives to patented B-frames. VVC/H.266, by contrast, inherits all of HEVC's patents plus new ones. Finally, VLC's position is explained: based in France, which rejects software patents, VLC is effectively shielded from most codec patent claims, as these are considered mathematical or idea patents and are invalid under European law.
Key Insights
- The speaker explains that each new codec generation — both in the AV and H.26x families — delivers approximately 30% better compression efficiency, though professionals may cite figures ranging from 25–35% depending on the specific use case, with screen recording sometimes yielding gains as high as 70–80%.
- The speaker argues that H.265/HEVC's licensing became practically unusable because features were deliberately pushed into the specification not for technical utility but solely so that patent holders could attach claims to them, resulting in multiple fragmented patent pools and uncapped royalty structures that could cost streaming companies hundreds of millions of dollars per year.
- The speaker states that the Alliance for Open Media was directly created because companies like Google, Meta, and Netflix concluded it was financially rational to build their own codec rather than pay escalating HEVC patent licensing fees, noting 'at 100 million per year you could create your own codec.'
- The speaker notes that AV1 and AV2 require 'double creativity' — engineers must not only improve compression efficiency but also deliberately design around existing patents, leading to constructs like 'golden frames' and 'S-frames' as royalty-free alternatives to patented B-frames.
- The speaker explains that VLC is headquartered in France specifically because France rejects software patents, meaning that most codec-related patents — which are essentially mathematical or idea patents — are legally invalid there, shielding VLC from licensing fees that the speaker calculated would otherwise exceed €200 per user.
Topics
Full transcript available for MurmurCast members
Sign Up to Access