The insane difficulty of reverse engineering video codecs | Lex Fridman Podcast
This segment of the Lex Fridman podcast discusses the extraordinary difficulty of reverse engineering proprietary video codecs, highlighting the work of developers like Costa who reverse engineered complex codecs like GoToMeeting. The conversation covers the technical process of binary reverse engineering, the importance of bit exactness, and the cultural community around open source multimedia development through FFmpeg.
Summary
The conversation begins with a historical overview of reverse engineering efforts in the video codec space, starting with proprietary formats like Windows Media and RealMedia in the 2000s, and progressing into the more complex era of the 2010s. A central figure discussed is Costa, a Ukrainian developer living in Germany, described as a borderline genius who reverse engineered binary blobs of 20-30 megabytes — a scale far beyond typical work, where even a 1 megabyte binary is estimated to take about a month of work.
The GoToMeeting codec is used as a case study. The speakers explain the practical problem: recordings made in GoToMeeting used a proprietary codec, and without the original player or decoder, the video could not be played. The reverse engineering process involved finding the relevant software module, hooking into it to extract raw YUV data as a reference, then opening a disassembler and using pattern recognition and intuition to identify structures like DCT transforms and entropy coding — all without documentation, treating the binary itself as the specification.
The discussion dives into the technical challenges: working entirely in a black box for long periods, debugging purely in memory, stepping through millions of instructions at the CPU level, sometimes needing to pause a virtual machine to dump memory due to DRM or encryption layers. A comparison is made to archaeology — reconstructing an entire human from tiny fragments with a small brush.
The concept of bit exactness is explained: from roughly the 2000s onward, most video codecs require that every compliant decoder produce exactly the same output bits for a given input. This was notably absent in MPEG-2 in the 1990s, acknowledged as one of the biggest mistakes of that era, with Yuri Resnik cited as someone who has publicly acknowledged this. The conversation also touches on the importance of sample collection — finding test files is critical, and sometimes a lucky find of thousands of files from a company that depended on the format enables thorough testing.
The segment closes with discussion of FFmpeg's unique position as a near-monopoly in open source multimedia decoding, where each new codec added increases the value of the entire library. Yuri Resnik is highlighted as a highly respected figure in the multimedia standards community, and the speakers reflect warmly on the community of reverse engineers and assembly-level developers who made much of this work possible.
Key Insights
- Costa, a Ukrainian developer in Germany, treated binary files as the complete specification — regularly using the phrase 'binary specification' — and reverse engineered 20-30 megabyte binary blobs, a scale where even 1 megabyte is estimated to take about a month of work.
- The reverse engineering process for GoToMeeting involved hooking into the module to extract raw YUV data as a ground-truth reference, then using a disassembler with pattern recognition to identify codec structures like DCT and entropy coding — all without any documentation.
- Reverse engineers often spend long periods in a complete black box, debugging purely in memory at the CPU instruction level, sometimes inside a virtual machine that can be paused to dump memory — necessary when codecs include DRM or encryption layers.
- The absence of bit exactness in MPEG-2 — meaning different decoder implementations could produce different outputs — is acknowledged as one of the biggest mistakes of the 1990s video industry, with Yuri Resnik cited as someone who has publicly recognized this error.
- FFmpeg has achieved a near winner-takes-all status in multimedia decoding because each new codec added increases the value of the entire library, unlike web browsers where multiple competing engines still coexist.
Topics
Transcript
[0:03] That's where a lot of the reverse engineering was happening. >> It start it started in the 2000s with the Windows media stuff because that was proprietary. It started with the real media with Benjamin Lson >> Kashiskov >> Kashiskov that era those were the key that was the key groundwork >> and then in in the 2010s was kind of the Paul Mahal Costa era building doing some of the most difficult codecs. Um JB maybe can talk about go to meeting four and go to meeting five and >> what >> these are what's the go to >> so so so so like let's talk about this [0:34] amazing Ukrainian guy called Costa who was uh at…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Anti-matter & nuclear weapons: Why technology is always a double-edge sword | Don Lincoln
Don Lincoln discusses how advanced energy sources like nuclear fusion, fission, and antimatter represent transformative but double-edged technologies. He argues that science's role is to understand nature, while society must collectively decide how to apply that knowledge. The conversation concludes with a celebration of humanity's innate curiosity as the driver of civilizational progress.
Why antimatter costs $63 trillion dollars to produce | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the extreme difficulty and cost of producing antimatter, noting that Fermilab could only produce about one nanogram per year. They explore the theoretical potential of antimatter as a propulsion system for space travel, while emphasizing that the core challenge is an engineering problem of concentrating energy, not a physics breakthrough.
Is loop quantum gravity wrong? - physicist explains | Don Lincoln and Lex Fridman
Don Lincoln explains the differences between loop quantum gravity and string theory, noting that loop quantum gravity attempts to quantize space itself rather than unify all forces. He discusses how an early prediction of loop quantum gravity — that light speed would vary by frequency — was disproven by gamma ray burst observations, but the theory adapted. He also highlights the landmark gravitational wave observation confirming that gravity travels at the speed of light.
Can antimatter be used as rocket fuel? | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the feasibility of using antimatter as rocket fuel, noting that while it is physically possible, the cost of production (estimated at $62-63 trillion per gram) and containment challenges make it impractical. Lincoln explains that antimatter propulsion is fundamentally an engineering problem rather than a physics mystery, and that breakthroughs would likely come from finding new ways to concentrate energy rather than new physics theory.
Speed of light explained: Was Einstein's theory correct? | Don Lincoln and Lex Fridman
Don Lincoln explains Einstein's special relativity, focusing on the two core premises: the universality of natural laws and the constant speed of light for all observers. He describes modern particle physics experiments that have empirically confirmed Einstein's conjecture. He also reflects on how understanding space-time makes the concept of a universal speed limit intuitive rather than bizarre.