The impossible task of testing FFmpeg code | Lex Fridman Podcast
A developer discusses FFmpeg's FATE (FFmpeg Automated Testing Environment) system, which uses volunteer-run machines to test an enormous matrix of compiler, OS, and architecture combinations. The conversation also touches on the speaker's company, Open Broadcast, which builds sports broadcasting equipment and deals with complex 10-bit video format conversion challenges. The scale of testing configurations is described as beyond a simple matrix — more like a multi-dimensional pivot table.
Summary
The conversation opens with a discussion of FFmpeg's automated testing system called FATE (FFmpeg Automated Testing Environment). Because FFmpeg runs across a vast number of operating systems and can be compiled with many different compilers, the number of test configurations is enormous. The speaker walks through the fate.ffmpeg.org website, showing combinations of compilers (e.g., Apple Clang, GCC, Intel compiler, Visual Studio), operating systems (Mac OS, iOS, tvOS), and instruction set architectures (PowerPC, RISC, x86). All of these test systems are run by volunteers, including machines the speaker himself hosts in his office.
The speaker explains that FATE is critical because complex C code in FFmpeg can sometimes be miscompiled — meaning the compiler itself produces incorrect output. This is particularly impactful in video processing because frames have dependencies on each other, so even a small error can cascade into significant visual glitches. FATE allows developers to catch issues that may not appear locally but break specific compiler versions on specific platforms.
The conversation then shifts to the speaker's day job at his company, Open Broadcast (clarified as unrelated to the free OBS streaming software). The company builds equipment for broadcasting sports matches between TV stadiums and studios. A major technical challenge they face is handling 10-bit video, which cannot be processed natively on a CPU in its native format and must be stored in 16-bit containers, wasting 6 bits. Various packing formats exist to use that space more efficiently, which matters greatly for network and PCI Express bus bandwidth. The speaker mentions they maintain roughly a 5x6 or 6x6 matrix of format-to-format conversions, all written in handwritten assembly and supporting different CPU generations — a process he describes as 'really traumatic.'
Key Insights
- The speaker explains that all FATE test systems are run by volunteers, including machines he personally hosts in his office, meaning the breadth of FFmpeg's cross-platform testing relies entirely on community participation.
- The speaker argues that FATE is essential because compilers can sometimes miscompile valid C code, and in video processing this is especially dangerous since frame dependencies mean even a small output error can cascade into large visible glitches.
- The speaker describes the FFmpeg test configuration space not as a simple matrix but as a 'pivot table of different combinations,' highlighting the combinatorial explosion of OS, compiler, and architecture variants.
- The speaker explains that 10-bit video cannot be processed natively on a CPU and must be packed into 16-bit storage, wasting 6 bits, which creates significant bandwidth pressure on interfaces like PCI Express — driving the need for specialized packing formats.
- The speaker reveals that his company maintains a roughly 5x5 or 6x6 matrix of every format-to-every-other-format conversion for 10-bit video, all written in handwritten assembly with support for multiple CPU generations, which he describes as 'really traumatic.'
Topics
Transcript
[0:02] That reminds me of a thing that you uh mentioned to me about fate testing and like the insanely rigorous process that's used to test everything that's incorporated into um into FFmpeg. Can you take me through the testing process? >> Yeah, so FFmpeg has a system called Fate FFmpeg automated testing environment. >> Because FFmpeg runs on so many different oss and can be compiled with so many different compilers, there's been a crazy number of configurations. So you can see the absurd combination of um compiler [0:34] variants, operating system variants, instruction sets. You can see at the top Mac OS has tons of different variants because it has iOS, it has tvOS. I'm looking at a…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Lex Clips
Anti-matter & nuclear weapons: Why technology is always a double-edge sword | Don Lincoln
Don Lincoln discusses how advanced energy sources like nuclear fusion, fission, and antimatter represent transformative but double-edged technologies. He argues that science's role is to understand nature, while society must collectively decide how to apply that knowledge. The conversation concludes with a celebration of humanity's innate curiosity as the driver of civilizational progress.
Why antimatter costs $63 trillion dollars to produce | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the extreme difficulty and cost of producing antimatter, noting that Fermilab could only produce about one nanogram per year. They explore the theoretical potential of antimatter as a propulsion system for space travel, while emphasizing that the core challenge is an engineering problem of concentrating energy, not a physics breakthrough.
Is loop quantum gravity wrong? - physicist explains | Don Lincoln and Lex Fridman
Don Lincoln explains the differences between loop quantum gravity and string theory, noting that loop quantum gravity attempts to quantize space itself rather than unify all forces. He discusses how an early prediction of loop quantum gravity — that light speed would vary by frequency — was disproven by gamma ray burst observations, but the theory adapted. He also highlights the landmark gravitational wave observation confirming that gravity travels at the speed of light.
Can antimatter be used as rocket fuel? | Don Lincoln and Lex Fridman
Don Lincoln and Lex Fridman discuss the feasibility of using antimatter as rocket fuel, noting that while it is physically possible, the cost of production (estimated at $62-63 trillion per gram) and containment challenges make it impractical. Lincoln explains that antimatter propulsion is fundamentally an engineering problem rather than a physics mystery, and that breakthroughs would likely come from finding new ways to concentrate energy rather than new physics theory.
Speed of light explained: Was Einstein's theory correct? | Don Lincoln and Lex Fridman
Don Lincoln explains Einstein's special relativity, focusing on the two core premises: the universality of natural laws and the constant speed of light for all observers. He describes modern particle physics experiments that have empirically confirmed Einstein's conjecture. He also reflects on how understanding space-time makes the concept of a universal speed limit intuitive rather than bizarre.