N4N002: Bandwidth and Latency Explained

Ethan Banks and Holly Metlitsky discuss a question from listener Douglas about how bandwidth and latency differ and why both matter. The hosts establish that bandwidth refers to the amount of data that can move through a network per unit time, measured in megabits or gigabits per second. Contrary to the highway speed analogy, increasing bandwidth doesn't make individual packets travel faster—packets already travel at the speed of light across fiber optics. Instead, higher bandwidth is achieved through increased clock cycles or channelization, where data is split across multiple parallel lanes.

The conversation introduces the concept of throughput, distinct from bandwidth, which represents how much data is actually being transmitted rather than the theoretical maximum. Throughput is affected by packet size, the capacity of various links along a route, and network congestion. The hosts explain that physical limitations imposed by physics affect how fast data can travel over copper cabling before attenuation and crosstalk degrade the signal.

Latency, defined as the time required for data to travel from sender to receiver, is explored through its impact on TCP (Transmission Control Protocol). TCP uses an acknowledgment mechanism where the sender waits for confirmation before transmitting more data. High latency significantly impacts throughput by creating delays in this acknowledgment process, preventing efficient use of available bandwidth. This problem has spawned numerous TCP variants and the emerging QUIC protocol, designed to overcome latency inefficiencies on the global internet.

The hosts discuss ultra-low latency, a specialized concern in data center environments, particularly for high-frequency trading where nanosecond-level delays matter economically. In contrast, UDP (User Datagram Protocol), an unacknowledged protocol, is appropriate for real-time applications like voice calls where timeliness is more critical than perfect delivery—losing some packets is preferable to buffering delays that degrade the user experience. The episode concludes by introducing related concepts: jitter (variable latency) and buffer bloat (excessive buffering causing congestion perception).