Skip to content
Documentation Background

TCP Congestion Control

TCP’s job is to fill the pipe without breaking the network. If every TCP sender pushed data at maximum rate simultaneously, routers would drop packets as queues overflow. TCP congestion control is the set of algorithms that make TCP self-regulate to avoid this.

The key insight: packet loss is TCP’s signal that the network is overloaded. TCP uses loss (and latency in newer algorithms) as feedback to slow down.

TCP Congestion Control

Congestion hurts in three ways, and each makes the next worse.

  • Queuing delay - as the arrival rate approaches a link’s capacity, queues grow and delay climbs steeply. Past capacity, buffers fill and packets are dropped.
  • Wasted retransmissions - a dropped packet gets sent again, so some of the link’s capacity carries data that has already been sent once. If a timer fires too early, the original is still in a queue and the link carries both copies.
  • Wasted upstream work - when a packet is dropped at the third router on a path, the capacity it used on the first two was spent for nothing.

Past a certain load the useful delivery rate (goodput) falls as the offered load rises. This is congestion collapse, and it is what the algorithms below prevent.

There are two ways to find out the network is congested. End-to-end control infers it from what the sender sees, such as loss and rising delay, with no help from routers. Network-assisted control has routers signal the senders directly. TCP is end-to-end by default. The ECN section below covers the network-assisted option.


Going deeper: TCP congestion theory works through the three congestion scenarios, a slow start and AIMD trace, the throughput formula for lossy paths, and why AIMD converges to a fair share.

VariableMeaning
CWND (Congestion Window)How many unacknowledged segments TCP is allowed to have in flight at once
SSTHRESH (Slow Start Threshold)The CWND size where TCP switches from exponential growth to linear growth
RTT (Round Trip Time)How long it takes for a packet + ACK round trip. Informs timing algorithms.
MSS (Maximum Segment Size)The largest data payload per TCP segment (typically 1460 bytes on Ethernet)

Despite the name, slow start is actually exponential growth - it just starts small:

CWND starts at 1 MSS (or 10 MSS in modern Linux, RFC 6928)
Each ACK received → CWND += 1 MSS
After 1 RTT → CWND doubles
RTT 0: CWND = 1 MSS (send 1 segment)
RTT 1: CWND = 2 MSS (send 2 segments)
RTT 2: CWND = 4 MSS (4)
RTT 3: CWND = 8 MSS (8)
...continues until CWND >= SSTHRESH

Once CWND hits SSTHRESH, TCP switches to linear growth (additive increase):

Each ACK received → CWND += 1/CWND (effectively +1 MSS per RTT)
This is "Additive Increase Multiplicative Decrease" (AIMD)
EventWhat TCP doesWhy
Packet loss (timeout)SSTHRESH = CWND/2, CWND = 1 MSS, restart slow startTimeout = severe congestion signal
3 duplicate ACKs (fast retransmit)SSTHRESH = CWND/2, CWND = SSTHRESH (TCP Reno)3 dupACKs = mild congestion, don’t restart from scratch
ECN signal (explicit congestion notification)Same as 3 dupACKs but without packet lossRouter flags packets before dropping them

TCP Lifecycle


The classic algorithm (TCP Reno/Cubic) reacts to loss. When links are fast, you need many seconds of data in flight before seeing loss - and then you slam the brakes. Faster links = worse efficiency with loss-based CC.

AlgorithmSignal UsedBest ForDefault On
TCP RenoPacket lossLow-bandwidth linksLegacy
TCP CubicPacket loss (cubic growth curve)High-bandwidth long-delay linksLinux default
TCP VegasRising RTT (delay)Research and controlled networksRarely used
TCP BBR (Bottleneck Bandwidth and RTT)Bandwidth + RTT (not loss)High-speed links, long distances, lossy linksGoogle uses it; opt-in on Linux
QUIC (HTTP/3)Custom (UDP-based, per-stream CC)Web, mobile, high packet loss scenariosChrome, Cloudflare, YouTube

Loss-based algorithms fill the bottleneck buffer until it overflows, so they keep queues full and latency high. Vegas backs off when RTT starts to rise, before any loss occurs, but loses bandwidth to loss-based flows on the same path, which is why it never spread. CUBIC grows the window along a cubic curve, probing quickly after a loss and slowly near the last known limit, which suits high-bandwidth long-distance links. BBR works differently: it estimates the bottleneck bandwidth and the minimum RTT, then paces packets at that rate and keeps about one BDP in flight. That keeps buffers short without waiting for loss.


Loss is a late and costly signal. A packet has to be dropped and recovered before the sender slows down. ECN lets a router mark packets as congested instead of dropping them.

  1. The sender sets two bits in the IP header to say “ECN-capable”.
  2. A router whose queue is building sets those bits to the congestion-experienced value instead of dropping the packet.
  3. The receiver sees the mark and sets the ECE flag in its ACKs.
  4. The sender reduces its congestion window as if it had detected a loss, and sets CWR in the next segment to tell the receiver it has reacted.

The sender backs off without a drop or a retransmission, so latency stays lower. ECN only works if both endpoints and the routers along the path support it, and routers mark packets only when they run an active queue management scheme, such as fq_codel, that decides when to mark. See Bufferbloat for the queue side.

Terminal window
sysctl net.ipv4.tcp_ecn
# net.ipv4.tcp_ecn = 2
# 0 = off, 1 = request ECN on outgoing connections and accept it on incoming ones,
# 2 = accept ECN when the peer asks (default), do not ask
# Watch for ECE and CWR flags in a capture
sudo tcpdump -nn -i eth0 'tcp[13] & 0xc0 != 0'

Data centers take this further. DCTCP marks packets at fine-grained queue thresholds, so senders reduce their rate in proportion to the extent of congestion instead of halving the window. It needs ECN-marking switches.

If several TCP connections share a bottleneck, each should get an equal share of its capacity. Additive increase with multiplicative decrease achieves that over time: all flows grow by the same amount per round trip, while the flows using more bandwidth lose more in each backoff, so their rates converge.

Fairness has limits in practice:

  • RTT bias - a flow with a short RTT grows its window faster than a long-RTT flow on the same link, so it takes a bigger share.
  • UDP traffic - a sender with no congestion control takes whatever it can, and TCP flows shrink to make room.
  • Parallel connections - an application that opens ten connections gets about ten times the share of one that opens one. This is part of why browsers open several connections per host.

Terminal window
# See CWND and other TCP state per connection
ss -tin
# Output includes:
# cwnd:10 ssthresh:7 bytes_acked:1448 rcv_rtt:11.563
# rto:324 rtt:123.456/12.345 ato:40 mss:1448 pmtu:1500
# Watch live CWND changes (requires ss with -e)
watch -n 1 'ss -tin | grep cwnd'
SymptomLikely cause
Slow file transfer on a fast linkCongestion window not opened fully; check RTT and SSTHRESH
Speed is good initially then dropsBuffer overflow causing loss; congestion kicks in
Some paths fast, others slowDifferent congestion levels per path
High CPU during transfersInterrupt coalescing / GRO settings; not CC
YouTube buffers but download is fineDifferent CC behavior for streaming vs. bulk

Why This Matters for Application Developers

Section titled “Why This Matters for Application Developers”
  • Small messages (RPC, API calls) often never leave slow start - size matters
  • HTTP/1.1 keep-alive reuses connections to preserve CWND state (vs. new connection = slow start again)
  • HTTP/2 multiplexing sends multiple streams on one connection - shares CWND efficiently
  • TCP_NODELAY disables Nagle’s algorithm (which buffers small packets) - use for interactive apps (SSH, gaming), not bulk transfers
Terminal window
# Check if TCP_NODELAY is set on a socket
ss -tino | grep nodelay