CDNs and video streaming
Serving a popular file from one data center fails in three ways. Long paths cross many networks, so some link along the way limits throughput. The same bytes travel the same expensive links again for every viewer. And one outage at the origin takes the whole service down. A content delivery network (CDN) copies content to many servers near the users and sends each request to a nearby copy.
Video made CDNs necessary, but the same design serves any large, shared, cacheable object: static assets, software packages, container image layers, and download artifacts.
Video as a network load
Section titled “Video as a network load”Video is a sequence of images shown at a fixed rate, usually 24 to 30 per second. Compression exploits similarity within a frame and between neighboring frames, so one source can be encoded at almost any target bitrate, trading quality against size.
| Quality tier | Typical bitrate |
|---|---|
| Low (mobile, small screen) | Under 1 Mbit/s |
| HD | 3 to 8 Mbit/s |
| Ultra HD | Up to about 25 Mbit/s |
One hour at 7 Mbit/s is about 3 GB per viewer. The metric that matters is sustained throughput. The path must deliver, on average, at least the bitrate being played, or playback stalls.
The player hides short dips with a playout buffer. It downloads ahead of the playback position, starts playing once enough is buffered, and keeps topping the buffer up while it plays.
Adaptive streaming
Section titled “Adaptive streaming”Downloading a single video file gives every viewer the same quality, whatever their connection. Adaptive streaming fixes that with three parts:
- Several renditions - the source is encoded at multiple bitrates and resolutions (a bitrate ladder).
- Segments - each rendition is cut into short pieces, typically 2 to 10 seconds, and every piece is a separate file with its own URL.
- A manifest - a small text file that lists the renditions and where their segments are.
The player fetches the manifest, then requests segments with ordinary HTTP GETs. Before each request it checks measured download speed and buffer level. If the buffer is full and speed is high, it asks for the next segment at a higher bitrate. If the buffer is draining, it steps down. All the intelligence is in the client, and the server is a plain web server.
#EXTM3U#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360360p/index.m3u8#EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720720p/index.m3u8#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x10801080p/index.m3u8This HLS master playlist is one manifest format. DASH is the open equivalent with an XML manifest. Because segments are static files fetched over HTTP, any HTTP cache can store them, and the caching rules in HTTP and web protocols apply unchanged.
# Inspect a manifestcurl -s https://video.example.com/vod/master.m3u8How a CDN works
Section titled “How a CDN works”Cluster placement
Section titled “Cluster placement”A CDN runs server clusters in many locations. There are two ways to place them.
| Strategy | What it means | Advantage | Cost |
|---|---|---|---|
| Enter deep | Many small clusters inside access ISPs, close to users | Fewest hops, lowest latency, highest throughput | Thousands of sites to run and keep consistent |
| Bring home | Fewer, larger clusters at internet exchange points or major interconnects | Easier to operate | Longer path to the user, lower throughput |
Large operators mix both. See ISP tiers, peering, and IXPs for why exchange points are natural cluster sites.
Routing requests to a cluster
Section titled “Routing requests to a cluster”The user types one hostname, and something has to turn it into the address of a good cluster.
DNS redirection is the most common method. The site’s authoritative DNS answers with a CNAME into the CDN’s domain. The CDN’s authoritative DNS then chooses a cluster and returns its address, using a short TTL so the choice can change quickly.
dig +noall +answer www.example.com
# www.example.com. 300 IN CNAME www.example.com.cdn.example.net.# www.example.com.cdn.example.net. 30 IN A 198.51.100.24Anycast announces the same IP address from many clusters over BGP, and the network delivers each client to the topologically closest one. No DNS decision is needed, and traffic follows routing changes automatically.
Direct assignment is possible when the client already talks to a control service. The service picks a cache and returns its URL or address in an API response or manifest, so DNS plays no part.
Choosing the cluster
Section titled “Choosing the cluster”The CDN usually sees the address of the client’s DNS resolver, not the client. It has three ways to choose.
- Geographic proximity - look up the resolver’s location in a geo-IP database and pick the closest cluster. This is simple, but physical distance is not path length, and a client using a distant public resolver looks like it is somewhere else.
- Measurement - probe from clusters toward resolvers and pick the lowest delay and loss. This follows real network conditions, but many resolvers ignore probes.
- Load - send some clients to a farther cluster when the nearest is overloaded.
The EDNS Client Subnet option lets a resolver pass part of the client’s address to the CDN, which fixes the resolver-location problem at some cost to privacy.
Filling the caches
Section titled “Filling the caches”A CDN can’t store every object everywhere, so it needs a rule for what goes where.
| Pull | Push | |
|---|---|---|
| How content arrives | On first request | Placed ahead of demand |
| On a miss | Fetch from a parent cache or the origin, store it, serve it | Rare, since the object is already there |
| Eviction | Least recently used objects are dropped when storage fills | Planned by the operator |
| Strength | No prediction needed, works for long-tail content | No first-viewer penalty for popular content |
| Weakness | The first viewer in each region waits for the origin | Requires forecasting demand and spare off-peak bandwidth |
Most CDNs use pull by default and push for content they can predict. Tiered caching adds a regional cache between the edge and the origin, so many edge misses for the same object turn into one origin fetch. This origin shield also protects the origin after a purge, when every edge would otherwise miss at once.
Two design patterns are common:
- Appliances inside ISPs - a control plane in a public cloud handles accounts, catalog, and encoding, and decides which cache serves each client. A private fleet of storage-heavy servers sits in ISP networks and at exchange points, filled by push during off-peak hours. The client gets a direct server address from the control plane.
- Private backbone with tiers - a few large data centers handle dynamic work such as search and personalization. Edge sites near ISPs serve static content and video from pull caches. A private WAN links the tiers, and DNS sends clients to the lowest-delay cluster, spread out to balance load.
Putting a CDN in front of your own service
Section titled “Putting a CDN in front of your own service”- Set
Cache-Controlon every response. A CDN without it falls back to its own defaults. - Use versioned asset names so you rarely need to purge.
- Exclude cookies and tracking query strings from the cache key for static paths. Otherwise every user is a separate cache entry.
- Restrict the origin to the CDN’s address ranges, or require a shared secret header, so attackers can’t bypass the cache and hit the origin directly.
- Use TLS between the CDN and the origin as well as to the user.
To tell whether an object came from the cache, check the response headers. Names vary by provider, but Age, Via, and an X-Cache style header are common.
curl -sI https://www.example.com/static/app.css | grep -iE '^(age|via|x-cache|cache-control)'
# cache-control: public, max-age=86400# age: 4172# x-cache: HITAn age above zero means the response was served from a cache and is that many seconds old.
Troubleshooting
Section titled “Troubleshooting”| Symptom | Likely cause | Fix |
|---|---|---|
| Old content after a deploy | Cached under the old name with a long TTL | Version filenames, or purge the CDN and shorten max-age |
| Low cache hit ratio | Cookies or unique query strings in the cache key, or private/no-store sent by the origin | Normalize the cache key and send public caching headers for static paths |
| Users served from a far-away cluster | Clients use a distant public resolver and the CDN lacks Client Subnet data | Enable EDNS Client Subnet, or use an anycast CDN |
| Origin overloaded right after a purge | Every edge misses at once | Enable tiered caching or an origin shield, and purge in stages |
| Video buffering for one ISP only | Congested interconnect between the CDN and that ISP | Compare mtr results with another network and raise it with the CDN |
| Origin returns 403 only for CDN traffic | Origin firewall does not allow the CDN’s address ranges | Allow the published CDN ranges or use the shared secret header |