Skip to content
Documentation Background

QoS and Traffic Shaping

Networks have finite bandwidth. When the pipe is full, packets queue and then get dropped - this affects all traffic equally by default. Quality of Service (QoS) lets you prioritize some traffic over others, ensuring critical traffic (VoIP calls, video conferencing) isn’t degraded by bulk traffic (backups, downloads).

Without QoS: With QoS:
┌────────────────┐ ┌────────────────┐
│ VoIP (20 Kbps) │ ──mix──▶ 🚧 │ VoIP (20 Kbps) │ ──fast lane──▶ ✅
│ Backup (100Mbps│ │ Backup (100Mbps) │ ──slow lane──▶ ✅
│ All suffer! │ │ Both work! │
└────────────────┘ └────────────────┘

DSCP is a 6-bit field in the IP header (within the former Type of Service byte). Routers read this field to determine how to treat the packet. The sender marks the packet, and every router along the path can honor that marking.

IPv4 Header - Type of Service (ToS) Byte:
┌──────────────────────────┬─────┬───┐
│ DSCP (6 bits) │ ECN │ │
│ Differentiated Services │ (2b)│ │
└──────────────────────────┴─────┴───┘
PHBDSCP ValueBinaryTraffic TypeDrop Probability
Default (BE)0000000Best Effort - no priorityStandard
CS18001000Background (lower than default)High
AF1110001010Bulk data - low priorityLow
AF2118010010Bulk data - medium priorityLow
AF3126011010Business critical appsLow
AF4134100010Video conferencingLow
CS540101000SignalingLow
EF46101110Expedited Forwarding - VoIPVery Low
CS648110000Network control (routing protocols)Very Low
CS756111000Network managementVery Low

Before a packet can be prioritized, it must be identified:

MethodHow
DSCP/TOS markingRead the DSCP field set by the sender
Port-basedTCP/UDP port number (SSH=22, SIP=5060)
IP addressSource or destination IP
ProtocolTCP, UDP, ICMP
Deep Packet Inspection (DPI)Look inside the payload (detects apps using non-standard ports)

Queuing determines what to do when the link is congested:

AlgorithmHow it worksBest for
FIFO (First In, First Out)No priority, pure orderSimple links, no QoS
WFQ (Weighted Fair Queuing)Multiple queues, each gets bandwidth shareGeneral QoS
CBWFQ (Class-Based WFQ)Admin-defined classes each get minimum bandwidth %Enterprise QoS
LLQ (Low-Latency Queuing)Strict priority queue for real-time + CBWFQ for restVoIP environments
HTB (Hierarchical Token Bucket)Linux’s class-based queuingLinux servers/routers
HFSCHierarchical fair service curvesComplex multi-class scenarios

When packets wait at an output port, the scheduler picks which one goes next. Four basic disciplines cover most of the table above.

DisciplineRuleRiskLinux qdisc
FIFOSend in arrival orderOne bulk flow delays everything behind itpfifo, bfifo
Strict priorityAlways serve the highest non-empty class firstLow classes starve if high-priority traffic never stopsprio
Round robinTake one packet per class in turnGives equal packet counts, not equal bytessfq, fq (per flow)
Weighted fair queuingRound robin where class i gets weight w_iNeeds sensible weightshtb, hfsc, fq_codel

All four are work-conserving: the link never idles while a packet is queued, and an empty class is skipped at once. Priority scheduling is also non-preemptive. A packet already on the wire finishes before a higher-priority packet is sent.

With weighted fair queuing, class i is guaranteed R x w_i / sum(w_j) while all classes have traffic queued. On a 100 Mbit/s link with weights 5, 3, and 2, the classes get 50, 30, and 20 Mbit/s under full load. If the third class goes idle, its share is split among the others in proportion to their weights. In htb the guarantee is the rate and the borrowing limit is the ceil.

Terminal window
# Which qdisc is attached, and what is the system default?
tc qdisc show dev eth0
sysctl net.core.default_qdisc
# net.core.default_qdisc = fq_codel
MechanismWhat it doesEffect on excess
Traffic ShapingBuffers excess traffic and sends later (smooth the burst)Delay (no drop if buffer not full)
Traffic PolicingDrops or re-marks packets exceeding the rateDrop or DSCP downgrade
Shaping: ████░░████░░████ → ██████████████
(smoothed, delayed)
Policing: ████░░████░░████ → ████ ████
(excess dropped)

The tc command (from iproute2) implements QoS on Linux. It can:

  • Set up queuing disciplines (qdiscs)
  • Create traffic classes
  • Add filters to classify traffic
Terminal window
# Limit outbound traffic to 10 Mbit/s total
sudo tc qdisc add dev eth0 root handle 1: htb default 10
# Add a class with 10 Mbit/s ceiling
sudo tc class add dev eth0 parent 1: classid 1:10 htb rate 10mbit
# Apply to all traffic on eth0
sudo tc filter add dev eth0 protocol ip parent 1:0 u32 match ip dst 0.0.0.0/0 flowid 1:10
# View current qdisc setup
tc qdisc show dev eth0
tc class show dev eth0
tc filter show dev eth0
Terminal window
# Create 3-priority prio qdisc
sudo tc qdisc add dev eth0 root handle 1: prio priomap 0 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2
# Band 0 (highest) = DSCP EF (VoIP)
# Band 1 (medium) = DSCP AF
# Band 2 (lowest) = best effort
# Filter: DSCP 46 (EF) → priority band 0
sudo tc filter add dev eth0 parent 1:0 protocol ip handle 0x2e/0xfc fw flowid 1:1
# View stats (bytes/packets per queue)
tc -s qdisc show dev eth0

Simple Bandwidth Cap with Token Bucket Filter

Section titled “Simple Bandwidth Cap with Token Bucket Filter”
Terminal window
# Limit a single IP to 1 Mbit/s (rate limiting for a client)
sudo tc qdisc add dev eth0 root handle 1: htb default 10
sudo tc class add dev eth0 parent 1: classid 1:10 htb rate 10mbit # default class
sudo tc class add dev eth0 parent 1: classid 1:20 htb rate 1mbit # limited class
# Filter: traffic to 192.168.1.100 → limited class
sudo tc filter add dev eth0 protocol ip parent 1:0 prio 1 \
u32 match ip dst 192.168.1.100/32 flowid 1:20
# Clean up
sudo tc qdisc del dev eth0 root

Terminal window
# Mark packets for QoS classification using iptables (DSCP marking)
# Mark all SSH traffic as DSCP CS6 (network control)
sudo iptables -t mangle -A OUTPUT -p tcp --dport 22 -j DSCP --set-dscp-class CS6
# Mark VoIP (SIP port 5060) as EF
sudo iptables -t mangle -A OUTPUT -p udp --dport 5060 -j DSCP --set-dscp 46
# Mark backup traffic as CS1 (background, below best-effort)
sudo iptables -t mangle -A OUTPUT -p tcp --dport 873 -j DSCP --set-dscp-class CS1 # rsync
# View marked packets
sudo iptables -t mangle -L OUTPUT -n -v

Terminal window
# Check if packets are being queued/dropped
tc -s qdisc show dev eth0
# Output includes: sent X bytes X pkt (dropped X, overlimits X requeues X)
# Check DSCP markings on packets leaving your machine
sudo tcpdump -nn -i eth0 -e 'ip[1] & 0xfc != 0'
# Shows all packets where DSCP != 0 (i.e., marked packets)
# Monitor per-class traffic rates
watch -n 1 'tc -s class show dev eth0'
# ss - check for large receive queues (signs of congestion)
ss -tnp | awk 'NR>1 && $2+0 > 0 {print "Recv-Q:", $2, $5}'

Bufferbloat happens when routers/modems have excessively large buffers. Traffic doesn’t get dropped fast enough, so it sits in the buffer causing high latency:

No buffer bloat: 20ms RTT
With bufferbloat: 500ms RTT (buffer filling = 480ms of queued data)

Fix: CoDel (Controlled Delay) or FQ-CoDel manages queues intelligently:

Terminal window
# Replace default qdisc with FQ-CoDel (reduced bufferbloat)
sudo tc qdisc replace dev eth0 root fq_codel
# Or cake (even better, does fairness + bufferbloat management together)
sudo tc qdisc replace dev eth0 root cake bandwidth 100mbit

A buffer absorbs bursts, which lowers loss. It also adds delay. Ten times the buffering can mean ten times the queueing delay, and a longer RTT makes TCP senders slower to notice congestion.

The standing queue comes from ACK clocking. A burst fills the queue. Each returning ACK lets the sender release one more packet, so the queue stays full for as long as the transfer runs. Latency-sensitive traffic such as calls and games shares that queue and waits behind the bulk transfer.

The classic rule sets the buffer to the bandwidth-delay product, B = RTT x C. A 10 Gbit/s link with a 250 ms RTT needs 2.5 Gbit. When many independent TCP flows share a backbone link, the aggregate smooths out and the need drops to B = RTT x C / sqrt(N). With 10,000 flows, that 2.5 Gbit falls to 25 Mbit. Edge devices have few flows, so they gain little from the second rule and are the usual place for bufferbloat. Router switching and queueing shows where both rules come from.

A drop-tail queue drops only when it is completely full, so senders learn about congestion late. Active queue management (AQM) drops or marks packets earlier, while the queue is still short.

AlgorithmSignal it reacts toNotes
REDAverage queue length. Drops with rising probabilityThe original AQM. Hard to tune, so rarely used on its own
CoDelTime packets spend in the queue (target about 5 ms)No tuning parameters. The basis of fq_codel
PIEEstimated queueing delayUsed by DOCSIS 3.1 cable modems

AQM can mark packets with ECN instead of dropping them, so the sender slows down without losing data. See TCP congestion control for the sender side.

Compare latency on an idle link with latency under load.

Terminal window
# Terminal 1: saturate the link
iperf3 -c iperf.example.net -t 30
# Terminal 2: watch latency while it runs
ping -i 0.2 -c 100 198.51.100.1
# Idle: ~20 ms. Under load with bufferbloat: 300 to 500 ms.
# Drops and ECN marks from the qdisc
tc -s qdisc show dev eth0