Skip to content
Documentation Background

Introduction to Networking

TCP/IP 5-Layer Model

Every network communication passes through a stack of layers. Each layer has a specific job, and each layer only talks to the layers directly above and below it.

LayerNameProtocol ExamplesWhat it does
5ApplicationHTTP, FTP, SMTP, DNS, SSHUser-facing data exchange - what apps actually send/receive
4TransportTCP, UDPReliable (TCP) or fast (UDP) delivery to the right application via ports
3NetworkIP (IPv4/IPv6)Routing across networks - getting data from network A to network B
2Data LinkEthernet, Wi-Fi (802.11)Getting data across a single link/segment - MAC addressing
1PhysicalCables, radio waves, fiberMoving raw bits (1s and 0s) across a wire, fiber, or air

Think of sending a package (for a full worked example across all five layers, see A web request, end to end):

  • Physical layer = the delivery truck and the roads
  • Data link layer = how the truck gets from one intersection to the next
  • Network layer = which roads to take from address A to address B
  • Transport layer = the driver knowing how to knock on your door to confirm delivery
  • Application layer = the contents of the package itself

Each layer works because both ends follow the same protocol: the message formats, their order, and what each side does on receipt. The IETF publishes internet protocols as numbered RFCs, such as RFC 791 for IPv4, RFC 9293 for TCP, and RFC 9110 for HTTP semantics. The IEEE 802 committees define link-layer standards: 802.3 is Ethernet and 802.11 is Wi-Fi. When two implementations disagree, the RFC is the tiebreaker.

Terminal window
# Read an RFC from the terminal
curl -s https://www.rfc-editor.org/rfc/rfc9293.txt | less

CAT Cables Comparison

The physical medium that carries data. Two main categories:

TypeMediumSpeedDistanceUse Case
Cat5Copper twisted pair100 Mbps100mLegacy (avoid for new installs)
Cat5eCopper twisted pair1 Gbps100mCommon in existing installs
Cat6Copper twisted pair10 Gbps (up to 55m)100m at 1GStandard for new installs
Cat6aCopper twisted pair10 Gbps100mData centers, PoE applications
Fiber (single-mode)Glass (light pulses)100+ GbpsUp to 80kmLong distance / WAN links
Fiber (multi-mode)Glass (light pulses)10-100 GbpsUp to 2kmData center interconnects

Key concepts:

  • Crosstalk - electrical signal on one wire bleeds onto an adjacent wire. Twisting the pairs together reduces this.
  • Copper uses electrical voltages to represent 1s and 0s
  • Fiber uses pulses of light - faster, longer range, immune to electromagnetic interference, but more expensive and fragile

Twisted pair cable types:

Twisted Pair Cable Types
TypeShieldingCostUse Case
UTP (Unshielded)None - just the twistLowestHome, office (most common)
STP (Shielded)Braided metal shield around all pairsMediumEnvironments with EMI
FTP (Foiled)Foil wrap around all pairsMediumSimilar to STP
Patch Panel
  • RJ-45 is the standard plug/socket for twisted pair Ethernet
  • Link light (on the port) = cable properly connected and both ends powered
  • Activity light (on the port) = data is actively flowing
  • Patch panel = a rack-mounted panel of RJ-45 ports where cable runs terminate. Cables from the panel then connect to switches.
Terminal window
# Check link status on Linux
ip link show
# eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
# ^^ UP = interface enabled, LOWER_UP = cable connected
# Detailed interface info (speed, duplex, link status)
ethtool eth0
# Speed: 1000Mb/s
# Duplex: Full
# Link detected: yes

Hubs, Switches, and Routers Comparison

These three devices form the hierarchy of network infrastructure:

DeviceOSI LayerIntelligenceHow it forwards
HubLayer 1 (Physical)NoneBroadcasts everything to all ports - every device sees every frame
SwitchLayer 2 (Data Link)Reads MAC addressesForwards frames only to the port where the destination MAC lives
RouterLayer 3 (Network)Reads IP addressesForwards packets between different networks based on routing tables

Hub - a “dumb repeater.” All ports share a single collision domain. If two devices send simultaneously, signals collide and both must retry after a random backoff. Hubs are obsolete.

Switch - inspects Ethernet frames, reads the destination MAC address, and forwards only to the correct port. Eliminates collision domains (each port is its own collision domain). This is why switches replaced hubs everywhere.

Router - connects separate networks (your LAN to the internet, office A to office B). Routers inspect IP headers to decide where to forward. They maintain routing tables and share route information with other routers via BGP (Border Gateway Protocol).

  • A server provides data to something requesting it
  • A client requests data from a server
  • The same device can be both: an email server is a client of a DNS server
  • Most devices are primarily one role but play both at different times

The physical layer’s job: move 1s and 0s from one end of a link to the other. Nothing more.

  • Bit = the smallest unit of data (a 1 or a 0)
  • On copper cables, bits are encoded as electrical voltage changes - a process called modulation (or more specifically, line coding)
  • On fiber cables, bits are encoded as pulses of light
  • On wireless links, bits are encoded as radio wave modulations

Twisted pair cables have multiple wire pairs, which enables simultaneous two-way communication:

ModeDirectionDescription
SimplexOne way onlyLike a loudspeaker - data flows in one direction
Half-duplexBoth ways, but not simultaneouslyLike a walkie-talkie - one side at a time
Full-duplexBoth ways simultaneouslyLike a phone call - both sides talk at once

Modern Ethernet runs full-duplex by default. Dedicated wire pairs handle each direction. If you see a link reporting half-duplex, something is wrong (duplex mismatch, bad cable, or a hub in the path).

Terminal window
# Check duplex mode
ethtool eth0 | grep -i duplex
# Duplex: Full
# If you see half-duplex on a modern gigabit link, investigate:
# - Cable problem? (check with cable tester)
# - Duplex mismatch? (both ends should auto-negotiate)
# - Hub in the path? (replace with a switch)

The data link layer solves the problem: multiple devices share the same physical medium - how do we avoid chaos?

Ethernet is the dominant data link protocol. It solved the collision problem with CSMA/CD (Carrier Sense Multiple Access with Collision Detection):

  1. Listen before sending - if the wire is quiet, go ahead and transmit
  2. If a collision is detected - both devices stop transmitting immediately
  3. Random backoff - each device waits a random interval before retrying (prevents repeated collisions)
MAC Address Anatomy

A MAC address (Media Access Control) is a 48-bit hardware address burned into every network interface. It’s globally unique.

A4:C3:F0:2B:7E:91
── ── ── ── ── ──
\__OUI__/ \__Device__/
First 3 octets = OUI (Organizationally Unique Identifier)
Assigned to the manufacturer by IEEE
Last 3 octets = Device ID (assigned by manufacturer, unique per NIC)
Terminal window
# View your MAC address
ip link show eth0
# link/ether a4:c3:f0:2b:7e:91 brd ff:ff:ff:ff:ff:ff
# Look up a MAC address OUI (manufacturer)
# First 3 octets identify the vendor
# https://maclookup.app/ or:
curl -s "https://api.maclookup.app/v2/macs/A4C3F0" | python3 -m json.tool
TypeDestinationWho receives it
UnicastOne specific MAC addressOnly the target device
MulticastA group MAC addressAll devices in the multicast group
BroadcastFF:FF:FF:FF:FF:FFEvery device on the LAN segment

The least significant bit of the first octet determines the type:

  • Bit = 0 -> unicast (destined for a single device)
  • Bit = 1 -> multicast (destined for a group)

Every piece of data on an Ethernet network is wrapped in a frame with this structure:

Ethernet Frame Structure
FieldSizePurpose
Preamble7 bytesAlternating 1s and 0s - lets the receiver sync its clock
SFD (Start Frame Delimiter)1 byteSignals: “the actual frame data starts now”
Destination MAC6 bytesWho should receive this frame
Source MAC6 bytesWho sent this frame
VLAN tag (optional)4 bytesIdentifies which VLAN the frame belongs to (802.1Q)
EtherType2 bytesWhat protocol is in the payload (e.g., 0x0800 = IPv4, 0x86DD = IPv6)
Payload46-1500 bytesThe actual data (IP packet, ARP request, etc.)
FCS (Frame Check Sequence)4 bytesCRC-32 checksum for error detection

A VLAN lets you create multiple logical networks on the same physical switch. Devices on VLAN 10 can’t see traffic on VLAN 20, even if they’re plugged into the same switch.

  • 802.1Q tag is inserted into the Ethernet frame header (4 bytes) to identify which VLAN a frame belongs to
  • Traffic between VLANs requires a router (inter-VLAN routing)
  • Common use: separate guest Wi-Fi (VLAN 100) from corporate network (VLAN 10) on the same physical infrastructure

Trunks, routing between VLANs, and the cloud-scale alternative, VXLAN, are in VLANs and overlays.

Terminal window
# Check VLAN configuration on Linux
ip -d link show
# Create a VLAN interface
sudo ip link add link eth0 name eth0.10 type vlan id 10
sudo ip addr add 192.168.10.1/24 dev eth0.10
sudo ip link set eth0.10 up
# View VLAN membership on a managed switch (varies by vendor)
# Cisco: show vlan brief
# Linux bridge: bridge vlan show
How a Frame Travels Through the Network

Putting it all together - when you browse example.com:

  1. Application layer generates an HTTP request
  2. Transport layer wraps it in a TCP segment (adds port numbers)
  3. Network layer wraps it in an IP packet (adds source/destination IP)
  4. Data link layer wraps it in an Ethernet frame (adds source/destination MAC)
  5. Physical layer converts the frame to electrical signals and pushes it onto the wire
  6. At each router hop, the Ethernet frame is unwrapped and rewrapped with new MAC addresses, but the IP packet inside stays the same until it reaches the destination network

Each layer treats what it receives from above as opaque payload and prepends its own header. The Ethernet layer also appends a trailer with the frame check sequence. The result is a set of nested envelopes, and the unit of data has a different name at each layer.

| Ethernet header | IP header | TCP header | HTTP request | FCS |
|<-- link ------->|<-- network ->|<-- transport -->|<-- application -->|
LayerUnitHeader carriesImplemented by
ApplicationMessageProtocol-specific fields (method, status, record type)Hosts
TransportSegment (TCP) or datagram (UDP)Ports, sequence and acknowledgment numbersHosts
NetworkPacket (datagram)Source and destination IP, TTL, protocolHosts and routers
Data linkFrameSource and destination MAC, frame check sequenceHosts, switches, routers (each hop)
PhysicalBitsNoneEvery device

The table also shows how far up the stack each device looks. A host implements all five layers. A router stops at layer 3: it rebuilds the frame on every hop but never reads the TCP header. A layer 2 switch stops at layer 2. This is why a switch can’t make routing decisions and a router can’t see ports (a firewall or load balancer that does is working above its layer).

tcpdump can print the nested headers directly. The -e flag adds the Ethernet header to the usual IP and TCP output.

Terminal window
sudo tcpdump -e -nn -c 1 -i eth0 'tcp port 443'
# Expected output (abbreviated):
# 10:42:07.118 aa:bb:cc:dd:ee:01 > aa:bb:cc:dd:ee:02, ethertype IPv4 (0x0800), length 74:
# 10.0.1.15.51832 > 203.0.113.10.443: Flags [S], seq 1021876, win 64240, length 0
# ^ link layer (MAC) ^ network layer (IP) ^ transport layer (TCP port)

Layering has a cost as well as a benefit.

  • Modularity - Swap Wi-Fi for fiber and HTTP never notices. Each layer has one job and a defined interface.
  • Duplicated work - Both the link layer and TCP detect errors, and TCP still has to, because a switch can corrupt a frame after the link check passes.
  • Hidden information - A layer can’t see what the others know. Application code can’t tell why a connection is slow, and a router can’t tell what the application cares about.

A host breaks a long message into chunks and wraps each in a header, producing packets. Routers and switches then move packets toward the destination one hop at a time. The complete chain of links and switches a packet crosses is its path.

A switch has to receive an entire packet before it can send the first bit onward. Sending a packet of L bits over a link of R bits per second takes L / R seconds, and that delay repeats on every link along the path.

delay over N links of equal rate = N x (L / R)
Example: 1500 bytes = 12,000 bits, 3 links at 100 Mbit/s
3 x (12,000 / 100,000,000) = 0.36 ms

Each outgoing link has an output queue. If packets arrive faster than the link can send them, they wait in the queue. If the queue is full, the next arriving packet is dropped. Loss and queuing delay are therefore the normal signs of a congested link, not signs of a fault.

Two jobs happen on every router, on very different timescales:

  • Forwarding - the per-packet decision. The router looks up the destination IP in its forwarding table and sends the packet out the matching interface.
  • Routing - the network-wide computation that fills those forwarding tables. Routing protocols run in the background and update the tables when links change.

See Forwarding and routing for the full comparison and Inside a router for the hardware view.

The classic telephone network reserved capacity end to end for the duration of a call. The internet shares capacity packet by packet.

Circuit switchingPacket switching
CapacityReserved per connection (a time slot or frequency band)Shared on demand
SetupSignaling before data flowsNone
Idle timeReserved capacity is wastedOthers use the idle capacity
OverloadNew connections are refusedQueues grow, then packets drop
Used byTraditional voice networksThe internet

Bursty traffic is why packet switching wins. Take a 1 Gbit/s link where each active user needs 100 Mbit/s but is active only 5% of the time. Circuit switching reserves capacity for each user, so it serves 10. With packet switching, 60 users can share the link: the chance that more than 10 are active at once is about 0.02%, so queues rarely build up. This is statistical multiplexing, and it is the economic basis of shared networks, including cloud networks.


The time a packet needs to cross one node is the sum of four delays:

d_nodal = d_proc + d_queue + d_trans + d_prop
DelayWhat it isDepends onTypical size
ProcessingReading the header, choosing the output link, checking errorsRouter hardwareMicroseconds
QueuingWaiting in the output queueTraffic loadMicroseconds to many milliseconds, varies
TransmissionPushing all bits onto the link: L / RPacket size and link rateMicroseconds on fast links
PropagationA bit crossing the medium: d / sDistance and medium (about 2 x 10^8 m/s in fiber or copper)About 5 ms per 1,000 km

Transmission and propagation get mixed up often. Transmission delay depends only on packet size and link rate. Propagation delay depends only on distance. Upgrading a link from 1 Gbit/s to 10 Gbit/s shrinks the first and does nothing for the second, which is why a faster link can’t fix latency between continents.

Queuing delay is the only variable component. It depends on traffic intensity, the ratio of arriving bits to the link’s capacity:

traffic intensity = (L x a) / R
L = average packet size in bits, a = packets arriving per second, R = link rate in bit/s
  • Below 1 - queues stay short while intensity is low. As it nears 1, delay climbs steeply, not gradually.
  • Above 1 - bits arrive faster than they leave. The queue grows until the buffer fills and packets drop.

In practice, a link at 90% utilization already shows latency spikes. If ping latency climbs only while a large transfer is running, the delay is in a queue on the path. See Bufferbloat for the fix.

Throughput is the rate at which data actually arrives at the receiver. It differs from bandwidth, which is the link’s capacity. For a path of several links in series, throughput can’t exceed the slowest one:

throughput = min(R1, R2, ..., RN)

For a single transfer, that slowest link is nearly always the access link at either end. The core is heavily over-provisioned. It becomes the bottleneck only when many flows share one link, each getting R / M for M flows.

Terminal window
# Measure achievable throughput between two hosts
iperf3 -s # on the server
iperf3 -c 10.0.1.20 -t 10 # on the client; reports bandwidth per interval
# Per-hop latency along the path
mtr --report --report-cycles 10 app.example.com

Design principles: narrow waist and end-to-end

Section titled “Design principles: narrow waist and end-to-end”

Many protocols sit above IP and many link technologies sit below it, but only one protocol sits at layer 3.

HTTP SMTP DNS QUIC SSH ... (many application protocols)
TCP UDP (two main transport protocols)
IP (one network protocol: the waist)
Ethernet Wi-Fi cellular fiber (many link technologies)

Any application that speaks IP can run over any network that carries IP. New applications don’t need new routers, and new link technologies don’t need new applications. That single required protocol is why the internet absorbed Wi-Fi, 5G, and virtual networks without redesign.

The same thinking puts intelligence at the edges. Reliability is the standard example. Even if every hop retransmits lost frames, a router can still crash with a packet in its queue, so only the endpoints can confirm that data arrived intact. TCP does that check end to end. Link-layer retransmission, as in Wi-Fi, is only a performance optimization on top.

NAT, firewalls, and deep packet inspection strain this model, because they rewrite or inspect ports and payloads inside the network. They exist for good operational reasons (conserving IPv4 addresses, filtering threats), but they are why a protocol change like QUIC has to be designed around middleboxes that expect TCP.