Skip to main content

Linux Networking Stack: From Packets to Applications

Master the Linux networking stack through interactive visualizations. Understand TCP/IP layers, sockets, iptables, routing, and network namespaces.

The Internet Post Office

Every time you browse a website, your computer performs an intricate dance involving multiple layers of wrapping, addressing, and routing. Think of it like a multinational postal system:

The Network as a Postal System

  • Your message

    → The letter content (HTTP request, file data)
  • TCP envelope

    → Tracking number + delivery confirmation (ensures nothing gets lost)

  • IP envelope

    → Street address (tells routers where to send it)
  • Ethernet envelope

    → Local mailroom routing (MAC addresses for the local network)

  • Security checkpoint

    → iptables/netfilter (inspects and filters every package)

Let's open each envelope and see how your data actually travels from application to wire.


Packet Encapsulation: The Russian Nesting Doll

When you send data, it gets wrapped in headers at each layer—like putting a letter in progressively larger envelopes. Watch the encapsulation process:

Packet Encapsulation Journey
Direction
Step 1 / 4
Data flows down through layers
Application
HTTP
12B

Your application writes "Hello World" to a socket

Transport
TCP
Network
IP
Data Link
Ethernet
Packet Structure
Data 12B
Payload
12B
Headers
0B
On wire
12B
Application data ready to send
ApplicationTransportNetworkData Link

Layer Responsibilities

Each layer has a specific job:

LayerProtocolWhat It AddsWhy
ApplicationHTTPRequest/responseYour actual data
TransportTCPPorts + sequencingWhich app, reliable delivery
NetworkIPIP addressesWhere to route globally
Data LinkEthernetMAC addressesWhere on local network
Note

Why So Much Overhead?

For a 12-byte "Hello World", you add 54 bytes of headers—that's 82% overhead! This is why small packets are inefficient, and why protocols like HTTP/2 batch multiple requests together. The overhead is the same whether you send 12 bytes or 1,400 bytes.


The TCP Handshake: Establishing Trust

Before any data flows, TCP performs a "three-way handshake" to establish a reliable connection. This is one of the most asked-about networking concepts:

TCP Three-Way Handshake
Mode
Step 1 / 5
Client
State
CLOSED
Server
State
LISTEN
Ready
Server is listening, client ready to connect

Why Three Packets?

The handshake proves that both sides can send AND receive:

text
Client                              Server
   |                                   |
   |  -------- SYN (seq=1000) ------>  |  Client proves: "I can send"
   |                                   |
   |  <-- SYN-ACK (seq=2000,ack=1001)  |  Server proves: "I can receive AND send"
   |                                   |
   |  -------- ACK (ack=2001) ------>  |  Client proves: "I can receive"
   |                                   |
   |        Connection ESTABLISHED     |

The sequence numbers (1000, 2000, etc.) are how TCP tracks every byte—they're essential for detecting lost or reordered packets.


Socket Programming: The Application Interface

Sockets are the API between applications and the network stack. Under the hood, every socket operation is a system call that transitions from user space into the kernel. Every socket is an integer file descriptor indexing a per-process kernel table: fd 0, 1, and 2 are stdin, stdout, and stderr; fd 3 and above can name sockets, files, or pipes. accept() returns a new fd for each client while the listening socket remains open. Follow the system calls that create, configure, and use a socket.

Socket Lifecycle Demo
Role
Step 1 / 8
server
CLOSED
No socket exists yet
Code:
Network terminal output
// Server starting up...
Role
server
State
CLOSED
File descriptor
not allocated
System Call Sequence:

Socket Types

c
// TCP Socket (reliable, ordered delivery)
int tcp_sock = socket(AF_INET, SOCK_STREAM, 0);

// UDP Socket (fast, no guarantees)
int udp_sock = socket(AF_INET, SOCK_DGRAM, 0);

// Raw Socket (direct IP access, requires root)
int raw_sock = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP);

// Unix Domain Socket (local IPC, no network overhead)
int unix_sock = socket(AF_UNIX, SOCK_STREAM, 0);

TCP Server Pattern

c
#include <sys/socket.h>
#include <netinet/in.h>

int create_server(int port) {
    int server_fd = socket(AF_INET, SOCK_STREAM, 0);

    // Allow port reuse (avoid "Address already in use")
    int opt = 1;
    setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    struct sockaddr_in addr = {
        .sin_family = AF_INET,
        .sin_addr.s_addr = INADDR_ANY,
        .sin_port = htons(port)
    };

    bind(server_fd, (struct sockaddr *)&addr, sizeof(addr));
    listen(server_fd, 128);  // Backlog queue

    while (1) {
        int client = accept(server_fd, NULL, NULL);
        handle_client(client);
        close(client);
    }
}

High-Performance I/O with epoll

For handling thousands of connections:

c
int epfd = epoll_create1(0);

struct epoll_event ev = {
    .events = EPOLLIN | EPOLLET,  // Edge-triggered
    .data.fd = socket_fd
};
epoll_ctl(epfd, EPOLL_CTL_ADD, socket_fd, &ev);

// Event loop
struct epoll_event events[MAX_EVENTS];
while (1) {
    int n = epoll_wait(epfd, events, MAX_EVENTS, -1);
    for (int i = 0; i < n; i++) {
        handle_event(events[i].data.fd);
    }
}

Routing and Forwarding

The kernel decides where to send each packet based on its routing table. Understanding how routes are selected is crucial for network debugging. The longest matching prefix wins: a /32 host route overrides a /24 network route, which overrides /16 and /8; 0.0.0.0/0 matches everything but is the fallback. Try the destination samples to see which route wins.

Routing Decision Demo
Destination
Step 1 / 6
Destination IP:
Destination IP in binary:
00001000000010000000100000001000
Routing Table (sorted by prefix length):
Ready
192.168.1.100/32eth0
direct • Host Route
192.168.1.0/24eth0
direct • Local Network
172.17.0.0/16docker0
direct • Docker Network
10.0.0.0/8vpn0
via 10.0.0.1 • VPN Network
0.0.0.0/0eth0
via 192.168.1.1 • Default Gateway

Viewing Routes

bash
# Modern way
ip route show

# Output:
# default via 192.168.1.1 dev eth0
# 192.168.1.0/24 dev eth0 scope link
# 172.17.0.0/16 dev docker0 scope link

Adding Routes

bash
# Static route to a network
ip route add 10.0.0.0/8 via 192.168.1.1 dev eth0

# Default gateway
ip route add default via 192.168.1.1

# Source-based routing (advanced)
ip rule add from 192.168.2.0/24 table custom
ip route add default via 10.0.0.1 table custom

Enable IP Forwarding (Router Mode)

bash
# Temporarily
echo 1 > /proc/sys/net/ipv4/ip_forward

# Permanently
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
sysctl -p

Netfilter: The Kernel's Security Checkpoint

Every packet entering or leaving your system passes through netfilter—a core part of the Linux kernel architecture. This is what iptables (and its successor nftables) uses for packet filtering. Try each packet path to see the hooks it traverses; the path determines where a filtering rule belongs.

Netfilter Packet Flow
Packet path
Step 1 / 3

HTTP request to your web server

Packet path
Network source
PREROUTING
INPUT
Local Process destination
Other hooks
FORWARD
OUTPUT
POSTROUTING
Path
PREROUTING → INPUT
Test Packets:
iptables Rules (INPUT chain):
INPUT-m state --state ESTABLISHED,RELATED
Allow established connections
ACCEPT
INPUT-p tcp --dport 22
Allow SSH
ACCEPT
INPUT-p tcp --dport 80
Allow HTTP
ACCEPT
INPUT-i lo
Allow loopback
ACCEPT
Quick Add Rule:

Understanding Netfilter Hooks

Packets traverse different hooks depending on their destination:

text
                          ┌─────────────────┐
                          │  Local Process  │
                          └────────┬────────┘
                                   │
    Network ──►  PREROUTING ──►  INPUT
        │                          ▲
        │                          │ (local destination)
        │                          │
        └──────►  FORWARD ─────────┴──►  POSTROUTING ──►  Network
                    │                         ▲
                    └─────────────────────────┘
                      (forwarded packets)

                          OUTPUT ────────────►  POSTROUTING ──►  Network
                            ▲
                            │
                    ┌───────┴───────┐
                    │ Local Process │
                    └───────────────┘

Common iptables Rules

bash
# Allow established connections (most efficient rule first!)
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Allow SSH from specific network
iptables -A INPUT -p tcp --dport 22 -s 192.168.1.0/24 -j ACCEPT

# Allow HTTP/HTTPS
iptables -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT

# Drop everything else (default deny)
iptables -P INPUT DROP
Order

Rule Order Matters!

iptables processes rules top-to-bottom—first match wins. If you put -j DROP before -j ACCEPT, packets will be dropped before reaching the accept rule. Always put your most-matched rules (like ESTABLISHED) first for performance.


Performance Tuning

Network performance depends heavily on memory management—the kernel allocates buffers (sk_buff structures) for every packet in flight. The most critical tuning parameter is the TCP buffer size. TCP can keep only one buffer of data in flight while it waits for acknowledgments. A 64KB buffer with a 600ms round trip limits throughput to 853 Kbps; a BDP-sized buffer can use the full link, while an oversized one wastes memory and may cause bufferbloat.

TCP Buffer Tuning Simulator
Network preset
100 Mbps
50 ms
131,072
BDP
100 Mbps × 50 ms = 610.4 KB
TCP buffer
128.0 KB
Achieved throughput
21.0 Mbps

BDP is the data in flight when the link is fully utilized.

Buffer Utilization100%
128.0 KB / 128.0 KB
Buffer too small — throughput limited!
Achieved Throughput21.0 Mbps
0 Mbps21% of max (100 Mbps)100 Mbps
Apply these settings:
sysctl commands
# Set maximum buffer sizes
sysctl -w net.core.rmem_max=131072
sysctl -w net.core.wmem_max=131072

# Set TCP buffer auto-tuning range
sysctl -w net.ipv4.tcp_rmem="4096 87380 131072"
sysctl -w net.ipv4.tcp_wmem="4096 65536 131072"

TCP Buffer Tuning Commands

bash
# Increase buffer sizes for high-bandwidth links
sysctl -w net.core.rmem_max=134217728
sysctl -w net.core.wmem_max=134217728
sysctl -w net.ipv4.tcp_rmem="4096 87380 134217728"
sysctl -w net.ipv4.tcp_wmem="4096 65536 134217728"

# Use BBR congestion control (Google's algorithm)
sysctl -w net.ipv4.tcp_congestion_control=bbr
sysctl -w net.core.default_qdisc=fq

Network Interface Tuning

bash
# Increase ring buffer
ethtool -G eth0 rx 4096 tx 4096

# Enable offloading
ethtool -K eth0 gso on gro on tso on

# Receive Packet Steering (multi-core)
echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus

Network Namespaces

Linux can create isolated network environments—this is how containers get their own network stack. Network namespaces work alongside other namespace types (PID, mount, user namespaces) to provide full container isolation.

bash
# Create namespace
ip netns add mycontainer

# Create virtual ethernet pair
ip link add veth0 type veth peer name veth1

# Move one end into namespace
ip link set veth1 netns mycontainer

# Configure host side
ip addr add 10.0.0.1/24 dev veth0
ip link set veth0 up

# Configure container side
ip netns exec mycontainer ip addr add 10.0.0.2/24 dev veth1
ip netns exec mycontainer ip link set veth1 up
ip netns exec mycontainer ip link set lo up

# Test connectivity
ping 10.0.0.2

Debugging Tools

Essential Commands

bash
# Active connections
ss -tunap

# Packet capture
tcpdump -i eth0 'tcp port 80'

# Route tracing
traceroute google.com
mtr google.com  # Better interactive version

# Performance testing
iperf3 -s        # Server
iperf3 -c host   # Client

Common Pitfalls

Common Networking Mistakes

  1. Small buffers on high-latency links. Default 64KB buffers throttle satellite/VPN connections. Calculate BDP and size buffers accordingly.
  2. iptables rule ordering mistakes. Putting DROP before ACCEPT, or putting expensive rules before ESTABLISHED. First match wins!
  3. Forgetting to enable ip_forward. Linux won't route packets between interfaces unless net.ipv4.ip_forward=1.
  4. Blocking I/O for high-concurrency servers. Use epoll/kqueue for thousands of connections. Blocking I/O means one thread per connection.
  5. Ignoring connection state tracking overhead. conntrack uses memory per connection. High-traffic servers may need to tune nf_conntrack_max.

If you found this explanation helpful, consider sharing it with others.

Mastodon