I/O Models Primer

A blocking read(fd, buf, n) on a socket: if data is available, copy it and return. If not, sleep the calling thread until data arrives or the connection closes. From the application's point of view, the call "just works"; the thread is suspended invisibly and resumed when I/O completes. Simple to program — but the suspended thread is a kernel resource, and you can't handle another connection on the same thread while waiting.

A non-blocking read (set viaO_NONBLOCK on the fd, or via fcntl) on the same socket: if data is available, copy it and return. If not, return immediately with errno = EAGAIN. The application must then decide what to do — try again later, poll for readiness with select/epoll/io_uring, or give up.

Why this dictates architecture

With blocking I/O, the natural concurrency primitive is "one thread per request": each connection's handler thread blocks on its own reads. Simple but heavy — kernel threads (process-thread primer) cap at a few thousand per machine.

With non-blocking I/O, the natural primitive is anevent loop: one thread (or a small pool) cycles through ready fds, doing a bit of work for each, then asking the kernel which fds are ready next. This is what enables a single process to handle 100K+ connections.

The cost of pretending

Many high-level runtimes hide non-blocking I/O behind a blocking-looking API. Go goroutines block on conn.Read, but the runtime translates that into non-blocking + park on epoll under the hood. Java virtual threads (Loom) do the same. Rust async / await ditto. The programmer writes synchronous-looking code; the runtime multiplexes many such "blocking" calls onto a small kernel-thread pool backed by epoll or io_uring.

The cost of this transparency: every blocking call that has no async equivalent (file system calls in many runtimes, third-party C libraries) pins the underlying kernel thread. If your goroutine pool of 10 OS threads has one of them blocked on a slow open(), you've lost 10% of your I/O capacity. Go uses a workaround (a separate pool for syscalls); JVM's Loom does too.

Three intermediate models

• Multiprocess (Apache prefork) — fork a process per request. Isolation, but huge per-request cost.
• Thread pool + queue — bounded threads pull from a request queue. Bounded resource use, but head-of-line blocking when one request stalls a thread.
• Reactor pattern — single thread runs an event loop dispatching events to handlers. Modern variant of the select/epoll-based event loop. Used by Node.js, nginx, Twisted.

The job: a single thread holds N open sockets and wants to be woken when any of them is ready to read or write. Three generations of API:

select() — 1980s

Caller passes three fd_set bitmaps (read-ready, write-ready, exception). Kernel blocks until any fd is ready, then returns the bitmaps with bits set for ready fds. Caller scans the bitmaps to find which ones to handle.

Problems: the bitmap has a compile-time-fixed size (FD_SETSIZE, usually 1024). The bitmap is copied user→kernel→user on every call. Userspace must scan the entire bitmap each call to find ready fds. Throughput collapses past ~1000 fds.

poll() — 1980s, BSD's answer

Caller passes an array of pollfd structs (fd + events-wanted + events-received). No fixed size. Otherwise the same problem: kernel walks the whole array each call to check readiness, and userspace walks the whole array to find what's ready. O(N) per call.

epoll — 2002 Linux

The breakthrough that made "C10K" (10K concurrent connections per server) feasible. Three syscalls:

• epoll_create — create an epoll instance, returning an fd that represents it.
• epoll_ctl — add / modify / remove fds from the set being watched. Called once per fd lifecycle, not per wakeup.
• epoll_wait — block until any registered fd is ready, returns just the ready ones (not the entire set).

The key win: the kernel maintains the set across calls (no copy) and returns only the ready fds (O(active), not O(N)). For 10K connections where only 100 are active at any given moment, epoll does ~100 units of work per wakeup; select would do 10000.

Level-triggered vs edge-triggered

Level-triggered (LT) — epoll reports any fd that is currently ready, every wakeup. If you don't fully drain the socket, the next epoll_wait reports it again. Default; easiest to use.

Edge-triggered (ET) — epoll reports an fd only when its readiness state changes. After it's reported once, you must read until EAGAIN (drain the socket) or you'll miss the next data. Slightly faster (no repeated wakeups for a stable-ready fd) but harder to use correctly. Used by performance- critical servers (HAProxy, Envoy).

kqueue — BSD's parallel evolution

FreeBSD/macOS solved the same problem with kqueue(1999, two years before epoll). API is more general — kqueue can watch arbitrary events (file changes, process events, timers, signals) not just fds. Performance is comparable to epoll. The portable cross-platform abstraction (libevent, libev, libuv) hides whether you're running on kqueue or epoll underneath.

io_uring (Linux 5.1, 2019) is built around two shared ring buffers between user code and the kernel:

• Submission Queue (SQ) — user code writes I/O requests (read this fd into this buffer, accept a connection on this socket, etc.) into ring slots. Each entry is a fixed-size sqe describing one operation.
• Completion Queue (CQ) — kernel writes completed operations' results into ring slots. Each entry is a fixed-size cqe describing one completed operation.

Both rings are mapped into both kernel and user address spaces. User code writes SQ entries and updates the SQ tail pointer with no syscall. The kernel polls or is notified via io_uring_enter (one syscall potentially submitting many ops); processes the ops; writes CQ entries; updates the CQ head pointer. User code reads CQ entries with no syscall.

What it changes

With epoll, every I/O operation needs a syscall (the actual read or write) plus periodic epoll_wait calls. With io_uring, you can submit and reap thousands of ops with zero or one syscall total. On a benchmark of 4 KB random disk reads, io_uring achieves ~2-3× the throughput of epoll + read with the same hardware, because the syscall overhead disappears.

It also natively supports things epoll never could: filesystem I/O (epoll only works on sockets, pipes, etc.), buffered operations, fixed buffers (pre-registered for less per-op copying), kernel polling (the kernel polls the SQ in a separate thread, eliminating even the io_uring_enter syscall under load).

Where it's used

• fio — the storage benchmarking tool grew an io_uring backend before most production code did.
• libuv (Node.js) — io_uring backend optional in recent versions.
• Tokio (Rust) — async runtime with optional io_uring backend via tokio-uring.
• Postgres 17+ — uses io_uring for some I/O paths.
• Production servers — adoption is slower than the benchmarks suggest. Security CVEs (several in 2022-2023) led some distros (Google, ChromeOS) to disable io_uring for unprivileged users. The API is also more complex than epoll, and the performance win is only meaningful past 100K ops/sec.

The mental model shift

epoll says: "tell me when fd X is ready, and I'll come do the I/O myself." io_uring says: "here's a list of operations, do them and tell me when they're done." The first is readiness-based (you do the work); the second is completion-based (the kernel does the work and reports back). Completion-based scales better but is harder to fit into existing reactor codebases.

Consider serving a 100 MB file over HTTP. The naive code is:

for (;;) {
    n = read(file_fd, buf, sizeof(buf));   // file → kernel page cache → user buf
    if (n <= 0) break;
    write(socket_fd, buf, n);              // user buf → kernel socket buf → NIC
}

Two copies per chunk, plus two syscalls. For a 100 MB file with 64 KB chunks: 1600 syscalls and 200 MB of CPU-driven memory copy.

sendfile()

sendfile(out_fd, in_fd, offset, count) tells the kernel to move bytes directly from the file's page cache to the socket's send queue, with no user-space copy. One syscall instead of N read+write pairs, half the memory copies eliminated. This is what nginx, Apache, and any static-file CDN use under the hood for response bodies.

On supported NICs with DMA scatter-gather, sendfile can be a true zero-copy operation: the NIC reads the data directly from kernel page cache via DMA, never staging through CPU.

splice() and tee()

splice(in_fd, in_offset, out_fd, out_offset, count, flags)is sendfile's more general cousin: move bytes between any two fds via an intermediate kernel pipe, without ever touching user space. Used for proxying (read from one socket, write to another without copying through user code). tee() duplicates the bytes between two pipes — used for logging or branching.

MSG_ZEROCOPY

For send() on a socket, MSG_ZEROCOPYflag tells the kernel to keep the user buffer pinned and DMA from it directly, without copying to a kernel buffer. The user code gets an asynchronous notification (via the socket's error queue) when the bytes have been sent and the buffer is reusable. Saves the user→kernel copy for large sends; cost is the notification handling and the constraint that the buffer can't be reused or freed early.

io_uring registered buffers

io_uring (previous section) supports pre-registered buffers (IORING_REGISTER_BUFFERS): the user gives the kernel a list of buffers up front; subsequent I/O ops reference them by index. The kernel pins them once (no per-op pinning cost) and knows they're safe to DMA into directly. This is the modern way to get zero-copy in a high-throughput async server.

When zero-copy helps and when it doesn't

• Helps: serving large static files (sendfile gives nginx the throughput edge over apps that read+write); high-throughput proxies (splice for connection-to-connection plumbing); video / file servers (DMA scatter-gather to NIC).
• Doesn't help: small messages where the syscall overhead dominates anyway; data that needs transformation between read and write (encryption, encoding) — you have to copy to transform; databases doing random small reads — the page cache already absorbs that.
• Modern alternative: DPDK / AF_XDP — userspace bypasses the kernel network stack entirely, polling the NIC directly. Used in extreme-performance scenarios (cloud-vendor load balancers, financial trading systems). Different paradigm — gives up generality for raw throughput.

Red flags in code review

• select() in new code. FD_SETSIZE caps at 1024 on most systems, O(N) scan per call. Use epoll on Linux, kqueue on BSD, or a portable abstraction (libuv).
• Edge-triggered epoll without drain-to-EAGAIN.Will miss data and look like silent message loss. Either carefully drain or switch to level-triggered.
• Calling blocking syscalls from a goroutine / async task.Pins the kernel thread; other goroutines on the same runtime thread block until it returns. Use the async equivalent or push to a worker thread.
• read+write loop for serving large static files.Use sendfile. Apache and nginx do this; your custom file server should too.
• Mixing blocking and non-blocking on the same fd.Forgetting to set O_NONBLOCK on an accepted socket while running an event loop means a single slow client can stall the entire loop on the next read.