I/O Models Primer

A socket read has two jobs: wait until bytes are in the receive queue, then copy them out. The copy is a few microseconds of honest work. The wait is unbounded, and it is the entire design problem.

So watch the wait itself. Drag the slider to move the moment the first byte lands, and watch the thread that is not running stretch to meet it:

Notice that the parked stretch is not the kernel doing anything. The 406 µs the call takes at the shipped setting is 6 µs of copying and 400 µs of nothing, and the thread paid two context switches — 2.6 µs — for the privilege of sleeping through it.

Non-blocking is the same call with the wait deleted. Flip the mode and the syscall returns EAGAIN immediately, and the time the thread keeps is exactly what the parked stretch was:

Watch what the mode switch actually changes: nothing about the data, nothing about the socket, only who owns the gap. Blocking hands it to the kernel and gets simplicity back. Non-blocking keeps it, and now owes an answer to the question the kernel used to answer — when is this descriptor worth asking again?

Blocking is not free just because it is easy. Each waiting connection needs its own thread, and each thread is 16 KiB of kernel stack plus a task_struct. Raise the connections and watch the fixed bill:

At ten thousand connections that is 249 MiB of kernel memory before a single byte moves, and 78 GiB of address space reserved by glibc's 8 MiB default stacks — only the pages a handler touches are real. Push it to a hundred thousand and the threads want 2.4 GiB that never swaps, 3.2 GiB resident all told — a fifth of a 16 GiB machine. Both halves of that 25.5 KiB are countable rather than estimated: 16 KiB is THREAD_SIZE on x86-64, four pages of kernel stack, and 9.5 KiB is what the task_struct slab reports in /proc/slabinfo on a 6.x kernel.

Memory is the bill you can see. The one that hides is the switch: 1.3 µs of scheduler and cold cache, twice per event. Raise the event rate and watch the curve cross the line that is one whole core:

Because two switches are 2.6 µs, the crossing lands at 385,000 events a second — a rate one connection can reach in a benchmark. Past it, a core is spent changing whose registers are loaded, and no application tuning touches it.

Non-blocking on its own does not fix that, because a loop with nothing to wait on has to re-ask. Tighten the interval between asks and watch the two bars trade against each other:

Neither end of that slider is a server. At 20 µs the loop burns 125% of a core answering EAGAIN; at 5 ms it answers with 2.5 ms of mean delay. What is missing is a way to be told, and the next section is twenty-five years of that one idea.

select answered it in 1983 with three bitmaps. You set a bit per descriptor you care about, the kernel hands them back with the ready bits set, and you scan. The bitmap is a fixed-size struct, which is where the trouble starts:

Drag the marker past 1024 and nothing complains. FD_SET is a macro over an array with no bounds check, so a descriptor past the end writes into whatever the caller put next on the stack. No error, no crash at the call — just a corrupted local, later, somewhere else.

poll fixed the ceiling with a caller-sized array, and left the real cost alone. Set the watched set against how many of them are ready, and read the two walks the kernel could have done:

At a thousand watched and ten ready, 990 of the kernel's 1,000 checks were spent on descriptors with nothing to say — and then userspace walked the same array again to find which ten. Both walks are O(n) in the size of the set, not in the size of the answer.

epoll, in Linux 2.5.44, changed one thing: the set lives in the kernel between calls. Step the transport through the three syscalls and watch what comes back in each:

Because the interest list stays registered, the wakeup does not have to describe the set — it only reports the descriptors that fired. The invariant this buys is worth stating exactly: at the top of every loop iteration each descriptor is either in the kernel's interest list or in the ready set the loop is currently draining. One in neither is invisible.

Keeping the set in the kernel is also where epoll stops behaving the way the descriptor number suggests. The interest list keys on the open file description, not on the integer. Step through a dup and a close of the number you registered:

Notice that the entry outlives the descriptor. Events keep arriving for a socket you believe you closed, and EPOLL_CTL_DEL on the closed number answers EBADF — you can no longer name the thing you registered. fork is the same trap from the other side: the child inherits the epoll fd, and both processes are now woken for one socket. Delete before you close, every time.

Set to answer size instead of set size is a different curve, not a better constant. Grow the watched set and watch the two lines separate:

At ten thousand descriptors with one percent ready, one wakeup is 250 µs against 2.8 µs — ninety-one times over. Turn the second slider up, though, and the gap closes: at fifty percent ready epoll is doing almost the same walk, because O(ready) and O(n) are the same when everything is ready.

One shared descriptor breaks the same way. Register a listening socket in n worker processes, then raise the worker count and turn EPOLLEXCLUSIVE on and off under one arriving connection:

Without the flag every waiter is woken; one wins the accept and the rest get EAGAIN and park again, each having paid two context switches for nothing. That is the accept thundering herd, and EPOLLEXCLUSIVE (Linux 4.5) wakes one waiter instead of all of them. It is level-triggered only, and it is not the general fix: SO_REUSEPORT, which gives each worker its own accept queue, is the one that scales.

epoll offers the answer in two shapes, and one of them is a trap. Level-triggered reports anything still readable; edge-triggered reports the arrival, once. Shorten the read and switch modes:

Watch the edge-triggered case at a 16 KiB read: 48 KiB stay in the socket, the loop parks, and no further wakeup is coming for bytes that already arrived. The connection hangs and the logs say nothing. That is the failure mode of this whole section — the invariant broken by a descriptor in neither list, and it fails silently.

Count the boundary crossings a batch of ready operations costs. Two hundred and fifty nanoseconds each with KPTI on, and the readiness loop pays one for the wakeup plus one per operation, where the ring pays one for the batch:

Notice that at a batch of one all three models cost about the same — 2.9 µs, 3.1 µs and 2.9 µs an operation — because the two context switches around the wakeup dominate everything else. The win is not the mechanism, it is the batch: at thirty-two, io_uring pays one crossing where epoll pays 33. The top row is worth reading twice, because it is the assumption the other two are measured against: a thread per connection has no batch at all. Thirty-two ready operations are thirty-two threads, each woken and each parked again, so the two switches are charged per operation and not per batch — which is why it costs 2.9 µs an operation while epoll, paying one crossing more, costs 339 ns.

It buys that with shared memory. The submission ring is mapped into both address spaces at once, so writing a request is a store, not a call. Drag the tail along the ring:

Notice the readout: entries are queued and no syscall has happened. Only two indices matter — the tail you advance and the head the kernel advances — and their difference is the occupancy. Push to eight and the ring is full; submission now blocks on the kernel, which is the worst single operation this model has.

The full cycle is four beats, and only one of them crosses. Step the transport through it:

Because the completions come back into mapped memory too, reaping is another load — the one crossing in the middle is the whole bill. This is the shift from readiness to completion: you no longer ask what is ready, you describe work and are told when it is done, which is also why io_uring covers regular files where epoll never could.

There is a mode that removes even that one call. With IORING_SETUP_SQPOLL a kernel thread spins on the ring, so the crossing never happens. Switch the poller on and read what it costs:

Watch both bars at once: the application's share of the boundary goes to zero, and a whole core is held whether the ring is busy or empty. Jens Axboe's 2019 paper measured 1.7 M IOPS this way against 608 K for libaio on the same device — a 2.8× that is real and that you rent a core to get.

Serving a file from the page cache to a socket is a read into a buffer and a write out of it. Step the transport through one chunk and watch where the payload actually goes:

Notice that the bytes are already in the kernel at step one and end up back in the kernel at step four. The user buffer is a round trip that the data did not need, and each leg of it is the CPU moving every word by hand at memory bandwidth.

Each kernel interface removes exactly one of those legs. Walk the slider from the loop to MSG_ZEROCOPY and watch a hop the CPU never makes take its place:

sendfile deletes the user buffer, so the page cache feeds the socket directly: one copy instead of two, one syscall instead of two. With scatter-gather DMA the card reads the page cache itself and the CPU never touches the payload at all. This is what nginx serves static files with.

Put a size on it. At 10 GB/s per core, drag the file up and read the three bills:

A 100 MiB file costs 22 ms of CPU under the loop, 11 ms under sendfile and 401 µs with scatter-gather — 54 times, and all of it is copying, because the 3,206 syscalls are only 0.8 ms of the total. At this size the boundary is noise and the bandwidth is everything — and the 54 does not move as you drag, because every term in it is per-byte. Only the bills change.

Which is exactly why it inverts for small messages. Shrink the message until the two lines cross:

Because MSG_ZEROCOPY has to pin the pages and reap a completion off the socket's error queue, it carries a flat 1,650 ns that a small message cannot amortise. The crossover is 13.7 KiB, close to the ~10 KiB the kernel documentation gives — under it, zero-copy is slower than the copy it removed, and at 1 KiB it is 1.3 µs slower.

The first axis is memory. An epoll registration is about 200 bytes; a thread is 25.5 KiB of kernel memory that never swaps and never shrinks.

Grow the connection count and watch one descriptor each stay flat under a line that is climbing away from it:

The ratio is flat at 128×; the absolute number is what bites. At ten thousand connections threads want 249 MiB against 2.0 MiB. C10K was a named problem in 1999 and is not one now, and the fix was never a faster thread — it was not having one.

Neither coloured line is what caps the box, which is the follow-up worth having ready. The grey one above both is socket buffers: Linux's default tcp_rmem is 4 KiB / 128 KiB / 6 MiB, and both models pay it per connection. Ten thousand active connections is 1.2 GiB of receive buffer — five times the thread cost. Dropping threads buys you the axis below it, not that one.

The second axis is the one people skip. An event loop is a single thread, so anything that blocks inside it blocks everyone. Drag the blocking call out from zero:

Because the loop cannot preempt itself, all 199 other ready connections wait out the whole call — a 10 ms getaddrinfo is 10 ms on every one of their p99s. A goroutine is the same shape: the runtime parks you on epoll for the I/O it knows about, and pins a kernel thread for everything it does not.

The third axis is whether you may run it at all. io_uring is the fastest thing here and the least available: Docker's default seccomp profile blocks io_uring_setup, Google turned it off across ChromeOS and Android in 2023, and kernels from 6.6 ship kernel.io_uring_disabled, which some distributions set to 2. Write the epoll path first.

/* EPOLLET, one read: 48 KiB stranded */
n = read(fd, buf, sizeof buf);

/* correct: drain until EAGAIN */
while ((n = read(fd, buf, sizeof buf)) > 0)
  handle(buf, n);

• select in new code. FD_SETSIZE is 1024 and the macro does not check it.
• Edge-triggered without a drain loop. Silent data loss; use level-triggered unless you have measured a reason.
• A blocking call inside an event loop. DNS, a file open, a lock — every ready connection pays for it.
• read plus write to serve a static file. Two copies and twice the syscalls; sendfile is one line.
• MSG_ZEROCOPY on small messages. Under about 10 KiB the pinning costs more than the copy.