Syscalls & Interrupts Primer

Three doorways exist, each with a different initiator and a different flavour of cost:

• Syscall — user code voluntarily asks the kernel for something. The instruction (syscall on x86-64, svc #0 on ARM64) is executed by the user; arguments live in well-known registers (assembly-isa primer covers the calling convention). Cost: ~150 ns including KPTI overhead. Used by every read, write, open, fork.
• Interrupt — hardware demands attention. A NIC received a packet, a timer fired, a disk completed an I/O. Comes from outside the CPU's instruction stream; can preempt any user (or kernel) code. Cost varies by handler but the entry/exit is similar to a syscall (~150 ns).
• Exception — the CPU faulted on the user's own instruction. Division by zero, dereferencing an unmapped address, executing an undefined opcode, or hitting a debug breakpoint. The CPU's state is similar to an interrupt entry, but the cause is the code itself rather than an external event.

What's common to all three

All three transitions atomically: (1) raise the privilege ring from 3 to 0; (2) switch to the per-CPU kernel stack; (3) save the user instruction pointer, stack pointer, and flags so the kernel can return; (4) jump to a kernel handler whose address is configured at boot. The hardware ensures user code cannot intercept this — the only way into ring 0 is via a controlled entry point the kernel wrote during initialisation.

Subtle differences

Synchronous vs asynchronous. Syscalls and exceptions are synchronous with user code — they happen because the current instruction stream did something. Interrupts are asynchronous; they can fire at any instruction boundary, including during another kernel handler (with nesting rules).

Resumability. A syscall always returns to the instruction after syscall. An interrupt returns to wherever it was when the IRQ fired, as if nothing happened (the kernel may have done a lot in between). An exception returns to the faulting instruction (retry, after the kernel fixed the cause — e.g. a page fault) or kills the process if not fixable.

Vector dispatch. All three use a CPU table called the Interrupt Descriptor Table (IDT, next section). Each vector (entry index 0-255) maps to a fixed kernel function. Syscalls use a separate fast-path mechanism (SYSCALL/SYSRET on x86-64, SVC on ARM64) that bypasses the IDT for speed, but conceptually it's still a vectored jump.

Why this matters in production

• Interrupt storms. A misbehaving NIC or a software bug that triggers a torrent of interrupts can starve user code. The fix is either fixing the source, IRQ affinity to isolate the storm, or interrupt coalescing (NIC firmware batches before raising the IRQ).
• Syscall reduction. The most common high-throughput-server technique. Each syscall is ~150 ns of pure overhead; batching with io_uring or epoll cuts the count by orders of magnitude.
• Page faults are exceptions, not just signals. The handler decides whether to allocate a page (good), swap one in (slow but correct), or deliver SIGSEGV (your code was wrong).

On x86-64, the IDT lives in memory and the CPU's IDTR register points to it. Each of the 256 entries is a 16-byte gate descriptor containing the handler's address, the privilege-ring transition rules, and a stack-switch hint. The first 32 entries are reserved for CPU-defined exceptions; the remaining 224 are software-assignable for hardware interrupts.

Vector assignments

• 0-31: CPU exceptions. Divide-by-zero (0), debug break (1), page fault (14), general protection (13), invalid opcode (6), and a handful more. Architecturally fixed; the kernel fills them in at boot.
• 32-127: external hardware interrupts (legacy). The IRQ controller (APIC) routes line-based interrupts to these vectors. Modern systems use Message-Signalled Interrupts (MSI / MSI-X) where the device writes a vector directly to memory.
• 128-255: software interrupts and modern MSI. Linux uses vector 128 for the legacy int $0x80 syscall mechanism (deprecated in favour of SYSCALL), and dynamically assigns higher numbers to MSI-driven devices.

How an interrupt actually fires

Walk through, say, a NIC interrupt:

1. NIC writes a Message-Signalled Interrupt: a magic value at a magic address that the CPU sees as "interrupt arriving on vector N". The vector is one of the numbers the kernel assigned to this NIC during PCIe enumeration.
2. CPU finishes the current instruction (interrupts are deferred to instruction boundaries to keep semantics sane), then looks up entry N in the IDT.
3. Entry N tells the CPU: handler address H, target privilege ring 0, load kernel stack from the per-CPU TSS. CPU pushes the user RIP / RSP / RFLAGS onto the kernel stack, then jumps to H.
4. H is the kernel's common-IRQ entry point. It dispatches based on the vector to the registered driver IRQ handler (the NIC's IRQ handler, in this case).

Why syscalls bypass the IDT

Early x86 used int $0x80 for syscalls — which goes through the IDT exactly like any other interrupt. Slow: ~200 cycles for the descriptor load and privilege check. AMD64 added the SYSCALL instruction, which uses MSRs (LSTAR for the handler address, STAR for the segment selectors) to do a direct jump without consulting the IDT. ~75 cycles. Linux switched to SYSCALL by default in the early 2.6 kernel; int $0x80still works for legacy code.

Per-CPU IDTs and APICs

On SMP systems every CPU has its own IDT (logically, though most kernels share the same table by setting all CPUs' IDTR to it). Routing of which CPU handles which IRQ is the job of the IO-APIC (line-based) or the LAPIC + MSI mechanism (modern). IRQ affinity (settable via /proc/irq/N/smp_affinity) pins specific IRQs to specific CPUs — important for performance, especially with high-rate NICs where bouncing the interrupt across CPUs kills cache locality.

The hardware interrupt itself (the IRQ handler, sometimes called the top half) runs with the current IRQ line disabled — no new interrupts of the same type can arrive until it returns. If it spent 10 μs processing the packet, no other packet could fire its interrupt for 10 μs, capping throughput at 100K packets/sec per IRQ line. A modern 100 Gb/s NIC can deliver millions of packets/sec.

The fix: keep the top half tiny. Just acknowledge the device, save the packet pointer somewhere the kernel can find it, schedule asoftirq to do the real work, and return. The softirq runs with interrupts enabled, so the next interrupt is no longer blocked. Real work that doesn't need the interrupt line locked happens in this lower-priority context.

NAPI — the polling trick

At very high packet rates, even one interrupt per packet is too expensive. NAPI (New API, ~2002) lets the NIC driver, after receiving the first interrupt of a burst, disable the device's interrupts and instead poll the RX ring for more packets. Once the ring is empty for some time, re-enable interrupts. This amortises one interrupt over potentially thousands of packets and is what makes 10/40/100 Gb/s NICs feasible.

Tasklets, workqueues, threaded IRQs

Three other deferred-work mechanisms live alongside softirqs:

• Tasklets — older API on top of softirqs, deprecated in newer code. Same idea, slightly different bookkeeping.
• Workqueues — work that can sleep (e.g. needs to allocate memory or wait on a lock) runs in a kernel-thread context via workqueues. Higher latency than softirqs but able to do things softirqs cannot.
• Threaded IRQs — newer mechanism where the IRQ handler is itself a kernel thread. Used by Linux RT preempt patches to make IRQ handling schedulable (so high-priority real-time work can preempt a slow IRQ handler).

Why this matters in user code

You don't write softirq handlers — they're a kernel concern. But three operational implications affect user code:

• ksoftirqd CPU%. If you seeksoftirqd/N burning CPU in top, your kernel is doing too much softirq work — usually from a network or disk firehose. Check NIC interrupt rates with perf top.
• IRQ affinity matters. Spreading NIC IRQs across CPUs (via set_irq_affinity.sh or RPS / RFS) lets softirqs run on multiple cores. Without it, one core handles all packets and bottlenecks.
• Latency in user code. A long-running softirq can delay scheduling of user threads by milliseconds. Real-time configurations enable PREEMPT_RT and threaded IRQs to bound this.

A signal is a small integer (1-31 for standard, 32-64 for real-time) delivered to a process to indicate an event. SIGTERM means "please exit cleanly"; SIGKILL means "the kernel is ending you now and you cannot resist"; SIGSEGV means "you accessed memory you shouldn't have"; SIGPIPE means "you wrote to a socket whose far end has closed." About 30 standard signals exist; most never matter for application code.

How a signal is delivered

When the kernel decides to deliver a signal (e.g. kill(pid, 15)from another process, or a SIGSEGV from a fault, or SIGCHLD when a child exits), it sets a bit in the target process's pending-signal mask. On the target process's next return from kernel mode to user mode, the kernel checks the mask, picks the highest-priority pending signal not currently blocked, and arranges for the signal handler to run instead of the next user instruction. After the handler returns, control resumes where the signal interrupted.

Default actions

Each signal has a default disposition: term (kill the process), core (kill and dump core), ignore (do nothing), stop, continue. The process can install a handler via signal (POSIX) or sigaction (POSIX with more knobs). Two signals cannot be caught or ignored: SIGKILL and SIGSTOP. Everything else is yours to override.

Async-signal-safety — the trap

A signal handler can fire at any instruction boundary, including while the process is in the middle of malloc, printf, or any other library call. If the handler then calls a function that takes a lock that's already held by the interrupted code, the process deadlocks. The POSIX standard lists ~30 functions that are async-signal-safe — guaranteed to be reentrant and not take locks. Everything else is unsafe to call from a signal handler.

Safe-from-signal-handler subset includes write (the syscall, not stdio), _exit, sigaction, signal-mask manipulation, and simple integer assignments. Not safe: malloc, printf, anything that locks, most of glibc. The standard idiom in production code is to do nothing in the handler except set a sig_atomic_t flag the main loop polls.

Real-time signals

Standard signals (1-31) are coalesced — if you send two SIGUSR1 to a process before it processes the first, you only get one delivery. Real-time signals (SIGRTMIN..SIGRTMAX, 32-64) are queued — each send delivers separately, and you can attach a small payload viasigqueue. Used for high-rate inter-process notifications where coalescing would lose information.

signalfd — making signals epoll-able

Signal handling interferes with the main event loop of any epoll-driven server. signalfd creates a file descriptor that becomes readable when a signal is pending; you can epoll on it like any other fd and handle signals as a normal readiness event. Modern servers (nginx, envoy, etc.) use this to avoid signal-handler entirely.

Red flags in code review

• Signal handler doing more than setting a flag.Calling stdio, malloc, anything that locks — race condition or deadlock waiting to happen. Move the work to the main loop.
• Tight loop full of syscalls. Each syscall is ~150 ns. If you're seeing 100K+ /sec, batch with epoll, io_uring, readv/writev, or hoist the work into the kernel via eBPF/AF_XDP.
• No SIGPIPE handler in a server. Writing to a client-closed socket defaults to killing your process via SIGPIPE. Ignore or use MSG_NOSIGNAL on send().
• Polling /proc/stat in a tight loop.Generates a syscall plus parsing overhead per iteration. Cache the values or use perf_event_open for actually-cheap counters.
• Custom SIGCHLD handler that doesn't reap children. Default action is "ignore but auto-reap"; if you install a handler and forget waitpid, you accumulate zombie processes until the process table fills.