Skip to main content

Linux System Calls: The User-Kernel Interface

Linux system calls visualized: how user programs communicate with the kernel, protection rings, argument registers, error handling, and syscall performance.

The Gateway to the Kernel

A file descriptor is useful because the kernel controls what it refers to. Your program can ask to read from it, but cannot directly run filesystem code with kernel privileges. A system call is the controlled entry point for that request.

The key distinction is between changing privilege and changing tasks. During a system call, the calling thread can execute kernel code and then return to user code. The scheduler may switch tasks if the operation blocks, but crossing the boundary does not require a task switch.

This page follows the native Linux x86-64 interface. Other architectures use different instructions, registers and syscall numbers. If the separation between application and kernel is new, start with kernel architecture.

Interactive System Call Visualization

Start with read(). Step forward to see the CPU enter kernel mode, then Linux switch the stack. Follow the request through its handler and back to the application. Choose Error to see where the request stops and how the raw kernel result becomes errno.

Cross the user–kernel boundary

Step 1 / 12

Start in the application

Task 1234 requests read(3, buffer, 1024). It is still executing user code.

User space · ring 3
Kernel space · ring 0
Task 1234

Executing in user mode

Stack: user

User buffer

Kernel file data

buffer

Unchanged

fd 3 → file

Awaiting read

Follow the bytes from the file into the application’s buffer.

Return value

Waiting for the handler

Step details and navigation

Request at entry · saved ABI values

rax · call number
—
rdi · file descriptor
—
rsi · buffer pointer
—
rdx · byte limit
—
  • A buffered file read. This successful example returns 1024 bytes; a real read can return fewer bytes or zero at EOF.
  • Addresses and process IDs here are illustrative. Step duration does not represent measured latency.
Native assembly and scope

A buffered file read. This successful example returns 1024 bytes; a real read can return fewer bytes or zero at EOF.

read.asm · ABI excerpt
mov eax, 0           ; SYS_read
mov edi, 3           ; file descriptor
mov rsi, buffer      ; writable user buffer
mov edx, 1024        ; maximum byte count
syscall              ; raw result in rax

Assembly excerpt; pointers refer to storage supplied by the caller. Raw assembly returns negative error numbers. The final step also shows how a libc-style wrapper reports that result.

Illustrated Linux x86-64 ABI. Steps, addresses and PIDs are examples; libc may use different syscalls.

Understanding System Calls

What Are System Calls?

System calls expose kernel services through a defined ABI: a syscall number, arguments, and a return value. Programs usually reach that interface through a C library function.

ServiceRepresentative kernel interfaces
Filesopenat, read, write, close
Processesclone, fork, execve, wait4
Memorymmap, brk, munmap
Networkingsocket, connect, sendto, recvfrom
Devicesioctl
Signalskill, rt_sigaction

A familiar function name does not guarantee an identical kernel entry. For example, modern glibc implements open() using openat() and fork() using clone(). The demo uses the native open and fork entries to make their ABI visible. open(2), fork(2).

Once resources are established, some work can proceed without a fresh system call: accessing an existing memory mapping or reading time through the vDSO are examples.

The System Call Table

A syscall number selects a kernel operation. For the native x86-64 ABI, read is 0, write is 1, open is 2, fork is 57, and execve is 59. These are ABI assignments, not universal Linux numbers. Linux x86-64 syscall definitions.

Think of this as a number-to-handler mapping. The dispatch implementation can change: current x86-64 Linux generates a switch in x64_sys_call(). Its sys_call_table is retained for tracing information rather than used directly for dispatch. Linux dispatch source.

CPU Protection Rings

Ring Architecture

For ordinary x86-64 Linux applications, user code runs at ring 3 and kernel code runs at ring 0. Rings 1 and 2 are not used for the usual application/kernel split.

  • User mode: page permissions restrict accessible memory, and privileged instructions cannot be used freely.
  • Kernel mode: privileged code manages page tables, interrupts and devices. Hardware permissions and protection features still govern memory access; ring 0 does not make every address valid.

The boundary is about authority. A pointer supplied by an application is still untrusted when a kernel handler receives it.

Mode Transitions

On native x86-64, the SYSCALL instruction saves the return instruction pointer in rcx and flags in r11, enters ring 0, and jumps to the configured kernel entry address. It leaves rsp unchanged. Linux entry code then saves the user stack pointer, selects the kernel stack and saves context.

On return, Linux handles exit work and restores the user context, including the user stack pointer. It can then use SYSRET for the fast path, or IRETQ when needed. SYSRET does not independently reload a saved user stack pointer. Linux entry and return code.

System Call Execution Flow

Step-by-Step: read(fd, buffer, count)

The demo separates the handoffs that are easy to conflate:

  1. Application → ABI: prepare the syscall number and arguments.
  2. CPU → Linux entry: enter kernel mode; Linux establishes the kernel stack and saved context.
  3. Entry → handler: apply entry checks and dispatch the request.
  4. Handler → file: look up the descriptor and obtain data through the file's operations.
  5. Kernel → user buffer: copy available bytes and prepare the result.
  6. Exit → application: restore the user context, return to user mode, then let the library report the result.

A buffered read can obtain data from the page cache. It does not always cause disk I/O. A successful read() returns the number of bytes actually read, which may be less than requested; zero means EOF for an ordinary file read. read(2).

Parameter Passing

x86-64 Calling Convention

The native syscall ABI differs from the ordinary C function-call ABI, notably in the fourth argument:

RegisterMeaning at syscall entry
raxSyscall number
rdiArgument 1
rsiArgument 2
rdxArgument 3
r10Argument 4
r8Argument 5
r9Argument 6

The return value arrives in rax; rcx and r11 are clobbered by the transition. The native interface has at most six argument registers. Interfaces needing larger input commonly pass a pointer to a structure; they do not add arbitrary syscall arguments on the user stack. syscall(2).

Example: open(filename, flags, mode)

This is an assembly excerpt for the native SYS_open entry. filename must refer to a valid NUL-terminated string. It does not describe the implementation of modern glibc open().

nasm
mov eax, 2                 ; native SYS_open
lea rdi, [rel filename]     ; pathname pointer
xor esi, esi               ; O_RDONLY
xor edx, edx               ; mode unused without O_CREAT
syscall
; rax: descriptor on success, negative error on failure

The pathname pointer remains an input. The newly allocated descriptor is a separate result; it does not replace that pointer in the ABI argument list.

Security and Validation

Why Validation is Critical

The kernel receives requests from programs with different credentials and potentially invalid inputs. It must check both whether an operation is allowed and whether its inputs can be accessed safely.

Validation Checks

  • Descriptors and flags: does the handle exist, and does the operation make sense for it?
  • Credentials and limits: do permissions, capabilities and resource limits allow the request?
  • Sizes and ranges: can lengths and offsets be handled without overflow?
  • User memory: can the relevant bytes actually be read or written through the kernel's user-access helpers?

An address-range check alone does not prove that memory is mapped or accessible. User access may fault; the kernel's access paths must handle that safely. Depending on the failure, a handler can return errors such as EFAULT, EINVAL, EACCES or EBADF.

Tracing System Calls

strace - System Call Tracer

Use strace to observe the kernel interface a program actually uses. Include openat when examining modern file-opening code. strace(1).

bash
# Trace a program's kernel calls
strace ls

# Focus on file opening, reading and closing
strace -e trace=open,openat,read,close cat /etc/hosts

# Show time spent in each observed call
strace -T ls

# Summarize calls and errors
strace -c ls

# Attach to a process, if permitted
strace -p 1234

# Save the trace
strace -o trace.log ls

Illustrative output, not a captured run:

text
openat(AT_FDCWD, "/etc/hosts", O_RDONLY) = 3
read(3, "127.0.0.1 localhost\n", 4096)   = 20
read(3, "", 4096)                        = 0
close(3)                                = 0

The library call and the kernel call can have different names. That distinction also explains why a library-call tracer can show a different view.

ltrace - Library Call Tracer

bash
# Observe supported dynamically linked library calls
ltrace ls

# Include system calls as well
ltrace -S ls

Coverage depends on linking, optimization and the tracing tool. Neither trace should be treated as a complete view of every internal function.

Performance Considerations

Syscall Overhead

Crossing the boundary adds work: entry and exit processing, checks, and the service itself. The cost depends on the CPU, kernel configuration, mitigations, workload and whether the call blocks. There is no single cycle count that describes all system calls.

Use tracing to find repeated operations and errors, then measure the actual workload. Tracing itself adds overhead, so a traced timing is not an uninstrumented benchmark.

vDSO: Virtual Dynamic Shared Object

Linux maps a small shared library into user processes. For supported operations, libc can use it to obtain information such as time without entering the kernel on every call. Availability depends on the architecture and requested operation; a library can still fall back to a syscall. vdso(7).

bash
# Show whether this process has a vDSO mapping
cat /proc/self/maps | grep '\[vdso\]'

A mapping proves availability, not that a particular call used it. Calls handled entirely through the vDSO do not appear as kernel entries in strace.

Batching Syscalls

Larger buffers can reduce the number of boundary crossings, but a large request does not guarantee a full buffer. This C excerpt consumes every successful read and handles interruption:

c
char buffer[64 * 1024];

for (;;) {
    ssize_t n = read(fd, buffer, sizeof buffer);
    if (n > 0) {
        consume(buffer, (size_t)n);  // application-defined processing
    } else if (n == 0) {
        break;                      // end of file
    } else if (errno != EINTR) {
        perror("read");
        break;
    }
    // EINTR: retry; short reads: process n bytes, then continue.
}

This assumes a blocking descriptor and the usual C/POSIX headers. readv() and writev() gather multiple buffers into one request. Their return values still need checking.

Common System Calls

File I/O

openat establishes a descriptor; read and write transfer bytes; lseek changes a seekable file's position; fstat retrieves metadata; close releases a descriptor. A library may use another compatible kernel entry for a particular API.

Process Management

fork and clone create execution contexts, execve replaces the current process image, wait4 collects child status, and kill sends a signal subject to permission checks. The raw exit syscall terminates a thread; library-level process termination has additional semantics. exit(2).

Memory Management

mmap creates a mapping, munmap removes one, mprotect changes protection, and brk adjusts the program break. An allocator such as malloc() manages blocks in user space and only sometimes needs a kernel request.

Network

socket, bind, listen, accept and connect establish communication endpoints. Data can travel through read/write, sendto/recvfrom, or message-oriented interfaces. Library APIs such as send() and recv() may map to different kernel entries by architecture.

Error Handling

Error Codes

At the raw x86-64 boundary, Linux uses the range −4095 through −1 for error results. A libc wrapper normally reports these as -1 and sets the calling thread's errno. glibc x86-64 error handling. Common errors include ENOENT (missing path), EACCES (permission denied), EINVAL (invalid argument), ENOMEM (insufficient memory), EFAULT (bad user address) and EBADF (invalid descriptor).

Check the documented return value before inspecting errno; successful calls need not clear it. For read():

c
ssize_t n = read(fd, buffer, sizeof buffer);
if (n == -1) {
    perror("read");
} else {
    consume(buffer, (size_t)n);  // n can be zero or a short read
}

The demo's Error option shows both representations: the raw negative error first, then -1 and errno after user-space wrapper handling. syscall(2).

Best Practices

  1. Check results: handle errors, partial transfers and EOF according to each interface's contract.
  2. Buffer where useful: reduce repeated small requests without assuming one call completes all work.
  3. Use library interfaces: they handle architecture details and often provide buffering or vDSO access.
  4. Measure the workload: use tracing to locate behavior, then benchmark with a clear measurement boundary.
  5. Choose concurrency deliberately: blocking, nonblocking and asynchronous interfaces solve different scheduling needs.

Modern Alternatives

io_uring (Linux 5.1+)

io_uring shares submission and completion queues between an application and the kernel. Multiple operations can be submitted together, and the application collects their completions later. It changes how requests are exchanged; it does not remove the kernel's responsibility to validate and perform them.

Setup and normal submission or waiting can still require system calls. Submission polling can avoid some entries under particular configurations, with wakeup and CPU-use tradeoffs. “Zero syscalls” is not a universal property of the interface. io_uring(7).

The same idea connects back to the walkthrough: decide what must cross the boundary, how often it crosses, and where completion becomes visible to the application.

If you found this explanation helpful, consider sharing it with others.

Mastodon