Traps and System Calls

16 · Traps and System Calls¶

The question¶

What does the hardware do when a program asks the kernel for something?

Part III ended with a program loaded into an address space and running. It cannot do anything useful on its own — it cannot read a file, write to the console, or obtain more memory — because none of those are things a user program is permitted to do. Everything interesting requires asking the kernel, and this chapter follows one such request all the way in and back out.

The material¶

A trap is not a call¶

The instruction is ecall, and it is not a function call into the kernel, though it is easy to read it as one. They differ in nearly every respect, and the differences matter.

A function call is an agreement. The caller knows it is calling, has arranged its registers accordingly, and the convention tells both sides who preserves what — that was ch14.

ecall is the one trap a program does agree to, and it gets nothing for agreeing, because it has no path of its own. One entry point serves all four of ecall, an illegal instruction, a page fault and a timer interrupt, and three of those arrive without being asked for: the interrupted code made no arrangements and has to find every register exactly as it left it. The path has to satisfy the worst of its callers, so it has no calling convention to lean on.

So the trap path cannot save “the callee-saved registers”. It has to save everything, and put everything back. The census at the end of this chapter shows the same point in a table: one counter, entered for several unrelated reasons.

The hardware’s half¶

ecall does a small, fixed amount of work, and it is small because this is the part implemented in silicon RISC-V International (n.d.). It records the address of the instruction that trapped, records why the trap happened, disables interrupts, switches the privilege level, and jumps to an address the kernel installed earlier.

It does not save registers, switch stacks, or change the page table. The processor’s contribution is to get control to a known address with the reason available, and everything else is software’s problem. The same division holds on every architecture, even where the names differ.

The software’s half, and the page it lives in¶

The diagram leaves out a problem the code has to solve. The kernel runs with a different page table from the process, so at some point the address space must change — and the instant it changes, the code that is running must still be mapped, or the next instruction fetch faults. A page table cannot be swapped from code that is only in one of the two.

xv6 solves it the way real kernels do: one page, the trampoline, mapped at the same virtual address in every address space, kernel and user alike. The switch happens inside that page, so whichever table is active the instruction after the switch is at an address that is still valid. ch17 explains page tables properly; this is the one place in Part IV where a chapter has to promise that something later will make sense.

How long is the path?¶

Conditions: target xv6 · qemu virt (-machine virt -bios none) · QEMU emulator version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.18) · xv6-riscv @ 9e3161a · riscv64-linux-gnu-gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0 · xv6's own CFLAGS · 2026-09-17 · Source: bench/results/traps-xv6.json, code hash f78d829cb10e7c2c.

Those counts are the state movement in full: the way in — uservec, the assembly in the trampoline page — saves thirty-one registers, the way out — userret — restores them, and neither is doing anything you asked for. Before usertrap, the C the assembly hands over to, has looked at why it was entered — before any argument has been examined or any work begun — the machine has executed over forty instructions of pure bookkeeping, and it will execute nearly as many again on the way out, after the work is finished.

uservec also does a handful of CSR operations, and one of them is the page-table switch. userret does fewer, because sret — supervisor mode’s mret — puts the privilege level and the interrupt state back in one instruction.

This is a count and not a cost, which is why it appears here rather than in Part V. What an instruction costs depends on a pipeline and a cache, and this target has neither. The path is this long on any machine. ch29 takes the same shape to hardware and puts a price on it.

Counting what actually happens¶

The kernel has been patched to count every trap by cause and print the table when you press Ctrl-T, the way it already prints the process table on Ctrl-P:

+static void
+census_note(uint64 scause, int user)
+{
+  uint64 code = scause & 0xf;
+  if (scause >> 63)
+    census.interrupt[code]++;
+  else if ((scause & ~(uint64)0xf) == 0)
+    census.exception[code]++;
+  else
+    census.unclassified++;
+
+  if (user)
+    census.from_user++;
+  else
+    census.from_kernel++;
+}
+
+// Called from syscall() once the number is known, so the census can say *which* calls a workload
+// made rather than only how many. The per-number counts are what the traps-and-system-calls chapter can reproduce exactly: a
+// program that asks for one thing a thousand times is counted a thousand times, whatever the
+// shell around it happened to do.
+void
+census_syscall(int number)
+{
+  if (number >= 0 && number < NSYSCALL)
+    census.syscall[number]++;
+}
+

Then run a workload and ask:

Conditions: target xv6 · qemu virt (-machine virt -bios none) · QEMU emulator version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.18) · xv6-riscv @ 9e3161a · riscv64-linux-gnu-gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0 · xv6's own CFLAGS · 2026-09-17 · Source: bench/results/traps-xv6.json, code hash f78d829cb10e7c2c.

Most of that table is a list rather than a count, because those counts were not reproducible — and the reason is worth more than the numbers would have been.

The obvious thing to measure was the shell: boot, run ls, and report how many system calls it took. That produced a satisfyingly large number — and a different one each time. How many times the shell calls read depends on how the console delivered its characters, which depends on timing, and elapsed time inside QEMU is a property of the laptop running it. Two runs of an identical workload disagreed, which is how this was found out rather than assumed.

An exception — a trap an instruction caused, as opposed to an interrupt — is reproducible for a fixed sequence of instructions, because the same instructions cause the same traps. The trouble is that “run the shell” is not a fixed sequence of instructions. So the workload is a program that removes the question:

int main(void) {
  int last = 0;
  for (int i = 0; i < SYSFS_TRAPLOAD_CALLS; i++) {
    last = getpid();
  }
  printf("trapload %d calls pid %d\n", SYSFS_TRAPLOAD_CALLS, last);
  exit(0);
}

Build it into the kernel, boot, and run it:

./run trapload

Nothing else in the system calls getpid, so that row of the census is a number this program decided and the shell’s noise lands elsewhere. It asks a thousand times and the kernel counts a thousand — which is a much smaller claim than the one that was almost printed, and unlike it, true on every machine.

This is where ch00’s discipline costs something. The large number was easy, impressive, and meaningless. ch29 counts the lot, on a machine where elapsed time is a fact about the machine.

The register census, and why it is not thirty-two¶

Thirty-one registers are saved. RV64 has thirty-two. The missing one is not an oversight and working out which it is — and why a trap path is allowed to ignore it — is one of this chapter’s problems.

What we measured¶

A path length, read out of the built kernel, and a trap census, read out of a running one. Neither is a duration and this target could not produce one honestly.

The census is reproducible because the workload is fixed and because only the deterministic half of it is recorded. CI re-runs it on every push and compares. A kernel patch that lost some of the workload’s thousand calls, or that introduced a cause this list does not have, would move the table — and moving it silently is exactly the failure the whole stamping scheme exists to prevent.

The check deliberately cannot catch a change in how often the shell asks the kernel for anything, because that was never recorded. Giving up that number bought the rest of the table its reproducibility, for the reasons the section above gives.

What this cannot tell you¶

What any of this costs. The path is eighty-odd instructions long; whether that is expensive depends on whether they hit in cache, whether the pipeline drains, and what the page-table switch does to the TLB (the hardware cache of address translations) — three questions this target has no opinion about whatsoever. It is entirely possible for the shorter of two paths to be the slower one, and ch29 is where that gets settled.

What a real kernel’s path looks like. xv6’s is short because xv6 is small. Linux’s does considerably more on the way in — checking for signals, handling seccomp filters, auditing — and its fast paths exist precisely to avoid some of it. The shape is the same; the length is not.

Why the timer fires when it does. The census shows timer interrupts occurred and nothing else about them, because their frequency here is an artefact of emulation. ch21 explains what the kernel does with them.

Anything about interrupts arriving during a trap. ecall disables them and usertrap turns them back on deliberately, at a point chosen for a reason the code comments explain. Nested traps, and what happens when one arrives at the wrong moment, are ch19’s subject.

Problems¶

Two, and the first is the one every xv6 reader should do once.

16.1 — Add a system call, end to end. The census is printed on Ctrl-T, which is no use to a program. Add trapcount() and a user program that calls it.

Five files have to agree before this works, and finding all five is the exercise: the number that identifies the call, the table that dispatches on it, the declaration the user program compiles against, the stub that issues ecall, and the kernel function itself. Leaving any one out produces an error that does not mention the others.

The test cannot check that your number is correct without reimplementing your counter, so it checks the two things that separate a working system call from a plausible stub: the value is not zero, and it grows between two runs. No constant passes both.

python3 -m pytest tests/traps_and_system_calls/test_problem_1_addcall.py

16.2 — Which registers must be saved, and which need not be? Count them, then say which ones uservec leaves alone and why each is allowed. There is more than one reason a register can end up on that list, and this chapter has given you two of them.

The test reads the count out of the kernel as built, so if you patch the trampoline you are graded against your own kernel rather than against the book’s.

python3 -m pytest tests/traps_and_system_calls/test_problem_2_registers.py

Where to go next¶

The RISC-V privileged specification RISC-V International (n.d.) defines exactly what ecall and sret do and what each CSR in the path is for. The section on trap handling is short, and reading it alongside trampoline.S is the fastest way to work out which half of the path is hardware.

xv6’s kernel/trampoline.S and kernel/trap.c Cox et al. (n.d.) are now readable in full — the assembly is the path this chapter counted, and usertrap is a short piece of C. Read the assembly first and the C second; the order matters, because the C makes no sense until you know what state it has been handed.

ch17 takes the promise this chapter made — that a page table is a thing, that the trampoline is mapped into two of them — and turns it into something you can print.