x86 Architecture
x86 is the instruction set architecture underlying essentially every desktop and server processor a hobby kernel is likely to target, tracing its lineage back to the Intel 8086 in 1978 and extended, generation after generation, while retaining backward compatibility with software written for its earliest ancestors. That history is directly visible in how a modern x86-64 CPU boots: it starts execution in a 16-bit mode functionally similar to the original 8086, and software has to explicitly transition it, one step at a time, through the intervening decades of architectural additions before reaching the full 64-bit environment most kernels actually run in.
CPU modes
Section titled “CPU modes”x86 defines several operating modes, each changing what the CPU is capable of and how it interprets memory addresses.
Real mode, the CPU’s power-on state, is a direct continuation of the original 8086’s 16-bit environment: registers are 16 bits wide, memory is addressed through 16-bit segment and offset pairs combined as segment × 16 + offset, and there is no memory protection or privilege separation of any kind: any code running in real mode can access any address and execute any instruction. Firmware and a bootloader’s earliest code run here, and the transition out of it is one of the first things a kernel’s boot code performs.
Protected mode, introduced with the 80286 and substantially extended by the 80386, adds memory protection through segment descriptors, four privilege levels (rings 0 through 3), and 32-bit registers and addressing on the 80386 and later. Segments here are looked up through descriptor tables rather than shifted directly, giving the CPU a place to enforce access permissions and privilege boundaries the way real mode cannot.
Long mode, added with x86-64, extends registers and virtual addresses to 64 bits and requires paging to be active as a precondition of entering it at all; unlike every earlier mode, long mode has no non-paged variant. Long mode further splits into 64-bit mode proper and compatibility mode, the latter allowing 32-bit and 16-bit application code to run largely unmodified under a 64-bit kernel, though the kernel itself, once in long mode, always executes as 64-bit code.
Reaching long mode from power-on is not a single step: real mode transitions to protected mode by loading a Global Descriptor Table and setting a control register bit, protected mode transitions to long mode by additionally enabling paging and a long-mode-enable bit in a model-specific register, and only after both of those does a far jump into a 64-bit code segment complete the switch. Each stage depends on the one before it, which is why a kernel’s earliest boot code is often its most architecture-specific and least portable.
General-purpose registers
Section titled “General-purpose registers”The original 8086 defined eight 16-bit registers: AX, BX, CX, DX, SI, DI, BP, SP, several with instruction-specific roles beyond general storage (CX as an implicit loop counter, SI/DI as source and destination pointers for string instructions). The 80386 extended each to 32 bits by prefixing an E (EAX, EBX, and so on), and x86-64 extended them again to 64 bits with an R prefix (RAX, RBX), while also adding eight entirely new registers, R8 through R15, with no legacy name at all. Critically, writing to a narrower register name does not clear the wider register’s upper bits except in one specific case: a 32-bit write under x86-64 (to EAX, for instance) zero-extends into the full 64-bit RAX, but a 16-bit or 8-bit write leaves the untouched upper bits exactly as they were, a frequent source of bugs when code assumes a narrow write clears the whole register.
Privilege levels
Section titled “Privilege levels”Protected mode and long mode both organize execution into four privilege levels, referred to as rings, numbered 0 (most privileged) through 3 (least). A kernel runs in ring 0, from which every instruction and every memory access is permitted; ordinary applications run in ring 3, where privileged instructions fault and memory access is restricted to whatever the current page tables’ permission bits allow. Rings 1 and 2 exist in the architecture but see essentially no use on modern systems; most operating systems, including every mainstream one, use only rings 0 and 3, leaving the intermediate levels defined but unused. Moving between rings happens through controlled transition points rather than an arbitrary jump: an interrupt or exception can raise privilege to ring 0 through the IDT, and a system call instruction provides a similar, faster-path transition specifically for voluntary requests from ring 3 into ring 0.
Descriptor tables
Section titled “Descriptor tables”Both segmentation and privilege enforcement are driven by tables of descriptors the CPU consults rather than by fixed rules baked into the instruction set. The Global Descriptor Table (GDT), loaded with the LGDT instruction, defines the segments available system-wide (even under long mode’s flat memory model, where segment base and limit are effectively ignored for most purposes, a minimal GDT is still required), since segment selectors continue to carry the privilege level information a ring transition depends on. An optional Local Descriptor Table (LDT) can supplement the GDT with per-task segments, though it sees little use in practice on systems that do not otherwise rely on hardware task switching.
Instruction encoding
Section titled “Instruction encoding”x86 instructions are variable-length and, particularly once every generation’s extensions are considered, encoded through a fairly intricate scheme: an optional set of prefix bytes (altering operand size, address size, adding lock semantics, or selecting a segment override), an opcode of one to three bytes, an optional ModR/M byte selecting operand addressing modes and register operands, an optional SIB (Scale-Index-Base) byte for complex memory addressing, an optional displacement, and an optional immediate value. Under long mode, an additional optional prefix byte, REX, sits between any legacy prefixes and the opcode, providing the extra bits needed to address the eight new 64-bit registers and to select a 64-bit operand size, since the base encoding scheme predates 64-bit operands existing at all.
Implementation notes
Section titled “Implementation notes”The mode transition sequence described above is unforgiving of ordering mistakes: enabling long mode’s control bit before paging is active, or attempting the final far jump into a 64-bit segment before the GDT has a valid 64-bit code descriptor loaded, generally produces an immediate triple fault rather than a diagnosable error, since the fault-handling machinery itself depends on structures that are not yet correctly set up at that point in boot. Segment registers also carry more hidden state than their 16-bit selector value suggests: loading a segment selector causes the CPU to cache the corresponding descriptor’s base, limit, and access rights internally, and this cached information, not a fresh table lookup, is what subsequent memory accesses through that segment actually use until the selector is reloaded.
References
Section titled “References”- ^ Intel, Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 1: the architectural overview this article summarizes.
- ^ AMD, AMD64 Architecture Programmer’s Manual, Volume 1: the corresponding long-mode-focused overview.
See also
Section titled “See also”- Protected Mode: a closer look at segmentation, the GDT, and privilege enforcement.
- Long Mode: the full transition sequence into 64-bit execution.