NVMe
NVMe (Non-Volatile Memory Express) is the storage protocol that replaced SATA/AHCI as the default for modern SSDs, built natively around PCIe rather than emulating the ATA register semantics AHCI still exposes underneath. Where AHCI organizes work around a fixed number of ports and command slots per port, NVMe is built around queue pairs, an entirely different command model designed from the outset for the parallelism flash storage can actually sustain.
The controller register set
Section titled “The controller register set”Locating an NVMe controller through PCI works exactly like locating any other device: read its BAR0, which is always memory-mapped for NVMe (there is no I/O-space alternative the way some other device classes offer), and map it. The registers found there include the controller’s capabilities, a Controller Configuration register used to enable it, a Controller Status register to check readiness, and the doorbell registers queue pairs use to signal new work.
struct nvme_regs { uint64_t cap; // capabilities uint32_t vs; // version uint32_t intms; // interrupt mask set uint32_t intmc; // interrupt mask clear uint32_t cc; // controller configuration uint32_t rsvd; uint32_t csts; // controller status uint32_t nssr; uint32_t aqa; // admin queue attributes uint64_t asq; // admin submission queue base address uint64_t acq; // admin completion queue base address // ... doorbell registers follow, spacing set by CAP.DSTRD ...};Submission and completion queues
Section titled “Submission and completion queues”Every operation, from the very first Identify command to an ordinary read or write, goes through a queue pair: a submission queue (SQ) the driver writes commands into, and a completion queue (CQ) the controller writes results into, each a simple circular buffer in host memory rather than a fixed set of hardware slots. The very first queue pair, the Admin Queue, is set up directly through the ASQ/ACQ/AQA registers before anything else can happen, and is used exclusively for controller management commands (Identify among them); ordinary data I/O goes through separate I/O queue pairs, created afterward via admin commands, and a driver can create many of them, commonly one pair per CPU core, to submit and complete commands from multiple cores in parallel with no shared-lock contention between them.
struct nvme_command { uint8_t opcode; uint8_t flags; uint16_t command_id; uint32_t nsid; // namespace ID (roughly, which "drive") uint64_t rsvd2[2]; uint64_t metadata; uint64_t prp1, prp2; // physical addresses of the data buffer uint32_t cdw10, cdw11, cdw12, cdw13, cdw14, cdw15; // opcode-specific};Submitting a command means writing it into the next free slot of the submission queue and then writing the new tail index to that queue’s doorbell register, a memory-mapped write the controller polls or is otherwise made aware of, notifying it that new work is ready without any register-by-register command sequence the way AHCI’s port registers require. The controller processes the command asynchronously and writes a completion entry into the corresponding completion queue, toggling a phase bit the driver checks to distinguish a genuinely new completion from stale data left over in a slot from a previous pass around the ring, then advances the completion queue’s own doorbell to acknowledge it.
Identify
Section titled “Identify”The Identify command (opcode 0x06) is the first real command issued after the admin queue is set up, and, similar to ATA’s own IDENTIFY in spirit though structurally unrelated, returns a large data structure describing the controller’s capabilities and, with a different sub-command value, a specific namespace’s geometry: its size in logical blocks, its logical block size, and various optional feature flags.
struct nvme_command identify_cmd = { .opcode = 0x06, .cdw10 = 1, // CNS = 1: identify the controller itself (0 = identify a namespace) .prp1 = data_buffer_phys_addr,};A namespace is NVMe’s rough equivalent of a “drive” from AHCI’s perspective, though the mapping isn’t always one-to-one: a single physical NVMe SSD can expose multiple namespaces, each independently addressable and sized, a capability with no direct parallel in AHCI’s simpler one-controller-per-physical-drive assumption.
Implementation notes
Section titled “Implementation notes”Because both submission and completion queues are ordinary host-memory buffers the controller reads and writes via DMA rather than fixed hardware structures, a driver has to allocate them as physically contiguous, properly aligned memory and never move them once registered with the controller, unlike AHCI’s fixed per-port command list whose location can be reprogrammed more freely between operations. The phase-bit mechanism specifically exists to let a completion queue be read purely by polling its memory contents, with interrupts as an optional optimization layered on top rather than a requirement, which is part of what makes NVMe’s completion path cheaper at high queue depths than a purely interrupt-driven design would be.
References
Section titled “References”- ^ NVM Express, Inc., NVM Express Base Specification: the formal specification defining the register set, queue model, and command set described above.