VirtIO and the Virtqueue

Every virtio device (network, block, GPU, and others) shares a single underlying structure regardless of what it actually does: the virtqueue, a ring of descriptors set up once and reused for every subsequent transfer between guest and host. PCI already identifies 0x1AF4 as the vendor ID QEMU and most hypervisors use for this device family; this article covers the virtqueue itself, the mechanism every specific virtio device (virtio-net, virtio-blk, and so on) is ultimately built from.

The descriptor table

A virtqueue’s core is a fixed-size array of descriptors, each one describing a single buffer: its guest-physical address, its length, a flags field, and, when a request spans more than one buffer, a next index chaining it to another descriptor in the same table.

struct virtq_desc {
    uint64_t addr;    // guest-physical address of the buffer
    uint32_t len;
    uint16_t flags;   // VIRTQ_DESC_F_NEXT, F_WRITE, F_INDIRECT
    uint16_t next;    // next descriptor index, if F_NEXT is set
};

A descriptor by itself is just a buffer description; nothing in the table alone says which descriptors currently represent a real, pending request versus which are simply unused. That bookkeeping is what the two rings surrounding the descriptor table exist to provide.

The available and used rings

The available ring is written by the guest and read by the host: it lists, in order, the index of the first descriptor (or the head of a chained descriptor list) for every request the guest has prepared and wants the host to process. The used ring runs in the opposite direction, written by the host and read by the guest, listing which descriptor chains the host has finished processing and how many bytes it actually wrote back for each, so the guest knows a given request completed without needing to poll every descriptor’s own state individually.

struct virtq_avail {
    uint16_t flags;
    uint16_t idx;          // next slot the guest will write
    uint16_t ring[];       // descriptor head indices
};

struct virtq_used_elem {
    uint32_t id;            // descriptor chain head that completed
    uint32_t len;            // bytes the host actually wrote
};

struct virtq_used {
    uint16_t flags;
    uint16_t idx;           // next slot the host will write
    struct virtq_used_elem ring[];
};

Submitting a request from the guest side means filling in one or more descriptors, writing the head descriptor’s index into the next free avail.ring slot, incrementing avail.idx, and then notifying the device (commonly through a dedicated MMIO or port write) that new work is available; the host, independently, watches used.idx advance to know a previously submitted request has actually completed and it’s safe to read whatever result the corresponding descriptor’s buffer now holds. Both rings are append-only, single-producer structures, wrapping around their fixed size rather than growing, which is what keeps updating either one a fast, lock-free operation on both the guest and host side.

Why this is simpler than emulating real hardware

A device emulator faithfully reproducing physical NIC or disk-controller register semantics (interrupt coalescing timers, DMA descriptor formats inherited from decades of real silicon, vendor-specific quirks) has to replicate all of that complexity in software just to be indistinguishable from the real chip it’s pretending to be. Virtio sidesteps this entirely by not pretending to be any physical device at all: the virtqueue format is designed from the outset to be efficient to implement on both sides of a virtual machine boundary, not to match a particular chip’s register layout, which is what makes a virtio-net or virtio-blk driver considerably simpler to write correctly than an equivalent driver for hardware emulated with full physical fidelity, and part of why virtio-net specifically is commonly the first network driver a hobby kernel implements, a subject its own dedicated article covers from the network side rather than the virtqueue side this one focuses on.

Implementation notes

A device can expose more than one virtqueue at once (virtio-net conventionally uses separate queues for transmit and receive, for instance), each independently sized and indexed, discovered and configured through the device’s own configuration space rather than a single shared queue serving every purpose. The VIRTQ_DESC_F_INDIRECT flag lets a single descriptor point at a further table of descriptors elsewhere in memory rather than chaining through the main table directly, useful for a request that would otherwise need more chained descriptors than comfortably fit given the queue’s fixed size. Because the guest and host observe each other’s ring updates without a full memory barrier automatically enforced by the transport itself in every configuration, a driver has to follow the specification’s documented ordering rules (writing a descriptor’s contents before publishing its index in the available ring, for instance) rather than assuming ordinary in-order execution is sufficient on every platform virtio might run on.

References

^ OASIS, Virtual I/O Device (VIRTIO) Specification: the formal specification defining the virtqueue layout and the available/used ring protocol described above.

VirtIO and the Virtqueue

The descriptor table

The available and used rings

Why this is simpler than emulating real hardware

Implementation notes

References

See also