Decode and Microcode

The previous chapters of Part V described the back end of a modern processor: pipelined, superscalar, out-of-order, with elaborate memory handling. The front end's job is to keep that back end fed with a steady stream of small, simple operations. For a clean RISC ISA with fixed-width instructions, the job is hard but tractable: fetch some bytes, identify which is which, decode them, send them on. For x86 and other CISC ISAs with variable-length, multi-operation instructions, the front end is one of the most intricate pieces of logic on the chip — a small assembly line in its own right that turns the architectural instruction stream into the internal instruction stream the back end actually executes.

This chapter is about that translation. We look at what µops (micro-operations) are and why modern processors use them, how variable-length instructions are decoded, how complex instructions are broken into µop sequences, where microcode fits in, and how front-end optimizations like the µop cache and macro-op fusion reduce the cost of decode.

01. What µops Are

Modern processors do not, internally, execute the instructions written in the ISA. They execute small simple operations called µops (also written uops, micro-ops, or RISC-style ops). The decoder's job is to translate each architectural instruction into one or more µops and feed them to the back end.

A simple instruction maps to a single µop:

(The destination physical register is named here because rename has already happened.)

A more complex instruction might map to multiple µops:

x86 has many instructions that are, internally, two or more µops. A read-modify-write memory instruction like add [rbx], rax decomposes into a load, an add, and a store:

The back end sees and schedules these µops independently. The load can issue early, the add can wait for the data, the store can hold in the SQ until retirement. The full instruction's behavior is preserved by the program-order ROB, but the internal scheduling is finer-grained than the original instruction.

For simple RISC ISAs, the µops correspond closely to the architectural instructions: each instruction maps to one µop, with little decomposition needed. For x86, the mapping can be one-to-many (a single architectural instruction generates several µops) or, in the most complex cases, hundreds-to-many (a single instruction triggers a microcoded sequence of dozens of µops).

The µop ISA is internal to the processor. It is not visible to software, not documented, and varies between micro-architectures. Two AMD processor generations and two Intel ones each have their own internal µop format, optimized for their specific back-end designs. Software sees only the architectural instructions; the decoder is the abstraction boundary.

Why µops?

Several reasons drive the µop translation.

Decoupling architecture from implementation. The back end can be designed to execute clean simple operations efficiently, without worrying about the messiness of the architectural instruction set. Adding new ISA features doesn't require redesigning the back end; just extend the decoder and (optionally) microcode.

Fixed format for the back end. µops have a fixed bit-layout, a fixed number of source and destination registers, and known latencies. The issue queue, the schedulers, and the execution units all work in the µop format.

Better OoO behavior. Decomposing complex instructions exposes their internal parallelism. The load, the add, and the store of add [rbx], rax can be scheduled independently in the OoO core, which finds more parallelism than treating the instruction as a monolithic op.

Simpler exception model. Each µop can be a separate retirement point internally, so faults are localized.

The cost is the decoder itself: it must produce µops at the rate the back end consumes them, and the variable-length and multi-µop nature of x86 instructions makes that hard.

02. Fixed-Width Decoding

For RISC ISAs (RISC-V, AArch64), decoding is comparatively simple. Every instruction is the same size — 32 bits in the standard encodings, with the option of 16-bit compressed instructions in some ISAs (RISC-V's C extension; AArch32's Thumb mode).

The fetch unit delivers a fixed number of bytes per cycle (typically 16 or 32 bytes — 4-8 instructions). A bank of parallel decoders, one per instruction position, decodes them all in parallel:

Each decoder is a fairly straightforward combinational circuit that takes a 32-bit instruction word and produces:

• The opcode (which operation).
• The source register numbers.
• The destination register number.
• Any immediate value, sign-extended.
• Control bits for the back end (issue port hints, etc.).

For most instructions, the decoder produces one µop. For a few — say, RISC-V's lr.d followed by sc.d for atomics, or AArch64's load-pair which writes two destinations — it may produce two µops.

A wrinkle for compressed instructions: RISC-V's C extension allows 16-bit instructions to mix with 32-bit ones. An aligned 16-byte fetch line might contain 4 32-bit instructions, or 8 16-bit ones, or some mixture. The decoder must determine instruction boundaries before parallel decode. This is a small extra stage that pre-classifies each 16-bit half-word as the start of a 16-bit instruction or part of a 32-bit one. The mix is much simpler than x86's full variable-length decode but still adds a small amount of front-end logic.

03. Variable-Length Decoding (x86)

x86 instructions are variable length, from 1 to 15 bytes. The boundaries between instructions are not marked in the bytes themselves: the only way to find them is to start at a known instruction boundary and walk forward, decoding each instruction's length before moving on.

This serial-by-nature problem is fundamentally awkward for a processor that wants to decode 4-6 instructions per cycle. The classic solutions:

Pre-decode bits in the cache. When a cache line is fetched, the front end runs a quick pass over it to mark instruction boundaries, storing the marks in extra bits attached to each cache line. Subsequent fetches use the pre-decoded marks to avoid re-discovering the boundaries.

Length decoders that process N bytes in parallel. Specialized circuits look at all bytes simultaneously and figure out, for each byte, whether it could be the start of an instruction, given multiple guesses at the previous boundaries. The right guess is selected once the actual previous boundary is known. This is essentially a parallel speculative scan.

Multiple decoders with different capabilities. Intel cores typically have one complex decoder that can handle any x86 instruction (including those that produce multiple µops or trigger microcode) and several simple decoders that handle only single-µop instructions. The complex decoder is on the first-instruction position; simple decoders take subsequent instructions in the fetch group.

The complete x86 front-end pipeline has multiple cycles devoted to decode. A simplified version:

Five cycles of front-end pipeline for x86, compared to perhaps two or three for a RISC ISA. This depth contributes to the misprediction penalty.

The decoders' output is buffered in a µop queue (Intel calls it the IDQ, Instruction Decode Queue) before being delivered to the rename stage. The queue smooths over decode-rate variations: if one cycle's fetch produces fewer µops than usual (because instructions are long), the queue's reserve covers the back end; if a cycle produces more µops, the queue absorbs the surplus.

04. The µop Cache

The variable-length decode is expensive in both energy and time. For code that runs in tight loops — common in real software — re-decoding the same instructions thousands of times is wasted effort. Modern Intel and AMD processors include a µop cache (Intel's DSB, Decoded Stream Buffer) that stores already-decoded µops keyed by the original instruction's PC.

The structure:

• The µop cache holds traces of decoded µops, organized by sequential instruction blocks.
• On a fetch, the cache is checked first. If the next several instructions are present, their µops are delivered directly to the µop queue, bypassing the decoders entirely.
• On a miss, the legacy decoders run; the resulting µops are also written to the µop cache for future use.

Modern µop caches deliver 6-8 µops per cycle, considerably faster than the legacy decoders (which max out at 5 µops/cycle on a typical cycle). They are also much lower power: the variable-length decode logic can be clock-gated when the µop cache is hitting.

The µop cache typically holds a few thousand µops — substantially smaller than the L1 I-cache by entry count but vastly faster on hit. Code that fits in the µop cache (small hot loops) runs faster and cooler than code that constantly re-decodes.

A subtle constraint: the µop cache's contents must be invalidated on certain conditions (page-table changes, JIT modifications to code, etc.). Most modern designs handle this transparently, but performance counters expose the µop-cache hit rate as a key metric.

05. The Loop Stream Detector

A still smaller and faster front-end structure exists for the most common case of all: tight loops whose entire body fits in the µop queue. Intel calls it the Loop Stream Detector (LSD); AMD has a similar mechanism in its op-cache region.

The LSD watches for a control-flow pattern in which a loop's body, after decode, fits entirely within the µop queue. When it detects one, it locks the relevant µops into the queue and replays them out of the queue on every iteration, rather than re-fetching, re-pre-decoding, or even re-reading from the µop cache. The branch predictor still steers iteration count; the loop body itself is delivered from the LSD.

The energy savings are substantial. The I-cache is not accessed; the pre-decode logic is gated; the legacy decoders and the µop cache are gated; only the rename and back-end stages do work. Tight numerical kernels, inner loops of memcpy and memset, the inner loops of cryptographic primitives, all benefit. Some Intel generations have aggressively expanded the LSD; others have de-emphasized it because of complications with security-related flushes or with branch-prediction interactions. The pattern is alive in current designs and should be expected in any energy-conscious x86 core.

A related idea on AArch64 is fetch-stream caching at finer granularity than the I-cache itself: small structures that hold the instructions of a hot loop body in a form pre-routed to the decoders. The Apple M-series cores reportedly use such a structure; the result is the very high decode width those cores achieve on small loops. The architectural fact is that the front end has its own multi-level cache hierarchy, parallel to the data-cache hierarchy, with the LSD as its smallest and fastest level.

06. Microcode Patching and Security Updates

The microcode ROM described above is read-only on the silicon but writable in a small patch RAM that overlays it. At boot time, the system firmware loads a microcode update file from flash or from the OS into the patch RAM; specific entries in the ROM are redirected to the patch. The mechanism predates Spectre by decades — microcode updates have always been used to fix functional errata — but the security vulnerabilities since 2018 have made it a routine part of system maintenance.

A microcode update is signed by the vendor and verified by the CPU before being applied; an unsigned update is rejected. The update format is opaque to the operating system; the OS just hands the binary to the CPU through a model-specific register write. Linux's intel-ucode and amd-ucode packages, and the equivalent Windows Update channel, deliver these blobs as part of the system's regular update flow.

The practical importance of microcode patching is that the front end is a post-silicon programmable component. Errata that would otherwise require a chip respin can be worked around by changing the µop sequence emitted for an instruction. Branch-prediction structures can be flushed at specific points by inserting new µops into security-critical instruction sequences. Performance can sometimes be improved (or, more often, slightly degraded) by tuning the schedules of microcoded instructions. The patches are not architecturally visible — software cannot tell whether an instruction is running on patched or original microcode — but they shape the performance and security profile of every machine that has been updated.

The security implication for system administrators is that microcode updates must be applied promptly to receive vendor-supplied mitigations, and that the applied microcode revision (visible in /proc/cpuinfo on Linux, wmic cpu on Windows) is the relevant version, not the silicon's original. We will return to security mitigations more broadly in Chapter 51.

07. Microcode

Some x86 instructions are too complex to decode into a static sequence of µops. Examples:

• REP MOVS — copy a string of bytes (loop with a counter).
• CALL through a memory operand with a complex addressing mode.
• DIV — large division operations.
• XSAVE / XRSTOR — save and restore the entire FPU/SIMD state.
• The transition into and out of x86's various operating modes.
• The complex side effects of RDMSR / WRMSR.

For these, the decoder cannot produce the µop sequence on its own. Instead, the decoder triggers microcode: it emits a single special µop that acts as a pointer into a microcode ROM (sometimes called the MSROM — microsequencer ROM). The microsequencer then emits the actual µops, one or several per cycle, until the operation is complete.

The microcode ROM is a small read-only memory containing pre-written µop sequences for each microcoded instruction. The sequences can be hundreds of µops long. The microsequencer steps through them, emitting µops, until reaching an end marker. During this time, the legacy decoders are stalled (the microsequencer has the bandwidth for itself).

A typical x86 implementation has:

• ~1500 to ~5000 different x86 instructions and addressing modes.
• ~50 to ~300 of those are microcoded.
• The microcode ROM is on the order of 10,000 to 50,000 µops.

Microcode is also where errata patches are delivered. When Intel or AMD discovers a bug in shipped silicon, they often distribute a microcode update — a small file of replacement µop sequences for affected instructions. The OS's loader (BIOS/UEFI or Linux's microcode driver) loads the update at boot time, patching the microcode ROM in RAM. This is one of the few ways processors can be modified after they ship; it is essential for security fixes (Spectre and Meltdown mitigations were partially delivered as microcode updates).

RISC ISAs and Microcode

RISC ISAs typically do not have microcode in the same elaborate sense. Most RISC instructions are single µops, and the decoder produces them directly. A few exceptions:

• Atomics. LR.D / SC.D on RISC-V, LDXR / STXR on AArch64, may be implemented as multiple µops with associated reservation-tracking state.
• Multi-register loads/stores. AArch64's LDP/STP produces two µops (one per register).
• Vector instructions. Long vector operations may be cracked into multiple µops.
• System instructions. The few complex system-level operations (TLB invalidation, cache maintenance) may invoke a microcoded sequence on some implementations.

The AArch64 architecture is intentionally designed to keep the µop count per instruction low; the standard expectation is that nearly every instruction produces one µop.

08. Decode-Time Optimizations

Modern decoders do more than just translate. They apply several optimizations as instructions flow through.

Macro-Op Fusion

Some pairs of consecutive instructions are common idioms that, semantically, are a single operation. The classic example: a compare followed by a conditional branch.

These are two architectural instructions, but a single conceptual operation: "branch to target if rax == rbx." The decoder can recognize the pair and emit a single fused µop that combines the compare and the branch. The back end issues, executes, and retires one µop instead of two; the front end's µop budget is conserved.

x86 cores have done this since Pentium M / Core 2 (mid-2000s). Modern Intel and AMD cores fuse:

• cmp + conditional branch.
• test + conditional branch.
• Some load-then-op patterns.
• Some shift-then-add patterns (lea-like).

The fusion is transparent to software: the architectural instructions are unchanged, but the internal µop count drops. This is one of the reasons modern x86 cores achieve high IPC on standard code: the apparent complexity at the ISA level is reduced internally.

AArch64 cores do similar fusion, less elaborately. ARM's macro-op fusion of compare-and-branch and other patterns is documented in some core implementations.

Move Elimination

A mov rax, rbx (register-to-register copy) is, internally, just a rename: the architectural register rax should now point to whatever physical register rbx points to. The back end does not need to actually execute an operation; the rename map can simply update.

Modern decoders recognize this and emit an eliminated µop that consumes no execution-port bandwidth. The rename stage handles the mapping; the back end never sees the op.

Move elimination saves execution slots, which are precious in tight code. The processor's effective IPC on register-shuffling code is higher than it would otherwise be.

Zeroing Idioms

A xor eax, eax zeros eax. The decoder recognizes this idiom, breaks the dependency on the previous value of eax, and emits a µop that simply writes zero. No actual XOR is performed. This is essential for correctness: if the previous eax was the result of a long-latency operation, the program may not want to wait for it just to zero the register.

The same applies to sub reg, reg and similar patterns. The decoder maintains a small list of zeroing idioms and special-cases them at decode time.

Constant Folding

Some decoders go further and fold constants. A mov eax, 1 followed by add eax, 2 could be fused into mov eax, 3 if both operations are simple enough. In practice this is rarely done; compilers usually fold constants at compile time, leaving little for the decoder to do at runtime. But the technique is implemented in some research designs and high-end cores for specific cases.

09. Stack Engine and Other Specialized Decoders

x86 has dedicated decode-time machinery for stack-pointer manipulation. The PUSH and POP instructions implicitly modify rsp (subtract or add 8 to it on each invocation). Decoded naively, every push or pop would consume execution-port bandwidth for the rsp adjustment.

Modern x86 cores have a stack engine (or stack pointer tracker) that keeps track of rsp's offset from its last actual update, accumulating pushes and pops without producing real µops for the rsp adjustments. When an instruction actually reads rsp (rather than implicitly modifying it via push/pop), the stack engine emits a single µop to bring the architectural rsp up to date.

The result is that a sequence of push rax; push rbx; push rcx produces three store µops but no rsp-adjustment µops; only when something later reads rsp does the adjustment µop get emitted. This dramatically reduces the µop count of function prologues and epilogues, which are dominated by push/pop sequences.

Similar specialized decode logic handles other x86 idioms: implicit operand reads in instructions like loop or rep, the special handling of segment registers in legacy 32-bit modes, the various prefix interactions.

10. Branch Identification at Decode

Branches must be identified as early as possible so the front end can redirect to the predicted target. The branch predictor, ideally, identifies branches before fetch even completes (using the BTB, indexed by PC). But sometimes the predictor misses, and the branch is only recognized at decode.

Decode-time branch identification:

• Detects branches that the BTB missed.
• Computes the direct branch target (a simple PC + offset operation).
• Redirects the front end if needed.

This is a short-distance redirect: the bubble it creates is the few cycles between fetch and decode (typically 2-4 cycles), much smaller than a misprediction's branch resolution at execute time. But it is still a bubble.

The decode-time redirect logic is a simple but useful addition. It helps the predictor's coverage gradually fill in: branches are identified, the BTB is updated, and subsequent fetches correctly anticipate them.

11. A Concrete x86 Decode Pipeline

Putting everything together, a modern Intel core's front end has roughly this structure:

Figure: Front end of a modern Intel core: branch predictor, I-cache and pre-decode, micro-op cache, simple and complex decoders, microsequencer, all feeding the IDQ

The branch predictor sits at the top, providing the next fetch address every cycle. The I-cache delivers 16 or 32 bytes; pre-decode finds boundaries; instructions are routed either to the legacy decoders or, on a hit, directly from the µop cache. Complex instructions trigger the microsequencer. All paths feed the µop queue, which delivers µops to the rename stage at up to 6-8 µops per cycle.

This is the structure of every recent Intel and AMD core. The exact sizes — number of decoders, µop cache size, µop queue size — vary by generation, but the conceptual structure has been stable for over a decade.

12. Front-End Bottlenecks

The front end can be a bottleneck in several situations.

Code that misses the µop cache. Large hot code regions that don't fit the µop cache fall back to the legacy decoders, which are slower. Compiler choices — code layout, inlining decisions, function ordering — affect µop-cache hit rates significantly.

Code with many microcoded instructions. A microcoded instruction stalls the legacy decoders while the microsequencer emits µops. If the program has many microcoded instructions back-to-back (e.g., REP MOVS for memcpy), the legacy decoders are mostly idle.

Branchy code with poor prediction. Mispredictions flush the front-end pipeline, costing many cycles. The µop queue's reserve helps absorb short flushes, but extended branch chaos overwhelms it.

Long instructions. x86 instructions can be up to 15 bytes long. A program with many long instructions has fewer instructions per cache line, fewer per fetch group, and lower decode bandwidth.

Code at the edge of cache lines. An instruction that straddles a cache-line boundary may take an extra cycle to fetch (both lines must be present). Aligning hot loops to cache-line boundaries is a common optimization.

Performance counters expose front-end stalls under various names: frontend bound in Intel's TopDown methodology, decoder stalls, µop cache misses. A program that is frontend-bound is wasting back-end capacity; the optimization target is the front end, not the algorithm.

13. RISC-V and AArch64 Decode

RISC-V and AArch64, with their fixed-width or near-fixed-width encodings, have much simpler front ends than x86. The decode pipeline is shorter (2-3 stages versus 5+ for x86), the decoders are simpler (each just decodes a 32-bit instruction), and there is no need for a µop cache (the legacy decoders are already fast and low-power).

Modern AArch64 high-performance cores still:

• Have multiple parallel decoders (typically 4-8 for performance cores).
• Apply macro-op fusion for compare-and-branch and other patterns.
• Eliminate moves at rename time.
• Recognize zeroing idioms.

But the structure is dramatically simpler than x86. The Apple M-series cores reportedly decode 8 instructions per cycle, taking advantage of the regular ISA to scale the decode width. x86 cores have struggled to exceed 4-5 instructions per cycle because of decode complexity, even with µop caches helping.

This is one of the reasons RISC ISAs are dominant in mobile and emerging in servers: the front end is much simpler at high width, which translates directly to performance and energy efficiency.

14. Summary

The front end of a modern processor turns architectural instructions into the µops the back end actually executes. For RISC ISAs, this is a relatively simple parallel decode of fixed-width instruction words; for x86, with its variable-length, multi-µop instructions, it is one of the most complex pieces of the machine, involving pre-decode for boundary identification, multiple parallel decoders, a microsequencer for complex instructions, and a µop cache that bypasses re-decode for hot code.

Decode-time optimizations — macro-op fusion, move elimination, zeroing-idiom recognition, the stack engine — reduce the effective µop count and improve throughput at no software cost. Microcode allows a small set of complex instructions to expand into long µop sequences and is also the mechanism by which post-silicon errata fixes (including security mitigations) are delivered.

Front-end bottlenecks — µop cache misses, microcoded instructions, mispredictions, long instructions — limit the rate at which the back end can be fed. On x86, the front end is often the constraining factor on peak throughput; RISC ISAs avoid much of this by virtue of their cleaner encoding.

This concludes Part V. We have walked from the abstract ISA to a complete modern micro-architecture: pipelined, superscalar, out-of-order, with sophisticated branch prediction, memory handling, and decode. Part VI takes a step back and looks at parallelism more broadly — across instructions, across data, and across threads — and at the consistency model that ties them all together.