Skip to content

필사 모드: A Single Instruction Can Take 62 Seconds — Latency Is a Property of the Path, Not of the Instruction

English
0%
정확도 0%
💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction — between 1 cycle and 198 billion cycles

Instruction latency tables are a basic tool for anyone doing optimization. Multiply is 3 cycles, divide is 20 cycles; you memorize that sort of thing and write your code.

The Assembly Hall of Shame, published in August 2026, goes in the opposite direction. Its subtitle is "Racing to the bottom of CPU performance." It is a leaderboard for the competition to make one single instruction as slow as possible.

The bottom entry, 27th place, is nop at 1 cycle. First place is fxrstor64 at 198 billion cycles, which in time is 62 seconds.

When there is a 200-billion-fold difference between things of the same kind, you should start by doubting the premise that you are measuring that kind on a single scale. This post reads the leaderboard from the bottom up, looking at what creates that difference and at what it means for your code and your benchmarks.

What the rules created — what counts as "one instruction"

In a competition like this, the rules are the thesis. The rules written in the repository are these.

  • Anything may be used in the setup, but only a single instruction is scored.
  • For instructions that trap, are emulated, or are virtualized, timing goes only up to the trap, not through the handler.
  • The instruction must not be interruptible. Things like rep movs or pause are disqualified.
  • Times are normalized against the CPU base clock.
  • All platforms must be at factory settings, and hardware modification is forbidden.

The third rule is what makes this competition meaningful. Allow interruptible instructions and you can manufacture any number you like with a repeat-prefixed string instruction. Requiring non-interruptibility means you count only the time the pipeline is genuinely tied up on that one instruction.

The second rule works the same way. Allow trapping instructions and what you are measuring is no longer the instruction but the operating system handler.

Thanks to those two rules the leaderboard becomes not a stunt but a record of microarchitectural observations.

The first band — where microcode steps in inside the core

The lower ranks are all things that happen inside the core. Most measurements were taken on an Intel Core i7-8559U.

RankInstructionCyclesWhat happens
27nop1nothing at all
26nop1620a long nop with seven data16 prefixes attached
25rdtsc49the reference point
24idiv77a 128-bit dividend burns the longest path through the divider
23enter112maximum nesting depth triggers a microcode display walk
22fldl133loading a denormal enters the FP microcode assist
20fsin257the special-value path for exponent 0x7ff
17fadd677FP microcode assist from a denormal operand
15fdiv883a denormal divisor
14cpuid1,248the leaf with the largest latency is selected

The common pattern in this band is visible. Give the hardware fast path an input it cannot handle and control passes to microcode.

The fadd entry is especially instructive. It is the same instruction, and yet purely because the operand is a denormal it takes 677 cycles. A normal floating-point addition is a handful of cycles. The instruction is unchanged and only the data changed, and a two-order-of-magnitude multiplier appeared.

The enter entry is fun too. Give it the maximum nesting depth of 31 and you take the microcode path that loads and pushes 30 display pointers. It is permitted by the instruction encoding but is a form no modern compiler would ever emit.

The second band — where coherency and the platform get involved

From the middle band onward, one core is no longer the whole story.

RankInstructionCyclesWhat happens
21clflush165eviction of a dirty line
19mfence326saturate the write-combining buffers with 16 movnti and then drain them all
18mov cr3352a full TLB invalidation
16split lock865a lock xaddl straddling a cache line forces an external bus lock
13rdrand5,579waiting for recovery after exhausting the hardware entropy pool
12wrmsr34,304writing MCG_CTL on Zen; presumed to synchronize units off the die
11out49,857a port write straddling a NIC register boundary stalls TX DMA
9wbinvd1,616,480fill the whole cache hierarchy with dirty data, then write back to DRAM

Split lock at 16th place is the entry you meet most often in practice. When the operand of an atomic operation straddles a cache line boundary, the CPU cannot use the fast MESI cache coherency path and has to assert an external bus lock. It takes 865 cycles, and other cores are affected in the meantime.

That wbinvd at 9th place is 1.6 million cycles is also worth noting. This is not because the instruction is complicated but because every piece of dirty data sitting in the cache has to be pushed out to DRAM. In other words, the cost of this instruction is decided not by the instruction but by the state of the cache at that moment.

The conjecture in the 12th-place wrmsr entry is worth quoting as well. The repository writes that this appears to be microcode halting and synchronizing machine check banks spread across multiple hardware units, some of which are off the die, so that it seems to require fabric-level communication rather than a simple local register write. The author states outright that this is a conjecture, so I carry it over as such.

The third band — the moment you leave the die

The upper ranks are of a completely different character.

RankInstructionCyclesTime
8inl (ACPI PM port)12,524,4153.92 milliseconds
7movl (MMIO GPU register)443,937,696139 milliseconds
6movq (8-byte MMIO)887,716,864278 milliseconds
5vmovdqu xmm (16 bytes)1,774,555,776556 milliseconds
4vmovdqu ymm (32 bytes)3,549,079,2961.111 seconds
3vmovdqu ymm (unaligned 32 bytes)4,453,212,2561.394 seconds

Everything here is a mov. An instruction that does nothing except move data takes more than a second.

The reason is in the address. These addresses are not DRAM but device registers beyond the PCIe fabric. The author used a separate tool called mmiotic to search MMIO space for the regions with the slowest responses.

And the pattern of increase from 6th to 3rd place lays bare the principle of this band. Widen the access from 8 bytes to 16 bytes to 32 bytes and the time roughly doubles each step. According to the repository description, that is because a single 8-byte MMIO read decomposes into two doubleword register accesses. Thirty-two bytes means eight, and misaligning it means nine.

The key point here is this. An MMIO read is a non-posted transaction. A write can be fired off and forgotten, but a read has to receive a response. So the round-trip time becomes the execution time of the instruction, directly. For 3rd place the repository notes that an unaligned 32-byte MMIO read is "technically not allowed but works anyway."

What the strategy at the top proves

Now for first place.

; CPU 0 - the instruction under measurement
movl $0xfcc68830, %rsi
fxrstor64 %rsi

; CPU 1..N - hammer loop pounding a different high-latency location
movl 0xfcc68858, %eax

fxrstor64 is the instruction that restores 512 bytes of FPU/MMX/XMM state from memory. Make it read those 512 bytes from the slowest MMIO region and you get an extension of the 3rd-place technique above. That alone yields 74,584,168,512 cycles, or 23.35 seconds.

First place adds one more layer. While that load is in flight, the remaining cores keep hammering a different high-latency MMIO register in 4-byte units. The PCIe root complex and the endpoint saturate with non-posted transactions, and the 512-byte load on CPU 0 has to queue behind them.

The result is 198,002,498,236 cycles, 62 seconds. In other words, the execution time of a single instruction changed by more than a factor of two depending on what the other cores were doing.

The proposition this experiment demonstrates is clear. The question "how many cycles is this instruction" has no answer unless you fix the surrounding state. The number written in a table is not a property of the instruction but an observation about the combination of the instruction and the state of the system.

Incidentally, this technique also produced a practical result. The repository writes that the unaligned ymm load at 3rd place was used to break the fundamental design of System Management Mode, and links a separate repository. If an interrupt can be stretched arbitrarily long, designs that assume atomicity break.

What this table says about your benchmarks

The leaderboard looks like extreme play, but the conclusions apply directly to ordinary performance work.

A microbenchmark measures system state, not an instruction. The same code produces entirely different numbers with a warm cache versus a cold one, with other cores idle versus busy, with normal data versus denormal data. This is why you must write down the conditions alongside any benchmark result you quote.

The worst case is not a multiple of the average. The bands in the table above are not continuous; they are separated by cliffs. However good your average latency, stepping on a single cliff makes that one request a hundred times slower. When you are dealing with tail latency, hunting down and removing cliffs is more effective than improving the average.

Even an identical instruction stream is affected by activity on other cores. The first-place case is extreme, but the principle is the same. Shared resources run from the last level cache through the memory controller and the interconnect all the way out to the I/O fabric. This is why a benchmark run in isolation fails to predict production performance.

Three things you will actually trip over

Finally, three entries from this table that touch working practice directly.

Denormals. Ranks 22, 17, and 15 are all this. It really shows up in code where values converge slowly toward zero, such as signal processing or physics simulation. The symptom is "the same code suddenly becomes slower depending on the input data." How to respond differs by hardware and by language — the flush-to-zero and denormals-are-zero modes on x86 are the representative case — so you have to check your own platform documentation. What matters here is that you can only find the cause if you know this cliff exists.

Split lock. Rank 16 in the table. The Linux kernel provides a feature for detecting this phenomenon. The kernel documentation defines a split lock as "any atomic operation whose operand crosses two cache lines," and a bus lock as "a split lock access to writeback memory or any locked access to non-writeback memory." The boot parameter split_lock_detect accepts the values off, warn, fatal, and ratelimit:N, and the default recorded in the documentation is warn. If related warnings are appearing in your kernel log, that is very likely a real performance problem.

Code that treats MMIO like memory. The whole upper part of the table is this story. The syntax is a single mov, but the cost can be a million times a DRAM access. If a driver or embedded code is reading device registers inside a loop, that loop is not computation but a repetition of I/O round trips. It is worth examining whether status polling can be replaced with interrupts, whether several registers can be read at once, or whether reads can be converted into writes.

The repository notes that only the x86 leaderboard is filled in so far, with ARM and RISC-V in preparation. Once the lists for other architectures are populated, it will become clearer which cliffs are specific to x86 and which are properties of modern SoCs in general.

References

I did not run a single entry from this leaderboard myself. A good many of them are manipulations that halt the system or destabilize devices, and the repository's measurements are values for specific hardware, mainly an Intel Core i7-8559U and an AMD Ryzen 7 5800H.

현재 단락 (1/80)

Instruction latency tables are a basic tool for anyone doing optimization. Multiply is 3 cycles, div...

작성 글자: 0원문 글자: 10,653작성 단락: 0/80