Sources this series builds on: Chili's (ChiliTomatoNoodle) multithreading
playlist,
CMU 15-213 (Computer Systems: A Programmer's Perspective), various CppCon talks,
and LLMs for restructuring and sanity-checking. Everything here is my own code,
compiled and run — not just transcribed.
Recap
Part 4 established why caching happens at all — spatial locality: if you
access one address, you're likely to want its neighbors soon, so hardware
fetches more than exactly what you asked for. This article answers the
obvious follow-up: how much more, exactly, and why that specific amount?
The answer is a fixed-size chunk called a cache line, and on most modern
CPUs that size is 64 bytes — always, no matter what you actually asked
for. This one fact is what the rest of this hardware track, and eventually
false sharing, is built entirely on top of.
What actually happens when you read one int
Say your code does int x = data[3]; and that triggers a cache miss — the
data isn't in cache yet, so it has to come from RAM. The natural mental
picture is the hardware reaching into RAM and pulling out exactly the 4
bytes that make up that one int. That is not what happens.
Instead:
The entire 64-byte block gets pulled in and cached as one unit, and your
int gets read out of that block afterward. You paid for 64 bytes whether
you needed them or not.
Why 64 bytes specifically — two independent reasons
Reason 1: the physical width of the connection
The hardware path between RAM and the CPU (and between cache levels) is
physically built to move a fixed number of bytes per transfer — its "width."
On most modern CPUs that width is 64 bytes. The critical part:
transferring 4 bytes over this path takes the same amount of time as
transferring all 64, because the limiting factor isn't the amount of data,
it's the fixed overhead of doing a transfer at all. Think of it like a
delivery truck that costs the same to dispatch whether it's carrying one box
or a full load — you fill the truck because the trip itself is the expensive
part, not the cargo.
Reason 2: spatial locality says it's worth it
Since nearby addresses are likely to be accessed soon anyway (Part 4), fetching
the surrounding 60 bytes "for free" — same transfer cost as fetching just 4 —
is a good bet almost all the time. Iterate over an array, and this pays off
constantly: fetching element 0 also fetches elements 1 through 15 (for 4-byte
ints), so the next several loop iterations are already cache hits before you
even ask.
Cache lines are aligned, not centered on what you asked for
A subtlety worth being precise about: the 64-byte chunk fetched isn't "the 64
bytes starting at the address you asked for." It's the 64-byte block that
address falls inside, where blocks are laid out in fixed segments starting
from address 0 (0-63, 64-127, 128-191, and so on). That's what aligned
means here.
So asking for the int at address 1004 doesn't fetch bytes 1004-1067 — it
fetches whichever fixed 64-byte segment contains address 1004 (addresses
960-1023, in this example). The requested bytes might sit anywhere inside
that segment — start, middle, or end.
This is the single most important detail in this article. Because lines
are fixed, aligned segments — not "64 bytes around whatever you asked for" —
any two variables that happen to land inside the same aligned 64-byte segment
get fetched and cached together as one unit, entirely regardless of whether
your program treats them as related at all. The hardware has no concept of
"these are two different variables." It only knows about lines.
How much fits in one line
64 bytes holds, for example:
- 16
ints (4 bytes each) - 8
doubles (8 bytes each) - A handful of small struct fields, packed together
So in practice, several "neighboring" variables in a real program — array
elements, adjacent struct fields, even separate global or static variables
that happen to land near each other by the compiler's own layout choices —
very commonly share a single cache line without anyone ever writing code
that groups them together on purpose.
Glossary added this part
| Term | Meaning |
|---|---|
| Cache line / cache block | The fixed-size chunk (typically 64 bytes) transferred and cached as one unit, every time |
| Bus width | The fixed amount of data a hardware connection moves per transfer, regardless of how much was actually requested |
| Aligned | Cache lines sit at fixed positions from address 0 (0-63, 64-127, ...), not centered on whatever address you asked for |
Takeaways
- A cache miss never fetches just what you asked for — it fetches the entire aligned 64-byte segment containing that address, every time.
- 64 bytes is fixed by hardware bus width (fetching 4 bytes costs the same as fetching 64, so hardware always fetches the full amount) and justified by spatial locality (the extra bytes are usually useful anyway).
- Cache lines are aligned to fixed 64-byte boundaries starting from address 0 — not centered on your specific address — which is why unrelated variables can end up sharing a line purely by accident of memory layout.
- The hardware has no concept of "your variables." It only tracks lines. Two
completely unrelated
ints can be neighbors in the same 64-byte block with neither programmer ever knowing.
