The problem
A former colleague once showed me a binary protocol that did everything right: fields ordered by access frequency, structs padded to land exactly on 64-byte cache-line boundaries, hot fields separated from cold ones — the kind of layout work covered in Binary Protocols: Designing Messages For Cache Lines, which took a message-processing pipeline from 14,700 to 50,100 messages/sec by fixing exactly this.
Except this second system had the identical struct layout and was still slow — not "8,000 msg/sec and the CPUs are asleep" slow, but a flat 30% overhead that profiling kept blaming on "GC pressure" and "allocation churn," with no obvious smoking gun. Cache-line alignment was right there in the struct definition. So why did the allocator show up in every flame graph?
Why it happens
Alignment tells the CPU where a field will land once it's in memory. It says nothing about whether the field is read from the buffer it arrived in, or copied somewhere else first.
The common pattern: bytes arrive off the wire into a receive buffer, and the very first thing the code does is deserialize — allocate a fresh struct on the heap, and field-by-field (or via memcpy) copy the wire bytes into it. Only then does application logic touch the "nice," cache-aligned struct.
That copy reintroduces almost everything the layout work was supposed to remove:
- An allocation for every message, which means allocator bookkeeping and, in managed runtimes, eventual GC pressure — the exact thing the flame graphs were pointing at.
- A second full read of the same bytes — once to copy them, once to actually use them — so you pay the cache-miss cost you just engineered around, just delayed by one step instead of eliminated.
-
A brand-new memory address for the copy, which may or may not land on your carefully-chosen 64-byte boundary, because the allocator — not your
__attribute__((aligned(64)))— decides where the copy lives. On some allocators you get lucky. On others, alignment guarantees you set at compile time quietly stop applying at runtime.
The struct in the original design is fast to access. Nothing in that design says anything about being fast to obtain. Those are two different problems, and it's easy to solve the first while never noticing you still have the second.
What to do about it
The fix is to stop deserializing and start viewing. Instead of copying wire bytes into a new object, cast a pointer directly into the receive buffer and read through it:
// Deserializing (the tax):
CacheOptimizedMessage* parse_message(const char* wire_bytes, size_t len) {
CacheOptimizedMessage* msg = malloc(sizeof(CacheOptimizedMessage)); // allocation
memcpy(msg, wire_bytes, sizeof(CacheOptimizedMessage)); // full copy
return msg; // caller must remember to free() this
}
// Viewing (zero-copy):
static inline const CacheOptimizedMessage* view_message(const char* wire_bytes, size_t len) {
if (len < sizeof(CacheOptimizedMessage)) return NULL; // bounds check
if ((uintptr_t)wire_bytes % _Alignof(CacheOptimizedMessage) != 0) // alignment check
return NULL; // or fall back to a copy for this one message
return (const CacheOptimizedMessage*)wire_bytes; // no copy, no allocation
}
Two things make this safe rather than just fast:
-
Bounds-check before you cast. A view is only as safe as the length check in front of it — you're trusting the buffer's declared length matches its actual contents, so validate
lenbefore you dereference anything, especially on untrusted input. -
Check alignment before you dereference, don't assume it. Receive buffers aren't always aligned the way your struct wants — network stacks,
mmap, and ring buffers each have their own alignment guarantees (or lack of them). If the buffer isn't aligned, either fall back to a copy for that one message (rare path, still correct) or use an accessor that does unaligned reads on purpose, rather than relying on the compiler to save you.
This is the same principle the original article's VariableSection uses for variable-length fields — get_variable_field() returns a pointer into the existing buffer via field_offsets[], not a freshly-copied string. Extend that same idea to the fixed-size header instead of just the variable tail, and the copy disappears from the entire message, not just the flexible part.
The trade-off: a view is only valid as long as the underlying buffer is. Copy-then-mutate code gets to outlive and modify its input freely; view-then-read code has to either finish before the buffer is reused (fine for most request/response and streaming pipelines) or explicitly copy out the one or two fields it needs to keep past that point.
Key takeaways
- Cache-line alignment makes a struct fast to read; it says nothing about whether you're paying to obtain it first via a hidden allocation and copy.
- If your flame graph blames the allocator on a "fast" binary protocol, check whether you're deserializing into a new object instead of viewing the wire buffer in place — that's a different bottleneck than the one alignment fixes.
- A zero-copy view needs a bounds check and an alignment check in front of every cast; skip either one and you've traded a performance bug for a memory-safety one.
- Apply the same in-place-pointer trick your variable-length fields probably already use to the fixed-size header too — it doesn't have to stay confined to the "flexible" part of the message.
