The circular_buffer trap: how a 'faster' data structure spiked my CPU from 2% to 25%

There's a special kind of bug that only shows up after you've made your code "better." You swap in the data structure everyone agrees is more efficient, the microbenchmark cheers, you commit — and then the real system quietly starts burning CPU. This is the story of how I replaced std::deque with boost::circular_buffer in my audio player, watched playback CPU jump from 1–3% to 20–30%, and ended up reverting the whole thing a week later.

The punchline up front: for a byte-streaming FIFO, std::deque beat boost::circular_buffer by at least 2×, and the "obvious" reasons circular buffers are supposed to be faster never applied.

The setup

Kalinka Player is a music player that runs on a Raspberry Pi. At its heart is a bounded producer/consumer buffer: a network/decoder thread writes audio bytes to the back, and the audio output thread reads and removes them from the front. The element type is uint8_t, capacities are big (the HTTP buffer is ~768 KB, FLAC ~1.5 MB), and data moves in large chunks (~16 KB, matching CURL_MAX_WRITE_SIZE).

The original buffer was backed by std::deque<T>. The two hot paths look like this:

size_t write(const T *source, size_t size) {
  std::lock_guard<std::mutex> lock(m);
  size_t n = std::min(size, capacity - data.size());
  data.insert(data.end(), source, source + n);   // append to back
  return n;
}

size_t read(T *dest, size_t size) {
  std::lock_guard<std::mutex> lock(m);
  size_t n = std::min(data.size(), size);
  std::copy_n(data.begin(), n, dest);             // copy from front
  data.erase(data.begin(), data.begin() + n);     // remove from front
  return n;
}

Look at that read(). A ring buffer is literally the textbook data structure for this access pattern. deque has to manage a map of fixed-size blocks and, so my reasoning went, all that front-erasing must be doing real work. A circular_buffer is a single fixed allocation with two indices — no allocation churn, perfect for a bounded FIFO. It felt like free performance.

I even wrote a synthetic benchmark to confirm it.

The synthetic benchmark that lied to me

The microbenchmark pushed fixed-size records onto the back of a bounded queue and drained them from the front — exactly the producer/consumer shape, no overwrite. circular_buffer won comfortably:

Bounded FIFO, 32-byte records, 20M ops (higher is better):

Case closed, right? circular_buffer is 2–3× faster, it's the correct data structure for a ring, and the benchmark agrees. On 2024-09-11 I committed:

bad7d04 — Switch to boost::circular_buffer, give DequeBuffer a generic name

Reality disagrees

The unit tests passed. Playback worked. And then I looked at top.

The KalinkaPlayer process, which used to sit at a comfortable 1–3% CPU during playback, was now churning at 20–30% — on a Raspberry Pi, where that actually matters for heat and battery. Nothing else had changed. The only difference was the buffer.

My first instinct was that I'd used the container naively. And I had: I'd left the read() path calling the generic erase(begin, begin + n):

data.erase(data.begin(), data.begin() + sizeToCopy);

boost::circular_buffer has a dedicated front-removal method, erase_begin(n), that just advances the internal start index instead of going through the general erase machinery. So on 2024-09-18 I fixed it:

- data.erase(data.begin(), data.begin() + sizeToCopy);
+ data.erase_begin(sizeToCopy);

402cdd1 — Improve CPU usage by changing erase with more efficient erase_begin call on circular buffer — time down from 2ms to 0.02ms

That's a 100× improvement on that single call — 2 ms down to 0.02 ms. Huge! Surely that was the CPU regression solved.

It helped. It was not enough. CPU was better but still nowhere near the old deque baseline. So I added a performance monitor (60976db) to actually measure the hot paths instead of guessing — and the numbers were damning.

Measuring the real workload

Synthetic record-pushing is not what the buffer actually does. The real buffer moves bulk byte ranges: insert(end, src, src+n) and copy_n(begin, n, dst) over ~16 KB chunks of uint8_t. So I rebuilt the benchmark around the actual Buffer<uint8_t> read/write hot paths, with the correct erase_begin for the ring, and verified both containers produced byte-identical output before trusting any number.

Buffer<uint8_t>, capacity 768 KB, 16 KB chunks, 4 GB streamed (higher is better):

Scenario A — fill to capacity, then drain to empty:

Scenario B — steady interleave, buffer kept near full:

The "faster" data structure was 1.6× to 9× slower on the workload that mattered. Even with the erase_begin fix in place. On 2024-09-18, the same day, I gave up:

5b281fc — Switch back to std::deque as buffer implementation as it is at least 2x faster than circular buffer.

Total lifespan of the circular_buffer experiment: one week.

Why the ring loses: the memmove fast path

Here's the mechanism, and it's the whole story.

For a trivially-copyable type like uint8_t, the standard library reduces bulk copies (std::copy, std::copy_n, and deque's range insert) to a single vectorized memmove — but only when it can prove the iterators are contiguous. Raw pointers qualify. std::deque's block iterators qualify (the library special-cases them, copying block-by-block via memmove).

boost::circular_buffer's iterator is not contiguous — its storage wraps, so every ++ does modular index arithmetic. The library can't take the memmove path, so both hot ops collapse to a scalar, per-element, unvectorizable loop:

• write: deque::insert(end, src, src+n) → memmove into the tail block. That's the 33 GB/s in Scenario B — essentially raw memcpy. The ring can only copy one byte at a time through its wrapping iterator.
• read: copy_n(begin, n, dst) → memmove out of the front blocks for deque; byte-at-a-time for the ring.

That single fact — no memmove for a wrapping buffer — accounts for the entire gap, on both reads and writes.

And every intuition that sent me down this path turned out to be irrelevant here:

• "No allocation churn." True, but at 16 KB chunks deque mostly reuses its already-allocated blocks — writes hit an existing tail block and memmove into it. There's almost no allocation to avoid.
• "Better locality — one contiguous buffer." Also true, and it genuinely favors the ring... but the win is buried under per-element iterator overhead that dwarfs any cache benefit.
• "erase_begin is the fix." It was a real 100× win on that one call, and still the ring lost overall — because the copy and insert around it were the actual bottleneck, not the erase.

The synthetic benchmark wasn't wrong — it faithfully measured element-by-element pushes of 32-byte records, and for that workload the ring really is faster. It just measured a workload my program never runs. My program moves big contiguous runs of bytes, and that's precisely where memmove makes deque untouchable.

Lessons

1. Benchmark the real operation, not the shape of the problem. "Bounded FIFO" made me benchmark pushes and pops of single records. The real workload was bulk memcpy of byte ranges. Same data structure, completely different performance regime.

2. Trivially-copyable + contiguous = memmove, and it's enormous. If your hot path is bulk-copying POD, the single most important question is whether the library can lower it to memmove. A container that breaks contiguity (a wrapping ring, a linked structure) forfeits that, and no amount of micro-optimizing the other calls buys it back.

3. "The textbook data structure for X" is an argument, not a measurement. A ring buffer is the canonical structure for a bounded FIFO. It was still the wrong choice here. The canonical answer optimizes for allocation and wraparound; my bottleneck was neither.

4. Watch the system, not just the unit test. The tests were green the whole time. The regression only showed up as CPU on a running Pi. A cheap always-on performance counter (60976db) turned "it feels sluggish" into "here are the microseconds," and that's what ended the debate.

std::deque has quietly backed KalinkaPlayer's audio buffer ever since. The best data structure was the one I already had.

The benchmarks in this article are reproducible. The full single-file C++ harnesses — one reproducing the Buffer<uint8_t> read/write hot paths across both containers (verifying byte-identical output before timing), and one for the synthetic record-push benchmark — are available as a GitHub Gist. Build with g++ -O2 -std=c++17 (Boost's circular_buffer is header-only) and run; they print the tables above.