The robot that couldn't hold a conversation
Picture this: a social robot standing in a lobby, listening for its wake word, chatting with whoever walks up. It's supposed to feel alive — responsive, natural, always listening. Instead, every so often, mid-sentence, it would just... die. SIGSEGV. The process gone, the conversation cut off like someone pulled the plug.
And the worst part? I couldn't reproduce it on demand. It happened more when the robot was busier — more people walking up, more wake words triggering, more conversations overlapping. Classic intermittent bug energy: the kind that makes you doubt your own sanity a little.
Sometimes it wasn't even a crash. The robot would just... not hear you. You'd say the wake word, nothing would happen, and you'd be left wondering if it was ignoring you or if something deeper was broken. Turns out — both, in a way.
The system: two ears, one microphone
Voxa, the robot's speech-to-speech pipeline, has two jobs running more or less independently:
- Wake-word detection — always listening in the background for its name.
- The conversation loop — once triggered, opens a live capture session to actually record and process what you're saying.
Both of these needed audio. Both of these, it turned out, were opening their own independent PortAudio capture stream to get it — two separate handles onto the same physical microphone.
Most of the time this was fine. Wake-word stream closes, conversation stream opens, no overlap, no problem. The failure only showed up in the gap between those two states — when a new utterance kicked off before the previous stream had actually finished tearing down.
Chasing a bug that only bites under load
The first instinct with a crash like this is to look at the crash site — the stack trace at the moment of SIGSEGV. But that only tells you where the memory got corrupted, not why two completely different code paths were touching the same resource at the same time.
The real signal was in the pattern, not the stack trace: this only got worse under load. That's usually not a logic bug — your code isn't suddenly wrong when there are more users. That's a race condition: code that's individually correct but breaks when two copies of it run concurrently and neither one knows the other exists.
Once I started looking at it through that lens, the pattern became obvious: wake-word detection and the conversation loop each owned a PortAudio stream, and neither one knew the other existed. There was no coordination — no lock, no shared state, nothing telling one "wait, the other one still has the mic." When a new utterance started fast enough, both streams ended up holding the device simultaneously. Sometimes that just corrupted the audio buffer silently. Sometimes it corrupted memory badly enough to take the whole process down with it.
The fix: give the microphone exactly one owner
The fix wasn't clever — it was structural. Instead of two components each managing their own capture stream, I unified audio capture onto a single PortAudio stream, with one explicit owner responsible for its entire lifecycle: opening it, closing it, and deciding who gets to consume from it.
Wake-word detection and the conversation loop still both need audio — but now they read from the one stream that owns the device, instead of each independently owning a competing one. There's no window anymore where two teardown paths can race each other, because there's only one teardown path.
before:
wake-word -> opens PortAudio stream A -> mic
convo loop -> opens PortAudio stream B -> mic
(both can hold the device at once — race)
after:
single owner -> opens PortAudio stream -> mic
wake-word -> reads from owned stream
convo loop -> reads from owned stream
(exactly one lifecycle, no race)
The result — and the feature it unlocked
The crash class disappeared completely. No more SIGSEGVs, no more leaked stream handles, no more mysterious "why didn't it hear me" reports.
But the more interesting payoff was what came after the fix. With a stable, contention-free capture path in place, I could finally tune barge-in detection — letting someone interrupt the robot mid-sentence, the way you'd naturally interrupt a person. That feature had been effectively unbuildable before, because you can't reliably detect an interruption on a microphone that occasionally forgets it's supposed to be listening.
Fixing the "boring" stability bug is what made the actually delightful feature possible.
Takeaways
- A shared hardware resource needs exactly one owner for its lifecycle. The moment two components each independently manage the same device, you have a race condition waiting for the right timing to expose it — not a convenience, a liability.
-
Bugs that scale with load are usually contention, not logic. If a bug gets worse the busier your system is, stop looking for a wrong
ifstatement and start looking for two things touching the same resource without knowing about each other. - Stability work often pays for itself in features you didn't plan for. Barge-in wasn't the goal when I started debugging crashes — it was a side effect of finally having a capture path solid enough to build on.
- Reproducing "under load" bugs on purpose beats waiting for them. If you suspect contention, write the thing that hammers the two code paths back-to-back instead of hoping it happens again in front of you.
Over to you
Race conditions are sneaky precisely because the code looks right in isolation — it's only wrong in relation to something else running at the same time. If you've hunted one of these before, I'd love to hear it: what was the tell that made you stop looking at logic and start looking at timing?
If you're building anything with real-time audio, sensor fusion, or just systems that are supposed to feel alive and responsive, I write about this stuff regularly — follow along, and feel free to reach out if you're wrestling with something similar.
