The first time someone tried the dubbing tool I built, they leaned in, heard a line of Italian, then waited. One second. Two. Nothing. They looked at me like the thing had broken, and honestly I started to believe them.
Every real-time dub demo you see online cuts the silence. Speaker talks, translation lands a heartbeat later, voices swap, everything glides. Watch closely and you'll notice those clips never show a natural conversation — they show a trailer where someone conveniently pauses between sentences. Real dialogue doesn't do that.
Here's the truth nobody puts in the demo: my local dubber waits until the speaker finishes before it translates. Which means you hear the original line in the language you don't understand, a short gap, then the dub. A sentence or two behind, always. I spent a long time treating that delay as a flaw to engineer away.
It isn't. It's how the thing has to work, and pretending otherwise is the actual lie.
Whisper streams the audio off the tab, but it's only confident in what it heard once the words are actually spoken. You can't transcribe a word that hasn't happened yet, and you definitely can't translate a sentence you've only got half of. The translator in LM Studio reads a complete thought. Neural TTS — Kokoro in my case — speaks it once it's ready. Every stage is sequential, and sequential takes time. That's not a workaround to be embarrassed about. That's the physics of understanding.
Translation tools that claim zero lag are either pre-subtiling everything (so it's not really real-time, it's karaoke with extra steps) or they're shoving partial sentences through and stitching together something that sounds like a phone line during a storm. I tried the fast path early on. The output was quicker and noticeably worse — mistranslations from half-context, voices tripping over each other. I threw it out.
The fix that felt right was to lean into the rhythm instead of fighting it. The original audio ducks down under the dub so you can still hear the speaker's tone underneath — the shouting, the sarcasm — and the gap becomes this natural breathing space where your brain catches up. After a couple of minutes of a foreign film, you stop noticing the beat. You start reading the emotion in the original voice and the meaning in the translation together, and that's the part I actually got excited about.
I also stopped stacking models to fake speed. Whisper for transcribing, a small local LLM in LM Studio for translation, Kokoro for the voice, a pitch check to pick a male or female speaker. The whole thing runs on my PC on Python 3.10, no GPU required (it helps, but it's not the gate). Learning the delay and building the rhythm around it was the difference between a demo that works and a tool people can actually watch a movie with.
So yeah, mine waits a sentence. I used to call that a limitation. Now I call it the thing that makes it honest — and honestly, I'd rather be honest and a second behind than fake and wrong. The tool is called LiveDub, it runs fully offline once you've set it up, and you can grab it at https://symshah.gumroad.com/l/livedub if you want to watch something with a breathing space in it.
