Shipping Local Speech-to-Text in a Desktop AI Assistant: What We Learned Bundling whisper.cpp

Originally published on cernalo.com.

The desktop app turns speech into text on your computer with a bundled whisper.cpp runtime. The question is answered only after you send the text.

The audio stays on the computer

I build Cernalo, an AI desktop assistant, on my own. This is an engineering write-up of one decision: transcribing your voice on your own device. It includes the version that did not work.

When you click the microphone in Cernalo, or press Ctrl+Shift+V, and ask a question out loud, the audio does not leave your computer to be transcribed. A small speech-to-text engine bundled with the app turns it into text locally. You see the transcript, and only when you send it does the text go to our service and on to the AI model that answers.

That sounds like a small product detail. It took two attempts, one of which I would now call a mistake, and it shaped how the whole desktop app is packaged. If you are thinking about putting local speech recognition into an Electron app, or any desktop app, this is what I would tell you.

Why transcription stays on the device

Three reasons, in order of importance.

Voice is the most sensitive input we handle. A screenshot is something you frame on purpose. A microphone picks up whatever is in the room: a colleague's side conversation, a phone call in the background, a child asking for snacks. If speech can become text without sending raw audio anywhere, that audio is not in flight and it is not stored on a server for transcription.

You should see what you said before anyone else does. Local transcription makes a review step natural. The transcript appears, you can correct a misheard word or decide not to send it at all, and only then does it become a request. That is the same "you choose the context" principle the rest of Cernalo is built on.

The current desktop app has one speech path: the bundled local runtime. It does not include a cloud speech option, so microphone audio is not uploaded to be transcribed. The text you review is the part you can choose to send.

Here is what local transcription is not: a claim that Cernalo is offline AI. Answering your question still uses hosted models through our service. I will come back to that split, because being vague about it is how desktop AI products lose trust.

Version one: a script subprocess

The first working prototype did what most tutorials suggest. A small worker script, written in a popular scripting language, loaded an open-source Whisper package and transcribed a temporary audio file. The Electron main process spawned it, waited, and read the text back. On my development machine it worked on the first afternoon.

Then I tried to package it.

A desktop app launched from the desktop does not inherit your developer shell. My terminal had the right interpreter, the right package manager, a virtual environment, and the model files on its PATH. A packaged app has none of that. The worker simply could not find its runtime.

The fix would have pushed my toolchain onto every customer. To make it work, a user would have had to install a language runtime, a package manager, a virtual environment, and the speech package, and keep them working through updates. That is not a product. It is a setup guide with a logo on it.

The worker could not run from inside the app archive. Electron apps ship most files inside a packed archive, app.asar. An external interpreter cannot execute a script that lives inside it. Workarounds exist, and every one of them felt like a new class of bug.

I had been tempted by the principle "do not rewrite a working subprocess". The packaging requirement overrode it: the shipped app has to transcribe using files that ship beside the app, not a toolchain the customer assembles. That sentence became the decision record.

Version two: bundled whisper.cpp and FFmpeg

The approach that shipped uses whisper.cpp, a native implementation of the Whisper model, plus a bundled FFmpeg for audio conversion. Both are plain executable files that sit in the app's resources directory, outside the packed archive: whisper/bin/whisper-cli, whisper/bin/ffmpeg, and whisper/models/ggml-<model>.en.bin.

The pipeline from the microphone to text is: the microphone runs only after you start it, audio is recorded in the app, a temporary local file is written, bundled FFmpeg converts it, bundled whisper.cpp transcribes it, the transcript is shown to you, you edit or send or discard it, and only then does a text request go to Cernalo's service.

Five implementation details made the difference between "works on my machine" and "works when installed".

Resolve paths and fail closed

The build copies the speech runtime into the app's resources outside the packed archive, using electron-builder extraResources. The main process resolves those exact paths at runtime. It does not search PATH and it does not look for a system copy of FFmpeg. If a binary is not where we put it, that is a packaging bug, and I want to see it as one.

When the packaged app cannot find the binary or the model, it fails with a clear error, internally RUNTIME_NOT_FOUND or MODEL_NOT_READY. It does not quietly try a system interpreter or a cloud service instead. Silent fallbacks are how a local feature turns into a non-local one without anyone noticing, and that is the failure I most wanted to rule out.

Spawn safely, pin the build, and keep licences honest

The transcriber is spawned with an argument array and shell set to false, so nothing from the audio path or settings is interpreted by a shell. Each run has a timeout, can be cancelled when you stop or close the app, and the temporary audio file is deleted afterwards. Small things, but they are the difference between a helper process and a liability.

The release build pins a specific whisper.cpp revision, a specific model file, and an FFmpeg build compiled with LGPL-only options. It checks their hashes during packaging. The model weights ship with the app. They are not downloaded the first time you speak. That makes installers larger, and in exchange the thing you install is the thing we tested.

FFmpeg's licensing depends on how it is built. Our packaging includes a guard that checks the FFmpeg build flags, and the third-party notices ship with the app. If you are bundling native tools, budget time for this. It is not optional.

Model choice and honest limits

We use a small English Whisper model by default. That choice comes with limits I would rather state than have you discover.

It is English-focused. Other languages and heavy code-switching will not transcribe as well with this model.

Performance depends on your hardware. Transcription runs on your computer, so an older machine will do the same work with more effort than a recent one. That is why the site says performance varies by hardware, rather than quoting a number.

It is basic speech-to-text. It turns what you said into text. It is not a meeting notetaker, and it does not identify speakers. Voice questions use the microphone you start. They do not take audio from other applications.

Accuracy is not perfect, which is exactly why the transcript is shown to you before it is sent.

Could we use a larger model? Yes, with a bigger download and more work for your machine on every question. For short spoken questions, the small model was the right balance for us. If you have a strong opinion, I would like to hear it.

What still goes to the cloud, and why

Being precise matters here. Microphone capture, when you start it, happens on your device. Speech-to-text happens on your device, with bundled whisper.cpp. Reviewing and editing the transcript happens on your device. Answering the question happens on Cernalo's service, then the configured AI model for your plan. Screen or region capture, when you start it, is captured on your device and sent with the request you submit.

The language models that answer your question are hosted. We broker every AI request through our own service, so provider credentials never sit on your machine and plan limits are enforced server-side. I have written separately about why API keys stay off your desktop.

Running a capable answering model locally on a typical computer is a different trade-off with different costs, and we do not claim to do it today.

So the honest summary is: your voice is transcribed locally, and your question is answered in the cloud.

A checklist if you are bundling whisper.cpp

If you are building something similar, here is the list I wish I had had.

Decide early whether local is a promise. If it is, design the failure mode first: what happens when the binary or model is missing? Our answer is to fail closed and show an error.
Ship binaries and models outside your app archive and resolve them by absolute path. Do not rely on PATH in a packaged app.
Spawn with argument arrays and no shell. Add a timeout and a cancel path. Delete temporary audio.
Pin versions and hashes for the runtime, the model, and FFmpeg. Verify them in the build, not by hand.
Check FFmpeg licensing for the exact build you ship, and include notices.
Show the transcript before sending. It handles both accuracy and consent.
Say plainly which parts are local and which are not. People notice when local quietly means partly local.
Test on modest hardware, not just your development machine.

Questions

Does Cernalo record in the background? No. The microphone runs only after you start it, and the app shows when it is active.

Can you use cloud speech-to-text instead? Not in the current desktop app. Transcription uses the bundled local runtime. There is no cloud speech switch. Only the text you send continues to the AI request.

Does local transcription mean Cernalo works offline? Transcription itself does not need a network. Getting an AI answer does, because answers come from hosted models through our service.

Which languages are supported? The default local model is English. Other languages may transcribe poorly.

The context you choose, in one place. You can try voice questions on the free plan from the download page. If you are an engineer and think I made the wrong call somewhere above, tell me: cernalo.ops@gmail.com.