I wanted to dictate on my PC without paying for a subscription. So I built Wisperno: hold a hotkey, speak, release, and the text is pasted into whatever app has focus.
It runs fully offline. Whisper large-v3-turbo transcribes on the GPU, and a small quantized language model cleans up the text. There are no network calls once the models are on disk, so no audio or text leaves the PC.
What it does
There are two ways to dictate. Direct Dictation uses no language model at all. A rule-based formatter capitalises pronouns, strips filler words and detects lists in well under 10 ms. Polish sends the text through the local model to fix grammar and filler. You pick per hotkey: tap Ctrl+Alt for direct dictation, Alt+C for polish.
A long pause (1.3 seconds or more) starts a new paragraph. I first tried segment-level timestamps from Whisper to detect the pause, and they could not see it at all. Word-level timestamps can.
How fast is it
I measured it on my laptop (RTX 4070 Laptop GPU, 8 GB) with tests/benchmark_models.py. Whisper stays loaded the whole time, so the VRAM numbers match the real app.
| Preset | VRAM | Decode speed | 100 words |
|---|---|---|---|
| Turbo Flagship (default) | 3491 MB | 113.8 tok/s | 0.82 s |
| Eco | 3621 MB | 103.3 tok/s | 0.89 s |
| Standard | 3691 MB | 91.7 tok/s | 1.04 s |
| Flagship | 5119 MB | 63.1 tok/s | 1.48 s |
The smaller presets are faster and use less memory. The bigger ones keep a little more fidelity: 97.6% for the two smallest, 98.8% for the two 3B models, measured on raw model output.
Upgrading llama-cpp-python from 0.2.90 to 0.3.26 made every preset about 20 to 30% faster with the same output. For the default preset that was 89.3 to 113.8 tokens per second. VRAM did not change.
Making a small model behave
A cleanup model can answer your dictation instead of cleaning it. If you say a question, it may reply to it. So the app runs every result through a guardrail (sanity_check()), and I wrote a suite of adversarial prompts meant to make the model answer back. With the guardrail on, that suite passes 100% on every preset.
A bug my own tests found
Live Transcription is a separate mode for meetings and lectures. My first design re-decoded only a sliding window of the latest audio. The test suite showed it silently dropped real words. The current version re-decodes the whole unconfirmed audio buffer every 0.6 seconds. It runs about 13.5 times faster than real time (RTF 0.074), with no second model loaded.
What it cannot do
- Windows only, and it needs an NVIDIA GPU. Below about 2 GB of VRAM it falls back to a slow CPU mode.
- It is a source install. The packaged build is about 4.7 GB, too big for GitHub.
- The fidelity scores above are on raw model output.
Code, benchmarks and install steps: https://github.com/Burthcer/Wisperno (MIT; the models keep their own licences).
