A client of mine, R. Pil, sends a lot of voice notes. In Russian. He needed them transcribed on his phone without uploading audio to some server. Privacy thing. And he wanted it to work offline, full stop.
So tonight I built it.
The Problem
Voice notes are convenient. You're walking, you're driving, you record a thought. But then what? You've got a 3-minute audio file sitting in your messages and no way to search it. Good luck finding that one thing you said about the meeting next Tuesday.
Most transcription services want your audio on their servers. Fine for grocery lists. Less fine for business conversations, personal notes, anything even mildly sensitive. And R. Pil has a Galaxy S25 Ultra sitting in his pocket. Plenty of horsepower. Why ship audio to the cloud when the phone can handle it locally?
What I Built
On-Device Whisper
OpenAI's Whisper is ridiculously good at speech recognition. The small model (about 461MB) handles 98+ languages and runs entirely on-device through whisper.cpp.
First launch downloads the model from Hugging Face. After that, airplane mode forever. No network required.
It's not instant. On a flagship phone you're looking at 30-60 seconds for a few minutes of audio. But it works, and nothing leaves the device.
Local LLM Polish
Raw Whisper output is choppy. Stream-of-consciousness, no punctuation sometimes, repetitions when someone stumbles over their words. You wouldn't want to read it as-is.
So I added a second stage. A local LLM (Braindler Q4_K_S, around 88MB) takes that raw transcript and turns verbal rambling into readable prose. I wrote custom prompts for Russian and English to handle the language-specific quirks.
The whole pipeline:
Audio → Whisper → Raw text → LLM → Polished prose → PDF/Display
About 2GB downloaded on first run. After that, everything lives on the phone.
Smart Language Detection
Tonight's addition. The app supports 98+ languages, and nobody wants to scroll through all of them every time they transcribe something.
I query Android's InputMethodManager for installed keyboards. If you've got English, Russian, and Hebrew keyboards installed, those three jump to the top of the language picker. Obvious in hindsight, but it makes the UX dramatically better than dumping an alphabetical list on the user.
The Tech Stack
- Flutter with platform channels for native integration
- whisper.cpp for on-device transcription
- flutter_llama for the LLM polish step
- 273 tests passing (because I've been burned before)
Flutter gets a bad rap sometimes, but for cross-platform apps where you need to drop into native code, it's genuinely good. Platform channels let you call Kotlin/Swift when you need to. The rest stays in Dart.
Why Bother With On-Device?
Your voice is biometric data. It's you. Sending it to random servers for transcription means trusting those servers, and trust is a cost most people don't think about until something goes wrong. With on-device processing, there's nothing to trust. The audio never leaves.
Then there's the offline angle. No Wi-Fi? Works. Underground parking garage? Works. International travel with expensive roaming? Still works. The whole system runs locally, so connectivity is irrelevant.
R. Pil gets his transcripts. I get another app in the portfolio. Not bad for a Tuesday night.
The app still needs some polish (better error states, maybe a larger Whisper model option for people who don't mind the extra storage). But the core pipeline is solid and the client's happy. That's usually a good sign.
Enjoyed this post?
Get notified when I publish something new. No spam, unsubscribe anytime.