Hold-to-Talk Dictation

Linux Whisper Voice Firefox GNOME Wayland Slack AI

Avoid RSI and improve your productivity.

After working in IT for 30 years, you start to develop a few aches and pains. I remember playing around with Dragon Dictate in the early 2000s, or even the late 90s for that matter, and it didn’t quite fit my style of work. Tapping the keyboard for 9 hours a day takes its toll. Time to try talking: hold a key, speak, release, and have the words appear wherever my cursor is. No cloud service, no subscription, nothing leaving the machine. Whisper runs happily on a laptop CPU these days, so the only real problem is plumbing.

The setup ended up in three layers that share one transcription server. The code is on GitHub: LewS/whispering-jack.

The Server

faster-whisper with the large-v3-turbo model, int8 on CPU. A small Python HTTP server loads the model once and accepts raw 16 kHz mono PCM, returning the transcript as JSON. Loading the model per request is what makes most home-grown dictation scripts feel sluggish, so keep it resident. Even with it resident, there is still a bit of lag with Whisper. Those with furious fingers probably wouldn’t be able to tolerate the 1.5 to 2 second time it takes to transcribe. I’m finding for myself it’s probably just about on par. Maybe a slight bit slower When not fat fingering, and slightly above par, when I am. Not to mention the lost artform of spelling.

Two small details worth copying:

  • Bind to an odd loopback address. Everything in 127.0.0.0/8 is loopback on Linux, so I used 127.0.10.8:8765 rather than the usual 127.0.0.1. It’s not security on its own, but it keeps the service off the well-known address that every other local tool and page probes.
  • Check the Origin header. Browsers set it themselves and a web page’s JavaScript can’t override it, so the server rejects any request whose origin isn’t a browser extension. That keeps random web pages from using your transcription server. It isn’t authentication: other local programs, or other extensions, can still send requests. The server only turns audio into text. It never touches the microphone and can’t type anything.

Run it as a systemd user service so it starts at login:

[Service]
ExecStart=%h/path/to/venv/bin/python %h/path/to/server.py
Restart=on-failure

[Install]
WantedBy=default.target
systemctl --user enable --now whispering-jack

It keeps the model in memory, so expect a couple of GB. If that’s too much, small.en is a fair trade.

Layer One: A Firefox Extension for Slack

In the Slack web client, hold Space in an empty message box and talk. A content script on app.slack.com waits about 300 ms to tell a tap from a hold, records from the mic, and on release posts the audio to the server and inserts the transcript. It never sends the message for you, so you can read what it heard before pressing Enter.

A few Firefox-specific gotchas:

  • Manifest V2. Firefox doesn’t support service-worker backgrounds, and MV3 host permissions aren’t granted at install. MV2 with a background script just works.
  • Sample rate. Firefox refuses to connect a mic stream to an AudioContext created at 16 kHz. Capture at the native rate and downsample yourself.
  • Make the request from the background script, not the content script, so Slack’s own page rules don’t get in the way.
  • Slack’s editor is contenteditable, so insert text with document.execCommand('insertText'), not by setting a value.

To keep the add-on across restarts you need Mozilla to sign it. An unlisted submission is free, and web-ext sign --channel=unlisted handles it. Run it from the extension folder and give it a good few minutes. Until then, loading it temporarily from about:debugging works, but it vanishes when Firefox restarts.

Layer Two: Everywhere Else

A browser extension can’t see the address bar, other apps, or anything outside a web page. For those I wanted hold-to-talk anywhere, and Wayland makes that awkward:

  • Apps can’t listen for global key presses.
  • GNOME custom shortcuts fire on key press only. No release event, so you can’t do true hold-to-talk with one.
  • xdotool and wtype don’t work on GNOME, so you can’t type into another window the usual way.

The pieces that do work:

  1. A GNOME Shell extension grabs Alt+Space. When it fires, the extension starts recording, then takes a brief modal keyboard grab so it can see the Space key-up. On release it stops the recording. It needs no special permissions. The alternative, adding yourself to the input group so a daemon can read raw key events, also lets every program you run read all your keystrokes. I didn’t like that.
  2. A small script records with pw-record (picking my Rode Podcaster MK II & failing over the built-in mic), posts the audio to the same server, copies the transcript to the clipboard and pastes it.
  3. ydotool does the pasting. It writes to /dev/uinput, so the kernel creates a virtual keyboard and GNOME treats it as real. It can only send input, not read it. It needs its ydotoold helper running as a user service, and wl-clipboard for the clipboard on Wayland:
sudo apt install wl-clipboard ydotool
systemctl --user enable --now ydotoold

Hold Alt+Space, talk, release, and the text appears in the address bar, Slack’s search, an AI chat box, or any other text field.

Small Details

  • Free up Alt+Space. GNOME uses it for the window menu. Clear it first:

    gsettings set org.gnome.desktop.wm.keybindings activate-window-menu "[]"

  • Don’t also bind a custom shortcut to the same keys. Two triggers on one key fire the script twice.

  • Terminals paste with Ctrl+Shift+V, not Ctrl+V. The extension knows which window has focus, so it tells the script when to use the terminal paste.

  • The clipboard gets overwritten by each transcript. That’s the cost of pasting via the clipboard.

  • GNOME caches extension code. On Wayland, changes to the extension itself need a log out and back in. Everything else (the script, the server) picks up changes immediately. That’s one more reason to keep the logic in the script and make the extension as thin as possible.

  • Left and right Alt both work. GNOME shortcuts can’t tell them apart.

  • Give Whisper a prompt. An initial prompt with your spelling and jargon steers it. Mine nudges it toward Australian English spelling.

Teaching It Your Words

Whisper picks the commonest word it knows, so names and jargon often come out wrong. Say “Kubernetes” fast and you might get “cooper netties”. Two server options fix this, and they work best together:

  • --hotwords "Kubernetes, Terraform" biases Whisper toward those spellings while it transcribes. It’s a nudge, not a guarantee.

  • --replacements words.json fixes the finished text. The file is a JSON object of "heard": "wanted" pairs:

    {
  "cooper netties": "Kubernetes",
  "terra form": "Terraform"
}

The replacements are whole-word and ignore case, so a fix for “terra” changes “Terra” and “Terra’s” but leaves “terrace” alone. The server re-reads the file when it changes, so a new word takes effect the moment you save it, with no restart. The catch is that it replaces every time, so a word you sometimes mean literally gets changed too. Keep your own list outside the repo, since it’s full of your names and jargon.

Verdict

It’s fast enough on CPU to be pleasant, entirely local, and works in the browser and out of it. The Wayland restrictions are a bit of a tax, but for good reasons, and also why it ended up thin and unprivileged. Combined with the fingerprint scanner, the keyboard use is reduced to a bare minimum. So hold a key, have your say and save your time & limbs ;-)