Rust · Voice infrastructure · Linux desktop

codex-voice

codex-voice is a Rust-native, Linux-first voice utility: hold a hotkey, speak, release, and the transcript lands in whatever app has focus. A second hotkey speaks your selected text aloud. Underneath the simple gesture is an 8-crate workspace, because the hard part — doing any of this on Wayland, where synthetic input is forbidden by design — is not simple at all.

The problem

Dictation tools assume they can read your keys and type into your windows. Wayland assumes they should not. There is no global keyboard hook, no synthetic input, and no "just XTestFakeKeyEvent" escape hatch — every capability has to be negotiated through desktop portals. Add the practical constraints — no separate transcription API key, no GPU-hungry local model, and other tools wanting to reuse the same capability — and the one-line utility turns out to have an infrastructure problem hiding inside it.

What I built

A trait-driven dictation engine in a pure core crate — recorder, transcriber, injector as swappable ports — with a state machine whose typed error stages always return to idle, and an event loop whose state guards discard hotkey presses arriving mid-transition.
A real-time-safe capture pipeline: the CPAL audio callback pushes buffers over a lock-free bounded channel to a dedicated WAV writer thread, recycling buffers through a pool so the callback never allocates.
Wayland integration done the sanctioned way: global shortcuts via the GlobalShortcuts portal, and text insertion via a RemoteDesktop portal session injecting raw keycodes for Ctrl+V — with the portal restore token persisted so the user approves exactly once.
Auth piggybacking instead of key management: transcription reuses the existing Codex/ChatGPT session, refreshing it by driving a spawned codex app-server over stdio JSON-RPC with a deadline-authoritative reader tested against hanging subprocesses.
A local OpenAI-compatible audio server on127.0.0.1:3845 — /v1/audio/transcriptionsand /v1/audio/speech — so any Whisper-shaped tool gets transcription and TTS for free, with health-probed fallback and ffmpeg chunking for oversized audio.
Persona-aware TTS across Google Gemini and ElevenLabs with a fallback policy that swaps providers on failure while preserving the voice persona, plus an installable web PWA embedded into the binary at build time.

How it works

The insertion path is the distinctive trick. Since Wayland will not let an app type into another app, codex-voice snapshots your clipboard, places the transcript on it, opens a RemoteDesktop portal keyboard session, sends the raw keycodes for Ctrl+V, waits 250ms, and restores your original clipboard. Reading a selection inverts the same idea with a twist: it writes a randomized sentinel to the clipboard, sends Ctrl+C, and checks whether the clipboard changed away from the sentinel — cleanly distinguishing "the user had a selection" from "the copy did nothing."

Dictation pathEvery stage that can fail has a typed error branch back to idle; a hotkey pressed mid-transcription is discarded, not queued into a surprise second recording.

The server turns a personal utility into shared infrastructure. Tools that already speak the OpenAI audio API point at localhost and get the same transcription and TTS the hotkeys use, discovered via a permission-locked state file. Audio beyond the backend's 24MiB cap is split by ffmpeg into 16kHz mono chunks and reassembled; a stale or failing local server falls back transparently to the direct backend path.

Local audio serverOne capability, three consumers: the hotkey engine, any OpenAI-compatible tool, and an installable PWA embedded in the same binary.

How it achieves the goal

A dictation tool succeeds when you stop thinking about it, and the engineering serving that is mostly about failure: recordings under 120ms are discarded, temp audio is deleted on every path, empty transcripts skip insertion, and the engine cannot wedge in a non-idle state. The discipline is enforced by 224 Rust tests plus Playwright coverage for the web app, and CI that runs fmt, clippy with warnings as errors, tests, and cargo-deny on every push.

What isn't done yet: Linux is the only proven target — the macOS port is explicitly on hold and Windows compiles but is unvalidated in a real session — and packaging is still a roadmap phase. I built the foundations in the order that survives: correct engine, sanctioned platform integration, shared server, and the packaging and extra platforms after.