After building a desktop-wide speech-to-text prototype for a family member with limited typing ability, I started wondering whether this is something that would make sense as a built-in GNOME feature.
What I did was use a local Whisper speech recognition backend running in a Podman container as a user service, together with a small Python D-Bus service that’s bound to a keyboard shortcut. Press the shortcut, speak, press to finish, and the recognised text is inserted into the currently focused application. It’s turned out to be very usable in day-to-day use, and it’s made me wonder whether this general idea is something the GNOME desktop itself should consider supporting.
It would be great if we could add a desktop speech-to-text service with a stable API and UI, while allowing different recognition engines to provide the backend (Whisper, Faster-Whisper, Vosk, etc.).
Some things such a service could provide:
A standard “Start/End Dictation” keyboard shortcut.
A small on-screen indicator while recording.
Desktop-wide text insertion into the focused application.
A D-Bus API that backends can implement or integrate with.
User-configurable choice of recognition engine.
Fully offline recognition for privacy and accessibility.
From an accessibility perspective, this feels like a capability that could benefit users with motor impairments or repetitive strain injuries, while also being generally useful for anyone who prefers dictation.
Would something like this fit within GNOME’s direction, or is there any related work happening that I’ve missed?
is there any related work happening that I’ve missed?
The closest I’ve seen to an OS-wide initiative is the recent Myna project from Ubuntu (which I’m not affiliated with in any way). I recommend contributing/keeping an eye on this project that is expected to launch in the next release and does pretty much what you are describing.
Meanwhile, I also invite you to check out Speed of Sound, which is another GNOME-first tool for voice dictation (this one I’m affiliated with because I built it :-). It uses local models for speech recognition, Whisper and others, and uses desktop portals for wide compatibility across DEs, X11, and Wayland.
@zugaldia Thanks! I somehow hadn’t come across Speed of Sound before, so I’ll definitely have a look at it. I might not have rolled my own solution if I’d come across it sooner
Myna also looks interesting, but I was imagining this as something the desktop itself could provide, with different speech recognition backends available underneath, rather than being tied to a particular distribution or implementation.
@adrianvovk Thanks, I’m taking a look at ibus-speech-to-text too. I do wonder whether dictation would work better as an accessibility feature than as a keyboard input method though?
In any case, thanks for the pointers—I’ve got some reading to do!
If this is an area GNOME is interested in, I’d be happy to help. (I’m new to contributing to the desktop itself, so I’d have a lot to learn, but I’d be interested in getting involved.)
EasySpeak is a GNOME- and Wayland-first application for voice control. It also covers dictation. The project is officially in alpha stage, but has seen a lot of movement recently.
Full disclosure: I’m one of the developers on the project.
