feat(autotyping): type transcription directly into active window #26
Conversation
Add cross-platform virtual keyboard support to type transcribed text directly into the currently focused window, as an alternative to clipboard-based pasting. All new dependencies are pure Rust - no C libraries required. Platform backends: - Wayland: wrtype (zwp_virtual_keyboard_v1 protocol) - X11: enigo 0.6 with x11rb (pure Rust X11 bindings) - macOS/Windows: enigo (CGEvent/SendInput) New configuration options in ui settings: - output_method: clipboard | type_to_window | both - typing_backend: auto | wrtype | enigo - typing_delay_ms: optional inter-character delay New CLI flag: - whis --type: type output into active window (overrides config) Architecture: - New typing.rs module in whis-core with platform detection - Auto-detection routes Wayland → wrtype, everything else → enigo - Feature-gated behind 'typing' feature (enabled by default) Platform notes: - macOS: requires Accessibility permission in System Settings - Windows: cannot type into elevated (admin) windows - Wayland: compositor must support virtual keyboard protocol Closes #15
Rename typing module to autotyping for clarity: - typing.rs -> autotyping.rs - TypingBackend -> AutotypeBackend - type_text -> autotype_text Add tool status detection for desktop UI: - get_autotype_tool_status() detects available tools (ydotool, wtype, etc.) - Provides compositor-aware recommendations - Shows installation hints when tools are missing Desktop UI improvements: - Move Output Method from Miscellaneous to Advanced section - Show [i] hint with install instructions when autotype tools needed
The root command uses Enter to stop recording which doesn't work when typing in another window. Move --autotype to `whis start` where hotkeys enable recording control from any window. - Add --autotype flag to Start command variant - Pass output_method_override through Service to finish_transcription - Remove autotype from OutputOptions and RecordConfig - Remove dead OutputMode::Autotype variant
|
Hola @martintrojer Built on your work from #20 , switched from wrtype to external tools. Basically I took a deep look in how https://handy.computer does it. Then I said, that works, lets see. ^^ Check it out, Would be sweet if you try it and give me your opinion. Curious how it works on your end and what do you think about the solution? PErsonally, I am not happy with the current settings.json so far. Because if I use the desktop it actives it and when I run the cli whis start is conflicts wit the settings. Have to give it some thought later on. Enjoy your night 😴 👋 |
This looks good! Relying on wtype is very reasonable, it's a better compromise than adding library dependencies on the whis executable. This (and the push to talk feature) makes whis perfect in my workflow. Thanks! |
Summary
Add cross-platform virtual keyboard support to type transcribed text directly into the currently focused window, as an alternative to clipboard-based pasting.
This PR builds on the work from #20 by @martintrojer but takes a different approach inspired by handy.computer - using external CLI tools with graceful fallback instead of the
wrtypecrate which required thexkbcommonC library dependency.Key differences from #20
wrtypecrate (Rust)xkbcommonPlatform backends
New configuration options
Desktop UI improvements
[i]hint with installation instructions when tools are missingCLI
whis --autotype: type output into active window (overrides config)Credits
Closes
Supersedes #20
Closes #15