fangorn/sunshine-qemu
public
ref:main
Phase 1c: keyboard, mouse, touch input to QEMU #5
open
Opened by cole.christensen@gmail.com
Pull requests
Links
Phase 1c: keyboard, mouse, touch input to QEMU
Parent: #1. Depends on #2 (QemuSession). Can run in parallel with #4.
Requirements
- REQ-INP-001: keyboard input delivered to the guest
- REQ-INP-002: mouse input delivered to the guest (relative, absolute, buttons, scroll)
- REQ-INP-003: touch input delivered to the guest
Background (verified)
- Sunshine:
src/input.cppdecodes Moonlight input and calls platform free functions:platf::move_mouse,abs_mouse(touch_port, x, y),button_mouse(button, release),scroll(high_res_distance),hscroll,keyboard_update(modcode, release, flags),unicode,touch_update,gamepad_update,alloc_gamepad… On Linux and macOS these live insrc/platform/virtualhid_input.cpp(~L1026-1100) and forward to libvirtualhid (host uinput).platf::input()is created atsrc/input.cpp:2277.keyboard_updatemodcodes are Windows virtual-key codes. - QEMU D-Bus (on
/org/qemu/Display1/Console_N):Keyboard.Press/Release(u keycode), where keycode is a QEMU qnum (xtkbd scancode with the high bit re-encoded), plus aModifiersproperty (Scroll/Num/Caps lock state).Mouse.Press/Release(u button)with Left=0 Middle=1 Right=2 WheelUp=3 WheelDown=4 Side=5 Extra=6;SetAbsPosition(u x, u y)in console pixels (error unlessIsAbsolute);RelMotion(i dx, i dy)(error ifIsAbsolute).MultiTouch.SendEvent(u kind Begin=0/Update=1/End=2/Cancel=3, t slot, d x, d y),MaxSlots.
IsAbsolutedepends on the guest’s devices: add-device virtio-tablet-pciorusb-tabletfor absolute mode; PS/2 mouse is relative. It can change at runtime, so watchPropertiesChanged.
Tasks
- Routing: when
capture == qemu,platf::input()creates a context holding aqemu::input_t(bound to the same QemuSession and console as capture), and the free functions dispatch to it. The gamepad functions keep going to libvirtualhid (see #7). Keep the diff tovirtualhid_input.cppsmall and obvious for upstream review: a dispatch branch, not interleaved logic. - Keymap:
keymap.{h,cpp}translates Windows VK → LinuxKEY_*→ QEMU qnum.- libvirtualhid or Sunshine already has a VK→Linux keycode table; find it and reuse it.
- The Linux→qnum table must be generated or ported from QEMU’s keycodemapdb (
ui/input-keymap.cusesqemu_input_map_linux_to_qcode, then qcode→qnum). keycodemapdb is GPL-2.0-or-later, so it’s compatible with GPL-3 Sunshine; record provenance in a README. - Handle extended keys (flags from Moonlight), left/right modifiers, and keys with no mapping (log once).
- Mouse:
- Absolute mode: convert Moonlight’s
touch_port-relative float coordinates to console pixels (use the consoleWidth/Height, honoring any capture offsets) →SetAbsPosition. - Relative mode: accumulate sub-pixel deltas →
RelMotion. - If the client sends absolute input while the guest is relative, or the reverse, convert rather than erroring: absolute→relative needs the last known guest cursor position from
MouseSet(#3), otherwise drop with a debug log. - Scroll: each 120 high-res units = one WheelUp/WheelDown press+release. Keep the remainder. Horizontal scroll has no D-Bus button; document it as unsupported.
- Buttons: Moonlight 1..5 → Left, Middle, Right, Side, Extra.
- Absolute mode: convert Moonlight’s
- Touch: map Moonlight touch events to MultiTouch events with slot allocation up to
MaxSlots; if the console lacks MultiTouch, fall back to absolute mouse emulation for single touch. - Keyboard state: release all pressed keys and buttons when a client disconnects, so no key stays stuck in the guest.
- All D-Bus calls are async on the GMainContext thread, preserving event order, so a slow bus never blocks Sunshine’s input thread.
Tests
- Unit, with a fake QEMU recording calls: VK→qnum table coverage for the full 104-key layout plus media keys; mouse abs scaling at different console sizes; rel accumulation; scroll remainder math; IsAbsolute switching; touch slot allocation; release-all on disconnect.
- E2E: a Linux test guest runs
evtestor a tiny evdev logger on its serial console (read from QEMU-serialto a file). The test client injects “a”, Shift+“A”, mouse move to a point, and a click, and the test asserts the guest logged the matching events.
Done when
REQ-INP-001..003 are linked to passing tests, typing and mouse work manually from a real Moonlight client in both PS/2 (relative) and virtio-tablet (absolute) guests, and anvil requirement status passes. PR Closes #5.
Progress on phase-1 (b5ae5cd6, 82c2d38e, cbc1a543):
- Keyboard, mouse and touch with capture = qemu go to the captured console through Keyboard/Mouse/MultiTouch (dispatch branch in virtualhid_input.cpp; gamepads unchanged). VK -> KEY_* table ported from libvirtualhid and extended; KEY_* -> qnum from keycodemapdb (third-party/qemu-keycodemapdb, provenance README).
- IsAbsolute tracking with abs<->rel conversion, scroll remainder, buttons 1..5, touch slots with mouse fallback, release-all on streaming stop, async ordered calls on a non-blocking GLib thread.
- 27 unit tests (fake QEMU records input calls) linked to REQ-INP-001..003; E2E_INPUT=absolute|relative|touch passes on QEMU 11.1.1 and 8.2.2 with the Linux guest’s evdev log.
- Open: manual check with a real Moonlight client (no GUI client on this host). Known limits: QEMU always exports MultiTouch, so VMs without virtio-multitouch-pci need native_pen_touch = disabled; with virtio-multitouch-pci QEMU sends mouse buttons to the touch device and drops wheel events. Details in JOURNAL.md.