@@ -330,3 +330,108 @@
- Not done here: a manual check with a real Moonlight client (no GUI client on this host), and
guests with more than two channels end to end (virtio-sound with a 5.1 guest stream).
## 2026-09-12 — Phase 1c (#5): keyboard, mouse and touch input
Same host (WSL2, KVM). Tested with QEMU 11.1.1 and the distro QEMU 8.2.2; both export
`org.qemu.Display1.Keyboard`, `Mouse` and `MultiTouch` on every console.
### Design
- **Routing.** `platf::input()` creates a `qemu::input_t` next to the libvirtualhid context when
`capture = qemu`; `move_mouse`, `abs_mouse`, `button_mouse`, `scroll`, `hscroll`,
`keyboard_update`, `unicode`, `touch_update` and `pen_update` in `virtualhid_input.cpp` have a
five-line `#ifdef SUNSHINE_BUILD_QEMU` dispatch at the top. Gamepads are untouched. With capture
= qemu `get_capabilities()` advertises pen/touch from `native_pen_touch` alone (no host touchscreen
needed), and `streaming_will_stop()` releases everything the QEMU inputs hold.
- **Console.** The capture calls `qemu::set_capture_console(session, console, store)` after it
registers; the input sends to that session and console and reads QEMU's `MouseSet` from that frame
store. Before any stream it uses the console `output_name` selects, like the capture will. A
capture re-created on the same console (mode change) only swaps the store; a different console or
session makes the input reconnect (and release what it held on the old console).
- **Never blocking.** Connecting (session + proxies with a property fetch) runs on a worker thread
of `input_t`; events that arrive before it is connected are dropped with a rate-limited debug log.
`session_t::open_input()` returns a `console_input_t` whose calls go into a FIFO drained on a third
GLib thread per session (created on the first `open_input`), which only makes asynchronous calls:
input never waits behind a blocking `RegisterListener` on the bus thread or behind display updates
on the listener thread. Property changes (`IsAbsolute`, `Width`/`Height`, `MaxSlots`) are delivered
on that thread into atomics. `QemuInputTest.InputNeverWaitsForABusyQemuAndKeepsItsOrder` stalls the
fake QEMU's main loop for 1 s: 100 moves return in well under 200 ms and arrive in order afterwards.
- **Keymap.** The Windows VK -> Linux step is not reusable code in this tree: Sunshine's Linux
table is `key_code_to_linux` inside an anonymous namespace of libvirtualhid's
`uhid_backend.cpp`. It is ported (MIT, commit noted) and extended with the keys it lacks (Pause,
Menu, media, browser, IME keys) from keycodemapdb's Win32 column. Linux -> qnum (and qnum -> Linux
for the tests) are unmodified `keymap-gen` output from keycodemapdb at the revision QEMU 11.1.1
pins, in `third-party/qemu-keycodemapdb` with a provenance README (GPL-2.0-or-later or BSD-3).
Moonlight has no extended-key bit (Sunshine strips the high byte), so the keypad Enter is Enter;
`SS_KBE_FLAG_NON_NORMALIZED` doesn't change the physical key. Keys without a QEMU number are logged
once per VK. A key held by several VKs (Sunshine's synthetic `VK_SHIFT` while the client holds
`VK_LSHIFT`) is released with the last of them; repeated presses are passed on as key repeat.
- **Mouse.** Absolute: `(x - offset) * console / touch_port`, floored and clamped to the last pixel
(QEMU refuses `x >= Width`). Relative guest + absolute input: motion from the believed pointer
position with a sub-pixel remainder; the position comes from `MouseSet` only for a **visible**
cursor and only after 100 ms without our own motion (reports during motion are stale). Without a
position the move is dropped. Absolute guest + relative input: a tracked position (last absolute
target, visible `MouseSet`, else the console center) plus the delta. Scroll: 120 units per wheel
press+release, remainder kept. `hscroll` and `unicode` log once and do nothing.
- **Touch.** Contacts get the lowest free slot below `MaxSlots`; cancel uses the contact's last
position (Moonlight only fills the pointer id). Without the `MultiTouch` interface, the first
contact drives the mouse (position + left button); pens always do.
### Findings
- QEMU sends `MouseSet(0, 0, hidden)` for virtio-gpu right after registration. The first version
took that as the pointer position, and Sunshine's connect nudge (`move_mouse(1,1)`,
`move_mouse(-1,-1)`) teleported the guest pointer to the top-left corner (seen in the absolute
E2E's evdev log: `ABS_X` 16409, 25, 0). Fixed by ignoring hidden-cursor reports; the unit test now
covers it.
- QEMU exports `MultiTouch` on every console whether or not the VM has a multi-touch device, so
touches from a native-touch Moonlight client are lost in a VM without `virtio-multitouch-pci`.
Sunshine can't detect the device over D-Bus. Documented: disable `native_pen_touch` for such VMs
(Moonlight then sends mouse input); the first touch logs a hint.
- With `virtio-multitouch-pci` next to `virtio-tablet-pci`, QEMU routes **button** events to the
multi-touch device (its handler has the BTN mask and wins `qemu_input_find_handler`) and wheel
events are lost: the absolute sequence against such a VM logged `BTN_LEFT` on "QEMU Virtio
MultiTouch" and no `REL_WHEEL`. Hence a separate `E2E_INPUT=touch` mode and a note in the docs.
- Sunshine's `release` path: `input::reset()` already releases the keys and buttons it tracks
through the platform calls on every disconnect; the QEMU input additionally releases its own
tracked keys, buttons and touch slots in `streaming_will_stop()` (last client gone). A per-client
hook would need a new `platf` function on every OS, so it wasn't added.
### TDD evidence and deviations
- Red checks: the five `QemuKeymapTest` tests against a stub keymap (4 failed, the "no mapping"
test passes trivially); 20 of 22 `QemuInputTest` tests against a stub `input_t` (the two
session-level tests were written together with `session_t::open_input()` and passed on their
first run). The E2E against the pre-change Sunshine (`cmake-build-release-cuda`, built before this
phase): `E2E_INPUT=absolute` FAIL, 0 evdev events in the guest (input went to libvirtualhid, which
has no uinput here).
- The hidden-cursor `MouseSet` fix came from the E2E log; its unit test assertion was added with the
fix, not run red first.
- Deviations from the issue text: the VK table is ported, not linked (see Design); release-all is
tied to `streaming_will_stop()` plus Sunshine's own per-disconnect releases rather than a new
per-client platform hook; touch in a VM without a multi-touch device needs `native_pen_touch =
disabled` (QEMU always exports `MultiTouch`); pens drive the mouse; `hscroll` and `unicode` only
log.
### Results
- Unit tests (cmake-build-debug, xvfb-run): `Qemu*` 120 passed, 2 skipped (the DMABUF GPU tests),
3 of 3 with `--gtest_repeat=3`; `QemuInput*` + `QemuKeymap*` (27 tests) 5 of 5 with
`--gtest_repeat=5`.
- Full `test_sunshine`: Debug 691 tests, 675 passed, 16 skipped, 0 failed tests;
`cmake-build-noqemu` 569 tests, 555 passed, 0 failed tests (same as the baseline). Both exit 1
only for the known AudioTest/MouseHIDTest/EncoderTest setup failures.
- E2E, KVM, Debug build, software encoding, all PASS:
| Mode (`E2E_INPUT`) | QEMU 11.1.1 | QEMU 8.2.2 | Guest log |
|--------------------|-------------|------------|-----------|
| absolute (virtio-tablet + keyboard) | PASS | PASS | keys 30/42 in order on "QEMU Virtio Keyboard"; ABS 16383,16383 then 18943,18431 (center + 100,50 through the rel->abs conversion); BTN_LEFT 1/0; REL_WHEEL 1 |
| relative (PS/2, vmport=off) | PASS | PASS | keys on "AT Translated Set 2 keyboard"; REL_X sum 120, REL_Y sum -75; no ABS; BTN_LEFT 1/0; REL_WHEEL 1 |
| touch (virtio-multitouch) | PASS | PASS | ABS_MT_POSITION 8191,24575 then 16383,16383; tracking id 0 then -1 on "QEMU Virtio MultiTouch" |
The pattern and audio E2E still pass on QEMU 11.1.1 with the `run_vm.sh` change.
- clang-format dry run clean, shellcheck clean on the changed scripts (one pre-existing info note in
`e2e_stream.sh`), doxygen 1.18.0 as upstream: exit 0, no warnings.
- Not done here: typing and mouse from a real Moonlight client (no GUI client on this host),
gcov coverage.