fangorn/sunshine-qemu
public
ref:main
Phase 1a: qemu capture source (shm + DMABUF zero-copy, mode changes, cursor) #3
open
Opened by cole.christensen@gmail.com
Pull requests
Links
Phase 1a: qemu capture source, complete (shm + DMABUF zero-copy, mode changes, cursor)
Parent: #1. Depends on #2 (QemuSession, shm capture spike, E2E harness).
Requirements
- REQ-CAP-002: capture shared-memory scanouts (finish what #2 started; all pixman formats QEMU emits)
- REQ-CAP-003: zero-copy DMABUF scanouts into VAAPI and CUDA encoders
- REQ-CAP-004: survive guest mode changes, Disable, and reboot
- REQ-CAP-005: composite the guest cursor into the stream
- REQ-CAP-006: selectable via
capture = qemu; console selection - REQ-NFR-001 (draft): capture-to-packet latency; this phase removes the capture pacing delay and records DMABUF latency
Background (verified)
- DMABUF scanouts need a GL-capable QEMU display:
-device virtio-vga-gl -display dbus,gl=on[,rendernode=/dev/dri/renderD128], orvfio-pci ...,display=onfor mdev/vGPU. QEMU sendsScanoutDMABUF(fd, w, h, stride, fourcc, modifier, y0_top)or, if our Listener lists the interface inInterfaces,ScanoutDMABUF2(ah fds, x, y, w, h, au offsets, au strides, num_planes, fourcc, backing_w, backing_h, modifier, y0_top), thenUpdateDMABUF(x, y, w, h)on damage. - Sunshine already imports DMABUFs:
egl::surface_descriptor_t(src/platform/linux/graphics.h:496: fds[4], fourcc, modifier, pitches[4], offsets[4]) andegl::import_source. See howkmsgrab.cppbuilds descriptors and implementsmake_avcodec_encode_device()for vram (kmsgrab.cpp:1731-1745:va::make_avcodec_encode_device(..., true),vk::make_avcodec_encode_device_vram, cuda). Model the qemu vram path on the kmsgrab vram display class. Don’t invent a new import path. - With DMABUF, the GPU that QEMU renders on must be the one Sunshine encodes on (same render node). Pick the render node from
rendernode=/adapter_nameconfig and fail with a clear error on mismatch. - Host for DMABUF work: native Linux with a DRM render node (
/dev/dri/renderD*). A GPU under WSL2 is not enough (checked 2026-09-12 on an RTX 4090 under WSL2): there’s no/dev/dri, and QEMU refuses-device virtio-vga-gl -display dbus,gl=onwith “egl: no drm render node available”. WSL2 can still run the shm path with NVENC (CUDA build). - Capture pacing dominates shm latency (measured in #2, see JOURNAL.md on
issue-2-phase0).qemu::display_t::capture()wakes on a fixed tick of one frame interval (platf::handle_pacing) and copies whatever damage arrived since. Damage lands uniformly in the interval, so it waits 0–16.7 ms at 60 fps. At 1080p60 the host latency was p50 11.1 ms / p95 20.4 ms with NVENC and p50 14.1 / p95 21.4 ms with software x264: pacing is ~8 ms p50 / ~16 ms p95 of that, and NVENC ~3 ms.
Tasks
- Two display classes, like kmsgrab:
qemu::display_ram_t(shm,mem_type_e::system) andqemu::display_vram_t(DMABUF → VAAPI/CUDA/Vulkan).platf::display()picks byhwdevice_type. If QEMU only sends shm while a vram encoder is requested, upload to GPU through the existing RAM→encode path rather than failing. - Add
org.qemu.Display1.Listener.Unix.ScanoutDMABUF2to the Listener’sInterfaces; handle both the DMABUF and DMABUF2 methods. Honory0_top(flip),x/yoffsets and backing size. - FD lifetime: dup/close the fds received in
GUnixFDListcorrectly. A new Scanout* replaces the previous buffer; release the old EGL image only after the encoder is done with it. - Pixman formats: at minimum
x8r8g8b8,a8r8g8b8,x8b8g8r8,a8b8g8r8,r5g6b5(fallback VGA). Unsupported formats are converted with pixman or logged once and ignored. - Mode changes (REQ-CAP-004): a size change in Scanout*/ScanoutMap returns
capture_e::reinitso the pipeline re-creates the encoder at the new size.Disableshows a black frame and keeps the session alive. The guest rebooting (Disable, then a firmware-size scanout, then an OS-size scanout) must not end the stream. - Cursor (REQ-CAP-005): store the
CursorDefineARGB image and hotspot and theMouseSet(x, y, on)state. When*cursoris true, blend onto frames: on the CPU for RAM, and with the existing cursor blend shader ingraphics.cppfor vram (kmsgrab/wlgrab already draw cursors; reuse that). - Console selection (REQ-CAP-006): #2 already selects consoles with Sunshine’s existing
output_name(id or label), anddisplay_names()returns the graphical console ids; there is noqemu_consolekey. Finish it: multi-head guests pick one console per stream; add tests for label and id selection throughoutput_name. - Damage-driven capture (REQ-NFR-001): replace the fixed-tick loop in
qemu::display_t::capture()with a wait on the frame store (condition variable, notified bytouch_locked()) that wakes as soon as new damage arrives, rate-limited so frames are never pushed more often than the client’s frame interval. Keep repeating the last frame at the client’s fps when there’s no damage, and never block the listener thread on the encoder. Target: shm + NVENC p95 ≤ 12 ms at 1080p60 (from 20.4 ms). Apply the same wakeup to the DMABUF display class.
Tests
- Unit, with the fake QEMU from #2: each Scanout variant; Update damage math; format conversion;
y0_topflip; cursor composite pixel checks; size change →reinit; Disable → black frame. The DMABUF unit tests can build DMABUFs withudmabuf(/dev/udmabuf) or GBM when available;GTEST_SKIP()with a clear message otherwise, and document which requirement the skip affects. - Unit, pacing: damage delivered mid-interval is pushed within a few ms, not at the next tick; bursts of damage still push at most one frame per client frame interval; no damage still repeats nothing (the encoder’s minimum FPS covers static screens).
- E2E: extend the #2 harness with a
gl=onvariant (hardware-gated; skipped in CI without/dev/dri) and a reboot-during-stream test (QMPsystem_reset, then assert the stream keeps producing frames). - Record latency for REQ-NFR-001 with
E2E_WIDTH=1920 E2E_HEIGHT=1080 E2E_FPS=60 E2E_FRAMES=600andE2E_ENCODER=nvenc(orvaapi): the shm path before and after damage-driven capture, and the DMABUF path. Update REQ-NFR-001 with the targets and set itactive.
Done when
All REQ-CAP-00x requirements are linked to passing tests, the E2E test (shm) is green, the DMABUF E2E is green on a native Linux GPU host with /dev/dri (note the GPU, driver and distro in the PR), the shm + hardware encoder p95 at 1080p60 is ≤ 12 ms, and anvil requirement status passes. PR Closes #3.
Progress on phase-1 (52f749fe..1f64e75f), from WSL2 with an RTX 4090 (no /dev/dri):
- Pixman formats: every packed 16/24/32 bpp format, r5g6b5 included, for Scanout/Update and ScanoutMap/UpdateMap.
- Damage-driven capture: 1080p60, NVENC, shm map. p95 went from 17.8-21.8 ms to 4.4-5.4 ms on a quiet host, and up to 10 ms under load. p50 went from 11-14 ms to about 4 ms. The <= 12 ms target is met. REQ-NFR-001 description is updated and stays draft until DMABUF is measured.
- REQ-CAP-004: Disable shows black, mode changes reinit. E2E_RESET_AFTER_FRAMES reboots the guest with QMP system_reset mid-stream. It passes on QEMU 11.1 and 8.2, with NVENC too. QEMU 8.2 never sends the text-mode scanout, so run it with E2E_RESET_MIN_DISPLAYS=1.
- REQ-CAP-005: CPU cursor compositing is unit-tested. The vram path hands the cursor to the existing GL shader, which is untested.
- REQ-CAP-006: output_name selects by id or label. Labels didn’t work through video::refresh_displays before.
- REQ-CAP-003: ScanoutDMABUF/ScanoutDMABUF2/UpdateDMABUF, display_vram_t (kmsgrab-style), y_invert flip on the GL path, RAM fallback and the render node check are unit-tested with memfds. Still unverified: the EGL import test and E2E_GL=1 skip here. That needs native Linux with /dev/dri.
Details in JOURNAL.md. Not closing: the DMABUF E2E and DMABUF latency are still open.