Anvil
fangorn/sunshine-qemu public
ref:main
← EPIC Epic: Stream QEMU VMs to Moonlight via a Sunshine QEMU D-Bus backend #1

Phase 0: dev environment, E2E harness, QemuSession core, capture spike #2

closed Opened by cole.christensen@gmail.com
REQ-CAP-001 REQ-CMP-001 REQ-E2E-001 REQ-NFR-001

Pull requests

Closed by (1)
References (1)

Links

Parent

Phase 0: dev environment, E2E harness, QemuSession core, capture spike

Parent: #1. Read the epic’s START HERE and Architecture sections first.

Primary journey first: this issue builds the end-to-end test that every later phase has to keep passing, plus the minimum code needed to make it pass on the simplest path: shared memory, software encoding, bus mode.

Requirements

  • REQ-E2E-001: a Moonlight client streams a QEMU VM with no guest agent
  • REQ-CAP-001: connect to the QEMU D-Bus display and discover consoles
  • REQ-CMP-001: unchanged behavior when capture != qemu
  • REQ-NFR-001 (draft): measure latency and propose a final target

Steps

0. Environment and baseline (no code changes)

  • Install Sunshine’s Linux build dependencies. The most reliable list is scripts/linux_build.sh or .github/workflows/ci-linux.yml; also install libglib2.0-dev (gio ≥ 2.64), dbus-daemon, qemu-system-x86 (≥ 9.0; 11.x preferred), ovmf.
  • cmake -B cmake-build-debug -G Ninja -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DBUILD_DOCS=OFF && ninja -C cmake-build-debug
  • Record the baseline: cmake-build-debug/tests/test_sunshine passes unmodified. If anything fails on a clean upstream tree, write it in JOURNAL.md so it isn’t blamed on us.
  • Check that QEMU was built with D-Bus display: qemu-system-x86_64 -display help | grep dbus and -audiodev help | grep dbus. If not, build QEMU from source with --enable-dbus-display and note the version.

1. Guest test image

  • Choose a tiny deterministic guest whose screen shows known content with no agent. Recommended: a small Alpine or buildroot image, or even just the OVMF/SeaBIOS boot screen for the first test.
    • Best option: a guest that draws a known pattern on the framebuffer, e.g. colored quadrants plus a frame counter, so pixel assertions are exact. Keep generation scripted under tests/e2e/qemu/guest/ and never commit large binaries; build or download them into a cache dir in the test script.
  • Standard invocation (put it in a script, tests/e2e/qemu/run_vm.sh): 
    dbus-daemon --session --print-address --fork --address=unix:path=$T/bus.sock

    qemu-system-x86_64 -accel kvm -accel tcg -m 512 -name e2e-guest \

      -device virtio-vga -display dbus,addr=unix:path=$T/bus.sock \

      -audiodev dbus,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 \

      -device virtio-tablet-pci -device virtio-keyboard-pci ...
    Use virtio-vga without GL here (the shared-memory path). The DMABUF path (virtio-vga-gl / -display dbus,gl=on) is for #3.

2. E2E harness (write first; it must fail red)

  • Headless Moonlight test client in tests/e2e/moonlight_client/:
    • Stream through third-party/moonlight-common-c (already a submodule): LiStartConnection with a decoder callback that decodes H.264/HEVC with FFmpeg (already a dependency) and keeps the last decoded frame.
    • Pairing and launch need GameStream HTTP (serverinfo, pair, applist, launch). moonlight-common-c doesn’t include this; moonlight-embedded’s libgamestream (GPL-3) does. Vendor or port a minimal version. Don’t use a GUI Moonlight.
    • Automate the PIN: the client generates it, and the harness sends it to Sunshine with POST https://localhost:<port+1>/api/pin (creds created via sunshine --creds user pass on a temp config).
  • Harness script tests/e2e/qemu/e2e_stream.sh (or a gtest-driven fixture):
    1. start the dbus-daemon and QEMU guest
    2. start Sunshine with a temp config: capture = qemu, qemu_dbus_address = unix:path=$T/bus.sock, encoder = software, a unique port, temp file_state/credentials_file/pkey/cert
    3. pair, launch the default “Desktop” app, and receive ≥ N frames
    4. assert that the decoded frame matches the guest’s known pattern within a tolerance (PSNR or per-quadrant mean color)
    5. tear everything down, including on failure
  • Annotate with // @tag requirements: [REQ-E2E-001] and link via anvil requirement link.

3. Vendored XML and generated bindings

  • third-party/qemu-dbus-display/: preprocessed dbus-display1.xml (produced with QEMU scripts/xml-preprocess.py, HOST_OS=linux), plus a README noting the QEMU commit, the exact command, and the license.
  • CMake: SUNSHINE_ENABLE_QEMU option; find_package(PkgConfig) + pkg_check_modules(GIO gio-2.0>=2.64 gio-unix-2.0); add_custom_command running gdbus-codegen --interface-prefix org.qemu. --c-namespace QemuDBus --glib-min-required 2.64 --generate-c-code dbus-display1. Compile the generated C as C, define SUNSHINE_BUILD_QEMU.

4. QemuSession core (TDD)

src/platform/linux/qemu/session.{h,cpp}:

  • Owns a GMainContext and a dedicated thread running a GMainLoop. All GDBus calls go through this context, and the public API is thread-safe.
  • connect(address): bus connection, proxy for /org/qemu/Display1/VM on name org.qemu, read Name, UUID, ConsoleIDs. For each console, read Label, Type, Width, Height, Interfaces.
  • register_listener(console_id, callbacks): socketpair(AF_UNIX, SOCK_STREAM), hand one end to Console.RegisterListener(h) with a GUnixFDList, and on our end run g_dbus_connection_new_sync(..., G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT) (QEMU is the auth server; see QEMU ui/dbus-console.c ~L312). Export a QemuDBusDisplay1Listener skeleton at /org/qemu/Display1/Listener. The Interfaces property must list org.qemu.Display1.Listener.Unix.Map (and ...ScanoutDMABUF2 once #3 supports it).
  • Detect VM disappearance (connection closed / name vanished) and report it so capture can return capture_e::reinit or error.
  • Unit tests: run a fake QEMU in-process. Export VM/Console skeletons on a private GTestDBus or a dbus-daemon started by the fixture, then drive Scanout/Update calls into our listener and assert what’s received. This gives fast, deterministic tests without real QEMU. Tag REQ-CAP-001.

5. Minimal capture spike (enough for REQ-E2E-001)

  • source::QEMU in src/platform/linux/misc.cpp: verify_qemu() returns true only when config::video.capture == "qemu" and a connection succeeds. Never auto-detect (auto-detect would break REQ-CMP-001). Add dispatch in display() and display_names(); display names are console IDs/labels.
  • New config keys in src/config.{h,cpp} with doxygen, plus en-only localization entries and web UI if the config-consistency integration test (tests/integration/test_config_consistency.cpp) requires them: qemu_dbus_address (string) and qemu_console (default the first graphical console).
  • display_t impl, shared-memory only: handle Scanout (full frame), Update (damage rect, blit into the retained frame), ScanoutMap/UpdateMap (mmap the fd; handle offset), and pixman formats x8r8g8b8/a8r8g8b8 at minimum. Hand frames out as mem_type_e::system images through pull_free_image_cb/push_captured_image_cb, paced to the client framerate by repeating the last frame when there’s no damage.
  • Unit tests with the fake QEMU from step 4.
  • REQ-CMP-001 test: with capture unset or kms, verify_qemu() is false and no D-Bus connection is attempted.

6. Latency measurement (REQ-NFR-001)

  • Add debug-level timing from listener callback receipt to encoder packet out, using the existing logging::time_delta_periodic_logger style.
  • Record p50/p95 at 1080p60 for shm+software, shm+VAAPI/NVENC if available, and later DMABUF (#3). Update REQ-NFR-001 with a measured target and set it active.

Done when

  • E2E test green locally: pair → stream → pixel assertion on a real QEMU guest
  • Unit tests for session and capture green; upstream tests still green
  • doxygen and clang-format clean; build passes with SUNSHINE_ENABLE_QEMU=ON and OFF
  • Requirements linked; anvil requirement status passes
  • PR on Anvil with Closes #2

Risks / notes

  • If GameStream HTTP pairing in the test client turns into a rabbit hole, stop and escalate. Don’t mock the Sunshine side of the stream; the point of this test is the real pipeline.
  • TCG without KVM is fine for correctness but useless for latency numbers.
Back to issues