Phase 1d: multi-VM deployment (ports, mDNS, systemd, libvirt)

Section C: libvirt investigation (read-only, answered before coding)

Sources read: libvirt git at tag v12.7.0 (plus git show v10.0.0: for Ubuntu 24.04’s version), QEMU 11.1.1 ui/dbus*.c and monitor/qmp-cmds.c, dbus 1.14.10, glib 2.80 gdbusobjectmanagerserver.c, and the Ubuntu noble debs libvirt-daemon-driver-qemu / libvirt-daemon-system 10.0.0-2ubuntu8.16 (unpacked, not installed).

Tags: [src] = read in the source. [inf] = inference, not tested.

Nothing here was run against a real libvirt. The dev host (WSL2) has no libvirt installed; no daemon was started and no VM was run through libvirt. Everything is static source reading.

1. Does libvirt start a private dbus-daemon per domain? Yes, unless p2p='yes' or address= is used

• When [src]: qemuExtDevicesStart() (src/qemu/qemu_extdevice.c:238-243) calls qemuDBusStart() for every <graphics type='dbus'> with p2p off and no user-given address= (fromConfig, domain_conf.c:12481). The same daemon also serves slirp helper / dbus-vmstate. <audio type='dbus'> never starts a daemon itself; it becomes -audiodev dbus bound to the display with -display dbus,...,audiodev=audioN (qemu_command.c:8765, 8334), and validation requires the linked audio to be dbus (qemu_validate.c:5005).
• Command line [src] (qemu_command.c:8742-8778): bus mode -display dbus,addr=unix:path=<sock>[,gl=on|off][,rendernode=..][,audiodev=..]; p2p mode -display dbus,p2p=on,.... An empty address is filled by qemuDBusGetAddress() (qemu_process.c:5462) and appears in the live XML.
• Socket path [src]: <stateDir>/dbus/<shortName>-dbus.sock (qemu_dbus.c:56, qemu_conf.c:248). shortName is "%d-%.20s" from the domain id and name (domain_conf.c:32027), so the path changes on every domain start.
  • system: /run/libvirt/qemu/dbus/<id>-<name20>-dbus.sock
  • session (qemu:///session): $XDG_RUNTIME_DIR/libvirt/qemu/run/dbus/... (qemu_conf.c:209)
  • discoverable via live virsh dumpxml (address='unix:path=...'), virsh domdisplay --type dbus (prints dbus+unix://<path>, since 8.4.0, tools/virsh-domain.c:12221), or the live XML a qemu hook gets on stdin.
• Owner / permissions [src]:
  • directory 0770 cfg->user:cfg->group (qemu_driver.c:651, spec %attr(0770, qemu_user, qemu_group)); Debian/Ubuntu libvirt-qemu:kvm, Fedora qemu:qemu.
  • config 0600 (qemu_dbus.c:154), chowned/labelled for the domain.
  • daemon runs via qemuSecurityCommandRun() with the domain’s labels: DAC uid/gid (security_dac.c:2392-2411), SELinux svirt_t:MCS (security_selinux.c:3110), AppArmor libvirt-<uuid> (security_apparmor.c:548); capabilities cleared.
  • socket: dbus-daemon chmod 0777 (dbus-sysdeps-unix.c:1254), then libvirt chowns and labels it to the domain (qemu_dbus.c:311). Net: srwxrwxrwx libvirt-qemu:kvm inside a 0770 dir [inf: combination of the two code paths].
  • bus policy (qemuDBusWriteConfig, qemu_dbus.c:119-155): EXTERNAL auth, allow-all send/receive/own, and <allow user='root'/> only when privileged and only since 11.2.0 (commit 37429530c). No other <allow user|group>.
  • dbus-daemon’s connect rule (bus/policy.c:460): only the bus owner’s uid, then <allow user|group> rules. So only the domain’s DAC user and (11.2.0+) root can connect. kvm group membership passes the directory but dbus-daemon still rejects the connection [src]. On 10.0.0 (Ubuntu 24.04) even root is refused.
• Lifetime [src]: started in qemuProcessLaunch (qemu_process.c:8368), before the start hook (8384) and before QEMU exec; stopped in qemuProcessStop via qemuDBusStop() (9404), before the stopped hook (9473). Survives a virtqemud restart (<dbusDaemon/> in status XML, qemu_domain.c:2731, 3387; reconnect qemu_process.c:9900). Binary from qemu.conf dbus_daemon since 9.4.0.

2. Versions [src: git describe --contains on introducing commits, NEWS.rst, formatdomain.rst]

3. How can a third-party process (Sunshine as its own user) get access?

• a) socket + group membership: does not work on system libvirt [src] (see §1). The .conf is rewritten on every start.
  • Remaining bus-mode options: run Sunshine as the domain’s DAC user, or give the domain <seclabel type='static' model='dac'><label>+UID:+GID</label> [inf]; run Sunshine as root (11.2.0+ only, not recommended) [src]; wrap dbus_daemon to add <allow group> (host-wide hack) [inf]; session libvirt with Sunshine as the same user [inf: uid matches].
• b) address= to a bus Sunshine owns: libvirt starts no daemon (qemu_extdevice.c:239) and does not label/authorise that path for QEMU [src] (virt-aa-helper only adds rendernode and listen sockets, virt-aa-helper.c:1053-1082). Under AppArmor/SELinux the confined QEMU is likely denied [inf].
• c) p2p via virDomainOpenGraphicsFD(dom, idx, 0): works by design, including svirt [src]:
  • libvirt makes the socketpair with the domain’s socket label (qemuSocketPair, qemu_driver.c:15237), passes one end to QEMU with QMP getfd fdname=graphicsfd + add_client protocol=@dbus-display fdname=graphicsfd skipauth=false (qemu_monitor.c:3149, qemu_monitor_json.c:3840, 4822), returns the other end over RPC.
  • Ubuntu’s AppArmor abstraction allows unix (send, receive) type=stream addr=none peer=(label=virtqemud|libvirtd) (abstractions/libvirt-qemu:247-250, 10.0.0 deb) - exactly this case.
  • Access control: needs a qemu:///system connection with domain:open_graphics. Default ACL = full RW access = libvirt group on Ubuntu (root-equivalent). With access_drivers = ["polkit"] it can be narrowed to org.libvirt.api.domain.open-graphics [inf: polkit rules not read].
  • Hook caveat: docs/hooks.rst - a hook must not call back into libvirt (deadlock). The hook should only systemctl start --no-block; Sunshine calls OpenGraphicsFD itself.
  • Unsupported alternative: virsh qemu-monitor-command --pass-fds (taints the domain).
• d) SELinux/AppArmor for bus mode [inf, untested]: the per-VM dbus-daemon runs confined; an unconfined client is generally allowed by SELinux; AppArmor pathname unix-socket mediation between confined daemon and unconfined peer is not covered by an explicit rule I could find - needs testing on a real Ubuntu host.

4. Does libvirt register as a display client? Do listeners coexist?

• libvirt itself registers no Console listener [src] (no RegisterListener in src/). Since 11.2.0 it holds a plain bus connection (qemu_dbus.c:88) for helper control and dbus-vmstate.
• But libvirt spawns display clients on the private bus: qemu-rdp (11.2.0+, needs non-p2p dbus, qemu_validate.c:5050) and qemu-vnc (12.5.0+, qemu_process.c:5611). QEMU 11.1.1 tools/qemu-vnc/display.c:307 calls RegisterListener [src]; qemu-rdp doing so is [inf].
• Multiple listeners coexist [src]: ui/dbus-console.c:45 keeps an unbounded GPtrArray *listeners; each RegisterListener appends a peer connection (line 332); removed when its connection closes. Any client’s SetUIInfo affects the shared console [inf].
• One D-Bus display per VM [src]: QEMU refuses a second dbus-display (ui/dbus.c:260), libvirt allows one dbus graphics (qemu_command.c:10504). Bus mode and p2p cannot be combined on one VM.
• p2p has a single control client [src]: dbus_display_add_client() (ui/dbus.c:376) calls g_dbus_object_manager_server_set_connection(), which (glib gdbusobjectmanagerserver.c:294-303) unexports everything from the previous connection. A second p2p client (e.g. virt-viewer) takes VM/Console/Mouse/Keyboard/Audio away from Sunshine’s control connection; already-registered listeners survive [inf from the code].

5. p2p sequence and reuse of QemuSession

• Sequence [src]: QEMU with -display dbus,p2p=on[,audiodev=..]. QMP getfd fdname=X (fd via SCM_RIGHTS), then add_client protocol=@dbus-display fdname=X (monitor/qmp-cmds.c:138-160). In bus mode add_client fails (“p2p connections not accepted in bus mode”). QEMU is the AUTHENTICATION_SERVER with DELAY_MESSAGE_PROCESSING (ui/dbus.c:416-429); client must be auth client. Via libvirt: virDomainOpenGraphicsFD(dom, idx, 0) does the same.
• Reuse: everything above the connection is transport independent (Console proxies, ConsoleIDs, RegisterListener socketpair, listener thread, frame store; QEMU names p2p listeners “p2p”, dbus-listener.c:1000; audio handles p2p, audio/dbusaudio.c:486). Required changes [inf: glib API rules + session.cpp reading]: connection from fd with AUTHENTICATION_CLIENT and no MESSAGE_BUS_CONNECTION; proxies with NULL name; no name_owner check or name watch (only closed); a new fd per reconnect.

Conclusion

• p2p is required for the common case (system libvirt, Sunshine as its own user). Socket + group cannot work.
• Plan for this issue: (A) raw QEMU / session libvirt as the same user → bus mode as today; (B) system libvirt → <graphics type='dbus' p2p='yes'> + a p2p connect path in QemuSession, with the fd obtained through libvirt virDomainOpenGraphicsFD or QMP getfd+add_client. The hook only does systemctl start/stop --no-block.
• To verify on real hosts (not possible here): Ubuntu with AppArmor enforcing and Fedora with SELinux enforcing, both routes; virt-manager manual check.

Progress on phase-1 (not closing)

Commits: 50eb6e47, 1c97e202, 4ffcec4b, 4fc4de64, e6a28789, e431a566, cfee508e.

A. Multi-instance. Two VMs, two Sunshine instances and two clients streamed at the same time with only config files. The one collision was the mDNS instance name, which came from the host name (the second instance became “ #2”). It now follows sunshine_name, and falls back to the host name when that is empty.

• tests/e2e/qemu/multi_instance.sh checks:
  • distinct mDNS names on each instance’s port, through a private rootless Avahi sandbox;
  • disjoint sockets inside port-5..port+21;
  • per-instance pairing;
  • each instance streams its own VM;
  • no PulseAudio sinks;
  • no shared ~/.config/sunshine state.
• It passes on QEMU 11.1.1 (Debug and Release NVENC at 1080p60) and on QEMU 8.2.2.

C. p2p (required, see the investigation comment). qemu_dbus_address now also accepts:

• qmp:<socket>: QMP getfd + add_client protocol=@dbus-display.
• libvirt:<domain>[?uri=]: libvirt.so.0 is loaded at run time. It uses virDomainOpenGraphicsFD for p2p='yes', otherwise the bus address from the live XML.

Tests:

• Unit tests use a fake p2p QEMU, a fake QMP server and a fake libvirt module.
• The E2E runs over QMP p2p (pattern, audio, input, reboot, reconnect) pass on real QEMU. The pattern run also passes on 8.2.2.

B. systemd + hook. In packaging/linux/qemu:

• sunshine-qemu@.service and a user unit
• sysusers file
• sunshine-qemu-wait-vm (ExecStartPre; Sunshine checks the display only once at startup and doesn’t recover when the VM appears later)
• sunshine-qemu-port
• libvirt-hook-qemu (opt-in = /etc/sunshine-qemu/<domain>.conf; uses systemctl --no-block)
• example.conf

tests/packaging/linux/qemu/test_sunshine_qemu.sh covers systemd-analyze verify for both units, the hook with a fake systemctl, the wait helper with fake gdbus/virsh, the port helper and shellcheck.

Docs. docs/qemu.md covers raw QEMU, several VMs, systemd, libvirt XML and permissions, the hook, virt-manager and troubleshooting. configuration.md is updated too.

Not verified here: there is no libvirt on this host. Still open:

• a real libvirt domain started from virt-manager (manual check);
• virDomainOpenGraphicsFD against a real libvirt;
• the polkit rule;
• SELinux and AppArmor;
• units under a real systemd instance;
• a GUI Moonlight client discovering both hosts.