ref:95b9419bfbf105e8e89f88e58a1cfb1a10e19c6e

feat(linux): composite the guest cursor, select consoles by label, survive reboots

- Store CursorDefine/MouseSet and blend the cursor into system-memory frames when the pipeline asks for it; cursor changes produce a frame. - Disable shows a black frame and keeps the capture running; guest mode changes and reboots re-create the display through reinit. - List the console that output_name names by label under that label, so video::refresh_displays selects it (labels fell back to the first console before). - E2E: E2E_RESET_AFTER_FRAMES reboots the guest with QMP system_reset mid-stream and requires the stream to keep producing frames and show the pattern again. The headless client gained --ready-file, --ready-frames and --wait-file for it. Refs #3 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPNw4PCgkEfhyCjQT19wsb
SHA: 95b9419bfbf105e8e89f88e58a1cfb1a10e19c6e
Author: Cole Christensen <cole.christensen@gmail.com>
Date: 2026-09-12 22:44
Parents: 225def9
14 files changed +645 -18
Type
JOURNAL.md +21 −0
@@ -141,3 +141,24 @@
26 ms at 30 fps) and passes now, `BurstsOfDamagePushAtMostOneFramePerInterval` and
`NoDamageRepeatsNoFrames` pass on both.
### Mode changes, cursor and console selection (REQ-CAP-004/005/006)
- **Reboot during a stream** (`E2E_RESET_AFTER_FRAMES=30 E2E_FRAMES=60`, Debug build, software
encoding): PASS on QEMU 11.1.1 (map path; the firmware's 720x400 text mode and the guest's
640x400 mode each re-create the display, longest gap between decoded frames 143-167 ms) and on
QEMU 8.2.2 (message path; longest gap 64 ms). On 8.2 Sunshine logged no size change during the
reset, so no display re-creation happened; I assume its VGA BIOS text mode is also 640x400 but
didn't check it with a screendump.
- Disable keeps the display and pushes a black frame; the next update at the same size brings the
picture back. Size changes still return `reinit` (unit tests replay Disable → firmware size → OS
size).
- The cursor is composited on the CPU for the RAM path (straight alpha, like the GL cursor shader's
`GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA`), clipped at the frame edges, with the hot spot
subtracted. `MouseSet`/`CursorDefine` changes count as frame changes so a moving cursor is
streamed without framebuffer damage (a hidden cursor moving doesn't). Not tested end to end: the
boot-sector guest has no hardware cursor.
- Console selection by label through `output_name` didn't actually work in the pipeline:
`video::refresh_displays` matches `output_name` against `display_names()`, which returned only
ids, so a label fell back to the first display. `qemu_display_names()` now lists the console
that `output_name` names by label under that label (all others by id).
src/platform/linux/qemu/capture.cpp +5 −4
@@ -134,7 +134,7 @@
}
std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out);
auto status = snapshot(pull_free_image_cb, img_out, cursor && *cursor);
switch (status) {
case platf::capture_e::reinit:
case platf::capture_e::error:
@@ -163,10 +163,11 @@
*
* @param pull_free_image_cb Callback providing an image to fill.
* @param img_out Filled image on `ok`.
* @param draw_cursor Whether to blend the guest cursor into the image.
* @return `ok` with a new frame, `timeout` when nothing changed, `reinit` when the size
* changed or QEMU went away, or `interrupted`.
*/
platf::capture_e snapshot(const pull_free_image_cb_t &pull_free_image_cb, std::shared_ptr<platf::img_t> &img_out) {
platf::capture_e snapshot(const pull_free_image_cb_t &pull_free_image_cb, std::shared_ptr<platf::img_t> &img_out, bool draw_cursor) {
if (!session->alive() || !store->connected() || store->width() != width || store->height() != height) {
return platf::capture_e::reinit;
}
@@ -179,7 +180,7 @@
}
std::chrono::steady_clock::time_point timestamp;
switch (store->copy_if_newer(copied_sequence, width, height, img_out->data, timestamp)) {
switch (store->copy_if_newer(copied_sequence, width, height, img_out->data, timestamp, draw_cursor)) {
case frame_status_e::new_frame:
img_out->frame_timestamp = timestamp;
damage_to_capture_logger.first_point(timestamp);
@@ -261,7 +262,7 @@
if (!session) {
return {};
}
auto names = qemu::graphic_console_names(session->vm());
auto names = qemu::graphic_console_names(session->vm(), config::video.output_name);
for (const auto &name : names) {
BOOST_LOG(info) << "qemu: found console "sv << name;
}
src/platform/linux/qemu/frame_store.cpp +81 −1
@@ -67,6 +67,36 @@
}
} // namespace
void blend_cursor(const cursor_state_t &cursor, std::uint8_t *frame, int width, int height) {
if (!cursor.drawable() || cursor.pixels.size() < (std::size_t) cursor.width * cursor.height * 4) {
return;
}
const int left = cursor.x - cursor.hot_x;
const int top = cursor.y - cursor.hot_y;
const int x0 = std::max(left, 0);
const int y0 = std::max(top, 0);
const int x1 = std::min(left + cursor.width, width);
const int y1 = std::min(top + cursor.height, height);
for (int y = y0; y < y1; ++y) {
auto src = cursor.pixels.data() + ((std::size_t) (y - top) * cursor.width + (x0 - left)) * 4;
auto dst = frame + ((std::size_t) y * width + x0) * 4;
for (int x = x0; x < x1; ++x, src += 4, dst += 4) {
const unsigned alpha = src[3];
if (alpha == 0xff) {
dst[0] = src[0];
dst[1] = src[1];
dst[2] = src[2];
} else if (alpha != 0) {
for (int c = 0; c < 3; ++c) {
dst[c] = (std::uint8_t) ((src[c] * alpha + dst[c] * (255 - alpha) + 127) / 255);
}
}
}
}
}
frame_store_t::~frame_store_t() {
std::lock_guard lock {mutex};
unmap_locked();
@@ -179,10 +209,57 @@
std::lock_guard lock {mutex};
std::ranges::fill(pixels, 0);
if (frame_width > 0) {
touch_locked();
}
}
void frame_store_t::mouse_set(std::int32_t x, std::int32_t y, bool visible) {
std::lock_guard lock {mutex};
const bool redraw = visible || cursor_state.visible;
cursor_state.x = x;
cursor_state.y = y;
cursor_state.visible = visible;
if (redraw && frame_width > 0) {
touch_locked();
}
}
void frame_store_t::cursor_define(std::int32_t width, std::int32_t height, std::int32_t hot_x, std::int32_t hot_y, std::span<const std::uint8_t> data) {
constexpr std::int32_t max_cursor_size = 1024;
if (width <= 0 || height <= 0 || width > max_cursor_size || height > max_cursor_size || data.size() < (std::size_t) width * height * 4) {
BOOST_LOG(warning) << "qemu: ignoring invalid "sv << width << 'x' << height << " cursor with "sv << data.size() << " bytes"sv;
return;
}
std::lock_guard lock {mutex};
cursor_state.width = width;
cursor_state.height = height;
cursor_state.hot_x = hot_x;
cursor_state.hot_y = hot_y;
cursor_state.pixels.assign(data.begin(), data.begin() + (std::ptrdiff_t) width * height * 4);
cursor_state.serial += 1;
if (cursor_state.visible && frame_width > 0) {
touch_locked();
}
}
cursor_state_t frame_store_t::cursor(std::uint64_t known_serial) const {
std::lock_guard lock {mutex};
cursor_state_t copy;
copy.x = cursor_state.x;
copy.y = cursor_state.y;
copy.visible = cursor_state.visible;
copy.width = cursor_state.width;
copy.height = cursor_state.height;
copy.hot_x = cursor_state.hot_x;
copy.hot_y = cursor_state.hot_y;
copy.serial = cursor_state.serial;
if (cursor_state.serial != known_serial) {
copy.pixels = cursor_state.pixels;
}
return copy;
}
void frame_store_t::disconnected() {
std::lock_guard lock {mutex};
is_disconnected = true;
@@ -224,7 +301,7 @@
return change_sequence;
}
frame_status_e frame_store_t::copy_if_newer(std::uint64_t &last_sequence, int width, int height, std::uint8_t *dst, std::chrono::steady_clock::time_point &timestamp) {
frame_status_e frame_store_t::copy_if_newer(std::uint64_t &last_sequence, int width, int height, std::uint8_t *dst, std::chrono::steady_clock::time_point &timestamp, bool draw_cursor) {
std::lock_guard lock {mutex};
if (is_disconnected) {
return frame_status_e::disconnected;
@@ -237,6 +314,9 @@
}
std::memcpy(dst, pixels.data(), pixels.size());
if (draw_cursor) {
blend_cursor(cursor_state, dst, frame_width, frame_height);
}
last_sequence = change_sequence;
timestamp = change_time;
return frame_status_e::new_frame;
src/platform/linux/qemu/frame_store.h +47 −1
@@ -29,6 +29,40 @@
};
/**
* @brief Guest cursor defined by `CursorDefine` and placed by `MouseSet`.
*/
struct cursor_state_t {
std::int32_t x {0}; ///< Pointer X position in frame pixels.
std::int32_t y {0}; ///< Pointer Y position in frame pixels.
bool visible {false}; ///< Whether QEMU reported the cursor as shown.
std::int32_t width {0}; ///< Cursor image width.
std::int32_t height {0}; ///< Cursor image height.
std::int32_t hot_x {0}; ///< Hot-spot X offset within the image.
std::int32_t hot_y {0}; ///< Hot-spot Y offset within the image.
std::uint64_t serial {0}; ///< Incremented on every `CursorDefine`; 0 before the first one.
std::vector<std::uint8_t> pixels; ///< B, G, R, A bytes (ARGB32 on little-endian hosts), straight alpha.
/**
* @brief Report whether there is a cursor to draw.
*
* @return True when the cursor is visible and has an image.
*/
[[nodiscard]] bool drawable() const {
return visible && serial > 0 && width > 0 && height > 0;
}
};
/**
* @brief Alpha-blend a cursor onto a B, G, R, X frame.
*
* @param cursor Cursor to draw; nothing happens when it isn't drawable.
* @param frame First byte of the frame, `4 * width` bytes per row.
* @param width Frame width.
* @param height Frame height.
*/
void blend_cursor(const cursor_state_t &cursor, std::uint8_t *frame, int width, int height);
/**
* @brief Reconstructs the guest framebuffer from QEMU listener calls.
* @details Listener calls arrive on the session thread; the capture thread copies frames out. The
* stored frame is always 4 bytes per pixel in B, G, R, X order with a stride of `4 * width`,
@@ -47,6 +81,8 @@
void scanout_map(fd_t fd, std::uint32_t offset, std::uint32_t width, std::uint32_t height, std::uint32_t stride, std::uint32_t format) override;
void update_map(std::int32_t x, std::int32_t y, std::int32_t width, std::int32_t height) override;
void disable() override;
void mouse_set(std::int32_t x, std::int32_t y, bool visible) override;
void cursor_define(std::int32_t width, std::int32_t height, std::int32_t hot_x, std::int32_t hot_y, std::span<const std::uint8_t> data) override;
void disconnected() override;
/**
@@ -104,10 +140,19 @@
* @param height Height of the destination buffer.
* @param dst Destination with `4 * width` bytes per row.
* @param timestamp Receipt time of the call that produced the copied frame; set on `new_frame`.
* @param draw_cursor Whether to blend the guest cursor into the copy.
* @return Poll result.
*/
frame_status_e copy_if_newer(std::uint64_t &last_sequence, int width, int height, std::uint8_t *dst, std::chrono::steady_clock::time_point &timestamp, bool draw_cursor = false);
frame_status_e copy_if_newer(std::uint64_t &last_sequence, int width, int height, std::uint8_t *dst, std::chrono::steady_clock::time_point &timestamp);
/**
* @brief Copy the cursor state.
*
* @param known_serial Serial of the image the caller already has; pixels are copied only when it differs.
* @return Cursor position, visibility, hot-spot and, when the image changed, its pixels.
*/
[[nodiscard]] cursor_state_t cursor(std::uint64_t known_serial) const;
private:
/**
* @brief Resize the frame, logging size or transport changes.
@@ -149,6 +194,7 @@
std::chrono::steady_clock::time_point change_time; ///< Receipt time of the latest change.
bool is_disconnected {false}; ///< Whether QEMU closed the listener connection.
bool logged_mapped {false}; ///< Whether the last logged scanout used the shared memory map.
cursor_state_t cursor_state; ///< Latest guest cursor.
fd_t map_fd; ///< Shared memory descriptor of the current map scanout.
const std::uint8_t *map_addr {nullptr}; ///< Mapping of `map_fd`.
src/platform/linux/qemu/session.cpp +9 −2
@@ -732,10 +732,17 @@
return *it;
}
std::vector<std::string> graphic_console_names(const vm_info_t &vm) {
std::vector<std::string> graphic_console_names(const vm_info_t &vm, std::string_view selected) {
auto selected_console = selected.empty() ? std::nullopt : find_console(vm, selected);
std::vector<std::string> names;
for (const auto &console : vm.consoles) {
if (!console.is_graphic()) {
continue;
if (console.is_graphic()) {
}
if (selected_console && selected_console->id == console.id) {
names.emplace_back(selected);
} else {
names.emplace_back(std::to_string(console.id));
}
}
src/platform/linux/qemu/session.h +6 −2
@@ -269,9 +269,13 @@
/**
* @brief List the names Sunshine uses for the graphical consoles.
* @details Sunshine selects a display by comparing `output_name` with these names, so the console
* that `selected` refers to by label is listed under that label; every other console is listed
* by id.
*
* @param vm VM information to list.
* @return Console ids as strings, for graphical consoles only.
* @param selected Configured `output_name`: a console id, a label, or empty.
* @return One name per graphical console, in QEMU order.
*/
std::vector<std::string> graphic_console_names(const vm_info_t &vm);
std::vector<std::string> graphic_console_names(const vm_info_t &vm, std::string_view selected = {});
} // namespace qemu
tests/e2e/moonlight_client/main.cpp +41 −2
@@ -10,6 +10,10 @@
* moonlight_e2e_client --port 47989 --api-user u --api-pass p --state-dir DIR \
* --expect "255,0,0;0,255,0;0,0,255;255,255,255" --out last.ppm --summary summary.json
* @endcode
*
* To test events during a stream (a guest reboot, for example), `--ready-file` is created once
* `--ready-frames` frames are decoded, and with `--wait-file` the `--frames` count and the picture
* check only start once that file exists.
*/
// standard includes
#include <algorithm>
@@ -64,6 +68,9 @@
int tolerance {48}; ///< Allowed per-channel difference.
std::string out; ///< PPM path for the last decoded frame.
std::string summary; ///< JSON path for the run summary.
std::string ready_file; ///< File created once `ready_frames` frames were decoded.
int ready_frames {1}; ///< Decoded frames before `ready_file` is created.
std::string wait_file; ///< Don't start counting `min_frames` until this file exists.
};
/**
@@ -76,6 +83,7 @@
AVPacket *packet {nullptr}; ///< Packet reused for decode units.
SwsContext *sws {nullptr}; ///< Converter to RGB24.
std::vector<std::uint8_t> rgb; ///< Last decoded picture, RGB24.
std::vector<std::chrono::steady_clock::time_point> decode_times; ///< When each frame was decoded.
int rgb_width {0}; ///< Width of `rgb`.
int rgb_height {0}; ///< Height of `rgb`.
int decoded {0}; ///< Decoded frame count.
@@ -143,6 +151,7 @@
state.rgb_width = w;
state.rgb_height = h;
state.decoded += 1;
state.decode_times.push_back(std::chrono::steady_clock::now());
}
return DR_OK;
}
@@ -336,6 +345,8 @@
{"--expect", &opts.expect},
{"--out", &opts.out},
{"--summary", &opts.summary},
{"--ready-file", &opts.ready_file},
{"--wait-file", &opts.wait_file},
};
std::map<std::string, int *> ints {
{"--port", &opts.port},
@@ -346,6 +357,7 @@
{"--frames", &opts.min_frames},
{"--timeout", &opts.timeout_s},
{"--tolerance", &opts.tolerance},
{"--ready-frames", &opts.ready_frames},
};
for (int i = 1; i < argc; ++i) {
std::string arg = argv[i];
@@ -476,12 +488,28 @@
}
bool matched = false;
bool ready_written = false;
std::optional<int> frames_at_wait;
std::optional<std::chrono::steady_clock::time_point> wait_start;
std::vector<std::array<int, 3>> last_colors;
const auto deadline = stream_start + std::chrono::seconds(opts.timeout_s);
while (std::chrono::steady_clock::now() < deadline && !state.terminated) {
std::this_thread::sleep_for(100ms);
std::lock_guard lock {state.mutex};
if (!opts.ready_file.empty() && !ready_written && state.decoded >= opts.ready_frames) {
std::ofstream {opts.ready_file} << state.decoded << "\n";
ready_written = true;
std::fprintf(stderr, "e2e: ready after %d frames\n", state.decoded);
}
if (!opts.wait_file.empty() && !frames_at_wait) {
if (!std::ifstream {opts.wait_file}) {
continue;
}
frames_at_wait = state.decoded;
wait_start = std::chrono::steady_clock::now();
std::fprintf(stderr, "e2e: %s appeared after %d frames\n", opts.wait_file.c_str(), state.decoded);
}
if (state.decoded < opts.min_frames || state.rgb.empty()) {
if (state.decoded - frames_at_wait.value_or(0) < opts.min_frames || state.rgb.empty()) {
continue;
}
if (!expected) {
@@ -502,10 +530,21 @@
std::vector<double> latencies;
int decoded = 0;
double max_gap_ms = 0;
{
std::lock_guard lock {state.mutex};
latencies = state.host_latency_ms;
decoded = state.decoded;
// the longest time without a decoded frame, from the first frame (or the wait file) to the end
auto previous = wait_start;
for (const auto &t : state.decode_times) {
if (previous && t > *previous) {
max_gap_ms = std::max(max_gap_ms, std::chrono::duration<double, std::milli>(t - *previous).count());
}
if (!wait_start || t > *wait_start) {
previous = t;
}
}
}
std::ostringstream summary;
@@ -513,7 +552,7 @@
for (std::size_t i = 0; i < last_colors.size(); ++i) {
summary << (i ? "," : "") << "[" << last_colors[i][0] << "," << last_colors[i][1] << "," << last_colors[i][2] << "]";
}
summary << "],\"terminated\":" << (state.terminated ? "true" : "false") << "}";
summary << "],\"frames_after_wait\":" << (decoded - frames_at_wait.value_or(0)) << ",\"max_frame_gap_ms\":" << max_gap_ms << ",\"terminated\":" << (state.terminated ? "true" : "false") << "}";
std::printf("%s\n", summary.str().c_str());
if (!opts.summary.empty()) {
std::ofstream {opts.summary} << summary.str() << "\n";
tests/e2e/qemu/e2e_stream.sh +38 −1
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# @tag requirements: [REQ-E2E-001, REQ-NFR-001]
# @tag requirements: [REQ-E2E-001, REQ-NFR-001, REQ-CAP-004]
# End-to-end test (REQ-E2E-001): an unmodified Moonlight client streams a QEMU VM that has no guest agent.
#
# 1. start a private dbus-daemon and a QEMU guest that draws a known pattern (run_vm.sh)
@@ -19,5 +19,8 @@
# E2E_WIDTH/E2E_HEIGHT/E2E_FPS stream mode (default: 1280x800 at 30 fps)
# E2E_FRAMES decoded frames to receive before checking (default: 30; raise for latency runs)
# E2E_ENCODER Sunshine encoder: software, nvenc, vaapi, vulkan (default: software)
# E2E_RESET_AFTER_FRAMES reboot the guest with QMP system_reset after this many decoded frames, then
# require E2E_FRAMES more frames and the pattern again (REQ-CAP-004)
# E2E_MAX_GAP_MS with E2E_RESET_AFTER_FRAMES, the longest allowed time without a decoded frame (default: 5000)
# E2E_KEEP set to 1 to keep the work directory
set -euo pipefail
@@ -35,6 +38,8 @@
fps="${E2E_FPS:-30}"
frames="${E2E_FRAMES:-30}"
encoder="${E2E_ENCODER:-software}"
reset_after="${E2E_RESET_AFTER_FRAMES:-}"
max_gap_ms="${E2E_MAX_GAP_MS:-5000}"
for bin in "${sunshine_bin}" "${client_bin}"; do
if [[ ! -x "${bin}" ]]; then
@@ -127,6 +132,21 @@
echo "e2e: Sunshine up on port ${port}" >&2
# 3-4. pair, stream, check the pattern (red, green, blue, white quadrants)
client_extra=()
if [[ -n "${reset_after}" ]]; then
client_extra=(--ready-file "${work}/stream_ready" --ready-frames "${reset_after}" --wait-file "${work}/reset_done")
(
for _ in $(seq 1 $((timeout_s * 10))); do
[[ -f "${work}/stream_ready" ]] && break
sleep 0.1
done
if [[ -f "${work}/stream_ready" ]]; then
echo "e2e: rebooting the guest (system_reset)" >&2
"${script_dir}/qmp.py" "${work}/vm/qmp.sock" system_reset >&2 && touch "${work}/reset_done"
fi
) &
fi
set +e
"${client_bin}" \
--port "${port}" \
@@ -140,9 +160,26 @@
--tolerance 48 \
--out "${artifacts}/last_frame.ppm" \
--summary "${artifacts}/summary.json" \
"${client_extra[@]}" \
2> "${work}/client.log"
result=$?
set -e
if [[ ${result} == 0 && -n "${reset_after}" ]]; then
# the stream must have kept producing frames through the reboot; QEMU 11 sends the firmware's
# 720x400 text mode (a display re-creation in Sunshine), QEMU 8.2 may coalesce it away
displays="$(sed -n '/CLIENT CONNECTED/,$p' "${work}/sunshine.log" | grep -c "qemu: streaming VM" || true)"
gap="$(sed -n 's/.*"max_frame_gap_ms":\([0-9.]*\).*/\1/p' "${artifacts}/summary.json")"
if [[ ! -f "${work}/reset_done" ]]; then
echo "e2e: FAIL: the guest was never reset" >&2
result=1
elif ! awk -v gap="${gap:-999999}" -v max="${max_gap_ms}" 'BEGIN { exit !(gap <= max) }'; then
echo "e2e: FAIL: no frame for ${gap} ms around the reset (limit ${max_gap_ms} ms)" >&2
result=1
else
echo "e2e: reset survived: ${displays} display(s) during the stream, longest frame gap ${gap} ms" >&2
fi
fi
if [[ ${result} == 0 ]]; then
echo "e2e: PASS $(cat "${artifacts}/summary.json")"
tests/e2e/qemu/qmp.py +48 −0
@@ -1,0 +1,48 @@
#!/usr/bin/env python3
"""Send one QMP command to a QEMU monitor socket and print the reply.
Usage: qmp.py <qmp-socket> <command> [json-arguments]
Exits non-zero when the connection fails or QEMU returns an error.
"""
import json
import socket
import sys
def main() -> int:
if len(sys.argv) < 3:
print(__doc__, file=sys.stderr)
return 2
path, command = sys.argv[1], sys.argv[2]
arguments = json.loads(sys.argv[3]) if len(sys.argv) > 3 else None
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock:
sock.settimeout(10)
sock.connect(path)
stream = sock.makefile("rw")
def receive():
# skip asynchronous events until a reply arrives
while True:
line = stream.readline()
if not line:
raise ConnectionError("QMP connection closed")
message = json.loads(line)
if "event" not in message:
return message
receive() # greeting
for request in ({"execute": "qmp_capabilities"}, {"execute": command, **({"arguments": arguments} if arguments else {})}):
stream.write(json.dumps(request) + "\n")
stream.flush()
reply = receive()
if "error" in reply:
print(json.dumps(reply), file=sys.stderr)
return 1
print(json.dumps(reply))
return 0
if __name__ == "__main__":
sys.exit(main())
tests/e2e/README.md +13 −1
@@ -11,7 +11,8 @@
| `qemu/guest/pattern.S` | 512-byte boot sector: VGA mode 13h with red, green, blue, white quadrants and a blinking square |
| `qemu/guest/build_guest.sh` | Builds the guest disk image with GNU `as`/`ld` (no binaries are committed) |
| `qemu/run_vm.sh` | Starts a private `dbus-daemon` and QEMU with `-display dbus,addr=...` and `-audiodev dbus` |
| `qemu/e2e_stream.sh` | The end-to-end test (REQ-E2E-001), also used for latency numbers (REQ-NFR-001) |
| `qemu/e2e_stream.sh` | The end-to-end test (REQ-E2E-001), also used for latency numbers (REQ-NFR-001) and the guest reboot test (REQ-CAP-004) |
| `qemu/qmp.py` | Sends one QMP command to the VM (used for `system_reset`) |
| `moonlight_client/` | Headless client: GameStream pairing/launch plus moonlight-common-c and FFmpeg decoding |
## Requirements
@@ -49,5 +50,16 @@
On failure, or when `E2E_ARTIFACTS` is set, the Sunshine, QEMU, dbus-daemon and client logs, the
last decoded frame (`last_frame.ppm`) and `summary.json` are kept. See the header of
`qemu/e2e_stream.sh` for all environment variables (binaries, port, stream mode, frame count).
## Guest reboot during a stream
```bash
E2E_RESET_AFTER_FRAMES=30 E2E_FRAMES=60 tests/e2e/qemu/e2e_stream.sh
```
After 30 decoded frames the script reboots the guest with QMP `system_reset`. The client then has
to decode 60 more frames and see the pattern again, and no gap between decoded frames may exceed
`E2E_MAX_GAP_MS` (5000 ms by default). The summary adds `frames_after_wait` and
`max_frame_gap_ms`.
## Latency runs
tests/unit/platform/linux/qemu/fake_qemu.h +41 −0
@@ -412,6 +412,47 @@
}
/**
* @brief Send a `MouseSet` call and wait for the reply.
*
* @param console_id Target console.
* @param x Pointer X position.
* @param y Pointer Y position.
* @param visible Whether the cursor is shown.
* @return True when the client acknowledged the call.
*/
bool mouse_set(std::uint32_t console_id, std::int32_t x, std::int32_t y, bool visible) {
auto proxy = listener_proxy(console_id);
if (!proxy) {
return false;
}
bool ok = qemu_dbus_display1_listener_call_mouse_set_sync(proxy, x, y, visible ? 1 : 0, G_DBUS_CALL_FLAGS_NONE, 5000, nullptr, nullptr);
g_object_unref(proxy);
return ok;
}
/**
* @brief Send a `CursorDefine` call and wait for the reply.
*
* @param console_id Target console.
* @param width Cursor width.
* @param height Cursor height.
* @param hot_x Hot-spot X.
* @param hot_y Hot-spot Y.
* @param pixels ARGB32 pixels.
* @return True when the client acknowledged the call.
*/
bool cursor_define(std::uint32_t console_id, std::int32_t width, std::int32_t height, std::int32_t hot_x, std::int32_t hot_y, const std::vector<std::uint8_t> &pixels) {
auto proxy = listener_proxy(console_id);
if (!proxy) {
return false;
}
auto v = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, pixels.data(), pixels.size(), 1);
bool ok = qemu_dbus_display1_listener_call_cursor_define_sync(proxy, width, height, hot_x, hot_y, v, G_DBUS_CALL_FLAGS_NONE, 5000, nullptr, nullptr);
g_object_unref(proxy);
return ok;
}
/**
* @brief Send a `ScanoutDMABUF` call, which the client does not support yet.
*
* @param console_id Target console.
tests/unit/platform/linux/qemu/test_capture.cpp +196 −4
@@ -14,6 +14,7 @@
#include <sys/socket.h>
#include <sys/un.h>
#include <thread>
#include <tuple>
// local includes
#include <src/config.h>
@@ -74,6 +75,7 @@
BaseTest::SetUp();
saved_capture = config::video.capture;
saved_address = config::video.qemu_dbus_address;
saved_output_name = config::video.output_name;
bus = std::make_unique<qemu_test::private_bus_t>();
if (!bus->ok()) {
@@ -93,11 +95,13 @@
config::video.capture = "qemu";
config::video.qemu_dbus_address = bus->address();
config::video.output_name.clear();
}
void TearDown() override {
config::video.capture = saved_capture;
config::video.qemu_dbus_address = saved_address;
config::video.output_name = saved_output_name;
fake.reset();
bus.reset();
BaseTest::TearDown();
@@ -114,8 +118,11 @@
* @return Display, or nullptr.
*/
std::shared_ptr<platf::display_t> open_display(const std::string &display_name, std::uint32_t console_id, int width, int height, int framerate = 60) {
const int before = fake->registrations(console_id);
std::thread sender {[&, console_id, width, height, before]() {
if (qemu_test::wait_until([&]() {
std::thread sender {[&, console_id, width, height]() {
if (fake->wait_for_listener(console_id)) {
return fake->registrations(console_id) > before;
})) {
fake->scanout(console_id, width, height, width * 4, qemu::pixman_format::x8r8g8b8, solid(width, height, 0x10, 0x20, 0x30));
}
}};
@@ -129,9 +136,10 @@
*
* @param display Display to capture from.
* @param on_frame Called for each pushed image; return false to stop.
* @param draw_cursor Whether the pipeline asks for the cursor to be drawn.
* @return Capture status; `ok` also when the tick limit stopped the capture.
*/
platf::capture_e run_capture(platf::display_t &display, const std::function<bool(std::shared_ptr<platf::img_t> &&, bool)> &on_frame, bool draw_cursor = false) {
platf::capture_e run_capture(platf::display_t &display, const std::function<bool(std::shared_ptr<platf::img_t> &&, bool)> &on_frame) {
int ticks = 0;
auto bounded = [&](std::shared_ptr<platf::img_t> &&img, bool frame_captured) {
return ++ticks < 600 && on_frame(std::move(img), frame_captured);
@@ -148,6 +156,6 @@
img_out = pool.back();
return true;
};
bool cursor = draw_cursor;
bool cursor = false;
return display.capture(bounded, pull, &cursor);
}
@@ -156,5 +164,6 @@
std::unique_ptr<qemu_test::fake_qemu_t> fake;
std::string saved_capture;
std::string saved_address;
std::string saved_output_name;
};
} // namespace
@@ -344,6 +353,189 @@
});
resizer.join();
EXPECT_EQ(status, platf::capture_e::reinit);
}
// @tag requirements: [REQ-CAP-004]
TEST_F(QemuCaptureTest, DisableShowsBlackFrameAndKeepsCapturing) {
auto display = open_display("1", 1, 4, 4);
ASSERT_NE(display, nullptr);
int captured = 0;
std::vector<std::uint8_t> last;
std::thread actor;
auto status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&img, bool frame_captured) {
if (!frame_captured) {
return true;
}
captured += 1;
last.assign(img->data, img->data + img->height * img->row_pitch);
if (captured == 1) {
EXPECT_EQ(last[0], 0x10);
actor = std::thread {[&]() {
fake->disable(1);
}};
return true;
}
if (captured == 2) {
// the display is off: a black frame, and the capture keeps running
EXPECT_EQ(last, std::vector<std::uint8_t>(4 * 4 * 4, 0));
actor.join();
actor = std::thread {[&]() {
fake->update(1, 0, 0, 4, 4, 16, qemu::pixman_format::x8r8g8b8, solid(4, 4, 0x55, 0x66, 0x77));
}};
return true;
}
// the guest turned the display back on at the same size
EXPECT_EQ(last[0], 0x55);
return false;
});
if (actor.joinable()) {
actor.join();
}
EXPECT_EQ(status, platf::capture_e::ok);
EXPECT_EQ(captured, 3);
}
// @tag requirements: [REQ-CAP-004]
TEST_F(QemuCaptureTest, SurvivesGuestRebootSequence) {
// the OS runs at 8x6; the guest reboots: Disable, firmware mode 4x2, then the OS mode 8x6 again
auto display = open_display("1", 1, 8, 6);
ASSERT_NE(display, nullptr);
std::thread reboot {[&]() {
std::this_thread::sleep_for(50ms);
fake->disable(1);
fake->scanout(1, 4, 2, 16, qemu::pixman_format::x8r8g8b8, solid(4, 2, 1, 2, 3));
}};
auto status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&, bool) {
return true;
});
reboot.join();
EXPECT_EQ(status, platf::capture_e::reinit) << "the firmware mode changes the size";
// the pipeline re-creates the display, as video.cpp does after reinit
display.reset();
display = open_display("1", 1, 4, 2);
ASSERT_NE(display, nullptr);
EXPECT_EQ(display->width, 4);
EXPECT_EQ(display->height, 2);
std::thread boot {[&]() {
std::this_thread::sleep_for(50ms);
fake->scanout(1, 8, 6, 32, qemu::pixman_format::x8r8g8b8, solid(8, 6, 9, 9, 9));
}};
status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&, bool) {
return true;
});
boot.join();
EXPECT_EQ(status, platf::capture_e::reinit);
display.reset();
display = open_display("1", 1, 8, 6);
ASSERT_NE(display, nullptr);
EXPECT_EQ(display->width, 8);
std::uint8_t first_byte = 0;
status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&img, bool frame_captured) {
if (frame_captured) {
first_byte = img->data[0];
}
return !frame_captured;
});
EXPECT_EQ(status, platf::capture_e::ok);
EXPECT_EQ(first_byte, 0x10);
}
// @tag requirements: [REQ-CAP-005]
TEST_F(QemuCaptureTest, CompositesGuestCursorWhenRequested) {
auto display = open_display("1", 1, 8, 8);
ASSERT_NE(display, nullptr);
// 2x2 opaque white cursor with its hot spot at (1, 1), pointer at (4, 4): covers (3..4, 3..4)
ASSERT_TRUE(fake->cursor_define(1, 2, 2, 1, 1, std::vector<std::uint8_t>(2 * 2 * 4, 0xff)));
ASSERT_TRUE(fake->mouse_set(1, 4, 4, true));
for (bool draw : {true, false}) {
std::vector<std::uint8_t> frame;
auto status = run_capture(
*display,
[&](std::shared_ptr<platf::img_t> &&img, bool frame_captured) {
if (!frame_captured) {
return true;
}
frame.assign(img->data, img->data + img->height * img->row_pitch);
return false;
},
draw
);
EXPECT_EQ(status, platf::capture_e::ok);
ASSERT_EQ(frame.size(), 8 * 8 * 4);
auto pixel = [&](int x, int y) {
return frame[(y * 8 + x) * 4];
};
EXPECT_EQ(pixel(3, 3), draw ? 0xff : 0x10) << "draw=" << draw;
EXPECT_EQ(pixel(4, 4), draw ? 0xff : 0x10) << "draw=" << draw;
EXPECT_EQ(pixel(2, 2), 0x10);
EXPECT_EQ(pixel(5, 5), 0x10);
// the cursor moving produces a new frame even without framebuffer damage
ASSERT_TRUE(fake->mouse_set(1, 4, 4, true));
}
}
// @tag requirements: [REQ-CAP-006]
TEST_F(QemuCaptureTest, OutputNameSelectsConsoleByIdOrLabel) {
for (auto [output_name, console_id, width] : {
std::tuple {std::string {"2"}, 2u, 32},
std::tuple {std::string {"virtio-gpu-pci.1"}, 2u, 32},
std::tuple {std::string {"1"}, 1u, 64},
std::tuple {std::string {"VGA"}, 1u, 64},
}) {
config::video.output_name = output_name;
// Sunshine picks the display whose name equals output_name (video::refresh_displays)
auto names = platf::qemu_display_names();
auto it = std::ranges::find(names, output_name);
ASSERT_NE(it, names.end()) << output_name;
EXPECT_EQ(names.size(), 2);
const int before = fake->registrations(console_id);
auto display = open_display(*it, console_id, width, 8);
ASSERT_NE(display, nullptr) << output_name;
EXPECT_EQ(display->width, width) << output_name;
EXPECT_EQ(fake->registrations(console_id), before + 1) << output_name;
}
}
// @tag requirements: [REQ-CAP-006]
TEST_F(QemuCaptureTest, MultiHeadGuestStreamsOneConsolePerDisplay) {
auto head0 = open_display("1", 1, 8, 4);
ASSERT_NE(head0, nullptr);
auto head1 = std::shared_ptr<platf::display_t> {};
{
std::thread sender {[&]() {
if (fake->wait_for_listener(2)) {
fake->scanout(2, 6, 2, 24, qemu::pixman_format::x8r8g8b8, solid(6, 2, 0x99, 0x88, 0x77));
}
}};
head1 = platf::qemu_display(platf::mem_type_e::system, "virtio-gpu-pci.1", stream_config());
sender.join();
}
ASSERT_NE(head1, nullptr);
EXPECT_EQ(head0->width, 8);
EXPECT_EQ(head1->width, 6);
for (auto [display, blue] : {std::pair {head0, 0x10}, std::pair {head1, 0x99}}) {
int value = -1;
auto status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&img, bool frame_captured) {
if (frame_captured) {
value = img->data[0];
}
return !frame_captured;
});
EXPECT_EQ(status, platf::capture_e::ok);
EXPECT_EQ(value, blue);
}
}
// @tag requirements: [REQ-CAP-001]
tests/unit/platform/linux/qemu/test_frame_store.cpp +81 −0
@@ -328,6 +328,87 @@
EXPECT_GT(store.sequence(), seq);
}
// @tag requirements: [REQ-CAP-005]
TEST(QemuFrameStoreTest, BlendsCursorWithHotSpotAlphaAndClipping) {
// 4x3 gray frame
qemu::frame_store_t store;
store.scanout(4, 3, 16, qemu::pixman_format::x8r8g8b8, std::vector<std::uint8_t>(4 * 3 * 4, 0x40));
// 2x2 cursor: opaque red, half-transparent white, fully transparent, opaque blue (B, G, R, A bytes)
const std::vector<std::uint8_t> image {0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0x80, 0x12, 0x34, 0x56, 0x00, 0xff, 0x00, 0x00, 0xff};
auto seq = store.sequence();
store.cursor_define(2, 2, 1, 1, image);
EXPECT_EQ(store.sequence(), seq) << "an invisible cursor doesn't change the frame";
store.mouse_set(2, 2, true); // hot spot (1, 1) at (2, 2): image covers (1..2, 1..2)
EXPECT_GT(store.sequence(), seq);
std::vector<std::uint8_t> frame(4 * 3 * 4);
std::uint64_t copied = 0;
std::chrono::steady_clock::time_point timestamp;
ASSERT_EQ(store.copy_if_newer(copied, 4, 3, frame.data(), timestamp, true), qemu::frame_status_e::new_frame);
EXPECT_EQ(bgr_at(frame, 4, 1, 1), (std::array<std::uint8_t, 3> {0x00, 0x00, 0xff}));
const std::uint8_t mixed = (0xff * 0x80 + 0x40 * 0x7f + 127) / 255;
EXPECT_EQ(bgr_at(frame, 4, 2, 1), (std::array<std::uint8_t, 3> {mixed, mixed, mixed}));
EXPECT_EQ(bgr_at(frame, 4, 1, 2), (std::array<std::uint8_t, 3> {0x40, 0x40, 0x40}));
EXPECT_EQ(bgr_at(frame, 4, 2, 2), (std::array<std::uint8_t, 3> {0xff, 0x00, 0x00}));
EXPECT_EQ(bgr_at(frame, 4, 0, 0), (std::array<std::uint8_t, 3> {0x40, 0x40, 0x40}));
// without cursor drawing, the frame is untouched
store.update(0, 0, 1, 1, 4, qemu::pixman_format::x8r8g8b8, std::vector<std::uint8_t>(4, 0x40));
ASSERT_EQ(store.copy_if_newer(copied, 4, 3, frame.data(), timestamp, false), qemu::frame_status_e::new_frame);
EXPECT_EQ(bgr_at(frame, 4, 1, 1), (std::array<std::uint8_t, 3> {0x40, 0x40, 0x40}));
// clipped at the bottom-right corner: only the opaque red pixel lands on (3, 2)
store.mouse_set(4, 3, true);
ASSERT_EQ(store.copy_if_newer(copied, 4, 3, frame.data(), timestamp, true), qemu::frame_status_e::new_frame);
EXPECT_EQ(bgr_at(frame, 4, 3, 2), (std::array<std::uint8_t, 3> {0x00, 0x00, 0xff}));
EXPECT_EQ(bgr_at(frame, 4, 2, 2), (std::array<std::uint8_t, 3> {0x40, 0x40, 0x40}));
// clipped at the top-left corner: only the opaque blue pixel lands on (0, 0)
store.mouse_set(0, 0, true);
ASSERT_EQ(store.copy_if_newer(copied, 4, 3, frame.data(), timestamp, true), qemu::frame_status_e::new_frame);
EXPECT_EQ(bgr_at(frame, 4, 0, 0), (std::array<std::uint8_t, 3> {0xff, 0x00, 0x00}));
// hiding the cursor redraws once; moving a hidden cursor doesn't
store.mouse_set(1, 1, false);
ASSERT_EQ(store.copy_if_newer(copied, 4, 3, frame.data(), timestamp, true), qemu::frame_status_e::new_frame);
EXPECT_EQ(bgr_at(frame, 4, 0, 0), (std::array<std::uint8_t, 3> {0x40, 0x40, 0x40}));
store.mouse_set(2, 2, false);
EXPECT_EQ(store.copy_if_newer(copied, 4, 3, frame.data(), timestamp, true), qemu::frame_status_e::unchanged);
}
// @tag requirements: [REQ-CAP-005]
TEST(QemuFrameStoreTest, CursorStateCopiesPixelsOnlyWhenTheImageChanged) {
qemu::frame_store_t store;
auto initial = store.cursor(0);
EXPECT_FALSE(initial.drawable());
store.cursor_define(1, 1, 0, 0, std::vector<std::uint8_t> {1, 2, 3, 4});
store.mouse_set(5, 6, true);
auto state = store.cursor(0);
EXPECT_TRUE(state.drawable());
EXPECT_EQ(state.x, 5);
EXPECT_EQ(state.y, 6);
EXPECT_EQ(state.serial, 1);
EXPECT_EQ(state.pixels, (std::vector<std::uint8_t> {1, 2, 3, 4}));
auto again = store.cursor(state.serial);
EXPECT_TRUE(again.pixels.empty());
EXPECT_EQ(again.serial, 1);
// invalid shapes are ignored
store.cursor_define(2, 2, 0, 0, std::vector<std::uint8_t> {1, 2, 3, 4});
store.cursor_define(0, 1, 0, 0, std::vector<std::uint8_t> {});
store.cursor_define(4096, 1, 0, 0, std::vector<std::uint8_t>(4096 * 4));
EXPECT_EQ(store.cursor(0).serial, 1);
// drawing an undefined or invisible cursor is a no-op
std::vector<std::uint8_t> frame(4, 0x33);
qemu::cursor_state_t hidden;
qemu::blend_cursor(hidden, frame.data(), 1, 1);
EXPECT_EQ(frame, std::vector<std::uint8_t>(4, 0x33));
}
// @tag requirements: [REQ-CAP-001]
TEST(QemuFrameStoreTest, DisconnectWakesWaitersAndIsReported) {
qemu::frame_store_t store;
tests/unit/platform/linux/qemu/test_session.cpp +18 −0
@@ -206,6 +206,24 @@
EXPECT_EQ(qemu::graphic_console_names(vm), (std::vector<std::string> {"1", "2"}));
}
// @tag requirements: [REQ-CAP-006]
TEST_F(QemuSessionTest, ConsoleNamesMatchOutputNameByIdOrLabel) {
qemu::vm_info_t vm;
vm.consoles = {
{0, "serial0", "Text"},
{1, "VGA", "Graphic"},
{2, "virtio-gpu-pci.1", "Graphic"},
};
EXPECT_EQ(qemu::graphic_console_names(vm, ""), (std::vector<std::string> {"1", "2"}));
EXPECT_EQ(qemu::graphic_console_names(vm, "2"), (std::vector<std::string> {"1", "2"}));
EXPECT_EQ(qemu::graphic_console_names(vm, "virtio-gpu-pci.1"), (std::vector<std::string> {"1", "virtio-gpu-pci.1"}));
EXPECT_EQ(qemu::graphic_console_names(vm, "VGA"), (std::vector<std::string> {"VGA", "2"}));
// a text console or an unknown name is never listed
EXPECT_EQ(qemu::graphic_console_names(vm, "serial0"), (std::vector<std::string> {"1", "2"}));
EXPECT_EQ(qemu::graphic_console_names(vm, "HDMI-1"), (std::vector<std::string> {"1", "2"}));
}
// @tag requirements: [REQ-CAP-001]
TEST_F(QemuSessionTest, RegisterListenerAdvertisesUnixMap) {
start_fake();