ref:225def94c915d1eb2b84e8018f14da37dfa01b86

feat(linux): push qemu frames on damage instead of a fixed capture tick

The capture loop slept to a fixed tick of one client frame interval and copied whatever damage arrived since, so damage waited 0-16.7 ms at 60 fps. Wait on the frame store's condition variable instead: push as soon as QEMU reports damage, at most once per client frame interval, and keep a heartbeat once per interval so the pipeline can stop the capture. The listener thread still never waits for capture. At 1080p60 with NVENC (shared memory map, RTX 4090) host latency p95 went from ~21 ms to ~10 ms and p50 from ~13 ms to ~4 ms; see JOURNAL.md. Refs #3 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPNw4PCgkEfhyCjQT19wsb
SHA: 225def94c915d1eb2b84e8018f14da37dfa01b86
Author: Cole Christensen <cole.christensen@gmail.com>
Date: 2026-09-12 22:40
Parents: 52f749f
6 files changed +212 -8
Type
JOURNAL.md +31 −0
@@ -110,3 +110,34 @@
- In the E2E runs, PulseAudio isn't running ("Failed to create client: Daemon not running") and
libvirtualhid gamepads are unavailable; video streaming is unaffected. #4 and #5 replace both.
## 2026-09-12 — Phase 1a (#3): qemu capture source
Same host as Phase 0 (WSL2, RTX 4090 through `/dev/dxg`, no `/dev/dri`, no `/dev/udmabuf`).
### Damage-driven capture (REQ-NFR-001)
`display_t::capture()` now waits on the frame store's condition variable (`wait_for_change`, woken
by every listener call that changes the frame) instead of sleeping to a fixed tick. It pushes as
soon as damage arrives, but never earlier than one client frame interval after the previous push.
Without damage it still returns to the pipeline once per interval with `frame_captured = false`,
so a stop or display switch is noticed; the encoder's minimum FPS repeats static frames. The
listener thread only notifies, it never waits for capture or the encoder.
Latency before and after, same harness and machine as Phase 0: Release + CUDA build
(`cmake-build-release-cuda`), `E2E_ENCODER=nvenc E2E_WIDTH=1920 E2E_HEIGHT=1080 E2E_FPS=60
E2E_FRAMES=600`, KVM, guest framebuffer 640x400 scaled by Sunshine, ~330 samples per run.
| Build | Transport (QEMU) | Encoder | p50 (3 runs) | p95 (3 runs) |
|----------------------------|----------------------------|---------------|--------------------|--------------------|
| before (`98cb3d22`) | shared memory map (11.1.1) | NVENC | 13.5 / 11.1 / 13.6 | 21.0 / 18.6 / 21.8 |
| after (damage-driven) | shared memory map (11.1.1) | NVENC | 4.0 / 4.5 / 3.9 | 9.8 / 10.0 / 9.4 |
| after (damage-driven) | D-Bus messages (8.2.2) | NVENC | 3.9 | 8.7 |
| after (damage-driven) | shared memory map (11.1.1) | software x264 | 4.4 | 6.4 |
The shm + hardware encoder target from #3 (p95 ≤ 12 ms at 1080p60) is met: p95 went from ~21 ms to
~10 ms. The remaining p95 is NVENC plus the RAM→CUDA upload of the 1080p frame; x264 is faster on
this mostly static 640x400 guest picture scaled up. Unit tests pin the behavior:
`QemuCaptureTest.PushesDamageWithinMillisecondsNotAtTheNextTick` fails on the old loop (median
26 ms at 30 fps) and passes now, `BurstsOfDamagePushAtMostOneFramePerInterval` and
`NoDamageRepeatsNoFrames` pass on both.
src/platform/linux/qemu/capture.cpp +24 −3
@@ -5,6 +5,7 @@
// standard includes
#include <cstring>
#include <mutex>
#include <thread>
// local includes
#include "frame_store.h"
@@ -106,12 +107,31 @@
return 0;
}
/**
* @brief Push frames as soon as the guest damages the display, at most one per client frame interval.
* @details Waits on the frame store instead of a fixed tick, so damage is copied right after
* QEMU reports it. Without damage, the pipeline gets a heartbeat once per frame interval (so it
* can stop or reconfigure the capture) and the encoder's minimum frame rate repeats the last
* frame. The listener thread never waits for this loop.
*
* @param push_captured_image_cb Callback receiving captured images and heartbeats.
* @param pull_free_image_cb Callback providing an image to fill.
* @param cursor Whether to draw the cursor.
* @return Why the capture stopped.
*/
platf::capture_e capture(const push_captured_image_cb_t &push_captured_image_cb, const pull_free_image_cb_t &pull_free_image_cb, bool *cursor) override {
// allow the first frame right away
auto next_frame = std::chrono::steady_clock::now();
sleep_overshoot_logger.reset();
auto last_push = std::chrono::steady_clock::now() - delay;
while (true) {
auto now = std::chrono::steady_clock::now();
if (store->wait_for_change(copied_sequence, now + delay)) {
// rate limit: never push more often than the client's frame interval
auto earliest = last_push + delay;
platf::handle_pacing(next_frame, delay, sleep_overshoot_logger);
if (std::chrono::steady_clock::now() < earliest) {
std::this_thread::sleep_until(earliest);
}
}
std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out);
@@ -126,6 +146,7 @@
}
break;
case platf::capture_e::ok:
last_push = std::chrono::steady_clock::now();
if (!push_captured_image_cb(std::move(img_out), true)) {
return platf::capture_e::ok;
}
src/platform/linux/qemu/frame_store.cpp +7 −0
@@ -197,6 +197,13 @@
return change_sequence > 0 && !is_disconnected;
}
bool frame_store_t::wait_for_change(std::uint64_t last_sequence, std::chrono::steady_clock::time_point deadline) {
std::unique_lock lock {mutex};
return frame_ready.wait_until(lock, deadline, [&]() {
return change_sequence != last_sequence || is_disconnected;
});
}
int frame_store_t::width() const {
std::lock_guard lock {mutex};
return frame_width;
src/platform/linux/qemu/frame_store.h +11 −0
@@ -58,6 +58,17 @@
bool wait_for_frame(std::chrono::milliseconds timeout);
/**
* @brief Wait until the frame changes past a sequence number, QEMU disconnects, or a deadline passes.
* @details Wakes as soon as a listener call changes the frame, so capture can push damage right
* away instead of at the next frame tick.
*
* @param last_sequence Sequence of the caller's last copy.
* @param deadline Time to give up waiting.
* @return True when the frame changed or QEMU disconnected; false when the deadline passed.
*/
bool wait_for_change(std::uint64_t last_sequence, std::chrono::steady_clock::time_point deadline);
/**
* @brief Current frame width.
*
* @return Width in pixels, 0 before the first scanout.
tests/unit/platform/linux/qemu/test_capture.cpp +112 −5
@@ -8,6 +8,8 @@
#include "fake_qemu.h"
// standard includes
#include <algorithm>
#include <atomic>
#include <cstring>
#include <sys/socket.h>
#include <sys/un.h>
@@ -29,14 +31,15 @@
namespace {
/**
* @brief A 60 fps stream configuration.
* @brief A 720p stream configuration.
*
* @param framerate Client frame rate.
* @return Video configuration.
*/
video::config_t stream_config() {
video::config_t stream_config(int framerate = 60) {
video::config_t config {};
config.width = 1280;
config.height = 720;
config.framerate = framerate;
config.framerate = 60;
return config;
}
@@ -107,15 +110,16 @@
* @param console_id Console the fake scans out on.
* @param width Scanout width.
* @param height Scanout height.
* @param framerate Client frame rate.
* @return Display, or nullptr.
*/
std::shared_ptr<platf::display_t> open_display(const std::string &display_name, std::uint32_t console_id, int width, int height) {
std::shared_ptr<platf::display_t> open_display(const std::string &display_name, std::uint32_t console_id, int width, int height, int framerate = 60) {
std::thread sender {[&, console_id, width, height]() {
if (fake->wait_for_listener(console_id)) {
fake->scanout(console_id, width, height, width * 4, qemu::pixman_format::x8r8g8b8, solid(width, height, 0x10, 0x20, 0x30));
}
}};
auto display = platf::qemu_display(platf::mem_type_e::system, display_name, stream_config());
auto display = platf::qemu_display(platf::mem_type_e::system, display_name, stream_config(framerate));
sender.join();
return display;
}
@@ -221,6 +225,109 @@
EXPECT_EQ(status, platf::capture_e::ok);
EXPECT_EQ(captured, 2);
EXPECT_GT(idle, 0);
}
// @tag requirements: [REQ-NFR-001]
TEST_F(QemuCaptureTest, PushesDamageWithinMillisecondsNotAtTheNextTick) {
// 30 fps: a fixed-tick capture loop would delay damage by 0-33 ms, 17 ms on average
auto display = open_display("1", 1, 4, 4, 30);
ASSERT_NE(display, nullptr);
std::vector<std::chrono::steady_clock::duration> delays;
std::atomic<bool> stop {false};
std::thread painter;
int captured = 0;
auto status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&img, bool frame_captured) {
if (!frame_captured) {
return true;
}
captured += 1;
if (captured == 1) {
painter = std::thread {[&]() {
// damage lands at varying points of the frame interval, always later than one interval after the last push
for (int i = 0; i < 12 && !stop; ++i) {
std::this_thread::sleep_for(std::chrono::milliseconds(40 + (i * 7) % 30));
fake->update(1, 0, 0, 1, 1, 4, qemu::pixman_format::x8r8g8b8, {(std::uint8_t) i, 0, 0, 0xff});
}
}};
return true;
}
delays.push_back(std::chrono::steady_clock::now() - *img->frame_timestamp);
return delays.size() < 10;
});
stop = true;
if (painter.joinable()) {
painter.join();
}
EXPECT_EQ(status, platf::capture_e::ok);
ASSERT_EQ(delays.size(), 10);
std::ranges::sort(delays);
EXPECT_LT(delays[5], 5ms) << "median damage-to-push delay";
EXPECT_LT(delays.back(), 16ms) << "worst damage-to-push delay";
}
// @tag requirements: [REQ-NFR-001]
TEST_F(QemuCaptureTest, BurstsOfDamagePushAtMostOneFramePerInterval) {
auto display = open_display("1", 1, 4, 4, 30);
ASSERT_NE(display, nullptr);
const auto interval = std::chrono::nanoseconds {1s} / 30;
std::vector<std::chrono::steady_clock::time_point> pushes;
std::atomic<bool> painting {true};
std::thread painter;
auto status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&, bool frame_captured) {
if (frame_captured) {
pushes.push_back(std::chrono::steady_clock::now());
if (pushes.size() == 1) {
painter = std::thread {[&]() {
const auto end = std::chrono::steady_clock::now() + 600ms;
for (std::uint8_t i = 0; std::chrono::steady_clock::now() < end; ++i) {
fake->update(1, 0, 0, 1, 1, 4, qemu::pixman_format::x8r8g8b8, {i, 0, 0, 0xff});
std::this_thread::sleep_for(2ms);
}
painting = false;
}};
}
}
return painting || pushes.size() < 2;
});
if (painter.joinable()) {
painter.join();
}
EXPECT_EQ(status, platf::capture_e::ok);
// 600 ms of damage every 2 ms is about 250 updates; at 30 fps that is at most 19 frames plus the first
EXPECT_GE(pushes.size(), 10);
EXPECT_LE(pushes.size(), 21);
for (std::size_t i = 1; i < pushes.size(); ++i) {
EXPECT_GE(pushes[i] - pushes[i - 1], interval - 2ms) << "push " << i;
}
}
// @tag requirements: [REQ-NFR-001]
TEST_F(QemuCaptureTest, NoDamageRepeatsNoFrames) {
auto display = open_display("1", 1, 4, 4, 60);
ASSERT_NE(display, nullptr);
int captured = 0;
int idle = 0;
std::chrono::steady_clock::time_point first;
auto status = run_capture(*display, [&](std::shared_ptr<platf::img_t> &&, bool frame_captured) {
if (frame_captured) {
captured += 1;
first = std::chrono::steady_clock::now();
return true;
}
idle += 1;
return captured == 0 || std::chrono::steady_clock::now() - first < 300ms;
});
EXPECT_EQ(status, platf::capture_e::ok);
EXPECT_EQ(captured, 1);
// the pipeline still gets a heartbeat about once per frame interval, so it can stop the capture
EXPECT_GE(idle, 10);
EXPECT_LE(idle, 40);
}
// @tag requirements: [REQ-CAP-001]
tests/unit/platform/linux/qemu/test_frame_store.cpp +27 −0
@@ -302,6 +302,32 @@
EXPECT_EQ(bgr_at(frame, 2, 1, 0), (std::array<std::uint8_t, 3> {0, 0, 0}));
}
// @tag requirements: [REQ-NFR-001]
TEST(QemuFrameStoreTest, WaitForChangeWakesOnDamageAndTimesOut) {
qemu::frame_store_t store;
store.scanout(1, 1, 4, qemu::pixman_format::x8r8g8b8, std::vector<std::uint8_t>(4));
const auto seq = store.sequence();
// nothing changes: the wait ends at the deadline
auto start = std::chrono::steady_clock::now();
EXPECT_FALSE(store.wait_for_change(seq, start + 30ms));
EXPECT_GE(std::chrono::steady_clock::now() - start, 30ms);
// a change that is already there returns immediately
EXPECT_TRUE(store.wait_for_change(seq - 1, std::chrono::steady_clock::now() + 5s));
// damage from another thread wakes the waiter long before the deadline
std::thread painter {[&]() {
std::this_thread::sleep_for(20ms);
store.update(0, 0, 1, 1, 4, qemu::pixman_format::x8r8g8b8, std::vector<std::uint8_t>(4, 0x11));
}};
start = std::chrono::steady_clock::now();
EXPECT_TRUE(store.wait_for_change(seq, start + 5s));
EXPECT_LT(std::chrono::steady_clock::now() - start, 1s);
painter.join();
EXPECT_GT(store.sequence(), seq);
}
// @tag requirements: [REQ-CAP-001]
TEST(QemuFrameStoreTest, DisconnectWakesWaitersAndIsReported) {
qemu::frame_store_t store;
@@ -313,6 +339,7 @@
EXPECT_FALSE(store.wait_for_frame(5s));
disconnector.join();
EXPECT_FALSE(store.connected());
EXPECT_TRUE(store.wait_for_change(store.sequence(), std::chrono::steady_clock::now() + 5s));
std::vector<std::uint8_t> frame(4);
std::uint64_t seq = 0;