ref:2920302546fcbc76477f9fc58bd5027905b3f83d

revert(linux): send the guest's audio as early as possible, not on a cadence

Pacing the frames to an even cadence was the wrong trade and the measurements say so: against a link with 1% loss and 8 ms of correlated jitter, it doubled the packets the client had to conceal, 101 and 103 against 49 and 60 unpaced over the same runs. A packet handed over as soon as the guest writes it reaches the client with up to a guest quantum of slack, which its jitter buffer spends absorbing a late arrival; a packet held back until the moment it plays arrives with none. The even cadence looked better in isolation because the measurement only saw arrival times on a loopback connection, where there is nothing to absorb. The end-to-end audio test now fails when the client conceals any packet at all on a local connection, which is what the cadence check should have been asking about. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPNw4PCgkEfhyCjQT19wsb
SHA: 2920302546fcbc76477f9fc58bd5027905b3f83d
Author: Cole Christensen <cole.christensen@gmail.com>
Date: 2026-09-18 03:38
Parents: f7bd16c
3 files changed +16 -69
Type
docs/qemu.md +6 −6
@@ -163,11 +163,11 @@
A guest that really plays 44.1 kHz content is fine: its own audio stack converts to the rate it opened the
device with, and only then does Sunshine see it.
Sunshine also hands the frames to the encoder at the rate they play. A guest writes its audio in whatever
quantum its own stack uses, usually 10 ms or more, so the frames arrive in bursts: two or three at once, then
nothing. The samples are correct either way, which is why this is inaudible in a recording of the stream and
plainly audible on a client -- its jitter buffer is fed unevenly and plays the gaps as a constant crackle.
Measured on the desktop guest, delivery went from half the packets arriving within a millisecond of the one
before to 3 in 26289.
Frames go to the encoder as soon as the guest writes them, which means in whatever quantum the guest's own
audio stack uses -- usually 10 ms, so two 5 ms packets at a time. That looks uneven, and handing them over at
an even cadence instead is tempting, but it is the wrong trade: a packet sent as early as possible reaches the
client with slack for its jitter buffer to absorb a late arrival, and one sent exactly when it plays has none.
Measured against a link with 1% loss and 8 ms of correlated jitter, pacing them evenly doubled the packets the
client had to conceal (101 and 103 against 49 and 60 over the same runs). Send early.
## The VM's display size
src/platform/linux/qemu/audio.cpp +6 −57
@@ -279,12 +279,9 @@
* @param output Output the mixer is attached to.
* @param mixer Attached mixer.
*/
mic_t(std::shared_ptr<audio_output_t> output, std::shared_ptr<audio_mixer_t> mixer, std::uint32_t frame_size, std::uint32_t sample_rate, std::chrono::nanoseconds frame_duration):
mic_t(std::shared_ptr<audio_output_t> output, std::shared_ptr<audio_mixer_t> mixer):
output {std::move(output)},
mixer {std::move(mixer)},
frame_size {frame_size},
sample_rate {sample_rate},
frame_duration {frame_duration} {
mixer {std::move(mixer)} {
}
~mic_t() override {
@@ -304,7 +301,6 @@
platf::capture_e sample(std::vector<float> &frame_buffer) override {
switch (mixer->read(frame_buffer, sample_timeout)) {
case read_status_e::ok:
pace();
return platf::capture_e::ok;
case read_status_e::timeout:
return platf::capture_e::timeout;
@@ -316,57 +312,8 @@
}
private:
/**
* @brief Hand out frames at the rate they play, not the rate the guest wrote them.
* @details A guest writes its audio in whatever quantum its own stack uses, usually 10 ms or more,
* so frames reach Sunshine in bursts: two or three at once, then nothing for a while. The samples
* are right either way, which is why only a player hears the difference -- its jitter buffer is fed
* unevenly and crackles. A sound card paces a real capture; this paces ours.
*
* The schedule follows the frames handed out, and restarts whenever it has fallen more than a frame
* behind, so a stall doesn't make the frames after it race out.
*/
void pace() {
// A guest whose clock runs a shade fast would build a backlog that only the buffer overflowing
// would clear, turning smooth delivery into added delay. Handing frames out as fast as they come
// while more than a couple are waiting keeps the delay where it started.
if (mixer->buffered_frames() > 2 * frame_size) {
next_frame = {};
return;
}
const auto now = std::chrono::steady_clock::now();
if (next_frame == std::chrono::steady_clock::time_point {} || now - next_frame > frame_duration) {
next_frame = now + frame_duration;
return;
}
if (now < next_frame) {
const auto wait = next_frame - now;
waited += wait;
std::this_thread::sleep_for(wait);
}
next_frame += frame_duration;
// what pacing costs: how long frames wait and how much audio is queued behind them
if (++paced_frames % report_every == 0) {
const auto queued = mixer->buffered_frames() * std::chrono::nanoseconds {std::chrono::seconds {1}} / sample_rate;
BOOST_LOG(debug) << "qemu: audio pacing held "sv
<< std::chrono::duration_cast<std::chrono::microseconds>(waited).count() / report_every
<< " us per frame over the last "sv << report_every << ", with "sv
<< std::chrono::duration_cast<std::chrono::microseconds>(queued).count() / 1000.0 << " ms queued"sv;
waited = {};
}
}
std::shared_ptr<audio_output_t> output; ///< Keeps the listener registered.
std::shared_ptr<audio_mixer_t> mixer; ///< Converts and buffers the guest audio.
std::uint32_t frame_size; ///< Frames handed out per read.
std::uint32_t sample_rate; ///< Frames per second of stream time.
std::chrono::nanoseconds waited {}; ///< Time spent waiting since the last report.
std::uint64_t paced_frames {0}; ///< Frames handed out through the schedule.
static constexpr std::uint64_t report_every = 1000; ///< Frames between reports, 5 s at 5 ms each.
std::chrono::nanoseconds frame_duration; ///< How long one frame of samples plays for.
std::chrono::steady_clock::time_point next_frame {}; ///< When the next frame is due out.
};
/**
@@ -417,8 +364,10 @@
}
BOOST_LOG(info) << "qemu: streaming guest audio as "sv << channels << " channel(s) at "sv << sample_rate << " Hz"sv << (continuous ? ", continuous"sv : ""sv);
auto mixer = output->attach({mapping, (std::size_t) channels}, sample_rate, frame_size, continuous);
// frames go out as soon as the guest has written them: a client's jitter buffer uses every
// millisecond of head start it gets, and holding them back to an even cadence measurably doubled
// the packets it had to conceal on a jittery link
const auto frame_duration = std::chrono::nanoseconds {std::chrono::seconds {1}} * frame_size / sample_rate;
return std::make_unique<mic_t>(output, std::move(mixer), frame_size, sample_rate, frame_duration);
return std::make_unique<mic_t>(output, std::move(mixer));
}
/**
tests/e2e/qemu/e2e_stream.sh +4 −6
@@ -308,7 +308,7 @@
if [[ ${result} == 0 && "${audio}" == 1 ]]; then
# Sunshine must have taken the audio from QEMU, and the tone must not have dropped out
silent="$(sed -n 's/.*"silent_blocks":\([0-9]*\).*/\1/p' "${artifacts}/summary.json")"
burst_fraction="$(sed -n 's/.*"burst_fraction":\([0-9.e+-]*\).*/\1/p' "${artifacts}/summary.json")"
concealed="$(sed -n 's/.*"lost_packets":\([0-9]*\).*/\1/p' "${artifacts}/summary.json")"
audio_streams="$(grep -c "qemu: streaming guest audio" "${work}/sunshine.log" || true)"
if (( audio_streams < sessions )); then
echo "e2e: FAIL: Sunshine captured the guest audio from QEMU ${audio_streams} time(s) for ${sessions} stream(s)" >&2
@@ -316,11 +316,9 @@
elif (( ${silent:-999999} > audio_max_silent )); then
echo "e2e: FAIL: ${silent} silent 10 ms block(s) in the tone (limit ${audio_max_silent})" >&2
result=1
elif (( $(awk -v f="${burst_fraction:-1}" 'BEGIN {print (f > 0.25) ? 1 : 0}') )); then
# packets that arrive in bursts feed a player's jitter buffer unevenly, which it plays as crackle,
# however clean the samples are; one frame per frame interval means almost none follow within a
# millisecond of the one before
echo "e2e: FAIL: ${burst_fraction} of the audio packets arrived in bursts (limit 0.25); the guest's audio isn't paced" >&2
elif (( ${concealed:-999999} > 0 )); then
# nothing is lost on a local link, so anything concealed here means the audio path dropped it
echo "e2e: FAIL: the client had to conceal ${concealed} audio packet(s) on a local connection" >&2
result=1
else
echo "e2e: guest tone received: $(grep -o '"audio":{[^}]*}' "${artifacts}/summary.json")" >&2