anvil-cli does not use prepare — move the cross-compile toolchain into it #43


      fangorn/anvil-cli

public

ref:main

← Linked to a private epic

anvil-cli does not use `prepare` — move the cross-compile toolchain into it #43

closed Opened by cole.christensen@gmail.com

Measurement

build-runner on the merged-main run took 1862s. From its own log:

Phase	Time
linux arm64 / amd64 / macos arm64 / amd64 compiles	31.8 + 19.0 + 29.0 + 29.5 = 109s
native `cargo build --release` (same run, calibration)	29s
everything else	~1750s

That ~1750s is: downloading zig, compiling cargo-zigbuild from source (~130 crates), fetching the macOS SDK, four rustup target add, and apt-get install bash curl jq xz-utils. Every run. It is also why the job exceeded its 1800s timeout and published a broken release (#40).

There is no emulation involved anywhere — cargo-zigbuild cross-compiles all four targets from one Linux host, and cross-compilation is not the bottleneck.

Fix

Give build-runner a step-level prepare (see fangorn/anvil#355) holding the toolchain install, and reduce ci/build-runner.sh to build + verify + publish. Step-level rather than pipeline-level so fmt (2s) and check (10s) don’t carry a macOS SDK.

One correctness requirement

ZIG_VERSION, ZIGBUILD_VERSION and MACOS_SDK_VERSION currently live as shell variables inside ci/build-runner.sh, and the prepared-image hash covers only the base image and the prepare command strings — not the contents of any script. If the installs move to prepare while the versions stay in the script, bumping a version would not invalidate the image, and the pipeline would keep silently using the old toolchain.

The version literals must therefore appear in .anvil.yml in the prepare commands themselves. This is a constraint of the cache key, not a style choice.

Also worth doing while in here

Delete the six cache: blocks — they are inert (fangorn/anvil#358) and misrepresent what the pipeline does.

Part of fangorn/anvil#354.

cole.christensen@gmail.com commented 2026-07-22 22:59

Revised plan: split builds by ISA across native-arch runners

Correcting the framing above. My earlier “compiles are only ~109s” was measured off a warm per-target target/ dir (the workspace persists per-worker with no git clean); the log shows only Compiling anvil-cli + Finished, i.e. every dependency including ring (which carries per-arch C + assembly) was already built. On a cold worker or any Cargo.lock change, cross-ISA targets recompile ring’s C/asm for the foreign arch via zig — genuinely slow. Same-ISA cross-OS (linux/arm64 → macos/arm64) reuses the arch codegen and is cheap.

So both levers are needed and they’re orthogonal:

prepare removes the ~1750s toolchain re-acquisition (this issue).
Native-ISA runners remove the cross-ISA codegen.

Target partition (zero cross-ISA compiles)

Runner (`runs_on`)	Native	Same-ISA cross-OS
`[linux, arm64]`	linux/arm64	macos/arm64
`[linux, amd64]`	linux/amd64	macos/amd64

Pipeline shape

build-arm64 — runs_on: [linux, arm64], step-level prepare (zig + SDK + targets), builds its two same-ISA binaries, anvil release uploads them.
build-amd64 — runs_on: [linux, amd64], same, its two.
publish — creates the release, then unifies checksums.

Blocker

Needs fangorn/anvil#361 (affinity must not deadlock a job the pinned worker can’t run). Until that ships, a run pins to whichever arch claims first and the other arch’s build job hangs :queued. This is the critical path for the multi-arch form; the single-runner zig-cross form (this issue as originally written) works today and doesn’t need it.

Fan-in is achievable today (no #312 needed)

The two build jobs are independent (no depends_on), so they don’t need portable build state. They converge only at publish, and that can go through the release-asset API: each build job uploads its own binaries; the publish job uses anvil release download to pull all four and compute a unified SHA256SUMS. The one genuine coupling (checksums spanning all four) is a few lines of text over an API that already exists.

Two cautions on the publish job:

It compounds the non-atomic publish problem (#40): with the release created before multiple uploaders finish, a mid-upload failure leaves a partial release. The release should be created last — build+upload binaries to a staging area or draft, publish only once all four + SHA256SUMS are present.
Ordering: whichever job creates the release must run before the uploads; simplest is a dedicated publish job depends_on: [build-arm64, build-amd64].

Prepared-image sharing is not on this critical path

Each runner builds and keeps its own prepared image (per-worker, no registry sharing), so fangorn/anvil-cli#44 (arch-tagged tags) only matters if we later adopt fangorn/anvil#359 to share them. Not required for the split.

cole.christensen@gmail.com commented 2026-07-26 20:08

Fixed by #44 and #45 (merged).

Verified on main (.anvil.yml): the runner build jobs carry step-level prepare: blocks holding the toolchain install (zig, cargo-zigbuild, macOS SDK, rustup targets), so fmt/check don’t drag a macOS SDK around.

The correctness requirement is satisfied: the version literals now live in the prepare commands themselves in .anvil.yml (e.g. https://ziglang.org/download/0.15.2/zig-$(uname -m)-linux-0.15.2.tar.xz and cargo install --locked cargo-zigbuild --version 0.23.0), not as shell variables inside ci/build-runner.sh — so a version bump changes the prepared-image hash and correctly invalidates the cache.

The inert cache: blocks are also deleted, with a note in place of them explaining why.

Back to issues