ref:main

feat(runner): production-ready tier 1 service management (#22) #27

merged colechristensen cole.christensen@gmail.com wants to merge feat/runner-production-ready into main

Summary

Closes #22. Makes the anvil runner production-deployable across the three tier-1 modes — standalone, user systemd, system systemd (and user launchd / system launchd on macOS) — with one coherent control surface and N parallel job slots first-class.

Single PR by user request — everything needed for production deployment in one release.

What lands

Runtime foundation

  • PID file + RAII Guard at `~/.anvil-runner/runner.pid`. Single-instance check on start (live → refuse, stale → warn + overwrite). `–pid-file` override.
  • SIGTERM + SIGINT both trigger graceful drain. SIGHUP + SIGPIPE absorbed so dropped terminals / broken pipes don’t kill the runner.
  • Cancel propagation to the executor — child gets SIGTERM, 5s grace, SIGKILL. Exit code 130 to distinguish from a normal failure.
  • N-slot drain — `–shutdown-timeout` (default 60s) is a single wall-clock budget for all slots to wind up.
  • Heartbeat reports capacity — `{"parallel": N, "slots_busy": M}` so the server can route jobs intelligently. Atomic counter, RAII reservation panic-safe.

New CLI subcommands

  • `anvil runner stop [–force] [–timeout]` — reads PID file, SIGTERM, polls for clean exit, optional SIGKILL fallback.
  • `anvil runner restart` — stop + `exec` into a fresh `runner start` (handles install.sh binary-upgrade case cleanly).

Service install overhauled

  • Explicit `–scope user|system` (default detected). No more brittle `getuid()` heuristic.
  • `–scope=system` requires `–user-account ` — refuses to install a system service running as root.
  • `~/.anvil-runner/service.json` records scope + unit path + config path + user account + parallel. `svc-*` commands read it; no more guessing.
  • macOS `–scope=system` writes `/Library/LaunchDaemons/` with `UserName`/`GroupName`. Previously only user-LaunchAgents was supported.

Hardened unit files

  • systemd: `TimeoutStopSec=90`, `Restart=always`, `RestartSec=5`, `KillSignal=SIGTERM`, `ProtectSystem=strict`, `ProtectHome=true`, `NoNewPrivileges=true`, `PrivateTmp=true`, `StandardOutput=journal`. System scope adds `User=`, `Group=`, `WorkingDirectory=/var/lib/anvil-runner`.
  • launchd: `ProcessType=Background`, `ThrottleInterval=5`, log paths under `~/Library/Logs/anvil-runner/` (user) or `/var/log/anvil-runner/` (system).

Test plan

  • `cargo test` — 102/102 pass
    • `pid_file`: absent / stale / dead-pid / live-pid / Guard lifecycle
    • `slot_counter`: increment + RAII decrement
    • `service_mode`: JSON round-trip
    • `commands/runner`: clap parsing for stop / restart / service-install flags
  • `cargo clippy –all-targets – -D warnings` clean
  • `cargo fmt` clean
  • CLI surface verified live: `anvil runner –help`, `runner stop –help`, `runner start –help`, `runner service install –help` all show the new flags/verbs

Out of scope (filed as follow-ups)

These would be useful but each is independently shippable and none blocks production deployment. Operators can use `systemctl status anvil-runner` and `journalctl -u anvil-runner` for now:

  • `runner status` runtime view + `runner logs` + `runner doctor`
  • `runner start –detach` standalone background mode
  • macOS modern launchctl `bootstrap`/`bootout` migration (current code uses `load`/`unload`, which still works as compatibility shim)
  • Multiple distinct runner identities on one host via `–service-name`
  • Server-side `runner_shutdown` cancel reason for jobs killed mid-run

Closes #22

Created Jun 05, 2026 at 01:14 UTC | Merged Jun 05, 2026 at 01:26 UTC by colechristensen cole.christensen@gmail.com