ref:main

Make CI service readiness honest: TCP health probe + fail on timeout #38

merged colechristensen cole.christensen@gmail.com wants to merge fix/29-ci-service-tcp-healthcheck into main

Why

CI jobs with a postgres service intermittently failed at DB setup:

tcp connect (localhost:5432): connection refused — :econnrefused
** (Mix) The database for <Repo> couldn't be created: killed

Deterministic with postgres:18 (fangorn/reader test job); postgres:16 (fangorn/mail) usually won the race. Not memory, not pg18 startup — a bare postgres:18 starts fine.

Root cause: service_manager.rs ran pg_isready -q with no -h, probing the Unix socket. The postgres image’s first-boot initdb runs a temporary server that listens on the socket ONLY (listen_addresses=''); the socket check passes before the real TCP listener is up, so the job connects over TCP and gets econnrefused. Secondary: wait_for_health warned and proceeded anyway on a real timeout.

What

  • postgres health check probes TCP (pg_isready -q -h 127.0.0.1) — during initdb’s socket-only phase TCP is refused, so the wait correctly continues until the real listener is accepting the connections the job will make.
  • wait_for_health returns bool; timeout is now a hard failure (cleanup + clear did not become healthy within 60s error) instead of silently proceeding. Window 30s → 60s for slower first-boot inits.
  • Regression test: postgres_health_check_probes_tcp_not_socket.

Shared-primitive fix — makes readiness honest for every postgres CI service, not a per-repo workaround. Verified: cargo fmt/clippy clean, 148+23 tests pass.

Follow-up (not in this PR)

Needs an anvil-cli release + runner update on carl/xps to take effect. Once deployed, fangorn/reader’s test job (postgres:18) should go green without any per-repo readiness hack.

Closes #29

🤖 Generated with Claude Code

Created Jul 17, 2026 at 06:11 UTC | Merged Jul 17, 2026 at 06:31 UTC by colechristensen cole.christensen@gmail.com