feat(runner): advertise local models and their capabilities on heartbeat
The routed model picker in Anvil is populated by the runner, not the
server: `Anvil.Agents.Inference.list_inference_models/1` unions
`metadata["inference_models"]` across the online inference runners in
scope. The heartbeat only ever sent `parallel` and `slots_busy`, so that
union was always empty and the picker rendered the configured model as
unchangeable text — indistinguishable from a provider that cannot
enumerate. The server side has been documented since the feature landed
(`docs/agents.md`, "Routed discovery has a runner-side half"); this is
that half.
Each entry carries the model name and the capabilities Ollama reports for
it:
{"name": "llama3.2:latest", "capabilities": ["completion", "tools"]}
Anvil filters on these. An agent chat always sends a tool set, so a model
without `tools` fails on its first call — offering it is offering a
guaranteed error.
A model whose capabilities Ollama does not report is sent with the key
omitted rather than an empty list: the server treats that as *unknown*
and keeps the model, which is the right behaviour when nothing is known.
A bare string remains a valid entry server-side, so a fleet running an
older runner keeps working.
Best-effort by design. Ollama being down or slow must never cost us a
heartbeat, so a failed lookup omits the key and the heartbeat goes out
regardless; the previously reported list stays in place server-side,
where reporting an empty list would retract it. The lookup has its own
3s timeout, well inside the heartbeat interval.
Verified end to end against a local Anvil dev instance: the picker lists
the local Ollama's models grouped by provider, selecting one routes the
run through this runner, and a model reporting only `["completion"]` is
correctly hidden when the chat requires `tools`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SHA:
7ffcb3e717da08d5bf688cf8aaeed3bf78d3d73f
Author:
Cole Christensen <cole.christensen@gmail.com>
Date:
2026-08-05 03:58
Parents:
2f5d136
1 files changed
+73
-1
| Type | ||
|---|---|---|
|
|
src/runner/heartbeat.rs | +73 −1 |
|
||