ObjectResolver memoizes pack lookups in the process dictionary:

defp cached_raw_index(%Repo{id: id} = repo, pack_sha) do
  case Process.get({@pack_raw_idx_key, id, pack_sha}) do
    nil -> ... Repo.storage_call(repo, :get_pack_index, [pack_sha]) ...

defp cached_pack_data(%Repo{id: repo_id} = repo, pack_sha) do
  case Process.get(key) do
    nil -> ... Repo.storage_call(repo, :get_pack, [pack_sha]) ...   # File.read/1 of the WHOLE .pack

ExGitObjectstore.blob_sizes/3 fans the SHA list out over Task.async_stream(max_concurrency: 16). Every task is a fresh process with an empty process dictionary, so the cache misses in each one: each task re-reads the pack .idx from disk, and cached_pack_data can File.read/1 the entire .pack.

Measured

Computing the (correct, three-dot) diff for fangorn/hephaestus#74 — 118 files, 159 unique blob SHAs — against the real repository, sampling :erlang.memory/0 at peak:

total            1889.8 MB
system           1859.9 MB
binary           1810.6 MB     <-- refc binaries
processes          29.8 MB     <-- every process heap, combined

1,810 MB of binary memory to produce a diff term of 7.9 MB. The sum of all per-file process heaps is 32.8 MB, so this is not diff data — it is repeated whole-file reads. eprof counted 320 calls to get_pack_index/3 for those 159 SHAs, against a 0.7 MB .idx and a 902.7 MB .pack.

Production is not paying this today

The prod hephaestus repo has 0 packs and 26,908 loose objects, so list_packs returns empty and this path never executes. That is the only reason a 3 GB droplet survives it.

It detonates the moment #324 (server-side gc/repack) lands — which is a precondition for the S3 work — on a box with 3 GB of RAM. Worth fixing before #324, not after.

Direction

A per-process cache is the wrong scope for something whose whole purpose is to be shared. The index is immutable once the pack exists (pack files are content-addressed and never rewritten in place), so it is safe to cache outside the process — the existing ExGitObjectstore.Cache.ETS is the obvious home, keyed {repo_id, pack_sha}.

Reading the entire .pack to serve point lookups is the second half and is the same problem #325 describes for the S3 backend; the fix there (ranged reads) applies to the Filesystem backend as :file.pread/3 against an open handle rather than File.read/1 of the whole file.

Found during the fangorn/anvil#367 performance audit. Related: #324, #325.