fangorn/ex_git_objectstore
public
perf: whole-pack escalation rebuilds O(n) SHA→offset maps (~183ms), dominates diff #73
Links
No links yet.
The adaptive ObjectResolver escalates a pack to whole-pack mode after 64 ranged reads. That escalation ran Index.parse (full sha→offset map, ~150ms) AND build_sha_cache (rebuilds it into a second map, ~33ms) over the repo’s 134,921 objects — ~183ms of O(n) work — even though Index.lookup_in_raw resolves an offset by binary search on the raw index in ~0.001ms.
Any op crossing 64 reads pays the full 183ms once. On a real 134k-object repo, diff HEAD~50..HEAD (151 reads): the single escalation read was 239ms of a 261ms diff (91%); every other read ~0.15ms.
Fix: whole_pack_read loads the pack once (still amortizes per-object HTTP for bulk S3 readers) but resolves offsets via the same lookup_in_raw binary search as the ranged path and reads objects from the in-memory pack (reusing read_object_ranged). Deletes cached_pack_index/cached_sha_cache/build_sha_cache.
Result: cold diff_commits 261ms → 45ms (native git 16.6ms); blob-read phase 263ms → 13.8ms. 982 tests pass; read_object_ranged ≡ read_object already asserted in reader_test. Fixed on feat/s3-pack-read-perf (PR #44).