Summary

Git LFS stores each large file as one opaque blob keyed by its whole-file SHA256. A single-byte change to a multi-GB file produces a brand-new OID, forcing full re-upload/re-download and storing a near-duplicate copy.

This issue adds content-defined chunking (CDC) using the FastCDC algorithm: files are split into variable-length chunks at content-defined boundaries, each unique chunk is stored once (dedup), and a file is represented by a manifest listing its chunk hashes in order. A local edit re-chunks only the affected region, so almost all chunk hashes stay identical — the client transfers only changed chunks and the backend dedups the rest.

Decisions (locked with user)

• FastCDC algorithm (normalized chunking, NC=2).
• A new chunk-aware transfer API (chunked) alongside the existing basic LFS transfer. Stock git-lfs clients keep working unchanged.
• Full vertical slice across all three backends (Memory / Filesystem / S3).
• Also: fork and enhance git-lfs (https://github.com/git-lfs/git-lfs) so a real client can drive the chunked transfer end-to-end.

Scope (server, ex_git_objectstore)

• Lfs.Chunker — FastCDC (deterministic compile-time gear table, streaming).
• Lfs.Manifest — parse/emit + integrity (reconstructed bytes hash to file OID; chunk sizes sum to total).
• Lfs.Chunk.Store — content-addressed chunk store with dedup + batch existence (exists_many/3).
• Lfs.Chunk — file-level put/get + transfer negotiation.
• Lfs.Chunk.Batch — chunk-aware batch endpoint advertising transfer chunked.

Scope (client, git-lfs fork)

• New chunked transfer adapter: client-side FastCDC matching server boundaries, negotiate via batch API, upload only missing chunks, download manifest + missing chunks, reconstruct with verified OID.
• Interop test driving the forked binary against this server.

Requirements

REQ-LFS-009 (chunker), REQ-LFS-010 (manifest), REQ-LFS-011 (chunk store dedup), REQ-LFS-012 (chunk-aware transfer/batch), plus client interop requirements.

Does not modify the basic transfer path.