Skip to content

Commit

Permalink
Introduce 'git backfill' to get missing blobs in a partial clone (#5172)
Browse files Browse the repository at this point in the history
This change introduces the `git backfill` command which uses the path
walk API to download missing blobs in a blobless partial clone.

By downloading blobs that correspond to the same file path at the same
time, we hope to maximize the potential benefits of delta compression
against multiple versions.

These downloads occur in a configurable batch size, presenting a
mechanism to perform "resumable" clones: `git clone --filter=blob:none`
gets the commits and trees, then `git backfill` will download all
missing blobs. If `git backfill` is interrupted partway through, it can
be restarted and will redownload only the missing objects.

When combining blobless partial clones with sparse-checkout, `git
backfill` will assume its `--sparse` option and download only the blobs
within the sparse-checkout. Users may want to do this as the repo size
will still be smaller than the full repo size, but commands like `git
blame` or `git log -L` will not suffer from many one-by-one blob
downloads.

Future directions should consider adding a pathspec or file prefix to
further focus which paths are being downloaded in a batch.
  • Loading branch information
dscho authored and Git for Windows Build Agent committed Jan 27, 2025
2 parents 717f037 + 76f6e8b commit 4ec6db9
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion Documentation/git-backfill.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ git-backfill - Download missing objects in a partial clone
SYNOPSIS
--------
[verse]
'git backfill' [--batch-size=<n>] [--[no-]sparse]
(EXPERIMENTAL) 'git backfill' [--batch-size=<n>] [--[no-]sparse]

DESCRIPTION
-----------
Expand Down
2 changes: 1 addition & 1 deletion builtin/backfill.c
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
#include "path-walk.h"

static const char * const builtin_backfill_usage[] = {
N_("git backfill [--batch-size=<n>] [--[no-]sparse]"),
N_("(EXPERIMENTAL) git backfill [--batch-size=<n>] [--[no-]sparse]"),
NULL
};

Expand Down

0 comments on commit 4ec6db9

Please sign in to comment.