maintenance: add new vfs-cache-move maintenance task #720
Conversation
mjcheetham
force-pushed
the
scalar-cache-task
branch
from
6a39733 to
1c29b72
Compare
Introduce a new maintenance task, `vfs-cache-move`, that operates on Scalar or VFS for Git repositories with a per-volume, shared object cache (specified by `gvfs.sharedCache`) to migrate packfiles from the repository object directory to the shared cache. Older versions of `microsoft/git` incorrectly placed packfiles in the repository object directory instead of the shared cache; this task will help clean up existing clones impacted by that issue. Signed-off-by: Matthew John Cheetham <[email protected]>
mjcheetham
force-pushed
the
scalar-cache-task
branch
from
1c29b72 to
4a5255d
Compare
| @@ -158,6 +159,13 @@ pack-refs:: | |||
| need to iterate across many references. See linkgit:git-pack-refs[1] | |||
| for more information. | |||
|
|
|||
| vfs-cache-move:: | |||
| The `vfs-cache-move` task operates only Scalar or VFS for Git | |||
There was a problem hiding this comment.
I think the word "on" is missing between "only" and "Scalar", not that that matters a lot...
| @@ -1345,6 +1345,154 @@ static int maintenance_task_incremental_repack(struct maintenance_run_opts *opts | |||
| return 0; | |||
| } | |||
|
|
|||
|
|
|||
| static void copy_file(const char *srcdir, const char *dstdir, const char *name) | |||
There was a problem hiding this comment.
There's already copy_file() in copy.h, can we use that instead?
| basename = xstrndup(pack_filename, strlen(pack_filename) - 5 /*.pack*/); | ||
| keep_filename = xstrfmt("%s.keep", basename); | ||
| rev_filename = xstrfmt("%s.rev", basename); | ||
| idx_filename = xstrfmt("%s.idx", basename); |
There was a problem hiding this comment.
What should we do with .pack files that have no associated .idx file?
(Could be an in-progress git fetch? Although that would probably have a tmp_* file at first, there is a small time window when it is already renamed to .pack but the .idx has not been renamed yet; But it could also be a left-over from some failed operation...).
| strbuf_release(&path); | ||
|
|
||
| /* Copy all but the index file, which we will *move* atomically */ | ||
| copy_file(srcdir, dstdir, pack_filename); |
There was a problem hiding this comment.
I would like to try hard-linking it first, i.e. the following strategy:
- create a hard-link of the
.packfile to the new location - create hard-links of the corresponding
.keep,.revand finally.idxfiles, too - remove the original
.idxfile, then the.pack,.revand.keepfile, too
Of course, hard-linking can fail, e.g. when the object directories are on different drives. In that instance, we do have to fall back to copying instead (but keeping the operations in the same order).
| touch .git/objects/pack/vfs-12345678.pack && | ||
| touch .git/objects/pack/vfs-12345678.keep && | ||
| touch .git/objects/pack/vfs-12345678.rev && | ||
| touch .git/objects/pack/vfs-12345678.idx && |
There was a problem hiding this comment.
I fear that we have to make this a proper .pack and .idx; This is relatively easy, though:
printf 'blob\ndata <<END\n%s\nEND\n\n' 1 2 3 4 5 |
git -c fastimport.unpackLimit=0 fast-importThe only non-obvious thing is to find out what the pack file is called, as fast-import does not report that ;-)
Introduce a new maintenance task,
vfs-cache-move, that operates on Scalar or VFS for Git repositories with a per-volume, shared object cache (specified bygvfs.sharedCache) to migrate packfiles from the repository object directory to the shared cache.Older versions of
microsoft/gitincorrectly placed packfiles in the repository object directory instead of the shared cache; this task will help clean up existing clones impacted by that issue.