From 1e951c647365be2e93bceddf7bd13173ed2cb1ce Mon Sep 17 00:00:00 2001
From: Jeff King <peff@peff.net>
Date: Wed, 28 Apr 2021 11:42:30 -0400
Subject: [PATCH 1/2] pack-bitmap: clean up include_check after use
When a bitmap walk has to traverse (to fill in non-bitmapped objects),
we use rev_info's include_check mechanism to let us stop the traversal
early. But after setting the function and its data parameter, we never
clean it up. This means that if the rev_info is used for a subsequent
traversal without bitmaps, it will unexpectedly call into our
include_check function (worse, it will do so pointing to a now-defunct
stack variable in include_check_data, likely resulting in a segfault).
There's no code which does this now, but it's an accident waiting to
happen. Let's clean up after ourselves in the bitmap code.
Reported-by: David Emett <dave@sp4m.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
pack-bitmap.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 1ebe0c8162..84a9195e72 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -630,6 +630,9 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git,
traverse_commit_list_filtered(filter, revs,
show_commit, show_object,
&show_data, NULL);
+
+ revs->include_check = NULL;
+ revs->include_check_data = NULL;
}
return base;
From 2ba582ba4c62fae506174ce326deab082d962e63 Mon Sep 17 00:00:00 2001
From: Jeff King <peff@peff.net>
Date: Wed, 28 Apr 2021 11:42:43 -0400
Subject: [PATCH 2/2] prune: save reachable-from-recent objects with bitmaps
We pass our prune expiration to mark_reachable_objects(), which will
traverse not only the reachable objects, but consider any recent ones as
tips for reachability; see d3038d22f9 (prune: keep objects reachable
from recent objects, 2014-10-15) for details.
However, this interacts badly with the bitmap code path added in
fde67d6896 (prune: use bitmaps for reachability traversal, 2019-02-13).
If we hit the bitmap-optimized path, we return immediately to avoid the
regular traversal, accidentally skipping the "also traverse recent"
code.
Instead, we should do an if-else for the bitmap versus regular
traversal, and then follow up with the "recent" traversal in either
case. This reuses the "rev_info" for a bitmap and then a regular
traversal, but that should work OK (the bitmap code clears the pending
array in the usual way, just like a regular traversal would).
Note that I dropped the comment above the regular traversal here. It
has little explanatory value, and makes the if-else logic much harder to
read.
Here are a few variants that I rejected:
- it seems like both the reachability and recent traversals could be
done in a single traversal. This was rejected by d3038d22f9 (prune:
keep objects reachable from recent objects, 2014-10-15), though the
balance may be different when using bitmaps. However, there's a
subtle correctness issue, too: we use revs->ignore_missing_links for
the recent traversal, but not the reachability one.
- we could try using bitmaps for the recent traversal, too, which
could possibly improve performance. But it would require some fixes
in the bitmap code, which uses ignore_missing_links for its own
purposes. Plus it would probably not help all that much in practice.
We use the reachable tips to generate bitmaps, so those objects are
likely not covered by bitmaps (unless they just became unreachable).
And in general, we expect the set of unreachable objects to be much
smaller anyway, so there's less to gain.
The test in t5304 detects the bug and confirms the fix.
I also beefed up the tests in t6501, which covers the mtime-checking
code more thoroughly, to handle the bitmap case (in addition to just
"loose" and "packed" cases). Interestingly, this test doesn't actually
detect the bug, because it is running "git gc", and not "prune"
directly. And "gc" will call "repack" first, which does not suffer the
same bug. So the old-but-reachable-from-recent objects get scooped up
into the new pack along with the actually-recent objects, which gives
both a recent mtime. But it seemed prudent to get more coverage of the
bitmap case for related code.
Reported-by: David Emett <dave@sp4m.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
reachable.c | 13 ++++---------
t/t5304-prune.sh | 16 ++++++++++++++++
t/t6501-freshen-objects.sh | 22 ++++++++++++++++------
3 files changed, 36 insertions(+), 15 deletions(-)
diff --git a/reachable.c b/reachable.c
index 77a60c70a5..a088717eb5 100644
--- a/reachable.c
+++ b/reachable.c
@@ -227,17 +227,12 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog,
if (bitmap_git) {
traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen);
free_bitmap_index(bitmap_git);
- return;
+ } else {
+ if (prepare_revision_walk(revs))
+ die("revision walk setup failed");
+ traverse_commit_list(revs, mark_commit, mark_object, &cp);
}
- /*
- * Set up the revision walk - this will move all commits
- * from the pending list to the commit walking list.
- */
- if (prepare_revision_walk(revs))
- die("revision walk setup failed");
- traverse_commit_list(revs, mark_commit, mark_object, &cp);
-
if (mark_recent) {
revs->ignore_missing_links = 1;
if (add_unseen_recent_objects_to_traversal(revs, mark_recent))
diff --git a/t/t5304-prune.sh b/t/t5304-prune.sh
index b447ce56a9..3475b06aeb 100755
--- a/t/t5304-prune.sh
+++ b/t/t5304-prune.sh
@@ -352,4 +352,20 @@ test_expect_success 'trivial prune with bitmaps enabled' '
test_must_fail git cat-file -e $blob
'
+test_expect_success 'old reachable-from-recent retained with bitmaps' '
+ git repack -adb &&
+ to_drop=$(echo bitmap-from-recent-1 | git hash-object -w --stdin) &&
+ test-tool chmtime -86400 .git/objects/$(test_oid_to_path $to_drop) &&
+ to_save=$(echo bitmap-from-recent-2 | git hash-object -w --stdin) &&
+ test-tool chmtime -86400 .git/objects/$(test_oid_to_path $to_save) &&
+ tree=$(printf "100644 blob $to_save\tfile\n" | git mktree) &&
+ test-tool chmtime -86400 .git/objects/$(test_oid_to_path $tree) &&
+ commit=$(echo foo | git commit-tree $tree) &&
+ git prune --expire=12.hours.ago &&
+ git cat-file -e $commit &&
+ git cat-file -e $tree &&
+ git cat-file -e $to_save &&
+ test_must_fail git cat-file -e $to_drop
+'
+
test_done
diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh
index 75210f012b..10662456ae 100755
--- a/t/t6501-freshen-objects.sh
+++ b/t/t6501-freshen-objects.sh
@@ -43,15 +43,25 @@ commit () {
}
maybe_repack () {
- if test -n "$repack"; then
+ case "$title" in
+ loose)
+ : skip repack
+ ;;
+ repack)
git repack -ad
- fi
+ ;;
+ bitmap)
+ git repack -adb
+ ;;
+ *)
+ echo >&2 "unknown test type in maybe_repack"
+ return 1
+ ;;
+ esac
}
-for repack in '' true; do
- title=${repack:+repack}
- title=${title:-loose}
-
+for title in loose repack bitmap
+do
test_expect_success "make repo completely empty ($title)" '
rm -rf .git &&
git init