Taylor Blau 4dc16e2cb0 gc: introduce `gc.recentObjectsHook` ... This patch introduces a new multi-valued configuration option, `gc.recentObjectsHook` as a means to mark certain objects as recent (and thus exempt from garbage collection), regardless of their age. When performing a garbage collection operation on a repository with unreachable objects, Git makes its decision on what to do with those object(s) based on how recent the objects are or not. Generally speaking, unreachable-but-recent objects stay in the repository, and older objects are discarded. However, we have no convenient way to keep certain precious, unreachable objects around in the repository, even if they have aged out and would be pruned. Our options today consist of: - Point references at the reachability tips of any objects you consider precious, which may be undesirable or infeasible if there are many such objects. - Track them via the reflog, which may be undesirable since the reflog's lifetime is limited to that of the reference it's tracking (and callers may want to keep those unreachable objects around for longer). - Extend the grace period, which may keep around other objects that the caller *does* want to discard. - Manually modify the mtimes of objects you want to keep. If those objects are already loose, this is easy enough to do (you can just enumerate and `touch -m` each one). But if they are packed, you will either end up modifying the mtimes of *all* objects in that pack, or be forced to write out a loose copy of that object, both of which may be undesirable. Even worse, if they are in a cruft pack, that requires modifying its `*.mtimes` file by hand, since there is no exposed plumbing for this. - Force the caller to construct the pack of objects they want to keep themselves, and then mark the pack as kept by adding a ".keep" file. This works, but is burdensome for the caller, and having extra packs is awkward as you roll forward your cruft pack. This patch introduces a new option to the above list via the `gc.recentObjectsHook` configuration, which allows the caller to specify a program (or set of programs) whose output is treated as a set of objects to treat as recent, regardless of their true age. The implementation is straightforward. Git enumerates recent objects via `add_unseen_recent_objects_to_traversal()`, which enumerates loose and packed objects, and eventually calls add_recent_object() on any objects for which `want_recent_object()`'s conditions are met. This patch modifies the recency condition from simply "is the mtime of this object more recent than the cutoff?" to "[...] or, is this object mentioned by at least one `gc.recentObjectsHook`?". Depending on whether or not we are generating a cruft pack, this allows the caller to do one of two things: - If generating a cruft pack, the caller is able to retain additional objects via the cruft pack, even if they would have otherwise been pruned due to their age. - If not generating a cruft pack, the caller is likewise able to retain additional objects as loose. A potential alternative here is to introduce a new mode to alter the contents of the reachable pack instead of the cruft one. One could imagine a new option to `pack-objects`, say `--extra-reachable-tips` that does the same thing as above, adding the visited set of objects along the traversal to the pack. But this has the unfortunate side-effect of altering the reachability closure of that pack. If parts of the unreachable object graph mentioned by one or more of the "extra reachable tips" programs is not closed, then the resulting pack won't be either. This makes it impossible in the general case to write out reachability bitmaps for that pack, since closure is a requirement there. Instead, keep these unreachable objects in the cruft pack (or set of unreachable, loose objects) instead, to ensure that we can continue to have a pack containing just reachable objects, which is always safe to write a bitmap over. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>		2023-06-12 14:12:20 -07:00
..
RelNotes	The sixteenth batch	2023-05-02 10:13:50 -07:00
config	gc: introduce `gc.recentObjectsHook`	2023-06-12 14:12:20 -07:00
howto	new-command.txt: update reference to builtin docs	2023-02-06 14:07:33 -08:00
includes	docs: add and use include template for config/* includes	2022-09-07 09:46:05 -07:00
mergetools	vimdiff: add tool documentation	2022-04-03 15:09:52 -07:00
technical	Merge branch 'ds/bundle-uri-5'	2023-02-15 17:11:52 -08:00
.gitattributes	…
.gitignore	doc: remove manpage-base-url workaround	2023-04-05 14:18:53 -07:00
CodingGuidelines	treewide: ensure one of the appropriate headers is sourced first	2023-02-23 17:25:28 -08:00
Makefile	Merge branch 'fc/doc-stop-using-manversion'	2023-04-21 15:35:04 -07:00
MyFirstContribution.txt	Merge branch 'la/mfc-markup-fix'	2023-04-17 18:05:12 -07:00
MyFirstObjectWalk.txt	Documentation: clean up a few misspelled word typos	2022-09-21 10:28:35 -07:00
ReviewingGuidelines.txt	Documentation: add ReviewingGuidelines	2022-09-19 14:36:08 -07:00
SubmittingPatches	SubmittingPatches: clarify MUA discussion with "the"	2023-04-05 14:50:25 -07:00
ToolsForGit.txt	Documentation/ToolsForGit.txt: Tools for developing Git	2022-04-21 09:50:25 -07:00
asciidoc.conf	doc: asciidoc: remove custom header macro	2023-04-05 21:37:45 -07:00
asciidoctor-extensions.rb	…
blame-options.txt	blame: use different author name for fake commit generated by --contents	2023-04-24 15:16:31 -07:00
build-docdep.perl	Documentation/build-docdep.perl: generate sorted output	2022-10-21 11:39:38 -07:00
cat-texi.perl	…
cmd-list.perl	cmd-list.perl: fix identifying man sections	2022-09-23 10:01:07 -07:00
config.txt	Merge branch 'ds/bundle-uri-3'	2022-10-30 21:04:44 -04:00
date-formats.txt	…
diff-format.txt	diff-format.txt: correct misleading wording	2022-06-13 11:57:03 -07:00
diff-generate-patch.txt	docs: link generating patch sections	2023-01-13 12:55:14 -08:00
diff-options.txt	diff: add --default-prefix option	2023-03-09 08:32:21 -08:00
doc-diff	…
docbook-xsl.css	…
docbook.xsl	…
everyday.txto	…
fetch-options.txt	Merge branch 'rc/fetch-refetch'	2022-04-04 10:56:23 -07:00
fix-texi.perl	…
fsck-msgids.txt	fsck: document the new `gitattributes` message IDs	2023-01-16 12:03:14 -08:00
git-add.txt	docs & comments: replace mentions of "git-add--interactive.perl"	2023-02-06 15:03:34 -08:00
git-am.txt	am: refer to format-patch in the documentation	2023-03-21 13:18:45 -07:00
git-annotate.txt	doc txt & -h consistency: make "annotate" consistent	2022-10-13 09:32:57 -07:00
git-apply.txt	Documentation: render dash correctly	2023-01-23 09:40:14 -08:00
git-archimport.txt	…
git-archive.txt	archive: add --mtime	2023-02-18 09:29:13 -08:00
git-bisect-lk2009.txt	git-bisect-lk2009: update nist report link	2023-01-13 11:58:51 -08:00
git-bisect.txt	…
git-blame.txt	blame: allow --contents to work with non-HEAD commit	2023-03-24 12:05:22 -07:00
git-branch.txt	branch, for-each-ref, tag: add option to omit empty lines	2023-04-13 08:07:45 -07:00
git-bugreport.txt	builtin/bugreport.c: create '--diagnose' option	2022-08-12 13:20:02 -07:00
git-bundle.txt	Merge branch 'jk/bundle-use-dash-for-stdfiles'	2023-03-19 15:03:12 -07:00
git-cat-file.txt	git-cat-file.txt: fix list continuations rendering literally	2023-01-18 08:24:39 -08:00
git-check-attr.txt	attr: add flag `--source` to work with tree-ish	2023-01-14 08:49:55 -08:00
git-check-ignore.txt	doc: check-ignore: code-quote an exclamation mark	2022-02-03 11:13:49 -08:00
git-check-mailmap.txt	…
git-check-ref-format.txt	…
git-checkout-index.txt	checkout-index: add --ignore-skip-worktree-bits option	2022-01-13 13:49:45 -08:00
git-checkout.txt	doc: git-checkout: reorganize examples	2023-04-18 15:47:13 -07:00
git-cherry-pick.txt	git-cherry-pick.txt: do not use 'ORIG_HEAD' in example	2023-01-13 09:55:45 -08:00
git-cherry.txt	…
git-citool.txt	…
git-clean.txt	Merge branch 'ab/doc-synopsis-and-cmd-usage'	2022-10-28 11:26:54 -07:00
git-clone.txt	clone: error specifically with --local and symlinked objects	2023-04-11 08:46:09 -07:00
git-column.txt	docs: add CONFIGURATION sections that fuzzy map to built-ins	2022-09-07 09:46:07 -07:00
git-commit-graph.txt	Merge branch 'ab/doc-synopsis-and-cmd-usage'	2022-10-28 11:26:54 -07:00
git-commit-tree.txt	…
git-commit.txt	docs: add CONFIGURATION sections that map to a built-in	2022-09-07 09:46:06 -07:00
git-config.txt	Documentation: define protected configuration	2022-07-14 15:08:29 -07:00
git-count-objects.txt	…
git-credential-cache--daemon.txt	doc txt & -h consistency: fix mismatching labels	2022-10-13 09:32:56 -07:00
git-credential-cache.txt	Documentation: clarify that cache forgets credentials if the system restarts	2023-01-29 09:21:07 -08:00
git-credential-store.txt	…
git-credential.txt	Merge branch 'mc/credential-helper-www-authenticate'	2023-03-17 14:03:10 -07:00
git-cvsexportcommit.txt	…
git-cvsimport.txt	…
git-cvsserver.txt	git-cvsserver: clarify directory list	2022-07-19 12:45:31 -07:00
git-daemon.txt	daemon: clarify directory arguments	2022-07-19 12:45:31 -07:00
git-describe.txt	…
git-diagnose.txt	builtin/diagnose.c: add '--mode' option	2022-08-12 13:20:02 -07:00
git-diff-files.txt	doc txt & -h consistency: correct padding around "[]()"	2022-10-13 09:32:55 -07:00
git-diff-index.txt	diff-index.txt: update raw output format in examples	2022-06-13 11:59:16 -07:00
git-diff-tree.txt	…
git-diff.txt	Merge branch 'rs/diff-caret-bang-with-parents'	2022-10-25 17:11:43 -07:00
git-difftool.txt	mergetool: new config guiDefault supports auto-toggling gui by DISPLAY	2023-04-05 21:03:29 -07:00
git-fast-export.txt	doc SYNOPSIS: consistently use ' for commands	2022-10-13 09:32:54 -07:00
git-fast-import.txt	docs: add CONFIGURATION sections that map to a built-in	2022-09-07 09:46:06 -07:00
git-fetch-pack.txt	builtin/fetch-pack: add --refetch option	2022-03-28 10:25:52 -07:00
git-fetch.txt	doc: fix non-existent config name	2023-01-13 17:33:32 -08:00
git-filter-branch.txt	…
git-fmt-merge-msg.txt	merge: allow to pretend a merge is made into a different branch	2021-12-20 14:55:02 -08:00
git-for-each-ref.txt	Merge branch 'ow/ref-filter-omit-empty'	2023-04-21 15:35:05 -07:00
git-for-each-repo.txt	…
git-format-patch.txt	Merge branch 'ah/format-patch-thread-doc'	2023-04-21 15:35:05 -07:00
git-fsck-objects.txt	…
git-fsck.txt	fsck: document msg-id	2022-10-25 15:44:18 -07:00
git-fsmonitor--daemon.txt	fsmonitor: add documentation for allowRemote and socketDir options	2022-10-05 11:05:23 -07:00
git-gc.txt	builtin/gc.c: make `gc.cruftPacks` enabled by default	2023-04-18 14:56:48 -07:00
git-get-tar-commit-id.txt	…
git-grep.txt	grep docs: de-duplicate configuration sections	2022-09-07 09:46:05 -07:00
git-gui.txt	…
git-hash-object.txt	doc txt & -h consistency: correct padding around "[]()"	2022-10-13 09:32:55 -07:00
git-help.txt	git docs: add a category for file formats, protocols and interfaces	2022-08-04 14:12:23 -07:00
git-hook.txt	hook: support a --to-stdin=<path> option	2023-02-08 12:50:03 -08:00
git-http-backend.txt	…
git-http-fetch.txt	…
git-http-push.txt	…
git-imap-send.txt	docs: add and use include template for config/* includes	2022-09-07 09:46:05 -07:00
git-index-pack.txt	index-pack: document and test the --promisor option	2022-03-09 10:25:26 -08:00
git-init-db.txt	…
git-init.txt	docs: add CONFIGURATION sections that fuzzy map to built-ins	2022-09-07 09:46:07 -07:00
git-instaweb.txt	…
git-interpret-trailers.txt	Merge branch 'ab/doc-synopsis-and-cmd-usage'	2022-10-28 11:26:54 -07:00
git-log.txt	Merge branch 'ab/dedup-config-and-command-docs'	2022-09-14 12:56:40 -07:00
git-ls-files.txt	ls-files: guide folks to --exclude-standard over other --exclude* options	2023-01-13 11:55:17 -08:00
git-ls-remote.txt	doc/ls-remote: clarify pattern format	2023-02-10 21:57:51 -08:00
git-ls-tree.txt	ls-tree doc: document interaction with submodules	2022-04-08 11:21:11 -07:00
git-mailinfo.txt	docs: add CONFIGURATION sections that map to a built-in	2022-09-07 09:46:06 -07:00
git-mailsplit.txt	…
git-maintenance.txt	maintenance: add option to register in a specific config	2022-11-14 22:39:25 -05:00
git-merge-base.txt	doc txt & -h consistency: correct padding around "[]()"	2022-10-13 09:32:55 -07:00
git-merge-file.txt	update documentation for new zdiff3 conflictStyle	2021-12-01 14:45:59 -08:00
git-merge-index.txt	…
git-merge-one-file.txt	…
git-merge-tree.txt	Merge branch 'as/doc-markup-fix'	2023-03-19 15:03:11 -07:00
git-merge.txt	git-merge.txt: mention 'ORIG_HEAD' in the Description	2023-01-13 09:55:46 -08:00
git-mergetool--lib.txt	…
git-mergetool.txt	mergetool: new config guiDefault supports auto-toggling gui by DISPLAY	2023-04-05 21:03:29 -07:00
git-mktag.txt	…
git-mktree.txt	fix typo in git-mktree.txt	2022-02-02 14:50:09 -08:00
git-multi-pack-index.txt	midx.c: prevent `expire` from removing the cruft pack	2022-09-21 10:21:46 -07:00
git-mv.txt	doc txt & -h consistency: fix mismatching labels	2022-10-13 09:32:56 -07:00
git-name-rev.txt	name-rev: deprecate --stdin in favor of --annotate-stdin	2022-01-10 09:39:26 -08:00
git-notes.txt	notes docs: de-duplicate and combine configuration sections	2022-09-07 09:46:06 -07:00
git-p4.txt	git-p4: improve encoding handling to support inconsistent encodings	2022-05-04 10:30:01 -07:00
git-pack-objects.txt	builtin/pack-objects.c: --cruft without expiration	2022-05-26 15:48:26 -07:00
git-pack-redundant.txt	pack-redundant: document deprecation	2023-03-30 07:50:43 -07:00
git-pack-refs.txt	…
git-patch-id.txt	builtin: patch-id: add --verbatim as a command mode	2022-10-24 15:44:20 -07:00
git-prune-packed.txt	doc txt & -h consistency: correct padding around "[]()"	2022-10-13 09:32:55 -07:00
git-prune.txt	…
git-pull.txt	…
git-push.txt	Doc: document push.recurseSubmodules=only	2022-11-14 16:55:50 -05:00
git-quiltimport.txt	…
git-range-diff.txt	range-diff: optionally accept pathspecs	2022-08-26 09:49:26 -07:00
git-read-tree.txt	Documentation: render dash correctly	2023-01-23 09:40:14 -08:00
git-rebase.txt	rebase: add a config option for --rebase-merges	2023-03-27 09:32:49 -07:00
git-receive-pack.txt	doc txt & -h consistency: use "git foo" form, not "git-foo"	2022-10-13 09:32:57 -07:00
git-reflog.txt	reflog doc: list real subcommands up-front	2022-10-13 09:32:58 -07:00
git-remote-ext.txt	…
git-remote-fd.txt	…
git-remote-helpers.txto	…
git-remote.txt	builtin/remote.c: teach `-v` to list filters for promisor remotes	2022-05-09 10:53:58 -07:00
git-repack.txt	builtin/repack.c: implement `--expire-to` for storing pruned objects	2022-10-24 13:39:42 -07:00
git-replace.txt	…
git-request-pull.txt	…
git-rerere.txt	doc txt & -h consistency: make "rerere" consistent	2022-10-13 09:32:56 -07:00
git-reset.txt	git-reset.txt: mention 'ORIG_HEAD' in the Description	2023-01-13 09:55:45 -08:00
git-restore.txt	update documentation for new zdiff3 conflictStyle	2021-12-01 14:45:59 -08:00
git-rev-list.txt	doc txt & -h consistency: add or fix optional "--" syntax	2022-10-13 09:32:56 -07:00
git-rev-parse.txt	fetch: support hideRefs to speed up connectivity checks	2023-02-27 09:27:03 -08:00
git-revert.txt	Merge branch 'ab/doc-synopsis-and-cmd-usage'	2022-11-29 10:41:06 +09:00
git-rm.txt	…
git-send-email.txt	e-mail workflow: Message-ID is spelled with ID in both capital letters	2023-04-03 08:55:43 -07:00
git-send-pack.txt	doc txt & -h consistency: add missing options and labels	2022-10-13 09:32:56 -07:00
git-sh-i18n--envsubst.txt	…
git-sh-i18n.txt	…
git-sh-setup.txt	…
git-shell.txt	…
git-shortlog.txt	shortlog: support arbitrary commit format `--group`s	2022-10-24 14:48:05 -07:00
git-show-branch.txt	Merge branch 'ab/doc-synopsis-and-cmd-usage'	2022-10-28 11:26:54 -07:00
git-show-index.txt	…
git-show-ref.txt	doc txt & -h consistency: correct padding around "[]()"	2022-10-13 09:32:55 -07:00
git-show.txt	…
git-sparse-checkout.txt	builtin/sparse-checkout: add check-rules command	2023-03-27 10:51:12 -07:00
git-stage.txt	…
git-stash.txt	Merge branch 'ab/doc-synopsis-and-cmd-usage'	2022-10-28 11:26:54 -07:00
git-status.txt	status: modernize git-status "slow untracked files" advice	2022-12-01 15:27:41 +09:00
git-stripspace.txt	…
git-submodule.txt	clone, submodule: pass partial clone filters to submodules	2022-02-09 15:38:36 -08:00
git-svn.txt	Merge branch 'ja/doc-cleanup'	2021-12-10 14:35:03 -08:00
git-switch.txt	docs: add CONFIGURATION sections that fuzzy map to built-ins	2022-09-07 09:46:07 -07:00
git-symbolic-ref.txt	symbolic-ref: teach "--[no-]recurse" option	2022-10-09 12:31:24 -07:00
git-tag.txt	branch, for-each-ref, tag: add option to omit empty lines	2023-04-13 08:07:45 -07:00
git-tools.txt	…
git-unpack-file.txt	…
git-unpack-objects.txt	…
git-update-index.txt	Documentation: fix various repeat word typos	2022-09-12 11:04:55 -07:00
git-update-ref.txt	…
git-update-server-info.txt	doc txt & -h consistency: add missing options and labels	2022-10-13 09:32:56 -07:00
git-upload-archive.txt	doc txt & -h consistency: fix mismatching labels	2022-10-13 09:32:56 -07:00
git-upload-pack.txt	Documentation: fix various repeat word typos	2022-09-12 11:04:55 -07:00
git-var.txt	var: add GIT_SEQUENCE_EDITOR variable	2022-12-18 11:48:26 +09:00
git-verify-commit.txt	doc txt & -h consistency: add missing options and labels	2022-10-13 09:32:56 -07:00
git-verify-pack.txt	doc txt & -h consistency: correct padding around "[]()"	2022-10-13 09:32:55 -07:00
git-verify-tag.txt	doc txt & -h consistency: add missing options and labels	2022-10-13 09:32:56 -07:00
git-version.txt	…
git-web--browse.txt	…
git-whatchanged.txt	…
git-worktree.txt	doc txt & -h consistency: word-wrap	2022-10-13 09:32:55 -07:00
git-write-tree.txt	…
git.txt	Merge branch 'ar/markup-em-dash'	2023-01-30 14:24:24 -08:00
gitattributes.txt	Merge branch 'jc/diff-algo-attribute'	2023-02-27 10:08:56 -08:00
gitcli.txt	git-cli.txt: clarify "options first and then args"	2022-01-17 11:42:25 -08:00
gitcore-tutorial.txt	…
gitcredentials.txt	credential: new attribute password_expiry_utc	2023-02-22 15:18:58 -08:00
gitcvs-migration.txt	…
gitdiffcore.txt	…
giteveryday.txt	…
gitfaq.txt	…
gitformat-bundle.txt	docs: move protocol-related docs to man section 5	2022-08-04 14:12:23 -07:00
gitformat-chunk.txt	docs: move pack format docs to man section 5	2022-08-04 14:12:24 -07:00
gitformat-commit-graph.txt	doc: use "commit-graph" hyphenation consistently	2022-10-30 19:58:40 -04:00
gitformat-index.txt	docs: document zero bits in index "mode"	2023-02-01 08:49:23 -08:00
gitformat-pack.txt	builtin/gc.c: make `gc.cruftPacks` enabled by default	2023-04-18 14:56:48 -07:00
gitformat-signature.txt	Merge branch 'gm/signature-format-doc'	2023-03-06 21:51:56 -08:00
gitglossary.txt	…
githooks.txt	send-email: export patch counters in validate environment	2023-04-14 10:41:15 -07:00
gitignore.txt	…
gitk.txt	…
gitmailmap.txt	…
gitmodules.txt	…
gitnamespaces.txt	…
gitprotocol-capabilities.txt	docs: fix a few recently broken links	2022-09-16 08:38:03 -07:00
gitprotocol-common.txt	docs: move protocol-related docs to man section 5	2022-08-04 14:12:23 -07:00
gitprotocol-http.txt	docs: move http-protocol docs to man section 5	2022-08-04 14:12:24 -07:00
gitprotocol-pack.txt	docs: move http-protocol docs to man section 5	2022-08-04 14:12:24 -07:00
gitprotocol-v2.txt	*: fix typos which duplicate a word	2023-01-08 10:28:34 +09:00
gitremote-helpers.txt	remote-curl: add 'get' capability	2022-08-10 14:07:37 -07:00
gitrepository-layout.txt	…
gitrevisions.txt	…
gitsubmodules.txt	…
gittutorial-2.txt	…
gittutorial.txt	…
gitweb.conf.txt	…
gitweb.txt	…
gitworkflows.txt	…
glossary-content.txt	glossary: add reachability bitmap description	2022-10-30 19:58:46 -04:00
howto-index.sh	…
i18n.txt	…
install-doc-quick.sh	…
install-webdoc.sh	…
line-range-format.txt	…
line-range-options.txt	…
lint-fsck-msgids.perl	Documentation: add lint-fsck-msgids	2022-10-25 15:44:19 -07:00
lint-gitlink.perl	…
lint-man-end-blurb.perl	…
lint-man-section-order.perl	git docs: add a category for file formats, protocols and interfaces	2022-08-04 14:12:23 -07:00
manpage-bold-literal.xsl	…
manpage-normal.xsl	…
manpage.xsl	…
merge-options.txt	…
merge-strategies.txt	…
object-format-disclaimer.txt	…
pretty-formats.txt	doc: pretty-formats note wide char limitations, and add tests	2023-01-19 14:35:15 -08:00
pretty-options.txt	…
pull-fetch-param.txt	…
ref-reachability-filters.txt	…
rerere-options.txt	doc: clarify rerere-autoupdate	2022-08-03 13:57:25 -07:00
rev-list-description.txt	…
rev-list-options.txt	Merge branch 'jk/document-rev-list-object-name'	2023-04-04 14:28:29 -07:00
revisions.txt	revisions.txt: be explicit about commands writing 'ORIG_HEAD'	2023-01-13 09:55:46 -08:00
scalar.txt	scalar: include in standard Git build & installation	2022-09-02 10:02:55 -07:00
sequencer.txt	…
signoff-option.txt	…
texi.xsl	…
trace2-target-values.txt	…
transfer-data-leaks.txt	…
urls-remotes.txt	Documentation: clarify multiple pushurls vs urls	2023-02-07 11:02:27 -08:00
urls.txt	…
user-manual.conf	…
user-manual.txt	docs: fix a few recently broken links	2022-09-16 08:38:03 -07:00