The path-walk API prunes trees and blobs when a sparse-checkout pattern
list is provided, which is the correct behavior for 'git backfill
--sparse' since it only needs to fill in objects at paths within the
sparse cone.
However, a future change will use the path-walk API with a sparse:<oid>
filter that restricts only blobs while retaining all reachable trees.
To support both behaviors, add a 'pl_sparse_trees' flag to
path_walk_info. When set (as in 'git backfill --sparse' and the
--stdin-pl test helper mode), the sparse patterns prune both trees and
blobs. When unset, only blobs are filtered and all trees are walked and
reported.
Additionally, move the SEEN flag assignment in add_tree_entries() to
after the sparse pattern and pathspec checks. Previously, SEEN was set
immediately upon discovering an object, before checking whether its path
matched the sparse patterns. When the same object ID appeared at
multiple paths (e.g. sibling directories with identical contents), the
first path to be visited would mark the object as SEEN. If that path was
outside the sparse cone, the object would be skipped there but also
never discovered at its in-cone path.
By deferring the SEEN flag until after the checks pass, objects that are
skipped due to sparse filtering remain discoverable at other paths where
they may be in scope.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Extend the path-walk API to handle the 'blob:limit=<size>' object
filter natively. This filter omits blobs whose size is equal to or
greater than the given limit, matching the semantics used by the
list-objects-filter machinery.
When revs->filter.choice is LOFC_BLOB_LIMIT, the prepare_filters()
method stores the limit value in info->blob_limit and clears the filter
from revs. If the limit is zero, this degenerates to blob:none (all
blobs excluded), so info->blobs is set to 0 instead.
During walk_path(), blob batches are filtered before being delivered to
the callback: each blob's size is checked via odb_read_object_info(),
and only blobs strictly smaller than the limit are included. Blobs whose
size cannot be determined (e.g. missing in a partial clone) are
conservatively included, matching the existing filter behavior. Empty
batches after filtering are skipped entirely.
The check for inclusion in the path batch looks a little strange at
first glance. We use odb_read_object_info() to read the object's size.
Based on all of the assumptions to this point, this _should_ return
OBJ_BLOB. Since we are focused on the size filter, we use a
short-circuited OR (||) to skip the size check if that method returns a
different object type.
Notice that this inspection of object sizes requires the content to be
present in the repository. The odb_read_object_info() call will download
a missing blob on-demand. This means that the use of the path-walk API
within 'git backfill' would not operate nicely with this filter type.
The intention of that command is to download missing blobs in batches.
Downloading objects one-by-one would go against the point. Update the
validation in 'git backfill' to add its own compatibility check on top
of path_walk_filter_compatible().
Add tests for blob:limit=0 (equivalent to blob:none) and blob:limit=3
(which exercises partial filtering within a batch where some blobs are
kept and others are excluded).
Co-authored-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'git backfill' command uses the path-walk API in a critical way: it
uses the objects output from the command to find the batches of missing
objects that should be requested from the server. Unlike 'git
pack-objects', we cannot fall back to another mechanism.
The previous change added the path_walk_filter_compatible() method that
we can reuse here. Use it during argument validation in cmd_backfill().
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'git pack-objects' command can opt-in to using the path-walk API for
scanning the objects. Currently, this option is dynamically disabled if
combined with '--filter=<X>', even when using a simple filter such as
'blob:none' to signal a blobless packfile. This is a common scenario for
repos at scale, so is worth integrating.
Also, users can opt-in to the '--path-walk' option by default through
the pack.usePathWalk=true config option. When using that in a blobless
partial clone, the following warning can appear even though the user did
not specify either option directly:
warning: cannot use --filter with --path-walk
Teach the path-walk API to handle the 'blob:none' object filter
natively. When revs->filter.choice is LOFC_BLOB_NONE, the path-walk
sets info->blobs to 0 (skipping all blob objects) and clears the
filter from revs so that prepare_revision_walk() does not reject the
configuration.
This check is implemented in the static prepare_filters() method, which
will simultaneously check if the input filters are compatible and will
make the appropriate mutations to the path_walk_info and filters if the
path_walk_info is non-NULL. This allows us to use this logic both in the
API method path_walk_filter_compatible() for use in
builtin/pack-objects.c and as a prep step in walk_objects_by_path().
Update the test helper (test-path-walk) to accept --filter=<spec>
as a test-tool option (before '--'), applying it to revs after
setup_revisions() to avoid the --objects requirement check. We can also
revert recent GIT_TEST_PACK_PATH_WALK overrides in t5620.
Also switch test-path-walk from REV_INFO_INIT with manual repo
assignment to repo_init_revisions(), which properly initializes
the filter_spec strbuf needed for filter parsing.
Add tests for blob:none with --all and with a single branch.
The performance test p5315 shows the impact of this change when using
blobless filters:
Test HEAD~1 HEAD
---------------------------------------------------------------------
5315.6: repack (blob:none) 13.53 13.87 +2.5%
5315.7: repack size (blob:none) 137.7M 137.8M +0.1%
5315.8: repack (blob:none, --path-walk) 13.51 23.43 +73.4%
5315.9: repack size (blob:none, --path-walk) 137.7M 115.2M -16.3%
These performance tests were run on the Git repository. The --path-walk
feature shows meaningful space savings (16% smaller for blobless packs)
at the cost of increased computation time due to the two compression
passes. This data demonstrates that the feature is engaged and provides
real compression benefits when --no-reuse-delta forces fresh deltas.
Co-Authored-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We are preparing to integrate the path-walk API with some --filter options
in 'git pack-objects', but there is a subtle issue that is revealed when
those are put together and the test suite is run with
GIT_TEST_PACK_PATH_WALK=1.
When a filter reduces the set of requested objects, this results in
filtering out directly-requested objects, such as in the download of needed
blobs in a blobless partial clone.
The root cause is that the scan of pending objects in the path-walk API
respects the filters set in the path_walk_info instead of overriding them
for pending objects.
We can tell that a path is part of the directly-referenced objects if its
path name starts with '/' (other paths, including root trees never have this
starting character). Create a path_is_for_direct_objects() to make this
meaning clear, especially as we add more references in the future as we
integrate the path-walk API with partial clone filter options.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add p5315-pack-objects-filter.sh to measure the performance of
'git pack-objects --revs --all' under different filter and traversal
combinations:
* no filter (baseline)
* --filter=blob:none (blobless)
* --filter=sparse:oid=<oid> (cone-mode sparse)
Each filter scenario is tested both with and without --path-walk,
producing paired measurements that show the impact of the path-walk
traversal for each filter type as we integrate the --path-walk feature
with different --filter options. It currently has no integration so
falls back to the standard revision walk. Thus, there are no significant
differences in the current results other than a full repack (and even
then, the --path-walk feature is not incredibly different for the
default Git repository):
Test HEAD
-----------------------------------------------------
5315.2: repack (no filter) 27.91
5315.3: repack size (no filter) 250.7M
5315.4: repack (no filter, --path-walk) 34.92
5315.5: repack size (no filter, --path-walk) 220.0M
5315.6: repack (blob:none) 13.63
5315.7: repack size (blob:none) 137.6M
5315.8: repack (blob:none, --path-walk) 13.48
5315.9: repack size (blob:none, --path-walk) 137.7M
5315.10: repack (sparse:oid) 72.67
5315.11: repack size (sparse:oid) 187.4M
5315.12: repack (sparse:oid, --path-walk) 72.47
5315.13: repack size (sparse:oid, --path-walk) 187.4M
The sparse filter definition is built automatically by sampling
depth-2 directories from the test repository, making the test work
on any repo passed via GIT_PERF_LARGE_REPO. For repos that lack
depth-2 directories, a single top-level directory is used; for flat
repos, the sparse tests are skipped via prerequisite.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When 'git pack-objects' has the --path-walk option enabled, it uses a
different set of revision walk parameters than normal. For one,
--objects was previously assumed by the path-walk API and could be
omitted. We also needed --boundary to allow discovering UNINTERESTING
objects to use as delta bases.
We will be updating the path-walk API soon to work with some filter
options. However, the revision machinery will trigger a fatal error:
fatal: object filtering requires --objects
The fix is easy: add the --objects option as an argument. This has no
effect on the path-walk API but does simplify the revision option
parsing for the objects filter.
We can remove the comment about "removing" the options because they were
never removed and instead not added. We still need to disable using
bitmaps.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The GIT_TEST_PACK_PATH_WALK test variable allows enabling the
--path-walk option to 'git pack-objects' by default. This sometimes
engages the warning that --path-walk is incompatible with the --filter
option. These tests in t5620 fail due to this warning over stderr in
this case. Disable this variable for this moment until these options
work together.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* en/backfill-fixes-and-edges:
backfill: default to grabbing edge blobs too
backfill: document acceptance of revision-range in more standard manner
backfill: reject rev-list arguments that do not make sense
6cc6d1b4c6 (Documentation: update add --force option + ignore=all
config, 2026-02-06) added text describing both the ignore=none and
ignore=all behaviors. The former had minor formatting and grammatical
errors, while the latter was a bit garbled. I have tried to tweak the
wording on the latter to make it read as I think was intended, and fixed
the minor grammatical issues with both as well.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
a8215a2051 (send-email: add client certificate options, 2026-03-02)
added documentation for sendemail.smtpSSLClientKey that says it works
"in conjunction with `sendemail.smtpSSLClientKey`" -- referring to
itself. It appears that `sendemail.smtpSSLClientCert` was the intended
reference; fix it.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix various issues in the release notes -- missing/wrong articles, typo,
indentation, quote consistency, and wording improvement or corrections.
Other than the indentation fix for "The way combined list-object filter
options...", this patch is much easier to view with --color-words.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As writing version 2 MIDX files by default breaks older versions of
Git and its reimplementations, use V2 only when necessary.
* jk/midx-write-v1-by-default:
MIDX: revert the default version to v1
We introduced midx version 2 in b2ec8e90c2 (midx: do not require packs
to be sorted in lexicographic order, 2026-02-24) and now write it by
default. The rationale was that older versions should ignore the v2 midx
and fall back to using the packs (just like we do for other midx
errors). Unfortunately this is not the case, as we have a hard die()
when we see an unknown midx version.
As a result, writing a midx with Git 2.54-rc2 puts the repository into a
state that is unusable with Git 2.53. And this midx write may happen
behind the scenes as part of normal operations, like fetch.
Let's switch back to writing v1 by default to avoid regressing the case
where multiple versions of Git are used on the same repository.
There is one gotcha, though: the v2 format is required for some new
features, like midx compaction, and running "git multi-pack-index
compact" will complain when asked to write a v1 index. The user must set
midx.version to "2" to make the feature work.
So instead of always using v1, we'll base the default on whether the
requested feature requires v2. That does mean that running midx
compaction will create a repository that can't be read by older versions
of Git. But we never do that by default; only people experimenting with
the new feature will be affected.
We have to adjust the test expectation in t5319, since it will now
generate v1 files. And our "auto-select v2" is covered by the tests in
t5335, which continue to check that compaction works without having to
set midx.version manually (and also explicitly check that asking for v1
with compaction reports the problem).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
- correct translation of pathspec msgs
Corrects cases where the “pathspec” is translated as if it was a
path
- correct translation of refspec msgs
Corrects cases where the “refspec” were not consistently translated
- correct translation of credential msgs
Corrects cases where the “credential” were not correctly translated
Signed-off-by: Stefan Björnelund <stefan.bjornelund.gnome@gmail.com>
Modified-by: Peter Krefting <peter@softwolves.pp.se>
Translate 198 previously fuzzy or untranslated messages, bringing the
total number of translated messages to 6226.
Reviewed-by: 依云 <lilydjwg@gmail.com>
Reviewed-by: Fangyi Zhou <me@fangyi.io>
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Commit 302aff0922 (backfill: accept revision arguments, 2026-03-26) added
support for accepting revision arguments to backfill. This allows users
to do things like
git backfill --remotes ^v2.3.0
and then run many commands without triggering on-demand downloads of
blobs. However, if they have topics based on v2.3.0, they will likely
still trigger on-demand downloads. Consider, for example, the command
git log -p v2.3.0..topic
This would still trigger on-demand blob loadings after the backfill
command above, because the commit(s) with A as a parent will need to
diff against the blobs in A. In fact, multiple commands need blobs from
the lower boundary of the revision range:
* git log -p A..B # After backfill A..B
* git replay --onto TARGET A..B # After backfill TARGET^! A..B
* git checkout A && git merge B # After backfill A...B
Add an extra --[no-]include-edges flag to allow grabbing blobs from
edge commits. Since the point of backfill is to prevent on-demand blob
loading and these are common commands, default to --include-edges.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
302aff0922 (backfill: accept revision arguments, 2026-03-26) added
support for passing revision arguments to 'git backfill' but documented
them only with a prose sentence:
You may also specify the commit limiting options from
git-rev-list(1).
No other command that accepts revision arguments documents them this
way. Commands like log, shortlog, and replay define a formal
<revision-range> entry and include rev-list-options.adoc. Commands like
bundle, fast-export, and filter-branch, which pass arguments through to
the revision machinery without including the full options file, still
define a formal <git-rev-list-args> entry explaining what is accepted.
Add a formal <revision-range> entry in the synopsis and OPTIONS section,
following the convention used by other commands, and mention that
commit-limiting options from git-rev-list(1) are also accepted.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some rev-list options accepted by setup_revisions() are silently
ignored or actively counterproductive when used with 'git backfill',
because the path-walk API has its own tree-walking logic that bypasses
the mechanisms these options rely on:
* -S/-G (pickaxe) and --diff-filter work by computing per-commit
diffs in get_revision_1() and filtering commits whose diffs don't
match. Since backfill's goal is to download all blobs reachable
from commits in the range, filtering out commits based on diff
content would silently skip blobs -- the opposite of what users
want.
* --follow disables path pruning (revs->prune) and only makes
sense for tracking a single file through renames in log output.
It has no useful interaction with backfill.
* -L (line-log) computes line-level diffs to track the evolution
of a function or line range. Like pickaxe, it filters commits
based on diff content, which would cause blobs to be silently
skipped.
* --diff-merges controls how merge commit diffs are displayed.
The path-walk API walks trees directly and never computes
per-commit diffs, so this option would be silently ignored.
* --filter (object filtering, e.g. --filter=blob:none) is used by
the list-objects traversal but is completely ignored by the
path-walk API, so it would silently do nothing.
Rather than letting users think these options are being honored,
reject them with a clear error message.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The glossary entry is a list of terms and their definitions, so
multi-paragraph definitions need "+" continuation lines to indicate
that they are part of a single entry.
When an entry contains a sub-list (say, a bulleted list), the final "+"
may become ambiguous: is it connecting the next paragraph to the final
entry of the sub-list, or to the original list of definition paragraphs?
Asciidoc generally connects it to the former, even when we mean the
latter, and you end up with the next paragraph indented incorrectly,
like this:
glob
...defines glob...
Two consecutive asterisks ("**") in patterns matched
against full pathname may have special meaning:
- ...some special meaning of **...
- ...another special meaning of **...
- Other consecutive asterisks are considered invalid.
Glob magic is incompatible with literal magic.
That final "Glob magic is incompatible" paragraph is in the wrong spot.
It should be at the same level as "Two consecutive asterisks", as it is
not part of the final "Other consecutive asterisks" bullet point.
The same problem appears in several other spots in the glossary.
Usually we'd fix this by using "--" markers, which put the sub-list into
its own block. But there's a catch: in some of these spots we are
already in an open block, and nesting open blocks is a problem. It seems
to work for me using Asciidoc 10.2.1, but Asciidoctor 2.0.26 makes a
mess of it (our intent to open a new block seems to close the old one).
Fortunately there's a work-around: when using a "+" list-continuation,
the number of empty lines above the continuation indicates which level
of parent list to continue. So by adding an empty line after our
unordered list (before the "+"), we should be able to continue the
definition list item.
But asciidoc being asciidoc, of course that is not the end of the story.
That technique works fine for the "glob" and "attr" lists in this patch,
but under the "refs" item it works for only 1 of the 2 lists! I can't
figure out why, and this may be an asciidoctor bug. But we can work
around it by using "--" open-block markers here, since we're not
already in an open block.
So using the extra blank line for the first two instances, and "--"
markers for the second two, this patch produces identical output from
"doc-diff HEAD^ HEAD" for both --asciidoctor and --asciidoc modes.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
GitHub Actions started complaining about use of Node.js 20 and I was
wondering why only one job uses actions/checkout@v4, while everybody
else already uses actions/checkout@v5.
It turns out that it is caused by a semantic mismerge between
e75cd059 (ci: check formatting of our Rust code, 2025-10-15) that
added a new use of actions/checkout@v4 that happened very close to
another change 63541ed9 (build(deps): bump actions/checkout from 4
to 5, 2025-10-16) that updated all uses of actions/checkout@v4 to
use vactions/checkout@v5.
Update the leftover and the last use of actions/checkout@v4 to use
actions/checkout@v5 to help ourselves to move away from Node.js 20.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I claimed in 3c18135b (doc: am: say that --message-id adds a trailer,
2026-02-09) that `git am --message-id` adds a Git trailer. But that
isn’t the case; for the case of a commit message with a subject, body,
and no trailer block:
<subject>
<paragrah>
It just appends the line right after `paragraph`:
<subject>
<paragraph>
Message-ID: <message-id_trailer.323@msgid.xyz>
It does work for two other cases though, namely subject-only and with an
existing trailer block.
This is at best an inconsistency and arguably a bug, but we’re at the
trailing end of the release cycle now. So reverting the doc is safer
than making msg-id act as a trailer, for now.
Revert this hunk from commit 3c18135b except the only useful
change (“Also use inline-verbatim for `Message-ID`”).
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We used writev() in limited code paths and supplied emulation for
platforms without working writev(), but the emulation was too
faithful to the spec to make the result useless to send even 64kB;
revert the topic and plan to restart the effort later.
* jc/no-writev-does-not-work:
Revert "compat/posix: introduce writev(3p) wrapper"
Revert "wrapper: introduce writev(3p) wrappers"
Revert "sideband: use writev(3p) to send pktlines"
Revert "cmake: use writev(3p) wrapper as needed"
Derrick Stolee