In partial clones, `git grep` fetches necessary blobs on-demand one
at a time, which can be very slow. In partial clones, add an extra
preliminary walk over the tree similar to grep_tree() which collects
the blobs of interest, and then prefetches them.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In partial clones, `git cherry` fetches necessary blobs on-demand one
at a time, which can be very slow. We would like to prefetch all
necessary blobs upfront. To do so, we need to be able to first figure
out which blobs are needed.
`git cherry` does its work in a two-phase approach: first computing
header-only IDs (based on file paths and modes), then falling back to
full content-based IDs only when header-only IDs collide -- or, more
accurately, whenever the oidhash() of the header-only object_ids
collide.
patch-ids.c handles this by creating an ids->patches hashmap that has
all the data we need, but the problem is that any attempt to query the
hashmap will invoke the patch_id_neq() function on any colliding objects,
which causes the on-demand fetching.
Insert a new prefetch_cherry_blobs() function before checking for
collisions. Use a temporary replacement on the ids->patches.cmpfn
in order to enumerate the blobs that would be needed without yet
fetching them, and then fetch them all at once, then restore the old
ids->patches.cmpfn.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
promisor_remote_get_direct() does not, on its happy path, filter out
OIDs that are already present in the local object store: every OID
the caller supplies is written to the fetch subprocess's stdin and
ends up in the response pack. The only filtering it performs is in
remove_fetched_oids(), and that only runs after a fetch failure when
falling back to a different configured promisor remote.
Almost every existing caller already filters locally-present OIDs out
itself (typically with odb_read_object_info_extended() and
OBJECT_INFO_FOR_PREFETCH, or odb_has_object() with no fetch flag). But
the existing API comment does not state this expectation, so a new
caller is easy to write incorrectly (I missed this originally and wrote
two problematic callers). Omitting the filter still "works" in the
sense that the desired objects end up local, but it silently makes the
fetch request -- and the response pack -- larger than necessary,
defeating part of the point of batching.
Spell the contract out so future callers know to filter (and
deduplicate) themselves, and point them at the helpers they should
use to check local presence without accidentally triggering a lazy
fetch.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The glossary entry is a list of terms and their definitions, so
multi-paragraph definitions need "+" continuation lines to indicate
that they are part of a single entry.
When an entry contains a sub-list (say, a bulleted list), the final "+"
may become ambiguous: is it connecting the next paragraph to the final
entry of the sub-list, or to the original list of definition paragraphs?
Asciidoc generally connects it to the former, even when we mean the
latter, and you end up with the next paragraph indented incorrectly,
like this:
glob
...defines glob...
Two consecutive asterisks ("**") in patterns matched
against full pathname may have special meaning:
- ...some special meaning of **...
- ...another special meaning of **...
- Other consecutive asterisks are considered invalid.
Glob magic is incompatible with literal magic.
That final "Glob magic is incompatible" paragraph is in the wrong spot.
It should be at the same level as "Two consecutive asterisks", as it is
not part of the final "Other consecutive asterisks" bullet point.
The same problem appears in several other spots in the glossary.
Usually we'd fix this by using "--" markers, which put the sub-list into
its own block. But there's a catch: in some of these spots we are
already in an open block, and nesting open blocks is a problem. It seems
to work for me using Asciidoc 10.2.1, but Asciidoctor 2.0.26 makes a
mess of it (our intent to open a new block seems to close the old one).
Fortunately there's a work-around: when using a "+" list-continuation,
the number of empty lines above the continuation indicates which level
of parent list to continue. So by adding an empty line after our
unordered list (before the "+"), we should be able to continue the
definition list item.
But asciidoc being asciidoc, of course that is not the end of the story.
That technique works fine for the "glob" and "attr" lists in this patch,
but under the "refs" item it works for only 1 of the 2 lists! I can't
figure out why, and this may be an asciidoctor bug. But we can work
around it by using "--" open-block markers here, since we're not
already in an open block.
So using the extra blank line for the first two instances, and "--"
markers for the second two, this patch produces identical output from
"doc-diff HEAD^ HEAD" for both --asciidoctor and --asciidoc modes.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
GitHub Actions started complaining about use of Node.js 20 and I was
wondering why only one job uses actions/checkout@v4, while everybody
else already uses actions/checkout@v5.
It turns out that it is caused by a semantic mismerge between
e75cd059 (ci: check formatting of our Rust code, 2025-10-15) that
added a new use of actions/checkout@v4 that happened very close to
another change 63541ed9 (build(deps): bump actions/checkout from 4
to 5, 2025-10-16) that updated all uses of actions/checkout@v4 to
use vactions/checkout@v5.
Update the leftover and the last use of actions/checkout@v4 to use
actions/checkout@v5 to help ourselves to move away from Node.js 20.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I claimed in 3c18135b (doc: am: say that --message-id adds a trailer,
2026-02-09) that `git am --message-id` adds a Git trailer. But that
isn’t the case; for the case of a commit message with a subject, body,
and no trailer block:
<subject>
<paragrah>
It just appends the line right after `paragraph`:
<subject>
<paragraph>
Message-ID: <message-id_trailer.323@msgid.xyz>
It does work for two other cases though, namely subject-only and with an
existing trailer block.
This is at best an inconsistency and arguably a bug, but we’re at the
trailing end of the release cycle now. So reverting the doc is safer
than making msg-id act as a trailer, for now.
Revert this hunk from commit 3c18135b except the only useful
change (“Also use inline-verbatim for `Message-ID`”).
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We used writev() in limited code paths and supplied emulation for
platforms without working writev(), but the emulation was too
faithful to the spec to make the result useless to send even 64kB;
revert the topic and plan to restart the effort later.
* jc/no-writev-does-not-work:
Revert "compat/posix: introduce writev(3p) wrapper"
Revert "wrapper: introduce writev(3p) wrappers"
Revert "sideband: use writev(3p) to send pktlines"
Revert "cmake: use writev(3p) wrapper as needed"
Earlier we timelined that we'd tune our build procedures to build
with Rust by default in Git 2.53, but we are already in prerelease
freeze for 2.54 now. Update the BreakingChanges document to delay
it until Git 2.55 (slated for the end of June 2026).
Noticed-by: brian m. carlson <sandals@crustytoothpaste.net>
Helped-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The emulation layer we added for writev(3p) tries to be too faithful
to the spec that on systems with SSIZE_MAX set to lower than 64kB to
fit a single sideband packet would fail just like the real system
writev(), which makes our use of writev() for sideband messages
unworkable.
Let's revert them and reboot the effort after the release. The
reverted commits are:
$ git log -Swritev --oneline 8023abc632^..v2.52.0-rc1
89152af176 cmake: use writev(3p) wrapper as needed
26986f4cba sideband: use writev(3p) to send pktlines
1970fcef93 wrapper: introduce writev(3p) wrappers
3b9b2c2a29 compat/posix: introduce writev(3p) wrapper
8023abc632 is the merge of ps/upload-pack-buffer-more-writes topic to
the mainline.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git rev-list --maximal-only" has been optimized by borrowing the
logic used by "git show-branch --independent", which computes the
same kind of information much more efficiently.
* ds/rev-list-maximal-only-optim:
rev-list: use reduce_heads() for --maximal-only
p6011: add perf test for rev-list --maximal-only
t6600: test --maximal-only and --independent
"git config list" is the official way to spell "git config -l" and
"git config --list". Use it to update the documentation.
* kh/doc-config-list:
doc: gitcvs-migration: rephrase “man page”
doc: replace git config --list/-l with `list`
Further work to adjust the codebase for C23 that changes functions
like strchr() that discarded constness when they return a pointer into
a const string to preserve constness.
* jk/c23-const-preserving-fixes-more:
git-compat-util: fix CONST_OUTPARAM typo and indentation
refs/files-backend: drop const to fix strchr() warning
http: drop const to fix strstr() warning
range-diff: drop const to fix strstr() warnings
pkt-line: make packet_reader.line non-const
skip_prefix(): check const match between in and out params
pseudo-merge: fix disk reads from find_pseudo_merge()
find_last_dir_sep(): convert inline function to macro
run-command: explicitly cast away constness when assigning to void
pager: explicitly cast away strchr() constness
transport-helper: drop const to fix strchr() warnings
http: add const to fix strchr() warnings
convert: add const to fix strchr() warnings
Clarify that --prefix is used as given and is not normalized,
and may include leading slashes or parent directory components.
Signed-off-by: Pushkar Singh <pushkarkumarsingh1970@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In commit ec0becacc9 (run-command: add stdin callback for
parallelization, 2026-01-28), we taught run_processes_parallel() to
ignore SIGPIPE, since we wouldn't want a write() to a broken pipe of one
of the children to take down the whole process.
But there's a subtle ordering issue. After we ignore SIGPIPE, we call
pp_init(), which installs its own cleanup handler for multiple signals
using sigchain_push_common(), which includes SIGPIPE. So if we receive
SIGPIPE while writing to a child, we'll trigger that handler first, pop
it off the stack, and then re-raise (which is then ignored because of
the SIG_IGN we pushed first).
But what does that handler do? It tries to clean up all of the child
processes, under the assumption that when we re-raise the signal we'll
be exiting the process!
So a hook that exits without reading all of its input will cause us to
get SIGPIPE, which will put us in a signal handler that then tries to
kill() that same child.
This seems to be mostly harmless on Linux. The process has already
exited by this point, and though kill() does not complain (since the
process has not been reaped with a wait() call), it does not affect the
exit status of the process.
However, this seems not to be true on all platforms. This case is
triggered by t5401.13, "pre-receive hook that forgets to read its
input". This test fails on NonStop since that hook was converted to the
run_processes_parallel() API.
We can fix it by reordering the code a bit. We should run pp_init()
first, and then push our SIG_IGN onto the stack afterwards, so that it
is truly ignored while feeding the sub-processes.
Note that we also reorder the popping at the end of the function, too.
This is not technically necessary, as we are doing two pops either way,
but now the pops will correctly match their pushes.
This also fixes a related case that we can't test yet. If we did have
more than one process to run, then one child causing SIGPIPE would cause
us to kill() all of the children (which might still actually be
running). But the hook API is the only user of the new feed_pipe
feature, and it does not yet support parallel hook execution. So for now
we'll always execute the processes sequentially. Once parallel hook
execution exists, we'll be able to add a test which covers this.
Reported-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
During Git 2.52 timeframe, we broke streaming computation of object
hash outside a repository, which has been corrected.
* jt/index-fd-wo-repo-regression-fix:
During Git 2.52 timeframe, we broke streaming computation of object
hash outside a repository, which has been corrected.
* jt/index-fd-wo-repo-regression-fix-maint:
object-file: avoid ODB transaction when not writing objects
The experimental `git replay` command learned the `--ref=<ref>` option
to allow specifying which ref to update, overriding the default behavior.
* tc/replay-ref:
replay: allow to specify a ref with option --ref
replay: use stuck form in documentation and help message
builtin/replay: mark options as not negatable
add_files_to_cache() used diff_files() to detect only the paths that
are different between the index and the working tree and add them,
which does not need rename detection, which interfered with unnecessary
conflicts.
* ng/add-files-to-cache-wo-rename:
read-cache: disable renames in add_files_to_cache
Update reftable library part with what is used in libgit2 to improve
portability to different target codebases and platforms.
* ps/reftable-portability:
reftable/system: add abstraction to mmap files
reftable/system: add abstraction to retrieve time in milliseconds
reftable/fsck: use REFTABLE_UNUSED instead of UNUSED
reftable/stack: provide fsync(3p) via system header
reftable: introduce "reftable-system.h" header
Various code clean-up around odb subsystem.
* ps/odb-cleanup:
odb: drop unneeded headers and forward decls
odb: rename `odb_has_object()` flags
odb: use enum for `odb_write_object` flags
odb: rename `odb_write_object()` flags
treewide: use enum for `odb_for_each_object()` flags
CodingGuidelines: document our style for flags
Add the missing &&'s so we properly propagate failures
between commands in the hook helper functions.
Also add a missing mkdir -p arg (found by adding the &&).
Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our description of the reftable format is that it is experimental and
subject to change, but that is no longer true. Remove this statement so
as not to mislead users.
In addition, the documentation says that the files format is the
default, but that is not true if breaking changes mode is on. Correct
this information with a conditional.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In ce1661f9da (odb: add transaction interface, 2025-09-16), existing
ODB transaction logic is adapted to create a transaction interface
at the ODB layer. The intent here is for the ODB transaction
interface to eventually provide an object source agnostic means to
manage transactions.
An unintended consequence of this change though is that
`object-file.c:index_fd()` may enter the ODB transaction path even
when no object write is requested. In non-repository contexts, this
can result in a NULL dereference and segfault. One such case occurs
when running git-diff(1) outside of a repository with
"core.bigFileThreshold" forcing the streaming path in `index_fd()`:
$ echo foo >foo
$ echo bar >bar
$ git -c core.bigFileThreshold=1 diff -- foo bar
In this scenario, the caller only needs to compute the object ID. Object
hashing does not require an ODB, so starting a transaction is both
unnecessary and invalid.
Fix the bug by avoiding the use of ODB transactions in `index_fd()` when
callers are only interested in computing the object hash.
Reported-by: Luca Stefani <luca.stefani.ge1@gmail.com>
Signed-off-by: Justin Tobler <jltobler@gmail.com>
[jc: adjusted to fd13909e (Merge branch 'jt/odb-transaction', 2025-10-02)]
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git backfill" is capable of auto-detecting a sparsely checked out
working tree, which was broken.
* th/backfill-auto-detect-sparseness-fix:
backfill: auto-detect sparse-checkout from config
The check in "receive-pack" to prevent a checked out branch from
getting updated via updateInstead mechanism has been corrected.
* ps/receive-pack-updateinstead-in-worktree:
receive-pack: use worktree HEAD for updateInstead
t5516: clean up cloned and new-wt in denyCurrentBranch and worktrees test
t5516: test updateInstead with worktree and unborn bare HEAD
Handling of signed commits and tags in fast-import has been made more
configurable.
* jt/fast-import-signed-modes:
fast-import: add 'abort-if-invalid' mode to '--signed-tags=<mode>'
fast-import: add 'sign-if-invalid' mode to '--signed-tags=<mode>'
fast-import: add 'strip-if-invalid' mode to '--signed-tags=<mode>'
fast-import: add 'abort-if-invalid' mode to '--signed-commits=<mode>'
fast-export: check for unsupported signing modes earlier
The way the "git log -L<range>:<file>" feature is bolted onto the
log/diff machinery is being reworked a bit to make the feature
compatible with more diff options, like -S/G.
* mm/line-log-use-standard-diff-output:
doc: note that -L supports patch formatting and pickaxe options
t4211: add tests for -L with standard diff options
line-log: route -L output through the standard diff pipeline
line-log: fix crash when combined with pickaxe options
Reduce dependency on `the_repository` in add-patch.c file.
* sp/add-patch-with-fewer-the-repository:
add-patch: use repository instance from add_i_state instead of the_repository
Elijah Newren