The 'remote-object-info' command has been added to 'git cat-file
--batch-command', allowing clients to request object metadata
(currently size) from a remote server via protocol v2 without
downloading the entire object. Format placeholders are dynamically
filtered on the client based on server-advertised capabilities,
returning empty strings for inapplicable or unsupported fields.
* ps/cat-file-remote-object-info:
cat-file: make remote-object-info allow-list dynamic
cat-file: validate remote atoms with an allow-list
cat-file: add remote-object-info to batch-command
transport: add client support for object-info
serve: advertise object-info feature
fetch-pack: move fetch initialization
connect: make write_fetch_command_and_capabilities() more generic
fetch-pack: move write_fetch_command_and_capabilities() to connect.c
fetch-pack: drop static advertise_sid variable
fetch-pack: fix hash_algo variable type
t1006: split test utility functions into new 'lib-cat-file.sh'
cat-file: declare loop counter inside for()
transport-helper: fix memory leak of helper on disconnect
The in-tree 'b4' cover letter template has been updated to include the
'change-id' trailer, ensuring that sent tags generated by 'b4' contain
the required tracking information for subsequent runs.
* cl/b4-cover-change-id:
b4: include change-id in cover template
The stream-based object signature verification path has been
corrected to avoid double-closing the stream on read errors.
* ps/odb-stream-double-close-fix:
object-file: fix closing object stream twice
The 'reprepare()' callback for object database sources has been
generalized into a 'prepare()' callback with an optional flush cache
flag, and a new 'odb_prepare()' wrapper has been introduced to allow
pre-opening object database sources.
* ps/odb-generalize-prepare:
odb: introduce `odb_prepare()`
odb/source: generalize `reprepare()` callback
The lazy priority queue optimization pattern (deferring actual removal
in 'prio_queue_get()' to allow get+put fusion) has been folded
directly into 'prio_queue' itself, speeding up commit traversal
workflows and simplifying callers.
* kk/prio-queue-get-put-fusion:
prio-queue: fold lazy_queue into prio_queue for automatic get+put fusion
prio-queue: rename .nr to .nr_ and add accessor helpers
When 'git push origin/main' or 'git branch origin main' is run, the
command is now recognized as a potential typo, and advice has been
added to offer a typo fix.
* hn/branch-push-slip-advice:
push: suggest <remote> <branch> for a slash slip
branch: suggest <remote>/<branch> on upstream slip
A memory leak in the '--base' handling of 'git format-patch' has been
plugged, and the leak reporting of the test suite when running under a
TAP harness has been improved.
* jk/format-patch-leakfix:
format-patch: fix leak of rev_info in prepare_bases()
t: move LSan errors from stdout to stderr
A memory leak in the 'reftable_writer_new()' initialization function
has been fixed by delaying the allocation of 'struct reftable_writer'
until after input options are validated.
* jk/reftable-leakfix:
reftable: fix unlikely leak on API error
The GPG and SSH signature parsing code has been corrected to strip
carriage return characters only when they immediately precede line
feeds, instead of unconditionally stripping all carriage returns.
* ad/gpg-strip-cr-before-lf:
gpg-interface: fix strip_cr_before_lf to only remove CR before LF
Various code paths have been hardened against potential NULL-pointer
dereferences and invalid file descriptor accesses flagged by
Coverity.
* js/coverity-fixes-null-safety:
shallow: give write_one_shallow() its own hex buffer
shallow: fix NULL dereference
bisect: ensure non-NULL `head` before using it
pack-bitmap: handle missing bitmap for base MIDX
revision: avoid dereferencing NULL in `add_parents_only()`
replay: die when --onto does not peel to a commit
bisect: handle NULL commit in `bisect_successful()`
mailsplit: move NULL check before first use of file handle
reftable/stack: guard against NULL list_file in stack_destroy
remote: guard `remote_tracking()` against NULL remote
diff: handle NULL return from repo_get_commit_tree()
diffcore-break: guard against NULLed queue entries in merge loop
The performance of ref updates and reads using the 'reftable' backend
in the presence of many deletion tombstone records has been optimized
by removing the tombstone suppression flag from the merged iterator
and instead skipping tombstones at higher-level call sites where
iteration bounds are known.
* kk/reftable-tombstone-quadratic-fix:
reftable: fix quadratic behavior in the presence of tombstones
t/perf: add perf test for ref tombstone scenarios
The 'topo_levels' slab was propagated only to the topmost layer of a
split commit-graph chain, causing incremental writes to recompute
topological levels for commits in base layers. This has been
corrected.
* kk/commit-graph-topo-levels-fix:
commit-graph: propagate topo_levels slab to all chain layers
commit-graph: add trace2 instrumentation for generation DFS
The global configuration variable 'ignore_case' (representing the
'core.ignorecase' configuration) has been migrated into 'struct
repo_config_values' to tie it to a specific repository instance.
* ty/migrate-ignorecase:
config: use repo_ignore_case() to access core.ignorecase
environment: move ignore_case into repo_config_values
The client-side parser of server-advertised bundle-URI list has been
updated to drain the remaining response in order to avoid protocol
desynchronization when the server sends a misconfigured list. Also,
the server-side has been taught to omit empty configuration values
instead of sending invalid key-value lines.
* tc/bundle-uri-empty-fix:
bundle-uri: stop sending invalid bundle configuration
bundle-uri: drain remaining response on invalid bundle-uri lines
The cache-scanning loop in 'next_cache_entry()' has been optimized
to avoid rescanning already-unpacked index entries, preventing a
quadratic performance slow-down when diffing the working tree
against a commit with a pathspec matching early index entries.
* hf/unpack-trees-quadratic-scan:
unpack-trees: avoid quadratic index scan in next_cache_entry()
The contributor guide has been updated to advise new contributors to
trim irrelevant quoted text when replying to review comments, matching
the existing advice given to reviewers.
* wy/doc-myfirstcontribution-trim-quotes:
MyFirstContribution: mention trimming quoted text in replies
The static allow-list in expand_atom() is hardcoded to only allow
"objectname" and "objectsize" for remote queries. This works because
up to this point all servers will either support object-info with name
and size or they do not support them at all, but we cannot expect that
in a future different servers with different git versions to have the
same object-info capabilities. Therefore, the allow_list needs to be
dynamic depending on what the server advertises.
The client will now:
1. Request the protocol option that the placeholder refers to (i.e.
"size" when "%(objectsize)").
2. Filters the request in fetch_object_info() dropping any option that
the server does not advertise.
3. After the fetching, the options that haven't been dropped are the ones
fetched and supported by the server, these supported options are
mapped and remote_allowed_atoms is populated with the placeholders.
4. expand_atom() checks remote_allowed_atoms with the same behaviour as
the static allow_list had.
Move object_info_options out of get_remote_info so the caller which has
data can select what options will be requested instead of requesting
always size.
Move batch_object_write() out so there will always be an output even if
all the placeholders are not supported by the server (returns an empty
line).
Include "type" in the object_info_options so once the server supports
it, the clients know already how to request it.
Mentored-by: Karthik Nayak <karthik.188@gmail.com>
Mentored-by: Chandra Pratap <chandrapratap3519@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
strstr() is not enough to validate the format placeholders in
remote-object-info causing two errors:
1. Atoms recognized by expand_atom() but the remote doesn't returns 1,
but data->type contains garbage causing segfault.
2. expand_atom() returns 0 for unknown atoms, calling
strbuf_expand_bad_format() which ends up dying, blocking local
queries if the same format is shared.
Add an allow-list with the supported atoms at the top of expand_atom().
In remote mode, unsupported atoms return 1 leaving the buffer empty,
honoring how for-each-ref handles known but inapplicable atoms.
As extra safety, initialize data->type to OBJ_BAD and add a NULL check
for type_name() so uninitialized data doesn't cause segfault.
Update tests that expect previous die() behavior to expect an empty
string and add an explicit test for empty string return on unknown
placeholder.
Update cat-file command documentation regarding remote-object-info.
Mentored-by: Karthik Nayak <karthik.188@gmail.com>
Mentored-by: Chandra Pratap <chandrapratap3519@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since the info command in cat-file --batch-command prints object
info for a given object, it is natural to add another command in
cat-file --batch-command to print object info for a given object
from a remote.
Add remote-object-info command to cat-file --batch-command.
While info takes object ids one at a time, this creates overhead when
making requests to a server. So remote-object-info instead can take
multiple object ids at once.
The cat-file --batch-command command is generally implemented in the
following manner:
- Receive and parse input from user
- Call respective function attached to command
- Get object info, print object info
In --buffer mode, this changes to:
- Receive and parse input from user
- Store respective function attached to command in a queue
- After flush, loop through commands in queue
- Call respective function attached to command
- Get object info, print object info
Notice how the getting and printing of object info is accomplished one
at a time. As described above, this creates a problem for making
requests to a server. Therefore, remote-object-info is implemented in
the following manner:
- Receive and parse input from user
If command is remote-object-info:
- Get object info from remote
- Loop through and print each object info
Else:
- Call respective function attached to command
- Parse input, get object info, print object info
And finally for --buffer mode remote-object-info:
- Receive and parse input from user
- Store respective function attached to command in a queue
- After flush, loop through commands in queue:
If command is remote-object-info:
- Get object info from remote
- Loop through and print each object info
Else:
- Call respective function attached to command
- Get object info, print object info
To summarize, remote-object-info gets object info from the remote and
then loops through the object info passed in, printing the info.
In order for remote-object-info to avoid remote communication
overhead in the non-buffer mode, the objects are passed in as such:
remote-object-info <remote> <oid> <oid> ... <oid>
rather than
remote-object-info <remote> <oid>
remote-object-info <remote> <oid>
...
remote-object-info <remote> <oid>
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sometimes, it is beneficial to retrieve information about an object
without downloading it entirely. The server-side logic for this
functionality was implemented in commit "a2ba162cda (object-info:
support for retrieving object info, 2021-04-20)." And the wire
format is documented at
https://git-scm.com/docs/protocol-v2#_object_info.
Introduce client-side support for the object-info capability.
Add its own function for object-info separate from existing fetch
infrastructure.
Currently, the client supports requesting a list of object IDs with
the size feature from a v2 server. If the server does not advertise
this feature (i.e., transfer.advertiseobjectinfo is set to false),
the client returns an error and exit.
Note that:
1. the entire request is written into req_buf before being sent to the
remote. This approach follows the pattern used in the
send_fetch_request() logic within 'fetch-pack.c'. Streaming the
request is not addressed in this patch.
2. When the server does not recognize an OID, following the v2 protocol,
the server returns "<OID> SP", when this happens,
fetch_object_info() sets the corresponding size pointer to NULL so
that callers can detect and handle it.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In order for a client to know what object-info components a server can
provide, advertise supported object-info features. This allows a client
to decide whether to query the server for object-info or fetch as a
fallback.
While at it, update the object-info section in 'gitprotocol-v2.adoc':
- Require full obj-oid explicitly.
- Fix parentheses.
- Define obj-size explicitly.
- Make obj-size optional in obj-info and document the behavior
for unrecognized object IDs.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are some variables initialized at the start of the
do_fetch_pack_v2() state machine. Currently, they are initialized in
FETCH_CHECK_LOCAL, which is the initial state set at the beginning
of the function.
However, a subsequent patch will allow for another initial state,
while still requiring these initialized variables.
Move the initialization to be before the state machine,
so that they are set regardless of the initial state.
Note that there is no change in behavior, because we're moving code
from the beginning of the first state to just before the execution of
the state machine.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor write_fetch_command_and_capabilities(), enabling it to serve
both fetch and additional commands.
In this context, "command" refers to the "operations" supported by
Git's wire protocol https://git-scm.com/docs/protocol-v2, such as a Git
subcommand (e.g., git-fetch(1)) or a server-side operation like
"object-info" as implemented in commit a2ba162
(object-info: support for retrieving object info, 2021-04-20).
Refactor the function signature to accept a command instead of the
hardcoded "fetch".
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
write_fetch_command_and_capabilities() is refactored in a subsequent
commit where it becomes a more general-purpose function, making it
more accessible to additional commands in the future.
Move write_fetch_command_and_capabilities() to 'connect.c', where
there are similar purpose functions.
Because string_list is only used as a pointer, use a forward
declaration [1].
[1]: https://lore.kernel.org/git/Z0RIqUAoEob8lGfM@pks.im/
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
write_fetch_command_and_capabilities() is moved to 'connect.c' in a
subsequent commit. To prepare for that, drop the static variable usage
of advertise_sid. Currently advertise_sid is used in two places:
1. In function do_fetch_pack():
if (!server_supports("session-id"))
advertise_sid = 0;
2. In function fetch_pack_config():
repo_config_get_bool("transfer.advertisesid", &advertise_sid);
About 1, it is only relevant for v0/v1 protocol, move it into
find_common().
About 2, call repo_config_get_bool() inside of
write_fetch_command_and_capabilities() and find_common() replacing the
static variable.
Because repo_config_get_bool() leaves advertise_sid as is if it is not
set, initialize it to 0 matching its default.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
hash_algo_by_name() returns "unsigned int", but the variable that it is
assigned to is "int".
Change hash_algo variable type to match hash_algo_by_name() type, also
make it const because it is never modified.
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This refactor extracts utility functions from the cat-file's test
script 't1006-cat-file.sh' into a new 'lib-cat-file.sh' dedicated
library file.
A subsequent commit will need this functions, the goal is to improve
code reuse and readability,enabling future tests to leverage these
utilities without duplicating code.
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some code used in this series declares variable i and only uses it
in a for loop, not in any other logic outside the loop.
Change the declaration of i to be inside the for loop for readability.
While at it, we also change its type from int to size_t where the
latter makes more sense.
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
disconnect_helper() only frees data inside of the if(data->helper) block
[1]. When the transport is disconnected without the helper being fully
started, data->name allocated in transport_helper_init()
is never freed.
Move FREE_AND_NULL(data->name) outside the conditional block so it's
always freed on disconnect.
[1]: https://lore.kernel.org/git/05fbadbae2184479c87c37675dde7bd79b3e32ab.1716465556.git.ps@pks.im/
Mentored-by: Karthik Nayak <karthik.188@gmail.com>
Mentored-by: Chandra Pratap <chandrapratap3519@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 10a6762719 (object-file: adapt `stream_object_signature()` to take a
stream, 2026-02-23), we have refactored `stream_object_signature()` so
that it doesn't create the stream ad-hoc anymore. Instead, callers are
expected to pass in a stream, which allows them to construct the streams
from different sources.
While the stream was previously managed by `stream_object_signature()`,
the full lifecycle is now owned by the caller. Hence, it's the caller's
responsibility to close the stream, and the called function shouldn't do
that anymore.
And while the mentioned commit did drop one call that closed the stream,
there's a second such call that was missed when reading from the stream
fails. The consequence of this can be a double free of the stream.
Fix the bug by dropping that leftover call to `odb_read_stream_close()`.
Note that it was originally discussed whether this should be treated as
a security vulnerability. But there are only two callers: once via
`parse_object_with_flags()`, and once via `verify_packfile()`. Neither
of these callers plays any role on the transport layer, so this issue is
only relevant for objects that are already available via the local
object database. Furthermore, a packfile that is corrupted in this way
would be detected when receiving the packfile, so it's not easy for an
adversary to plant such a packfile, either. Consequently, we decided
that this is not covered as part of our threat model.
Reported-by: xuqing yang <rigelyoung@icloud.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With b4 0.15.2, I hit a local failure after sending a series with the
in-tree cover template. The generated sent/<change-id>-vN tag contained
base-commit, but did not contain change-id, and later b4 commands failed
when trying to read it:
CRITICAL: Tag sent/... does not contain change-id info
Looking at b4's source, the sent tag message is derived from the rendered
cover letter. The same code later parses that tag and expects both
base-commit and change-id to be present. The default b4 cover template
has both trailers, but our in-tree template only has base-commit.
Add the missing change-id trailer next to base-commit so sent tags
produced from the project template remain readable by b4's reroll and
comparison logic.
Signed-off-by: Chen Linxuan <me@black-desk.cn>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The pipelines in 't1410-reflog.sh' have been replaced with the
'test_stdout_line_count' helper to avoid suppressing the exit code of
'git' commands, ensuring failures are not hidden from the test suite.
* gr/t1410-reflog-exit-code:
t1410-reflog.sh: avoid suppressing git's exit code in pipelines
The test suite has been updated to use the 'test_grep' helper instead
of bare 'grep' for test assertions, allowing file contents to be
printed on failure for easier debugging. A new 'greplint' linter has
been introduced to detect and prevent new bare 'grep' assertions from
being added to the test suite.
* mm/test-grep-lint:
t: add greplint to detect bare grep assertions
t: convert grep assertions to test_grep
t: fix Lexer line count for $() inside double-quoted strings
t: extract chainlint's parser into shared module
t: fix grep assertions missing file arguments
t/README: document test_grep helper
The build system has been updated to support building universal macOS
binaries when 'Rust' is enabled, by compiling separate static archives
for each target triple listed in 'RUST_TARGETS' and combining them
using the macOS 'lipo' tool. The 'git-credential-osxkeychain' helper
has been updated to link against '$(RUST_LIB)' when 'Rust' is enabled.
* sn/osxkeychain-rust-universal:
contrib: wire up osxkeychain in contrib/Makefile on macOS
Makefile: support universal macOS builds via RUST_TARGETS
Makefile: add $(RUST_LIB) prerequisite to osxkeychain
Option parsing with 'git rev-parse --parseopt' and in most 'git'
subcommands has been updated to exit with 0 (instead of 129) when the
help option ('-h' or '--help') is requested directly by the user,
aligning with standard Unix convention.
* bc/parse-options-exit-0-on-help:
parse-options: exit 0 on -h
rev-parse: have --parseopt callers exit 0 on --help
parse-options: add a separate case for help output on error
t1517: skip svn tests if svn is not installed
The 'SubmittingPatches' document has been updated to explicitly
describe the expectation for contributors to retract or abandon their
patch series when they are no longer pursuing it.
* jc/submitting-patches-abandoning:
SubmittingPatches: document how to retract a topic
The early-exit optimization in 'paint_down_to_common()' has been gated
on the queue being generation-ordered, fixing a bug where 'git merge-
base' (without '--all') could return incorrect results on repositories
with v1 commit graphs and clock skew.
* kk/commit-reach-find-all-fix:
commit-reach: guard !FIND_ALL early exit with generation ordering check
t6600: add test for merge-base early exit with clock skew
A description in the release notes for Git 2.55.0 has been
retroactively updated to clarify that Rust support is enabled by
default, but still optional, and will become mandatory in Git 3.0.
* jc/relnotes-2.55-rust-fix:
Rust: fix description in Release Notes to 2.55
The repository discovery and repository configuration phases, which
were previously intertwined in 'setup.c', have been split. Repository
discovery has been updated to populate a 'struct repo_discovery'
without modifying the repository state, which is then taken by
repository configuration to initialize the repository, paving the way
for clean unification of repository configuration.
* ps/setup-split-discovery-and-setup:
setup: mark `set_git_work_tree()` as file-local
setup: pass worktree to `init_db()`
setup: drop redundant configuration of `startup_info->have_repository`
setup: make repository discovery self-contained
setup: propagate prefix via repository discovery
setup: drop static `cwd` variable
setup: move prefix into repository
setup: embed repository format in discovery
setup: introduce explicit repository discovery
setup: split up concerns of `setup_git_env_internal()`
setup: unify setup of shallow file
setup: mark bogus worktree in `apply_repository_format()`
setup: rename `check_repository_format_gently()`
The reftable code has been hardened against corrupted tables by
fixing out-of-bounds writes, out-of-bounds reads, and abort calls
during parsing.
* ps/reftable-hardening:
reftable/table: fix OOB read on truncated table
reftable/table: fix NULL pointer access when seeking to bogus offsets
reftable/block: fix OOB read with bogus restart offset
reftable/block: fix use of uninitialized memory when binsearch fails
reftable/block: fix OOB read with bogus restart count
reftable/block: fix OOB read with bogus block size
reftable/block: fix OOB write with bogus inflated log size
t/unit-tests: introduce test helper to write reftable blocks
reftable/record: don't abort when decoding invalid ref value type
reftable/basics: fix OOB read on binary search of empty range
oss-fuzz: add fuzzer for parsing reftables
meson: support building fuzzers with libFuzzer
When many tombstones are present in a reftable, operations that need
to look up or iterate over refs exhibit quadratic behavior. With
8000 refs deleted and re-created, update-ref takes ~15s, quadrupling
for each doubling of input size.
The root cause is the merged iterator's suppress_deletions flag.
When set, merged_iter_next_void() silently consumes tombstone records
in a tight internal loop before returning to the caller. This
prevents higher-level code from checking iteration bounds (such as
prefix or refname comparisons) until after all tombstones have been
scanned.
This affects any code path that seeks into a range containing
tombstones, including:
- refs_verify_refnames_available() seeks to "refs/tags/foo-1/" to
check for D/F conflicts and must scan through all subsequent
tombstones before the caller can see that they are past the prefix
of interest.
- reftable_backend_read_ref() seeks to a specific refname and must
scan through all subsequent tombstones before returning "not
found", because the merged iterator skips the matching tombstone
and searches for the next live record.
Fix this by making suppress_deletions configurable via
reftable_stack_options instead of unconditionally enabling it. Git
no longer sets the flag, so tombstones are now returned to callers in
the reftable backend, which skip them after their existing bounds
checks. This allows iteration to terminate as soon as a tombstone
past the relevant bound is encountered.
Downstream users of the reftable library (e.g. libgit2) can still
enable suppress_deletions through the stack options to retain the
previous behavior.
This also requires adding deletion checks to the log iteration paths,
since suppress_deletions applied to both ref and log iterators.
Both tests in p1401 go from ~13s to ~0.2s with this change.
Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add performance tests for update-ref when many tombstones are present
in a reftable.
The first test exercises the scenario where all refs are deleted
(creating tombstones) and then re-created with the same names, which
currently exhibits quadratic behavior.
The second test uses a separate repository with an asymmetric variant
where refs are deleted and then new, differently-named refs are
created. When the tombstones sort after the new refs, every create
scans all tombstones, making this case even worse than re-creating
the same refs.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous fix reuses the local `hex` variable that is already
computed at the top of `write_one_shallow()`. That works today, but
`oid_to_hex()` returns a pointer into a small rotating buffer, so it is
not stable across an unrelated call to `oid_to_hex()` from the same
thread. A future edit that adds such a call between the assignment and
the last user of `hex` would silently corrupt the output.
Move `write_one_shallow()` off the rotating buffer entirely by using a
local buffer instead. The current users of that `hex` variable are
unchanged.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Assisted-by: Claude Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After `write_one_shallow()` calls `lookup_commit()` to find the commit
object for a shallow graft entry, it then checks `if (!c || ...)`.
Inside that block, when the VERBOSE flag is set, it prints the OID being
removed, via `c->object.oid`. But `c` can be NULL (the first condition
in the `||` check).
This happens when a shallow graft entry references a commit object that
is not in the object store (e.g., after a partial fetch or in a
corrupted repository). In that case, `lookup_commit()` returns NULL
because the object cannot be found, the SEEN_ONLY check correctly
decides to remove this entry from .git/shallow, but the verbose message
crashes before the removal can complete.
Use `graft->oid` instead of `c->object.oid` for the message. The graft
entry's OID is the same value (it was used as the lookup key) and is
always available regardless of whether the commit object exists.
Pointed out by Coverity.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When `refs_resolve_ref_unsafe()` is called to resolve HEAD, and returns
NULL (e.g., HEAD does not exist as a proper ref), the code falls back to
`repo_get_oid("HEAD")` to try to resolve the OID directly. If that
succeeds, execution continues with `head` still set to NULL.
Later, that variable is passed to `repo_get_oid()` and `starts_with()`,
both of which would dereference the NULL pointer.
A concrete trigger for `refs_resolve_ref_unsafe()` returning NULL while
`repo_get_oid()` succeeds could not be constructed against the ref
backends currently in the tree; the naive case (a symbolic HEAD pointing
at a nonexistent branch, in either the files or the reftable backend)
fails in both calls consistently and returns via the existing
`error(_("bad HEAD - I need a HEAD"))` path. Coverity, however, flags
the leftover use of `head` after the outer `if (!head)` on a formal
reading: `head` is still NULL at that point, and both `starts_with(head,
...)` and the second `repo_get_oid(..., head, ...)` in the else-branch
would dereference it if that state were ever reached.
Removing the outer check would risk regressing to a crash if a future
ref backend ever manages to hit the "returns NULL for HEAD but has a
valid OID for HEAD" state. Assigning the literal string "HEAD" as a
safe fallback documents the intent and satisfies the analyzer without
changing behavior in any code path we can currently reach.
Assisted-by: Claude Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When `prepare_midx_bitmap_git()` is called to load the bitmap for a
chained MIDX's base layer, if the base MIDX does not have an associated
bitmap file (e.g., it was not generated, or was deleted by gc), the
return value is NULL. It is then stored in `bitmap_git->base` and
immediately dereferenced on the next line.
This can happen in practice with incremental MIDX chains: the base MIDX
may have been written without `--write-bitmap-index`, or the bitmap may
have been pruned while the incremental layer's bitmap still references
it.
Check the return value and go to the cleanup label (which unmaps the
current bitmap and returns -1) so the caller falls back to non-bitmap
object enumeration, matching the handling of other bitmap loading
failures in the same function.
Pointed out by Coverity.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This function resolves revision suffixes like commit^@ (all parents),
commit^! (commit minus parents), and commit^-N (exclude Nth parent). It
calls `get_reference()` in a loop to peel through tag objects until it
reaches a commit.
The existing NULL check after `get_reference()` only handles the
ignore_missing case, but get_reference() can return NULL through three
distinct paths:
1. revs->ignore_missing: the caller asked to silently skip missing
objects.
2. revs->exclude_promisor_objects: the object is a lazy promisor
object that should be excluded from the walk.
3. revs->do_not_die_on_missing_objects: the caller wants to record
missing OIDs for later reporting (used by `git rev-list
--missing=print`) rather than dying.
In the latter two instances, the code falls through to dereference the
NULL pointer.
Handle all three cases explicitly:
- ignore_missing: return 0, matching the existing behavior and
the pattern in `handle_revision_arg()`.
- do_not_die_on_missing_objects: return 0. The missing OID has already
been recorded in `revs->missing_commits` by `get_reference()`.
Returning 0 is consistent with `handle_revision_arg()` and
`process_parents()`, both of which continue without error when this flag
is set. The broader codebase pattern for this flag is "record and
continue": list-objects.c, builtin/rev-list.c, and process_parents
all skip the die/error and keep walking.
- everything else (only the `exclude_promisor_objects` case in
practice): return -1, consistent with `handle_revision_arg()` where
the condition only matches `ignore_missing` or
`do_not_die_on_missing_objects`, falling through to ret = -1 for the
promisor case.
Note: the callers of `add_parents_only()` in
`handle_revision_pseudo_opt()` treat any nonzero return as "handled"
(`if (add_parents_only(...)) { ret = 0; }`), so the -1 for the promisor
case is indistinguishable from success there. This means a
promisor-excluded tag target referenced via commit^@ would be silently
skipped rather than producing an error. This is a pre-existing
limitation of the caller's return value handling and not made worse by
this change; the alternative (a NULL dereference crash) _would be_
strictly worse.
Pointed out by Coverity.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `peel_committish()` function calls `repo_peel_to_type()` to convert
the given object to a commit, but does not check the return value. When
the object exists but cannot be peeled to a commit (e.g., a tree or blob
OID is passed as --onto), the return value is NULL. Add an explicit NULL
check and die with a descriptive message in that case.
Pointed out by Coverity.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano