The fetch.writeCommitGraph behavior has been optimized to
incrementally update the commit graph using only the newly fetched
commits as seeds. This avoids a full reachability scan of all
references and drastically reduces the post-fetch commit-graph
update time for repositories with many references.
* kk/fetch-write-commit-graph-incremental:
fetch: write commit-graph using updated refs only
test-tool read-graph: add commit-info subcommand
The routine that rewrites the packed-refs file has been optimized to
directly pass unmodified reference records verbatim to the output
via fwrite(3p), instead of redundantly re-formatting them with
fprintf(3p).
* kn/packed-refs-verbatim-write:
packed-refs: use `fwrite()` when passing refs verbatim
Validation of the 'fetch.followRemoteHEAD' configuration variable
has been deferred to when it is actually used, preventing irrelevant
warnings during 'git fetch' invocations that do not consult it.
* ch/fetch-followremotehead-lazy-validation:
fetch.c: defer fetch.followRemoteHEAD validation
Closing a packfile did not evict the entries belonging to it from
the delta base cache, which could lead to data corruption when a
newly allocated packfile happened to reuse the same memory address.
The cache eviction logic has been updated to remove these stale
entries upon closing a pack
* ps/packfile-stale-delta-base-cache-fix:
packfile: fix corruption due to stale delta base cache entries
packfile: move around `close_pack()`
Replace a link to a site that disappeared with a link to web archive
of the last known good copy.
* dk/fix-shattered-io-link:
doc: fix shattered.io link
A test script (t0450) has been updated to use the test_path_is_file
and test_path_is_missing helper functions, which provides better
diagnostic output on failure.
* ma/t0450-use-test-path-is-file:
t0450: use test_path_is_file and test_path_is_missing
The requirement for a 'SYNOPSIS' section in section 7 manual pages
has been relaxed by updating the linter script, and dummy
'SYNOPSIS' sections from existing section 7 documentation have been
removed to improve the rendered output
* je/doc-section-7-synopsis:
doc: don't require a SYNOPSIS in section 7
The shell quoting of a lazy prereq script was quoted incorrectly and
allowed variables dereferenced at the wrong moment, breaking the
curl version detection.
* jk/t5551-test-shell-quote-fix:
t5551: fix quoting in curl version bug prereq
The documentation for core.pager has been updated to provide
examples of overriding the LESS environment variable for custom
pagers by prefixing the pager command in the configuration.
* tz/doc-override-less-env:
doc: add more examples of overriding LESS in core.pager
The commit map populated from the state branch in 'git
filter-branch' has been corrected to parse the map using the
original-to-rewritten order.
* gm/filter-branch-state-map-fix:
filter-branch: fix commit map init from state branch
Fix definition for the "--all" option to use symbolic constant
in the options[] array passed to parse_options API.
* pw/stash-all-parseopt-fix:
stash: use named constant when parsing "--all"
When fetch.writeCommitGraph was introduced in 50f26bd035 (fetch: add
fetch.writeCommitGraph config setting, 2019-09-02), the stated goal
was to stay updated with the latest commits after fetching new
objects. The implementation used write_commit_graph_reachable()
because it was the only API available, but two things have changed
since then:
1. write_commit_graph() was added, and it accepts an explicit set of
commits as seeds, enabling more targeted commit-graph updates.
2. The ref-scanning callback add_ref_to_set() became more expensive
in 630cd5194e (commit-graph.c: peel refs in 'add_ref_to_set',
2020-07-22) when it started to validate the refs against the odb
for correctness. On a repository with many refs, this makes the
full reachable scan unnecessarily costly for a targeted fetch.
Optimize the commit-graph write by using only the newly updated refs
as seeds instead of scanning all refs after every fetch. To keep
this change small, skip the optimization for multi-remote fetches
(since that would require propagating the set of refs across process
boundaries).
This relies on the commit-graph write being additive, keeping the
commits that are already in the graph. fetch already operates in
this mode (COMMIT_GRAPH_WRITE_SPLIT) and now that becomes
required for correctness. Without that mode, the write would
replace the commit-graph and lose other commits.
After fetch_one() returns, call prepare_commit_graph() (which is
made non-static by this commit) to determine the graph-write mode:
- If no commit-graph exists yet, fall back to the full reachable
scan so the first graph creation covers all refs.
- If a commit-graph exists and the fetch updated at least one ref,
write incrementally using only the new refs as seeds.
- If a commit-graph exists but the fetch is a no-op, skip the
commit-graph write entirely.
- For the multi-remote path (fetch --all), where child processes
do the actual fetching, fall back to the full reachable scan.
Full commit-graph coverage of all refs remains the responsibility
of "git maintenance", "git gc" and "git commit-graph write".
Regular Git operations may trigger "git maintenance run --auto",
which periodically rebuilds the commit-graph from all reachable
refs.
The effect was measured on a synthetic setup: git.git with 200K
extra packed refs (~206K total), a local file:// remote, an existing
split commit-graph and a warm page cache. The times below are the
median of 9 runs of the trace2 region fetch/write-commit-graph:
scenario before after
no-op fetch 380 ms (skipped)
1 ref updated 357 ms 9.3 ms
10 refs updated 359 ms 8.9 ms
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test infrastructure has no way to check whether a specific commit
is present in the commit-graph, making it hard to verify graph state
after operations like fetch.
Add a "commit-info" subcommand to test-tool read-graph that queries
whether specific commits are present in the commit-graph and prints
their generation numbers. Returns 1 if any commit is not found.
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'format-patch' command has been updated with options to
configure notes specifically for range-diff output, allowing them to
differ from the notes displayed on the patches themselves.
* kh/format-patch-range-diff-notes:
format-patch: learn --[no-]range-diff-notes
format-patch: simplify get_notes_arg parameters
The documentation for 'git interpret-trailers' has been updated to
fix the grammatical errors in the descriptions of command examples.
* kh/doc-trailers-cmd-examples:
doc: interpret-trailers: fix cmd examples
The documentation for 'git remote set-head' has been updated to
clarify that it only modifies the local repository's remote-tracking
symbolic reference, and does not alter the target remote
repository's own 'HEAD'.
* mg/doc-remote-set-head:
doc: remote: say that it only affects the local repository
The parse-options API has been updated to support grouping
subcommands, and the 'git refs' command has been updated to use this
new feature to categorize its subcommands in its help output.
* ps/parse-options-subcommand-groups:
builtin/refs: introduce subcommand groups
parse-options: allow grouping subcommands
parse-options: extract functions to print single option
parse-options: fix completion format when first option is skipped
The @{upstream} notation had short-hand @{u} from the beginning,
and @{push} learned corresponding @{p} to match.
* hn/object-name-push-short:
object-name: accept @{p} as short for @{push}
The 'git-contacts' program (in contrib/) has been taught to read
patches from the standard input stream when an explicit dash ('-') is
passed in place of a file.
* bc/git-contacts-stdin:
git-contacts: allow inputting patch via stdin
The 'git var' command has been updated to output individual components
of the author and committer identities, take multiple keys on the
command-line, and support NUL-delimited output with the '-z' option.
* ap/var-broken-down-idents:
var: add broken-out identity variables
var: accept more than one variable
var: add "-z" output mode
var: represent multi-valued variables with a string_list
The handling of index conflicts in 'git stash apply --index' has
been optimized. Instead of relying on expensive external
subprocesses using 'git diff-tree' and 'git apply', the index is now
merged in-core using the merge-ort machinery, which fixes a
concurrent lock file issue with autostash.
* dk/stash-apply-index-incore:
builtin/stash: merge index in-core
t3903: test failed "stash apply --index"
stash: prepare merge options earlier
builtin/stash: remove unused header
The internal diff and merge machinery has been updated to use
standard size_t types rather than signed longs for buffer
allocations and offsets, preventing subtle overflow issues on LLP64
architectures, and plugging uninitialized memory leaks along error
paths.
* jk/xdiff-size_t:
merge-ll: report an error when reading external merge results fails
merge-ll: handle external driver status before reading result
merge-ll: use read_mmfile() to read external merge results
xdiff: NUL-terminate buffers read by read_mmfile()
xdiff: use size_t for buffer sizes
xdiff: replace mmbuffer_t with mmfile_t
xdiff: clean up read_mmfile() allocations on error
The internal representation of timezones in the reflog for the
reftable format has been fixed to match the specification, which
dictates an offset in minutes rather than the parsed "+HHMM" integer
representation.
* ps/reftable-reflog-timezone:
refs/reftable: fix on-disk representation of reflog timezones
t/helper: fix segfault in "dump-reftable -t"
date: add helpers to convert between "+HHMM" timezones and minutes
The `t5520` test script has been updated to disable reflog expiration.
This prevents test flakiness caused by the auto-maintenance
reflog-expire task, which would otherwise immediately expire the
test's reflogs due to them carrying hardcoded timestamps from 2005.
* tb/t5520-reflog-expire:
t5520: don't expire reflogs where it matters
CI resource exhaustion during test runs on GitHub Actions has been
mitigated by switching a comparison of a huge output to use a binary
comparison, and by capping the parallel jobs on Linux to the number
of available CPUs.
* td/ci-large-test-resources:
ci: use twice the CPU count on both providers
t4205: compare huge output without diff
The build configurations for Meson have been optimized to avoid
recompiling HTTP sources multiple times and to utilize precompiled
headers for test-helpers and unit tests, reducing clean build times.
Additionally, the shell completion tests and GitLab CI MSVC runner
jobs have been fixed.
* ps/meson-improvements:
gitlab-ci: fix hanging MSVC jobs
meson: update wrappers
meson: fix outdated completion helpers
meson: use precompiled headers for unit tests
meson: use precompiled headers for our test-helper
meson: don't recompile git-remote-http(1) multiple times for tests
meson: avoid recompiling HTTP sources several times
The `write_with_updates()` function uses a `struct ref_iterator` to
iterate over all refs to write to the temporary packed-refs file. It
receives the iterator from `packed_ref_iterator_begin()` which takes a
snapshot of the 'packed-refs' file.
While writing to the new packed-refs file, writes are routed via
`write_packed_entry()` which uses `fprintf()`. Even for references which
haven't changed, we use the same mechanism. This is slow since we need
to do string parsing and formatting. Instead, let's track the position
of unchanged references in the snapshot iterator and directly write
verbatim via `fwrite()`.
With this, any sanitization which was happening as a side effect of
reformatting is now lost. But that was never the job of this section of
the code, since the main intention is to simply rewrite the remaining
refs post deletion of the selective few.
This removes the unnecessary formatting operation involved. We can see a
consistent ~20% performance improvement when deleting from packed
references.
Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
Time (mean ± σ): 28.7 ms ± 1.7 ms [User: 22.5 ms, System: 5.9 ms]
Range (min … max): 26.7 ms … 33.3 ms 46 runs
Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
Time (mean ± σ): 23.8 ms ± 1.2 ms [User: 17.5 ms, System: 6.0 ms]
Range (min … max): 22.1 ms … 27.7 ms 56 runs
Summary
update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
1.21 ± 0.09 times faster than update-ref: delete ref (refcount = 100000, revision = master)
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since we published this document the domain has been taken over and no
longer reflects the original content. Link to the last good Web Archive
snapshot so folks can still retrieve the SHAttered information.
Suggested-by: Mae Eckert <mae.eckert@albellus.de>
Signed-off-by: D. Ben Knoble <ben.knoble@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the SYNOPSIS section from the section 7 man pages where
appropriate, to avoid having a section that contains no information.
It's not the norm in section 7 to always require a SYNOPSIS.
Update the perl script with a special case for section 7.
Tested by running `make lint-docs`, and looked at the renaming synopses
with this fish script snippet:
for i in *.7
echo $i; grep SYNOPSIS -A 5 (string replace .7 .adoc $i)
end
Co-authored-by: Tuomas Ahola <taahol@utu.fi>
Signed-off-by: Tuomas Ahola <taahol@utu.fi>
Signed-off-by: Julia Evans <julia@jvns.ca>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace raw 'test -f' and '! test -f' assertions with the test helper
functions 'test_path_is_file' and 'test_path_is_missing' to provide
diagnostic output when an assertion fails.
Signed-off-by: Muhammed Dilshad A <dilsheddilu123@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We have a prereq snippet that invokes awk. The awk script's $1, etc,
variables need to be quoted to avoid shell interpolation. We correctly
use a single-quote inside the prereq snippet, but the snippet itself is
contained in double-quotes. So we interpolate "$1" into whatever value
that happens to have in the outer shell, and eval nonsense like:
awk '(--some-garbage == 7 && --other-garbage >= 88) ...'
As a result, we don't think we have a buggy curl version even when we
do, and run the test anyway. But of course it's easy not to notice,
since this prereq was protecting us from a racy bug. It only breaks
sometimes.
There are a few options for fixing the quoting:
1. Backslash-escaping the dollar signs. This is perhaps the least-ugly
version, but it's a minor hassle to remember if somebody touches
the code later.
2. Single-quote the snippet, then quote interior single-quotes as
'\''. Reasonably obvious, but ugly.
3. Use the '<<\EOT' here-doc trick to specify the snippet. This would
look nice, but we don't yet support it for prereqs. ;)
This patch uses (2), and we can circle back to (3) to make it look nicer
later.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The timestamp used for checking the reflog of a remote-tracking
branch during 'git push --force-if-includes' was left uninitialized
when the reflog was completely empty, which has been corrected.
* as/push-force-if-includes-no-reflog:
push: fix --force-if-includes when remote-tracking ref has no reflog
The performance test for `git fetch` pack-dir rescans has been
updated to use the `--no-deref` option when cleaning up
remote-tracking branches. This prevents `git update-ref` from
failing with a multiple updates error when both a symref and its
target are deleted in the same transaction.
* js/p5551-fetch-rescan-fix:
p5551: fix repeated runs with update-ref --no-deref
The HTTP transport has been updated to handle changes in recent curl
versions that strip credentials from the effective URL, preventing
unnecessary credential prompts during redirects.
* jk/http-curl-strip-creds:
http: handle curl stripping creds from effective url
The external merge driver in 'git merge' now uses the tempfile API
to create its temporary files. This ensures that these temporaries
are reliably cleaned up even when the merge driver or its parent Git
process is terminated abruptly.
* jk/merge-ll-tempfile-cleanup:
merge-ll: use tempfile API for external driver files
merge-ll: catch close() errors when writing external tempfiles
The Windows GitLab CI job has been updated to provision and use the
GCC-targeting Rust toolchain for MinGW builds, fixing job failures
caused by incomplete Rust setup and missing linker support.
* js/gitlab-ci-windows-rust:
ci(gitlab,windows): provide GNU Rust's host-linker support
ci(gitlab,windows): fix Rust setup for GitLab's MinGW build
ci(gitlab,windows): preserve exclusions during dependency setup
ci(gitlab,windows): provision GNU Rust for SDK-based MinGW builds
The value of the fetch.followRemoteHEAD configuration variable is
validated while the configuration file is being parsed, which
produces a warning even when this particular "git fetch" invocation
will never consult it.
Store the raw config string instead, and resolve/validate it lazily
at the one place in do_fetch() that actually uses it, so a mistyped
value only warns, and a missing value only dies, when this fetch
would have consulted it.
remote.c's handle_config() has the same problem for
remote.<name>.followRemoteHEAD, but is left unaddressed here since
it touches shared remote-parsing infrastructure used well beyond
fetch. Leave a NEEDSWORK comment at remote.c:handle_config()
that has a defect similar to what is fixed by this patch,
so the remaining scope is easy to find for a follow-up patch.
Signed-off-by: Colin Hinton <colinlewishinton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The delta base cache is a process-global hashmap that is keyed by the
address of the `struct packed_git` plus the offset of the base object
within that pack. Entries part of the cache are never removed when a
pack is closed, and neither when the pack is subsequently freed. As a
consequence, the cache may contain stale entries.
For a long time, the worst consequence of this leaking cache was that we
held on to memory that we could've released. But the reason for this was
that we didn't even free the packfiles, either. That has changed in
6f1e9394e2 (object: fix leaking packfiles when closing object store,
2024-08-08), where we plugged that leak.
Now that we free them, a new packfile may be allocated using the exact
same address as a previously allocated one. And if the new packfile has
both the same address and a similar layout, it may happen that a
preexisting entry from a previously-allocated in the delta base cache
would have the exact same key.
All of this sounds very theoretical, but we can actually trigger this
bug somewhat reliably! When doing a merge in a repository with lots of
submodules that have similar-looking packfiles we end up opening and
then closing the object databases of each of the submodules in sequence.
Because of the above mentioned commit we would close and free each of
the packfiles part of the respective databases, but we wouldn't evict
their delta base entries from the cache.
When using glibc, one of the packfiles will eventually get the exact
same address, and that will then cause Git to read the wrong entry from
the cache. Git detects this and aborts with an error:
$ git merge branch-b
error: Could not read 584ef938be4a749bfa13f68d5ac5545bc029e529
error: could not parse commit 584ef938be4a749bfa13f68d5ac5545bc029e529
error: failed to merge submodule G (repository corrupt)
Now in this case we're lucky that Git detects this error because we try
to read a commit from a different submodule via an object database that
doesn't have it. But potentially, in an even more contrived scenario, we
might even silently yield wrong data from the cache.
Fix this bug by evicting cache entries that belong to a specific pack
when closing it.
Note that the added test reliably reproduces the above bug on my machine
that uses NixOS at c59305bab206 (cosmic-applets: add missing runtime
dependency (#566040), 2026-10-01) with glibc 2.44-25. But as we rely on
specific allocation behaviour of glibc it is very likely that the test
will not work on other platforms.
Reported-by: Guillaume Chauvel <guillaume.chauvel@gmail.com>
Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the next commit we'll want to access the delta base cache in
`close_pack()`. Move the function after the declaration of the cache so
that we won't need a forward declaration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation for 'git-name-rev' has been updated to refer to
object IDs instead of SHA-1, to more accurately reflect that the
active hash algorithm is used.
* jk/name-rev-update-hash-descriptions:
name-rev: update hash descriptions
The command-line parsing of revision options has been updated to
properly distinguish between malformed and unknown options, and to
correctly report the name of unknown options after they have been
moved during the option parsing process.
* jk/parse-revision-opt-fixes:
revision: handle argv movement in parse_revision_opt()
revision: avoid reporting known options as unknown on error
The directory traversal logic has been updated to correctly evaluate
pathspecs for ignored directories containing nested repositories,
and has been fortified to gracefully handle null pathspecs during
checkout operations.
* rs/dir-skip-nested-repo-fix:
dir: skip excluded directory with nested repo on prefix match
Various documentation files have been updated to use explicit
'<<TARGET,TARGET>>' AsciiDoc cross-references instead of plain text
references. This improves rendering on the web and avoids broken
links when manual pages are included into other pages.
* je/doc-asciidoc-xrefs:
doc: add more AsciiDoc cross-references
The command-line completion script (in 'contrib/') has been taught
to complete the 'repair' subcommand for 'git worktree'.
* yn/worktree-repair-completion:
completion: complete 'git worktree repair'
The CI script has been updated to check for one-indexed slice numbers
when deciding to run unit tests on Windows, mirroring the behaviour in
the modern GitHub and GitLab workflows.
* kn/ci-unit-tests-on-windows-fix:
ci: fix unit tests not running on windows
The default expiry periods for both reachable and unreachable reflog
entries have been corrected from a past regression that swapped
them.
* ps/reflog-expiry-fix:
reflog: fix default expiry periods
Junio C Hamano