Commit Graph

78893 Commits (8c1ce2204cc755bdafec85aaa4ac9c5a686a8bf4)

Author SHA1 Message Date
Karthik Nayak 8c1ce2204c maintenance: add checking logic in `pack_refs_condition()`
The 'git-maintenance(1)' command supports an '--auto' flag. Usage of the
flag ensures to run maintenance tasks only if certain thresholds are
met. The heuristic is defined on a task level, wherein each task defines
an 'auto_condition', which states if the task should be run.

The 'pack-refs' task is hard-coded to return 1 as:
1. There was never a way to check if the reference backend needs to be
optimized without actually performing the optimization.
2. We can pass in the '--auto' flag to 'git-pack-refs(1)' which would
optimize based on heuristics.

The previous commit added a `refs_optimize_required()` function, which
can be used to check if a reference backend required optimization. Use
this within `pack_refs_condition()`.

This allows us to add a 'git maintenance is-needed' subcommand which can
notify the user if maintenance is needed without actually performing the
optimization. Without this change, the reference backend would always
state that optimization is needed.

Since we import 'revision.h', we need to remove the definition for
'SEEN' which is duplicated in the included header.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-10 09:28:48 -08:00
Karthik Nayak f6c5ca387a refs: add a `optimize_required` field to `struct ref_storage_be`
To allow users of the refs namespace to check if the reference backend
requires optimization, add a new field `optimize_required` field to
`struct ref_storage_be`. This field is of type `optimize_required_fn`
which is also introduced in this commit.

Modify the debug, files, packed and reftable backend to implement this
field. A following commit will expose this via 'git pack-refs' and 'git
refs optimize'.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-10 09:28:48 -08:00
Karthik Nayak e35155588a reftable/stack: add function to check if optimization is required
The reftable backend performs auto-compaction as part of its regular
flow, which is required to keep the number of tables part of a stack at
bay. This allows it to stay optimized.

Compaction can also be triggered voluntarily by the user via the 'git
pack-refs' or the 'git refs optimize' command. However, currently there
is no way for the user to check if optimization is required without
actually performing it.

Extract out the heuristics logic from 'reftable_stack_auto_compact()'
into an internal function 'update_segment_if_compaction_required()'.
Then use this to add and expose `reftable_stack_compaction_required()`
which will allow users to check if the reftable backend can be
optimized.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-10 09:28:47 -08:00
Karthik Nayak 135f491f83 reftable/stack: return stack segments directly
The `stack_table_sizes_for_compaction()` function returns individual
sizes of each reftable table. This function is only called by
`reftable_stack_auto_compact()` to decide which tables need to be
compacted, if any.

Modify the function to directly return the segments, which avoids the
extra step of receiving the sizes only to pass it to
`suggest_compaction_segment()`.

A future commit will also add functionality for checking whether
auto-compaction is necessary without performing it. This change allows
code re-usability in that context.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-10 09:28:47 -08:00
Junio C Hamano 31177a8bb6 Merge branch 'kn/refs-optim-cleanup' into kn/maintenance-is-needed
* kn/refs-optim-cleanup:
  t/pack-refs-tests: move the 'test_done' to callees
  refs: rename 'pack_refs_opts' to 'refs_optimize_opts'
  refs: move to using the '.optimize' functions
2025-11-04 07:38:48 -08:00
Junio C Hamano 4a1442a336 Merge branch 'ps/ref-peeled-tags' into kn/maintenance-is-needed
* ps/ref-peeled-tags: (23 commits)
  t7004: do not chdir around in the main process
  ref-filter: fix stale parsed objects
  ref-filter: parse objects on demand
  ref-filter: detect broken tags when dereferencing them
  refs: don't store peeled object IDs for invalid tags
  object: add flag to `peel_object()` to verify object type
  refs: drop infrastructure to peel via iterators
  refs: drop `current_ref_iter` hack
  builtin/show-ref: convert to use `reference_get_peeled_oid()`
  ref-filter: propagate peeled object ID
  upload-pack: convert to use `reference_get_peeled_oid()`
  refs: expose peeled object ID via the iterator
  refs: refactor reference status flags
  refs: fully reset `struct ref_iterator::ref` on iteration
  refs: introduce `.ref` field for the base iterator
  refs: introduce wrapper struct for `each_ref_fn`
  builtin/repo: add progress meter for structure stats
  builtin/repo: add keyvalue and nul format for structure stats
  builtin/repo: add object counts in structure output
  builtin/repo: introduce structure subcommand
  ...
2025-11-04 07:38:27 -08:00
Karthik Nayak c113f4ca4d t/pack-refs-tests: move the 'test_done' to callees
In ac0bad0af4 (t0601: refactor tests to be shareable, 2025-09-19), we
refactored 't/t0601-reffiles-pack-refs.sh' to move all of the tests to
't/pack-refs-tests.sh', which became a common test suite which was also
used by 't/t1463-refs-optimize.sh'.

This also moved the 'test_done' directive to 't/pack-refs-tests.sh'.
Which inhibits additional tests from being added to either of the tests.
Let's move the directive out to both the tests, so that we can add
additional specific tests to them. Also the test flow logic shouldn't be
part of tests which can be embedded in other test scripts.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:35:12 -08:00
Karthik Nayak 2cd99d9841 refs: rename 'pack_refs_opts' to 'refs_optimize_opts'
The previous commit removed all references to 'pack_refs()' within
the refs subsystem. Continue this cleanup by also renaming
'pack_refs_opts' to 'refs_optimize_opts' and the respective flags
accordingly. Keeping the naming consistent will make the code easier to
maintain.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:35:12 -08:00
Karthik Nayak 9b93ab8a9c refs: move to using the '.optimize' functions
The `struct ref_store` variable exposes two ways to optimize a reftable
backend:

  1. pack_refs
  2. optimize

The former was specific to the 'files' + 'packed' refs backend. The
latter is more generic and covers all backends. While the naming is
different, both of these functions perform the same functionality.

Consolidate this code to only maintain the 'optimize' functions. Do this
by modifying the backends so that they exclusively implement the
`optimize` callback, only. All users of the refs subsystem already use
the 'optimize' function so there is no changes needed on the callee
side. Finally, cleanup all references to the 'pack_refs' field of the
structure and code around it.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:35:12 -08:00
Junio C Hamano aec5adb4b7 Merge branch 'ps/ref-peeled-tags' into kn/refs-optim-cleanup
* ps/ref-peeled-tags: (92 commits)
  t7004: do not chdir around in the main process
  ref-filter: fix stale parsed objects
  ref-filter: parse objects on demand
  ref-filter: detect broken tags when dereferencing them
  refs: don't store peeled object IDs for invalid tags
  object: add flag to `peel_object()` to verify object type
  refs: drop infrastructure to peel via iterators
  refs: drop `current_ref_iter` hack
  builtin/show-ref: convert to use `reference_get_peeled_oid()`
  ref-filter: propagate peeled object ID
  upload-pack: convert to use `reference_get_peeled_oid()`
  refs: expose peeled object ID via the iterator
  refs: refactor reference status flags
  refs: fully reset `struct ref_iterator::ref` on iteration
  refs: introduce `.ref` field for the base iterator
  refs: introduce wrapper struct for `each_ref_fn`
  builtin/repo: add progress meter for structure stats
  builtin/repo: add keyvalue and nul format for structure stats
  builtin/repo: add object counts in structure output
  builtin/repo: introduce structure subcommand
  ...
2025-11-04 07:33:41 -08:00
Junio C Hamano 61ac8ba0f0 t7004: do not chdir around in the main process
Move down to no-contains subdirectory inside a subshell, just like
the previous step that created and used it does.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt bea37f1d64 ref-filter: fix stale parsed objects
In 054f5f457e (ref-filter: parse objects on demand, 2025-10-23) we have
started to skip parsing some objects in case we don't need to access
their values in the first place. This was done by introducing a new
member `struct expand_data::maybe_object` that gets populated on demand
via `get_or_parse_object()`.

This has led to a regression though where the object now gets reused
because we don't reset it properly. The `oi` structure is declared in
global scope, and there is no single place where we reset it before
invoking `get_object()`. The consequence is that the `maybe_object`
member doesn't get reset across calls, so subsequent calls will end up
reusing the same object.

This is only an issue for a subset of retrieved values, as not all of
the infrastructure ends up calling `get_or_parse_object()`. So the
effect is limited, which is probably why the issue wasn't detected
earlier.

Fix the issue by resetting `maybe_object` in `get_object()`.

Reported-by: Junio C Hamano <gitster@pobox.com>
Based-on-patch-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt a29e2e8fe7 ref-filter: parse objects on demand
When formatting an arbitrary object we parse that object regardless of
whether or not we actually need any parsed data. In fact, many of the
atoms we have don't require any.

Refactor the code so that we parse the data on demand when we see an
atom that wants to access the objects. This leads to a small speedup,
for example in the Chromium repository with around 40000 refs:

    Benchmark 1: for-each-ref --format='%(raw)' (HEAD~)
      Time (mean ± σ):     388.7 ms ±   1.1 ms    [User: 322.2 ms, System: 65.0 ms]
      Range (min … max):   387.3 ms … 390.8 ms    10 runs

    Benchmark 2: for-each-ref --format='%(raw)' (HEAD)
      Time (mean ± σ):     344.7 ms ±   0.7 ms    [User: 287.8 ms, System: 55.1 ms]
      Range (min … max):   343.9 ms … 345.7 ms    10 runs

    Summary
      for-each-ref --format='%(raw)' (HEAD) ran
        1.13 ± 0.00 times faster than for-each-ref --format='%(raw)' (HEAD~)

With this change, we now spend ~90% of the time decompressing objects,
which is almost as good as it gets regarding git-for-each-ref(1)'s own
infrastructure.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt e66077ae45 ref-filter: detect broken tags when dereferencing them
Users can ask git-for-each-ref(1) to peel tags and return information of
the tagged object by adding an asterisk to the format, like for example
"%(*$objectname)". If so, git-for-each-ref(1) peels that object to the
first non-tag object and then returns its values.

As mentioned in preceding commits, it can happen that the tagged object
type and the claimed object type differ, effectively resulting in a
corrupt tag. git-for-each-ref(1) would notice this mismatch, print an
error and then bail out when trying to peel the tag.

But we only notice this corruption in some very specific edge cases!
While we have a test in "t/for-each-ref-tests.sh" that verifies the
above scenario, this test is specifically crafted to detect the issue at
hand. Namely, we create two tags:

  - One tag points to a specific object with the correct type.

  - The other tag points to the *same* object with a different type.

The fact that both tags point to the same object is important here:
`peel_object()` wouldn't notice the corruption if the tagged objects
were different.

The root cause is that `peel_object()` calls `lookup_${type}()`
eventually, where the type is the same type declared in the tag object.
Consequently, when we have two tags pointing to the same object but with
different declared types we'll call two different lookup functions. The
first lookup will store the object with an unverified type A, whereas
the second lookup will try to look up the object with a different
unverified type B. And it is only now that we notice the discrepancy in
object types, even though type A could've already been the wrong type.

Fix the issue by verifying the object type in `populate_value()`. With
this change we'll also notice type mismatches when only dereferencing a
tag once.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt 6ec4c0b45b refs: don't store peeled object IDs for invalid tags
Both the "files" and "reftable" backend store peeled object IDs for
references that point to tags:

  - The "files" backend stores the value when packing refs, where each
    peeled object ID is prefixed with "^".

  - The "reftable" backend stores the value whenever writing a new
    reference that points to a tag via a special ref record type.

Both of these backends use `peel_object()` to find the peeled object ID.
But as explained in the preceding commit, that function does not detect
the case where the tag's tagged object and its claimed type mismatch.

The consequence of storing these bogus peeled object IDs is that we're
less likely to detect such corruption in other parts of Git.
git-for-each-ref(1) for example does not notice anymore that the tag is
broken when using "--format=%(*objectname)" to dereference tags.

One could claim that this is good, because it still allows us to mostly
use the tag as intended. But the biggest problem here is that we now
have different behaviour for such a broken tag depending on whether or
not we have its peeled value in the refdb.

Fix the issue by verifying the object type when peeling the object. If
that verification fails we simply skip storing the peeled value in
either of the reference formats.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt 7ec85185b1 object: add flag to `peel_object()` to verify object type
When peeling a tag to a non-tag object we repeatedly call
`parse_object()` on the tagged object until we find the first object
that isn't a tag. While this feels sensible at first, there is a big
catch here: `parse_object()` doesn't actually verify the type of the
tagged object.

The relevant code path here eventually ends up in `parse_tag_buffer()`.
Here, we parse the various fields of the tag, including the "type". Once
we've figured out the type and the tagged object ID, we call one of the
`lookup_${type}()` functions for whatever type we have found. There is
two possible outcomes in the successful case:

  1. The object is already part of our cached objects. In that case we
     double-check whether the type we're trying to look up matches the
     type that was cached.

  2. The object is _not_ part of our cached objects. In that case, we
     simply create a new object with the expected type, but we don't
     parse that object.

In the first case we might notice type mismatches, but only in the case
where our cache has the object with the correct type. In the second
case, we'll blindly assume that the type is correct and then go with it.
We'll only notice that the type might be wrong when we try to parse the
object at a later point.

Now arguably, we could change `parse_tag_buffer()` to verify the tagged
object's type for us. But that would have the effect that such a tag
cannot be parsed at all anymore, and we have a small bunch of tests for
exactly this case that assert we still can open such tags. So this
change does not feel like something we can retroactively tighten, even
though one shouldn't ever hit such corrupted tags.

Instead, add a new `flags` field to `peel_object()` that allows the
caller to opt in to strict object verification. This will be wired up at
a subset of callsites over the next few commits.

Note that this change also inlines `deref_tag_noverify()`. There's only
been two callsites of that function, the one we're changing and one in
our test helpers. The latter callsite can trivially use `deref_tag()`
instead, so by inlining the function we avoid having to pass down the
flag.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt 705114772e refs: drop infrastructure to peel via iterators
Now that the peeled object ID gets propagated via the `struct reference`
there is no need anymore to call into the reference iterator itself to
dereference an object. Remove this infrastructure.

Most of the changes are straight-forward deletions of code. There is one
exception though in `refs/packed-backend.c::write_with_updates()`. Here
we stop peeling the iterator and instead just pass the peeled object ID
of that iterator directly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt 5a5c7359f7 refs: drop `current_ref_iter` hack
In preceding commits we have refactored all callers of
`peel_iterated_oid()` to instead use `reference_get_peeled_oid()`. This
allows us to thus get rid of the former function.

Getting rid of that function is nice, but even nicer is that this also
allows us to get rid of the `current_ref_iter` hack. This global
variable tracked the currently-active ref iterator so that we can use it
to peel an object ID. Now that the peeled object ID is propagated via
`struct reference` though we don't have to depend on this hack anymore,
which makes for a more robust and easier-to-understand infrastructure.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt feaaea4c12 builtin/show-ref: convert to use `reference_get_peeled_oid()`
The git-show-ref(1) command has multiple different modes:

  - It knows to show all references matching a pattern.

  - It knows to list all references that are an exact match to whatever
    the user has provided.

  - It knows to check for reference existence.

The first two commands use mostly the same infrastructure to print the
references via `show_one()`. But while the former mode uses a proper
iterator and thus has a `struct reference` available in its context, the
latter calls `refs_read_ref()` and thus doesn't. Consequently, we cannot
easily use `reference_get_peeled_oid()` to print the peeled value.

Adapt the code so that we manually construct a `struct reference` when
verifying refs. We wouldn't ever have the peeled value available anyway
as we're not using an iterator here, so we can simply plug in the values
we _do_ have.

With this change we now have a `struct reference` available at both
callsites of `show_one()` and can thus pass it, which allows us to use
`reference_get_peeled_oid()` instead of `peel_iterated_oid()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt 70b783c3a1 ref-filter: propagate peeled object ID
When queueing a reference in the "ref-filter" subsystem we end up
creating a new ref array item that contains the reference's info. One
bit of info that we always discard though is the peeled object ID, and
because of that we are forced to use `peel_iterated_oid()`.

Refactor the code to propagate the peeled object ID via the ref array,
if available. This allows us to manually peel tags without having to go
through the object database.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt adecd5f0b6 upload-pack: convert to use `reference_get_peeled_oid()`
The `write_v0_ref()` callback is invoked from two callsites:

  - Once via `send_ref()` which is a callback passed to
    `for_each_namespaced_ref_1()` and `refs_head_ref_namespaced()`.

  - Once manually to announce capabilities.

When sending references to the client we also send the peeled value of
tags. As we don't have a `struct reference` available in the second
case, we cannot easily peel by calling `reference_get_peeled_oid()`, but
we instead have to depend on on global state via `peel_iterated_oid()`.

We do have a reference available though in the first case, it's only the
second case that keeps us from using `reference_get_peeled_oid()`. But
that second case only announces capabilities anyway, so we're not really
handling a reference at all here.

Adapt that case to construct a reference manually and pass that to
`write_v0_ref()`. Start to use `reference_get_peeled_oid()` now that we
always have a `struct reference` available.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt f898661637 refs: expose peeled object ID via the iterator
Both the "files" and "reftable" backend are able to store peeled values
for tags in the respective formats. This allows for a more efficient
lookup of the target object of such a tag without having to manually
peel via the object database.

The infrastructure to access these peeled object IDs is somewhat funky
though. When iterating through objects, we store a pointer reference to
the current iterator in a global variable. The callbacks invoked by that
iterator are then expected to call `peel_iterated_oid()`, which checks
whether the globally-stored iterator's current reference refers to the
one handed into that function. If so, we ask the iterator to peel the
object, otherwise we manually peel the object via the object database.
Depending on global state like this is somewhat weird and also quite
fragile.

Introduce a new `struct reference::peeled_oid` field that can be
populated by the reference backends. This field can be accessed via a
new function `reference_get_peeled_oid()` that either uses that value,
if set, or alternatively peels via the ODB. With this change we don't
have to rely on global state anymore, but make the peeled object ID
available to the callback functions directly.

Adjust trivial callers that already have a `struct reference` available.
Remaining callers will be adjusted in subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt eb2934d94b refs: refactor reference status flags
The reference flags encode information like whether or not a reference
is a symbolic reference or whether it may be broken. This information is
stored in a `int flags` bitfield, which is in conflict with our modern
best practices; we tend to use an unsigned integer to store flags.

Change the type of the field to be `unsigned`. While at it, refactor the
individual flags to be part of an `enum` instead of using preprocessor
defines.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt 4cea042287 refs: fully reset `struct ref_iterator::ref` on iteration
With the introduction of the `struct ref_iterator::ref` field it now is
a whole lot easier to introduce new fields that become accessible to the
caller without having to adapt every single callsite. But there's a
downside: when a new field is introduced we always have to adapt all
backends to set that field.

This isn't something we can avoid in the general case: when the new
field is expected to be populated by all backends we of course cannot
avoid doing so. But new fields may be entirely optional, in which case
we'd still have such churn. And furthermore, it is very easy right now
to leak state from a previous iteration into the next iteration.

Address this issue by ensuring that the reference backends all fully
reset the field on every single iteration. This ensures that no state
from previous iterations can leak into the next one. And it ensures that
any newly introduced fields will be zeroed out by default.

Note that we don't have to explicitly adapt the "files" backend, as it
uses the `cache_ref_iterator` internally. Furthermore, other "wrapping"
iterators like for example the `prefix_ref_iterator` copy around the
whole reference, so these don't need to be adapted either.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt 89baa52da6 refs: introduce `.ref` field for the base iterator
The base iterator has a couple of fields that tracks the name, target,
object ID and flags for the current reference. Due to this design we
have to create a new `struct reference` whenever we want to hand over
that reference to the callback function, which is tedious and not very
efficient.

Convert the structure to instead contain a `struct reference` as member.
This member is expected to be populated by the implementations of the
iterator and is handed over to the callback directly.

While at it, simplify `should_pack_ref()` to take a `struct reference`
directly instead of passing its respective fields.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:25 -08:00
Patrick Steinhardt bdbebe5714 refs: introduce wrapper struct for `each_ref_fn`
The `each_ref_fn` callback function type is used across our code base
for several different functions that iterate through reference. There's
a bunch of callbacks implementing this type, which makes any changes to
the callback signature extremely noisy. An example of the required churn
is e8207717f1 (refs: add referent to each_ref_fn, 2024-08-09): adding a
single argument required us to change 48 files.

It was already proposed back then [1] that we might want to introduce a
wrapper structure to alleviate the pain going forward. While this of
course requires the same kind of global refactoring as just introducing
a new parameter, it at least allows us to more change the callback type
afterwards by just extending the wrapper structure.

One counterargument to this refactoring is that it makes the structure
more opaque. While it is obvious which callsites need to be fixed up
when we change the function type, it's not obvious anymore once we use
a structure. That being said, we only have a handful of sites that
actually need to populate this wrapper structure: our ref backends,
"refs/iterator.c" as well as very few sites that invoke the iterator
callback functions directly.

Introduce this wrapper structure so that we can adapt the iterator
interfaces more readily.

[1]: <ZmarVcF5JjsZx0dl@tanuki>

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-04 07:32:24 -08:00
Junio C Hamano a99f379adf The 27th batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-30 08:00:20 -07:00
Junio C Hamano be414e17e5 Merge branch 'rz/bisect-help-unknown'
"git bisect" command did not react correctly to "git bisect help"
and "git bisect unknown", which has been corrected.

* rz/bisect-help-unknown:
  bisect: fix handling of `help` and invalid subcommands
2025-10-30 08:00:20 -07:00
Junio C Hamano ee335b9f81 Merge branch 'kf/log-shortlog-completion-fix'
"git shortlog" knows "--committer" and "--author" options, which
the command line completion (in contrib/) did not handle well,
which has been corrected.

* kf/log-shortlog-completion-fix:
  completion: complete some 'git log' options
2025-10-30 08:00:20 -07:00
Junio C Hamano 27a1735807 Merge branch 'ly/diff-name-only-with-diff-from-content'
Regression fixes for a topic that has already been merged.

* ly/diff-name-only-with-diff-from-content:
  diff: stop output garbled message in dry run mode
2025-10-30 08:00:20 -07:00
Junio C Hamano 5554738038 Merge branch 'ps/remove-packfile-store-get-packs'
Two slightly different ways to get at "all the packfiles" in API
has been cleaned up.

* ps/remove-packfile-store-get-packs:
  packfile: rename `packfile_store_get_all_packs()`
  packfile: introduce macro to iterate through packs
  packfile: drop `packfile_store_get_packs()`
  builtin/grep: simplify how we preload packs
  builtin/gc: convert to use `packfile_store_get_all_packs()`
  object-name: convert to use `packfile_store_get_all_packs()`
2025-10-30 08:00:19 -07:00
Junio C Hamano c43d4cf762 Merge branch 'ob/gpg-interface-cleanup'
strbuf_split*() to split a string into multiple strbufs is often a
wrong API to use.  A few uses of it have been removed by
simplifying the code.

* ob/gpg-interface-cleanup:
  gpg-interface: do not use misdesigned strbuf_split*()
  gpg-interface: do not use misdesigned strbuf_split*()
2025-10-30 08:00:19 -07:00
Junio C Hamano 48d0b6545a Merge branch 'ps/symlink-symref-deprecation'
"Symlink symref" has been added to the list of things that will
disappear at Git 3.0 boundary.

* ps/symlink-symref-deprecation:
  refs/files: deprecate writing symrefs as symbolic links
2025-10-30 08:00:19 -07:00
Junio C Hamano 923436e23d Merge branch 'ey/commit-graph-changed-paths-config'
A new configuration variable commitGraph.changedPaths allows to
turn "--changed-paths" on by default for "git commit-graph".

* ey/commit-graph-changed-paths-config:
  commit-graph: add new config for changed-paths & recommend it in scalar
2025-10-30 08:00:19 -07:00
Junio C Hamano dc70283dfc The 26th batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-29 12:40:39 -07:00
Junio C Hamano c1b23bd8aa Merge branch 'tb/incremental-midx-part-3.1'
Clean-up "git repack" machinery to prepare for incremental update
of midx files.

* tb/incremental-midx-part-3.1: (49 commits)
  builtin/repack.c: clean up unused `#include`s
  repack: move `write_cruft_pack()` out of the builtin
  repack: move `write_filtered_pack()` out of the builtin
  repack: move `pack_kept_objects` to `struct pack_objects_args`
  repack: move `finish_pack_objects_cmd()` out of the builtin
  builtin/repack.c: pass `write_pack_opts` to `finish_pack_objects_cmd()`
  repack: extract `write_pack_opts_is_local()`
  repack: move `find_pack_prefix()` out of the builtin
  builtin/repack.c: use `write_pack_opts` within `write_cruft_pack()`
  builtin/repack.c: introduce `struct write_pack_opts`
  repack: 'write_midx_included_packs' API from the builtin
  builtin/repack.c: inline packs within `write_midx_included_packs()`
  builtin/repack.c: pass `repack_write_midx_opts` to `midx_included_packs`
  builtin/repack.c: inline `remove_redundant_bitmaps()`
  builtin/repack.c: reorder `remove_redundant_bitmaps()`
  repack: keep track of MIDX pack names using existing_packs
  builtin/repack.c: use a string_list for 'midx_pack_names'
  builtin/repack.c: extract opts struct for 'write_midx_included_packs()'
  builtin/repack.c: remove ref snapshotting from builtin
  repack: remove pack_geometry API from the builtin
  ...
2025-10-29 12:38:24 -07:00
Junio C Hamano 57da342c78 The 25th batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-28 10:29:09 -07:00
Junio C Hamano fe95c55549 Merge branch 'ps/ci-rust'
CI improvements to handle the recent Rust integration better.

* ps/ci-rust:
  rust: support for Windows
  ci: verify minimum supported Rust version
  ci: check for common Rust mistakes via Clippy
  rust/varint: add safety comments
  ci: check formatting of our Rust code
  ci: deduplicate calls to `apt-get update`
2025-10-28 10:29:09 -07:00
Junio C Hamano 3deb97fe24 Merge branch 'cc/fast-import-strip-signed-tags'
"git fast-import" is taught to handle signed tags, just like it
recently learned to handle signed commits, in different ways.

* cc/fast-import-strip-signed-tags:
  fast-import: add '--signed-tags=<mode>' option
  fast-export: handle all kinds of tag signatures
  t9350: properly count annotated tags
  lib-gpg: allow tests with GPGSM or GPGSSH prereq first
  doc: git-tag: stop focusing on GPG signed tags
2025-10-28 10:29:09 -07:00
Junio C Hamano 54ac3809c3 Merge branch 'ds/sparse-checkout-clean'
"git sparse-checkout" subcommand learned a new "clean" action to
prune otherwise unused working-tree files that are outside the
areas of interest.

* ds/sparse-checkout-clean:
  sparse-index: improve advice message instructions
  t: expand tests around sparse merges and clean
  sparse-index: point users to new 'clean' action
  sparse-checkout: add --verbose option to 'clean'
  dir: add generic "walk all files" helper
  sparse-checkout: match some 'clean' behavior
  sparse-checkout: add basics of 'clean' command
  sparse-checkout: remove use of the_repository
2025-10-28 10:29:09 -07:00
Junio C Hamano 419c72cb8a Git 2.51.2
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4fA2sf7nIh/HeOzvsLXohpav5ssFAmj+4TIACgkQsLXohpav
 5svscw/+M/bCf9KO33uDLcWETGlM22OSa42Sxh7L+BWzI9YS3GVYOp0qRu4QFet+
 hB3XuNYtmeuVqXUw3OrvlK56oXPD9UocUgNLSCIxleuaY4RahHhCcsJxlcoJRWkb
 t6xv+2npAAdEBmjWa4OI62mK8O2wvs+m/aTA4Jtdx1Dg5XpZPRYwe8LY/e22hVt3
 KpxQ0QzLCjc/RJKHxslEpQYUoTOAsyVrS64n4hK4bQqrn4iyMBFuS7yWavTwu3V3
 Rl571hm71A2nxwbV2nVNnV1gyypL5z6F+19W9g1alBcefmGR5RnXmFO2twd2ufWK
 /oAEXx+zDbCaplz9BrvX+HdDv+YcBEMpYi1GQQEeEgYQ0t7ADPI7wnW/1UU8kEYl
 0AHgsaCeWbHPYdPWpIFGjXIxuoTmHL115Gj3IQPOakghfqVjnTor0DAZl8LpG8Co
 Lqr+G2+TP01/aEhxFy9jFfoibSftIvQNQbSbQ7wsjn0XfE2B/MqeZHOXsHDtxQX0
 YjSPfPAZWZNaC5lQ20V0B0FMiWJSNqIsTdXZmlJDT6n9E8oCWkI/Q4oHlzoZCOMH
 rZPAe67EP9MtIxgwmfntINVK0LryjxE72UyeR0bk6dmdJ+a15BnvjXl/Ad5MocVw
 Jht7gybYNKXfPMu08wBJ2VRFLbayDjteN2N5rarx9YeWPrvWY2o=
 =N5et
 -----END PGP SIGNATURE-----

Sync with Git 2.51.2

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-26 20:09:24 -07:00
Junio C Hamano bb5c624209 Git 2.51.2
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-26 19:48:21 -07:00
Junio C Hamano b42b995d22 Merge branch 'so/t2401-use-test-path-helpers' into maint-2.51
Test modernization.

* so/t2401-use-test-path-helpers:
  t2401: update path checks using test_path helpers
2025-10-26 19:48:21 -07:00
Junio C Hamano 476b2407be Merge branch 'js/ci-github-actions-update' into maint-2.51
CI update.

* js/ci-github-actions-update:
  build(deps): bump actions/github-script from 7 to 8
  build(deps): bump actions/setup-python from 5 to 6
  build(deps): bump actions/checkout from 4 to 5
  build(deps): bump actions/download-artifact from 4 to 5
2025-10-26 19:48:20 -07:00
Junio C Hamano 3b9055c369 Merge branch 'kh/doc-continued-paragraph-fix' into maint-2.51
Doc mark-up fixes.

* kh/doc-continued-paragraph-fix:
  doc: fix accidental literal blocks
2025-10-26 19:48:20 -07:00
Junio C Hamano 4b67e53fd6 Merge branch 'js/unreachable-workaround-for-no-symlink-head' into maint-2.51
Code clean-up.

* js/unreachable-workaround-for-no-symlink-head:
  refs: forbid clang to complain about unreachable code
2025-10-26 19:48:20 -07:00
Junio C Hamano ed931ebe18 Merge branch 'ps/t7528-ssh-agent-uds-workaround' into maint-2.51
Recent OpenSSH creates the Unix domain socket to communicate with
ssh-agent under $HOME instead of /tmp, which causes our test to
fail doe to overly long pathname in our test environment, which has
been worked around by using "ssh-agent -T".

* ps/t7528-ssh-agent-uds-workaround:
  t7528: work around ETOOMANY in OpenSSH 10.1 and newer
2025-10-26 19:48:20 -07:00
Junio C Hamano 2ad0fc2add Merge branch 'tb/unicode-width-table-17' into maint-2.51
Unicode width table update.

* tb/unicode-width-table-17:
  unicode: update the width tables to Unicode 17
2025-10-26 19:48:19 -07:00
Junio C Hamano 3d638cb389 Merge branch 'jk/status-z-short-fix' into maint-2.51
The "--short" option of "git status" that meant output for humans
and "-z" option to show NUL delimited output format did not mix
well, and colored some but not all things.  The command has been
updated to color all elements consistently in such a case.

* jk/status-z-short-fix:
  status: make coloring of "-z --short" consistent
2025-10-26 19:48:19 -07:00
Junio C Hamano 2319fbae48 Merge branch 'jk/diff-no-index-with-pathspec-fix' into maint-2.51
An earlier addition to "git diff --no-index A B" to limit the
output with pathspec after the two directories misbehaved when
these directories were given with a trailing slash, which has been
corrected.

* jk/diff-no-index-with-pathspec-fix:
  diff --no-index: fix logic for paths ending in '/'
2025-10-26 19:48:19 -07:00