The global variable 'fetch_if_missing' has been moved to a member in
'struct repository', continuing the libification process and
allowing per-repository control (such as for submodules).
* ty/repository-fetch-if-missing:
repository: move fetch_if_missing into struct repository
The pack-objects command has been updated to record the total bytes
written to pack files in trace2 output, allowing performance
analysis of different compression settings by comparing the
resulting pack sizes.
* fr/pack-objects-trace-pack-bytes:
pack-objects: trace pack bytes written
The mechanism to generate a packfile corresponding to the result of
a fetch/push has been made pluggable through a set of object
database callback functions, removing hardcoded references to
'pack-objects' and enabling alternative ODBs to serve packfiles
themselves.
* ps/odb-pluggable-pack-generation:
bundle: generate packfiles via the object database
bundle: get (mostly) rid of `the_repository`
builtin/bundle: refactor option handling for progress meter
send-pack: generate packfiles via the object database
upload-pack: generate packfiles via the object database
odb: introduce interface to generate packfiles
The 'git receive-pack' command has been updated to use a new ODB
transaction interface for writing incoming packfiles, making it more
backend-agnostic.
* jt/receive-pack-pluggable-writes:
odb/transaction: add transaction interface to write packfiles
odb: return temporary ODB source when set
builtin/receive-pack: explicitly pass packfile fd
builtin/receive-pack: report unpack errors via strbuf
builtin/receive-pack: lift global state out of unpack()
builtin/receive-pack: read unpack limit config lazily
builtin/receive-pack: pass shallow file explicitly
odb/transaction: add transaction finalize interface
builtin/receive-pack: properly clean up keep files
The object database layer has been simplified by eagerly loading
alternate object directories upon initialization, instead of
deferring it to the first object lookup. This eliminates the need
for scattered lazy-loading calls throughout the codebase and paves
the way for integrating alternates with the pluggable backends.
* ps/odb-eagerly-load-alternates:
odb: drop `alternates_db` field
odb: drop `loaded_alternates` field
odb: eagerly initialize alternates
odb: decouple source path comparisons from `the_repository`
setup: create ref and object databases after config is written
The object database (odb) API has been refactored to distinguish
between missing objects and corrupt ones by returning more
descriptive error statuses. Both the packed and loose backends now
faithfully propagate error details using a generic strbuf error
mechanism, removing backend-specific leakage from central lookup
paths.
* ps/odb-generic-corrupt-objects:
odb: handle `OBJECT_INFO_DIE_IF_CORRUPT` generically
odb/source: allow `read_object_info()` to bubble up error messages
odb/source: let callers discern missing and corrupt objects
odb/source: introduce error status when reading objects
odb/source-packed: flag known-bad objects as corrupt and not missing
The DWIM logic in 'git worktree add' sometimes tried to infer a
remote-tracking branch when an explicit '-b' or '-B' option was
given to create a new branch, causing the explicit branch name to
be ignored, which has been corrected.
* yn/worktree-add-no-dwim-with-b:
worktree add: shouldn't dwim if -b or -B is given
git-bundle(1) spawns git-pack-objects(1) directly to generate the pack
data that gets appended to the bundle header. While bundles are not
part of the wire protocol, they are a transfer mechanism for packs all
the same, so convert them to use the pack generation interface of the
object database as well.
This makes the pack generator the single spawn point for all pack
streams that leave the repository, leaving only local maintenance tasks
like git-repack(1) with direct knowledge of git-pack-objects(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-bundle(1) command has a couple of command line options that
relate to whether or not progress should be reported. These options
match the options that git-pack-objects(1) expects, and consequently
they mostly get passed through to it directly.
This results in somewhat of a confusing interface: there are four
different options that relate to whether or not progress should be
displayed and how verbose it should be. But in reality, there's really
only two modes:
- "--progress" and "--all-progress" result in the same outcome, which
is also documented as such.
- "--all-progress-implied" does nothing as we pass that argument to
git-pack-objects(1) unconditionally anyway.
So in the end, the options only control whether or not progress should
be displayed at all, nothing else.
Refactor the interface to instead use a simple `progress` boolean. This
makes argument handling a lot more straight-forward and it prepares us
for the next commit, where we're migrating git-bundle(1) to the generic
interface for generating a packfile.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In git-receive-pack(1), the incoming packfile is written to the ODB via
`unpack()`, which spawns git-index-pack(1) or git-unpack-objects(1)
directly. With pluggable object databases, an alternative backend may
need to handle writing packfile data differently though.
Introduce `odb_transaction_write_pack()` as a generic interface to
handle writing a packfile to a transaction and use the logic from
`unpack()` as the "files" backend implementation. Note that when storing
the objects as a packfile, git-index-pack(1) also writes a ".keep"
lockfile next to it to prevent a concurrent repack from removing the new
pack prior to reference updates being performed. The "files" transaction
backend is responsible for managing these ".keep" files and removes them
post-commit once the transaction is finalized.
Call sites in git-receive-pack(1) are updated accordingly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When processing the incoming packfile in git-receive-pack(1), `unpack()`
assumes it should always read it from stdin. In preparation for
`unpack()` logic being moved behind a generic ODB transaction interface,
update the function signature to take the an explicit fd provided by
callers to read the incoming packfile from instead. Call sites are
updated accordingly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When writing packfiles via `unpack()`, error messages are returned
directly by the function. In preparation for `unpack()` logic being
moved behind a generic ODB transaction interface, update the function to
instead write any error messages to a caller provided strbuf and return
a negative value on error. Call sites are updated to use the error
strbuf accordingly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In git-receive-pack(1), writing the packfile to the transaction is
handled via `unpack()` which relies on global variables to decide how to
invoke the underlying git-index-pack(1) or git-unpack-objects(1) child
processes. In a subsequent commit, the `unpack()` logic is moved behind
a generic ODB transaction interface to handle writing packfiles and thus
can no longer rely on these globals.
Lift the global state out of `unpack()` by instead storing this state in
a `struct unpack_opts` that gets passed to the function explicitly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In git-receive-pack(1), the `receive.unpackLimit` and
`transfer.unpackLimit` configuration decides whether an incoming
packfile should be exploded into loose objects or kept as a packfile
on-disk. In a subsequent commit, the logic to write the incoming
packfile is made ODB backend agnostic and moved behind a pluggable ODB
transaction interface. Consequently, whether to explode a packfile is a
detail of how a particular backend stores objects and should not be a
part of the generic interface itself.
In preparation for this, instead resolve the unpack limit lazily inside
`unpack()` by reading the configuration directly. The now-unused unpack
limit globals are dropped accordingly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If shallow information is provided during `unpack()`, a temporary
shallow file is created and stored in global state. In a subsequent
commit, the `unpack()` logic is moved behind a generic ODB transaction
interface to handle writing packfiles and thus can no longer rely on
such global state. Lift the setup of the temporary shallow file out of
`unpack()` and wire it through to its call sites explicitly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When committing an ODB transaction via `odb_transaction_commit()`, the
staged objects are made visible and the underlying transaction is freed
at the same time. Coupling these two steps does not leave room for any
post-commit transaction operations to be introduced though. Such a
capability is useful if an ODB transaction backend needs to hold on to
lockfiles after transaction commit until references are updated, as is
the case with the existing "files" backend in git-receive-pack(1).
Stop freeing the transaction in `odb_transaction_commit()` and introduce
`odb_transaction_finalize()` to explicitly clean up the transaction
accordingly. Note that the finalize interface also provides an optional
callback for any backend-specific deferred cleanup. In a subsequent
commit, the "files" transaction backend will use this to remove ".keep"
files generated for packfiles received via git-receive-pack(1) after
references have been updated. In preparation for this, the
`odb_transaction_finalize()` call site in git-receive-pack(1) is made
after the reference updates are finished.
All other callers commit a transaction and immediately finalize it
without any work happening in between those two operations.
Consequently, they cannot meaningfully recover in case either of them
would fail, and spelling out these two separate steps with proper error
handling would be quite repetitive and pointless. Introduce a helper
`odb_transaction_commit_and_finalize_or_die()` for those call sites and
update them accordingly.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When git-receive-pack(1) stores an incoming packfile with
git-index-pack(1), a ".keep" file is written alongside it in the
transaction quarantine directory and also gets migrated to the main ODB
when the ODB transaction is committed. This keep lockfile ensures the
packfile remains in place until the references have been updated and is
removed afterwards. The path used to remove it is derived via
`index_pack_lockfile()` from the repository's primary object directory.
In bdee7b3013 (builtin/receive-pack: stage incoming objects via ODB
transactions, 2026-07-10), git-receive-pack(1) started using the ODB
transaction interfaces instead of managing a temporary directory
directly. When starting an ODB transaction, the sources list is
reordered to insert the newly created transaction source first as the
primary to ensure writes are routed to it accordingly.
Prior to using ODB transactions, git-receive-pack(1) would only set the
temporary directory as the primary source for the child
git-index-pack(1) and git-unpack-objects(1) processes it spawned and the
parent process would set the temporary directory set as an alternate
only. By using ODB transactions, the ODB source list is also reordered
for the parent process which results in `index_pack_lockfile()` deriving
the ".keep" path relative to the temporary directory instead of the
actual main ODB source path. Consequently, this prevents the ".keep"
file from being properly removed after being migrated into the main ODB
source post-commit.
Update `index_pack_lockfile()` to operate on an ODB source explicitly
provided to it and update call sites accordingly to pass the expected
ODB source.
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git worktree add <path> <branch>' DWIMs <branch> to a
remote-tracking branch when neither -b, -B, nor --detach
is given.
However, 'git worktree add -b <new-branch> <path> <branch>' can
still DWIM <branch>, causing <new-branch> to be ignored. This is a
regression introduced by 128e5496b3 (worktree add: extend DWIM to
infer --orphan, 2023-05-17), which appeared in Git 2.42.
Signed-off-by: Yoichi NAKAYAMA <yoichi.nakayama@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We want to measure how compression settings affect push performance on
the client. Different settings can produce different-sized packs from
the same objects. Trace2 records the object count, but we also need the
pack size to compare those settings.
Add a write_pack_file/wrote_bytes Trace2 datum alongside
write_pack_file/wrote. Count packs written to stdout or disk, including
each pack's header and trailing checksum. When pack.packSizeLimit splits
the output, report the sum of the pack sizes.
Signed-off-by: Friel <friel@openai.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'git branch' command has been taught the '--delete-merged' option
to remove local branches that are already merged into their tracked
remote-tracking branches.
* hn/branch-delete-merged:
branch: add --dry-run for --delete-merged
branch: add branch.<name>.deleteMerged opt-out
branch: add --delete-merged <pattern>
branch: prepare delete_branches for a bulk caller
branch: let delete_branches skip unmerged branches on bulk refusal
branch: convert delete_branches() to a flags argument
branch: add --forked filter for --list mode
When reading an object fails even though it exists, the sources know
best what exactly went wrong and where the corrupt object is located.
This information is lost though when bubbling up the error to the object
database layer, which forces that layer to reconstruct it after the
fact. This is exactly what `do_oid_object_info_extended()` does via
`has_packed_and_bad()`, but that function only really knows to handle
the "files" backend by reaching into its internals.
Introduce a new `errmsg` parameter for the `read_object_info()` callback
that sources are expected to populate with a human-readable message in
case reading the object has failed. Adapt the packed and loose sources
to populate the buffer with the messages that we ultimately want to
surface to the user.
For now, all callers are adapted to pass a `NULL` pointer. We will add a
user of this new infrastructure in a subsequent commit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When reading packed objects we know to tell apart missing objects and
corrupt objects by returning a positive error code in the former case,
and a negative one in the latter case. We do that by distinguishing
between errors returned by `find_pack_entry()`, which yields the offset
of the object, and `packed_object_info()`, which reads the object
contents.
But even though we already distinguish those cases when reading packed
objects, the logic is broken in case a caller tries to read an object
that has been marked as corrupt. In that case, `find_pack_entry()` will
tell us that the object in question does not exist, and consequently
we'll not flag the object as corrupt but as missing.
Fix this issue by bubbling up whether the object is corrupt and, if so,
which packfile contains the corrupted object.
Note that we don't yet need the information about the specific packfile,
so we could've just as well made this a `bool *corrupted` pointer. But
we'll need information about the containing packfile in a subsequent
commit so that we can generate a proper error message telling the user
which packfile contains the broken object.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git repack' has been taught '--drop-filtered' to delete local
promisor blobs exceeding a limit (currently 'blob:limit=') in partial
clones, reclaiming space. Guards prevent running during other
operations or if referenced by the index.
* ss/repack-drop-filtered:
builtin/repack: add guards for --drop-filtered
builtin/repack: actually drop filtered promisor blobs
builtin/repack: enumerate promisor blobs for --drop-filtered
repack-promisor: allow excluding objects from the rebuilt promisor pack
list-objects-filter: add list_objects_filter__filter_oidset()
builtin/repack: add --drop-filtered and --dry-run options
The 'git bisect' command has been taught a
'--reset-when-found[=<where>]' option that tells the command to
automatically run 'git bisect reset' to jump back to the original
state or to the found culprit.
* hn/bisect-reset-when-found:
bisect: add --reset-when-found to leave when done
bisect: let bisect_reset() optionally check out quietly
A compatibility wrapper for writev(3p) has been reintroduced,
including fixes for CMake build and 'MAX_IO_SIZE' limits on NonStop.
Calls to write(3p) in send_sideband() and cat_blob() have been
refactored to use writev(3p) wrappers to reduce syscall overhead.
* ps/writev:
fast-import: use writev(3p) to send cat-blob responses
sideband: use writev(3p) to send pktlines
wrapper: properly handle MAX_IO_SIZE in writev(3p)
wrapper: introduce writev(3p) wrappers
compat/posix: introduce writev(3p) wrapper
The 'pack-objects' and delta-encoding code paths have been updated to
use 'size_t' instead of 'unsigned long' for object sizes and offset
limits, avoiding potential truncation issues on 64-bit Windows.
* js/pack-objects-delta-size-t:
packfile: widen `unpack_object_header_buffer()` to `size_t`
git-zlib: widen `git_deflate_bound()` to `size_t`
t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`
http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
diff: widen `deflate_it()`'s bound local from int to `size_t`
archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
delta: widen `create_delta()` and `diff_delta()` to `size_t`
pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`
pack-objects: widen `free_unpacked()` return to `size_t`
pack-objects: widen delta-cache accounting to `size_t`
delta: widen `create_delta_index()` parameter to `size_t`
diff-delta: widen `struct delta_index`' size fields to `size_t`
A handful of code paths have been corrected to check return values
from functions like curl_easy_duphandle(), deflateInit(), lseek(),
dup(), and strbuf_getline_lf(), resolving several Coverity warnings
about unchecked returns.
* js/coverity-unchecked-returns-fix:
bisect: handle dup() failure when redirecting stdout
bisect: check get_terms return at all call sites
bisect: check strbuf_getline_lf return when reading terms
transport-helper: warn when export-marks file cannot be finalized
transport-helper: check dup() return in get_exporter
compat/pread: check initial lseek for errors
last-modified: handle repo_parse_commit() failures
reftable tests: check reftable_table_init_ref_iterator() return
reftable/block: check deflateInit() return value
reftable: handle block-writer initialization errors
config: propagate launch_editor() failure in show_editor()
http: die on curl_easy_duphandle failure in get_active_slot
When creating the object database we initialize the main object database
source, but we don't yet initialize its alternates. Instead, we have
many calls to `odb_prepare_alternates()` cluttered around the code base
whenever we are about to iterate through the sources.
This lazy loading doesn't really add much value: the moment where we
read any object we _have_ to load the alternates anyway. So given that
most of our commands would access the object database this optimization
is not really buying us much in the first place. Quite on the contrary,
it makes the code harder to understand and is a potential source of bugs
in case any callsite forgot to prepare alternates before we iterate
through the sources.
Historically though there was a reason why we deferred lazy-loading: it
may happen that the repository has "core.ignoreCase" configured, and we
use that to deduplicate the list of alternates in case we had the same
alternate configured multiple times, but with different casing. We used
to initialize the object database before we had fully configured the
owning repository though, and consequently we couldn't access that
configuration yet. This has changed in the preceding commit though where
we started to parse "core.ignoreCase" manually.
Eagerly prepare alternates both when creating the object database and
when flushing its caches. Drop the now-unneeded calls to prepare the
alternates that are scattered across the code base.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'struct odb_read_stream' and 'struct odb_write_stream'
structures have been consolidated into a single unified 'struct
odb_stream' structure, simplifying object database streaming APIs
and enabling streaming of arbitrary object types.
* ps/odb-streams:
odb/streaming: unify function names to create new streams
odb/streaming: rename `struct input_zstream_data`
odb/streaming: rename `struct read_object_fd_data`
odb/streaming: consolidate read and write streams
odb/streaming: rename `struct odb_read_stream`
odb/streaming: support streaming arbitrary object types
odb/streaming: drop `is_finished` field
odb/streaming: track write stream size in the structure
The usage string of 'git fast-import' has been updated to use the
parse_options() API for displaying help, and its SYNOPSIS in the
documentation has been standardized to match.
* cc/fast-import-usage:
fast-import: remove useless from_stream argument
fast-import: use parse_options() for command line options
fast-import: use callbacks to parse some options
fast-import: use struct option for usage string
fast-import: move command state globals into 'struct fast_import_state'
fast-import: introduce 'struct fast_import_state'
fast-import: factor out option_*() functions
fast-import: use int for some bool flags
fast-import: localize 'i' into the 'for' loops using it
api-parse-options.adoc: document hidden and OPT_*_F option macros
api-parse-options.adoc: document per-option flags
parse-options: introduce OPT_HIDDEN_GROUP
The 'remote-object-info' command for 'git cat-file --batch-command'
has been extended to support the '%(objecttype)' placeholder.
* ps/cat-file-remote-object-info-type:
cat-file: unify default format
serve: advertise type capability
fetch-object-info: parse type from server response
protocol-caps: add type support to object-info
transport: drop remote object-info fields from transport struct
fetch-object-info: die() on the remaining error path
fetch-object-info: use dedicated struct for the results
fetch-object-info: pass arguments directly instead of a struct
fetch-object-info: detect malformed server responses
t5701: use test_file_size() to get the size of a file
'git add' has been taught a new '--resolved' option to stage
conflict-resolved paths, while leaving unrelated local changes
unstaged. It scans the unmerged paths for leftover conflict
markers and aborts if any are found.
* jc/add-resolved:
add: introduce '--resolved' option
read-cache: add remove_file_from_index_with_flags()
merge-ll: consolidate conflict marker scanning logic
read-cache: reindent
The global variable 'fetch_if_missing' controls whether a missing
object check should prompt a lazy fetch from a promisor remote.
In order to continue the libification effort, move it into
'struct repository' and initialize it to 1 by default to keep the
previous behavior.
builtin/fetch-pack.c, builtin/fsck.c, and builtin/rev-list.c are
entered via commands marked RUN_SETUP in git.c:commands[]. Their
'repo' parameter is only NULL when '-h' is given outside of a
repository, in which case either show_usage_if_asked() or
parse_options()'s own '-h' handling exits the process before
returning. We can therefore drop their UNUSED markers and assign
to 'repo' directly.
builtin/index-pack.c is entered via RUN_SETUP_GENTLY, so its
'repo' pointer can be NULL any time it is run outside of a
repository, not only with '-h'. We keep a NULL check there and fall
back to 'the_repository'.
builtin/pack-objects.c needs two adjustments to make 'repo' reach
every 'fetch_if_missing' call site: 'read_stdin_packs()' now takes a
'struct repository *'; 'option_parse_missing_action()', which
is registered as an OPT_CALLBACK, receives a 'repo' through the
option's 'value' field now.
Additionally, update the partial clone documentation to reflect
that this is now a per-repository flag.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com>
Mentored-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
--drop-filtered removes local promisor blobs. That is only safe when the
repository is not mid-operation and when the blobs are not actively in
use, so add two guards, both skipped for bare repositories which have
neither a worktree nor an index.
First, refuse to run while a merge, rebase, am, cherry-pick, revert, or
bisect is in progress. During these operations the working tree and
index are in an intermediate state, and rewriting packs and deleting
objects underneath a half-finished operation is unsafe.
Second, refuse to drop a blob that the current index references. Such a
blob is needed by the working tree, so dropping it would only cause the
next command that touches the worktree to lazy-fetch it straight back,
reclaiming nothing. The offending path is reported so the user can see
why the drop was refused.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Siddharth Asthana <siddharthasthana31@gmail.com>
Signed-off-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make --drop-filtered remove the enumerated promisor blobs instead of
only listing them.
The drop set is computed before repack_promisor_objects() runs, and on
a real run it is passed in so the rebuilt promisor pack omits those
blobs. --drop-filtered implies -d so the old promisor packs, which
still contain the dropped blobs, are removed. Without this the blobs
would survive in the redundant packs. The existing repack machinery
performs the write-before-delete and fsync, so the drop is crash-safe.
The dropped blobs become absent locally but remain recoverable from the
promisor remote, so a later access lazy-fetches them back
transparently. --dry-run keeps its previous behavior, i.e. it lists the
candidates and changes nothing.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Siddharth Asthana <siddharthasthana31@gmail.com>
Signed-off-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add enumeration logic for --drop-filtered. In --dry-run mode, print
the OIDs of locally-held promisor blobs that exceed the filter
threshold, as candidates for removal.
Reading from write_filtered_pack() cannot work for partial clones.
git repack routes promisor objects through a separate path:
repack_promisor_objects() repacks them first, and the main
pack-objects run uses --exclude-promisor-objects. By the time
write_filtered_pack() runs, the promisor blobs are already consumed by
the main pack. The filtered pack is always empty on a partial clone.
Instead, walk promisor objects directly via odb_for_each_object() with
ODB_FOR_EACH_OBJECT_PROMISOR_ONLY, collecting all promisor blobs into
an oidset. The blobs exceeding the filter threshold are then selected
using list_objects_filter__filter_oidset().
Every object enumerated this way is a promisor object, so it is
recoverable from the promisor remote in the same sense as the rest of a
partial clone, as long as the remote still has it. This holds without a
separate is_promisor_object() check. A future implementation can verify
availability against the remote directly once a client-side
remote-object-info query exists.
OBJECT_INFO_SKIP_FETCH_OBJECT is passed to every object info query so
enumeration never triggers a lazy fetch.
The enumeration collects candidates into a caller-provided oidset and
--dry-run prints them. Actually removing the objects, together with the
required promisor-remote verification, is written in a later commit.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Siddharth Asthana <siddharthasthana31@gmail.com>
Signed-off-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a to_drop oidset parameter to repack_promisor_objects(). When it is
non-NULL, write_oid() omits those objects from the rebuilt promisor
pack. This is the mechanism --drop-filtered will use to remove promisor
blobs, i.e. rebuild the promisor pack without them.
All existing callers pass NULL, so behavior is unchanged.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Siddharth Asthana <siddharthasthana31@gmail.com>
Signed-off-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add two new command-line options to 'git-repack':
--drop-filtered: intended to eventually delete objects that match
the filter specification. Requires --filter and -a,
and is incompatible with --filter-to.
--dry-run: show which objects would be dropped without making any
changes. Only meaningful with --drop-filtered.
Keep --dry-run as a separate option rather than folding it into
--drop-filtered (e.g. --drop-filtered=dry-run), to stay consistent with
the --dry-run option other Git commands already provide and to leave
room for it to describe other repack behavior later. A
--drop-filtered=<mode> form can still be added later if more
drop-specific modes are needed.
--drop-filtered also requires a promisor remote to be configured, since
dropping objects without a remote to fetch them back from would be
permanent data loss.
--drop-filtered is incompatible with bitmap writing: filtering breaks
the "all objects in one pack" closure that bitmaps require. Detect an
explicit -b/--write-bitmap-index on the command line with a dedicated
option callback that sets a "write_bitmaps_given" flag, so it can be
distinguished from a repack.writeBitmaps configuration value even when
config already enables bitmaps. An explicit -b is reported as a conflict,
while a config-provided default is silently disabled for the duration
of the command.
These options currently only perform validation. The actual enumeration
and deletion will be added in follow-up commits.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Siddharth Asthana <siddharthasthana31@gmail.com>
Signed-off-by: Siddharth Shrimali <r.siddharth.shrimali@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'git branch' command has been taught the '--delete-merged' option
to remove local branches that are already merged into their tracked
remote-tracking branches.
* hn/branch-delete-merged:
branch: add --dry-run for --delete-merged
branch: add branch.<name>.deleteMerged opt-out
branch: add --delete-merged <pattern>
branch: prepare delete_branches for a bulk caller
branch: let delete_branches skip unmerged branches on bulk refusal
branch: convert delete_branches() to a flags argument
branch: add --forked filter for --list mode
As part of the ongoing effort to replace `unsigned long` data types with
`size_t` wherever appropriate (mainly to fix all those problems on
Windows with objects larger than 4GB), let's also adjust the return type
and the type of the `len` parameter of this function.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Bundling the two widenings: four call sites pass `&stream.avail_in`
directly to `use_pack()`, and widening either type fencepost alone would
force a bridge variable at each. Doing both together is the simpler end
state and is the prerequisite for the `do_compress()` widening in the
next commit, which is what lets `write_no_reuse_object()` lose its last
`cast_size_t_to_ulong()` shim.
The unsigned-long locals widened at the other `use_pack()` callers
(avail / remaining / left) hold pack-window sizes bounded by
`core.packedGitWindowSize`, so the change is type consistency rather
than a new >4GB capability. `git_zstream.avail_in`/`avail_out` likewise
reach zlib's `uInt` fields only after `zlib_buf_cap()`'s 1 GiB cap, so
the wrapper already accepted `size_t`-shaped inputs in practice.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Last stop in the delta-encoding API widening for >4 GiB blobs on
Windows: with `create_delta_index()` done in the prior commit and
`create_delta()`/`diff_delta()` finished here, every byte count that
crosses delta.h is now `size_t`. The struct fields they store into have
been `size_t` since the diff-delta struct widening.
The API change must move with all callers in the same commit (the build
only passes when every `&delta_size` matches the new `size_t*`). Caller
updates are kept minimal:
* builtin/pack-objects.c `get_delta()` and `try_delta()`: widen only
the local `delta_size` variable; the surrounding unsigned-long
locals and their `cast_size_t_to_ulong()` shims are out of scope
here and will be cleaned up in their own commits.
* builtin/fast-import.c, diff.c, t/helper/test-pack-deltas.c:
keep the local unsigned-long delta size (each feeds a still-
unsigned-long downstream consumer: zlib's `avail_in`,
`deflate_it()`, the test helper's own `do_compress()`), and bridge
via a temporary `size_t` plus `cast_size_t_to_ulong()`. The new
casts are paid back in later topics that widen those consumers.
* t/helper/test-delta.c: widen the local outright (no downstream
consumer beyond the test's own `out_size`, which is already
`size_t`).
Note that GCC struggles a bit to figure out that `deltalen` is always
initialized before it is used; To help it along, we initialize it to 0.
This work-around will go away in a later patch series when `deltalen`
can be widened to `size_t`.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The pair must move together because `find_deltas()` passes `&mem_usage`
to `try_delta()`: widening either alone breaks the type match.
`mem_usage` accumulates per-object byte counts already computed in
`size_t` (`SIZE()` and `sizeof_delta_index()` reach here through
`free_unpacked()`, now `size_t`), and was the last 32-bit-on-Windows
narrowing point in the delta-window memory accounting chain. With this
commit, that chain uses `size_t` consistently except for
`sizeof_delta_index()`'s still-narrow return, whose value is bounded by
`create_delta_index()`'s entries cap.
`window_memory_limit` (config-driven via `git_config_ulong()`) stays
`unsigned long`: it is only compared against `mem_usage` and promotes.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`free_unpacked()` sums two byte counts: `sizeof_delta_index()` and
`SIZE(n->entry)`. The latter has been `size_t` since the prior topic
"More work supporting objects larger than 4GB on Windows" widened
`SIZE()`/`oe_size()` to `size_t`, so accumulating it into an `unsigned
long` return was a silent Windows-only truncation on a packing run with
many large objects.
The sole caller, `find_deltas()`, still holds its own `mem_usage` in an
`unsigned long` for now, and therefore still truncates silently.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These three are a single accounting tuple (the globals tracking
cumulative cached-delta bytes, plus the helper that compares them
against an incoming delta size) and are latently 32-bit on Windows where
`unsigned long` != `size_t`: a pack with many large cached deltas could
wrap silently.
The widening is internally consistent on its own: the additions and
subtractions against delta_cache_size already come from `size_t` sources
(`DELTA_SIZE()` returns `size_t`), and `delta_cacheable()`'s sole caller
in `try_delta()` still passes `unsigned long`, which promotes.
Prerequisite for dropping `try_delta()`'s `cast_size_t_to_ulong()`
shims, which becomes possible once 1create_delta()` and `diff_delta()`
are widened in a later commit.
Note: since `max_delta_cache_size` changes data type to `size_t`, a pair
of new helpers is introduced to parse config values of that type, too.
Assisted-by: Opus 4.7
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To capture the output of each verdict command, bisect_run()
temporarily redirects stdout to a temporary file via the classic
dup(1) / dup2() pair, restoring it afterwards. The return value of
dup(1) is not checked, however. When it fails, the saved descriptor
is -1, which is then passed to close() (the issue Coverity flags),
and the matching dup2() that is meant to restore stdout also fails,
leaving the process with stdout still pointing at the temporary file
for the remainder of the run.
Treat a failed dup(1) or dup2(..., 1) as a fatal error for this bisect
step: close the temporary file descriptor, report the error via
error_errno(), and break out of the loop so the existing cleanup path
handles the rest, just as on other failure paths in this function.
Reported by Coverity as CID 1508242 ("Improper use of negative
value").
Assisted-by: Opus 4.7
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Six callers of get_terms() silently discard its return value. When
get_terms fails (missing or truncated BISECT_TERMS file), the term
strings remain NULL or empty, causing confusing downstream
behavior: commands like "bisect next" or "bisect run" proceed with
empty term strings, producing nonsensical ref names (refs/bisect/
with no suffix) and misleading error messages.
Let's not discard the return value, but handle an error with the same
message `bisect_terms()` already uses when reading the terms failed.
Pointed out by Coverity.
There is one slight complication here: One caller _needs_ the return
value to indicate an error when the `BISECT_TERMS` file is absent, all
the other call sites are totally okay with a "missing" `BISECT_TERMS`
file. To address that, extend the function signature of `get_terms()` to
indicate which behavior the caller wants.
Assisted-by: Claude Opus 4.6
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
get_terms() in builtin/bisect.c and read_bisect_terms() in
bisect.c both read the BISECT_TERMS file but do not check the
strbuf_getline_lf() return values. If the file is truncated
(e.g., a partial write from a crash or disk-full condition),
strbuf_getline_lf returns EOF and the strbuf remains empty.
strbuf_detach then returns an empty string, and the term names
silently become "" instead of the expected "bad"/"good" or
custom terms.
In get_terms(), check for EOF and return -1 on truncation,
matching the existing -1 return for a missing file.
In read_bisect_terms(), die with a descriptive message when a
line cannot be read, consistent with the die_errno for a
non-ENOENT open failure in the same function. Unlike get_terms(),
read_bisect_terms() returns void and uses die() for all error
paths, so the die is the appropriate error handling here.
Pointed out by Coverity.
Assisted-by: Claude Opus 4.6
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
last_modified_run() and process_parent() call repo_parse_commit()
without checking the return value at three sites. When a commit
object is corrupt or unavailable (e.g., a shallow clone boundary
or a missing object in a partial clone), the parse fails and the
commit's internal fields (parents, tree, date) are not populated.
The consequences depend on which call site fails:
At line 417 (the main walk loop), c->parents stays NULL after a
failed parse. The parent-walking loop at line 440 simply does not
execute, silently treating the unparsable commit as a root commit.
This produces incorrect "last modified" results: paths changed in
ancestors beyond the corrupt commit are attributed to the wrong
commit or not reported at all.
At line 423 (the --not exclusion walk), n->parents stays NULL,
causing the exclusion walk to stop prematurely. Commits that
should be excluded from the output may be incorrectly included.
At line 293 (process_parent), the parent's tree and parents are
unavailable, so diff operations against it produce wrong results
and the parent's own ancestors are never enqueued for walking.
Skip unparsable commits by checking the return value and
continuing to the next iteration (or returning early in
process_parent). This matches the defensive pattern used in other
revision walkers such as limit_list() and get_revision_internal().
Pointed out by Coverity.
Assisted-by: Claude Opus 4.6
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
show_editor() calls launch_editor() to open the user's editor on
the configuration file, but discards the return value and
unconditionally returns 0 (success). When the editor fails to
launch (e.g., $EDITOR is not found, or the editor exits with a
nonzero status), the caller receives no indication that anything
went wrong.
This affects "git config edit" and "git config --edit": the
command silently succeeds even when the editor could not be
started. In contrast, other editor-launching paths in git (such
as "git commit" and "git rebase --edit-todo") properly propagate
editor failures and exit with an error.
Check the return value and propagate the failure by returning -1.
The two callers (cmd_config_edit at line 1315 and the legacy
cmd_config at line 1478) both propagate this return to
handle_builtin, which translates negative returns into an error
exit.
Pointed out by Coverity.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano