The mechanism to register in-memory alternate object sources has
been removed, as submodule object databases are now accessed
natively via their own repository structures. This simplifies
object database management and prepares the codebase for migrating
alternate tracking into the files backend.
* ps/odb-stop-registering-in-memory-sources:
odb: remove the ability to link sources ad-hoc
t/helper: stop registering alternates in "ref-store" command
t/helper: adapt read-midx to not link ad-hoc source anymore
builtin/multi-pack-index: refuse unknown sources with "--object-dir="
odb/packed: fix memory leaks when freeing source
tmp-objdir: drop unused function to register alternate
odb: remove infrastructure to register submodule sources
builtin/grep: stop registering submodule ODB as source
submodule-config: stop registering submodule sources
submodule-config: stop using `the_hash_algo`
submodule-config: remove uses of `the_repository`
cache-tree: remove dependency on `the_repository`
cache-tree: drop `the_repository` in `cache_tree_fully_valid()`
Teach 'am', 'revert', and 'rebase' that running 'commit --amend' or a
partial 'commit <paths>' makes no sense during operations that stop
and return control to the user to resolve conflicts left in the
working tree, just like 'cherry-pick' and 'merge' do.
* en/no-amend-during-conflicts:
commit: refuse partial commits during conflict resolution
commit: refuse to amend during conflict resolution
commit: reword the empty-commit rebase amend error
commit: allow a partial commit when a rebase pick becomes empty
commit: clarify FROM_REBASE_PICK and is_from_rebase() names
The consistency checks for the object database (fsck) have been
decoupled from the generic builtin implementation and moved into the
backend-specific object source layers, making them pluggable for
different object storage formats.
* ps/odb-pluggable-fsck:
builtin/fsck: move loose object verification into the loose source
builtin/fsck: move multi-pack index verification into the packed source
builtin/fsck: move bitmap verification into the packed source
builtin/fsck: move reverse index verification into the packed source
builtin/fsck: move packfile verification into the packed source
odb: provide infrastructure for pluggable fsck checks
builtin/fsck: don't check alternates with "--no-full"
builtin/fsck: de-globalize option handling
builtin/fsck: merge `fsck_obj_buffer()` and `fsck_obj()`
builtin/fsck: use `fsck_obj_buffer()` when checking loose objects
The setup of alternates has been deferred to object database
creation time during clone, which drops the unused ad-hoc alternate
writing API, simplifying the object database backend interface.
* ps/odb-alternates-at-creation:
odb/source: remove the ability to write alternates
builtin/clone: write alternates via `odb_create_on_disk()`
odb/source: support writing alternates when creating the database
builtin/clone: move setup of alternates for non-shared local clones
builtin/clone: move setup of alternates for shared local clones
builtin/clone: refactor handling of "--reference{,-if-able}"
builtin/clone: move around `setup_reference()`
builtin/clone: defer setup of the object database
The git config --global read operations have been updated to respect
both $HOME/.gitconfig and $XDG_CONFIG_HOME/git/config, fixing an
inconsistency where only the former was read when both configuration
files are present.
* dw/config-read-both-global:
config: read global scope via config_sequence
config: let sequence require a successful file
path: use forward slashes in XDG config on Windows
The --recurse-submodules option in 'git branch' has been fixed to
avoid a crash when the start point is not a reference (e.g., a raw
object ID). The creation path now skips setting up tracking and
properly forwards the absent tracking name to the submodule helper.
* vv/branch-recurse-no-start-ref:
branch: allow recursion with no tracking name
branch: do not track a start point with no ref
The codebase has been updated to use the newer utimensat() POSIX
function instead of the obsolescent utime(), allowing
high-precision timestamps while preserving fallback compatibility.
* as/utimensat-utimes:
compat/posix: drop legacy <utime.h> header and shims
treewide: use utimensat(2) instead of legacy utime(3p)
compat/posix: introduce utimensat(2) wrapper
The autostash fallback in 'git checkout -m' has been refined to only
retry when there are local changes. Additionally, a blank line now
visually separates autostash conflict advice from the subsequent
branch-switch message.
* hn/checkout-m-autostash-refine:
checkout: separate autostash conflict advice from branch-switch message
stash: reserve exit status 1 for conflicts
The object lookup machinery has been taught to gracefully recover
when a multi-pack-index points to an owning pack that was removed
during a concurrent geometric repack, and 'git replay' has been
fixed to not segfault when reading such missing objects.
* en/midx-missing-pack-fallback:
packfile: recover when a multi-pack-index names a removed pack
mktree: do not use OBJECT_INFO_QUICK when checking objects
mktree: plug per-tree leak in --batch mode
replay: fail gracefully when a merge input is unreadable
The build-time knob 'USE_NSEC' for nanosecond stat precision has been
converted to a runtime configuration 'core.useNanosec', allowing
distributions to bundle one binary that adapts to filesystem
capabilities dynamically.
* dk/use-nsec-runtime:
core: convert build-time USE_NSEC into runtime core.useNanosec
environment: align repo_config_values_init with struct declaration
meson: expose knob for xmlto relative links in manuals
The string extraction logic for the branch name and worktree name
from the given path in 'git worktree add' has been corrected and
simplified to avoid out-of-bounds reads and improper handling of
trailing slashes.
* rs/worktree-add-basename-fixes:
worktree add: let worktree_basename() return string copy
worktree add: trim slashes when deriving branch name from path
worktree add: reject separator-only path
worktree add: don't read out of bounds in worktree_basename()
The 'git replay' command has been taught the '--linearize' option to
drop merge commits and linearize the replayed history, mimicking 'git
rebase --no-rebase-merges'.
* tc/replay-linearize:
replay: offer an option to linearize the commit topology
replay: resolve the replay base outside pick_regular_commit()
replay: add helper to put entry into replayed_commits
The application of the edited patch in 'git add -e' has been
refactored to use the internal apply API directly, avoiding the need
to spawn a 'git apply' subprocess.
* gr/add-e-use-apply-api:
builtin/add.c: replace run_command() with direct apply_all_patches() call
'git worktree add' did not prevent DWIM behavior when '-b' or '-B' was
specified, which has been corrected.
* yn/worktree-ambiguous-remote-advice:
worktree add: treat multiple matches with --guess-remote as an error
worktree add: improve message for ambiguous remote branch name
checkout: improve message for ambiguous remote branch name
checkout: extract function to display advice for ambiguous remotes
"git checkout -m" stashes the user's local changes when it cannot
perform the checkout, and then applies the stash. When applying the
stash results in conflicts, the advice on how to deal with them is
printed directly on top of the branch-switch message ("Switched to
branch ..."), making the two hard to tell apart. Print a blank line
in between so that the advice and the branch-switch message are
visually distinct.
apply_autostash_ref() reports whether applying the stash resulted in
conflicts via its enum stash_apply_result return value, so only print
the blank line in the conflicted case.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git stash apply", "pop" and "branch" exit with status 1 both when
applying the stash entry resulted in conflicts and when they fail for
other reasons, so callers cannot tell the two apart.
Follow the convention of "git merge-tree" and the merge strategies,
which exit with status 1 to indicate conflicts and with a different
non-zero status for errors: those subcommands now exit with status 1
only when applying the stash entry resulted in conflicts, in which
case the stash entry is left in place, and exit with status 128, the
status die() uses, when they fail for other reasons. Document the
exit statuses.
cmd_stash() used to collapse the return values of the subcommand
implementations to a boolean. It now maps negative values, which
signal a failure, to 128 and passes everything else through as-is.
The only implementations that return a positive value are "apply",
"pop" and "branch", which return the value of do_apply_stash():
"apply" returns it directly, and "pop" and "branch" drop the stash
entry, via do_drop_stash(), which always returns 0, only when the
application succeeded. The positive value is always 1, as
do_apply_stash() only returns a positive value when the three-way
merge was unclean.
Make the convention explicit by introducing enum stash_apply_result
with the values STASH_APPLY_CLEAN, STASH_APPLY_CONFLICT and
STASH_APPLY_ERROR, and use it for the in-process autostash helpers,
too. They spawn "git stash apply" and can now tell conflicts apart
from other failures, e.g. a crash or death by signal of the child,
which map to exit statuses above 1. Since we know the stash entry
was saved, tell users so in the error message instead of leaving them
wondering what happened to their stashed changes.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can tell git-multi-pack-index(1) to access multi-pack indices that
are stored in a different object directory via the "--object-dir="
option. This allows them to for example write or verify a multi-pack
index other than the one located in the main object directory in case a
repository has alternates with multiple multi-pack indices.
But while the documentation explicitly points out that the specified
object directory must be an alternate of the current repository, we
never verify that property. Instead, starting with 017db7bb14 (midx:
load multi-pack indices via their source, 2025-08-11), we now construct
an ad-hoc source and link it to the main object directory.
Besides contradicting the documentation, it's dubious that this really
ought to work in the first place: creating a multi-pack index (and
potentially a bitmap) for a completely foreign object directory is of
questionable value, as bitmap commit selection operates on the invoking
repository's refs. Furthermore, this is the only remaining caller
outside of our test helpers that constructs an ad-hoc source and links
it to the database, and we want to get rid of this mechanism as part of
this series.
Stop constructing the ad-hoc source and instead refuse the operation.
While this results in a change in behaviour, this restriction has been
documented as such ever since f57a739691 (midx: avoid opening multiple
MIDXs when writing, 2021-09-01).
Note that this change requires us to adapt one test chain in t5319, as
it creates an object directory that is not connected to any repository
and then uses it via "--object-dir=". The setup itself already documents
this and does the necessary gymnastics to link the object directory to a
temporary repository, but subsequent tests don't. Adapt those tests to
retain and reuse the temporary repository.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Same as with the preceding commit, git-grep(1) registers each
submodule's object database as an in-memory source of the main object
database before grepping it. This was introduced as an eager alternate
registration and converted into the lazy mechanism via 8d33c3af0b (grep:
use submodule-ODB-as-alternate lazy-addition, 2021-08-16).
Starting with 0693806bf8 (grep: add repository to OID grep sources,
2021-08-16), the command instead knows to pass submodule repositories to
our workers, which means that those now use that repository to look up
objects, too. As a consequence, registering submodule sources as
alternates is not required anymore.
Remove the logic to register submodule sources. Unfortunately, this does
not allow us to get rid of the object read lock as initializing the
subrepository is still racy.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Several functions in the submodule-config subsystem implicitly depend
on `the_repository`. Refactor these to take a `struct repository` as
parameter and adapt callers accordingly.
Note that as usual with these refactorings, callers simply pass
`the_repository` even if they already have a different repository
available in the calling context. This simplifies the migration and
ensures that we don't have a change in behaviour.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `cache_tree_fully_valid()` verifies whether the cache tree
owned by the index is valid or not. As part of that, the function checks
whether the objects referenced by the cache all exist. But because the
function has no repository available, it is using the object database of
`the_repository` instead.
We could of course adapt callers to pass in a repository as parameter
explicitly to get rid of this implicit dependency on global state. But
all of them pass the cache tree owned by a `struct index_state`, and
that structure already has a reference to its owning repository.
So instead, adapt the function to accept a `struct index_state`, which
ensures that callers will implicitly always pass the correct repository.
Adapt callers accordingly.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The global variable 'fetch_if_missing' has been moved to a member in
'struct repository', continuing the libification process and
allowing per-repository control (such as for submodules).
* ty/repository-fetch-if-missing:
repository: move fetch_if_missing into struct repository
Similar to the previous commit, just as `git commit --amend` is a
foot-gun during conflict resolution, so is a partial commit (`git commit
<paths>`). Recording a conflict resolution is about capturing the state
of the entire tree on top of HEAD, not a subset of paths. For many years
we have rejected partial commits in the middle of
- a merge
- a cherry-pick
but, just like amending, this was never extended to the other operations
that can also leave conflicts to resolve:
- an `am` operation
- a revert
- a rebase that stopped for conflict resolution
Reuse sequencer_ongoing_operation(), introduced for the analogous
`--amend` check, to detect these and refuse the partial commit. A rebase
that stopped because a pick became empty is not conflict resolution and,
as an earlier patch established, is deliberately left permitted.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Running `git commit --amend` during conflict resolution is an ugly
foot-gun. For many years, we have rejected amending during conflict
resolution in the middle of
- a merge
- a cherry-pick
However, this was never extended to other operations that can also
produce conflicts:
- an `am` operation
- a revert
- a rebase
Extend it to handle these other cases now.
Extending to `am`, revert, and the apply backend of rebase are fairly
straightforward. However, with the merge backend of rebase we have to
be more careful, since it powers interactive rebases and
- the interactive machinery internally uses `git commit --amend` for
`squash` and `reword` directives
- users are expected to `git commit --amend` after hitting an `edit`
or `break` directive
So, we need to be careful with rebase to only reject amending when doing
conflict resolution.
A few files under the rebase-merge/ directory provide us the necessary
information:
- stopped-sha is written only when the rebase stops and hands control
back to the user, so its presence marks a genuine stop -- as opposed
to the sequencer's own internal `git commit --amend` while applying
a squash, fixup, or reword, during which no stopped-sha exists.
- amend is written only when the rebase stops with HEAD already
pointing at the commit the user is meant to amend: a clean `edit`,
or a fast-forward `reword`. Its absence at a stop therefore means
the commit did not apply, so HEAD is the previously-applied commit
rather than the one being rebased -- exactly the case we refuse.
So for the merge backend we die when stopped-sha exists and amend does
not. This covers a plain conflicted pick as well as a conflicted `edit`
(both leave HEAD on the previously-applied commit), while still allowing
a clean `edit` or `reword` stop and a `break` stop (no stopped-sha).
stopped-sha is unlinked at the start of the resume loop, so a resumed
squash's internal amend is unaffected.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a rebase applies a commit that becomes empty, it stops and asks the
user to decide whether to keep it or drop it. HEAD still points at the
previously-applied commit at that point, so amending is refused, with:
You are in the middle of a rebase -- cannot amend.
That message would suggest that amending is not allowed during an 'edit'
or 'break' stop, which is misleading, plus it lacks the specificity that
might help the user know why their particular case is a problem: the
commit they intended to amend became empty and was dropped, so amending
would affect the wrong commit. Reword the error accordingly.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For years, we disallowed partial commits during merges or cherry-picks.
In commit 430b75f720 (commit: give correct advice for empty commit
during a rebase, 2019-12-06) it was noted that the "cannot do a partial
commit during a cherry-pick" message was also printed when rebasing a
commit that became empty, and rather than drop the check in that case,
that commit opted to make the message print the actual operation that
was in progress.
Since a commit that has become empty comes without conflicts, a new
partial commit poses no problems; remove the error in that case.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 430b75f720 (commit: give correct advice for empty commit during
a rebase, 2019-12-06) introduced a FROM_REBASE_PICK enum value and an
is_from_rebase() function. Those names failed to convey that they were
specifically about hitting a commit that becomes empty when rebasing.
Clarify their names now.
While at it, change `whence == FROM_REBASE_NOW_EMPTY` to use
`is_from_rebase_now_empty(whence)`.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Racy Git problems persist today, manifesting themselves in the
performance of commands like "git diff" in new worktrees [1]. We have
long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond
precision when available, which mitigates most if not all racy issues,
but most builds we know about don't use it. In part, that's because
someone distributing Git can't safely enable it at compile-time if they
don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code.
Regardless, we want to give users the ability to benefit from it. This
requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of
USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag
"--ignore-space-change" may be particularly helpful when viewing changes
from this patch.
Signed-off-by: D. Ben Knoble <ben.knoble@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
One of the stated goals of git-replay(1) is to allow implementing the
git-rebase(1) functionality on the server side.
The default mode of git-rebase(1) is to act as if `--no-rebase-merges`
was given. This mode drops merge commits instead of replaying them, and
linearizes the history into a sequence of regular (single-parent)
commits.
Add option `--linearize` to git-replay(1) to do the same. Each replayed
commit is stacked on top of the previously replayed one. When a merge is
encountered, the commits reachable from all of its sides are replayed
into the single line and the merge itself is dropped.
If a ref was pointing to a merge commit, that ref is updated to the
merge's last replayed ancestor.
git-replay(1) accepts multiple branches, for example:
$ git replay --onto main topic1 topic2
Without `--linearize` this replays 'topic1' and 'topic2' onto 'main'
(keeping shared portions of history shared and divergent parts
divergent) and updates both refs.
Due to current implementation limitations, replaying multiple branches
with `--linearize` is disallowed to avoid concatenating unrelated
histories into a single line. For the same reason disallow the use of
`--contained` with `--linearize`.
Users who want to linearize multiple branches are advised to do this in
separate git-replay(1) invocations. Linearizing multiple branches at
once might be added later.
Note that `--linearize` is not modeled after git-rebase(1)'s
`--rebase-merges[=<mode>]` interface. Recreating merges, by preserving
their topology, is a distinct operation that would be a separate mode.
`--linearize` only drops merges and replays commits linearly. So
git-replay(1) uses its own option rather than reusing that interface.
Based-on-patches-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Toon Claes <toon@iotcl.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The pack-objects command has been updated to record the total bytes
written to pack files in trace2 output, allowing performance
analysis of different compression settings by comparing the
resulting pack sizes.
* fr/pack-objects-trace-pack-bytes:
pack-objects: trace pack bytes written
The mechanism to generate a packfile corresponding to the result of
a fetch/push has been made pluggable through a set of object
database callback functions, removing hardcoded references to
'pack-objects' and enabling alternative ODBs to serve packfiles
themselves.
* ps/odb-pluggable-pack-generation:
bundle: generate packfiles via the object database
bundle: get (mostly) rid of `the_repository`
builtin/bundle: refactor option handling for progress meter
send-pack: generate packfiles via the object database
upload-pack: generate packfiles via the object database
odb: introduce interface to generate packfiles
The 'git receive-pack' command has been updated to use a new ODB
transaction interface for writing incoming packfiles, making it more
backend-agnostic.
* jt/receive-pack-pluggable-writes:
odb/transaction: add transaction interface to write packfiles
odb: return temporary ODB source when set
builtin/receive-pack: explicitly pass packfile fd
builtin/receive-pack: report unpack errors via strbuf
builtin/receive-pack: lift global state out of unpack()
builtin/receive-pack: read unpack limit config lazily
builtin/receive-pack: pass shallow file explicitly
odb/transaction: add transaction finalize interface
builtin/receive-pack: properly clean up keep files
The object database layer has been simplified by eagerly loading
alternate object directories upon initialization, instead of
deferring it to the first object lookup. This eliminates the need
for scattered lazy-loading calls throughout the codebase and paves
the way for integrating alternates with the pluggable backends.
* ps/odb-eagerly-load-alternates:
odb: drop `alternates_db` field
odb: drop `loaded_alternates` field
odb: eagerly initialize alternates
odb: decouple source path comparisons from `the_repository`
setup: create ref and object databases after config is written
The object database (odb) API has been refactored to distinguish
between missing objects and corrupt ones by returning more
descriptive error statuses. Both the packed and loose backends now
faithfully propagate error details using a generic strbuf error
mechanism, removing backend-specific leakage from central lookup
paths.
* ps/odb-generic-corrupt-objects:
odb: handle `OBJECT_INFO_DIE_IF_CORRUPT` generically
odb/source: allow `read_object_info()` to bubble up error messages
odb/source: let callers discern missing and corrupt objects
odb/source: introduce error status when reading objects
odb/source-packed: flag known-bad objects as corrupt and not missing
The DWIM logic in 'git worktree add' sometimes tried to infer a
remote-tracking branch when an explicit '-b' or '-B' option was
given to create a new branch, causing the explicit branch name to
be ignored, which has been corrected.
* yn/worktree-add-no-dwim-with-b:
worktree add: shouldn't dwim if -b or -B is given
The consistency checks for loose objects are hosted by "builtin/fsck.c".
These checks are obviously specific to the "loose" backend.
Move the logic into `odb_source_loose_fsck()`. Introduce a new "verbose"
flag so that we can properly retain semantics around whether or not we
want to print some status messages.
Note that this fixes a bug as a side effect: the progress meter was
captured in the callback data before `start_progress()` was even called,
so the per-subdirectory progress updates always operated on a NULL
pointer and the meter jumped straight from 0 to 256 upon completion. The
new code only sets up the callback data's progress meter after it has
been created, so the progress display now advances incrementally again.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for multi-pack indexes are hosted in `cmd_fsck()` directly.
These checks are obviously specific to the "packed" backend.
Move the logic into `odb_source_packed_fsck()`. As in preceding commits,
this means that we now properly honor both "--connectivity-only" and
"--no-full". Furthermore, we drop the dedicated `ERROR_MULTI_PACK_INDEX`
bit and instead use the generic `ERROR_OBJECT` bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for bitmaps live in `verify_bitmap_files()`, which is called
by "builtin/fsck.c". These checks are obviously specific to the "packed"
backend.
Move the logic into `odb_source_packed_fsck()`. As in preceding commits,
this means that we now properly honor both "--connectivity-only" and
"--no-full". Furthermore, we drop the dedicated `ERROR_BITMAP` bit and
instead use the generic `ERROR_OBJECT` bit.
Note that this change also adapts `verify_bitmap_files()` to be
focused on a single "packed" source instead of verifying bitmaps from
all sources. This change is required as we already know to loop around
the sources in `odb_fsck()` itself.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for reverse indexes live in `check_pack_rev_indexes()`, which
is hosted in "builtin/fsck.c". These checks are obviously specific to
the "packed" backend.
Move the logic into `odb_source_packed_fsck()`. As in the preceding
commit, drop the dedicated `ERROR_PACK_REV_INDEX` bit and instead use
the generic `ERROR_OBJECT` bit.
Note that this changes behaviour in two ways:
- The checks are now skipped when "--connectivity-only" was passed.
This is because we don't even run `odb_fsck()` at all when that
flag has been passed by the user, and not verifying data structures
of the object database matches the documented intent of that flag,
which is to only check the connectivity of reachable objects.
- The checks are now skipped for non-local sources when "--no-full"
was passed. This is, again, in line with the documented intent of
that flag.
Add a test to cast these semantics into stone.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the packfile verification out of `cmd_fsck()` and into the "packed"
source. While doing so, thread the progress meter and object callback
through the newly introduced `struct odb_fsck_options` so that the
caller's preferences are honoured without exposing those details at the
"builtin/fsck.c" level.
Note that the old code reported failures when verifying packfiles with
the `ERROR_PACK` bit, which gets returned to the caller via the exit
code. This bit is neither exercised in our test suite nor is it
documented anywhere in our codebase. Furthermore, this bit is highly
specific to the object storage backend, which makes it a bad fit for the
new pluggable infrastructure. So instead of retaining these semantics,
we drop them and return the generic `ERROR_OBJECT` bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The on-disk consistency checks in git-fsck(1) are conceptually
backend-specific: while connectivity checks and object-level parsing
checks are generic, verifying the physical integrity of packfiles and
loose objects is meaningful only to backends that use these formats:
Having these checks live in "builtin/fsck.c" violates that layering,
because it forces the command to reach directly into format-specific
internals.
Provide new infrastructure to make these format-specific checks
pluggable and implement stubs for the different source types we already
have. In subsequent commits we'll move functionality over piece by
piece.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
According to git-fsck(1), the "--full" option behaves in the following
way:
Check not just objects in GIT_OBJECT_DIRECTORY ($GIT_DIR/objects), but
also the ones found in alternate object pools listed in
GIT_ALTERNATE_OBJECT_DIRECTORIES or $GIT_DIR/objects/info/alternates,
and in packed Git archives found in $GIT_DIR/objects/pack and
corresponding pack subdirectories in alternate object pools.
So ultimately, it is supposed to control two things: (1) whether we only
check the main object directory, and (2) whether we check packfiles.
In its current state though, the flag only controls whether we check
packfiles or not, and if so we verify packfiles of all attached sources.
But we also have checks for loose objects in git-fsck(1), and here we
unconditionally check them in all sources.
The flag is arguably conflating two unrelated concerns with one another,
and it really should be split up into two flags: one that controls how
thorough we want to check individual sources, and one that controls
which sources we want to check in the first place. So ideally, we would
have:
- "--include-alternates": check all sources, not only the local one.
- "--include-optimized-objects": check not only loose objects, but
also those that have been packed. Note that we explicitly don't say
"--include-packed-objects" here to be more backend-agnostic.
- "--full": implies both of the above flags.
This feels out of scope for this series though. So for now, simply fix
the code by honoring locality of the sources for loose objects.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In subsequent commits we're about to rework some of the option handling
in git-fsck(1) a bit. It is currently a bit of a mess though due to lots
of global state that makes it hard to see which flags are used where
exactly.
Refactor the code by moving the fsck options into `cmd_fsck()`. This
allows us to convert some of the options into function-local variables.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The interfaces of the functions `fsck_obj()` and `fsck_obj_buffer()` are
somewhat similar to one another. The only difference between those two
is that `fsck_obj()` takes an already-parsed object as input, whereas
`fsck_obj_buffer()` parses the buffer and then calls `fsck_obj()`.
Furthermore, `fsck_obj()` has no callers other than `fsck_obj_buffer()`.
Refactor the code by merging those two functions. This makes it obvious
which function does what, and it allows us to get rid of the early
return in `fsck_obj()` in case `SEEN` is set as the only caller
unconditionally clears that bit before calling it anyway.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When checking loose objects we manually parse the object buffer we have
read from the on-disk file, mark the object and then call `fsck_obj()`.
The exact same steps are also performed by `fsck_obj_buffer()`.
Stop open-coding this logic and call `fsck_obj_buffer()` instead.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When creating a repository with alternates we first initialize the
object database and then write alternates to it in a separate step. This
is unfortunate due to a couple of reasons:
- It requires us to have a `write_alternates()` callback, which is
unfortunate as we never even write alternates to an object database
after it has been created.
- We're about to make alternates an implementation detail of the
object database's backend in a future patch series, so having this
callback is suboptimal there.
- The backend has more flexibility with how exactly alternates are
configured when it itself is in full control over their setup at the
time where it creates the object database itself.
We have thus introduced the ability to write alternates right at
creation time in the preceding commits, and we have unified setup of
alternates into a single location. All that's left to do for us now is
to wire up alternates as an option for the database creation.
Do so.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar as in the preceding commit, move the setup of alternates for
local clones with "--no-shared" into `collect_alternates()`. With this
step, the complete setup of alternates is now handled by that function.
Note that besides moving stuff around, it also fixes a bug: previously,
we did not know to resolve the referenced repository's common directory.
Consequently, when referencing a worktree we failed to resolve
alternates. But as `collect_alternates()` already knows to resolve the
commondir for "--local" we can simply reuse this resolved path for our
purpose.
Add two tests, the first one of which exercises this bug to avoid future
regressions. The second patch ensures that we properly handle relative
alternates for a referenced worktree.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning a local repository with "--shared" we add that repository
to the new repository's alternates. This is done in `clone_local()`,
which is responsible for performing local clones.
Move the logic into `collect_alternates()` to unify our setup of
alternates. Furthermore, this will allow us to set up alternates right
at creation time of the object database.
Note that the logic for cloning a local repository with "--no-shared" is
not yet part of `collect_alternates()`. This will be handled in the next
commit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can pass "--reference{,-if-able}" to git-clone(1) to instruct it
to set up alternates for the newly created repository. This allows it to
reuse objects from the source repository so that in the best case we
don't have to clone all objects over.
Those options are handled by the confusingly named `setup_reference()`
function -- without the above context, one might rightfully believe that
it was about refs, not about alternates. The function itself is rather
simple: we loop through all provided alternate paths and then, if such
an alternate is valid, we write it to the object database.
In subsequent commits we're about to consolidate the complete setup of
alternates into this function, and furthermore we'll refactor the setup
of the object database to handle doing this for us instead of writing
the alterantes into it one by one.
Prepare for this refactoring by collecting the alternates into a strvec.
Rename the function to `collect_alternates()` to clarify its scope.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a subsequent commit, `setup_reference()` will start to call
`copy_alternates()`. Prepare for this by moving the function further
down so that we can avoid adding a declaration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano