The git config --global read operations have been updated to respect
both $HOME/.gitconfig and $XDG_CONFIG_HOME/git/config, fixing an
inconsistency where only the former was read when both configuration
files are present.
* dw/config-read-both-global:
config: read global scope via config_sequence
config: let sequence require a successful file
path: use forward slashes in XDG config on Windows
The --recurse-submodules option in 'git branch' has been fixed to
avoid a crash when the start point is not a reference (e.g., a raw
object ID). The creation path now skips setting up tracking and
properly forwards the absent tracking name to the submodule helper.
* vv/branch-recurse-no-start-ref:
branch: allow recursion with no tracking name
branch: do not track a start point with no ref
The codebase has been updated to use the newer utimensat() POSIX
function instead of the obsolescent utime(), allowing
high-precision timestamps while preserving fallback compatibility.
* as/utimensat-utimes:
compat/posix: drop legacy <utime.h> header and shims
treewide: use utimensat(2) instead of legacy utime(3p)
compat/posix: introduce utimensat(2) wrapper
'git pull' has been taught to fail more gracefully instead of
segfaulting when lookup_commit_reference() fails on unparseable
objects.
* jk/pull-null-merge-head:
pull: avoid segfault when commit lookup fails
The build-time knob 'USE_NSEC' for nanosecond stat precision has been
converted to a runtime configuration 'core.useNanosec', allowing
distributions to bundle one binary that adapts to filesystem
capabilities dynamically.
* dk/use-nsec-runtime:
core: convert build-time USE_NSEC into runtime core.useNanosec
environment: align repo_config_values_init with struct declaration
meson: expose knob for xmlto relative links in manuals
The terminology regarding reference storage formats has been unified
across command-line options, environment variables, configuration
variables, and source code, standardizing on the phrase "ref storage
format" (e.g., `--ref-storage-format`, `'GIT_REF_STORAGE_FORMAT'`).
Additionally, the `--ref-storage-format` option has been updated to
accept payloads in the form `<format>://<payload>`.
* ps/ref-storage-format:
setup: allow "--ref-storage-format=" to specify a payload
setup: rename "init.defaultRefFormat" to "init.defaultRefStorageFormat"
t: rename GIT_TEST_DEFAULT_REF_FORMAT
setup: rename ref storage format environment variables
setup: refactor how we configure the ref storage format
refs: expose function to parse reference URIs
help: rename "default-ref-format" to "default-ref-storage-format"
builtin/rev-parse: rename "--show-ref-format" to "--show-ref-storage-format"
builtin/submodule: rename "--ref-format=" to "--ref-storage-format="
builtin/refs: rename "--ref-format=" to "--ref-storage-format="
builtin/clone: rename "--ref-format=" to "--ref-storage-format="
builtin/init: rename "--ref-format=" to "--ref-storage-format="
parse-options: allow for hidden aliases
A new hook 'report' is added to 'git receive-pack', which runs after
reference updates and allows the server to filter or modify the
packet-line status report sent back to the client.
* kn/receive-report-hook:
receive-pack: coccinelle fix
hook: introduce the receive-report hook
receive-pack: move message generation to separate function
receive-pack: drop static variables to track report status version
doc: add proc-receive hook info in 'git-receive-pack.adoc'
The experimental 'git history' command has been taught a new 'squash'
subcommand to fold a range of commits into a single commit, with any
descendants replayed on top.
* hn/history-squash:
history: support editing squashed commit messages
history: create squashed commits without editing
history: protect branches when squashing a range
history: validate squash revision ranges
history: add skeleton for squash subcommand
sequencer: share the squash message marker helpers and flags
history: give commit_tree_ext a message template
history: extract helper for a commit's parent tree
The setup of alternates has been deferred to object database
creation time during clone, which drops the unused ad-hoc alternate
writing API, simplifying the object database backend interface.
* ps/odb-alternates-at-creation:
odb/source: remove the ability to write alternates
builtin/clone: write alternates via `odb_create_on_disk()`
odb/source: support writing alternates when creating the database
builtin/clone: move setup of alternates for non-shared local clones
builtin/clone: move setup of alternates for shared local clones
builtin/clone: refactor handling of "--reference{,-if-able}"
builtin/clone: move around `setup_reference()`
builtin/clone: defer setup of the object database
setup: split up concerns of `init_db()`
get_can_ff() and already_up_to_date() pass the result of
lookup_commit_reference() straight to commit_list_insert() and
repo_is_descendant_of() without checking it. When the object
behind HEAD or one of the merge heads cannot be parsed, e.g. because
a loose object was left truncated by a fetch or gc racing on the
same repository, lookup_commit_reference() returns NULL and
"git pull" segfaults instead of reporting the corruption.
Treat a failed lookup as "cannot fast-forward" and "not up to date",
so that the caller falls through to the normal merge path, which
already diagnoses the broken object and fails cleanly.
An alternative would be to report the breakage at each lookup site,
which could give a more precise diagnosis. The minimal guards are
preferred because they do no more than is needed to avoid the
crash, and will be easy to drop once "git pull" is reworked to
resolve object names into commit objects early and pass those
around, at which point there will not be multiple lookups of the
same object name to guard in the first place.
The test corrupts the loose object in place rather than removing
it: a missing object that is still recorded in the commit-graph is
caught by the consistency check in fetch-pack before "git pull"
reaches the fast-forward check, so removing it would not exercise
the crash.
Signed-off-by: Jiri Kuncar <jiri.kuncar@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The memory leak caused by not unusing the commit buffer returned by
repo_logmsg_reencode() during the rewording operation in 'git
history' has been plugged.
* ks/history-commit-leakfix:
builtin/history: unuse the commit buffer after use
The consistency checks for the object database (fsck) have been
decoupled from the generic builtin implementation and moved into the
backend-specific object source layers, making them pluggable for
different object storage formats.
* ps/odb-pluggable-fsck:
builtin/fsck: move loose object verification into the loose source
builtin/fsck: move multi-pack index verification into the packed source
builtin/fsck: move bitmap verification into the packed source
builtin/fsck: move reverse index verification into the packed source
builtin/fsck: move packfile verification into the packed source
odb: provide infrastructure for pluggable fsck checks
builtin/fsck: don't check alternates with "--no-full"
builtin/fsck: de-globalize option handling
builtin/fsck: merge `fsck_obj_buffer()` and `fsck_obj()`
builtin/fsck: use `fsck_obj_buffer()` when checking loose objects
The mechanism to register in-memory alternate object sources has
been removed, as submodule object databases are now accessed
natively via their own repository structures. This simplifies
object database management and prepares the codebase for migrating
alternate tracking into the files backend.
* ps/odb-stop-registering-in-memory-sources:
odb: remove the ability to link sources ad-hoc
t/helper: stop registering alternates in "ref-store" command
t/helper: adapt read-midx to not link ad-hoc source anymore
builtin/multi-pack-index: refuse unknown sources with "--object-dir="
odb/packed: fix memory leaks when freeing source
tmp-objdir: drop unused function to register alternate
odb: remove infrastructure to register submodule sources
builtin/grep: stop registering submodule ODB as source
submodule-config: stop registering submodule sources
submodule-config: stop using `the_hash_algo`
submodule-config: remove uses of `the_repository`
cache-tree: remove dependency on `the_repository`
cache-tree: drop `the_repository` in `cache_tree_fully_valid()`
Every call to repo_logmsg_reencode() must be paired with a call to
repo_unuse_commit_buffer(), or we may leak an allocated buffer. We
have such a leak in "git history", which we can fix by adding an unuse
call.
The leak-checking tests don't detect this because we only allocate a
fresh buffer sometimes: when the message is reencoded, or when we had
to load it fresh from the odb (e.g., because the commit was parsed
from the commit graph rather than the object contents). But you can
see it by running:
make SANITIZE=leak
cd t
GIT_TEST_COMMIT_GRAPH=1 ./t3451-history-reword.sh
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Let's not check the nullness of cmd->error_string_owned before
calling FREE_AND_NULL(cmd->error_string_owned). It is cheap and
safe to call FREE_AND_NULL(variable) for a variable that has NULL
in it.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git rev-list command has been augmented with a '--missing-only'
option that filters the output to only show missing objects,
stripping the leading '?' character and suppressing present objects,
which is useful when used in combination with '--missing=print' or
'--missing=print-info'.
* sa/rev-list-missing-only:
rev-list: add --missing-only option to filter output
"git maintenance" triggered "rerere gc" in unappropriate times and
interfered with "git rebase" etc. too much. The conditions "rerere
gc" gets triggered have been tweaked.
* ps/tune-rerere-gc:
builtin/maintenance: improve heuristic for "rerere gc"
rerere: extract logic to determine whether entries are stale
The autostash fallback in 'git checkout -m' has been refined to only
retry when there are local changes. Additionally, a blank line now
visually separates autostash conflict advice from the subsequent
branch-switch message.
* hn/checkout-m-autostash-refine:
checkout: separate autostash conflict advice from branch-switch message
stash: reserve exit status 1 for conflicts
Running "git history" in a corrupt repository can (unsurprisingly)
segfault when a necessary tree object is not found.
* jc/history-missing-tree-errorfix:
history: do not dereference NULL when parent tree is missing
The error path in 'git submodule--helper' has been updated to plug a
memory leak when a repository handle could not be obtained,
leveraging an updated idempotent repo_clear().
* jk/submodule-error-leak:
submodule--helper: free URL when repository setup fails
repository: make repo_clear() idempotent
Teach 'am', 'revert', and 'rebase' that running 'commit --amend' or a
partial 'commit <paths>' makes no sense during operations that stop
and return control to the user to resolve conflicts left in the
working tree, just like 'cherry-pick' and 'merge' do.
* en/no-amend-during-conflicts:
commit: refuse partial commits during conflict resolution
commit: refuse to amend during conflict resolution
commit: reword the empty-commit rebase amend error
commit: allow a partial commit when a rebase pick becomes empty
commit: clarify FROM_REBASE_PICK and is_from_rebase() names
When running 'git-receive-pack(1)', there is no way for the server to
intercept and modify the status report before it is sent back to the
client. Servers with custom logic may need to transform or gate the
report based on the outcome of external logic post reference updates.
This is specially needed for our usecase at GitLab where we have custom
MVCC logic on top of Git which creates a new version for each push
operation. The new version is only committed when certain external
operations post reference transaction succeed. So reporting the correct
message based on the outcome of these operations is important.
The outcome of these operations is only known after `execute_commands()`
has returned and before the report is written. There is no point in
receive-pack where the server can act on that.
We cannot use any of the existing hooks as:
- The pre-receive hook runs too early, as we haven't updated
references at that point yet and we need to have the full view of
all resulting updates (both objects and references).
- The update hook is too inefficient as it runs once per reference,
and we cannot trivially determine the last update.
- The reference-transaction hook is not suited for this. It fires from
within `ref_transaction_commit()`, which is before the outcome we
need to report is known, so there is no phase at which it could give
us the answer. It also does not contain any knowledge regarding the
push and cannot communicate with the clients.
- The proc-receive hook replaces execute_commands() for references
matching 'receive.procReceiveRefs'. We need to gate the report for
the push as a whole.
- The post-receive and post-update hooks cannot be used as they run
too late, at the point where we have already reported success to the
client.
Introduce a new 'receive-report' hook. The hook receives the complete
pkt-line encoded status report on standard input, after all ref updates
have been applied to the repository by execute_commands() but before the
report is sent to the client. See linkgit:gitprotocol-pack[5] details on
the protocol structure.
The hook's stdout fully replaces the report sent to the client.
receive-pack fully buffers the hook's stdout before acting on the exit
status, so the exit code is known before the client receives anything.
This gives two distinct behaviors depending on exit status:
- Exit 0: the hook's stdout is used as the report. The hook can
rewrite 'ok' lines to 'ng' lines to signal per-ref rejection to the
client while receive-pack itself exits cleanly. The client marks
rejected refs as '[remote rejected]' and exits with a non-zero
status if any ref is 'ng'.
- Non-zero exit: the hook's stdout is discarded, receive-pack modifies
all references to be rejected with a 'receive-report hook failed'
error.
In both cases, any output the hook writes to standard error is
forwarded to the client over the sideband channel and appears as
'remote:' lines on the client terminal. Writing to stderr alone does
not affect the push outcome.
Reference updates applied by execute_commands() are not rolled back in
either failure mode. The hook can cause the client to perceive the push
as failed, but cannot undo server-side changes. This creates a
divergence that the server cannot resolve: the client leaves its
remote-tracking reference at the old value while the update is in fact
applied, and a later fetch may reveal the update that the push reported
as rejected.
The hook is therefore only appropriate for servers which can guarantee
that a rejected update is not observable by any reader. In our case the
transaction committed by execute_commands() produces a candidate version
which is not visible to other readers and is only published once the
subsequent operations succeed, so a report of 'ng' corresponds to a
version that is discarded rather than published. On a repository where a
committed reference update is immediately visible, rejecting a push from
this hook would instead leave the pusher with a view that does not match
the server.
This hook does not use the config-based hook infrastructure, which
supports running multiple scripts per hook event. This hook is a
bidirectional filter: it receives the report on stdin and writes a
modified version to stdout. Running multiple such scripts sequentially
would require piping the output of one into the input of the next,
which the current hook infrastructure does not support. A single-script
design is therefore a natural fit, and is consistent with how
'proc-receive' is structured for the same reason.
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After git-receive-pack(1) has committed the reference updates, we call
either `report()` or `report_v2()` to report to the client which of the
references we have updated successfully and which updates have failed.
The only difference between those two functions is that the latter also
knows to provide a more detailed report about how exactly a given
reference was updated.
With this, also drop `report_v2()` as both report functions now are
similar in structure with only the `report_status_version`
differentiating them.
In the next commit we're about to add another site that wants to
generate these reports. Refactor the logic into a shared function that
can easily be reused.
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 'git-receive-pack(1)', to track the report status version, we use the
static variables `report_status` and `report_status_v2`. As the report
status version is mutually exclusive, using an enum better suits the
requirement. switch to using a new `enum report_status_version`, while
also dropping the static variable to make the flow easier to understand.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Racy Git problems persist today, manifesting themselves in the
performance of commands like "git diff" in new worktrees [1]. We have
long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond
precision when available, which mitigates most if not all racy issues,
but most builds we know about don't use it. In part, that's because
someone distributing Git can't safely enable it at compile-time if they
don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code.
Regardless, we want to give users the ability to benefit from it. This
requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of
USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag
"--ignore-space-change" may be particularly helpful when viewing changes
from this patch.
Signed-off-by: D. Ben Knoble <ben.knoble@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The consistency checks for loose objects are hosted by "builtin/fsck.c".
These checks are obviously specific to the "loose" backend.
Move the logic into `odb_source_loose_fsck()`. Introduce a new "verbose"
flag so that we can properly retain semantics around whether or not we
want to print some status messages.
Note that this fixes a bug as a side effect: the progress meter was
captured in the callback data before `start_progress()` was even called,
so the per-subdirectory progress updates always operated on a NULL
pointer and the meter jumped straight from 0 to 256 upon completion. The
new code only sets up the callback data's progress meter after it has
been created, so the progress display now advances incrementally again.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for multi-pack indexes are hosted in `cmd_fsck()` directly.
These checks are obviously specific to the "packed" backend.
Move the logic into `odb_source_packed_fsck()`. As in preceding commits,
this means that we now properly honor both "--connectivity-only" and
"--no-full". Furthermore, we drop the dedicated `ERROR_MULTI_PACK_INDEX`
bit and instead use the generic `ERROR_OBJECT` bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for bitmaps live in `verify_bitmap_files()`, which is called
by "builtin/fsck.c". These checks are obviously specific to the "packed"
backend.
Move the logic into `odb_source_packed_fsck()`. As in preceding commits,
this means that we now properly honor both "--connectivity-only" and
"--no-full". Furthermore, we drop the dedicated `ERROR_BITMAP` bit and
instead use the generic `ERROR_OBJECT` bit.
Note that this change also adapts `verify_bitmap_files()` to be
focused on a single "packed" source instead of verifying bitmaps from
all sources. This change is required as we already know to loop around
the sources in `odb_fsck()` itself.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for reverse indexes live in `check_pack_rev_indexes()`, which
is hosted in "builtin/fsck.c". These checks are obviously specific to
the "packed" backend.
Move the logic into `odb_source_packed_fsck()`. As in the preceding
commit, drop the dedicated `ERROR_PACK_REV_INDEX` bit and instead use
the generic `ERROR_OBJECT` bit.
Note that this changes behaviour in two ways:
- The checks are now skipped when "--connectivity-only" was passed.
This is because we don't even run `odb_fsck()` at all when that
flag has been passed by the user, and not verifying data structures
of the object database matches the documented intent of that flag,
which is to only check the connectivity of reachable objects.
- The checks are now skipped for non-local sources when "--no-full"
was passed. This is, again, in line with the documented intent of
that flag.
Add a test to cast these semantics into stone.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the packfile verification out of `cmd_fsck()` and into the "packed"
source. While doing so, thread the progress meter and object callback
through the newly introduced `struct odb_fsck_options` so that the
caller's preferences are honoured without exposing those details at the
"builtin/fsck.c" level.
Note that the old code reported failures when verifying packfiles with
the `ERROR_PACK` bit, which gets returned to the caller via the exit
code. This bit is neither exercised in our test suite nor is it
documented anywhere in our codebase. Furthermore, this bit is highly
specific to the object storage backend, which makes it a bad fit for the
new pluggable infrastructure. So instead of retaining these semantics,
we drop them and return the generic `ERROR_OBJECT` bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The on-disk consistency checks in git-fsck(1) are conceptually
backend-specific: while connectivity checks and object-level parsing
checks are generic, verifying the physical integrity of packfiles and
loose objects is meaningful only to backends that use these formats:
Having these checks live in "builtin/fsck.c" violates that layering,
because it forces the command to reach directly into format-specific
internals.
Provide new infrastructure to make these format-specific checks
pluggable and implement stubs for the different source types we already
have. In subsequent commits we'll move functionality over piece by
piece.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
According to git-fsck(1), the "--full" option behaves in the following
way:
Check not just objects in GIT_OBJECT_DIRECTORY ($GIT_DIR/objects), but
also the ones found in alternate object pools listed in
GIT_ALTERNATE_OBJECT_DIRECTORIES or $GIT_DIR/objects/info/alternates,
and in packed Git archives found in $GIT_DIR/objects/pack and
corresponding pack subdirectories in alternate object pools.
So ultimately, it is supposed to control two things: (1) whether we only
check the main object directory, and (2) whether we check packfiles.
In its current state though, the flag only controls whether we check
packfiles or not, and if so we verify packfiles of all attached sources.
But we also have checks for loose objects in git-fsck(1), and here we
unconditionally check them in all sources.
The flag is arguably conflating two unrelated concerns with one another,
and it really should be split up into two flags: one that controls how
thorough we want to check individual sources, and one that controls
which sources we want to check in the first place. So ideally, we would
have:
- "--include-alternates": check all sources, not only the local one.
- "--include-optimized-objects": check not only loose objects, but
also those that have been packed. Note that we explicitly don't say
"--include-packed-objects" here to be more backend-agnostic.
- "--full": implies both of the above flags.
This feels out of scope for this series though. So for now, simply fix
the code by honoring locality of the sources for loose objects.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In subsequent commits we're about to rework some of the option handling
in git-fsck(1) a bit. It is currently a bit of a mess though due to lots
of global state that makes it hard to see which flags are used where
exactly.
Refactor the code by moving the fsck options into `cmd_fsck()`. This
allows us to convert some of the options into function-local variables.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The interfaces of the functions `fsck_obj()` and `fsck_obj_buffer()` are
somewhat similar to one another. The only difference between those two
is that `fsck_obj()` takes an already-parsed object as input, whereas
`fsck_obj_buffer()` parses the buffer and then calls `fsck_obj()`.
Furthermore, `fsck_obj()` has no callers other than `fsck_obj_buffer()`.
Refactor the code by merging those two functions. This makes it obvious
which function does what, and it allows us to get rid of the early
return in `fsck_obj()` in case `SEEN` is set as the only caller
unconditionally clears that bit before calling it anyway.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When checking loose objects we manually parse the object buffer we have
read from the on-disk file, mark the object and then call `fsck_obj()`.
The exact same steps are also performed by `fsck_obj_buffer()`.
Stop open-coding this logic and call `fsck_obj_buffer()` instead.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can tell git-multi-pack-index(1) to access multi-pack indices that
are stored in a different object directory via the "--object-dir="
option. This allows them to for example write or verify a multi-pack
index other than the one located in the main object directory in case a
repository has alternates with multiple multi-pack indices.
But while the documentation explicitly points out that the specified
object directory must be an alternate of the current repository, we
never verify that property. Instead, starting with 017db7bb14 (midx:
load multi-pack indices via their source, 2025-08-11), we now construct
an ad-hoc source and link it to the main object directory.
Besides contradicting the documentation, it's dubious that this really
ought to work in the first place: creating a multi-pack index (and
potentially a bitmap) for a completely foreign object directory is of
questionable value, as bitmap commit selection operates on the invoking
repository's refs. Furthermore, this is the only remaining caller
outside of our test helpers that constructs an ad-hoc source and links
it to the database, and we want to get rid of this mechanism as part of
this series.
Stop constructing the ad-hoc source and instead refuse the operation.
While this results in a change in behaviour, this restriction has been
documented as such ever since f57a739691 (midx: avoid opening multiple
MIDXs when writing, 2021-09-01).
Note that this change requires us to adapt one test chain in t5319, as
it creates an object directory that is not connected to any repository
and then uses it via "--object-dir=". The setup itself already documents
this and does the necessary gymnastics to link the object directory to a
temporary repository, but subsequent tests don't. Adapt those tests to
retain and reuse the temporary repository.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Same as with the preceding commit, git-grep(1) registers each
submodule's object database as an in-memory source of the main object
database before grepping it. This was introduced as an eager alternate
registration and converted into the lazy mechanism via 8d33c3af0b (grep:
use submodule-ODB-as-alternate lazy-addition, 2021-08-16).
Starting with 0693806bf8 (grep: add repository to OID grep sources,
2021-08-16), the command instead knows to pass submodule repositories to
our workers, which means that those now use that repository to look up
objects, too. As a consequence, registering submodule sources as
alternates is not required anymore.
Remove the logic to register submodule sources. Unfortunately, this does
not allow us to get rid of the object read lock as initializing the
subrepository is still racy.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Several functions in the submodule-config subsystem implicitly depend
on `the_repository`. Refactor these to take a `struct repository` as
parameter and adapt callers accordingly.
Note that as usual with these refactorings, callers simply pass
`the_repository` even if they already have a different repository
available in the calling context. This simplifies the migration and
ensures that we don't have a change in behaviour.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `cache_tree_fully_valid()` verifies whether the cache tree
owned by the index is valid or not. As part of that, the function checks
whether the objects referenced by the cache all exist. But because the
function has no repository available, it is using the object database of
`the_repository` instead.
We could of course adapt callers to pass in a repository as parameter
explicitly to get rid of this implicit dependency on global state. But
all of them pass the cache tree owned by a `struct index_state`, and
that structure already has a reference to its owning repository.
So instead, adapt the function to accept a `struct index_state`, which
ensures that callers will implicitly always pass the correct repository.
Adapt callers accordingly.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When creating a repository with alternates we first initialize the
object database and then write alternates to it in a separate step. This
is unfortunate due to a couple of reasons:
- It requires us to have a `write_alternates()` callback, which is
unfortunate as we never even write alternates to an object database
after it has been created.
- We're about to make alternates an implementation detail of the
object database's backend in a future patch series, so having this
callback is suboptimal there.
- The backend has more flexibility with how exactly alternates are
configured when it itself is in full control over their setup at the
time where it creates the object database itself.
We have thus introduced the ability to write alternates right at
creation time in the preceding commits, and we have unified setup of
alternates into a single location. All that's left to do for us now is
to wire up alternates as an option for the database creation.
Do so.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar as in the preceding commit, move the setup of alternates for
local clones with "--no-shared" into `collect_alternates()`. With this
step, the complete setup of alternates is now handled by that function.
Note that besides moving stuff around, it also fixes a bug: previously,
we did not know to resolve the referenced repository's common directory.
Consequently, when referencing a worktree we failed to resolve
alternates. But as `collect_alternates()` already knows to resolve the
commondir for "--local" we can simply reuse this resolved path for our
purpose.
Add two tests, the first one of which exercises this bug to avoid future
regressions. The second test ensures that we properly handle relative
alternates for a referenced worktree.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning a local repository with "--shared" we add that repository
to the new repository's alternates. This is done in `clone_local()`,
which is responsible for performing local clones.
Move the logic into `collect_alternates()` to unify our setup of
alternates. Furthermore, this will allow us to set up alternates right
at creation time of the object database.
Note that the logic for cloning a local repository with "--no-shared" is
not yet part of `collect_alternates()`. This will be handled in the next
commit, but means that at this step, we may compute `commondir` without
it being used. It will become used in the next step though.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can pass "--reference{,-if-able}" to git-clone(1) to instruct it
to set up alternates for the newly created repository. This allows it to
reuse objects from the source repository so that in the best case we
don't have to clone all objects over.
Those options are handled by the confusingly named `setup_reference()`
function -- without the above context, one might rightfully believe that
it was about refs, not about alternates. The function itself is rather
simple: we loop through all provided alternate paths and then, if such
an alternate is valid, we write it to the object database.
In subsequent commits we're about to consolidate the complete setup of
alternates into this function, and furthermore we'll refactor the setup
of the object database to handle doing this for us instead of writing
the alternates into it one by one.
Prepare for this refactoring by collecting the alternates into a strvec.
Rename the function to `collect_alternates()` to clarify its scope, as
it does not set up the references itself anymore.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a subsequent commit, `setup_reference()` will start to call
`copy_alternates()`. Prepare for this by moving the function further
down so that we can avoid adding a declaration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning a repository we defer initialization of the reference
database. This is because we don't yet know all details required for us
to initialize the refdb in the first place. Most importantly, what we
are missing is information about the object hash.
We don't do the same thing for the object database yet, but here we
essentially have the same problem. While the "files" database does not
need any information about the object format at creation time, alternate
backends are likely to require that information so that they can
properly set up their data structures.
Besides this forward-looking future proofing though, we also have a
second use case for deferring initialization of the object database,
namely alternates. When initializing the object database we do not yet
know whether we'll need alternates or not because this depends on the
repository we're about to clone from. If it is a local repository and
the user has passed "--reference{,-if-able}", then we will end up
writing alternates into the object database.
The ugly part though is that we cannot determine where the repository is
getting cloned from before it has been initialized. While we of course
already have access to the user-provided URI, that URI can be very well
rewritten via "url.<base>.insteadOf". We can of course read the global-
and system-level configuration to resolve it. But we explicitly resolve
the URI a second time after we have initialized the repository because
it can happen that we copy a ".git/config" over from our templates, and
that file may cause us to rewrite the path.
In a subsequent commit though we'll start to write alternates as part of
the repository initialization, so we'll need to have the URI properly
resolved before we can initialize the object database. This is ugly, but
as mentioned above it makes sense for us to defer its initialization
anyway so that we also know about the object hash already.
Defer creation of the object database until after we have resolved the
URI.
Note that this also requires us to defer the call to `setup_reference()`
until after we have created the object database. While you might think
that this function has something to do with references ("refs/*"), it is
in fact responsible for setting up alternates. Consequently, we can only
call it after we have created the object database already.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `init_db()` is responsible for creating the on-disk
directory structure required for a Git repository. It is used by both
git-init(1) and git-clone(1), and because their expected behaviour is
different we support a couple of flags:
- The `QUIET` flag controls whether the command is quiet or not. For
git-init(1) this is user-controllable, whereas for git-clone(1)
we're always quiet.
- The `EXIST_OK` flag controls whether a preexisting repository is
okay or not. For git-init(1) it is, for git-clone(1) it's not.
- The `SKIP_REFDB` flag controls whether the reference database should
already be created or not. For git-init(1) we do, but for
git-clone(1) we don't because it does not yet know about the default
branch and about the remote object hash.
Furthermore, we're about to add another divergence in behaviour, where
we have to also skip creation of the object database in git-clone(1).
This is becoming quite cumbersome though.
Instead of introducing another flag, start to split up concerns of the
function so that we never create the reference or object database. This
becomes the responsibility of the caller, which is thus free to defer
their creation to a later point in time. This lets us get rid of most of
the divergent behaviour:
- We don't need the `SKIP_REFDB` and a potential `SKIP_ODB` flags
anymore.
- We don't need the `QUIET` flag anymore, as nothing prints output
except for the final status message that tells the user that the
repository has been (re)initialized. But as this message is specific
to git-init(1), we can easily move it there.
The only piece of information we still have to convey is whether or not
reinitialization of a preexisting repository is okay. This is handled by
a new `reinit_ok` pointer that, if non-`NULL`, indicates that it is okay
to reinitialize the repository. Furthermore, the pointer will be written
to to indicate whether the repository was reinitialized or not, which we
need in git-init(1) to print the correct initialization message.
With these refactorings, `init_db()` is named quite misleadingly though,
as we don't create any of the reference or object databases anymore.
Rename it to `create_repository()`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The object lookup machinery has been taught to gracefully recover
when a multi-pack-index points to an owning pack that was removed
during a concurrent geometric repack, and 'git replay' has been
fixed to not segfault when reading such missing objects.
* en/midx-missing-pack-fallback:
packfile: recover when a multi-pack-index names a removed pack
mktree: do not use OBJECT_INFO_QUICK when checking objects
mktree: plug per-tree leak in --batch mode
replay: fail gracefully when a merge input is unreadable
The 'git replay' command has been taught the '--linearize' option to
drop merge commits and linearize the replayed history, mimicking 'git
rebase --no-rebase-merges'.
* tc/replay-linearize:
replay: offer an option to linearize the commit topology
replay: resolve the replay base outside pick_regular_commit()
replay: add helper to put entry into replayed_commits
The string extraction logic for the branch name and worktree name
from the given path in 'git worktree add' has been corrected and
simplified to avoid out-of-bounds reads and improper handling of
trailing slashes.
* rs/worktree-add-basename-fixes:
worktree add: let worktree_basename() return string copy
worktree add: trim slashes when deriving branch name from path
worktree add: reject separator-only path
worktree add: don't read out of bounds in worktree_basename()
Reference storage backends can be configured with a payload via the
"extensions.refStorage" config key and the "GIT_REF_STORAGE_FORMAT"
environment variable, both of which accept a URI in the format
"<format>://<payload>". The payload may contain backend-specific
information, for example an alternate refs directory or which database
references should be stored in.
The `--ref-storage-format=` option of git-init(1) and git-clone(1) does
not know about payloads though: its value is parsed as a plain format
name, so backends that require a payload cannot be conveniently set up
at initialization time via the command line.
Teach the option to accept the same URI syntax. Also, document the
optional payloads for both the "files" and "reftable" backends.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano