The terminology regarding reference storage formats has been unified
across command-line options, environment variables, configuration
variables, and source code, standardizing on the phrase "ref storage
format" (e.g., `--ref-storage-format`, `'GIT_REF_STORAGE_FORMAT'`).
Additionally, the `--ref-storage-format` option has been updated to
accept payloads in the form `<format>://<payload>`.
* ps/ref-storage-format:
setup: allow "--ref-storage-format=" to specify a payload
setup: rename "init.defaultRefFormat" to "init.defaultRefStorageFormat"
t: rename GIT_TEST_DEFAULT_REF_FORMAT
setup: rename ref storage format environment variables
setup: refactor how we configure the ref storage format
refs: expose function to parse reference URIs
help: rename "default-ref-format" to "default-ref-storage-format"
builtin/rev-parse: rename "--show-ref-format" to "--show-ref-storage-format"
builtin/submodule: rename "--ref-format=" to "--ref-storage-format="
builtin/refs: rename "--ref-format=" to "--ref-storage-format="
builtin/clone: rename "--ref-format=" to "--ref-storage-format="
builtin/init: rename "--ref-format=" to "--ref-storage-format="
parse-options: allow for hidden aliases
The build-time knob 'USE_NSEC' for nanosecond stat precision has been
converted to a runtime configuration 'core.useNanosec', allowing
distributions to bundle one binary that adapts to filesystem
capabilities dynamically.
* dk/use-nsec-runtime:
core: convert build-time USE_NSEC into runtime core.useNanosec
environment: align repo_config_values_init with struct declaration
meson: expose knob for xmlto relative links in manuals
A new hook 'report' is added to 'git receive-pack', which runs after
reference updates and allows the server to filter or modify the
packet-line status report sent back to the client.
* kn/receive-report-hook:
receive-pack: coccinelle fix
hook: introduce the receive-report hook
receive-pack: move message generation to separate function
receive-pack: drop static variables to track report status version
doc: add proc-receive hook info in 'git-receive-pack.adoc'
The experimental 'git history' command has been taught a new 'squash'
subcommand to fold a range of commits into a single commit, with any
descendants replayed on top.
* hn/history-squash:
history: support editing squashed commit messages
history: create squashed commits without editing
history: protect branches when squashing a range
history: validate squash revision ranges
history: add skeleton for squash subcommand
sequencer: share the squash message marker helpers and flags
history: give commit_tree_ext a message template
history: extract helper for a commit's parent tree
The setup of alternates has been deferred to object database
creation time during clone, which drops the unused ad-hoc alternate
writing API, simplifying the object database backend interface.
* ps/odb-alternates-at-creation:
odb/source: remove the ability to write alternates
builtin/clone: write alternates via `odb_create_on_disk()`
odb/source: support writing alternates when creating the database
builtin/clone: move setup of alternates for non-shared local clones
builtin/clone: move setup of alternates for shared local clones
builtin/clone: refactor handling of "--reference{,-if-able}"
builtin/clone: move around `setup_reference()`
builtin/clone: defer setup of the object database
setup: split up concerns of `init_db()`
Every call to repo_logmsg_reencode() must be paired with a call to
repo_unuse_commit_buffer(), or we may leak an allocated buffer. We
have such a leak in "git history", which we can fix by adding an unuse
call.
The leak-checking tests don't detect this because we only allocate a
fresh buffer sometimes: when the message is reencoded, or when we had
to load it fresh from the odb (e.g., because the commit was parsed
from the commit graph rather than the object contents). But you can
see it by running:
make SANITIZE=leak
cd t
GIT_TEST_COMMIT_GRAPH=1 ./t3451-history-reword.sh
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Let's not check the nullness of cmd->error_string_owned before
calling FREE_AND_NULL(cmd->error_string_owned). It is cheap and
safe to call FREE_AND_NULL(variable) for a variable that has NULL
in it.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The consistency checks for the object database (fsck) have been
decoupled from the generic builtin implementation and moved into the
backend-specific object source layers, making them pluggable for
different object storage formats.
* ps/odb-pluggable-fsck:
builtin/fsck: move loose object verification into the loose source
builtin/fsck: move multi-pack index verification into the packed source
builtin/fsck: move bitmap verification into the packed source
builtin/fsck: move reverse index verification into the packed source
builtin/fsck: move packfile verification into the packed source
odb: provide infrastructure for pluggable fsck checks
builtin/fsck: don't check alternates with "--no-full"
builtin/fsck: de-globalize option handling
builtin/fsck: merge `fsck_obj_buffer()` and `fsck_obj()`
builtin/fsck: use `fsck_obj_buffer()` when checking loose objects
When running 'git-receive-pack(1)', there is no way for the server to
intercept and modify the status report before it is sent back to the
client. Servers with custom logic may need to transform or gate the
report based on the outcome of external logic post reference updates.
This is specially needed for our usecase at GitLab where we have custom
MVCC logic on top of Git which creates a new version for each push
operation. The new version is only committed when certain external
operations post reference transaction succeed. So reporting the correct
message based on the outcome of these operations is important.
The outcome of these operations is only known after `execute_commands()`
has returned and before the report is written. There is no point in
receive-pack where the server can act on that.
We cannot use any of the existing hooks as:
- The pre-receive hook runs too early, as we haven't updated
references at that point yet and we need to have the full view of
all resulting updates (both objects and references).
- The update hook is too inefficient as it runs once per reference,
and we cannot trivially determine the last update.
- The reference-transaction hook is not suited for this. It fires from
within `ref_transaction_commit()`, which is before the outcome we
need to report is known, so there is no phase at which it could give
us the answer. It also does not contain any knowledge regarding the
push and cannot communicate with the clients.
- The proc-receive hook replaces execute_commands() for references
matching 'receive.procReceiveRefs'. We need to gate the report for
the push as a whole.
- The post-receive and post-update hooks cannot be used as they run
too late, at the point where we have already reported success to the
client.
Introduce a new 'receive-report' hook. The hook receives the complete
pkt-line encoded status report on standard input, after all ref updates
have been applied to the repository by execute_commands() but before the
report is sent to the client. See linkgit:gitprotocol-pack[5] details on
the protocol structure.
The hook's stdout fully replaces the report sent to the client.
receive-pack fully buffers the hook's stdout before acting on the exit
status, so the exit code is known before the client receives anything.
This gives two distinct behaviors depending on exit status:
- Exit 0: the hook's stdout is used as the report. The hook can
rewrite 'ok' lines to 'ng' lines to signal per-ref rejection to the
client while receive-pack itself exits cleanly. The client marks
rejected refs as '[remote rejected]' and exits with a non-zero
status if any ref is 'ng'.
- Non-zero exit: the hook's stdout is discarded, receive-pack modifies
all references to be rejected with a 'receive-report hook failed'
error.
In both cases, any output the hook writes to standard error is
forwarded to the client over the sideband channel and appears as
'remote:' lines on the client terminal. Writing to stderr alone does
not affect the push outcome.
Reference updates applied by execute_commands() are not rolled back in
either failure mode. The hook can cause the client to perceive the push
as failed, but cannot undo server-side changes. This creates a
divergence that the server cannot resolve: the client leaves its
remote-tracking reference at the old value while the update is in fact
applied, and a later fetch may reveal the update that the push reported
as rejected.
The hook is therefore only appropriate for servers which can guarantee
that a rejected update is not observable by any reader. In our case the
transaction committed by execute_commands() produces a candidate version
which is not visible to other readers and is only published once the
subsequent operations succeed, so a report of 'ng' corresponds to a
version that is discarded rather than published. On a repository where a
committed reference update is immediately visible, rejecting a push from
this hook would instead leave the pusher with a view that does not match
the server.
This hook does not use the config-based hook infrastructure, which
supports running multiple scripts per hook event. This hook is a
bidirectional filter: it receives the report on stdin and writes a
modified version to stdout. Running multiple such scripts sequentially
would require piping the output of one into the input of the next,
which the current hook infrastructure does not support. A single-script
design is therefore a natural fit, and is consistent with how
'proc-receive' is structured for the same reason.
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After git-receive-pack(1) has committed the reference updates, we call
either `report()` or `report_v2()` to report to the client which of the
references we have updated successfully and which updates have failed.
The only difference between those two functions is that the latter also
knows to provide a more detailed report about how exactly a given
reference was updated.
With this, also drop `report_v2()` as both report functions now are
similar in structure with only the `report_status_version`
differentiating them.
In the next commit we're about to add another site that wants to
generate these reports. Refactor the logic into a shared function that
can easily be reused.
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 'git-receive-pack(1)', to track the report status version, we use the
static variables `report_status` and `report_status_v2`. As the report
status version is mutually exclusive, using an enum better suits the
requirement. switch to using a new `enum report_status_version`, while
also dropping the static variable to make the flow easier to understand.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Racy Git problems persist today, manifesting themselves in the
performance of commands like "git diff" in new worktrees [1]. We have
long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond
precision when available, which mitigates most if not all racy issues,
but most builds we know about don't use it. In part, that's because
someone distributing Git can't safely enable it at compile-time if they
don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code.
Regardless, we want to give users the ability to benefit from it. This
requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of
USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag
"--ignore-space-change" may be particularly helpful when viewing changes
from this patch.
Signed-off-by: D. Ben Knoble <ben.knoble@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The mechanism to register in-memory alternate object sources has
been removed, as submodule object databases are now accessed
natively via their own repository structures. This simplifies
object database management and prepares the codebase for migrating
alternate tracking into the files backend.
* ps/odb-stop-registering-in-memory-sources:
odb: remove the ability to link sources ad-hoc
t/helper: stop registering alternates in "ref-store" command
t/helper: adapt read-midx to not link ad-hoc source anymore
builtin/multi-pack-index: refuse unknown sources with "--object-dir="
odb/packed: fix memory leaks when freeing source
tmp-objdir: drop unused function to register alternate
odb: remove infrastructure to register submodule sources
builtin/grep: stop registering submodule ODB as source
submodule-config: stop registering submodule sources
submodule-config: stop using `the_hash_algo`
submodule-config: remove uses of `the_repository`
cache-tree: remove dependency on `the_repository`
cache-tree: drop `the_repository` in `cache_tree_fully_valid()`
The consistency checks for loose objects are hosted by "builtin/fsck.c".
These checks are obviously specific to the "loose" backend.
Move the logic into `odb_source_loose_fsck()`. Introduce a new "verbose"
flag so that we can properly retain semantics around whether or not we
want to print some status messages.
Note that this fixes a bug as a side effect: the progress meter was
captured in the callback data before `start_progress()` was even called,
so the per-subdirectory progress updates always operated on a NULL
pointer and the meter jumped straight from 0 to 256 upon completion. The
new code only sets up the callback data's progress meter after it has
been created, so the progress display now advances incrementally again.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for multi-pack indexes are hosted in `cmd_fsck()` directly.
These checks are obviously specific to the "packed" backend.
Move the logic into `odb_source_packed_fsck()`. As in preceding commits,
this means that we now properly honor both "--connectivity-only" and
"--no-full". Furthermore, we drop the dedicated `ERROR_MULTI_PACK_INDEX`
bit and instead use the generic `ERROR_OBJECT` bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for bitmaps live in `verify_bitmap_files()`, which is called
by "builtin/fsck.c". These checks are obviously specific to the "packed"
backend.
Move the logic into `odb_source_packed_fsck()`. As in preceding commits,
this means that we now properly honor both "--connectivity-only" and
"--no-full". Furthermore, we drop the dedicated `ERROR_BITMAP` bit and
instead use the generic `ERROR_OBJECT` bit.
Note that this change also adapts `verify_bitmap_files()` to be
focused on a single "packed" source instead of verifying bitmaps from
all sources. This change is required as we already know to loop around
the sources in `odb_fsck()` itself.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checks for reverse indexes live in `check_pack_rev_indexes()`, which
is hosted in "builtin/fsck.c". These checks are obviously specific to
the "packed" backend.
Move the logic into `odb_source_packed_fsck()`. As in the preceding
commit, drop the dedicated `ERROR_PACK_REV_INDEX` bit and instead use
the generic `ERROR_OBJECT` bit.
Note that this changes behaviour in two ways:
- The checks are now skipped when "--connectivity-only" was passed.
This is because we don't even run `odb_fsck()` at all when that
flag has been passed by the user, and not verifying data structures
of the object database matches the documented intent of that flag,
which is to only check the connectivity of reachable objects.
- The checks are now skipped for non-local sources when "--no-full"
was passed. This is, again, in line with the documented intent of
that flag.
Add a test to cast these semantics into stone.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the packfile verification out of `cmd_fsck()` and into the "packed"
source. While doing so, thread the progress meter and object callback
through the newly introduced `struct odb_fsck_options` so that the
caller's preferences are honoured without exposing those details at the
"builtin/fsck.c" level.
Note that the old code reported failures when verifying packfiles with
the `ERROR_PACK` bit, which gets returned to the caller via the exit
code. This bit is neither exercised in our test suite nor is it
documented anywhere in our codebase. Furthermore, this bit is highly
specific to the object storage backend, which makes it a bad fit for the
new pluggable infrastructure. So instead of retaining these semantics,
we drop them and return the generic `ERROR_OBJECT` bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The on-disk consistency checks in git-fsck(1) are conceptually
backend-specific: while connectivity checks and object-level parsing
checks are generic, verifying the physical integrity of packfiles and
loose objects is meaningful only to backends that use these formats:
Having these checks live in "builtin/fsck.c" violates that layering,
because it forces the command to reach directly into format-specific
internals.
Provide new infrastructure to make these format-specific checks
pluggable and implement stubs for the different source types we already
have. In subsequent commits we'll move functionality over piece by
piece.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
According to git-fsck(1), the "--full" option behaves in the following
way:
Check not just objects in GIT_OBJECT_DIRECTORY ($GIT_DIR/objects), but
also the ones found in alternate object pools listed in
GIT_ALTERNATE_OBJECT_DIRECTORIES or $GIT_DIR/objects/info/alternates,
and in packed Git archives found in $GIT_DIR/objects/pack and
corresponding pack subdirectories in alternate object pools.
So ultimately, it is supposed to control two things: (1) whether we only
check the main object directory, and (2) whether we check packfiles.
In its current state though, the flag only controls whether we check
packfiles or not, and if so we verify packfiles of all attached sources.
But we also have checks for loose objects in git-fsck(1), and here we
unconditionally check them in all sources.
The flag is arguably conflating two unrelated concerns with one another,
and it really should be split up into two flags: one that controls how
thorough we want to check individual sources, and one that controls
which sources we want to check in the first place. So ideally, we would
have:
- "--include-alternates": check all sources, not only the local one.
- "--include-optimized-objects": check not only loose objects, but
also those that have been packed. Note that we explicitly don't say
"--include-packed-objects" here to be more backend-agnostic.
- "--full": implies both of the above flags.
This feels out of scope for this series though. So for now, simply fix
the code by honoring locality of the sources for loose objects.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In subsequent commits we're about to rework some of the option handling
in git-fsck(1) a bit. It is currently a bit of a mess though due to lots
of global state that makes it hard to see which flags are used where
exactly.
Refactor the code by moving the fsck options into `cmd_fsck()`. This
allows us to convert some of the options into function-local variables.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The interfaces of the functions `fsck_obj()` and `fsck_obj_buffer()` are
somewhat similar to one another. The only difference between those two
is that `fsck_obj()` takes an already-parsed object as input, whereas
`fsck_obj_buffer()` parses the buffer and then calls `fsck_obj()`.
Furthermore, `fsck_obj()` has no callers other than `fsck_obj_buffer()`.
Refactor the code by merging those two functions. This makes it obvious
which function does what, and it allows us to get rid of the early
return in `fsck_obj()` in case `SEEN` is set as the only caller
unconditionally clears that bit before calling it anyway.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When checking loose objects we manually parse the object buffer we have
read from the on-disk file, mark the object and then call `fsck_obj()`.
The exact same steps are also performed by `fsck_obj_buffer()`.
Stop open-coding this logic and call `fsck_obj_buffer()` instead.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can tell git-multi-pack-index(1) to access multi-pack indices that
are stored in a different object directory via the "--object-dir="
option. This allows them to for example write or verify a multi-pack
index other than the one located in the main object directory in case a
repository has alternates with multiple multi-pack indices.
But while the documentation explicitly points out that the specified
object directory must be an alternate of the current repository, we
never verify that property. Instead, starting with 017db7bb14 (midx:
load multi-pack indices via their source, 2025-08-11), we now construct
an ad-hoc source and link it to the main object directory.
Besides contradicting the documentation, it's dubious that this really
ought to work in the first place: creating a multi-pack index (and
potentially a bitmap) for a completely foreign object directory is of
questionable value, as bitmap commit selection operates on the invoking
repository's refs. Furthermore, this is the only remaining caller
outside of our test helpers that constructs an ad-hoc source and links
it to the database, and we want to get rid of this mechanism as part of
this series.
Stop constructing the ad-hoc source and instead refuse the operation.
While this results in a change in behaviour, this restriction has been
documented as such ever since f57a739691 (midx: avoid opening multiple
MIDXs when writing, 2021-09-01).
Note that this change requires us to adapt one test chain in t5319, as
it creates an object directory that is not connected to any repository
and then uses it via "--object-dir=". The setup itself already documents
this and does the necessary gymnastics to link the object directory to a
temporary repository, but subsequent tests don't. Adapt those tests to
retain and reuse the temporary repository.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Same as with the preceding commit, git-grep(1) registers each
submodule's object database as an in-memory source of the main object
database before grepping it. This was introduced as an eager alternate
registration and converted into the lazy mechanism via 8d33c3af0b (grep:
use submodule-ODB-as-alternate lazy-addition, 2021-08-16).
Starting with 0693806bf8 (grep: add repository to OID grep sources,
2021-08-16), the command instead knows to pass submodule repositories to
our workers, which means that those now use that repository to look up
objects, too. As a consequence, registering submodule sources as
alternates is not required anymore.
Remove the logic to register submodule sources. Unfortunately, this does
not allow us to get rid of the object read lock as initializing the
subrepository is still racy.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Several functions in the submodule-config subsystem implicitly depend
on `the_repository`. Refactor these to take a `struct repository` as
parameter and adapt callers accordingly.
Note that as usual with these refactorings, callers simply pass
`the_repository` even if they already have a different repository
available in the calling context. This simplifies the migration and
ensures that we don't have a change in behaviour.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `cache_tree_fully_valid()` verifies whether the cache tree
owned by the index is valid or not. As part of that, the function checks
whether the objects referenced by the cache all exist. But because the
function has no repository available, it is using the object database of
`the_repository` instead.
We could of course adapt callers to pass in a repository as parameter
explicitly to get rid of this implicit dependency on global state. But
all of them pass the cache tree owned by a `struct index_state`, and
that structure already has a reference to its owning repository.
So instead, adapt the function to accept a `struct index_state`, which
ensures that callers will implicitly always pass the correct repository.
Adapt callers accordingly.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When creating a repository with alternates we first initialize the
object database and then write alternates to it in a separate step. This
is unfortunate due to a couple of reasons:
- It requires us to have a `write_alternates()` callback, which is
unfortunate as we never even write alternates to an object database
after it has been created.
- We're about to make alternates an implementation detail of the
object database's backend in a future patch series, so having this
callback is suboptimal there.
- The backend has more flexibility with how exactly alternates are
configured when it itself is in full control over their setup at the
time where it creates the object database itself.
We have thus introduced the ability to write alternates right at
creation time in the preceding commits, and we have unified setup of
alternates into a single location. All that's left to do for us now is
to wire up alternates as an option for the database creation.
Do so.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar as in the preceding commit, move the setup of alternates for
local clones with "--no-shared" into `collect_alternates()`. With this
step, the complete setup of alternates is now handled by that function.
Note that besides moving stuff around, it also fixes a bug: previously,
we did not know to resolve the referenced repository's common directory.
Consequently, when referencing a worktree we failed to resolve
alternates. But as `collect_alternates()` already knows to resolve the
commondir for "--local" we can simply reuse this resolved path for our
purpose.
Add two tests, the first one of which exercises this bug to avoid future
regressions. The second test ensures that we properly handle relative
alternates for a referenced worktree.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning a local repository with "--shared" we add that repository
to the new repository's alternates. This is done in `clone_local()`,
which is responsible for performing local clones.
Move the logic into `collect_alternates()` to unify our setup of
alternates. Furthermore, this will allow us to set up alternates right
at creation time of the object database.
Note that the logic for cloning a local repository with "--no-shared" is
not yet part of `collect_alternates()`. This will be handled in the next
commit, but means that at this step, we may compute `commondir` without
it being used. It will become used in the next step though.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can pass "--reference{,-if-able}" to git-clone(1) to instruct it
to set up alternates for the newly created repository. This allows it to
reuse objects from the source repository so that in the best case we
don't have to clone all objects over.
Those options are handled by the confusingly named `setup_reference()`
function -- without the above context, one might rightfully believe that
it was about refs, not about alternates. The function itself is rather
simple: we loop through all provided alternate paths and then, if such
an alternate is valid, we write it to the object database.
In subsequent commits we're about to consolidate the complete setup of
alternates into this function, and furthermore we'll refactor the setup
of the object database to handle doing this for us instead of writing
the alternates into it one by one.
Prepare for this refactoring by collecting the alternates into a strvec.
Rename the function to `collect_alternates()` to clarify its scope, as
it does not set up the references itself anymore.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a subsequent commit, `setup_reference()` will start to call
`copy_alternates()`. Prepare for this by moving the function further
down so that we can avoid adding a declaration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning a repository we defer initialization of the reference
database. This is because we don't yet know all details required for us
to initialize the refdb in the first place. Most importantly, what we
are missing is information about the object hash.
We don't do the same thing for the object database yet, but here we
essentially have the same problem. While the "files" database does not
need any information about the object format at creation time, alternate
backends are likely to require that information so that they can
properly set up their data structures.
Besides this forward-looking future proofing though, we also have a
second use case for deferring initialization of the object database,
namely alternates. When initializing the object database we do not yet
know whether we'll need alternates or not because this depends on the
repository we're about to clone from. If it is a local repository and
the user has passed "--reference{,-if-able}", then we will end up
writing alternates into the object database.
The ugly part though is that we cannot determine where the repository is
getting cloned from before it has been initialized. While we of course
already have access to the user-provided URI, that URI can be very well
rewritten via "url.<base>.insteadOf". We can of course read the global-
and system-level configuration to resolve it. But we explicitly resolve
the URI a second time after we have initialized the repository because
it can happen that we copy a ".git/config" over from our templates, and
that file may cause us to rewrite the path.
In a subsequent commit though we'll start to write alternates as part of
the repository initialization, so we'll need to have the URI properly
resolved before we can initialize the object database. This is ugly, but
as mentioned above it makes sense for us to defer its initialization
anyway so that we also know about the object hash already.
Defer creation of the object database until after we have resolved the
URI.
Note that this also requires us to defer the call to `setup_reference()`
until after we have created the object database. While you might think
that this function has something to do with references ("refs/*"), it is
in fact responsible for setting up alternates. Consequently, we can only
call it after we have created the object database already.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `init_db()` is responsible for creating the on-disk
directory structure required for a Git repository. It is used by both
git-init(1) and git-clone(1), and because their expected behaviour is
different we support a couple of flags:
- The `QUIET` flag controls whether the command is quiet or not. For
git-init(1) this is user-controllable, whereas for git-clone(1)
we're always quiet.
- The `EXIST_OK` flag controls whether a preexisting repository is
okay or not. For git-init(1) it is, for git-clone(1) it's not.
- The `SKIP_REFDB` flag controls whether the reference database should
already be created or not. For git-init(1) we do, but for
git-clone(1) we don't because it does not yet know about the default
branch and about the remote object hash.
Furthermore, we're about to add another divergence in behaviour, where
we have to also skip creation of the object database in git-clone(1).
This is becoming quite cumbersome though.
Instead of introducing another flag, start to split up concerns of the
function so that we never create the reference or object database. This
becomes the responsibility of the caller, which is thus free to defer
their creation to a later point in time. This lets us get rid of most of
the divergent behaviour:
- We don't need the `SKIP_REFDB` and a potential `SKIP_ODB` flags
anymore.
- We don't need the `QUIET` flag anymore, as nothing prints output
except for the final status message that tells the user that the
repository has been (re)initialized. But as this message is specific
to git-init(1), we can easily move it there.
The only piece of information we still have to convey is whether or not
reinitialization of a preexisting repository is okay. This is handled by
a new `reinit_ok` pointer that, if non-`NULL`, indicates that it is okay
to reinitialize the repository. Furthermore, the pointer will be written
to to indicate whether the repository was reinitialized or not, which we
need in git-init(1) to print the correct initialization message.
With these refactorings, `init_db()` is named quite misleadingly though,
as we don't create any of the reference or object databases anymore.
Rename it to `create_repository()`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Running "git history" in a corrupt repository can (unsurprisingly)
segfault when a necessary tree object is not found.
* jc/history-missing-tree-errorfix:
history: do not dereference NULL when parent tree is missing
Reference storage backends can be configured with a payload via the
"extensions.refStorage" config key and the "GIT_REF_STORAGE_FORMAT"
environment variable, both of which accept a URI in the format
"<format>://<payload>". The payload may contain backend-specific
information, for example an alternate refs directory or which database
references should be stored in.
The `--ref-storage-format=` option of git-init(1) and git-clone(1) does
not know about payloads though: its value is parsed as a plain format
name, so backends that require a payload cannot be conveniently set up
at initialization time via the command line.
Teach the option to accept the same URI syntax. Also, document the
optional payloads for both the "files" and "reftable" backends.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the same reasoning as for git-init(1), rename "--show-ref-format"
to "--show-ref-storage-format" and keep the old name as an alias.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the same reasoning as for git-init(1), rename "--ref-format=" to
"--ref-storage-format=" and keep the old name as an alias.
Note that this commit is a bit more complex compared to the others as we
also need to adapt the submodule helper for consistency. But overall,
the changes are straight-forward and in the same spirit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the same reasoning as for git-init(1), rename "--ref-format=" to
"--ref-storage-format=" and keep the old name as an alias.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the same reasoning as for git-init(1), rename "--ref-format=" to
"--ref-storage-format=" and keep the old name as an alias.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Back when we gained support for reftables we of course introduced the
ability to control the reference storage format that is used by newly
created repositories. This infrastructure has grown over time, and
unfortunately without consistency:
- The command line parameter to specify the ref storage format is
called "--ref-format=", while the corresponding repository extension
is called "refStorage".
- In most cases we refer to the "ref storage format" in our docs, so
calling it "--ref-format=" is being inconsistent with them.
- It is possible to override the ref storage format via an environment
variable that is called "GIT_REFERENCE_BACKEND", which is not even
remotely consistent with anything else.
- There is also an "object format", but that format does not control
how we store objects but rather whether we use SHA1 or SHA256.
So in summary, it's a huge mess.
This problem is about to become even worse though, as we're soon going
to introduce an object storage extension. This extension is the
equivalent to the ref storage extension, and of course we also want
users to be able to control which object storage format new repositories
are using. But we cannot properly name that parameter without creating
even more inconsistencies:
- "--object-format=" would match "--ref-format=", but that parameter
name is already taken to specify the hash function.
- "--object-storage=" would be a good fit, but be inconsistent with
"--ref-format=". Asking the user to execute `git init --ref-format=
--object-storage=` just feels extremely awkward.
Instead, this and subsequent patches will fix the mess by consistently
referring to the ref storage format as such throughout all options,
environment variables and config settings. This new name much more
closely indicates that it is about how we store data and finally brings
consistency into this area. We will keep the old names working of course
for the sake of backwards compatibility.
Start with git-init(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Open the editor by default when squashing and provide --no-edit as the
opt-out. Record the exact commits selected by the revision walk,
rearrange that todo list with the sequencer's autosquash machinery, and
build the message template from the resulting order.
Match interactive rebase's treatment of marker messages: comment out
fixup! messages, retain squash! bodies, and let amend! replace its target
unless a preceding squash! requires both bodies. This keeps message
editing aligned with the marker validation used by the no-edit path.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create one replacement commit from the resolved range when --no-edit is
selected. Preserve the authorship and all parents of the oldest commit,
use the tip tree, and replay descendants through the existing history
rewrite machinery. Record the complete revision expression in the
reflog and retain dry-run and update-refs behavior.
Resolve fixup!, squash! and amend! subjects while walking the range.
Reject markers whose targets are not selected and refuse any no-edit
fold that would discard a squash! or amend! message. A range made
entirely from related markers can still be consolidated, with the last
applicable amend! body supplying the message.
Inspired-by: Sergey Chernov <serega.morph@gmail.com>
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A local branch that descends from the selected graph without containing
its tip cannot be replayed as a descendant of the squashed commit. Find
those branches with ref-filter before creating any replacement objects
and refuse the operation unless --update-refs=head was requested.
Limit this protection to local branches, matching the refs that the
default history rewrite mode updates; tags and remote-tracking refs
remain untouched. Sort the blocking refs and print their short branch
names so the user can decide whether to move them or leave them behind.
Add advice.historyUpdateRefs for the hint that points to
--update-refs=head.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Walk the selected commits in topological order from oldest to newest and
mark each one as it is seen. Every parent after the oldest commit must
already be selected or also be a parent of the oldest commit. This
accepts merges contained by the range while rejecting a merge arm that
entered it from elsewhere.
Track the remaining graph tips during the same walk and require exactly
one. Also reject empty and single-commit ranges and any selection that
reaches a root commit. These checks identify the oldest commit whose
parents will be preserved and the single tip whose tree will be used by
the rewrite.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the entry point and option parsing for "git history squash". Pass
the remaining arguments through setup_revisions() so the command accepts
revision ranges and rev-list options, while restoring the ordering and
simplification settings required by the fold if an option changes them.
Require at least one BOTTOM revision. The squashed commit needs a commit
outside the selected range to serve as its base, so a single positive
revision is not a sufficient range.
Keep this step limited to defining the revision input contract so graph
validation and the rewrite can be added independently.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit_tree_ext() reuses the message of the commit it is handed. A
caller that folds several commits together wants to seed the message
from more than that single commit, so add an optional message_template
parameter. When NULL, the behavior is unchanged.
Pass NULL from the existing fixup and split callers.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Three places resolve the tree of a commit's first parent, falling back
to the empty tree for a root commit, each repeating the same parse and
oidcpy dance. Extract a first_parent_tree_oid() helper and route the
existing callers through it.
No change in behavior.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git rev-list command has been augmented with a '--missing-only'
option that filters the output to only show missing objects,
stripping the leading '?' character and suppressing present objects,
which is useful when used in combination with '--missing=print' or
'--missing=print-info'.
* sa/rev-list-missing-only:
rev-list: add --missing-only option to filter output
Junio C Hamano