Commit Graph

150 Commits (afa4ea56fedc3ca04d3b00a3ef930833e7ad68ea)

Author SHA1 Message Date
Jeff King 9b204b825b hash: use git_hash_init() consistently
We'd like to add more logic to git_hash_init(), but many callers skip it
and call algop->init_fn() directly. Let's make sure we're consistently
using the wrapper by adding a coccinelle rule.

Besides the coccinelle file itself, this is a purely mechanical
conversion based on the patch it generates. There should be no bare
init_fn() calls left (except for the one in the wrapper).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-07 21:56:00 -07:00
Junio C Hamano f08ece0e2c Merge branch 'jk/hash-algo-leak-fixes' into jk/git-hash-cleanups
* jk/hash-algo-leak-fixes:
  hash: add platform-specific discard functions
  hash: fix memory leak copying sha256 gcrypt handles
  http: discard hash in dumb-http http_object_request
  check_stream_oid(): discard hash on read error
  patch-id: discard hash when done
  csum-file: provide a function to release checkpoints
  csum-file: always finalize or discard hash
  hash: add discard primitive
  csum-file: drop discard_hashfile()
2026-07-07 10:52:48 -07:00
Jeff King 46ba44e1fd csum-file: provide a function to release checkpoints
A hashfile_checkpoint struct is basically just a copy of the hash_ctx
state at a given point in the file. As such, it contains its own
git_hash_ctx which may (depending on the underlying hash implementation)
need to be discarded when we're done with it.

Let's add a "release" function which cleans up the hash context it
holds. I chose "release" here and not "discard" because you'd use this
to clean up every checkpoint, whether you used it or not. As opposed to
git_hash_discard(), which is needed only if you didn't call
git_hash_final().

There are only two callers which use hashfile_checkpoints, and we can
add release calls to both. When built with "SANITIZE=leak
OPENSSL_SHA1_UNSAFE=1", this makes both t1050 and t9300 leak-free.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-02 09:50:47 -07:00
Junio C Hamano 02bb39c5cb Merge branch 'js/objects-larger-than-4gb-on-windows-more'
* js/objects-larger-than-4gb-on-windows-more:
  odb: use size_t for object_info.sizep and the size APIs
  packfile,delta: drop the `cast_size_t_to_ulong()` wrappers
  pack-objects: use size_t for in-core object sizes
  packfile: widen unpack_entry()'s size out-parameter to size_t
  pack-objects(check_pack_inflate()): use size_t instead of unsigned long
  patch-delta: use size_t for sizes
  compat/msvc: use _chsize_s for ftruncate
2026-06-21 16:41:38 -07:00
Johannes Schindelin c6a4629e32 odb: use size_t for object_info.sizep and the size APIs
When `js/objects-larger-than-4gb-on-windows` widened the streaming,
index-pack and unpack-objects code paths, in the interest of keeping the
patches somewhat reasonably-sized, it left the public ODB API still
typed in `unsigned long`. In particular `struct object_info::sizep` and
the four wrappers built on top of it (`odb_read_object`,
`odb_read_object_peeled`, `odb_read_object_info`, `odb_pretend_object`)
still return the unpacked size through `unsigned long *`, so on Windows
`cat-file -s` and the `git add` / `git status` paths for a >4 GiB blob
silently cap at 4 GiB.

Widen the field and the four wrappers. The previous commits already
widened the `unpack_entry()` cascade and pack-objects' in-core size
accessors, so most of the cascade arrives here with no further work: the
temporary shims in `packed_object_info_with_index_pos()` and in
`unpack_entry()`'s delta-base recovery path go away, the two
`SET_SIZE(entry, cast_size_t_to_ulong(canonical_size))` calls in
`check_object()` and the matching one in `drop_reused_delta()` collapse
to plain `SET_SIZE`, and `oe_get_size_slow()`'s tail
`cast_size_t_to_ulong()` is gone too.

What remains narrow are the boundaries this series does not
intend to touch: the diff, blame, textconv and fast-import machinery.

Even so, this patch is unfortunately quite large.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-15 07:45:41 -07:00
Johannes Schindelin 2d83cc3f84 packfile: widen unpack_entry()'s size out-parameter to size_t
The topic `js/objects-larger-than-4gb-on-windows` widened the streaming,
index-pack and unpack-objects paths to `size_t` but deliberately stopped
at the in-memory `unpack_entry()` cascade, which still hands back the
unpacked size through `unsigned long *`.  On Windows that boundary
truncates above 4 GiB because that data type is only 32 bits wide on
that platform.

Widen the code path. Except `packed_object_info_with_index_pos()`: It
cannot yet pass `oi->sizep` directly because the field is still
`unsigned long *`; bridge it with a `size_t` temporary that narrows
back, and let a later commit drop the bridge once the field is wide
too. `gfi_unpack_entry()` keeps its narrow signature because fast-import
tracks sizes through `unsigned long` everywhere it crosses subsystem
boundaries, keeping its signature allows the scope of this commit to be
somewhat reasonable, still.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-15 07:45:40 -07:00
Olamide Caleb Bello 8cd7402acc environment: move "pack_compression_level" into `struct repo_config_values`
The `pack_compression_level` configuration is currently stored in the
global variable `pack_compression_level`, which makes it shared across
repository instances within a single process.

Store it instead in `repo_config_values`, where eagerly‑parsed
repository configuration lives. `pack_compression_level` is parsed
eagerly because it influences packfile compression, a core operation
where a lazy parse could cause inconsistent behavior and hamper
libification. This preserves the existing eager‑parsing behavior while
tying the value to the repository from which it was read, avoiding
cross‑repository state leakage and continuing the effort to reduce
reliance on global configuration state.

Update all references to use `repo_config_values()`.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Usman Akinyemi <usmanakinyemi202@gmail.com>
Signed-off-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-03 08:36:48 +09:00
Junio C Hamano f1edda9bfb Merge branch 'jt/fast-import-signed-modes'
Handling of signed commits and tags in fast-import has been made more
configurable.

* jt/fast-import-signed-modes:
  fast-import: add 'abort-if-invalid' mode to '--signed-tags=<mode>'
  fast-import: add 'sign-if-invalid' mode to '--signed-tags=<mode>'
  fast-import: add 'strip-if-invalid' mode to '--signed-tags=<mode>'
  fast-import: add 'abort-if-invalid' mode to '--signed-commits=<mode>'
  fast-export: check for unsupported signing modes earlier
2026-04-07 14:59:27 -07:00
Junio C Hamano 0a39ec283c Merge branch 'vp/http-rate-limit-retries'
The HTTP transport learned to react to "429 Too Many Requests".

* vp/http-rate-limit-retries:
  http: add support for HTTP 429 rate limit retries
  strbuf_attach: fix call sites to pass correct alloc
  strbuf: pass correct alloc to strbuf_attach() in strbuf_reencode()
2026-04-01 10:28:18 -07:00
Justin Tobler ddd7c7ab12 fast-import: add 'abort-if-invalid' mode to '--signed-tags=<mode>'
In git-fast-import(1), the 'abort-if-invalid' mode for the
'--signed-commits' option verifies commit signatures during import and
aborts the entire operation when verification fails. Extend the same
behavior to signed tag objects by introducing an 'abort-if-invalid' mode
for the '--signed-tags' option.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-26 12:42:58 -07:00
Justin Tobler 2b1546c03c fast-import: add 'sign-if-invalid' mode to '--signed-tags=<mode>'
With ee66c793f8 (fast-import: add mode to sign commits with invalid
signatures, 2026-03-12), git-fast-import(1) learned to verify commit
signatures during import and replace signatures that fail verification
with a newly generated one. Extend the same behavior to signed tag
objects by introducing a 'sign-if-invalid' mode for the '--signed-tags'
option.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-26 12:42:58 -07:00
Justin Tobler 817b042879 fast-import: add 'strip-if-invalid' mode to '--signed-tags=<mode>'
With c20f112e51 (fast-import: add 'strip-if-invalid' mode to
--signed-commits=<mode>, 2025-11-17), git-fast-import(1) learned to
verify commit signatures during import and strip signatures that fail
verification. Extend the same behavior to signed tag objects by
introducing a 'strip-if-invalid' mode for the '--signed-tags' option.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-26 12:42:57 -07:00
Justin Tobler 4c36345e04 fast-import: add 'abort-if-invalid' mode to '--signed-commits=<mode>'
The '--signed-commits=<mode>' option for git-fast-import(1) configures
how signed commits are handled when encountered. In cases where an
invalid commit signature is encountered, a user may wish to abort the
operation entirely. Introduce an 'abort-if-invalid' mode to do so.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-26 12:42:57 -07:00
Junio C Hamano 410ab9a257 Merge branch 'jt/fast-import-sign-again'
"git fast-import" learned to optionally replace signature on
commits whose signatures get invalidated due to replaying by
signing afresh.

* jt/fast-import-sign-again:
  fast-import: add mode to sign commits with invalid signatures
  gpg-interface: allow sign_buffer() to use default signing key
  commit: remove unused forward declaration
2026-03-24 12:31:31 -07:00
Vaidas Pilkauskas a4fddb01c5 strbuf_attach: fix call sites to pass correct alloc
strbuf_attach(sb, buf, len, alloc) requires alloc > len (the buffer
must have at least len+1 bytes to hold the NUL). Several call sites
passed alloc == len, relying on strbuf_grow(sb, 0) inside strbuf_attach
to reallocate. Fix these in mailinfo, am, refs/files-backend,
fast-import, and trailer by passing len+1 when the buffer is a
NUL-terminated string (or from strbuf_detach).

Signed-off-by: Vaidas Pilkauskas <vaidas.pilkauskas@shopify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-17 09:14:19 -07:00
Justin Tobler ee66c793f8 fast-import: add mode to sign commits with invalid signatures
With git-fast-import(1), handling of signed commits is controlled via
the `--signed-commits=<mode>` option. When an invalid signature is
encountered, a user may want the option to sign the commit again as
opposed to just stripping the signature. To facilitate this, introduce a
"sign-if-invalid" mode for the `--signed-commits` option. Optionally, a
key ID may be explicitly provided in the form
`sign-if-invalid[=<keyid>]` to specify which signing key should be used
when signing invalid commit signatures.

Note that to properly support interoperability mode when signing commit
signatures, the commit buffer must be created in both the repository and
compatability object formats to generate the appropriate signatures
accordingly. As currently implemented, the commit buffer for the
compatability object format is not reconstructed and thus signing
commits in interoperability mode is not yet supported. Support may be
added in the future.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-12 21:28:20 -07:00
Patrick Steinhardt d9ecf268ef odb: embed base source in the "files" backend
The "files" backend is implemented as a pointer in the `struct
odb_source`. This contradicts our typical pattern for pluggable backends
like we use it for example in the ref store or for object database
streams, where we typically embed the generic base structure in the
specialized implementation. This pattern has a couple of small benefits:

  - We avoid an extra allocation.

  - We hide implementation details in the generic structure.

  - We can easily downcast from a generic backend to the specialized
    structure and vice versa because the offsets are known at compile
    time.

  - It becomes trivial to identify locations where we depend on backend
    specific logic because the cast needs to be explicit.

Refactor our "files" object database source to do the same and embed the
`struct odb_source` in the `struct odb_source_files`.

There are still a bunch of sites in our code base where we do have to
access internals of the "files" backend. The intent is that those will
go away over time, but this will certainly take a while. Meanwhile,
provide a `odb_source_files_downcast()` function that can convert a
generic source into a "files" source.

As we only have a single source the downcast succeeds unconditionally
for now. Eventually though the intent is to make the cast `BUG()` in
case the caller requests to downcast a non-"files" backend to a "files"
backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-05 11:45:15 -08:00
Patrick Steinhardt cb506a8a69 odb: introduce "files" source
Introduce a new "files" object database source. This source encapsulates
access to both loose object files and the packfile store, similar to how
the "files" backend for refs encapsulates access to loose refs and the
packed-refs file.

Note that for now the "files" source is still a direct member of a
`struct odb_source`. This architecture will be reversed in the next
commit so that the files source contains a `struct odb_source`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-05 11:45:14 -08:00
Patrick Steinhardt 84f0e60b28 packfile: move packfile store into object source
The packfile store is a member of `struct object_database`, which means
that we have a single store per database. This doesn't really make much
sense though: each source connected to the database has its own set of
packfiles, so there is a conceptual mismatch here. This hasn't really
caused much of a problem in the past, but with the advent of pluggable
object databases this is becoming more of a problem because some of the
sources may not even use packfiles in the first place.

Move the packfile store down by one level from the object database into
the object database source. This ensures that each source now has its
own packfile store, and we can eventually start to abstract it away
entirely so that the caller doesn't even know what kind of store it
uses.

Note that we only need to adjust a relatively small number of callers,
way less than one might expect. This is because most callers are using
`repo_for_each_pack()`, which handles enumeration of all packfiles that
exist in the repository. So for now, none of these callers need to be
adapted. The remaining callers that iterate through the packfiles
directly and that need adjustment are those that are a bit more tangled
with packfiles. These will be adjusted over time.

Note that this patch only moves the packfile store, and there is still a
bunch of functions that seemingly operate on a packfile store but that
end up iterating over all sources. These will be adjusted in subsequent
commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-09 06:40:07 -08:00
Junio C Hamano 1b40ddc1a5 Merge branch 'cc/fast-import-strip-if-invalid'
"git fast-import" learns "--strip-if-invalid" option to drop
invalid cryptographic signature from objects.

* cc/fast-import-strip-if-invalid:
  fast-import: add 'strip-if-invalid' mode to --signed-commits=<mode>
  commit: refactor verify_commit_buffer()
  fast-import: refactor finalize_commit_buffer()
2025-12-05 14:49:58 +09:00
Christian Couder c20f112e51 fast-import: add 'strip-if-invalid' mode to --signed-commits=<mode>
Tools like `git filter-repo`[1] use `git fast-export` and
`git fast-import` to rewrite repository history. When rewriting
history using one such tool though, commit signatures might become
invalid because the commits they sign changed due to the changes
in the repository history made by the tool between the fast-export
and the fast-import steps.

Note that as far as signature handling goes:

  * Since fast-export doesn't know what changes filter-repo may make
to the stream, it can't know whether the signatures will still be
valid.

  * Since filter-repo doesn't know what history canonicalizations
fast-export performed (and it performs a few), it can't know whether
the signatures will still be valid.

  * Therefore, fast-import is the only process in the pipeline that
can know whether a specified signature remains valid.

Having invalid signatures in a rewritten repository could be
confusing, so users rewritting history might prefer to simply
discard signatures that are invalid at the fast-import step.

For example a common use case is to rewrite only "recent" history.
While specifying commit ranges corresponding to "recent" commits
could work, users worry about getting it wrong and want to just
automatically rewrite everything, expecting older commit signatures
to be untouched.

To let them do that, let's add a new 'strip-if-invalid' mode to the
`--signed-commits=<mode>` option of `git fast-import`.

It would be interesting for the `--signed-tags=<mode>` option to
have this mode too, but we leave that for a future improvement.

It might also be possible for `git fast-export` to have such a mode
in its `--signed-commits=<mode>` and `--signed-tags=<mode>`
options, but the use cases for it are much less clear, so we also
leave that for possible future improvements.

For now let's just die() if 'strip-if-invalid' is passed to these
options where it hasn't been implemented yet.

[1]: https://github.com/newren/git-filter-repo

Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-26 08:43:44 -08:00
Junio C Hamano 7a75e549b2 Merge branch 'ps/packed-git-in-object-store'
The list of packfiles used in a running Git process is moved from
the packed_git structure into the packfile store.

* ps/packed-git-in-object-store:
  packfile: track packs via the MRU list exclusively
  packfile: always add packfiles to MRU when adding a pack
  packfile: move list of packs into the packfile store
  builtin/pack-objects: simplify logic to find kept or nonlocal objects
  packfile: fix approximation of object counts
  http: refactor subsystem to use `packfile_list`s
  packfile: move the MRU list into the packfile store
  packfile: use a `strmap` to store packs by name
2025-11-19 10:55:37 -08:00
Christian Couder 388517c14c fast-import: refactor finalize_commit_buffer()
In a following commit we are going to finalize commit buffers with or
without signatures in order to check the signatures and possibly drop
them.

To do so easily and without duplication, let's refactor the current
code that finalizes commit buffers into a new finalize_commit_buffer()
function.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-16 20:56:35 -08:00
Junio C Hamano e569dced68 Merge branch 'cc/fast-import-export-i18n-cleanup'
Messages from fast-import/export are now marked for i18n.

* cc/fast-import-export-i18n-cleanup:
  gpg-interface: mark a string for translation
  fast-import: mark strings for translation
  fast-export: mark strings for translation
  gpg-interface: use left shift to define GPG_VERIFY_*
  gpg-interface: simplify ssh fingerprint parsing
2025-11-06 15:17:01 -08:00
Junio C Hamano 5554738038 Merge branch 'ps/remove-packfile-store-get-packs'
Two slightly different ways to get at "all the packfiles" in API
has been cleaned up.

* ps/remove-packfile-store-get-packs:
  packfile: rename `packfile_store_get_all_packs()`
  packfile: introduce macro to iterate through packs
  packfile: drop `packfile_store_get_packs()`
  builtin/grep: simplify how we preload packs
  builtin/gc: convert to use `packfile_store_get_all_packs()`
  object-name: convert to use `packfile_store_get_all_packs()`
2025-10-30 08:00:19 -07:00
Patrick Steinhardt 589127caa7 packfile: move list of packs into the packfile store
Move the list of packs into the packfile store. This follows the same
logic as in a previous commit, where we moved the most-recently-used
list of packs, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-30 07:09:53 -07:00
Christian Couder c295115ec6 fast-import: mark strings for translation
Some error or warning messages in "builtin/fast-import.c" are marked
for translation, but many are not.

To be more consistent and provide a better experience to people using a
translated version, let's mark all the remaining error or warning
messages for translation.

While at it, let's make the following small changes:

  - replace "GIT" or "git" in a few error messages to just "Git",
  - replace "Expected from command, got %s" to "expected 'from'
    command, got '%s'", which makes it clearer that "from" is a command
    and should not be translated,
  - downcase error and warning messages that start with an uppercase,
  - fix test cases in "t9300-fast-import.sh" that broke because an
    error or warning message was downcased,
  - split error and warning messages that are too long,
  - adjust the indentation of some arguments of the error functions.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-30 07:06:58 -07:00
Junio C Hamano 3deb97fe24 Merge branch 'cc/fast-import-strip-signed-tags'
"git fast-import" is taught to handle signed tags, just like it
recently learned to handle signed commits, in different ways.

* cc/fast-import-strip-signed-tags:
  fast-import: add '--signed-tags=<mode>' option
  fast-export: handle all kinds of tag signatures
  t9350: properly count annotated tags
  lib-gpg: allow tests with GPGSM or GPGSSH prereq first
  doc: git-tag: stop focusing on GPG signed tags
2025-10-28 10:29:09 -07:00
Junio C Hamano ed3305fff7 Merge branch 'ps/remove-packfile-store-get-packs' into ps/packed-git-in-object-store
* ps/remove-packfile-store-get-packs: (55 commits)
  packfile: rename `packfile_store_get_all_packs()`
  packfile: introduce macro to iterate through packs
  packfile: drop `packfile_store_get_packs()`
  builtin/grep: simplify how we preload packs
  builtin/gc: convert to use `packfile_store_get_all_packs()`
  object-name: convert to use `packfile_store_get_all_packs()`
  builtin/repack.c: clean up unused `#include`s
  repack: move `write_cruft_pack()` out of the builtin
  repack: move `write_filtered_pack()` out of the builtin
  repack: move `pack_kept_objects` to `struct pack_objects_args`
  repack: move `finish_pack_objects_cmd()` out of the builtin
  builtin/repack.c: pass `write_pack_opts` to `finish_pack_objects_cmd()`
  repack: extract `write_pack_opts_is_local()`
  repack: move `find_pack_prefix()` out of the builtin
  builtin/repack.c: use `write_pack_opts` within `write_cruft_pack()`
  builtin/repack.c: introduce `struct write_pack_opts`
  repack: 'write_midx_included_packs' API from the builtin
  builtin/repack.c: inline packs within `write_midx_included_packs()`
  builtin/repack.c: pass `repack_write_midx_opts` to `midx_included_packs`
  builtin/repack.c: inline `remove_redundant_bitmaps()`
  ...
2025-10-28 10:00:56 -07:00
Patrick Steinhardt ecad863c12 packfile: rename `packfile_store_get_all_packs()`
In a preceding commit we have removed `packfile_store_get_packs()`. With
this function removed it's somewhat useless to still have the "all"
infix in `packfile_store_get_all_packs()`. Rename the latter to drop
that infix.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-16 14:42:40 -07:00
Christian Couder d8ce08aa13 fast-import: add '--signed-tags=<mode>' option
Recently, eaaddf5791 (fast-import: add '--signed-commits=<mode>'
option, 2025-09-17) added support for controlling how signed commits
are handled by `git fast-import`, but there is no option yet to
decide about signed tags.

To remediate that, let's add a '--signed-tags=<mode>' option to
`git fast-import` too.

With this, both `git fast-export` and `git fast-import` have both
a '--signed-tags=<mode>' and a '--signed-commits=<mode>' supporting
the same <mode>s.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-13 08:51:42 -07:00
Junio C Hamano 8c13c31404 Merge branch 'ps/packfile-store'
Code clean-up around the in-core list of all the pack files and
object database(s).

* ps/packfile-store:
  packfile: refactor `get_packed_git_mru()` to work on packfile store
  packfile: refactor `get_all_packs()` to work on packfile store
  packfile: refactor `get_packed_git()` to work on packfile store
  packfile: move `get_multi_pack_index()` into "midx.c"
  packfile: introduce function to load and add packfiles
  packfile: refactor `install_packed_git()` to work on packfile store
  packfile: split up responsibilities of `reprepare_packed_git()`
  packfile: refactor `prepare_packed_git()` to work on packfile store
  packfile: reorder functions to avoid function declaration
  odb: move kept cache into `struct packfile_store`
  odb: move MRU list of packfiles into `struct packfile_store`
  odb: move packfile map into `struct packfile_store`
  odb: move initialization bit into `struct packfile_store`
  odb: move list of packfiles into `struct packfile_store`
  packfile: introduce a new `struct packfile_store`
2025-10-07 12:25:27 -07:00
Patrick Steinhardt d2779beb36 packfile: refactor `get_all_packs()` to work on packfile store
The `get_all_packs()` function prepares the packfile store and then
returns its packfiles. Refactor it to accept a packfile store instead of
a repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-24 11:53:51 -07:00
Patrick Steinhardt d67530f6bb packfile: introduce function to load and add packfiles
We have a recurring pattern where we essentially perform an upsert of a
packfile in case it isn't yet known by the packfile store. The logic to
do so is non-trivial as we have to reconstruct the packfile's key, check
the map of packfiles, then create the new packfile and finally add it to
the store.

Introduce a new function that does this dance for us. Refactor callsites
to use it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-24 11:53:50 -07:00
Patrick Steinhardt f6f236d926 packfile: refactor `install_packed_git()` to work on packfile store
The `install_packed_git()` functions adds a packfile to a specific
object store. Refactor it to accept a packfile store instead of a
repository to clarify its scope.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-24 11:53:50 -07:00
Christian Couder eaaddf5791 fast-import: add '--signed-commits=<mode>' option
A '--signed-commits=<mode>' option is already available when using
`git fast-export` to decide what should be done at export time about
commit signatures. At import time though, there is no option, or
other way, in `git fast-import` to decide about commit signatures.

To remediate that, let's add a '--signed-commits=<mode>' option to
`git fast-import` too.

For now the supported <mode>s are the same as those supported by
`git fast-export`.

The code responsible for consuming a signature is refactored into
the import_one_signature() and discard_one_signature() functions,
which makes it easier to follow the logic and add new modes in the
future.

In the 'strip' and 'warn-strip' modes, we deliberately use
discard_one_signature() to discard the signature without parsing it.
This ensures that even malformed signatures, which would cause the
parser to fail, can be successfully stripped from a commit.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-17 11:18:28 -07:00
Junio C Hamano 4ce0caa7cc Merge branch 'ps/object-file-wo-the-repository'
Reduce implicit assumption and dependence on the_repository in the
object-file subsystem.

* ps/object-file-wo-the-repository:
  object-file: get rid of `the_repository` in index-related functions
  object-file: get rid of `the_repository` in `force_object_loose()`
  object-file: get rid of `the_repository` in `read_loose_object()`
  object-file: get rid of `the_repository` in loose object iterators
  object-file: remove declaration for `for_each_file_in_obj_subdir()`
  object-file: inline `for_each_loose_file_in_objdir_buf()`
  object-file: get rid of `the_repository` when writing objects
  odb: introduce `odb_write_object()`
  loose: write loose objects map via their source
  object-file: get rid of `the_repository` in `finalize_object_file()`
  object-file: get rid of `the_repository` in `loose_object_info()`
  object-file: get rid of `the_repository` when freshening objects
  object-file: inline `check_and_freshen()` functions
  object-file: get rid of `the_repository` in `has_loose_object()`
  object-file: stop using `the_hash_algo`
  object-file: fix -Wsign-compare warnings
2025-08-05 11:53:55 -07:00
Junio C Hamano 540aaa607c Merge branch 'ps/config-wo-the-repository'
The config API had a set of convenience wrapper functions that
implicitly use the_repository instance; they have been removed and
inlined at the calling sites.

* ps/config-wo-the-repository: (21 commits)
  config: fix sign comparison warnings
  config: move Git config parsing into "environment.c"
  config: remove unused `the_repository` wrappers
  config: drop `git_config_set_multivar()` wrapper
  config: drop `git_config_get_multivar_gently()` wrapper
  config: drop `git_config_set_multivar_in_file_gently()` wrapper
  config: drop `git_config_set_in_file_gently()` wrapper
  config: drop `git_config_set()` wrapper
  config: drop `git_config_set_gently()` wrapper
  config: drop `git_config_set_in_file()` wrapper
  config: drop `git_config_get_bool()` wrapper
  config: drop `git_config_get_ulong()` wrapper
  config: drop `git_config_get_int()` wrapper
  config: drop `git_config_get_string()` wrapper
  config: drop `git_config_get_string()` wrapper
  config: drop `git_config_get_string_multi()` wrapper
  config: drop `git_config_get_value()` wrapper
  config: drop `git_config_get_value()` wrapper
  config: drop `git_config_get()` wrapper
  config: drop `git_config_clear()` wrapper
  ...
2025-08-04 08:10:33 -07:00
Junio C Hamano 5216bcbc84 Merge branch 'cc/fast-import-export-signature-names'
Clean up the way how signature on commit objects are exported to
and imported from fast-import stream.

* cc/fast-import-export-signature-names:
  fast-(import|export): improve on commit signature output format
2025-07-23 15:45:16 -07:00
Patrick Steinhardt d57f078e37 config: drop `git_config_get_ulong()` wrapper
In 036876a106 (config: hide functions using `the_repository` by
default, 2024-08-13) we have moved around a bunch of functions in the
config subsystem that depend on `the_repository`. Those function have
been converted into mere wrappers around their equivalent function that
takes in a repository as parameter, and the intent was that we'll
eventually remove those wrappers to make the dependency on the global
repository variable explicit at the callsite.

Follow through with that intent and remove `git_config_get_ulong()`. All
callsites are adjusted so that they use
`repo_config_get_ulong(the_repository, ...)` instead. While some
callsites might already have a repository available, this mechanical
conversion is the exact same as the current situation and thus cannot
cause any regression. Those sites should eventually be cleaned up in a
later patch series.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-23 08:15:20 -07:00
Patrick Steinhardt 3fda14d86d config: drop `git_config_get_int()` wrapper
In 036876a106 (config: hide functions using `the_repository` by
default, 2024-08-13) we have moved around a bunch of functions in the
config subsystem that depend on `the_repository`. Those function have
been converted into mere wrappers around their equivalent function that
takes in a repository as parameter, and the intent was that we'll
eventually remove those wrappers to make the dependency on the global
repository variable explicit at the callsite.

Follow through with that intent and remove `git_config_get_int()`. All
callsites are adjusted so that they use
`repo_config_get_int(the_repository, ...)` instead. While some callsites
might already have a repository available, this mechanical conversion is
the exact same as the current situation and thus cannot cause any
regression. Those sites should eventually be cleaned up in a later patch
series.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-23 08:15:20 -07:00
Patrick Steinhardt 9ce196e86b config: drop `git_config()` wrapper
In 036876a106 (config: hide functions using `the_repository` by
default, 2024-08-13) we have moved around a bunch of functions in the
config subsystem that depend on `the_repository`. Those function have
been converted into mere wrappers around their equivalent function that
takes in a repository as parameter, and the intent was that we'll
eventually remove those wrappers to make the dependency on the global
repository variable explicit at the callsite.

Follow through with that intent and remove `git_config()`. All callsites
are adjusted so that they use `repo_config(the_repository, ...)`
instead. While some callsites might already have a repository available,
this mechanical conversion is the exact same as the current situation
and thus cannot cause any regression. Those sites should eventually be
cleaned up in a later patch series.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-23 08:15:18 -07:00
Patrick Steinhardt cbb388f3e5 object-file: get rid of `the_repository` in `finalize_object_file()`
We implicitly depend on `the_repository` when moving an object file into
place in `finalize_object_file()`. Get rid of this global dependency by
passing in a repository.

Note that one might be pressed to inject an object database instead of a
repository. But the function doesn't really care about the ODB at all.
All it does is to move a file into place while checking whether there is
any collision. As such, the functionality it provides is independent of
the object database and only needs the repository as parameter so that
it can adjust permissions of the file we are about to finalize.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-16 22:16:14 -07:00
Christian Couder b5b3ddbe5c fast-(import|export): improve on commit signature output format
A recent commit, d9cb0e6ff8 (fast-export, fast-import: add support for
signed-commits, 2025-03-10), added support for signed commits to
fast-export and fast-import.

When a signed commit is processed, fast-export can output either
"gpgsig sha1" or "gpgsig sha256" depending on whether the signed
commit uses the SHA-1 or SHA-256 Git object format.

However, this implementation has a number of limitations:

  - the output format was not properly described in the documentation,
  - the output format is not very informative as it doesn't even say
    if the signature is an OpenPGP, an SSH, or an X509 signature,
  - the implementation doesn't support having both one signature on
    the SHA-1 object and one on the SHA-256 object.

Let's improve on these limitations by improving fast-export and
fast-import so that:

  - all the signatures are exported,
  - at most one signature on the SHA-1 object and one on the SHA-256
    are imported,
  - if there is more than one signature on the SHA-1 object or on
    the SHA-256 object, fast-import emits a warning for each
    additional signature,
  - the output format is "gpgsig <git-hash-algo> <signature-format>",
    where <git-hash-algo> is the Git object format as before, and
    <signature-format> is the signature type ("openpgp", "x509",
    "ssh" or "unknown"),
  - the output is properly documented.

About the output format:

  - <git-hash-algo> allows to know which representation of the commit
    was signed (the SHA-1 or the SHA-256 version) which helps with
    both signature verification and interoperability between repos
    with different hash functions,

  - <signature-format> helps tools that process the fast-export
    stream, so they don't have to parse the ASCII armor to identify
    the signature type.

It could be even better to be able to import more than one signature
on the SHA-1 object and on the SHA-256 object, but other parts of
Git don't handle that well for now, so this is left for future
improvements.

Helped-by: brian m. carlson <sandals@crustytoothpaste.net>
Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-09 16:08:29 -07:00
Patrick Steinhardt 841a03b404 odb: rename `read_object_with_reference()`
Rename `read_object_with_reference()` to `odb_read_object_peeled()` to
match other functions related to the object database and our modern
coding guidelines. Furthermore though, the old name didn't really
describe very well what this function actually does, which is to walk
down any commit and tag objects until an object of the required type has
been found. This is generally referred to as "peeling", so the new name
should be way more descriptive.

No compatibility wrapper is introduced as the function is not used a lot
throughout our codebase.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:39 -07:00
Patrick Steinhardt d4ff88aee3 odb: rename `repo_read_object_file()`
Rename `repo_read_object_file()` to `odb_read_object()` to match other
functions related to the object database and our modern coding
guidelines.

Introduce a compatibility wrapper so that any in-flight topics will
continue to compile.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:38 -07:00
Patrick Steinhardt e989dd96b8 odb: rename `oid_object_info()`
Rename `oid_object_info()` to `odb_read_object_info()` as well as their
`_extended()` variant to match other functions related to the object
database and our modern coding guidelines.

Introduce compatibility wrappers so that any in-flight topics will
continue to compile.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:37 -07:00
Patrick Steinhardt 1b1679c688 odb: get rid of `the_repository` in `odb_mkstemp()`
Get rid of our dependency on `the_repository` in `odb_mkstemp()` by
passing in the object database as a parameter and adjusting all callers.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:35 -07:00
Patrick Steinhardt 8f49151763 object-store: rename files to "odb.{c,h}"
In the preceding commits we have renamed the structures contained in
"object-store.h" to `struct object_database` and `struct odb_backend`.
As such, the code files "object-store.{c,h}" are confusingly named now.
Rename them to "odb.{c,h}" accordingly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:34 -07:00
Patrick Steinhardt 0b8ed25b66 object-store: move and rename `odb_pack_keep()`
The function `odb_pack_keep()` creates a file at the passed-in path. If
this fails, then the function re-tries by first creating any potentially
missing leading directories and then trying to create the file once
again. As such, this function doesn't host any kind of logic that is
specific to the object store, but is rather a generic helper function.

Rename the function to `safe_create_file_with_leading_directories()` and
move it into "path.c". While at it, refactor it so that it loses its
dependency on `the_repository`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-29 10:08:12 -07:00