Commit Graph

871 Commits (49263c6e855f8ea54405c8d4fbbccede478b54af)

Author SHA1 Message Date
Junio C Hamano 1a4c4dac18 Merge branch 'ps/odb-move-loose-object-writing' into jch
The logic to write loose objects has been refactored and moved from
'object-file.c' to the loose backend source file 'odb/source-loose.c',
making the loose backend more self-contained.  This is achieved by
first refactoring force_object_loose() to use generic ODB write
interfaces instead of loose-backend internals.

* ps/odb-move-loose-object-writing:
  object-file: move logic to write loose objects
  object-file: move `force_object_loose()`
  object-file: force objects loose via generic interface
  object-file: fix memory leak in `force_object_loose()`
  odb: support setting mtime when writing objects
  odb: lift object existence check out of the "loose" backend
  odb: compute object hash in `odb_write_object_ext()`
  t/u-odb-inmemory: implement wrapper for writing objects
  odb: compute compat object ID in `odb_write_object_ext()`
2026-07-25 14:27:15 -07:00
Junio C Hamano 17a2bdf582 Merge branch 'ty/migrate-trust-executable-bit' into jch
The 'trust_executable_bit' (coming from the 'core.filemode'
configuration) has been migrated into 'struct repo_config_values' to
tie it to a specific repository instance.

* ty/migrate-trust-executable-bit:
  environment: move has_symlinks into repo_config_values
  environment: move trust_executable_bit into repo_config_values
  read-cache: pass 'repo' to 'ce_mode_from_stat()'
  read-cache: remove redundant extern declarations
2026-07-25 14:27:12 -07:00
Junio C Hamano 55f961a555 Merge branch 'jt/receive-pack-use-odb-transactions'
'git receive-pack' has been refactored to use ODB transaction
interfaces instead of directly managing 'tmp_objdir' for staging
incoming objects, bringing it closer to being ODB backend agnostic.

* jt/receive-pack-use-odb-transactions:
  builtin/receive-pack: stage incoming objects via ODB transactions
  builtin/receive-pack: drop redundant tmpdir env
  odb/transaction: introduce ODB transaction flags
  odb/transaction: add transaction env interface
  odb/transaction: propagate commit errors
  odb/transaction: propagate begin errors
  object-file: propagate files transaction errors
  object-file: drop check for inflight transactions
  object-file: embed transaction flush logic in commit function
  object-file: rename files transaction fsync function
  object-file: rename files transaction prepare function
2026-07-22 10:30:53 -07:00
Tian Yuchen df2bc04e69 environment: move has_symlinks into repo_config_values
Move the global 'has_symlinks' configuration into the
repository-specific 'repo_config_values' struct.

Introduce 'repo_has_symlinks()' getter for readability.
Callers access this configuration by passing in 'repo'
when possible, and explicitly fall back to
'the_repository' the rest of the time.

Introduce 'platform_has_symlinks()' macro to allow
platform specific-customization, primarily to help MinGW.
Platforms can override this in their respective headers.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com>
Mentored-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-20 07:37:05 -07:00
Tian Yuchen 99c79dabb0 environment: move trust_executable_bit into repo_config_values
Move the global 'trust_executable_bit' configuration
into the repository-specific 'repo_config_values'
struct.

To ensure code readability, the getter function
'repo_trust_executable_bit()' has been introduced.
Callers access this configuration by passing in 'repo'
when possible, and explicitly fall back to 'the_repository'
the rest of time.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com>
Mentored-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-20 07:37:05 -07:00
Tian Yuchen b7b6e9e02f read-cache: pass 'repo' to 'ce_mode_from_stat()'
The ce_mode_from_stat() function is a performance-critical static
inline helper in 'read-cache.h'. As we migrate configuration
variables into the repository struct, this helper needs access
to the repository context.

Update the signature of ce_mode_from_stat() to take a 'struct
repository *' parameter, and update all callers to pass the
appropriate repository instance.

To prepare for the overhead of replacing cheap global variable
accesses with getter functions, the boolean expressions are
reordered to evaluate 'S_ISREG(mode)' first.

While at it, add a comment for ce_mode_from_stat().

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com>
Mentored-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-20 07:37:05 -07:00
Tian Yuchen 8817d7931a read-cache: remove redundant extern declarations
The 'read-cache.c' file already includes 'environment.h', which provides
the extern declarations for variables like 'trust_executable_bit' and
'has_symlinks'.

Remove the redundant extern declarations inside 'st_mode_from_ce()' to
clean up the code.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com>
Mentored-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-20 07:37:05 -07:00
Junio C Hamano a336bfa205 Merge branch 'ty/migrate-ignorecase'
The global configuration variable 'ignore_case' (representing the
'core.ignorecase' configuration) has been migrated into 'struct
repo_config_values' to tie it to a specific repository instance.

* ty/migrate-ignorecase:
  config: use repo_ignore_case() to access core.ignorecase
  environment: move ignore_case into repo_config_values
2026-07-19 10:42:19 -07:00
Patrick Steinhardt 8d9135dce3 odb: support setting mtime when writing objects
The function `force_object_loose()` is used to loosen packed objects
before repacking. It passes the pack's mtime along so that the newly
written loose object inherits the same timestamp. This matters for
object pruning, which uses the mtime to determine whether an object is
old enough to be pruned.

In a subsequent commit, `force_object_loose()` will be converted to use
the generic `odb_source_write_object()` interface instead of calling
`write_loose_object()` directly. But the generic interface doesn't yet
support setting a specific mtime, which makes it impossible to implement
the logic as of now.

Prepare for the change by introducing a new `mtime` parameter to this
function that we plumb through the stack. If set, the backends are
instructed to set the object's mtime accordingly. If unset, the backends
are expected to use the current time instead.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-18 19:03:48 -07:00
Junio C Hamano 9306f4f379 Merge branch 'jt/receive-pack-use-odb-transactions' into HEAD
'git receive-pack' has been refactored to use ODB transaction
interfaces instead of directly managing 'tmp_objdir' for staging
incoming objects, bringing it closer to being ODB backend agnostic.

* jt/receive-pack-use-odb-transactions:
  builtin/receive-pack: stage incoming objects via ODB transactions
  builtin/receive-pack: drop redundant tmpdir env
  odb/transaction: introduce ODB transaction flags
  odb/transaction: add transaction env interface
  odb/transaction: propagate commit errors
  odb/transaction: propagate begin errors
  object-file: propagate files transaction errors
  object-file: drop check for inflight transactions
  object-file: embed transaction flush logic in commit function
  object-file: rename files transaction fsync function
  object-file: rename files transaction prepare function
2026-07-18 19:02:51 -07:00
Junio C Hamano e7e0872c6a Merge branch 'jk/git-hash-cleanups'
The 'git_hash_*()' wrappers have been updated to be used consistently
across the codebase instead of direct calls to members of 'struct
git_hash_algo', and 'git_hash_discard()' has been made idempotent to
simplify cleanups.

* jk/git-hash-cleanups:
  hash: check ctx->active flag in all wrapper functions
  http: use idempotent git_hash_discard()
  csum-file: use idempotent git_hash_discard()
  hash: make git_hash_discard() idempotent
  hash: document function pointers and wrappers
  hash: convert remaining direct function calls
  hash: use git_hash_init() consistently
2026-07-16 23:05:48 -07:00
Junio C Hamano 955b276e05 Merge branch 'ps/history-drop'
The experimental 'git history' command has been taught a new 'drop'
subcommand to remove a commit, with its descendants replayed onto its
parent.

* ps/history-drop:
  builtin/history: implement "drop" subcommand
  builtin/history: split handling of ref updates into two phases
  replay: expose `replay_result_queue_update()`
  reset: stop assuming that the caller passes in a clean index
  reset: allow the caller to specify the current HEAD object
  reset: introduce ability to skip updating HEAD
  reset: introduce dry-run mode
  reset: modernize flags passed to `reset_working_tree()`
  reset: rename `reset_head()`
  reset: drop `USE_THE_REPOSITORY_VARIABLE`
  read-cache: split out function to drop unmerged entries to stage 0
2026-07-15 13:24:19 -07:00
Justin Tobler 48d730a16a odb/transaction: introduce ODB transaction flags
The temporary directory used by git-receive-pack(1) to write objects is
managed slightly differently than how it is done via ODB transactions:

  - The temporary directory is eagerly created upfront, instead of
    waiting for the first object write.

  - The prefix name of the temporary directory is "incoming" instead of
    "bulk-fsync".

In a subsequent commit, git-receive-pack(1) will use ODB transactions
instead of `tmp_objdir` directly. To provide a means to configure the
same transaction behavior, introduce `enum odb_transaction_flags` and
the ODB_TRANSACTION_RECEIVE flag intended as a signal for ODB
transactions using the "files" backend to be set up for
git-receive-pack(1). Transaction call sites are updated accordingly to
provide the required flag parameter.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-10 13:21:53 -07:00
Justin Tobler 4576d6c522 odb/transaction: propagate begin errors
When `odb_transaction_begin()` is invoked, the function returns the
transaction pointer directly. There is no way for the backend to
signal that it failed to set up its state, such as when creating the
temporary object directory backing the transaction.

In a subsequent commit, git-receive-pack(1) starts using ODB
transactions and needs to be able to report such failures rather
than silently ignore them. Refactor `odb_transaction_begin()` to
return an int error code and write the resulting transaction into an
out parameter. Also introduce `odb_transaction_begin_or_die()` as a
convenience for callsites that do not need to handle errors
explicitly.

Note that `odb_transaction_begin()` now returns an error when the ODB
already has an inflight transaction pending. ODB transaction call sites
that may encounter an inflight transaction are updated to explicitly
handle this case.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-10 13:21:53 -07:00
Jeff King 9b204b825b hash: use git_hash_init() consistently
We'd like to add more logic to git_hash_init(), but many callers skip it
and call algop->init_fn() directly. Let's make sure we're consistently
using the wrapper by adding a coccinelle rule.

Besides the coccinelle file itself, this is a purely mechanical
conversion based on the patch it generates. There should be no bare
init_fn() calls left (except for the one in the wrapper).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-07 21:56:00 -07:00
Junio C Hamano 13f5190881 Merge branch 'ty/move-protect-hfs-ntfs'
The global configuration variables protect_hfs and protect_ntfs have
been migrated into struct repo_config_values to tie them to
per-repository configuration state.

* ty/move-protect-hfs-ntfs:
  environment: use 'repo->initialized' for repo_protect_hfs() and repo_protect_ntfs()
  environment: move 'protect_hfs' and 'protect_ntfs' into 'repo_config_values'
2026-07-06 15:50:21 -07:00
Patrick Steinhardt ca39c3511d read-cache: split out function to drop unmerged entries to stage 0
In `repo_read_index_unmerged()` we read the index and then drop any
unmerged entries to stage 0. In a subsequent commit we'll want to
perform this operation on arbitrary indexes, not only the one of the
given repository.

Prepare for this by splitting out the functionality into a new function
that can act on an arbitrary index.

While at it, fix a signedness mismatch when iterating through the index
cache entries.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-01 09:56:52 -07:00
Tian Yuchen e6a79c9eb8 config: use repo_ignore_case() to access core.ignorecase
Replace the accesses to the global 'ignore_case' variable with
calls to 'repo_ignore_case(the_repository)'. This step eliminates
the 'ignore_case' global state.

Note on compat/win32/path-utils.c:
To eliminate the global state, several helper functions
(e.g. 'win32_fspathncmp()') now read from
'repo_ignore_case(the_repository)'. While this introduces
dependency on 'repository.h' into the 'compat/', it avoids massive
refactoring of the signatures across the codebase.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com>
Mentored-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-19 10:04:56 -07:00
Johannes Schindelin c6a4629e32 odb: use size_t for object_info.sizep and the size APIs
When `js/objects-larger-than-4gb-on-windows` widened the streaming,
index-pack and unpack-objects code paths, in the interest of keeping the
patches somewhat reasonably-sized, it left the public ODB API still
typed in `unsigned long`. In particular `struct object_info::sizep` and
the four wrappers built on top of it (`odb_read_object`,
`odb_read_object_peeled`, `odb_read_object_info`, `odb_pretend_object`)
still return the unpacked size through `unsigned long *`, so on Windows
`cat-file -s` and the `git add` / `git status` paths for a >4 GiB blob
silently cap at 4 GiB.

Widen the field and the four wrappers. The previous commits already
widened the `unpack_entry()` cascade and pack-objects' in-core size
accessors, so most of the cascade arrives here with no further work: the
temporary shims in `packed_object_info_with_index_pos()` and in
`unpack_entry()`'s delta-base recovery path go away, the two
`SET_SIZE(entry, cast_size_t_to_ulong(canonical_size))` calls in
`check_object()` and the matching one in `drop_reused_delta()` collapse
to plain `SET_SIZE`, and `oe_get_size_slow()`'s tail
`cast_size_t_to_ulong()` is gone too.

What remains narrow are the boundaries this series does not
intend to touch: the diff, blame, textconv and fast-import machinery.

Even so, this patch is unfortunately quite large.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-15 07:45:41 -07:00
Tian Yuchen 71386c21df environment: move 'protect_hfs' and 'protect_ntfs' into 'repo_config_values'
Move the global 'protect_hfs' and 'protect_ntfs' configurations
into the repository-specific 'repo_config_values' struct.
This will help with the elimination of 'the_repository'

To ensure code readability, the getter functions
'repo_protect_hfs()' and 'repo_protect_ntfs()'
have been introduced.

For now, associated functions access this configuration by
explicitly falling back to 'the_repository', which needs to
be addressed in the future.

Note: In 't/helper/test-path-utils.c', there is a function
'protect_ntfs_hfs_benchmark()' where these two global
variables are used as loop iterators. New local variables
have been created to replace them.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com>
Mentored-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-12 05:45:09 -07:00
Junio C Hamano 2f952b81ed Merge branch 'jt/odb-transaction-write'
ODB transaction interface is being reworked to explicitly handle
object writes.

* jt/odb-transaction-write:
  odb/transaction: make `write_object_stream()` pluggable
  object-file: generalize packfile writes to use odb_write_stream
  object-file: avoid fd seekback by checking object size upfront
  object-file: remove flags from transaction packfile writes
  odb: update `struct odb_write_stream` read() callback
  odb/transaction: use pluggable `begin_transaction()`
  odb: split `struct odb_transaction` into separate header
2026-05-27 14:15:45 +09:00
Justin Tobler 5f6744d3eb odb: split `struct odb_transaction` into separate header
The current ODB transaction interface is colocated with other ODB
interfaces in "odb.{c,h}". Subsequent commits will expand `struct
odb_transaction` to support write operations on the transaction
directly. To keep things organized and prevent "odb.{c,h}" from becoming
more unwieldy, split out `struct odb_transaction` into a separate
header.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-15 04:44:39 +09:00
Junio C Hamano d8c553bbed Merge branch 'ng/add-files-to-cache-wo-rename'
add_files_to_cache() used diff_files() to detect only the paths that
are different between the index and the working tree and add them,
which does not need rename detection, which interfered with unnecessary
conflicts.

* ng/add-files-to-cache-wo-rename:
  read-cache: disable renames in add_files_to_cache
2026-04-08 10:19:18 -07:00
Nick Golden c0ce43376b read-cache: disable renames in add_files_to_cache
add_files_to_cache() refreshes the index from worktree changes and does
not need rename detection. When unmerged entries and a deleted stage-0
path are present together, rename detection can pair them and rewrite an
unmerged diff pair to point at the deleted path.

That later makes "git commit -a" and "git add -u" try to stat the
deleted path and die with "unable to stat". Disable rename detection in
this callback-driven staging path and add a regression test covering the
crash.

Signed-off-by: Nick Golden <blindmansion@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-01 12:45:06 -07:00
Jayesh Daga a7aca15677 read-cache: use istate->repo for trace2 logging
trace2 calls in read-cache.c use the global 'the_repository',
even though the relevant index_state provides an explicit
repository pointer via 'istate->repo'.

Using the global repository can result in incorrect trace2
output when multiple repository instances are in use, as
events may be attributed to the wrong repository.

Use 'istate->repo' instead to ensure correct repository
attribution.

Signed-off-by: Jayesh Daga <jayeshdaga99@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-30 13:02:29 -07:00
Junio C Hamano 3fe08b8fd1 Merge branch 'cs/add-skip-submodule-ignore-all'
"git add <submodule>" has been taught to honor
submodule.<name>.ignore that is set to "all" (and requires "git add
-f" to override it).

* cs/add-skip-submodule-ignore-all:
  Documentation: update add --force option + ignore=all config
  tests: fix existing tests when add an ignore=all submodule
  tests: t2206-add-submodule-ignored: ignore=all and add --force tests
  read-cache: submodule add need --force given ignore=all configuration
  read-cache: update add_files_to_cache take param ignored_too
2026-03-09 14:36:55 -07:00
Claus Schneider(Eficode) a16c4a245a read-cache: submodule add need --force given ignore=all configuration
Submodules configured with ignore=all are now skipped during add operations
unless overridden by --force and the submodule path is explicitly specified.

A message is printed (like ignored files) guiding the user to use the
--force flag if the user explicitly wants to update the submodule reference.

The reason for the change is to support branch tracking in submodules
with configuration `submdule.<name>.branch` or similar workflows where the
user is not interested in tracking each update of the sha1 in the submdule.
You can additionally set `submodule.<name>.ignore=all` and the `git status`
will state nothing and, with this patch, the `git add` does not either - as
the default behaviour. This patch changes the workflow to a more logical
behaviour and similar to workflow for ignored files.

The patch gives more scenarios for submodules to be used effectively with
less friction similar to the "repo" tool. A submodule can be added for many
different reasons than a hard dependency. It can be added as loosely
coupled dependencies whereas the user wants the latest based on the
configuration `submoule.<name>.branch`, but are not interested to track
each commit in the `super-repo`. Currently it gives friction of handling
conflicts between branches even the sha1's are fast-forward and the user
just wants the latest in any way. The user can still add a sha1 explicitly
to track updates.

Signed-off-by: Claus Schneider(Eficode) <claus.schneider@eficode.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-06 09:43:26 -08:00
Claus Schneider(Eficode) 37196d8995 read-cache: update add_files_to_cache take param ignored_too
The ignored_too parameter is added to the function
add_files_to_cache for usage of explicit updating the index for the updated
submodule using the explicit patchspec to the submodule.

Signed-off-by: Claus Schneider(Eficode) <claus.schneider@eficode.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-06 09:43:26 -08:00
Junio C Hamano 26f50ef98f Merge branch 'js/symlink-windows'
Upstream symbolic link support on Windows from Git-for-Windows.

* js/symlink-windows:
  mingw: special-case index entries for symlinks with buggy size
  mingw: emulate `stat()` a little more faithfully
  mingw: try to create symlinks without elevated permissions
  mingw: add support for symlinks to directories
  mingw: implement basic `symlink()` functionality (file symlinks only)
  mingw: implement `readlink()`
  mingw: allow `mingw_chdir()` to change to symlink-resolved directories
  mingw: support renaming symlinks
  mingw: handle symlinks to directories in `mingw_unlink()`
  mingw: add symlink-specific error codes
  mingw: change default of `core.symlinks` to false
  mingw: factor out the retry logic
  mingw: compute the correct size for symlinks in `mingw_lstat()`
  mingw: teach dirent about symlinks
  mingw: let `mingw_lstat()` error early upon problems with reparse points
  mingw: drop the separate `do_lstat()` function
  mingw: implement `stat()` with symlink support
  mingw: don't call `GetFileAttributes()` twice in `mingw_lstat()`
2026-01-23 13:34:37 -08:00
René Scharfe ec7a16b145 cocci: convert parse_tree functions to repo_ variants
Add and apply a semantic patch to convert calls to parse_tree() and
friends to the corresponding variant that takes a repository argument,
to allow the functions that implicitly use the_repository to be retired
once all potential in-flight topics are settled and converted as well.

The changes in .c files were generated by Coccinelle, but I fixed a
whitespace bug it would have introduced to builtin/commit.c.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-09 18:36:18 -08:00
Johannes Schindelin 44af34bde7 mingw: special-case index entries for symlinks with buggy size
In https://github.com/git-for-windows/git/pull/2637, we fixed a bug
where symbolic links' target path sizes were recorded incorrectly in the
index. The downside of this fix was that every user with tracked
symbolic links in their checkouts would see them as modified in `git
status`, but not in `git diff`, and only a `git add <path>` (or `git add
-u`) would "fix" this.

Let's do better than that: we can detect that situation and simply
pretend that a symbolic link with a known bad size (or a size that just
happens to be that bad size, a _very_ unlikely scenario because it would
overflow our buffers due to the trailing NUL byte) means that it needs
to be re-checked as if we had just checked it out.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-09 18:32:55 -08:00
Junio C Hamano d91d79f26d Merge branch 'bc/submodule-force-same-hash'
Adding a repository that uses a different hash function is a no-no,
but "git submodule add" did nt prevent it, which has been corrected.

* bc/submodule-force-same-hash:
  read-cache: drop submodule check from add_to_cache()
  object-file: disallow adding submodules of different hash algo
2025-11-24 15:46:40 -08:00
Jeff King 6fe288bfbc read-cache: drop submodule check from add_to_cache()
In add_to_cache(), we treat any directories as submodules, and complain
if we can't resolve their HEAD. This call to resolve_gitlink_ref() was
added by f937bc2f86 (add: error appropriately on repository with no
commits, 2019-04-09), with the goal of improving the error message for
empty repositories.

But we already resolve the submodule HEAD in index_path(), which is
where we find the actual oid we're going to use. Resolving it again here
introduces some downsides:

  1. It's more work, since we have to open up the submodule repository's
     files twice.

  2. There are call paths that get to index_path() without going through
     add_to_cache(). For instance, we'd want a similar informative
     message if "git diff empty" finds that it can't resolve the
     submodule's HEAD. (In theory we can also get there through
     update-index, but AFAICT it refuses to consider directories as
     submodules at all, and just complains about them).

  3. The resolution in index_path() catches more errors that we don't
     handle here. In particular, it will validate that the object format
     for the submodule matches that of the superproject. This isn't a
     bug, since our call in add_to_cache() throws away the oid it gets
     without looking at it. But it certainly caused confusion for me
     when looking at where the object-format check should go.

So instead of resolving the submodule HEAD in add_to_cache(), let's just
teach the call in index_path() to actually produce an error message
(which it already does for other cases). That's probably what f937bc2f86
should have done in the first place, and it gives us a single point of
resolution when adding a submodule to the index.

The resulting output is slightly more verbose, as we propagate the error
up the call stack, but I think that's OK (and again, matches many other
errors we get when indexing fails).

I've left the text of the error message as-is, though it is perhaps
overly specific.  There are many reasons that resolving the submodule
HEAD might fail, though outside of corruption or system errors it is
probably most likely that the submodule HEAD is simply on an unborn
branch.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-15 21:18:49 -08:00
Junio C Hamano 75f8dfabaa Merge branch 'ps/rust-balloon'
Dip our toes a bit to (optionally) use Rust implemented helper
called from our C code.

* ps/rust-balloon:
  ci: enable Rust for breaking-changes jobs
  ci: convert "pedantic" job into full build with breaking changes
  BreakingChanges: announce Rust becoming mandatory
  varint: reimplement as test balloon for Rust
  varint: use explicit width for integers
  help: report on whether or not Rust is enabled
  Makefile: introduce infrastructure to build internal Rust library
  Makefile: reorder sources after includes
  meson: add infrastructure to build internal Rust library
2025-10-08 12:17:55 -07:00
Patrick Steinhardt f366bfe16b varint: use explicit width for integers
The varint subsystem currently uses implicit widths for integers. On the
one hand we use `uintmax_t` for the actual value. On the other hand, we
use `int` for the length of the encoded varint.

Both of these have known maximum values, as we only support at most 16
bytes when encoding varints. Thus, we know that we won't ever exceed
`uint64_t` for the actual value and `uint8_t` for the prefix length.

Refactor the code to use explicit widths. Besides making the logic
platform-independent, it also makes our life a bit easier in the next
commit, where we reimplement "varint.c" in Rust.

Suggested-by: Ezekiel Newren <ezekielnewren@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-02 09:32:32 -07:00
Justin Tobler ce1661f9da odb: add transaction interface
Transactions are managed via the {begin,end}_odb_transaction() function
in the object-file subsystem and its implementation is specific to the
files object source. Introduce odb_transaction_{begin,commit}() in the
odb subsystem to provide an eventual object source agnostic means to
manage transactions.

Update call sites to instead manage transactions through the odb
subsystem. Also rename {begin,end}_odb_transaction() functions to
object_file_transaction_{begin,commit}() to clarify the object source it
supports.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-16 11:37:06 -07:00
Justin Tobler 78839e9cde object-file: relocate ODB transaction code
The bulk-checkin subsystem provides various functions to manage ODB
transactions. Apart from {begin,end}_odb_transaction(), these functions
are only used by the object-file subsystem to manage aspects of a
transaction implementation specific to the files object source.

Relocate all the transaction code in bulk-checkin to object-file. This
simplifies the exposed transaction interface by reducing it to only
{begin,end}_odb_transaction(). Function and type names are adjusted in
the subsequent commit to better fit the new location.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-16 11:37:05 -07:00
Justin Tobler b336144725 bulk-checkin: remove global transaction state
Object database transactions in the bulk-checkin subsystem rely on
global state to track transaction status. Stop relying on global state
and instead store the transaction in the `struct object_database`.
Functions that operate on transactions are updated to now wire
transaction state.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-08-25 09:48:13 -07:00
Junio C Hamano 4ce0caa7cc Merge branch 'ps/object-file-wo-the-repository'
Reduce implicit assumption and dependence on the_repository in the
object-file subsystem.

* ps/object-file-wo-the-repository:
  object-file: get rid of `the_repository` in index-related functions
  object-file: get rid of `the_repository` in `force_object_loose()`
  object-file: get rid of `the_repository` in `read_loose_object()`
  object-file: get rid of `the_repository` in loose object iterators
  object-file: remove declaration for `for_each_file_in_obj_subdir()`
  object-file: inline `for_each_loose_file_in_objdir_buf()`
  object-file: get rid of `the_repository` when writing objects
  odb: introduce `odb_write_object()`
  loose: write loose objects map via their source
  object-file: get rid of `the_repository` in `finalize_object_file()`
  object-file: get rid of `the_repository` in `loose_object_info()`
  object-file: get rid of `the_repository` when freshening objects
  object-file: inline `check_and_freshen()` functions
  object-file: get rid of `the_repository` in `has_loose_object()`
  object-file: stop using `the_hash_algo`
  object-file: fix -Wsign-compare warnings
2025-08-05 11:53:55 -07:00
Patrick Steinhardt 5d215a7b3e config: drop `git_config_get_bool()` wrapper
In 036876a106 (config: hide functions using `the_repository` by
default, 2024-08-13) we have moved around a bunch of functions in the
config subsystem that depend on `the_repository`. Those function have
been converted into mere wrappers around their equivalent function that
takes in a repository as parameter, and the intent was that we'll
eventually remove those wrappers to make the dependency on the global
repository variable explicit at the callsite.

Follow through with that intent and remove `git_config_get_bool()`. All
callsites are adjusted so that they use
`repo_config_get_bool(the_repository, ...)` instead. While some
callsites might already have a repository available, this mechanical
conversion is the exact same as the current situation and thus cannot
cause any regression. Those sites should eventually be cleaned up in a
later patch series.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-23 08:15:20 -07:00
Patrick Steinhardt ab1c6e1d12 odb: introduce `odb_write_object()`
We do not have a backend-agnostic way to write objects into an object
database. While there is `write_object_file()`, this function is rather
specific to the loose object format.

Introduce `odb_write_object()` to plug this gap. For now, this function
is a simple wrapper around `write_object_file()` and doesn't even use
the passed-in object database yet. This will change in subsequent
commits, where `write_object_file()` is converted so that it works on
top of an `odb_source`. `odb_write_object()` will then become
responsible for deciding which source an object shall be written to.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-16 22:16:15 -07:00
Junio C Hamano 51b50c55a9 Merge branch 'ps/object-store'
Code clean-up around object access API.

* ps/object-store:
  odb: rename `read_object_with_reference()`
  odb: rename `pretend_object_file()`
  odb: rename `has_object()`
  odb: rename `repo_read_object_file()`
  odb: rename `oid_object_info()`
  odb: trivial refactorings to get rid of `the_repository`
  odb: get rid of `the_repository` when handling submodule sources
  odb: get rid of `the_repository` when handling the primary source
  odb: get rid of `the_repository` in `for_each()` functions
  odb: get rid of `the_repository` when handling alternates
  odb: get rid of `the_repository` in `odb_mkstemp()`
  odb: get rid of `the_repository` in `assert_oid_type()`
  odb: get rid of `the_repository` in `find_odb()`
  odb: introduce parent pointers
  object-store: rename files to "odb.{c,h}"
  object-store: rename `object_directory` to `odb_source`
  object-store: rename `raw_object_store` to `object_database`
2025-07-15 15:18:18 -07:00
Han Young 9455397a5c read-cache: report lock error when refreshing index
In the repo_refresh_and_write_index of read-cache.c, we return -1 to
indicate that writing the index to disk failed.
However, callers do not use this information. Commands such as stash print
  "could not write index"
and then exit, which does not help to discover the exact problem.

We can let repo_hold_locked_index print the error message if the locking
failed.

Signed-off-by: Han Young <hanyang.tony@bytedance.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-07 06:49:07 -07:00
Patrick Steinhardt d4ff88aee3 odb: rename `repo_read_object_file()`
Rename `repo_read_object_file()` to `odb_read_object()` to match other
functions related to the object database and our modern coding
guidelines.

Introduce a compatibility wrapper so that any in-flight topics will
continue to compile.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:38 -07:00
Patrick Steinhardt e989dd96b8 odb: rename `oid_object_info()`
Rename `oid_object_info()` to `odb_read_object_info()` as well as their
`_extended()` variant to match other functions related to the object
database and our modern coding guidelines.

Introduce compatibility wrappers so that any in-flight topics will
continue to compile.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:37 -07:00
Patrick Steinhardt 8f49151763 object-store: rename files to "odb.{c,h}"
In the preceding commits we have renamed the structures contained in
"object-store.h" to `struct object_database` and `struct odb_backend`.
As such, the code files "object-store.{c,h}" are confusingly named now.
Rename them to "odb.{c,h}" accordingly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01 14:46:34 -07:00
Johannes Schindelin 655268452c has_dir_name(): make code more obvious
One thing that might be non-obvious to readers (or to analyzers like
CodeQL) is that the function essentially does nothing when the Git index
is empty, and in particular that it does not look at the value of
`len_eq_last` (which would be uninitialized at that point).

Let's make this much easier to understand, by returning early if the Git
index is empty, and by avoiding empty `else` blocks.

This commit changes indentation and is hence best viewed using
`--ignore-space-change`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-05-15 13:46:46 -07:00
Junio C Hamano 36d8035d27 Merge branch 'ps/object-file-cleanup'
Code clean-up.

* ps/object-file-cleanup:
  object-store: merge "object-store-ll.h" and "object-store.h"
  object-store: remove global array of cached objects
  object: split out functions relating to object store subsystem
  object-file: drop `index_blob_stream()`
  object-file: split up concerns of `HASH_*` flags
  object-file: split out functions relating to object store subsystem
  object-file: move `xmmap()` into "wrapper.c"
  object-file: move `git_open_cloexec()` to "compat/open.c"
  object-file: move `safe_create_leading_directories()` into "path.c"
  object-file: move `mkdir_in_gitdir()` into "path.c"
2025-04-24 17:25:33 -07:00
Junio C Hamano 480ddc566f Merge branch 'js/range-check-codeql-workaround'
Work around false positive from CodeQL checker.

* js/range-check-codeql-workaround:
  read-cache: check range before dereferencing an array element
2025-04-23 13:58:51 -07:00
Patrick Steinhardt 68cd492a3e object-store: merge "object-store-ll.h" and "object-store.h"
The "object-store-ll.h" header has been introduced to keep transitive
header dependendcies and compile times at bay. Now that we have created
a new "object-store.c" file though we can easily move the last remaining
additional bit of "object-store.h", the `odb_path_map`, out of the
header.

Do so. As the "object-store.h" header is now equivalent to its low-level
alternative we drop the latter and inline it into the former.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-15 08:24:37 -07:00