Commit Graph

3248 Commits (bbcc0ae3e94be45320ca7bbd31dbc2a2ce58ea8d)

Author SHA1 Message Date
Junio C Hamano 686213114e Merge branch 'mm/git-url-parse'
The internal URL parsing logic has been made accessible via a new
subcommand "git url-parse".

* mm/git-url-parse:
  t9904: add tests for the new url-parse builtin
  doc: describe the url-parse builtin
  builtin: create url-parse command
  urlmatch: define url_parse function
  url: return URL_SCHEME_UNKNOWN instead of dying
  url: move scheme detection to URL header/source
  url: move url_is_local_not_ssh to url.h
  connect: rename enum protocol to url_scheme
2026-05-21 12:06:48 +09:00
Junio C Hamano 91ddfe3d5c Merge branch 'js/mingw-no-nedmalloc'
Stop using unmaintained custom allocator in Windows build which was
the last user of the code.

* js/mingw-no-nedmalloc:
  mingw: remove the vendored compat/nedmalloc/ subtree
  mingw: drop the build-system plumbing for nedmalloc
  mingw: stop using nedmalloc
2026-05-20 10:30:56 +09:00
Junio C Hamano a6876b2068 Merge branch 'js/objects-larger-than-4gb-on-windows'
Update code paths that assumed "unsigned long" was long enough for
"size_t".

* js/objects-larger-than-4gb-on-windows:
  ci: run expensive tests on push builds to integration branches
  t5608: mark >4GB tests as EXPENSIVE
  test-tool synthesize: add precomputed SHA-256 pack for 4 GiB + 1
  test-tool synthesize: precompute pack for 4 GiB + 1
  test-tool synthesize: use the unsafe hash for speed
  t5608: add regression test for >4GB object clone
  test-tool: add a helper to synthesize large packfiles
  delta, packfile: use size_t for delta header sizes
  odb, packfile: use size_t for streaming object sizes
  git-zlib: handle data streams larger than 4GB
  index-pack, unpack-objects: use size_t for object size
2026-05-20 10:30:56 +09:00
Junio C Hamano 056472b82d Merge branch 'kh/name-rev-custom-format'
A new builtin "git format-rev" is introduced for pretty formatting
one revision expression per line or commit object names found in
running text.

* kh/name-rev-custom-format:
  format-rev: introduce builtin for on-demand pretty formatting
  name-rev: make dedicated --annotate-stdin --name-only test
  name-rev: factor code for sharing with a new command
  name-rev: run clang-format before factoring code
  name-rev: wrap both blocks in braces
2026-05-19 09:57:44 +09:00
Kristoffer Haugsbakk 19e3106c45 format-rev: introduce builtin for on-demand pretty formatting
Introduce a new builtin for pretty formatting one revision expression
per line or commit object names found in running text.

Sometimes you want to format commits. Most of the time you’re
walking the graph, e.g. getting a range of commits like
`master..topic`. That’s a job for git-log(1).

But there are times when you want to format commits that you encounter
on demand:

• Full hashes in running text that you might want to pretty-print
• git-last-modified(1) outputs full hashes that you can do the same
  with
• git-cherry(1) has `-v` for commit subject, but maybe you want
  something else?

But now you can’t use git-log(1), git-show(1), or git-rev-list(1):

• You can’t feed commits piecemeal to these commands, one input
  for one output; they block until standard in is closed
• You can’t feed a list of possibly duplicate commits, like the output
  of git-last-modified(1); they effectively deduplicate the output

Beyond these two points there’s also the input massage problem: you
cannot feed mixed input (revisions mixed with arbitrary text).

One might hope that git-cat-file(1) can save us. But it doesn’t
support pretty formats.

But there is one command that already both handles revisions as
arguments, revisions on standard input, and even revisions mixed in
with arbitrary text. Namely git-name-rev(1): the command for outputting
symbolic names for commits.

We made some room in `builtin/name-rev.c` two commits ago. Let’s
now add this new git-format-rev(1) command. Taking inspiration from
git-name-rev(1), there are two modes:

• revs: like git-name-rev(1) in argv mode, but one revision per line
  on standard in
• text: like git-name-rev(1) with `--annotate-stdin`

***

We need to add this command to the exception list in
`t/t1517-outside-repo.sh` because it uses “EXPERIMENTAL!”
in the usage line.

Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-12 09:09:51 +09:00
Junio C Hamano bd5c2827b2 Merge branch 'bc/rust-by-default'
Rust support is enabled by default (but still allows opting out) in
some future version of Git.

* bc/rust-by-default:
  Enable Rust by default
  Linux: link against libdl
  ci: install cargo on Alpine
  docs: update version with default Rust support
2026-05-11 10:05:54 +09:00
Junio C Hamano 718db095c2 Merge branch 'ar/parallel-hooks'
Hook scripts defined via the configuration system can now be
configured to run in parallel.

* ar/parallel-hooks:
  t1800: test SIGPIPE with parallel hooks
  hook: allow hook.jobs=-1 to use all available CPU cores
  hook: add hook.<event>.enabled switch
  hook: move is_known_hook() to hook.c for wider use
  hook: warn when hook.<friendly-name>.jobs is set
  hook: add per-event jobs config
  hook: add -j/--jobs option to git hook run
  hook: mark non-parallelizable hooks
  hook: allow pre-push parallel execution
  hook: allow parallel hook execution
  hook: parse the hook.jobs config
  config: add a repo_config_get_uint() helper
  repository: fix repo_init() memleak due to missing _clear()
2026-05-11 10:05:53 +09:00
Johannes Schindelin 438886aecb test-tool: add a helper to synthesize large packfiles
To test Git's behavior with very large pack files, we need a way to
generate such files quickly.

A naive approach using only readily-available Git commands would take
over 10 hours for a 4GB pack file, which is prohibitive.

Side-stepping Git's machinery and actual zlib compression by writing
uncompressed content with the appropriate zlib header makes things
much faster. The fastest method using this approach generates many
small, unreachable blob objects and takes about 1.5 minutes for 4GB.
However, this cannot be used because we need to test git clone, which
requires a reachable commit history.

Generating many reachable commits with small, uncompressed blobs takes
about 4 minutes for 4GB. But this approach 1) does not reproduce the
issues we want to fix (which require individual objects larger than
4GB) and 2) is comparatively slow because of the many SHA-1
calculations.

The approach taken here generates a single large blob (filled with NUL
bytes), along with the trees and commits needed to make it reachable.
This takes about 2.5 minutes for 4.5GB, which is the fastest option
that produces a valid, clonable repository with an object large enough
to trigger the bugs we want to test.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-09 11:25:32 +09:00
Johannes Schindelin cefcada1d3 mingw: drop the build-system plumbing for nedmalloc
With the previous commit removing every opt-in, the build-system
plumbing for nedmalloc has nothing left to switch on. Remove it so
that the upcoming deletion of the compat/nedmalloc/ tree is a pure
file removal.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-09 11:19:23 +09:00
Matheus Afonso Martins Moreira 533eb14798 builtin: create url-parse command
Git commands can accept a rather wide variety of URLs syntaxes.
The range of accepted inputs might expand even more in the future.
This makes the parsing of URL components difficult since standard URL
parsers cannot be used. Extracting the components of a git URL would
require implementing all the schemes that git itself supports, not to
mention tracking its development continuously in case new URL schemes
are added.

The url-parse builtin command is designed to solve this problem
by exposing git's native URL parsing facilities as a plumbing command.
Other programs can then call upon git itself to parse the git URLs
and extract their components. This should be quite useful for scripts.

Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-06 09:48:28 +09:00
Adrian Ratiu 2eb541e8f2 hook: move is_known_hook() to hook.c for wider use
Move is_known_hook() from builtin/hook.c (static) into hook.c and
export it via hook.h so it can be reused.

Make it return bool and the iterator `h` for clarity (iterate hooks).

Both meson.build and the Makefile are updated to reflect that the
header is now used by libgit, not the builtin sources.

The next commit will use this to reject hook friendly-names that
collide with known event names.

Co-authored-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-10 07:58:54 -07:00
brian m. carlson 32d5b90590 Enable Rust by default
Our breaking changes document says that we'll enable Rust by default in
Git 2.54.  Adjust the Makefile to switch the option from WITH_RUST to
NO_RUST to enable it by default and update the help text accordingly.
Similarly, for Meson, enable the option by default and do not
automatically disable it if Cargo is missing, since the goal is to help
users find where they are likely to have problems in the future.

Update our CI tests to swap out the single Linux job with Rust to a
single job without, both for Makefile and Meson.  Similarly, update the
Windows Makefile job to not use Rust, while the Meson job (which does
not build with ci/lib.sh) will default to having it enabled.

Move the check for Cargo in the Meson build because it is no longer
needed in the main script.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-09 17:25:36 -07:00
Junio C Hamano 7798034171 Revert "compat/posix: introduce writev(3p) wrapper"
This reverts commit 3b9b2c2a29a1d529ca9884fa0a6529f6e2496abe; let's
not use writev() for now.
2026-04-09 14:48:24 -07:00
Junio C Hamano 0cd4fb9f46 Merge branch 'ar/config-hook-cleanups'
Code clean-up around the recent "hooks defined in config" topic.

* ar/config-hook-cleanups:
  hook: reject unknown hook names in git-hook(1)
  hook: show disabled hooks in "git hook list"
  hook: show config scope in git hook list
  hook: introduce hook_config_cache_entry for per-hook data
  t1800: add test to verify hook execution ordering
  hook: make consistent use of friendly-name in docs
  hook: replace hook_list_clear() -> string_list_clear_func()
  hook: detect & emit two more bugs
  hook: rename cb_data_free/alloc -> hook_data_free/alloc
  hook: fix minor style issues
  builtin/receive-pack: properly init receive_hook strbuf
  hook: move unsorted_string_list_remove() to string-list.[ch]
2026-04-03 13:01:09 -07:00
Junio C Hamano d1f07dd500 Merge branch 'ps/build-tweaks'
Tweak the build infrastructure by moving tools around.

* ps/build-tweaks:
  meson: precompile "git-compat-util.h"
  meson: compile compatibility sources separately
  git-compat-util.h: move warning infra to prepare for PCHs
  builds: move build scripts into "tools/"
  contrib: move "update-unicode.sh" script into "tools/"
  contrib: move "coverage-diff.sh" script into "tools/"
  contrib: move "coccinelle/" directory into "tools/"
  Introduce new "tools/" directory
2026-03-27 11:00:01 -07:00
Adrian Ratiu 5c58dbc887 hook: reject unknown hook names in git-hook(1)
Teach "git hook run" and "git hook list" to reject hook event names
that are not recognized by Git. This helps catch typos such as
"prereceive" when "pre-receive" was intended, since in 99% of the
cases users want known (already-existing) hook names.

The list of known hooks is derived from the generated hook-list.h
(built from Documentation/githooks.adoc). This is why the Makefile
is updated, so builtin/hook.c depends on hook-list.h. In meson the
header is already a dependency for all builtins, no change required.

The "--allow-unknown-hook-name" flag can be used to bypass this check.

Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-25 14:00:48 -07:00
Junio C Hamano 8023abc632 Merge branch 'ps/upload-pack-buffer-more-writes'
Reduce system overhead "git upload-pack" spends on relaying "git
pack-objects" output to the "git fetch" running on the other end of
the connection.

* ps/upload-pack-buffer-more-writes:
  builtin/pack-objects: reduce lock contention when writing packfile data
  csum-file: drop `hashfd_throughput()`
  csum-file: introduce `hashfd_ext()`
  sideband: use writev(3p) to send pktlines
  wrapper: introduce writev(3p) wrappers
  compat/posix: introduce writev(3p) wrapper
  upload-pack: reduce lock contention when writing packfile data
  upload-pack: prefer flushing data over sending keepalive
  upload-pack: adapt keepalives based on buffering
  upload-pack: fix debug statement when flushing packfile data
2026-03-24 12:31:34 -07:00
Patrick Steinhardt a767f2fd6c builds: move build scripts into "tools/"
We have a bunch of scripts used by our different build systems that are
all located in the top-level directory. Now that we have introduced the
new "tools/" directory though we have a better home for them.

Move the scripts into the "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-19 06:40:09 -07:00
Patrick Steinhardt 8ca1b4472c contrib: move "coccinelle/" directory into "tools/"
The Coccinelle tool is an ingrained part of our build infrastructure. It
is executed by our CI to detect antipatterns and is used to detect
misuses of certain interfaces. It's presence in "contrib/" is thus
rather misleading.

Promote the configuration into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-19 06:40:08 -07:00
Patrick Steinhardt 8872941fd2 Introduce new "tools/" directory
According to its readme, the "contrib/" directory's main intent is to
collect stuff that is not an official part of Git, either because it is
too specialized or because it is still considered experimental. The
reality tells a bit of a different story though: while it _does_ contain
such things, it also contains other things:

  - Our credential helpers, which are being distributed by many
    packagers nowadays and which can be considered "stable".

  - A bunch of tooling that relates to our build and test
    infrastructure.

Especially the second category is somewhat of a sore spot. You really
wouldn't expect build-related tooling to be considered an optional part
of Git. Quite the opposite.

Create a new top-level "tools/" directory to fix this discrepancy. This
directory will contain all kind of tools that are related to our build
infrastructure and that Git developers are likely to use day to day.

For now, this directory doesn't contain anything yet except for a
readme and a Meson skeleton. This will change in subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-19 06:40:08 -07:00
Junio C Hamano 2eec0f5115 Merge branch 'jk/unleak-mmap'
Plug a few leaks where mmap'ed memory regions are not unmapped.

* jk/unleak-mmap:
  meson: turn on NO_MMAP when building with LSan
  Makefile: turn on NO_MMAP when building with LSan
  object-file: fix mmap() leak in odb_source_loose_read_object_stream()
  pack-revindex: avoid double-loading .rev files
  check_connected(): fix leak of pack-index mmap
  check_connected(): delay opening new_pack
2026-03-16 10:48:15 -07:00
Patrick Steinhardt 3b9b2c2a29 compat/posix: introduce writev(3p) wrapper
In a subsequent commit we're going to add the first caller to
writev(3p). Introduce a compatibility wrapper for this syscall that we
can use on systems that don't have this syscall.

The syscall exists on modern Unixes like Linux and macOS, and seemingly
even for NonStop according to [1]. It doesn't seem to exist on Windows
though.

[1]: http://nonstoptools.com/manuals/OSS-SystemCalls.pdf
[2]: https://www.gnu.org/software/gnulib/manual/html_node/writev.html

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-13 08:54:14 -07:00
Junio C Hamano c89a495ce4 Merge branch 'ps/odb-sources'
The object source API is getting restructured to allow plugging new
backends.

* ps/odb-sources:
  odb/source: make `begin_transaction()` function pluggable
  odb/source: make `write_alternate()` function pluggable
  odb/source: make `read_alternates()` function pluggable
  odb/source: make `write_object_stream()` function pluggable
  odb/source: make `write_object()` function pluggable
  odb/source: make `freshen_object()` function pluggable
  odb/source: make `for_each_object()` function pluggable
  odb/source: make `read_object_stream()` function pluggable
  odb/source: make `read_object_info()` function pluggable
  odb/source: make `close()` function pluggable
  odb/source: make `reprepare()` function pluggable
  odb/source: make `free()` function pluggable
  odb/source: introduce source type for robustness
  odb: move reparenting logic into respective subsystems
  odb: embed base source in the "files" backend
  odb: introduce "files" source
  odb: split `struct odb_source` into separate header
2026-03-12 14:09:07 -07:00
Junio C Hamano 8194f1795b Merge branch 'bc/sha1-256-interop-02'
The code to maintain mapping between object names in multiple hash
functions is being added, written in Rust.

* bc/sha1-256-interop-02:
  object-file-convert: always make sure object ID algo is valid
  rust: add a small wrapper around the hashfile code
  rust: add a new binary object map format
  rust: add functionality to hash an object
  rust: add a build.rs script for tests
  rust: fix linking binaries with cargo
  hash: expose hash context functions to Rust
  write-or-die: add an fsync component for the object map
  csum-file: define hashwrite's count as a uint32_t
  rust: add additional helpers for ObjectID
  hash: add a function to look up hash algo structs
  rust: add a hash algorithm abstraction
  rust: add a ObjectID struct
  hash: use uint32_t for object_id algorithm
  conversion: don't crash when no destination algo
  repository: require Rust support for interoperability
2026-03-12 10:56:02 -07:00
Jeff King 00611d86c6 Makefile: turn on NO_MMAP when building with LSan
The past few commits fixed some cases where we leak memory allocated by
mmap(). Building with SANITIZE=leak doesn't detect these because it
covers only heap buffers allocated by malloc().

But if we build with NO_MMAP, our compat mmap() implementation will
allocate a heap buffer and pread() into it. And thus Lsan will detect
these leaks for free.

Using NO_MMAP is less performant, of course, since we have to use extra
memory and read in the whole file, rather than faulting in pages from
disk. But LSan builds are already slow, and this doesn't make them
measurably worse. Getting extra coverage for our leak-checking is worth
it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-06 21:12:10 -08:00
Patrick Steinhardt cb506a8a69 odb: introduce "files" source
Introduce a new "files" object database source. This source encapsulates
access to both loose object files and the packfile store, similar to how
the "files" backend for refs encapsulates access to loose refs and the
packed-refs file.

Note that for now the "files" source is still a direct member of a
`struct odb_source`. This architecture will be reversed in the next
commit so that the files source contains a `struct odb_source`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-05 11:45:14 -08:00
Patrick Steinhardt ba1c21d343 odb: split `struct odb_source` into separate header
Subsequent commits will expand the `struct odb_source` to become a
generic interface for accessing an object database source. As part of
these refactorings we'll add a set of function pointers that will
significantly expand the structure overall.

Prepare for this by splitting out the `struct odb_source` into a
separate header. This keeps the high-level object database interface
detached from the low-level object database sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-05 11:45:14 -08:00
Junio C Hamano efd5fdbcf9 Merge branch 'dk/meson-regen-config-list'
Fix dependency screw-up in meson-based builds.

* dk/meson-regen-config-list:
  build: regenerate config-list.h when Documentation changes
2026-03-04 10:53:00 -08:00
Junio C Hamano 22c9b6bd93 Merge branch 'kn/osxkeychain-buildfix'
Simplify build procedure for oxskeychain (in contrib/).

* kn/osxkeychain-buildfix:
  osxkeychain: define build targets in the top-level Makefile.
2026-03-04 10:52:59 -08:00
Junio C Hamano 6b5ad01886 Merge branch 'cc/lop-filter-auto'
"auto filter" logic for large-object promisor remote.

* cc/lop-filter-auto:
  fetch-pack: wire up and enable auto filter logic
  promisor-remote: change promisor_remote_reply()'s signature
  promisor-remote: keep advertised filters in memory
  list-objects-filter-options: support 'auto' mode for --filter
  doc: fetch: document `--filter=<filter-spec>` option
  fetch: make filter_options local to cmd_fetch()
  clone: make filter_options local to cmd_clone()
  promisor-remote: allow a client to store fields
  promisor-remote: refactor initialising field lists
2026-02-25 11:54:17 -08:00
D. Ben Knoble ebeea3c471 build: regenerate config-list.h when Documentation changes
The Meson-based build doesn't know when to rebuild config-list.h, so the
header is sometimes stale.

For example, an old build directory might have config-list.h from before
4173df5187 (submodule: introduce extensions.submodulePathConfig,
2026-01-12), which added submodule.<name>.gitdir to the list. Without
it, t9902-completion.sh fails. Regenerating the config-list.h artifact
from sources fixes the artifact and the test.

Since Meson does not have (or want) builtin support for globbing like
Make, teach generate-configlist.sh to also generate a list of
Documentation files its output depends on, and incorporate that into the
Meson build. We honor the undocumented GCC/Clang contract of outputting
empty targets for all the dependencies (like they do with -MP). That is,
generate lines like

    build/config-list.h: $SOURCE_DIR/Documentation/config.adoc
    $SOURCE_DIR/Documentation/config.adoc:

We assume that if a user adds a new file under
Documentation/config then they will also edit one of the existing files
to include that new file, and that will trigger a rebuild. Also mark the
generator script as a dependency.

While we're at it, teach the Makefile to use the same "the script knows
it's dependencies" logic.

For Meson, combining the following commands helps debug dependencies:

    ninja -C <builddir> -t deps config-list.h
    ninja -C <builddir> -t browse config-list.h

The former lists all the dependencies discovered from our output ".d"
file (the config documentation) and the latter shows the dependency on
the script itself, among other useful edges in the dependency graph.

Helped-by: Patrick Steinhardt <ps@pks.im>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: D. Ben Knoble <ben.knoble+github@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-24 10:51:54 -08:00
Koji Nakamaru 3e9cc24e68 osxkeychain: define build targets in the top-level Makefile.
The fix for git-credential-osxkeychain in 4580bcd235 (osxkeychain: avoid
incorrectly skipping store operation, 2025-11-14) introduced linkage
with libgit.a, and its Makefile was adjusted accordingly. However, the
build fails as of 864f55e190 because several macOS-specific refinements
were applied to the top-level Makefile and config.mak.uname, such as:

  - 363837afe7 (macOS: make Homebrew use configurable, 2025-12-24)
  - cee341e9dd (macOS: use iconv from Homebrew if needed and present,
    2025-12-24)
  - d281241518 (utf8.c: enable workaround for iconv under macOS 14/15,
    2026-01-12)

Since libgit.a and its corresponding header files depend on many flags
defined in the top-level Makefile, these flags must be consistently
defined when building git-credential-osxkeychain. Continuing to manually
adjust the git-credential-osxkeychain Makefile is cumbersome and
fragile.

Define the build targets for git-credential-osxkeychain in the top-level
Makefile and modify its local Makefile to simply rely on those targets.

Helped-by: Junio C Hamano <gitster@pobox.com>
Reported-by: D. Ben Knoble <ben.knoble@gmail.com>
Helped-by: Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com>
Signed-off-by: Koji Nakamaru <koji.nakamaru@gree.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-20 08:10:59 -08:00
Christian Couder cd1a89838a list-objects-filter-options: support 'auto' mode for --filter
In a following commit, we are going to allow passing "auto" as a
<filterspec> to the `--filter=<filterspec>` option, but only for some
commands. Other commands that support the `--filter=<filterspec>`
option should still die() when 'auto' is passed.

Let's set up the "list-objects-filter-options.{c,h}" infrastructure to
support that:

- Add a new `unsigned int allow_auto_filter : 1;` flag to
  `struct list_objects_filter_options` which specifies if "auto" is
  accepted or not by the current command.
- Change gently_parse_list_objects_filter() to parse "auto" if it's
  accepted.
- Make sure we die() if "auto" is combined with another filter.
- Update list_objects_filter_release() to preserve the
  allow_auto_filter flag, as this function is often called (via
  opt_parse_list_objects_filter) to reset the struct before parsing a
  new value.

Let's also update `list-objects-filter.c` to recognize the new
`LOFC_AUTO` choice. Since "auto" must be resolved to a concrete filter
before filtering actually begins, initializing a filter with
`LOFC_AUTO` is invalid and will trigger a BUG().

Note that ideally combining "auto" with "auto" could be allowed, but in
practice, it's probably not worth the added code complexity. And if we
really want it, nothing prevents us to allow it in future work.

If we ever want to give a meaning to combining "auto" with a different
filter too, nothing prevents us to do that in future work either.

Also note that the new `allow_auto_filter` flag depends on the command,
not user choices, so it should be reset to the command default when
`struct list_objects_filter_options` instances are reset.

While at it, let's add a new "u-list-objects-filter-options.c" file for
`struct list_objects_filter_options` related unit tests. For now it
only tests gently_parse_list_objects_filter() though.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-17 11:46:40 -08:00
Junio C Hamano e10d5fcad0 Merge branch 'jc/ci-test-contrib-too'
Test contrib/ things in CI to catch breakages before they enter the
"next" branch.

* jc/ci-test-contrib-too:
  : Some of our downstream folks run more tests than we do and catch
  : breakages in them, namely, where contrib/*/Makefile has "test" target.
  : Let's make sure we fail upon accepting a new topic that break them in
  : 'seen'.
  ci: ubuntu: use GNU coreutils for dirname
  test: optionally test contrib in CI
2026-02-13 13:39:26 -08:00
Junio C Hamano 7bf3785d09 Merge branch 'ps/history'
"git history" history rewriting UI.

* ps/history:
  builtin/history: implement "reword" subcommand
  builtin: add new "history" command
  wt-status: provide function to expose status for trees
  replay: support updating detached HEAD
  replay: support empty commit ranges
  replay: small set of cleanups
  builtin/replay: move core logic into "libgit.a"
  builtin/replay: extract core logic to replay revisions
2026-02-09 12:09:09 -08:00
brian m. carlson 39e4dcf77d rust: add a small wrapper around the hashfile code
Our new binary object map code avoids needing to be intimately involved
with file handling by simply writing data to an object implement Write.
This makes it very easy to test by writing to a Cursor wrapping a Vec
for tests, and thus decouples it from intimate knowledge about how we
handle files.

However, we will actually want to write our data to an actual file,
since that's the most practical way to persist data.  Implement a
wrapper around the hashfile code that implements the Write trait so that
we can write our object map into a file.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-07 17:41:03 -08:00
brian m. carlson 40a1b4fb2b rust: add a new binary object map format
Our current loose object format has a few problems.  First, it is not
efficient: the list of object IDs is not sorted and even if it were,
there would not be an efficient way to look up objects in both
algorithms.

Second, we need to store mappings for things which are not technically
loose objects but are not packed objects, either, and so cannot be
stored in a pack index.  These kinds of things include shallows, their
parents, and their trees, as well as submodules. Yet we also need to
implement a sensible way to store the kind of object so that we can
prune unneeded entries.  For instance, if the user has updated the
shallows, we can remove the old values.

For these reasons, introduce a new binary object map format.  The
careful reader will notice that it resembles very closely the pack index
v3 format.  Add an in-memory object map as well, and allow writing to a
batched map, which can then be written later as one of the binary object
maps.  Include several tests for round tripping and data lookup across
algorithms.

Note that the use of this code elsewhere in Git will involve some C code
and some C-compatible code in Rust that will be introduced in a future
commit.  Thus, for example, we ignore the fact that if there is no
current batch and the caller asks for data to be written, this code does
nothing, mostly because this code also does not involve itself with
opening or manipulating files.  The C code that we will add later will
implement this functionality at a higher level and take care of this,
since the code which is necessary for writing to the object store is
deeply involved with our C abstractions and it would require extensive
work (which would not be especially valuable at this point) to port
those to Rust.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-07 17:41:03 -08:00
brian m. carlson 3bb0b0afab rust: add a build.rs script for tests
Cargo uses the build.rs script to determine how to compile and link a
binary.  The only binary we're generating, however, is for our tests,
but in a future commit, we're going to link against libgit.a for some
functionality and we'll need to make sure the test binaries are
complete.

Add a build.rs file for this case and specify the files we're going to
be linking against.  Because we cannot specify different dependencies
when building our static library versus our tests, update the Makefile
to specify these dependencies for our static library to avoid race
conditions during build.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-07 17:41:03 -08:00
brian m. carlson b674d1036a rust: add a ObjectID struct
We'd like to be able to write some Rust code that can work with object
IDs.  Add a structure here that's identical to struct object_id in C,
for easy use in sharing across the FFI boundary.  We will use this
structure in several places in hot paths, such as index-pack or
pack-objects when converting between algorithms, so prioritize efficient
interchange over a more idiomatic Rust approach.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-07 17:41:01 -08:00
Junio C Hamano 3c6162ea5c test: optionally test contrib in CI
Recently it was reported that a topic merged to 'next' broke build
and test for contrib/subtree part of the system.

Instead of having those who run 'next' or 'master' to hit the build
and test breakage and report to us, make sure we notice breakages in
contrib/ area before they hit my tree at all, during their own
presubmit testing.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-05 09:26:18 -08:00
Patrick Steinhardt a675183d48 builtin: add new "history" command
When rewriting history via git-rebase(1) there are a few very common use
cases:

  - The ordering of two commits should be reversed.

  - A commit should be split up into two commits.

  - A commit should be dropped from the history completely.

  - Multiple commits should be squashed into one.

  - Editing an existing commit that is not the tip of the current
    branch.

While these operations are all doable, it often feels needlessly kludgey
to do so by doing an interactive rebase, using the editor to say what
one wants, and then perform the actions. Also, some operations like
splitting up a commit into two are way more involved than that and
require a whole series of commands.

Rebases also do not update dependent branches. The use of stacked
branches has grown quite common with competing version control systems
like Jujutsu though, so it clearly is a need that users have. While
rebases _can_ serve this use case if one always works on the latest
stacked branch, it is somewhat awkward and very easy to get wrong.

Add a new "history" command to plug these gaps. This command will have
several different subcommands to imperatively rewrite history for common
use cases like the above.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-13 05:41:17 -08:00
Patrick Steinhardt 6aeda3cf5b builtin/replay: move core logic into "libgit.a"
Move the core logic used to replay commits into "libgit.a" so that it
can be easily reused by other commands. It will be used in a subsequent
commit where we're about to introduce a new git-history(1) command.

Note that with this change we have no sign-comparison warnings anymore,
and neither do we depend on `the_repository`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-13 05:41:16 -08:00
Torsten Bögershausen d281241518 utf8.c: enable workaround for iconv under macOS 14/15
The previous commit introduced a workaround in utf8.c to deal
with broken iconv implementations.

It is enabled when a MacOS version is used that has a buggy
iconv library and there is no external library provided
(and linked against) from neither MacPorts nor Homebrew nor Fink.
For Homebrew, MacPorts and Fink we check if libiconv exist.
Introduce 2 new macros: HAS_GOOD_LIBICONV and NEEDS_GOOD_LIBICONV.

For Homebrew HAS_GOOD_LIBICONV is set when the libiconv directory
exist.
MacPorts can be installed with or without libiconv, so check if
libiconv.dylib exists (which is a softlink)

Fink compiles and installs libiconv by default.
Note that a fresh installation of Fink now defaults to /opt/sw.
Older versions used /sw as default, so leave the check and setting
of BASIC_CFLAGS and BASIC_LDFLAGS as is.
For the new default check for the existance of /opt/sw as well.
Add a check for /opt/sw/lib/libiconv.dylib which sets HAS_GOOD_LIBICONV

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-12 12:00:39 -08:00
Junio C Hamano b39aad0b0d Merge branch 'rs/macos-iconv-workaround'
Workaround the "iconv" shipped as part of macOS, which is broken
handling stateful ISO/IEC 2022 encoded strings.

* rs/macos-iconv-workaround:
  macOS: use iconv from Homebrew if needed and present
  macOS: make Homebrew use configurable
2026-01-06 16:33:52 +09:00
René Scharfe cee341e9dd macOS: use iconv from Homebrew if needed and present
The library function iconv(3) supplied with macOS versions 15.7.2
(Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:

  not ok 17 - ISO-2022-JP should be shown in UTF-8 now
  not ok 25 - ISO-2022-JP should be shown in UTF-8 now
  not ok 38 - commit --fixup into ISO-2022-JP from UTF-8

As a workaround, use libiconv from Homebrew, if available.  Search it in
its default locations: /opt/homebrew for Apple Silicon and /usr/local
for macOS Intel, with the former taking precedence.  Respect ICONVDIR if
already set by the user, though.

Helped-by: Koji Nakamaru <koji.nakamaru@gree.net>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-12-25 16:43:10 +09:00
René Scharfe 363837afe7 macOS: make Homebrew use configurable
On macOS we opportunistically use Homebrew-installed versions of
gettext(3) and msgfmt(1).  Make that behavior configurable by providing
make variables to disable Homebrew usage (NO_HOMEBREW) and to allow
using a non-default installation location (HOMEBREW_PREFIX).

Include and link only the gettext keg via the symlink opt/gettext
pointing to its installed version instead of using the Homebrew prefix.
This is simpler and prevents accidentally including other libraries.

Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Suggested-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-12-25 16:43:09 +09:00
Junio C Hamano 86ebd83e6a Merge branch 'jc/memzero-array'
Further application of MEMZERO_ARRAY() macro to the rest of the
code base.

* jc/memzero-array:
  cocci: use MEMZERO_ARRAY() a bit more
  coccicheck: emit the contents of cocci patch
2025-12-23 11:33:16 +09:00
Junio C Hamano 396df67739 Merge branch 'tc/memzero-array'
MEMZERO_ARRAY() helper is introduced to avoid clearing only the
first N bytes of an N-element array whose elements are larger than
a byte.

* tc/memzero-array:
  contrib/coccinelle: pass include paths to spatch(1)
  git-compat-util: introduce MEMZERO_ARRAY() macro
2025-12-23 11:33:16 +09:00
Junio C Hamano 448673412d Merge branch 'jc/macports-darwinports'
Makefile in-comment doc update.

* jc/macports-darwinports:
  Makefile: help macOS novices by mentioning MacPorts
2025-12-22 14:57:48 +09:00
Junio C Hamano 91bfbf49b6 Merge branch 'rs/ban-mktemp'
Rewrite the only use of "mktemp()" that is subject to TOCTOU race
and Stop using the insecure "mktemp()" function.

* rs/ban-mktemp:
  compat: remove gitmkdtemp()
  banned.h: ban mktemp(3)
  compat: remove mingw_mktemp()
  compat: use git_mkdtemp()
  wrapper: add git_mkdtemp()
2025-12-16 11:08:35 +09:00