Commit Graph

3294 Commits (seen)

Author SHA1 Message Date
Junio C Hamano 82f4a74b59 Merge branch 'mh/rust-crate-subdir' into seen
The Rust code and its 'Cargo.toml' file have been moved from the
top-level repository root and 'src/' directory into a dedicated
'rust/' subdirectory.  This avoids confusing Cargo's packaging
mechanism when the Git repository is included as a submodule in
other Rust projects.

* mh/rust-crate-subdir:
  move rust gitcore crate to a different subdirectory
2026-09-22 10:36:58 -07:00
Junio C Hamano 715eb495b1 Merge branch 'bs/runtime-prefix-obsd-getexecpath' into seen
Git on OpenBSD historically lacked an authoritative mechanism to
resolve its executable path, as it doesn't support the
KERN_PROC_PATHNAME sysctl.  With OpenBSD 8.0 introducing
getexecpath(3), it is now utilized to provide proper RUNTIME_PREFIX
resolution instead of relying on argv[0] fallback.

* bs/runtime-prefix-obsd-getexecpath:
  exec_cmd: RUNTIME_PREFIX on OpenBSD systems
2026-09-22 10:36:57 -07:00
Junio C Hamano e237308b48 Merge branch 'mm/diff-process-hunks' into seen
A new 'diff.<driver>.process' configuration has been introduced to
allow a long-running external process to act as a hunk provider,
enabling external tools to control which lines Git considers changed
while leaving all output formatting (word diff, color, blame, etc.) to
Git's standard pipeline.

* mm/diff-process-hunks:
  fixup! diff: consult oid-only hunk providers via diff.<driver>.process
  diff: consult oid-only hunk providers via diff.<driver>.process
  userdiff: add diff.<driver>.process config
  sub-process: add a gentle status read
  sub-process: separate process lifecycle from hashmap management
  blame: read precomputed hunks
  diff: read precomputed hunks for stat output
  diff: record precomputed hunks during stat output
  diff-hunks: add the store format, library, and command
  diff: introduce a hunk provider interface
  gitattributes: document how external diff drivers relate to diff features
2026-09-22 10:36:55 -07:00
Junio C Hamano d30134abaa Merge branch 'ws/squelch-svn-migrate' into seen
* ws/squelch-svn-migrate:
  Makefile: add NO_GIT_SVN knob to skip building/installing git-svn
  git-svn: don't print v1-layout migration noise when there's nothing to migrate
2026-09-22 10:36:53 -07:00
Junio C Hamano 008c530ac1 Merge branch 'as/utimensat-utimes' into jch
The codebase has been updated to use the newer utimensat() POSIX
function instead of the obsolescent utime(), allowing
high-precision timestamps while preserving fallback compatibility.

* as/utimensat-utimes:
  compat/posix: drop legacy <utime.h> header and shims
  treewide: use utimensat(2) instead of legacy utime(3p)
  compat/posix: introduce utimensat(2) wrapper
2026-09-22 10:36:38 -07:00
Junio C Hamano 37edba3319 Merge branch 'sg/precompile-git-compat-util' into jch
The 'Makefile' has been taught to precompile 'git-compat-util.h' to
speed up overall compilation, while excluding sources that do not
include the compatibility header.

* sg/precompile-git-compat-util:
  Makefile: precompile "git-compat-util.h"
  Makefile: reintroduce REFTABLE_OBJS
  cmake: remove any "$(*_OBJS)" variables when parsing Makefile for sources
  Makefile: remove XDIFF_OBJS initialization
2026-09-22 10:36:35 -07:00
Junio C Hamano 549a4ba707 Merge branch 'dk/use-nsec-runtime' into jch
The build-time knob 'USE_NSEC' for nanosecond stat precision has been
converted to a runtime configuration 'core.useNanosec', allowing
distributions to bundle one binary that adapts to filesystem
capabilities dynamically.

* dk/use-nsec-runtime:
  core: convert build-time USE_NSEC into runtime core.useNanosec
  environment: align repo_config_values_init with struct declaration
  meson: expose knob for xmlto relative links in manuals
2026-09-22 10:36:34 -07:00
Mike Hommey c78f853e37 move rust gitcore crate to a different subdirectory
Having `Cargo.toml` at the top-level of the repository implies that one
can run `cargo build` directly, but this doesn't produce anything useful
on its own.

Additionally, when including the git source as a submodule of a Rust
project, it prevents the git source from being included at all in the
crate package because cargo skips directories that contain a Cargo.toml,
assuming that everything in the directory is relevant to the crate.

Move all Rust-specific files into a dedicated `rust/` subdirectory.

Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-17 13:01:05 -07:00
Brad Smith 49ea669075 exec_cmd: RUNTIME_PREFIX on OpenBSD systems
Enable Git to resolve its own binary location using getexecpath().

Signed-off-by: Brad Smith <brad@comstyle.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-17 11:30:27 -07:00
Junio C Hamano 3a055174da Merge branch 'js/rust-in-windows-ci'
Update Windows CI build to support Rust.

* js/rust-in-windows-ci:
  ci(windows): build with Rust
  rust: pick a GCC-compatible Cargo target under MSYS2/MinGW
2026-09-16 09:04:55 -07:00
SZEDER Gábor 8d3a673147 Makefile: precompile "git-compat-util.h"
This patch follows the idea of 671df48df8 (meson: precompile
"git-compat-util.h", 2026-03-19) to make it faster to build Git using
"make".  The notable differences are the boilerplate needed to wire up
the precompiled header with "make", and the selection of object files
that are built using the precompiled header:

  - Add a new rule to precompile "tools/precompiled.h" into
    "tools/precompiled.h.gch".

    This rule and its dependencies are basically the same as our
    existing rule and its dependencies for compiling object files from
    our source files and, except that in this rule we don't use the
    EXTRA_CPPFLAGS target-specific variable.  This is partly because
    any compiler flags in that target-specific variable can't possibly
    make sense for the project-wide "git-compat-util.h", and partly
    because it could run afoul of "make"'s quirks with target-specific
    variables and GCC's "-Winvalid-pch" compiler option [1].

    Our Makefile always writes object files next to the source files
    they were compiled from.  Since a precompiled header is in many
    ways similar to an object file, let's follow suit and write it
    next to the header file it was created from as well.  671df48df8
    doesn't mention why "precompiled.h" was placed in "tools/".

    Use the ".gch" suffix for the precompiled header file, because
    that's the only suffix GCC looks for when searching for a
    precompiled header.  Clang apparently looks for both ".pch" and
    ".gch" suffixes.  If we ever encounter a compiler which only
    supports a different precompiled header suffix, then we might make
    the suffix configurable via a Makefile knob; but until then it
    remains hard-coded for the sake of simplicity.

  - Declare the precompiled header's dependencies, so it gets rebuilt
    when any of our header files it includes is modified.

    When using computed header dependencies, then the compiler can
    generate the Makefile snippet containing the header files inlcuded
    by the precompiled header, just like when compiling a C source
    file into an object file.

    When not using computed header dependencies, then make the
    precompiled header depend on the same header files that our object
    files depend on, i.e. those listed in LIB_H and GENERATED_H.  This
    is not ideal, because the precompiled header will be rebuilt even
    when a header file it doesn't actually depend on changes; but in
    that case all object files will be rebuilt as well anyway.

  - List the object files that are built using the precompiled header
    in the PRECOMPILED_HEADER_USERS variable:

    - The precompiled header should not change what actually gets
      compiled.  Therefore, use the precompiled header only when
      compiling source files that start with including
      "git-compat-util.h" (directly or indirectly, e.g. via
      "builtin.h"), or its inclusion is only preceeded by #define
      directives that don't influence "git-compat-util.h" between its
      include guards [2] (currently DISABLE_SIGN_COMPARE_WARNINGS,
      USE_THE_REPOSITORY_VARIABLE or GIT_TEST_PROGRESS_ONLY). [3]

      Several (but not all) object or source files listed in the
      COMPAT_OBJS, REFTABLE_OBJS and THIRD_PARTY_SOURCES variables
      don't include "git-compat-util.h", therefore, for the sake of
      simplicity, none of the files listed in these variables are
      built with the precompiled header. [4]

      Since 671df48df8 the Meson build uses the precompiled header
      even when compiling those reftable source files that don't
      include "git-compat-util.h" at all, although this change is not
      mentioned in the commit message.

    - But other than that, use the precompiled header when compiling
      all other source files, including e.g. source files for
      standalone executables (e.g. "daemon.c" for "git-daemon"), or
      those in the directories "t/helper/" and "t/unit-tests/" as
      well.

      The Meson build, however, only uses the precompiled header for
      compiling objects in "libgit_sources" and "builtin_sources",
      despite 671df48df8 giving the impression that the precompiled
      header is included in all compilation units.

    - In short, PRECOMPILED_HEADER_USERS contains all object files
      listed in OBJECTS, except those that are listed in COMPAT_OBJS,
      REFTABLE_OBJS or THIRD_PARTY_SOURCES as well.

  - Add a new rule to build object files listed in
    PRECOMPILED_HEADER_USERS with the precompiled header.

    This rule and its dependencies are basically the same as our
    existing rule and its dependencies compiling object files from our
    source files, except:

    - This rule depends on the precompiled header as well, to make
      sure that it's built before it is used.

    - Use the "-include tools/precompiled.h" option to make the
      precompiled header the first header in the compilation unit,
      because only then is it used to speed things up.

    - Use the "-Winvalid-pch" option to catch any issues when the
      precompiled header is present but can't be used for whatever
      reason.  While being unable to use the precompiled header would
      normally only result in a slower build, it is probably better
      that developers are made aware that simething is not quite
      right.  Meson uses this option as well when compiling a source
      file using the precompiled header.

      Note, that in case of such an issue with "-Winvalid-pch" we
      would get a warning for each object file that is build using the
      precompiled header, but together with "-Werror" (e.g. with
      DEVELOPER=1) it would fail the build.

    Object files not listed in PRECOMPILED_HEADER_USERS are built with
    the existing (and unchanged) rule for object files.

  - This way the precompiled header is compiled only once during
    the whole build process.

    The Meson build, however, currently compiles the precompiled
    header twice: once for "libgit_sources" and once for
    "builtin_sources", despite 671df48df8 giving the impression that
    it's compiled only once.  And, by the looks of it, it would have
    to be compiled once more for each new library or executable where
    we declare the use of the precompiled header.

  - Add the Makefile knob NO_PRECOMPILED_HEADER to make it possible to
    build without using the precompiled header for testing purposes,
    or for any compiler that might be out there that doesn't support
    the "-Winvalid-pch" option.

    Note, that when this knob is set then we merely leave
    PRECOMPILED_HEADER_USERS empty: this way nothing is built with the
    precompiled header, but "make clean NO_PRECOMPILED_HEADER=1" would
    still remove a leftover precompiled header file.

With this patch series on top of v2.55.0 I got the following build
time improvement:

  Benchmark 1: make -j12 (rev = v2.55.0)
    Time (mean ± σ):     29.414 s ±  0.031 s    [User: 254.786 s, System: 47.447 s]
    Range (min … max):   29.370 s … 29.470 s    10 runs

  Benchmark 2: make -j12 (rev = precompile)
    Time (mean ± σ):     21.725 s ±  0.047 s    [User: 186.047 s, System: 35.109 s]
    Range (min … max):   21.643 s … 21.768 s    10 runs

  Summary
    'make -j12 (rev = precompile)' ran
      1.35 ± 0.00 times faster than 'make -j12 (rev = v2.55.0)'

[1] "make" applies target-specific variables to the first target's
    dependencies as well.  So, if the rule creating the precompiled
    header included $(EXTRA_CPPFLAGS) as well, then we could get this:

      $ make -s clean
      $ make V=1 git
      gcc -o tools/precompiled.h.gch -c -MF [...] -DSHELL_PATH='"/bin/sh"' '-DGIT_HTML_PATH="share/doc/git-doc"' '-DGIT_MAN_PATH="share/man"' '-DGIT_INFO_PATH="share/info"' tools/precompiled.h
      gcc -o git.o -c -include tools/precompiled.h -Winvalid-pch -MF [...] -DSHELL_PATH='"/bin/sh"' '-DGIT_HTML_PATH="share/doc/git-doc"' '-DGIT_MAN_PATH="share/man"' '-DGIT_INFO_PATH="share/info"' git.c
      gcc -o builtin/add.o -c -include tools/precompiled.h -Winvalid-pch -MF [...] -DSHELL_PATH='"/bin/sh"'  builtin/add.c
      cc1: error: ./tools/precompiled.h.gch: not used because `GIT_MAN_PATH' not defined [-Werror=invalid-pch]
      cc1: all warnings being treated as errors
      make: *** [Makefile:2940: builtin/add.o] Error 1

    So in this case "git.o" is the first object file to be build, and
    since it has a target-specific EXTRA_CPPFLAGS and since it depends
    on the precompiled header, the precompiled header is built with
    the same EXTRA_CPPFLAGS as well.  But then "builtin/add.o" is
    built with empty EXTRA_CPPFLAGS, and GCC's sanity checks for the
    -Winvalid-pch option complain.  Clang apparently doesn't consider
    this to be an issue, and builds "git" just fine.

    Note, however, that if the first object file to be built is not
    "git.o" (or one of the other object files with target-specific
    EXTRA_CPPFLAGS), which is usually the case with e.g. "make all",
    then the precompiled header is built with empty EXTRA_CPPFLAGS,
    and then all is well.  Weird.  We already had issues with "make"
    and target-specific variables in the past, see a673cfede6
    (Makefile: Fix occasional GIT-CFLAGS breakage, 2010-03-19).
    The Meson build doesn't use target-specific variables, and the
    whole project needs to be rebuilt when e.g. "mandir" is changed,
    although this is not mentioned in 904339edbd (Introduce support
    for the Meson build system, 2024-12-06).

[2] See baa61e46da (git-compat-util.h: move warning infra to prepare
    for PCHs, 2026-03-19).

[3] This can be verified by simply scanning through the output of:

    {
            printf '%s\n' 'print-%:'
            printf '\t%s\n' '@printf "%s\n" $($*)'
    } >/tmp/printvars.mak
    for f in $(make -f /tmp/printvars.mak -f Makefile print-PRECOMPILED_HEADER_USERS)
    do
            f="${f%o}c"
            sed -n -E -e "s%# *(define|include).*%&    $f%p" -e '/# *include/q' "$f"
    done | sort

[4] "make"'s flexibility would allow us to separately list those
    object files that don't include "git-compat-util.h", but then on
    my (I assume fairly typical) Linux box the number of object files
    built using the precompiled header increases only by 9 (from 536
    to 545), which reduces the build time by about 1% (0.2s).
    Therefore, I don't think it's worth the churn.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-15 09:18:06 -07:00
SZEDER Gábor af36bb91f9 Makefile: reintroduce REFTABLE_OBJS
In the next commit we are about to precompile "git-compat-util.h" with
"make" to reduce build times.  But using the precompiled header should
not change what actually gets compiled, therefore a source file can
only be compiled using the precompiled header if the first included
header file is "git-compat-util.h".

The reftable source files are kind of special, because the reftable
implementation is supposed to be easily includable in other projects.
Therefore, the reftable source files don't include
"git-compat-util.h", with the sole exception of the purposefully
project-specific "reftable/system.c".  Consequently, they shouldn't be
compiled with our precompiled header.

List object files under "reftable" in the REFTABLE_OBJS Makefile
variable, so in the next commit we'll be able to easily filter them
out and keep building them the old way, without the precompiled
header.

Note that object files under "reftable/" used to be listed in
REFTABLE_OBJS so we could build a static library from them.  This
static library was removed in f3b4c89d59 (make: delete REFTABLE_LIB,
add reftable to LIB_OBJS, 2025-10-02), along with filling
REFTALBE_OBJS with object files.  This change essentially reverts the
removal of REFTABLE_OBJS, but not the building of that static library.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-15 09:18:05 -07:00
SZEDER Gábor 8f2f265622 Makefile: remove XDIFF_OBJS initialization
Object files under 'xlib/' used to be listed in the XDIFF_OBJS
Makefile variable so we could build a static library from them.  This
static library was removed in cf680cdb95 (make: delete XDIFF_LIB, add
xdiff to LIB_OBJS, 2025-10-02), along with filling XDIFF_OBJS with
object files.

But the initial empty initalization of XDIFF_OBJS remained, so remove
it now.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-15 09:18:05 -07:00
Johannes Schindelin ce6f9ffaa7 rust: pick a GCC-compatible Cargo target under MSYS2/MinGW
When Git is built under MSYS2/MinGW with Rust support enabled, the
Makefile expects `cargo build` to drop a `target/release/libgitcore.a`
that is linkable by the same MinGW GCC used for every other object. With
Rust installed via `rustup` (the way it ships on the GitHub-hosted
`windows-2022` and `windows-11-arm` runners that build git/git and its
forks), the default toolchain targets the MSVC ABI; cargo then writes
`target/release/gitcore.lib` instead, which the MinGW `ld.exe` cannot
consume:

    LINK git-shell.exe
    D:\git-sdk-64-minimal\mingw64\bin/ld.exe: cannot find target/release/libgitcore.a: No such file or directory
    collect2.exe: error: ld returned 1 exit status

See https://github.com/microsoft/git/actions/runs/27341625000 for a
full example log.

Let's define the correct target, using the `CARGO_BUILD_TARGET` variable
that will be picked up by Rust, see
https://dirname.github.io/rust-std-doc/cargo/reference/environment-variables.html#:~:text=CARGO%5FBUILD%5FTARGET

Re-use (and fix) the existing `HOST_CPU` variable to determine the
correct value. Avoid relying on environment variables that are simply
not defined in Git for Windows' minimal SDK that Git uses in its CI
runs.

Assisted-by: Claude Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-13 15:48:08 -07:00
D. Ben Knoble 3a21fe8cf9 core: convert build-time USE_NSEC into runtime core.useNanosec
Racy Git problems persist today, manifesting themselves in the
performance of commands like "git diff" in new worktrees [1]. We have
long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond
precision when available, which mitigates most if not all racy issues,
but most builds we know about don't use it. In part, that's because
someone distributing Git can't safely enable it at compile-time if they
don't know exactly what platforms their distribution will be used on.

[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com

These days, most platforms are likely to be safe for the USE_NSEC code.
Regardless, we want to give users the ability to benefit from it. This
requires exposing the compile-time gated code as a runtime option.

In addition, update the Racy Git documentation and other mentions of
USE_NSEC in the code.

Due to the conversion from #ifdef to runtime check, using the flag
"--ignore-space-change" may be particularly helpful when viewing changes
from this patch.

Signed-off-by: D. Ben Knoble <ben.knoble@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-11 11:10:01 -07:00
Michael Montalbo 74fcf26527 diff: consult oid-only hunk providers via diff.<driver>.process
The provider chain so far holds the diff-hunks store in front of the
terminal builtin computation.  Open it to external processes: a pair on
a path whose driver configures diff.<driver>.process is answered by a
long-running process speaking a pkt-line protocol (following the filter
process protocol), registered at the head of the chain and consulted
before the store and before any blob is loaded.

The protocol starts with the smallest request that can carry an answer:
object names alone.  A request is the pathname and the pair's
old-oid/new-oid, with no content.  The process answers with hunk lines,
with a zero-hunk success that asserts the blobs equivalent (trailing
newlines included), or with status=need-content, on which the pair
falls through to the builtin answer.  This serves the two shapes that
need no content pushed to them: a cache keyed on the blob pair, and a
process that fetches the blobs itself (for example over "git cat-file
--batch").  A pair whose side is not a stored blob carries a NULL id;
the provider sends no request and passes it.  Because Git holds no
content for the exchange, the answer is used as sent: hunks are
validated for order, overlap, lockstep alignment, and magnitude, then
replayed without the normalization xdiff applies to diffs it computes
itself.  The magnitude bound is the blobs' sizes, read from the object
database without loading content: a blob of N bytes holds at most N
lines.

Because the process's answer is authoritative, it outranks the store,
and its head-of-chain position says so.  A pair the process answers
never reaches the store and is never recorded, so nothing it produces
enters the store, which holds the builtin answer only.  A request it
does not answer, whether need-content, a missing capability, or a
missing id, passes down the chain to the builtin answer, which is what
the store serves, so the store may serve such a pair and a warming run
may record it.  Entries recorded before a process was configured are
not purged; a pair the process answers ignores them, and "git diff-hunks
clear" discards them.

The provider gates itself per request.  The driver is looked up by the
old-side path, so a renamed file resolves to the same driver, and by
the repository-relative path, so a diff.relative run from a
subdirectory names the pair the same way.  Options the process is never
told about select no process: the whitespace-ignoring options, -I,
--anchored, and an algorithm forced by option or configuration (blame
routes its algorithm through xdl_opts, so --histogram is covered).  The
request gains its last field, the path; the consumers change only by
filling it, and neither names the process.

The provider's state is its repository's pool of running processes,
keyed by the configured command, so drivers sharing a command share a
process, a submodule speaks to its own, and releasing the provider
(from repo_clear()) stops them.  The pool owns a copy of each command
string, so an entry outlives a config re-read.  A command that fails
stays as an entry that is not retried: its request and every later one
pass, so the store may serve the path for the rest of the command.

A protocol error in a response never kills the command.  The response
is read through a packet reader gentle about framing, so an error takes
one path: a single warning, the process stopped and marked failed, and
the builtin diff for the rest of the command.  That covers garbage
bytes, a truncated response, an empty packet, a bare status, and an
unrecognized status.  Semantically invalid coordinates cost only their
pair: the response is drained, the pair is computed, and the process
stays alive.  A path the protocol cannot carry (an embedded newline, or
one too long for a packet) falls back per path rather than costing the
command its process.  The handshake keeps one fatal check: a process
that announces a capability Git did not request aborts the command, as
the long-running filter protocol does.

Consulting is allowed per command, following the allow_textconv
precedent.  "git diff", "git log" and "git show", and "git blame" set
allow_diff_process; the plumbing diff commands and the interactive-patch
machinery never set it, so scripted and staging output stays builtin.
The options adjust the flag:

- --no-ext-diff clears it and --ext-diff sets it;
- --diff-process and --no-diff-process set and clear it alone, leaving
  external diff drivers as they were;
- format-patch clears it unconditionally, so a generated patch applies
  for recipients without the process;
- range-diff passes --no-ext-diff to the "git log" it compares.

git blame and the summary formats consult the process.  For blame, a
pair reported equivalent emits no hunks, so the whole commit passes to
its parent.  In the stat formats such a pair sums to a zero-count entry,
which the "nothing changed" rule omits, as under -w.  The subprocess is
long-running: one startup cost across a traversal, one round-trip per
consulted pair.  Answers travel in struct xdl_hunk, new in
xdiff-interface.h, holding xdiff's 1-based coordinates; nothing feeds
them back to xdiff, since only coordinate consumers consult.

A content-carrying request is the natural extension: it would serve
sides that are not stored blobs and processes that want content pushed
to them, and bring patch output and log -L's range tracking to the same
answer.  As it stands, a process's answers show in blame and the summary
formats while patch output stays builtin.

t4080 exercises the protocol, the per-command gate, and the error paths:

- each adversarial response shape warns and falls back to builtin, the
  request log proving which failures disable the process and which keep
  it alive (a malformed hunk line, coordinates past the blob size, a
  count overflowing strtol(), overlapping or misaligned hunks, an
  unrecognized status, a bare status, an empty packet, a mid-response
  crash, and raw garbage);
- a capability-less process and status=abort degrade without noise, and
  a failed start warns once and returns the path to the store;
- a trailing token on a hunk line is ignored, pinning field
  appendability;
- positive consults for git diff, git show, and diff-tree under
  --ext-diff and --diff-process; textconv output and gitlink sides are
  never identified; a diff.relative run consults by the repo-relative
  path;
- the equivalence answer is pinned from both consumers, and a warming
  run past a deferring process records the pair for a later read.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-03 07:43:11 -07:00
Michael Montalbo 3dc1d97529 diff-hunks: add the store format, library, and command
Blame and "git log --stat" recover hunk coordinates by diffing blob
pairs, and recompute them on every run.  Add a cache of those
coordinates at $GIT_DIR/objects/info/diff-hunks, beside the
commit-graph, so a later run can look them up instead of decompressing
the blobs and running xdiff again.

The store is a single chunk-format file (see gitformat-chunk(5)): an
8-byte header, a DHIX index of fixed-size entries sorted by key, a DHDT
segment of hunk records, and a trailing hash checksum.  An entry is
keyed by the two blob object ids and the xdl_opts the pair was diffed
under, so a stored result is served only where that exact key recurs,
independent of path.  A zero-context diff trims unchanged lines from
hunk edges and can pick a different but equally valid set of hunks than
an untrimmed diff, so a recording caller stores a pair only when its
trimmed and untrimmed diffs are identical; such an entry answers any
consumer at any context, and the rare divergent pair is always
computed.  Identical hunk blocks are interned once and shared across
keys.

The library provides a reader (repo_diff_hunks_store and _replay, gated
by core.diffHunks), loaded once and cached on the object database as the
commit-graph is, and a writer that accumulates entries and flushes them
in one atomic pass.  An absent, corrupt, or disabled store reads as all
misses.  A record with no hunks is invalid too: replaying it would claim
the pair equivalent, which the store never asserts, so it reads as a
miss.

Ordinary reads are diagnostic-free.  Loading parses the chunk table
through read_table_of_contents_quiet(), new in chunk-format, which
prints nothing on a malformed table and takes the repository's hash
algorithm rather than the_hash_algo, so the file is bounds-checked under
the algorithm it is keyed by.

The flush closes the repository's mmapped store and forgets that loading
was attempted before committing the lockfile.  A warming run that also
reads may hold the file it is replacing mapped, and the rename must not
land on a live mapping, which Windows refuses; a read after the flush
then observes the committed file.  commit-graph closes its graph before
committing for the same reason.

Writing is off by default, enabled per run by GIT_DIFF_HUNKS_WRITE or
persistently by diffHunks.write, the environment winning.  A writer
seeds from the existing store, so a flush merges rather than replaces.
The seed's checksum is verified first: a corrupt store is discarded, not
rewritten with a fresh checksum verify could no longer catch.  An entry
that fails the shared diff_provider_check_hunk() or names no blob is
dropped with a warning, since it would only ever read as a miss.  A seed
that discarded or dropped anything forces the flush even when the
warming run computed nothing new.  The writer fsyncs through a new
diff-hunks core.fsync component.

"git diff-hunks" inspects and manages the file: "verify" checks the
checksum, chunk table, sort order, entry bounds, and every entry's hunk
sequence against that shared check, so a store whose entries could only
read as misses fails verify; "clear" removes the file.  Later patches
wire the readers and the writer into the diff and blame paths.

Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-03 07:43:08 -07:00
Michael Montalbo 3dc7f5d1e3 diff: introduce a hunk provider interface
To learn which line ranges changed between two blobs, every consumer in
the diff machinery loads both blobs and runs xdiff.  There is no other
way to supply that answer, even when it is known elsewhere: a cache may
hold the ranges from the last time the pair was diffed, and a
format-aware process may have its own idea of which lines changed.
Either could answer from the blob object ids alone, but the loading and
computing are hard-wired into each consumer, so such an answer has no
place to enter.

Introduce the hunk provider interface, diff-provider.h, between asking
the question and computing the answer.  A provider answers a request
made of the pair's identity, its blob object ids and the parameters
that determine the diff.  A provider is either authoritative, so its
answer may deliberately differ from the builtin diff, or not, so its
answer must reproduce the builtin result exactly.  Every answer served
from identity passes diff_provider_check_hunk() before a consumer sees
it: coordinates fit int32, hunks are ordered and non-overlapping, and
the unchanged runs between them match on both sides.  A failing answer
is discarded and the pair falls through as unanswered.

Providers are repository-lifecycle objects.  Each repository owns a
chain of them, built on first consultation and released from
repo_clear(), so a submodule gets its own providers and no provider
state outlives the repository it serves.  The chain has a fixed
composition, and each provider gates itself per request, passing when
it does not apply.  Chain order is the authority: the first answer
wins.  A provider may instead refuse a pair whose request is shaped by
parameters its recording key cannot express.  After a refusal, no later
provider answers the pair from identity, and the consumer must not
record what it computes for it.  The last provider is the builtin
computation, the only one that computes rather than answering from
identity, so a walk given a fill callback always ends in an answer,
refusal or not.

The walk in diff-provider.c maps a provider's four dispositions
(answer, pass, fail, refuse) onto the consumer-facing outcomes, and
checks with BUG() that only the computing provider fails and that it
passes on a walk with no fill callback.  The implementor contract, the
provider struct, its dispositions, and the shared check, lives in
diff-provider-internal.h, as refs/refs-internal.h is to refs.h;
consumers see only diff-provider.h.

The consumer surface is two types.  struct diff_provider_request names
what is diffed and under which parameters; each later commit that
consults on more state adds the field it keys on (the object ids and
diff options, then the path).  enum diff_provider_outcome flattens two
dependent axes into four points: the response state (answered,
unanswered, failed) and, only when unanswered, whether the caller may
record what it computes.  The record rule rides in the outcome, not a
separate flag, so -Wswitch forces every consumer to place the no-record
arm.  A provider added later maps onto these values inside the walk, so
consumer code is written once.

diff_provider_emit_hunks() is the consumer entry: the caller states the
request, a hunk callback, and a content-loading callback that reaches
the terminal provider only when the ranges are computed.  Blame's
pass_blame_to_parent() is the first consumer, since it knows both blob
ids before reading either blob; its loads move into the fill callback.
With only the terminal provider registered, every request still
computes, so behavior is unchanged.  (Blame's -C/-M split detection
diffs partial buffers with no blob identity and stays on xdi_diff().)

Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-09-03 07:42:20 -07:00
Wesley Schwengle c618e8f8bd Makefile: add NO_GIT_SVN knob to skip building/installing git-svn
This option also implies that NO_SVN_TESTS is enabled.

Signed-off-by: Wesley Schwengle <wesleys@opperschaap.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-08-27 21:08:03 -07:00
Alexey Samsonov be2b0a8f00 compat/posix: introduce utimensat(2) wrapper
In POSIX.1-2008, utime(3p) was marked as obsolescent in favor of
utimensat(2) and futimens(2). In the recent POSIX.1-2024 (Issue 8)
specification, <utime.h> and utime(3p) were officially removed.

utimensat(2) operates on `struct timespec` rather than the second-only
`struct utimbuf`, allowing sub-second timestamp updates while also
providing support for UTIME_NOW and UTIME_OMIT flags to selectively
update or preserve individual access and modification timestamps.

Introduce a compatibility layer for utimensat(2):
- Provide fallback definitions for AT_FDCWD, UTIME_NOW, and UTIME_OMIT
  in case the system headers lack them.
- Introduce `ST_ATIME_NSEC(st)` to complement `ST_MTIME_NSEC(st)` and
  `ST_CTIME_NSEC(st)`.
- Implement `git_utimensat()` in `compat/utimensat.c` as a fallback using
  utimes(2) on platforms that define NO_UTIMENSAT.
- Implement `mingw_utimensat()` in `compat/mingw.c` converting `struct
  timespec` to Windows FILETIME with 100ns precision.
- Wire up NO_UTIMENSAT support in Makefile, meson.build,
  contrib/buildsystems/CMakeLists.txt, and configure.ac.

Subsequent commits will migrate callers across the codebase to
utimensat(2) and drop the legacy <utime.h> header.

Signed-off-by: Alexey Samsonov <vonosmas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-08-21 10:24:33 -07:00
Junio C Hamano 3beb8bb742 Merge branch 'ps/writev'
A compatibility wrapper for writev(3p) has been reintroduced,
including fixes for CMake build and 'MAX_IO_SIZE' limits on NonStop.
Calls to write(3p) in send_sideband() and cat_blob() have been
refactored to use writev(3p) wrappers to reduce syscall overhead.

* ps/writev:
  fast-import: use writev(3p) to send cat-blob responses
  sideband: use writev(3p) to send pktlines
  wrapper: properly handle MAX_IO_SIZE in writev(3p)
  wrapper: introduce writev(3p) wrappers
  compat/posix: introduce writev(3p) wrapper
2026-08-18 09:31:36 -07:00
Patrick Steinhardt 8b0ab33247 compat/posix: introduce writev(3p) wrapper
In a subsequent commit we're going to add the first caller to
writev(3p). Introduce a compatibility wrapper for this syscall that we
can use on systems that don't have this syscall.

The syscall exists on modern Unixes like Linux and macOS, and seemingly
even for NonStop according to [1]. It doesn't seem to exist on Windows
though.

[1]: http://nonstoptools.com/manuals/OSS-SystemCalls.pdf
[2]: https://www.gnu.org/software/gnulib/manual/html_node/writev.html

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-08-07 08:57:25 -07:00
Junio C Hamano e1fb641cfa Merge branch 'ps/cat-file-remote-object-info'
The 'remote-object-info' command has been added to 'git cat-file
--batch-command', allowing clients to request object metadata
(currently size) from a remote server via protocol v2 without
downloading the entire object.  Format placeholders are dynamically
filtered on the client based on server-advertised capabilities,
returning empty strings for inapplicable or unsupported fields.

* ps/cat-file-remote-object-info:
  cat-file: make remote-object-info allow-list adapt to the server
  cat-file: add remote-object-info to batch-command
  transport: add client support for object-info
  serve: advertise object-info feature
  protocol-caps: check object existence regardless of the attributes requested
  fetch-pack: move fetch initialization
  connect: make write_fetch_command_and_capabilities() more generic
  fetch-pack: move write_fetch_command_and_capabilities() to connect.c
  fetch-pack: use unsigned int for hash_algo variable
  fetch-pack: drop the static advertise_sid variable
  t1006: extract helper functions into new 'lib-cat-file.sh'
  cat-file: declare loop counter inside for()
  transport-helper: fix memory leak of helper on disconnect
2026-07-30 10:32:04 -07:00
Calvin Wan 12a19eec1d transport: add client support for object-info
Sometimes, it is beneficial to retrieve information about an object
without downloading it entirely. The server-side logic for this
functionality was implemented in commit "a2ba162cda (object-info:
support for retrieving object info, 2021-04-20)." And the wire
format is documented at
https://git-scm.com/docs/protocol-v2#_object_info.

Introduce client-side support for the object-info capability.

Add its own function for object-info separate from existing fetch
infrastructure.

Currently, the client supports requesting a list of OIDs with the size
attribute from a v2 server. If the server does not advertise this
feature (i.e., transfer.advertiseobjectinfo is set to false), the client
returns an error and exits.

Note that:

1. The entire request is written into req_buf before being sent to the
   remote. This approach follows the pattern used in the
   send_fetch_request() logic within 'fetch-pack.c'. Streaming the
   request is not addressed in this patch.

2. A new field 'unrecognized' has been added to object_info. This new
   field is set at fetch_object_info() when the object is unrecognized
   by the server.

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-24 08:46:59 -07:00
Junio C Hamano f8ec2c6aee Merge branch 'sn/osxkeychain-rust-universal'
The build system has been updated to support building universal macOS
binaries when 'Rust' is enabled, by compiling separate static archives
for each target triple listed in 'RUST_TARGETS' and combining them
using the macOS 'lipo' tool.  The 'git-credential-osxkeychain' helper
has been updated to link against '$(RUST_LIB)' when 'Rust' is enabled.

* sn/osxkeychain-rust-universal:
  contrib: wire up osxkeychain in contrib/Makefile on macOS
  Makefile: support universal macOS builds via RUST_TARGETS
  Makefile: add $(RUST_LIB) prerequisite to osxkeychain
2026-07-19 10:42:17 -07:00
Junio C Hamano 65f37b632a Merge branch 'ps/reftable-hardening'
The 'reftable' code has been hardened against corrupted tables by
fixing out-of-bounds writes, out-of-bounds reads, and abort calls
during parsing.

* ps/reftable-hardening:
  reftable/table: fix OOB read on truncated table
  reftable/table: fix NULL pointer access when seeking to bogus offsets
  reftable/block: fix OOB read with bogus restart offset
  reftable/block: fix use of uninitialized memory when binsearch fails
  reftable/block: fix OOB read with bogus restart count
  reftable/block: fix OOB read with bogus block size
  reftable/block: fix OOB write with bogus inflated log size
  t/unit-tests: introduce test helper to write reftable blocks
  reftable/record: don't abort when decoding invalid ref value type
  reftable/basics: fix OOB read on binary search of empty range
  oss-fuzz: add fuzzer for parsing reftables
  meson: support building fuzzers with libFuzzer
2026-07-19 10:42:15 -07:00
Shardul Natu 924dfced6b Makefile: support universal macOS builds via RUST_TARGETS
On macOS, Universal Binaries contain native executable code for
multiple architectures (such as Intel x86_64 and Apple Silicon arm64)
bundled into a single file. This is standard practice for macOS
distribution and CI packaging (such as internal distribution packages
or tooling like Burrito/Homebrew), allowing a single build artifact
to run natively across all Macs without Rosetta emulation or
maintaining separate packages.

When building Git C code for multiple architectures on macOS, the
Apple toolchain (clang) natively supports universal builds via
CFLAGS/LDFLAGS. When "-arch x86_64 -arch arm64" is passed, clang
automatically compiles and links universal binaries for all C object
files and executables out of the box.

Cargo and rustc, however, do not support multiple "-arch" flags or
emitting universal binaries in a single invocation. Instead, Cargo
requires invoking each target triple independently (e.g., passing
"--target x86_64-apple-darwin" and "--target aarch64-apple-darwin").

To bridge this gap when Rust is enabled:
  1. Allow specifying space-separated target triples in RUST_TARGETS.
  2. Introduce declarative pattern rules (target/%/...) to compile
     each target-specific library slice via Cargo.
  3. On macOS, if multiple targets are specified, use "lipo" (part of
     the mandatory Xcode Command Line Tools) to combine the resulting
     static libraries into target/release/libgitcore.a.

Once $(RUST_LIB) is compiled into a universal static archive, the
standard C linker seamlessly links it with the C object files to
produce universal Git executables.

Signed-off-by: Shardul Natu <snatu@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-07 21:58:54 -07:00
Shardul Natu 87bd9bd40e Makefile: add $(RUST_LIB) prerequisite to osxkeychain
When Rust is enabled, the git-credential-osxkeychain helper depends on
Rust symbols compiled into $(RUST_LIB). While commit 522ea8ef7d
("osxkeychain: fix build with Rust") updated the linker command line to
use $(LIBS), it omitted $(RUST_LIB) from the target prerequisite list.
Without this prerequisite, running a parallel build ("make -j") from a
clean working tree can fail because Make does not know to invoke Cargo
to build libgitcore.a before linking git-credential-osxkeychain.

Note that we depend explicitly on $(LIB_FILE) and $(RUST_LIB) rather
than $(GITLIBS). Unlike standard Git builtins and programs like scalar
(which define cmd_main() and rely on common-main.o to supply main()),
git-credential-osxkeychain.c defines its own standalone int main().
If $(GITLIBS) were used, $(filter %.o,$^) in the link recipe would
match both git-credential-osxkeychain.o and common-main.o, causing a
duplicate symbol linking error for _main on macOS.

Additionally, wrap the definitions of $(RUST_LIB) and the "rust" build
target in "ifndef NO_RUST". This ensures that when NO_RUST=1 is
specified, $(RUST_LIB) evaluates to empty, making the Rust dependency a
clean no-op without needing intermediate variables.

Signed-off-by: Shardul Natu <snatu@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-07 21:58:54 -07:00
Junio C Hamano a7f8e84772 Merge branch 'ps/odb-source-packed'
The packed object source has been refactored into a proper struct
odb_source.

* ps/odb-source-packed:
  odb/source-packed: drop pointer to "files" parent source
  midx: refactor interfaces to work on "packed" source
  odb/source-packed: stub out remaining functions
  odb/source-packed: wire up `freshen_object()` callback
  odb/source-packed: wire up `find_abbrev_len()` callback
  odb/source-packed: wire up `count_objects()` callback
  odb/source-packed: wire up `for_each_object()` callback
  odb/source-packed: wire up `read_object_stream()` callback
  odb/source-packed: wire up `read_object_info()` callback
  packfile: use higher-level interface to implement `has_object_pack()`
  odb/source-packed: wire up `reprepare()` callback
  odb/source-packed: wire up `close()` callback
  odb/source-packed: start converting to a proper `struct odb_source`
  odb/source-packed: store pointer to "files" instead of generic source
  packfile: move packed source into "odb/" subsystem
  packfile: split out packfile list logic
  packfile: rename `struct packfile_store` to `odb_source_packed`
2026-07-06 15:50:21 -07:00
Patrick Steinhardt adf45165e6 oss-fuzz: add fuzzer for parsing reftables
Add a new fuzzer that exercises our parsing of reftables. Fallout from
this fuzzer will be fixed over subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-03 11:08:02 -07:00
Junio C Hamano 88c737a9e6 Merge branch 'js/osxkeychain-build-wo-rust'
Build fix.

* js/osxkeychain-build-wo-rust:
  osxkeychain: fix build with Rust
2026-06-17 05:39:13 -07:00
Patrick Steinhardt c71a121415 packfile: move packed source into "odb/" subsystem
In subsequent patches we'll be turning `struct odb_source_packed` into a
proper `struct odb_source`. As a first step towards this goal, move its
struct out of "packfile.{c,h}" and into "odb/source-packed.{c,h}".

This detaches the implementation of the packfile object source from the
generic packfile code, following the same convention already used by the
"files" and "in-memory" sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-17 05:00:00 -07:00
Patrick Steinhardt cf86a18ac3 packfile: split out packfile list logic
In the next commit we're about to introduce the "packed" object database
source. This source will embed a packfile list, and consequently we'll
have to include "packfile.h" to make the struct definition available.
This will unfortunately lead to a cyclic dependency that we cannot
resolve with a forward declaration.

Split out the code that relates to the packfile list into a separate
compilation unit so that both "packfile.h" and "odb/source-packed.h" can
include it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-17 05:00:00 -07:00
Johannes Schindelin 522ea8ef7d osxkeychain: fix build with Rust
Without NO_RUST defined, the varint encoder/decoder lives in the
RUST_LIB, which needs to be linked. Symptom:

cc [... -o contrib/credential/osxkeychain/git-credential-osxkeychain [...]
Undefined symbols for architecture x86_64:
  "_decode_varint", referenced from:
      _read_untracked_extension in libgit.a[x86_64][63](dir.o)
      _read_untracked_extension in libgit.a[x86_64][63](dir.o)
      _read_one_dir in libgit.a[x86_64][63](dir.o)
      _read_one_dir in libgit.a[x86_64][63](dir.o)
      _load_cache_entry_block in libgit.a[x86_64][174](read-cache.o)
  "_encode_varint", referenced from:
      _write_untracked_extension in libgit.a[x86_64][63](dir.o)
      _write_untracked_extension in libgit.a[x86_64][63](dir.o)
      _write_untracked_extension in libgit.a[x86_64][63](dir.o)
      _write_one_dir in libgit.a[x86_64][63](dir.o)
      _write_one_dir in libgit.a[x86_64][63](dir.o)
      _do_write_index in libgit.a[x86_64][174](read-cache.o)
ld: symbol(s) not found for architecture x86_64

While it is curious why these functions are needed at all (osxkeychain
does not read or write the index), the compile error is a real problem.

Instead of trying to play games to add `GITLIBS` while filtering out
`common-main.o`, replace the `$(LIB_FILE) $(EXTLIBS)` construct with the
much shorter `$(LIBS)` construct that _already_ filters out
`common-main.o` and adds the Rust library when needed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-17 04:54:51 -07:00
Junio C Hamano 06f63df846 Merge branch 'ps/odb-source-loose'
The loose object source has been refactored into a proper `struct
odb_source`.

* ps/odb-source-loose:
  odb/source-loose: drop pointer to the "files" source
  odb/source-loose: stub out remaining callbacks
  odb/source-loose: wire up `write_object_stream()` callback
  object-file: refactor writing objects to use loose source
  odb/source-loose: wire up `write_object()` callback
  loose: refactor object map to operate on `struct odb_source_loose`
  odb/source-loose: wire up `freshen_object()` callback
  odb/source-loose: drop `odb_source_loose_has_object()`
  odb/source-loose: wire up `count_objects()` callback
  odb/source-loose: wire up `find_abbrev_len()` callback
  odb/source-loose: wire up `for_each_object()` callback
  odb/source-loose: wire up `read_object_stream()` callback
  odb/source-loose: wire up `read_object_info()` callback
  odb/source-loose: wire up `close()` callback
  odb/source-loose: wire up `reprepare()` callback
  odb/source-loose: start converting to a proper `struct odb_source`
  odb/source-loose: store pointer to "files" instead of generic source
  odb/source-loose: move loose source into "odb/" subsystem
2026-06-11 04:31:18 -07:00
Junio C Hamano f985a6ec65 Merge branch 'ps/odb-source-loose' into ps/odb-source-packed
* ps/odb-source-loose:
  odb/source-loose: drop pointer to the "files" source
  odb/source-loose: stub out remaining callbacks
  odb/source-loose: wire up `write_object_stream()` callback
  object-file: refactor writing objects to use loose source
  odb/source-loose: wire up `write_object()` callback
  loose: refactor object map to operate on `struct odb_source_loose`
  odb/source-loose: wire up `freshen_object()` callback
  odb/source-loose: drop `odb_source_loose_has_object()`
  odb/source-loose: wire up `count_objects()` callback
  odb/source-loose: wire up `find_abbrev_len()` callback
  odb/source-loose: wire up `for_each_object()` callback
  odb/source-loose: wire up `read_object_stream()` callback
  odb/source-loose: wire up `read_object_info()` callback
  odb/source-loose: wire up `close()` callback
  odb/source-loose: wire up `reprepare()` callback
  odb/source-loose: start converting to a proper `struct odb_source`
  odb/source-loose: store pointer to "files" instead of generic source
  odb/source-loose: move loose source into "odb/" subsystem
2026-06-05 22:26:06 +09:00
Patrick Steinhardt 514f039c90 odb/source-loose: move loose source into "odb/" subsystem
In subsequent patches we'll be turning `struct odb_source_loose` into a
proper `struct odb_source`. As a first step towards this goal, move its
struct out of "object-file.c" and into "odb/source-loose.c".

This detaches the implementation of the loose object source from the
generic object file code, following the same convention already used by
the "files" and "in-memory" sources.

No functional changes are intended.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01 18:47:17 +09:00
Junio C Hamano 4d11b9c218 Merge branch 'pt/fsmonitor-linux'
The fsmonitor daemon has been implemented for Linux.

* pt/fsmonitor-linux:
  fsmonitor: convert shown khash to strset in do_handle_client
  fsmonitor: add tests for Linux
  fsmonitor: add timeout to daemon stop command
  fsmonitor: close inherited file descriptors and detach in daemon
  run-command: add close_fd_above_stderr option
  fsmonitor: implement filesystem change listener for Linux
  fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c
  fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c
  fsmonitor: use pthread_cond_timedwait for cookie wait
  compat/win32: add pthread_cond_timedwait
  fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon
  fsmonitor: fix khash memory leak in do_handle_client
  t9210, t9211: disable GIT_TEST_SPLIT_INDEX for scalar clone tests
2026-05-31 10:00:38 +09:00
Junio C Hamano 0839e56957 Merge branch 'ps/odb-in-memory'
Add a new odb "in-memory" source that is meant to only hold
tentative objects (like the virtual blob object that represents the
working tree file used by "git blame").

* ps/odb-in-memory:
  t/unit-tests: add tests for the in-memory object source
  odb: generic in-memory source
  odb/source-inmemory: stub out remaining functions
  odb/source-inmemory: implement `freshen_object()` callback
  odb/source-inmemory: implement `count_objects()` callback
  odb/source-inmemory: implement `find_abbrev_len()` callback
  odb/source-inmemory: implement `for_each_object()` callback
  odb/source-inmemory: convert to use oidtree
  oidtree: add ability to store data
  cbtree: allow using arbitrary wrapper structures for nodes
  odb/source-inmemory: implement `write_object_stream()` callback
  odb/source-inmemory: implement `write_object()` callback
  odb/source-inmemory: implement `read_object_stream()` callback
  odb/source-inmemory: implement `read_object_info()` callback
  odb: fix unnecessary call to `find_cached_object()`
  odb/source-inmemory: implement `free()` callback
  odb: introduce "in-memory" source
2026-05-27 14:15:46 +09:00
Junio C Hamano 2f952b81ed Merge branch 'jt/odb-transaction-write'
ODB transaction interface is being reworked to explicitly handle
object writes.

* jt/odb-transaction-write:
  odb/transaction: make `write_object_stream()` pluggable
  object-file: generalize packfile writes to use odb_write_stream
  object-file: avoid fd seekback by checking object size upfront
  object-file: remove flags from transaction packfile writes
  odb: update `struct odb_write_stream` read() callback
  odb/transaction: use pluggable `begin_transaction()`
  odb: split `struct odb_transaction` into separate header
2026-05-27 14:15:45 +09:00
Junio C Hamano 9ebc19b760 Merge branch 'ps/odb-in-memory' into ps/odb-source-loose
* ps/odb-in-memory: (24 commits)
  t/unit-tests: add tests for the in-memory object source
  odb: generic in-memory source
  odb/source-inmemory: stub out remaining functions
  odb/source-inmemory: implement `freshen_object()` callback
  odb/source-inmemory: implement `count_objects()` callback
  odb/source-inmemory: implement `find_abbrev_len()` callback
  odb/source-inmemory: implement `for_each_object()` callback
  odb/source-inmemory: convert to use oidtree
  oidtree: add ability to store data
  cbtree: allow using arbitrary wrapper structures for nodes
  odb/source-inmemory: implement `write_object_stream()` callback
  odb/source-inmemory: implement `write_object()` callback
  odb/source-inmemory: implement `read_object_stream()` callback
  odb/source-inmemory: implement `read_object_info()` callback
  odb: fix unnecessary call to `find_cached_object()`
  odb/source-inmemory: implement `free()` callback
  odb: introduce "in-memory" source
  odb/transaction: make `write_object_stream()` pluggable
  object-file: generalize packfile writes to use odb_write_stream
  object-file: avoid fd seekback by checking object size upfront
  ...
2026-05-21 22:34:55 +09:00
Junio C Hamano 686213114e Merge branch 'mm/git-url-parse'
The internal URL parsing logic has been made accessible via a new
subcommand "git url-parse".

* mm/git-url-parse:
  t9904: add tests for the new url-parse builtin
  doc: describe the url-parse builtin
  builtin: create url-parse command
  urlmatch: define url_parse function
  url: return URL_SCHEME_UNKNOWN instead of dying
  url: move scheme detection to URL header/source
  url: move url_is_local_not_ssh to url.h
  connect: rename enum protocol to url_scheme
2026-05-21 12:06:48 +09:00
Junio C Hamano 91ddfe3d5c Merge branch 'js/mingw-no-nedmalloc'
Stop using unmaintained custom allocator in Windows build which was
the last user of the code.

* js/mingw-no-nedmalloc:
  mingw: remove the vendored compat/nedmalloc/ subtree
  mingw: drop the build-system plumbing for nedmalloc
  mingw: stop using nedmalloc
2026-05-20 10:30:56 +09:00
Junio C Hamano a6876b2068 Merge branch 'js/objects-larger-than-4gb-on-windows'
Update code paths that assumed "unsigned long" was long enough for
"size_t".

* js/objects-larger-than-4gb-on-windows:
  ci: run expensive tests on push builds to integration branches
  t5608: mark >4GB tests as EXPENSIVE
  test-tool synthesize: add precomputed SHA-256 pack for 4 GiB + 1
  test-tool synthesize: precompute pack for 4 GiB + 1
  test-tool synthesize: use the unsafe hash for speed
  t5608: add regression test for >4GB object clone
  test-tool: add a helper to synthesize large packfiles
  delta, packfile: use size_t for delta header sizes
  odb, packfile: use size_t for streaming object sizes
  git-zlib: handle data streams larger than 4GB
  index-pack, unpack-objects: use size_t for object size
2026-05-20 10:30:56 +09:00
Junio C Hamano 056472b82d Merge branch 'kh/name-rev-custom-format'
A new builtin "git format-rev" is introduced for pretty formatting
one revision expression per line or commit object names found in
running text.

* kh/name-rev-custom-format:
  format-rev: introduce builtin for on-demand pretty formatting
  name-rev: make dedicated --annotate-stdin --name-only test
  name-rev: factor code for sharing with a new command
  name-rev: run clang-format before factoring code
  name-rev: wrap both blocks in braces
2026-05-19 09:57:44 +09:00
Patrick Steinhardt d2902a4549 t/unit-tests: add tests for the in-memory object source
While the in-memory object source is a full-fledged source, our code
base only exercises parts of its functionality because we only use it in
git-blame(1). Implement unit tests to verify that the yet-unused
functionality of the backend works as expected.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-15 04:50:45 +09:00
Patrick Steinhardt 822d403651 odb: introduce "in-memory" source
Next to our typical object database sources, each object database also
has an implicit source of "cached" objects. These cached objects only
exist in memory and some use cases:

  - They contain evergreen objects that we expect to always exist, like
    for example the empty tree.

  - They can be used to store temporary objects that we don't want to
    persist to disk, which is used by git-blame(1) to create a fake
    worktree commit.

Overall, their use is somewhat restricted though. For example, we don't
provide the ability to use it as a temporary object database source that
allows the user to write objects, but discard them after Git exists. So
while these cached objects behave almost like a source, they aren't used
as one.

This is about to change over the following commits, where we will turn
cached objects into a new "in-memory" source. This will allow us to use
it exactly the same as any other source by providing the same common
interface as the "files" source.

For now, the in-memory source only hosts the cached objects and doesn't
provide any logic yet. This will change with subsequent commits, where
we move respective functionality into the source.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-15 04:50:44 +09:00
Junio C Hamano 2f124686e8 Merge branch 'jt/odb-transaction-write' into ps/odb-in-memory
* jt/odb-transaction-write:
  odb/transaction: make `write_object_stream()` pluggable
  object-file: generalize packfile writes to use odb_write_stream
  object-file: avoid fd seekback by checking object size upfront
  object-file: remove flags from transaction packfile writes
  odb: update `struct odb_write_stream` read() callback
  odb/transaction: use pluggable `begin_transaction()`
  odb: split `struct odb_transaction` into separate header
2026-05-15 04:50:31 +09:00
Justin Tobler 5f6744d3eb odb: split `struct odb_transaction` into separate header
The current ODB transaction interface is colocated with other ODB
interfaces in "odb.{c,h}". Subsequent commits will expand `struct
odb_transaction` to support write operations on the transaction
directly. To keep things organized and prevent "odb.{c,h}" from becoming
more unwieldy, split out `struct odb_transaction` into a separate
header.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-15 04:44:39 +09:00
Kristoffer Haugsbakk 19e3106c45 format-rev: introduce builtin for on-demand pretty formatting
Introduce a new builtin for pretty formatting one revision expression
per line or commit object names found in running text.

Sometimes you want to format commits. Most of the time you’re
walking the graph, e.g. getting a range of commits like
`master..topic`. That’s a job for git-log(1).

But there are times when you want to format commits that you encounter
on demand:

• Full hashes in running text that you might want to pretty-print
• git-last-modified(1) outputs full hashes that you can do the same
  with
• git-cherry(1) has `-v` for commit subject, but maybe you want
  something else?

But now you can’t use git-log(1), git-show(1), or git-rev-list(1):

• You can’t feed commits piecemeal to these commands, one input
  for one output; they block until standard in is closed
• You can’t feed a list of possibly duplicate commits, like the output
  of git-last-modified(1); they effectively deduplicate the output

Beyond these two points there’s also the input massage problem: you
cannot feed mixed input (revisions mixed with arbitrary text).

One might hope that git-cat-file(1) can save us. But it doesn’t
support pretty formats.

But there is one command that already both handles revisions as
arguments, revisions on standard input, and even revisions mixed in
with arbitrary text. Namely git-name-rev(1): the command for outputting
symbolic names for commits.

We made some room in `builtin/name-rev.c` two commits ago. Let’s
now add this new git-format-rev(1) command. Taking inspiration from
git-name-rev(1), there are two modes:

• revs: like git-name-rev(1) in argv mode, but one revision per line
  on standard in
• text: like git-name-rev(1) with `--annotate-stdin`

***

We need to add this command to the exception list in
`t/t1517-outside-repo.sh` because it uses “EXPERIMENTAL!”
in the usage line.

Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-12 09:09:51 +09:00