The Windows runners used by Git's GitHub workflow's `windows-build` job
ship `rustup` plus a `*-pc-windows-msvc` default toolchain (see
https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md
and
https://github.com/actions/partner-runner-images/blob/main/images/arm-windows-11-image.md),
but no precompiled `std` for `*-pc-windows-gnu` or
`*-pc-windows-gnullvm`. With the Makefile now picking a GCC-compatible
target triple based on `$(MSYSTEM)`, the build step needs that
precompiled `std` to be installed before invoking `make`, otherwise
`cargo build --target <triple>` fails to find a usable `std` for the
chosen target.
Add a step between the SDK setup and the `make` invocation that selects
the matching triple from `$MSYSTEM` (which
`git-for-windows/setup-git-for-windows-sdk` exports for every subsequent
step) and runs `rustup target add` for it. The mapping mirrors what
`config.mak.uname` derives from `$(MSYSTEM)` and `$(HOST_CPU)`, just
enumerated explicitly here since CI has direct knowledge of which MSYS2
subsystems the matrix actually exercises (`CLANGARM64` for the ARM64
runner, `MINGW64` for the x86_64 runner). Technically, we only need to
handle MINGW64 at present, but the switch to UCRT64 is imminent, and the
other case arms serve as a very fine documentation of what people should
do for other MSYSTEM values.
For a `staticlib` crate-type `cargo build` does not invoke an external
linker, so no further toolchain components (e.g. the `gnullvm` LLVM
linker) need to be installed; `rustup target add` alone is sufficient.
Assisted-by: Claude Opus 4.7
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When Git is built under MSYS2/MinGW with Rust support enabled, the
Makefile expects `cargo build` to drop a `target/release/libgitcore.a`
that is linkable by the same MinGW GCC used for every other object. With
Rust installed via `rustup` (the way it ships on the GitHub-hosted
`windows-2022` and `windows-11-arm` runners that build git/git and its
forks), the default toolchain targets the MSVC ABI; cargo then writes
`target/release/gitcore.lib` instead, which the MinGW `ld.exe` cannot
consume:
LINK git-shell.exe
D:\git-sdk-64-minimal\mingw64\bin/ld.exe: cannot find target/release/libgitcore.a: No such file or directory
collect2.exe: error: ld returned 1 exit status
See https://github.com/microsoft/git/actions/runs/27341625000 for a
full example log.
Let's define the correct target, using the `CARGO_BUILD_TARGET` variable
that will be picked up by Rust, see
https://dirname.github.io/rust-std-doc/cargo/reference/environment-variables.html#:~:text=CARGO%5FBUILD%5FTARGET
Re-use (and fix) the existing `HOST_CPU` variable to determine the
correct value. Avoid relying on environment variables that are simply
not defined in Git for Windows' minimal SDK that Git uses in its CI
runs.
Assisted-by: Claude Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These days, the test cases are less free-form than in the wild old days
of the Git project.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git for Windows wants to add `git.exe` to the users' `PATH`, without
cluttering the latter with unnecessary executables such as `wish.exe`.
To that end, it invented the concept of its "Git wrapper", i.e. a tiny
executable located in `C:\Program Files\Git\cmd\git.exe` (originally a
CMD script) whose sole purpose is to set up a couple of environment
variables and then spawn the _actual_ `git.exe` (which nowadays lives in
`C:\Program Files\Git\mingw64\bin\git.exe` for 64-bit, and the obvious
equivalent for 32-bit installations).
Currently, the following environment variables are set unless already
initialized:
- `MSYSTEM`, to make sure that the MSYS2 Bash and the MSYS2 Perl
interpreter behave as expected, and
- `PLINK_PROTOCOL`, to force PuTTY's `plink.exe` to use the SSH
protocol instead of Telnet,
- `PATH`, to make sure that the `bin` folder in the user's home
directory, as well as the `/mingw64/bin` and the `/usr/bin`
directories are included. The trick here is that the `/mingw64/bin/`
and `/usr/bin/` directories are relative to the top-level installation
directory of Git for Windows (which the included Bash interprets as
`/`, i.e. as the MSYS pseudo root directory).
Using the absence of `MSYSTEM` as a tell-tale, we can detect in
`git.exe` whether these environment variables have been initialized
properly. Therefore we can call `C:\Program Files\Git\mingw64\bin\git`
in-place after this change, without having to call Git through the Git
wrapper.
Obviously, above-mentioned directories must be _prepended_ to the `PATH`
variable, otherwise we risk picking up executables from unrelated Git
installations. We do that by constructing the new `PATH` value from
scratch, appending `$HOME/bin` (if `HOME` is set), then the MSYS2 system
directories, and then appending the original `PATH`.
Side note: this modification of the `PATH` variable is independent of
the modification necessary to reach the executables and scripts in
`/mingw64/libexec/git-core/`, i.e. the `GIT_EXEC_PATH`. That
modification is still performed by Git, elsewhere, long after making the
changes described above.
While we _still_ cannot simply hard-link `mingw64\bin\git.exe` to `cmd`
(because the former depends on a couple of `.dll` files that are only in
`mingw64\bin`, i.e. calling `...\cmd\git.exe` would fail to load due to
missing dependencies), at least we can now avoid that extra process of
running the Git wrapper (which then has to wait for the spawned
`git.exe` to finish) by calling `...\mingw64\bin\git.exe` directly, via
its absolute path.
Testing this is in Git's test suite tricky: we set up a "new" MSYS
pseudo-root and copy the `git.exe` file into the appropriate location,
then verify that `MSYSTEM` is set properly, and also that the `PATH` is
modified so that scripts can be found in `$HOME/bin`, `/mingw64/bin/`
and `/usr/bin/`.
This addresses https://github.com/git-for-windows/git/issues/2283
Note: This keeps the same, hard-coded MSYSTEM platform support for CMake
as before, and introduces an `msystem' and `mingw-prefix` knob for Meson
(read: neither CMake nor Meson will automatically inherit the setting
from the current build environment).
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A change between versions 2.4.1 and 2.6.0 of the MSYS2 runtime modified
how Cygwin's runtime (and hence Git for Windows' MSYS2 runtime
derivative) handles locales: d16a56306d (Consolidate wctomb/mbtowc calls
for POSIX-1.2008, 2016-07-20).
An unintended side-effect is that "cold-calling" into the POSIX
emulation will start with a locale based on the current code page,
something that Git for Windows is very ill-prepared for, as it expects
to be able to pass a command-line containing non-ASCII characters to the
shell without having those characters munged.
One symptom of this behavior: when `git clone` or `git fetch` shell out
to call `git-upload-pack` with a path that contains non-ASCII
characters, the shell tried to interpret the entire command-line
(including command-line parameters) as executable path, which obviously
must fail.
This fixes https://github.com/git-for-windows/git/issues/1036
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Special-casing even more configurations simply does not make sense.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is merely a historical wart that, say, `git-commit` exists in the
`libexec/git-core/` directory, a tribute to the original idea to let Git
be essentially a bunch of Unix shell scripts revolving around very few
"plumbing" (AKA low-level) commands.
Git has evolved a lot from there. These days, most of Git's
functionality is contained within the `git` executable, in the form of
"built-in" commands.
To accommodate for scripts that use the "dashed" form of Git commands,
even today, Git provides hard-links that make the `git` executable
available as, say, `git-commit`, just in case that an old script has not
been updated to invoke `git commit`.
Those hard-links do not come cheap: they take about half a minute for
every build of Git on Windows, they are mistaken for taking up huge
amounts of space by some Windows Explorer versions that do not
understand hard-links, and therefore many a "bug" report had to be
addressed.
The "dashed form" has been officially deprecated in Git version 1.5.4,
which was released on February 2nd, 2008, i.e. a very long time ago.
This deprecation was never finalized by skipping these hard-links, but
we can start the process now, in Git for Windows.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
MSYS2 defines some helpful environment variables, e.g. `MSYSTEM`. There
is code in Git for Windows to ensure that that `MSYSTEM` variable is
set, hard-coding a default.
However, the existing solution jumps through hoops to reconstruct the
proper default, and is even incomplete doing so, as we found out when we
extended it to support CLANGARM64.
This is absolutely unnecessary because there is already a perfectly
valid `MSYSTEM` value we can use at build time. This is even true when
building the MINGW32 variant on a MINGW64 system because `makepkg-mingw`
will override the `MSYSTEM` value as per the `MINGW_ARCH` array.
The same is equally true for the `/mingw64`, `/mingw32` and
`/clangarm64` prefix: those values are already available via the
`MINGW_PREFIX` environment variable, and we just need to pass that
setting through.
Only when `MINGW_PREFIX` is not set (as is the case in Git for Windows'
minimal SDK, where only `MSYSTEM` is guaranteed to be set correctly), we
use as fall-back the top-level directory whose name is the down-cased
value of the `MSYSTEM` variable.
Incidentally, this also broadens the support to all the configurations
supported by the MSYS2 project, i.e. clang64 & ucrt64, too.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The tell-tale is the presence of the `MSYSTEM` value while compiling, of
course. In that case, we want to ensure that `MSYSTEM` is set when
running `git.exe`, and also enable the magic MSYS2 tty detection.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
MSYS2 already defines a couple of helpful environment variables, and we
can use those to infer the installation location as well as the CPU. No
need for hard-coding ;-)
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In bf2d5d8239e (Don't let ld strip relocations, 2016-01-16) (picked from
https://github.com/git-for-windows/git/pull/612/commits/6a237925bf10),
Git for Windows introduced the `-Wl,-pic-executable` flag, specifying
the exact entry point via `-e`. This required discerning between i686
and x86_64 code because the former required the symbol to be prefixed
with an underscore, the latter did not.
As per https://sourceware.org/bugzilla/show_bug.cgi?id=10865, the
specified symbols are already the default, though.
So let's drop the overly-specific definition.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
That option only matters there, and is in fact only really understood in
those builds; UCRT64 versions of GCC, for example, do not know what to
do with that option.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This option was added in fa93bb20d7 (MinGW: Fix stat definitions to
work with MinGW runtime version 4.0, 2013-09-11), i.e. a _long_ time
ago. So long, in fact, that it still targeted MinGW. But we switched to
mingw-w64 in 2015, which seems not to share the problem, and therefore
does not require a fix.
Even worse: This flag is incompatible with UCRT64, which we are about to
support by way of upstreaming `mingw-w64-git` to the MSYS2 project, see
https://github.com/msys2/MINGW-packages/pull/26470 for details.
So let's send that option into its well-deserved retirement.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is no longer true in general, not with supporting Clang out of the
box.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While Git for Windows does not _ship_ Python (in order to save on
bandwidth), MSYS2 provides very fine Python interpreters that users can
easily take advantage of, by using Git for Windows within its SDK.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The remote-matching logic for submodules has been corrected to resolve
'url.*.insteadOf' aliases before comparing the inventoried URL from
'.gitmodules' with the URLs of configured remotes.
* en/submodule-insteadof-remote-match:
submodule: resolve insteadOf aliases when matching remote
rewrites_release() in 'remote.c' has been updated to free 'struct
rewrite' instances, their '.instead_of' arrays, and their contents.
* jc/remote-insteadof-leakfix:
remote: plug memory leaks
A segfault when 'git clone --revision' talks to a server that does not
support protocol v2 (falling back to protocol v0) has been corrected.
* af/clone-revision-v0-segfault-fix:
builtin/clone: fix segfault when using --revision with protocol v0
Traversals with '--exclude-first-parent-only' have been corrected
to properly stop after the first parent even when it has already
been marked as 'SEEN'.
* jc/exclude-first-parent-seen:
revision: honor --exclude-first-parent-only with SEEN first parent
When the push remote is specified as a URL, the fetch refspec of a
uniquely matching configured remote is now used to find and update
the remote-tracking branch (e.g., '@{push}').
* hn/url-push-tracking:
remote: find tracking branches for URL push destinations
remote: pass repository to push tracking helper
The logic to write loose objects has been refactored and moved from
'object-file.c' to the loose backend source file 'odb/source-loose.c',
making the loose backend more self-contained. This is achieved by
first refactoring force_object_loose() to use generic ODB write
interfaces instead of loose-backend internals.
* ps/odb-move-loose-object-writing:
object-file: move logic to write loose objects
object-file: move `force_object_loose()`
object-file: force objects loose via generic interface
object-file: fix memory leak in `force_object_loose()`
odb: support setting mtime when writing objects
odb: lift object existence check out of the "loose" backend
odb: compute object hash in `odb_write_object_ext()`
t/u-odb-inmemory: implement wrapper for writing objects
odb: compute compat object ID in `odb_write_object_ext()`
The object ID shortening and linking in the 'commitdiff' view of
'gitweb' has been corrected to work even when the index line carries
a trailing file mode.
* tl/gitweb-shorten-hashes-with-modes:
gitweb: shorten index hashes with trailing file modes
The 'remote-object-info' command has been added to 'git cat-file
--batch-command', allowing clients to request object metadata
(currently size) from a remote server via protocol v2 without
downloading the entire object. Format placeholders are dynamically
filtered on the client based on server-advertised capabilities,
returning empty strings for inapplicable or unsupported fields.
* ps/cat-file-remote-object-info:
cat-file: make remote-object-info allow-list adapt to the server
cat-file: add remote-object-info to batch-command
transport: add client support for object-info
serve: advertise object-info feature
protocol-caps: check object existence regardless of the attributes requested
fetch-pack: move fetch initialization
connect: make write_fetch_command_and_capabilities() more generic
fetch-pack: move write_fetch_command_and_capabilities() to connect.c
fetch-pack: use unsigned int for hash_algo variable
fetch-pack: drop the static advertise_sid variable
t1006: extract helper functions into new 'lib-cat-file.sh'
cat-file: declare loop counter inside for()
transport-helper: fix memory leak of helper on disconnect
Configuration file locking has been updated to retry for a short
period, avoiding failures when multiple processes attempt to update
the configuration simultaneously.
* jt/config-lock-timeout:
config: retry acquiring config.lock, configurable via core.configLockTimeout
The 'git stash push' command has been optimized to avoid unnecessary
sparse index expansion when pathspecs are wholly inside the
sparse-checkout cone. Also, a potential out-of-bounds read in the
sparse-index expansion check helper pathspec_needs_expanded_index()
has been fixed by consistently using the parsed, prefixed path.
* tn/stash-avoid-sparse-index-expansion:
stash: avoid sparse-index expansion for in-cone paths
pathspec: use match for sparse-index expansion checks
Userdiff patterns for Swift have been added, with support for
Swift-specific constructs such as attributes, modifiers, failable
initializers, and generics.
* sk/userdiff-swift:
userdiff: add support for Swift
The 'excludes_file' and various other global configuration variables
(including 'editor_program', 'pager_program', 'askpass_program', and
'push_default') have been migrated into the per-repository structure.
* ty/migrate-excludes-file:
repository: adjust the comment of config_values_private_
environment: move object_creation_mode into repo_config_values
environment: move autorebase into repo_config_values
environment: move push_default into repo_config_values
environment: migrate apply_default_whitespace and apply_default_ignorewhitespace
environment: move askpass_program into repo_config_values
environment: move pager_program into repo_config_values
environment: move editor_program into repo_config_values
environment: move excludes_file into repo_config_values
repository: introduce repo_config_values_clear()
The 'trust_executable_bit' (coming from the 'core.filemode'
configuration) has been migrated into 'struct repo_config_values' to
tie it to a specific repository instance.
* ty/migrate-trust-executable-bit:
environment: move has_symlinks into repo_config_values
environment: move trust_executable_bit into repo_config_values
read-cache: pass 'repo' to 'ce_mode_from_stat()'
read-cache: remove redundant extern declarations
The tempfile and lockfile APIs have been refactored to stop depending
on the 'the_repository' global variable, and their callers have been
updated to use the repository-aware variants.
* rs/tempfile-wo-the-repository:
use repo_hold_lock_file_for_update{,_mode,_timeout}() with custom repos
tempfile: stop using the_repository
lockfile: add repo_hold_lock_file_for_update{,_timeout}{,_mode}()
refs/packed: use repo_create_tempfile()
tempfile: add repo_create_tempfile{,_mode}()
A few memory problems in the Rust interface to C hash functions have
been corrected. The 'Clone' implementation of 'CryptoHasher' now
properly initializes the context before cloning, and its 'Drop'
implementation now discards the context to prevent leaks.
* bc/rust-hash-cleanups:
rust: discard hash context when finished
hash: initialize context before cloning
The rebase post-rewrite notes-copying logic has been corrected. When
a commit is dropped during rebase (e.g., because its changes are
already upstream), it is no longer recorded as rewritten, preventing
its notes from being copied to an unrelated commit.
* pw/rebase-drop-notes-with-commit:
sequencer: do not record dropped commits as rewritten
sequencer: use an enum to represent result of picking a commit
sequencer: simplify pick_one_commit()
sequencer: remove unnecessary condition in pick_one_commit()
sequencer: simplify handling of fixup with conflicts
sequencer: remove unnecessary "or" in pick_one_commit()
sequencer: never reschedule on failed commit
sequencer: be more careful with external merge
t3400: restore coverage for note copying with apply backend
The copy_file() and copy_file_with_time() functions have been
refactored to take a repository parameter, allowing the removal of the
implicit dependency on the global 'the_repository' variable in
'copy.c'.
* ps/copy-wo-the-repository:
copy: drop dependency on `the_repository`
The enumeration of untracked and ignored files in 'git status' has
been optimized by avoiding quadratic complexity when inserting into
string lists, reducing the construction cost from O(n^2) to O(n log
n).
* sc/wt-status-avoid-quadratic-insertion:
wt-status: avoid repeated insertion for untracked paths
The dependency on the global 'the_repository' variable in the
'refspec.c' API has been removed by passing the hash algorithm
explicitly to refspec-parsing functions and storing it in 'struct
refspec'.
* ps/refspec-wo-the-repository:
refspec: stop depending on `the_repository`
refspec: let callers pass in hash algorithm when parsing items
refspec: group related structures and functions
The passing of push destination specifications in the 'remote-curl'
helper has been simplified by removing the explicit 'count' parameter
and relying on the NULL-termination of the array.
* rs/remote-curl-simplify-push-specs:
remote-curl: simplify passing of push specs
'git branch --contains' and 'git for-each-ref --contains' have been
optimized to use the memoized commit traversal previously used only by
'git tag --contains', significantly speeding up connectivity checks
across many candidate refs with shared history.
* td/ref-filter-memoize-contains:
commit-reach: die on contains walk errors
ref-filter: memoize --contains with generations
commit-reach: reject cycles in contains walk
The ref subsystem and the worktree API have been refactored to pass a
repository pointer down the call chain, allowing them to drop
references to the global 'the_repository' variable. As part of this,
the handling of the 'core.packedRefsTimeout' configuration has been
moved into the per-repository ref store structure.
* ps/refs-wo-the-repository:
refs: remove remaining uses of `the_repository`
worktree: pass repository to public functions
worktree: pass repository to file-local functions
worktree: refactor code to use available repositories
refs/files: drop `USE_THE_REPOSITORY_VARIABLE`
refs/packed: de-globalize handling of "core.packedRefsTimeout"
An accidental use of the '%zu' format specifier in 'git
submodule--helper' has been corrected to use 'PRIuMAX' and cast the
value to 'uintmax_t' to avoid portability issues.
* jc/submodule-helper-avoid-zu:
submodule--helper: avoid use of %zu for now
'git log --graph' has been modified to visually distinguish parentless
'root' commits (and commits that become roots due to history
simplification) by indenting them, preventing them from appearing
falsely related to unrelated commits rendered immediately above them.
* ps/shift-root-in-graph:
graph: add --[no-]graph-indent and log.graphIndent
graph: move config reading into graph_read_config()
graph: wrap cascading commits after 4 columns
graph: indent visual root in graph
graph: add a 2 commit buffer for lookahead
revision: add next_commit_to_show()
lib-log-graph: move check_graph function
The in-core data structure used to keep track of
'url.<real>.{insteadOf,pushInsteadOf} = <alias>' settings is not
properly cleaned up when the process is done with it.
'struct rewrites' is embedded in 'remote_state' and serves as the
top level of the rewrite data. This holds an array of a variable
number of pointers to 'struct rewrite' allocated individually on the
heap. Each 'struct rewrite' holds a '.base' string and an array of
'struct counted_string' called '.instead_of', which is allocated
contiguously on the heap. Each 'struct counted_string' has a
pointer to a string allocated on the heap.
Amid these pointers, rewrites_release() fails to free everything
other than 'struct rewrite''s '.base' member and the 'struct rewrite'
instances themselves.
Fix rewrites_release() to also free the contiguous array storing
'.instead_of', the string pointers within each '.instead_of' element,
and each 'struct rewrite' instance individually allocated on the heap.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Servers supporting protocol v2 do not advertise excess refs and honor
`transport_ls_refs_options.ref_prefixes` when
$ git clone --revision=refs/heads/main $URL
contacts them, but when talking to a server that does not support
protocol v2 the client segfaults. This can also be observed when v0 is
enforced for example by
$ git -c protocol.version=0 clone --revision=refs/heads/main $URL
In the protocol v2 case the server honors
`transport_ls_refs_options.ref_prefixes` and in `cmd_clone()` the linked
list `refs` returned by `transport_get_remote_refs()` only contains a
single item, which is the ref requested with the --revision argument.
Both `remote_head` returned by `find_ref_by_name()` and
`remote_head_points_at` returned by `guess_remote_head()` are NULL. The
guard in `update_remote_refs()` skips a the affected code because
`remote_head_points_at` is NULL.
In the protocol v0 case in `cmd_clone()` the linked list `refs` returned
by `transport_get_remote_refs()` contains many items, amongst others
"HEAD". `remote_head` returned by `find_ref_by_name()` is not NULL and
`remote_head_points_at` returned by `guess_remote_head()` is not NULL
but its field `peer_ref` is NULL. Because `remote_head_points_at` is not
NULL the guard in `update_remote_refs()` does not skip the affected code
and `remote_head_points_at->peer_ref->name` is accessed, which causes a
segfault later on.
Signed-off-by: Adrian Friedli <adrian.friedli@mt.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The static allow-list in expand_atom() is hardcoded to allow only
"objectname" and "objectsize" for remote queries. This works because,
up to this point, servers will either support object-info with name
and size or they do not support them at all.
As object-info gains new capabilities, we cannot expect different
servers with different Git versions to have the same object-info
capabilities. Therefore, the client needs to adapt its allow-list to
what the server advertises.
The client now:
1. Requests the protocol option that the placeholder refers to (i.e.
"size" for "%(objectsize)").
2. Drops any requested option that the server does not advertise in
fetch_object_info().
3. Maps the remaining advertised options back to their placeholders and
populates remote_allowed_atoms.
4. Uses remote_allowed_atoms in expand_atom(), preserving the previous
behavior for supported placeholders.
For example, if the client requests "%(objectsize) %(objecttype)" and
the server only supports 'size', then the client only requests 'size'.
The server returns the size (i.e "42") "%(objectsize)" is expanded
normally while "%(objecttype)" expands to an empty string:
"42 "
Note that the empty string expansion is only for known but unsupported
placeholders. "%(objectcolor)" which doesn't exist would die().
This honors what for-each-ref does for known but inapplicable atoms
(placeholders).
Move object_info_options out of get_remote_info() so the caller which
has data can select what options will be requested instead of requesting
always size.
Move batch_object_write() out so output is always produced.
If there are no supported attributes, the output is a blank line.
Include "type" in the object_info_options even though the client does
not yet know how to parse the server's "type" capability.
As a result, "type" is always filtered out, allowing the tests to verify
that known but unsupported placeholders expand to an empty string.
Since the filter removes options by swapping with the last element,
the list is no longer kept sorted. Drop the pre-sort in
fetch_object_info_via_pack() and use the unsorted string_list lookup
for the response header. This has no effect in performance as the list
can only be two entries long ('size' and 'type').
Mentored-by: Karthik Nayak <karthik.188@gmail.com>
Mentored-by: Chandra Pratap <chandrapratap3519@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since the info command in cat-file --batch-command prints object
info for a given object, it is natural to add another command in
cat-file --batch-command to print object info for a given object
from a remote.
Add remote-object-info command to cat-file --batch-command.
While info takes object ids one at a time, this creates overhead when
making requests to a server. So remote-object-info instead can take
multiple object ids at once.
The cat-file --batch-command command is generally implemented in the
following manner:
- Receive and parse input from user
- Call respective function attached to command
- Get object info, print object info
In --buffer mode, this changes to:
- Receive and parse input from user
- Store respective function attached to command in a queue
- After flush, loop through commands in queue
- Call respective function attached to command
- Get object info, print object info
Notice how the getting and printing of object info is accomplished one
at a time. As described above, this creates a problem for making
requests to a server. Therefore, remote-object-info is implemented in
the following manner:
- Receive and parse input from user
If command is remote-object-info:
- Get object info from remote
- Loop through and print each object info
Else:
- Call respective function attached to command
- Parse input, get object info, print object info
And finally for --buffer mode remote-object-info:
- Receive and parse input from user
- Store respective function attached to command in a queue
- After flush, loop through commands in queue:
If command is remote-object-info:
- Get object info from remote
- Loop through and print each object info
Else:
- Call respective function attached to command
- Get object info, print object info
To summarize, remote-object-info gets object info from the remote and
then loops through the object info passed in, printing the info.
In order for remote-object-info to avoid remote communication
overhead in the non-buffer mode, the objects are passed in as such:
remote-object-info <remote> <oid> <oid> ... <oid>
rather than
remote-object-info <remote> <oid>
remote-object-info <remote> <oid>
...
remote-object-info <remote> <oid>
Placeholders in the format are validated against an allow-list of the
atoms the remote path supports: "objectname" and "objectsize".
Unsupported atoms expand to an empty string, honoring how for-each-ref
handles known but inapplicable atoms.
Without this, atoms like %(objecttype) would mark data->info.typep and
because the server only sends size, type_name() would later crash.
As extra safety, even outside of the remote path, initialize
expand_data's type to OBJ_BAD and handle type_name() returning NULL.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Mentored-by: Karthik Nayak <karthik.188@gmail.com>
Mentored-by: Chandra Pratap <chandrapratap3519@gmail.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
[pablo: added the atom allow-list validation]
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sometimes, it is beneficial to retrieve information about an object
without downloading it entirely. The server-side logic for this
functionality was implemented in commit "a2ba162cda (object-info:
support for retrieving object info, 2021-04-20)." And the wire
format is documented at
https://git-scm.com/docs/protocol-v2#_object_info.
Introduce client-side support for the object-info capability.
Add its own function for object-info separate from existing fetch
infrastructure.
Currently, the client supports requesting a list of OIDs with the size
attribute from a v2 server. If the server does not advertise this
feature (i.e., transfer.advertiseobjectinfo is set to false), the client
returns an error and exits.
Note that:
1. The entire request is written into req_buf before being sent to the
remote. This approach follows the pattern used in the
send_fetch_request() logic within 'fetch-pack.c'. Streaming the
request is not addressed in this patch.
2. A new field 'unrecognized' has been added to object_info. This new
field is set at fetch_object_info() when the object is unrecognized
by the server.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In order for a client to know what object-info components a server can
provide, advertise supported object-info features. This allows a client
to decide whether to query the server for object-info or fetch as a
fallback.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, send_info() only checks for existence when the attribute
'size' is also requested. Requesting a bare OID, without attributes only
echoes back the OID.
Extract the existence check to be done regardless of the number of
attributes requested.
While at it, introduce a wrapper called get_object_info() similar to
odb_read_object_info() that returns OBJ_BAD on fail and adds
OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK flags.
OBJECT_INFO_SKIP_FETCH_OBJECT is so a server with a partial clone
doesn't trigger fetching objects when it gets an object-info request
with an OID that is not available locally. A server should only report
what it has locally.
Tighten the condition used to determine whether an object is
recognized. get_object_info() returns OBJ_BAD for unknown objects,
but OBJ_NONE (0) can also mean "not found". Change the check from '< 0'
to '<= OBJ_NONE' to cover both as unrecognized.
With this patch, a bare OID has two possible responses:
1. Recognized OID: the server answers with "<OID>"
2. Unrecognized OID: the server answers with "<OID> SP"
Update the object-info section in 'gitprotocol-v2.adoc':
- Require full obj-oid explicitly.
- Fix parentheses.
- Define obj-size explicitly.
- Make obj-size optional in obj-info and document the behavior
for unrecognized object IDs.
- Describe the attr header as zero or more pkt-lines, one per attribute,
matching what the server implements. A request with no attributes gets
no header.
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are some variables initialized at the start of the
do_fetch_pack_v2() state machine. Currently, they are initialized in
FETCH_CHECK_LOCAL, which is the initial state set at the beginning
of the function.
However, a subsequent patch will allow for another initial state,
while still requiring these initialized variables.
Move the initialization to be before the state machine,
so that they are set regardless of the initial state.
Note that there is no change in behavior, because we're moving code
from the beginning of the first state to just before the execution of
the state machine.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Eric Ju <eric.peijian@gmail.com>
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Johannes Schindelin