Commit Graph

73324 Commits (v2.45.4)

Author SHA1 Message Date
Jeff King e69ac42fcc docs: document security issues around untrusted .git dirs
For a long time our general philosophy has been that it's unsafe to run
arbitrary Git commands if you don't trust the hooks or config in .git,
but that running upload-pack should be OK. E.g., see 1456b043fc (Remove
post-upload-hook, 2009-12-10), or the design of uploadpack.packObjectsHook.

But we never really documented this (and even the discussions that led
to 1456b043fc were not on the public list!). Let's try to make our
approach more clear, but also be realistic that even upload-pack carries
some risk.

Helped-by: Filip Hejsek <filip.hejsek@gmail.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 22:29:57 +02:00
Jeff King 7b70e9efb1 upload-pack: disable lazy-fetching by default
The upload-pack command tries to avoid trusting the repository in which
it's run (e.g., by not running any hooks and not using any config that
contains arbitrary commands). But if the server side of a fetch or a
clone is a partial clone, then either upload-pack or its child
pack-objects may run a lazy "git fetch" under the hood. And it is very
easy to convince fetch to run arbitrary commands.

The "server" side can be a local repository owned by someone else, who
would be able to configure commands that are run during a clone with the
current user's permissions. This issue has been designated
CVE-2024-32004.

The fix in this commit's parent helps in this scenario, as well as in
related scenarios using SSH to clone, where the untrusted .git directory
is owned by a different user id. But if you received one as a zip file,
on a USB stick, etc, it may be owned by your user but still untrusted.

This has been designated CVE-2024-32465.

To mitigate the issue more completely, let's disable lazy fetching
entirely during `upload-pack`. While fetching from a partial repository
should be relatively rare, it is certainly not an unreasonable workflow.
And thus we need to provide an escape hatch.

This commit works by respecting a GIT_NO_LAZY_FETCH environment variable
(to skip the lazy-fetch), and setting it in upload-pack, but only when
the user has not already done so (which gives us the escape hatch).

The name of the variable is specifically chosen to match what has
already been added in 'master' via e6d5479e7a (git: extend
--no-lazy-fetch to work across subprocesses, 2024-02-27). Since we're
building this fix as a backport for older versions, we could cherry-pick
that patch and its earlier steps. However, we don't really need the
niceties (like a "--no-lazy-fetch" option) that it offers. By using the
same name, everything should just work when the two are eventually
merged, but here are a few notes:

  - the blocking of the fetch in e6d5479e7a is incomplete! It sets
    fetch_if_missing to 0 when we setup the repository variable, but
    that isn't enough. pack-objects in particular will call
    prefetch_to_pack() even if that variable is 0. This patch by
    contrast checks the environment variable at the lowest level before
    we call the lazy fetch, where we can be sure to catch all code
    paths.

    Possibly the setting of fetch_if_missing from e6d5479e7a can be
    reverted, but it may be useful to have. For example, some code may
    want to use that flag to change behavior before it gets to the point
    of trying to start the fetch. At any rate, that's all outside the
    scope of this patch.

  - there's documentation for GIT_NO_LAZY_FETCH in e6d5479e7a. We can
    live without that here, because for the most part the user shouldn't
    need to set it themselves. The exception is if they do want to
    override upload-pack's default, and that requires a separate
    documentation section (which is added here)

  - it would be nice to use the NO_LAZY_FETCH_ENVIRONMENT macro added by
    e6d5479e7a, but those definitions have moved from cache.h to
    environment.h between 2.39.3 and master. I just used the raw string
    literals, and we can replace them with the macro once this topic is
    merged to master.

At least with respect to CVE-2024-32004, this does render this commit's
parent commit somewhat redundant. However, it is worth retaining that
commit as defense in depth, and because it may help other issues (e.g.,
symlink/hardlink TOCTOU races, where zip files are not really an
interesting attack vector).

The tests in t0411 still pass, but now we have _two_ mechanisms ensuring
that the evil command is not run. Let's beef up the existing ones to
check that they failed for the expected reason, that we refused to run
upload-pack at all with an alternate user id. And add two new ones for
the same-user case that both the restriction and its escape hatch.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 22:29:56 +02:00
Johannes Schindelin f4aa8c8bb1 fetch/clone: detect dubious ownership of local repositories
When cloning from somebody else's repositories, it is possible that,
say, the `upload-pack` command is overridden in the repository that is
about to be cloned, which would then be run in the user's context who
started the clone.

To remind the user that this is a potentially unsafe operation, let's
extend the ownership checks we have already established for regular
gitdir discovery to extend also to local repositories that are about to
be cloned.

This protection extends also to file:// URLs.

The fixes in this commit address CVE-2024-32004.

Note: This commit does not touch the `fetch`/`clone` code directly, but
instead the function used implicitly by both: `enter_repo()`. This
function is also used by `git receive-pack` (i.e. pushes), by `git
upload-archive`, by `git daemon` and by `git http-backend`. In setups
that want to serve repositories owned by different users than the
account running the service, this will require `safe.*` settings to be
configured accordingly.

Also note: there are tiny time windows where a time-of-check-time-of-use
("TOCTOU") race is possible. The real solution to those would be to work
with `fstat()` and `openat()`. However, the latter function is not
available on Windows (and would have to be emulated with rather
expensive low-level `NtCreateFile()` calls), and the changes would be
quite extensive, for my taste too extensive for the little gain given
that embargoed releases need to pay extra attention to avoid introducing
inadvertent bugs.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 22:29:54 +02:00
Filip Hejsek 5c5a4a1c05 t0411: add tests for cloning from partial repo
Cloning from a partial repository must not fetch missing objects into
the partial repository, because that can lead to arbitrary code
execution.

Add a couple of test cases, pretending to the `upload-pack` command (and
to that command only) that it is working on a repository owned by
someone else.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Filip Hejsek <filip.hejsek@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 22:29:53 +02:00
Xing Xin 93e2ae1c95 midx: disable replace objects
We observed a series of clone failures arose in a specific set of
repositories after we fully enabled the MIDX bitmap feature within our
Codebase service. These failures were accompanied with error messages
such as:

    Cloning into bare repository 'clone.git'...
    remote: Enumerating objects: 8, done.
    remote: Total 8 (delta 0), reused 0 (delta 0), pack-reused 8 (from 1)
    Receiving objects: 100% (8/8), done.
    fatal: did not receive expected object ...
    fatal: fetch-pack: invalid index-pack output

Temporarily disabling the MIDX feature eliminated the reported issues.
After some investigation we found that all repositories experiencing
failures contain replace references, which seem to be improperly
acknowledged by the MIDX bitmap generation logic.

A more thorough explanation about the root cause from Taylor Blau says:

Indeed, the pack-bitmap-write machinery does not itself call
disable_replace_refs(). So when it generates a reachability bitmap, it
is doing so with the replace refs in mind. You can see that this is
indeed the cause of the problem by looking at the output of an
instrumented version of Git that indicates what bits are being set
during the bitmap generation phase.

With replace refs (incorrectly) enabled, we get:

    [2, 4, 6, 8, 13, 3, 6, 7, 3, 4, 6, 8]

and doing the same after calling disable_replace_refs(), we instead get:

    [2, 5, 6, 13, 3, 6, 7, 3, 4, 6, 8]

Single pack bitmaps are unaffected by this issue because we generate
them from within pack-objects, which does call disable_replace_refs().

This patch updates the MIDX logic to disable replace objects within the
multi-pack-index builtin, and a test showing a clone (which would fail
with MIDX bitmap) is added to demonstrate the bug.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Xing Xin <xingxin.xx@bytedance.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-17 12:35:41 -07:00
Patrick Steinhardt 7bf3057d9c builtin/receive-pack: convert to use git-maintenance(1)
In 850b6edefa (auto-gc: extract a reusable helper from "git fetch",
2020-05-06), we have introduced a helper function `run_auto_gc()` that
kicks off `git gc --auto`. The intent of this function was to pass down
the "--quiet" flag to git-gc(1) as required without duplicating this at
all callsites. In 7c3e9e8cfb (auto-gc: pass --quiet down from am,
commit, merge and rebase, 2020-05-06) we then converted callsites that
need to pass down this flag to use the new helper function. This has the
notable omission of git-receive-pack(1), which is the only remaining
user of `git gc --auto` that sets up the proccess manually. This is
probably because it unconditionally passes down the `--quiet` flag and
thus didn't benefit much from the new helper function.

In a95ce12430 (maintenance: replace run_auto_gc(), 2020-09-17) we then
replaced `run_auto_gc()` with `run_auto_maintenance()` which invokes
git-maintenance(1) instead of git-gc(1). This command is the modern
replacement for git-gc(1) and is both more thorough and also more
flexible because administrators can configure which tasks exactly to run
during maintenance.

But due to git-receive-pack(1) not using `run_auto_gc()` in the first
place it did not get converted to use git-maintenance(1) like we do
everywhere else now. Address this oversight and start to use the newly
introduced function `prepare_auto_maintenance()`. This will also make it
easier for us to adapt this code together with all the other callsites
that invoke auto-maintenance in the future.

This removes the last internal user of `git gc --auto`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-17 08:42:26 -07:00
Patrick Steinhardt b396ee6bed run-command: introduce function to prepare auto-maintenance process
The `run_auto_maintenance()` function is responsible for spawning a new
`git maintenance run --auto` process. To do so, it sets up the `sturct
child_process` and then runs it by executing `run_command()` directly.
This is rather inflexible in case callers want to modify the child
process somewhat, e.g. to redirect stderr or stdout.

Introduce a new `prepare_auto_maintenance()` function to plug this gap.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-17 08:42:26 -07:00
Linus Arver 8882ee9d68 mailmap: change primary address for Linus Arver
Linus will lose access to his work email soon.

Signed-off-by: Linus Arver <linusa@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-16 22:25:11 -07:00
Patrick Steinhardt 1204e1a824 builtin/clone: refuse local clones of unsafe repositories
When performing a local clone of a repository we end up either copying
or hardlinking the source repository into the target repository. This is
significantly more performant than if we were to use git-upload-pack(1)
and git-fetch-pack(1) to create the new repository and preserves both
disk space and compute time.

Unfortunately though, performing such a local clone of a repository that
is not owned by the current user is inherently unsafe:

  - It is possible that source files get swapped out underneath us while
    we are copying or hardlinking them. While we do perform some checks
    here to assert that we hardlinked the expected file, they cannot
    reliably thwart time-of-check-time-of-use (TOCTOU) style races. It
    is thus possible for an adversary to make us copy or hardlink
    unexpected files into the target directory.

    Ideally, we would address this by starting to use openat(3P),
    fstatat(3P) and friends. Due to platform compatibility with Windows
    we cannot easily do that though. Furthermore, the scope of these
    fixes would likely be quite broad and thus not fit for an embargoed
    security release.

  - Even if we handled TOCTOU-style races perfectly, hardlinking files
    owned by a different user into the target repository is not a good
    idea in general. It is possible for an adversary to rewrite those
    files to contain whatever data they want even after the clone has
    completed.

Address these issues by completely refusing local clones of a repository
that is not owned by the current user. This reuses our existing infra we
have in place via `ensure_valid_ownership()` and thus allows a user to
override the safety guard by adding the source repository path to the
"safe.directory" configuration.

This addresses CVE-2024-32020.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 02:17:40 +02:00
Patrick Steinhardt 8c9c051bef setup.c: introduce `die_upon_dubious_ownership()`
Introduce a new function `die_upon_dubious_ownership()` that uses
`ensure_valid_ownership()` to verify whether a repositroy is safe for
use, and causes Git to die in case it is not.

This function will be used in a subsequent commit.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:01:26 +02:00
Patrick Steinhardt d1bb66a546 builtin/clone: abort when hardlinked source and target file differ
When performing local clones with hardlinks we refuse to copy source
files which are symlinks as a mitigation for CVE-2022-39253. This check
can be raced by an adversary though by changing the file to a symlink
after we have checked it.

Fix the issue by checking whether the hardlinked destination file
matches the source file and abort in case it doesn't.

This addresses CVE-2024-32021.

Reported-by: Apple Product Security <product-security@apple.com>
Suggested-by: Linus Torvalds <torvalds@linuxfoundation.org>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:01:25 +02:00
Patrick Steinhardt 150e6b0aed builtin/clone: stop resolving symlinks when copying files
When a user performs a local clone without `--no-local`, then we end up
copying the source repository into the target repository directly. To
optimize this even further, we try to hardlink files into place instead
of copying data over, which helps both disk usage and speed.

There is an important edge case in this context though, namely when we
try to hardlink symlinks from the source repository into the target
repository. Depending on both platform and filesystem the resulting
behaviour here can be different:

  - On macOS and NetBSD, calling link(3P) with a symlink target creates
    a hardlink to the file pointed to by the symlink.

  - On Linux, calling link(3P) instead creates a hardlink to the symlink
    itself.

To unify this behaviour, 36596fd2df (clone: better handle symlinked
files at .git/objects/, 2019-07-10) introduced logic to resolve symlinks
before we try to link(3P) files. Consequently, the new behaviour was to
always create a hard link to the target of the symlink on all platforms.

Eventually though, we figured out that following symlinks like this can
cause havoc when performing a local clone of a malicious repository,
which resulted in CVE-2022-39253. This issue was fixed via 6f054f9fb3
(builtin/clone.c: disallow `--local` clones with symlinks, 2022-07-28),
by refusing symlinks in the source repository.

But even though we now shouldn't ever link symlinks anymore, the code
that resolves symlinks still exists. In the best case the code does not
end up doing anything because there are no symlinks anymore. In the
worst case though this can be abused by an adversary that rewrites the
source file after it has been checked not to be a symlink such that it
actually is a symlink when we call link(3P). Thus, it is still possible
to recreate CVE-2022-39253 due to this time-of-check-time-of-use bug.

Remove the call to `realpath()`. This doesn't yet address the actual
vulnerability, which will be handled in a subsequent commit.

Reported-by: Apple Product Security <product-security@apple.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:01:25 +02:00
Johannes Schindelin 9e06401098 Merge branch 'js/github-actions-update'
Update remaining GitHub Actions jobs to avoid warnings against
using deprecated version of Node.js.

* js/github-actions-update:
  ci(linux32): add a note about Actions that must not be updated
  ci: bump remaining outdated Actions versions

With this backport, `maint-2.39`'s CI builds are finally healthy again.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-17 00:01:11 +02:00
Johannes Schindelin 64f35baa34 Merge branch 'jc/maint-github-actions-update'
* jc/maint-github-actions-update:
  GitHub Actions: update to github-script@v7
  GitHub Actions: update to checkout@v4

Yet another thing to help `maint-2.39`'s CI builds to become healthy
again.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-17 00:01:00 +02:00
Johannes Schindelin 213958f248 ci(linux32): add a note about Actions that must not be updated
The Docker container used by the `linux32` job comes without Node.js,
and therefore the `actions/checkout` and `actions/upload-artifact`
Actions cannot be upgraded to the latest versions (because they use
Node.js).

One time too many, I accidentally tried to update them, where
`actions/checkout` at least fails immediately, but the
`actions/upload-artifact` step is only used when any test fails, and
therefore the CI run usually passes even though that Action was updated
to a version that is incompatible with the Docker container in which
this job runs.

So let's add a big fat warning, mainly for my own benefit, to avoid
running into the very same issue over and over again.

Backported-from: 20e0ff8835 (ci(linux32): add a note about Actions that must not be updated, 2024-02-11)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:00:58 +02:00
Junio C Hamano f6bed64ce2 GitHub Actions: update to github-script@v7
We seem to be getting "Node.js 16 actions are deprecated." warnings
for jobs that use github-script@v6.  Update to github-script@v7,
which is said to use Node.js 20.

Backported-from: c4ddbe043e (GitHub Actions: update to github-script@v7, 2024-02-02)
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:00:57 +02:00
Johannes Schindelin 7e1bcc8d63 ci: bump remaining outdated Actions versions
After activating automatic Dependabot updates in the
git-for-windows/git repository, Dependabot noticed a couple
of yet-unaddressed updates.  They avoid "Node.js 16 Actions"
deprecation messages by bumping the following Actions'
versions:

- actions/upload-artifact from 3 to 4
- actions/download-artifact from 3 to 4
- actions/cache from 3 to 4

Backported-from: 820a340085 (ci: bump remaining outdated Actions versions, 2024-02-11)
Helped-by: Matthias Aßhauer <mha1993@live.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:00:57 +02:00
Junio C Hamano ce47f7c85f GitHub Actions: update to checkout@v4
We seem to be getting "Node.js 16 actions are deprecated." warnings
for jobs that use checkout@v3.  Except for the i686 containers job
that is kept at checkout@v1 [*], update to checkout@v4, which is
said to use Node.js 20.

[*] 6cf4d908 (ci(main): upgrade actions/checkout to v3, 2022-12-05)
    refers to https://github.com/actions/runner/issues/2115 and
    explains why container jobs are kept at checkout@v1.  We may
    want to check the current status of the issue and move it to the
    same version as other jobs, but that is outside the scope of
    this step.

Backported-from: e94dec0c1d (GitHub Actions: update to checkout@v4, 2024-02-02)
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:00:57 +02:00
Johannes Schindelin 6133e3a77e Merge branch 'quicker-asan-lsan'
This patch speeds up the `asan`/`lsan` jobs that are really slow enough
already.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:00:56 +02:00
Johannes Schindelin ea094eec54 Merge branch 'jk/test-lsan-denoise-output'
Tests with LSan from time to time seem to emit harmless message
that makes our tests unnecessarily flakey; we work it around by
filtering the uninteresting output.

* jk/test-lsan-denoise-output:
  test-lib: ignore uninteresting LSan output

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-17 00:00:54 +02:00
Johannes Schindelin b12dcab61d Merge branch 'js/ci-use-macos-13'
Replace macos-12 used at GitHub CI with macos-13.

* js/ci-use-macos-13:
  ci: upgrade to using macos-13

This is another backport to `maint-2.39` to allow less CI jobs to break.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16 23:59:03 +02:00
Johannes Schindelin c7db432de6 Merge branch 'backport/jk/libcurl-8.7-regression-workaround' into maint-2.39
Fix was added to work around a regression in libcURL 8.7.0 (which has
already been fixed in their tip of the tree).

* jk/libcurl-8.7-regression-workaround:
  remote-curl: add Transfer-Encoding header only for older curl
  INSTALL: bump libcurl version to 7.21.3
  http: reset POSTFIELDSIZE when clearing curl handle

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16 23:58:53 +02:00
Johannes Schindelin e1813a335c Merge branch 'jk/redact-h2h3-headers-fix' into maint-2.42
HTTP Header redaction code has been adjusted for a newer version of
cURL library that shows its traces differently from earlier
versions.

* jk/redact-h2h3-headers-fix:
  http: update curl http/2 info matching for curl 8.3.0
  http: factor out matching of curl http/2 trace lines

This backport to `maint-2.39` is needed to bring the following test
cases back to a working state in conjunction with recent libcurl
versions:

- t5559.17 GIT_TRACE_CURL redacts auth details
- t5559.18 GIT_CURL_VERBOSE redacts auth details
- t5559.38 cookies are redacted by default

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16 23:58:48 +02:00
Johannes Schindelin ef0fc42829 Merge branch 'jk/httpd-test-updates'
Test update.

* jk/httpd-test-updates:
  t/lib-httpd: increase ssl key size to 2048 bits
  t/lib-httpd: drop SSLMutex config
  t/lib-httpd: bump required apache version to 2.4
  t/lib-httpd: bump required apache version to 2.2

This is a backport onto the `maint-2.39` branch, to improve the CI
health of that branch.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16 23:58:40 +02:00
Johannes Schindelin e3cbeb9673 Merge branch 'jk/http-test-fixes'
Various fix-ups on HTTP tests.

* jk/http-test-fixes:
  t5559: make SSL/TLS the default
  t5559: fix test failures with LIB_HTTPD_SSL
  t/lib-httpd: enable HTTP/2 "h2" protocol, not just h2c
  t/lib-httpd: respect $HTTPD_PROTO in expect_askpass()
  t5551: drop curl trace lines without headers
  t5551: handle v2 protocol in cookie test
  t5551: simplify expected cookie file
  t5551: handle v2 protocol in upload-pack service test
  t5551: handle v2 protocol when checking curl trace
  t5551: stop forcing clone to run with v0 protocol
  t5551: handle HTTP/2 when checking curl trace
  t5551: lower-case headers in expected curl trace
  t5551: drop redundant grep for Accept-Language
  t5541: simplify and move "no empty path components" test
  t5541: stop marking "used receive-pack service" test as v0 only
  t5541: run "used receive-pack service" test earlier

This is a backport onto the `maint-2.39` branch, starting to take care
of making that branch's CI builds healthy again.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16 23:58:06 +02:00
Johannes Schindelin cf5dcd817a ci(linux-asan/linux-ubsan): let's save some time
Every once in a while, the `git-p4` tests flake for reasons outside of
our control. It typically fails with "Connection refused" e.g. here:
https://github.com/git/git/actions/runs/5969707156/job/16196057724

	[...]
	+ git p4 clone --dest=/home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git //depot
	  Initialized empty Git repository in /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git/.git/
	  Perforce client error:
		Connect to server failed; check $P4PORT.
		TCP connect to localhost:9807 failed.
		connect: 127.0.0.1:9807: Connection refused
	  failure accessing depot: could not run p4
	  Importing from //depot into /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git
	 [...]

This happens in other jobs, too, but in the `linux-asan`/`linux-ubsan`
jobs it hurts the most because those jobs often take an _awfully_ long
time to run, therefore re-running a failed `linux-asan`/`linux-ubsan`
jobs is _very_ costly.

The purpose of the `linux-asan`/`linux-ubsan` jobs is to exercise the C
code of Git, anyway, and any part of Git's source code that the `git-p4`
tests run and that would benefit from the attention of ASAN/UBSAN are
run better in other tests anyway, as debugging C code run via Python
scripts can get a bit hairy.

In fact, it is not even just `git-p4` that is the problem (even if it
flakes often enough to be problematic in the CI builds), but really the
part about Python scripts. So let's just skip any Python parts of the
tests from being run in that job.

For good measure, also skip the Subversion tests because debugging C
code run via Perl scripts is as much fun as debugging C code run via
Python scripts. And it will reduce the time this very expensive job
takes, which is a big benefit.

Backported to `maint-2.39` as another step to get that branch's CI
builds back to a healthy state.

Backported-from: 6ba913629f (ci(linux-asan-ubsan): let's save some time, 2023-08-29)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-16 23:58:04 +02:00
Jeff King c67cf4c434 test-lib: ignore uninteresting LSan output
When I run the tests in leak-checking mode the same way our CI job does,
like:

  make SANITIZE=leak \
       GIT_TEST_PASSING_SANITIZE_LEAK=true \
       GIT_TEST_SANITIZE_LEAK_LOG=true \
       test

then LSan can racily produce useless entries in the log files that look
like this:

  ==git==3034393==Unable to get registers from thread 3034307.

I think they're mostly harmless based on the source here:

  7e0a52e8e9/compiler-rt/lib/lsan/lsan_common.cpp (L414)

which reads:

    PtraceRegistersStatus have_registers =
        suspended_threads.GetRegistersAndSP(i, &registers, &sp);
    if (have_registers != REGISTERS_AVAILABLE) {
      Report("Unable to get registers from thread %llu.\n", os_id);
      // If unable to get SP, consider the entire stack to be reachable unless
      // GetRegistersAndSP failed with ESRCH.
      if (have_registers == REGISTERS_UNAVAILABLE_FATAL)
        continue;
      sp = stack_begin;
    }

The program itself still runs fine and LSan doesn't cause us to abort.
But test-lib.sh looks for any non-empty LSan logs and marks the test as
a failure anyway, under the assumption that we simply missed the failing
exit code somehow.

I don't think I've ever seen this happen in the CI job, but running
locally using clang-14 on an 8-core machine, I can't seem to make it
through a full run of the test suite without having at least one
failure. And it's a different one every time (though they do seem to
often be related to packing tests, which makes sense, since that is one
of our biggest users of threaded code).

We can hack around this by only counting LSan log files that contain a
line that doesn't match our known-uninteresting pattern.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-16 23:58:04 +02:00
Johannes Schindelin 3167b60e5b ci: upgrade to using macos-13
In April, GitHub announced that the `macos-13` pool is available:
https://github.blog/changelog/2023-04-24-github-actions-macos-13-is-now-available/.
It is only a matter of time until the `macos-12` pool is going away,
therefore we should switch now, without pressure of a looming deadline.

Since the `macos-13` runners no longer include Python2, we also drop
specifically testing with Python2 and switch uniformly to Python3, see
https://github.com/actions/runner-images/blob/HEAD/images/macos/macos-13-Readme.md
for details about the software available on the `macos-13` pool's
runners.

Also, on macOS 13, Homebrew seems to install a `gcc@9` package that no
longer comes with a regular `unistd.h` (there seems only to be a
`ssp/unistd.h`), and hence builds would fail with:

    In file included from base85.c:1:
    git-compat-util.h:223:10: fatal error: unistd.h: No such file or directory
      223 | #include <unistd.h>
          |          ^~~~~~~~~~
    compilation terminated.

The reason why we install GCC v9.x explicitly is historical, and back in
the days it was because it was the _newest_ version available via
Homebrew: 176441bfb5 (ci: build Git with GCC 9 in the 'osx-gcc' build
job, 2019-11-27).

To reinstate the spirit of that commit _and_ to fix that build failure,
let's switch to the now-newest GCC version: v13.x.

Backported-from: 682a868f67 (ci: upgrade to using macos-13, 2023-11-03)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-04-16 23:58:03 +02:00
Johannes Schindelin bddc176e79 Merge branch 'jh/fsmonitor-darwin-modernize'
Stop using deprecated macOS API in fsmonitor.

* jh/fsmonitor-darwin-modernize:
  fsmonitor: eliminate call to deprecated FSEventStream function

This backport to `maint-2.39` is needed to be able to build on
`macos-13`, which we need to update to as we restore the CI health of
that branch.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2024-04-16 23:55:55 +02:00
Junio C Hamano 21306a098c The twentieth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-16 14:50:31 -07:00
Junio C Hamano 93e3f9df7a Merge branch 'pw/t3428-cleanup'
Test cleanup.

* pw/t3428-cleanup:
  t3428: restore coverage for "apply" backend
  t3428: use test_commit_message
  t3428: modernize test setup
2024-04-16 14:50:31 -07:00
Junio C Hamano 51c15ac1b6 Merge branch 'ba/osxkeychain-updates'
Update osxkeychain backend with features required for the recent
credential subsystem.

* ba/osxkeychain-updates:
  osxkeychain: store new attributes
  osxkeychain: erase matching passwords only
  osxkeychain: erase all matching credentials
  osxkeychain: replace deprecated SecKeychain API
2024-04-16 14:50:30 -07:00
Junio C Hamano 82a31ec324 Merge branch 'jt/reftable-geometric-compaction'
The strategy to compact multiple tables of reftables after many
operations accumulate many entries has been improved to avoid
accumulating too many tables uncollected.

* jt/reftable-geometric-compaction:
  reftable/stack: use geometric table compaction
  reftable/stack: add env to disable autocompaction
  reftable/stack: expose option to disable auto-compaction
2024-04-16 14:50:30 -07:00
Junio C Hamano 2b49e41155 Merge branch 'tb/make-indent-conditional-with-non-spaces'
Adjust to an upcoming changes to GNU make that breaks our Makefiles.

* tb/make-indent-conditional-with-non-spaces:
  Makefile(s): do not enforce "all indents must be done with tab"
  Makefile(s): avoid recipe prefix in conditional statements
2024-04-16 14:50:29 -07:00
Junio C Hamano a7589384d5 Merge branch 'rs/usage-fallback-to-show-message-format'
vreportf(), which is usede by error() and friends, has been taught
to give the error message printf-format string when its vsnprintf()
call fails, instead of showing nothing useful to identify the
nature of the error.

* rs/usage-fallback-to-show-message-format:
  usage: report vsnprintf(3) failure
2024-04-16 14:50:29 -07:00
Junio C Hamano 107313eb11 Merge branch 'rs/date-mode-pass-by-value'
The codepaths that reach date_mode_from_type() have been updated to
pass "struct date_mode" by value to make them thread safe.

* rs/date-mode-pass-by-value:
  date: make DATE_MODE thread-safe
2024-04-16 14:50:29 -07:00
Junio C Hamano 2d642afb0a Merge branch 'sj/userdiff-c-sharp'
The userdiff patterns for C# has been updated.

Acked-by: Johannes Sixt <j6t@kdbg.org>
cf. <c2154457-3f2f-496e-9b8b-c8ea7257027b@kdbg.org>

* sj/userdiff-c-sharp:
  userdiff: better method/property matching for C#
2024-04-16 14:50:28 -07:00
Junio C Hamano 625ef1c6f1 Merge branch 'tb/t7700-fixup'
Test fix.

* tb/t7700-fixup:
  t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1 `
2024-04-16 14:50:28 -07:00
Junio C Hamano 92e8388bd3 Merge branch 'jc/local-extern-shell-rules'
Document and apply workaround for a buggy version of dash that
mishandles "local var=val" construct.

* jc/local-extern-shell-rules:
  t1016: local VAR="VAL" fix
  t0610: local VAR="VAL" fix
  t: teach lint that RHS of 'local VAR=VAL' needs to be quoted
  t: local VAR="VAL" (quote ${magic-reference})
  t: local VAR="VAL" (quote command substitution)
  t: local VAR="VAL" (quote positional parameters)
  CodingGuidelines: quote assigned value in 'local var=$val'
  CodingGuidelines: describe "export VAR=VAL" rule
2024-04-16 14:50:27 -07:00
René Scharfe 20fee9af9e apply: avoid using fixed-size buffer in write_out_one_reject()
On some systems PATH_MAX is not a hard limit.  Support longer paths by
building them on the heap instead of using static buffers.

Take care to work around (arguably buggy) implementations of free(3)
that change errno by calling it only after using the errno value.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-16 13:38:35 -07:00
Marcel Röthke 167395bb47 rerere: fix crashes due to unmatched opening conflict markers
When rerere handles a conflict with an unmatched opening conflict marker
in a file with other conflicts, it will fail create a preimage and also
fail allocate the status member of struct rerere_dir. Currently the
status member is allocated after the error handling. This will lead to a
SEGFAULT when the status member is accessed during cleanup of the failed
parse.

Additionally, in subsequent executions of rerere, after removing the
MERGE_RR.lock manually, rerere crashes for a similar reason. MERGE_RR
points to a conflict id that has no preimage, therefore the status
member is not allocated and a SEGFAULT happens when trying to check if a
preimage exists.

Solve this by making sure the status field is allocated correctly and add
tests to prevent the bug from reoccurring.

This does not fix the root cause, failing to parse stray conflict
markers, but I don't think we can do much better than recognizing it,
printing an error, and moving on gracefully.

Signed-off-by: Marcel Röthke <marcel@roethke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-16 08:42:36 -07:00
Junio C Hamano 548fe35913 The ninteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-15 14:11:44 -07:00
Junio C Hamano cb25f97eab Merge branch 'jc/t2104-style-fixes'
Test style fixes.

* jc/t2104-style-fixes:
  t2104: style fixes
2024-04-15 14:11:44 -07:00
Junio C Hamano b415f15b49 Merge branch 'jc/unleak-core-excludesfile'
The variable that holds the value read from the core.excludefile
configuration variable used to leak, which has been corrected.

* jc/unleak-core-excludesfile:
  config: do not leak excludes_file
2024-04-15 14:11:44 -07:00
Junio C Hamano eba498a774 Merge branch 'jk/libcurl-8.7-regression-workaround'
Fix was added to work around a regression in libcURL 8.7.0 (which has
already been fixed in their tip of the tree).

* jk/libcurl-8.7-regression-workaround:
  remote-curl: add Transfer-Encoding header only for older curl
  INSTALL: bump libcurl version to 7.21.3
  http: reset POSTFIELDSIZE when clearing curl handle
2024-04-15 14:11:44 -07:00
Junio C Hamano 372aabe912 Merge branch 'ps/t0610-umask-fix'
The "shared repository" test in the t0610 reftable test failed
under restrictive umask setting (e.g. 007), which has been
corrected.

* ps/t0610-umask-fix:
  t0610: execute git-pack-refs(1) with specified umask
  t0610: make `--shared=` tests reusable
2024-04-15 14:11:43 -07:00
Junio C Hamano d75ec4c627 Merge branch 'gt/add-u-commit-i-pathspec-check'
"git add -u <pathspec>" and "git commit [-i] <pathspec>" did not
diagnose a pathspec element that did not match any files in certain
situations, unlike "git add <pathspec>" did.

* gt/add-u-commit-i-pathspec-check:
  builtin/add: error out when passing untracked path with -u
  builtin/commit: error out when passing untracked path with -i
  revision: optionally record matches with pathspec elements
2024-04-15 14:11:43 -07:00
Junio C Hamano 6c142bc846 Merge branch 'ds/fetch-config-parse-microfix'
A config parser callback function fell through instead of returning
after recognising and processing a variable, wasting cycles, which
has been corrected.

* ds/fetch-config-parse-microfix:
  fetch: return when parsing submodule.recurse
2024-04-15 14:11:43 -07:00
Junio C Hamano ce729ea9ba Merge branch 'rs/apply-reject-fd-leakfix'
A file descriptor leak in an error codepath, used when "git apply
--reject" fails to create the *.rej file, has been corrected.

* rs/apply-reject-fd-leakfix:
  apply: don't leak fd on fdopen() error
2024-04-15 14:11:43 -07:00
Junio C Hamano c7a9ec4728 Merge branch 'rs/apply-lift-path-length-limit'
"git apply" has been updated to lift the hardcoded pathname length
limit, which in turn allowed a mksnpath() function that is no
longer used.

* rs/apply-lift-path-length-limit:
  path: remove mksnpath()
  apply: avoid fixed-size buffer in create_one_file()
2024-04-15 14:11:42 -07:00