Commit Graph

369 Commits (master)

Author SHA1 Message Date
Karthik Nayak 73d8380e56 clang-format: add 'RemoveBracesLLVM' to the main config
In 1b8f306612 (ci/style-check: add `RemoveBracesLLVM` in CI job,
2024-07-23) we added 'RemoveBracesLLVM' to the CI job of running the
clang formatter.

This rule checks and warns against using braces on simple
single-statement bodies of statements. Since we haven't had any issues
regarding this rule, we can now move it into the main clang-format
config and remove it from being CI exclusive.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-02 09:05:29 -07:00
Junio C Hamano 1551145edb Merge branch 'js/ci-buildsystems-cleanup'
Code clean-up around stale CI elements and building with Visual Studio.

* js/ci-buildsystems-cleanup:
  config.mak.uname: drop the `vcxproj` target
  contrib/buildsystems: drop support for building . vcproj/.vcxproj files
  ci: stop linking the `prove` cache
2025-05-13 14:05:06 -07:00
Junio C Hamano 03284715a8 Merge branch 'ps/ci-test-aggreg-fix-for-meson'
Test result aggregation did not work in Meson based CI jobs.

* ps/ci-test-aggreg-fix-for-meson:
  ci: fix aggregation of test results with Meson
2025-05-13 14:05:06 -07:00
Junio C Hamano a9d67d67e3 Merge branch 'jc/ci-skip-unavailable-external-software'
Further refinement on CI messages when an optional external
software is unavailable (e.g. due to third-party service outage).

* jc/ci-skip-unavailable-external-software:
  ci: download JGit from maven, not eclipse.org
  ci: update the message for unavailble third-party software
2025-05-12 14:22:49 -07:00
Junio C Hamano 791db2c7ba Merge branch 'kn/meson-hdr-check'
Add an equivalent to "make hdr-check" target to meson based builds.

* kn/meson-hdr-check:
  makefile/meson: add 'check-headers' as alias for 'hdr-check'
  meson: add support for 'hdr-check'
  meson: rename 'third_party_sources' to 'third_party_excludes'
  meson: move headers definition from 'contrib/coccinelle'
  coccinelle: meson: rename variables to be more specific
  ci/github: install git before checking out the repository
2025-05-05 14:56:25 -07:00
Patrick Steinhardt bd38ed5be1 ci: fix aggregation of test results with Meson
Our CI needs to be aware of the location of the test output directory so
that it knows where to find test results. Some of our CI jobs achieve
this by setting the `TEST_OUTPUT_DIRECTORY` environment variable, which
ensures that the output will be written to that directory. Other jobs,
especially on GitHub Workflows, don't set that environment variable and
instead expect test results to be located in the source directory in
"t/".

The latter logic does not work with Meson though, as the test results
are not written into the source directory by default, but instead into
the build directory. As such, any job that uses Meson without setting
the environment variable will be unable to locate and aggregate results.

Fix this by explicitly setting the test output directory when we set up
the Meson build directory. Like this, we can easily default to "t/" in
the source directory when the value hasn't been set explicitly.

Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-05-05 14:19:10 -07:00
Johannes Schindelin 9c1ce1271d ci: stop linking the `prove` cache
It is not useful because we do not have any persisted directory anymore,
not since dropping our Travis CI support.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-05-05 14:15:19 -07:00
Junio C Hamano 956acbefbd ci: download JGit from maven, not eclipse.org
As Matthias Sohn, JGit maintainer, recommends, update the JGit
download link from repo.eclipse.org to a one in maven.org

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-25 08:39:05 -07:00
Junio C Hamano cbc1d8e265 ci: update the message for unavailble third-party software
An earlier fix added an extra message immediately after failing to
download a third-party package.  But near the end of the script,
their availability is checked again and given a message.

Remove the new ones added with a recent fix, as they are redundant.
If we were to add more places to download these software (e.g. for
other platforms we currently do not download them on), the existing
warnning near the end of the script will also trigger.

While at it, as Dscho suggests, rewrite the WARNING: label on the
warning message to :⚠️:, which presumably should be shown a
bit more prominently in the CI summary.

Suggested-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-25 08:39:05 -07:00
Junio C Hamano cbe2267d3c Merge branch 'jc/ci-skip-unavailable-external-software'
Make sure outage of third-party sites that supply P4, Git-LFS, and
JGit we use for testing would not prevent our CI jobs from running
at all.

* jc/ci-skip-unavailable-external-software:
  ci: skip unavailable external software
2025-04-24 17:25:32 -07:00
Junio C Hamano a910fda6b0 Merge branch 'jc/ci-skip-unavailable-external-software' into maint-2.49
Make sure outage of third-party sites that supply P4, Git-LFS, and
JGit we use for testing would not prevent our CI jobs from running
at all.

* jc/ci-skip-unavailable-external-software:
  ci: skip unavailable external software
2025-04-24 17:13:31 -07:00
Junio C Hamano 051923e5ce Merge branch 'js/ci-fedora-gawk' into maint-2.49
Work around CI breakage due to fedora base image getting updated.

* js/ci-fedora-gawk:
  ci(pedantic): ensure that awk is installed
2025-04-24 17:13:31 -07:00
Junio C Hamano b0026daf1e ci: skip unavailable external software
The ci/install-dependencies.sh script used in a very early phase of
our CI jobs downloads Perforce, Git-LFS, and JGit, used for running
the test scripts.  The test framework is prepared to properly skip
the tests that depend on these external software, but the CI script
is unnecessarily strict (due to its use of "set -e" in ci/lib.sh)
and fails the entire CI run before even starting to test the rest of
the system.

Notice a failure to download to any of these external software, but
keep going.  We need to be careful about cleaning after a failed
wget, as a later part of the script that does:

        if type jgit >/dev/null 2>&1
        then
                echo "$(tput setaf 6)JGit Version$(tput sgr0)"
                jgit version
        else
                echo >&2 "WARNING: JGit wasn't installed, see above for clues why"
        fi

will (surprise!) succeed running "type jgit", and then fail with
"jgit version", taking the whole thing down due to "set -e".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-24 16:12:09 -07:00
Karthik Nayak 04a13ed8a7 makefile/meson: add 'check-headers' as alias for 'hdr-check'
The 'hdr-check' target in Meson and makefile is used to check if headers
can be compiled individually. The naming however isn't readable as 'hdr'
is not a common shortforme for 'header', neither is it an abbreviation.

Let's introduce 'check-headers' as an alternative target for 'hdr-check'
and add a `TODO` to deprecate the latter after 2 releases. Since this
is an internal tool, we can use a shorter deprecation cycle.

Change existing usage of 'hdr-check' in 'ci/run-static-analysis.sh' to
also use 'check-headers'.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-23 10:36:42 -07:00
Junio C Hamano c3ebf18eb2 Merge branch 'en/merge-recursive-debug'
Remove remnants of the recursive merge strategy backend, which was
superseded by the ort merge strategy.

* en/merge-recursive-debug:
  builtin/{merge,rebase,revert}: remove GIT_TEST_MERGE_ALGORITHM
  tests: remove GIT_TEST_MERGE_ALGORITHM and test_expect_merge_algorithm
  merge-recursive.[ch]: thoroughly debug these
  merge, sequencer: switch recursive merges over to ort
  sequencer: switch non-recursive merges over to ort
  merge-ort: enable diff-algorithms other than histogram
  builtin/merge-recursive: switch to using merge_ort_generic()
  checkout: replace merge_trees() with merge_ort_nonrecursive()
2025-04-17 10:28:18 -07:00
Junio C Hamano 959760dc42 Merge branch 'js/ci-fedora-gawk'
Work around CI breakage due to fedora base image getting updated.

* js/ci-fedora-gawk:
  ci(pedantic): ensure that awk is installed
2025-04-16 13:54:17 -07:00
Junio C Hamano 03d96fc61d Merge branch 'js/ci-github-update-ubuntu'
Adjust to the deprecation of use of Ubuntu 20.04 GitHub Actions CI.

* js/ci-github-update-ubuntu:
  ci: upgrade `sparse` to supported build agents
2025-04-16 13:54:16 -07:00
Johannes Schindelin 8a471a663b ci(pedantic): ensure that awk is installed
The image pointed to by the fedora:latest tag has moved from fedora
41 to 42. The fedora 41 container images have awk installed while
the fedora 42 images do not.  That change is most likely just part
of reducing the size of the base container images.

In both AlmaLinux and Fedora (as well as other RHEL
derivatives/relatives), awk is provided by the gawk package.

On Fedora, `dnf install awk` would work, by using the package
filelist data to determine that /usr/bin/awk is provided by gawk and
installs gawk as a result.

On AlmaLinux (8 & 9, by quick testing by Todd), that is not the case
and you'd need to use `dnf install gawk` or `dnf install '*bin/awk'`
to get it installed. Having said that, awk _is_ included in the
current AlmaLinux 8 and 9 images, so it isn't strictly needed.  But
it's probably better to be explicit that we need it installed, as a
defense against some future change to the AlmaLinux container
removing awk.

Because we know that on both of these distros, our scripts that call
for 'awk' had been using 'gawk' that was installed as part of the
base image, let's make sure that we explicitly install 'gawk'.  If
the image already has it, it would be a no-op that does not cause
breakage.

Suggested-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-16 07:06:02 -07:00
Johannes Schindelin 832d9f6d0b ci: upgrade `sparse` to supported build agents
The `sparse` job still uses the `ubuntu-20.04` runner pool, but that
pool is about to go away, so let's stop using it.

There is no `sparse-22.04` artifact provided by the "Build sparse for
Ubuntu" Azure Pipeline, but that is not necessary anyway because Ubuntu
22.04 has the `sparse` package: https://packages.ubuntu.com/jammy/sparse

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-09 08:44:26 -07:00
Elijah Newren bfbd201e39 tests: remove GIT_TEST_MERGE_ALGORITHM and test_expect_merge_algorithm
Both of these existed to allow us to reuse all the merge-related tests
in the testsuite while easily flipping between the 'recursive' and the
'ort' backends.  Now that we have removed merge-recursive and remapped
'recursive' to mean 'ort', we don't need this scaffolding anymore.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-08 13:59:13 -07:00
Junio C Hamano ed7d9bd73f Merge branch 'ps/ci-meson-check-build-docs'
CI update.

* ps/ci-meson-check-build-docs:
  ci: perform build and smoke tests for Meson docs
2025-04-08 11:43:14 -07:00
Elijah Newren 85e4f762c2 ci: add build checking for side-effects in assert() calls
It is a big no-no to have side-effects in an assertion, because if the
assert() is compiled out, you don't get that side-effect, leading to the
code behaving differently.  That can be a large headache to debug.

We have roughly 566 assert() calls in our codebase (my grep might have
picked up things that aren't actually assert() calls, but most appeared
to be).  All but 9 of them can be determined by gcc to be free of side
effects with a clever redefine of assert() provided by Bruno De Fraine
(from
https://stackoverflow.com/questions/10593492/catching-assert-with-side-effects),
who upon request has graciously placed his two-liner into the public
domain without warranty of any kind.  The current 9 assert() calls
flagged by this clever redefinition of assert() appear to me to be free
of side effects as well, but are too complicated for a compiler/linker
to figure that since each assertion involves some kind of function call.
Add a CI job which will find and report these possibly problematic
assertions, and have the job suggest to the user that they replace these
with ASSERT() calls.

Example output from running:

```
ERROR: The compiler could not verify the following assert()
       calls are free of side-effects.  Please replace with
       ASSERT() calls.
/home/newren/floss/git/diffcore-rename.c:1409
	assert(!dir_rename_count || strmap_empty(dir_rename_count));
/home/newren/floss/git/merge-ort.c:1645
			assert(renames->deferred[side].trivial_merges_okay &&
			       !strset_contains(&renames->deferred[side].target_dirs,
						path));
/home/newren/floss/git/merge-ort.c:794
	assert(omittable_hint ==
	       (!starts_with(type_short_descriptions[type], "CONFLICT") &&
		!starts_with(type_short_descriptions[type], "ERROR")) ||
	       type == CONFLICT_DIR_RENAME_SUGGESTED);
/home/newren/floss/git/merge-recursive.c:1200
	assert(!merge_remote_util(commit));
/home/newren/floss/git/object-file.c:2709
	assert(would_convert_to_git_filter_fd(istate, path));
/home/newren/floss/git/parallel-checkout.c:280
	assert(is_eligible_for_parallel_checkout(pc_item->ce, &pc_item->ca));
/home/newren/floss/git/scalar.c:244
	assert(have_fsmonitor_support());
/home/newren/floss/git/scalar.c:254
	assert(have_fsmonitor_support());
/home/newren/floss/git/sequencer.c:4968
		assert(!(opts->signoff || opts->no_commit ||
			 opts->record_origin || should_edit(opts) ||
			 opts->committer_date_is_author_date ||
			 opts->ignore_date));
```

Note that if there are possibly problematic assertions, not necessarily
all of them will be shown in a single run, because the compiler errors
may include something like "ld: ... more undefined references to
`not_supposed_to_survive' follow" instead of listing each individually.
But in such cases, once you clean up a few that are shown in your first
run, subsequent runs will show (some of) the ones that remain, allowing
you to iteratively remove them all.

Helped-by: Bruno De Fraine <defraine@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Acked-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-03-21 03:32:04 -07:00
Patrick Steinhardt 9451446c61 ci: perform build and smoke tests for Meson docs
Our "documentation" CI job performs a couple of tests against our
documentation. Part of these tests is to check whether documentation
builds at all and whether it spits out the expected set of files. We
don't yet have such a test for Meson, which means that we wouldn't
notice at all if building the documentation were to break. As a result,
breakages as fixed by 87eccc3a81 (meson: fix building technical and
howto docs, 2025-03-02) are easy to go unnoticed.

Address this test gap by starting to build both manpages and HTML sites
as part of the CI job.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-03-12 10:21:53 -07:00
Patrick Steinhardt 63a597dd94 ci: exercise credential helpers
Wire up credential helpers in our CI runs so that we can rest assured
that they compile and (if tests are available) function correctly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-20 07:49:52 -08:00
Patrick Steinhardt 235fe77c29 ci: fix propagating UTF-8 test locale in musl-based Meson job
The musl-based Meson job is supposed to explicitly specify the UTF-8
locale used for testing, which has been introduced with 84bb5eeace (ci:
switch linux-musl to use Meson, 2025-01-28). That commit had two issues
though:

  - We continue to refer to "linux-musl", even though the job has been
    renamed in the same commit to "linux-musl-meson".

  - We use the wrong option name to specify the locale. This was not
    noticed though due to the first issue.

Fix both of these issues by fixing both the job and option naems.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-20 07:49:52 -08:00
Junio C Hamano 9d0e81e2ae Merge branch 'ps/zlib-ng'
The code paths to interact with zlib has been cleaned up in
preparation for building with zlib-ng.

* ps/zlib-ng:
  ci: make "linux-musl" job use zlib-ng
  ci: switch linux-musl to use Meson
  compat/zlib: allow use of zlib-ng as backend
  git-zlib: cast away potential constness of `next_in` pointer
  compat/zlib: provide stubs for `deflateSetHeader()`
  compat/zlib: provide `deflateBound()` shim centrally
  git-compat-util: move include of "compat/zlib.h" into "git-zlib.h"
  compat: introduce new "zlib.h" header
  git-compat-util: drop `z_const` define
  compat: drop `uncompress2()` compatibility shim
2025-02-06 14:56:45 -08:00
Junio C Hamano 2bf3c7fab1 Merge branch 'ps/ci-misc-updates'
CI updates (containerization, dropping stale ones, etc.).

* ps/ci-misc-updates:
  ci: remove stale code for Azure Pipelines
  ci: use latest Ubuntu release
  ci: stop special-casing for Ubuntu 16.04
  gitlab-ci: add linux32 job testing against i386
  gitlab-ci: remove the "linux-old" job
  github: simplify computation of the job's distro
  github: convert all Linux jobs to be containerized
  github: adapt containerized jobs to be rootless
  t7422: fix flaky test caused by buffered stdout
  t0060: fix EBUSY in MinGW when setting up runtime prefix
2025-02-06 14:56:44 -08:00
Junio C Hamano f49905d47d Merge branch 'ps/build-meson-fixes'
More build fixes and enhancements on meson based build procedure.

* ps/build-meson-fixes:
  ci: wire up Visual Studio build with Meson
  ci: raise error when Meson generates warnings
  meson: fix compilation with Visual Studio
  meson: make the CSPRNG backend configurable
  meson: wire up fuzzers
  meson: wire up generation of distribution archive
  meson: wire up development environments
  meson: fix dependencies for generated headers
  meson: populate project version via GIT-VERSION-GEN
  GIT-VERSION-GEN: allow running without input and output files
  GIT-VERSION-GEN: simplify computing the dirty marker
2025-02-03 10:23:34 -08:00
Junio C Hamano 803b5acaa7 Merge branch 'ps/3.0-remote-deprecation'
Following the procedure we established to introduce breaking
changes for Git 3.0, allow an early opt-in for removing support of
$GIT_DIR/branches/ and $GIT_DIR/remotes/ directories to configure
remotes.

* ps/3.0-remote-deprecation:
  remote: announce removal of "branches/" and "remotes/"
  builtin/pack-redundant: remove subcommand with breaking changes
  ci: repurpose "linux-gcc" job for deprecations
  ci: merge linux-gcc-default into linux-gcc
  Makefile: wire up build option for deprecated features
2025-02-03 10:23:33 -08:00
Patrick Steinhardt 78cdeed4c7 ci: make "linux-musl" job use zlib-ng
We don't yet have any test coverage for the new zlib-ng backend as part
of our CI. Add it by installing zlib-ng in Alpine Linux, which causes
Meson to pick it up automatically.

Note that we are somewhat limited with regards to where we run that job:
Debian-based distributions don't have zlib-ng in their repositories,
Fedora has it but doesn't run tests, and Alma Linux doesn't have the
package either. Alpine Linux does have it available and is running our
test suite, which is why it was picked.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-28 13:03:23 -08:00
Patrick Steinhardt 84bb5eeace ci: switch linux-musl to use Meson
Switch over the "linux-musl" job to use Meson instead of Makefiles. This
is done due to multiple reasons:

  - It simplifies our CI infrastructure a bit as we don't have to
    manually specify a couple of build options anymore.

  - It verifies that Meson detects and sets those build options
    automatically.

  - It makes it easier for us to wire up a new CI job using zlib-ng as
    backend.

One platform compatibility that Meson cannot easily detect automatically
is the `GIT_TEST_UTF8_LOCALE` variable used in tests. Wire up a build
option for it, which we set via a new "MESONFLAGS" environment variable.

Note that we also drop the CC variable, which is set to "gcc". We
already default to GCC when CC is unset in "ci/lib.sh", so this is not
needed.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-28 13:03:23 -08:00
Patrick Steinhardt a8179952e1 ci: raise error when Meson generates warnings
Meson prints warnings in several cases, like for example when using a
feature supported by the current version of Meson, but not yet supported
by the minimum required version as declared by the project. These
warnings will not cause the setup to fail by default, which makes it
quite easy to miss them.

Improve this by passing `--fatal-meson-warnings` to `meson setup` so
that our CI jobs will fail on warnings.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22 12:37:34 -08:00
Patrick Steinhardt 28911f7dca meson: wire up fuzzers
Meson does not yet know to build our fuzzers. Introduce a new build
option "fuzzers" and wire up the fuzzers in case it is enabled. Adapt
our CI jobs so that they build the fuzzers by default.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22 12:37:33 -08:00
Patrick Steinhardt 4b5073c64b ci: repurpose "linux-gcc" job for deprecations
The "linux-gcc" job isn't all that interesting by itself and can be
considered more or less the "standard" job: it is running with a
reasonably up-to-date image and uses GCC as a compiler, both of which we
already cover in other jobs.

There is one exception though: we change the default branch to be "main"
instead of "master", so it is forging ahead a bit into the future to
make sure that this change does not cause havoc. So let's expand on this
a bit and also add the new "WITH_BREAKING_CHANGES" flag to the mix.

Rename the job to "linux-breaking-changes" accordingly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22 12:28:28 -08:00
Patrick Steinhardt 04c29bdea0 ci: merge linux-gcc-default into linux-gcc
The "linux-gcc-default" job is mostly doing the same as the "linux-gcc"
job, except for a couple of minor differences:

  - We use an explicit GCC version instead of the default version
    provided by the distribution. We have other jobs that test with
    "gcc-8", making this distinction pointless.

  - We don't set up the Python version explicitly, and instead use the
    default Python version. Python 2 has been end-of-life for quite a
    while now though, making this distinction less interesting.

  - We set up the default branch name to be "main" in "linux-gcc". We
    have other testcases that don't and also some that explicitly use
    "master".

  - We use "ubuntu:20.04" in one job and "ubuntu:latest" in another. We
    already have a couple other jobs testing these respectively.

So overall, the job does not add much to our test coverage.

Drop the "linux-gcc-default" job and adapt "linux-gcc" to start using
the default GCC compiler, effectively merging those two jobs into one.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22 12:28:27 -08:00
Patrick Steinhardt e39e332e50 ci: remove stale code for Azure Pipelines
Support for Azure Pipelines has been retired in 6081d3898f (ci: retire
the Azure Pipelines definition, 2020-04-11) in favor of GitHub Actions.
Our CI library still has some infrastructure left for Azure though that
is now unused. Remove it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10 09:15:39 -08:00
Patrick Steinhardt 678b22f528 ci: stop special-casing for Ubuntu 16.04
With c85bcb5de1 (gitlab-ci: switch from Ubuntu 16.04 to 20.04,
2024-10-31) we have adapted the last CI job to stop using Ubuntu 16.04
in favor of Ubuntu 20.04. Remove the special-casing we still have in our
CI scripts.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10 09:15:38 -08:00
Patrick Steinhardt 4ad71b16cd gitlab-ci: add linux32 job testing against i386
Add another job to GitLab CI that tests against the i386 architecture.
This job is equivalent to the same job in GitHub Workflows.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10 09:15:38 -08:00
Patrick Steinhardt b133d3071a github: simplify computation of the job's distro
We explicitly list the distro of Linux-based jobs, but it is equivalent
to the name of the image in almost all cases, except that colons are
replaced with dashes. Drop the redundant information and massage it in
our CI scripts, which is equivalent to how we do it in GitLab CI.

There are a couple of exceptions:

  - The "linux32" job, whose distro name is different than the image
    name. This is handled by adapting all sites to use the new name.

  - The "alpine" and "fedora" jobs, neither of which specify a tag for
    their image. This is handled by adding the "latest" tag.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10 09:15:37 -08:00
Patrick Steinhardt 2a21098b98 github: adapt containerized jobs to be rootless
The containerized jobs in GitHub Actions run as root, giving them
special permissions to for example delete files even when the user
shouldn't be able to due to file permissions. This limitation keeps us
from using containerized jobs for most of our Ubuntu-based jobs as it
causes a number of tests to fail.

Adapt the jobs to create a separate user that executes the test suite.
This follows similar infrastructure that we already have in GitLab CI.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10 09:15:37 -08:00
Junio C Hamano effbef2beb Merge branch 'jk/lsan-race-ignore-false-positive'
CI jobs that run threaded programs under LSan has been giving false
positives from time to time, which has been worked around.

This is an alternative to the jk/lsan-race-with-barrier topic with
much smaller change to the production code.

* jk/lsan-race-ignore-false-positive:
  test-lib: ignore leaks in the sanitizer's thread code
  test-lib: check leak logs for presence of DEDUP_TOKEN
  test-lib: simplify leak-log checking
  test-lib: rely on logs to detect leaks
  Revert barrier-based LSan threading race workaround
2025-01-02 13:37:08 -08:00
Junio C Hamano fc89d14c63 Revert barrier-based LSan threading race workaround
The extra "barrier" approach was too much code whose sole purpose
was to work around a race that is not even ours (i.e. in LSan's
teardown code).

In preparation for queuing a solution taking a much-less-invasive
approach, let's revert them.
2025-01-01 14:13:01 -08:00
Junio C Hamano d893741e02 Merge branch 'jk/lsan-race-with-barrier'
CI jobs that run threaded programs under LSan has been giving false
positives from time to time, which has been worked around.

* jk/lsan-race-with-barrier:
  grep: work around LSan threading race with barrier
  index-pack: work around LSan threading race with barrier
  thread-utils: introduce optional barrier type
  Revert "index-pack: spawn threads atomically"
  test-lib: use individual lsan dir for --stress runs
2025-01-01 09:21:15 -08:00
Junio C Hamano 98422943f0 Merge branch 'ps/weak-sha1-for-tail-sum-fix'
An earlier "csum-file checksum does not have to be computed with
sha1dc" topic had a few code paths that had initialized an
implementation of a hash function to be used by an unmatching hash
by mistake, which have been corrected.

* ps/weak-sha1-for-tail-sum-fix:
  ci: exercise unsafe OpenSSL backend
  builtin/fast-import: fix segfault with unsafe SHA1 backend
  bulk-checkin: fix segfault with unsafe SHA1 backend
2025-01-01 09:21:14 -08:00
Patrick Steinhardt 599a63409b ci: exercise unsafe OpenSSL backend
In the preceding commit we have fixed a segfault when using an unsafe
SHA1 backend that is different from the safe one. This segfault only
went by unnoticed because we never set up an unsafe backend in our CI
systems. Fix this ommission by setting `OPENSSL_SHA1_UNSAFE` in our
TEST-vars job.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-30 06:46:30 -08:00
Jeff King 7d0037b59a thread-utils: introduce optional barrier type
One thread primitive we don't yet support is a barrier: it waits for all
threads to reach a synchronization point before letting any of them
continue. This would be useful for avoiding the LSan race we see in
index-pack (and other places) by having all threads complete their
initialization before any of them start to do real work.

POSIX introduced a pthread_barrier_t in 2004, which does what we want.
But if we want to rely on it:

  1. Our Windows pthread emulation would need a new set of wrapper
     functions. There's a Synchronization Barrier primitive there, which
     was introduced in Windows 8 (which is old enough for us to depend
     on).

  2. macOS (and possibly other systems) has pthreads but not
     pthread_barrier_t. So there we'd have to implement our own barrier
     based on the mutex and cond primitives.

Those are do-able, but since we only care about avoiding races in our
LSan builds, there's an easier way: make it a noop on systems without a
native pthread barrier.

This patch introduces a "maybe_thread_barrier" API. The clunky name
(rather than just using pthread_barrier directly) should hopefully clue
people in that on some systems it will do nothing. It's wired to a
Makefile knob which has to be triggered manually, and we enable it for
the linux-leaks CI jobs (since we know we'll have it there).

There are some other possible options:

  - we could turn it on all the time for Linux systems based on uname.
    But we really only care about it for LSan builds, and there is no
    need to add extra code to regular builds.

  - we could turn it on only for LSan builds. But that would break
    builds on non-Linux platforms (like macOS) that otherwise should
    support sanitizers.

  - we could trigger only on the combination of Linux and LSan together.
    This isn't too hard to do, but the uname check isn't completely
    accurate. It is really about what your libc supports, and non-glibc
    systems might not have it (though at least musl seems to).

    So we'd risk breaking builds on those systems, which would need to
    add a new knob. Though the upside would be that running local "make
    SANITIZE=leak test" would be protected automatically.

And of course none of this protects LSan runs from races on systems
without pthread barriers. It's probably OK in practice to protect only
our CI jobs, though. The race is rare-ish and most leak-checking happens
through CI.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-30 06:18:57 -08:00
Junio C Hamano 83c8f76235 Merge branch 'ps/ci-meson'
The meson-build procedure is integrated into CI to catch and
prevent bitrotting.

* ps/ci-meson:
  ci: wire up Meson builds
  t: introduce compatibility options to clar-based tests
  t: fix out-of-tree tests for some git-p4 tests
  Makefile: detect missing Meson tests
  meson: detect missing tests at configure time
  t/unit-tests: rename clar-based unit tests to have a common prefix
  Makefile: drop -DSUPPRESS_ANNOTATED_LEAKS
  ci/lib: support custom output directories when creating test artifacts
2024-12-23 09:32:25 -08:00
Junio C Hamano 19fbad7918 Merge branch 'ps/ci-gitlab-update'
GitLab CI updates.

* ps/ci-gitlab-update:
  ci/lib: fix "CI setup" sections with GitLab CI
  ci/lib: do not interpret escape sequences in `group ()` arguments
  ci/lib: remove duplicate trap to end "CI setup" group
  gitlab-ci: update macOS images to Sonoma
2024-12-23 09:32:07 -08:00
Junio C Hamano 29e5596eb8 Merge branch 'ps/build'
Build procedure update plus introduction of Meson based builds.

* ps/build: (24 commits)
  Introduce support for the Meson build system
  Documentation: add comparison of build systems
  t: allow overriding build dir
  t: better support for out-of-tree builds
  Documentation: extract script to generate a list of mergetools
  Documentation: teach "cmd-list.perl" about out-of-tree builds
  Documentation: allow sourcing generated includes from separate dir
  Makefile: simplify building of templates
  Makefile: write absolute program path into bin-wrappers
  Makefile: allow "bin-wrappers/" directory to exist
  Makefile: refactor generators to be PWD-independent
  Makefile: extract script to generate gitweb.js
  Makefile: extract script to generate gitweb.cgi
  Makefile: extract script to massage Python scripts
  Makefile: extract script to massage Shell scripts
  Makefile: use "generate-perl.sh" to massage Perl library
  Makefile: extract script to massage Perl scripts
  Makefile: consistently use PERL_PATH
  Makefile: generate doc versions via GIT-VERSION-GEN
  Makefile: generate "git.rc" via GIT-VERSION-GEN
  ...
2024-12-15 17:54:33 -08:00
Junio C Hamano cd0a222f08 Merge branch 'es/oss-fuzz'
Backport oss-fuzz tests for us to our codebase.

* es/oss-fuzz:
  fuzz: port fuzz-url-decode-mem from OSS-Fuzz
  fuzz: port fuzz-parse-attr-line from OSS-Fuzz
  fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz
2024-12-13 07:33:42 -08:00