From fc1bac6b57a99cefe2c0febbe57bd19bd3e03ab9 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:23:56 +0200 Subject: [PATCH 1/9] README: add GitLab CI badge to make it more discoverable The Git project uses CI systems from both GitHub and GitLab. While both of these systems are extensively used in day-to-day work, we only have a link to the GitHub Workflows in our README, which makes the GitLab CI hard to discover. Improve the situation by adding a second badge for GitLab CI to our README. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index d87bca1b8c..46489b0971 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ -[![Build status](https://github.com/git/git/workflows/CI/badge.svg)](https://github.com/git/git/actions?query=branch%3Amaster+event%3Apush) +[![GitHub build status](https://github.com/git/git/workflows/CI/badge.svg)](https://github.com/git/git/actions?query=branch%3Amaster+event%3Apush) +[![GitLab build status](https://gitlab.com/git-scm/git/badges/master/pipeline.svg)](https://gitlab.com/git-scm/git/-/pipelines?ref=master) Git - fast, scalable, distributed revision control system ========================================================= From 9769449fc8c9dc508a3cfd4e0af6d182fc6cf619 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:23:57 +0200 Subject: [PATCH 2/9] t0021: skip EXPENSIVE test that is broken without SIZE_T_IS_64BIT One of the tests in t0021 writes a 2GB file and then roundtrips it through the clean/sumdge filters. This test is broken on 32 bit platforms because they typically don't handle files larger then `SSIZE_MAX` well at all. While our CI has a "linux32" job that should in theory hit this issue, we never noticed it because we didn't use to run EXPENSIVE tests until 7a094d68a2 (ci: run expensive tests on push builds to integration branches, 2026-05-08). And after that commit, the test does not fail but instead hangs completely. Ideally, we'd of course properly detect this situation and then test for it. In practice, this turns out to be hard as the test failure are not reliable as they often (but not always) run into ENOMEM errors. Instead, skip the test altogether. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/t0021-conversion.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh index 033b00a364..7b9a0ca877 100755 --- a/t/t0021-conversion.sh +++ b/t/t0021-conversion.sh @@ -296,7 +296,7 @@ test_expect_success 'filter that does not read is fine' ' test_cmp expect actual ' -test_expect_success EXPENSIVE 'filter large file' ' +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'filter large file' ' test_config filter.largefile.smudge cat && test_config filter.largefile.clean cat && test_seq -f "%1048576d" 1 2048 >2GB && From f0598d079afa3110ef662fd543a83923cf72a5f3 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:23:58 +0200 Subject: [PATCH 3/9] t4141: fix inefficient use of dd(1) In t4141 we generate a patch that is roughly 1GB in size to verify that git-apply(1) indeed rejects that patch. We generate that patch by prepending a patch header and then executing `test-tool genzeros` without a limit. This causes us to print infinitely many zeros, and we limit the overall amount of generated bytes via `test_copy_bytes`. This test setup is extremely expensive, as `test_copy_bytes` is implemented via `dd ibs=1 count="$1"`, which copies data one byte at a time. So as we write 1GB of data, we end up doing 1 billion reads and writes. This naturally takes a while: it takes 6 minutes on my system, and around 40 minutes in some CI jobs! We can do much better though, as genzeros already knows to handle an optional limit of how much data it is supposed to write, which allows us to remove the call to `test_copy_bytes`. Furthermore, it has already been optimized to generate the data fast. And indeed, doing this conversion drops the test execution to less than a second on my machine. That means that in theory it becomes feasible to drop the EXPENSIVE prerequisite now. But git-apply(1) still soaks up 1GB of data into memory, which may count as being expensive. Consequently, we keep the prerequisite intact. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/t4141-apply-too-large.sh | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/t/t4141-apply-too-large.sh b/t/t4141-apply-too-large.sh index eac6f7e151..9dbed940db 100755 --- a/t/t4141-apply-too-large.sh +++ b/t/t4141-apply-too-large.sh @@ -5,7 +5,6 @@ test_description='git apply with too-large patch' . ./test-lib.sh test_expect_success EXPENSIVE 'git apply rejects patches that are too large' ' - sz=$((1024 * 1024 * 1023)) && { cat <<-\EOF && diff --git a/file b/file @@ -14,8 +13,8 @@ test_expect_success EXPENSIVE 'git apply rejects patches that are too large' ' +++ b/file @@ -0,0 +1 @@ EOF - test-tool genzeros - } | test_copy_bytes $sz | test_must_fail git apply 2>err && + test-tool genzeros $((1024 * 1024 * 1023)) + } | test_must_fail git apply 2>err && grep "patch too large" err ' From bc1854f525ecc07043ccf131d18217adb926c876 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:23:59 +0200 Subject: [PATCH 4/9] t5608: reduce maximum disk usage The tests in t5608 perform a couple of clones of repositories that are somewhat large. Ultimately, we end up creating: - A setup repository that contains 2GB of uncompressed pack data. - A bare clone that contains the same 2GB of data. - A clone with worktree writes a 2GB packfile and a 2GB worktree. - A second setup repository that contains a 4GB packfile. - Two 4GB clone of that repository. Some of these clones ultimately hardlink files, which ensures that we at least don't end up with more than 20GB of data. But at the end of the test we still have around 16GB of data, which is only a tiny bit better. Refactor the test to prune repositories after they have no use anymore. This reduced the peak disk usage of this test to 8GB. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/t5608-clone-2gb.sh | 66 +++++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 31 deletions(-) diff --git a/t/t5608-clone-2gb.sh b/t/t5608-clone-2gb.sh index 4f8a95ddda..5d56debf1c 100755 --- a/t/t5608-clone-2gb.sh +++ b/t/t5608-clone-2gb.sh @@ -10,45 +10,47 @@ then fi test_expect_success 'setup' ' - - git config pack.compression 0 && - git config pack.depth 0 && - blobsize=$((100*1024*1024)) && - blobcount=$((2*1024*1024*1024/$blobsize+1)) && - i=1 && - (while test $i -le $blobcount - do - printf "Generating blob $i/$blobcount\r" >&2 && - printf "blob\nmark :$i\ndata $blobsize\n" && - #test-tool genrandom $i $blobsize && - printf "%-${blobsize}s" $i && - echo "M 100644 :$i $i" >> commit && - i=$(($i+1)) || - echo $? > exit-status - done && - echo "commit refs/heads/main" && - echo "author A U Thor 123456789 +0000" && - echo "committer C O Mitter 123456789 +0000" && - echo "data 5" && - echo ">2gb" && - cat commit) | - git fast-import --big-file-threshold=2 && - test ! -f exit-status - + git init 2gb-repo && + ( + cd 2gb-repo && + git config pack.compression 0 && + git config pack.depth 0 && + blobsize=$((100*1024*1024)) && + blobcount=$((2*1024*1024*1024/$blobsize+1)) && + i=1 && + (while test $i -le $blobcount + do + printf "Generating blob $i/$blobcount\r" >&2 && + printf "blob\nmark :$i\ndata $blobsize\n" && + #test-tool genrandom $i $blobsize && + printf "%-${blobsize}s" $i && + echo "M 100644 :$i $i" >> commit && + i=$(($i+1)) || + echo $? > exit-status + done && + echo "commit refs/heads/main" && + echo "author A U Thor 123456789 +0000" && + echo "committer C O Mitter 123456789 +0000" && + echo "data 5" && + echo ">2gb" && + cat commit) | + git fast-import --big-file-threshold=2 && + test ! -f exit-status + ) ' test_expect_success 'clone - bare' ' - - git clone --bare --no-hardlinks . clone-bare - + test_when_finished rm -rf clone-bare && + git clone --bare --no-hardlinks 2gb-repo clone-bare ' test_expect_success 'clone - with worktree, file:// protocol' ' - - git clone "file://$(pwd)" clone-wt - + test_when_finished rm -rf clone-wt && + git clone "file://$(pwd)/2gb-repo" clone-wt ' +rm -rf 2gb-repo 2>/dev/null + test_expect_success SIZE_T_IS_64BIT,EXPENSIVE 'set up repo with >4GB object' ' large_blob_size=$((4*1024*1024*1024+1)) && git init --bare 4gb-repo && @@ -61,6 +63,7 @@ test_expect_success SIZE_T_IS_64BIT,EXPENSIVE 'set up repo with >4GB object' ' ' test_expect_success SIZE_T_IS_64BIT,EXPENSIVE 'clone >4GB object via unpack-objects' ' + test_when_finished rm -rf 4gb-clone-unpack && # The synthesized pack has five objects, so a large unpack limit keeps # fetch-pack on the unpack-objects path. git -c fetch.unpackLimit=100 clone --bare \ @@ -77,6 +80,7 @@ test_expect_success SIZE_T_IS_64BIT,EXPENSIVE 'clone >4GB object via unpack-obje ' test_expect_success SIZE_T_IS_64BIT,EXPENSIVE 'clone with >4GB object via index-pack' ' + test_when_finished rm -rf 4gb-clone-index && # Force fetch-pack to hand the pack to index-pack instead. git -c fetch.unpackLimit=1 clone --bare \ "file://$(pwd)/4gb-repo" 4gb-clone-index && From 8f276d146c55247eea4f5304d6b5e9ef293cefaf Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:24:00 +0200 Subject: [PATCH 5/9] t7508: skip EXPENSIVE test that is broken without SIZE_T_IS_64BIT One of the tests in t7508 is marked as EXPENSIVE because it ends up creating and adding files that are multiple gigabytes in size. This takes a while to complete, hence the EXPENSIVE prerequisite. Besides being expensive though the test can only work on systems where `size_t` is at least 64 bit. This is because one of the created files is larger than 4GB, and because Git tracks object size via `size_t` it will eventually blow up. This test has also been blowing up in the "linux32" CI job in GitHub Workflows since 7a094d68a2 (ci: run expensive tests on push builds to integration branches, 2026-05-08). But that job doesn't only fail, it also hangs, and that has been concealing the failure. Fix the issue by marking the test as requiring 64 bit `size_t`. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/t7508-status.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t7508-status.sh b/t/t7508-status.sh index c2057bc94c..dfdd78b6fe 100755 --- a/t/t7508-status.sh +++ b/t/t7508-status.sh @@ -1773,7 +1773,7 @@ test_expect_success 'slow status advice when core.untrackedCache true, and fsmon ) ' -test_expect_success EXPENSIVE 'status does not re-read unchanged 4 or 8 GiB file' ' +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'status does not re-read unchanged 4 or 8 GiB file' ' ( mkdir large-file && cd large-file && From aa4ee1f0a86e2a01f68af63560d94679fc8dd4f5 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:24:01 +0200 Subject: [PATCH 6/9] t7900: clean up large EXPENSIVE repository One of the tests in t7900 is marked with EXPENSIVE because we create a repository with 2GB of data that we end up repacking. We never clean up that repository though, so we occupy the full 2GB of data until the end of the test suite. Besides clogging our disk, having an EXPENSIVE test that alters the repository's state used by subsequent tests is also a bad idea, as it can easily have an impact on the heuristics used by other maintenance tasks. Adapt the test so that we create the data in a standalone repository that we clean up at the end of the test. While at it, also disable auto-maintenance so that it does not race with our manual maintenance. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/t7900-maintenance.sh | 56 +++++++++++++++++++++++------------------- 1 file changed, 31 insertions(+), 25 deletions(-) diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index d7f82e1bec..8a7e1306d0 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -461,36 +461,42 @@ test_expect_success 'incremental-repack task' ' ' test_expect_success EXPENSIVE 'incremental-repack 2g limit' ' - test_config core.compression 0 && + test_when_finished rm -rf expensive-repo && + git init expensive-repo && + ( + cd expensive-repo && + git config set core.compression 0 && + git config set maintenance.auto false && - for i in $(test_seq 1 5) - do - test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big || - return 1 - done && - git add big && - git commit -qm "Add big file (1)" && + for i in $(test_seq 1 5) + do + test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big || + return 1 + done && + git add big && + git commit -qm "Add big file (1)" && - # ensure any possible loose objects are in a pack-file - git maintenance run --task=loose-objects && + # ensure any possible loose objects are in a pack-file + git maintenance run --task=loose-objects && - rm big && - for i in $(test_seq 6 10) - do - test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big || - return 1 - done && - git add big && - git commit -qm "Add big file (2)" && + rm big && + for i in $(test_seq 6 10) + do + test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big || + return 1 + done && + git add big && + git commit -qm "Add big file (2)" && - # ensure any possible loose objects are in a pack-file - git maintenance run --task=loose-objects && + # ensure any possible loose objects are in a pack-file + git maintenance run --task=loose-objects && - # Now run the incremental-repack task and check the batch-size - GIT_TRACE2_EVENT="$(pwd)/run-2g.txt" git maintenance run \ - --task=incremental-repack 2>/dev/null && - test_subcommand git multi-pack-index repack \ - --no-progress --batch-size=2147483647 /dev/null && + test_subcommand git multi-pack-index repack \ + --no-progress --batch-size=2147483647 Date: Mon, 6 Jul 2026 08:24:02 +0200 Subject: [PATCH 7/9] t: use `test_bool_env` to parse GIT_TEST_LONG It's currently hard to explicitly disable GIT_TEST_LONG by setting it to `false`. Fix this by using `test_bool_env` instead. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- ci/lib.sh | 2 +- t/test-lib.sh | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index b939110a6e..01a0bc6b75 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -321,7 +321,7 @@ export SKIP_DASHED_BUILT_INS=YesPlease # enable the long tests for pushes to the integration branches as well. case "$GITHUB_EVENT_NAME,$CI_BRANCH" in pull_request,*|push,*next*|push,*master*|push,*main*|push,*maint*) - export GIT_TEST_LONG=YesPlease + export GIT_TEST_LONG=true ;; esac diff --git a/t/test-lib.sh b/t/test-lib.sh index ceefb99bff..623fcfb747 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -210,7 +210,7 @@ parse_option () { -i|--i|--im|--imm|--imme|--immed|--immedi|--immedia|--immediat|--immediate) immediate=t ;; -l|--l|--lo|--lon|--long|--long-|--long-t|--long-te|--long-tes|--long-test|--long-tests) - GIT_TEST_LONG=t; export GIT_TEST_LONG ;; + GIT_TEST_LONG=true; export GIT_TEST_LONG ;; -r) mark_option_requires_arg "$opt" run_list ;; @@ -1849,7 +1849,7 @@ test_lazy_prereq AUTOIDENT ' ' test_lazy_prereq EXPENSIVE ' - test -n "$GIT_TEST_LONG" + test_bool_env GIT_TEST_LONG false ' test_lazy_prereq EXPENSIVE_ON_WINDOWS ' From 3e35bf7eff61ac43c8ae3fabaf617c6fba8cf5ed Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:24:03 +0200 Subject: [PATCH 8/9] gitlab-ci: disable RAM disk on macOS jobs When we added the macOS jobs to GitLab CI in 56090a35ab (ci: add macOS jobs to GitLab CI, 2024-01-18) we had to work around some very slow disks. This workaround essentially creates a RAM disk that we mount, where all test data is being written into RAM instead of the real disk. In the next commit though we're about to enable "GIT_TEST_LONG", which will make tests run that are marked with the "EXPENSIVE" prerequisite. This change will make a couple of tests run that write up to 8GB of data into the test output directory. As our RAM disk is only 4GB in size, this change will cause ENOSPC errors. We could accommodate for this by increasing the size of the RAM disk. In c9d708b7fc (gitlab-ci: upgrade macOS runners, 2026-05-21) we have upgraded our runners to use the "large" runners, which have 16GB of RAM available. So we could easily expand the RAM disk to a capacity of for example 12GB. But some test runs have shown that this is still quite flaky overall, as we get quite close to our limits. Instead, drop the workaround completely. This does indeed slow down execution of the test jobs: - osx-clang goes from 18 minutes to 25 minutes - osx-meson goes from 21 minutes to 33 minutes - osx-reftable stays at 21 minutes The last one seems like an outlier. The only explanation that I have is that we end up writing significantly less files with the reftable backend, which ultimately causes less I/O. Overall though, it's preferable to have something that works with the least amount of flakiness compared to having something else that is faster but unstable. Despite that, the macOS jobs aren't even the slowest jobs, so this doesn't extend the overall pipeline's length. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- .gitlab-ci.yml | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 1a8e90932c..a4aebe8b71 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -88,13 +88,8 @@ test:osx: tags: - saas-macos-large-m2pro variables: - TEST_OUTPUT_DIRECTORY: "/Volumes/RAMDisk" + TEST_OUTPUT_DIRECTORY: "/tmp/test-output" before_script: - # Create a 4GB RAM disk that we use to store test output on. This small hack - # significantly speeds up tests by more than a factor of 2 because the - # macOS runners use network-attached storage as disks, which is _really_ - # slow with the many small writes that our tests do. - - sudo diskutil apfs create $(hdiutil attach -nomount ram://8192000) RAMDisk - ./ci/install-dependencies.sh script: - ./ci/run-build-and-tests.sh From 84248444ad9577ba7f0bf0679d57c629166eb903 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Mon, 6 Jul 2026 08:24:04 +0200 Subject: [PATCH 9/9] gitlab-ci: enable "GIT_TEST_LONG" Starting with 7a094d68a2 (ci: run expensive tests on push builds to integration branches, 2026-05-08) we run expensive tests in our CI for certain events. So far, this has only been wired up for GitHub Workflows though, which creates a test gap for GitLab CI. Plug this gap by also making this work for the latter. Note that these tests cannot be run on the Windows runners, as they only have 7.5GB of RAM. This is insufficient for some of the EXPENSIVE tests, so we explicitly disable "GIT_TEST_LONG" on these jobs. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- .gitlab-ci.yml | 6 ++++++ ci/lib.sh | 12 ++++++++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a4aebe8b71..1c4d04da9d 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -147,6 +147,9 @@ test:mingw64: needs: - job: "build:mingw64" artifacts: true + variables: + # Windows runners don't have enough RAM to run EXPENSIVE tests. + GIT_TEST_LONG: false before_script: - *windows_before_script - git-sdk/usr/bin/bash.exe -l -c 'tar xf artifacts/artifacts.tar.gz' @@ -195,6 +198,9 @@ test:msvc-meson: script: - | & "C:/Program Files/Git/usr/bin/bash.exe" -l -c 'ci/run-test-slice-meson.sh build $CI_NODE_INDEX $CI_NODE_TOTAL' + variables: + # Windows runners don't have enough RAM to run EXPENSIVE tests. + GIT_TEST_LONG: false after_script: - | if ($env:CI_JOB_STATUS -ne "success") { diff --git a/ci/lib.sh b/ci/lib.sh index 01a0bc6b75..6c52154eac 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -215,6 +215,7 @@ then test macos != "$CI_OS_NAME" || CI_OS_NAME=osx CI_REPO_SLUG="$GITHUB_REPOSITORY" CI_JOB_ID="$GITHUB_RUN_ID" + CI_EVENT="$GITHUB_EVENT_NAME" CC="${CC_PACKAGE:-${CC:-gcc}}" DONT_SKIP_TAGS=t handle_failed_tests () { @@ -239,6 +240,13 @@ then CI_BRANCH="$CI_COMMIT_REF_NAME" CI_COMMIT="$CI_COMMIT_SHA" + case "$CI_PIPELINE_SOURCE" in + merge_request_event) + CI_EVENT=pull_request;; + *) + CI_EVENT="$CI_PIPELINE_SOURCE";; + esac + case "$OS,$CI_JOB_IMAGE" in Windows_NT,*) CI_OS_NAME=windows @@ -319,9 +327,9 @@ export SKIP_DASHED_BUILT_INS=YesPlease # enable "expensive" tests for PR events. # In order to catch bugs introduced at integration time by mismerges, # enable the long tests for pushes to the integration branches as well. -case "$GITHUB_EVENT_NAME,$CI_BRANCH" in +case "$CI_EVENT,$CI_BRANCH" in pull_request,*|push,*next*|push,*master*|push,*main*|push,*maint*) - export GIT_TEST_LONG=true + export GIT_TEST_LONG=${GIT_TEST_LONG:-true} ;; esac