kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Johannes Schindelin	4463ce75b7	ci: do not skip tagged revisions in GitHub workflows When `master` is tagged, and then both `master` and the tag are pushed, Travis CI will happily build both. That is a waste of energy, which is why we skip the build for `master` in that case. Our GitHub workflow is also triggered by tags. However, the run would fail because the `windows-test` jobs are _not_ skipped on tags, but the `windows-build` job _is skipped (and therefore fails to upload the build artifacts needed by the test jobs). In addition, we just added logic to our GitHub workflow that will skip runs altogether if there is already a successful run for the same commit or at least for the same tree. Let's just change the GitHub workflow to no longer specifically skip tagged revisions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Johannes Schindelin	ef60e9f74b	ci: stop linking built-ins to the dashed versions Since `e4597aae65` (run test suite without dashed git-commands in PATH, 2009-12-02), we stopped running our tests with `git-foo` binaries found at the top-level directory of a freshly built source tree; instead we have placed only `git` and selected `git-foo` commands that must be on `$PATH` in `bin-wrappers/` and prepended that `bin-wrappers/` to the `PATH` used in the test suite. We did that to catch the tests and scripted Git commands that still try to use the dashed form. Since CI jobs will not install the built Git to anywhere, and the hardlinks we make at the top-level of the source tree for `git-add` and friends are not even used during tests, they are pure waste of resources these days. Thanks to the newly invented `SKIP_DASHED_BUILT_INS` knob, we can now skip creating these links in the source tree. So let's do that. Note that this change introduces a subtle change of behavior: when Git's `cmd_main()` calls `setup_path()`, it inserts the value of `GIT_EXEC_PATH` (defaulting to `<prefix>/libexec/git-core`) at the beginning of the environment variable `PATH`. This is necessary to find e.g. scripted commands that are installed in that location. For the purposes of Git's test suite, the `bin-wrappers/` scripts override `GIT_EXEC_PATH` to point to the top-level directory of the source code. In other words, if a scripted command had used a dashed invocation of a built-in Git command, it would not have been caught previously, which is fixed by this change. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
SZEDER Gábor	60e47f6773	ci: use absolute PYTHON_PATH in the Linux jobs In our test suite, when 'git p4' invokes a Git command as a subprocesses, then it should run the 'git' binary we are testing. Unfortunately, this is not the case in the 'linux-clang' and 'linux-gcc' jobs on Travis CI, where 'git p4' runs the system '/usr/bin/git' instead. Travis CI's default Linux image includes 'pyenv', and all Python invocations that involve PATH lookup go through 'pyenv', e.g. our 'PYTHON_PATH=$(which python3)' sets '/opt/pyenv/shims/python3' as PYTHON_PATH, which in turn will invoke '/usr/bin/python3'. Alas, the 'pyenv' version included in this image is buggy, and prepends the directory containing the Python binary to PATH even if that is a system directory already in PATH near the end. Consequently, 'git p4' in those jobs ends up with its PATH starting with '/usr/bin', and then runs '/usr/bin/git'. So use the absolute paths '/usr/bin/python{2,3}' explicitly when setting PYTHON_PATH in those Linux jobs to avoid the PATH lookup and thus the bogus 'pyenv' from interfering with our 'git p4' tests. Don't bother with special-casing Travis CI: while this issue doesn't affect the corresponding Linux jobs on GitHub Actions, both CI systems use Ubuntu LTS-based images, so we can safely rely on these Python paths. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Junio C Hamano	4024295568	Revert "ci: add a problem matcher for GitHub Actions" This reverts commit 676eb0c1ce0d380478eb16bdc5a3f2a7bc01c1d2; as we will be reverting the change to show these extra output tokens under bash, the pattern would not match anything. Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	676eb0c1ce	ci: add a problem matcher for GitHub Actions With this patch, test failures will be annotated with a helpful, clickable message in GitHub Actions. For details, see https://github.com/actions/toolkit/blob/master/docs/problem-matchers.md Note: we need to set `TEST_SHELL_PATH` to Bash so that the problem matcher is fed a file and line number for each test failure. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	87b68db3ac	ci: fix the `jobname` of the `GETTEXT_POISON` job In `6cdccfce1e` (i18n: make GETTEXT_POISON a runtime option, 2018-11-08), the `jobname` was adjusted to have the `GIT_TEST_` prefix, but that prefix makes no sense in this context. Co-authored-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Đoàn Trần Công Danh	855c158e81	ci/lib: set TERM environment variable if not exist GitHub Action doesn't set TERM environment variable, which is required by "tput". Fallback to dumb if it's not set. Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	a3f2eec862	ci/lib: allow running in GitHub Actions For each CI system we support, we need a specific arm in that if/else construct in ci/lib.sh. Let's add one for GitHub Actions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	5127e8cf7a	ci/lib: if CI type is unknown, show the environment variables This should help with adding new CI-specific if-else arms. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Đoàn Trần Công Danh	e0f8690dee	travis: build and test on Linux with musl libc and busybox Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	d2fae19e0f	ci: make MAKEFLAGS available inside the Docker container in the Linux32 job Once upon a time we ran 'make --jobs=2 ...' to build Git, its documentation, or to apply Coccinelle semantic patches. Then commit `eaa62291ff` (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests, 2019-01-27) came along, and started using the MAKEFLAGS environment variable to centralize setting the number of parallel jobs in 'ci/libs.sh'. Alas, it forgot to update 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container running the 32 bit Linux job, and, consequently, since then that job builds Git sequentially, and it ignores any Makefile knobs that we might set in MAKEFLAGS (though we don't set any for the 32 bit Linux job at the moment). So update the 'docker run' invocation in 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container as well. Set CC=gcc for the 32 bit Linux job, because that's the compiler installed in the 32 bit Linux Docker image that we use (Travis CI nowadays sets CC=clang by default, but clang is not installed in this image). Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	6bb40ed20a	ci: use python3 in linux-gcc and osx-gcc and python2 elsewhere Python2 reached end of life, and we have been preparing our Python scripts to work with Python3. 'git p4', the main in-tree user of Python, has just received a number of compatibility updates. Our other notable Python script 'contrib/svn-fe/svnrdump_sim.py' is only used in 't9020-remote-svn.sh', and is apparently already compatible with both Python2 and 3. Our CI jobs currently only use Python2. We want to make sure that these Python scripts do indeed work with Python3, and we also want to make sure that these scripts keep working with Python2 as well, for the sake of some older LTS/Enterprise setups. Therefore, pick two jobs and use Python3 there, while leaving other jobs to still stick to Python2 for now. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	176441bfb5	ci: build Git with GCC 9 in the 'osx-gcc' build job Our 'osx-gcc' build job on Travis CI relied on GCC 8 being installed (but not linked) in the image we use [1]. Alas, since the last update of this image a few days ago this is not the case anymore, and now it contains GCC 9 (installed and linked) instead of GCC 8. The results are failed 'osx-gcc' jobs, because they can't find the 'gcc-8' command [2]. Let's move on to use GCC 9, with hopefully better error reporting and improved -Wfoo flags and what not. On Travis CI this has the benefit that we can spare a few seconds while installing dependencies, because it already comes pre-installed, at least for now. The Azure Pipelines OSX image doesn't include GCC, so we have to install it ourselves anyway, and then we might as well install the newer version. In a vain attempt I tried to future-proof this a bit: - Install 'gcc@9' specifically, so we'll still get what we want even after GCC 10 comes out, and the "plain" 'gcc' package starts to refer to 'gcc@10'. - Run both 'brew install gcc@9' and 'brew link gcc@9'. If 'gcc@9' is already installed and linked, then both commands are noop and exit with success. But as we saw in the past, sometimes the image contains the expected GCC package installed but not linked, so maybe it will happen again in the future as well. In that case 'brew install' is still a noop, and instructs the user to run 'brew link' instead, so that's what we'll do. And if 'gcc@9' is not installed, then 'brew install' will install it, and the subsequent 'brew link' becomes a noop. An additional benefit of this patch is that from now on we won't unnecessarily install GCC and its dependencies in the 'osx-clang' jobs on Azure Pipelines. [1] `7d4733c501` (ci: fix GCC install in the Travis CI GCC OSX job, 2019-10-24) [2] https://travis-ci.org/git/git/jobs/615442297#L333 Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	a85efb5985	t5608-clone-2gb.sh: turn GIT_TEST_CLONE_2GB into a bool The GIT_TEST_CLONE_2GB environment variable is only ever checked with 'test -z' in 't5608-clone-2gb.sh', so any non-empty value is interpreted as "yes, run these expensive tests", even 'GIT_TEST_CLONE_2GB=NoThanks'. Similar GIT_TEST_* environment variables have already been turned into bools in `3b072c577b` (tests: replace test_tristate with "git env--helper", 2019-06-21), so let's turn GIT_TEST_CLONE_2GB into a bool as well, to follow suit. Our CI builds set GIT_TEST_CLONE_2GB=YesPlease, so adjust them accordingly, thus removing the last 'YesPlease' from our CI scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	c46ebc2496	travis-ci: do not skip successfully tested trees in debug mode Travis CI offers shell access to its virtual machine environment running the build jobs, called "debug mode" [1]. After restarting a build job in debug mode and logging in, the first thing I usually do is to install dependencies, i.e. run './ci/install-dependencies.sh'. This works just fine when I restarted a failed build job in debug mode. However, after restarting a successful build job in debug mode our CI scripts get all clever, and exit without doing anything useful, claiming that "This commit's tree has already been built and tested successfully" [2]. Our CI scripts are right, and we do want to skip building and testing already known good trees in "regular" CI builds. In debug mode, however, this is a nuisiance, because one has to delete the cache (or at least the 'good-trees' file in the cache) to proceed. Let's update our CI scripts, in particular the common 'ci/lib.sh', to not skip previously successfully built and tested trees in debug mode, so all those scripts will do what there were supposed to do even when a successful build job was restarted in debug mode. [1] https://docs.travis-ci.com/user/running-build-in-debug-mode/ [2] `9cc2c76f5e` (travis-ci: record and skip successfully built trees, 2017-12-31) Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	3960290675	ci: restore running httpd tests Once upon a time GIT_TEST_HTTPD was a tristate variable and we exported 'GIT_TEST_HTTPD=YesPlease' in our CI scripts to make sure that we run the httpd tests in the Linux Clang and GCC build jobs, or error out if they can't be run for any reason [1]. Then `3b072c577b` (tests: replace test_tristate with "git env--helper", 2019-06-21) came along, turned GIT_TEST_HTTPD into a bool, but forgot to update our CI scripts accordingly. So, since GIT_TEST_HTTPD is set explicitly, but its value is not one of the standardized true values, our CI jobs have been simply skipping the httpd tests in the last couple of weeks. Set 'GIT_TEST_HTTPD=true' to restore running httpd tests in our CI jobs. [1] `a1157b76eb` (travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh', 2017-12-12) Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	37a2e35395	ci/lib.sh: update a comment about installed P4 and Git-LFS versions A comment in 'ci/lib.sh' claims that the "OS X build installs the latest available versions" of P4 and Git-LFS, but since `f2f47150` ("ci: don't update Homebrew", 2019-07-03) that's no longer the case, as it will install the versions which were recorded in the image's Homebrew database when the image was created. Update this comment accordingly. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	1ff750b128	tests: make GIT_TEST_GETTEXT_POISON a boolean Change the GIT_TEST_GETTEXT_POISON variable from being "non-empty?" to being a more standard boolean variable. Since it needed to be checked in both C code and shellscript (via test -n) it was one of the remaining shellscript-like variables. Now that we have "env--helper" we can change that. There's a couple of tricky edge cases that arise because we're using git_env_bool() early, and the config-reading "env--helper". If GIT_TEST_GETTEXT_POISON is set to an invalid value die_bad_number() will die, but to do so it would usually call gettext(). Let's detect the special case of GIT_TEST_GETTEXT_POISON and always emit that message in the C locale, lest we infinitely loop. As seen in the updated tests in t0017-env-helper.sh there's also a caveat related to "env--helper" needing to read the config for trace2 purposes. Since the C_LOCALE_OUTPUT prerequisite is lazy and relies on "env--helper" we could get invalid results if we failed to read the config (e.g. because we'd loop on includes) when combined with e.g. "test_i18ngrep" wanting to check with "env--helper" if GIT_TEST_GETTEXT_POISON was true or not. I'm crossing my fingers and hoping that a test similar to the one I removed in the earlier "config tests: simplify include cycle test" change in this series won't happen again, and testing for this explicitly in "env--helper"'s own tests. This change breaks existing uses of e.g. GIT_TEST_GETTEXT_POISON=YesPlease, which we've documented in po/README and other places. As noted in [1] we might want to consider also accepting "YesPlease" in "env--helper" as a special-case. But as the lack of uproar over `6cdccfce1e` ("i18n: make GETTEXT_POISON a runtime option", 2018-11-08) demonstrates the audience for this option is a really narrow set of git developers, who shouldn't have much trouble modifying their test scripts, so I think it's better to deal with that minor headache now and make all the relevant GIT_TEST_* variables boolean in the same way than carry the "YesPlease" special-case forward. 1. https://public-inbox.org/git/xmqqtvckm3h8.fsf@gitster-ct.c.googlers.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Junio C Hamano	a8c51f77d1	ci: clear and mark MAKEFLAGS exported just once Clearing it once upfront, and turning all the assignment into appending, would future-proof the code even more, to prevent mistakes the previous one fixed from happening again. Also, mark the variable exported just once at the beginning. There is no point in marking it exported repeatedly. Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	406f93ae48	ci: make sure we build Git parallel Commit `2c8921db2b` (travis-ci: build with the right compiler, 2019-01-17) started to use MAKEFLAGS to specify which compiler to use to build Git. A bit later, and in a different topic branch commit `eaa62291ff` (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests, 2019-01-27) started to use MAKEFLAGS as well. Unfortunately, there is a semantic conflict between these two commits: both of them set MAKEFLAGS, and since the line adding CC from `2c8921db2b` comes later in 'ci/lib.sh', it overwrites the number of parallel jobs added in `eaa62291ff`. Consequently, since both commits have been merged all our CI jobs have been building Git, building its documentation, and applying semantic patches sequentially, making all build jobs a bit slower. Running the test suite is unaffected, because the number of test jobs comes from GIT_PROVE_OPTS. Append to MAKEFLAGS when setting the compiler to use, to ensure that the number of parallel jobs to use is preserved. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	a87e427e35	ci: speed up Windows phase As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite. Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	6141a2edc9	ci/lib.sh: add support for Azure Pipelines This patch introduces a conditional arm that defines some environment variables and a function that displays the URL given the job id (to identify previous runs for known-good trees). Because Azure Pipeline's macOS agents already have git-lfs and gettext installed, we can leave `BREW_INSTALL_PACKAGES` empty (unlike in Travis' case). Note: this patch does not introduce an Azure Pipelines definition yet; That is left for the next patch. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	eaa62291ff	ci: inherit --jobs via MAKEFLAGS in run-build-and-tests Let's not decide in the generic ci/ part how many jobs to run in parallel; different CI configurations would favor a different number of parallel jobs, and it is easy enough to hand that information down via the `MAKEFLAGS` variable. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	b011fabd6e	ci/lib.sh: encapsulate Travis-specific things The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI. Also, the configurations of the available agents are different. For example, Travis' and Azure Pipelines' macOS agents are set up differently, so that on Travis, we have to install the git-lfs and gettext Homebrew packages, and on Azure Pipelines we do not need to. Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor installed. Finally, on Azure Pipelines the natural way is not to base64-encode tar files of the trash directories of failed tests, but to publish build artifacts instead. Therefore, that code to log those base64-encoded tar files is guarded to be Travis-specific. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	c2160f2d19	ci: rename the library of common functions The name is hard-coded to reflect that we use Travis CI for continuous testing. In the next commits, we will extend this to be able use Azure DevOps, too. So let's adjust the name to make it more generic. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	4096a98d79	travis: fix skipping tagged releases When building a PR, TRAVIS_BRANCH refers to the target branch. Therefore, if a PR targets `master`, and `master` happened to be tagged, we skipped the build by mistake. Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the source branch) when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also known as "push builds"). Let's give it a new variable name, too: CI_BRANCH (as it is different from TRAVIS_BRANCH). This also prepares for the upcoming patches which will make our ci/* code a bit more independent from Travis and open it to other CI systems (in particular to Azure Pipelines). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	2c8921db2b	travis-ci: build with the right compiler Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This CC variable can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to be on the platform. Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang both on Linux and macOS from the very beginning (`522354d70f` (Add Travis CI support, 2015-11-27)). Alas, this never really worked as supposed to, because Travis CI specifies the compiler for those build jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for projects built with './configure && make'). Consequently, our 'linux-clang' build job has always used GCC, because that's where 'cc' points at in Travis CI's Linux images, while the 'osx-gcc' build job has always used Clang. Furthermore, `37fa4b3c78` (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an attempt to build with a more modern compiler, but to no avail. Set MAKEFLAGS with CC based on the $CC environment variable, so 'make' will run the "right" compiler. The Xcode 10.1 macOS image on Travis CI already contains the gcc@8 package from Homebrew, but we have to 'brew link' it first to be able to use it. So with this patch our build jobs will build Git with the following compiler versions: linux-clang: clang version 5.0.0 (tags/RELEASE_500/final) linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0 osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5) osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0 GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4 Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	6cdccfce1e	i18n: make GETTEXT_POISON a runtime option Change the GETTEXT_POISON compile-time + runtime GIT_GETTEXT_POISON test parameter to only be a GIT_TEST_GETTEXT_POISON=<non-empty?> runtime parameter, to be consistent with other parameters documented in "Running tests with special setups" in t/README. When I added GETTEXT_POISON in `bb946bba76` ("i18n: add GETTEXT_POISON to simulate unfriendly translator", 2011-02-22) I was concerned with ensuring that the _() function would get constant folded if NO_GETTEXT was defined, and likewise that GETTEXT_POISON would be compiled out unless it was defined. But as the benchmark in my [1] shows doing a one-off runtime getenv("GIT_TEST_[...]") is trivial, and since GETTEXT_POISON was originally added the GIT_TEST_* env variables have become the common idiom for turning on special test setups. So change GETTEXT_POISON to work the same way. Now the GETTEXT_POISON=YesPlease compile-time option is gone, and running the tests with GIT_TEST_GETTEXT_POISON=[YesPlease\|] can be toggled on/off without recompiling. This allows for conditionally amending tests to test with/without poison, similar to what `859fdc0c3c` ("commit-graph: define GIT_TEST_COMMIT_GRAPH", 2018-08-29) did for GIT_TEST_COMMIT_GRAPH. Do some of that, now we e.g. always run the t0205-gettext-poison.sh test. I did enough there to remove the GETTEXT_POISON prerequisite, but its inverse C_LOCALE_OUTPUT is still around, and surely some tests using it can be converted to e.g. always set GIT_TEST_GETTEXT_POISON=. Notes on the implementation: * We still compile a dedicated GETTEXT_POISON build in Travis CI. Perhaps this should be revisited and integrated into the "linux-gcc" build, see `ae59a4e44f` ("travis: run tests with GIT_TEST_SPLIT_INDEX", 2018-01-07) for prior art in that area. Then again maybe not, see [2]. * We now skip a test in t0000-basic.sh under GIT_TEST_GETTEXT_POISON=YesPlease that wasn't skipped before. This test relies on C locale output, but due to an edge case in how the previous implementation of GETTEXT_POISON worked (reading it from GIT-BUILD-OPTIONS) wasn't enabling poison correctly. Now it does, and needs to be skipped. * The getenv() function is not reentrant, so out of paranoia about code of the form: printf(_("%s"), getenv("some-env")); call use_gettext_poison() in our early setup in git_setup_gettext() so we populate the "poison_requested" variable in a codepath that's won't suffer from that race condition. * We error out in the Makefile if you're still saying GETTEXT_POISON=YesPlease to prompt users to change their invocation. * We should not print out poisoned messages during the test initialization itself to keep it more readable, so the test library hides the variable if set in $GIT_TEST_GETTEXT_POISON_ORIG during setup. See [3]. See also [4] for more on the motivation behind this patch, and the history of the GETTEXT_POISON facility. 1. https://public-inbox.org/git/871s8gd32p.fsf@evledraar.gmail.com/ 2. https://public-inbox.org/git/20181102163725.GY30222@szeder.dev/ 3. https://public-inbox.org/git/20181022202241.18629-2-szeder.dev@gmail.com/ 4. https://public-inbox.org/git/878t2pd6yu.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	aea8879a6a	travis-ci: include the trash directories of failed tests in the trace log The trash directory of a failed test might contain invaluable information about the cause of the failure, but we have no access to the trash directories of Travis CI build jobs. The only feedback we get from there is the build job's trace log, so... Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the trash directory of each failed test, encode that archive with base64, and print the resulting block of ASCII text, so it gets embedded in the trace log. Furthermore, run tests with '--immediate' to faithfully preserve the failed state. Extracting the trash directories from the trace log turned out to be a bit of a hassle, partly because of the size of these logs (usually resulting in several hundreds or even thousands of lines of base64-encoded text), and partly because these logs have CRLF, CRCRLF and occasionally even CRCRCRLF line endings, which cause 'base64 -d' from coreutils to complain about "invalid input". For convenience add a small script 'ci/util/extract-trash-dirs.sh', which will extract and unpack all base64-encoded trash directories embedded in the log fed to its standard input, and include an example command to be copy-pasted into a terminal to do it all at the end of the failure report. A few of our tests create sizeable trash directories, so limit the size of each included base64-encoded block, let's say, to 1MB. And just in case something fundamental gets broken and a lot of tests fail at once, don't include trash directories when the combined size of the included base64-encoded blocks would exceed 1MB. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Nguyễn Thái Ngọc Duy	37fa4b3c78	travis-ci: run gcc-8 on linux-gcc jobs Switch from gcc-4.8 to gcc-8. Newer compilers come with more warning checks (usually in -Wextra). Since -Wextra is enabled in developer mode (which is also enabled in travis), this lets travis report more warnings before other people do it. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	aedffe9525	travis-ci: run tests with '-x' tracing Now that the test suite runs successfully with '-x' tracing even with /bin/sh, enable it on Travis CI in order to - get more information about test failures, and - catch constructs breaking '-x' with /bin/sh sneaking into our test suite. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	b2cbaa091c	travis-ci: don't repeat the path of the cache directory Some of our 'ci/*' scripts repeat the name or full path of the Travis CI cache directory, and the following patches will add new places using that path. Use a variable to refer to the path of the cache directory instead, so it's hard-coded only in a single place. Pay extra attention to the 32 bit Linux build: it runs in a Docker container, so pass the path of the cache directory from the host to the container in an environment variable. Note that an environment variable passed this way is exported inside the container, therefore its value is directly available in the 'su' snippet even though that snippet is single quoted. Furthermore, use the variable in the container only if it's been assigned a non-empty value, to prevent errors when someone is running or debugging the Docker build locally, because in that case the variable won't be set as there won't be any Travis CI cache. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	b92cb86ea1	travis-ci: check that all build artifacts are .gitignore-d Every once in a while our explicit .gitignore files get out of sync when our build process learns to create new artifacts, like test helper executables, but the .gitignore files are not updated accordingly. Use Travis CI to help catch such issues earlier: check that there are no untracked files at the end of any build jobs building Git (i.e. the 64 bit Clang and GCC Linux and OSX build jobs, plus the GETTEXT_POISON and 32 bit Linux build jobs) or its documentation, and fail the build job if there are any present. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	88e00b7033	travis-ci: don't store P4 and Git LFS in the working tree The Clang and GCC 64 bit Linux build jobs download and store the P4 and Git LFS executables under the current directory, which is the working tree that we are about to build and test. This means that Git commands like 'status' or 'ls-files' would list these files as untracked. The next commit is about to make sure that there are no untracked files present after the build, and the downloaded executables in the working tree are interfering with those upcoming checks. Therefore, let's download P4 and Git LFS in the home directory, outside of the working tree. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	9cc2c76f5e	travis-ci: record and skip successfully built trees Travis CI dutifully builds and tests each new branch tip, even if its tree has previously been successfully built and tested. This happens often enough in contributors' workflows, when a work-in-progress branch is rebased changing e.g. only commit messages or the order or number of commits while leaving the resulting code intact, and is then pushed to a Travis CI-enabled GitHub fork. This is wasting Travis CI's resources and is sometimes scary-annoying when the new tip commit with a tree identical to the previous, successfully tested one is suddenly reported in red, because one of the OSX build jobs happened to exceed the time limit yet again. So extend our Travis CI build scripts to skip building commits whose trees have previously been successfully built and tested. Use the Travis CI cache feature to keep a record of the object names of trees that tested successfully, in a plain and simple flat text file, one line per tree object name. Append the current tree's object name at the end of every successful build job to this file, along with a bit of additional info about the build job (commit object name, Travis CI job number and id). Limit the size of this file to 1000 records, to prevent it from growing too large for git/git's forever living integration branches. Check, using a simple grep invocation, in each build job whether the current commit's tree is already in there, and skip the build if it is. Include a message in the skipped build job's trace log, containing the URL to the build job successfully testing that tree for the first time and instructions on how to force a re-build. Catch the case when a build job, which successfully built and tested a particular tree for the first time, is restarted and omit the URL of the previous build job's trace log, as in this case it's the same build job and the trace log has just been overwritten. Note: this won't kick in if two identical trees are on two different branches, because Travis CI caches are not shared between build jobs of different branches. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	b4a2fdc9bd	travis-ci: create the cache directory early in the build process It seems that Travis CI creates the cache directory for us anyway, even when a previous cache doesn't exist for the current build job. Alas, this behavior is not explicitly documented, therefore we don't rely on it and create the cache directory ourselves in those build jobs that read/write cached data (currently only the prove state). In the following commit we'll start to cache additional data in every build job, and will access the cache much earlier in the build process. Therefore move creating the cache directory to 'ci/lib-travisci.sh' to make sure that it exists at the very beginning of every build job. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	495ea6cd41	travis-ci: print the "tip of branch is exactly at tag" message in color To make this info message stand out from the regular build job trace output. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	a8b8b6b87d	travis-ci: fine tune the use of 'set -x' in 'ci/' scripts The change in commit `4f2636667` (travis-ci: use 'set -x' in 'ci/' scripts for extra tracing output, 2017-12-12) left a couple of rough edges: - 'ci/run-linux32-build.sh' is executed in a Docker container and therefore doesn't source 'ci/lib-travisci.sh', which would enable tracing executed commands. Enable 'set -x' in this script, too. - 'ci/print-test-failures.sh' iterates over all the files containing the exit codes of all the executed test scripts. Since there are over 800 such files, the loop produces way too much noise with tracing executed commands enabled, so disable 'set -x' for this script. - 'ci/run-windows-build.sh' busily waits in a loop for the result of the Windows build, producing too much noise with tracing executed commands enabled as well. Disable 'set -x' for the duration of that loop. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	4f26366679	travis-ci: use 'set -x' in 'ci/' scripts for extra tracing output While the build logic was embedded in our '.travis.yml', Travis CI used to produce a nice trace log including all commands executed in those embedded scriptlets. Since `657343a60` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10), however, we only see the name of the dedicated scripts, but not what those scripts are actually doing, resulting in a less useful trace log. A patch later in this series will move setting environment variables from '.travis.yml' to the 'ci/' scripts, so not even those will be included in the trace log. Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other 'ci/*' scripts, so we get trace log about the commands executed in all of those scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	a1157b76eb	travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh' Commit `657343a60` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10) converted '.travis.yml's default 'before_install' scriptlet to the 'ci/install-dependencies.sh' script, and while doing so moved setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs to that script. This is wrong for two reasons: - The purpose of that script is, as its name suggests, to install dependencies, not to set any environment variables influencing which tests should be run (though, arguably, this was already an issue with the original 'before_install' scriptlet). - Setting the variable has no effect anymore, because that script is run in a separate shell process, and the variable won't be visible in any of the other scripts, notably in 'ci/run-tests.sh' responsible for, well, running the tests. Luckily, this didn't have a negative effect on our Travis CI build jobs, because GIT_TEST_HTTPD is a tri-state variable defaulting to "auto" and a functioning web server was installed in those Linux build jobs, so the httpd tests were run anyway. Apparently the httpd tests run just fine without GIT_TEST_HTTPD being set, therefore we could simply remove this environment variable. However, if a bug were to creep in to change the Travis CI build environment to run the tests as root or to not install Apache, then the httpd tests would be skipped and the build job would still succeed. We would only notice if someone actually were to look through the build job's trace log; but who would look at the trace log of a successful build job?! Since httpd tests are important, we do want to run them and we want to be loudly reminded if they can't be run. Therefore, move setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs to 'ci/lib-travisci.sh' to ensure that the build job fails when the httpd tests can't be run. (We could set it in 'ci/run-tests.sh' just as well, but it's better to keep all environment variables in one place in 'ci/lib-travisci.sh'.) Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	e3371e9260	travis-ci: move setting environment variables to 'ci/lib-travisci.sh' Our '.travis.yml's 'env.global' section sets a bunch of environment variables for all build jobs, though none of them actually affects all build jobs. It's convenient for us, and in most cases it works just fine, because irrelevant environment variables are simply ignored. However, $GIT_SKIP_TESTS is an exception: it tells the test harness to skip the two test scripts that are prone to occasional failures on OSX, but as it's set for all build jobs those tests are not run in any of the build jobs that are capable to run them reliably, either. Therefore $GIT_SKIP_TESTS should only be set in the OSX build jobs, but those build jobs are included in the build matrix implicitly (i.e. by combining the matrix keys 'os' and 'compiler'), and there is no way to set an environment variable only for a subset of those implicit build jobs. (Unless we were to add new scriptlets to '.travis.yml', which is exactly the opposite direction that we took with commit `657343a60` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10)). So move setting $GIT_SKIP_TESTS to 'ci/lib-travisci.sh', where it can trivially be set only for the OSX build jobs. Furthermore, move setting all other environment variables from '.travis.yml' to 'ci/lib-travisci.sh', too, because a couple of environment variables are already set there, and this way all environment variables will be set in the same place. All the logic controlling our builds is already in the 'ci/*' scripts anyway, so there is really no good reason to keep the environment variables separately. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	bf427a9451	travis-ci: introduce a $jobname variable for 'ci/' scripts A couple of 'ci/' scripts are shared between different build jobs: 'ci/lib-travisci.sh', being a common library, is sourced from almost every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh' and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang Linux and OSX build jobs, and the latter two scripts are used in the GETTEXT_POISON Linux build job as well. Our builds could benefit from these shared scripts being able to easily tell which build job they are taking part in. Now, it's already quite easy to tell apart Linux vs OSX and GCC vs Clang build jobs, but it gets trickier with all the additional Linux-based build jobs included explicitly in the build matrix. Unfortunately, Travis CI doesn't provide much help in this regard. The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of which is two dot-separated integers, where the second integer indicates a particular build job. While it would be possible to use that second number to identify the build job in our shared scripts, it doesn't seem like a good idea to rely on that: - Though the build job numbering sequence seems to be stable so far, Travis CI's documentation doesn't explicitly states that it is indeed stable and will remain so in the future. And even if it were stable, - if we were to remove or insert a build job in the middle, then the job numbers of all subsequent build jobs would change accordingly. So roll our own means of simple build job identification and introduce the $jobname environment variable in our builds, setting it in the environments of the explicitly included jobs in '.travis.yml', while constructing one in 'ci/lib-travisci.sh' as the combination of the OS and compiler name for the GCC and Clang Linux and OSX build jobs. Use $jobname instead of $TRAVIS_OS_NAME in scripts taking different actions based on the OS and build job (when installing P4 and Git LFS dependencies and including them in $PATH). The following two patches will also rely on $jobname. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	83d1efe5d4	travis-ci: fix running P4 and Git LFS tests in Linux build jobs Linux build jobs on Travis CI skip the P4 and Git LFS tests since commit `657343a60` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10), claiming there are no P4 or Git LFS installed. The reason is that P4 and Git LFS binaries are not installed to a directory in the default $PATH, but their directories are prepended to $PATH. This worked just fine before said commit, because $PATH was set in a scriptlet embedded in our '.travis.yml', thus its new value was visible during the rest of the build job. However, after these embedded scriptlets were moved into dedicated scripts executed in separate shell processes, any variable set in one of those scripts is only visible in that single script but not in any of the others. In this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees Travis CI's default $PATH. Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all other 'ci/' scripts, so all those scripts will see the updated $PATH value. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Lars Schneider	8376eb4a8f	travis-ci: fix "skip_branch_tip_with_tag()" string comparison `09f5e97` ("travis-ci: skip a branch build if equal tag is present", 2017-09-17) introduced the "skip_branch_tip_with_tag" function with a broken string comparison. Fix it! Reported-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Lars Schneider	09f5e9746c	travis-ci: skip a branch build if equal tag is present If we push a branch and a tag pointing to the HEAD of this branch, then Travis CI would run the build twice. This wastes resources and slows the testing. Add a function to detect this situation and skip the build the branch if appropriate. Invoke this function on every build. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Lars Schneider	657343a602	travis-ci: move Travis CI code into dedicated scripts Most of the Travis CI commands are in the '.travis.yml'. The yml format does not support functions and therefore code duplication is necessary to run commands across all builds. To fix this, add a library for common CI functions. Move all Travis CI code into dedicated scripts and make them call the library first. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago

30 Commits (6f054f9fb3a501c35b55c65e547a244f14c38d56)