kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Ævar Arnfjörð Bjarmason	f15e00b463	ci: use https, not http to download binaries from perforce.com Since `522354d70f` (Add Travis CI support, 2015-11-27) the CI has used http://filehost.perforce.com/perforce/ to download binaries from filehost.perforce.com, they were then moved to this script in `657343a602` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10). Let's use https instead for good measure. I don't think we need to worry about the DNS or network between the GitHub CI and perforce.com being MitM'd, but using https gives us extra validation of the payload at least, and is one less thing to worry about when checking where else we rely on non-TLS'd http connections. Also, use the same download site at perforce.com for Linux and macOS tarballs for consistency. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Carlo Marcelo Arenas Belón	49af448197	ci: reintroduce prevention from perforce being quarantined in macOS `5ed9fc3fc8` (ci: prevent `perforce` from being quarantined, 2020-02-27) introduces this prevention for brew, but brew has been removed in a previous commit, so reintroduce an equivalent option to avoid a possible regression. This doesn't affect github actions (as configure now) and is therefore done silently to avoid any possible scary irrelevant messages. Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Carlo Marcelo Arenas Belón	d1c9195116	ci: avoid brew for installing perforce Perfoce's cask in brew is meant[1] to be used only by humans, so replace its use from the CI with a scripted binary download which is less likely to fail, as it is done in Linux. Kept the logic together so it will be less likely to break when moved around as on the fly code changes in this area are settled, at which point it will also feasable to ammend it to avoid some of the hardcoded values by using similar variables to the ones Linux does. In that same line, a POSIX sh syntax is used instead of the similar one used in Linux in preparation for an unrelated future change that might change the shell currently configured for it. This change reintroduces the risk that the installed binaries might not work because of being quarantined that was fixed with `5ed9fc3fc8` (ci: prevent `perforce` from being quarantined, 2020-02-27) but fixing that now was also punted for simplicity and since the affected cloud provider is scheduled to be retired with an on the fly change, but should be addressed if that other change is not integrated further. The discussion on the need to keep 2 radically different versions of the binaries to be tested with Linux vs macOS or how to upgrade to newer versions now that brew won't do that automatically for us has been punted for now as well. On that line the now obsolete comment about it in lib.sh was originally being updated by this change but created conflicts as it is moved around by other on the fly changes, so will be addressed independently as well. [1] https://github.com/Homebrew/homebrew-cask/pull/122347#discussion_r856026584 Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Carlo Marcelo Arenas Belón	cde6b9b78d	ci: make failure to find perforce more user friendly In preparation for a future change that will make perforce installation optional in macOS, make sure that the check for it is done without triggering scary looking errors and add a user friendly message instead. All other existing uses of 'type <cmd>' in our shell scripts that check the availability of a command <cmd> send both standard output and error stream to /dev/null to squelch "<cmd> not found" diagnostic output, but this script left the standard error stream shown. Redirect it just like everybody else to squelch this error message that we fully expect to see. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	707d2f2fe8	CI: use "$runs_on_pool", not "$jobname" to select packages & config Change the setup hooks for the CI to use "$runs_on_pool" for the "$regular" job. Now we won't need as much boilerplate when adding new jobs to the "regular" matrix, see `956d2e4639` (tests: add a test mode for SANITIZE=leak, run it in CI, 2021-09-23) for the last such commit. I.e. now instead of needing to enumerate each jobname when we select packages we can install things depending on the pool we're running in. That we didn't do this dates back to the now gone dependency on Travis CI, but even if we add a new CI target in the future this'll be easier to port over, since we can probably treat "ubuntu-latest" as a stand-in for some recent Linux that can run "apt" commands. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	4a6e4b9602	CI: remove Travis CI support Remove support for running the CI in travis. The last builds in it are from 5 months ago[1] (as of 2021-11-19), and our documentation has referred to GitHub CI instead since `f003a91f5c` (SubmittingPatches: replace discussion of Travis with GitHub Actions, 2021-07-22). We'll now run the "t9810 t9816" and tests on OSX. We didn't before, as we'd carried the Travis exclusion of them forward from `522354d70f` (Add Travis CI support, 2015-11-27). Let's hope whatever issue there was with them was either Travis specific, or fixed since then (I'm not sure). The "apt-add-repository" invocation (which we were doing in GitHub CI) isn't needed, it was another Travis-only case that was carried forward into more general code. See `0f0c51181d` (travis-ci: install packages in 'ci/install-dependencies.sh', 2018-11-01). Remove the "linux-gcc-4.8" job added in `fb9d7431cf` (travis-ci: build with GCC 4.8 as well, 2019-07-18), it only ran in Travis CI. 1. https://travis-ci.org/github/git/git/builds Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	956d2e4639	tests: add a test mode for SANITIZE=leak, run it in CI While git can be compiled with SANITIZE=leak, we have not run regression tests under that mode. Memory leaks have only been fixed as one-offs without structured regression testing. This change adds CI testing for it. We'll now build and small set of whitelisted t00.sh tests under Linux with a new job called "linux-leaks". The CI target uses a new GIT_TEST_PASSING_SANITIZE_LEAK=true test mode. When running in that mode, we'll assert that we were compiled with SANITIZE=leak. We'll then skip all tests, except those that we've opted-in by setting "TEST_PASSES_SANITIZE_LEAK=true". A test setting "TEST_PASSES_SANITIZE_LEAK=true" setting can in turn make use of the "SANITIZE_LEAK" prerequisite, should they wish to selectively skip tests even under "GIT_TEST_PASSING_SANITIZE_LEAK=true". In the preceding commit we started doing this in "t0004-unwritable.sh" under SANITIZE=leak, now it'll combine nicely with "GIT_TEST_PASSING_SANITIZE_LEAK=true". This is how tests that don't set "TEST_PASSES_SANITIZE_LEAK=true" will be skipped under GIT_TEST_PASSING_SANITIZE_LEAK=true: $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0001-init.sh 1..0 # SKIP skip all tests in t0001 under SANITIZE=leak, TEST_PASSES_SANITIZE_LEAK not set The intent is to add more TEST_PASSES_SANITIZE_LEAK=true annotations as follow-up change, but let's start small to begin with. In ci/run-build-and-tests.sh we make use of the default "" case to run "make test" without any GIT_TEST_* modes. SANITIZE=leak is known to fail in combination with GIT_TEST_SPLIT_INDEX=true in t0016-oidmap.sh, and we're likely to have other such failures in various GIT_TEST_* modes. Let's focus on getting the base tests passing, we can expand coverage to GIT_TEST_* modes later. It would also be possible to implement a more lightweight version of this by only relying on setting "LSAN_OPTIONS". See <YS9OT/pn5rRK9cGB@coredump.intra.peff.net>[1] and <YS9ZIDpANfsh7N+S@coredump.intra.peff.net>[2] for a discussion of that. I've opted for this approach of adding a GIT_TEST_* mode instead because it's consistent with how we handle other special test modes. Being able to add a "!SANITIZE_LEAK" prerequisite and calling "test_done" early if it isn't satisfied also means that we can more incrementally add regression tests without being forced to fix widespread and hard-to-fix leaks at the same time. We have tests that do simple checking of some tool we're interested in, but later on in the script might be stressing trace2, or common sources of leaks like "git log" in combination with the tool (e.g. the commit-graph tests). To be clear having a prerequisite could also be accomplished by using "LSAN_OPTIONS" directly. On the topic of "LSAN_OPTIONS": It would be nice to have a mode to aggregate all failures in our various scripts, see [2] for a start at doing that which sets "log_path" in "LSAN_OPTIONS". I've punted on that for now, it can be added later. As of writing this we've got major regressions between master..seen, i.e. the t000*.sh tests and more fixed since `31f9acf9ce` (Merge branch 'ah/plugleaks', 2021-08-04) have regressed recently. See the discussion at <87czsv2idy.fsf@evledraar.gmail.com>[3] about the lack of this sort of test mode, and `0e5bba53af` (add UNLEAK annotation for reducing leak false positives, 2017-09-08) for the initial addition of SANITIZE=leak. See also `09595ab381` (Merge branch 'jk/leak-checkers', 2017-09-19), `7782066f67` (Merge branch 'jk/apache-lsan', 2019-05-19) and the recent `936e58851a` (Merge branch 'ah/plugleaks', 2021-05-07) for some of the past history of "one-off" SANITIZE=leak (and more) fixes. As noted in [5] we can't support this on OSX yet until Clang 14 is released, at that point we'll probably want to resurrect that "osx-leaks" job. 1. https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer 2. https://lore.kernel.org/git/YS9OT%2Fpn5rRK9cGB@coredump.intra.peff.net/ 3. https://lore.kernel.org/git/87czsv2idy.fsf@evledraar.gmail.com/ 4. https://lore.kernel.org/git/YS9ZIDpANfsh7N+S@coredump.intra.peff.net/ 5. https://lore.kernel.org/git/20210916035603.76369-1-carenas@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jeff King	27f45ccf33	ci/install-dependencies: handle "sparse" job package installs This just matches the style/location of the package installation for other jobs. There should be no functional change. I did flip the order of the options and command-name ("-y update" instead of "update -y") for consistency with other lines in the same file. Note also that we have to reorder the dependency install with the "checkout" action, so that we actually have the "ci" scripts available. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	6c280b4142	ci: remove GETTEXT_POISON jobs A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option. We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job in the sense that it's the job that was otherwise run with the default compiler, no other GIT_TEST_* options etc. So let's keep it under the name "linux-gcc-default". This means we can remove the initial "make test" from the "linux-gcc" job (it does another one after setting a bunch of GIT_TEST_* variables). I'm not doing that because it would conflict with the in-flight `334afbc76f` (tests: mark tests relying on the current default for `init.defaultBranch`, 2020-11-18) (currently on the "seen" branch, so the SHA-1 will almost definitely change). It's going to use that "make test" again for different reasons, so let's preserve it for now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Junio C Hamano	3831132ace	ci/install-depends: attempt to fix "brew cask" stuff We run "git pull" against "$cask_repo"; clarify that we are expecting not to have any of our own modifications and running "git pull" to merely update, by passing "--ff-only" on the command line. Also, the "brew cask install" command line triggers an error message that says: Error: Calling brew cask install is disabled! Use brew install [--cask] instead. In addition, "brew install caskroom/cask/perforce" step triggers an error that says: Error: caskroom/cask was moved. Tap homebrew/cask instead. Attempt to see if blindly following the suggestion in these error messages gets us into a better shape. Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Johannes Schindelin	4fef6321a5	ci: run gem with sudo to install asciidoctor In a later patch, we will run Documentation job in GitHub Actions. The job will run without elevated permission. Run `gem` with `sudo` to elevate permission in order to be able to install to system location. This will also keep this installation in-line with other installation in our Linux system for CI. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> [Danh: reword commit message] Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Đoàn Trần Công Danh	61432dd630	ci: explicit install all required packages In a later patch, we will support GitHub Action. Explicitly install all of our build dependencies on Linux. Since GitHub Action's Linux VM hasn't installed our build dependencies. And there're no harm to reinstall them (in Travis) Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	5ed9fc3fc8	ci: prevent `perforce` from being quarantined The most recent Azure Pipelines macOS agents enable what Apple calls "System Integrity Protection". This makes `p4d -V` hang: there is some sort of GUI dialog waiting for the user to acknowledge that the copied binaries are legit and may be executed, but on build agents, there is no user who could acknowledge that. Let's ask Homebrew specifically to _not_ quarantine the Perforce binaries. Helped-by: Aleksandr Chebotov Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	176441bfb5	ci: build Git with GCC 9 in the 'osx-gcc' build job Our 'osx-gcc' build job on Travis CI relied on GCC 8 being installed (but not linked) in the image we use [1]. Alas, since the last update of this image a few days ago this is not the case anymore, and now it contains GCC 9 (installed and linked) instead of GCC 8. The results are failed 'osx-gcc' jobs, because they can't find the 'gcc-8' command [2]. Let's move on to use GCC 9, with hopefully better error reporting and improved -Wfoo flags and what not. On Travis CI this has the benefit that we can spare a few seconds while installing dependencies, because it already comes pre-installed, at least for now. The Azure Pipelines OSX image doesn't include GCC, so we have to install it ourselves anyway, and then we might as well install the newer version. In a vain attempt I tried to future-proof this a bit: - Install 'gcc@9' specifically, so we'll still get what we want even after GCC 10 comes out, and the "plain" 'gcc' package starts to refer to 'gcc@10'. - Run both 'brew install gcc@9' and 'brew link gcc@9'. If 'gcc@9' is already installed and linked, then both commands are noop and exit with success. But as we saw in the past, sometimes the image contains the expected GCC package installed but not linked, so maybe it will happen again in the future as well. In that case 'brew install' is still a noop, and instructs the user to run 'brew link' instead, so that's what we'll do. And if 'gcc@9' is not installed, then 'brew install' will install it, and the subsequent 'brew link' becomes a noop. An additional benefit of this patch is that from now on we won't unnecessarily install GCC and its dependencies in the 'osx-clang' jobs on Azure Pipelines. [1] `7d4733c501` (ci: fix GCC install in the Travis CI GCC OSX job, 2019-10-24) [2] https://travis-ci.org/git/git/jobs/615442297#L333 Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Junio C Hamano	0dbc4a0edf	ci(osx): update homebrew-cask repository with less noise The OSX CI build procedure updates the homebrew-cask repository before attempting to install perforce again, after seeing an installation failure. This involves a "git pull" that by default computes and outputs diffstat, which would only grow as the time goes by and the repository cast in stone in the CI build image becomes more and more stale relative to the upstream repository in the outside world. Suppress the diffstat to both save cycles to generate it, and strain on the eyeballs to skip it. Reported-by: SZEDER Gábor <szeder.dev@gmail.com> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	7d4733c501	ci: fix GCC install in the Travis CI GCC OSX job A few days ago Travis CI updated their existing OSX images, including the Homebrew database in the xcode10.1 OSX image that we use. Since then installing dependencies in the 'osx-gcc' job fails when it tries to link gcc@8: + brew link gcc@8 Error: No such keg: /usr/local/Cellar/gcc@8 GCC8 is still installed but not linked to '/usr/local' in the updated image, as it was before this update, but now we have to link it by running 'brew link gcc'. So let's do that then, and fall back to linking gcc@8 if it doesn't, just to be sure. Our builds on Azure Pipelines are unaffected by this issue. The OSX image over there doesn't contain the gcc@8 package, so we have to 'brew install' it, which already takes care of linking it to '/usr/local'. After that the 'brew link gcc' command added by this patch fails, but the \|\|-chained fallback 'brew link gcc@8' command succeeds with an "already linked" warning. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	0eb3671ed9	ci(osx): use new location of the `perforce` cask The Azure Pipelines builds are failing for macOS due to a change in the location of the perforce cask. The command outputs the following error: + brew install caskroom/cask/perforce Error: caskroom/cask was moved. Tap homebrew/cask-cask instead. So let's try to call `brew cask install perforce` first (which is what that error message suggests, in a most round-about way). Prior to `672f51cb` we used to install the 'perforce' package with 'brew install perforce' (note: no 'cask' in there). The justification for `672f51cb` was that the command 'brew install perforce' simply stopped working, after Homebrew folks decided that it's better to move the 'perforce' package to a "cask". Their justification for this move was that 'brew install perforce' "can fail due to a checksum mismatch ...", and casks can be installed without checksum verification. And indeed, both 'brew cask install perforce' and 'brew install caskroom/cask/perforce' printed something along the lines of: ==> No checksum defined for Cask perforce, skipping verification It is unclear why `672f51cb` used 'brew install caskroom/cask/perforce' instead of 'brew cask install perforce'. It appears (by running both commands on old Travis CI macOS images) that both commands worked all the same already back then. In any case, as the error message at the top of this commit message shows, 'brew install caskroom/cask/perforce' has stopped working recently, but 'brew cask install perforce' still does, so let's use that. CI servers are typically fresh virtual machines, but not always. To accommodate for that, let's try harder if `brew cask install perforce` fails, by specifically pulling the latest `master` of the `homebrew-cask` repository. This will still fail, of course, when `homebrew-cask` falls behind Perforce's release schedule. But once it is updated, we can now simply re-run the failed jobs and they will pick up that update. As for updating `homebrew-cask`: the beginnings of automating this in https://dev.azure.com/gitgitgadget/git/_build?definitionId=11&_a=summary will be finished once the next Perforce upgrade comes around. Helped-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	411e4f4735	ci: run `hdr-check` as part of the `Static Analysis` job Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
brian m. carlson	f6461b82b9	Documentation: fix build with Asciidoctor 2 Our documentation toolchain has traditionally been built around DocBook 4.5. This version of DocBook is the last DTD-based version of DocBook. In 2009, DocBook 5 was introduced using namespaces and its syntax is expressed in RELAX NG, which is more expressive and allows a wider variety of syntax forms. Asciidoctor, one of the alternatives for building our documentation, moved support for DocBook 4.5 out of core in its recent 2.0 release and now only supports DocBook 5 in the main release. The DocBoook 4.5 converter is still available as a separate component, but this is not available in most distro packages. This would not be a problem but for the fact that we use xmlto, which is still stuck in the DocBook 4.5 era. xmlto performs DTD validation as part of the build process. This is not problematic for DocBook 4.5, which has a valid DTD, but it clearly cannot work for DocBook 5, since no DTD can adequately express its full syntax. In addition, even if xmlto did support RELAX NG validation, that wouldn't be sufficient because it uses the libxml2-based xmllint to do so, which has known problems with validating interleaves in RELAX NG. Fortunately, there's an easy way forward: ask Asciidoctor to use its DocBook 5 backend and tell xmlto to skip validation. Asciidoctor has supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported skipping validation for probably longer than that. We also need to teach xmlto how to use the namespaced DocBook XSLT stylesheets instead of the non-namespaced ones it usually uses. Normally these stylesheets are interchangeable, but the non-namespaced ones have a bug that causes them not to strip whitespace automatically from certain elements when namespaces are in use. This results in additional whitespace at the beginning of list elements, which is jarring and unsightly. We can do this by passing a custom stylesheet with the -x option that simply imports the namespaced stylesheets via a URL. Any system with support for XML catalogs will automatically look this URL up and reference a local copy instead without us having to know where this local copy is located. We know that anyone using xmlto will already have catalogs set up properly since the DocBook 4.5 DTD used during validation is also looked up via catalogs. All major Linux distributions distribute the necessary stylesheets and have built-in catalog support, and Homebrew does as well, albeit with a requirement to set an environment variable to enable catalog support. On the off chance that someone lacks support for catalogs, it is possible for xmlto (via xmllint) to download the stylesheets from the URLs in question, although this will likely perform poorly enough to attract attention. People still have the option of using the prebuilt documentation that we ship, so happily this should not be an impediment. Finally, we need to filter out some messages from other stylesheets that occur when invoking dblatex in the CI job. This tool strips namespaces much like the unnamespaced DocBook stylesheets and prints similar messages. If we permit these messages to be printed to standard error, our documentation CI job will fail because we check standard error for unexpected output. Due to dblatex's reliance on Python 2, we may need to revisit its use in the future, in which case this problem may go away, but this can be delayed until a future patch. The final message we filter is due to libxslt on modern Debian and Ubuntu. The patch which they use to implement reproducible ID generation also prints messages about the ID generation. While this doesn't affect our current CI images since they use Ubuntu 16.04 which lacks this patch, if we upgrade to Ubuntu 18.04 or a modern Debian, these messages will appear and, like the above messages, cause a CI failure. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	af8ed04778	ci: disable Homebrew's auto cleanup Lately Homebrew learned to automagically clean up information about outdated packages during other 'brew' commands, which might be useful for the avarage user, but is a waste of time in CI build jobs, because the next build jobs will start from the exact same image containing the same outdated packages anyway. Export HOMEBREW_NO_INSTALL_CLEANUP=1 to disable this auto cleanup feature, shaving off about 20-30s from the time needed to install dependencies in our macOS build jobs on Travis CI. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	f2f4715033	ci: don't update Homebrew Lately our GCC macOS build job on Travis CI has been erroring out while installing dependencies with: +brew link gcc@8 Error: No such keg: /usr/local/Cellar/gcc@8 The command "ci/install-dependencies.sh" failed and exited with 1 during . Now, while gcc@8 is still pre-installed (but not linked) and would be perfectly usable in the Travis CI macOS image we use [1], it's at version 8.2. However, when installing dependencies we first explicitly run 'brew update', which spends over two minutes to update itself and information about the available packages, and it learns about GCC 8.3. After that point gcc@8 exclusively refers to v8.3, and, unfortunately, 'brew' is just too dumb to be able to do anything with the still installed 8.2 package, and the subsequent 'brew link gcc@8' fails. (Even 'brew uninstall gcc@8' fails with the same error!) Don't run 'brew update' to keep the already installed GCC 8.2 'brew link'-able. Note that in addition we have to 'export HOMEBREW_NO_AUTO_UPDATE=1' first, because 'brew' is so very helpful that it would implicitly run update for us on the next 'brew install <pkg>' otherwise. Disabling 'brew update' has additional benefits: - It shaves off 2-3mins from the ~4mins currently spent on installing dependencies, and the macOS build jobs have always been prone to exceeding the time limit on Travis CI. - Our builds won't suddenly break because of the occasional Homebrew breakages [2]. The drawback is that we'll be stuck with slightly older versions of the packages that we install via Homebrew (Git-LFS 2.5.2 and Perforce 2018.1; they are currently at 2.7.2 and 2019.1, respectively). We might want to reconsider this decision as time goes on and/or switch to a more recent macOS image as they become available. [1] `2000ac9fbf` (travis-ci: switch to Xcode 10.1 macOS image, 2019-01-17) [2] See e.g. `a1ccaedd62` (travis-ci: make the OSX build jobs' 'brew update' more quiet, 2019-02-02) or https://public-inbox.org/git/20180907032002.23366-1-szeder.dev@gmail.com/T/#+u Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	db864306cf	ci: install 'libsvn-perl' instead of 'git-svn' Since `e7e9f5e7a1` (travis-ci: enable Git SVN tests t91xx on Linux, 2016-05-19) some of our Travis CI build jobs install the 'git-svn' package, because it was a convenient way to install its dependencies, which are necessary to run our 'git-svn' tests (we don't actually need the 'git-svn' package itself). However, from those dependencies, namely the 'libsvn-perl', 'libyaml-perl', and 'libterm-readkey-perl' packages, only 'libsvn-perl' is necessary to run those tests, the others arent, not even to fulfill some prereqs. So update 'ci/install-dependencies.sh' to install only 'libsvn-perl' instead of 'git-svn' and its additional dependencies. Note that this change has more important implications than merely not installing three unnecessary packages, as it keeps our builds working with Travis CI's Xenial images. In our '.travis.yml' we never explicitly specified which Linux image we want to use to run our Linux build jobs, and so far they have been run on the default Ubuntu 14.04 Trusty image. However, 14.04 just reached its EOL, and Travis CI has already began the transition to use 16.04 Xenial as the default Linux build environment [1]. Alas, our Linux Clang and GCC build jobs can't simply 'apt-get install git-svn' in the current Xenial images [2], like they did in the Trusty images, and, consequently, fail. Installing only 'libsvn-perl' avoids this issue, while the 'git svn' tests are still run as they should. [1] https://blog.travis-ci.com/2019-04-15-xenial-default-build-environment [2] 'apt-get install git-svn' in the Xenial image fails with: The following packages have unmet dependencies: git-svn : Depends: git (< 1:2.7.4-.) E: Unable to correct problems, you have held broken packages. The reason is that both the Trusty and Xenial images contain the 'git' package installed from 'ppa:git-core/ppa', so it's considerably newer than the 'git' package in the corresponding standard Ubuntu package repositories. The difference is that the Trusty image still contains these third-party apt repositories, so the 'git-svn' package was installed from the same PPA, and its version matched the version of the already installed 'git' package. In the Xenial image, however, these third-party apt-repositories are removed (to reduce the risk of unrelated interference and faster 'apt-get update') [3], and the version of the 'git-svn' package coming from the standard Ubuntu package repositories doesn't match the much more recent version of the 'git' package installed from the PPA, resulting in this dependecy error. Adding back the 'ppa:git-core/ppa' package repository would solve this dependency issue as well, but since the troublesome package happens to be unnecessary, not installing it in the first place is better. [3] https://docs.travis-ci.com/user/reference/xenial/#third-party-apt-repositories-removed Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	615a6c37e1	ci: stick with Asciidoctor v1.5.8 for now The recent release of Asciidoctor v2.0.0 broke our documentation build job on Travis CI, where we 'gem install asciidoctor', which always brings us the latest and (supposedly) greatest. Alas, we are not ready for that just yet, because it removed support for DocBook 4.5, and we have been requiring that particular DocBook version to build 'user-manual.xml' with Asciidoctor, resulting in: ASCIIDOC user-manual.xml asciidoctor: FAILED: missing converter for backend 'docbook45'. Processing aborted. Use --trace for backtrace make[1]: *** [user-manual.xml] Error 1 Unfortunately, we can't simply switch to DocBook 5 right away, as doing so leads to validation errors from 'xmlto', and working around those leads to yet another errors... [1] So let's stick with Asciidoctor v1.5.8 (latest stable release before v2.0.0) in our documentation build job on Travis CI for now, until we figure out how to deal with the fallout from Asciidoctor v2.0.0. [1] https://public-inbox.org/git/20190324162131.GL4047@pobox.com/ Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	f34a1bd96c	ci: install Asciidoctor in 'ci/install-dependencies.sh' When our '.travis.yml' was split into several 'ci/*' scripts [1], the installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'. Install it in 'ci/install-dependencies.sh', where we install other dependencies of the Documentation build job as well (asciidoc, xmlto). [1] `657343a602` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10) Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	a1ccaedd62	travis-ci: make the OSX build jobs' 'brew update' more quiet Before installing the necessary dependencies, our OSX build jobs run 'brew update --quiet'. This is problematic for two reasons: - This '--quiet' flag apparently broke overnight, resulting in errored builds: +brew update --quiet ==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz ######################################################################## 100.0% ==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz Usage: brew update_report [--preinstall] The Ruby implementation of brew update. Never called manually. --preinstall Run in 'auto-update' mode (faster, less output). -f, --force Override warnings and enable potentially unsafe operations. -d, --debug Display any debugging information. -v, --verbose Make some output more verbose. -h, --help Show this message. Error: invalid option: --quiet The command "ci/install-dependencies.sh" failed and exited with 1 during . I belive that this breakage will be noticed and fixed soon-ish, so we could probably just wait a bit for this issue to solve itself, but: - 'brew update --quiet' wasn't really quiet in the first place, as it listed over about 2000 lines worth of available packages that we absolutely don't care about, see e.g. one of the latest 'master' builds: https://travis-ci.org/git/git/jobs/486134962#L113 So drop this '--quiet' option and redirect 'brew update's standard output to /dev/null to make it really quiet, thereby making the OSX builds work again despite the above mentioned breakage. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	b011fabd6e	ci/lib.sh: encapsulate Travis-specific things The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI. Also, the configurations of the available agents are different. For example, Travis' and Azure Pipelines' macOS agents are set up differently, so that on Travis, we have to install the git-lfs and gettext Homebrew packages, and on Azure Pipelines we do not need to. Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor installed. Finally, on Azure Pipelines the natural way is not to base64-encode tar files of the trash directories of failed tests, but to publish build artifacts instead. Therefore, that code to log those base64-encoded tar files is guarded to be Travis-specific. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	c2160f2d19	ci: rename the library of common functions The name is hard-coded to reflect that we use Travis CI for continuous testing. In the next commits, we will extend this to be able use Azure DevOps, too. So let's adjust the name to make it more generic. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	2c8921db2b	travis-ci: build with the right compiler Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This CC variable can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to be on the platform. Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang both on Linux and macOS from the very beginning (`522354d70f` (Add Travis CI support, 2015-11-27)). Alas, this never really worked as supposed to, because Travis CI specifies the compiler for those build jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for projects built with './configure && make'). Consequently, our 'linux-clang' build job has always used GCC, because that's where 'cc' points at in Travis CI's Linux images, while the 'osx-gcc' build job has always used Clang. Furthermore, `37fa4b3c78` (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an attempt to build with a more modern compiler, but to no avail. Set MAKEFLAGS with CC based on the $CC environment variable, so 'make' will run the "right" compiler. The Xcode 10.1 macOS image on Travis CI already contains the gcc@8 package from Homebrew, but we have to 'brew link' it first to be able to use it. So with this patch our build jobs will build Git with the following compiler versions: linux-clang: clang version 5.0.0 (tags/RELEASE_500/final) linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0 osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5) osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0 GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4 Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	0f0c51181d	travis-ci: install packages in 'ci/install-dependencies.sh' Ever since we started using Travis CI, we specified the list of packages to install in '.travis.yml' via the APT addon. While running our builds on Travis CI's container-based infrastructure we didn't have another choice, because that environment didn't support 'sudo', and thus we didn't have permission to install packages ourselves. With the switch to the VM-based infrastructure in the previous patch we do get a working 'sudo', so we can install packages by running 'sudo apt-get -y install ...' as well. Let's make use of this and install necessary packages in 'ci/install-dependencies.sh', so all the dependencies (i.e. both packages and "non-packages" (P4 and Git-LFS)) are handled in the same file. Install gcc-8 only in the 'linux-gcc' build job; so far it has been unnecessarily installed in the 'linux-clang' build job as well. Print the versions of P4 and Git-LFS conditionally, i.e. only when they have been installed; with this change even the static analysis and documentation build jobs start using 'ci/install-dependencies.sh' to install packages, and neither of these two build jobs depend on and thus install those. This change will presumably be beneficial for the upcoming Azure Pipelines integration [1]: preliminary versions of that patch series run a couple of 'apt-get' commands to install the necessary packages before running 'ci/install-dependencies.sh', but with this patch it will be sufficient to run only 'ci/install-dependencies.sh'. [1] https://public-inbox.org/git/1a22efe849d6da79f2c639c62a1483361a130238.1539598316.git.gitgitgadget@gmail.com/ Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	a1157b76eb	travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh' Commit `657343a60` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10) converted '.travis.yml's default 'before_install' scriptlet to the 'ci/install-dependencies.sh' script, and while doing so moved setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs to that script. This is wrong for two reasons: - The purpose of that script is, as its name suggests, to install dependencies, not to set any environment variables influencing which tests should be run (though, arguably, this was already an issue with the original 'before_install' scriptlet). - Setting the variable has no effect anymore, because that script is run in a separate shell process, and the variable won't be visible in any of the other scripts, notably in 'ci/run-tests.sh' responsible for, well, running the tests. Luckily, this didn't have a negative effect on our Travis CI build jobs, because GIT_TEST_HTTPD is a tri-state variable defaulting to "auto" and a functioning web server was installed in those Linux build jobs, so the httpd tests were run anyway. Apparently the httpd tests run just fine without GIT_TEST_HTTPD being set, therefore we could simply remove this environment variable. However, if a bug were to creep in to change the Travis CI build environment to run the tests as root or to not install Apache, then the httpd tests would be skipped and the build job would still succeed. We would only notice if someone actually were to look through the build job's trace log; but who would look at the trace log of a successful build job?! Since httpd tests are important, we do want to run them and we want to be loudly reminded if they can't be run. Therefore, move setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs to 'ci/lib-travisci.sh' to ensure that the build job fails when the httpd tests can't be run. (We could set it in 'ci/run-tests.sh' just as well, but it's better to keep all environment variables in one place in 'ci/lib-travisci.sh'.) Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	bf427a9451	travis-ci: introduce a $jobname variable for 'ci/' scripts A couple of 'ci/' scripts are shared between different build jobs: 'ci/lib-travisci.sh', being a common library, is sourced from almost every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh' and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang Linux and OSX build jobs, and the latter two scripts are used in the GETTEXT_POISON Linux build job as well. Our builds could benefit from these shared scripts being able to easily tell which build job they are taking part in. Now, it's already quite easy to tell apart Linux vs OSX and GCC vs Clang build jobs, but it gets trickier with all the additional Linux-based build jobs included explicitly in the build matrix. Unfortunately, Travis CI doesn't provide much help in this regard. The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of which is two dot-separated integers, where the second integer indicates a particular build job. While it would be possible to use that second number to identify the build job in our shared scripts, it doesn't seem like a good idea to rely on that: - Though the build job numbering sequence seems to be stable so far, Travis CI's documentation doesn't explicitly states that it is indeed stable and will remain so in the future. And even if it were stable, - if we were to remove or insert a build job in the middle, then the job numbers of all subsequent build jobs would change accordingly. So roll our own means of simple build job identification and introduce the $jobname environment variable in our builds, setting it in the environments of the explicitly included jobs in '.travis.yml', while constructing one in 'ci/lib-travisci.sh' as the combination of the OS and compiler name for the GCC and Clang Linux and OSX build jobs. Use $jobname instead of $TRAVIS_OS_NAME in scripts taking different actions based on the OS and build job (when installing P4 and Git LFS dependencies and including them in $PATH). The following two patches will also rely on $jobname. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	83d1efe5d4	travis-ci: fix running P4 and Git LFS tests in Linux build jobs Linux build jobs on Travis CI skip the P4 and Git LFS tests since commit `657343a60` (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10), claiming there are no P4 or Git LFS installed. The reason is that P4 and Git LFS binaries are not installed to a directory in the default $PATH, but their directories are prepended to $PATH. This worked just fine before said commit, because $PATH was set in a scriptlet embedded in our '.travis.yml', thus its new value was visible during the rest of the build job. However, after these embedded scriptlets were moved into dedicated scripts executed in separate shell processes, any variable set in one of those scripts is only visible in that single script but not in any of the others. In this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees Travis CI's default $PATH. Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all other 'ci/' scripts, so all those scripts will see the updated $PATH value. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Junio C Hamano	f67242c10d	travis: dedent a few scripts that are indented overly deeply Signed-off-by: Junio C Hamano <gitster@pobox.com>	8 years ago
Lars Schneider	657343a602	travis-ci: move Travis CI code into dedicated scripts Most of the Travis CI commands are in the '.travis.yml'. The yml format does not support functions and therefore code duplication is necessary to run commands across all builds. To fix this, add a library for common CI functions. Move all Travis CI code into dedicated scripts and make them call the library first. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	8 years ago

42 Commits (fdbfac60fd889d4e55244958ce7febd61cb53f9d)