kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Denton Liu	788db145c7	t/README: avoid poor-man's small caps GIT In `48a8c26c62` (Documentation: avoid poor-man's small caps GIT, 2013-01-21), the documentation was amended to spell Git's name as Git when talking about the system as a whole. However, t/README was skipped over when the treatment was applied. Bring t/README into conformance with the CodingGuidelines by casing "Git" properly. While we're at it, fix a small typo. Change "the git internal" to "the Git internals". Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Garima Singh	d5b873c832	commit-graph: add GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS test flag Add GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS test flag to the test setup suite in order to toggle writing Bloom filters when running any of the git tests. If set to true, we will compute and write Bloom filters every time a test calls `git commit-graph write`, as if the `--changed-paths` option was passed in. The test suite passes when GIT_TEST_COMMIT_GRAPH and GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS are enabled. Helped-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Garima Singh <garima.singh@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Junio C Hamano	7cc112dc95	t/README: suggest how to leave test early with failure Over time, we added the support to our test framework to make it easy to leave a test early with failure, but it was not clearly documented in t/README to help developers writing new tests. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Matheus Tavares	78dc08875c	test-lib: allow short options to be bundled When debugging a test (or a set of tests), it's common to execute it with some combination of short options, such as: $ ./txxx-testname.sh -d -x -i In cases like this, CLIs usually allow the short options to be bundled in a single argument, for convenience and agility. Let's add this feature to test-lib, allowing the above command to be run as: $ ./txxx-testname.sh -dxi (or any other permutation, e.g. '-ixd') Note: Short options that require an argument can also be used in a bundle, in any position. So, for example, '-r 5 -x', '-xr 5' and '-rx 5' are all valid and equivalent. A special case would be having a bundle with more than one of such options. To keep things simple, this case is not allowed for now. This shouldn't be a major limitation, though, as the only short option that requires an argument today is '-r'. And concatenating '-r's as in '-rr 5 6' would probably not be very practical: its unbundled format would be '-r 5 -r 6', for which test-lib currently considers only the last argument. Therefore, if '-rr 5 6' were to be allowed, it would have the same effect as just typing '-r 6'. Note: the test-lib currently doesn't support '-r5', as an alternative for '-r 5', so the former is not supported in bundles as well. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Derrick Stolee	2d657ab95f	pack-objects: flip the use of GIT_TEST_PACK_SPARSE The environment variable GIT_TEST_PACK_SPARSE was previously used to allow testing the --sparse option for "git pack-objects" in the test suite. This allowed interesting cases of "git push" to also test this algorithm. Since pack.useSparse is now true by default, we do not need this variable to _enable_ the --sparse option, but instead to _disable_ it. This flips how we work with the variable a bit. When checking for the variable, default to a value of -1 for "unset". If unset, then take the default from the repo settings, which is currently 1. Then, the --[no-]sparse command-line option will override either of these settings. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Thomas Gummerer	8a2cd3f512	stash: remove the stash.useBuiltin setting Remove the stash.useBuiltin setting which was added as an escape hatch to disable the builtin version of stash first released with Git 2.22. Carrying the legacy version is a maintenance burden, and has in fact become out of date failing a test since the 2.23 release, without anyone noticing until now. So users would be getting a hint to fall back to a potentially buggy version of the tool. We used to shell out to git config to get the useBuiltin configuration to avoid changing any global state before spawning legacy-stash. However that is no longer necessary, so just use the 'git_config' function to get the setting instead. Similar to what we've done in `d03ebd411c` ("rebase: remove the rebase.useBuiltin setting", 2019-03-18), where we remove the corresponding setting for rebase, we leave the documentation in place, so people can refer back to it when searching for it online, and so we can refer to it in the commit message. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jonathan Nieder	33166f3a1f	protocol test: let protocol.version override GIT_TEST_PROTOCOL_VERSION The GIT_TEST_PROTOCOL_VERSION environment variable can be used to upgrade the version of Git protocol used in tests. If both GIT_TEST_PROTOCOL_VERSION and 'protocol.version' are set, the higher value wins. For usage within tests, these semantics are too complex. Instead, always use the value from protocol.version configuration when it is set, falling back to GIT_TEST_PROTOCOL_VERSION. This way, the envvar provides a reliable preview of what will happen if the default protocol version is changed. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	43a2afee82	tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values Since `3b072c577b` (tests: replace test_tristate with "git env--helper", 2019-06-21) we get the normalized bool values of various GIT_TEST_* environment variables via 'git env--helper'. Now, while the 'git env--helper' command itself does catch invalid values in the environment variable or in the given --default and exits with error (exit code 128 or 129, respectively), it's invoked in conditions like 'if ! git env--helper ...', which means that all invalid bool values are interpreted the same as the ordinary 'false' (exit code 1). This has led to inadvertently skipped httpd tests in our CI builds for a couple of weeks, see `3960290675` (ci: restore running httpd tests, 2019-09-06). Let's be more careful about what the test suite accepts as bool values in GIT_TEST_* environment variables, and error out loud and clear on invalid values instead of simply skipping tests. Add the 'test_bool_env' helper function to encapsulate the invocation of 'git env--helper' and the verification of its exit code, and replace all invocations of that command in our test framework and test suite with a call to this new helper (except in 't0017-env-helper.sh', of course). $ GIT_TEST_GIT_DAEMON=YesPlease ./t5570-git-daemon.sh fatal: bad numeric config value 'YesPlease' for 'GIT_TEST_GIT_DAEMON': invalid unit error: test_bool_env requires bool values both for $GIT_TEST_GIT_DAEMON and for the default fallback Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	f83dff60a7	Start to implement a built-in version of `git add --interactive` Unlike previous conversions to C, where we started with a built-in helper, we start this conversion by adding an interception in the `run_add_interactive()` function when the new opt-in `add.interactive.useBuiltin` config knob is turned on (or the corresponding environment variable `GIT_TEST_ADD_I_USE_BUILTIN`), and calling the new internal API function `run_add_i()` that is implemented directly in libgit.a. At this point, the built-in version of `git add -i` only states that it cannot do anything yet. In subsequent patches/patch series, the `run_add_i()` function will gain more and more functionality, until it is feature complete. The whole arc of the conversion can be found in the PRs #170-175 at https://github.com/gitgitgadget/git. The "--helper approach" can unfortunately not be used here: on Windows we face the very specific problem that a `system()` call in Perl seems to close `stdin` in the parent process when the spawned process consumes even one character from `stdin`. Which prevents us from implementing the main loop in C and still trying to hand off to the Perl script. The very real downside of the approach we have to take here is that the test suite won't pass with `GIT_TEST_ADD_I_USE_BUILTIN=true` until the conversion is complete (the `--helper` approach would have let it pass, even at each of the incremental conversion steps). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Ævar Arnfjörð Bjarmason	c740039921	tests: make GIT_TEST_FAIL_PREREQS a boolean Change the GIT_TEST_FAIL_PREREQS variable from being "non-empty?" to being a more standard boolean variable. I recently added the variable in `dfe1a17df9` ("tests: add a special setup where prerequisites fail", 2019-05-13), having to add another "non-empty?" special-case is what prompted me to write the "git env--helper" utility being used here. Converting this one is a bit tricky since we use it so early and frequently in the guts of the test code itself, so let's set a GIT_TEST_FAIL_PREREQS_INTERNAL which can be tested with the old "test -n" for the purposes of the shell code, and change the user-exposed and documented GIT_TEST_FAIL_PREREQS variable to a boolean. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	423b05e102	tests README: re-flow a previously changed paragraph A previous change to the "GIT_TEST_GETTEXT_POISON" variable left this paragraph needing to be re-flowed. Let's do that in this separate change to make it easy to see that there's no change here when viewed with "--word-diff". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	1ff750b128	tests: make GIT_TEST_GETTEXT_POISON a boolean Change the GIT_TEST_GETTEXT_POISON variable from being "non-empty?" to being a more standard boolean variable. Since it needed to be checked in both C code and shellscript (via test -n) it was one of the remaining shellscript-like variables. Now that we have "env--helper" we can change that. There's a couple of tricky edge cases that arise because we're using git_env_bool() early, and the config-reading "env--helper". If GIT_TEST_GETTEXT_POISON is set to an invalid value die_bad_number() will die, but to do so it would usually call gettext(). Let's detect the special case of GIT_TEST_GETTEXT_POISON and always emit that message in the C locale, lest we infinitely loop. As seen in the updated tests in t0017-env-helper.sh there's also a caveat related to "env--helper" needing to read the config for trace2 purposes. Since the C_LOCALE_OUTPUT prerequisite is lazy and relies on "env--helper" we could get invalid results if we failed to read the config (e.g. because we'd loop on includes) when combined with e.g. "test_i18ngrep" wanting to check with "env--helper" if GIT_TEST_GETTEXT_POISON was true or not. I'm crossing my fingers and hoping that a test similar to the one I removed in the earlier "config tests: simplify include cycle test" change in this series won't happen again, and testing for this explicitly in "env--helper"'s own tests. This change breaks existing uses of e.g. GIT_TEST_GETTEXT_POISON=YesPlease, which we've documented in po/README and other places. As noted in [1] we might want to consider also accepting "YesPlease" in "env--helper" as a special-case. But as the lack of uproar over `6cdccfce1e` ("i18n: make GETTEXT_POISON a runtime option", 2018-11-08) demonstrates the audience for this option is a really narrow set of git developers, who shouldn't have much trouble modifying their test scripts, so I think it's better to deal with that minor headache now and make all the relevant GIT_TEST_* variables boolean in the same way than carry the "YesPlease" special-case forward. 1. https://public-inbox.org/git/xmqqtvckm3h8.fsf@gitster-ct.c.googlers.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	dfe1a17df9	tests: add a special setup where prerequisites fail As discussed in [1] there's a regression in the "pu" branch now because a new test implicitly assumed that a previous test guarded by a prerequisite had been run. Add a "GIT_TEST_FAIL_PREREQS" special test setup where we'll skip (nearly) all tests guarded by prerequisites, allowing us to easily emulate those platform where we don't run these tests. As noted in the documentation I'm adding I'm whitelisting the SYMLINKS prerequisite for now. A lot of tests started failing if we lied about not supporting symlinks. It's also unlikely that we'll have a failing test due to a hard dependency on symlinks without that being the obvious cause, so for now it's not worth the effort to make it work. 1. https://public-inbox.org/git/nycvar.QRO.7.76.6.1905131531000.44@tvgsbejvaqbjf.bet/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	b02e7d5d70	tests: disallow the use of abbreviated options (by default) Git's command-line parsers support uniquely abbreviated options, e.g. `git init --ba` would automatically expand `--ba` to `--bare`. This is a very convenient feature in every day life for Git users, in particular when tab completion is not available. However, it is not a good idea to rely on that in Git's test suite, as something that is a unique abbreviation of a command line option today might no longer be a unique abbreviation tomorrow. For example, if a future contribution added a new mode `git init --babyproofing` and a previously-introduced test case used the fact that `git init --ba` expanded to `git init --bare`, that future contribution would now have to touch seemingly unrelated tests just to keep the test suite from failing. So let's disallow abbreviated options in the test suite by default. Note: for ease of implementation, this patch really only touches the `parse-options` machinery: more and more hand-rolled option parsers are converted to use that internal API, and more and more scripts are converted to built-ins (naturally using the parse-options API, too), so in practice this catches most issues, and is definitely the biggest bang for the buck. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	d03ebd411c	rebase: remove the rebase.useBuiltin setting Remove the rebase.useBuiltin setting, which was added as an escape hatch to disable the builtin version of rebase first released with Git 2.20. See [1] for the initial implementation of rebase.useBuiltin, and [2] and [3] for the documentation and corresponding GIT_TEST_REBASE_USE_BUILTIN option. Carrying the legacy version is a maintenance burden as seen in `7e097e27d3` ("legacy-rebase: backport -C<n> and --whitespace=<option> checks", 2018-11-20) and `9aea5e9286` ("rebase: fix regression in rebase.useBuiltin=false test mode", 2019-02-13). Since the built-in version has been shown to be stable enough let's remove the legacy version. As noted in [3] having use_builtin_rebase() shell out to get its config doesn't make any sense anymore, that was done for the purposes of spawning the legacy rebase without having modified any global state. Let's instead handle this case in rebase_config(). There's still a bunch of references to git-legacy-rebase in po/*.po, but those will be dealt with in time by the i18n effort. Even though this configuration variable only existed two releases let's not entirely delete the entry from the docs, but note its absence. Individual versions of git tend to be around for a while due to distro packaging timelines, so e.g. if we're "lucky" a given version like 2.21 might be installed on say OSX for half a decade. That'll mean some people probably setting this in config, and then when they later wonder if it's needed they can Google search the config option name or check it in git-config. It also allows us to refer to the docs from the warning for details. 1. `55071ea248` ("rebase: start implementing it as a builtin", 2018-08-07) 2. `d8d0a546f0` ("rebase doc: document rebase.useBuiltin", 2018-11-14) 3. `62c23938fa` ("tests: add a special setup where rebase.useBuiltin is off", 2018-11-14) 3. https://public-inbox.org/git/nycvar.QRO.7.76.6.1903141544110.41@tvgsbejvaqbjf.bet/ Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	900721e15c	test-lib: introduce 'test_atexit' When running Apache, 'git daemon', or p4d, we want to kill them at the end of the test script, otherwise a leftover daemon process will keep its port open indefinitely, and thus will interfere with subsequent executions of the same test script. So far, we stop these daemon processes "manually", i.e.: - by registering functions or commands in the trap on EXIT to stop the daemon while preserving the last seen exit code before the trap (to deal with a failure when run with '--immediate' or with interrupts by ctrl-C), - and by invoking these functions/commands last thing before 'test_done' (and sometimes restoring the test framework's default trap on EXIT, to prevent the daemons from being killed twice). On one hand, we do this inconsistently, e.g. 'git p4' tests invoke different functions in the trap on EXIT and in the last test before 'test_done', and they neither restore the test framework's default trap on EXIT nor preserve the last seen exit code. On the other hand, this is error prone, because, as shown in a previous patch in this series, any output from the cleanup commands in the trap on EXIT can prevent a proper cleanup when a test script run with '--verbose-log' and certain shells, notably 'dash', is interrupted. Let's introduce 'test_atexit', which is loosely modeled after 'test_when_finished', but has a broader scope: rather than running the commands after the current test case, run them when the test script finishes, and also run them when the test is interrupted, or exits early in case of a failure while the '--immediate' option is in effect. When running the cleanup commands at the end of a successful test, then they will be run in 'test_done' before it removes the trash directory, i.e. the cleanup commands will still be able to access any pidfiles or socket files in there. When running the cleanup commands after an interrupt or failure with '--immediate', then they will be run in the trap on EXIT. In both cases they will be run in 'test_eval_', i.e. both standard error and output of all cleanup commands will go where they should according to the '-v' or '--verbose-log' options, and thus won't cause any troubles when interrupting a test script run with '--verbose-log'. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Jonathan Tan	8cbeba0632	tests: define GIT_TEST_PROTOCOL_VERSION Define a GIT_TEST_PROTOCOL_VERSION environment variable meant to be used from tests. When set, this ensures protocol.version is at least the given value, allowing the entire test suite to be run as if this configuration is in place for all repositories. As of this patch, all tests pass whether GIT_TEST_PROTOCOL_VERSION is unset or set to 0. Some tests fail when GIT_TEST_PROTOCOL_VERSION is set to 1 or 2, but this will be dealt with in subsequent patches. This is based on work by Ævar Arnfjörð Bjarmason. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	7906af0cb8	tests: add a special setup where stash.useBuiltin is off Add a GIT_TEST_STASH_USE_BUILTIN=false test mode which is equivalent to running with stash.useBuiltin=false. This is needed to spot that we're not introducing any regressions in the legacy stash version while we're carrying both it and the new built-in version. This imitates the equivalent treatment for the built-in rebase in `62c23938fa` (tests: add a special setup where rebase.useBuiltin is off, 2018-11-14). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	f545737144	tests: introduce --stress-jobs=<N> The --stress option currently accepts an argument, but it is confusing to at least this user that the argument does not define the maximal number of stress iterations, but instead the number of jobs to run in parallel per stress iteration. Let's introduce a separate option for that, whose name makes it more obvious what it is about, and let --stress=<N> error out with a helpful suggestion about the two options tha could possibly have been meant. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	de69e6f6c9	tests: let --stress-limit=<N> imply --stress It does not make much sense that running a test with --stress-limit=<N> seemingly ignores that option because it does not stress test at all. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Junio C Hamano	11f470aee7	test: caution on our version of 'yes' During a review of a patch, we noticed that we use our own imitation of 'yes' with the limit of 99 lines. It is very tempting to lift this arbitrary limit, but the limit is there for a reason. Add an in-code comment to prevent future developers from wasting their time. Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	76e27fbfd9	test-lib: make '--stress' more bisect-friendly Let's suppose that a test somehow becomes flaky between 'master' and 'pu', and tends to fail within the first 50 repetitions when run with '--stress'. In such a case we could use 'git bisect' to find the culprit: if the test script fails with '--stress', then the commit is definitely bad, but if it survives, say, 300 repetitions, then we could consider it good with reasonable confidence. Unfortunately, all this could only be done manually, because '--stress' would run the test script repeatedly for all eternity on a good commit, and it would exit with success even when it found a failure on a bad commit. So let's make '--stress' usable with 'git bisect run': - Make it exit with failure if a failure is found. - Add the '--stress-limit=<N>' option to repeat the test script at most N times in each of the parallel jobs, and exit with success when the limit is reached. And then we could simply run something like: $ git bisect start origin/pu master $ git bisect run sh -c 'make && cd t && ./t1234-foo.sh --stress --stress-limit=300' Sure, as a brand new feature it won't be any useful right now, but in a release or three most cooking topics will already contain this, so we could automatically bisect at least newly introduced flakiness. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	dd167a3001	tests: optionally skip bin-wrappers/ This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Derrick Stolee	99dbbfa8dd	pack-objects: create GIT_TEST_PACK_SPARSE Create a test variable GIT_TEST_PACK_SPARSE to enable the sparse object walk algorithm by default during the test suite. Enabling this variable ensures coverage in many interesting cases, such as shallow clones, partial clones, and missing objects. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Jonathan Tan	07c3c2aa16	tests: define GIT_TEST_SIDEBAND_ALL Define a GIT_TEST_SIDEBAND_ALL environment variable meant to be used from tests. When set to true, this overrides uploadpack.allowsidebandall to true, allowing the entire test suite to be run as if this configuration is in place for all repositories. As of this patch, all tests pass whether GIT_TEST_SIDEBAND_ALL is unset or set to 1. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	fb7d1e3ac8	test-lib: add the '--stress' option to run a test repeatedly under load Unfortunately, we have a few flaky tests, whose failures tend to be hard to reproduce. We've found that the best we can do to reproduce such a failure is to run the test script repeatedly while the machine is under load, and wait in the hope that the load creates enough variance in the timing of the test's commands that a failure is evenually triggered. I have a command to do that, and I noticed that two other contributors have rolled their own scripts to do the same, all choosing slightly different approaches. To help reproduce failures in flaky tests, introduce the '--stress' option to run a test script repeatedly in multiple parallel jobs until one of them fails, thereby using the test script itself to increase the load on the machine. The number of parallel jobs is determined by, in order of precedence: the number specified as '--stress=<N>', or the value of the GIT_TEST_STRESS_LOAD environment variable, or twice the number of available processors (as reported by the 'getconf' utility), or 8. Make '--stress' imply '--verbose -x --immediate' to get the most information about rare failures; there is really no point in spending all the extra effort to reproduce such a failure, and then not know which command failed and why. To prevent the several parallel invocations of the same test from interfering with each other: - Include the parallel job's number in the name of the trash directory and the various output files under 't/test-results/' as a '.stress-<Nr>' suffix. - Add the parallel job's number to the port number specified by the user or to the test number, so even tests involving daemons listening on a TCP socket can be stressed. - Redirect each parallel test run's verbose output to 't/test-results/$TEST_NAME.stress-<nr>.out', because dumping the output of several parallel running tests to the terminal would create a big ugly mess. For convenience, print the output of the failed test job at the end, and rename its trash directory to end with the '.stress-failed' suffix, so it's easy to find in a predictable path (OTOH, all absolute paths recorded in the trash directory become invalid; we'll see whether this causes any issues in practice). If, in an unlikely case, more than one jobs were to fail nearly at the same time, then print the output of all failed jobs, and rename the trash directory of only the last one (i.e. with the highest job number), as it is the trash directory of the test whose output will be at the bottom of the user's terminal. Based on Jeff King's 'stress' script. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	62c23938fa	tests: add a special setup where rebase.useBuiltin is off Add a GIT_TEST_REBASE_USE_BUILTIN=false test mode which is equivalent to running with rebase.useBuiltin=false. This is needed to spot that we're not introducing any regressions in the legacy rebase version while we're carrying both it and the new builtin version. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	6cdccfce1e	i18n: make GETTEXT_POISON a runtime option Change the GETTEXT_POISON compile-time + runtime GIT_GETTEXT_POISON test parameter to only be a GIT_TEST_GETTEXT_POISON=<non-empty?> runtime parameter, to be consistent with other parameters documented in "Running tests with special setups" in t/README. When I added GETTEXT_POISON in `bb946bba76` ("i18n: add GETTEXT_POISON to simulate unfriendly translator", 2011-02-22) I was concerned with ensuring that the _() function would get constant folded if NO_GETTEXT was defined, and likewise that GETTEXT_POISON would be compiled out unless it was defined. But as the benchmark in my [1] shows doing a one-off runtime getenv("GIT_TEST_[...]") is trivial, and since GETTEXT_POISON was originally added the GIT_TEST_* env variables have become the common idiom for turning on special test setups. So change GETTEXT_POISON to work the same way. Now the GETTEXT_POISON=YesPlease compile-time option is gone, and running the tests with GIT_TEST_GETTEXT_POISON=[YesPlease\|] can be toggled on/off without recompiling. This allows for conditionally amending tests to test with/without poison, similar to what `859fdc0c3c` ("commit-graph: define GIT_TEST_COMMIT_GRAPH", 2018-08-29) did for GIT_TEST_COMMIT_GRAPH. Do some of that, now we e.g. always run the t0205-gettext-poison.sh test. I did enough there to remove the GETTEXT_POISON prerequisite, but its inverse C_LOCALE_OUTPUT is still around, and surely some tests using it can be converted to e.g. always set GIT_TEST_GETTEXT_POISON=. Notes on the implementation: * We still compile a dedicated GETTEXT_POISON build in Travis CI. Perhaps this should be revisited and integrated into the "linux-gcc" build, see `ae59a4e44f` ("travis: run tests with GIT_TEST_SPLIT_INDEX", 2018-01-07) for prior art in that area. Then again maybe not, see [2]. * We now skip a test in t0000-basic.sh under GIT_TEST_GETTEXT_POISON=YesPlease that wasn't skipped before. This test relies on C locale output, but due to an edge case in how the previous implementation of GETTEXT_POISON worked (reading it from GIT-BUILD-OPTIONS) wasn't enabling poison correctly. Now it does, and needs to be skipped. * The getenv() function is not reentrant, so out of paranoia about code of the form: printf(_("%s"), getenv("some-env")); call use_gettext_poison() in our early setup in git_setup_gettext() so we populate the "poison_requested" variable in a codepath that's won't suffer from that race condition. * We error out in the Makefile if you're still saying GETTEXT_POISON=YesPlease to prompt users to change their invocation. * We should not print out poisoned messages during the test initialization itself to keep it more readable, so the test library hides the variable if set in $GIT_TEST_GETTEXT_POISON_ORIG during setup. See [3]. See also [4] for more on the motivation behind this patch, and the history of the GETTEXT_POISON facility. 1. https://public-inbox.org/git/871s8gd32p.fsf@evledraar.gmail.com/ 2. https://public-inbox.org/git/20181102163725.GY30222@szeder.dev/ 3. https://public-inbox.org/git/20181022202241.18629-2-szeder.dev@gmail.com/ 4. https://public-inbox.org/git/878t2pd6yu.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	a5f52c6dab	test-lib: introduce the '-V' short option for '--verbose-log' '--verbose-log' is one of the most useful and thus most frequently used test options, but due to its length it's a pain to type on the command line. Let's introduce the corresponding short option '-V' to save some keystrokes. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Derrick Stolee	0465a50506	multi-pack-index: define GIT_TEST_MULTI_PACK_INDEX The multi-pack-index feature is tested in isolation by t5319-multi-pack-index.sh, but there are many more interesting scenarios in the test suite surrounding pack-file data shapes and interactions. Since the multi-pack-index is an optional data structure, it does not make sense to include it by default in those tests. Instead, add a new GIT_TEST_MULTI_PACK_INDEX environment variable that enables core.multiPackIndex and writes a multi-pack-index after each 'git repack' command. This adds extra test coverage when needed. There are a few spots in the test suite that need to react to this change: * t5319-multi-pack-index.sh: there is a test that checks that 'git repack' deletes the multi-pack-index. Disable the environment variable to ensure this still happens. * t5310-pack-bitmaps.sh: One test moves a pack-file from the object directory to an alternate. This breaks the multi-pack-index, so delete the multi-pack-index at this point, if it exists. * t9300-fast-import.sh: One test verifies the number of files in the .git/objects/pack directory is exactly 8. Exclude the multi-pack-index from this count so it is still 8 in all cases. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ben Peart	c780b9cfe8	config: add new index.threads config setting Add support for a new index.threads config setting which will be used to control the threading code in do_read_index(). A value of 0 will tell the index code to automatically determine the correct number of threads to use. A value of 1 will make the code single threaded. A value greater than 1 will set the maximum number of threads to use. For testing purposes, this setting can be overwritten by setting the GIT_TEST_INDEX_THREADS=<n> environment variable to a value greater than 0. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Matthew DeVore	a378fee5b0	Documentation: add shell guidelines Add the following guideline to Documentation/CodingGuidelines: Break overlong lines after "&&", "\|\|", and "\|", not before them; that way the command can continue to subsequent lines without backslash at the end. And the following to t/README (since it is specific to writing tests): Pipes and $(git ...) should be avoided when they swallow exit codes of Git processes Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Matthew DeVore	441ee35d83	t/README: reformat Do, Don't, Keep in mind lists The list of Don'ts for test writing has grown large such that it is hard to see at a glance which section an item is in. In other words, if I ignore a little bit of surrounding context, the "don'ts" look like "do's." To make the list more readable, prefix "Don't" in front of every first sentence in the items. Also, the "Keep in mind" list is out of place and awkward, because it was a very short "list" beneath two very long ones, and it seemed easy to miss under the list of "don'ts," and it only had one item. So move this item to the list of "do's" and phrase as "Remember..." Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ben Peart	5765d97b71	preload-index: update GIT_FORCE_PRELOAD_TEST support Rename GIT_FORCE_PRELOAD_TEST to GIT_TEST_PRELOAD_INDEX for consistency with the other GIT_TEST_ special setups and properly document its use. Add logic in t/test-lib.sh to give a warning when the old variable is set to let people know they need to update their environment to use the new variable. Signed-off-by: Ben Peart <Ben.Peart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ben Peart	1f357b045b	read-cache: update TEST_GIT_INDEX_VERSION support Rename TEST_GIT_INDEX_VERSION to GIT_TEST_INDEX_VERSION for consistency with the other GIT_TEST_ special setups and properly document its use. Add logic in t/test-lib.sh to give a warning when the old variable is set to let people know they need to update their environment to use the new variable. Signed-off-by: Ben Peart <Ben.Peart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ben Peart	4cb54d0aa8	fsmonitor: update GIT_TEST_FSMONITOR support Rename GIT_FSMONITOR_TEST to GIT_TEST_FSMONITOR for consistency with the other GIT_TEST_ special setups and properly document its use. Add logic in t/test-lib.sh to give a warning when the old variable is set to let people know they need to update their environment to use the new variable. Remove the outdated instructions on how to run the test suite utilizing fsmonitor now that it is properly documented in t/README. Signed-off-by: Ben Peart <Ben.Peart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ben Peart	ac6e12f9b7	t/README: correct spelling of "uncommon" Correct a spelling error in the documentation for GIT_TEST_OE_DELTA_SIZE. Signed-off-by: Ben Peart <Ben.Peart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
brian m. carlson	2c02b110da	t: add test functions to translate hash-related values Add several test functions to make working with various hash-related values easier. Add test_oid_init, which loads common hash-related constants and placeholder object IDs from the newly added files in t/oid-info. Provide values for these constants for both SHA-1 and SHA-256. Add test_oid_cache, which accepts data on standard input in the form of hash-specific key-value pairs that can be looked up later, using the same format as the files in t/oid-info. Document this format in a t/oid-info/README directory so that it's easier to use in the future. Add test_oid, which is used to specify look up a per-hash value (produced on standard output) based on the key specified as its argument. Usually the data to be looked up will be a hash-related constant (such as the size of the hash in binary or hexadecimal), a well-known or placeholder object ID (such as the all-zeros object ID or one consisting of "deadbeef" repeated), or something similar. For these reasons, test_oid will usually be used within a command substitution. Consequently, redirect the error output to standard error, since otherwise it will not be displayed. Add test_detect_hash, which currently only detects SHA-1, and test_set_hash, which can be used to set a different hash algorithm for test purposes. In the future, test_detect_hash will learn to actually detect the hash depending on how the testsuite is to be run. Use the local keyword within these functions to avoid overwriting other shell variables. We have had a test balloon in place for a couple of releases to catch shells that don't have this keyword and have not received any reports of failure. Note that the varying usages of local used here are supported by all common open-source shells supporting the local keyword. Test these new functions as part of t0000, which also serves to demonstrate basic usage of them. In addition, add documentation on how to format the lookup data and how to use the test functions. Implement two basic lookup charts, one for common invalid or synthesized object IDs, and one for various facts about the hash function in use. Provide versions of the data for both SHA-1 and SHA-256. Since we use shell variables for storage, names used for lookup can currently consist only of shell identifier characters. If this is a problem in the future, we can hash the names before use. Improved-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Derrick Stolee	859fdc0c3c	commit-graph: define GIT_TEST_COMMIT_GRAPH The commit-graph feature is tested in isolation by t5318-commit-graph.sh and t6600-test-reach.sh, but there are many more interesting scenarios involving commit walks. Many of these scenarios are covered by the existing test suite, but we need to maintain coverage when the optional commit-graph structure is not present. To allow running the full test suite with the commit-graph present, add a new test environment variable, GIT_TEST_COMMIT_GRAPH. Similar to GIT_TEST_SPLIT_INDEX, this variable makes every Git command try to load the commit-graph when parsing commits, and writes the commit-graph file after every 'git commit' command. There are a few tests that rely on commits not existing in pack-files to trigger important events, so manually set GIT_TEST_COMMIT_GRAPH to false for the necessary commands. There is one test in t6024-recursive-merge.sh that relies on the merge-base algorithm picking one of two ambiguous merge-bases, and the commit-graph feature changes which merge-base is picked. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Nguyễn Thái Ngọc Duy	5f4436a721	Document update for nd/unpack-trees-with-cache-tree Fix an incorrect comment in the new code added in `b4da37380b` (unpack-trees: optimize walking same trees with cache-tree - 2018-08-18) and document about the new test variable that is enabled by default in test-lib.sh in `4592e6080f` (cache-tree: verify valid cache-tree in the test suite - 2018-08-18) Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Nguyễn Thái Ngọc Duy	9ac3f0e5b3	pack-objects: fix performance issues on packing large deltas Let's start with some background about oe_delta_size() and oe_set_delta_size(). If you already know, skip the next paragraph. These two are added in `0aca34e826` (pack-objects: shrink delta_size field in struct object_entry - 2018-04-14) to help reduce 'struct object_entry' size. The delta size field in this struct is reduced to only contain max 1MB. So if any new delta is produced and larger than 1MB, it's dropped because we can't really save such a large size anywhere. Fallback is provided in case existing packfiles already have large deltas, then we can retrieve it from the pack. While this should help small machines repacking large repos without large deltas (i.e. less memory pressure), dropping large deltas during the delta selection process could end up with worse pack files. And if existing packfiles already have >1MB delta and pack-objects is instructed to not reuse deltas, all of them will be dropped on the floor, and the resulting pack would be definitely bigger. There is also a regression in terms of CPU/IO if we have large on-disk deltas because fallback code needs to parse the pack every time the delta size is needed and just access to the mmap'd pack data is enough for extra page faults when memory is under pressure. Both of these issues were reported on the mailing list. Here's some numbers for comparison. Version Pack (MB) MaxRSS(kB) Time (s) ------- --------- ---------- -------- 2.17.0 5498 43513628 2494.85 2.18.0 10531 40449596 4168.94 This patch provides a better fallback that is - cheaper in terms of cpu and io because we won't have to read existing pack files as much - better in terms of pack size because the pack heuristics is back to 2.17.0 time, we do not drop large deltas at all If we encounter any delta (on-disk or created during try_delta phase) that is larger than the 1MB limit, we stop using delta_size_ field for this because it can't contain such size anyway. A new array of delta size is dynamically allocated and can hold all the deltas that 2.17.0 can. This array only contains delta sizes that delta_size_ can't contain. With this, we do not have to drop deltas in try_delta() anymore. Of course the downside is we use slightly more memory, even compared to 2.17.0. But since this is considered an uncommon case, a bit more memory consumption should not be a problem. Delta size limit is also raised from 1MB to 16MB to better cover common case and avoid that extra memory consumption (99.999% deltas in this reported repo are under 12MB; Jeff noted binary artifacts topped out at about 3MB in some other private repos). Other fields are shuffled around to keep this struct packed tight. We don't use more memory in common case even with this limit update. A note about thread synchronization. Since this code can be run in parallel during delta searching phase, we need a mutex. The realloc part in packlist_alloc() is not protected because it only happens during the object counting phase, which is always single-threaded. Access to e->delta_size_ (and by extension pack->delta_size[e - pack->objects]) is unprotected as before, the thread scheduler in pack-objects must make sure "e" is never updated by two different threads. The area under the new lock is as small as possible, avoiding locking at all in common case, since lock contention with high thread count could be expensive (most blobs are small enough that delta compute time is short and we end up taking the lock very often). The previous attempt to always hold a lock in oe_delta_size() and oe_set_delta_size() increases execution time by 33% when repacking linux.git with with 40 threads. Reported-by: Elijah Newren <newren@gmail.com> Helped-by: Elijah Newren <newren@gmail.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Nguyễn Thái Ngọc Duy	ac77d0c370	pack-objects: shrink size field in struct object_entry It's very very rare that an uncompressed object is larger than 4GB (partly because Git does not handle those large files very well to begin with). Let's optimize it for the common case where object size is smaller than this limit. Shrink size field down to 31 bits and one overflow bit. If the size is too large, we read it back from disk. As noted in the previous patch, we need to return the delta size instead of canonical size when the to-be-reused object entry type is a delta instead of a canonical one. Add two compare helpers that can take advantage of the overflow bit (e.g. if the file is 4GB+, chances are it's already larger than core.bigFileThreshold and there's no point in comparing the actual value). Another note about oe_get_size_slow(). This function MUST be thread safe because SIZE() macro is used inside try_delta() which may run in parallel. Outside parallel code, no-contention locking should be dirt cheap (or insignificant compared to i/o access anyway). To exercise this code, it's best to run the test suite with something like make test GIT_TEST_OE_SIZE=4 which forces this code on all objects larger than 3 bytes. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Nguyễn Thái Ngọc Duy	43fa44fa3b	pack-objects: move in_pack out of struct object_entry Instead of using 8 bytes (on 64 bit arch) to store a pointer to a pack. Use an index instead since the number of packs should be relatively small. This limits the number of packs we can handle to 1k. Since we can't be sure people can never run into the situation where they have more than 1k pack files. Provide a fall back route for it. If we find out they have too many packs, the new in_pack_by_idx[] array (which has at most 1k elements) will not be used. Instead we allocate in_pack[] array that holds nr_objects elements. This is similar to how the optional in_pack_pos field is handled. The new simple test is just to make sure the too-many-packs code path is at least executed. The true test is running make test GIT_TEST_FULL_IN_PACK_ARRAY=1 to take advantage of other special case tests. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Nguyễn Thái Ngọc Duy	4c2db93807	read-cache.c: make $GIT_TEST_SPLIT_INDEX boolean While at there, document about this special mode when running the test suite. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	94201a2b00	t/README: add a note about don't saving stderr of compound commands Explain in 't/README' why it is a bad idea to redirect and verify the stderr of compound commands, in the hope that future contributions will follow this advice and the test suite will keep working with '-x' tracing and /bin/sh. While at it, since we can now run the test suite with '-x' without needing a Bash version supporting BASH_XTRACEFD, remove the now outdated caution note about non-Bash shells from the description of the '-x' option. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	5fc98e79fc	t: add means to disable '-x' tracing for individual test scripts The previous patch resolved most of the test failures caused by running our test suite with '-x' tracing and /bin/sh, and the following patches in this series will resolve almost all of the remaining failures. Unfortunately, not yet all. Add means to disable '-x' tracing for individual test scripts by setting the $test_untraceable variable to a non-empty value in the test script before sourcing 'test-lib.sh'. However, since '-x' tracing is not an issue with recent Bash versions supporting BASH_XTRACEFD, i.e. v4.1 and later, don't disable tracing when the test script is run with such a Bash version even when $test_untraceable is set. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	12e31a6b12	t: document 'test_must_fail ok=<signal-name>' Since 'test_might_fail' is implemented as a thin wrapper around 'test_must_fail', it also accepts the same options. Mention this in the docs as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Thomas Gummerer	5a0526264b	t/README: document test_cmp_rev test_cmp_rev is a useful function that's used in quite a few test scripts. It is however not documented in t/README. Document it. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Thomas Gummerer	51b7a52522	t/README: remove mention of adding copyright notices We generally no longer include copyright notices in new test scripts. However t/README still mentions it as something to include at the top of every new script. Remove that mention as it's outdated. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Ævar Arnfjörð Bjarmason	ce9a257031	test-lib: add LIBPCRE1 & LIBPCRE2 prerequisites Add LIBPCRE1 and LIBPCRE2 prerequisites which are true when git is compiled with USE_LIBPCRE1=YesPlease or USE_LIBPCRE2=YesPlease, respectively. The syntax of PCRE1 and PCRE2 isn't the same in all cases (see pcresyntax(3) and pcre2syntax(3)). If test are added that test for those they'll need to be guarded by these new prerequisites. The subsequent patch will make use of LIBPCRE2, so LIBPCRE1 isn't strictly needed for now, but let's add it for consistency and so that checking for it doesn't have to be done with the less obvious "PCRE, !LIBPCRE2", which while semantically the same is more confusing, and would lead to bugs if PCRE v3 is ever released as the tests would mean v1, not any non-v2 version. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago

1 2 3 4

179 Commits (12026f46e73ba09b8c403079bccc72a3b85e8dc9)