kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Ævar Arnfjörð Bjarmason	2e54907e83	test-lib tests: get rid of copy/pasted mock test code Now that we've split up the write_sub_test_lib_test() and run_sub_test_lib_test() functions let's fix those tests in t0000-basic.sh that were verbosely copy/pasting earlier tests. That we caught all of them was asserted with a follow-up change that's not part of this series[1], we might add such a duplication check at some later time, but for now let's just one-off remove the duplicate boilerplate. 1. https://lore.kernel.org/git/patch-v3-6.9-bc79b29f3c-20210805T103237Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	c3ff7be6fb	test-lib tests: don't provide a description for the sub-tests Change the $test_description provided for the generated subtests to be constant, since the only purpose of having it is that test-lib.sh will barf if it isn't supplied. The other purpose of having it was to effectively split up the test description between the argument to test_expect_success and the argument to "write_and_run_sub_test_lib_test". Let's only use one of the two. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	9f0a45208d	test-lib tests: split up "write and run" into two functions Refactor the function to write and run tests of the test-lib.sh output into two functions. When this was added back in `565b6fa87b` (tests: refactor mechanics of testing in a sub test-lib, 2012-12-16) there was no reason to do this, but since we started supporting test arguments in `517cd55fd5` (test-lib: self-test that --verbose works, 2013-06-23) we've started to write out duplicate tests simply to test different arguments, now we'll be able to re-use them. This change doesn't consolidate any of those tests yet, it just makes it possible to do so. All the changes in t0000-basic.sh are a simple search-replacement. Since the _run_sub_test_lib_test_common() function doesn't handle running the test anymore we can do away with the sub-shell, which was used to scope an "unset" and "export" shell variables. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Carlo Marcelo Arenas Belón	66c0c44df6	t0000: avoid masking git exit value through pipes `9af0b8dbe2` (t0000-basic: more commit-tree tests., 2006-04-26) adds tests for commit-tree that mask the return exit from git as described in `a378fee5b0` (Documentation: add shell guidelines, 2018-10-05). Fix the tests, to avoid pipes by using a temporary file instead. Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	866a3014de	test-lib tests: move "run_sub_test" to a new lib-subtest.sh Move the "check_sub_test_lib_test()" and its sister functions to a new lib-subtest.sh. In the future (not in this series) I'd like to test test-lib's output in a more targeted and smaller test, and I'll need these functions to do that. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	ade1552598	t0000: fix test if run with TEST_OUTPUT_DIRECTORY Testcases in t0000 are quite special given that they many of them run nested testcases to verify that testing functionality itself works as expected. These nested testcases are realized by writing a new ad-hoc test script which again sources test-lib.sh, where the new script is created in a nested subdirectory located beneath the current trash directory. We then execute the new test script with the nested subdirectory as current working directory and explicitly re-export TEST_OUTPUT_DIRECTORY to point to that directory. While this works as expected in the general case, it falls apart when the developer has TEST_OUTPUT_DIRECTORY explicitly defined either via the environment or via config.mak and runs "make test". In that case, test-lib.sh will clobber the value that we've just carefully set up to instead contain what the developer has defined. As a result, the TEST_OUTPUT_DIRECTORY continues to point at the root output directory, not at the nested one. This issue causes breakage in the 'test_atexit is run' test case: the nested test case writes files into "../../", which is assumed to be the parent's trash directory. But because TEST_OUTPUT_DIRECTORY already points to to the root output directory, we instead end up writing those files outside of the output directory. The parent test case will then try to check whether those files still exist in its own trash directory, which thus must fail now. Fix the issue by adding a new TEST_OUTPUT_DIRECTORY_OVERRIDE variable. If set, then we'll always override the TEST_OUTPUT_DIRECTORY with its value after sourcing GIT-BUILD-OPTIONS. Signed-off-by: Patrick Steinhardt <ps@pks.im> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Jeff King	ac223c4047	t0000: clear GIT_SKIP_TESTS before running sub-tests In t0000, we run several fake "sub-test" suites to verify the behavior of the test suite. But because we don't clear the parent environment completely, the sub-tests can be fooled by variables meant for the parent. For example: GIT_SKIP_TESTS=t1234 ./t0000-basic.sh fails when a sub-test expects its fake t1234 to actually run. This particular pattern is unlikely in practice; we're running a single script, and there is no t1234 in the real test suite anyway (not yet, at least). A more real-world example is: GIT_SKIP_TESTS=t[^0]* make test to run only the t0* tests. The fix is conceptually simple: we should clear the GIT_SKIP_TESTS variable when running the sub-tests, because its contents (if any) will be meant for the main test suite. This is easy to do centrally in our sub-test helper. But there's a catch: some of our tests do set GIT_SKIP_TESTS intentionally to test the feature. We need to allow them to continue to set it, but clear it for all the other tests. And the sub-test helper can't tell if the GIT_SKIP_TESTS it sees is from a test or not. We can handle this by adding a new option to the helper to let callers specify the skip list. I considered adding a more general "--eval" option to let callers set up the env for the sub-test however they like. That would cover this case and possible future ones. But the quoting gets awkward for the callers (since we're now 2 layers deep in evals!), so I went with the simpler more specific solution. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	b57913f205	test-lib tests: remove dead GIT_TEST_FRAMEWORK_SELFTEST variable Stop setting the GIT_TEST_FRAMEWORK_SELFTEST variable. This was originally needed back in `4231d1ba99` (t0000: do not get self-test disrupted by environment warnings, 2018-09-20). It hasn't been needed since I deleted the relevant code in test-lib.sh in `c0eedbc009` (test-lib: remove check_var_migration, 2021-02-09), I just didn't notice that it was set here. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	a926c4b904	tests: remove most uses of C_LOCALE_OUTPUT As a follow-up to `d162b25f95` (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) remove those uses of the now always true C_LOCALE_OUTPUT prerequisite from those tests which declare it as an argument to test_expect_{success,failure}. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Jeff King	30291525d9	t0000: consistently use single quotes for outer tests When we use the sub-test helpers, we end up defining one shell snippet inside another shell snippet. So if we use single-quotes for the outer snippet, we have to use double-quotes within the inner snippet (it's included as here-doc within the outer snippet, but using a single quote would end the outer snippet early). Or vice versa we can use double quotes for the outer snippet, but then single quotes in the inner. We have some of each in the script, and neither is wrong. But it would be nice to be consistent unless there is a good reason not to. Using single quotes for the outer script is preferable, because it requires less metacharacter quoting overall. For example, in: test_expect_success 'outer' ' run_sub_test_lib_test ... <<-\EOF echo $foo && test_expect_success "inner" " echo \$bar " EOF ' we need only quote inside "inner", but not inside "outer" or the here-doc. Whereas if we flip them, we have to quote in both places: test_expect_success 'outer' " run_sub_test_lib_test ... <<-\EOF echo \$foo && test_expect_success 'inner' ' echo \$bar ' EOF " The exception is when we need a literal single-quote in an expected output here-doc. There we can either use outer double-quotes, or just use ${SQ} within the doc. I chose the latter for consistency (within this test, but also with other test scripts that face the same problem). There is one other interesting case, which is some tests that do: test_expect_success ... " do_something --run='"'!3'"' " This is rather confusing to read, but is correct. The outer script sees '!3' in single-quotes, as does the eval'd snippet. This is perhaps being overly cautious. In many interactive shells, an exclamation triggers history expansion even inside double quotes, but that is not generally true in non-interactive shells. There's some conflicting information here. Commit `784ce03d55` (t4216: avoid unnecessary subshell in test_bloom_filters_not_used, 2020-05-19) reports it as a problem with OpenBSD 6.7's /bin/sh. However, we have many instances in this script of prereqs like !LAZY_TRUE, which haven't been a problem. I left them un-escaped here to test out this theory. It's much nicer if we can not worry about this as a portability issue, so it's worth knowing. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Jeff King	080e295248	t0000: run cleaning test inside sub-test Our check of test_when_finished is done directly in the main script, and if we failed to clean, we complain and exit immediately. It's nicer to signal a test failure here, for a few reasons: - this gives better output to the user when run under a TAP harness like "prove" - constency; it's the only test left in the file that behaves this way - half of its "if" conditional is nonsense anyway; it picked up a reference to GIT_TEST_FAIL_PREREQS_INTERNAL in `dfe1a17df9` (tests: add a special setup where prerequisites fail, 2019-05-13) along with its neighbors, even though it has nothing to do with that flag We could actually do this without a sub-test at all, and just put our two tests (one to do cleanup, and one to check that it happened) in the main script. But doing it in a subtest is conceptually cleaner (from the perspective of the main test script, we are checking only one thing), and it remains consistent with the "cleanup when failing" test directly after it, which has to happen in a sub-test (to avoid the main script complaining of the failed test). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Jeff King	efd2600e6f	t0000: run prereq tests inside sub-test We test the behavior of prerequisites in t0000 by setting up fake ones in the main test script, trying to run some tests, and then seeing if those tests impacted the environment correctly. If they didn't, then we write a message and manually call exit. Instead, let's push these down into a sub-test, like many of the other tests covering the framework itself. This has a few advantages: - it does not pollute the test output with mention of skipped tests (that we know are uninteresting -- the point of the test was to see that these are skipped). - when running in a TAP harness, we get a useful test failure message (whereas when the script exits early, a tool like "prove" simply says "Dubious, test returned 1"). - we do not have to worry about different test environments, such as when GIT_TEST_FAIL_PREREQS_INTERNAL is set. Our sub-test helpers already give us a known environment. - the tests themselves are a bit easier to read, as we can just check the test-framework output to see what happened (and get the usual test_cmp diff if it failed) A few notes on the implementation: - we could do one sub-test per each individual test_expect_success. I broke it up here into a few logical groups, as I think this makes it more readable - the original tests modified environment variables inside the test bodies. Instead, I've used "true" as the body of a test we expect to run and "false" otherwise. Technically this does not confirm that the body of the "true" test actually ran. We are trusting the framework output to believe that it truly ran, which is sufficient for these tests. And I think the end result is much simpler to follow. - the nested_prereq test uses a few bare "test -f" calls; I converted these to our usual test_path_is_* helpers while moving the code around. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Jeff King	03efadb774	t0000: keep clean-up tests together We check that test_when_finished cleans up after a test, and that it runs even after a failure. Those two were originally adjacent, but got split apart by the new test added in `477dcaddb6` (tests: do not let lazy prereqs inside `test_expect_*` turn off tracing, 2020-03-26), and then further by more lazy-prereq tests. Let's move them back together. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
SZEDER Gábor	53ff3b96a8	tests: make sure nested lazy prereqs work reliably Some test prereqs depend on other prereqs, so in a couple of cases we have nested prereqs that look something like this: test_lazy_prereq FOO ' test_have_prereq BAR && check-foo ' This can be problematic, because lazy prereqs are evaluated in the '$TRASH_DIRECTORY/prereq-test-dir' directory, which is the same for every prereq, and which is automatically removed after the prereq has been evaluated. So if the inner prereq (BAR above) is a lazy prereq that hasn't been evaluated yet, then after its evaluation the 'prereq-test-dir' shared with the outer prereq will be removed. Consequently, 'check-foo' will find itself in a non-existing directory, and won't be able to create/access any files in its cwd, which could result in an unfulfilled outer prereq. Luckily, this doesn't affect any of our current nested prereqs, either because the inner prereq is not a lazy prereq (e.g. MINGW, CYGWIN or PERL), or because the outer prereq happens to be checked without touching any paths in its cwd (GPGSM and RFC1991 in 'lib-gpg.sh'). So to prevent nested prereqs from interfering with each other let's evaluate each prereq in its own dedicated directory by appending the prereq's name to the directory name, e.g. 'prereq-test-dir-SYMLINKS'. In the test we check not only that the prereq test dir is still there, but also that the inner prereq can't mess with the outer prereq's files. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Elijah Newren	f21ac368f1	test-lib: allow selecting tests by substring/glob with --run Many of our test scripts have several "setup" tests. It's a lot easier to say ./t0050-filesystem.sh --run=setup,9 in order to run all the setup tests as well as test #9, than it is to track down what all the setup tests are and enter all their numbers in the list. Also, I often find myself wanting to run just one or a couple tests from the test file, but I don't know the numbering of any of the tests -- to get it I either have to first run the whole test file (or start counting by hand or figure out some other clever but non-obvious tricks). It's really convenient to be able to just look at the test description(s) and then run ./t6416-recursive-corner-cases.sh --run=symlink or ./t6402-merge-rename.sh --run='setup,unnecessary update' Add such an ability to test selection which relies on merely matching against the test description. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Caleb Tillman	ac9b547548	t0000: use test_path_is_file instead of "test -f" Signed-off-by: Caleb Tillman <caleb.tillman@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
brian m. carlson	e023ff0691	t: remove test_oid_init in tests Now that we call test_oid_init in the setup for all test scripts, there's no point in calling it individually. Remove all of the places where we've done so to help keep tests tidy. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
brian m. carlson	ceaa4b3ad7	t: add test_oid option to select hash algorithm In some tests, we have data files which are written with a particular hash algorithm. Instead of keeping two copies of the test files, we can keep one, and translate the value on the fly. In order to do so, we'll need to read both the source algorithm and the current algorithm, so add an optional flag to the test_oid helper that lets us look up a value for a specified hash algorithm. This should not cause any conflicts with existing tests, since key arguments to test_oid are allowed to contains only shell identifier characters. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Denton Liu	6a67c75948	test-lib-functions: restrict test_must_fail usage In previous commits, we removed the usage of test_must_fail() for most commands except for a set of pre-approved commands. Since that's done, only allow test_must_fail() to run those pre-approved commands. Obviously, we should allow `git`. We allow `__git` as some completion functions return an error code that comes from a git invocation. It's good to avoid using test_must_fail unnecessarily but it wouldn't hurt to err on the side of caution when we're potentially wrapping a git command (like in these cases). We also allow `test-tool` and `test-svn-fe` because these are helper commands that are written by us and we want to catch their failure. Finally, we allow `test_terminal` because `test_terminal` just wraps around git commands. Also, we cannot rewrite `test_must_fail test_terminal` as `test_terminal test_must_fail` because test_must_fail() is a shell function and as a result, it cannot be invoked from the test-terminal Perl script. We opted to explicitly list the above tools instead of using a catch-all such as `test[-_]` because we want to be as restrictive as possible so that in the future, someone would not accidentally introduce an unrelated usage of test_must_fail() on an "unapproved" command. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Carlo Marcelo Arenas Belón	0555e4af58	t/t0000-basic: make sure subtests also use TEST_SHELL_PATH `3f824e91c8` (t/Makefile: introduce TEST_SHELL_PATH, 2017-12-08) allows for setting a shell for running the tests, but the generated subtests weren't updated. Correct that and while at it update it to use write_script. Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jeff King	797e2cfd78	t0000: disable GIT_TEST_FAIL_PREREQS in sub-tests The test added by `477dcaddb6` (tests: do not let lazy prereqs inside `test_expect_*` turn off tracing, 2020-03-26) runs a sub-test script that traces a test with a lazy prereq, like: test_have_prereq LAZY && echo trace That won't work if GIT_TEST_FAIL_PREREQS is set in the environment, because our have_prereq will report failure, and we won't run the echo at all. We could work around this by avoiding the &&-chain, but we can fix this and any future tests at once by unsetting that variable for our sub-tests. These are meant to be controlled environments where we test the test-suite itself; the outer test snippet should be in charge of the sub-test environment, not whatever mode the user happens to be running in. Reported-by: Son Luong Ngoc <sluongng@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	477dcaddb6	tests: do not let lazy prereqs inside `test_expect_` turn off tracing The `test_expect_` functions use `test_eval_` and so does `test_run_lazy_prereq_`. If tracing is enabled via the `-x` option, `test_eval_` turns on tracing while evaluating the code block, and turns it off directly after it. This is unwanted for nested invocations. One somewhat surprising example of this is when running a test that calls `test_i18ngrep`: that function requires the `C_LOCALE_OUTPUT` prereq, and that prereq is a lazy one, so it is evaluated via `test_eval_`, the command tracing is turned off, and the test case continues to run _without tracing the commands_. Another somewhat surprising example is when one lazy prereq depends on another lazy prereq: the former will call `test_have_prereq` with the latter one, which in turn calls `test_eval_` and -- you guessed it -- tracing (if enabled) will be turned off _before_ returning to evaluating the other lazy prereq. As we will introduce just such a scenario with the GPG, GPGSM and RFC1991 prereqs, let's fix that by introducing a variable that keeps track of the current trace level: nested `test_eval_` calls will increment and then decrement the level, and only when it reaches 0, the tracing will _actually_ be turned off. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Denton Liu	7717242014	t0000: replace test_must_fail with run_sub_test_lib_test_err() The test_must_fail function should only be used for git commands since we should assume that external commands work sanely. We use test_must_fail to test run_sub_test_lib_test() but that function does not invoke any git commands internally. Even better, we have a function that's exactly meant to be used when we expect to have a failing test suite: run_sub_test_lib_test_err()! Replace `test_must_fail run_sub_test_lib_test` with `run_sub_test_lib_test_err`. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
SZEDER Gábor	43a2afee82	tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values Since `3b072c577b` (tests: replace test_tristate with "git env--helper", 2019-06-21) we get the normalized bool values of various GIT_TEST_* environment variables via 'git env--helper'. Now, while the 'git env--helper' command itself does catch invalid values in the environment variable or in the given --default and exits with error (exit code 128 or 129, respectively), it's invoked in conditions like 'if ! git env--helper ...', which means that all invalid bool values are interpreted the same as the ordinary 'false' (exit code 1). This has led to inadvertently skipped httpd tests in our CI builds for a couple of weeks, see `3960290675` (ci: restore running httpd tests, 2019-09-06). Let's be more careful about what the test suite accepts as bool values in GIT_TEST_* environment variables, and error out loud and clear on invalid values instead of simply skipping tests. Add the 'test_bool_env' helper function to encapsulate the invocation of 'git env--helper' and the verification of its exit code, and replace all invocations of that command in our test framework and test suite with a call to this new helper (except in 't0017-env-helper.sh', of course). $ GIT_TEST_GIT_DAEMON=YesPlease ./t5570-git-daemon.sh fatal: bad numeric config value 'YesPlease' for 'GIT_TEST_GIT_DAEMON': invalid unit error: test_bool_env requires bool values both for $GIT_TEST_GIT_DAEMON and for the default fallback Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Denton Liu	8cb7980382	t0000: test multiple local assignment According to POSIX enhancement request '0000767: Add built-in "local"'[1], dash only allows one variable in a local definition; it permits assignment though it doesn't document that clearly. however, this isn't true since t0000 still passes with this patch applied on dash 0.5.10.2. Needless to say, since `local` isn't POSIX standardized, it is not exactly clear what `local` entails on different versions of different shells. We currently already have many instances of multiple local assignments in our codebase. Ensure that this is actually supported by explicitly testing that it is sane. [1]: http://austingroupbugs.net/bug_view_page.php?bug_id=767 Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Elijah Newren	7a40cf1553	Fix spelling errors in comments of testcases Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Denton Liu	b05b40930e	t0000: cover GIT_SKIP_TESTS blindspots Currently, the tests for GIT_SKIP_TESTS do not cover the situation where we skip an entire test suite. The tests also do not cover the situation where we have GIT_SKIP_TESTS defined but the test suite does not match. Add two test cases so we cover this blindspot. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Jeff King	7f0b590875	t0000: reword comments for "local" test Commit `01d3a526ad` (t0000: check whether the shell supports the "local" keyword, 2017-10-26) added a test to gather data on whether people run the test suite with shells that don't support "local". After almost two years, nobody has complained, and several other uses have cropped up in test-lib-functions.sh. Let's declare it acceptable to use. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	ffe1afe67c	tests: show the test name and number at the start of verbose output The verbose output of every test looks something like this: expecting success: echo content >file && git add file && git commit -m "add file" [master (root-commit) d1fbfbd] add file Author: A U Thor <author@example.com> 1 file changed, 1 insertion(+) create mode 100644 file ok 1 - commit works i.e. first an "expecting success" (or "checking known breakage") line followed by the commands to be executed, then the output of those comamnds, and finally an "ok"/"not ok" line containing the test name. Note that the test's name is only shown at the very end. With '-x' tracing enabled and/or in longer tests the verbose output might be several screenfulls long, making it harder than necessary to find where the output of the test with a given name starts (especially when the outputs to different file descriptors are racing, and the "expecting success"/command block arrives earlier than the "ok" line of the previous test). Print the test name at the start of the test's verbose output, i.e. at the end of the "expecting success" and "checking known breakage" lines, to make the start of a particular test a bit easier to recognize. Also print the test script and test case numbers, to help those poor souls who regularly have to scan through the combined verbose output of several test scripts. So the dummy test above would start like this: expecting success of 9999.1 'commit works': echo content >file && [...] Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
SZEDER Gábor	96f3ccc2ab	t0000-basic: use realistic test script names in the verbose tests Our test scripts are named something like 't1234-command.sh', but the script names used in 't0000-basic.sh' don't follow this naming convention. Normally this doesn't matter, because the test scripts themselves don't care how they are called. However, the next patch will start to include the test number in the test's verbose output, so the test script's name will matter in the two tests checking the verbose output. Update the tests 'test --verbose' and 'test --verbose-only' to follow out test script naming convention. Leave the other tests in 't0000' unchanged: changing the names of their test scripts would be only pointless code churn. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	c740039921	tests: make GIT_TEST_FAIL_PREREQS a boolean Change the GIT_TEST_FAIL_PREREQS variable from being "non-empty?" to being a more standard boolean variable. I recently added the variable in `dfe1a17df9` ("tests: add a special setup where prerequisites fail", 2019-05-13), having to add another "non-empty?" special-case is what prompted me to write the "git env--helper" utility being used here. Converting this one is a bit tricky since we use it so early and frequently in the guts of the test code itself, so let's set a GIT_TEST_FAIL_PREREQS_INTERNAL which can be tested with the old "test -n" for the purposes of the shell code, and change the user-exposed and documented GIT_TEST_FAIL_PREREQS variable to a boolean. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	dfe1a17df9	tests: add a special setup where prerequisites fail As discussed in [1] there's a regression in the "pu" branch now because a new test implicitly assumed that a previous test guarded by a prerequisite had been run. Add a "GIT_TEST_FAIL_PREREQS" special test setup where we'll skip (nearly) all tests guarded by prerequisites, allowing us to easily emulate those platform where we don't run these tests. As noted in the documentation I'm adding I'm whitelisting the SYMLINKS prerequisite for now. A lot of tests started failing if we lied about not supporting symlinks. It's also unlikely that we'll have a failing test due to a hard dependency on symlinks without that being the obvious cause, so for now it's not worth the effort to make it work. 1. https://public-inbox.org/git/nycvar.QRO.7.76.6.1905131531000.44@tvgsbejvaqbjf.bet/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	900721e15c	test-lib: introduce 'test_atexit' When running Apache, 'git daemon', or p4d, we want to kill them at the end of the test script, otherwise a leftover daemon process will keep its port open indefinitely, and thus will interfere with subsequent executions of the same test script. So far, we stop these daemon processes "manually", i.e.: - by registering functions or commands in the trap on EXIT to stop the daemon while preserving the last seen exit code before the trap (to deal with a failure when run with '--immediate' or with interrupts by ctrl-C), - and by invoking these functions/commands last thing before 'test_done' (and sometimes restoring the test framework's default trap on EXIT, to prevent the daemons from being killed twice). On one hand, we do this inconsistently, e.g. 'git p4' tests invoke different functions in the trap on EXIT and in the last test before 'test_done', and they neither restore the test framework's default trap on EXIT nor preserve the last seen exit code. On the other hand, this is error prone, because, as shown in a previous patch in this series, any output from the cleanup commands in the trap on EXIT can prevent a proper cleanup when a test script run with '--verbose-log' and certain shells, notably 'dash', is interrupted. Let's introduce 'test_atexit', which is loosely modeled after 'test_when_finished', but has a broader scope: rather than running the commands after the current test case, run them when the test script finishes, and also run them when the test is interrupted, or exits early in case of a failure while the '--immediate' option is in effect. When running the cleanup commands at the end of a successful test, then they will be run in 'test_done' before it removes the trash directory, i.e. the cleanup commands will still be able to access any pidfiles or socket files in there. When running the cleanup commands after an interrupt or failure with '--immediate', then they will be run in the trap on EXIT. In both cases they will be run in 'test_eval_', i.e. both standard error and output of all cleanup commands will go where they should according to the '-v' or '--verbose-log' options, and thus won't cause any troubles when interrupting a test script run with '--verbose-log'. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Ævar Arnfjörð Bjarmason	6cdccfce1e	i18n: make GETTEXT_POISON a runtime option Change the GETTEXT_POISON compile-time + runtime GIT_GETTEXT_POISON test parameter to only be a GIT_TEST_GETTEXT_POISON=<non-empty?> runtime parameter, to be consistent with other parameters documented in "Running tests with special setups" in t/README. When I added GETTEXT_POISON in `bb946bba76` ("i18n: add GETTEXT_POISON to simulate unfriendly translator", 2011-02-22) I was concerned with ensuring that the _() function would get constant folded if NO_GETTEXT was defined, and likewise that GETTEXT_POISON would be compiled out unless it was defined. But as the benchmark in my [1] shows doing a one-off runtime getenv("GIT_TEST_[...]") is trivial, and since GETTEXT_POISON was originally added the GIT_TEST_* env variables have become the common idiom for turning on special test setups. So change GETTEXT_POISON to work the same way. Now the GETTEXT_POISON=YesPlease compile-time option is gone, and running the tests with GIT_TEST_GETTEXT_POISON=[YesPlease\|] can be toggled on/off without recompiling. This allows for conditionally amending tests to test with/without poison, similar to what `859fdc0c3c` ("commit-graph: define GIT_TEST_COMMIT_GRAPH", 2018-08-29) did for GIT_TEST_COMMIT_GRAPH. Do some of that, now we e.g. always run the t0205-gettext-poison.sh test. I did enough there to remove the GETTEXT_POISON prerequisite, but its inverse C_LOCALE_OUTPUT is still around, and surely some tests using it can be converted to e.g. always set GIT_TEST_GETTEXT_POISON=. Notes on the implementation: * We still compile a dedicated GETTEXT_POISON build in Travis CI. Perhaps this should be revisited and integrated into the "linux-gcc" build, see `ae59a4e44f` ("travis: run tests with GIT_TEST_SPLIT_INDEX", 2018-01-07) for prior art in that area. Then again maybe not, see [2]. * We now skip a test in t0000-basic.sh under GIT_TEST_GETTEXT_POISON=YesPlease that wasn't skipped before. This test relies on C locale output, but due to an edge case in how the previous implementation of GETTEXT_POISON worked (reading it from GIT-BUILD-OPTIONS) wasn't enabling poison correctly. Now it does, and needs to be skipped. * The getenv() function is not reentrant, so out of paranoia about code of the form: printf(_("%s"), getenv("some-env")); call use_gettext_poison() in our early setup in git_setup_gettext() so we populate the "poison_requested" variable in a codepath that's won't suffer from that race condition. * We error out in the Makefile if you're still saying GETTEXT_POISON=YesPlease to prompt users to change their invocation. * We should not print out poisoned messages during the test initialization itself to keep it more readable, so the test library hides the variable if set in $GIT_TEST_GETTEXT_POISON_ORIG during setup. See [3]. See also [4] for more on the motivation behind this patch, and the history of the GETTEXT_POISON facility. 1. https://public-inbox.org/git/871s8gd32p.fsf@evledraar.gmail.com/ 2. https://public-inbox.org/git/20181102163725.GY30222@szeder.dev/ 3. https://public-inbox.org/git/20181022202241.18629-2-szeder.dev@gmail.com/ 4. https://public-inbox.org/git/878t2pd6yu.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Matthew DeVore	dcbaa0b361	t/*: fix ordering of expected/observed arguments Fix various places where the ordering was obviously wrong, meaning it was easy to find with grep. Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Junio C Hamano	4231d1ba99	t0000: do not get self-test disrupted by environment warnings The test framework test-lib.sh itself would want to give warnings and hints, e.g. when it sees a deprecated environment variable is in use that we want to encourage users to migrate to another variable. The self-test of test framework done in t0000 however do not expect to see these warnings and hints, so depending on the settings of environment variables, a running test may or may not produce these messages to the standard error output, breaking the expectations of self-test test framework does on itself. Here is what we see: $ TEST_GIT_INDEX_VERSION=4 sh t0000-basic.sh -i -v ... 'err' is not empty, it contains: warning: TEST_GIT_INDEX_VERSION is now GIT_TEST_INDEX_VERSION hint: set GIT_TEST_INDEX_VERSION too during the transition period not ok 5 - pretend we have a fully passing test suite The following quick attempt to work it around does not work, because some tests in t0000 do want to see expected errors from the test framework itself. t/t0000-basic.sh \| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh index 850f651e4e..88c6ed4696 100755 --- a/t/t0000-basic.sh +++ b/t/t0000-basic.sh @@ -88,7 +88,7 @@ _run_sub_test_lib_test_common () { ' # Point to the t/test-lib.sh, which isn't in ../ as usual - . "\$TEST_DIRECTORY"/test-lib.sh + . "\$TEST_DIRECTORY"/test-lib.sh >/dev/null 2>&1 EOF cat >>"$name.sh" && chmod +x "$name.sh" && There are a few possible ways to work this around: * We could strip the warning: and hint: unconditionally from the error output before the error messages are checked in the self-test (helper functions check_sub_test_lib_test_err and check_sub_test_lib_test); the problem with this approach is that it will make it impossible to write self-tests to ensure that right warnings and hints are given. * We could force a sane environment settings before the test helper _run_sub_test_lib_test_common dot-sources test-lib.sh; the problem with this approach is that _run_sub_test_lib_test_common now needs to be aware of what pairs of environment variables are checked in test-lib.sh using check_var_migration helper. The final patch I came up with is probably the solution that is least bad. Set a variable to tell test-lib.sh that we are running a self-test, so that various pieces in test-lib.sh can react to keep the output stable. Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
brian m. carlson	e483e1441a	t0000: update tests for SHA-256 Test t0000 tests the "basics of the basics" and as such, checks that we have various fixed hard-coded object IDs. The tests relying on these assertions have been marked with the SHA1 prerequisite, as they will obviously not function in their current form with SHA-256. Use the test_oid helper to update these assertions and provide values for both SHA-1 and SHA-256. These object IDs were synthesized using a set of scripts that created the objects for both SHA-1 and SHA-256 using the same method to ensure that they are indeed the correct values. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
brian m. carlson	cdd1e17f87	t0000: use hash translation table If the hash we're using is 32 bytes in size, attempting to insert a 20-byte object name won't work. Since these are synthesized objects that are almost all zeros, look them up in a translation table. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
brian m. carlson	2c02b110da	t: add test functions to translate hash-related values Add several test functions to make working with various hash-related values easier. Add test_oid_init, which loads common hash-related constants and placeholder object IDs from the newly added files in t/oid-info. Provide values for these constants for both SHA-1 and SHA-256. Add test_oid_cache, which accepts data on standard input in the form of hash-specific key-value pairs that can be looked up later, using the same format as the files in t/oid-info. Document this format in a t/oid-info/README directory so that it's easier to use in the future. Add test_oid, which is used to specify look up a per-hash value (produced on standard output) based on the key specified as its argument. Usually the data to be looked up will be a hash-related constant (such as the size of the hash in binary or hexadecimal), a well-known or placeholder object ID (such as the all-zeros object ID or one consisting of "deadbeef" repeated), or something similar. For these reasons, test_oid will usually be used within a command substitution. Consequently, redirect the error output to standard error, since otherwise it will not be displayed. Add test_detect_hash, which currently only detects SHA-1, and test_set_hash, which can be used to set a different hash algorithm for test purposes. In the future, test_detect_hash will learn to actually detect the hash depending on how the testsuite is to be run. Use the local keyword within these functions to avoid overwriting other shell variables. We have had a test balloon in place for a couple of releases to catch shells that don't have this keyword and have not received any reports of failure. Note that the varying usages of local used here are supported by all common open-source shells supporting the local keyword. Test these new functions as part of t0000, which also serves to demonstrate basic usage of them. In addition, add documentation on how to format the lookup data and how to use the test functions. Implement two basic lookup charts, one for common invalid or synthesized object IDs, and one for various facts about the hash function in use. Provide versions of the data for both SHA-1 and SHA-256. Since we use shell variables for storage, names used for lookup can currently consist only of shell identifier characters. If this is a problem in the future, we can hash the names before use. Improved-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	ec10b018e7	tests: use 'test_must_be_empty' instead of '! test -s' Using 'test_must_be_empty' is preferable to '! test -s', because it gives a helpful error message if the given file is unexpectedly not empty, while the latter remains completely silent. Furthermore, it also catches cases when the given file unexpectedly does not exist at all. This patch was basically created by: sed -i -e 's/! test -s/test_must_be_empty/' t[0-9]*.sh with the following notable exceptions: - The '! test -s' check in '.gitmodules ignore=dirty suppresses submodules with untracked content' in 't7508-status.sh' is left as-is, because it's bogus and, therefore, it's subject of a dedicated patch. - The '! test -s' checks in 't9131-git-svn-empty-symlink.sh' and 't9135-git-svn-moved-branch-empty-file.sh' are immediately preceeded by a 'test -f' to ensure that the files exist in the first place. 'test_must_be_empty' ensures that as well, so those 'test -f' commands are removed as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Eric Sunshine	75651fd783	t0000-t0999: fix broken &&-chains Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
brian m. carlson	5b9ba9bd80	t0000: annotate with SHA1 prerequisite Since this is a core test that tests basic functionality, annotate the assertions that have dependencies on SHA-1 with the appropriate prerequisite. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Michael Haggerty	01d3a526ad	t0000: check whether the shell supports the "local" keyword Add a test balloon to see if we get complaints from anybody who is using a shell that doesn't support the "local" keyword. If so, this test can be reverted. If not, we might want to consider using "local" in shell code throughout the git code base. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Nguyễn Thái Ngọc Duy	f9e7d9f8c3	test-lib.sh: introduce and use $EMPTY_TREE This is a special SHA1. Let's keep it at one place, easier to replace later when the hash change comes, easier to recognize. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Li Peng	832c0e5e63	typofix: assorted typofixes in comments, documentation and messages Many instances of duplicate words (e.g. "the the path") and a few typoes are fixed, originally in multiple patches. wildmatch: fix duplicate words of "the" t: fix duplicate words of "output" transport-helper: fix duplicate words of "read" Git.pm: fix duplicate words of "return" path: fix duplicate words of "look" pack-protocol.txt: fix duplicate words of "the" precompose-utf8: fix typo of "sequences" split-index: fix typo worktree.c: fix typo remote-ext: fix typo utf8: fix duplicate words of "the" git-cvsserver: fix duplicate words Signed-off-by: Li Peng <lip@dtdream.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Jeff King	99094a7ad4	t: fix trivial &&-chain breakage These are tests which are missing a link in their &&-chain, but during a setup phase. We may fail to notice failure in commands that build the test environment, but these are typically not expected to fail at all (but it's still good to double-check that our test environment is what we expect). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	10 years ago
Ramsay Jones	7e28c16fdb	t0000-*.sh: fix the GIT_SKIP_TESTS sub-tests Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>	11 years ago
Ilya Bobyr	0445e6f0a1	test-lib: '--run' to run only specific tests Allow better control of the set of tests that will be executed for a single test suite. Mostly useful while debugging or developing as it allows to focus on a specific test. Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	11 years ago
Ilya Bobyr	ef2ac68def	test-lib: tests skipped by GIT_SKIP_TESTS say so We used to show "(missing )" next to tests skipped because they are specified in GIT_SKIP_TESTS. Use "(GIT_SKIP_TESTS)" instead. Plus tests that check basic GIT_SKIP_TESTS functions. Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	11 years ago
Jeff King	738a8beac4	t0000: drop "known breakage" test Having a simulated "known breakage" test means that the test suite will always tell us there is a bug to be fixed, even though it is only simulated. The right way to test this is in a sub-test, that can also check that we provide the correct exit status and output. Fortunately, we already have such a test (added much later by `5ebf89e`). We could arguably get rid of the simulated success test immediately above, as well, as it is also redundant with the tests added in `5ebf89e`. However, it does not have the annoying behavior of the "known breakage" test. It may also be easier to debug if the test suite is truly broken, since it is not a test-within-a-test, as the later tests are. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	11 years ago

1 2 3

131 Commits (0d7131763e7762dc34c407f32e3c63123001a292)