Otherwise the created test repositories will be affected by users ~/.gitconfig.
For example, setting core.logAllrefupdates in users config will make all
calls to "git config --unset core.logAllrefupdates" fail which will break
the first test which uses the statement and expects it to succeed.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In commit 95a1d12e9b ("tests: scrub environment of GIT_* variables") all
environment variables starting with "GIT_" were unset for the tests using
a perl script rather than unsetting them one by one. Only three exceptions
were made to make them work as before: "GIT_TRACE*", "GIT_DEBUG*" and
"GIT_USE_LOOKUP".
Unfortunately some environment variables used by the test framework itself
were not added to the exceptions and thus stopped working when given
before the make command instead of after it. Those are:
- GIT_NOTES_TIMING_TESTS
- GIT_PATCHID_TIMING_TESTS
- GIT_PROVE_OPTS
- GIT_REMOTE_SVN_TEST_BIG_FILES
- GIT_SKIP_TESTS
- GIT_TEST*
- GIT_VALGRIND_OPTIONS
I noticed that when skipping a test the way I was used to suddenly failed:
GIT_SKIP_TESTS='t1234' GIT_TEST_OPTS='--root=/dev/shm' make -j10 test
This should work according to t/README, but didn't anymore, so let's fix
that by adding them to the exception list. And to avoid having a long
regexp put the exceptions in a separate variable using nicer formatting.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Thanks-to: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Variables from the inherited environment that are meaningful to git
can break tests in undesirable ways. For example,
GIT_PAGER=more sh t5400-send-pack.sh -v -i
hangs. So unset all environment variables in the GIT_ namespace in
test-lib, with a few exceptions:
- GIT_TRACE* are useful for tracking down bugs exhibited by a failing
test;
- GIT_DEBUG* are GIT_TRACE variables by another name, practically
speaking. They should probably be tweaked to follow the
GIT_TRACE_foo scheme and use trace_printf machinery some time.
- GIT_USE_LOOKUP from v1.5.6-rc0~134^2~1 (sha1-lookup: more memory
efficient search in sorted list of SHA-1, 2007-12-29) is about
trying an alternate implementation strategy rather than changing
semantics and it can be useful to compare performance with and
without it set.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that test-lib sets $HOME to protect against pollution from user
settings, GIT_CONFIG_NOGLOBAL is not needed for use by the test
suite any more. And as luck would have it, a quick code search
reveals no other users in the wild.
This patch does not affect GIT_CONFIG_NOSYSTEM, which is still
needed.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Set GIT_ATTR_NOSYSTEM in test-lib to make tests more reliable in two
ways:
- an invalid GIT_ATTR_NOSYSTEM setting should not cause tests to fail
with "fatal: bad config value for 'GIT_ATTR_NOSYSTEM'".
- /etc/gitattributes should not change the outcome of tests.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Improved-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After v0.99.7~99 (Retire support for old environment variables,
2005-09-09), there is no more need to unset a stray AUTHOR_NAME
variable that might have entered the test environment.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tweak the GETTEXT_POISON facility so it is activated at run time
instead of compile time. If the GIT_GETTEXT_POISON environment
variable is set, _(msg) will result in gibberish as before; but if the
GIT_GETTEXT_POISON variable is not set, it will return the message for
human-readable output. So the behavior of mistranslated and
untranslated git can be compared without rebuilding git in between.
For simplicity we always set the GIT_GETTEXT_POISON variable in tests.
This does not affect builds without the GETTEXT_POISON compile-time
option set, so non-i18n git will not be slowed down.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a new GETTEXT_POISON compile-time parameter to make _(msg) always
return gibberish. So now you can run
make GETTEXT_POISON=YesPlease
to get a copy of git that functions correctly (one hopes) but produces
output that is in nobody's native language at all.
This is a debugging aid for people who are working on the i18n part of
the system, to make sure that they are not marking plumbing messages
that should never be translated with _().
As new strings get marked for translation, naturally a number of tests
will be broken in this mode. Tests that depend on output from
Porcelain will need to be marked with the new C_LOCALE_OUTPUT test
prerequisite. Newly failing tests that do not depend on output from
Porcelain would be bugs due to messages that should not have been
marked for translation.
Note that the string we're using ("# GETTEXT POISON #") intentionally
starts the pound sign. Some of Git's tests such as
t3404-rebase-interactive.sh rely on interactive editing with a fake
editor, and will needlessly break if the message doesn't start with
something the interactive editor considers a comment.
A future patch will fix fix the underlying cause of that issue by
adding "#" characters to the commit advice automatically.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These variables should generally not be set in one's
environment, but they do get set by rebase, which means
doing an interactive rebase like:
pick abcd1234 foo
exec make test
will cause false negatives in the test suite.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The BSLASHPSPEC tests (11-13) fail on cygwin, since you can't
create files containing an backslash character in the name.
In order to skip these tests, we simply stop (incorrectly)
asserting the BSLASHPSPEC prerequisite in test-lib.sh.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The tests using the fuzz_conflict helper function (tests 5-6)
fail on cygwin in the same way they used to on MinGW, prior
to commit ca02ad3. The solution is also the same; passing the
-b (--binary) option to sed, using the SED_OPTIONS variable.
We introduce a new prerequisite SED_STRIPS_CR to use in the
conditional initialisation of SED_OPTIONS, rather than MINGW.
The new prerequisite is set in test-lib.sh for both MinGW and
Cygwin.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
POSIX awk seems to explicitly not support hexadecimal escape sequences.
From http://pubs.opengroup.org/onlinepubs/009695399/:
Regular expressions in awk have been extended somewhat...
One sequence that is not supported is hexadecimal value escapes
beginning with '\x'.
This affects the awk on IRIX 6.5, and causes t4015.56 to fail.
Use octal instead.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests check their output with code like the following:
test "$(git ls-files -u B | wc -l)" -eq 3 || {
echo "BAD: should have left stages for B"
return 1
}
The verbose failure condition is used because test, unlike
diff, does not print any useful information about the
nature of the failure when it fails.
Introduce a test_line_count function to help. If used like
git ls-files -u B >output &&
test_line_count -eq 3 output
it will produce output like
test_line_count: line count for output !-eq 3
100644 b023018cabc396e7692c70bbf5784a93d3f738ab 2 hi.c
100644 45b983be36b73c0788dc9cbcb76cbb80fc7bb057 3 hi.c
on failure.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Enhance the test_decode_color function to handle all common color codes,
including background colors and escapes that contain multiple codes.
This change necessitates changing <WHITE> to <BOLD>, so update t4034
as well.
This change is necessary for the next commit in order to test
background colors properly.
Signed-off-by: Kevin Ballard <kevin@sb.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is plumbing to prepare helpers like test_terminal to notice buggy
test scripts that do not declare all of the necessary prerequisites.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code paths for showing commits in "git log" and "git
rev-list --graph" correctly handle embedded NULs by looking
only at the resulting strbuf's length, and never treating it
as a C string. The code path for regular rev-list, however,
used printf("%s"), which resulted in truncated output. This
patch uses fwrite instead, like the --graph code path.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change test_expect_code to be a normal test command instead of a
top-level command.
As a top-level command it would fail in cases like:
test_expect_code 1 'phoney' '
foo && bar && (exit 1)
'
Here the test might incorrectly succeed if "foo" or "bar" happened to
fail with exit status 1. Instead we now do:
test_expect_success 'phoney' '
foo && bar && test_expect_code 1 "(exit 1)"
'
Which will only succeed if "foo" and "bar" return status 0, and "(exit
1)" returns status 1. Note that test_expect_code has been made slightly
noisier, as it reports the exit code it receives even upon success.
Some test code in t0000-basic.sh relied on the old semantics of
test_expect_code to test the test_when_finished command. I've
converted that code to use an external test similar to the TODO test I
added in v1.7.3-rc0~2^2~3.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
By default, MSYS grep reads in text-mode and converts CRLF into LF line
endings. For testing HTTP use binary mode (-U) as checking is done for
CR in HTTP headers
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
These two tests fail on msysGit because /dev/null is an alias for nul on
Windows and when reading the value back from git config the alias does
not match the real filename. Also the HOME environment variable has a
unix-style path but git returns a native equivalent path for '~'. As
these are platform-dependent equivalent results it seems simplest to
skip the test entirely.
Moves the NOT_MINGW prereq from t5503 into the test library.
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
The same pattern is used in many tests, and makes it easy for new ones to
rely on $HOME being a trashable, clean, directory.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the test_create_repo code added in v1.2.2~6 to use a subshell
instead of keeping track of the old working directory and cd-ing back
when it's done.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a test has no prerequisites satisfied (the usual case), instead
of "missing THING of THING", just say "missing THING". This does not
affect the output when a test is skipped due to a missing
prerequisites if another prerequisite is satisfied.
For example: instead of
ok 8 # skip notes work (missing EXPENSIVE of EXPENSIVE)
ok 9 # skip notes timing with /usr/bin/time (missing EXPENSIVE of USR_BIN_TIME,EXPENSIVE)
write
ok 8 # skip notes work (missing EXPENSIVE)
ok 9 # skip notes timing with /usr/bin/time (missing EXPENSIVE of USR_BIN_TIME,EXPENSIVE)
Cc: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tests that test the test-lib.sh itself need to be executed in the
dynamically created trash directory, so we can't assume
$TEST_DIRECTORY is ../ for those.
As a side benefit this change also makes it easy for us to move the
t/*.sh tests into subdirectories if we ever want to do that.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change code that used $TEST_DIRECTORY/.. to use $GIT_BUILD_DIR
instead, the two are equivalent, but the latter is easier to read.
This required moving the assignment od GIT_BUILD_DIR to earlier in the
test-lib.sh file.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the redundant calls to $(pwd) to use $TEST_DIRECTORY
instead. None of these were being executed after we cd'd somewhere
else so they weren't actually needed.
This also makes it easier to add support for overriding the test
library location and run tests in a different directory than t/.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce a new variable $GIT_BUILD_DIR which can be used to locate
data that resides under the build directory, and use that instead.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Detect and report hard-to-notice spelling mistakes like
test_might_fail "git config --unset whatever"
(the extra quotes prevent the shell from running git as intended;
instead, the shell looks for a "git config --unset whatever" file).
Cc: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Let test_might_fail say something about its failures for consistency
with test_must_fail.
Cc: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The point of it is to run a command that produces failure. A
missing command is more likely an error in the test script
(e.g., using 'test_must_fail "command with arguments"', or
relying on a missing command).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Because test_must_fail fails when a command succeeds, the
command frequently does not produce any output (since, after
all, it thought it was succeeding). So let's have
test_must_fail itself report that a problem occurred.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The support for multiple test prerequisites added by me in "test-lib:
Add support for multiple test prerequisites" was broken.
The for iterated over each prerequisite and returned true/false within
a case statement, but since it missed a return statement only the last
prerequisite in the list of prerequisites was ever considered, the
rest were ignored.
Fix that by changing the test_have_prereq code to something less
clever that keeps a count of the total prereqs and the ones we have
and compares the count at the end.
This comes with the added advantage that it's easy to list the missing
prerequisites in the test output, implement that while I'm at it.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests depend on not being able to write to files after chmod
-w. This doesn't work when running the tests as root.
Change test-lib.sh to test if this works, and if so it sets a new
SANITY test prerequisite. The tests that use this previously failed
when run under root.
There was already a test for this in t3600-rm.sh, added by Junio C
Hamano in 2283645 in 2006. That check now uses the new SANITY
prerequisite.
Some of this was resurrected from the "Tests in Cygwin" thread in May
2009:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the test output to print needed prerequisites as part of the
TAP. This makes it easy to see at a glance why a test was
skipped. Before:
ok 7 # skip <message>
ok 9 # skip <message>
After:
ok 7 # skip <message> (prereqs: DONTHAVEIT)
ok 9 # skip <message> (prereqs: HAVEIT,DONTHAVEIT)
This'll also be useful for smoke testing output, where the developer
reading the output may not be familiar with the system where tests are
being skipped.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the test_have_prereq function in test-lib.sh to support a
comma-separated list of prerequisites. This is useful for tests that
need e.g. both POSIXPERM and SANITY.
The implementation was stolen from Junio C Hamano and Johannes Sixt,
the tests and documentation were not. See the "Tests in Cygwin" thread
in May 2009 for the originals:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
TAP harnesses don't need to read test-results/*, since they keep track
of the number of passing/failing tests internally. Skip the generation
of these files when HARNESS_ACTIVE is set.
It's now possible to run the Git test suite without writing anything
to the t/ directory at all if you use a TAP harness and the --root
switch:
cd t
sudo mount -t tmpfs none /tmp/memory -o size=300m
prove -j9 ./t[0-9]*.sh :: --root=/tmp/memory
The I/O that the ~500 test-results/* files contributed was very
minimal, but I thought this was worth mentioning.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The helper functions are implemented, documented, and used in a few
places to validate them, but not everywhere to avoid useless code churn.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The --no-python option was added to test-lib.sh by Johannes Schindelin
in early 2006 in abb7c7b3. It was later turned into a no-op by Junio C
Hamano in 7cdbff14 the same year.
Over three years is long enough before removing this old wart which
was retained for backwards compatibility. Our tests have been using
NO_PYTHON and "test_have_prereq PYTHON" for a long time now.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Running the tests with --quiet under a TAP harness will always fail,
since a TAP harness always needs actual test output to go along with
the plan that's being emitted.
Change the test-lib.sh to ignore the --quiet option under
HARNESS_ACTIVE to work around this. Then users that have --quiet in
their GIT_TEST_OPTS can run tests under prove(1) without everything
breaking.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Place setup commands in test_expect_success blocks. This makes the
rare event of the setup commands breaking on some platform easier to
diagnose, and more importantly, it visually distinguishes where
each test begins and ends.
Instead of running test -z against the result of "git diff" command
substitution, use "git diff --exit-code", to improve output when
running with the "-v" option.
Use test_cmp in place of "test $(foo) = $(bar)" for similar reasons.
Remove whitespace after the > and < redirection operators for
consistency with other tests.
The order of arguments to test_cmp is "test_cmp expected actual".
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current make target 'aggregate-results' scanned all files matching
test-results/t*-*. Normally these are only the test counts (and the
exit values, which are ignored), but with --tee the suite also dumps
all output. Furthermore, with --verbose t1450 contains several lines
starting with "broken link from ..." which matches the criteria used
by aggregate-results.sh.
Rename the counts output files to *.counts, and only scan those.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Acked-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make the output TAP compliant for tests skipped on request (GIT_SKIP_TESTS).
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
04ece59 (GIT_SKIP_TESTS: allow users to omit tests that are known to break, 2006-12-28)
introduced GIT_SKIP_TESTS, and since then we have had two nested loops
iterating over GIT_SKIP_TESTS with the same loop variable.
Reduce this to one loop.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests in the testsuite will emit a line that doesn't end with a
newline, right before we're about to output "ok" or "not ok". This
breaks the TAP output with "Tests out of sequence" errors since a TAP
harness can't understand this:
ok 1 - A test
[some output here]ok 2 - Another test
ok 3 - Yet another test
Work around it by emitting an empty line before we're about to say
"ok" or "not ok", but only if we're running under --verbose and
HARNESS_ACTIVE=1 is set, which'll only be the case when running under
a harnesses like prove(1).
I think it's better to do this than fix each tests by adding `&& echo'
everywhere. More tests might be added that break TAP in the future,
and a human isn't going to look at the extra whitespace, since
HARNESS_ACTIVE=1 always means a harness is reading it.
The tests that had issues were:
t1007, t3410, t3413, t3409, t3414, t3415, t3416, t3412, t3404,
t5407, t7402, t7003, t9001
With this workaround the entire test suite runs without errors under:
prove -j 10 ./t[0-9]*.sh :: --verbose
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Before TAP we just ran the Perl test and assumed that it failed if
nothing was printed on STDERR. Continue doing that, but introduce a
`test_external_has_tap' variable which tests can set to indicate that
they're outputting TAP.
If it's set we won't output a test plan, but trust the external test
to do so. That way we can make external tests work with a TAP harness,
but still maintain compatibility with test-lib's own way of tracking
tests through the test-results directory.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
$ prove ./t0005-signals.sh
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)
Result: PASS
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
All this patch does is re-arrange the output a bit so that it conforms
with the TAP spec, everything that the test suite did before continues
to work. That includes aggregating results in t/test-results/, the
--verbose, --debug and other options for tests, and the test color
output.
TAP harnesses ignore everything that they don't know about, so running
the tests with --verbose works:
$ prove ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh .. Terminated
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU)
Result: PASS
Just supply the -v option to prove itself to get all the verbose
output that it suppresses:
$ prove -v ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh ..
Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/
expecting success:
test-sigchain >actual
case "$?" in
143) true ;; # POSIX w/ SIGTERM=15
3) true ;; # Windows
*) false ;;
esac &&
test_cmp expect actual
Terminated
ok 1 - sigchain works
# passed all 1 test(s)
1..1
ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU)
Result: PASS
As a further example, consider this test script that uses a lot of
test-lib.sh features by Jakub Narebski:
#!/bin/sh
test_description='this is a sample test.
This test is here to see various test outputs.'
. ./test-lib.sh
say 'diagnostic message'
test_expect_success 'true test' 'true'
test_expect_success 'false test' 'false'
test_expect_failure 'true test (todo)' 'true'
test_expect_failure 'false test (todo)' 'false'
test_debug 'echo "debug message"'
test_done
The output of that was previously:
* diagnostic message # yellow
* ok 1: true test
* FAIL 2: false test # bold red
false
* FIXED 3: true test (todo)
* still broken 4: false test (todo) # bold green
* fixed 1 known breakage(s) # green
* still have 1 known breakage(s) # bold red
* failed 1 among remaining 3 test(s) # bold red
But is now:
diagnostic message # yellow
ok 1 - true test
not ok - 2 false test # bold red
# false
ok 3 - true test (todo) # TODO known breakage
not ok 4 - false test (todo) # TODO known breakage # bold green
# fixed 1 known breakage(s) # green
# still have 1 known breakage(s) # bold red
# failed 1 among remaining 3 test(s) # bold red
1..4
All the coloring is preserved when the test is run manually. Under
prove(1) the test performs as expected, even with --debug and
--verbose options:
$ prove ./example.sh :: --debug --verbose
./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/4 subtests
(1 TODO test unexpectedly succeeded)
Test Summary Report
-------------------
./example.sh (Wstat: 256 Tests: 4 Failed: 1)
Failed test: 2
TODO passed: 3
Non-zero exit status: 1
Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.00 cusr 0.01 csys = 0.03 CPU)
Result: FAIL
The TAP harness itself doesn't get confused by the color output, they
aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Otherwise running individual tests from t/ directory may lack the definition
of $DIFF, $GIT_TEST_CMP and friends.
Noticed and initial patch provided by Thomas Rast, alternative solution
suggested by Brandon Casey, which this patch implements.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Thomas Rast <trast@student.ethz.ch>
In 3bf7886 (test-lib: Let tests specify commands to be run at end of
test, 2010-05-02), the git test harness learned to run cleanup
commands unconditionally at the end of a test. During each test,
the intended cleanup actions are collected in the test_cleanup variable
and evaluated. That variable looks something like this:
eval_ret=$?; clean_something && (exit "$eval_ret")
eval_ret=$?; clean_something_else && (exit "$eval_ret")
eval_ret=$?; final_cleanup && (exit "$eval_ret")
eval_ret=$?
All cleanup actions are run unconditionally but if one of them fails
it is properly reported through $eval_ret.
On FreeBSD, unfortunately, $? is set at the beginning of an ‘eval’
to 0 instead of the exit status of the previous command. This results
in tests using test_expect_code appearing to fail and all others
appearing to pass, unless their cleanup fails. Avoid the problem by
setting eval_ret before the ‘eval’ begins.
Thanks to Jeff King for the explanation.
Cc: Jeff King <peff@peff.net>
Cc: Johannes Sixt <j6t@kdbg.org>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Certain actions can imply that if the test fails early, recovery from
within other tests is too much to expect:
- creating unwritable directories, like the EACCESS test in t0001-init
- setting unusual configuration, like user.signingkey in t7004-tag
- crashing and leaving the index lock held, like t3600-rm once did
Some test scripts work around this by running cleanup actions outside
the supervision of the test harness, with the unfortunate consequence
that those commands are not appropriately echoed and their output not
suppressed. Others explicitly save exit status, clean up, and then
reset the exit status within the tests, which has excellent behavior
but makes the tests hard to read. Still others ignore the problem.
Allow tests a fourth option: by calling this function, tests can
stack up commands they would like to be run to clean up.
Commands passed to test_when_finished during a test are
unconditionally run in the test environment immediately before the
test is completed, in last-in-first-out order. If some cleanup
command fails, then the other cleanup commands are still run before
the failure is reported and the test script allowed to continue.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Alex Riesen