kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Brandon Casey	971ecbd1f8	t/README: clarify test_must_fail description Some have found the wording of the description to be somewhat ambiguous with respect to when it is desirable to use test_must_fail instead of "! <git-command>". Tweak the wording somewhat to hopefully clarify that it is _because_ test_must_fail can detect segmentation fault that it is desirable to use it instead of "! <git-command>". Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Jonathan Nieder	c9667456d2	t/README: document more test helpers There is no documentation in t/README for test_must_fail, test_might_fail, test_cmp, or test_when_finished. Reported-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Junio C Hamano	6fd45295ae	t/README: proposed rewording... Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	20873f45e7	t/README: Document the do's and don'ts of tests Add a "Do's, don'ts & things to keep in mind" subsection to the "Writing Tests" documentation. Much of this is based on Junio C Hamano's "Test your stuff" section in <7vhbkj2kcr.fsf@alter.siamese.dyndns.org>. I turned it into a list of do's and don'ts to make it easier to skim it, and integrated my note that a TAP harness will get confused if you print "ok" or "not ok" at the beginning of a line. Thad had to be fixed in `335f87871f` when TAP support was introduced. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	b5500d16cd	t/README: Add a section about skipping tests Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	97d9fd925b	t/README: Document test_expect_code test_expect_code (which was introduced in `d3bfdb75`) never had any documentation. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	2fac6a4b93	t/README: Document test_external* There was do documentation for the test_external_without_stderr and test_external functions. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	9a897893a7	t/README: Document the prereq functions, and 3-arg test_* There was no documentation for the test_set_prereq and test_have_prereq functions, or the three-arg form of test_expect_success and test_expect_failure. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	85b0b34ea4	t/README: Typo: paralell -> parallel Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	e1ca1c9d9b	t/README: The trash is in 't/trash directory.$name' There's a unique trash directory for each test, not a single directory as the previous documentation suggested. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Ævar Arnfjörð Bjarmason	5099b99d25	test-lib: Adjust output to be valid TAP format TAP, the Test Anything Protocol, is a simple text-based interface between testing modules in a test harness. test-lib.sh's output was already very close to being valid TAP. This change brings it all the way there. Before: $ ./t0005-signals.sh * ok 1: sigchain works * passed all 1 test(s) And after: $ ./t0005-signals.sh ok 1 - sigchain works # passed all 1 test(s) 1..1 The advantage of using TAP is that any program that reads the format (a "test harness") can run the tests. The most popular of these is the prove(1) utility that comes with Perl. It can run tests in parallel, display colored output, format the output to console, file, HTML etc., and much more. An example: $ prove ./t0005-signals.sh ./t0005-signals.sh .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU) Result: PASS prove(1) gives you human readable output without being too verbose. Running the test suite in parallel with `make test -j15` produces a flood of text. Running them with `prove -j 15 ./t[0-9].sh` makes it easy to follow what's going on. All this patch does is re-arrange the output a bit so that it conforms with the TAP spec, everything that the test suite did before continues to work. That includes aggregating results in t/test-results/, the --verbose, --debug and other options for tests, and the test color output. TAP harnesses ignore everything that they don't know about, so running the tests with --verbose works: $ prove ./t0005-signals.sh :: --verbose --debug ./t0005-signals.sh .. Terminated ./t0005-signals.sh .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU) Result: PASS Just supply the -v option to prove itself to get all the verbose output that it suppresses: $ prove -v ./t0005-signals.sh :: --verbose --debug ./t0005-signals.sh .. Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/ expecting success: test-sigchain >actual case "$?" in 143) true ;; # POSIX w/ SIGTERM=15 3) true ;; # Windows ) false ;; esac && test_cmp expect actual Terminated ok 1 - sigchain works # passed all 1 test(s) 1..1 ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU) Result: PASS As a further example, consider this test script that uses a lot of test-lib.sh features by Jakub Narebski: #!/bin/sh test_description='this is a sample test. This test is here to see various test outputs.' . ./test-lib.sh say 'diagnostic message' test_expect_success 'true test' 'true' test_expect_success 'false test' 'false' test_expect_failure 'true test (todo)' 'true' test_expect_failure 'false test (todo)' 'false' test_debug 'echo "debug message"' test_done The output of that was previously: * diagnostic message # yellow * ok 1: true test * FAIL 2: false test # bold red false * FIXED 3: true test (todo) * still broken 4: false test (todo) # bold green * fixed 1 known breakage(s) # green * still have 1 known breakage(s) # bold red * failed 1 among remaining 3 test(s) # bold red But is now: diagnostic message # yellow ok 1 - true test not ok - 2 false test # bold red # false ok 3 - true test (todo) # TODO known breakage not ok 4 - false test (todo) # TODO known breakage # bold green # fixed 1 known breakage(s) # green # still have 1 known breakage(s) # bold red # failed 1 among remaining 3 test(s) # bold red 1..4 All the coloring is preserved when the test is run manually. Under prove(1) the test performs as expected, even with --debug and --verbose options: $ prove ./example.sh :: --debug --verbose ./example.sh .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/4 subtests (1 TODO test unexpectedly succeeded) Test Summary Report ------------------- ./example.sh (Wstat: 256 Tests: 4 Failed: 1) Failed test: 2 TODO passed: 3 Non-zero exit status: 1 Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.00 cusr 0.01 csys = 0.03 CPU) Result: FAIL The TAP harness itself doesn't get confused by the color output, they aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Thomas Rast	0d4dbcd35e	t/README: document --root option We've had this option since `f423ef5` (tests: allow user to specify trash directory location, 2009-08-09). Make it easier to look up :-) Signed-off-by: Thomas Rast <trast@student.ethz.ch> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Matthew Ogilvie	e4597aae65	run test suite without dashed git-commands in PATH Only put bin-wrappers in the PATH (not GIT_EXEC_PATH), to emulate the default installed user environment, and ensure all the programs run correctly in such an environment. This is now the default, although it can be overridden with a --with-dashes test option when running tests. Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Matthew Ogilvie	e160da7f60	t/README: Document GIT_TEST_INSTALLED and GIT_TEST_EXEC_PATH These were added without documentation in 2009-03-16 (`6720721`). Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago
Johannes Schindelin	3da9365234	Tests: let --valgrind imply --verbose and --tee It does not make much sense to run the (expensive) valgrind tests and not look at the output. To prevent output from scrolling out of reach, the parameter --tee is implied, too. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
Johannes Schindelin	44138559e8	test-lib.sh: optionally output to test-results/$TEST.out, too When tests are run in parallel and a few tests fail, it does not help that the output of the terminal is totally confusing, as you rarely know which test which line came from. So introduce the option '--tee' which triggers that the output of the tests will be written to t/test-results/$TEST.out in addition to the terminal, where $TEST is the basename of the script. Unfortunately, there seems to be no way to redirect a given file descriptor to a specified subprocess in POSIX shell, only redirection to a file is supported via 'exec > $FILE'. At least with bash, one might think that 'exec >($COMMAND)' would work as intended, but it does not. The common way to work around the lack of proper tools support is to work with named pipes, alas, one of our most beloved platforms does not really support named pipes. Besides, we would need a pipe for every script, as the whole point of this patch is to allow parallel execution. Therefore, we handle the redirection in the following way: when '--tee' was passed to the test script, the variable GIT_TEST_TEE_STARTED is set (to avoid triggering that code path again) and the script is started _again_, in a subshell, redirected to the command "tee". Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
Johannes Schindelin	4e1be63c3b	Add valgrind support in test scripts This patch adds the ability to use valgrind's memcheck tool to diagnose memory problems in Git while running the test scripts. It requires valgrind 3.4.0 or newer. It works by creating symlinks to a valgrind script, which have the same name as our Git binaries, and then putting that directory in front of the test script's PATH as well as set GIT_EXEC_PATH to that directory. Git scripts are symlinked from that directory directly. That way, Git binaries called by Git scripts are valgrinded, too. Valgrind can be used by specifying "GIT_TEST_OPTS=--valgrind" in the make invocation. Any invocation of git that finds any errors under valgrind will exit with failure code 126. Any valgrind output will go to the usual stderr channel for tests (i.e., /dev/null, unless -v has been specified). If you need to pass options to valgrind -- you might want to run another tool than memcheck, for example -- you can set the environment variable GIT_VALGRIND_OPTIONS. A few default suppressions are included, since libz seems to trigger quite a few false positives. We'll assume that libz works and that we can ignore any errors which are reported there. Note: it is safe to run the valgrind tests in parallel, as the links in t/valgrind/bin/ are created using proper locking. Initial patch and all the hard work by Jeff King. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
Johannes Schindelin	008849689e	test-lib.sh: introduce test_commit() and test_merge() helpers Often we just need to add a commit with a given (short) name, that will be tagged with the same name. Now, relatively complicated graphs can be constructed easily and in a clear fashion: test_commit A && test_commit B && git checkout A && test_commit C && test_merge D B will construct this graph: A - B \ \ C - D For simplicity, files whose name is the lower case version of the commit message (to avoid a warning about ambiguous names) will be committed, with the corresponding commit messages as contents. If you need to provide a different file/different contents, you can use the more explicit form test_commit $MESSAGE $FILENAME $CONTENTS Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
Jakub Narebski	fbd458a3f6	t/README: Add 'Skipping Tests' section below 'Running Tests' Add description of GIT_SKIP_TESTS variable, taken almost verbatim (adjusting for conventions in t/README) from the commit message in `04ece59` (GIT_SKIP_TESTS: allow users to omit tests that are known to break) Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	17 years ago
Lea Wiemann	5e2c08c6f0	test-lib.sh: add --long-tests option Add a --long-tests option to test-lib.sh, which enables tests to selectively run more exhaustive (longer running, potentially brute-force) tests. Such exhaustive tests would only be useful if one works on the specific module that is being tested -- for a general "cd t/; make" to check whether everything is OK, such exhaustive tests shouldn't be run by default since the longer it takes to run the tests, the less often they are actually run. Signed-off-by: Lea Wiemann <LeWiemann@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	17 years ago
Brandon Casey	9231e3a953	t/Makefile: "trash" directory was renamed recently Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	17 years ago
Junio C Hamano	41ac414ea2	Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test should succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>	17 years ago
Nicolas Pitre	5c94f87e6b	use 'init' instead of 'init-db' for shipped docs and tools While 'init-db' still is and probably will always remain a valid git command for obvious backward compatibility reasons, it would be a good idea to move shipped tools and docs to using 'init' instead. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>	18 years ago
Jakub Narebski	8757749ecb	Add info about new test families (8 and 9) to t/README Signed-off-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <junkio@cox.net>	18 years ago
Junio C Hamano	8f4a9b62ee	t/README: start testing porcelainish Signed-off-by: Junio C Hamano <junkio@cox.net>	19 years ago
Junio C Hamano	215a7ad1ef	Big tool rename. As promised, this is the "big tool rename" patch. The primary differences since 0.99.6 are: (1) git-*-script are no more. The commands installed do not have any such suffix so users do not have to remember if something is implemented as a shell script or not. (2) Many command names with 'cache' in them are renamed with 'index' if that is what they mean. There are backward compatibility symblic links so that you and Porcelains can keep using the old names, but the backward compatibility support is expected to be removed in the near future. Signed-off-by: Junio C Hamano <junkio@cox.net>	20 years ago
Junio C Hamano	776566000f	[PATCH] Prevent t6000 series from dropping useless sed.script in t/ The Makefile in the test suite directory considers any file matching t[0-9][0-9][0-9][0-9]-*.sh as the top-level test script to be executed. Unfortunately this was not documented, and the common test library, t6000-lib.sh was named to match that pattern. This caused t6000-lib.sh to be called from Makefile as the top-level program, causing it to leave t/sed.script file behind. Rename it to t6000lib.sh to prevent this, and document the naming convention a bit more clearly. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	14cd1ff396	[PATCH 4/4] Trivial test harness fixes. The documentation of the test harness still refer to old numbering and also contains an obvious typo. Also "make test" should be run after making sure we have built all binaries, since test is designed to test the newly built ones. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Petr Baudis <pasky@ucw.cz>	20 years ago
Petr Baudis	f50c9f76ca	Rename some test scripts and describe the naming convention First digit: "family", e.g. the absolute basics and global stuff (0), the basic db-side commands (read-tree, write-tree, commit-tree), the basic working-tree-side commands (checkout-cache, update-cache), the other basic commands (ls-files), the diff commands, the pull commands, exporting commands, revision tree commands... Second digit: the particular command we are testing Third digit: (optionally) the particular switch or group of switches we are testing Freeform part: commandname-details Described in the README. mv t1000-checkout-cache.sh t2000-checkout-cache-clash.sh mv t1001-checkout-cache.sh t2001-checkout-cache-clash.sh mv t0200-update-cache.sh t2010-update-cache-badpath.sh mv t0400-ls-files.sh t3000-ls-files-others.sh mv t0500-ls-files.sh t3010-ls-files-killed.sh	20 years ago
Junio C Hamano	986aa7f17e	[PATCH 2/2] Test framework documentation. This adds instruction for running tests, and writing new tests. Signed-off-by: Junio C Hamano <junkio@cox.net> Updated to the new tidied up output style. Signed-off-by: Petr Baudis <pasky@ucw.cz>	20 years ago

31 Commits (77b5be2abab69a69618dd40a6ca8b754388b8216)