Merge branch 'ab/tap'

* ab/tap:
  t/README: document more test helpers
  t/README: proposed rewording...
  t/README: Document the do's and don'ts of tests
  t/README: Add a section about skipping tests
  t/README: Document test_expect_code
  t/README: Document test_external*
  t/README: Document the prereq functions, and 3-arg test_*
  t/README: Typo: paralell -> parallel
  t/README: The trash is in 't/trash directory.$name'
  t/t9700/test.pl: don't access private object members, use public access methods
  t9700: Use Test::More->builder, not $Test::Builder::Test
  tests: Say "pass" rather than "ok" on empty lines for TAP
  tests: Skip tests in a way that makes sense under TAP
  test-lib: output a newline before "ok" under a TAP harness
  test-lib: Make the test_external_* functions TAP-aware
  test-lib: Adjust output to be valid TAP format

t/README

@@ -18,25 +18,48 @@ The easiest way to run tests is to say "make".  This runs all
 the tests.
 
     *** t0000-basic.sh ***
-    *   ok 1: .git/objects should be empty after git-init in an empty repo.
+    ok 1 - .git/objects should be empty after git init in an empty repo.
-    *   ok 2: .git/objects should have 256 subdirectories.
+    ok 2 - .git/objects should have 3 subdirectories.
-    *   ok 3: git-update-index without --add should fail adding.
+    ok 3 - success is reported like this
-    ...
-    *   ok 23: no diff after checkout and git-update-index --refresh.
-    * passed all 23 test(s)
-    *** t0100-environment-names.sh ***
-    *   ok 1: using old names should issue warnings.
-    *   ok 2: using old names but having new names should not issue warnings.
     ...
+    ok 43 - very long name in the index handled sanely
+    # fixed 1 known breakage(s)
+    # still have 1 known breakage(s)
+    # passed all remaining 42 test(s)
+    1..43
+    *** t0001-init.sh ***
+    ok 1 - plain
+    ok 2 - plain with GIT_WORK_TREE
+    ok 3 - plain bare
 
-Or you can run each test individually from command line, like
+Since the tests all output TAP (see http://testanything.org) they can
-this:
+be run with any TAP harness. Here's an example of parallel testing
+powered by a recent version of prove(1):
 
-    $ sh ./t3001-ls-files-killed.sh
+    $ prove --timer --jobs 15 ./t[0-9]*.sh
-    *   ok 1: git-update-index --add to add various paths.
+    [19:17:33] ./t0005-signals.sh ................................... ok       36 ms
-    *   ok 2: git-ls-files -k to show killed files.
+    [19:17:33] ./t0022-crlf-rename.sh ............................... ok       69 ms
-    *   ok 3: validate git-ls-files -k output.
+    [19:17:33] ./t0024-crlf-archive.sh .............................. ok      154 ms
-    * passed all 3 test(s)
+    [19:17:33] ./t0004-unwritable.sh ................................ ok      289 ms
+    [19:17:33] ./t0002-gitfile.sh ................................... ok      480 ms
+    ===(     102;0  25/?  6/?  5/?  16/?  1/?  4/?  2/?  1/?  3/?  1... )===
+
+prove and other harnesses come with a lot of useful options. The
+--state option in particular is very useful:
+
+    # Repeat until no more failures
+    $ prove -j 15 --state=failed,save ./t[0-9]*.sh
+
+You can also run each test individually from command line, like this:
+
+    $ sh ./t3010-ls-files-killed-modified.sh
+    ok 1 - git update-index --add to add various paths.
+    ok 2 - git ls-files -k to show killed files.
+    ok 3 - validate git ls-files -k output.
+    ok 4 - git ls-files -m to show modified files.
+    ok 5 - validate git ls-files -m output.
+    # passed all 5 test(s)
+    1..5
 
 You can pass --verbose (or -v), --debug (or -d), and --immediate
 (or -i) command line argument to the test, or by setting GIT_TEST_OPTS
@@ -198,15 +221,101 @@ This test harness library does the following things:
  - If the script is invoked with command line argument --help
    (or -h), it shows the test_description and exits.
 
- - Creates an empty test directory with an empty .git/objects
+ - Creates an empty test directory with an empty .git/objects database
-   database and chdir(2) into it.  This directory is 't/trash directory'
+   and chdir(2) into it.  This directory is 't/trash
-   if you must know, but I do not think you care.
+   directory.$test_name_without_dotsh', with t/ subject to change by
+   the --root option documented above.
 
  - Defines standard test helper functions for your scripts to
    use.  These functions are designed to make all scripts behave
    consistently when command line arguments --verbose (or -v),
    --debug (or -d), and --immediate (or -i) is given.
 
+Do's, don'ts & things to keep in mind
+-------------------------------------
+
+Here are a few examples of things you probably should and shouldn't do
+when writing tests.
+
+Do:
+
+ - Put all code inside test_expect_success and other assertions.
+
+   Even code that isn't a test per se, but merely some setup code
+   should be inside a test assertion.
+
+ - Chain your test assertions
+
+   Write test code like this:
+
+	git merge foo &&
+	git push bar &&
+	test ...
+
+   Instead of:
+
+	git merge hla
+	git push gh
+	test ...
+
+   That way all of the commands in your tests will succeed or fail. If
+   you must ignore the return value of something (e.g. the return
+   value of export is unportable) it's best to indicate so explicitly
+   with a semicolon:
+
+	export HLAGH;
+	git merge hla &&
+	git push gh &&
+	test ...
+
+Don't:
+
+ - exit() within a <script> part.
+
+   The harness will catch this as a programming error of the test.
+   Use test_done instead if you need to stop the tests early (see
+   "Skipping tests" below).
+
+ - Break the TAP output
+
+   The raw output from your test may be interpreted by a TAP harness. TAP
+   harnesses will ignore everything they don't know about, but don't step
+   on their toes in these areas:
+
+   - Don't print lines like "$x..$y" where $x and $y are integers.
+
+   - Don't print lines that begin with "ok" or "not ok".
+
+   TAP harnesses expect a line that begins with either "ok" and "not
+   ok" to signal a test passed or failed (and our harness already
+   produces such lines), so your script shouldn't emit such lines to
+   their output.
+
+   You can glean some further possible issues from the TAP grammar
+   (see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar)
+   but the best indication is to just run the tests with prove(1),
+   it'll complain if anything is amiss.
+
+Keep in mind:
+
+ - Inside <script> part, the standard output and standard error
+   streams are discarded, and the test harness only reports "ok" or
+   "not ok" to the end user running the tests. Under --verbose, they
+   are shown to help debugging the tests.
+
+
+Skipping tests
+--------------
+
+If you need to skip all the remaining tests you should set skip_all
+and immediately call test_done. The string you give to skip_all will
+be used as an explanation for why the test was skipped. for instance:
+
+	if ! test_have_prereq PERL
+	then
+	    skip_all='skipping perl interface tests, perl not available'
+	    test_done
+	fi
 
 End with test_done
 ------------------
@@ -222,9 +331,9 @@ Test harness library
 There are a handful helper functions defined in the test harness
 library for your script to use.
 
- - test_expect_success <message> <script>
+ - test_expect_success [<prereq>] <message> <script>
 
-   This takes two strings as parameter, and evaluates the
+   Usually takes two strings as parameter, and evaluates the
    <script>.  If it yields success, test is considered
    successful.  <message> should state what it is testing.
 
@@ -234,7 +343,14 @@ library for your script to use.
 	    'git-write-tree should be able to write an empty tree.' \
 	    'tree=$(git-write-tree)'
 
- - test_expect_failure <message> <script>
+   If you supply three parameters the first will be taken to be a
+   prerequisite, see the test_set_prereq and test_have_prereq
+   documentation below:
+
+	test_expect_success TTY 'git --paginate rev-list uses a pager' \
+	    ' ... '
+
+ - test_expect_failure [<prereq>] <message> <script>
 
    This is NOT the opposite of test_expect_success, but is used
    to mark a test that demonstrates a known breakage.  Unlike
@@ -243,6 +359,16 @@ library for your script to use.
    success and "still broken" on failure.  Failures from these
    tests won't cause -i (immediate) to stop.
 
+   Like test_expect_success this function can optionally use a three
+   argument invocation with a prerequisite as the first argument.
+
+ - test_expect_code [<prereq>] <code> <message> <script>
+
+   Analogous to test_expect_success, but pass the test if it exits
+   with a given exit <code>
+
+ test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'
+
  - test_debug <script>
 
    This takes a single argument, <script>, and evaluates it only
@@ -275,6 +401,85 @@ library for your script to use.
    Merges the given rev using the given message.  Like test_commit,
    creates a tag and calls test_tick before committing.
 
+ - test_set_prereq SOME_PREREQ
+
+   Set a test prerequisite to be used later with test_have_prereq. The
+   test-lib will set some prerequisites for you, e.g. PERL and PYTHON
+   which are derived from ./GIT-BUILD-OPTIONS (grep test_set_prereq
+   test-lib.sh for more). Others you can set yourself and use later
+   with either test_have_prereq directly, or the three argument
+   invocation of test_expect_success and test_expect_failure.
+
+ - test_have_prereq SOME PREREQ
+
+   Check if we have a prerequisite previously set with
+   test_set_prereq. The most common use of this directly is to skip
+   all the tests if we don't have some essential prerequisite:
+
+	if ! test_have_prereq PERL
+	then
+	    skip_all='skipping perl interface tests, perl not available'
+	    test_done
+	fi
+
+ - test_external [<prereq>] <message> <external> <script>
+
+   Execute a <script> with an <external> interpreter (like perl). This
+   was added for tests like t9700-perl-git.sh which do most of their
+   work in an external test script.
+
+	test_external \
+	    'GitwebCache::*FileCache*' \
+	    "$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_interface.pl
+
+   If the test is outputting its own TAP you should set the
+   test_external_has_tap variable somewhere before calling the first
+   test_external* function. See t9700-perl-git.sh for an example.
+
+	# The external test will outputs its own plan
+	test_external_has_tap=1
+
+ - test_external_without_stderr [<prereq>] <message> <external> <script>
+
+   Like test_external but fail if there's any output on stderr,
+   instead of checking the exit code.
+
+	test_external_without_stderr \
+	    'Perl API' \
+	    "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
+
+ - test_must_fail <git-command>
+
+   Run a git command and ensure it fails in a controlled way.  Use
+   this instead of "! <git-command>" to fail when git commands
+   segfault.
+
+ - test_might_fail <git-command>
+
+   Similar to test_must_fail, but tolerate success, too.  Use this
+   instead of "<git-command> || :" to catch failures due to segv.
+
+ - test_cmp <expected> <actual>
+
+   Check whether the content of the <actual> file matches the
+   <expected> file.  This behaves like "cmp" but produces more
+   helpful output when the test is run with "-v" option.
+
+ - test_when_finished <script>
+
+   Prepend <script> to a list of commands to run to clean up
+   at the end of the current test.  If some clean-up command
+   fails, the test will not pass.
+
+   Example:
+
+	test_expect_success 'branch pointing to non-commit' '
+		git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
+		test_when_finished "git update-ref -d refs/heads/invalid" &&
+		...
+	'
+
+
 Tips for Writing Tests
 ----------------------
 

t/lib-git-svn.sh

@@ -5,11 +5,11 @@ git_svn_id=git""-svn-id
 
 if test -n "$NO_SVN_TESTS"
 then
-	say 'skipping git svn tests, NO_SVN_TESTS defined'
+	skip_all='skipping git svn tests, NO_SVN_TESTS defined'
 	test_done
 fi
 if ! test_have_prereq PERL; then
-	say 'skipping git svn tests, perl not available'
+	skip_all='skipping git svn tests, perl not available'
 	test_done
 fi
 

t/lib-httpd.sh

@@ -5,8 +5,7 @@
 
 if test -z "$GIT_TEST_HTTPD"
 then
-	say "skipping test, network testing disabled by default"
+	skip_all="Network testing disabled (define GIT_TEST_HTTPD to enable)"
-	say "(define GIT_TEST_HTTPD to enable)"
 	test_done
 fi
 

t/t1020-subdirectory.sh

@@ -24,18 +24,18 @@ test_expect_success 'update-index and ls-files' '
 	cd "$HERE" &&
 	git update-index --add one &&
 	case "`git ls-files`" in
-	one) echo ok one ;;
+	one) echo pass one ;;
 	*) echo bad one; exit 1 ;;
 	esac &&
 	cd dir &&
 	git update-index --add two &&
 	case "`git ls-files`" in
-	two) echo ok two ;;
+	two) echo pass two ;;
 	*) echo bad two; exit 1 ;;
 	esac &&
 	cd .. &&
 	case "`git ls-files`" in
-	dir/two"$LF"one) echo ok both ;;
+	dir/two"$LF"one) echo pass both ;;
 	*) echo bad; exit 1 ;;
 	esac
 '
@@ -58,17 +58,17 @@ test_expect_success 'diff-files' '
 	echo a >>one &&
 	echo d >>dir/two &&
 	case "`git diff-files --name-only`" in
-	dir/two"$LF"one) echo ok top ;;
+	dir/two"$LF"one) echo pass top ;;
 	*) echo bad top; exit 1 ;;
 	esac &&
 	# diff should not omit leading paths
 	cd dir &&
 	case "`git diff-files --name-only`" in
-	dir/two"$LF"one) echo ok subdir ;;
+	dir/two"$LF"one) echo pass subdir ;;
 	*) echo bad subdir; exit 1 ;;
 	esac &&
 	case "`git diff-files --name-only .`" in
-	dir/two) echo ok subdir limited ;;
+	dir/two) echo pass subdir limited ;;
 	*) echo bad subdir limited; exit 1 ;;
 	esac
 '

t/t1304-default-acl.sh

@@ -15,9 +15,12 @@ umask 077
 # is a good candidate: exists on all unices, and it has permission
 # anyway, so we don't create a security hole running the testsuite.
 
-if ! setfacl -m u:root:rwx .; then
+setfacl_out="$(setfacl -m u:root:rwx . 2>&1)"
-    say "Skipping ACL tests: unable to use setfacl"
+setfacl_ret=$?
-    test_done
+
+if [ $setfacl_ret != 0 ]; then
+	skip_all="Skipping ACL tests: unable to use setfacl (output: '$setfacl_out'; return code: '$setfacl_ret')"
+	test_done
 fi
 
 check_perms_and_acl () {

t/t1509-root-worktree.sh

@@ -99,17 +99,17 @@ test_foobar_foobar() {
 }
 
 if ! test_have_prereq POSIXPERM || ! [ -w / ]; then
-	say "Dangerous test skipped. Read this test if you want to execute it"
+	skip_all="Dangerous test skipped. Read this test if you want to execute it"
 	test_done
 fi
 
 if [ "$IKNOWWHATIAMDOING" != "YES" ]; then
-	say "You must set env var IKNOWWHATIAMDOING=YES in order to run this test"
+	skip_all="You must set env var IKNOWWHATIAMDOING=YES in order to run this test"
 	test_done
 fi
 
 if [ "$UID" = 0 ]; then
-	say "No you can't run this with root"
+	skip_all="No you can't run this with root"
 	test_done
 fi
 

t/t2007-checkout-symlink.sh

@@ -8,7 +8,7 @@ test_description='git checkout to switch between branches with symlink<->dir'
 
 if ! test_have_prereq SYMLINKS
 then
-	say "symbolic links not supported - skipping tests"
+	skip_all="symbolic links not supported - skipping tests"
 	test_done
 fi
 

t/t2102-update-index-symlinks.sh

@@ -24,7 +24,7 @@ git update-index symlink'
 test_expect_success \
 'the index entry must still be a symbolic link' '
 case "`git ls-files --stage --cached symlink`" in
-120000" "*symlink) echo ok;;
+120000" "*symlink) echo pass;;
 *) echo fail; git ls-files --stage --cached symlink; (exit 1);;
 esac'
 

t/t3300-funny-names.sh

@@ -26,7 +26,7 @@ echo 'Foo Bar Baz' >"$p2"
 
 test -f "$p1" && cmp "$p0" "$p1" || {
 	# since FAT/NTFS does not allow tabs in filenames, skip this test
-	say 'Your filesystem does not allow tabs in filenames, test skipped.'
+	skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
 	test_done
 }
 

t/t3302-notes-index-expensive.sh

@@ -8,7 +8,7 @@ test_description='Test commit notes index (expensive!)'
 . ./test-lib.sh
 
 test -z "$GIT_NOTES_TIMING_TESTS" && {
-	say Skipping timing tests
+	skip_all="Skipping timing tests"
 	test_done
 	exit
 }

t/t3600-rm.sh

@@ -39,7 +39,7 @@ if test -f test-file
 then
 	test_set_prereq RO_DIR
 else
-	say 'skipping removal failure test (perhaps running as root?)'
+	skip_all='skipping removal failure test (perhaps running as root?)'
 fi
 chmod 775 .
 rm -f test-file

t/t3700-add.sh

@@ -26,7 +26,7 @@ test_expect_success \
 	 chmod 755 xfoo1 &&
 	 git add xfoo1 &&
 	 case "`git ls-files --stage xfoo1`" in
-	 100644" "*xfoo1) echo ok;;
+	 100644" "*xfoo1) echo pass;;
 	 *) echo fail; git ls-files --stage xfoo1; (exit 1);;
 	 esac'
 
@@ -35,7 +35,7 @@ test_expect_success SYMLINKS 'git add: filemode=0 should not get confused by sym
 	ln -s foo xfoo1 &&
 	git add xfoo1 &&
 	case "`git ls-files --stage xfoo1`" in
-	120000" "*xfoo1) echo ok;;
+	120000" "*xfoo1) echo pass;;
 	*) echo fail; git ls-files --stage xfoo1; (exit 1);;
 	esac
 '
@@ -47,7 +47,7 @@ test_expect_success \
 	 chmod 755 xfoo2 &&
 	 git update-index --add xfoo2 &&
 	 case "`git ls-files --stage xfoo2`" in
-	 100644" "*xfoo2) echo ok;;
+	 100644" "*xfoo2) echo pass;;
 	 *) echo fail; git ls-files --stage xfoo2; (exit 1);;
 	 esac'
 
@@ -56,7 +56,7 @@ test_expect_success SYMLINKS 'git add: filemode=0 should not get confused by sym
 	ln -s foo xfoo2 &&
 	git update-index --add xfoo2 &&
 	case "`git ls-files --stage xfoo2`" in
-	120000" "*xfoo2) echo ok;;
+	120000" "*xfoo2) echo pass;;
 	*) echo fail; git ls-files --stage xfoo2; (exit 1);;
 	esac
 '
@@ -67,7 +67,7 @@ test_expect_success SYMLINKS \
 	 ln -s xfoo2 xfoo3 &&
 	 git update-index --add xfoo3 &&
 	 case "`git ls-files --stage xfoo3`" in
-	 120000" "*xfoo3) echo ok;;
+	 120000" "*xfoo3) echo pass;;
 	 *) echo fail; git ls-files --stage xfoo3; (exit 1);;
 	 esac'
 
@@ -172,7 +172,7 @@ test_expect_success 'git add --refresh' '
 	test -z "`git diff-index HEAD -- foo`" &&
 	git read-tree HEAD &&
 	case "`git diff-index HEAD -- foo`" in
-	:100644" "*"M	foo") echo ok;;
+	:100644" "*"M	foo") echo pass;;
 	*) echo fail; (exit 1);;
 	esac &&
 	git add --refresh -- foo &&

t/t3701-add-interactive.sh

@@ -4,7 +4,7 @@ test_description='add -i basic tests'
 . ./test-lib.sh
 
 if ! test_have_prereq PERL; then
-	say 'skipping git add -i tests, perl not available'
+	skip_all='skipping git add -i tests, perl not available'
 	test_done
 fi
 
@@ -154,7 +154,7 @@ rm -f .gitignore
 
 if test "$(git config --bool core.filemode)" = false
 then
-	say 'skipping filemode tests (filesystem does not properly support modes)'
+	say '# skipping filemode tests (filesystem does not properly support modes)'
 else
 	test_set_prereq FILEMODE
 fi

t/t3902-quoted.sh

@@ -17,7 +17,7 @@ DQ='"'
 echo foo 2>/dev/null > "Name and an${HT}HT"
 test -f "Name and an${HT}HT" || {
 	# since FAT/NTFS does not allow tabs in filenames, skip this test
-	say 'Your filesystem does not allow tabs in filenames, test skipped.'
+	skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
 	test_done
 }
 

t/t4004-diff-rename-symlink.sh

@@ -14,7 +14,7 @@ by an edit for them.
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t4011-diff-symlink.sh

@@ -11,7 +11,7 @@ test_description='Test diff of symlinks.
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t4016-diff-quote.sh

@@ -14,7 +14,7 @@ P2='pathname with SP'
 P3='pathname
 with LF'
 : 2>/dev/null >"$P1" && test -f "$P1" && rm -f "$P1" || {
-	say 'Your filesystem does not allow tabs in filenames, test skipped.'
+	skip_all='Your filesystem does not allow tabs in filenames, test skipped.'
 	test_done
 }
 

t/t4023-diff-rename-typechange.sh

@@ -6,7 +6,7 @@ test_description='typechange rename detection'
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t4114-apply-typechange.sh

@@ -11,7 +11,7 @@ test_description='git apply should not get confused with type changes.
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t4115-apply-symlink.sh

@@ -11,7 +11,7 @@ test_description='git apply symlinks and partial files
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t4122-apply-symlink-inside.sh

@@ -5,7 +5,7 @@ test_description='apply to deeper directory without getting fooled with symlink'
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t5302-pack-index.sh

@@ -74,7 +74,7 @@ if msg=$(git verify-pack -v "test-3-${pack3}.pack" 2>&1) ||
 then
 	test_set_prereq OFF64_T
 else
-	say "skipping tests concerning 64-bit offsets"
+	say "# skipping tests concerning 64-bit offsets"
 fi
 
 test_expect_success OFF64_T \

t/t5503-tagfollow.sh

@@ -6,7 +6,7 @@ test_description='test automatic tag following'
 
 case $(uname -s) in
 *MINGW*)
-	say "GIT_DEBUG_SEND_PACK not supported - skipping tests"
+	skip_all="GIT_DEBUG_SEND_PACK not supported - skipping tests"
 	test_done
 esac
 

t/t5522-pull-symlink.sh

@@ -6,7 +6,7 @@ test_description='pulling from symlinked subdir'
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t5540-http-push.sh

@@ -11,7 +11,7 @@ This test runs various sanity checks on http-push.'
 
 if git http-push > /dev/null 2>&1 || [ $? -eq 128 ]
 then
-	say "skipping test, USE_CURL_MULTI is not defined"
+	skip_all="skipping test, USE_CURL_MULTI is not defined"
 	test_done
 fi
 

t/t5541-http-push.sh

@@ -7,7 +7,7 @@ test_description='test smart pushing over http via http-backend'
 . ./test-lib.sh
 
 if test -n "$NO_CURL"; then
-	say 'skipping test, git built without http support'
+	skip_all='skipping test, git built without http support'
 	test_done
 fi
 

t/t5550-http-fetch.sh

@@ -4,7 +4,7 @@ test_description='test dumb fetching over http via static file'
 . ./test-lib.sh
 
 if test -n "$NO_CURL"; then
-	say 'skipping test, git built without http support'
+	skip_all='skipping test, git built without http support'
 	test_done
 fi
 

t/t5551-http-fetch.sh

@@ -4,7 +4,7 @@ test_description='test smart fetching over http via http-backend'
 . ./test-lib.sh
 
 if test -n "$NO_CURL"; then
-	say 'skipping test, git built without http support'
+	skip_all='skipping test, git built without http support'
 	test_done
 fi
 

t/t5561-http-backend.sh

@@ -4,7 +4,7 @@ test_description='test git-http-backend'
 . ./test-lib.sh
 
 if test -n "$NO_CURL"; then
-	say 'skipping test, git built without http support'
+	skip_all='skipping test, git built without http support'
 	test_done
 fi
 

t/t5705-clone-2gb.sh

@@ -4,7 +4,7 @@ test_description='Test cloning a repository larger than 2 gigabyte'
 . ./test-lib.sh
 
 test -z "$GIT_TEST_CLONE_2GB" &&
-say "Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t" &&
+skip_all="Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t" &&
 test_done &&
 exit
 

t/t6035-merge-dir-to-symlink.sh

@@ -5,7 +5,7 @@ test_description='merging when a directory was replaced with a symlink'
 
 if ! test_have_prereq SYMLINKS
 then
-	say 'Symbolic links not supported, skipping tests.'
+	skip_all='Symbolic links not supported, skipping tests.'
 	test_done
 fi
 

t/t7004-tag.sh

@@ -583,7 +583,7 @@ test_expect_success \
 # subsequent tests require gpg; check if it is available
 gpg --version >/dev/null 2>/dev/null
 if [ $? -eq 127 ]; then
-	say "gpg not found - skipping tag signing and verification tests"
+	say "# gpg not found - skipping tag signing and verification tests"
 else
 	# As said here: http://www.gnupg.org/documentation/faqs.html#q6.19
 	# the gpg version 1.0.6 didn't parse trust packets correctly, so for

t/t7006-pager.sh

@@ -37,7 +37,7 @@ then
 	}
 	test_set_prereq TTY
 else
-	say no usable terminal, so skipping some tests
+	say "# no usable terminal, so skipping some tests"
 fi
 
 test_expect_success 'setup' '

t/t7800-difftool.sh

@@ -11,7 +11,7 @@ Testing basic diff tool invocation
 . ./test-lib.sh
 
 if ! test_have_prereq PERL; then
-	say 'skipping difftool tests, perl not available'
+	skip_all='skipping difftool tests, perl not available'
 	test_done
 fi
 

t/t9001-send-email.sh

@@ -4,7 +4,7 @@ test_description='git send-email'
 . ./test-lib.sh
 
 if ! test_have_prereq PERL; then
-	say 'skipping git send-email tests, perl not available'
+	skip_all='skipping git send-email tests, perl not available'
 	test_done
 fi
 
@@ -58,7 +58,7 @@ test_no_confirm () {
 # Exit immediately to prevent hang if a no-confirm test fails
 check_no_confirm () {
 	test -f no_confirm_okay || {
-		say 'No confirm test failed; skipping remaining tests to prevent hanging'
+		skip_all='confirm test failed; skipping remaining tests to prevent hanging'
 		test_done
 	}
 }

t/t9100-git-svn-basic.sh

@@ -15,7 +15,7 @@ case "$GIT_SVN_LC_ALL" in
 	test_set_prereq UTF8
 	;;
 *)
-	say "UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)"
+	say "# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)"
 	;;
 esac
 

t/t9119-git-svn-info.sh

@@ -13,7 +13,7 @@ case $v in
 1.[456].*)
 	;;
 *)
-	say "skipping svn-info test (SVN version: $v not supported)"
+	skip_all="skipping svn-info test (SVN version: $v not supported)"
 	test_done
 	;;
 esac

t/t9129-git-svn-i18n-commitencoding.sh

@@ -23,7 +23,7 @@ if test -n "$a_utf8_locale"
 then
 	test_set_prereq UTF8
 else
-	say "UTF-8 locale not available, some tests are skipped"
+	say "# UTF-8 locale not available, some tests are skipped"
 fi
 
 compare_svn_head_with () {

t/t9143-git-svn-gc.sh

@@ -43,7 +43,7 @@ then
 		 gunzip .git/svn/refs/remotes/git-svn/unhandled.log.gz
 		'
 else
-	say "Perl Compress::Zlib unavailable, skipping gunzip test"
+	say "# Perl Compress::Zlib unavailable, skipping gunzip test"
 fi
 
 test_expect_success 'git svn gc does not change unhandled.log files' '

t/t9200-git-cvsexportcommit.sh

@@ -7,14 +7,14 @@ test_description='Test export of commits to CVS'
 . ./test-lib.sh
 
 if ! test_have_prereq PERL; then
-	say 'skipping git cvsexportcommit tests, perl not available'
+	skip_all='skipping git cvsexportcommit tests, perl not available'
 	test_done
 fi
 
 cvs >/dev/null 2>&1
 if test $? -ne 1
 then
-    say 'skipping git cvsexportcommit tests, cvs not found'
+    skip_all='skipping git cvsexportcommit tests, cvs not found'
     test_done
 fi
 

t/t9400-git-cvsserver-server.sh

@@ -11,17 +11,17 @@ cvs CLI client via git-cvsserver server'
 . ./test-lib.sh
 
 if ! test_have_prereq PERL; then
-	say 'skipping git cvsserver tests, perl not available'
+	skip_all='skipping git cvsserver tests, perl not available'
 	test_done
 fi
 cvs >/dev/null 2>&1
 if test $? -ne 1
 then
-    say 'skipping git-cvsserver tests, cvs not found'
+    skip_all='skipping git-cvsserver tests, cvs not found'
     test_done
 fi
 "$PERL_PATH" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {
-    say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'
+    skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'
     test_done
 }
 

t/t9401-git-cvsserver-crlf.sh

@@ -41,16 +41,16 @@ not_present() {
 cvs >/dev/null 2>&1
 if test $? -ne 1
 then
-    say 'skipping git-cvsserver tests, cvs not found'
+    skip_all='skipping git-cvsserver tests, cvs not found'
     test_done
 fi
 if ! test_have_prereq PERL
 then
-    say 'skipping git-cvsserver tests, perl not available'
+    skip_all='skipping git-cvsserver tests, perl not available'
     test_done
 fi
 "$PERL_PATH" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {
-    say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'
+    skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'
     test_done
 }
 

t/t9600-cvsimport.sh

@@ -4,7 +4,7 @@ test_description='git cvsimport basic tests'
 . ./lib-cvs.sh
 
 if ! test_have_prereq PERL; then
-	say 'skipping git cvsimport tests, perl not available'
+	skip_all='skipping git cvsimport tests, perl not available'
 	test_done
 fi
 

t/t9700-perl-git.sh

@@ -7,12 +7,12 @@ test_description='perl interface (Git.pm)'
 . ./test-lib.sh
 
 if ! test_have_prereq PERL; then
-	say 'skipping perl interface tests, perl not available'
+	skip_all='skipping perl interface tests, perl not available'
 	test_done
 fi
 
 "$PERL_PATH" -MTest::More -e 0 2>/dev/null || {
-	say "Perl Test::More unavailable, skipping test"
+	skip_all="Perl Test::More unavailable, skipping test"
 	test_done
 }
 
@@ -46,6 +46,9 @@ test_expect_success \
      git config --add test.int 2k
      '
 
+# The external test will outputs its own plan
+test_external_has_tap=1
+
 test_external_without_stderr \
     'Perl API' \
     "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl

t/t9700/test.pl

@@ -7,6 +7,13 @@ use strict;
 
 use Test::More qw(no_plan);
 
+BEGIN {
+	# t9700-perl-git.sh kicks off our testing, so we have to go from
+	# there.
+	Test::More->builder->current_test(1);
+	Test::More->builder->no_ending(1);
+}
+
 use Cwd;
 use File::Basename;
 
@@ -105,3 +112,8 @@ my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));
 like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');
 my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');
 isnt($last_commit, $dir_commit, 'log . does not show last commit');
+
+printf "1..%d\n", Test::More->builder->current_test;
+
+my $is_passing = eval { Test::More->is_passing };
+exit($is_passing ? 0 : 1) unless $@ =~ /Can't locate object method/;

t/test-lib.sh

@@ -160,7 +160,7 @@ if test -n "$color"; then
 			*) test -n "$quiet" && return;;
 		esac
 		shift
-		printf "* %s" "$*"
+		printf "%s" "$*"
 		tput sgr0
 		echo
 		)
@@ -169,7 +169,7 @@ else
 	say_color() {
 		test -z "$1" && test -n "$quiet" && return
 		shift
-		echo "* $*"
+		echo "$*"
 	}
 fi
 
@@ -206,6 +206,8 @@ test_fixed=0
 test_broken=0
 test_success=0
 
+test_external_has_tap=0
+
 die () {
 	code=$?
 	if test -n "$GIT_EXIT_OK"
@@ -339,25 +341,25 @@ test_have_prereq () {
 
 test_ok_ () {
 	test_success=$(($test_success + 1))
-	say_color "" "  ok $test_count: $@"
+	say_color "" "ok $test_count - $@"
 }
 
 test_failure_ () {
 	test_failure=$(($test_failure + 1))
-	say_color error "FAIL $test_count: $1"
+	say_color error "not ok - $test_count $1"
 	shift
-	echo "$@" | sed -e 's/^/	/'
+	echo "$@" | sed -e 's/^/#	/'
 	test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }
 }
 
 test_known_broken_ok_ () {
 	test_fixed=$(($test_fixed+1))
-	say_color "" "  FIXED $test_count: $@"
+	say_color "" "ok $test_count - $@ # TODO known breakage"
 }
 
 test_known_broken_failure_ () {
 	test_broken=$(($test_broken+1))
-	say_color skip "  still broken $test_count: $@"
+	say_color skip "not ok $test_count - $@ # TODO known breakage"
 }
 
 test_debug () {
@@ -369,6 +371,9 @@ test_run_ () {
 	eval >&3 2>&4 "$1"
 	eval_ret=$?
 	eval >&3 2>&4 "$test_cleanup"
+	if test "$verbose" = "t" && test -n "$HARNESS_ACTIVE"; then
+		echo ""
+	fi
 	return 0
 }
 
@@ -390,7 +395,7 @@ test_skip () {
 	case "$to_skip" in
 	t)
 		say_color skip >&3 "skipping test: $@"
-		say_color skip "skip $test_count: $1"
+		say_color skip "ok $test_count: # skip $1"
 		: true
 		;;
 	*)
@@ -456,7 +461,7 @@ test_expect_code () {
 # test_external runs external test scripts that provide continuous
 # test output about their progress, and succeeds/fails on
 # zero/non-zero exit code.  It outputs the test output on stdout even
-# in non-verbose mode, and announces the external script with "* run
+# in non-verbose mode, and announces the external script with "# run
 # <n>: ..." before running it.  When providing relative paths, keep in
 # mind that all scripts run in "trash directory".
 # Usage: test_external description command arguments...
@@ -471,7 +476,7 @@ test_external () {
 	then
 		# Announce the script to reduce confusion about the
 		# test output that follows.
-		say_color "" " run $test_count: $descr ($*)"
+		say_color "" "# run $test_count: $descr ($*)"
 		# Export TEST_DIRECTORY, TRASH_DIRECTORY and GIT_TEST_LONG
 		# to be able to use them in script
 		export TEST_DIRECTORY TRASH_DIRECTORY GIT_TEST_LONG
@@ -481,9 +486,19 @@ test_external () {
 		"$@" 2>&4
 		if [ "$?" = 0 ]
 		then
-			test_ok_ "$descr"
+			if test $test_external_has_tap -eq 0; then
+				test_ok_ "$descr"
+			else
+				say_color "" "# test_external test $descr was ok"
+				test_success=$(($test_success + 1))
+			fi
 		else
-			test_failure_ "$descr" "$@"
+			if test $test_external_has_tap -eq 0; then
+				test_failure_ "$descr" "$@"
+			else
+				say_color error "# test_external test $descr failed: $@"
+				test_failure=$(($test_failure + 1))
+			fi
 		fi
 	fi
 }
@@ -499,19 +514,30 @@ test_external_without_stderr () {
 	[ -f "$stderr" ] || error "Internal error: $stderr disappeared."
 	descr="no stderr: $1"
 	shift
-	say >&3 "expecting no stderr from previous command"
+	say >&3 "# expecting no stderr from previous command"
 	if [ ! -s "$stderr" ]; then
 		rm "$stderr"
-		test_ok_ "$descr"
+
+		if test $test_external_has_tap -eq 0; then
+			test_ok_ "$descr"
+		else
+			say_color "" "# test_external_without_stderr test $descr was ok"
+			test_success=$(($test_success + 1))
+		fi
 	else
 		if [ "$verbose" = t ]; then
-			output=`echo; echo Stderr is:; cat "$stderr"`
+			output=`echo; echo "# Stderr is:"; cat "$stderr"`
 		else
 			output=
 		fi
 		# rm first in case test_failure exits.
 		rm "$stderr"
-		test_failure_ "$descr" "$@" "$output"
+		if test $test_external_has_tap -eq 0; then
+			test_failure_ "$descr" "$@" "$output"
+		else
+			say_color error "# test_external_without_stderr test $descr failed: $@: $output"
+			test_failure=$(($test_failure + 1))
+		fi
 	fi
 }
 
@@ -620,18 +646,24 @@ test_done () {
 
 	if test "$test_fixed" != 0
 	then
-		say_color pass "fixed $test_fixed known breakage(s)"
+		say_color pass "# fixed $test_fixed known breakage(s)"
 	fi
 	if test "$test_broken" != 0
 	then
-		say_color error "still have $test_broken known breakage(s)"
+		say_color error "# still have $test_broken known breakage(s)"
 		msg="remaining $(($test_count-$test_broken)) test(s)"
 	else
 		msg="$test_count test(s)"
 	fi
 	case "$test_failure" in
 	0)
-		say_color pass "passed all $msg"
+		# Maybe print SKIP message
+		[ -z "$skip_all" ] || skip_all=" # SKIP $skip_all"
+
+		if test $test_external_has_tap -eq 0; then
+			say_color pass "# passed all $msg"
+			say "1..$test_count$skip_all"
+		fi
 
 		test -d "$remove_trash" &&
 		cd "$(dirname "$remove_trash")" &&
@@ -640,7 +672,11 @@ test_done () {
 		exit 0 ;;
 
 	*)
-		say_color error "failed $test_failure among $msg"
+		if test $test_external_has_tap -eq 0; then
+			say_color error "# failed $test_failure among $msg"
+			say "1..$test_count"
+		fi
+
 		exit 1 ;;
 
 	esac