The test suite has been updated to use the 'test_grep' helper instead
of bare 'grep' for test assertions, allowing file contents to be
printed on failure for easier debugging. A new 'greplint' linter has
been introduced to detect and prevent new bare 'grep' assertions from
being added to the test suite.
* mm/test-grep-lint:
t: add greplint to detect bare grep assertions
t: convert grep assertions to test_grep
t: fix Lexer line count for $() inside double-quoted strings
t: extract chainlint's parser into shared module
t: fix grep assertions missing file arguments
t/README: document test_grep helper
The standard philosophy for Unix software when a help option (such as
--help) is specified is that the software should exit 0, printing the
help output to standard output, since the standard output is for
user-requested output and the program performed the requested task
successfully. If the user specifies an incorrect option, then the help
output should be printed to standard error (since the user has made a
mistake) and it should exit unsuccessfully.
Most of our commands currently exit 129 on receiving the -h option to
print the short help, which does not line up with the standard
philosophy above. Let's change that to exit 0 instead.
This requires changes to a variety of tests which previously wanted the
129 exit code, so update them. Note that because git diff does its own
option parsing, it still exits with 129, so update some of the tests to
expect either exit status.
Some commands also now pass with -h but not --help-all, so handle those
cases differently for those commands.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The standard philosophy for Unix software when a help option (such as
--help) is specified is that the software should exit 0, printing the
help output to standard output, since the standard output is for
user-requested output and the program performed the requested task
successfully. If the user specifies an incorrect option, then the help
output should be printed to standard error (since the user has made a
mistake) and it should exit unsuccessfully.
git rev-parse --parseopt properly directs the output in both of these
cases, but it currently exits 129 when it receives a --help or -h option
on the command line, which causes its invoking script to do the same.
This is not in line with the usual behavior and it causes scripts using
this command to exit unsuccessfully on --help as well.
Note that Git subcommands implemented using scripts, such as git
submodule, don't have this problem because Git itself intercepts the
--help option and runs man (or a similar tool), which then exits 0.
However, this still affects the myriad scripts that use this
functionality because Git is widespread and the --parseopt functionality
is a good way to get sensible option parsing across shells in a portable
way.
Because git rev-parse --parseopt is intended to be eval'd by the shell,
when help output is to be printed to standard output, Git actually
prints a cat command with a heredoc since the standard output is being
evaluated by the shell. Thus, to do the right thing, simply add an
"exit 0" right after the end of the heredoc, which will cause the
invoking program to exit successfully.
The usual invocation recommended by the manual page is this:
eval "$(echo "$OPTS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?)"
Thus, the fact that git rev-parse --parseopt still exits 129 in this
case is irrelevant, since the "echo exit $?" will print "exit 129", but
that will be after the "exit 0" printed by Git—and thus ignored, since
the shell will have already exited successfully.
Update the tests for this case. Note that we no longer need to delete
only the first and last lines in some tests, so add a command to delete
the end of the heredoc as well. We could do something clever with sed
to delete all but the last two lines or switch to head and tail, but
those would be more complicated and less readable, so just stick with
the simple approach.
In t1517, add three shell scripts to the failure case because they no
longer return 129 as expected. In a future commit, we'll change the
expected result to exit 0 and these will become successful again.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace bare grep with test_grep in test assertions across the
suite, including sourced test helpers (lib-*.sh, *-tests.sh).
test_grep prints the contents of the file being searched on
failure, making debugging easier than a bare grep which fails
silently.
Only assertion-style greps are converted: grep used as a filter
in pipelines, command substitutions, conditionals, or with
redirected I/O is left as-is with a "# lint-ok" annotation.
Existing '! test_grep' calls are rewritten to 'test_grep !' so
that the diagnostic output is preserved on failure.
test_grep requires the file it reads to exist, so '! grep'
assertions that inspect a file whose presence is conditional need
care. In t5537 the '.git/shallow' file is still present after the
repack (the client remains shallow), so the assertion is
converted like any other. In t1400 the '.git/packed-refs' file
exists only with the files backend, so its check is guarded with a
REFFILES prerequisite; the backend-agnostic 'git show-ref' check
that follows still runs under every backend. In t7450 'git~2' is
the NTFS 8.3 short name of a '..git' file and only exists
when 8.3 short-name generation is enabled, so its check is guarded
with a 'test -f' on the path and uses test_grep inside the guard,
the same shape as t1400 (a plain test_grep would BUG when the
short name is absent).
The conversion was generated using a grep-assertion linter
(greplint.pl, added in the following commit) to identify bare
grep calls at command position. To reproduce, from the t/
directory:
# Step 1: annotate the two data-filter greps (grep produces
# data, not a verdict) so the linter skips them.
sed -i '/grep -vf before commits\.raw/s/$/ # lint-ok: data filter/' \
t5326-multi-pack-bitmaps.sh
sed -i '/grep -E "^\[0-9a-f\].*|| :/s/$/ # lint-ok: data filter/' \
t5702-protocol-v2.sh
# Step 1b: two '! grep' assertions need more than a mechanical
# conversion; handle them by hand before the linter-driven steps
# below so it leaves them alone.
#
# t1400: '.git/packed-refs' is absent under reftable, so guard the
# check with REFFILES (a plain test_grep would BUG on the missing
# file):
#
# git update-ref -d HEAD $B &&
# - ! grep "$m" .git/packed-refs &&
# + if test_have_prereq REFFILES
# + then
# + test_grep ! "$m" .git/packed-refs
# + fi &&
# test_must_fail git show-ref --verify -q $m
#
# t7450: git~2 is an NTFS 8.3 short name that exists only when
# short-name generation is enabled, so guard the check on its
# presence with 'test -f' and note in a comment why the path can
# be absent (a plain test_grep would BUG when it is):
#
# - ! grep gitdir squatting-clone/d/a/git~2
# + if test -f squatting-clone/d/a/git~2
# + then
# + test_grep ! gitdir squatting-clone/d/a/git~2
# + fi
# Step 2: reorder pre-existing '! test_grep' to 'test_grep !'
# (must come before steps 3-4 so greplint does not see them)
sed -i 's/! test_grep/test_grep !/' t0031-lockfile-pid.sh
sed -i 's/! test_grep/test_grep !/' t5300-pack-object.sh
sed -i 's/! test_grep/test_grep !/' t5319-multi-pack-index.sh
# Step 3: convert '! grep' -> 'test_grep !'
perl greplint.pl *.sh 2>&1 | cut -d: -f1,2 |
while IFS=: read f l; do
sed -i "${l}s/! *grep/test_grep !/" "$f"
done
# Step 4: convert remaining 'grep' -> 'test_grep'
perl greplint.pl *.sh 2>&1 | cut -d: -f1,2 |
while IFS=: read f l; do
sed -i "${l}s/grep/test_grep/" "$f"
done
To verify, run: make -C t test-greplint
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that the default value for TEST_PASSES_SANITIZE_LEAK is `true` there
is no longer a need to have that variable declared in all of our tests.
Drop it.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `--parseopt` mode allows shell scripts to have the same option
parsing mode as we have in C builtins. It soaks up a set of option
descriptions via stdin and massages them into proper `struct option`s
that we can then use to parse a set of arguments.
We only partially free those options when done though, creating a memory
leak. Interestingly, we only end up free'ing the first option's help,
which is of course wrong.
Fix this by freeing all option's help fields as well as their `argh`
fields to plug this memory leak.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git currently does not detect the ambiguity of an option that starts
with "no" like --notes and its negated form if given just --n or --no.
All Git commands with such options have other negatable options, and
we detect the ambiguity with them, so that's currently only a potential
problem for scripts that use git rev-parse --parseopt.
Let's fix it nevertheless, as there's no need for that confusion. To
detect the ambiguity we have to loosen the check in register_abbrev(),
as an option is considered an alias of itself. Add non-matching
negation flags as a criterion to recognize an option being ambiguous
with its negated form.
And we need to keep going after finding a non-negated option as an
abbreviated candidate and perform the negation checks in the same
loop.
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a "[no-]" prefix to options without the flag PARSE_OPT_NONEG to
document the fact that you can negate them.
This looks a bit strange for options that already start with "no-", e.g.
for the option --no-name of git show-branch:
--[no-]no-name suppress naming strings
You can actually use --no-no-name as an alias of --name, so the short
help is not wrong. If we strip off any of the "no-"s, we lose either
the ability to see if the remaining one belongs to the documented
variant or to see if it can be negated.
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add tests for checking the "git rev-parse --parseopt" flag "!" and
whether options can be negated with a "no-" prefix.
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git rev-parse --parseopt" shows the short help with its description of
all recognized options twice: When called with -h or --help, and after
reporting an unknown option. Move the one for optionspec into a file
and use it in two tests to deduplicate that part.
"git rev-parse --parseopt -- --h" wraps the help text in "cat <<\EOF"
and "EOF". Keep that part in the file to use it as is in the test that
needs it and simply remove it in the other one using sed.
Disable whitespace checking for the file using an attribute, as we need
to keep its spaces intact and wouldn't want a stray --whitespace=fix
turn them into tabs.
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git rev-parse --parseopt" handles the built-in options -h and --help,
but not --no-help. Make test definitions and documentation examples
more realistic by disabling negation.
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Three tests in file t1502-rev-parse-parseopt.sh use three redirections
with invocation of "git rev-parse --parseopt --". All three tests
redirect standard output to file "out" and file "spec" to standard
input. Two of the tests redirect standard output a second time to file
"actual", and the third test redirects standard error to file "err".
These tests check contents of files "actual" and "err", but don't use
the files named "out" for assertions. The two tests that redirect to
standard output twice might also be confusing to the reader.
Don't redirect standard output of "git rev-parse" to file "out" in
t1502-rev-parse-parseopt.sh to avoid creating unnecessary files.
Acked-by: Øystein Walle <oystwa@gmail.com>
Signed-off-by: Andrei Rybak <rybak.a.v@gmail.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After 2d893dff4c (rev-parse --parseopt: allow [*=?!] in argument hints,
2015-07-14) updated the parser, a line in parseopts's input can start
with one of the flag characters and be erroneously parsed as a opt-spec
where the short name of the option is the flag character itself and the
long name is after the end of the string. This makes Git want to
allocate SIZE_MAX bytes of memory at this line:
o->long_name = xmemdupz(sb.buf + 2, s - sb.buf - 2);
Since s and sb.buf are equal the second argument is -2 (except unsigned)
and xmemdupz allocates len + 1 bytes, ie. -1 meaning SIZE_MAX.
Avoid this by checking whether a flag character was found in the zeroth
position.
Reported-by: Ingy dot Net <ingy@ingy.net>
Signed-off-by: Øystein Walle <oystwa@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add tests for the "usagestr" passed to parse-options.c
usage_with_options_internal() through cmd_parseopt().
These test for edge cases in the existing behavior related to the
"--parseopt" interface doing its own line-splitting with
strbuf_getline(), and the native C interface expecting and potentially
needing to handle newlines within the strings in the array it
accepts. The results are probably something that wasn't anticipated,
but let's make sure we stay backwards compatible with it.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As a follow-up to d162b25f95 (tests: remove support for
GIT_TEST_GETTEXT_POISON, 2021-01-20) remove most uses of test_i18ncmp
via a simple s/test_i18ncmp/test_cmp/g search-replacement.
I'm leaving t6300-for-each-ref.sh out due to a conflict with in-flight
changes between "master" and "seen", as well as the prerequisite
itself due to other changes between "master" and "next/seen" which add
new test_i18ncmp uses.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, when parse_options() produces a help message it always emits
a blank line after the usage text to separate it from the options text.
If the option spec does not define any switches, or only defines hidden
switches that will not be displayed, then the help text will end up with
two trailing blank lines instead of one. Let's defer emitting the blank
line between the usage text and the options text until it is clear that
the options section will not be empty.
Fixes t1502.5, t1502.6.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When commit 54e6dc7 added translation support to parse-options, an
fprintf was mistakenly replaced by a call to putchar(). Let's use fputc
instead.
Fixes t0040.11, t0040.12, t0040.33, and t1502.8.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the option spec contains no switches or only hidden switches,
parse_options will emit an extra blank line at the end of help output so
that the help text will end in two blank lines instead of one.
When parse_options produces internal help output after an error has
occurred it will emit blank lines within the usage string to stdout
instead of stderr.
Update t/helper/test-parse-options.c to have a description body in the
usage string to exercise this second bug and mark tests as failing in
t0040.
Add tests to t1502 to demonstrate both of these problems.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, rev-parse only interprets a space ' ' character as the
delimiter between the option spec and the help text. So if a tab
character is placed between the option spec and the help text, it will
be interpreted as part of the long option name or as part of the arg
hint. If it is interpreted as part of the long option name, then
rev-parse will produce what will be interpreted as multiple arguments
on the command line.
For example, the following option spec (note: there is a <tab> between
"frotz" and "enable"):
frotz enable frotzing
will produce the following set expression when --frotz is used:
set -- --frotz --
instead of this:
set -- --frotz enable --
Mark t1502.2 as fixed.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When searching for flag characters in the option spec, we should ensure
the search stays within the bounds of the option spec and does not enter
the help text portion of the spec. So when we find the boundary white
space marking the start of the help text, let's mark it with a nul
character. Then when we search for flag characters starting from the
beginning of the string we'll stop at the nul and won't enter the help
text.
Now, the following option spec:
exclame this does something!
will produce this 'set' expression when --exclame is specified:
set -- --exclame --
instead of this one:
set -- --exclame this does something --
Mark t1502.4 and t1502.5 as fixed.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit 2d893df rev-parse will scan forward from the beginning of
the option string looking for a flag character. If there are no flag
characters then the scan will spill over into the help text and will
interpret the characters preceding the "flag" as part of the option-spec
i.e. the long option name.
For example, the following option spec:
exclame this does something!
will produce this 'set' expression when --exclame is specified:
set -- --exclame this does something --
which will be interpreted as four separate parameters by the shell. And
will produce a help string that looks like:
--exclame this does something
this does something!
git-rebase.sh has such an option (--autosquash), and so will add extra
parameters to the 'set' expression when --autosquash is used.
git-rebase continues to work correctly though because when it parses the
arguments, it ignores ones that it does not recognize.
Also, rev-parse --parseopt does not currently interpret a tab character
as a delimiter between the option spec and the help text. If a tab is
used at the end of the option spec, before the help text, and before a
space has been specified, then rev-parse will interpret the tab as part
of the preceding component (either the long name or the arg hint).
For example, the following option spec (note: there is a <tab> between
"frotz" and "enable"):
frotz enable frotzing
will produce this 'set' expression when --frotz is specified:
set -- --frotz enable --
which will be interpreted as 2 separate arguments by the shell.
git-rebase.sh has one of these too (--keep-empty). In this case the tab
is immediately followed by spaces so there are no additional parameters
produced on the command line. The only side-effect is misalignment in
the help text.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A line in the input to "rev-parse --parseopt" describes an option by
listing a short and/or long name, optional flags [*=?!], argument hint,
and then whitespace and help string.
We did not allow any of the [*=?!] characters in the argument hints.
The following input
pair=key=value equals sign in the hint
used to generate a help line like this:
--pair=key <value> equals sign in the hint
and used to expect "pair=key" as the argument name.
That is not very helpful as we generally do not want any of the [*=?!]
characters in the argument names. But we do want to use at least the
equals sign in the argument hints.
Update the parser to make long argument names stop at the first [*=?!]
character.
Add test case with equals sign in the argument hint and update the test
to perform all the operations in test_expect_success matching the
t/README requirements and allowing commands like
./t1502-rev-parse-parseopt.sh --run=1-2
to stop at the test case 2 without any further modification of the test
state area.
Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is easy to overlook an already assigned single-letter option name
and try to use it for a new one. Help the developer to catch it
before such a mistake escapes the lab.
This retroactively forbids any short option name (which is defined
to be of type "int") outside the ASCII printable range. We might
want to do one of two things:
- tighten the type of short_name member to 'char', and further
update optbug() to protect it against doing "'%c'" on a funny
value, e.g. negative or above 127.
- drop the check (even the "duplicate" check) for an option whose
short_name is either negative or above 255, to allow clever folks
to take advantage of the fact that such a short_name cannot be
parsed from the command line and the member can be used to store
some extra information.
Helped-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The expected output from the argument help use runs of SPs to align
the description of each option; a careless use of --whitespace=fix
can turn leading parts of them into appropriate number of HTs.
Prevent such a breakage by prefixing all the expected lines with
leading vertical bars in the original and stripping them with a
small sed script.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Built-in commands can specify names for option arguments when usage text
is generated for a command. sh based commands should be able to do the
same.
Option argument name hint is any text that comes after [*=?!] after the
argument name up to the first whitespace.
Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the --stuck-long option to output the options in their long form
if available, and with their arguments stuck.
Contrary to the default form (non stuck arguments and short options),
this can be parsed unambiguously when using options with optional
arguments :
- in the non stuck form, when an option is taking an optional argument
you cannot know if the next argument is its optional argument, or the
next option.
- the long options form allows to differentiate between an empty argument
'--option=' and an unset argument '--option', which is not possible
with short options.
Signed-off-by: Nicolas Vigier <boklm@mars-attacks.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use the i18n-specific test functions in test scripts for parseopt tests.
This issue was was introduced in v1.7.10.1-488-g54e6d:
54e6d i18n: parseopt: lookup help and argument translations when showing usage
and been broken under GETTEXT_POISON=YesPlease since.
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rev-parse --parseopt exits with code 129 (usage error) when asked
to dump usage with -h on behalf of another command. Scripts can
take advantage of this to avoid trying to parse usage information
as though it were the regular output from some git command.
Noticed with an &&-chaining tester.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
backmerge a few more fixes to 1.7.1.X series
rev-parse: fix --parse-opt --keep-dashdash --stop-at-non-option
fix git branch -m in presence of cross devices
Conflicts:
RelNotes
builtin/rev-parse.c
The ?: operator has a lower priority than |, so the implicit associativity
made the 6th argument of parse_options be PARSE_OPT_KEEP_DASHDASH if
keep_dashdash was true discarding PARSE_OPT_STOP_AT_NON_OPTION and
PARSE_OPT_SHELL_EVAL.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
9c7304e (print the usage string on stdout instead of stderr,
2010-05-17) broke rev-parse --parseopt: when run with -h, the usage
notice on stdout ended up in the shell eval.
Wrap the usage in a cat <<\EOF ... EOF block when printing to stdout.
I do not expect any usage lines to ever start with EOF so this
shouldn't be an undue burden.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When -h is used, print usage messages on stdout. If a command is invoked with
wrong arguments then print the usage messages on stderr.
Signed-off-by: Giuseppe Scrivano <gscrivano@gnu.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Extend parse-options test suite
api-parse-options.txt: Introduce documentation for parse options API
parse-options.c: fix documentation syntax of optional arguments
api-builtin.txt: update and fix typo
When an argument for an option is optional, short options don't need a
space between the option and the argument, and long options need a "=".
Otherwise, arguments are misinterpreted.
Signed-off-by: Michele Ballabio <barra_cuda@katamail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When outputting a usage message with a blank line in the
header, we would output a line with four spaces. Make this
truly a blank line.
This helps us remove trailing whitespace from a test vector.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As a general principle, we should not use "git diff" to validate the
results of what git command that is being tested has done. We would not
know if we are testing the command in question, or locating a bug in the
cute hack of "git diff --no-index".
Rather use test_cmp for that purpose.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A non-empty line containing no spaces should be treated by --parseopt as
an option group header, but was causing a bus error. Also added a test
script for rev-parse --parseopt.
Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano