Code clean-up.
* jk/test-hashmap-updates:
test-hashmap: use "unsigned int" for hash storage
test-hashmap: simplify alloc_test_entry
test-hashmap: use strbuf_getline rather than fgets
test-hashmap: use xsnprintf rather than snprintf
test-hashmap: check allocation computation for overflow
test-hashmap: use ALLOC_ARRAY rather than bare malloc
Code to unquote single-quoted string (used in the parser for
configuration files, etc.) did not diagnose bogus input correctly
and produced bogus results instead.
* jk/sq-dequote-on-bogus-input:
sq_dequote: fix extra consumption of source string
"git describe $garbage" stopped giving any errors when the garbage
happens to be a string with 40 hexadecimal letters.
* sb/describe-blob:
describe: confirm that blobs actually exist
"git check-ignore" with multiple paths got confused when one is a
file and the other is a directory, which has been fixed.
* rs/check-ignore-multi:
check-ignore: fix mix of directories and other file types
"git rebase -p" mangled log messages of a merge commit, which is
now fixed.
* js/fix-merge-arg-quoting-in-rebase-p:
rebase -p: fix incorrect commit message when calling `git merge`.
"git add" files in the same directory, but spelling the directory
path in different cases on case insensitive filesystem, corrupted
the name hash data structure and led to unexpected results. This
has been corrected.
* bp/name-hash-dirname-fix:
name-hash: properly fold directory names in adjust_dirname_case()
Some bugs around "untracked cache" feature have been fixed.
* nd/fix-untracked-cache-invalidation:
dir.c: ignore paths containing .git when invalidating untracked cache
dir.c: stop ignoring opendir() error in open_cached_dir()
dir.c: fix missing dir invalidation in untracked code
dir.c: avoid stat() in valid_cached_dir()
status: add a failing test showing a core.untrackedCache bug
Test fixes.
* sg/test-i18ngrep:
t: make 'test_i18ngrep' more informative on failure
t: validate 'test_i18ngrep's parameters
t: move 'test_i18ncmp' and 'test_i18ngrep' to 'test-lib-functions.sh'
t5536: let 'test_i18ngrep' read the file without redirection
t5510: consolidate 'grep' and 'test_i18ngrep' patterns
t4001: don't run 'git status' upstream of a pipe
t6022: don't run 'git merge' upstream of a pipe
t5812: add 'test_i18ngrep's missing filename parameter
t5541: add 'test_i18ngrep's missing filename parameter
"git format-patch" learned to give 72-cols to diffstat, which is
consistent with other line length limits the subcommand uses for
its output meant for e-mails.
* nd/format-patch-stat-width:
format-patch: reduce patch diffstat width to 72
format-patch: keep cover-letter diffstat wrapped in 72 columns
More tests for wildmatch functions.
* ab/wildmatch-tests:
wildmatch test: mark test as EXPENSIVE_ON_WINDOWS
test-lib: add an EXPENSIVE_ON_WINDOWS prerequisite
wildmatch test: create & test files on disk in addition to in-memory
wildmatch test: perform all tests under all wildmatch() modes
wildmatch test: use test_must_fail, not ! for test-wildmatch
wildmatch test: remove dead fnmatch() test code
wildmatch test: use a paranoia pattern from nul_match()
wildmatch test: don't try to vertically align our output
wildmatch test: use more standard shell style
wildmatch test: indent with tabs, not spaces
* en/merge-recursive-fixes:
merge-recursive: add explanation for src_entry and dst_entry
merge-recursive: fix logic ordering issue
Tighten and correct a few testcases for merging and cherry-picking
There are a small number of misspellings, ".gitmodule", scattered
throughout the code base, correct them ... no apparent functional
changes.
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This fixes a (probably harmless) parsing problem in
sq_dequote_step(), in which we parse some bogus input
incorrectly rather than complaining that it's bogus.
Our shell-dequoting function is very strict: it can unquote
everything generated by sq_quote(), but not arbitrary
strings. In particular, it only allows characters outside of
the single-quoted string if they are immediately backslashed
and then the single-quoted string is resumed. So:
'foo'\''bar'
is OK. But these are not:
'foo'\'bar
'foo'\'
'foo'\'\''bar'
even though they are all valid shell. The parser has a funny
corner case here. When we see a backslashed character, we
keep incrementing the "src" pointer as we parse it. For a
single sq_dequote() call, that's OK; our next step is to
bail with an error, and we don't care where "src" points.
But if we're parsing multiple strings with sq_dequote_to_argv(),
then our next step is to see if the string is followed by
whitespace. Because we erroneously incremented the "src"
pointer, we don't barf on the bogus backslash that we
skipped. Instead, we may find whitespace that immediately
follows it, and continue as if all is well (skipping the
backslashed character completely!).
In practice, this shouldn't be a big deal. The input is
bogus, and our sq_quote() would never generate this bogus
input. In all but one callers, we are parsing input created
by an earlier call to sq_quote(). That final case is "git
shell", which parses shell-quoting generated by the client.
And in that case we use the singular sq_quote(), which has
always behaved correctly.
One might also wonder if you could provoke a read past the
end of the string. But the answer is no; we still parse
character by character, and would never advance past a NUL.
This patch implements the minimal fix, along with
documenting the restriction (which confused at least me
while reading the code). We should possibly consider
being more liberal in accepting valid shell-quoted words. I
suspect the code may actually be simpler, and it would be
more friendly to anybody generating or editing input by
hand. But I wanted to fix just the immediate bug in this
patch.
We don't have a direct way to unit-test the sq_dequote()
functions, but we can do this by feeding input to
GIT_CONFIG_PARAMETERS (which is not normally a user-facing
interface, but serves here as it expects to see sq_quote()
input from "git -c"). I've included both a bogus example,
and a related "good" one to confirm that we still parse it
correctly.
Noticed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The hashmap API always use an unsigned value for storing
and comparing hashes. Whereas this test code uses "int".
This works out in practice since one can typically
round-trip between "int" and "unsigned int". But since this
is essentially reference code for the hashmap API, we should
model using the correct types.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This function takes two ptr/len pairs, which implies that
they can be arbitrary buffers. But internally, it assumes
that each "ptr" is NUL-terminated at "len" (because we
memcpy an extra byte to pick up the NUL terminator).
In practice this works because each caller only ever passes
strlen(ptr) as the length. But let's drop the "len"
parameters to make our expectations clear.
Note that we can get rid of the "l1" and "l2" variables from
cmd_main() as a further cleanup, since they are now mostly
used to check whether the p1 and p2 arguments are present
(technically the length parameters conflated NULL with the
empty string, which we no longer do, but I think that is
actually an improvement).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using fgets() with a fixed-size buffer can lead to lines
being accidentally split across two calls if they are larger
than the buffer size.
As this is just a test helper, this is unlikely to be a
problem in practice. But since people may look at test
helpers as reference code, it's a good idea for them to
model the preferred behavior.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In general, using a bare snprintf can truncate the resulting
buffer, leading to confusing results. In this case we know
that our buffer is sized large enough to accommodate our
loop, so there's no bug. However, we should use xsnprintf()
to document (and check) that assumption, and to model good
practice to people reading the code.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we allocate the test_entry flex-struct, we have to add
up all of the elements that go into the flex array. If these
were to overflow a size_t, this would allocate a too-small
buffer, which we would then overflow in our memcpy steps.
Since this is just a test-helper, it probably doesn't matter
in practice, but we should model the correct technique by
using the st_add() macros.
Unfortunately, we cannot use the FLEX_ALLOC() macros here,
because we are stuffing two different buffers into a single
flex array.
While we're here, let's also swap out "malloc" for our
error-checking "xmalloc", and use the preferred
"sizeof(*var)" instead of "sizeof(type)".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These two array allocations have several minor flaws:
- they use bare malloc, rather than our error-checking
xmalloc
- they do a bare multiplication to determine the total
size (which in theory can overflow, though in this case
the sizes are all constants)
- they use sizeof(type), but the type in the second one
doesn't match the actual array (though it's "int" versus
"unsigned int", which are guaranteed by C99 to have the
same size)
None of these are likely to be problems in practice, and
this is just a test helper. But since people often look at
test helpers as reference code, we should do our best to
model the recommended techniques.
Switching to ALLOC_ARRAY fixes all three.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The sequencer infrastructure is shared across "git cherry-pick",
"git rebase -i", etc., and has always spawned "git commit" when it
needs to create a commit. It has been taught to do so internally,
when able, by reusing the codepath "git commit" itself uses, which
gives performance boost for a few tens of percents in some sample
scenarios.
* pw/sequencer-in-process-commit:
sequencer: run 'prepare-commit-msg' hook
t7505: add tests for cherry-pick and rebase -i/-p
t7505: style fixes
sequencer: assign only free()able strings to gpg_sign
sequencer: improve config handling
t3512/t3513: remove KNOWN_FAILURE_CHERRY_PICK_SEES_EMPTY_COMMIT=1
sequencer: try to commit without forking 'git commit'
sequencer: load commit related config
sequencer: simplify adding Signed-off-by: trailer
commit: move print_commit_summary() to libgit
commit: move post-rewrite code to libgit
Add a function to update HEAD after creating a commit
commit: move empty message checks to libgit
t3404: check intermediate squash messages
Code clean-up.
* nd/shared-index-fix:
read-cache: don't write index twice if we can't write shared index
read-cache.c: move tempfile creation/cleanup out of write_shared_index
read-cache.c: change type of "temp" in write_shared_index()
The split-index mode had a few corner case bugs fixed.
* tg/split-index-fixes:
travis: run tests with GIT_TEST_SPLIT_INDEX
split-index: don't write cache tree with null oid entries
read-cache: fix reading the shared index for other repos
The http tracing code, often used to debug connection issues,
learned to redact potentially sensitive information from its output
so that it can be more safely sharable.
* jt/http-redact-cookies:
http: support omitting data from traces
http: support cookie redaction when tracing
The tracing machinery learned to report tweaking of environment
variables as well.
* nd/trace-with-env:
run-command.c: print new cwd in trace_run_command()
run-command.c: print env vars in trace_run_command()
run-command.c: print program 'git' when tracing git_cmd mode
run-command.c: introduce trace_run_command()
trace.c: move strbuf_release() out of print_trace_line()
trace: avoid unnecessary quoting
sq_quote_argv: drop maxlen parameter
The machinery to clone & fetch, which in turn involves packing and
unpacking objects, have been told how to omit certain objects using
the filtering mechanism introduced by the jh/object-filtering
topic, and also mark the resulting pack as a promisor pack to
tolerate missing objects, taking advantage of the mechanism
introduced by the jh/fsck-promisors topic.
* jh/partial-clone:
t5616: test bulk prefetch after partial fetch
fetch: inherit filter-spec from partial clone
t5616: end-to-end tests for partial clone
fetch-pack: restore save_commit_buffer after use
unpack-trees: batch fetching of missing blobs
clone: partial clone
partial-clone: define partial clone settings in config
fetch: support filters
fetch: refactor calculation of remote list
fetch-pack: test support excluding large blobs
fetch-pack: add --no-filter
fetch-pack, index-pack, transport: partial clone
upload-pack: add object filtering for partial clone
In preparation for implementing narrow/partial clone, the machinery
for checking object connectivity used by gc and fsck has been
taught that a missing object is OK when it is referenced by a
packfile specially marked as coming from trusted repository that
promises to make them available on-demand and lazily.
* jh/fsck-promisors:
gc: do not repack promisor packfiles
rev-list: support termination at promisor objects
sha1_file: support lazily fetching missing objects
introduce fetch-object: fetch one promisor object
index-pack: refactor writing of .keep files
fsck: support promisor objects as CLI argument
fsck: support referenced promisor objects
fsck: support refs pointing to promisor objects
fsck: introduce partialclone extension
extension.partialclone: introduce partial clone extension
The build procedure for perl/ part has been greatly simplified by
weaning ourselves off of MakeMaker.
* ab/simplify-perl-makefile:
perl: treat PERLLIB_EXTRA as an extra path again
perl: avoid *.pmc and fix Error.pm further
Makefile: replace perl/Makefile.PL with simple make rules
Small changes in messages to fit the style and typography of rest.
Reuse already translated messages if possible.
Do not translate messages aimed at developers of git.
Fix unit tests depending on the original string.
Use `test_i18ngrep` for tests with translatable strings.
Change and verify rest of tests via `make GETTEXT_POISON=1 test`.
Signed-off-by: Alexander Shopov <ash@kambanaria.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In check_ignore(), the first pathspec item determines the dtype for any
subsequent ones. That means that a pathspec matching a regular file can
prevent following pathspecs from matching directories, which makes no
sense. Fix that by determining the dtype for each pathspec separately,
by passing the value DT_UNKNOWN to last_exclude_matching() each time.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Prior to 644eb60bd0 (builtin/describe.c: describe a blob,
2017-11-15), we noticed and complained about missing
objects, since they were not valid commits:
$ git describe 0000000000000000000000000000000000000000
fatal: 0000000000000000000000000000000000000000 is not a valid 'commit' object
After that commit, we feed any non-commit to lookup_blob(),
and complain only if it returns NULL. But the lookup_*
functions do not actually look at the on-disk object
database at all. They return an entry from the in-memory
object hash if present (and if it matches the requested
type), and otherwise auto-create a "struct object" of the
requested type.
A missing object would hit that latter case: we create a
bogus blob struct, walk all of history looking for it, and
then exit successfully having produced no output.
One reason nobody may have noticed this is that some related
cases do still work OK:
1. If we ask for a tree by sha1, then the call to
lookup_commit_referecne_gently() would have parsed it,
and we would have its true type in the in-memory object
hash.
2. If we ask for a name that doesn't exist but isn't a
40-hex sha1, then get_oid() would complain before we
even look at the objects at all.
We can fix this by replacing the lookup_blob() call with a
check of the true type via sha1_object_info(). This is not
quite as efficient as we could possibly make this check. We
know in most cases that the object was already parsed in the
earlier commit lookup, so we could call lookup_object(),
which does auto-create, and check the resulting struct's
type (or NULL). However it's not worth the fragility nor
code complexity to save a single object lookup.
The new tests cover this case, as well as that of a
tree-by-sha1 (which does work as described above, but was
not explicitly tested).
Noticed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This ancient test script does a lot of manual checking of
test conditions with "if" blocks. We can simplify this
by relying on helpers like test_must_fail.
Note that a failing "grep" call here won't produce any
verbose output, but that's OK. These days we rely on "-x" to
tell us about such commands. And in addition, these greps
are soon to be converted to test_i18ngrep (which is itself
soon learning to be more verbose).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 'test_might_fail' is implemented as a thin wrapper around
'test_must_fail', it also accepts the same options. Mention this in
the docs as well.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit dd6fb0053 ("rebase -p: fix quoting when calling `git
merge`"), commit message of the merge commit being rebased is passed to
the merge command using a subshell executing 'git rev-parse --sq-quote'.
Double quotes are needed around this subshell so that, newlines are
kept for the git merge command.
Before this patch, following merge message:
"Merge mybranch into mynewbranch
Awesome commit."
becomes:
"Merge mybranch into mynewbranch Awesome commit."
after a rebase -p.
Fixes: "dd6fb0053 rebase -p: fix quoting when calling `git merge`"
Reported-by: Jamie Iles <jamie.iles@oracle.com>
Suggested-by: Vegard Nossum <vegard.nossum@oracle.com>
Suggested-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Correct the pointer arithmetic in adjust_dirname_case() so that it calls
find_dir_entry() with the correct string length. Previously passing in
"dir1/foo" would pass a length of 6 instead of the correct 4. This resulted in
find_dir_entry() never finding the entry and so the subsequent memcpy that would
fold the name to the version with the correct case never executed.
Add a test to validate the corrected behavior with name folding of directories.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When 'test_i18ngrep' can't find the expected pattern, it exits
completely silently; when its negated form does find the pattern that
shouldn't be there, it prints the matching line(s) but otherwise exits
without any error message. This leaves the developer puzzled about
what could have gone wrong.
Make 'test_i18ngrep' more informative on failure by printing an error
message including the invoked 'grep' command and the contents of the
file it had to scan through.
Note that this "dump the scanned file" part is not quite perfect, as
it dumps only the file specified as the function's last positional
parameter, thus assuming that there is only a single file parameter.
I think that's a reasonable assumption to make, one that holds true in
the current code base. And even if someone were to scan multiple
files at once in the future, the worst thing that could happen is that
the verbose error message won't include the contents of all those
files, only the last one. Alas, we can't really do any better than
this, because checking whether the other positional parameters match a
filename can result in false positives: 't3400-rebase.sh' and
't3404-rebase-interactive.sh' contain one test each, where the
'test_i18ngrep's pattern verbatimly matches a file in the trash
directory.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some of the previous patches in this series fixed bogus
'test_i18ngrep' invocations:
- Two invocations where the tested git command's standard output is
directly piped into 'test_i18ngrep'. While convenient, this is an
antipattern, because the pipe hides the git command's exit code,
and the test could continue even if the command exited with error.
- Two invocations that had neither a filename parameter nor anything
piped into their standard input, yet both managed to remain
unnoticed for years. A third similarly bogus invocation is
currently lurking in 'pu' for a couple of weeks now.
Prevent similar mistakes in the future by validating 'test_i18ngrep's
parameters requiring that
- The last parameter names an existing file to be read, effectively
forbidding piping into 'test_i18ngrep'.
Note that this change will also forbid cases where 'test_i18ngrep'
would legitimately read its standard input, e.g. when its standard
input is redirected from a file, or when a git command's standard
output is first written to an intermediate file, which is then
preprocessed by a non-git command before the results are piped
into 'test_i18ngrep'. See two of the previous patches for the
only such cases we had in our test suite. However, reliably
preventing the piping antipattern is arguably more important than
supporting these cases, which can be easily worked around by
opening the file directly or using an intermediate file anyway.
- There are at least two parameters, not including the optional '!'
to negate the pattern. This ought to catch corner cases when
'test_i18ngrep' looks for the name of an existing file on its
standard input; the above check would miss this case becase the
filename as pattern would be the last parameter.
Note that this is not quite perfect, as it doesn't account for any
'grep --options' given as parameters. However, doing so would be
far too complicated, considering that patterns can start with
dashes as well, and in the majority of the cases we don't use any
such options anyway.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Both 'test_i18ncmp' and 'test_i18ngrep' helper functions are supposed
to be called from our test scripts, so they should be in
'test-lib-functions.sh'.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Redirecting 'test_i18ngrep's standard input from a file will interfere
with the linting that will be added in a later patch.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
One of the tests in 't5510-fetch.sh' checks the output of 'git fetch'
using 'test_i18ngrep', and while doing so it prefilters the output
with 'grep' before piping the result into 'test_i18ngrep'.
This prefiltering is unnecessary, with the appropriate pattern
'test_i18ngrep' can do it all by itself. Furthermore, piping data
into 'test_i18ngrep' will interfere with the linting that will be
added in a later patch.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano