kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Alexandr Miloslavskiy	3d7747e318	real_path: remove unsafe API Returning a shared buffer invites very subtle bugs due to reentrancy or multi-threading, as demonstrated by the previous patch. There was an unfinished effort to abolish this [1]. Let's finally rid of `real_path()`, using `strbuf_realpath()` instead. This patch uses a local `strbuf` for most places where `real_path()` was previously called. However, two places return the value of `real_path()` to the caller. For them, a `static` local `strbuf` was added, effectively pushing the problem one level higher: read_gitfile_gently() get_superproject_working_tree() [1] https://lore.kernel.org/git/1480964316-99305-1-git-send-email-bmwill@google.com/ Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	d2c84dad1c	mingw: refuse to access paths with trailing spaces or periods When creating a directory on Windows whose path ends in a space or a period (or chains thereof), the Win32 API "helpfully" trims those. For example, `mkdir("abc ");` will return success, but actually create a directory called `abc` instead. This stems back to the DOS days, when all file names had exactly 8 characters plus exactly 3 characters for the file extension, and the only way to have shorter names was by padding with spaces. Sadly, this "helpful" behavior is a bit inconsistent: after a successful `mkdir("abc ");`, a `mkdir("abc /def")` will actually _fail_ (because the directory `abc ` does not actually exist). Even if it would work, we now have a serious problem because a Git repository could contain directories `abc` and `abc `, and on Windows, they would be "merged" unintentionally. As these paths are illegal on Windows, anyway, let's disallow any accesses to such paths on that Operating System. For practical reasons, this behavior is still guarded by the config setting `core.protectNTFS`: it is possible (and at least two regression tests make use of it) to create commits without involving the worktree. In such a scenario, it is of course possible -- even on Windows -- to create such file names. Among other consequences, this patch disallows submodules' paths to end in spaces on Windows (which would formerly have confused Git enough to try to write into incorrect paths, anyway). While this patch does not fix a vulnerability on its own, it prevents an attack vector that was exploited in demonstrations of a number of recently-fixed security bugs. The regression test added to `t/t7417-submodule-path-url.sh` reflects that attack vector. Note that we have to adjust the test case "prevent git~1 squatting on Windows" in `t/t7415-submodule-names.sh` because of a very subtle issue. It tries to clone two submodules whose names differ only in a trailing period character, and as a consequence their git directories differ in the same way. Previously, when Git tried to clone the second submodule, it thought that the git directory already existed (because on Windows, when you create a directory with the name `b.` it actually creates `b`), but with this patch, the first submodule's clone will fail because of the illegal name of the git directory. Therefore, when cloning the second submodule, Git will take a different code path: a fresh clone (without an existing git directory). Both code paths fail to clone the second submodule, both because the the corresponding worktree directory exists and is not empty, but the error messages are worded differently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>	5 years ago
Garima Singh	a62f9d1ace	test-path-utils: offer to run a protectNTFS/protectHFS benchmark In preparation to flipping the default on `core.protectNTFS`, let's have some way to measure the speed impact of this config setting reliably (and for comparison, the `core.protectHFS` config setting). For now, this is a manual performance benchmark: ./t/helper/test-path-utils protect_ntfs_hfs [arguments...] where the arguments are an optional number of file names to test with, optionally followed by minimum and maximum length of the random file names. The default values are one million, 3 and 20, respectively. Just like `sqrti()` in `bisect.c`, we introduce a very simple function to approximation the square root of a given value, in order to avoid having to introduce the first user of `<math.h>` in Git's source code. Note: this is _not_ implemented as a Unix shell script in t/perf/ because we really care about _very_ precise timings here, and Unix shell scripts are simply unsuited for precise and consistent benchmarking. Signed-off-by: Garima Singh <garima.singh@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>	5 years ago
Johannes Schindelin	b819f1d2ce	ci: parallelize testing on Windows The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice. For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines. To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them. The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the `test-tool`. To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one). We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	af9912efaf	tests: include detailed trace logs with --write-junit-xml upon failure The JUnit XML format lends itself to be presented in a powerful UI, where you can drill down to the information you are interested in very quickly. For test failures, this usually means that you want to see the detailed trace of the failing tests. With Travis CI, we passed the `--verbose-log` option to get those traces. However, that seems excessive, as we do not need/use the logs in almost all of those cases: only when a test fails do we have a way to include the trace. So let's do something different when using Azure DevOps: let's run all the tests with `--quiet` first, and only if a failure is encountered, try to trace the commands as they are executed. Of course, we cannot turn on `--verbose-log` after the fact. So let's just re-run the test with all the same options, adding `--verbose-log`. And then munging the output file into the JUnit XML on the fly. Note: there is an off chance that re-running the test in verbose mode "fixes" the failures (and this does happen from time to time!). That is a possibility we should be able to live with. Ideally, we would label this as "Passed upon rerun", and Azure Pipelines even know about that outcome, but it is not available when using the JUnit XML format for now: https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	5868bd8620	tests: avoid calling Perl just to determine file sizes It is a bit ridiculous to spin up a full-blown Perl instance (especially on Windows, where that means spinning up a full POSIX emulation layer, AKA the MSYS2 runtime) just to tell how large a given file is. So let's just use the test-tool to do that job instead. This command will also be used over the next commits, to allow for cutting out individual test cases' verbose log from the file generated via --verbose-log. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	dc2d9ba318	is_{hfs,ntfs}_dotgitmodules: add tests This tests primarily for NTFS issues, but also adds one example of an HFS+ issue. Thanks go to Congyi Wu for coming up with the list of examples where NTFS would possibly equate the filename with `.gitmodules`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jeff King <peff@peff.net>	7 years ago
Nguyễn Thái Ngọc Duy	b8d5cf4f96	t/helper: merge test-path-utils into test-tool Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
René Scharfe	29c2eda80b	test-path-utils: handle const parameter of basename and dirname The parameter to basename(3) and dirname(3) traditionally had the type "char ", but on OpenBSD it's been "const char " for years. That causes (at least) Clang to throw an incompatible-pointer-types warning for test-path-utils, where we try to pass around pointers to these functions. Avoid this warning (which is fatal in DEVELOPER mode) by ignoring the promise of OpenBSD's implementations to keep input strings unmodified and enclosing them in POSIX-compatible wrappers. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	8 years ago
Nguyễn Thái Ngọc Duy	e6e7530d10	test helpers: move test-* to t/helper/ subdirectory This keeps top dir a bit less crowded. And because these programs are for testing purposes, it makes sense that they stay somewhere in t/ Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Jeff King	62f17513e7	test-path-utils: fix normalize_path_copy output buffer size The normalize_path_copy function needs an output buffer that is at least as long as its input (it may shrink the path, but never expand it). However, this test program feeds it static PATH_MAX-sized buffers, which have no relation to the input size. In the normalize_ceiling_entry case, we do at least check the size against PATH_MAX and die(), but that case is even more convoluted. We normalize into a fixed-size buffer, free the original, and then replace it with a strdup'd copy of the result. But normalize_path_copy explicitly allows normalizing in-place, so we can simply do that. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Jeff King	7b11a18a2e	test-path-utils: use xsnprintf in favor of strcpy This strcpy will never overflow because it's copying from baked-in test data. But we would prefer to avoid strcpy entirely, as it makes it harder to audit for real security bugs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Johannes Schindelin	371471cea3	t0060: loosen overly strict expectations The dirname() tests file were developed and tested on only the five platforms available to the developer at the time, namely: Linux (both 32 and 64bit), Windows XP 32-bit (MSVC), MinGW 32-bit and Cygwin 32-bit. http://pubs.opengroup.org/onlinepubs/9699919799/functions/basename.html (i.e. the POSIX spec) says, in part: If the string pointed to by path consists entirely of the '/' character, basename() shall return a pointer to the string "/". If the string pointed to by path is exactly "//", it is implementation-defined whether "/" or "//" is returned. The thinking behind testing precise, OS-dependent output values was to document that different setups produce different values. However, as the test failures on MacOSX illustrated eloquently: hardcoding pretty much each and every setup's expectations is pretty fragile. This is not limited to the "//" vs "/" case, of course, other inputs are also allowed to produce multiple outputs by the POSIX specs. So let's just test for all allowed values and be done with it. This still documents that Git cannot rely on one particular output value in those cases, so the intention of the original tests is still met. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Johannes Schindelin	7d1aaa684d	t0060: verify that basename() and dirname() work as expected Unfortunately, some libgen implementations yield outcomes different from what Git expects. For example, mingw-w64-crt provides a basename() function, that shortens `path0/` to `path`! So let's verify that the basename() and dirname() functions we use conform to what Git expects. Derived-from-code-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Ray Donnelly	b2a7123b99	test-path-utils.c: remove incorrect assumption In normalize_ceiling_entry(), we test that normalized paths end with slash, unless the path to be normalized was already the root directory. However, normalize_path_copy() does not even enforce this condition. Even worse: on Windows, the root directory gets translated into a Windows directory by the Bash before being passed to `git.exe` (or `test-path-utils.exe`), which means that we cannot even know whether the path that was passed to us was the root directory to begin with. This issue has already caused endless hours of trying to "fix" the MSYS2 runtime, only to break other things due to MSYS2 ensuring that the converted path maintains the same state as the input path with respect to any final '/'. So let's just forget about this test. It is non-essential to Git's operation, anyway. Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Ray Donnelly <mingw.android@gmail.com>	10 years ago
Sebastian Schuberth	7ffd18fce1	path-utils test: rename mingw_path function to print_path mingw_path was introduced in `abd4284` to output a mangled path as it is passed as an argument to main(). But the name is misleading because mangling does not come from MinGW, but from MSYS [1]. As `abd4284` does not introduce any MSYS or MinGW specific code but just prints out argv[2] as it is passed to main(), give the function the more generic and less confusing name "print_path". [1] http://www.mingw.org/wiki/Posix_path_conversion Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>	12 years ago
Jiang Xin	abd4284bc6	test: run testcases with POSIX absolute paths on Windows Some test cases are skipped on Windows by marking with POSIX prereq. This is because arguments look like absolute paths (such as /a/b) for regular Windows programs (*.exe executables, no bash scripts) are changed to Windows paths (like C:/msysgit/a/b). There is no cygpath nor equivalent on msysGit, but it is easy to write one. New subcommand "mingw_path" is added in test-path-utils, so that we can get the expected absolute paths on Windows. E.g. COMMAND LINE Linux output Windows output ================================== ============ =============== test-path-utils mingw_path / / C:/msysgit test-path-utils mingw_path /a/b/ /a/b/ C:/msysgit/a/b/ With this utility, most skipped test cases in t0060 can be turned on to be tested correctly on Windows. Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	12 years ago
Jiang Xin	e02ca72f70	path.c: refactor relative_path(), not only strip prefix Original design of relative_path() is simple, just strip the prefix (base) from the absolute path (abs). In most cases, we need a real relative path, such as: ../foo, ../../bar. That's why there is another reimplementation (path_relative()) in quote.c. Borrow some codes from path_relative() in quote.c to refactor relative_path() in path.c, so that it could return real relative path, and user can reuse this function without reimplementing his/her own. The function path_relative() in quote.c will be substituted, and I would use the new relative_path() function when implementing the interactive git-clean later. Different results for relative_path() before and after this refactor: abs path base path relative (original) relative (refactor) ======== ========= =================== =================== /a/b /a/b . ./ /a/b/ /a/b . ./ /a /a/b/ /a ../ / /a/b/ / ../../ /a/c /a/b/ /a/c ../c /x/y /a/b/ /x/y ../../x/y a/b/ a/b/ . ./ a/b/ a/b . ./ a a/b a ../ x/y a/b/ x/y ../../x/y a/c a/b a/c ../c (empty) (null) (empty) ./ (empty) (empty) (empty) ./ (empty) /a/b (empty) ./ (null) (null) (null) ./ (null) (empty) (null) ./ (null) /a/b (segfault) ./ You may notice that return value "." has been changed to "./". It is because: * Function quote_path_relative() in quote.c will show the relative path as "./" if abs(in) and base(prefix) are the same. * Function relative_path() is called only once (in setup.c), and it will be OK for the return value as "./" instead of ".". Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	12 years ago
Jiang Xin	203439b284	test: add test cases for relative_path Add subcommand "relative_path" in test-path-utils, and add test cases in t0060. Johannes tested an earlier version of this patch on Windows, and found that some relative_path tests should be skipped on Windows. This is because the bash on Windows rewrites arguments of regular Windows programs, such as git and the test helpers, if the arguments look like absolute POSIX paths. As a consequence, the actual tests performed are not what the tests scripts expect. The tests that need not be skipped are those where the two paths passed to 'test-path-utils relative_path' have the same prefix and the result is expected to be a relative path. This is because the rewriting changes "/a/b" to "D:/Src/MSysGit/a/b", and when both inputs are extended the same way, this just cancels out in the relative path computation. Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>	12 years ago
Michael Haggerty	9e2326c7e1	longest_ancestor_length(): require prefix list entries to be normalized Move the responsibility for normalizing prefixes from longest_ancestor_length() to its callers. Use slightly different normalizations at the two callers: In setup_git_directory_gently_1(), use the old normalization, which ignores paths that are not usable. In the next commit we will change this caller to also resolve symlinks in the paths from GIT_CEILING_DIRECTORIES as part of the normalization. In "test-path-utils longest_ancestor_length", use the old normalization, but die() if any paths are unusable. Also change t0060 to only pass normalized paths to the test program (no empty entries or non-absolute paths, strip trailing slashes from the paths, and remove tests that thereby become redundant). The point of this change is to reduce the scope of the ancestor_length tests in t0060 from testing normalization+longest_prefix to testing only mostly longest_prefix. This is necessary because when setup_git_directory_gently_1() starts resolving symlinks as part of its normalization, it will not be reasonable to do the same in the test suite, because that would make the test results depend on the contents of the root directory of the filesystem on which the test is run. HOWEVER: under Windows, bash mangles arguments that look like absolute POSIX paths into DOS paths. So we have to retain the level of normalization done by normalize_path_copy() to convert the bash-mangled DOS paths (which contain backslashes) into paths that use forward slashes. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>	13 years ago
Michael Haggerty	31171d9e45	longest_ancestor_length(): take a string_list argument for prefixes Change longest_ancestor_length() to take the prefixes argument as a string_list rather than as a colon-separated string. This will make it easier for the caller to alter the entries before calling longest_ancestor_length(). Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net>	13 years ago
Michael Haggerty	9e8137238d	test-path-utils: Add subcommand "prefix_path" Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>	14 years ago
Michael Haggerty	87a246e1b5	test-path-utils: Add subcommand "absolute_path" Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>	14 years ago
Carlos Martín Nieto	e2a57aac8a	Name make__path functions more accurately Rename the make__path functions so it's clearer what they do, in particlar make clear what the differnce between make_absolute_path and make_nonrelative_path is by renaming them real_path and absolute_path respectively. make_relative_path has an understandable name and is renamed to relative_path to maintain the name convention. The function calls have been replaced 1-to-1 in their usage. Signed-off-by: Carlos Martín Nieto <cmn@elego.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	14 years ago
Johannes Schindelin	4fcc86b07d	Introduce the function strip_path_suffix() The function strip_path_suffix() will try to strip a given suffix from a given path. The suffix must start at a directory boundary (i.e. "core" is not a path suffix of "libexec/git-core", but "git-core" is). Arbitrary runs of directory separators ("slashes") are assumed identical. Example: strip_path_suffix("C:\\msysgit/\\libexec\\git-core", "libexec///git-core", &prefix) will set prefix to "C:\\msysgit" and return 0. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Acked-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
Johannes Sixt	f42302b493	Test and fix normalize_path_copy() This changes the test-path-utils utility to invoke normalize_path_copy() instead of normalize_absolute_path() because the latter is about to be removed. The test cases in t0060 are adjusted in two regards: - normalize_path_copy() more often leaves a trailing slash in the result. This has no negative side effects because the new user of this function, longest_ancester_length(), already accounts for this behavior. - The function can fail. The tests uncover a flaw in normalize_path_copy(): If there are sufficiently many '..' path components so that the root is reached, such as in "/d1/s1/../../d2", then the leading slash was lost. This manifested itself that (assuming there is a repository at /tmp/foo) $ git add /d1/../tmp/foo/some-file reported 'pathspec is outside repository'. This is now fixed. Moreover, the test case descriptions of t0060 now include the test data and expected outcome. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
Johannes Sixt	2cd85c40a9	Make test-path-utils more robust against incorrect use Previously, this test utility happily returned with exit code 0 if garbage was thrown at it. Now it reports failure if an unknown function name was given on the command line. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
Johannes Schindelin	b8469ad057	test-path-utils: Fix off by one, found by valgrind When normalizing an absolute path, we might have to add a slash _and_ a NUL to the buffer, so the buffer was one too small. Let's just future proof the code and alloc PATH_MAX + 1 bytes. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	16 years ago
David Reiss	0454dd93bf	Add support for GIT_CEILING_DIRECTORIES Make git recognize a new environment variable that prevents it from chdir'ing up into specified directories when looking for a GIT_DIR. Useful for avoiding slow network directories. For example, I use git in an environment where homedirs are automounted and "ls /home/nonexistent" takes about 9 seconds. Setting GIT_CEILING_DIRS="/home" allows "git help -a" (for bash completion) and "git symbolic-ref" (for my shell prompt) to run in a reasonable time. Signed-off-by: David Reiss <dreiss@facebook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	17 years ago
David Reiss	d553e73789	Fold test-absolute-path into test-path-utils Signed-off-by: David Reiss <dreiss@facebook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	17 years ago
David Reiss	ae299be0e5	Implement normalize_absolute_path normalize_absolute_path removes several oddities form absolute paths, giving nice clean paths like "/dir/sub1/sub2". Also add a test case for this utility, based on a new test program (in the style of test-sha1). Signed-off-by: David Reiss <dreiss@facebook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	17 years ago

15 Commits (5602b500c3cd9ac308bf9af0d5f0a79bd2195346)