kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Johannes Schindelin	9a780a384d	mingw: spawned processes need to inherit only standard handles By default, CreateProcess() does not inherit any open file handles, unless the bInheritHandles parameter is set to TRUE. Which we do need to set because we need to pass in stdin/stdout/stderr to talk to the child processes. Sadly, this means that all file handles (unless marked via O_NOINHERIT) are inherited. This lead to problems in VFS for Git, where a long-running read-object hook is used to hydrate missing objects, and depending on the circumstances, might only be called after Git opened a file handle. Ideally, we would not open files without O_NOINHERIT unless really necessary (i.e. when we want to pass the opened file handle as standard handle into a child process), but apparently it is all-too-easy to introduce incorrect open() calls: this happened, and prevented updating a file after the read-object hook was started because the hook still held a handle on said file. Happily, there is a solution: as described in the "Old New Thing" https://blogs.msdn.microsoft.com/oldnewthing/20111216-00/?p=8873 there is a way, starting with Windows Vista, that lets us define precisely which handles should be inherited by the child process. And since we bumped the minimum Windows version for use with Git for Windows to Vista with v2.10.1 (i.e. a long time ago), we can use this method. So let's do exactly that. We need to make sure that the list of handles to inherit does not contain duplicates; Otherwise CreateProcessW() would fail with ERROR_INVALID_ARGUMENT. While at it, stop setting errno to ENOENT unless it really is the correct value. Also, fall back to not limiting handle inheritance under certain error conditions (e.g. on Windows 7, which is a lot stricter in what handles you can specify to limit to). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Johannes Schindelin	eea4a7f4b3	mingw: demonstrate that all file handles are inherited by child processes When spawning child processes, we really should be careful which file handles we let them inherit. This is doubly important on Windows, where we cannot rename, delete, or modify files if there is still a file handle open. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Alexandr Miloslavskiy	71f4960b91	t0061: fix test for argv[0] with spaces (MINGW only) The test was originally designed for the case where user reported that setting GIT_SSH to a .bat file with spaces in path fails on Windows: https://github.com/git-for-windows/git/issues/692 The test has two different problems: 1. It succeeds with AND without fix `eb7c7863` that addressed user's problem. This happens because the core problem was misunderstood, leading to conclusion that git is unable to start any programs with spaces in path on Win7. But in fact a) Bug only affected cmd.exe scripts, such as .bat scripts b) Bug only happened when cmd.exe received at least two quoted args c) Bug happened on any Windows (verified on Win10). Therefore, correct test must involve .bat script and two quoted args. 2. In Visual Studio build, it fails to run, because 'test-fake-ssh.exe' is copied away from its dependencies 'libiconv.dll' and 'zlib1.dll'. Fix both problems by using .bat script instead of 'test-fake-ssh.exe'. NOTE: With this change, the test now correctly fails without `eb7c7863`. Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	eb7c786314	mingw: support spawning programs containing spaces in their names On some older Windows versions (e.g. Windows 7), the CreateProcessW() function does not really support spaces in its first argument, lpApplicationName. But it supports passing NULL as lpApplicationName, which makes it figure out the application from the (possibly quoted) first argument of lpCommandLine. Let's use that trick (if we are certain that the first argument matches the executable's path) to support launching programs whose path contains spaces. We will abuse the test-fake-ssh.exe helper to verify that this works and does not regress. This fixes https://github.com/git-for-windows/git/issues/692 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	06718d4a1e	t0061: workaround issues with --with-dashes and RUNTIME_PREFIX When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect a system prefix. The reason is that the RUNTIME_PREFIX feature wants to use the location of the Git executable to determine where the support files can be found, e.g. system-wide Git config or the translations. This does not make any sense for the test helpers, though, as they are distinctly not in a directory structure resembling the final installation location of Git. That is the reason why the test helpers rely on environment variables to indicate the location of the needed support files, e.g. GIT_TEXTDOMAINDIR. If this information is missing, the output will contain warnings like this one: RUNTIME_PREFIX requested, but prefix computation failed. [...] In t0061, we did not expect that to happen, and it actually does not happen in the regular case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool needs to know anything about any runtime prefix). However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly instead. So let's just ignore the RUNTIME_PREFIX warning. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Johannes Schindelin	9e9da23c27	mingw: special-case arguments to `sh` The MSYS2 runtime does its best to emulate the command-line wildcard expansion and de-quoting which would be performed by the calling Unix shell on Unix systems. Those Unix shell quoting rules differ from the quoting rules applying to Windows' cmd and Powershell, making it a little awkward to quote command-line parameters properly when spawning other processes. In particular, git.exe passes arguments to subprocesses that are not intended to be interpreted as wildcards, and if they contain backslashes, those are not to be interpreted as escape characters, e.g. when passing Windows paths. Note: this is only a problem when calling MSYS2 executables, not when calling MINGW executables such as git.exe. However, we do call MSYS2 executables frequently, most notably when setting the use_shell flag in the child_process structure. There is no elegant way to determine whether the .exe file to be executed is an MSYS2 program or a MINGW one. But since the use case of passing a command line through the shell is so prevalent, we need to work around this issue at least when executing sh.exe. Let's introduce an ugly, hard-coded test whether argv[0] is "sh", and whether it refers to the MSYS2 Bash, to determine whether we need to quote the arguments differently than usual. That still does not fix the issue completely, but at least it is something. Incidentally, this also fixes the problem where `git clone \\server\repo` failed due to incorrect handling of the backslashes when handing the path to the git-upload-pack process. Further, we need to take care to quote not only whitespace and backslashes, but also curly brackets. As aliases frequently go through the MSYS2 Bash, and as aliases frequently get parameters such as HEAD@{yesterday}, this is really important. As an early version of this patch broke this, let's make sure that this does not regress by adding a test case for that. Helped-by: Kim Gybels <kgybels@infogroep.be> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Junio C Hamano	e5a329a279	run-command: report exec failure In `321fd823` ("run-command: mark path lookup errors with ENOENT", 2018-10-24), we rewrote the logic to execute a command by looking in the directories on $PATH; as a side effect, a request to run a command that is not found on $PATH is noticed even before a child process is forked to execute it. We however stopped to report an exec failure in such a case by mistake. Add a logic to report the error unless silent-exec-failure is requested, to match the original code. Reported-by: John Passaro <john.a.passaro@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Junio C Hamano	89ba9a79ae	t0061: do not fail test if '.' is part of $PATH t0061 creates a script with an unlikely name in the current directory and asks the run_command() API to run it without an explicit path, expecting that the script does not get run. This obviously would not work if the $PATH does contain such an element. Check if the running shell picks up the script without an explicit path to it, and skip the test when it does, as the run_command() API should also run the script in such an (insane) environment. Reported-by: "H.Merijn Brand" <h.m.brand@xs4all.nl> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	6 years ago
Jeff King	321fd82389	run-command: mark path lookup errors with ENOENT Since commit `e3a434468f` (run-command: use the async-signal-safe execv instead of execvp, 2017-04-19), prepare_cmd() does its own PATH lookup for any commands we run (on non-Windows platforms). However, its logic does not match the old execvp call when we fail to find a matching entry in the PATH. Instead of feeding the name directly to execv, execvp would consider that an ENOENT error. By continuing and passing the name directly to execv, we effectively behave as if "." was included at the end of the PATH. This can have confusing and even dangerous results. The fix itself is pretty straight-forward. There's a new test in t0061 to cover this explicitly, and I've also added a duplicate of the ENOENT test to ensure that we return the correct errno for this case. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
SZEDER Gábor	1c5e94f459	tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>' Using 'test_must_be_empty' is shorter and more idiomatic than >empty && test_cmp empty out as it saves the creation of an empty file. Furthermore, sometimes the expected empty file doesn't have such a descriptive name like 'empty', and its creation is far away from the place where it's finally used for comparison (e.g. in 't7600-merge.sh', where two expected empty files are created in the 'setup' test, but are used only about 500 lines later). These cases were found by instrumenting 'test_cmp' to error out the test script when it's used to compare empty files, and then converted manually. Note that even after this patch there still remain a lot of cases where we use 'test_cmp' to check empty files: - Sometimes the expected output is not hard-coded in the test, but 'test_cmp' is used to ensure that two similar git commands produce the same output, and that output happens to be empty, e.g. the test 'submodule update --merge - ignores --merge for new submodules' in 't7406-submodule-update.sh'. - Repetitive common tasks, including preparing the expected results and running 'test_cmp', are often extracted into a helper function, and some of this helper's callsites expect no output. - For the same reason as above, the whole 'test_expect_success' block is within a helper function, e.g. in 't3070-wildmatch.sh'. - Or 'test_cmp' is invoked in a loop, e.g. the test 'cvs update (-p)' in 't9400-git-cvsserver-server.sh'. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Dan Jacques	226c0ddd0d	exec_cmd: RUNTIME_PREFIX on some POSIX systems Enable Git to resolve its own binary location using a variety of OS-specific and generic methods, including: - procfs via "/proc/self/exe" (Linux) - _NSGetExecutablePath (Darwin) - KERN_PROC_PATHNAME sysctl on BSDs. - argv0, if absolute (all, including Windows). This is used to enable RUNTIME_PREFIX support for non-Windows systems, notably Linux and Darwin. When configured with RUNTIME_PREFIX, Git will do a best-effort resolution of its executable path and automatically use this as its "exec_path" for relative helper and data lookups, unless explicitly overridden. Small incidental formatting cleanup of "exec_cmd.c". Signed-off-by: Dan Jacques <dnj@google.com> Thanks-to: Robbie Iannucci <iannucci@google.com> Thanks-to: Junio C Hamano <gitster@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Nguyễn Thái Ngọc Duy	ae6a51f5a1	t/helper: merge test-run-command into test-tool Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Nguyễn Thái Ngọc Duy	c61a975df1	run-command.c: print env vars in trace_run_command() Occasionally submodule code could execute new commands with GIT_DIR set to some submodule. GIT_TRACE prints just the command line which makes it hard to tell that it's not really executed on this repository. Print the env delta (compared to parent environment) in this case. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	7 years ago
Brandon Williams	940283101c	run-command: restrict PATH search to executable files In some situations run-command will incorrectly try (and fail) to execute a directory instead of an executable file. This was observed by having a directory called "ssh" in $PATH before the real ssh and trying to use ssh protoccol, reslting in the following: $ git ls-remote ssh://url fatal: cannot exec 'ssh': Permission denied It ends up being worse and run-command will even try to execute a non-executable file if it preceeds the executable version of a file on the PATH. For example, if PATH=~/bin1:~/bin2:~/bin3 and there exists a directory 'git-hello' in 'bin1', a non-executable file 'git-hello' in bin2 and an executable file 'git-hello' (which prints "Hello World!") in bin3 the following will occur: $ git hello fatal: cannot exec 'git-hello': Permission denied This is due to only checking 'access()' when locating an executable in PATH, which doesn't distinguish between files and directories. Instead use 'is_executable()' which check that the path is to a regular, executable file. Now run-command won't try to execute the directory or non-executable file 'git-hello': $ git hello Hello World! which matches what execvp(3) would have done when asked to execute git-hello with such a $PATH. Reported-by: Brian Hatfield <bhatfield@google.com> Signed-off-by: Brandon Williams <bmwill@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	8 years ago
Brandon Williams	c2d3119d7b	t0061: run_command executes scripts without a #! line Add a test to 't0061-run-command.sh' to ensure that run_command can continue to execute scripts which don't include a '#!' line. As shell scripts are not natively executable on Windows, we use a workaround to check "#!" when running scripts from Git. As this test requires the platform (not with Git's help) to run scripts without "#!", skipt it on Windows. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	8 years ago
Stefan Beller	c553c72eed	run-command: add an asynchronous parallel child processor This allows to run external commands in parallel with ordered output on stderr. If we run external commands in parallel we cannot pipe the output directly to the our stdout/err as it would mix up. So each process's output will flow through a pipe, which we buffer. One subprocess can be directly piped to out stdout/err for a low latency feedback to the user. Example: Let's assume we have 5 submodules A,B,C,D,E and each fetch takes a different amount of time as the different submodules vary in size, then the output of fetches in sequential order might look like this: time --> output: \|---A---\| \|-B-\| \|-------C-------\| \|-D-\| \|-E-\| When we schedule these submodules into maximal two parallel processes, a schedule and sample output over time may look like this: process 1: \|---A---\| \|-D-\| \|-E-\| process 2: \|-B-\| \|-------C-------\| output: \|---A---\|B\|---C-------\|DE So A will be perceived as it would run normally in the single child version. As B has finished by the time A is done, we can dump its whole progress buffer on stderr, such that it looks like it finished in no time. Once that is done, C is determined to be the visible child and its progress will be reported in real time. So this way of output is really good for human consumption, as it only changes the timing, not the actual output. For machine consumption the output needs to be prepared in the tasks, by either having a prefix per line or per block to indicate whose tasks output is displayed, because the output order may not follow the original sequential ordering: \|----A----\| \|--B--\| \|-C-\| will be scheduled to be all parallel: process 1: \|----A----\| process 2: \|--B--\| process 3: \|-C-\| output: \|----A----\|CB This happens because C finished before B did, so it will be queued for output before B. To detect when a child has finished executing, we check interleaved with other actions (such as checking the liveliness of children or starting new processes) whether the stderr pipe still exists. Once a child closed its stderr stream, we assume it is terminating very soon, and use `finish_command()` from the single external process execution interface to collect the exit status. By maintaining the strong assumption of stderr being open until the very end of a child process, we can avoid other hassle such as an implementation using `waitpid(-1)`, which is not implemented in Windows. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	9 years ago
Junio C Hamano	eae69530ae	tests: correct misuses of POSIXPERM POSIXPERM requires that a later call to stat(2) (hence "ls -l") faithfully reproduces what an earlier chmod(2) did. Some filesystems cannot satisify this. SANITY requires that a file or a directory is indeed accessible (or inaccessible) when its permission bits would say it ought to be accessible (or inaccessible). Running tests as root would lose this prerequisite for obvious reasons. Fix a few tests that misuse POSIXPERM. t0061-run-command.sh has two uses of POSIXPERM. - One checks that an attempt to execute a file that is marked as unexecutable results in a failure with EACCES; I do not think having root-ness or any other capability that busts the filesystem permission mode bits will make you run an unexecutable file, so this should be left as-is. The test does not have anything to do with SANITY. - The other one expects 'git nitfol' runs the alias when an alias.nitfol is defined and a directory on the PATH is marked as unreadable and unsearchable. I _think_ the test tries to reject the alternative expectation that we want to refuse to run the alias because it would break "no alias may mask a command" rule if a file 'git-nitfol' exists in the unreadable directory but we cannot even determine if that is the case. Under !SANITY that busts the permission bits, this test no longer checks that, so it must be protected with SANITY. t1509-root-worktree.sh expects to be run on a / that is writable by the user and sees if Git behaves "sensibly" when /.git is the repository to govern a worktree that is the whole filesystem, and also if Git behaves "sensibly" when / itself is a bare repository with refs, objects, and friends (I find the definition of "behaves sensibly" under these conditions hard to fathom, but it is a different matter). The implementation of the test is very much problematic. - It requires POSIXPERM, but it does not do chmod or checks modes in any way. - It runs "rm /*" and "rm -fr /refs /objects ..." in one of the tests, and also does "cd / && git init --bare". If done on a live system that takes advantages of the "feature" being tested, these obviously will clobber the system. But there is no guard against such a breakage. - It uses "test $UID = 0" to see rootness, which now should be spelled "! test_have_prereq NOT_ROOT" Signed-off-by: Junio C Hamano <gitster@pobox.com>	10 years ago
Jeff King	38f865c27d	run-command: treat inaccessible directories as ENOENT When execvp reports EACCES, it can be one of two things: 1. We found a file to execute, but did not have permissions to do so. 2. We did not have permissions to look in some directory in the $PATH. In the former case, we want to consider this a permissions problem and report it to the user as such (since getting this for something like "git foo" is likely a configuration error). In the latter case, there is a good chance that the inaccessible directory does not contain anything of interest. Reporting "permission denied" is confusing to the user (and prevents our usual "did you mean...?" lookup). It also prevents git from trying alias lookup, since we do so only when an external command does not exist (not when it exists but has an error). This patch detects EACCES from execvp, checks whether we are in case (2), and if so converts errno to ENOENT. This behavior matches that of "bash" (but not of simpler shells that use execvp more directly, like "dash"). Test stolen from Junio. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	13 years ago
Jonathan Nieder	c0f19bf3b9	tests: check error message from run_command In git versions starting at v1.7.5-rc0~29^2 until v1.7.5-rc3~2 (Revert "run-command: prettify -D_FORTIFY_SOURCE workaround", 2011-04-18) fixed it, the run_command facility would write a truncated error message when the command is present but cannot be executed for some other reason. For example, if I add a 'hello' command to git: $ echo 'echo hello' >git-hello $ chmod +x git-hello $ PATH=.:$PATH git hello hello and make it non-executable, this is what I normally get: $ chmod -x git-hello $ git hello fatal: cannot exec 'git-hello': Permission denied But with the problematic versions, we get disturbing output: $ PATH=.:$PATH git hello fatal: $ Add some tests to make sure it doesn't happen again. The hello-script used in these tests uses cat instead of echo because on Windows the bash spawned by git converts LF to CRLF in text written by echo while the bash running tests does not, causing the test to fail if "echo" is used. Thanks to Hannes for noticing. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Improved-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>	14 years ago
Johannes Sixt	2b541bf8be	start_command: detect execvp failures early Previously, failures during execvp could be detected only by finish_command. However, in some situations it is beneficial for the parent process to know earlier that the child process will not run. The idea to use a pipe to signal failures to the parent process and the test case were lifted from patches by Ilari Liusvaara. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>	15 years ago

25 Commits (883b98efade212bf1496a93dd8a691355a31daf0)