Commit ffe256f9ba ("git-svn: Speed up fetch")
introduced changes that create a temporary file for each object fetched by
svn. These files should be deleted automatically, but perl apparently
doesn't do this until the process exits (or perhaps when its garbage
collector runs).
This means that on a large fetch, especially with lots of branches, we
sometimes fill up /tmp completely, which prevents the next temp file from
being written completely. This is aggravated by the fact that a new temp
file is created for each updated file, even if that update produces a file
identical to one already in git. Thus, it can happen even if there's lots
of disk space to store the finished repository.
We weren't adequately checking for write errors, so this would result in an
invalid file getting committed, which caused git-svn to fail later with an
invalid checksum.
This patch adds a check to syswrite() so similar problems don't lead to
corruption in the future. It also unlink()'s each temp file explicitly
when we're done with it, so the disk doesn't need to fill up.
Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
Tested-by: Björn Steinbrink <B.Steinbrink@gmx.de>
Acked-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When run in batch mode, git cat-file never frees the memory for the blob
contents it is printing. This quickly adds up and causes git-svn to be
hardly usable for imports of large svn repos, because it uses cat-file in
batch mode and cat-file's memory usage easily reaches several hundred MB
without any good reason.
Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the previous patch, not configuring any encryption (either on or
off) would leave $smtp_encryption undefined. We simply set it to the
empty string in that case.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-config expects a space, not '=' between option and value.
Also, quote the value since it contains globs, which some shells will not
pass through unchanged, or will abort if the glob doesn't expand.
Signed-off-by: Joey Hess <joey@kitenet.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make sure that buf has enough space to store the trailing \0 of
the command line argument, too.
Signed-off-by: Jochen Voss <voss@seehuhn.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sometimes, the easiest way to fix up a patch is to edit it directly, even
adding or deleting lines. Now, many people are not as divine as certain
benevolent dictators as to update the hunk headers correctly at the first
try.
So teach the tool to do it for us.
[jc: with tests]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is no really good reason to have a merge with more than 16
parents, but we have a history of giving our users rope.
Combined with the fact that there was no good reason for that
arbitrary limit in the first place, here is an all-too-easy to fix.
Kind of wished-for by Len Brown.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A lot of modules that have nothing to do with git-shell functionality
were linked in, bloating git-shell more than 8 times.
This patch cuts off redundant dependencies by:
1. providing stubs for three functions that make no sense for git-shell;
2. moving quote_path_fully from environment.c to quote.c to make the
later self sufficient;
3. moving make_absolute_path into a new separate file.
The following numbers have been received with the default optimization
settings on master using GCC 4.1.2:
Before:
text data bss dec hex filename
143915 1348 93168 238431 3a35f git-shell
After:
text data bss dec hex filename
17670 788 8232 26690 6842 git-shell
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Before this patch, something like "git help tutorial" did not work,
people had to use "git help gittutorial" which is not very intuitive.
This patch uses the "is_git_command" function to test early if the
argument passed to "git help" is a git command, and if this is not the
case then we prefix the argument with "git" instead of "git-".
This way, things like "git help tutorial" or "git help glossary" will
work fine.
The little downside of this patch is that the "is_git_command" is a
little bit slow.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When working with a lot of people who backport patches all day long, every
once in a while I get a patch that modifies the same file more than once
inside the same patch. git-apply either fails if the second change relies
on the first change or silently drops the first change if the second change
is independent.
The silent part is the scary scenario for us. Also this behaviour is
different from the patch-utils.
I have modified git-apply to create a table of the filenames of files it
modifies such that if a later patch chunk modifies a file in the table it
will buffer the previously changed file instead of reading the original file
from disk.
Logic has been put in to handle creations/deletions/renames/copies. All the
relevant tests of git-apply succeed.
A new test has been added to cover the cases I addressed.
The fix is relatively straight-forward.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, if there was an error while storing a local
tracking ref, the low-level functions would report an error,
but fetch's status output wouldn't indicate any problem.
E.g., imagine you have an old "refs/remotes/origin/foo/bar" but
upstream has deleted "foo/bar" in favor of a new branch
"foo". You would get output like this:
error: there are still refs under 'refs/remotes/origin/foo'
From $url_of_repo
* [new branch] foo -> origin/foo
With this patch, the output takes into account the status of
updating the local ref:
error: there are still refs under 'refs/remotes/origin/foo'
From $url_of_repo
! [new branch] foo -> origin/foo (unable to update local ref)
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we call "git clone" with a url that has a rewrite rule in either
$HOME/.gitconfig or /etc/gitconfig, the URL can be different from
what the command line expects it to be.
So, let's use the URL as the remote structure has it, not the literal
string from the command line.
Noticed by Pieter de Bie.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Acked-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a very well established command line convention that old residents
of the git mailing list knew by heart and nobody even thought about
documenting it explicitly, which was not very nice.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now "diff --check" can detect not just whitespace errors but also notices
leftover conflict marker lines, we can use it in the sample pre-commit
hook script.
These days the object layer knows about the empty tree object without
actually having one in the repository, so we can run the test even for the
initial commit.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This teaches "diff --check" to detect and complain if the change
adds lines that look like leftover conflict markers.
We should be able to remove the old Perl script used in the sample
pre-commit hook and modernize the script with this facility.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a patch adds new blank lines at the end, "git apply --whitespace"
warns. This teaches "diff --check" to do the same.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function name was too bland and not explicit enough as to what it is
checking. Split it into two, and call the one that checks if there is a
whitespace breakage "ws_check()", and call the other one that checks and
emits the line after color coding "ws_check_emit()".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All other codepaths refrain from running textual diff when either the old
or the new side is binary, but this function only checks the new side. I
was almost going to change it to check both, but that would be a bad
change. Explain why to prevent future mistakes.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a backport of 0a47dc110e
to 'maint' to be included in 1.5.6.2 so that older server side
can accept dashless form of request when clients are updated.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git diff --check" should return non-zero when there was any whitespace
error but the code only paid attention to the error status of the last
new line in the patch.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, we did a sanity check by doing for-each-ref
using each possible format atom. However, we never checked
the actual output produced by that atom, which recently let
an obvious bug go undetected for some time.
While we're at it, also clean up a few '!' into
test_must_fail.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I'm working with a perforce repo using git-p4. There are some config
files which I need to change locally according to my environment. I'm
using a 'local' git branch to park these changes. And I want to avoid
accidentally checking them into p4 just by doing "git p4 submit"
mindlessly without realizing which branch I'm actually on.
This patch adds a new git config, 'git-p4.allowSubmit', which is a
whitelist of branch names. "git p4 submit" will only allow submissions
from local branches on the list. Useful for preventing inadvertently
submitting from a strictly local branch.
For backward compatibility, if this config is not set at all,
submissions from all branches are allowed.
Signed-off-by: Jing Xue <jingxue@digizenstudio.com>
Acked-By: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
read_in_full()'s is used in compat/pread.c. read_in_full() is
declared in cache.h. But we can't include cache.h because too
many macros are defined there. Using read_in_full() without
including cache.h is dangerous because we wouldn't recognize if
its prototyp changed. gcc issues a warning about that.
This commit adds a forward declaration to git-compat-util.h.
git-compat-util.h is included by compat/pread.c _and_ cache.h.
Hence, changes in cache.h would be detected.
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
On Windows, ntohl() returns unsigned long. On Unix it returns
uint32_t. This makes choosing a suitable printf format string
hard.
This commit introduces a mingw specific helper function
git_ntohl() that casts to unsigned int before returning. This
makes gcc's printf format check happy. It should be safe because
we expect ntohl to use 32-bit numbers.
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
git help -a scans the PATH for git commands. On Windows it failed for two
reasons:
- The PATH separator is ';', not ':' on Windows.
- stat() does not set the executable bit.
We now open the file and guess whether it is executable.
The result of the guess is good enough for the list of git commands, but
it is of no use for a general stat() implementation because (1) it is a
guess, (2) the user has no way to influence the outcome (via chmod or
similar), and (3) it would reduce stat() performance by an unacceptable
amount. Therefore, this strategy is a special-case local to help.c.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
On Windows, write() is implemented using WriteFile(). After the reader
closed its end of the pipe, the first WriteFile() returns
ERROR_BROKEN_PIPE (which translates to EPIPE), subsequent WriteFile()s
return ERROR_NO_DATA, which is translated to EINVAL.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Since we have neither fork() nor exec(), we have to spawn the pager and
feed it with the program's output.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Since the Makefile in the template/ subdirectory is only used to install
the templates, we do not simply pass down the setting of template_dir
when it is relative, but construct the intended destination in a new
variable: A relative template_dir is relative to gitexecdir.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
With this definition the templates and system config file will be found
irrespective of the installation location.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Since on Windows the user is fairly free where to install programs, we
cannot rely on a hard-coded path. We use the program name to derive the
installation directory and use that as exec_path.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
builtin_exec_path returns the hard-coded installation path, which is used
as the ultimate fallback to look for git commands. Making it into a function
enables us in a follow-up patch to return a computed value instead of just
a constant string.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Windows's struct stat does not have a st_blocks member. Since we already
have our own stat/lstat/fstat implementations, we can just as well use
a customized struct stat. This patch introduces just that, and also fills
in the st_blocks member. On the other hand, we don't provide members that
are never used.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
This is a necessary pendant to our lstat implementation: MSVCRT's
implementations of lstat and utime do some adjustments if daylight
saving time is in effect, but our lstat implementation doesn't do these
adjustments and report the correct UTC time. With this implementation
we omit the adjustments in utime() as well and always write UTC.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
This gives us a significant speedup when adding, committing and stat'ing files.
Also, since Windows doesn't really handle symlinks, we let stat just uses lstat.
We also need to replace fstat, since our implementation and the standard stat()
functions report slightly different timestamps, possibly due to timezones.
We simply report UTC in our implementation, and do our FILETIME to time_t
conversion based on the document at http://support.microsoft.com/kb/167296.
With Moe's repo structure (100K files in 100 dirs, containing 2-4 bytes)
mkdir bummer && cd bummer; for ((i=0;i<100;i++)); do
mkdir $i && pushd $i;
for ((j=0;j<1000;j++)); do echo "$j" >$j; done;
popd;
done
We get the following performance boost:
With normal lstat & stat Custom lstat/fstat
------------------------ ------------------------
Command: git init Command: git init
------------------------ ------------------------
real 0m 0.047s real 0m 0.063s
user 0m 0.031s user 0m 0.015s
sys 0m 0.000s sys 0m 0.015s
------------------------ ------------------------
Command: git add . Command: git add .
------------------------ ------------------------
real 0m19.390s real 0m12.031s 1.6x
user 0m 0.015s user 0m 0.031s
sys 0m 0.030s sys 0m 0.000s
------------------------ ------------------------
Command: git commit -a.. Command: git commit -a..
------------------------ ------------------------
real 0m30.812s real 0m16.875s 1.8x
user 0m 0.015s user 0m 0.015s
sys 0m 0.000s sys 0m 0.015s
------------------------ ------------------------
3x Command: git-status 3x Command: git-status
------------------------ ------------------------
real 0m11.860s real 0m 5.266s 2.2x
user 0m 0.015s user 0m 0.015s
sys 0m 0.015s sys 0m 0.015s
real 0m11.703s real 0m 5.234s
user 0m 0.015s user 0m 0.015s
sys 0m 0.000s sys 0m 0.000s
real 0m11.672s real 0m 5.250s
user 0m 0.031s user 0m 0.015s
sys 0m 0.000s sys 0m 0.000s
------------------------ ------------------------
Command: git commit... Command: git commit...
(single file) (single file)
------------------------ ------------------------
real 0m14.234s real 0m 7.735s 1.8x
user 0m 0.015s user 0m 0.031s
sys 0m 0.000s sys 0m 0.000s
Signed-off-by: Marius Storm-Olsen <mstormo_git@storm-olsen.com>
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
The problem with Windows's own implementation is that it tries to be
clever when a console program is invoked from a GUI application: In this
case it sometimes automatically allocates a new console window. As a
consequence, the IO channels of the spawned program are directed to the
console, but the invoking application listens on channels that are now
directed to nowhere.
In this implementation we use the lowlevel facilities of CreateProcess(),
which offers a flag to tell the system not to open a console. As a side
effect, only stdin, stdout, and stderr channels will be accessible from
C programs that are spawned. Other channels (file handles, pipe handles,
etc.) are still inherited by the spawned program, but it doesn't get
enough information to access them.
Johannes Schindelin integrated path quoting and unified the various
*execv* and *spawnv* helpers. Eric Raible suggested to also quote '{'.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
gethostbyname() is the first function that calls into the Winsock library,
and it is wrapped only to initialize the library.
socket() is wrapped for two reasons:
- Windows's socket() creates things that are like low-level file handles,
and they must be converted into file descriptors first.
- And these handles cannot be used with plain ReadFile()/WriteFile()
because they are opened for "overlapped IO". We have to use WSASocket()
to create non-overlapped IO sockets.
connect() must be wrapped because Windows's connect() expects the low-level
sockets, not file descriptors, and we must first unwrap the file descriptor
before we can pass it on to Windows's connect().
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
If the PATH lists the Windows system directories before the MSYS
directories, Windows's own incompatible sort and find commands would be
picked up. We implement these commands as functions and call the real
tools by absolute path.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
In upload-pack we must explicitly close the output channel of rev-list.
(On Unix, the channel is closed automatically because process that runs
rev-list terminates.)
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
If on Windows a path is specified as C:/path, then this is also a valid
SSH URL. To disambiguate between the two interpretations we take an URL
that looks like a path with a drive letter as a local URL.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Avery Pennarun