From 60890cc60ccfc7000791a47f1f3d69fdb8884dd7 Mon Sep 17 00:00:00 2001 From: Ian McLean Date: Thu, 20 May 2010 20:57:51 +0200 Subject: [PATCH 1/3] Fix "Out of memory? mmap failed" for files larger than 4GB on Windows The git_mmap implementation was broken for file sizes that wouldn't fit into a size_t (32 bits). This was caused by intermediate variables that were only 32 bits wide when they should be 64 bits. Signed-off-by: Johannes Sixt Signed-off-by: Junio C Hamano --- compat/win32mmap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/compat/win32mmap.c b/compat/win32mmap.c index 1c5a14922f..b58aa69fa0 100644 --- a/compat/win32mmap.c +++ b/compat/win32mmap.c @@ -4,19 +4,19 @@ void *git_mmap(void *start, size_t length, int prot, int flags, int fd, off_t of { HANDLE hmap; void *temp; - size_t len; + off_t len; struct stat st; uint64_t o = offset; uint32_t l = o & 0xFFFFFFFF; uint32_t h = (o >> 32) & 0xFFFFFFFF; if (!fstat(fd, &st)) - len = xsize_t(st.st_size); + len = st.st_size; else die("mmap: could not determine filesize"); if ((length + offset) > len) - length = len - offset; + length = xsize_t(len - offset); if (!(flags & MAP_PRIVATE)) die("Invalid usage of mmap when built with USE_WIN32_MMAP"); From fc012c2810c237dd2299a3fc85b18c2beb60b730 Mon Sep 17 00:00:00 2001 From: bert Dvornik Date: Thu, 20 May 2010 20:57:52 +0200 Subject: [PATCH 2/3] start_command: close cmd->err descriptor when fork/spawn fails Fix the problem where the cmd->err passed into start_command wasn't being properly closed when certain types of errors occurr. (Compare the affected code with the clean shutdown code later in the function.) On Windows, this problem would be triggered if mingw_spawnvpe() failed, which would happen if the command to be executed was malformed (e.g. a text file that didn't start with a #! line). If cmd->err was a pipe, the failure to close it could result in a hang while the other side was waiting (forever) for either input or pipe close, e.g. while trying to shove the output into the side band. On msysGit, this problem was causing a hang in t5516-fetch-push. [J6t: With a slight adjustment of the test case, the hang is also observed on Linux.] Signed-off-by: bert Dvornik Signed-off-by: Johannes Sixt Signed-off-by: Junio C Hamano --- run-command.c | 2 ++ t/t5516-fetch-push.sh | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/run-command.c b/run-command.c index eb5c575629..c7793f50fb 100644 --- a/run-command.c +++ b/run-command.c @@ -383,6 +383,8 @@ fail_pipe: close(cmd->out); if (need_err) close_pair(fderr); + else if (cmd->err) + close(cmd->err); errno = failed_errno; return -1; } diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh index 2de98e6561..6a37a4d993 100755 --- a/t/t5516-fetch-push.sh +++ b/t/t5516-fetch-push.sh @@ -528,7 +528,7 @@ test_expect_success 'push does not update local refs on failure' ' mk_test heads/master && mk_child child && mkdir testrepo/.git/hooks && - echo exit 1 >testrepo/.git/hooks/pre-receive && + echo "#!/no/frobnication/today" >testrepo/.git/hooks/pre-receive && chmod +x testrepo/.git/hooks/pre-receive && (cd child && git pull .. master From c8b296450e5148c576697ea4709072b7855aacd5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Scharfe?= Date: Thu, 20 May 2010 20:57:53 +0200 Subject: [PATCH 3/3] Fix checkout of large files to network shares on Windows XP Bigger writes to network drives on Windows XP fail. Cap them at 31MB to allow them to succeed. Callers need to be prepared for write() calls that do less work than requested anyway. On local drives, write() calls are translated to WriteFile() calls with a cap of 64KB on Windows XP and 256KB on Vista. Thus a cap of 31MB won't affect the number of WriteFile() calls which do the actual work. There's still room for some other version of Windows to use a chunk size of 1MB without increasing the number of system calls. Signed-off-by: Rene Scharfe Signed-off-by: Johannes Sixt Signed-off-by: Junio C Hamano --- compat/mingw.c | 17 +++++++++++++++++ compat/mingw.h | 3 +++ t/t5705-clone-2gb.sh | 12 +++++++++--- 3 files changed, 29 insertions(+), 3 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index f90a114b02..9a8e336582 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -140,6 +140,23 @@ int mingw_open (const char *filename, int oflags, ...) return fd; } +#undef write +ssize_t mingw_write(int fd, const void *buf, size_t count) +{ + /* + * While write() calls to a file on a local disk are translated + * into WriteFile() calls with a maximum size of 64KB on Windows + * XP and 256KB on Vista, no such cap is placed on writes to + * files over the network on Windows XP. Unfortunately, there + * seems to be a limit of 32MB-28KB on X64 and 64MB-32KB on x86; + * bigger writes fail on Windows XP. + * So we cap to a nice 31MB here to avoid write failures over + * the net without changing the number of WriteFile() calls in + * the local case. + */ + return write(fd, buf, min(count, 31 * 1024 * 1024)); +} + #undef fopen FILE *mingw_fopen (const char *filename, const char *otype) { diff --git a/compat/mingw.h b/compat/mingw.h index 7c2ab64cb4..0e3e743041 100644 --- a/compat/mingw.h +++ b/compat/mingw.h @@ -170,6 +170,9 @@ int link(const char *oldpath, const char *newpath); int mingw_open (const char *filename, int oflags, ...); #define open mingw_open +ssize_t mingw_write(int fd, const void *buf, size_t count); +#define write mingw_write + FILE *mingw_fopen (const char *filename, const char *otype); #define fopen mingw_fopen diff --git a/t/t5705-clone-2gb.sh b/t/t5705-clone-2gb.sh index adfaae8c5b..8afbdd4de2 100755 --- a/t/t5705-clone-2gb.sh +++ b/t/t5705-clone-2gb.sh @@ -12,7 +12,7 @@ test_expect_success 'setup' ' git config pack.compression 0 && git config pack.depth 0 && - blobsize=$((20*1024*1024)) && + blobsize=$((100*1024*1024)) && blobcount=$((2*1024*1024*1024/$blobsize+1)) && i=1 && (while test $i -le $blobcount @@ -36,9 +36,15 @@ test_expect_success 'setup' ' ' -test_expect_success 'clone' ' +test_expect_success 'clone - bare' ' - git clone --bare --no-hardlinks . clone + git clone --bare --no-hardlinks . clone-bare + +' + +test_expect_success 'clone - with worktree, file:// protocol' ' + + git clone file://. clone-wt '