From 00ddc9d13cadb66d51e72107109baef1b3ca7c70 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 9 May 2017 14:53:21 +0200 Subject: [PATCH 1/6] Fix build with core.autocrlf=true On Windows, the default line endings are denoted by a Carriage Return byte followed by a Line Feed byte, while Linux and MacOSX use a single Line Feed byte to denote a line ending. To help with this situation, Git introduced several mechanisms over the last decade, most prominently the `core.autocrlf` setting. Sometimes, however, a single setting is incorrect, e.g. when certain files in the source code are to be consumed by software that can handle only LF line endings, while other files can use whatever is appropriate for the current platform. To allow for that, Git added the `eol` option to its .gitattributes handling, expecting every user of Git to mark their source code appropriately. Bash assumes that line-endings of scripts are denoted by a single Line Feed byte. Therefore, shell scripts in Git's source code are one example where that `eol=lf` option is *required*. When generating common-cmds.h, the Unix tools we use generally operate on the assumption that input and output deliminate their lines using LF-only line endings. Consequently, they would happily copy the CR byte verbatim into the strings in common-cmds.h, which in turn makes the C preprocessor barf (that interprets them as MacOS-style line endings). Therefore, we have to mark the input files as LF-only: command-list.txt and Documentation/git-*.txt. Quite a bit belatedly, this patch brings Git's own source code in line with those expectations by setting those attributes to allow for a correct build even when core.autocrlf=true. This patch can be validated even on Linux, by using this cadence: git config core.autocrlf true rm .git/index && git stash make -j15 DEVELOPER=1 Signed-off-by: Johannes Schindelin Reviewed-by: Jonathan Nieder Signed-off-by: Junio C Hamano --- .gitattributes | 8 +++++++- git-gui/.gitattributes | 1 + 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/.gitattributes b/.gitattributes index 320e33c327..8ce9c6b888 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,3 +1,9 @@ * whitespace=!indent,trail,space *.[ch] whitespace=indent,trail,space diff=cpp -*.sh whitespace=indent,trail,space +*.sh whitespace=indent,trail,space eol=lf +*.perl eol=lf +*.pm eol=lf +/Documentation/git-*.txt eol=lf +/command-list.txt eol=lf +/GIT-VERSION-GEN eol=lf +/mergetools/* eol=lf diff --git a/git-gui/.gitattributes b/git-gui/.gitattributes index 33d07c06bd..59cd41dbff 100644 --- a/git-gui/.gitattributes +++ b/git-gui/.gitattributes @@ -2,3 +2,4 @@ * encoding=US-ASCII git-gui.sh encoding=UTF-8 /po/*.po encoding=UTF-8 +/GIT-VERSION-GEN eol=lf From bee286f8f7c2e08a9c7fef28136796f6fd99a8ab Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 9 May 2017 14:53:25 +0200 Subject: [PATCH 2/6] git-new-workdir: mark script as LF-only Bash does not handle scripts with CR/LF line endings correctly, therefore they *have* to be forced to LF-only line endings. Funnily enough, this fixes t3000-ls-files-others and t1021-rerere-in-workdir when git.git was checked out with core.autocrlf=true, as these test still use git-new-workdir (once `git worktree` is no longer marked as experimental, both scripts probably want to be ported to using that command instead). Signed-off-by: Johannes Schindelin Reviewed-by: Jonathan Nieder Signed-off-by: Junio C Hamano --- contrib/workdir/.gitattributes | 1 + 1 file changed, 1 insertion(+) create mode 100644 contrib/workdir/.gitattributes diff --git a/contrib/workdir/.gitattributes b/contrib/workdir/.gitattributes new file mode 100644 index 0000000000..1f78c5d1bd --- /dev/null +++ b/contrib/workdir/.gitattributes @@ -0,0 +1 @@ +/git-new-workdir eol=lf From cedf4e27de019915a0386cb8f9a4721f2c26e7f7 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 9 May 2017 14:54:20 +0200 Subject: [PATCH 3/6] completion: mark bash script as LF-only Without this change, the completion script does not work, as Bash expects its scripts to have line feeds as end-of-line markers (this is particularly prominent in quoted multi-line strings, where carriage returns would slip into the strings as verbatim characters otherwise). This change is required to let t9902-completion pass when Git's source code is checked out with `core.autocrlf = true`. Signed-off-by: Johannes Schindelin Reviewed-by: Jonathan Nieder Signed-off-by: Junio C Hamano --- contrib/completion/.gitattributes | 1 + 1 file changed, 1 insertion(+) create mode 100644 contrib/completion/.gitattributes diff --git a/contrib/completion/.gitattributes b/contrib/completion/.gitattributes new file mode 100644 index 0000000000..19116944c1 --- /dev/null +++ b/contrib/completion/.gitattributes @@ -0,0 +1 @@ +*.bash eol=lf From 8a7f8bedf361f6af761863ee4ea6206f1cf3bc19 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 9 May 2017 14:54:24 +0200 Subject: [PATCH 4/6] t3901: move supporting files into t/t3901/ The current convention is to either generate files on the fly in tests, or to use supporting files taken from a t/tNNNN/ directory (where NNNN matches the test's number, or the number of the test from which we borrow supporting files). The test t3901-i18n-patch.sh was obviously introduced before that convention was in full swing, hence its supporting files still lived in t/t3901-8859-1.txt and t/t3901-utf8.txt, respectively. Let's adjust to the current convention. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t0203-gettext-setlocale-sanity.sh | 4 +-- t/t3901-i18n-patch.sh | 38 ++++++++++++------------ t/{t3901-8859-1.txt => t3901/8859-1.txt} | 0 t/{t3901-utf8.txt => t3901/utf8.txt} | 0 t/t9350-fast-export.sh | 2 +- t/t9500-gitweb-standalone-no-errors.sh | 4 +-- 6 files changed, 24 insertions(+), 24 deletions(-) rename t/{t3901-8859-1.txt => t3901/8859-1.txt} (100%) rename t/{t3901-utf8.txt => t3901/utf8.txt} (100%) diff --git a/t/t0203-gettext-setlocale-sanity.sh b/t/t0203-gettext-setlocale-sanity.sh index a212460081..71b0d74b4d 100755 --- a/t/t0203-gettext-setlocale-sanity.sh +++ b/t/t0203-gettext-setlocale-sanity.sh @@ -8,7 +8,7 @@ test_description="The Git C functions aren't broken by setlocale(3)" . ./lib-gettext.sh test_expect_success 'git show a ISO-8859-1 commit under C locale' ' - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && test_commit "iso-c-commit" iso-under-c && git show >out 2>err && ! test -s err && @@ -16,7 +16,7 @@ test_expect_success 'git show a ISO-8859-1 commit under C locale' ' ' test_expect_success GETTEXT_LOCALE 'git show a ISO-8859-1 commit under a UTF-8 locale' ' - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && test_commit "iso-utf8-commit" iso-under-utf8 && LANGUAGE=is LC_ALL="$is_IS_locale" git show >out 2>err && ! test -s err && diff --git a/t/t3901-i18n-patch.sh b/t/t3901-i18n-patch.sh index f663d567c8..923eb01f0e 100755 --- a/t/t3901-i18n-patch.sh +++ b/t/t3901-i18n-patch.sh @@ -31,7 +31,7 @@ test_expect_success setup ' # use UTF-8 in author and committer name to match the # i18n.commitencoding settings - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && test_tick && echo "$GIT_AUTHOR_NAME" >mine && @@ -55,7 +55,7 @@ test_expect_success setup ' # the second one on the side branch is ISO-8859-1 git config i18n.commitencoding ISO8859-1 && # use author and committer name in ISO-8859-1 to match it. - . "$TEST_DIRECTORY"/t3901-8859-1.txt + . "$TEST_DIRECTORY"/t3901/8859-1.txt fi && test_tick && echo Yet another >theirs && @@ -100,7 +100,7 @@ test_expect_success 'rebase (U/U)' ' # The result will be committed by GIT_COMMITTER_NAME -- # we want UTF-8 encoded name. - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git checkout -b test && git rebase master && @@ -110,7 +110,7 @@ test_expect_success 'rebase (U/U)' ' test_expect_success 'rebase (U/L)' ' git config i18n.commitencoding UTF-8 && git config i18n.logoutputencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard side && git rebase master && @@ -122,7 +122,7 @@ test_expect_success !MINGW 'rebase (L/L)' ' # In this test we want ISO-8859-1 encoded commits as the result git config i18n.commitencoding ISO8859-1 && git config i18n.logoutputencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard side && git rebase master && @@ -135,7 +135,7 @@ test_expect_success !MINGW 'rebase (L/U)' ' # to get ISO-8859-1 results. git config i18n.commitencoding ISO8859-1 && git config i18n.logoutputencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard side && git rebase master && @@ -148,7 +148,7 @@ test_expect_success 'cherry-pick(U/U)' ' git config i18n.commitencoding UTF-8 && git config i18n.logoutputencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard master && git cherry-pick side^ && @@ -163,7 +163,7 @@ test_expect_success !MINGW 'cherry-pick(L/L)' ' git config i18n.commitencoding ISO8859-1 && git config i18n.logoutputencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard master && git cherry-pick side^ && @@ -178,7 +178,7 @@ test_expect_success 'cherry-pick(U/L)' ' git config i18n.commitencoding UTF-8 && git config i18n.logoutputencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard master && git cherry-pick side^ && @@ -194,7 +194,7 @@ test_expect_success !MINGW 'cherry-pick(L/U)' ' git config i18n.commitencoding ISO8859-1 && git config i18n.logoutputencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard master && git cherry-pick side^ && @@ -207,7 +207,7 @@ test_expect_success !MINGW 'cherry-pick(L/U)' ' test_expect_success 'rebase --merge (U/U)' ' git config i18n.commitencoding UTF-8 && git config i18n.logoutputencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard side && git rebase --merge master && @@ -218,7 +218,7 @@ test_expect_success 'rebase --merge (U/U)' ' test_expect_success 'rebase --merge (U/L)' ' git config i18n.commitencoding UTF-8 && git config i18n.logoutputencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard side && git rebase --merge master && @@ -230,7 +230,7 @@ test_expect_success 'rebase --merge (L/L)' ' # In this test we want ISO-8859-1 encoded commits as the result git config i18n.commitencoding ISO8859-1 && git config i18n.logoutputencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard side && git rebase --merge master && @@ -243,7 +243,7 @@ test_expect_success 'rebase --merge (L/U)' ' # to get ISO-8859-1 results. git config i18n.commitencoding ISO8859-1 && git config i18n.logoutputencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard side && git rebase --merge master && @@ -254,7 +254,7 @@ test_expect_success 'rebase --merge (L/U)' ' test_expect_success 'am (U/U)' ' # Apply UTF-8 patches with UTF-8 commitencoding git config i18n.commitencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard master && git am out-u1 out-u2 && @@ -265,7 +265,7 @@ test_expect_success 'am (U/U)' ' test_expect_success !MINGW 'am (L/L)' ' # Apply ISO-8859-1 patches with ISO-8859-1 commitencoding git config i18n.commitencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard master && git am out-l1 out-l2 && @@ -276,7 +276,7 @@ test_expect_success !MINGW 'am (L/L)' ' test_expect_success 'am (U/L)' ' # Apply ISO-8859-1 patches with UTF-8 commitencoding git config i18n.commitencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard master && # am specifies --utf8 by default. @@ -288,7 +288,7 @@ test_expect_success 'am (U/L)' ' test_expect_success 'am --no-utf8 (U/L)' ' # Apply ISO-8859-1 patches with UTF-8 commitencoding git config i18n.commitencoding UTF-8 && - . "$TEST_DIRECTORY"/t3901-utf8.txt && + . "$TEST_DIRECTORY"/t3901/utf8.txt && git reset --hard master && git am --no-utf8 out-l1 out-l2 2>err && @@ -303,7 +303,7 @@ test_expect_success 'am --no-utf8 (U/L)' ' test_expect_success !MINGW 'am (L/U)' ' # Apply UTF-8 patches with ISO-8859-1 commitencoding git config i18n.commitencoding ISO8859-1 && - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && git reset --hard master && # mailinfo will re-code the commit message to the charset specified by diff --git a/t/t3901-8859-1.txt b/t/t3901/8859-1.txt similarity index 100% rename from t/t3901-8859-1.txt rename to t/t3901/8859-1.txt diff --git a/t/t3901-utf8.txt b/t/t3901/utf8.txt similarity index 100% rename from t/t3901-utf8.txt rename to t/t3901/utf8.txt diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index b5149fde6e..8dcb05c4a5 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -70,7 +70,7 @@ test_expect_success 'iso-8859-1' ' git config i18n.commitencoding ISO8859-1 && # use author and committer name in ISO-8859-1 to match it. - . "$TEST_DIRECTORY"/t3901-8859-1.txt && + . "$TEST_DIRECTORY"/t3901/8859-1.txt && test_tick && echo rosten >file && git commit -s -m den file && diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh index 6d06ed96cb..cc8d463e01 100755 --- a/t/t9500-gitweb-standalone-no-errors.sh +++ b/t/t9500-gitweb-standalone-no-errors.sh @@ -519,7 +519,7 @@ test_expect_success \ test_expect_success \ 'encode(commit): utf8' \ - '. "$TEST_DIRECTORY"/t3901-utf8.txt && + '. "$TEST_DIRECTORY"/t3901/utf8.txt && test_when_finished "GIT_AUTHOR_NAME=\"A U Thor\"" && test_when_finished "GIT_COMMITTER_NAME=\"C O Mitter\"" && echo "UTF-8" >> file && @@ -529,7 +529,7 @@ test_expect_success \ test_expect_success \ 'encode(commit): iso-8859-1' \ - '. "$TEST_DIRECTORY"/t3901-8859-1.txt && + '. "$TEST_DIRECTORY"/t3901/8859-1.txt && test_when_finished "GIT_AUTHOR_NAME=\"A U Thor\"" && test_when_finished "GIT_COMMITTER_NAME=\"C O Mitter\"" && echo "ISO-8859-1" >> file && From 614f4f0f350aef57473b0311b0a36b13e91e1310 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 9 May 2017 14:54:27 +0200 Subject: [PATCH 5/6] Fix the remaining tests that failed with core.autocrlf=true The test suite is mainly developed on Linux and MacOSX, which is the reason that nobody thought to mark files as LF-only as needed. The symptom is a test suite that fails left and right when being checked out using Git for Windows (which defaults to core.autocrlf=true). Mostly, the problems stem from Git's (LF-only) output being compared to hard-coded files that are checked out with line endings according to core.autocrlf (which is of course incorrect). This includes the two test files in t/diff-lib/, README and COPYING. This patch can be validated even on Linux by using this cadence: git config core.autocrlf true rm .git/index && git stash make -j15 DEVELOPER=1 test Signed-off-by: Johannes Schindelin Reviewed-by: Jonathan Nieder Signed-off-by: Junio C Hamano --- t/.gitattributes | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/t/.gitattributes b/t/.gitattributes index 2d44088f56..11e5fe3728 100644 --- a/t/.gitattributes +++ b/t/.gitattributes @@ -1,2 +1,21 @@ t[0-9][0-9][0-9][0-9]/* -whitespace -t0110/url-* binary +/diff-lib/* eol=lf +/t0110/url-* binary +/t3900/*.txt eol=lf +/t3901/*.txt eol=lf +/t4034/*/* eol=lf +/t4013/* eol=lf +/t4018/* eol=lf +/t4100/* eol=lf +/t4101/* eol=lf +/t4109/* eol=lf +/t4110/* eol=lf +/t4135/* eol=lf +/t4211/* eol=lf +/t4252/* eol=lf +/t5100/* eol=lf +/t5515/* eol=lf +/t556x_common eol=lf +/t7500/* eol=lf +/t8005/*.txt eol=lf +/t9*/*.dump eol=lf From 2779f665054c3ba85c1ee298929e640149d84b1e Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 9 May 2017 14:54:32 +0200 Subject: [PATCH 6/6] t4051: mark supporting files as requiring LF-only line endings The test t4051-diff-function-context.sh passes on Linux when core.autocrlf=true even without marking its support files as LF-only, but they fail when core.autocrlf=true in Git for Windows' SDK. The reason is that `grep ... >file.c.new` will keep CR/LF line endings on Linux (obviously treating CRs as if they were regular characters), but will be converted to LF-only line endings with MSYS2's grep that is used in Git for Windows. As we do not want to validate the way the available `grep` works, let's just mark the input as LF-only and move on. Signed-off-by: Johannes Schindelin Reviewed-by: Jonathan Nieder Signed-off-by: Junio C Hamano --- t/.gitattributes | 1 + 1 file changed, 1 insertion(+) diff --git a/t/.gitattributes b/t/.gitattributes index 11e5fe3728..3bd959ae52 100644 --- a/t/.gitattributes +++ b/t/.gitattributes @@ -6,6 +6,7 @@ t[0-9][0-9][0-9][0-9]/* -whitespace /t4034/*/* eol=lf /t4013/* eol=lf /t4018/* eol=lf +/t4051/* eol=lf /t4100/* eol=lf /t4101/* eol=lf /t4109/* eol=lf