Go to file
Tao Klerks f7b5ff607f git-p4: improve encoding handling to support inconsistent encodings
git-p4 is designed to run correctly under python2.7 and python3, but
its functional behavior wrt importing user-entered text differs across
these environments:

Under python2, git-p4 "naively" writes the Perforce bytestream into git
metadata (and does not set an "encoding" header on the commits); this
means that any non-utf-8 byte sequences end up creating invalidly-encoded
commit metadata in git.

Under python3, git-p4 attempts to decode the Perforce bytestream as utf-8
data, and fails badly (with an unhelpful error) when non-utf-8 data is
encountered.

Perforce clients (especially p4v) encourage user entry of changelist
descriptions (and user full names) in OS-local encoding, and store the
resulting bytestream to the server unmodified - such that different
clients can end up creating mutually-unintelligible messages. The most
common inconsistency, in many Perforce environments, is likely to be utf-8
(typical in linux) vs cp-1252 (typical in windows).

Make the changelist-description- and user-fullname-handling code
python-runtime-agnostic, introducing three "strategies" selectable via
config:
- 'passthrough', behaving as previously under python2,
- 'strict', behaving as previously under python3, and
- 'fallback', favoring utf-8 but supporting a secondary encoding when
utf-8 decoding fails, and finally escaping high-range bytes if the
decoding with the secondary encoding also fails.

Keep the python2 default behavior as-is ('legacy' strategy), but switch
the python3 default strategy to 'fallback' with default fallback encoding
'cp1252'.

Also include tests exercising these encoding strategies, documentation for
the new config, and improve the user-facing error messages when decoding
does fail.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:30:01 -07:00
.github
Documentation git-p4: improve encoding handling to support inconsistent encodings 2022-05-04 10:30:01 -07:00
block-sha1
builtin i18n: fix some badly formatted i18n strings 2022-04-11 14:13:46 -07:00
ci
compat Git 2.35.2 2022-04-11 16:44:45 -07:00
contrib Merge branch 'ab/make-optim-noop' 2022-04-06 15:21:58 -07:00
ewah
git-gui
gitk-git
gitweb
mergetools
negotiator
perl
po l10n: sv.po: Update Swedish translation (5282t0f0u) 2022-04-17 18:13:34 +01:00
ppc
refs Revert "Merge branch 'ps/avoid-unnecessary-hook-invocation-with-packed-refs'" 2022-04-13 15:51:33 -07:00
reftable reftable: make assignments portable to AIX xlc v12.01 2022-03-28 13:58:10 -07:00
sha1collisiondetection@855827c583
sha1dc
sha256
t git-p4: improve encoding handling to support inconsistent encodings 2022-05-04 10:30:01 -07:00
templates
trace2
xdiff
.cirrus.yml
.clang-format
.editorconfig
.gitattributes
.gitignore fsmonitor--daemon: add a built-in fsmonitor daemon 2022-03-25 16:04:15 -07:00
.gitmodules
.mailmap
.tsan-suppressions
CODE_OF_CONDUCT.md
COPYING
GIT-VERSION-GEN Git 2.36 2022-04-17 22:21:51 -07:00
INSTALL
LGPL-2.1
Makefile Merge branch 'bc/csprng-mktemps' 2022-04-06 15:21:59 -07:00
README.md
RelNotes Git 2.35.3 2022-04-13 15:21:34 -07:00
SECURITY.md
abspath.c
aclocal.m4
add-interactive.c Merge branch 'pw/single-key-interactive' 2022-03-21 15:14:24 -07:00
add-interactive.h
add-patch.c
advice.c Merge branch 'tk/ambiguous-fetch-refspec' 2022-04-04 10:56:24 -07:00
advice.h Merge branch 'tk/ambiguous-fetch-refspec' 2022-04-04 10:56:24 -07:00
alias.c
alias.h
alloc.c
alloc.h
apply.c Merge branch 'ab/object-file-api-updates' 2022-03-16 17:53:08 -07:00
apply.h
archive-tar.c
archive-zip.c
archive.c
archive.h
attr.c Merge branch 'ab/refs-various-fixes' 2022-03-29 12:22:02 -07:00
attr.h
banned.h
base85.c
bisect.c
bisect.h
blame.c
blame.h
blob.c
blob.h
bloom.c
bloom.h
branch.c Merge branch 'tk/ambiguous-fetch-refspec' 2022-04-04 10:56:24 -07:00
branch.h
builtin.h fsmonitor--daemon: add a built-in fsmonitor daemon 2022-03-25 16:04:15 -07:00
bulk-checkin.c Merge branch 'ns/core-fsyncmethod' 2022-03-25 16:38:24 -07:00
bulk-checkin.h
bundle.c bundle: output hash information in 'verify' 2022-03-23 13:14:09 -07:00
bundle.h
cache-tree.c
cache-tree.h
cache.h Merge branch 'jh/builtin-fsmonitor-part2' 2022-04-04 10:56:24 -07:00
cbtree.c
cbtree.h
chdir-notify.c
chdir-notify.h
check-builtins.sh
checkout.c
checkout.h
chunk-format.c
chunk-format.h
color.c
color.h
column.c
column.h
combine-diff.c
command-list.txt
commit-graph.c Merge branch 'ns/core-fsyncmethod' 2022-03-25 16:38:24 -07:00
commit-graph.h
commit-reach.c
commit-reach.h
commit-slab-decl.h
commit-slab-impl.h
commit-slab.h
commit.c Merge branch 'ab/racy-hooks' 2022-03-30 18:01:10 -07:00
commit.h shallow: reset commit grafts when shallow is reset 2022-03-17 17:44:38 -07:00
common-main.c
config.c Merge branch 'jh/builtin-fsmonitor-part2' 2022-04-04 10:56:24 -07:00
config.h fsmonitor: config settings are repository-specific 2022-03-25 16:04:15 -07:00
config.mak.dev config.mak.dev: alternative workaround to gcc 12 warning in http.c 2022-04-15 16:27:04 -07:00
config.mak.in
config.mak.uname Merge branch 'jh/builtin-fsmonitor-part2' 2022-04-04 10:56:24 -07:00
configure.ac configure.ac: fix HAVE_SYNC_FILE_RANGE definition 2022-04-06 09:36:19 -07:00
connect.c
connect.h
connected.c
connected.h
convert.c Merge branch 'ab/object-file-api-updates' 2022-03-16 17:53:08 -07:00
convert.h
copy.c
credential.c
credential.h
csum-file.c
csum-file.h
ctype.c
daemon.c
date.c
date.h
decorate.c
decorate.h
delta-islands.c
delta-islands.h
delta.h
detect-compiler
diff-delta.c
diff-lib.c
diff-merges.c
diff-merges.h
diff-no-index.c
diff.c Merge branch 'ab/plug-random-leaks' 2022-03-23 14:09:31 -07:00
diff.h
diffcore-break.c
diffcore-delta.c
diffcore-order.c
diffcore-pickaxe.c
diffcore-rename.c
diffcore-rotate.c
diffcore.h
dir-iterator.c
dir-iterator.h
dir.c Merge branch 'vd/sparse-read-tree' 2022-03-16 17:53:08 -07:00
dir.h
editor.c
entry.c
entry.h
environment.c Merge branch 'jh/builtin-fsmonitor-part2' 2022-04-04 10:56:24 -07:00
environment.h
exec-cmd.c
exec-cmd.h
fetch-negotiator.c fetch-negotiator: add specific noop initializer 2022-03-28 10:25:52 -07:00
fetch-negotiator.h fetch-negotiator: add specific noop initializer 2022-03-28 10:25:52 -07:00
fetch-pack.c fetch-pack: add refetch 2022-03-28 10:25:52 -07:00
fetch-pack.h fetch-pack: add refetch 2022-03-28 10:25:52 -07:00
fmt-merge-msg.c
fmt-merge-msg.h
fsck.c
fsck.h
fsmonitor--daemon.h fsmonitor--daemon: use a cookie file to sync with file system 2022-03-25 16:04:17 -07:00
fsmonitor-ipc.c fsmonitor-ipc: create client routines for git-fsmonitor--daemon 2022-03-25 16:04:15 -07:00
fsmonitor-ipc.h fsmonitor-ipc: create client routines for git-fsmonitor--daemon 2022-03-25 16:04:15 -07:00
fsmonitor-settings.c fsmonitor: config settings are repository-specific 2022-03-25 16:04:15 -07:00
fsmonitor-settings.h fsmonitor: config settings are repository-specific 2022-03-25 16:04:15 -07:00
fsmonitor.c fsmonitor: force update index after large responses 2022-03-25 16:04:17 -07:00
fsmonitor.h fsmonitor: config settings are repository-specific 2022-03-25 16:04:15 -07:00
fuzz-commit-graph.c
fuzz-pack-headers.c
fuzz-pack-idx.c
generate-cmdlist.sh
generate-configlist.sh
generate-hooklist.sh
gettext.c
gettext.h
git-add--interactive.perl
git-archimport.perl
git-bisect.sh
git-compat-util.h Git 2.35.2 2022-04-11 16:44:45 -07:00
git-curl-compat.h
git-cvsexportcommit.perl
git-cvsimport.perl
git-cvsserver.perl
git-difftool--helper.sh
git-filter-branch.sh
git-instaweb.sh
git-merge-octopus.sh
git-merge-one-file.sh
git-merge-resolve.sh
git-mergetool--lib.sh
git-mergetool.sh
git-p4.py git-p4: improve encoding handling to support inconsistent encodings 2022-05-04 10:30:01 -07:00
git-quiltimport.sh
git-request-pull.sh
git-send-email.perl i18n: fix some badly formatted i18n strings 2022-04-11 14:13:46 -07:00
git-sh-i18n.sh
git-sh-setup.sh
git-submodule.sh submodule: move core cmd_update() logic to C 2022-03-16 15:07:43 -07:00
git-svn.perl
git-web--browse.sh
git.c fsmonitor--daemon: add a built-in fsmonitor daemon 2022-03-25 16:04:15 -07:00
git.rc
gpg-interface.c
gpg-interface.h
graph.c
graph.h
grep.c
grep.h
hash-lookup.c
hash-lookup.h
hash.h
hashmap.c
hashmap.h
help.c Merge branch 'jh/builtin-fsmonitor-part2' 2022-04-04 10:56:24 -07:00
help.h
hex.c
hook.c
hook.h
http-backend.c
http-fetch.c
http-push.c
http-walker.c
http.c
http.h
ident.c
imap-send.c Merge branch 'bc/csprng-mktemps' 2022-04-06 15:21:59 -07:00
iterator.h
json-writer.c
json-writer.h
khash.h
kwset.c
kwset.h
levenshtein.c
levenshtein.h
line-log.c
line-log.h
line-range.c
line-range.h
linear-assignment.c
linear-assignment.h
list-objects-filter-options.c pack-objects: lazily set up "struct rev_info", don't leak 2022-03-28 09:57:21 -07:00
list-objects-filter-options.h pack-objects: lazily set up "struct rev_info", don't leak 2022-03-28 09:57:21 -07:00
list-objects-filter.c
list-objects-filter.h
list-objects.c
list-objects.h
list.h
ll-merge.c
ll-merge.h
lockfile.c
lockfile.h
log-tree.c Merge branch 'ab/object-file-api-updates' 2022-03-16 17:53:08 -07:00
log-tree.h
ls-refs.c
ls-refs.h
mailinfo.c
mailinfo.h
mailmap.c
mailmap.h
match-trees.c
mem-pool.c
mem-pool.h
merge-blobs.c
merge-blobs.h
merge-ort-wrappers.c
merge-ort-wrappers.h
merge-ort.c Merge branch 'ab/string-list-count-in-size-t' 2022-03-16 17:53:09 -07:00
merge-ort.h
merge-recursive.c Merge branch 'ab/object-file-api-updates' 2022-03-16 17:53:08 -07:00
merge-recursive.h
merge.c
mergesort.c
mergesort.h
midx.c
midx.h
name-hash.c
notes-cache.c
notes-cache.h
notes-merge.c misc *.c: use designated initializers for struct assignments 2022-03-17 10:36:42 -07:00
notes-merge.h
notes-utils.c
notes-utils.h
notes.c
notes.h
object-file.c object-file: pass filename to fsync_or_die 2022-03-30 14:46:47 -07:00
object-name.c
object-store.h
object.c
object.h
oid-array.c
oid-array.h
oidmap.c
oidmap.h
oidset.c
oidset.h
oidtree.c
oidtree.h
pack-bitmap-write.c
pack-bitmap.c
pack-bitmap.h
pack-check.c
pack-objects.c
pack-objects.h
pack-revindex.c
pack-revindex.h
pack-write.c
pack.h
packfile.c
packfile.h
pager.c
parallel-checkout.c
parallel-checkout.h
parse-options-cb.c
parse-options.c
parse-options.h
patch-delta.c
patch-ids.c
patch-ids.h
path.c Sync with 2.33.2 2022-03-24 00:31:36 +01:00
path.h
pathspec.c
pathspec.h
pkt-line.c
pkt-line.h
preload-index.c
pretty.c
pretty.h
prio-queue.c
prio-queue.h
progress.c
progress.h
promisor-remote.c
promisor-remote.h
prompt.c
prompt.h
protocol-caps.c
protocol-caps.h
protocol.c
protocol.h
prune-packed.c
prune-packed.h
quote.c
quote.h
range-diff.c
range-diff.h
reachable.c
reachable.h
read-cache.c Merge branch 'vd/mv-refresh-stat' 2022-04-04 10:56:24 -07:00
rebase-interactive.c
rebase-interactive.h
rebase.c
rebase.h
ref-filter.c
ref-filter.h
reflog-walk.c
reflog-walk.h
reflog.c Merge branch 'ab/reflog-parse-options' 2022-04-04 10:56:21 -07:00
reflog.h
refs.c Revert "Merge branch 'ps/avoid-unnecessary-hook-invocation-with-packed-refs'" 2022-04-13 15:51:33 -07:00
refs.h Revert "Merge branch 'ps/avoid-unnecessary-hook-invocation-with-packed-refs'" 2022-04-13 15:51:33 -07:00
refspec.c
refspec.h
remote-curl.c Merge branch 'rc/fetch-refetch' 2022-04-04 10:56:23 -07:00
remote.c
remote.h
replace-object.c
replace-object.h
repo-settings.c compat/fsmonitor/fsm-listen-win32: stub in backend for Windows 2022-03-25 16:04:15 -07:00
repository.c
repository.h Merge branch 'jh/builtin-fsmonitor-part2' 2022-04-04 10:56:24 -07:00
rerere.c
rerere.h
reset.c
reset.h
resolve-undo.c
resolve-undo.h
revision.c list-objects-filter: remove CL_ARG__FILTER 2022-03-23 13:13:17 -07:00
revision.h
run-command.c
run-command.h
send-pack.c
send-pack.h
sequencer.c Revert "Merge branch 'ps/avoid-unnecessary-hook-invocation-with-packed-refs'" 2022-04-13 15:51:33 -07:00
sequencer.h
serve.c
serve.h
server-info.c
setup.c Git 2.35.3 2022-04-13 15:26:32 -07:00
sh-i18n--envsubst.c
sha1dc_git.c
sha1dc_git.h
shallow.c shallow: reset commit grafts when shallow is reset 2022-03-17 17:44:38 -07:00
shallow.h
shared.mak Makefile: use ' ', not non-existing $(wspfx_SQ) 2022-03-21 09:03:45 -07:00
shell.c
shortlog.h
sideband.c
sideband.h
sigchain.c
sigchain.h
simple-ipc.h
sparse-index.c
sparse-index.h
split-index.c
split-index.h
stable-qsort.c
strbuf.c
strbuf.h
streaming.c
streaming.h
string-list.c
string-list.h
strmap.c
strmap.h
strvec.c
strvec.h
sub-process.c
sub-process.h
submodule-config.c
submodule-config.h
submodule.c Merge branch 'jt/reset-grafts-when-resetting-shallow' 2022-03-29 12:22:03 -07:00
submodule.h submodule: fix latent check_has_commit() bug 2022-03-16 16:08:59 -07:00
symlinks.c
tag.c
tag.h
tar.h
tempfile.c
tempfile.h
thread-utils.c
thread-utils.h
tmp-objdir.c
tmp-objdir.h
trace.c
trace.h
trace2.c trace2: add stats for fsync operations 2022-03-30 11:15:55 -07:00
trace2.h
trailer.c
trailer.h
transport-helper.c fetch: add --refetch option 2022-03-28 10:25:52 -07:00
transport-internal.h
transport.c Merge branch 'rc/fetch-refetch' 2022-04-04 10:56:23 -07:00
transport.h fetch: add --refetch option 2022-03-28 10:25:52 -07:00
tree-diff.c
tree-walk.c
tree-walk.h
tree.c
tree.h
unicode-width.h
unimplemented.sh
unix-socket.c
unix-socket.h
unix-stream-server.c
unix-stream-server.h
unpack-trees.c Revert "unpack-trees: improve performance of next_cache_entry" 2022-03-17 09:13:21 -07:00
unpack-trees.h
upload-pack.c
upload-pack.h
url.c
url.h
urlmatch.c
urlmatch.h
usage.c
userdiff.c Merge branch 'jd/userdiff-kotlin' 2022-03-23 14:09:29 -07:00
userdiff.h
utf8.c
utf8.h
varint.c
varint.h
version.c
version.h
versioncmp.c
walker.c
walker.h
wildmatch.c
wildmatch.h
worktree.c
worktree.h
wrap-for-bin.sh
wrapper.c trace2: add stats for fsync operations 2022-03-30 11:15:55 -07:00
write-or-die.c
ws.c
wt-status.c Merge branch 'ab/string-list-count-in-size-t' 2022-03-16 17:53:09 -07:00
wt-status.h
xdiff-interface.c
xdiff-interface.h
zlib.c

README.md

Build status

Git - fast, scalable, distributed revision control system

Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.

Git is an Open Source project covered by the GNU General Public License version 2 (some parts of it are under different licenses, compatible with the GPLv2). It was originally written by Linus Torvalds with help of a group of hackers around the net.

Please read the file INSTALL for installation instructions.

Many Git online resources are accessible from https://git-scm.com/ including full documentation and Git related tools.

See Documentation/gittutorial.txt to get started, then see Documentation/giteveryday.txt for a useful minimum set of commands, and Documentation/git-<commandname>.txt for documentation of each command. If git has been correctly installed, then the tutorial can also be read with man gittutorial or git help tutorial, and the documentation of each command with man git-<commandname> or git help <commandname>.

CVS users may also want to read Documentation/gitcvs-migration.txt (man gitcvs-migration or git help cvs-migration if git is installed).

The user discussion and development of Git take place on the Git mailing list -- everyone is welcome to post bug reports, feature requests, comments and patches to git@vger.kernel.org (read Documentation/SubmittingPatches for instructions on patch submission and Documentation/CodingGuidelines).

Those wishing to help with error message, usage and informational message string translations (localization l10) should see po/README.md (a po file is a Portable Object file that holds the translations).

To subscribe to the list, send an email with just "subscribe git" in the body to majordomo@vger.kernel.org (not the Git list). The mailing list archives are available at https://lore.kernel.org/git/, http://marc.info/?l=git and other archival sites.

Issues which are security relevant should be disclosed privately to the Git Security mailing list git-security@googlegroups.com.

The maintainer frequently sends the "What's cooking" reports that list the current status of various development topics to the mailing list. The discussion following them give a good reference for project status, development direction and remaining tasks.

The name "git" was given by Linus Torvalds when he wrote the very first version. He described the tool as "the stupid content tracker" and the name as (depending on your mood):

  • random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of "get" may or may not be relevant.
  • stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang.
  • "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.
  • "goddamn idiotic truckload of sh*t": when it breaks