3ce9e5f293
The previous change added a --path-walk option to 'git pack-objects'.
Create a performance test that demonstrates the time and space benefits
of the feature.
In order to get an appropriate comparison, we need to avoid reusing
deltas and recompute them from scratch.
Compare the creation of a thin pack representing a small push and the
creation of a relatively large non-thin pack.
Running on my copy of the Git repository results in this data (removing
the repack tests for --name-hash-version):
Test this tree
------------------------------------------------------------------------
5313.2: thin pack with --name-hash-version=1 0.02(0.01+0.01)
5313.3: thin pack size with --name-hash-version=1 1.6K
5313.4: big pack with --name-hash-version=1 2.55(4.20+0.26)
5313.5: big pack size with --name-hash-version=1 16.4M
5313.6: shallow fetch pack with --name-hash-version=1 1.24(2.03+0.08)
5313.7: shallow pack size with --name-hash-version=1 12.2M
5313.10: thin pack with --name-hash-version=2 0.03(0.01+0.01)
5313.11: thin pack size with --name-hash-version=2 1.6K
5313.12: big pack with --name-hash-version=2 1.91(3.23+0.20)
5313.13: big pack size with --name-hash-version=2 16.4M
5313.14: shallow fetch pack with --name-hash-version=2 1.06(1.57+0.10)
5313.15: shallow pack size with --name-hash-version=2 12.5M
5313.18: thin pack with --path-walk 0.03(0.01+0.01)
5313.19: thin pack size with --path-walk 1.6K
5313.20: big pack with --path-walk 2.05(3.24+0.27)
5313.21: big pack size with --path-walk 16.3M
5313.22: shallow fetch pack with --path-walk 1.08(1.66+0.07)
5313.23: shallow pack size with --path-walk 12.4M
This can be reformatted as follows:
Pack Type Hash v1 Hash v2 Path Walk
---------------------------------------------------
thin pack (time) 0.02s 0.03s 0.03s
(size) 1.6K 1.6K 1.6K
big pack (time) 2.55s 1.91s 2.05s
(size) 16.4M 16.4M 16.3M
shallow pack (time) 1.24s 1.06s 1.08s
(size) 12.2M 12.5M 12.4M
Note that the timing is slower because there is no threading in the
--path-walk case (yet). Also, the shallow pack cases are really not
using the --path-walk logic right now because it is disabled until some
additions are made to the path walk API.
The cases where the --path-walk option really shines is when the default
name-hash is overwhelmed with unhelpful collisions. An open source
example can be found in the microsoft/fluentui repo [1] at a certain
commit [2].
[1] https://github.com/microsoft/fluentui
[2] e70848ebac1cd720875bccaa3026f4a9ed700e08
Running the tests on this repo results in the following comparison table:
Pack Type Hash v1 Hash v2 Path Walk
---------------------------------------------------
thin pack (time) 0.36s 0.12s 0.08s
(size) 1.2M 22.0K 18.4K
big pack (time) 2.00s 2.90s 2.21s
(size) 20.4M 25.9M 19.5M
shallow pack (time) 1.41s 1.80s 1.65s
(size) 34.4M 33.7M 33.6M
Notice in particular that in the small thin pack, the time performance
has improved from 0.36s for --name-hash-version=1 to 0.08s and this is
likely due to the improved size of the resulting pack: 18.4K instead of
1.2M. The relatively new --name-hash-version=2 is competitive with
--path-walk (0.12s and 22.0K) but not quite as successful.
Finally, running this on a copy of the Linux kernel repository results
in these data points:
Pack Type Hash v1 Hash v2 Path Walk
---------------------------------------------------
thin pack (time) 0.03s 0.13s 0.03s
(size) 4.6K 4.6K 4.6K
big pack (time) 15.29s 12.32s 13.92s
(size) 201.1M 159.1M 158.5M
shallow pack (time) 10.88s 22.93s 22.74s
(size) 269.2M 273.8M 267.7M
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
||
|---|---|---|
| .github | ||
| Documentation | ||
| bin-wrappers | ||
| block-sha1 | ||
| builtin | ||
| ci | ||
| compat | ||
| contrib | ||
| ewah | ||
| git-gui | ||
| gitk-git | ||
| gitweb | ||
| mergetools | ||
| negotiator | ||
| oss-fuzz | ||
| perl | ||
| po | ||
| refs | ||
| reftable | ||
| sha1 | ||
| sha1collisiondetection@855827c583 | ||
| sha1dc | ||
| sha256 | ||
| subprojects | ||
| t | ||
| templates | ||
| trace2 | ||
| xdiff | ||
| .cirrus.yml | ||
| .clang-format | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .gitmodules | ||
| .mailmap | ||
| .tsan-suppressions | ||
| CODE_OF_CONDUCT.md | ||
| COPYING | ||
| GIT-BUILD-OPTIONS.in | ||
| GIT-VERSION-FILE.in | ||
| GIT-VERSION-GEN | ||
| INSTALL | ||
| LGPL-2.1 | ||
| Makefile | ||
| README.md | ||
| RelNotes | ||
| SECURITY.md | ||
| abspath.c | ||
| abspath.h | ||
| aclocal.m4 | ||
| add-interactive.c | ||
| add-interactive.h | ||
| add-patch.c | ||
| advice.c | ||
| advice.h | ||
| alias.c | ||
| alias.h | ||
| alloc.c | ||
| alloc.h | ||
| apply.c | ||
| apply.h | ||
| archive-tar.c | ||
| archive-zip.c | ||
| archive.c | ||
| archive.h | ||
| attr.c | ||
| attr.h | ||
| banned.h | ||
| base85.c | ||
| base85.h | ||
| bisect.c | ||
| bisect.h | ||
| blame.c | ||
| blame.h | ||
| blob.c | ||
| blob.h | ||
| bloom.c | ||
| bloom.h | ||
| branch.c | ||
| branch.h | ||
| builtin.h | ||
| bulk-checkin.c | ||
| bulk-checkin.h | ||
| bundle-uri.c | ||
| bundle-uri.h | ||
| bundle.c | ||
| bundle.h | ||
| cache-tree.c | ||
| cache-tree.h | ||
| cbtree.c | ||
| cbtree.h | ||
| chdir-notify.c | ||
| chdir-notify.h | ||
| check-builtins.sh | ||
| checkout.c | ||
| checkout.h | ||
| chunk-format.c | ||
| chunk-format.h | ||
| color.c | ||
| color.h | ||
| column.c | ||
| column.h | ||
| combine-diff.c | ||
| command-list.txt | ||
| commit-graph.c | ||
| commit-graph.h | ||
| commit-reach.c | ||
| commit-reach.h | ||
| commit-slab-decl.h | ||
| commit-slab-impl.h | ||
| commit-slab.h | ||
| commit.c | ||
| commit.h | ||
| common-exit.c | ||
| common-init.c | ||
| common-init.h | ||
| common-main.c | ||
| config.c | ||
| config.h | ||
| config.mak.dev | ||
| config.mak.in | ||
| config.mak.uname | ||
| configure.ac | ||
| connect.c | ||
| connect.h | ||
| connected.c | ||
| connected.h | ||
| convert.c | ||
| convert.h | ||
| copy.c | ||
| copy.h | ||
| credential.c | ||
| credential.h | ||
| csum-file.c | ||
| csum-file.h | ||
| ctype.c | ||
| daemon.c | ||
| date.c | ||
| date.h | ||
| decorate.c | ||
| decorate.h | ||
| delta-islands.c | ||
| delta-islands.h | ||
| delta.h | ||
| detect-compiler | ||
| diagnose.c | ||
| diagnose.h | ||
| diff-delta.c | ||
| diff-lib.c | ||
| diff-merges.c | ||
| diff-merges.h | ||
| diff-no-index.c | ||
| diff.c | ||
| diff.h | ||
| diffcore-break.c | ||
| diffcore-delta.c | ||
| diffcore-order.c | ||
| diffcore-pickaxe.c | ||
| diffcore-rename.c | ||
| diffcore-rotate.c | ||
| diffcore.h | ||
| dir-iterator.c | ||
| dir-iterator.h | ||
| dir.c | ||
| dir.h | ||
| editor.c | ||
| editor.h | ||
| entry.c | ||
| entry.h | ||
| environment.c | ||
| environment.h | ||
| exec-cmd.c | ||
| exec-cmd.h | ||
| fetch-negotiator.c | ||
| fetch-negotiator.h | ||
| fetch-pack.c | ||
| fetch-pack.h | ||
| fmt-merge-msg.c | ||
| fmt-merge-msg.h | ||
| fsck.c | ||
| fsck.h | ||
| fsmonitor--daemon.h | ||
| fsmonitor-ipc.c | ||
| fsmonitor-ipc.h | ||
| fsmonitor-ll.h | ||
| fsmonitor-path-utils.h | ||
| fsmonitor-settings.c | ||
| fsmonitor-settings.h | ||
| fsmonitor.c | ||
| fsmonitor.h | ||
| generate-cmdlist.sh | ||
| generate-configlist.sh | ||
| generate-hooklist.sh | ||
| generate-perl.sh | ||
| generate-python.sh | ||
| generate-script.sh | ||
| gettext.c | ||
| gettext.h | ||
| git-archimport.perl | ||
| git-compat-util.h | ||
| git-curl-compat.h | ||
| git-cvsexportcommit.perl | ||
| git-cvsimport.perl | ||
| git-cvsserver.perl | ||
| git-difftool--helper.sh | ||
| git-filter-branch.sh | ||
| git-instaweb.sh | ||
| git-merge-octopus.sh | ||
| git-merge-one-file.sh | ||
| git-merge-resolve.sh | ||
| git-mergetool--lib.sh | ||
| git-mergetool.sh | ||
| git-p4.py | ||
| git-quiltimport.sh | ||
| git-request-pull.sh | ||
| git-send-email.perl | ||
| git-sh-i18n.sh | ||
| git-sh-setup.sh | ||
| git-submodule.sh | ||
| git-svn.perl | ||
| git-web--browse.sh | ||
| git-zlib.c | ||
| git-zlib.h | ||
| git.c | ||
| git.rc.in | ||
| gpg-interface.c | ||
| gpg-interface.h | ||
| graph.c | ||
| graph.h | ||
| grep.c | ||
| grep.h | ||
| hash-lookup.c | ||
| hash-lookup.h | ||
| hash.h | ||
| hashmap.c | ||
| hashmap.h | ||
| help.c | ||
| help.h | ||
| hex-ll.c | ||
| hex-ll.h | ||
| hex.c | ||
| hex.h | ||
| hook.c | ||
| hook.h | ||
| http-backend.c | ||
| http-fetch.c | ||
| http-push.c | ||
| http-walker.c | ||
| http.c | ||
| http.h | ||
| ident.c | ||
| ident.h | ||
| imap-send.c | ||
| iterator.h | ||
| json-writer.c | ||
| json-writer.h | ||
| khash.h | ||
| kwset.c | ||
| kwset.h | ||
| levenshtein.c | ||
| levenshtein.h | ||
| line-log.c | ||
| line-log.h | ||
| line-range.c | ||
| line-range.h | ||
| linear-assignment.c | ||
| linear-assignment.h | ||
| list-objects-filter-options.c | ||
| list-objects-filter-options.h | ||
| list-objects-filter.c | ||
| list-objects-filter.h | ||
| list-objects.c | ||
| list-objects.h | ||
| list.h | ||
| lockfile.c | ||
| lockfile.h | ||
| log-tree.c | ||
| log-tree.h | ||
| loose.c | ||
| loose.h | ||
| ls-refs.c | ||
| ls-refs.h | ||
| mailinfo.c | ||
| mailinfo.h | ||
| mailmap.c | ||
| mailmap.h | ||
| match-trees.c | ||
| match-trees.h | ||
| mem-pool.c | ||
| mem-pool.h | ||
| merge-blobs.c | ||
| merge-blobs.h | ||
| merge-ll.c | ||
| merge-ll.h | ||
| merge-ort-wrappers.c | ||
| merge-ort-wrappers.h | ||
| merge-ort.c | ||
| merge-ort.h | ||
| merge-recursive.c | ||
| merge-recursive.h | ||
| merge.c | ||
| merge.h | ||
| mergesort.h | ||
| meson.build | ||
| meson_options.txt | ||
| midx-write.c | ||
| midx.c | ||
| midx.h | ||
| name-hash.c | ||
| name-hash.h | ||
| notes-cache.c | ||
| notes-cache.h | ||
| notes-merge.c | ||
| notes-merge.h | ||
| notes-utils.c | ||
| notes-utils.h | ||
| notes.c | ||
| notes.h | ||
| object-file-convert.c | ||
| object-file-convert.h | ||
| object-file.c | ||
| object-file.h | ||
| object-name.c | ||
| object-name.h | ||
| object-store-ll.h | ||
| object-store.h | ||
| object.c | ||
| object.h | ||
| oid-array.c | ||
| oid-array.h | ||
| oidmap.c | ||
| oidmap.h | ||
| oidset.c | ||
| oidset.h | ||
| oidtree.c | ||
| oidtree.h | ||
| pack-bitmap-write.c | ||
| pack-bitmap.c | ||
| pack-bitmap.h | ||
| pack-check.c | ||
| pack-mtimes.c | ||
| pack-mtimes.h | ||
| pack-objects.c | ||
| pack-objects.h | ||
| pack-revindex.c | ||
| pack-revindex.h | ||
| pack-write.c | ||
| pack.h | ||
| packfile.c | ||
| packfile.h | ||
| pager.c | ||
| pager.h | ||
| parallel-checkout.c | ||
| parallel-checkout.h | ||
| parse-options-cb.c | ||
| parse-options.c | ||
| parse-options.h | ||
| parse.c | ||
| parse.h | ||
| patch-delta.c | ||
| patch-ids.c | ||
| patch-ids.h | ||
| path-walk.c | ||
| path-walk.h | ||
| path.c | ||
| path.h | ||
| pathspec.c | ||
| pathspec.h | ||
| pkt-line.c | ||
| pkt-line.h | ||
| preload-index.c | ||
| preload-index.h | ||
| pretty.c | ||
| pretty.h | ||
| prio-queue.c | ||
| prio-queue.h | ||
| progress.c | ||
| progress.h | ||
| promisor-remote.c | ||
| promisor-remote.h | ||
| prompt.c | ||
| prompt.h | ||
| protocol-caps.c | ||
| protocol-caps.h | ||
| protocol.c | ||
| protocol.h | ||
| prune-packed.c | ||
| prune-packed.h | ||
| pseudo-merge.c | ||
| pseudo-merge.h | ||
| quote.c | ||
| quote.h | ||
| range-diff.c | ||
| range-diff.h | ||
| reachable.c | ||
| reachable.h | ||
| read-cache-ll.h | ||
| read-cache.c | ||
| read-cache.h | ||
| rebase-interactive.c | ||
| rebase-interactive.h | ||
| rebase.c | ||
| rebase.h | ||
| ref-filter.c | ||
| ref-filter.h | ||
| reflog-walk.c | ||
| reflog-walk.h | ||
| reflog.c | ||
| reflog.h | ||
| refs.c | ||
| refs.h | ||
| refspec.c | ||
| refspec.h | ||
| remote-curl.c | ||
| remote.c | ||
| remote.h | ||
| replace-object.c | ||
| replace-object.h | ||
| repo-settings.c | ||
| repo-settings.h | ||
| repository.c | ||
| repository.h | ||
| rerere.c | ||
| rerere.h | ||
| reset.c | ||
| reset.h | ||
| resolve-undo.c | ||
| resolve-undo.h | ||
| revision.c | ||
| revision.h | ||
| run-command.c | ||
| run-command.h | ||
| sane-ctype.h | ||
| scalar.c | ||
| send-pack.c | ||
| send-pack.h | ||
| sequencer.c | ||
| sequencer.h | ||
| serve.c | ||
| serve.h | ||
| server-info.c | ||
| server-info.h | ||
| setup.c | ||
| setup.h | ||
| sh-i18n--envsubst.c | ||
| sha1dc_git.c | ||
| sha1dc_git.h | ||
| shallow.c | ||
| shallow.h | ||
| shared.mak | ||
| shell.c | ||
| shortlog.h | ||
| sideband.c | ||
| sideband.h | ||
| sigchain.c | ||
| sigchain.h | ||
| simple-ipc.h | ||
| sparse-index.c | ||
| sparse-index.h | ||
| split-index.c | ||
| split-index.h | ||
| stable-qsort.c | ||
| statinfo.c | ||
| statinfo.h | ||
| strbuf.c | ||
| strbuf.h | ||
| streaming.c | ||
| streaming.h | ||
| string-list.c | ||
| string-list.h | ||
| strmap.c | ||
| strmap.h | ||
| strvec.c | ||
| strvec.h | ||
| sub-process.c | ||
| sub-process.h | ||
| submodule-config.c | ||
| submodule-config.h | ||
| submodule.c | ||
| submodule.h | ||
| symlinks.c | ||
| symlinks.h | ||
| tag.c | ||
| tag.h | ||
| tar.h | ||
| tempfile.c | ||
| tempfile.h | ||
| thread-utils.c | ||
| thread-utils.h | ||
| tmp-objdir.c | ||
| tmp-objdir.h | ||
| trace.c | ||
| trace.h | ||
| trace2.c | ||
| trace2.h | ||
| trailer.c | ||
| trailer.h | ||
| transport-helper.c | ||
| transport-internal.h | ||
| transport.c | ||
| transport.h | ||
| tree-diff.c | ||
| tree-walk.c | ||
| tree-walk.h | ||
| tree.c | ||
| tree.h | ||
| unicode-width.h | ||
| unimplemented.sh | ||
| unix-socket.c | ||
| unix-socket.h | ||
| unix-stream-server.c | ||
| unix-stream-server.h | ||
| unpack-trees.c | ||
| unpack-trees.h | ||
| upload-pack.c | ||
| upload-pack.h | ||
| url.c | ||
| url.h | ||
| urlmatch.c | ||
| urlmatch.h | ||
| usage.c | ||
| userdiff.c | ||
| userdiff.h | ||
| utf8.c | ||
| utf8.h | ||
| varint.c | ||
| varint.h | ||
| version-def.h.in | ||
| version.c | ||
| version.h | ||
| versioncmp.c | ||
| versioncmp.h | ||
| walker.c | ||
| walker.h | ||
| wildmatch.c | ||
| wildmatch.h | ||
| worktree.c | ||
| worktree.h | ||
| wrapper.c | ||
| wrapper.h | ||
| write-or-die.c | ||
| write-or-die.h | ||
| ws.c | ||
| ws.h | ||
| wt-status.c | ||
| wt-status.h | ||
| xdiff-interface.c | ||
| xdiff-interface.h | ||
README.md
Git - fast, scalable, distributed revision control system
Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.
Git is an Open Source project covered by the GNU General Public License version 2 (some parts of it are under different licenses, compatible with the GPLv2). It was originally written by Linus Torvalds with help of a group of hackers around the net.
Please read the file INSTALL for installation instructions.
Many Git online resources are accessible from https://git-scm.com/ including full documentation and Git related tools.
See Documentation/gittutorial.adoc to get started, then see
Documentation/giteveryday.adoc for a useful minimum set of commands, and
Documentation/git-<commandname>.adoc for documentation of each command.
If git has been correctly installed, then the tutorial can also be
read with man gittutorial or git help tutorial, and the
documentation of each command with man git-<commandname> or git help <commandname>.
CVS users may also want to read Documentation/gitcvs-migration.adoc
(man gitcvs-migration or git help cvs-migration if git is
installed).
The user discussion and development of Git take place on the Git mailing list -- everyone is welcome to post bug reports, feature requests, comments and patches to git@vger.kernel.org (read Documentation/SubmittingPatches for instructions on patch submission and Documentation/CodingGuidelines).
Those wishing to help with error message, usage and informational message
string translations (localization l10) should see po/README.md
(a po file is a Portable Object file that holds the translations).
To subscribe to the list, send an email to git+subscribe@vger.kernel.org (see https://subspace.kernel.org/subscribing.html for details). The mailing list archives are available at https://lore.kernel.org/git/, https://marc.info/?l=git and other archival sites.
Issues which are security relevant should be disclosed privately to the Git Security mailing list git-security@googlegroups.com.
The maintainer frequently sends the "What's cooking" reports that list the current status of various development topics to the mailing list. The discussion following them give a good reference for project status, development direction and remaining tasks.
The name "git" was given by Linus Torvalds when he wrote the very first version. He described the tool as "the stupid content tracker" and the name as (depending on your mood):
- random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of "get" may or may not be relevant.
- stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang.
- "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.
- "goddamn idiotic truckload of sh*t": when it breaks