Go to file
Kirill Smelkov 8518ff8fab combine-diff: optimize combine_diff_path sets intersection
When generating combined diff, for each commit, we intersect diff
paths from diff(parent_0,commit) to diff(parent_i,commit) comparing
all paths pairs, i.e. doing it the quadratic way. That is correct,
but could be optimized.

Paths come from trees in sorted (= tree) order, and so does diff_tree()
emits resulting paths in that order too. Now if we look at diffcore
transformations, all of them, except diffcore_order, preserve resulting
path ordering:

    - skip_stat_unmatch, grep, pickaxe, filter
                            -- just skip elements -> order stays preserved

    - break                 -- just breaks diff for a path, adding path
                               dup after the path -> order stays preserved

    - detect rename/copy    -- resulting paths are emitted sorted
                               (verified empirically)

So only diffcore_order changes diff paths ordering.

But diffcore_order meaning affects only presentation - i.e. only how to
show the diff, so we could do all the internal computations without
paths reordering, and order only resultant paths set. This is faster,
since, if we know two paths sets are all ordered, their intersection
could be done in linear time.

This patch does just that.

Timings for `git log --raw --no-abbrev --no-renames` without `-c` ("git log")
and with `-c` ("git log -c") before and after the patch are as follows:

                linux.git v3.10..v3.11

            log     log -c

    before  1.9s    20.4s
    after   1.9s    16.6s

                navy.git    (private repo)

            log     log -c

    before  0.83s   15.6s
    after   0.83s    2.1s

P.S.

I think linux.git case is sped up not so much as the second one, since
in navy.git, there are more exotic (subtree, etc) merges.

P.P.S.

My tracing showed that the rest of the time (16.6s vs 1.9s) is usually
spent in computing huge diffs from commit to second parent. Will try to
deal with it, if I'll have time.

P.P.P.S.

For combine_diff_path, ->len is not needed anymore - will remove it in
the next noisy cleanup path, to maintain good signal/noise ratio here.

Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-02-24 14:44:57 -08:00
Documentation Git 1.9-rc1 2014-01-27 11:01:35 -08:00
block-sha1
builtin Merge branch 'sb/repack-in-c' 2014-01-27 10:45:49 -08:00
compat Merge branch 'ef/mingw-write' 2014-01-27 10:44:59 -08:00
contrib Merge branch 'jk/complete-merge-base' 2014-01-27 10:43:55 -08:00
git-gui git-gui 0.19.0 2014-01-21 13:16:17 -08:00
gitk-git Merge git://ozlabs.org/~paulus/gitk 2014-01-23 08:50:50 -08:00
gitweb gitweb: Denote non-heads, non-remotes branches 2013-12-12 12:37:37 -08:00
mergetools
perl git-svn: memoize _rev_list and rebuild 2014-01-23 02:54:26 +00:00
po l10n: Bulgarian translation of git (222t21f1967u) 2014-01-29 14:29:15 +02:00
ppc
t diff test: add tests for combine-diff with orderfile 2014-02-24 14:44:57 -08:00
templates
vcs-svn strbuf: introduce starts_with() and ends_with() 2013-12-05 14:12:52 -08:00
xdiff
.gitattributes
.gitignore Merge branch 'jk/remove-deprecated' 2013-12-12 14:18:34 -08:00
.mailmap
COPYING
GIT-VERSION-GEN Git 1.9-rc2 2014-01-31 14:16:06 -08:00
INSTALL
LGPL-2.1
Makefile Merge branch 'jk/revision-o-is-in-libgit-a' 2014-01-27 10:45:52 -08:00
README
RelNotes Git 1.8.5.3 2014-01-13 11:28:26 -08:00
abspath.c Merge branch 'ap/path-max' 2014-01-10 10:32:18 -08:00
aclocal.m4
advice.c
advice.h
alias.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
alloc.c
archive-tar.c
archive-zip.c
archive.c
archive.h
argv-array.c
argv-array.h
attr.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
attr.h
base85.c
bisect.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
bisect.h
blob.c
blob.h
branch.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
branch.h
builtin.h builtin/help.c: speed up is_git_command() by checking for builtin commands first 2014-01-06 11:26:31 -08:00
bulk-checkin.c
bulk-checkin.h
bundle.c bundle: use argv-array 2013-11-12 13:32:11 -08:00
bundle.h
cache-tree.c
cache-tree.h
cache.h Merge branch 'mh/safe-create-leading-directories' 2014-01-27 10:45:33 -08:00
check-builtins.sh
check-racy.c
check_bindir
color.c
color.h
column.c
column.h
combine-diff.c combine-diff: optimize combine_diff_path sets intersection 2014-02-24 14:44:57 -08:00
command-list.txt peek-remote: remove deprecated alias of ls-remote 2013-11-12 14:10:22 -08:00
commit-slab.h commit-slab: sizeof() the right type in xrealloc 2013-12-02 12:46:01 -08:00
commit.c Merge branch 'vm/octopus-merge-bases-simplify' 2014-01-10 10:33:45 -08:00
commit.h Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
config.c Merge branch 'cc/starts-n-ends-with' 2013-12-17 12:02:44 -08:00
config.mak.in
config.mak.uname Add MirBSD support to the build system. 2014-01-02 10:19:14 -08:00
configure.ac
connect.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
connect.h connect.c: refactor url parsing 2013-12-09 14:54:48 -08:00
connected.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
connected.h connected.c: add new variant that runs with --shallow-file 2013-12-10 16:14:18 -08:00
convert.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
convert.h
copy.c
credential-cache--daemon.c
credential-cache.c
credential-store.c
credential.c
credential.h
csum-file.c Merge branch 'jk/sha1write-void' 2014-01-10 10:33:09 -08:00
csum-file.h Merge branch 'jk/sha1write-void' 2014-01-10 10:33:09 -08:00
ctype.c
daemon.c Merge branch 'nd/daemon-informative-errors-typofix' 2014-01-10 10:32:59 -08:00
date.c Merge branch 'jk/date-c-double-semicolon' 2013-10-30 12:11:01 -07:00
decorate.c
decorate.h
delta.h
diff-delta.c
diff-lib.c
diff-no-index.c diff: move no-index detection to builtin/diff.c 2013-12-12 12:23:02 -08:00
diff.c Merge branch 'jk/diff-filespec-cleanup' 2014-01-27 10:45:03 -08:00
diff.h Merge branch 'tg/diff-no-index-refactor' 2013-12-27 14:58:17 -08:00
diffcore-break.c
diffcore-delta.c
diffcore-order.c diffcore-order: export generic ordering interface 2014-02-24 14:44:57 -08:00
diffcore-pickaxe.c
diffcore-rename.c
diffcore.h diffcore-order: export generic ordering interface 2014-02-24 14:44:57 -08:00
dir.c Merge branch 'mh/safe-create-leading-directories' 2014-01-27 10:45:33 -08:00
dir.h
editor.c
entry.c
environment.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
exec_cmd.c
exec_cmd.h
fast-import.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
fetch-pack.c Merge branch 'jk/allow-fetch-onelevel-refname' 2014-01-27 10:44:14 -08:00
fetch-pack.h Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
fmt-merge-msg.h
fsck.c
fsck.h
generate-cmdlist.sh
gettext.c gettext.c: detect the vsnprintf bug at runtime 2013-12-04 16:10:51 -08:00
gettext.h
git-add--interactive.perl
git-am.sh Merge branch 'jk/replace-perl-in-built-scripts' 2013-12-05 12:58:21 -08:00
git-archimport.perl
git-bisect.sh typofixes: fix misspelt comments 2013-11-12 09:24:27 -08:00
git-compat-util.h Merge branch 'bs/mirbsd' 2014-01-10 10:33:14 -08:00
git-cvsexportcommit.perl
git-cvsimport.perl
git-cvsserver.perl Merge branch 'ak/cvsserver-stabilize-use-of-hash-keys' 2013-11-04 14:58:05 -08:00
git-difftool--helper.sh difftool: display the number of files in the diff queue in the prompt 2013-12-06 14:00:27 -08:00
git-difftool.perl
git-filter-branch.sh
git-instaweb.sh use @@PERL@@ in built scripts 2013-10-29 12:41:17 -07:00
git-merge-octopus.sh
git-merge-one-file.sh
git-merge-resolve.sh
git-mergetool--lib.sh remove #!interpreter line from shell libraries 2013-11-26 14:23:56 -08:00
git-mergetool.sh
git-p4.py git p4: fix an error message when "p4 where" fails 2014-01-22 08:06:19 -08:00
git-parse-remote.sh remove #!interpreter line from shell libraries 2013-11-26 14:23:56 -08:00
git-pull.sh Merge branch 'jk/pull-rebase-using-fork-point' 2014-01-17 12:04:29 -08:00
git-quiltimport.sh
git-rebase--am.sh remove #!interpreter line from shell libraries 2013-11-26 14:23:56 -08:00
git-rebase--interactive.sh remove #!interpreter line from shell libraries 2013-11-26 14:23:56 -08:00
git-rebase--merge.sh remove #!interpreter line from shell libraries 2013-11-26 14:23:56 -08:00
git-rebase.sh rebase: fix fork-point with zero arguments 2014-01-09 15:05:26 -08:00
git-relink.perl
git-remote-testgit.sh
git-request-pull.sh Merge branch 'jk/replace-perl-in-built-scripts' 2013-12-05 12:58:21 -08:00
git-send-email.perl Merge branch 'rk/send-email-ssl-cert' 2014-01-27 10:44:34 -08:00
git-sh-i18n.sh remove #!interpreter line from shell libraries 2013-11-26 14:23:56 -08:00
git-sh-setup.sh Merge branch 'jn/pager-lv-default-env' 2014-01-13 11:33:35 -08:00
git-stash.sh stash: handle specifying stashes with $IFS 2014-01-07 10:51:04 -08:00
git-submodule.sh Merge branch 'fp/submodule-checkout-mode' 2014-01-17 12:21:20 -08:00
git-svn.perl
git-web--browse.sh web--browse: Add support for xdg-open 2013-10-30 13:54:15 -07:00
git.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
git.rc Makefile: Fix compilation of Windows resource file 2014-01-23 10:00:28 -08:00
git.spec.in
gpg-interface.c
gpg-interface.h
graph.c
graph.h
grep.c
grep.h
hash.c
hash.h
help.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
help.h
hex.c
http-backend.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
http-fetch.c
http-push.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
http-walker.c
http.c Merge branch 'cc/starts-n-ends-with' 2013-12-17 12:02:44 -08:00
http.h Merge branch 'bc/http-100-continue' 2013-12-05 12:58:59 -08:00
ident.c Merge branch 'jk/split-broken-ident' into maint 2013-11-07 14:34:51 -08:00
imap-send.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
kwset.c
kwset.h
levenshtein.c
levenshtein.h
line-log.c line-log: convert to using diff_tree_sha1() 2014-02-05 10:50:36 -08:00
line-log.h
line-range.c
line-range.h
list-objects.c Merge branch 'jk/mark-edges-uninteresting' 2014-01-27 10:45:08 -08:00
list-objects.h
ll-merge.c
ll-merge.h
lockfile.c
log-tree.c Merge branch 'bc/log-decoration' 2014-01-10 10:32:39 -08:00
log-tree.h
mailmap.c
mailmap.h
match-trees.c
merge-blobs.c
merge-blobs.h
merge-recursive.c Merge branch 'mh/safe-create-leading-directories' 2014-01-27 10:45:33 -08:00
merge-recursive.h
merge.c
mergesort.c
mergesort.h
name-hash.c
notes-cache.c
notes-cache.h
notes-merge.c
notes-merge.h
notes-utils.c Merge branch 'cc/starts-n-ends-with' 2013-12-17 12:02:44 -08:00
notes-utils.h
notes.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
notes.h
object.c
object.h
pack-check.c
pack-revindex.c
pack-revindex.h
pack-write.c Merge branch 'jk/sha1write-void' 2014-01-10 10:33:09 -08:00
pack.h pack-objects: name pack files after trailer hash 2013-12-05 15:40:11 -08:00
pager.c Merge branch 'jn/pager-lv-default-env' 2014-01-13 11:33:35 -08:00
parse-options-cb.c
parse-options.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
parse-options.h parse-options: remove OPT_BOOLEAN 2013-12-09 11:24:16 -08:00
patch-delta.c
patch-ids.c
patch-ids.h
path.c
pathspec.c Merge branch 'nd/negative-pathspec' 2014-01-10 10:31:48 -08:00
pathspec.h Support pathspec magic :(exclude) and its short form :! 2013-12-06 13:00:39 -08:00
pkt-line.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
pkt-line.h
preload-index.c
pretty.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
prio-queue.c
prio-queue.h
progress.c
progress.h
prompt.c Revert "prompt: clean up strbuf usage" 2014-01-02 10:21:40 -08:00
prompt.h
quote.c
quote.h
reachable.c
reachable.h
read-cache.c
reflog-walk.c
reflog-walk.h
refs.c Merge branch 'mh/safe-create-leading-directories' 2014-01-27 10:45:33 -08:00
refs.h Merge branch 'sb/refs-code-cleanup' 2013-11-01 07:38:58 -07:00
remote-curl.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
remote-testsvn.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
remote.c Merge branch 'mh/retire-ref-fetch-rules' 2014-01-27 10:44:07 -08:00
remote.h Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
replace_object.c replace_object: don't check read_replace_refs twice 2013-12-12 11:53:48 -08:00
rerere.c
rerere.h
resolve-undo.c
resolve-undo.h
revision.c revision: convert to using diff_tree_sha1() 2014-02-05 10:51:16 -08:00
revision.h Merge branch 'jc/ref-excludes' 2013-12-05 12:59:09 -08:00
run-command.c run-command: trivial style fixes 2013-10-31 13:48:26 -07:00
run-command.h
send-pack.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
send-pack.h remote.h: replace struct extra_have_objects with struct sha1_array 2013-12-10 16:14:15 -08:00
sequencer.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
sequencer.h
server-info.c
setup.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
sh-i18n--envsubst.c
sha1-array.c
sha1-array.h
sha1-lookup.c
sha1-lookup.h
sha1_file.c Merge branch 'ss/safe-create-leading-dir-with-slash' 2014-01-27 10:45:37 -08:00
sha1_name.c Merge branch 'jk/interpret-branch-name-fix' 2014-01-27 10:44:21 -08:00
shallow.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
shell.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
shortlog.h
show-index.c
sideband.c
sideband.h
sigchain.c
sigchain.h
strbuf.c strbuf: introduce starts_with() and ends_with() 2013-12-05 14:12:52 -08:00
strbuf.h
streaming.c Merge branch 'ef/mingw-write' 2014-01-27 10:44:59 -08:00
streaming.h
string-list.c
string-list.h
submodule.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
submodule.h
symlinks.c
tag.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
tag.h
tar.h
test-chmtime.c
test-ctype.c
test-date.c
test-delta.c
test-dump-cache-tree.c
test-genrandom.c
test-index-version.c
test-line-buffer.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
test-match-trees.c
test-mergesort.c
test-mktemp.c
test-parse-options.c
test-path-utils.c
test-prio-queue.c
test-read-cache.c
test-regex.c
test-revision-walking.c
test-run-command.c
test-scrap-cache-tree.c
test-sha1.c
test-sha1.sh
test-sigchain.c
test-string-list.c replace {pre,suf}fixcmp() with {starts,ends}_with() 2013-12-05 14:13:21 -08:00
test-subprocess.c
test-svn-fe.c
test-urlmatch-normalization.c
test-wildmatch.c
thread-utils.c
thread-utils.h
trace.c shallow.c: the 8 steps to select new commits for .git/shallow 2013-12-10 16:14:16 -08:00
transport-helper.c Merge branch 'ef/mingw-write' 2014-01-27 10:44:59 -08:00
transport.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
transport.h fetch: add --update-shallow to accept refs that update .git/shallow 2013-12-10 16:14:17 -08:00
tree-diff.c tree-diff: convert diff_root_tree_sha1() to just call diff_tree_sha1 with old=NULL 2014-02-05 10:49:07 -08:00
tree-walk.c tree-walk: finally switch over tree descriptors to contain a pre-parsed entry 2014-02-24 14:43:29 -08:00
tree-walk.h tree-walk: finally switch over tree descriptors to contain a pre-parsed entry 2014-02-24 14:43:29 -08:00
tree.c
tree.h
unimplemented.sh
unix-socket.c
unix-socket.h
unpack-trees.c Merge branch 'jk/two-way-merge-corner-case-fix' into maint 2013-12-17 11:32:17 -08:00
unpack-trees.h
upload-pack.c Merge branch 'nd/shallow-clone' 2014-01-17 12:21:20 -08:00
url.c
url.h
urlmatch.c
urlmatch.h
usage.c
userdiff.c
userdiff.h
utf8.c
utf8.h
varint.c
varint.h
version.c
version.h
walker.c
walker.h
wildmatch.c
wildmatch.h
wrap-for-bin.sh
wrapper.c typofixes: fix misspelt comments 2013-11-12 09:24:27 -08:00
write_or_die.c
ws.c
wt-status.c Merge branch 'cc/starts-n-ends-with' 2013-12-17 12:02:44 -08:00
wt-status.h commit -v: strip diffs and submodule shortlogs from the commit message 2013-12-05 14:39:11 -08:00
xdiff-interface.c
xdiff-interface.h
zlib.c

README

////////////////////////////////////////////////////////////////

	Git - the stupid content tracker

////////////////////////////////////////////////////////////////

"git" can mean anything, depending on your mood.

 - random three-letter combination that is pronounceable, and not
   actually used by any common UNIX command.  The fact that it is a
   mispronunciation of "get" may or may not be relevant.
 - stupid. contemptible and despicable. simple. Take your pick from the
   dictionary of slang.
 - "global information tracker": you're in a good mood, and it actually
   works for you. Angels sing, and a light suddenly fills the room.
 - "goddamn idiotic truckload of sh*t": when it breaks

Git is a fast, scalable, distributed revision control system with an
unusually rich command set that provides both high-level operations
and full access to internals.

Git is an Open Source project covered by the GNU General Public
License version 2 (some parts of it are under different licenses,
compatible with the GPLv2). It was originally written by Linus
Torvalds with help of a group of hackers around the net.

Please read the file INSTALL for installation instructions.

See Documentation/gittutorial.txt to get started, then see
Documentation/everyday.txt for a useful minimum set of commands, and
Documentation/git-commandname.txt for documentation of each command.
If git has been correctly installed, then the tutorial can also be
read with "man gittutorial" or "git help tutorial", and the
documentation of each command with "man git-commandname" or "git help
commandname".

CVS users may also want to read Documentation/gitcvs-migration.txt
("man gitcvs-migration" or "git help cvs-migration" if git is
installed).

Many Git online resources are accessible from http://git-scm.com/
including full documentation and Git related tools.

The user discussion and development of Git take place on the Git
mailing list -- everyone is welcome to post bug reports, feature
requests, comments and patches to git@vger.kernel.org (read
Documentation/SubmittingPatches for instructions on patch submission).
To subscribe to the list, send an email with just "subscribe git" in
the body to majordomo@vger.kernel.org. The mailing list archives are
available at http://news.gmane.org/gmane.comp.version-control.git/,
http://marc.info/?l=git and other archival sites.

The maintainer frequently sends the "What's cooking" reports that
list the current status of various development topics to the mailing
list.  The discussion following them give a good reference for
project status, development direction and remaining tasks.