git/builtin
Orgad Shaneh 7ea0c2f44d fetch: do not look for submodule changes in unchanged refs
When fetching recursively with submodules, for each ref in the
superproject, we call check_for_new_submodule_commits() which collects all
the objects that have to be checked for submodule changes on
calculate_changed_submodule_paths(). On the first call, it also collects all
the existing refs for excluding them from the scan.

calculate_changed_submodule_paths() creates an argument array with all the
collected new objects, followed by --not and all the old objects. This argv
is passed to setup_revisions, which parses each argument, converts it back
to an oid and resolves the object. The parsing itself also does redundant
work, because it is treated like user input, while in fact it is a full
oid. So it needlessly attempts to look it up as ref (checks if it has ^, ~
etc.), checks if it is a file name etc.

For a repository with many refs, all of this is expensive. But if the fetch
in the superproject did not update the ref (i.e. the objects that are
required to exist in the submodule did not change), there is no need to
include it in the list.

Before commit be76c212 (fetch: ensure submodule objects fetched,
2018-12-06), submodule reference changes were only detected for refs that
were changed, but not for new refs. This commit covered also this case, but
what it did was to just include every ref.

This change should reduce the number of scanned refs by about half (except
the case of a no-op fetch, which will not scan any ref), because all the
existing refs will still be listed after --not.

The regression was reported here:
https://public-inbox.org/git/CAGHpTBKSUJzFSWc=uznSu2zB33qCSmKXM-
iAjxRCpqNK5bnhRg@mail.gmail.com/

Signed-off-by: Orgad Shaneh <orgads@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-06 09:50:49 -07:00
..
add.c dir: fix problematic API to avoid memory leaks 2020-08-18 17:17:31 -07:00
am.c Merge branch 'pw/rebase-i-more-options' 2020-09-03 12:37:01 -07:00
annotate.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
apply.c
…
archive.c
…
bisect--helper.c bisect: swap command-line options in documentation 2020-08-28 14:06:06 -07:00
blame.c messages: avoid SHA-1 in end-user facing messages 2020-08-14 09:33:37 -07:00
branch.c Merge branch 'es/get-worktrees-unsort' 2020-07-06 22:09:15 -07:00
bugreport.c Merge branch 'jk/slimmed-down' 2020-09-03 12:37:02 -07:00
bundle.c Merge branch 'bc/sha-256-part-3' 2020-08-11 18:04:11 -07:00
cat-file.c Merge branch 'cc/cat-file-usage-update' into master 2020-07-09 14:00:41 -07:00
check-attr.c
…
check-ignore.c dir: fix problematic API to avoid memory leaks 2020-08-18 17:17:31 -07:00
check-mailmap.c
…
check-ref-format.c
…
checkout-index.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
checkout.c Merge branch 'rs/checkout-no-overlay-pathspec-fix' 2020-08-31 15:49:50 -07:00
clean.c dir: fix problematic API to avoid memory leaks 2020-08-18 17:17:31 -07:00
clone.c Merge branch 'jk/strvec' 2020-08-10 10:23:57 -07:00
column.c
…
commit-graph.c Merge branch 'ds/commit-graph-bloom-updates' into master 2020-07-30 13:20:31 -07:00
commit-tree.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
commit.c Merge branch 'pw/rebase-i-more-options' 2020-09-03 12:37:01 -07:00
config.c worktree: drop get_worktrees() unused 'flags' argument 2020-06-22 10:31:15 -07:00
count-objects.c
…
credential-cache--daemon.c make credential helpers builtins 2020-08-13 11:02:08 -07:00
credential-cache.c Merge branch 'jc/undash-in-tree-git-callers' 2020-09-03 12:37:03 -07:00
credential-store.c make credential helpers builtins 2020-08-13 11:02:08 -07:00
credential.c
…
describe.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
diff-files.c diff-files: treat "i-t-a" files as "not-in-index" 2020-06-22 10:46:45 -07:00
diff-index.c
…
diff-tree.c diff-tree.c: load notes machinery when required 2020-04-20 18:22:54 -07:00
diff.c Merge branch 'ct/diff-with-merge-base-clarification' into master 2020-07-09 14:00:43 -07:00
difftool.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
env--helper.c
…
fast-export.c fast-export: use local array to store anonymized oid 2020-06-25 14:19:23 -07:00
fast-import.c Merge branch 'jk/slimmed-down' 2020-09-03 12:37:02 -07:00
fetch-pack.c fetch-pack: remove no_dependents code 2020-08-18 16:46:53 -07:00
fetch.c fetch: do not look for submodule changes in unchanged refs 2020-09-06 09:50:49 -07:00
fmt-merge-msg.c
…
for-each-ref.c Merge branch 'jk/for-each-ref-multi-key-sort-fix' 2020-05-08 14:25:04 -07:00
fsck.c fsck: do not lazy fetch known non-promisor object 2020-08-06 13:01:03 -07:00
gc.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
get-tar-commit-id.c
…
grep.c dir: fix problematic API to avoid memory leaks 2020-08-18 17:17:31 -07:00
hash-object.c
…
help.c help: drop usage of 'common' and 'useful' for guides 2020-08-04 18:34:01 -07:00
index-pack.c index-pack: adjust default threading cap 2020-08-21 12:02:36 -07:00
init-db.c Merge branch 'es/init-no-separate-git-dir-in-bare' 2020-08-24 14:54:28 -07:00
interpret-trailers.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
log.c Merge branch 'jk/log-fp-implies-m' 2020-08-17 17:02:49 -07:00
ls-files.c dir: fix problematic API to avoid memory leaks 2020-08-18 17:17:31 -07:00
ls-remote.c Merge branch 'jk/unleak-fixes' 2020-08-24 14:54:30 -07:00
ls-tree.c
…
mailinfo.c
…
mailsplit.c
…
merge-base.c
…
merge-file.c
…
merge-index.c
…
merge-ours.c
…
merge-recursive.c
…
merge-tree.c
…
merge.c sequencer: treat CHERRY_PICK_HEAD as a pseudo ref 2020-08-21 11:20:10 -07:00
mktag.c
…
mktree.c
…
multi-pack-index.c
…
mv.c git-mv: improve error message for conflicted file 2020-07-20 14:35:43 -07:00
name-rev.c messages: avoid SHA-1 in end-user facing messages 2020-08-14 09:33:37 -07:00
notes.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
pack-objects.c Merge branch 'jc/object-names-are-not-sha-1' 2020-08-19 16:14:52 -07:00
pack-redundant.c
…
pack-refs.c
…
patch-id.c
…
prune-packed.c
…
prune.c Merge branch 'tb/shallow-cleanup' 2020-05-13 12:19:18 -07:00
pull.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
push.c Merge branch 'dl/push-recurse-submodules-fix' 2020-05-05 14:54:28 -07:00
range-diff.c strvec: convert builtin/ callers away from argv_array name 2020-07-28 15:02:18 -07:00
read-tree.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
rebase.c Merge branch 'pw/rebase-i-more-options' 2020-09-03 12:37:01 -07:00
receive-pack.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
reflog.c Merge branch 'es/get-worktrees-unsort' 2020-07-06 22:09:15 -07:00
remote-ext.c strvec: convert builtin/ callers away from argv_array name 2020-07-28 15:02:18 -07:00
remote-fd.c
…
remote.c refspec: make sure stack refspec_item variables are zeroed 2020-08-17 10:39:21 -07:00
repack.c strvec: fix indentation in renamed calls 2020-07-28 15:02:18 -07:00
replace.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
rerere.c
…
reset.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
rev-list.c bisect: combine args passed to find_bisection() 2020-08-07 15:13:03 -07:00
rev-parse.c Merge branch 'tb/shallow-cleanup' 2020-05-13 12:19:18 -07:00
revert.c
…
rm.c
…
send-pack.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
shortlog.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
show-branch.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
show-index.c builtin/show-index: provide options to determine hash algo 2020-05-27 10:07:07 -07:00
show-ref.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
sparse-checkout.c Merge branch 'xl/upgrade-repo-format' 2020-06-29 14:17:24 -07:00
stash.c dir: fix problematic API to avoid memory leaks 2020-08-18 17:17:31 -07:00
stripspace.c
…
submodule--helper.c Merge branch 'jk/leakfix' 2020-08-27 14:04:49 -07:00
symbolic-ref.c
…
tag.c Merge branch 'jk/for-each-ref-multi-key-sort-fix' 2020-05-08 14:25:04 -07:00
unpack-file.c
…
unpack-objects.c
…
update-index.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
update-ref.c strvec: rename files from argv-array to strvec 2020-07-28 15:02:17 -07:00
update-server-info.c
…
upload-archive.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
upload-pack.c
…
var.c
…
verify-commit.c
…
verify-pack.c Merge branch 'bc/sha-256-part-3' 2020-08-11 18:04:11 -07:00
verify-tag.c
…
worktree.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
write-tree.c
…