From e53e6b4433f264250c2e586167caf61721b0185c Mon Sep 17 00:00:00 2001 From: Brian Downing Date: Thu, 10 Jun 2010 21:59:07 -0500 Subject: [PATCH] unpack-trees: Make index lookahead less pessimal When traversing trees with an index, the current index pointer (o->cache_bottom) occasionally has to be temporarily advanced forwards to match the traversal order of the tree, which is not the same as the sort order of the index. The existing algorithm that did this (introduced in 730f72840cc50c523fe4cdd796ea2d2fc4571a28) would get "stuck" when the cache_bottom was popped and then repeatedly check the same index entries over and over. This represents a serious performance regression for large repositories compared to the old "broken" traversal order. This commit makes a simple change to mitigate this. Whenever find_cache_pos sees that the current pos is also the cache_bottom, and it has already been unpacked, it advances the cache_bottom as well as the current pos. This prevents the above "sticking" behavior without dramatically changing the algorithm. In addition, this commit moves the unpacked check above the ce_in_traverse_path() check. The simple bitmask check is cheaper, and in the case described above will be firing quite a bit to advance the cache_bottom after a tree pop. This yields considerable performance improvements for large trees. The following are the number of function calls for "git diff HEAD" on the Linux kernel tree, with 33,307 files: Symbol Calls Before Calls After ------------------- ------------ ----------- unpack_callback 35,332 35,332 find_cache_pos 37,357 37,357 ce_in_traverse_path 4,979,473 37,357 do_compare_entry 6,828,181 251,925 df_name_compare 6,828,181 251,925 And on a repository of 187,456 files: Symbol Calls Before Calls After ------------------- ------------ ----------- unpack_callback 197,958 197,958 find_cache_pos 208,460 208,460 ce_in_traverse_path 37,308,336 208,460 do_compare_entry 156,950,469 2,690,626 df_name_compare 156,950,469 2,690,626 On the latter repository, user time for "git diff HEAD" was reduced from 5.58 to 0.42 seconds. This is compared to 0.30 seconds before the traversal order fix was implemented. Signed-off-by: Brian Downing Signed-off-by: Junio C Hamano --- unpack-trees.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 75f54cac97..fe2340892a 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -528,9 +528,17 @@ static int find_cache_pos(struct traverse_info *info, const char *ce_name, *ce_slash; int cmp, ce_len; - if (!ce_in_traverse_path(ce, info)) + if (ce->ce_flags & CE_UNPACKED) { + /* + * cache_bottom entry is already unpacked, so + * we can never match it; don't check it + * again. + */ + if (pos == o->cache_bottom) + ++o->cache_bottom; continue; - if (ce->ce_flags & CE_UNPACKED) + } + if (!ce_in_traverse_path(ce, info)) continue; ce_name = ce->name + pfxlen; ce_slash = strchr(ce_name, '/');