Tree:
a1b19aa5d4
main
maint
master
next
seen
todo
gitgui-0.10.0
gitgui-0.10.1
gitgui-0.10.2
gitgui-0.11.0
gitgui-0.12.0
gitgui-0.13.0
gitgui-0.14.0
gitgui-0.15.0
gitgui-0.16.0
gitgui-0.17.0
gitgui-0.18.0
gitgui-0.19.0
gitgui-0.20.0
gitgui-0.21.0
gitgui-0.6.0
gitgui-0.6.1
gitgui-0.6.2
gitgui-0.6.3
gitgui-0.6.4
gitgui-0.6.5
gitgui-0.7.0
gitgui-0.7.0-rc1
gitgui-0.7.1
gitgui-0.7.2
gitgui-0.7.3
gitgui-0.7.4
gitgui-0.7.5
gitgui-0.8.0
gitgui-0.8.1
gitgui-0.8.2
gitgui-0.8.3
gitgui-0.8.4
gitgui-0.9.0
gitgui-0.9.1
gitgui-0.9.2
gitgui-0.9.3
junio-gpg-pub
v0.99
v0.99.1
v0.99.2
v0.99.3
v0.99.4
v0.99.5
v0.99.6
v0.99.7
v0.99.7a
v0.99.7b
v0.99.7c
v0.99.7d
v0.99.8
v0.99.8a
v0.99.8b
v0.99.8c
v0.99.8d
v0.99.8e
v0.99.8f
v0.99.8g
v0.99.9
v0.99.9a
v0.99.9b
v0.99.9c
v0.99.9d
v0.99.9e
v0.99.9f
v0.99.9g
v0.99.9h
v0.99.9i
v0.99.9j
v0.99.9k
v0.99.9l
v0.99.9m
v0.99.9n
v1.0.0
v1.0.0a
v1.0.0b
v1.0.1
v1.0.10
v1.0.11
v1.0.12
v1.0.13
v1.0.2
v1.0.3
v1.0.4
v1.0.5
v1.0.6
v1.0.7
v1.0.8
v1.0.9
v1.0rc1
v1.0rc2
v1.0rc3
v1.0rc4
v1.0rc5
v1.0rc6
v1.1.0
v1.1.1
v1.1.2
v1.1.3
v1.1.4
v1.1.5
v1.1.6
v1.2.0
v1.2.1
v1.2.2
v1.2.3
v1.2.4
v1.2.5
v1.2.6
v1.3.0
v1.3.0-rc1
v1.3.0-rc2
v1.3.0-rc3
v1.3.0-rc4
v1.3.1
v1.3.2
v1.3.3
v1.4.0
v1.4.0-rc1
v1.4.0-rc2
v1.4.1
v1.4.1-rc1
v1.4.1-rc2
v1.4.1.1
v1.4.2
v1.4.2-rc1
v1.4.2-rc2
v1.4.2-rc3
v1.4.2-rc4
v1.4.2.1
v1.4.2.2
v1.4.2.3
v1.4.2.4
v1.4.3
v1.4.3-rc1
v1.4.3-rc2
v1.4.3-rc3
v1.4.3.1
v1.4.3.2
v1.4.3.3
v1.4.3.4
v1.4.3.5
v1.4.4
v1.4.4-rc1
v1.4.4-rc2
v1.4.4.1
v1.4.4.2
v1.4.4.3
v1.4.4.4
v1.4.4.5
v1.5.0
v1.5.0-rc0
v1.5.0-rc1
v1.5.0-rc2
v1.5.0-rc3
v1.5.0-rc4
v1.5.0.1
v1.5.0.2
v1.5.0.3
v1.5.0.4
v1.5.0.5
v1.5.0.6
v1.5.0.7
v1.5.1
v1.5.1-rc1
v1.5.1-rc2
v1.5.1-rc3
v1.5.1.1
v1.5.1.2
v1.5.1.3
v1.5.1.4
v1.5.1.5
v1.5.1.6
v1.5.2
v1.5.2-rc0
v1.5.2-rc1
v1.5.2-rc2
v1.5.2-rc3
v1.5.2.1
v1.5.2.2
v1.5.2.3
v1.5.2.4
v1.5.2.5
v1.5.3
v1.5.3-rc0
v1.5.3-rc1
v1.5.3-rc2
v1.5.3-rc3
v1.5.3-rc4
v1.5.3-rc5
v1.5.3-rc6
v1.5.3-rc7
v1.5.3.1
v1.5.3.2
v1.5.3.3
v1.5.3.4
v1.5.3.5
v1.5.3.6
v1.5.3.7
v1.5.3.8
v1.5.4
v1.5.4-rc0
v1.5.4-rc1
v1.5.4-rc2
v1.5.4-rc3
v1.5.4-rc4
v1.5.4-rc5
v1.5.4.1
v1.5.4.2
v1.5.4.3
v1.5.4.4
v1.5.4.5
v1.5.4.6
v1.5.4.7
v1.5.5
v1.5.5-rc0
v1.5.5-rc1
v1.5.5-rc2
v1.5.5-rc3
v1.5.5.1
v1.5.5.2
v1.5.5.3
v1.5.5.4
v1.5.5.5
v1.5.5.6
v1.5.6
v1.5.6-rc0
v1.5.6-rc1
v1.5.6-rc2
v1.5.6-rc3
v1.5.6.1
v1.5.6.2
v1.5.6.3
v1.5.6.4
v1.5.6.5
v1.5.6.6
v1.6.0
v1.6.0-rc0
v1.6.0-rc1
v1.6.0-rc2
v1.6.0-rc3
v1.6.0.1
v1.6.0.2
v1.6.0.3
v1.6.0.4
v1.6.0.5
v1.6.0.6
v1.6.1
v1.6.1-rc1
v1.6.1-rc2
v1.6.1-rc3
v1.6.1-rc4
v1.6.1.1
v1.6.1.2
v1.6.1.3
v1.6.1.4
v1.6.2
v1.6.2-rc0
v1.6.2-rc1
v1.6.2-rc2
v1.6.2.1
v1.6.2.2
v1.6.2.3
v1.6.2.4
v1.6.2.5
v1.6.3
v1.6.3-rc0
v1.6.3-rc1
v1.6.3-rc2
v1.6.3-rc3
v1.6.3-rc4
v1.6.3.1
v1.6.3.2
v1.6.3.3
v1.6.3.4
v1.6.4
v1.6.4-rc0
v1.6.4-rc1
v1.6.4-rc2
v1.6.4-rc3
v1.6.4.1
v1.6.4.2
v1.6.4.3
v1.6.4.4
v1.6.4.5
v1.6.5
v1.6.5-rc0
v1.6.5-rc1
v1.6.5-rc2
v1.6.5-rc3
v1.6.5.1
v1.6.5.2
v1.6.5.3
v1.6.5.4
v1.6.5.5
v1.6.5.6
v1.6.5.7
v1.6.5.8
v1.6.5.9
v1.6.6
v1.6.6-rc0
v1.6.6-rc1
v1.6.6-rc2
v1.6.6-rc3
v1.6.6-rc4
v1.6.6.1
v1.6.6.2
v1.6.6.3
v1.7.0
v1.7.0-rc0
v1.7.0-rc1
v1.7.0-rc2
v1.7.0.1
v1.7.0.2
v1.7.0.3
v1.7.0.4
v1.7.0.5
v1.7.0.6
v1.7.0.7
v1.7.0.8
v1.7.0.9
v1.7.1
v1.7.1-rc0
v1.7.1-rc1
v1.7.1-rc2
v1.7.1.1
v1.7.1.2
v1.7.1.3
v1.7.1.4
v1.7.10
v1.7.10-rc0
v1.7.10-rc1
v1.7.10-rc2
v1.7.10-rc3
v1.7.10-rc4
v1.7.10.1
v1.7.10.2
v1.7.10.3
v1.7.10.4
v1.7.10.5
v1.7.11
v1.7.11-rc0
v1.7.11-rc1
v1.7.11-rc2
v1.7.11-rc3
v1.7.11.1
v1.7.11.2
v1.7.11.3
v1.7.11.4
v1.7.11.5
v1.7.11.6
v1.7.11.7
v1.7.12
v1.7.12-rc0
v1.7.12-rc1
v1.7.12-rc2
v1.7.12-rc3
v1.7.12.1
v1.7.12.2
v1.7.12.3
v1.7.12.4
v1.7.2
v1.7.2-rc0
v1.7.2-rc1
v1.7.2-rc2
v1.7.2-rc3
v1.7.2.1
v1.7.2.2
v1.7.2.3
v1.7.2.4
v1.7.2.5
v1.7.3
v1.7.3-rc0
v1.7.3-rc1
v1.7.3-rc2
v1.7.3.1
v1.7.3.2
v1.7.3.3
v1.7.3.4
v1.7.3.5
v1.7.4
v1.7.4-rc0
v1.7.4-rc1
v1.7.4-rc2
v1.7.4-rc3
v1.7.4.1
v1.7.4.2
v1.7.4.3
v1.7.4.4
v1.7.4.5
v1.7.5
v1.7.5-rc0
v1.7.5-rc1
v1.7.5-rc2
v1.7.5-rc3
v1.7.5.1
v1.7.5.2
v1.7.5.3
v1.7.5.4
v1.7.6
v1.7.6-rc0
v1.7.6-rc1
v1.7.6-rc2
v1.7.6-rc3
v1.7.6.1
v1.7.6.2
v1.7.6.3
v1.7.6.4
v1.7.6.5
v1.7.6.6
v1.7.7
v1.7.7-rc0
v1.7.7-rc1
v1.7.7-rc2
v1.7.7-rc3
v1.7.7.1
v1.7.7.2
v1.7.7.3
v1.7.7.4
v1.7.7.5
v1.7.7.6
v1.7.7.7
v1.7.8
v1.7.8-rc0
v1.7.8-rc1
v1.7.8-rc2
v1.7.8-rc3
v1.7.8-rc4
v1.7.8.1
v1.7.8.2
v1.7.8.3
v1.7.8.4
v1.7.8.5
v1.7.8.6
v1.7.9
v1.7.9-rc0
v1.7.9-rc1
v1.7.9-rc2
v1.7.9.1
v1.7.9.2
v1.7.9.3
v1.7.9.4
v1.7.9.5
v1.7.9.6
v1.7.9.7
v1.8.0
v1.8.0-rc0
v1.8.0-rc1
v1.8.0-rc2
v1.8.0-rc3
v1.8.0.1
v1.8.0.2
v1.8.0.3
v1.8.1
v1.8.1-rc0
v1.8.1-rc1
v1.8.1-rc2
v1.8.1-rc3
v1.8.1.1
v1.8.1.2
v1.8.1.3
v1.8.1.4
v1.8.1.5
v1.8.1.6
v1.8.2
v1.8.2-rc0
v1.8.2-rc1
v1.8.2-rc2
v1.8.2-rc3
v1.8.2.1
v1.8.2.2
v1.8.2.3
v1.8.3
v1.8.3-rc0
v1.8.3-rc1
v1.8.3-rc2
v1.8.3-rc3
v1.8.3.1
v1.8.3.2
v1.8.3.3
v1.8.3.4
v1.8.4
v1.8.4-rc0
v1.8.4-rc1
v1.8.4-rc2
v1.8.4-rc3
v1.8.4-rc4
v1.8.4.1
v1.8.4.2
v1.8.4.3
v1.8.4.4
v1.8.4.5
v1.8.5
v1.8.5-rc0
v1.8.5-rc1
v1.8.5-rc2
v1.8.5-rc3
v1.8.5.1
v1.8.5.2
v1.8.5.3
v1.8.5.4
v1.8.5.5
v1.8.5.6
v1.9-rc0
v1.9-rc1
v1.9-rc2
v1.9.0
v1.9.0-rc3
v1.9.1
v1.9.2
v1.9.3
v1.9.4
v1.9.5
v2.0.0
v2.0.0-rc0
v2.0.0-rc1
v2.0.0-rc2
v2.0.0-rc3
v2.0.0-rc4
v2.0.1
v2.0.2
v2.0.3
v2.0.4
v2.0.5
v2.1.0
v2.1.0-rc0
v2.1.0-rc1
v2.1.0-rc2
v2.1.1
v2.1.2
v2.1.3
v2.1.4
v2.10.0
v2.10.0-rc0
v2.10.0-rc1
v2.10.0-rc2
v2.10.1
v2.10.2
v2.10.3
v2.10.4
v2.10.5
v2.11.0
v2.11.0-rc0
v2.11.0-rc1
v2.11.0-rc2
v2.11.0-rc3
v2.11.1
v2.11.2
v2.11.3
v2.11.4
v2.12.0
v2.12.0-rc0
v2.12.0-rc1
v2.12.0-rc2
v2.12.1
v2.12.2
v2.12.3
v2.12.4
v2.12.5
v2.13.0
v2.13.0-rc0
v2.13.0-rc1
v2.13.0-rc2
v2.13.1
v2.13.2
v2.13.3
v2.13.4
v2.13.5
v2.13.6
v2.13.7
v2.14.0
v2.14.0-rc0
v2.14.0-rc1
v2.14.1
v2.14.2
v2.14.3
v2.14.4
v2.14.5
v2.14.6
v2.15.0
v2.15.0-rc0
v2.15.0-rc1
v2.15.0-rc2
v2.15.1
v2.15.2
v2.15.3
v2.15.4
v2.16.0
v2.16.0-rc0
v2.16.0-rc1
v2.16.0-rc2
v2.16.1
v2.16.2
v2.16.3
v2.16.4
v2.16.5
v2.16.6
v2.17.0
v2.17.0-rc0
v2.17.0-rc1
v2.17.0-rc2
v2.17.1
v2.17.2
v2.17.3
v2.17.4
v2.17.5
v2.17.6
v2.18.0
v2.18.0-rc0
v2.18.0-rc1
v2.18.0-rc2
v2.18.1
v2.18.2
v2.18.3
v2.18.4
v2.18.5
v2.19.0
v2.19.0-rc0
v2.19.0-rc1
v2.19.0-rc2
v2.19.1
v2.19.2
v2.19.3
v2.19.4
v2.19.5
v2.19.6
v2.2.0
v2.2.0-rc0
v2.2.0-rc1
v2.2.0-rc2
v2.2.0-rc3
v2.2.1
v2.2.2
v2.2.3
v2.20.0
v2.20.0-rc0
v2.20.0-rc1
v2.20.0-rc2
v2.20.1
v2.20.2
v2.20.3
v2.20.4
v2.20.5
v2.21.0
v2.21.0-rc0
v2.21.0-rc1
v2.21.0-rc2
v2.21.1
v2.21.2
v2.21.3
v2.21.4
v2.22.0
v2.22.0-rc0
v2.22.0-rc1
v2.22.0-rc2
v2.22.0-rc3
v2.22.1
v2.22.2
v2.22.3
v2.22.4
v2.22.5
v2.23.0
v2.23.0-rc0
v2.23.0-rc1
v2.23.0-rc2
v2.23.1
v2.23.2
v2.23.3
v2.23.4
v2.24.0
v2.24.0-rc0
v2.24.0-rc1
v2.24.0-rc2
v2.24.1
v2.24.2
v2.24.3
v2.24.4
v2.25.0
v2.25.0-rc0
v2.25.0-rc1
v2.25.0-rc2
v2.25.1
v2.25.2
v2.25.3
v2.25.4
v2.25.5
v2.26.0
v2.26.0-rc0
v2.26.0-rc1
v2.26.0-rc2
v2.26.1
v2.26.2
v2.26.3
v2.27.0
v2.27.0-rc0
v2.27.0-rc1
v2.27.0-rc2
v2.27.1
v2.28.0
v2.28.0-rc0
v2.28.0-rc1
v2.28.0-rc2
v2.28.1
v2.29.0
v2.29.0-rc0
v2.29.0-rc1
v2.29.0-rc2
v2.29.1
v2.29.2
v2.29.3
v2.3.0
v2.3.0-rc0
v2.3.0-rc1
v2.3.0-rc2
v2.3.1
v2.3.10
v2.3.2
v2.3.3
v2.3.4
v2.3.5
v2.3.6
v2.3.7
v2.3.8
v2.3.9
v2.30.0
v2.30.0-rc0
v2.30.0-rc1
v2.30.0-rc2
v2.30.1
v2.30.2
v2.30.3
v2.30.4
v2.30.5
v2.30.6
v2.30.7
v2.30.8
v2.30.9
v2.31.0
v2.31.0-rc0
v2.31.0-rc1
v2.31.0-rc2
v2.31.1
v2.31.2
v2.31.3
v2.31.4
v2.31.5
v2.31.6
v2.31.7
v2.31.8
v2.32.0
v2.32.0-rc0
v2.32.0-rc1
v2.32.0-rc2
v2.32.0-rc3
v2.32.1
v2.32.2
v2.32.3
v2.32.4
v2.32.5
v2.32.6
v2.32.7
v2.33.0
v2.33.0-rc0
v2.33.0-rc1
v2.33.0-rc2
v2.33.1
v2.33.2
v2.33.3
v2.33.4
v2.33.5
v2.33.6
v2.33.7
v2.33.8
v2.34.0
v2.34.0-rc0
v2.34.0-rc1
v2.34.0-rc2
v2.34.1
v2.34.2
v2.34.3
v2.34.4
v2.34.5
v2.34.6
v2.34.7
v2.34.8
v2.35.0
v2.35.0-rc0
v2.35.0-rc1
v2.35.0-rc2
v2.35.1
v2.35.2
v2.35.3
v2.35.4
v2.35.5
v2.35.6
v2.35.7
v2.35.8
v2.36.0
v2.36.0-rc0
v2.36.0-rc1
v2.36.0-rc2
v2.36.1
v2.36.2
v2.36.3
v2.36.4
v2.36.5
v2.36.6
v2.37.0
v2.37.0-rc0
v2.37.0-rc1
v2.37.0-rc2
v2.37.1
v2.37.2
v2.37.3
v2.37.4
v2.37.5
v2.37.6
v2.37.7
v2.38.0
v2.38.0-rc0
v2.38.0-rc1
v2.38.0-rc2
v2.38.1
v2.38.2
v2.38.3
v2.38.4
v2.38.5
v2.39.0
v2.39.0-rc0
v2.39.0-rc1
v2.39.0-rc2
v2.39.1
v2.39.2
v2.39.3
v2.4.0
v2.4.0-rc0
v2.4.0-rc1
v2.4.0-rc2
v2.4.0-rc3
v2.4.1
v2.4.10
v2.4.11
v2.4.12
v2.4.2
v2.4.3
v2.4.4
v2.4.5
v2.4.6
v2.4.7
v2.4.8
v2.4.9
v2.40.0
v2.40.0-rc0
v2.40.0-rc1
v2.40.0-rc2
v2.40.1
v2.41.0
v2.41.0-rc0
v2.41.0-rc1
v2.41.0-rc2
v2.5.0
v2.5.0-rc0
v2.5.0-rc1
v2.5.0-rc2
v2.5.0-rc3
v2.5.1
v2.5.2
v2.5.3
v2.5.4
v2.5.5
v2.5.6
v2.6.0
v2.6.0-rc0
v2.6.0-rc1
v2.6.0-rc2
v2.6.0-rc3
v2.6.1
v2.6.2
v2.6.3
v2.6.4
v2.6.5
v2.6.6
v2.6.7
v2.7.0
v2.7.0-rc0
v2.7.0-rc1
v2.7.0-rc2
v2.7.0-rc3
v2.7.1
v2.7.2
v2.7.3
v2.7.4
v2.7.5
v2.7.6
v2.8.0
v2.8.0-rc0
v2.8.0-rc1
v2.8.0-rc2
v2.8.0-rc3
v2.8.0-rc4
v2.8.1
v2.8.2
v2.8.3
v2.8.4
v2.8.5
v2.8.6
v2.9.0
v2.9.0-rc0
v2.9.0-rc1
v2.9.0-rc2
v2.9.1
v2.9.2
v2.9.3
v2.9.4
v2.9.5
${ noResults }
209 Commits (a1b19aa5d43c46fc570f6946ab5ad001dccc8cdf)
Author | SHA1 | Message | Date |
---|---|---|---|
Patrick Steinhardt | 0a0fbbe3ff |
refs: remove lookup cache for reference-transaction hook
When adding the reference-transaction hook, there were concerns about
the performance impact it may have on setups which do not make use of
the new hook at all. After all, it gets executed every time a reftx is
prepared, committed or aborted, which linearly scales with the number of
reference-transactions created per session. And as there are code paths
like `git push` which create a new transaction for each reference to be
updated, this may translate to calling `find_hook()` quite a lot.
To address this concern, a cache was added with the intention to not
repeatedly do negative hook lookups. Turns out this cache caused a
regression, which was fixed via
|
4 years ago |
Jeff King | 218389b9f3 |
p5302: count up to online-cpus for thread tests
When PERF_EXTRA is enabled, p5302 checks the performance of index-pack with various numbers of threads. This can be useful for deciding what the default should be (which is currently capped at 3 threads based on the results of this script). However, we only go up to 8 threads, and modern machines may have more. Let's get the number of CPUs from test-tool, and test various numbers of threads between one and that maximum. Note that the current tests aren't all identical, as we have to set GIT_FORCE_THREADS for the --threads=1 test (which measures the overhead of starting a single worker thread versus the "0" case of using the main thread). To keep the loop simple, we'll keep the "0" case out of it, and set GIT_FORCE_THREADS=1 for all of the other cases (it's a noop for all but the "1" case, since numbers higher than 1 would always need threads). Note also that we could skip running "test-tool" if PERF_EXTRA isn't set. However, there's some small value in knowing the number of threads, so that we can mark each test as skipped in the output. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
4 years ago |
Jeff King | 47274251a4 |
p5302: disable thread-count parameter tests by default
The primary function of the perf suite is to detect regressions (or improvements) between versions of Git. The only numbers we show a direct comparison for are timings between the same test run on two different versions. However, it can sometimes be used to collect other information. For instance, p5302 runs the same index-pack operation with different thread counts. The output doesn't directly compare these, but anybody interested in working on index-pack can manually compare the results. For a normal regression run of the full perf-suite, though, this incurs a significant cost to generate numbers nobody will actually look at; about 25% of the total time of the test suite is spent in p5302. And the low-thread-count runs are the most expensive part of it, since they're (unsurprisingly) not using as many threads. Let's skip these tests by default, but make it possible for people working on index-pack to still run them by setting an environment variable. Rather than make this specific to p5302, let's introduce a generic mechanism. This makes it possible to run the full suite with every possible test if somebody really wants to burn some CPU. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
4 years ago |
Patrick Steinhardt | 6754159767 |
refs: implement reference transaction hook
The low-level reference transactions used to update references are currently completely opaque to the user. While certainly desirable in most usecases, there are some which might want to hook into the transaction to observe all queued reference updates as well as observing the abortion or commit of a prepared transaction. One such usecase would be to have a set of replicas of a given Git repository, where we perform Git operations on all of the repositories at once and expect the outcome to be the same in all of them. While there exist hooks already for a certain subset of Git commands that could be used to implement a voting mechanism for this, many others currently don't have any mechanism for this. The above scenario is the motivation for the new "reference-transaction" hook that reaches directly into Git's reference transaction mechanism. The hook receives as parameter the current state the transaction was moved to ("prepared", "committed" or "aborted") and gets via its standard input all queued reference updates. While the exit code gets ignored in the "committed" and "aborted" states, a non-zero exit code in the "prepared" state will cause the transaction to be aborted prematurely. Given the usecase described above, a voting mechanism can now be implemented via this hook: as soon as it gets called, it will take all of stdin and use it to cast a vote to a central service. When all replicas of the repository agree, the hook will exit with zero, otherwise it will abort the transaction by returning non-zero. The most important upside is that this will catch _all_ commands writing references at once, allowing to implement strong consistency for reference updates via a single mechanism. In order to test the impact on the case where we don't have any "reference-transaction" hook installed in the repository, this commit introduce two new performance tests for git-update-refs(1). Run against an empty repository, it produces the following results: Test origin/master HEAD -------------------------------------------------------------------- 1400.2: update-ref 2.70(2.10+0.71) 2.71(2.10+0.73) +0.4% 1400.3: update-ref --stdin 0.21(0.09+0.11) 0.21(0.07+0.14) +0.0% The performance test p1400.2 creates, updates and deletes a branch a thousand times, thus averaging runtime of git-update-refs over 3000 invocations. p1400.3 instead calls `git-update-refs --stdin` three times and queues a thousand creations, updates and deletes respectively. As expected, p1400.3 consistently shows no noticeable impact, as for each batch of updates there's a single call to access(3P) for the negative hook lookup. On the other hand, for p1400.2, one can see an impact caused by this patchset. But doing five runs of the performance tests where each one was run with GIT_PERF_REPEAT_COUNT=10, the overhead ranged from -1.5% to +1.1%. These inconsistent performance numbers can be explained by the overhead of spawning 3000 processes. This shows that the overhead of assembling the hook path and executing access(3P) once to check if it's there is mostly outweighed by the operating system's overhead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 9639474b6d |
pack-bitmap: pass object filter to fill-in traversal
Sometimes a bitmap traversal still has to walk some commits manually, because those commits aren't included in the bitmap packfile (e.g., due to a push or commit since the last full repack). If we're given an object filter, we don't pass it down to this traversal. It's not necessary for correctness because the bitmap code has its own filters to post-process the bitmap result (which it must, to filter out the objects that _are_ mentioned in the bitmapped packfile). And with blob filters, there was no performance reason to pass along those filters, either. The fill-in traversal could omit them from the result, but it wouldn't save us any time to do so, since we'd still have to walk each tree entry to see if it's a blob or not. But now that we support tree filters, there's opportunity for savings. A tree:depth=0 filter means we can avoid accessing trees entirely, since we know we won't them (or any of the subtrees or blobs they point to). The new test in p5310 shows this off (the "partial bitmap" state is one where HEAD~100 and its ancestors are all in a bitmapped pack, but HEAD~100..HEAD are not). Here are the results (run against linux.git): Test HEAD^ HEAD ------------------------------------------------------------------------------------------------- [...] 5310.16: rev-list with tree filter (partial bitmap) 0.19(0.17+0.02) 0.03(0.02+0.01) -84.2% The absolute number of savings isn't _huge_, but keep in mind that we only omitted 100 first-parent links (in the version of linux.git here, that's 894 actual commits). In a more pathological case, we might have a much larger proportion of non-bitmapped commits. I didn't bother creating such a case in the perf script because the setup is expensive, and this is plenty to show the savings as a percentage. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Taylor Blau | b0a8d4820b |
pack-bitmap.c: support 'tree:0' filtering
In the previous patch, we made it easy to define other filters that exclude all objects of a certain type. Use that in order to implement bitmap-level filtering for the '--filter=tree:<n>' filter when 'n' is equal to 0. The general case is not helped by bitmaps, since for values of 'n > 0', the object filtering machinery requires a full-blown tree traversal in order to determine the depth of a given tree. Caching this is non-obvious, too, since the same tree object can have a different depth depending on the context (e.g., a tree was moved up in the directory hierarchy between two commits). But, the 'n = 0' case can be helped, and this patch does so. Running p5310.11 in this tree and on master with the kernel, we can see that this case is helped substantially: Test master this tree -------------------------------------------------------------------------------- 5310.11: rev-list count with tree:0 10.68(10.39+0.27) 0.06(0.04+0.01) -99.4% Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | d8410a816b |
fast-import: replace custom hash with hashmap.c
We use a custom hash in fast-import to store the set of objects we've imported so far. It has a fixed set of 2^16 buckets and chains any collisions with a linked list. As the number of objects grows larger than that, the load factor increases and we degrade to O(n) lookups and O(n^2) insertions. We can scale better by using our hashmap.c implementation, which will resize the bucket count as we grow. This does incur an extra memory cost of 8 bytes per object, as hashmap stores the integer hash value for each entry in its hashmap_entry struct (which we really don't care about here, because we're just reusing the embedded object hash). But I think the numbers below justify this (and our per-object memory cost is already much higher). I also looked at using khash, but it seemed to perform slightly worse than hashmap at all sizes, and worse even than the existing code for small sizes. It's also awkward to use here, because we want to look up a "struct object_entry" from a "struct object_id", and it doesn't handle mismatched keys as well. Making a mapping of object_id to object_entry would be more natural, but that would require pulling the embedded oid out of the object_entry or incurring an extra 32 bytes per object. In a synthetic test creating as many cheap, tiny objects as possible perl -e ' my $bits = shift; my $nr = 2**$bits; for (my $i = 0; $i < $nr; $i++) { print "blob\n"; print "data 4\n"; print pack("N", $i); } ' $bits | git fast-import I got these results: nr_objects master khash hashmap 2^20 0m4.317s 0m5.109s 0m3.890s 2^21 0m10.204s 0m9.702s 0m7.933s 2^22 0m27.159s 0m17.911s 0m16.751s 2^23 1m19.038s 0m35.080s 0m31.963s 2^24 4m18.766s 1m10.233s 1m6.793s which points to hashmap as the winner. We didn't have any perf tests for fast-export or fast-import, so I added one as a more real-world case. It uses an export without blobs since that's significantly cheaper than a full one, but still is an interesting case people might use (e.g., for rewriting history). It will emphasize this change in some ways (as a percentage we spend more time making objects and less shuffling blob bytes around) and less in others (the total object count is lower). Here are the results for linux.git: Test HEAD^ HEAD ---------------------------------------------------------------------------- 9300.1: export (no-blobs) 67.64(66.96+0.67) 67.81(67.06+0.75) +0.3% 9300.2: import (no-blobs) 284.04(283.34+0.69) 198.09(196.01+0.92) -30.3% It only has ~5.2M commits and trees, so this is a larger effect than I expected (the 2^23 case above only improved by 50s or so, but here we gained almost 90s). This is probably due to actually performing more object lookups in a real import with trees and commits, as opposed to just dumping a bunch of blobs into a pack. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 14d277879c |
p5310: stop timing non-bitmap pack-to-disk
Commit |
5 years ago |
Jeff King | 3ab3185f99 |
pack-objects: support filters with bitmaps
Just as rev-list recently learned to combine filters and bitmaps, let's do the same for pack-objects. The infrastructure is all there; we just need to pass along our filter options, and the pack-bitmap code will decide to use bitmaps or not. This unsurprisingly makes things faster for partial clones of large repositories (here we're cloning linux.git): Test HEAD^ HEAD ------------------------------------------------------------------------------ 5310.11: simulated partial clone 38.94(37.28+5.87) 11.06(11.27+4.07) -71.6% Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 84243da129 |
pack-bitmap: implement BLOB_LIMIT filtering
Just as the previous commit implemented BLOB_NONE, we can support BLOB_LIMIT filters by looking at the sizes of any blobs in the result and unsetting their bits as appropriate. This is slightly more expensive than BLOB_NONE, but still produces a noticeable speedup (these results are on git.git): Test HEAD~2 HEAD ------------------------------------------------------------------------------------ 5310.9: rev-list count with blob:none 1.80(1.77+0.02) 0.22(0.20+0.02) -87.8% 5310.10: rev-list count with blob:limit=1k 1.99(1.96+0.03) 0.29(0.25+0.03) -85.4% The implementation is similar to the BLOB_NONE one, with the exception that we have to go object-by-object while walking the blob-type bitmap (since we can't mask out the matches, but must look up the size individually for each blob). The trick with using ctz64() is taken from show_objects_for_type(), which likewise needs to find individual bits (but wants to quickly skip over big chunks without blobs). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 4f3bd5606a |
pack-bitmap: implement BLOB_NONE filtering
We can easily support BLOB_NONE filters with bitmaps. Since we know the types of all of the objects, we just need to clear the result bits of any blobs. Note two subtleties in the implementation (which I also called out in comments): - we have to include any blobs that were specifically asked for (and not reached through graph traversal) to match the non-bitmap version - we have to handle in-pack and "ext_index" objects separately. Arguably prepare_bitmap_walk() could be adding these ext_index objects to the type bitmaps. But it doesn't for now, so let's match the rest of the bitmap code here (it probably wouldn't be an efficiency improvement to do so since the cost of extending those bitmaps is about the same as our loop here, but it might make the code a bit simpler). Here are perf results for the new test on git.git: Test HEAD^ HEAD -------------------------------------------------------------------------------- 5310.9: rev-list count with blob:none 1.67(1.62+0.05) 0.22(0.21+0.02) -86.8% Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 4eb707ebd6 |
rev-list: allow commit-only bitmap traversals
Ever since we added reachability bitmap support, we've been able to use it with rev-list to get the full list of objects, like: git rev-list --objects --use-bitmap-index --all But you can't do so without --objects, since we weren't ready to just show the commits. However, the internals of the bitmap code are mostly ready for this: they avoid opening up trees when walking to fill in the bitmaps. We just need to actually pass in the rev_info to traverse_bitmap_commit_list() so it knows which types to bother triggering our callback for. For completeness, the perf test now covers both the existing --objects case, as well as the new commits-only behavior (the objects one got way faster when we introduced bitmaps, but obviously isn't improved now). Here are numbers for linux.git: Test HEAD^ HEAD ------------------------------------------------------------------------ 5310.7: rev-list (commits) 8.29(8.10+0.19) 1.76(1.72+0.04) -78.8% 5310.8: rev-list (objects) 8.06(7.94+0.12) 8.14(7.94+0.13) +1.0% That run was cheating a little, as I didn't have any commit-graph in the repository, and we'd built it by default these days when running git-gc. Here are numbers with a commit-graph: Test HEAD^ HEAD ------------------------------------------------------------------------ 5310.7: rev-list (commits) 0.70(0.58+0.12) 0.51(0.46+0.04) -27.1% 5310.8: rev-list (objects) 6.20(6.09+0.10) 6.27(6.16+0.11) +1.1% Still an improvement, but a lot less impressive. We could have the perf script remove any commit-graph to show the out-sized effect, but it probably makes sense to leave it in what would be a more typical setup. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Colin Stolley | ec48540fe8 |
packfile.c: speed up loading lots of packfiles
When loading packfiles on start-up, we traverse the internal packfile list once per file to avoid reloading packfiles that have already been loaded. This check runs in quadratic time, so for poorly maintained repos with a large number of packfiles, it can be pretty slow. Add a hashmap containing the packfile names as we load them so that the average runtime cost of checking for already-loaded packs becomes constant. Add a perf test to p5303 to show speed-up. The existing p5303 test runtimes are dominated by other factors and do not show an appreciable speed-up. The new test in p5303 clearly exposes a speed-up in bad cases. In this test we create 10,000 packfiles and measure the start-up time of git rev-parse, which does little else besides load in the packs. Here are the numbers for the new p5303 test: Test HEAD^ HEAD --------------------------------------------------------------------- 5303.12: load 10,000 packs 1.03(0.92+0.10) 0.12(0.02+0.09) -88.3% Signed-off-by: Colin Stolley <cstolley@runbox.com> Helped-by: Jeff King <peff@peff.net> [jc: squashed the change to call hashmap in install_packed_git() by peff] Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 528d9e6d01 |
t/perf: don't depend on Git.pm
The perf suite's aggregate.perl depends on Git.pm, which is a mild annoyance if you've built git with NO_PERL. It turns out that the only thing we use it for is a single call of the command_oneline() helper. We can just replace this with backticks or similar. Annoyingly, perl has no backtick equivalent that avoids a shell eval, which means our $arg would require quoting. This probably doesn't matter for our purposes, but it's better to be safe and model good style. So we'll just provide a short helper around open(), which takes its arguments as a list. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | b8dcc45387 |
perf-lib: use a single filename for all measurement types
The perf tests write files recording the results of tests. These
results are later aggregated by 'aggregate.perl'. If the tests are run
multiple times, those results are overwritten by the new results. This
works just fine as long as there are only perf tests measuring the
times, whose results are stored in "$base".times files.
However
|
5 years ago |
Jeff King | f66e0401ab |
pack-objects: avoid pointless oe_map_new_pack() calls
This patch fixes an extreme slowdown in pack-objects when you have more
than 1023 packs. See below for numbers.
Since
|
5 years ago |
Elijah Newren | 96c0caf5e3 |
Fix spelling errors in messages shown to users
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 362f8b280c |
t/perf: rename duplicate-numbered test script
There are two perf scripts numbered p5600, but with otherwise different names ("clone-reference" versus "partial-clone"). We store timing results in files named after the whole script, so internally we don't get confused between the two. But "aggregate.perl" just prints the test number for each result, giving multiple entries for "5600.3". It also makes it impossible to skip one test but not the other with GIT_SKIP_TESTS. Let's renumber the one that appeared later (by date -- the source of the problem is that the two were developed on independent branches). For the non-perf test suite, our test-lint rule would have complained about this when the two were merged, but t/perf never learned that trick. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
5 years ago |
Jeff King | 39b44ba771 |
check_everything_connected: assume alternate ref tips are valid
When we receive a remote ref update to sha1 "X", we want to check that we have all of the objects needed by "X". We can assume that our repository is not currently corrupted, and therefore if we have a ref pointing at "Y", we have all of its objects. So we can stop our traversal from "X" as soon as we hit "Y". If we make the same non-corruption assumption about any repositories we use to store alternates, then we can also use their ref tips to shorten the traversal. This is especially useful when cloning with "--reference", as we otherwise do not have any local refs to check against, and have to traverse the whole history, even though the other side may have sent us few or no objects. Here are results for the included perf test (which shows off more or less the maximal savings, getting one new commit and sharing the whole history): Test HEAD^ HEAD -------------------------------------------------------------------- [on git.git] 5600.3: clone --reference 2.94(2.86+0.08) 0.09(0.08+0.01) -96.9% [on linux.git] 5600.3: clone --reference 45.74(45.34+0.41) 0.36(0.30+0.08) -99.2% Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Ævar Arnfjörð Bjarmason | 82b7eb231d |
perf-lib.sh: forbid the use of GIT_TEST_INSTALLED
As noted in preceding commits setting GIT_TEST_INSTALLED has never
been supported or documented, and as noted in an earlier t/perf/README
change to the extent that it's been documented nobody's notices that
the example hasn't worked since
|
6 years ago |
Ævar Arnfjörð Bjarmason | fab80ee79d |
perf tests: add "bindir" prefix to git tree test results
Change the output file names in test-results/ to be "test-results/bindir_<munged dir>" rather than just "test-results/<munged dir>". This is for consistency with the "build_" directories we have for built revisions, i.e. "test-results/build_<SHA-1>". There's no user-visible functional changes here, it just makes it easier to see at a glance what "test-results" files are of what "type" as they're all explicitly grouped together now, and to grep this code to find both the run_dirs_helper() implementation and its corresponding aggregate.perl code. Note that we already guarantee that the rest of the PERF_RESULTS_PREFIX is an absolute path, and since it'll start with e.g. "/" which we munge to "_" we'll up with a readable string like "bindir_home_avar[...]". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> |
6 years ago |
Ævar Arnfjörð Bjarmason | df0f502195 |
perf-lib.sh: remove GIT_TEST_INSTALLED from perf-lib.sh
Follow-up my preceding change which fixed the immediate "./run
<revisions>" regression in
|
6 years ago |
Ævar Arnfjörð Bjarmason | 90e38154ee |
perf-lib.sh: make "./run <revisions>" use the correct gits
Fix a really bad regression in |
6 years ago |
Ævar Arnfjörð Bjarmason | c43b7e6089 |
perf aggregate: remove GIT_TEST_INSTALLED from --codespeed
Remove the setting of the "environment" from the --codespeed output. I don't think this is useful, and it helps with a later refactoring where we GIT_TEST_INSTALLED stop munging/reading GIT_TEST_INSTALLED in the perf tests in so many places. This was added in |
6 years ago |
Ævar Arnfjörð Bjarmason | 9bb81452ff |
perf README: correct docs for 3c8f12c96c regression
Since |
6 years ago |
Jeff King | 1bb10d4f7c |
t/perf: add perf script for partial clones
We don't cover the partial clone feature at all in t/perf. Let's at least run a few basic tests so that we'll notice any regressions. We'll do a no-blob clone, and split it into two parts: the actual object transfer, and the subsequent checkout (which will of course require another transfer to get the blobs). That will help us more clearly assess the performance of each. There are obviously a lot more possibilities besides just a no-blob partial clone, but this should serve as a canary that alerts us to any generic slow-downs (and we can add more tests later for cases that aren't exercised here). There are a few non-ideal things here that make this not an entirely accurate test, but are probably OK for our purposes: 1. We have to do some extra prep/cleanup work inside the timing tests, since they impact the on-disk state and the perf harness may run each one multiple times. In practice this is probably OK, since these bits should be much less expensive than the operations we are measuring. 2. The clone time is likely to be dominated by the server's object enumeration. In the real world, a repo large enough to drive people to partial clones is likely to have reachability bitmaps enabled. And in the opposite direction, our object transfer is happening at the speed of a local pipe, whereas in the real world it would bottle-neck on the network. So any percentage speedups should be taken with a grain of salt. But hopefully any regressions will produce enough of an effect to be noticeable. This script also demonstrates the recent improvement from |
6 years ago |
Jeff King | f2e875d6df |
t/perf: depend on perl JSON only when using --codespeed
Commit
|
6 years ago |
Jeff King | 775c71e16d |
p5302: create the repo in each index-pack test
The p5302 script runs "index-pack --stdin" in each timing test. It does two things to try to get good timings: 1. we do the repo creation in a separate (non-timed) setup test, so that our timing is purely the index-pack run 2. we use a separate repo for each test; this is important because the presence of existing objects in the repo influences the result (because we'll end up doing collision checks against them) But this forgets one thing: we generally run each timed test multiple times to reduce the impact of noise. Which means that repeats of each test after the first will be subject to the collision slowdown from point 2, and we'll generally just end up taking the first time anyway. Instead, let's create the repo in the test (effectively undoing point 1). That does add a constant amount of extra work to each iteration, but it's quite small compared to the actual effects we're interested in measuring. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Jeff King | 8320b1dbe7 |
revision: use a prio_queue to hold rewritten parents
This patch fixes a quadratic list insertion in rewrite_one() when
pathspec limiting is combined with --parents. What happens is something
like this:
1. We see that some commit X touches the path, so we try to rewrite
its parents.
2. rewrite_one() loops forever, rewriting parents, until it finds a
relevant parent (or hits the root and decides there are none). The
heavy lifting is done by process_parent(), which uses
try_to_simplify_commit() to drop parents.
3. process_parent() puts any intermediate parents into the
&revs->commits list, inserting by commit date as usual.
So if commit X is recent, and then there's a large chunk of history that
doesn't touch the path, we may add a lot of commits to &revs->commits.
And insertion by commit date is O(n) in the worst case, making the whole
thing quadratic.
We tried to deal with this long ago in
|
6 years ago |
Jeff King | 0baf78e7bc |
perf-lib.sh: rely on test-lib.sh for --tee handling
Since its inception, the perf-lib.sh script has manually handled the "--tee" option (and other options which imply it, like "--valgrind") with a cut-and-pasted block from test-lib.sh. That block has grown stale over the years, and has at least three problems: 1. It uses $SHELL to re-exec the script, whereas the version in test-lib.sh learned to use $TEST_SHELL_PATH. 2. It does an ad-hoc search of the "$*" string, whereas test-lib.sh learned to carefully parse the arguments left to right. 3. It never learned about --verbose-log (which also implies --tee), so it would not trigger for that option. This last one was especially annoying, because t/perf/run uses the GIT_TEST_OPTS from your config.mak to run the perf scripts. So if you've set, say, "-x --verbose-log" there, it will be passed as part of most perf runs. And while this script doesn't recognize the option, the test-lib.sh that we source _does_, and the behavior ends up being much more annoying: - as the comment at the top of the block says, we have to run this tee code early, before we start munging variables (it says GIT_BUILD_DIR, but the problematic variable is actually GIT_TEST_INSTALLED). - since we don't recognize --verbose-log, we don't trigger the block. We go on to munge GIT_TEST_INSTALLED, converting it from a relative to an absolute path. - then we source test-lib.sh, which _does_ recognize --verbose-log. It re-execs the script, which runs again. But this time with an absolute version of GIT_TEST_INSTALLED. - As a result, we copy the absolute version of GIT_TEST_INSTALLED into perf_results_prefix. Instead of writing our results to the expected "test-results/build_1234abcd.p1234-whatever.times", we instead write them to "test-results/_full_path_to_repo_t_perf_build_1234...". The aggregate.perl script doesn't expect this, and so it prints "<missing>" for each result (even though it spent considerable time running the tests!). We can solve all of these in one blow by just deleting our custom handling, and relying on the inclusion of test-lib.sh to handle --tee, --verbose-log, etc. There's one catch, though. We want to handle GIT_TEST_INSTALLED after we've included test-lib.sh, since we want it un-munged in the re-exec'd version of the script. But if we want to convert it from a relative to an absolute path, we must do so before we load test-lib.sh, since it will change our working directory. So we compute the absolute directory first, store it away, then include test-lib.sh, and finally assign to GIT_TEST_INSTALLED as appropriate. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Jeff King | d4316604f8 |
pack-objects: default to writing bitmap hash-cache
Enabling pack.writebitmaphashcache should always be a performance win. It costs only 4 bytes per object on disk, and the timings in |
6 years ago |
Jeff King | fde67d6896 |
prune: use bitmaps for reachability traversal
Pruning generally has to traverse the whole commit graph in order to see which objects are reachable. This is the exact problem that reachability bitmaps were meant to solve, so let's use them (if they're available, of course). Here are timings on git.git: Test HEAD^ HEAD ------------------------------------------------------------------------ 5304.6: prune with bitmaps 3.65(3.56+0.09) 1.01(0.92+0.08) -72.3% And on linux.git: Test HEAD^ HEAD -------------------------------------------------------------------------- 5304.6: prune with bitmaps 35.05(34.79+0.23) 3.00(2.78+0.21) -91.4% The tests show a pretty optimal case, as we'll have just repacked and should have pretty good coverage of all refs with our bitmaps. But that's actually pretty realistic: normally prune is run via "gc" right after repacking. A few notes on the implementation: - the change is actually in reachable.c, so it would improve reachability traversals by "reflog expire --stale-fix", as well. Those aren't performed regularly, though (a normal "git gc" doesn't use --stale-fix), so they're not really worth measuring. There's a low chance of regressing that caller, since the use of bitmaps is totally transparent from the caller's perspective. - The bitmap case could actually get away without creating a "struct object", and instead the caller could just look up each object id in the bitmap result. However, this would be a marginal improvement in runtime, and it would make the callers much more complicated. They'd have to handle both the bitmap and non-bitmap cases separately, and in the case of git-prune, we'd also have to tweak prune_shallow(), which relies on our SEEN flags. - Because we do create real object structs, we go through a few contortions to create ones of the right type. This isn't strictly necessary (lookup_unknown_object() would suffice), but it's more memory efficient to use the correct types, since we already know them. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Jeff King | d55a30bb1d |
prune: lazily perform reachability traversal
The general strategy of "git prune" is to do a full reachability walk, then for each loose object see if we found it in our walk. But if we don't have any loose objects, we don't need to do the expensive walk in the first place. This patch postpones that walk until the first time we need to see its results. Note that this is really a specific case of a more general optimization, which is that we could traverse only far enough to find the object under consideration (i.e., stop the traversal when we find it, then pick up again when asked about the next object, etc). That could save us in some instances from having to do a full walk. But it's actually a bit tricky to do with our traversal code, and you'd need to do a full walk anyway if you have even a single unreachable object (which you generally do, if any objects are actually left after running git-repack). So in practice this lazy-load of the full walk catches one easy but common case (i.e., you've just repacked via git-gc, and there's nothing unreachable). The perf script is fairly contrived, but it does show off the improvement: Test HEAD^ HEAD ------------------------------------------------------------------------- 5304.4: prune with no objects 3.66(3.60+0.05) 0.00(0.00+0.00) -100.0% and would let us know if we accidentally regress this optimization. Note also that we need to take special care with prune_shallow(), which relies on us having performed the traversal. So this optimization can only kick in for a non-shallow repository. Since this is easy to get wrong and is not covered by existing tests, let's add an extra test to t5304 that covers this case explicitly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
SZEDER Gábor | 165293af3c |
tests: send "bug in the test script" errors to the script's stderr
Some of the functions in our test library check that they were invoked properly with conditions like this: test "$#" = 2 || error "bug in the test script: not 2 parameters to test-expect-success" If this particular condition is triggered, then 'error' will abort the whole test script with a bold red error message [1] right away. However, under certain circumstances the test script will be aborted completely silently, namely if: - a similar condition in a test helper function like 'test_line_count' is triggered, - which is invoked from the test script's "main" shell [2], - and the test script is run manually (i.e. './t1234-foo.sh' as opposed to 'make t1234-foo.sh' or 'make test') [3] - and without the '--verbose' option, because the error message is printed from within 'test_eval_', where standard output is redirected either to /dev/null or to a log file. The only indication that something is wrong is that not all tests in the script are executed and at the end of the test script's output there is no "# passed all N tests" message, which are subtle and can easily go unnoticed, as I had to experience myself. Send these "bug in the test script" error messages directly to the test scripts standard error and thus to the terminal, so those bugs will be much harder to overlook. Instead of updating all ~20 such 'error' calls with a redirection, let's add a BUG() function to 'test-lib.sh', wrapping an 'error' call with the proper redirection and also including the common prefix of those error messages, and convert all those call sites [4] to use this new BUG() function instead. [1] That particular error message from 'test_expect_success' is printed in color only when running with or without '--verbose'; with '--tee' or '--verbose-log' the error is printed without color, but it is printed to the terminal nonetheless. [2] If such a condition is triggered in a subshell of a test, then 'error' won't be able to abort the whole test script, but only the subshell, which in turn causes the test to fail in the usual way, indicating loudly and clearly that something is wrong. [3] Well, 'error' aborts the test script the same way when run manually or by 'make' or 'prove', but both 'make' and 'prove' pay attention to the test script's exit status, and even a silently aborted test script would then trigger those tools' usual noticable error messages. [4] Strictly speaking, not all those 'error' calls need that redirection to send their output to the terminal, see e.g. 'test_expect_success' in the opening example, but I think it's better to be consistent. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Alban Gruin | 5aa24d71d8 |
p3400: replace calls to `git checkout -b' by `git checkout -B'
p3400 makes a copy of the current repository to test git-rebase performance, and creates new branches in the copy with `git checkout -b'. If the original repository has branches with the same name as the script is trying to create, this operation will fail. This replaces these calls by `git checkout -B' to force the creation and update of these branches. Signed-off-by: Alban Gruin <alban.gruin@gmail.com> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
René Scharfe | 01e0d545ab |
fsck: add a performance test for skipList
Create a performance test to see how the skipList implementation performs. First we setup N bad commits, then we see how progressively working our way up to 0..N in increments of 10x does. I.e. the needle(s) in the haystack get progressively more numerous. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Ævar Arnfjörð Bjarmason | 6cb173b5b6 |
fsck: add a performance test
Add a plain performance test for "fsck". This test will not be used to / referred to in any upcoming commit of mine in this series, but having a simple test for fsck performance is valuable, so let's add it while we're at it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Jeff King | 198b349da8 |
t/perf: add perf tests for fetches from a bitmapped server
A server with bitmapped packs can serve a clone very quickly. However, fetches are not necessarily made any faster, because we spend a lot less time in object traversal (which is what bitmaps help with) and more time finding deltas (because we may have to throw out on-disk deltas if the client does not have the base). As a first step to making this faster, this patch introduces a new perf script to measure fetches into a repo of various ages from a fully-bitmapped server. We separately measure the work done by the server (in pack-objects) and that done by the client (in index-pack). Furthermore, we measure the size of the resulting pack. Breaking it down like this (instead of just doing a regular "git fetch") lets us see how much each side benefits from any changes. And since we know the pack size, if we estimate the network speed, then one could calculate a complete wall-clock time for the operation (though the script does not do this automatically). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Jeff King | 22bec79d1a |
t/perf: add infrastructure for measuring sizes
The main objective of scripts in the perf framework is to run "test_perf", which measures the time it takes to run some operation. However, it can also be interesting to see the change in the output size of certain operations. This patch introduces test_size, which records a single numeric output from the test and shows it in the aggregated output (with pretty printing and relative size comparison). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Jeff King | 5a924a62bb |
t/perf: factor out percent calculations
This will let us reuse the code when we add new values to aggregate besides times. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Jeff King | 968e77a5f8 |
t/perf: factor boilerplate out of test_perf
About half of test_perf() is boilerplate preparing to run _any_ test, and the other half is specifically running a timing test. Let's split it into two functions, so that we can reuse the boilerplate in future commits. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
6 years ago |
Christian Couder | d9ea451ab6 |
perf/bisect_run_script: disable codespeed
When bisecting a performance regression using a config file, `./bisect_regression --config my_perf.conf` for example, the config file can contain Codespeed configuration which would instruct the 'aggregate.perl' script called by the 'run' script to output results in the Codespeed format and maybe to try to send this output to a Codespeed server. This is unfortunate because the 'bisect_run_script' relies on the regular output from 'aggregate.perl' to mesure performance, so let's disable Codespeed output and sending results to a Codespeed server. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Christian Couder | 38368cba26 |
perf/aggregate: use Getopt::Long for option parsing
When passing an option '--foo' that it does not recognize, the aggregate.perl script should die with an helpful error message like: Unknown option: foo ./aggregate.perl [options] [--] [<dir_or_rev>...] [--] \ [<test_script>...] > Options: --codespeed * Format output for Codespeed --reponame <str> * Send given reponame to codespeed --sort-by <str> * Sort output (only "regression" \ criteria is supported) rather than: fatal: Needed a single revision rev-parse --verify --foo: command returned error: 128 To implement that let's use Getopt::Long for option parsing instead of the current manual and sloppy parsing. This should save some code and make option parsing simpler, tighter and safer. This will avoid something like 'foo--sort-by=regression' to be handled as if '--sort-by=regression' had been used, for example. As Getopt::Long eats '--' at the end of options, this changes a bit the way '--' is handled as we can now have '--' both after the options and before the scripts. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Christian Couder | 297e685cba |
t/perf: add scripts to bisect performance regressions
The new bisect_regression script can be used to automatically bisect performance regressions. It will pass the new bisect_run_script to `git bisect run`. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Christian Couder | 8796b307ea |
perf/run: add --subsection option
This new option makes it possible to run perf tests as defined in only one subsection of a config file. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Christian Couder | 2e3efd0613 |
perf/aggregate: add --sort-by=regression option
One of the most interesting thing one can be interested in when looking at performance test results is possible performance regressions. This new option makes it easy to spot such possible regressions. This new option is named '--sort-by=regression' to make it possible and easy to add other ways to sort the results, like for example '--sort-by=utime'. If we would like to sort according to how much the stime regressed we could also add a new option called '--sort-by=regression:stime'. Then '--sort-by=regression' could become a synonym for '--sort-by=regression:rtime'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Christian Couder | c94b6ac50f |
perf/aggregate: add display_dir()
This new helper function will be reused in a subsequent commit. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Nguyễn Thái Ngọc Duy | c81f843d09 |
t/helper: merge test-write-cache into test-tool
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Nguyễn Thái Ngọc Duy | c932a5ff28 |
t/helper: merge test-string-list into test-tool
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |
Nguyễn Thái Ngọc Duy | 5fbe600cb5 |
t/helper: merge test-read-cache into test-tool
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> |
7 years ago |