Go to file
Taylor Blau dab60934e3 pack-bitmap: pass `bitmapped_pack` struct to pack-reuse functions
When trying to assemble a pack with bitmaps using `--use-bitmap-index`,
`pack-objects` asks the pack-bitmap machinery for a bitmap which
indicates the set of objects we can "reuse" verbatim from on-disk.

This set is roughly comprised of: a prefix of objects in the bitmapped
pack (or preferred pack, in the case of a multi-pack reachability
bitmap), plus any other objects not included in the prefix, excluding
any deltas whose base we are not sending in the resulting pack.

The pack-bitmap machinery is responsible for computing this bitmap, and
does so with the following functions:

  - reuse_partial_packfile_from_bitmap()
  - try_partial_reuse()

In the existing implementation, the first function is responsible for
(a) marking the prefix of objects in the reusable pack, and then (b)
calling try_partial_reuse() on any remaining objects to ensure that they
are also reusable (and removing them from the bitmapped set if they are
not).

Likewise, the `try_partial_reuse()` function is responsible for checking
whether an isolated object (that is, an object from the bitmapped
pack/preferred pack not contained in the prefix from earlier) may be
reused, i.e. that it isn't a delta of an object that we are not sending
in the resulting pack.

These functions are based on two core assumptions, which we will unwind
in this and the following commits:

  1. There is only a single pack from the bitmap which is eligible for
     verbatim pack-reuse. For single-pack bitmaps, this is trivially the
     bitmapped pack. For multi-pack bitmaps, this is (currently) the
     MIDX's preferred pack.

  2. The pack eligible for reuse has its first object in bit position 0,
     and all objects from that pack follow in pack-order from that first
     bit position.

In order to perform verbatim pack reuse over multiple packs, we must
unwind these two assumptions. Most notably, in order to reuse bits from
a given packfile, we need to know the first bit position occupied by
an object form that packfile. To propagate this information around, pass
a `struct bitmapped_pack *` anywhere we previously passed a `struct
packed_git *`, since the former contains the bitmap position we're
interested in (as well as a pointer to the latter).

As an additional step, factor out a sub-routine from the main
`reuse_partial_packfile_from_bitmap()` function, called
`reuse_partial_packfile_from_bitmap_1()`. This new function will be
responsible for figuring out which objects may be reused from a single
pack, and the existing function will dispatch multiple calls to its new
helper function for each reusable pack.

Consequently, `reuse_partial_packfile_from_bitmap()` will now maintain
an array of reusable packs instead of a single such pack. We currently
expect that array to have only a single element, so this awkward state
is short-lived. It will serve as useful scaffolding in subsequent
commits as we begin to work towards enabling multi-pack reuse.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-12-14 14:38:07 -08:00
.github ci: upgrade to using macos-13 2023-11-03 18:52:02 +09:00
Documentation midx: implement `BTMP` chunk 2023-12-14 14:38:07 -08:00
block-sha1
builtin pack-objects: free packing_data in more places 2023-12-14 14:38:07 -08:00
ci Merge branch 'js/ci-discard-prove-state' 2023-12-09 16:37:48 -08:00
compat
contrib Merge branch 'tz/send-email-negatable-options' 2023-12-09 16:37:51 -08:00
ewah
git-gui
gitk-git
gitweb perl: bump the required Perl version to 5.8.1 from 5.8.0 2023-11-17 07:26:32 +09:00
mergetools
negotiator
oss-fuzz
perl perl: bump the required Perl version to 5.8.1 from 5.8.0 2023-11-17 07:26:32 +09:00
po Merge branch 'l10n/zh-TW/2023-11-19' of github.com:l10n-tw/git-po 2023-11-20 07:57:09 +08:00
refs files-backend.c: avoid stat in 'loose_fill_ref_dir' 2023-10-09 15:53:14 -07:00
reftable
sha1
sha1collisiondetection@855827c583
sha1dc
sha256
t midx: implement `BTMP` chunk 2023-12-14 14:38:07 -08:00
templates
trace2
xdiff
.cirrus.yml ci: run unit tests in CI 2023-11-10 08:15:32 +09:00
.clang-format clang-format: fix typo in comment 2023-11-01 12:24:19 +09:00
.editorconfig
.gitattributes
.gitignore
.gitlab-ci.yml ci: add support for GitLab CI 2023-11-09 18:56:10 +09:00
.gitmodules
.mailmap mailmap: change primary address for Derrick Stolee 2023-10-12 10:59:36 -07:00
.tsan-suppressions
CODE_OF_CONDUCT.md
COPYING
GIT-VERSION-GEN Start the 2.44 cycle 2023-12-09 16:37:51 -08:00
INSTALL perl: bump the required Perl version to 5.8.1 from 5.8.0 2023-11-17 07:26:32 +09:00
LGPL-2.1
Makefile Merge branch 'ps/ban-a-or-o-operator-with-test' 2023-12-09 16:37:50 -08:00
README.md
RelNotes Start the 2.44 cycle 2023-12-09 16:37:51 -08:00
SECURITY.md
abspath.c
abspath.h
aclocal.m4
add-interactive.c
add-interactive.h
add-patch.c
advice.c
advice.h
alias.c
alias.h
alloc.c
alloc.h
apply.c
apply.h
archive-tar.c
archive-zip.c
archive.c
archive.h
attr.c Merge branch 'jc/attr-tree-config' 2023-10-30 07:09:55 +09:00
attr.h attr: add attr.tree for setting the treeish to read attributes from 2023-10-13 11:43:29 -07:00
banned.h
base85.c
base85.h
bisect.c
bisect.h
blame.c
blame.h
blob.c
blob.h
bloom.c commit-graph: detect out-of-order BIDX offsets 2023-10-09 15:55:02 -07:00
bloom.h
branch.c
branch.h
builtin.h
bulk-checkin.c Merge branch 'eb/limit-bulk-checkin-to-blobs' 2023-10-10 11:39:14 -07:00
bulk-checkin.h
bundle-uri.c
bundle-uri.h
bundle.c
bundle.h
cache-tree.c
cache-tree.h
cbtree.c
cbtree.h
chdir-notify.c
chdir-notify.h
check-builtins.sh
checkout.c
checkout.h
chunk-format.c chunk-format: drop pair_chunk_unsafe() 2023-10-09 15:55:02 -07:00
chunk-format.h chunk-format: drop pair_chunk_unsafe() 2023-10-09 15:55:02 -07:00
color.c
color.h
column.c
column.h
combine-diff.c
command-list.txt
commit-graph.c Merge branch 'jk/chunk-bounds-more' 2023-12-09 16:37:48 -08:00
commit-graph.h Merge branch 'ps/do-not-trust-commit-graph-blindly-for-existence' 2023-11-08 11:03:59 +09:00
commit-reach.c
commit-reach.h
commit-slab-decl.h
commit-slab-impl.h
commit-slab.h
commit.c commit: detect commits that exist in commit-graph but not in the ODB 2023-11-01 12:04:06 +09:00
commit.h
common-main.c
config.c Merge branch 'jc/attr-tree-config' 2023-10-30 07:09:55 +09:00
config.h
config.mak.dev
config.mak.in
config.mak.uname
configure.ac global: convert trivial usages of `test <expr> -a/-o <expr>` 2023-11-11 09:21:00 +09:00
connect.c
connect.h
connected.c
connected.h
convert.c
convert.h
copy.c
copy.h
credential.c
credential.h
csum-file.c
csum-file.h
ctype.c
daemon.c
date.c
date.h
decorate.c
decorate.h
delta-islands.c
delta-islands.h
delta.h
detect-compiler
diagnose.c dir.[ch]: add 'follow_symlink' arg to 'get_dtype' 2023-10-09 15:53:13 -07:00
diagnose.h
diff-delta.c
diff-lib.c Merge branch 'js/diff-cached-fsmonitor-fix' into maint-2.42 2023-11-02 16:53:19 +09:00
diff-merges.c diff-merges: introduce '--dd' option 2023-10-09 12:47:29 -07:00
diff-merges.h
diff-no-index.c Merge branch 'pw/diff-no-index-from-named-pipes' into maint-2.42 2023-11-02 16:53:18 +09:00
diff.c Merge branch 'jk/diff-result-code-cleanup' into maint-2.42 2023-11-02 16:53:16 +09:00
diff.h
diffcore-break.c
diffcore-delta.c
diffcore-order.c
diffcore-pickaxe.c
diffcore-rename.c
diffcore-rotate.c
diffcore.h
dir-iterator.c
dir-iterator.h
dir.c attr: enable attr pathspec magic for git-add and git-stash 2023-11-04 17:00:27 +09:00
dir.h dir.[ch]: add 'follow_symlink' arg to 'get_dtype' 2023-10-09 15:53:13 -07:00
editor.c
editor.h
entry.c
entry.h
environment.c max_tree_depth: lower it for MSVC to avoid stack overflows 2023-11-02 08:58:28 +09:00
environment.h
exec-cmd.c
exec-cmd.h
fetch-negotiator.c
fetch-negotiator.h
fetch-pack.c Merge branch 'ts/unpacklimit-config-fix' into maint-2.42 2023-11-02 16:53:16 +09:00
fetch-pack.h
fmt-merge-msg.c
fmt-merge-msg.h
fsck.c
fsck.h
fsmonitor--daemon.h
fsmonitor-ipc.c
fsmonitor-ipc.h
fsmonitor-ll.h
fsmonitor-path-utils.h
fsmonitor-settings.c
fsmonitor-settings.h
fsmonitor.c
fsmonitor.h
generate-cmdlist.sh
generate-configlist.sh
generate-hooklist.sh
gettext.c
gettext.h
git-archimport.perl perl: bump the required Perl version to 5.8.1 from 5.8.0 2023-11-17 07:26:32 +09:00
git-compat-util.h Merge branch 'mp/rebase-label-length-limit' into maint-2.42 2023-11-02 16:53:14 +09:00
git-curl-compat.h
git-cvsexportcommit.perl perl: bump the required Perl version to 5.8.1 from 5.8.0 2023-11-17 07:26:32 +09:00
git-cvsimport.perl perl: bump the required Perl version to 5.8.1 from 5.8.0 2023-11-17 07:26:32 +09:00
git-cvsserver.perl perl: bump the required Perl version to 5.8.1 from 5.8.0 2023-11-17 07:26:32 +09:00
git-difftool--helper.sh
git-filter-branch.sh
git-instaweb.sh
git-merge-octopus.sh
git-merge-one-file.sh
git-merge-resolve.sh
git-mergetool--lib.sh
git-mergetool.sh
git-p4.py git-p4 shouldn't attempt to store symlinks in LFS 2023-10-19 10:57:44 -07:00
git-quiltimport.sh
git-request-pull.sh
git-send-email.perl Merge branch 'tz/send-email-negatable-options' 2023-12-09 16:37:51 -08:00
git-sh-i18n.sh
git-sh-setup.sh
git-submodule.sh
git-svn.perl Merge branch 'tz/send-email-negatable-options' 2023-12-09 16:37:51 -08:00
git-web--browse.sh
git-zlib.c
git-zlib.h
git.c
git.rc
gpg-interface.c
gpg-interface.h
graph.c
graph.h
grep.c
grep.h
hash-ll.h
hash-lookup.c
hash-lookup.h
hash.h
hashmap.c
hashmap.h
help.c
help.h
hex-ll.c
hex-ll.h
hex.c
hex.h
hook.c
hook.h
http-backend.c
http-fetch.c
http-push.c
http-walker.c
http.c Merge branch 'jk/redact-h2h3-headers-fix' into maint-2.42 2023-11-02 16:53:19 +09:00
http.h
ident.c
ident.h
imap-send.c
iterator.h
json-writer.c
json-writer.h
khash.h
kwset.c
kwset.h
levenshtein.c
levenshtein.h
line-log.c
line-log.h
line-range.c
line-range.h
linear-assignment.c
linear-assignment.h
list-objects-filter-options.c
list-objects-filter-options.h
list-objects-filter.c
list-objects-filter.h
list-objects.c Merge branch 'tb/rev-list-unpacked-fix' 2023-11-08 15:04:42 +09:00
list-objects.h
list.h
lockfile.c
lockfile.h
log-tree.c
log-tree.h
ls-refs.c
ls-refs.h
mailinfo.c
mailinfo.h
mailmap.c
mailmap.h
match-trees.c
match-trees.h
mem-pool.c
mem-pool.h
merge-blobs.c
merge-blobs.h
merge-ll.c
merge-ll.h
merge-ort-wrappers.c
merge-ort-wrappers.h
merge-ort.c Merge branch 'wx/merge-ort-comment-typofix' into maint-2.42 2023-11-02 16:53:27 +09:00
merge-ort.h
merge-recursive.c merge: introduce {copy|clear}_merge_options() 2023-10-11 13:37:47 -07:00
merge-recursive.h merge: introduce {copy|clear}_merge_options() 2023-10-11 13:37:47 -07:00
merge.c
merge.h
mergesort.h
midx.c midx: implement `midx_locate_pack()` 2023-12-14 14:38:07 -08:00
midx.h midx: implement `midx_locate_pack()` 2023-12-14 14:38:07 -08:00
name-hash.c
name-hash.h
notes-cache.c
notes-cache.h
notes-merge.c
notes-merge.h
notes-utils.c
notes-utils.h
notes.c
notes.h
object-file.c
object-file.h
object-name.c
object-name.h
object-store-ll.h
object-store.h
object.c
object.h
oid-array.c
oid-array.h
oidmap.c
oidmap.h
oidset.c
oidset.h
oidtree.c
oidtree.h
pack-bitmap-write.c pack-bitmap-write: deep-clear the `bb_commit` slab 2023-12-14 14:38:07 -08:00
pack-bitmap.c pack-bitmap: pass `bitmapped_pack` struct to pack-reuse functions 2023-12-14 14:38:07 -08:00
pack-bitmap.h midx: implement `BTMP` chunk 2023-12-14 14:38:07 -08:00
pack-check.c
pack-mtimes.c
pack-mtimes.h
pack-objects.c pack-objects: free packing_data in more places 2023-12-14 14:38:07 -08:00
pack-objects.h pack-objects: free packing_data in more places 2023-12-14 14:38:07 -08:00
pack-revindex.c Merge branch 'jk/chunk-bounds' 2023-10-23 13:56:36 -07:00
pack-revindex.h
pack-write.c
pack.h
packfile.c
packfile.h
pager.c
pager.h
parallel-checkout.c
parallel-checkout.h
parse-options-cb.c
parse-options.c parse-options: make CMDMODE errors more precise 2023-10-29 09:15:18 +09:00
parse-options.h parse-options: make CMDMODE errors more precise 2023-10-29 09:15:18 +09:00
parse.c
parse.h
patch-delta.c
patch-ids.c
patch-ids.h
path.c
path.h
pathspec.c attr: enable attr pathspec magic for git-add and git-stash 2023-11-04 17:00:27 +09:00
pathspec.h
pkt-line.c
pkt-line.h
preload-index.c
preload-index.h
pretty.c pretty: fix ref filtering for %(decorate) formats 2023-10-09 11:25:13 -07:00
pretty.h
prio-queue.c
prio-queue.h
progress.c
progress.h
promisor-remote.c
promisor-remote.h
prompt.c
prompt.h
protocol-caps.c
protocol-caps.h
protocol.c
protocol.h
prune-packed.c
prune-packed.h
quote.c
quote.h
range-diff.c
range-diff.h
reachable.c
reachable.h
read-cache-ll.h
read-cache.c
read-cache.h
rebase-interactive.c
rebase-interactive.h
rebase.c
rebase.h
ref-filter.c Merge branch 'vd/for-each-ref-unsorted-optimization' 2023-12-09 16:37:50 -08:00
ref-filter.h ref-filter.h: add functions for filter/format & format-only 2023-11-16 14:02:59 +09:00
reflog-walk.c
reflog-walk.h
reflog.c
reflog.h
refs.c
refs.h
refspec.c
refspec.h
remote-curl.c
remote.c
remote.h
replace-object.c
replace-object.h
repo-settings.c
repository.c
repository.h
rerere.c
rerere.h
reset.c
reset.h
resolve-undo.c
resolve-undo.h
revision.c rev-list: add commit object support in `--missing` option 2023-11-01 12:07:18 +09:00
revision.h rev-list: add commit object support in `--missing` option 2023-11-01 12:07:18 +09:00
run-command.c
run-command.h
sane-ctype.h
scalar.c Merge branch 'ds/scalar-updates' into maint-2.42 2023-11-02 16:53:15 +09:00
send-pack.c
send-pack.h
sequencer.c Merge branch 'ob/sequencer-remove-dead-code' into maint-2.42 2023-11-02 16:53:24 +09:00
sequencer.h
serve.c
serve.h
server-info.c
server-info.h
setup.c setup: fix leaking repository format 2023-11-07 08:51:41 +09:00
setup.h Merge branch 'ds/scalar-updates' into maint-2.42 2023-11-02 16:53:15 +09:00
sh-i18n--envsubst.c
sha1dc_git.c
sha1dc_git.h
shallow.c shallow: fix memory leak when registering shallow roots 2023-11-07 08:51:41 +09:00
shallow.h
shared.mak
shell.c
shortlog.h
sideband.c
sideband.h
sigchain.c
sigchain.h
simple-ipc.h
sparse-index.c
sparse-index.h
split-index.c
split-index.h
stable-qsort.c
statinfo.c Prevent git from rehashing 4GiB files 2023-10-13 13:33:35 -07:00
statinfo.h
strbuf.c
strbuf.h
streaming.c
streaming.h
string-list.c
string-list.h
strmap.c
strmap.h
strvec.c strvec: drop unnecessary include of hex.h 2023-11-03 08:26:55 +09:00
strvec.h
sub-process.c
sub-process.h
submodule-config.c
submodule-config.h
submodule.c
submodule.h
symlinks.c
symlinks.h
tag.c
tag.h
tar.h
tempfile.c
tempfile.h
thread-utils.c
thread-utils.h
tmp-objdir.c
tmp-objdir.h
trace.c
trace.h
trace2.c
trace2.h
trailer.c
trailer.h
transport-helper.c
transport-internal.h
transport.c
transport.h
tree-diff.c
tree-walk.c
tree-walk.h
tree.c
tree.h
unicode-width.h
unimplemented.sh
unix-socket.c
unix-socket.h
unix-stream-server.c
unix-stream-server.h
unpack-trees.c Merge branch 'cw/prelim-cleanup' 2023-10-10 11:39:14 -07:00
unpack-trees.h
upload-pack.c Merge branch 'ds/upload-pack-error-sequence-fix' into maint-2.42 2023-11-02 16:53:20 +09:00
upload-pack.h
url.c
url.h
urlmatch.c
urlmatch.h
usage.c
userdiff.c
userdiff.h
utf8.c
utf8.h
varint.c
varint.h
version.c
version.h
versioncmp.c
versioncmp.h
walker.c
walker.h
wildmatch.c
wildmatch.h
worktree.c
worktree.h
wrap-for-bin.sh
wrapper.c
wrapper.h
write-or-die.c
write-or-die.h
ws.c
ws.h
wt-status.c Merge branch 'jk/diff-result-code-cleanup' into maint-2.42 2023-11-02 16:53:16 +09:00
wt-status.h
xdiff-interface.c
xdiff-interface.h

README.md

Build status

Git - fast, scalable, distributed revision control system

Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.

Git is an Open Source project covered by the GNU General Public License version 2 (some parts of it are under different licenses, compatible with the GPLv2). It was originally written by Linus Torvalds with help of a group of hackers around the net.

Please read the file INSTALL for installation instructions.

Many Git online resources are accessible from https://git-scm.com/ including full documentation and Git related tools.

See Documentation/gittutorial.txt to get started, then see Documentation/giteveryday.txt for a useful minimum set of commands, and Documentation/git-<commandname>.txt for documentation of each command. If git has been correctly installed, then the tutorial can also be read with man gittutorial or git help tutorial, and the documentation of each command with man git-<commandname> or git help <commandname>.

CVS users may also want to read Documentation/gitcvs-migration.txt (man gitcvs-migration or git help cvs-migration if git is installed).

The user discussion and development of Git take place on the Git mailing list -- everyone is welcome to post bug reports, feature requests, comments and patches to git@vger.kernel.org (read Documentation/SubmittingPatches for instructions on patch submission and Documentation/CodingGuidelines).

Those wishing to help with error message, usage and informational message string translations (localization l10) should see po/README.md (a po file is a Portable Object file that holds the translations).

To subscribe to the list, send an email with just "subscribe git" in the body to majordomo@vger.kernel.org (not the Git list). The mailing list archives are available at https://lore.kernel.org/git/, http://marc.info/?l=git and other archival sites.

Issues which are security relevant should be disclosed privately to the Git Security mailing list git-security@googlegroups.com.

The maintainer frequently sends the "What's cooking" reports that list the current status of various development topics to the mailing list. The discussion following them give a good reference for project status, development direction and remaining tasks.

The name "git" was given by Linus Torvalds when he wrote the very first version. He described the tool as "the stupid content tracker" and the name as (depending on your mood):

  • random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of "get" may or may not be relevant.
  • stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang.
  • "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.
  • "goddamn idiotic truckload of sh*t": when it breaks