140 lines
6.1 KiB
Plaintext
140 lines
6.1 KiB
Plaintext
Git commit graph format
|
|
=======================
|
|
|
|
The Git commit graph stores a list of commit OIDs and some associated
|
|
metadata, including:
|
|
|
|
- The generation number of the commit. Commits with no parents have
|
|
generation number 1; commits with parents have generation number
|
|
one more than the maximum generation number of its parents. We
|
|
reserve zero as special, and can be used to mark a generation
|
|
number invalid or as "not computed".
|
|
|
|
- The root tree OID.
|
|
|
|
- The commit date.
|
|
|
|
- The parents of the commit, stored using positional references within
|
|
the graph file.
|
|
|
|
- The Bloom filter of the commit carrying the paths that were changed between
|
|
the commit and its first parent, if requested.
|
|
|
|
These positional references are stored as unsigned 32-bit integers
|
|
corresponding to the array position within the list of commit OIDs. Due
|
|
to some special constants we use to track parents, we can store at most
|
|
(1 << 30) + (1 << 29) + (1 << 28) - 1 (around 1.8 billion) commits.
|
|
|
|
== Commit graph files have the following format:
|
|
|
|
In order to allow extensions that add extra data to the graph, we organize
|
|
the body into "chunks" and provide a binary lookup table at the beginning
|
|
of the body. The header includes certain values, such as number of chunks
|
|
and hash type.
|
|
|
|
All multi-byte numbers are in network byte order.
|
|
|
|
HEADER:
|
|
|
|
4-byte signature:
|
|
The signature is: {'C', 'G', 'P', 'H'}
|
|
|
|
1-byte version number:
|
|
Currently, the only valid version is 1.
|
|
|
|
1-byte Hash Version
|
|
We infer the hash length (H) from this value:
|
|
1 => SHA-1
|
|
2 => SHA-256
|
|
If the hash type does not match the repository's hash algorithm, the
|
|
commit-graph file should be ignored with a warning presented to the
|
|
user.
|
|
|
|
1-byte number (C) of "chunks"
|
|
|
|
1-byte number (B) of base commit-graphs
|
|
We infer the length (H*B) of the Base Graphs chunk
|
|
from this value.
|
|
|
|
CHUNK LOOKUP:
|
|
|
|
(C + 1) * 12 bytes listing the table of contents for the chunks:
|
|
First 4 bytes describe the chunk id. Value 0 is a terminating label.
|
|
Other 8 bytes provide the byte-offset in current file for chunk to
|
|
start. (Chunks are ordered contiguously in the file, so you can infer
|
|
the length using the next chunk position if necessary.) Each chunk
|
|
ID appears at most once.
|
|
|
|
The remaining data in the body is described one chunk at a time, and
|
|
these chunks may be given in any order. Chunks are required unless
|
|
otherwise specified.
|
|
|
|
CHUNK DATA:
|
|
|
|
OID Fanout (ID: {'O', 'I', 'D', 'F'}) (256 * 4 bytes)
|
|
The ith entry, F[i], stores the number of OIDs with first
|
|
byte at most i. Thus F[255] stores the total
|
|
number of commits (N).
|
|
|
|
OID Lookup (ID: {'O', 'I', 'D', 'L'}) (N * H bytes)
|
|
The OIDs for all commits in the graph, sorted in ascending order.
|
|
|
|
Commit Data (ID: {'C', 'D', 'A', 'T' }) (N * (H + 16) bytes)
|
|
* The first H bytes are for the OID of the root tree.
|
|
* The next 8 bytes are for the positions of the first two parents
|
|
of the ith commit. Stores value 0x70000000 if no parent in that
|
|
position. If there are more than two parents, the second value
|
|
has its most-significant bit on and the other bits store an array
|
|
position into the Extra Edge List chunk.
|
|
* The next 8 bytes store the generation number of the commit and
|
|
the commit time in seconds since EPOCH. The generation number
|
|
uses the higher 30 bits of the first 4 bytes, while the commit
|
|
time uses the 32 bits of the second 4 bytes, along with the lowest
|
|
2 bits of the lowest byte, storing the 33rd and 34th bit of the
|
|
commit time.
|
|
|
|
Extra Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional]
|
|
This list of 4-byte values store the second through nth parents for
|
|
all octopus merges. The second parent value in the commit data stores
|
|
an array position within this list along with the most-significant bit
|
|
on. Starting at that array position, iterate through this list of commit
|
|
positions for the parents until reaching a value with the most-significant
|
|
bit on. The other bits correspond to the position of the last parent.
|
|
|
|
Bloom Filter Index (ID: {'B', 'I', 'D', 'X'}) (N * 4 bytes) [Optional]
|
|
* The ith entry, BIDX[i], stores the number of bytes in all Bloom filters
|
|
from commit 0 to commit i (inclusive) in lexicographic order. The Bloom
|
|
filter for the i-th commit spans from BIDX[i-1] to BIDX[i] (plus header
|
|
length), where BIDX[-1] is 0.
|
|
* The BIDX chunk is ignored if the BDAT chunk is not present.
|
|
|
|
Bloom Filter Data (ID: {'B', 'D', 'A', 'T'}) [Optional]
|
|
* It starts with header consisting of three unsigned 32-bit integers:
|
|
- Version of the hash algorithm being used. We currently only support
|
|
value 1 which corresponds to the 32-bit version of the murmur3 hash
|
|
implemented exactly as described in
|
|
https://en.wikipedia.org/wiki/MurmurHash#Algorithm and the double
|
|
hashing technique using seed values 0x293ae76f and 0x7e646e2 as
|
|
described in https://doi.org/10.1007/978-3-540-30494-4_26 "Bloom Filters
|
|
in Probabilistic Verification"
|
|
- The number of times a path is hashed and hence the number of bit positions
|
|
that cumulatively determine whether a file is present in the commit.
|
|
- The minimum number of bits 'b' per entry in the Bloom filter. If the filter
|
|
contains 'n' entries, then the filter size is the minimum number of 64-bit
|
|
words that contain n*b bits.
|
|
* The rest of the chunk is the concatenation of all the computed Bloom
|
|
filters for the commits in lexicographic order.
|
|
* Note: Commits with no changes or more than 512 changes have Bloom filters
|
|
of length one, with either all bits set to zero or one respectively.
|
|
* The BDAT chunk is present if and only if BIDX is present.
|
|
|
|
Base Graphs List (ID: {'B', 'A', 'S', 'E'}) [Optional]
|
|
This list of H-byte hashes describe a set of B commit-graph files that
|
|
form a commit-graph chain. The graph position for the ith commit in this
|
|
file's OID Lookup chunk is equal to i plus the number of commits in all
|
|
base graphs. If B is non-zero, this chunk must exist.
|
|
|
|
TRAILER:
|
|
|
|
H-byte HASH-checksum of all of the above.
|