Teach builtin_diffstat() to consult the hunk provider interface through
diff_provider_consult(), new here: the consult-only entry that answers
without loading content or computing, so it never returns
DIFF_PROVIDER_ERROR. On an answer, the summing callback accumulates the
provided counts directly into the diffstat entry; the blobs were already
loaded for the binary check, so an answer saves the diff run, not the
content load (blame, taught next, skips its loads too). On an
unanswered outcome it computes as before and, with a writer attached,
records what it computed; on unanswered-no-record it computes without
recording.
The provider behind the consult is the diff-hunks store, registered in
front of the terminal builtin computation. Its consult serves a
recorded pair through diff_hunks_replay(), which validates the sequence
before any hunk reaches the callback, so direct accumulation is safe.
The request gains the pair's object ids and the diff options read by the
exclusions below. A side whose bytes are not a stored blob, such as a
working-tree file or a gitlink, has a NULL id; the store passes it by and
the terminal provider computes it. diff_provider_emit_hunks() walks the
same chain, so blame's requests follow these rules the moment blame
supplies identity. The walk also insists, as a BUG check, that a
request's diff options belong to the repository whose chain it walks.
Each exclusion lives with the provider whose key cannot express it. -I
patterns and --anchored shape the diff outside the store key, and break
detection (-B) rescores the pair outside it; the store's consult maps
all three to stop-no-record, so such a request is neither served nor
recorded for any consumer. The consumer-side guard the recording commit
carried for those three comes out here. The compile-time assert on
xpparam_t's layout sits next to that decision, forcing an explicit
keying decision whenever a diff parameter is added. The stat consumer
keeps only the exclusion that is not about the key: --ignore-blank-lines
is part of the key but coalesces hunks differently between the
text-emitting and coordinate-callback paths, so the consumer returns
before consulting. A "log -L" range-scoped stat neither reads nor
records; the line-range filter computes it as before.
"git diff", "git log", "git show", and "git diff-tree" with the --stat,
--numstat, and --shortstat formats consult the interface. Reading is
controlled by core.diffHunks.
An answer is invisible in the output, so the store counts the pairs it
serves and the consultations it cannot, and diff_hunks_read_stats()
reports both; the stat path emits the hits as a trace2 "read-hits" datum
for tests and tuning. The counters live on the store because only the
store knows whether a consultation reached it, and none of its exclusion
legs reaches the replay, so none counts as a miss.
Extend t4220 with the read half:
- output parity with and without the store, at several context lengths
and both directions, and reversed pairs keying apart;
- the consultation made visible through the read-hits datum, and the
trim-divergent pair correct at every context;
- the settings that must bypass the store doing so in both directions
(-I, -B, --anchored, --ignore-blank-lines), asserted through the trace
rather than output parity alone, which a coincidentally equal count
could satisfy;
- a driver-forced algorithm keying apart rather than bypassing: it is
part of the key, so a read under it misses the default entries and a
warm records under its own.
A "log -L" range-scoped stat neither reads nor records.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The diff-hunks store has a writer, but nothing fills it. Teach
builtin_diffstat() to do so: on a warming run (a writer is attached), a
modified pair's stat is produced by collecting the pair's hunk
coordinates instead of emitting text, the counts are summed from those
hunks, and the pair is recorded. A run without a writer is unchanged,
and nothing reads the store yet; the read side arrives next.
The store records one context-free entry per pair, and only for a
trim-stable pair: one whose zero-context trimmed diff (what blame will
read) and untrimmed diff (whose counts a nonzero-context stat matches)
are identical. The warming path computes both and hands them to
diff_hunks_writer_record_stable(), new here, which records only when
they agree; a divergent pair is never recorded and every consumer
computes it. The warming run displays the counts it shows a store-less
run: the trimmed ones, since xdi_diff trims at zero context, while the
untrimmed counts serve only the stability comparison.
Not everything the stat path computes may be recorded.
--ignore-blank-lines is part of the key, but it coalesces hunks
differently between the text-emitting and coordinate-callback paths, so
a recorded entry would not match a store-less run's --stat. -I
patterns, --anchored, and break detection (-B) shape the diff outside
the key entirely; the guard for those three sits in this consumer for
now and moves into the store's own provider when it registers, next. A
"log -L" range-scoped stat is not the whole-pair diff the key describes,
so it does not record. Recording also requires both sides to be valid
regular files whose blobs the key can name: a working-tree side,
textconv output, or a gitlink has no usable id.
"git diff", "git log", "git show", and "git diff-tree" with the --stat,
--numstat, and --shortstat formats attach a writer when writing is
enabled and flush it when the traversal finishes, so a warming run such
as
GIT_DIFF_HUNKS_WRITE=1 git log --all --stat >/dev/null
fills the cache as a side effect of the diff work the command already
does. Writing is controlled by diffHunks.write and GIT_DIFF_HUNKS_WRITE.
Add the write half of t4220:
- ordinary commands never create the store, and creation is gated off
by default, the environment overriding the config;
- a warming run builds a store that verifies, and a second refreshes it
in place;
- a warming run displays parity at zero context on a trim-divergent
pair, committed as a fixture (small synthetic pairs cannot diverge:
minimal diffs add and delete equal counts, and trimming preserves
that);
- binary and mode-only pairs do not break the writer;
- a corrupt store is discarded at seed;
- verify and clear run against the files a warming run builds.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Blame and "git log --stat" recover hunk coordinates by diffing blob
pairs, and recompute them on every run. Add a cache of those
coordinates at $GIT_DIR/objects/info/diff-hunks, beside the
commit-graph, so a later run can look them up instead of decompressing
the blobs and running xdiff again.
The store is a single chunk-format file (see gitformat-chunk(5)): an
8-byte header, a DHIX index of fixed-size entries sorted by key, a DHDT
segment of hunk records, and a trailing hash checksum. An entry is
keyed by the two blob object ids and the xdl_opts the pair was diffed
under, so a stored result is served only where that exact key recurs,
independent of path. A zero-context diff trims unchanged lines from
hunk edges and can pick a different but equally valid set of hunks than
an untrimmed diff, so a recording caller stores a pair only when its
trimmed and untrimmed diffs are identical; such an entry answers any
consumer at any context, and the rare divergent pair is always
computed. Identical hunk blocks are interned once and shared across
keys.
The library provides a reader (repo_diff_hunks_store and _replay, gated
by core.diffHunks), loaded once and cached on the object database as the
commit-graph is, and a writer that accumulates entries and flushes them
in one atomic pass. An absent, corrupt, or disabled store reads as all
misses. A record with no hunks is invalid too: replaying it would claim
the pair equivalent, which the store never asserts, so it reads as a
miss.
Ordinary reads are diagnostic-free. Loading parses the chunk table
through read_table_of_contents_quiet(), new in chunk-format, which
prints nothing on a malformed table and takes the repository's hash
algorithm rather than the_hash_algo, so the file is bounds-checked under
the algorithm it is keyed by.
The flush closes the repository's mmapped store and forgets that loading
was attempted before committing the lockfile. A warming run that also
reads may hold the file it is replacing mapped, and the rename must not
land on a live mapping, which Windows refuses; a read after the flush
then observes the committed file. commit-graph closes its graph before
committing for the same reason.
Writing is off by default, enabled per run by GIT_DIFF_HUNKS_WRITE or
persistently by diffHunks.write, the environment winning. A writer
seeds from the existing store, so a flush merges rather than replaces.
The seed's checksum is verified first: a corrupt store is discarded, not
rewritten with a fresh checksum verify could no longer catch. An entry
that fails the shared diff_provider_check_hunk() or names no blob is
dropped with a warning, since it would only ever read as a miss. A seed
that discarded or dropped anything forces the flush even when the
warming run computed nothing new. The writer fsyncs through a new
diff-hunks core.fsync component.
"git diff-hunks" inspects and manages the file: "verify" checks the
checksum, chunk table, sort order, entry bounds, and every entry's hunk
sequence against that shared check, so a store whose entries could only
read as misses fails verify; "clear" removes the file. Later patches
wire the readers and the writer into the diff and blame paths.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Michael Montalbo