kernel
/
git
mirror of https://git.kernel.org/pub/scm/git/git.git
1
0
Fork 0
Code Releases Activity

fast-import: use writev(3p) to send cat-blob responses 

When answering a `cat-blob` command, `cat_blob()` issues three separate
calls to write(3p) on the cat-blob fd: one for the header line, one for
the full blob payload, and one for the trailing newline. Frontends like
git-filter-repo issue these commands in bulk, once per rewritten blob,
so the syscall overhead adds up.

Use `writev_in_full()` to send all three parts with a single syscall.

This can be benchmarked with the following setup:

    $ git cat-file --unordered --filter=object:type=blob
        --batch-check='cat-blob %(objectname)' --batch-all-objects >request
    $ git fast-import --cat-blob-fd=3 <request

Executing this with 100,000 objects in linux.git:

  Benchmark 1: HEAD~
    Time (mean ± σ):      1.320 s ±  0.003 s    [User: 1.154 s, System: 0.161 s]
    Range (min … max):    1.314 s …  1.324 s    10 runs

  Benchmark 2: HEAD
    Time (mean ± σ):      1.270 s ±  0.022 s    [User: 1.133 s, System: 0.132 s]
    Range (min … max):    1.209 s …  1.282 s    10 runs

  Summary
    HEAD ran
      1.04 ± 0.02 times faster than HEAD~

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
main
Patrick Steinhardt 2026-08-07 08:18:07 +02:00 committed by Junio C Hamano
parent 21db416cd2
commit 5bd4f43456
1 changed files with 15 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
builtin/fast-import.c
View File

@ -3332,6 +3332,7 @@ static void cat_blob_write(const char *buf, unsigned long size)
static void cat_blob(struct object_entry *oe, struct object_id *oid)
{
struct strbuf line = STRBUF_INIT;
struct iovec iov[3];
unsigned long size;
enum object_type type = 0;
char *buf;
@ -3365,10 +3366,21 @@ static void cat_blob(struct object_entry *oe, struct object_id *oid)
strbuf_reset(&line);
strbuf_addf(&line, "%s %s %"PRIuMAX"\n", oid_to_hex(oid),
type_name(type), (uintmax_t)size);
cat_blob_write(line.buf, line.len);

/*
* Write the header, the payload and the trailing newline with a
* single writev(3p) call instead of three separate write(3p) calls.
*/
iov[0].iov_base = line.buf;
iov[0].iov_len = line.len;
iov[1].iov_base = buf;
iov[1].iov_len = size;
iov[2].iov_base = (void *) "\n";
iov[2].iov_len = 1;

if (writev_in_full(cat_blob_fd, iov, ARRAY_SIZE(iov)) < 0)
die_errno(_("write to frontend failed"));
strbuf_release(&line);
cat_blob_write(buf, size);
cat_blob_write("\n", 1);
if (oe && oe->pack_id == pack_id) {
last_blob.offset = oe->idx.offset;
strbuf_attach(&last_blob.data, buf, size, size + 1);