git/odb
Patrick Steinhardt a316d615da odb: introduce interface to generate packfiles
Packfiles have two primary use cases:

  - They are used to store objects at rest in a Git repository.

  - They are used on the transport layer to transfer objects between two
    repositories.

The first class is closely tied to a given object database backend, and
as such this use is highly specific to how such a backend decides to
store its data. This shows in git-pack-objects(1), which is used by
git-repack(1) et al to optimize the object database, which supports lots
of options that are closely coupled with how data is stored.

But the second class is quite a lot more generic: we don't care about
specifics of how the object database stores its objects, but to generate
the packfiles we only care about the object graph itself. Still, this
use case is also coupled with git-pack-objects(1).

Unfortunately, because git-pack-objects(1) covers both classes, the
result is that it is very hard to port the whole command to properly
support pluggable object databases. There are simply way too many
options that an alternative implementation will have a very hard time to
support in the first place.

And despite being hard to implement, it's also quite unnecessary to
implement those backend-specific options. Optimizing the object database
has already been made pluggable, and an alternative implementation is
unlikely to care about cruft packs, unpacked objects, keep packs and the
like. But we still need to make at least _parts_ of the packfile
generation pluggable so that backends can generate packfiles for the
transport layer itself.

Introduce a new interface that lets backends generate a new packfile and
implement that interface for the "files" backend. The options supported
by the callback are exactly the set of options that are required for the
transport layer, but nothing more.

This means that git-pack-objects(1) itself cannot be ported over to this
new interface, but as explained above that's a hard feat to pull off due
to the backend-specific features. Ideally though, we should expose the
ability to generate arbitrary packfiles using this interface. The intent
of this is to eventually introduce a git-objects(1) subcommand (similar
to git-refs(1)) that exposes generic interfaces for accessing everything
related to the object database. In that case, we are able to expose only
those options that are generic.

Subsequent commits will convert git-upload-pack(1), git-send-pack(1) and
git-bundle(1) to use this interface.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-08-21 09:01:54 -07:00
..
source-files.c odb: introduce interface to generate packfiles 2026-08-21 09:01:54 -07:00
source-files.h odb: make optimizations pluggable 2026-07-13 08:13:17 -07:00
source-inmemory.c Merge branch 'jt/receive-pack-use-odb-transactions' 2026-07-22 10:30:53 -07:00
source-inmemory.h odb/source-inmemory: convert to use oidtree 2026-05-15 04:50:45 +09:00
source-loose.c Merge branch 'jt/receive-pack-use-odb-transactions' 2026-07-22 10:30:53 -07:00
source-loose.h odb/source-loose: drop pointer to the "files" source 2026-06-01 18:47:19 +09:00
source-packed.c Merge branch 'ps/odb-move-loose-object-writing' 2026-07-30 10:32:05 -07:00
source-packed.h Merge branch 'ps/odb-generalize-prepare' 2026-07-13 08:27:28 -07:00
source.c odb/source: make `free()` function pluggable 2026-03-05 11:45:15 -08:00
source.h odb: introduce interface to generate packfiles 2026-08-21 09:01:54 -07:00
streaming.c odb: use size_t for object_info.sizep and the size APIs 2026-06-15 07:45:41 -07:00
streaming.h Merge branch 'ps/odb-in-memory' into ps/odb-source-loose 2026-05-21 22:34:55 +09:00
transaction.c odb/transaction: introduce ODB transaction flags 2026-07-10 13:21:53 -07:00
transaction.h odb/transaction: introduce ODB transaction flags 2026-07-10 13:21:53 -07:00