Commit Graph

74067 Commits (c3d034df1606beca8a4ba3bc8a8266f4c711f23c)

Author SHA1 Message Date
Junio C Hamano 1e1586e4ed The sixteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-24 16:39:16 -07:00
Junio C Hamano 532083fd16 Merge branch 'kl/attr-read-attr-fromindex-msan-workaround'
Code clarification to avoid an appearance of using an uninitialized
variable.

* kl/attr-read-attr-fromindex-msan-workaround:
  attr: fix msan issue in read_attr_from_index
2024-06-24 16:39:15 -07:00
Junio C Hamano 107ed55103 Merge branch 'jc/worktree-git-path'
Code cleanup.

* jc/worktree-git-path:
  worktree_git_path(): move the declaration to path.h
2024-06-24 16:39:15 -07:00
Junio C Hamano e5ff701d4c Merge branch 'tb/commit-graph-use-tempfile'
"git update-server-info" and "git commit-graph --write" have been
updated to use the tempfile API to avoid leaving cruft after
failing.

* tb/commit-graph-use-tempfile:
  server-info.c: remove temporary info files on exit
  commit-graph.c: remove temporary graph layers on exit
2024-06-24 16:39:15 -07:00
Junio C Hamano 2c4aa7ad74 Merge branch 'jc/add-i-retire-usebuiltin-config'
For over a year, setting add.interactive.useBuiltin configuration
variable did nothing but giving a "this does not do anything"
warning.  Finally remove it.

* jc/add-i-retire-usebuiltin-config:
  add-i: finally retire add.interactive.useBuiltin
2024-06-24 16:39:14 -07:00
Junio C Hamano ae2f21b560 Merge branch 'jc/no-default-attr-tree-in-bare'
Earlier we stopped using the tree of HEAD as the default source of
attributes in a bare repository, but failed to document it.  This
has been corrected.

* jc/no-default-attr-tree-in-bare:
  attr.tree: HEAD:.gitattributes is no longer the default in a bare repo
2024-06-24 16:39:14 -07:00
Junio C Hamano f0a462ecd5 Merge branch 'tb/precompose-getcwd'
We forgot to normalize the result of getcwd() to NFC on macOS where
all other paths are normalized, which has been corrected.  This still
does not address the case where core.precomposeUnicode configuration
is not defined globally.

* tb/precompose-getcwd:
  macOS: ls-files path fails if path of workdir is NFD
2024-06-24 16:39:14 -07:00
Junio C Hamano ffa47b75cf Merge branch 'tb/pseudo-merge-reachability-bitmap'
The pseudo-merge reachability bitmap to help more efficient storage
of the reachability bitmap in a repository with too many refs has
been added.

* tb/pseudo-merge-reachability-bitmap: (26 commits)
  pack-bitmap.c: ensure pseudo-merge offset reads are bounded
  Documentation/technical/bitmap-format.txt: add missing position table
  t/perf: implement performance tests for pseudo-merge bitmaps
  pseudo-merge: implement support for finding existing merges
  ewah: `bitmap_equals_ewah()`
  pack-bitmap: extra trace2 information
  pack-bitmap.c: use pseudo-merges during traversal
  t/test-lib-functions.sh: support `--notick` in `test_commit_bulk()`
  pack-bitmap: implement test helpers for pseudo-merge
  ewah: implement `ewah_bitmap_popcount()`
  pseudo-merge: implement support for reading pseudo-merge commits
  pack-bitmap.c: read pseudo-merge extension
  pseudo-merge: scaffolding for reads
  pack-bitmap: extract `read_bitmap()` function
  pack-bitmap-write.c: write pseudo-merge table
  pseudo-merge: implement support for selecting pseudo-merge commits
  config: introduce `git_config_double()`
  pack-bitmap: make `bitmap_writer_push_bitmapped_commit()` public
  pack-bitmap: implement `bitmap_writer_has_bitmapped_object_id()`
  pack-bitmap-write: support storing pseudo-merge commits
  ...
2024-06-24 16:39:13 -07:00
René Scharfe 0f4b0d4cf0 diff: allow --color-moved with --no-ext-diff
We ignore the option --color-moved if an external diff program is
configured, presumably because its overhead is unnecessary in that case.
Respect the option if we don't actually use the external diff, though.

Reported-by: lolligerhans@gmx.de
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-24 13:49:41 -07:00
Eric Wong 493fdae046 object-file: fix leak on conversion failure
I'm not sure exactly how to trigger the leak, but it seems fairly
obvious that the `content' buffer should be freed even if
convert_object_file() fails.  Noticed while working in this area
on unrelated things.

Signed-off-by: Eric Wong <e@80x24.org>
Acked-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-24 09:07:21 -07:00
Johannes Sixt c1db988093 Merge branch 'pk/swedish-translation'
* pk/swedish-translation:
  git-gui: sv.po: Update Swedish translation (576t0f0u)
2024-06-23 10:25:57 +02:00
Johannes Sixt 1f9693afb2 Merge branch 'bc/french-translation'
* bc/french-translation:
  git-gui: po: fix typo in French "aperçu"
2024-06-23 10:25:41 +02:00
Junio C Hamano 4e66b5a990 fuzz: minimum fuzzers environment lacks libcURL
The "fuzz smoke test" job compiles various .o files to create
libgit.a and others, but the final build product of the fuzzer build
is *not* "git".  Since the job is not interested in building a
working "git", it does not define any build flags, and among the
notable ones that are missing is NO_CURL---even though the CI
environment that runs the job does not have libcURL development
package installed.

This obviously leads to a build failure.

Pass NO_CURL=NoThanks to "make" to make sure things will build
correctly, if we add any conditional compilation with "#ifdef
NO_CURL ... #endif" in the codebase.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-21 22:12:13 -07:00
Randall S. Becker 57139818bf version: teach --build-options to reports zlib version information
Show ZLIB_VERSION, if defined, in "git version --build-options"
output.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-21 16:43:58 -07:00
Randall S. Becker 2e2203163d version: teach --build-options to reports libcurl version information
Show LIBCURL_VERSION, if defined, in "git version --build-options"
output.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-21 16:40:43 -07:00
Junio C Hamano 9005149a4a The fifteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 15:45:17 -07:00
Junio C Hamano 892fd8b89f Merge branch 'jc/heads-are-branches'
The "--heads" option of "ls-remote" and "show-ref" has been been
deprecated; "--branches" replaces "--heads".

* jc/heads-are-branches:
  show-ref: introduce --branches and deprecate --heads
  ls-remote: introduce --branches and deprecate --heads
  refs: call branches branches
2024-06-20 15:45:17 -07:00
Junio C Hamano 166cdd8915 Merge branch 'ps/document-breaking-changes'
The structure of the document that records longer-term project
decisions to deprecate/remove/update various behaviour has been
outlined.

* ps/document-breaking-changes:
  BreakingChanges: document that we do not plan to deprecate git-checkout
  BreakingChanges: document removal of grafting
  BreakingChanges: document upcoming change from "sha1" to "sha256"
  docs: introduce document to announce breaking changes
2024-06-20 15:45:16 -07:00
Junio C Hamano 83ac567781 Merge branch 'pw/rebase-i-error-message'
When the user adds to "git rebase -i" instruction to "pick" a merge
commit, the error experience is not pleasant.  Such an error is now
caught earlier in the process that parses the todo list.

* pw/rebase-i-error-message:
  rebase -i: improve error message when picking merge
  rebase -i: pass struct replay_opts to parse_insn_line()
2024-06-20 15:45:15 -07:00
Junio C Hamano e4ecba994c Merge branch 'ds/ahead-behind-fix'
Fix for a progress bar.

* ds/ahead-behind-fix:
  commit-graph: increment progress indicator
2024-06-20 15:45:14 -07:00
Junio C Hamano 4401639f96 Merge branch 'ps/abbrev-length-before-setup-fix'
Setting core.abbrev too early before the repository set-up
(typically in "git clone") caused segfault, which as been
corrected.

* ps/abbrev-length-before-setup-fix:
  object-name: don't try to abbreviate to lengths greater than hexsz
  parse-options-cb: stop clamping "--abbrev=" to hash length
  config: fix segfault when parsing "core.abbrev" without repo
2024-06-20 15:45:13 -07:00
Junio C Hamano 9071453ef6 Merge branch 'rj/format-patch-auto-cover-with-interdiff'
"git format-patch --interdiff" for multi-patch series learned to
turn on cover letters automatically (unless told never to enable
cover letter with "--no-cover-letter" and such).

* rj/format-patch-auto-cover-with-interdiff:
  format-patch: assume --cover-letter for diff in multi-patch series
  t4014: cleanups in a few tests
2024-06-20 15:45:12 -07:00
Junio C Hamano 5f14d20984 Merge branch 'kn/update-ref-symref'
"git update-ref --stdin" learned to handle transactional updates of
symbolic-refs.

* kn/update-ref-symref:
  update-ref: add support for 'symref-update' command
  reftable: pick either 'oid' or 'target' for new updates
  update-ref: add support for 'symref-create' command
  update-ref: add support for 'symref-delete' command
  update-ref: add support for 'symref-verify' command
  refs: specify error for regular refs with `old_target`
  refs: create and use `ref_update_expects_existing_old_ref()`
2024-06-20 15:45:12 -07:00
Junio C Hamano c1322ca474 Merge branch 'gt/unit-test-oidtree'
"oidtree" tests were rewritten to use the unit test framework.

* gt/unit-test-oidtree:
  t/: migrate helper/test-oidtree.c to unit-tests/t-oidtree.c
2024-06-20 15:45:10 -07:00
Junio C Hamano 393879d473 Merge branch 'tb/multi-pack-reuse-fix'
Assorted fixes to multi-pack-index code paths.

* tb/multi-pack-reuse-fix:
  pack-revindex.c: guard against out-of-bounds pack lookups
  pack-bitmap.c: avoid uninitialized `pack_int_id` during reuse
  midx-write.c: do not read existing MIDX with `packs_to_include`
2024-06-20 15:45:10 -07:00
Junio C Hamano f4788a577b Merge branch 'ps/make-append-to-cflags'
To help developers, the build procedure now allows builders to use
CFLAGS_APPEND to specify additional CFLAGS.

* ps/make-append-to-cflags:
  Makefile: add ability to append to CFLAGS and LDFLAGS
2024-06-20 15:45:09 -07:00
Junio C Hamano 8ba7dbdefb Merge branch 'rs/diff-exit-code-with-external-diff'
"git diff --exit-code --ext-diff" learned to take the exit status
of the external diff driver into account when deciding the exit
status of the overall "git diff" invocation when configured to do
so.

* rs/diff-exit-code-with-external-diff:
  diff: let external diffs report that changes are uninteresting
  userdiff: add and use struct external_diff
  t4020: test exit code with external diffs
2024-06-20 15:45:08 -07:00
Junio C Hamano e631115ae5 Merge branch 'ds/doc-add-interactive-singlekey'
Doc update.

* ds/doc-add-interactive-singlekey:
  doc: interactive.singleKey is disabled by default
2024-06-20 15:45:08 -07:00
Randall S. Becker 8b731b8d06 version: --build-options reports OpenSSL version information
This change uses the OpenSSL supplied OPENSSL_VERSION_TEXT #define supplied
for this purpose by that project. If the #define is not present, the version
is not reported.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 13:02:31 -07:00
René Scharfe 28dc26dc33 commit: remove find_header_mem()
cfc5cf428b (receive-pack.c: consolidate find header logic, 2022-01-06)
introduced find_header_mem() and turned find_commit_header() into a thin
wrapper.  Since then, the latter has become the last remaining caller of
the former.  Remove it to restore find_commit_header() to the state
before cfc5cf428b, get rid of a strlen(3) call and resolve a NEEDSWORK
note in the process.

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 11:12:40 -07:00
Jeff King 40d817875d t5500: fix mistaken $SERVER reference in helper function
The end of t5500 contains two tests which use a single helper function,
fetch_filter_blob_limit_zero(). It takes a parameter to point to the
path of the server repository, which we store locally as $SERVER. The
first caller uses the relative path "server", while the second points
into the httpd document root.

Commit 07ef3c6604 (fetch test: use more robust test for filtered
objects, 2019-12-23) refactored some lines, but accidentally switched
"$SERVER" to "server" in one spot. That means the second caller is
looking at the server directory from the previous test rather than its
own.

This happens to work out because the "server" directory from the first
test is still hanging around, and the contents of the two are identical.
But it was clearly not the intended behavior, and is fragile to cleaning
up the leftovers from the first test.

Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 11:06:45 -07:00
Johannes Schindelin 3c295c87c2 mingw: drop bogus (and unneeded) declaration of `_pgmptr`
In 08809c09aa (mingw: add a helper function to attach GDB to the
current process, 2020-02-13), I added a declaration that was not needed.
Back then, that did not matter, but now that the declaration of that
symbol was changed in mingw-w64's headers, it causes the following
compile error:

      CC compat/mingw.o
  compat/mingw.c: In function 'open_in_gdb':
  compat/mingw.c:35:9: error: function declaration isn't a prototype [-Werror=strict-prototypes]
     35 |         extern char *_pgmptr;
        |         ^~~~~~
  In file included from C:/git-sdk-64/usr/src/git/build-installers/mingw64/lib/gcc/x86_64-w64-mingw32/14.1.0/include/mm_malloc.h:27,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/lib/gcc/x86_64-w64-mingw32/14.1.0/include/xmmintrin.h:34,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/lib/gcc/x86_64-w64-mingw32/14.1.0/include/immintrin.h:31,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/lib/gcc/x86_64-w64-mingw32/14.1.0/include/x86intrin.h:32,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/include/winnt.h:1658,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/include/minwindef.h:163,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/include/windef.h:9,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/include/windows.h:69,
                   from C:/git-sdk-64/usr/src/git/build-installers/mingw64/include/winsock2.h:23,
                   from compat/../git-compat-util.h:215,
                   from compat/mingw.c:1:
  compat/mingw.c:35:22: error: '__p__pgmptr' redeclared without dllimport attribute: previous dllimport ignored [-Werror=attributes]
     35 |         extern char *_pgmptr;
        |                      ^~~~~~~

Let's just drop the declaration and get rid of this compile error.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:59:42 -07:00
Jeff King 96a6621d25 fetch-pack: fix segfault when fscking without --lock-pack
The fetch-pack internals have multiple options related to creating
".keep" lock-files for the received pack:

  - if args.lock_pack is set, then we tell index-pack to create a .keep
    file. In the fetch-pack plumbing command, this is triggered by
    passing "-k" twice.

  - if the caller passes in a pack_lockfiles string list, then we use it
    to record the path of the keep-file created by index-pack. We get
    that name by reading the stdout of index-pack. In the fetch-pack
    command, this is triggered by passing the (undocumented) --lock-pack
    option; without it, we pass in a NULL string list.

So it's possible to ask index-pack to create the lock-file (using "-k
-k") but not ask to record it (by avoiding "--lock-pack"). This worked
fine until 5476e1efde (fetch-pack: print and use dangling .gitmodules,
2021-02-22), but now it causes a segfault.

Before that commit, if pack_lockfiles was NULL, we wouldn't bother
reading the output from index-pack at all. But since that commit,
index-pack may produce extra output if we asked it to fsck. So even if
nobody cares about the lockfile path, we still need to read it to skip
to the output we do care about.

We correctly check that we didn't get a NULL lockfile path (which can
happen if we did not ask it to create a .keep file at all), but we
missed the case where the lockfile path is not NULL (due to "-k -k") but
the pack_lockfiles string_list is NULL (because nobody passed
"--lock-pack"), and segfault trying to add to the NULL string-list.

We can fix this by skipping the append to the string list when either
the value or the list is NULL. In that case we must also free the
lockfile path to avoid leaking it when it's non-NULL.

Nobody noticed the bug for so long because the transport code used by
"git fetch" always passes in a pack_lockfiles pointer, and remote-curl
(the main user of the fetch-pack plumbing command) always passes
--lock-pack.

Reported-by: Kirill Smelkov <kirr@nexedi.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:58:00 -07:00
Elijah Newren f19b916535 merge-ort: convert more error() cases to path_msg()
merge_submodule() stores errors using path_msg(), whereas other call
sites make use of the error() function.  This is inconsistent, and
moving towards path_msg() seems more friendly for libification efforts
since it will allow the caller to determine whether the error messages
need to be printed.

Note that this deferred handling of error messages changes the error
message in a recursive merge from
  error: failed to execute internal merge
to
  From inner merge:  error: failed to execute internal merge
which provides a little more information about the error which may be
useful.  Since the recursive merge strategy still only shows the older
error, we had to adjust the new testcase introduced a few commits ago to
just search for the older message somewhere in the output.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:35:25 -07:00
Elijah Newren 14949d91b6 merge-ort: upon merge abort, only show messages causing the abort
When something goes wrong enough that we need to abort early and not
even attempt merging the remaining files, it probably does not make
sense to report conflicts messages for the subset of files we processed
before hitting the fatal error.  Instead, only show the messages
associated with paths where we hit the fatal error.  Also, print these
messages to stderr rather than stdout.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:35:25 -07:00
Elijah Newren c55c3f20b1 merge-ort: loosen commented requirements
The comment above type_short_descriptions claimed that the order had to
match what was found in the conflict_info_and_types enum.  Since
type_short_descriptions uses designated initializers, the order should
not actually matter; I am guessing that positional initializers may have
been under consideration when that comment was added, but the comment
was not updated when designated initializers were chosen.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:35:25 -07:00
Elijah Newren 5fadf1f933 merge-ort: clearer propagation of failure-to-function from merge_submodule
The 'clean' member variable is somewhat of a tri-state (1 = clean, 0 =
conflicted, -1 = failure-to-determine), but we often like to think of
it as binary (ignoring the possibility of a negative value) and use
constructs like '!clean' to reflect this.  However, these constructs
can make codepaths more difficult to understand, unless we handle the
negative case early and return pre-emptively; do that in
handle_content_merge() to make the code a bit easier to read.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:35:24 -07:00
Elijah Newren 9ed8e17d8a merge-ort: fix type of local 'clean' var in handle_content_merge ()
handle_content_merge() returns an int.  Every caller of
handle_content_merge() expects an int.  However, we declare a local
variable 'clean' that we use for the return value to be unsigned.  To
make matters worse, we also assign 'clean' the return value of
merge_submodule() in one codepath, which is defined to return an int.
It seems that the only reason to have 'clean' be unsigned was to allow a
cutesy bit manipulation operation to be well-defined.  Fix the type of
the 'clean' local in handle_content_merge().

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:35:24 -07:00
Elijah Newren 0b4f726cde merge-ort: maintain expected invariant for priv member
The calling convention for the merge machinery is
   One call to          init_merge_options()
   One or more calls to merge_incore_[non]recursive()
   One call to          merge_finalize()
      (possibly indirectly via merge_switch_to_result())
Both merge_switch_to_result() and merge_finalize() expect
   opt->priv == NULL && result->priv != NULL
which is supposed to be set up by our move_opt_priv_to_result_priv()
function.  However, two codepaths dealing with error cases did not
execute this necessary logic, which could result in assertion failures
(or, if assertions were compiled out, could result in segfaults).  Fix
the oversight and add a test that would have caught one of these
problems.

While at it, also tighten an existing test for a non-recursive merge
to verify that it fails with appropriate status.  Most merge tests in
the testsuite check either for success or conflicts; those testing for
neither are rare and it is good to ensure they support the invariant
assumed by builtin/merge.c in this comment:
    /*
     * The backend exits with 1 when conflicts are
     * left to be resolved, with 2 when it does not
     * handle the given merge at all.
     */
So, explicitly check for the exit status of 2 in these cases.

Reported-by: Matt Cree <matt.cree@gearset.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:35:24 -07:00
Elijah Newren e79bdb426c merge-ort: extract handling of priv member into reusable function
In preparation for a subsequent commit which will ensure we do not
forget to maintain our invariants for the priv member in error
codepaths, extract the necessary functionality out into a separate
function.  This change is cosmetic at this point, and introduces no
changes beyond an extra assertion sanity check.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:35:24 -07:00
Xing Xin 63d903ff52 unbundle: extend object verification for fetches
The existing fetch.fsckObjects and transfer.fsckObjects configurations
were not fully applied to bundle-involved fetches, including direct
bundle fetches and bundle-uri enabled fetches. Furthermore, there was no
object verification support for unbundle.

This commit extends object verification support in `bundle.c:unbundle`
by adding the `VERIFY_BUNDLE_FSCK` option to `verify_bundle_flags`. When
this option is enabled, we append the `--fsck-objects` flag to
`git-index-pack`.

The `VERIFY_BUNDLE_FSCK` option is now used by bundle-involved fetches,
where we use `fetch-pack.c:fetch_pack_fsck_objects` to determine whether
to enable this option for `bundle.c:unbundle`, specifically in:

- `transport.c:fetch_refs_from_bundle` for direct bundle fetches.
- `bundle-uri.c:unbundle_from_file` for bundle-uri enabled fetches.

This addition ensures a consistent logic for object verification during
fetches. Tests have been added to confirm functionality in the scenarios
mentioned above.

Reviewed-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Xing Xin <xingxin.xx@bytedance.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:30:08 -07:00
Xing Xin d0cbc75680 fetch-pack: expose fsckObjects configuration logic
Currently, we can use "transfer.fsckObjects" and the more specific
"fetch.fsckObjects" to control checks for broken objects in received
packs during fetches. However, these configurations were only
acknowledged by `fetch-pack.c:get_pack` and did not take effect in
direct bundle fetches or fetches with _bundle-uri_ enabled.

This commit exposes the fetch-then-transfer configuration logic by
adding a new function `fetch_pack_fsck_objects` in fetch-pack.h. This
new function is used to replace the assignment for `fsck_objects` in
`fetch-pack.c:get_pack`. In the next commit, this function will also be
used to extend fsck support for bundle-involved fetches.

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Xing Xin <xingxin.xx@bytedance.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:30:07 -07:00
Xing Xin 3079026fc1 bundle-uri: verify oid before writing refs
When using the bundle-uri mechanism with a bundle list containing
multiple interrelated bundles, we encountered a bug where tips from
downloaded bundles were not discovered, thus resulting in rather slow
clones. This was particularly problematic when employing the
"creationTokens" heuristic.

To reproduce this issue, consider a repository with a single branch
"main" pointing to commit "A". Firstly, create a base bundle with:

  git bundle create base.bundle main

Then, add a new commit "B" on top of "A", and create an incremental
bundle for "main":

  git bundle create incr.bundle A..main

Now, generate a bundle list with the following content:

  [bundle]
      version = 1
      mode = all
      heuristic = creationToken

  [bundle "base"]
      uri = base.bundle
      creationToken = 1

  [bundle "incr"]
      uri = incr.bundle
      creationToken = 2

A fresh clone with the bundle list above should result in a reference
"refs/bundles/main" pointing to "B" in the new repository. However, git
would still download everything from the server, as if it had fetched
nothing locally.

So why the "refs/bundles/main" is not discovered? After some digging I
found that:

1. Bundles in bundle list are downloaded to local files via
   `bundle-uri.c:download_bundle_list` or via
   `bundle-uri.c:fetch_bundles_by_token` for the "creationToken"
   heuristic.
2. Each bundle is unbundled via `bundle-uri.c:unbundle_from_file`, which
   is called by `bundle-uri.c:unbundle_all_bundles` or called within
   `bundle-uri.c:fetch_bundles_by_token` for the "creationToken"
   heuristic.
3. To get all prerequisites of the bundle, the bundle header is read
   inside `bundle-uri.c:unbundle_from_file` to by calling
   `bundle.c:read_bundle_header`.
4. Then it calls `bundle.c:unbundle`, which calls
   `bundle.c:verify_bundle` to ensure the repository contains all the
   prerequisites.
5. `bundle.c:verify_bundle` calls `parse_object`, which eventually
   invokes `packfile.c:prepare_packed_git` or
   `packfile.c:reprepare_packed_git`, filling
   `raw_object_store->packed_git` and setting `packed_git_initialized`.
6. If `bundle.c:unbundle` succeeds, it writes refs via
   `refs.c:refs_update_ref` with `REF_SKIP_OID_VERIFICATION` set. Here
   bundle refs which can target arbitrary objects are written to the
   repository.
7. Finally, in `fetch-pack.c:do_fetch_pack_v2`, the functions
   `fetch-pack.c:mark_complete_and_common_ref` and
   `fetch-pack.c:mark_tips` are called with `OBJECT_INFO_QUICK` set to
   find local tips for negotiation. The `OBJECT_INFO_QUICK` flag
   prevents `packfile.c:reprepare_packed_git` from being called,
   resulting in failures to parse OIDs that reside only in the latest
   bundle.

In the example above, when unbunding "incr.bundle", "base.pack" is added
to `packed_git` due to prerequisites verification. However, "B" cannot
be found for negotiation because it exists in "incr.pack", which is not
included in `packed_git`.

Fix the bug by removing `REF_SKIP_OID_VERIFICATION` flag when writing
bundle refs. When `refs.c:refs_update_ref` is called to write the
corresponding bundle refs, it triggers `refs.c:ref_transaction_commit`.
This, in turn, invokes `refs.c:ref_transaction_prepare`, which calls
`transaction_prepare` of the refs storage backend. For files backend, it
is `files-backend.c:files_transaction_prepare`, and for reftable
backend, it is `reftable-backend.c:reftable_be_transaction_prepare`.
Both functions eventually call `object.c:parse_object`, which can invoke
`packfile.c:reprepare_packed_git` to refresh `packed_git`. This ensures
that bundle refs point to valid objects and that all tips from bundle
refs are correctly parsed during subsequent negotiations.

A set of negotiation-related tests for cloning with bundle-uri has been
included to demonstrate that downloaded bundles are utilized to
accelerate fetching.

Additionally, another test has been added to show that bundles with
incorrect headers, where refs point to non-existent objects, do not
result in any bundle refs being created in the repository.

Reviewed-by: Karthik Nayak <karthik.188@gmail.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Xing Xin <xingxin.xx@bytedance.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:30:07 -07:00
Eric Wong 75daa42ddf t1006: ensure cat-file info isn't buffered by default
While working on buffering changes to `git cat-file' in a
separate patch, I inadvertently made the output of --batch-check
and the `info' command of --batch-command buffered as if
opt->buffer_output is turned on by default.

Buffering by default breaks some 3rd-party Perl scripts using
cat-file, but this breakage was not detected anywhere in our
test suite.  Add a small Perl snippet to test this problem since
(AFAIK) other equivalent ways to test this behavior from Bourne
shell and/or awk would require racy sleeps, non-portable FIFOs
or tedious C code.

Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-20 10:28:46 -07:00
Peter Krefting 47d2691ae9 git-gui: sv.po: Update Swedish translation (576t0f0u)
Signed-off-by: Peter Krefting <peter@softwolves.pp.se>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
2024-06-20 19:27:35 +02:00
Kyle Zhao 2e5a636593 merge: avoid write merge state when unable to write index
Writing the merge state after the index write fails is meaningless and
could potentially cause Git to lose changes.

Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-18 08:13:35 -07:00
Junio C Hamano 66ac6e4bcd The fourteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-17 15:55:59 -07:00
Junio C Hamano 4216329457 Merge branch 'ps/no-writable-strings'
Building with "-Werror -Wwrite-strings" is now supported.

* ps/no-writable-strings: (27 commits)
  config.mak.dev: enable `-Wwrite-strings` warning
  builtin/merge: always store allocated strings in `pull_twohead`
  builtin/rebase: always store allocated string in `options.strategy`
  builtin/rebase: do not assign default backend to non-constant field
  imap-send: fix leaking memory in `imap_server_conf`
  imap-send: drop global `imap_server_conf` variable
  mailmap: always store allocated strings in mailmap blob
  revision: always store allocated strings in output encoding
  remote-curl: avoid assigning string constant to non-const variable
  send-pack: always allocate receive status
  parse-options: cast long name for OPTION_ALIAS
  http: do not assign string constant to non-const field
  compat/win32: fix const-correctness with string constants
  pretty: add casts for decoration option pointers
  object-file: make `buf` parameter of `index_mem()` a constant
  object-file: mark cached object buffers as const
  ident: add casts for fallback name and GECOS
  entry: refactor how we remove items for delayed checkouts
  line-log: always allocate the output prefix
  line-log: stop assigning string constant to file parent buffer
  ...
2024-06-17 15:55:58 -07:00
Junio C Hamano 72576d139d Merge branch 'jk/imap-send-plug-all-msgs-leak'
A leak in "git imap-send" that somehow escapes LSan has been
plugged.

* jk/imap-send-plug-all-msgs-leak:
  imap-send: free all_msgs strbuf in "out" label
2024-06-17 15:55:58 -07:00
Junio C Hamano 42b8b5bfd0 Merge branch 'jk/am-retry'
"git am" has a safety feature to prevent it from starting a new
session when there already is a session going.  It reliably
triggers when a mbox is given on the command line, but it has to
rely on the tty-ness of the standard input.  Add an explicit way to
opt out of this safety with a command line option.

* jk/am-retry:
  test-terminal: drop stdin handling
  am: add explicit "--retry" option
2024-06-17 15:55:56 -07:00