The `git fetch` and `git clone` commands have been optimized to
download packfile URIs in parallel when the new
`fetch.packfileURIThreads` configuration is set, significantly
speeding up fetches from servers that advertise multiple packfiles.
* ps/fetch-packfile-uris-parallel:
fetch-pack: allow parallelizing packfile URI fetches
fetch-pack: prepare for threaded fetching of packfile URIs
'git worktree add' did not prevent DWIM behavior when '-b' or '-B' was
specified, which has been corrected.
* yn/worktree-ambiguous-remote-advice:
worktree add: treat multiple matches with --guess-remote as an error
worktree add: improve message for ambiguous remote branch name
checkout: improve message for ambiguous remote branch name
checkout: extract function to display advice for ambiguous remotes
When 'git worktree add <path>' is invoked without <commit-ish> and
with the --guess-remote option (or when worktree.guessRemote is set to
true), it tries to find a remote-tracking branch matching the basename
of <path>.
Currently, the behavior when multiple matches are found is the same as
when no match is found: it falls back to creating a branch from
HEAD. This has been the behavior since 71d6682d8c (worktree: add
--guess-remote option to add subcommand, 2017-11-29), when the option
was first introduced.
However, if the specified <path> matches any remote-tracking branch,
we infer that the user intended to use one of the remote-tracking
branches as the start-point rather than HEAD. So we abort the creation
of the branch and worktree when there are multiple matches, and
instruct the user to choose the start-point.
Signed-off-by: Yoichi NAKAYAMA <yoichi.nakayama@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning from a server that supports packfile URIs we may see
multiple URIs being announced by the server. If so, the expectation is
that the client will download all of those packfiles. This is being done
sequentially, where we fetch one packfile after the other.
In many cases this should be fine, but there are scenarios where it's
not. When packfiles are for example hosted by object storage (think AWS
S3 or GCS) then the way to achieve high performance is typically to
parallelize downloading the data as a single connection is often capped
at a certain bandwidth. Furthermore, when the server announces a bunch
of smaller packfiles, then the overhead of establishing the connection
may eventually add up.
Despite the limitations caused by the network bandwidth and latency, Git
also runs git-index-pack(1) on all of the fetched packfiles. This is
another task that can be easily parallelized for another speedup.
All of these limitations can be addressed by parallelizing the fetch.
Introduce a new configuration option that allows the user to ask for
this: by default we continue to not parallelize the fetch to retain the
status quo. But when configured to 0 (where we auto-detect the number of
cores) or a value larger than 1 we perform the fetches concurrently.
With this infrastructure in place we can significantly speed up such
fetches. Using a local HTTP server demonstrates the speedup when using a
throttled connection of 2MB/s and downloading 8x1MB packfiles:
Benchmark 1: 2MB/s, 8x1MB packfiles, 1 threads
Time (mean ± σ): 4.321 s ± 0.003 s [User: 0.195 s, System: 0.113 s]
Range (min … max): 4.318 s … 4.325 s 5 runs
Benchmark 2: 2MB/s, 8x1MB packfiles, 2 threads
Time (mean ± σ): 2.284 s ± 0.241 s [User: 0.191 s, System: 0.114 s]
Range (min … max): 2.173 s … 2.714 s 5 runs
Benchmark 3: 2MB/s, 8x1MB packfiles, 4 threads
Time (mean ± σ): 1.212 s ± 0.238 s [User: 0.192 s, System: 0.105 s]
Range (min … max): 1.102 s … 1.638 s 5 runs
Benchmark 4: 2MB/s, 8x1MB packfiles, 8 threads
Time (mean ± σ): 569.9 ms ± 2.5 ms [User: 183.4 ms, System: 116.0 ms]
Range (min … max): 566.5 ms … 573.4 ms 5 runs
Summary
2MB/s, 8x1MB packfiles, 8 threads ran
2.13 ± 0.42 times faster than 2MB/s, 8x1MB packfiles, 4 threads
4.01 ± 0.42 times faster than 2MB/s, 8x1MB packfiles, 2 threads
7.58 ± 0.03 times faster than 2MB/s, 8x1MB packfiles, 1 threads
Quite unsurprisingly, we scale almost linearly with the number of
threads in this case as we're limited by the bandwidth of a single
connection. But we can also demonstrate a speedup on an unthrottled
connection when downloading slightly larger packfiles:
Benchmark 1: unthrottled, 8x16MB packfiles, 1 threads
Time (mean ± σ): 2.434 s ± 0.031 s [User: 2.067 s, System: 0.329 s]
Range (min … max): 2.381 s … 2.460 s 5 runs
Benchmark 2: unthrottled, 8x16MB packfiles, 2 threads
Time (mean ± σ): 1.353 s ± 0.129 s [User: 2.025 s, System: 0.328 s]
Range (min … max): 1.288 s … 1.583 s 5 runs
Benchmark 3: unthrottled, 8x16MB packfiles, 4 threads
Time (mean ± σ): 702.9 ms ± 23.9 ms [User: 1732.5 ms, System: 313.9 ms]
Range (min … max): 660.7 ms … 718.5 ms 5 runs
Benchmark 4: unthrottled, 8x16MB packfiles, 8 threads
Time (mean ± σ): 455.1 ms ± 7.7 ms [User: 1730.0 ms, System: 372.3 ms]
Range (min … max): 442.8 ms … 462.8 ms 5 runs
Summary
unthrottled, 8x16MB packfiles, 8 threads ran
1.54 ± 0.06 times faster than unthrottled, 8x16MB packfiles, 4 threads
2.97 ± 0.29 times faster than unthrottled, 8x16MB packfiles, 2 threads
5.35 ± 0.11 times faster than unthrottled, 8x16MB packfiles, 1 threads
In this case, the speedup is caused by us running git-index-pack(1) in
parallel. The improvement isn't linear, but still quite significant.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When performing auto-maintenance with geometric repacking we have two
conditions that may trigger a repack:
- Either the geometric sequence of packfiles is invalidated.
- Or we have too many loose objects.
The first condition shouldn't trigger all that often: it may be hit when
we fetch a new packfile, but users tend to not do that all the time. The
second condition is what typically triggers more regularly though, as
every command that ends up writing new objects may cause us to cross the
threshold of loose objects. It is thus preferable to not be too
aggressive here, as otherwise we may end up repacking objects quite
often.
For the geometric-repacking strategy though we have a default of 100
objects, only. As we're approximating the count of objects by only
reading the "objects/17/" shared, we'd only need 2 objects in there
before we perform a repack by default, which is quite aggressive.
git-gc(1) on the other hand has a default of 6700, so it is quite a bit
more conservative here.
Being this aggressive is also causing problems as reported by our users.
When running lots of concurrent writers, those writes will constantly
end up spawning maintenance jobs that end up repacking objects. As we
also prune objects, a concurrently running process that tries to write
an object may see that the sharding directories get removed under their
feet. While we try re-creating such leading directories, we only do so a
single time, and it may happen that the directory vanishes again before
we had the chance to create the loose object. This is not a new problem,
but it is exacerbated by us running maintenance this aggressively.
Improve the status quo by reducing the frequency at which we pack loose
objects to the same frequency that git-gc(1) uses.
Reported-by: Stefan Haller <lists@haller-berlin.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'git branch' command has been taught the '--delete-merged' option
to remove local branches that are already merged into their tracked
remote-tracking branches.
* hn/branch-delete-merged:
branch: add --dry-run for --delete-merged
branch: add branch.<name>.deleteMerged opt-out
branch: add --delete-merged <pattern>
branch: prepare delete_branches for a bulk caller
branch: let delete_branches skip unmerged branches on bulk refusal
branch: convert delete_branches() to a flags argument
branch: add --forked filter for --list mode
Documentation for 'git replay' has been updated to refer to its
configuration variables.
* kh/doc-replay-config:
doc: replay: move “default” to the right-hand side
doc: replay: use a nested description list
doc: replay: improve config description
doc: link to config for git-replay(1)
Synopsis and options in the documentation for 'git format-patch',
'git imap-send', 'git send-email', and 'git request-pull' have been
updated to the modern style.
* ja/doc-synopsis-style-yet-more:
doc: convert git-request-pull synopsis and options to new style
doc: convert git-send-email synopsis and options to new style
doc: convert git-format-patch synopsis and options to new style
doc: convert git-imap-send synopsis and options to new style
Setting branch.<name>.deleteMerged=false exempts that branch from
"git branch --delete-merged", which is useful for a topic you want
to keep developing after an early round of it has been merged
upstream. Unless --quiet is given, each skip is reported so the
user knows why their topic was kept.
Explicit deletion with "git branch -d" still uses the normal merge
check and ignores this setting.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the push remote is specified as a URL, the fetch refspec of a
uniquely matching configured remote is now used to find and update
the remote-tracking branch (e.g., '@{push}').
* hn/url-push-tracking:
remote: find tracking branches for URL push destinations
remote: pass repository to push tracking helper
Configuration file locking has been updated to retry for a short
period, avoiding failures when multiple processes attempt to update
the configuration simultaneously.
* jt/config-lock-timeout:
config: retry acquiring config.lock, configurable via core.configLockTimeout
This is now a description list (see previous commit) and parentheticals
like this do not go on the left-hand side. Moving it to the other side
makes it stand out just as much and is also more consistent with the
rest of the documentation.
Let’s also do the same for the `replay.refAction` description list.
That makes the two desc. lists identical in the first sentence. Let’s
add a comment about that for future editors.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
First of all, this unordered list for `replay.refAction` introduces
a term with a colon. This is exactly what a description list is,
structurally. Let’s be stylistically consistent and use the desc.
list markup construct. Let’s also drop the harmless but unneeded
indentation.
We can reuse the `::` delimiter since we use an open block.
But for consistency use the typical nested description list
delimiter, namely `;;`.
Second, let’s replace the inline-verbatim `git replay` with a link
to git-replay(1), since we are naming the command. But make that
conditional so that we avoid a self-link inside git-replay(1).[1]
† 1: See e.g. e7b3a768 (doc: git-init: rework config item
init.templateDir, 2024-03-10) for another example of
avoiding self-linking
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git log --graph' has been modified to visually distinguish parentless
'root' commits (and commits that become roots due to history
simplification) by indenting them, preventing them from appearing
falsely related to unrelated commits rendered immediately above them.
* ps/shift-root-in-graph:
graph: add --[no-]graph-indent and log.graphIndent
graph: move config reading into graph_read_config()
graph: wrap cascading commits after 4 columns
graph: indent visual root in graph
graph: add a 2 commit buffer for lookahead
revision: add next_commit_to_show()
lib-log-graph: move check_graph function
Replace [verse] with [synopsis] in the SYNOPSIS block and remove
single-quote formatting from the command name.
Backtick-quote all option terms in the OPTIONS section, convert
the standalone placeholder _<encoding>_ in prose, fix single-quoted
server names to backtick form, and convert mbox(5) to `mbox`(5).
Also update config/sendemail.adoc: backtick-quote all configuration
key terms and fix a bare sendemail.aliasesFile reference in prose.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert git-imap-send from [verse]/single-quote style to the modern
synopsis-block style:
- Replace [verse] with [synopsis] in SYNOPSIS block
- Backtick-quote all OPTIONS terms
- Backtick-quote all config keys in config/imap.adoc
- Backtick-quote bare config key references in prose
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git accepts a repository URL as branch.<name>.pushRemote and can push
to it. This branch setting takes precedence over remote.pushDefault.
A branch can be configured with a URL-valued pushRemote before any push
occurs. If the remotes are later rearranged with "git remote rename" and
"git remote add", the newly added remote may use that URL. The URL value
is unaffected by the rename and continues to take precedence over
remote.pushDefault. The URL and the remote then point to the same
repository, but Git does not connect them for tracking. Pushing works,
but @{push} cannot identify the remote's tracking branch. As a result,
"git status" cannot show the push branch, and an up-to-date push can
leave its tracking information stale.
When exactly one configured remote would push to the same URL, use that
remote for push tracking. Continue to push to the URL so the configured
remote's push settings do not change existing behavior. Keep the current
behavior when no remote matches or multiple remotes match.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some users may prefer to not have graph indentation.
Add "log.graphIndent" config variable to graph_read_config() to read the
default preference. By default is graph indentation is true.
Add --graph-indent and --no-graph-indent options to overwrite the
default preference.
Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When 'git push origin/main' or 'git branch origin main' is run, the
command is now recognized as a potential typo, and advice has been
added to offer a typo fix.
* hn/branch-push-slip-advice:
push: suggest <remote> <branch> for a slash slip
branch: suggest <remote>/<branch> on upstream slip
"git log --follow" has been updated to better handle non-linear
history, in which the path being tracked gets renamed differently in
multiple history lines.
* mv/log-follow-mergy:
log: improve --follow following renames for non-linear history
The `fetch.followRemoteHEAD` configuration variable has been added to
provide a default for the per-remote `remote.<name>.followRemoteHEAD`
setting.
* mh/fetch-follow-remote-head-config:
fetch: fixup a misaligned comment
fetch: add configuration variable fetch.followRemoteHEAD
fetch: refactor do_fetch handling of followRemoteHEAD
fetch: return 0 on known git_fetch_config
fetch: rename function report_set_head
t5510: cleanup remote in followRemoteHEAD dangling ref test
doc: explain fetchRemoteHEADWarn advice
fetch: fixup set_head advice for warn-if-not-branch
The handling of promisor-remote protocol capability has been updated
to allow the other side to add to the list of promisor remotes via the
'promisor.acceptFromServerURL' configuration variable.
* cc/promisor-auto-config-url-more:
doc: promisor: improve acceptFromServer entry
promisor-remote: auto-configure unknown remotes
promisor-remote: trust known remotes matching acceptFromServerUrl
promisor-remote: introduce promisor.acceptFromServerUrl
promisor-remote: add 'local_name' to 'struct promisor_info'
urlmatch: add url_normalize_pattern() helper
urlmatch: change 'allow_globs' arg to bool
t5710: simplify 'mkdir X' followed by 'git -C X init'
When pushing the 'main' branch to the remote 'origin', i.e.,
$ git push origin main
it is easy to mistakenly write
$ git push origin/main
That is parsed as the repository to push to, and since 'origin/main'
is neither a configured remote nor a path it dies with:
fatal: 'origin/main' does not appear to be a git repository
Often 'origin/main' does not exist as a repository, so the command
fails without doing any harm, but it gives no hint that a space was
meant instead of a slash and can leave the user puzzled.
When the argument is not an existing path or configured remote but
its part before the first slash names one, suggest the intended
'<remote> <branch>' form:
$ git push origin main
The suggestion is shown as advice so it can be silenced with
advice.pushRepoLooksLikeRef.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Have a repo with a subtree merge, do a 'git log --follow prefix/test.c',
the output only contains history in the outer repo, not commits that
were merged via a subtree merge.
What happens is that 'git log --follow' stores the followed path only in
opt->diffopt.pathspec, so in case the commit history is non-linear, and
multiple parents have renames to the followed path, then the end result
isn't really defined: the first commit that happens to be visited in one
of the parents update opt->diffopt.pathspec, and from that point, only
that updated path is visited.
Fix the problem by introducing a commit -> path map
(follow_pathspec_slab) that stores what will be a path to follow when
visiting that parent. At the top of log_tree_commit(), if the slab has
an entry for this commit, we replace opt->diffopt.pathspec with a path
from this entry, so the correct path is followed, even if an unrelated
sub-tree changed the path to be followed to something else. After
log_tree_diff() runs, we record each parent's path in the slab. As a
result, the walk order doesn't matter, which was exactly the source of
problems previously.
This helps with subtree merges (rename happens inside the merge commit),
but also fixes the general case when the rename happens in the history
of parents, not in the merge commit itself.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'fetch.followRemoteHEAD' is added as a generic setting used by all
remotes for which 'remote.<name>.followRemoteHEAD' is undefined. If
both variables are undefined, a builtin default of "create" is in
effect, matching the previous behavior.
As mentioned in the previous patch, 'fetch.followRemoteHEAD' supports
all of the values that its 'remote' counterpart does _except_
warn-if-not-$branch, due to its tighter coupling to individual remote
repositories.
This setting interacts with the do_fetch mechanism in the same way as
the previous does, but there are opportunities for improved
user-experience discussed in [1]. See the included NEEDSWORK comment as
well.
Documentation and advice messages for both of the followRemoteHEAD
variables are reworded to better capture the relationship between the
two.
The added tests assert feature parity between the two followRemoteHEAD
variables, as well as the fact that 'remote.<name>.followRemoteHEAD'
always supersedes this new configurable default.
[1]: https://lore.kernel.org/git/xmqqh5n213bw.fsf@gitster.g/
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Matt Hunter <m@lfurio.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the user sets 'remote.<name>.followRemoteHEAD' to
'warn[-if-not-$branch]', git-fetch will report when a fetched HEAD
disagrees with the locally-configured remote's HEAD. This additional
advice instructs the user how to deal with these warnings, but was
previously undocumented in git-config.
Signed-off-by: Matt Hunter <m@lfurio.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Various AsciiDoc markup fixes in 'git config' documentation and
related files to ensure lists and formatting are rendered correctly.
* ta/doc-config-adoc-fixes:
doc: git-config: escape erroneous highlight markup
doc: config/sideband: fix description list delimiter
doc: config: terminate runaway lists
There are many places in git-config(1) where paragraphs that should
logically come after a list are instead appended to the last item of
the list. This is a well-documented quirk of AsciiDoc, and can be
mitigated by enclosing the list in an open block:
--
* first item
* last item
--
+
New paragraph after the list.
Fix the issue accordingly.
Signed-off-by: Tuomas Ahola <taahol@utu.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation for `push.default = simple` has been clarified to
better explain its behavior, making it clear that it pushes the
current branch to a same-named branch on the remote, and detailing
the upstream requirements for centralized workflows.
* ib/doc-push-default-simple:
doc: clarify push.default=simple behavior
A batch of documentation pages has been updated to use the modern
synopsis style.
* ja/doc-synopsis-style-again:
doc: convert git-imap-send synopsis and options to new style
doc: convert git-apply synopsis and options to new style
doc: convert git-am synopsis and options to new style
doc: convert git-grep synopsis and options to new style
doc: git bisect: clarify the usage of the synopsis vs actual command
doc: convert git-bisect to synopsis style
Fix some typos and grammar errors in comments and documentation files.
Signed-off-by: Tuomas Ahola <taahol@utu.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The fsmonitor daemon has been implemented for Linux.
* pt/fsmonitor-linux:
fsmonitor: convert shown khash to strset in do_handle_client
fsmonitor: add tests for Linux
fsmonitor: add timeout to daemon stop command
fsmonitor: close inherited file descriptors and detach in daemon
run-command: add close_fd_above_stderr option
fsmonitor: implement filesystem change listener for Linux
fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c
fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c
fsmonitor: use pthread_cond_timedwait for cookie wait
compat/win32: add pthread_cond_timedwait
fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon
fsmonitor: fix khash memory leak in do_handle_client
t9210, t9211: disable GIT_TEST_SPLIT_INDEX for scalar clone tests
The entry for the `promisor.acceptFromServer` in
"Documentation/config/promisor.adoc" has a number of issues:
- it's not clear if new remotes and URLs can be created,
- it looks like a big block of text,
- it's not easy to see all the options,
- it's not easy to see which option is the default one,
- for "knownName", it says "advertised by the client" instead of
"advertised by the server",
- it doesn't refer to the new related `acceptFromServerUrl`
option.
Let's address all these issues by rewording large parts of it
and using bullet points for the different options.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previous commits have introduced the `promisor.acceptFromServerUrl`
config variable to allowlist some URLs advertised by a server through
the "promisor-remote" protocol capability.
However the new `promisor.acceptFromServerUrl` mechanism, like the old
`promisor.acceptFromServer` mechanism, still requires a remote to
already exist in the client's local configuration before it can be
accepted. This places a significant manual burden on users to
pre-configure these remotes, and creates friction for administrators
who have to troubleshoot or manually provision these setups for their
teams.
To eliminate this burden, let's automatically create a new `[remote]`
section in the client's config when a server advertises an unknown
remote whose URL matches a `promisor.acceptFromServerUrl` glob pattern.
Concretely, let's add four helpers:
- sanitize_remote_name(): turn an arbitrary URL-derived string into a
valid remote name by replacing non-alphanumeric characters,
collapsing runs of '-', and prepending "promisor-auto-".
- promisor_remote_name_from_url(): normalize the URL and extract
host+port+path to build a human-readable base name, then pass it
through sanitize_remote_name().
- configure_auto_promisor_remote(): write the remote.*.url,
remote.*.promisor and remote.*.advertisedAs keys to the repo
config.
- handle_matching_allowed_url(): pick the final name (user-supplied
alias or auto-generated), handle collisions by appending "-1",
"-2", etc., then call configure_auto_promisor_remote().
Let's also add should_accept_new_remote_url() which reuses the
url_matches_accept_list() helper introduced in a previous commit to
find a matching pattern, then delegates to handle_matching_allowed_url()
to create the remote.
And then let's call should_accept_new_remote_url() from the '!item'
(unknown remote) branch of should_accept_remote(), setting
`reload_config` so that the newly-written config is picked up.
Finally let's document all that by:
- expanding the `promisor.acceptFromServerUrl` entry to describe
auto-creation, the optional "name=" prefix syntax, the
"promisor-auto-*" generation rules, and numeric-suffix collision
handling, and by
- adding a "remote.<name>.advertisedAs" entry to "remote.adoc".
Also let's extend the precedence paragraph added by a previous commit
to mention this new acceptance path: until now, the only way for
`promisor.acceptFromServerUrl` to trigger acceptance was to allow
field updates for a known remote. With this commit, it can also trigger
auto-creation of a previously-unknown remote whose advertised URL
matches the allowlist.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A previous commit introduced the `promisor.acceptFromServerUrl` config
variable along with the machinery to parse and validate the URL glob
patterns and optional remote name prefixes it contains. However, these
URL patterns are not yet tied into the client's acceptance logic.
When a promisor remote is already configured locally, its fields (like
authentication tokens) may occasionally need to be refreshed by the
server. If `promisor.acceptFromServer` is set to the secure default
("None"), these updates are rejected, potentially causing future
fetches to fail.
To enable such targeted updates for trusted URLs, let's use the URL
patterns from `promisor.acceptFromServerUrl` as an additional URL
based allowlist.
Concretely, let's check the advertised URLs against the URL glob
patterns by introducing a new small helper function called
url_matches_accept_list(), which iterates over the glob patterns and
returns the first matching allowed_url entry (or NULL).
The URL matching is done component by component: scheme and port are
compared exactly, the host and path are matched with wildmatch().
Before matching, the advertised URL is passed through url_normalize()
so that case variations in the scheme/host, percent-encoding tricks,
and ".." path segments cannot bypass the allowlist.
The username and password components of the URL are intentionally
ignored during matching to allow servers to rotate them, though using
the 'token' field of the capability is preferred over embedding
credentials in the URL.
Let's then use this helper in should_accept_remote() so that a known
remote whose URL matches the allowlist is accepted.
To prepare for this new logic, let's also:
- Add an 'accept_urls' parameter to should_accept_remote().
- Replace the BUG() guard in the ACCEPT_KNOWN_URL case with an
explicit 'if (accept == ACCEPT_KNOWN_URL) return' and a new
BUG() guard in the ACCEPT_NONE case.
- Call accept_from_server_url() from filter_promisor_remote()
and relax its early return so that the function is entered when
`accept_urls` has entries even if `accept == ACCEPT_NONE`.
With this, many organizations may only need something like:
git config set --global \
promisor.acceptFromServerUrl "https://my-org.com/*"
to accept only their own remotes. And if they need to accept additional
remotes in some specific repos, they can also set:
git config set promisor.acceptFromServer knownUrl
and configure the additional remote manually only in the repos where
they are needed.
Let's then properly document `promisor.acceptFromServerUrl` in
"promisor.adoc" as an additive security allowlist for known remotes,
including the URL normalization behavior and the component-wise
matching, and let's mention it in "gitprotocol-v2.adoc".
Also let's clarify in the documentation how
`promisor.acceptFromServerUrl` interacts with
`promisor.acceptFromServer`:
- Precedence: when both options are set,
`promisor.acceptFromServerUrl` is consulted first. If a matching
pattern leads to acceptance, the remote is accepted regardless of
`promisor.acceptFromServer`. Otherwise the decision is left to
`promisor.acceptFromServer`.
- URL-mismatch guard: even when the advertised URL matches the
allowlist, an already-existing client-side remote whose configured
URL differs from the advertised one is not accepted through
`promisor.acceptFromServerUrl`. `promisor.acceptFromServer=all` and
`=knownName` keep their pre-existing, looser semantics.
The precedence paragraph is intentionally scoped here to known remotes
only (field updates). A following commit that introduces auto-creation
of unknown remotes will extend it to cover that case as well.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The repacking code has been refactored and compaction of MIDX layers
have been implemented, and incremental strategy that does not require
all-into-one repacking has been introduced.
* tb/incremental-midx-part-3.3:
repack: allow `--write-midx=incremental` without `--geometric`
repack: introduce `--write-midx=incremental`
repack: implement incremental MIDX repacking
packfile: ensure `close_pack_revindex()` frees in-memory revindex
builtin/repack.c: convert `--write-midx` to an `OPT_CALLBACK`
repack-geometry: prepare for incremental MIDX repacking
repack-midx: extract `repack_fill_midx_stdin_packs()`
repack-midx: factor out `repack_prepare_midx_command()`
midx: expose `midx_layer_contains_pack()`
repack: track the ODB source via existing_packs
midx: support custom `--base` for incremental MIDX writes
midx: introduce `--no-write-chain-file` for incremental MIDX writes
midx: use `strvec` for `keep_hashes`
midx: build `keep_hashes` array in order
midx: use `strset` for retained MIDX files
midx-write: handle noop writes when converting incremental chains
The negotiation tip options in "git fetch" have been reworked to
allow requiring certain refs to be sent as "have" lines, and to
restrict negotiation to a specific set of refs.
* ds/fetch-negotiation-options:
send-pack: pass negotiation config in push
remote: add remote.*.negotiationInclude config
fetch: add --negotiation-include option for negotiation
negotiator: add have_sent() interface
remote: add remote.*.negotiationRestrict config
transport: rename negotiation_tips
fetch: add --negotiation-restrict option
t5516: fix test order flakiness
The documentation for the 'simple' push mode currently singles out
the centralized workflow, which can cause confusion about its
behavior in other scenarios, such as triangular workflows.
Clarify that 'simple' always pushes the current branch to a branch
of the same name, but only enforces the strict upstream tracking
requirement when pushing back to the same remote being pulled from.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ivan Baluta <ivanbaluta.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert git-imap-send from [verse]/single-quote style to the modern
synopsis-block style:
- Replace [verse] with [synopsis] in SYNOPSIS block
- Backtick-quote all OPTIONS terms
- Backtick-quote all config keys in config/imap.adoc
- Backtick-quote bare config key references in prose
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert git-apply from [verse]/single-quote style to the modern
synopsis-block style:
- Replace [verse] with [synopsis] in SYNOPSIS block
- Backtick-quote all OPTIONS terms and config keys in config/apply.adoc
- Convert single-quoted inline commands ('git apply', 'diff', etc.)
- Wrap standalone placeholders in underscores (<n>, <root>, <action>)
- Backtick-quote `*.rej` and GNU `patch` tool references
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert git-am from [verse]/single-quote style to the modern
synopsis-block style:
- Replace [verse] with [synopsis] in SYNOPSIS block
- Backtick-quote all OPTIONS terms
- Convert inline man page refs
- Convert inline command refs
- Convert prose placeholders:
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Do not link to git-hook(1) from the config options when we already are
in that doc.
This implementation is similar to the updates to git-init(1) and
git-commit(1), implemented in [1] and [2], respectively.
† 1: e7b3a768 (doc: git-init: rework config item init.templateDir,
2024-03-10)
† 2: 819fdd6e (doc: convert git commit config to new format, 2025-01-15)
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When push.negotiate is enabled, 'git push' spawns a child 'git fetch
--negotiate-only' process to find common commits. Pass
--negotiation-include and --negotiation-restrict options from the
'remote.<name>.negotiationInclude' and
'remote.<name>.negotiationRestrict' config keys to this child process.
When negotiationRestrict is configured, it replaces the default
behavior of using all remote refs as negotiation tips. This allows
the user to control which local refs are used for push negotiation.
When negotiationInclude is configured, the specified ref patterns
are passed as --negotiation-include to ensure their tips are always
sent as 'have' lines during push negotiation.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a new 'remote.<name>.negotiationInclude' multi-valued config option that
provides default values for --negotiation-include when no
--negotiation-include arguments are specified over the command line. This
is a mirror of how 'remote.<name>.negotiationRestrict' specifies defaults
for the --negotiation-restrict arguments.
Each value is either an exact ref name or a glob pattern whose tips should
always be sent as 'have' lines during negotiation. The config values are
resolved through the same resolve_negotiation_include() codepath as the CLI
options.
This option is additive with the normal negotiation process: the negotiation
algorithm still runs and advertises its own selected commits, but the refs
matching the config are sent unconditionally on top of those heuristically
selected commits.
Similar to the negotiationRestrict config, an empty value resets the value
list to allow ignoring earlier config values, such as those that might be
set in system or global config.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a previous change, the --negotiation-restrict command-line option of 'git
fetch' was added as a synonym of --negotiation-tip. Both of these options
restrict the set of 'haves' the client can send as part of negotiation.
This was previously not available via a configuration option. Add a new
'remote.<name>.negotiationRestrict' multi-valued config option that updates
'git fetch <name>' to use these restrictions by default.
If the user provides even one --negotiation-restrict argument, then the
config is ignored.
An empty value resets the value list to allow ignoring earlier config
values, such as those that might be set in system or global config.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano