Close FETCH_HEAD and release the string url even if we have to leave the
function store_updated_refs() early.
Reported-by: Chris Wilson <cwilson@vigilantsw.com>
Helped-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Extract the helper function and the type definition of the iterator
function it uses out of builtin/fetch.c into a separate source and a
header file.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We will be using the same "rev-list --verify-objects" logic to add a
sanity check to the receiving end of "git push" in the same way, but the
list of commits that are checked come from a structure with a different
shape over there.
Update the function to take an iterator to make it easier to reuse it in
different contexts.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "git fetch" command works in two phases. The remote side tells us what
objects are at the tip of the refs we are fetching from, and transfers the
objects missing from our side. After storing the objects in our repository,
we update our remote tracking branches to point at the updated tips of the
refs.
A broken or malicious remote side could send a perfectly well-formed pack
data during the object transfer phase, but there is no guarantee that the
given data actually fill the gap between the objects we originally had and
the refs we are updating to.
Although this kind of breakage can be caught by running fsck after a
fetch, it is much cheaper to verify that everything that is reachable from
the tips of the refs we fetched are indeed fully connected to the tips of
our current set of refs before we update them.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It makes no sense to do the - possibly very expensive - call to "rev-list
<new-ref-sha1> --not --all" in check_for_new_submodule_commits() when
there aren't any submodules configured.
Leave check_for_new_submodule_commits() early when no name <-> path
mappings for submodules are found in the configuration. To make that work
reading the configuration had to be moved further up in cmd_fetch(), as
doing that after the actual fetch of the superproject was too late.
Reported-by: Martin Fick <mfick@codeaurora.org>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "git fetch" command works in two phases. The remote side tells us what
objects are at the tip of the refs we are fetching from, and transfers the
objects missing from our side. After storing the objects in our repository,
we update our remote tracking branches to point at the updated tips of the
refs.
A broken or malicious remote side could send a perfectly well-formed pack
data during the object transfer phase, but there is no guarantee that the
given data actually fill the gap between the objects we originally had and
the refs we are updating to.
Although this kind of breakage can be caught by running fsck after a
fetch, it is much cheaper to verify that everything that is reachable from
the tips of the refs we fetched are indeed fully connected to the tips of
our current set of refs before we update them.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Split up the "(non-fast-forward)" message from printf directives and
make it translatable.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Translate a "[rejected]" message spotted by Jeff King, and other
things in update_local_ref along with it.
Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Translate some of the formatting messages that appear on git-fetch
showing how branches/tags etc. were updated.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Until now the --recurse-submodules option could only be used to either
fetch all populated submodules recursively or to disable recursion
completely. As fetch and pull now by default just fetch those submodules
for which new commits have been fetched in the superproject, a command
line option to enforce that behavior is needed to be able to override
configuration settings.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To be able to access all commits of populated submodules referenced by the
superproject it is sufficient to only then let "git fetch" recurse into a
submodule when the new commits fetched in the superproject record new
commits for it. Having these commits present is extremely useful when
using the "--submodule" option to "git diff" (which is what "git gui" and
"gitk" do since 1.6.6), as all submodule commits needed for creating a
descriptive output can be accessed. Also merging submodule commits (added
in 1.7.3) depends on the submodule commits in question being present to
work. Last but not least this enables disconnected operation when using
submodules, as all commits necessary for a successful "git submodule
update -N" will have been fetched automatically. So we choose this mode as
the default for fetch and pull.
Before a new or changed ref from upstream is updated in update_local_ref()
"git rev-list <new-sha1> --not --branches --remotes" is used to determine
all newly fetched commits. These are then walked and diffed against their
parent(s) to see if a submodule has been changed. If that is the case, its
path is stored to be fetched after the superproject fetch is completed.
Using the "--recurse-submodules" or the "--no-recurse-submodules" option
disables the examination of the fetched refs because the result will be
ignored anyway.
There is currently no infrastructure for storing deleted and new
submodules in the .git directory of the superproject. That's why fetch and
pull for now only fetch submodules that are already checked out and are
not renamed.
In t7403 the "--no-recurse-submodules" argument had to be added to "git
pull" to avoid failure because of the moved upstream submodule repo.
Thanks-to: Jonathan Nieder <jrnieder@gmail.com>
Thanks-to: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This shows a trace of all packets coming in or out of a given
program. This can help with debugging object negotiation or
other protocol issues.
To keep the code changes simple, we operate at the lowest
level, meaning we don't necessarily understand what's in the
packets. The one exception is a packet starting with "PACK",
which causes us to skip that packet and turn off tracing
(since the gigantic pack data will not be interesting to
read, at least not in the trace format).
We show both written and read packets. In the local case,
this may mean you will see packets twice (written by the
sender and read by the receiver). However, for cases where
the other end is remote, this allows you to see the full
conversation.
Packet tracing can be enabled with GIT_TRACE_PACKET=<foo>,
where <foo> takes the same arguments as GIT_TRACE.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add OPT__FORCE as a helper macro in the same spirit as OPT__VERBOSE
et.al. to simplify defining -f/--force options.
Signed-off-by: Rene Scharfe <rene.scharfe@lstfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This new boolean option can be used to override the default for "git
fetch" and "git pull", which is to not recurse into populated submodules
and fetch all new commits there too.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Until now you had to call "git submodule update" (without -N|--no-fetch
option) or something like "git submodule foreach git fetch" to fetch
new commits in populated submodules from their remote.
This could lead to "(commits not present)" messages in the output of
"git diff --submodule" (which is used by "git gui" and "gitk") after
fetching or pulling new commits in the superproject and is an obstacle for
implementing recursive checkout of submodules. Also "git submodule
update" cannot fetch changes when disconnected, so it was very easy to
forget to fetch the submodule changes before disconnecting only to
discover later that they are needed.
This patch adds the "--recurse-submodules" option to recursively fetch
each populated submodule from the url configured in the .git/config of the
submodule at the end of each "git fetch" or during "git pull" in the
superproject. The submodule paths are taken from the index.
The hidden option "--submodule-prefix" is added to "git fetch" to be able
to print out the full paths of nested submodules.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
(Just like we did for documentation already)
In the process, we change "non-remote branch" to "branch outside the
refs/remotes/ hierarchy" to avoid the ugly "non-remote-tracking branch".
The new formulation actually corresponds to how the code detects this
case (i.e. prefixcmp(refname, "refs/remotes")).
Also, we use 'remote-tracking branch' in generated merge messages (by
merge an fmt-merge-msg).
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
One more step towards consistancy. We change the documentation and the C
code in a single patch, since the only instances in the C code are in
comment and usage strings.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove some stray usage of other bracket types and asterisks for the
same purpose.
Signed-off-by: Štěpán Němec <stepnem@gmail.com>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Save future readers the trouble of tracing code to determine that the two
uses of branch->remote_name are safe when has_merge is set, by adding a
comment explaining that it is so.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When 'git fetch' is supplied a single argument, it tries to match it
against a configured remote and then fetch the refs specified by the
named remote's fetchspec. Additionally, or alternatively, if the current
branch has a merge ref configured, and if the name of the remote supplied
to fetch matches the one in the branch's configuration, then git also adds
the merge ref to the list of refs to update.
If the argument to fetch does not specify a named remote, or if the name
supplied does not match the remote configured for the current branch, then
the current branch's merge configuration should not be considered.
git currently mishandles the case when the argument to fetch specifies a
GIT URL(i.e. not a named remote) and the current branch has a configured
merge ref. In this case, fetch should ignore the branch's merge ref and
attempt to fetch from the remote repository's HEAD branch. But, since
fetch only checks _whether_ the current branch has a merge ref configured,
and does _not_ check whether the branch's configured remote matches the
command line argument (until later), it will mistakenly enter the wrong
branch of an 'if' statement and will not fall back to fetch the HEAD branch.
The fetch ends up doing nothing and returns with a successful zero status.
Fix this by comparing the remote repository's name to the branch's remote
name, in addition to whether it has a configured merge ref, sooner, so that
fetch can correctly decide whether the branch's configuration is interesting
or not, and fall back to fetching from the remote's HEAD branch when
appropriate.
This fixes the test in t5510.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Originally, if remote.<name>.tagopt was set, the --tags and option would
have no effect when given to git fetch. So if
tagopt="--no-tags"
git fetch --tags
would not actually fetch tags.
This patch changes this behavior to only follow what is written in the
config if there is no option passed by the command line.
Signed-off-by: Daniel Johnson <ComputerDruid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The message is especially confusing when "git fetch" is ran from "git
pull", for users not aware of "git fetch". The new message makes it clear
that "fetch" means "fetch new revisions", and gives hint on the solution.
We don't add a advice.* configuration option since this message doesn't
appear in normal use, and shouldn't disturb advanced users.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The rule for selecting the candidates for conversion is: if the callback
function returns only 0 (the condition for for_each_string_list to exit
early), than it can be safely converted to the macro.
A notable exception are the callers in builtin/remote.c. If converted, the
readability in the file will suffer greately. Besides, the code is not very
performance critical (at the moment, at least): it does output formatting of
the list of remotes.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update the definition and callers of string_list_append to use the
string_list as the first argument. This helps make the string_list
API easier to use by being more consistent.
Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update the definition and callers of string_list_lookup to use the
string_list as the first argument. This helps make the string_list
API easier to use by being more consistent.
Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update the definition and callers of string_list_insert to use the
string_list as the first argument. This helps make the string_list
API easier to use by being more consistent.
Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update the definition and callers of for_each_string_list to use the
string_list as the first argument. This helps make the string_list
API easier to use by being more consistent.
Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Unfortunately, there are still plenty of production systems with
vendor compilers that choke unless all compound declarations can be
determined statically at compile time, for example hpux10.20 (I can
provide a comprehensive list of our supported platforms that exhibit
this problem if necessary).
This patch simply breaks apart any compound declarations with dynamic
initialisation expressions, and moves the initialisation until after
the last declaration in the same block, in all the places necessary to
have the offending compilers accept the code.
Signed-off-by: Gary V. Vaughan <gary@thewrittenword.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This shrinks the top-level directory a bit, and makes it much more
pleasant to use auto-completion on the thing. Instead of
[torvalds@nehalem git]$ em buil<tab>
Display all 180 possibilities? (y or n)
[torvalds@nehalem git]$ em builtin-sh
builtin-shortlog.c builtin-show-branch.c builtin-show-ref.c
builtin-shortlog.o builtin-show-branch.o builtin-show-ref.o
[torvalds@nehalem git]$ em builtin-shor<tab>
builtin-shortlog.c builtin-shortlog.o
[torvalds@nehalem git]$ em builtin-shortlog.c
you get
[torvalds@nehalem git]$ em buil<tab> [type]
builtin/ builtin.h
[torvalds@nehalem git]$ em builtin [auto-completes to]
[torvalds@nehalem git]$ em builtin/sh<tab> [type]
shortlog.c shortlog.o show-branch.c show-branch.o show-ref.c show-ref.o
[torvalds@nehalem git]$ em builtin/sho [auto-completes to]
[torvalds@nehalem git]$ em builtin/shor<tab> [type]
shortlog.c shortlog.o
[torvalds@nehalem git]$ em builtin/shortlog.c
which doesn't seem all that different, but not having that annoying
break in "Display all 180 possibilities?" is quite a relief.
NOTE! If you do this in a clean tree (no object files etc), or using an
editor that has auto-completion rules that ignores '*.o' files, you
won't see that annoying 'Display all 180 possibilities?' message - it
will just show the choices instead. I think bash has some cut-off
around 100 choices or something.
So the reason I see this is that I'm using an odd editory, and thus
don't have the rules to cut down on auto-completion. But you can
simulate that by using 'ls' instead, or something similar.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The following functions are (almost) identical:
verify_remote_names
update_tracking_ref
refs_pushed
print_push_status
Move common versions of these functions to transport.c and rename
them, as suggested by Jeff King and Junio C Hamano.
These functions have been removed entirely from builtin-send-pack.c,
since they are only used internally by print_push_status():
print_ref_status
status_abbrev
print_ok_ref_status
print_one_push_status
Also, move #define SUMMARY_WIDTH to transport.h and rename it
TRANSPORT_SUMMARY_WIDTH as it is used in builtin-fetch.c and
transport.c
Signed-off-by: Michael Lukashov <michael.lukashov@gmail.com>
Acked-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For fetch and ls-remote, which use the first url of a remote, have
transport_get() determine this by passing a remote and passing NULL
for the url. For push, which uses every url of a remote, use each url
in turn if there are any, and use NULL if there are none.
This will allow the transport code to do something different if the
location is not specified with a url.
Also, have the message for a fetch say "foreign" if there is no url.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach fetch --dry-run as users of "git remote prune" switching to "git fetch
--prune" may expect it. Unfortunately OPT__DRY_RUN() cannot be used as fetch
already uses "-n" for something else.
Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: Björn Gustavsson <bgustavsson@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach fetch to cull stale remote tracking branches after fetching via --prune.
Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: Björn Gustavsson <bgustavsson@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Implement the configuration skipFetchAll option to allow
certain remotes to be skipped when doing 'git fetch --all' and
'git remote update'. The existing skipDefaultUpdate variable
is still honored (by 'git fetch --all' and 'git remote update').
(If both are set in the configuration file with different values,
the value of the last occurrence will be used.)
Signed-off-by: Björn Gustavsson <bgustavsson@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the --multiple option to specify that all arguments are either
groups or remotes. The primary reason for adding this option is
to allow us to re-implement 'git remote update' using fetch.
It would have been nice if this option was not needed, but since
the colon in a refspec is optional, it is in general not possible
to know whether a single, colon-less argument is a remote or a
refspec.
Signed-off-by: Björn Gustavsson <bgustavsson@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git remote' is meant for managing remotes and 'git fetch' is meant
for actually fetching data from remote repositories. Therefore, it is
not logical that you must use 'git remote update' to fetch from
more than one repository at once.
Add the --all option to 'git fetch', to tell it to attempt to fetch
from all remotes. Also, if --all is not given, the <repository>
argument is allowed to be the name of a group, to allow fetching
from all repositories in the group.
Other options except -v and -q are silently ignored.
Signed-off-by: Björn Gustavsson <bgustavsson@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Helpers might want a higher level of verbosity than just +1 (the
porcelain default setting) and +2 (-v -v). Expand the field to
allow verbosity in the range -1..3.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When there are large numbers of refs, calling read_ref for each ref is
inefficent (and infact downright slow) - so instead use for_each_ref
to build up a string list of all the refs that we currently have,
which significantly improves the volume.
Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When trying to get a list of remote tags to see if we need to fetch
any we were doing a linear search for the matching tag ref for the
tag^{} commit entries. This proves to be incredibly slow for large
numbers of tags. Rewrite the function so that we build up a
string_list of refs to fetch and then process that instead.
As an extreme example, for a repository with 50000 tags (and just a
single commit on a single branch), a fetch that does nothing goes from
~1m50s to ~4.1s.
Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 2d14d65 (Use a clearer style to issue commands to remote helpers,
2009-09-03) I happened to notice two changes like this:
- write_in_full(helper->in, "list\n", 5);
+
+ strbuf_addstr(&buf, "list\n");
+ write_in_full(helper->in, buf.buf, buf.len);
+ strbuf_reset(&buf);
IMHO, it would be better to define a new function,
static inline ssize_t write_str_in_full(int fd, const char *str)
{
return write_in_full(fd, str, strlen(str));
}
and then use it like this:
- strbuf_addstr(&buf, "list\n");
- write_in_full(helper->in, buf.buf, buf.len);
- strbuf_reset(&buf);
+ write_str_in_full(helper->in, "list\n");
Thus not requiring the added allocation, and still avoiding
the maintenance risk of literal string lengths.
These days, compilers are good enough that strlen("literal")
imposes no run-time cost.
Transformed via this:
perl -pi -e \
's/write_in_full\((.*?), (".*?"), \d+\)/write_str_in_full($1, $2)/'\
$(git grep -l 'write_in_full.*"')
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
quickfetch() calls rev-list to check whether the objects we are about to
fetch are already present in the repo (if so, we can skip the object fetch).
However, when there are many (~1000) refs to be fetched, the rev-list
command line grows larger than the maximum command line size on some systems
(32K in Windows). This causes rev-list to fail, making quickfetch() return
non-zero, which unnecessarily triggers the transport machinery. This somehow
causes fetch to fail with an exit code.
By using the --stdin option to rev-list (and feeding the object list to its
standard input), we prevent the overflow of the rev-list command line,
which causes quickfetch(), and subsequently the overall fetch, to succeed.
However, using rev-list --stdin is not entirely straightforward: rev-list
terminates immediately when encountering an unknown object, which can
trigger SIGPIPE if we are still writing object's to its standard input.
We therefore temporarily ignore SIGPIPE so that the fetch process is not
terminated.
The patch also contains a testcase to verify the fix (note that before
the patch, the testcase would only fail on msysGit).
Signed-off-by: Johan Herland <johan@herland.net>
Improved-by: Johannes Sixt <j6t@kdbg.org>
Improved-by: Alex Riesen <raa.lkml@gmail.com>
Tested-by: Peter Krefting <peter@softwolves.pp.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we fail to store a fetched ref, we recommend that the
user try running "git prune" to remove up any old refs that
have been deleted by the remote, which would clear up any DF
conflicts. However, ref storage might fail for other
reasons (e.g., permissions problems) in which case the
advice is useless and misleading.
This patch detects when there is an actual DF situation and
only issues the advice when one is found.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To give OPT_FILENAME the prefix, we pass the prefix to parse_options()
which passes the prefix to parse_options_start() which sets the prefix
member of parse_opts_ctx accordingly. If there isn't a prefix in the
calling context, passing NULL will suffice.
Signed-off-by: Stephen Boyd <bebarino@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In preparation to be used when the ref object is not available
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The fmt-merge-msg does a strong syntax checking of its input and fails
with if it is incorrect. The LF character is the only character
important for fmt-merge-msg. As the url in FETCH_HEAD plays only
informational role, a quoted representation of the url should be good
and true enough.
The url often comes from either user-editable config or command line,
so it is reasonable to expect all kinds of characters in it, including
the characters which the format of FETCH_HEAD considers special (line
separator in this case).
Noticed and reported by Hugo Mildenberger.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tay Ray Chuan