test_when_finished has no effect in a subshell. Since the cmp_marks
function is only used once, inline it at its call site and move the
test_when_finished invocation to the start of the test.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A typical remote helper will return a `list` of refs containing a symbolic
ref HEAD, pointing to, e.g. refs/heads/master. In the case of a clone, all
the refs are being requested through `fetch` or `import`, including the
symbolic ref.
While this works properly, in some cases of a fetch, like `git fetch url`
or `git fetch origin HEAD`, or any fetch command involving a symbolic ref
without also fetching the corresponding ref it points to, the fetch command
fails with:
fatal: bad object 0000000000000000000000000000000000000000
error: <remote> did not send all necessary objects
(in the case the remote helper returned '?' values to the `list` command).
This is because there is only one ref given to fetch(), and it's not
further resolved to something at the end of fetch_with_import().
While this can be somehow handled in the remote helper itself, by adding
a refspec for the symbolic ref, and storing an explicit ref in a private
namespace, and then handling the `import` for that symbolic ref
specifically, very few existing remote helpers are actually doing that.
So, instead of requesting the exact list of wanted refs to remote helpers,
treat symbolic refs differently and request the ref they point to instead.
Then, resolve the symbolic refs values based on the pointed ref.
This assumes there is no more than one level of indirection (a symbolic
ref doesn't point to another symbolic ref).
Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of showing a warning and working as before, fail and show
the message and force immediate upgrade from their upstream
repositories when these tools are run, per request from their
primary author.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit d508e4a8e2,
reversing changes made to e42552135a.
The author of the original topic says he broke the upcoming 2.0
release with something that relates to "synchronization crash
regression" while refusing to give further specifics, so this would
unfortunately be the safest option for the upcoming release.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For remote-helpers that use 'export' to push.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
By using fast-export's new --refspec option.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 512477b (tests: use "env" to run commands with temporary env-var
settings) missed some variables in the remote-helpers test. Also
standardize these.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a remote helper crashes while pushing we should revert back to the
state before the push, however, it's possible that `git fast-export`
already finished its job, and therefore has exported the marks already.
This creates a synchronization problem because from that moment on
`git fast-{import,export}` will have marks that the remote helper is not
aware of and all further commands fail (if those marks are referenced).
The fix is to tell `git fast-export` to export to a temporary file, and
only after the remote helper has finishes successfully, move to the
final destination.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ordinarily, we would say "VAR=VAL command" to execute a tested
command with environment variable(s) set only for that command.
This however does not work if 'command' is a shell function (most
notably 'test_must_fail'); the result of the assignment is retained
and affects later commands.
To avoid this, we used to assign and export environment variables
and run such a test in a subshell, like so:
(
VAR=VAL && export VAR &&
test_must_fail git command to be tested
)
But with "env" utility, we should be able to say:
test_must_fail env VAR=VAL git command to be tested
which is much shorter and easier to read.
Signed-off-by: David Tran <unsignedzero@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Otherwise they cannot know when to force the push or not (other than
hacks).
Tests-by: Richard Hansen <rhansen@bbn.com>
Documentation-by: Richard Hansen <rhansen@bbn.com>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 664059fb (transport-helper: update remote helper namespace,
2013-04-17), a 'push' operation on a remote helper updates the
private ref by default. This is often a good thing, but it can also
be desirable to disable this update to force the next 'pull' to
re-import the pushed revisions.
Allow remote-helpers to disable the automatic update by introducing a new
capability.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Prior to commit 81d340d4, we did not print any error message
if a remote transport helper died unexpectedly. If a helper
did not print any error message (e.g., because it crashed),
the user could be left confused. That commit tried to
rectify the situation by printing a note that the helper
exited unexpectedly.
However, this makes a much more common case worse: when a
helper does die with a useful message, we print the extra
"Reading from 'git-remote-foo failed" message. This can also
end up confusing users, as they may not even know what
remote helpers are (e.g., the fact that http support comes
through git-remote-https is purely an implementation detail
that most users do not know or care about).
Since we do not have a good way of knowing whether the
helper printed a useful error, and since the common failure
mode is for it to do so, let's default to remaining quiet.
Debuggers can dig further by setting GIT_TRANSPORT_HELPER_DEBUG.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 664059f (transport-helper: update remote helper namespace)
updates the namespace when the push succeeds or not; we should do it
only when it succeeded.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If there's already a remote-helper tracking ref, we can fetch the
SHA-1 to report proper push messages (as opposed to always reporting
[new branch]).
The remote-helper currently can specify the old SHA-1 to avoid this
problem, but there's no point in forcing all remote-helpers to be aware
of git commit ids; they should be able to be agnostic of them.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Just like all the other shell scripts, replace the shebang line to
make sure it runs under the shell the user specified.
As this no longer depends on bashisms, t5801 does not have to say
bash must be available somewhere on the system.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is not a portable expectation that a single-shot environment
variable assignment works when calling a shell function, not a
command.
Set and export the variable before calling "test_must_fail git push"
instead. This change would not hurt because this is the last
command in the subprocess and the environment will not seep through
to later tests without using a single-shot assignment.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When pushing, the remote namespace is updated correctly
(e.g. refs/origin/master), but not the remote helper's
(e.g. refs/testgit/origin/master), which currently is only
updated while fetching.
Since the remote namespace is used to tell fast-export which commits
to avoid (because they were already imported/exported), it makes
sense to have them in sync so they don't get generated twice. If the
remote helper was implemented properly, they would be ignored, if
not, they probably would end up repeated.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For the modes that need it. In the future we should probably error out,
instead of providing half-assed support.
The reason we want to do this is because if it's not present, the remote
helper might be updating refs/heads/*, or refs/remotes/origin/*,
directly, and in the process fetch will get confused trying to update
refs that are already updated, or older than what they should be. We
shouldn't be messing with the rest of git.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This has never worked, since it's inception the code simply skips all
the refs, essentially telling fast-export to do nothing.
Let's at least tell the user what's going on.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The *:* refspec doesn't work, and never has, clarify the code and
documentation to reflect that. This in effect reverts commit 9e7673e
(gitremote-helpers(1): clarify refspec behaviour).
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows a remote helper using the 'export' protocol to specify that
it supports signed tags, changing the handing from 'warn-strip' to
'verbatim'.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, attempting to push a signed tag to a remote helper which uses
fast-export results in the remote helper failing because the default
fast-export action for signed tags is "abort". This is not helpful for
users because there is no way to pass additional arguments to
fast-export here, either from the remote helper or from the command
line.
In general, the signature will be invalidated by whatever transformation
a remote helper performs on a tag to push it to a repository in a
different format so the correct behaviour is to strip the tag. Doing
this silently may surprise people, so use "warn-strip" to issue a
warning when a signed tag is encountered.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If there's already a remote-helper tracking ref, we can fetch the SHA-1
to report proper push messages (as opposed to always reporting
[new branch]).
The remote-helper currently can specify the old SHA-1 to avoid this
problem, but there's no point in forcing all remote-helpers to be aware
of git commit ids; they should be able to be agnostic of them.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we try to read from a remote-helper and get EOF or an
error, we print a message indicating that the helper died.
However, users may not know that a remote helper was in use
(e.g., when using git-over-http), or even what a remote
helper is.
Let's print the name of the helper (e.g., "git-remote-https");
this makes it more obvious what the program is for, and
provides a useful token for reporting bugs or searching for
more information (e.g., in manpages).
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a push fails because the remote-helper died (with fast-export),
the user may not see any error message. We do correctly die with a
failed exit code, as we notice that the helper has died while
reading back the ref status from the helper. However, we don't print
any message. This is OK if the helper itself printed a useful error
message, but we cannot count on that; let's let the user know that
the helper failed.
In the long run, it may make more sense to propagate the error back
up to push, so that it can present the usual status table and give a
nicer message. But this is a much simpler fix that can help
immediately.
While we're adding tests, let's also confirm that the remote-helper
dying is also detected when importing refs. We currently do so
robustly when the helper uses the "done" feature (and that is what
we test). We cannot do so reliably when the helper does not use the
"done" feature, but it is not even worth testing; the right solution
is for the helper to start using "done".
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When an object has already been exported (and thus is in the marks) it's
flagged as SHOWN, so it will not be exported again, even if in a later
time it's exported through a different ref.
We don't need the object to be exported again, but we want the ref
updated, which doesn't happen.
Since we can't know if a ref was exported or not, let's just assume that
if the commit was marked (flags & SHOWN), the user still wants the ref
updated.
IOW: If it's specified in the command line, it will get updated,
regardless of whether or not the object was marked.
So:
% git branch test master
% git fast-export $mark_flags master
% git fast-export $mark_flags test
Would export 'test' properly.
Additionally, this fixes issues with remote helpers; now they can push
refs whose objects have already been exported, and a few other issues as
well. Update the tests accordingly.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
They have been marked as UNINTERESTING for a reason, lets respect
that. Currently the first ref is handled properly, but not the
rest. Assuming that all the refs point at the same commit in the
following example:
% git fast-export master ^uninteresting ^foo ^bar
reset refs/heads/bar
from :0
reset refs/heads/foo
from :0
reset refs/heads/uninteresting
from :0
% git fast-export ^uninteresting ^foo ^bar master
reset refs/heads/master
from :0
reset refs/heads/bar
from :0
reset refs/heads/foo
from :0
Clearly this is wrong; the negative refs should be ignored.
After this patch:
% git fast-export ^uninteresting ^foo ^bar master
# nothing
% git fast-export master ^uninteresting ^foo ^bar
# nothing
And even more, it would only happen if the ref is pointing to exactly
the same commit, but not otherwise:
% git fast-export ^next next
reset refs/heads/next
from :0
% git fast-export ^next next^{commit}
# nothing
% git fast-export ^next next~0
# nothing
% git fast-export ^next next~1
# nothing
% git fast-export ^next next~2
# nothing
The reason this happens is that before traversing the commits,
fast-export checks if any of the refs point to the same object, and any
duplicated ref gets added to a list in order to issue 'reset' commands
after the traversing. Unfortunately, it's not even checking if the
commit is flagged as UNINTERESTING. The fix of course, is to check it.
However, in order to do it properly we need to get the UNINTERESTING
flag from the command line, not from the commit object, because
"^foo bar" will mark the commit 'bar' uninteresting if foo and bar
points at the same commit. rev_cmdline_info, which was introduced
exactly to handle this situation, contains all the information we
need for get_tags_and_duplicates(), plus the ref flag. This way the
rest of the positive refs will remain untouched; it's only the
negative ones that change in behavior.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Unfortunately a lot of these tests fail.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We don't need a bare 'server' and an intermediary 'public'. The repos
can talk to each other directly; that's what we want to exercise.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This was only to cover a bug that was fixed in remote-testpy not to
resurface.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This only makes sense for the python remote helpers framework. The tests
don't exercise any feature of transport helper. Remove them.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Exercising the python remote helper framework is for another tool and
another test. This is about testing the remote-helper interface.
It's way simpler, it exercises the same features of remote helpers, it's
easy to read and understand, and it doesn't depend on python.
For now let's just copy the old remote-helpers test script, although
some of those tests don't make sense. In addition, this script would be
able to test other features not currently being tested.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This script is not really exercising the remote-helper functionality,
but more the python framework for remote helpers that live in
git_remote_helpers.
It's also not a good example of how to write remote-helpers, unless you
are planning to use python, and even then you might not want to use this
framework.
So let's use a more appropriate name: git-remote-testpy.
A patch that replaces git-remote-testgit with a simpler version is on
the way.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'feature done' was missing, which allowed fast-import exit properly, and
transport-helper to continue checking for refs and what not when in fact
the remote-helper died.
Let's enable that, and make sure the error paths are triggered.
Now transport-helper correctly detects the errors from fast-import,
unfortunately, not from fast-export because it might finish before
detecting a SIGPIPE. This means transport-helper will quit silently and
the user will not see any errors, which is bad. Hopefully the helper
will print the error before dying anyway, so not all is lost.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
These tests just want a bit-for-bit identical copy; they do not need
even -H (there is no symbolic link involved) nor -p (there is no
funny permission or ownership issues involved).
Just use "cp -R" instead.
Signed-off-by: Ben Walton <bdwalton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Test "pushing to local repo" in t5800-remote-helpers can hang
due to a race condition in git-remote-testgit. Fix it by
setting stdin to unbuffered.
On the writer side, "git push" invokes push_refs_with_export(),
which sends to stdout the command "export\n" and immediately
starts up "git fast-export". The latter writes its output stream
to the same stdout.
On the reader side, remote helper "git-remote-testgit" reads from
stdin to get its next command. It uses getc() to read characters
from libc up until \n. Libc has buffered a potentially much
larger chunk of stdin. When it sees the "export\n" command, it
forks "git fast-import" to read the stream.
If fast-export finishes before git fast-import starts, the
fast-export output can end up in libc's buffer in
git-remote-testgit, rather than in git fast-import. The latter
hangs indefinitely on a now-empty stdin.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the helper must somehow guess how many import statements to
read before it starts outputting its fast-export stream. This is
because the remote helper infrastructure runs fast-import only once,
so the helper is forced to output one stream for all import commands
it will receive. The only reason this worked in the past was because
only one ref was imported at a time.
Change the semantics of the import statement such that it matches
that of the push statement. That is, the import statement is followed
by a series of import statements that are terminated by a '\n'.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a remote helper exports to a non-local git repo, the
steps are roughly:
1. fast-export into a local staging area; the set of
interesting refs is defined by what is in the fast-export
stream
2. git push from the staging area to the non-local repo
In the second step, we should explicitly push all refs, not
just matching ones. This will let us push refs that do not
yet exist in the remote repo.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we want to push to a remote helper that has the
"export" capability, we collect all of the refs we want to
push and then feed them to fast-export.
However, the list of refs is actually a list of remote refs,
not local refs. The mapped local refs are included via the
peer_ref pointer. So when we add an argument to our
fast-export command line, we must be sure to use the local
peer_ref name (and if there is no local name, it is because
we are not actually sending that ref, or we may not even
have the ref at all).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Upon receiving an "import" command, the testgit remote
helper would ignore the ref asked for by git and generate a
fast-export stream based on HEAD. Instead, we should
actually give git the ref it asked for.
This requires adding a new parameter to the export_repo
method in the remote-helpers python library, which may be
used by code outside of git.git. We use a default parameter
so that callers without the new parameter will get the same
behavior as before.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These are all things one might expect to work in a helper
that is capable of handling multiple branches (which our
testgit helper in theory should be able to do, as it is
backed by git). All of these bugs are specific to the
import/export codepaths, so they don't affect helpers like
git-remote-curl that use fetch/push commands.
The first and fourth tests are about fetching and pushing
new refs, and demonstrate bugs in the git_remote_helpers
library (so they would be most likely to impact helpers for
other VCSs which import/export git).
The second test is about importing multiple refs; it
demonstrates a bug in git-remote-testgit, which is mostly
for exercising the test code. Therefore it probably doesn't
affect anyone in practice.
The third test demonstrates a bug in git's side of the
helper code when the upstream has added refs that we do not
have locally. This could impact git users who use remote
helpers to access foreign VCSs.
All of those bugs have fixes later in this series.
The fifth test is the most complex, and does not have a fix
in this series. It tests pushing a ref via the export
mechanism to a new name on the remote side (i.e.,
"git push $remote old:new").
The problem is that we push all of the work of generating
the export stream onto fast-export, but we have no way of
communicating to fast-export that this name mapping is
happening. So we tell fast-export to generate a stream with
the commits for "old", but we can't tell it to label them
all as "new".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All tests require python 2.4 or higher.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These are a little hard to read, and I'm about to add more
just like them. Plus the failure output is nicer if we use
test_cmp than a comparison with "test".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the t/t5800-remote-helpers.sh test to skip with the the
three-arg prereq form of test_expect_success instead of bailing out.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change tests to skip with skip_all=* + test_done instead of using say
+ test_done.
This is a follow-up to "tests: Skip tests in a way that makes sense
under TAP" (fadb5156e4). I missed these cases when prepearing that
patch, hopefully this is all of them.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The following python 2.5 features were worked around:
* the sha module is used as a fallback when the hashlib module is
not available
* the 'any' built-in method was replaced with a 'for' loop
* a conditional expression was replaced with an 'if' statement
* the subprocess.check_call method was replaced by a call to
subprocess.Popen followed by a call to subprocess.wait with a
check of its return status
These changes allow the python infrastructure to be used with python 2.4
which is distributed with RedHat's RHEL 5, for example.
t5800 was updated to check for python >= 2.4 to reflect these changes.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This test script depends on the git-remote-testgit python script. This
python script makes use of the hashlib module which was released in python
version 2.5. So, add a new pre-requisite named PYTHON_2_5_OR_NEWER to
test-lib.sh and check for it in t5800.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
John Keeping