Many call-sites of parse_object assume that they will get a
non-NULL return value; this is not the case if we encounter
an error while parsing the object.
This patch adds a wrapper function around parse_object that
handles dying automatically, and uses it anywhere we
immediately try to access the return value as a non-NULL
pointer (i.e., anywhere that we would currently segfault).
This wrapper may also be useful in other places. The most
obvious one is code like:
o = parse_object(sha1);
if (!o)
die(...);
However, these should not be mechanically converted to
parse_object_or_die, as the die message is sometimes
customized. Later patches can address these sites on a
case-by-case basis.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These slightly improve the reading flow by making it obvious that a list
follows.
Also, make the wording of both headings consistent by changing "contains
%d ref(s)" to "contains this ref"/"contains these %d refs".
Signed-off-by: Lukas Fleischer <git@cryptocrack.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A more informative message for "complete" bundles was added in commit
8c3710fd30 (tweak "bundle verify" of a complete history, 2012-06-04).
However, the prerequisites ref list is currently read *after* we
check if it equals zero, which means we never actually use the
number of prerequisite refs to decide when to print the newly
introduced message. The code incorrectly uses the number of
references recorded in the bundle instead.
Signed-off-by: Lukas Fleischer <git@cryptocrack.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A bundle that records a complete history without prerequiste is a
useful way to sneakernet the sources of your configuration files
under your home directory, etc. E.g.
$ GIT_DIR=/srv/git/homesrc.git git bundle create x.bndl HEAD master
Running "git bundle verify" on such a "complete" bundle, however,
gives somewhat a funny output.
$ git bundle verify x.bndl
The bundle contains 2 refs
b2611f37ebc7ed6435a72d77fbc5f8b48a7d7146 HEAD
b2611f37ebc7ed6435a72d77fbc5f8b48a7d7146 refs/heads/master
The bundle requires these 0 refs
x.bndl is okay
Reword "requires these 0 refs" to say "The bundle records a complete
history" instead.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After running rev-list --boundary to retrieve the list of boundary
commits, "git bundle create" runs its own revision walk. If in this
stage git encounters an unfamiliar option, it writes a message with an
unbalanced quotation mark:
error: unrecognized argument: --foo'
Drop the stray quote to match the "unrecognized argument: %s" message
used elsewhere and save translators some work.
This is mostly a futureproofing measure: for now, the "rev-list
--boundary" command catches most strange arguments on its own and the
above message is not seen unless you try something esoteric like "git
bundle create test.bundle --header HEAD".
Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'name' field passed to add_pending_object() is used to later
deduplicate in object_array_remove_duplicates().
git-bundle had a bug in this area since 18449ab (git-bundle: avoid
packing objects which are in the prerequisites, 2007-03-08): it passed
the name of each boundary object in a static buffer. In other words,
all that object_array_remove_duplicates() saw was the name of the
*last* added boundary object.
The recent switch to a strbuf in bc2fed4 (bundle: use a strbuf to scan
the log for boundary commits, 2012-02-22) made this slightly worse: we
now free the buffer at the end, so it is not even guaranteed that it
still points into addressable memory by the time object_array_remove_
duplicates looks at it. On the plus side however, it was now
detectable by valgrind.
The fix is easy: pass a copy of the string to add_pending_object.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The first part of the bundle header contains the boundary commits, and
could be approximated by
# v2 git bundle
$(git rev-list --pretty=oneline --boundary <ARGS> | grep ^-)
git-bundle actually spawns exactly this rev-list invocation, and does
the grepping internally.
There was a subtle bug in the latter step: it used fgets() with a
1024-byte buffer. If the user has sufficiently long subjects (e.g.,
by not adhering to the git oneline-subject convention in the first
place), the 'oneline' format can easily overflow the buffer. fgets()
then returns the rest of the line in the next call(s). If one of
these remaining parts started with '-', git-bundle would mistakenly
insert it into the bundle thinking it was a boundary commit.
Fix it by using strbuf_getwholeline() instead, which handles arbitrary
line lengths correctly.
Note that on the receiving side in parse_bundle_header() we were
already using strbuf_getwholeline_fd(), so that part is safe.
Reported-by: Jannis Pohlmann <jannis.pohlmann@codethink.co.uk>
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The comment even said that it should eventually go there. While at
it, match the calling convention and name of the function to the
strbuf_get*line family. So it now is strbuf_getwholeline_fd.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-repo if interupted at the exact wrong time will generate zero
length bundles- literal empty files. git-repo is wrong here, but
git fetch shouldn't effectively spin loop if pointed at a zero
length bundle.
Signed-off-by: Brian Harring <ferringb@chromium.org>
Helped-by: Johannes Sixt
Helped-by: Nguyen Thai Ngoc Duy
Signed-off-by: Junio C Hamano <gitster@pobox.com>
resolve_ref() may return a pointer to a static buffer, which is not
safe for long-term use because if another resolve_ref() call happens,
the buffer may be changed. Many call sites though do not care about
this buffer. They simply check if the return value is NULL or not.
Convert all these call sites to new wrappers to reduce resolve_ref()
calls from 57 to 34. If we change resolve_ref() prototype later on
to avoid passing static buffer out, this helps reduce changes.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move most of the code from read_bundle_header() to parse_bundle_header()
that takes a file descriptor that is already opened for reading, and make
the former responsible only for opening the file and noticing errors.
As a logical consequence of this, is_bundle() helper function can be
implemented as a non-complaining variant of read_bundle_header() that
does not return an open file descriptor, and can be used to tighten
the check used to decide the use of bundle transport in transport_get()
function.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We wished that "git bundle" to eventually learn to read from a network
socket which is not seekable. The current code opens with fopen(), reads
the file halfway and run ftell(), and reopens the same file with open()
and seeks, to skip the header.
This patch by itself does not reach that goal yet, but I think it is a
right step in that direction.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Factor out the code to clear the commit marks for a whole struct
object_array from builtin/checkout.c into its own exported function
clear_commit_marks_for_object_array and use it in bisect and bundle
as well. It handles tags and commits and ignores objects of any
other type.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of creating a copy of the list of pending objects, copy the
struct object_array that points to it, turn on leak_pending, and thus
cause prepare_revision_walk to leave it to us. And free it once
we're done.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the usual "git" transport, a large-ish transfer with "git fetch" and
"git pull" give progress eye-candy to avoid boring users. However, not
when they are reading from a bundle. I.e.
$ git pull ../git-bundle.bndl master
This teaches bundle.c:unbundle() to give "-v" option to index-pack and
tell it to give progress bar when transport decides it is necessary.
The operation in the other direction, "git bundle create", could also
learn to honor --quiet but that is a separate issue.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-bundle first appeared in 2e0afafe ("Add git-bundle") in Feb 2007,
and first shipped in Git 1.5.1.
However, OFS_DELTA is an even earlier invention, coming about in
eb32d236 ("introduce delta objects with offset to base") in Sep 2006,
and first shipped in Git 1.4.4.5.
OFS_DELTA is smaller, about 3.2%-5% smaller, and is typically faster
to access than REF_DELTA because the exact location of the delta base
is available after parsing the object header. Since all bundle aware
versions of Git are also OFS_DELTA aware, just make it the default.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
bundle command silently died with no sign of failure if it
could not create the bundle file. (Eg.: its path resovles to a directory,
or the parent dir is sticky while file already exists and is owned
by someone else.)
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the --all-progress flag is used to use force progress display
during the writing object phase even if output goes to stdout which is
primarily the case during a push operation. This has the unfortunate
side effect of forcing progress display even if stderr is not a
terminal.
Let's introduce the --all-progress-implied argument which has the same
intent except for actually forcing the activation of any progress
display. With this, progress display will be automatically inhibited
whenever stderr is not a terminal, or full progress display will be
included otherwise. This should let people use 'git push' within a cron
job without filling their logs with useless percentage displays.
Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Tested-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the logic to read revs from standard input that rev-list knows about
from it to revision machinery, so that all the users of setup_revisions()
can feed the list of revs from the standard input when "--stdin" is used
on the command line.
Allow some users of the revision machinery that want different semantics
from the "--stdin" option to disable it by setting an option in the
rev_info structure.
This also cleans up the kludge made to bundle.c via cut and paste.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There were several unchecked use of fdopen(); replace them with xfdopen()
that checks and dies.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is convention that argv should be terminated with NULL, even if
argc is used to specify the size of argv. setup_revisions() requires
this and may segfault otherwise.
This patch makes sure that all argv (that I can find) is NULL terminated.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git bundle create x master master" used to create a bundle that lists
the same branch (master) twice. Cloning from such a bundle resulted in
a needless warning "warning: Duplicated ref: refs/remotes/origin/master".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With options such as "--all --since=2.weeks.ago", annotated tags used to
be included, when they should have been excluded. The reason is that we
heavily abuse the revision walker to determine what needs to be included
or excluded. And the revision walker does not show tags at all (and
therefore never marks tags as uninteresting).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These were found using gcc 4.3.2-1ubuntu11 with the warning:
warning: format not a string literal and no format arguments
Incorporated suggestions from Brandon Casey <casey@nrlssc.navy.mil>.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This changes the "die_on_error" boolean parameter to a mere "flags", and
changes the existing callers of hold_lock_file_for_update/append()
functions to pass LOCK_DIE_ON_ERROR.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch allows the caller to feed the revision parameters to git-bundle
from its standard input. This way, a script do not have to worry about
limitation of the length of command line.
Documentation/git-bundle.txt says that git-bundle takes arguments acceptable
to git-rev-list. Obviously some arguments that git-rev-list handles don't
make sense for git-bundle (e.g. --bisect) but --stdin is pretty reasonable.
Signed-off-by: Adam Brewster <adambrewster@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Callers of start_command() can set the members .in and .out of struct
child_process to a value > 0 to specify that this descriptor is used as
the stdin or stdout of the child process.
Previously, if start_command() was successful, this descriptor was closed
upon return. Here we now make sure that the descriptor is also closed in
case of failures. All callers are updated not to close the file descriptor
themselves after start_command() was called.
Note that earlier run_gpg_verify() of git-verify-tag set .out = 1, which
worked because start_command() treated this as a special case, but now
this is incorrect because it closes the descriptor. The intent here is to
inherit stdout to the child, which is achieved by .out = 0.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
By setting .in, .out, or .err members of struct child_process to -1, the
callers of start_command() can request that a pipe is allocated that talks
to the child process and one end is returned by replacing -1 with the
file descriptor.
Previously, a flag was set (for .in and .out, but not .err) to signal
finish_command() to close the pipe end that start_command() had handed out,
so it was optional for callers to close the pipe, and many already do so.
Now we make it mandatory to close the pipe.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A failure in prepare_revision_walk can be caused by
a not parseable object.
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I noticed some unchecked writes. This fixes them.
* bundle.c (create_bundle): Die upon write failure.
* fast-import.c (keep_pack): Die upon write or close failure.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A NUL byte at beginning of file, or just after a newline
would provoke an invalid buf[-1] access in a few places.
* builtin-grep.c (cmd_grep): Don't access buf[-1].
* builtin-pack-objects.c (get_object_list): Likewise.
* builtin-rev-list.c (read_revisions_from_stdin): Likewise.
* bundle.c (read_bundle_header): Likewise.
* server-info.c (read_pack_info_file): Likewise.
* transport.c (insert_packed_refs): Likewise.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The transport specific stuff was moved into libgit.a, and the
bundle specific stuff will not be left behind.
This is a big code move, with one exception: the function
unbundle() no longer outputs the list of refs. You have to call
list_bundle_refs() yourself for that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This eliminates all use of byte-at-a-time reading of data in this
function: as Junio noted, a bundle file is seekable so we can
reset the file position to the first part of the pack-file using lseek
after reading the header.
Signed-off-by: Mark Levedahl <mdl123@verizon.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-bundle create on cygwin was nearly unusable due to 1 character
at a time (unbuffered) reading from an exec'ed process. Fix by using
fdopen to get a buffered stream.
Results for "time git bundle create test.bdl v1.0.3..v1.5.2" are:
before this patch:
cygwin linux
real 1m38.828s 0m3.578s
user 0m12.122s 0m2.896s
sys 1m28.215s 0m0.692s
after this patch:
real 0m3.688s 0m2.835s
user 0m3.075s 0m2.731s
sys 0m1.075s 0m0.149s
Signed-off-by: Mark Levedahl <mdl123@verizon.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Mark Levedahl <mlevedahl@gmail.com> writes:
> Junio C Hamano wrote:
>> While "git bundle" was a useful way to sneakernet incremental
>> changes, we did not allow:
>>
> Thanks - I've been thinking for months I could fix this bug, never
> figured it out and didn't want to nag Dscho one more time. I confirm
> that this allows creation of bundles with arbitrary refs, not just
> those under refs/heads. Yahoo!
Actually, there is another bug nearby.
If you do:
git bundle create v2.6-20-v2.6.22.bndl v2.6.20..v2.6.22
the bundle records that it requires v2.6.20^0 commit (correct)
and gives you tag v2.6.22 (incorrect); the bug is that the
object it lists in fact is the commit v2.6.22^0, not the tag.
This is because the revision range operation .. is always about
set of commits, but the code near where my patch touches does
not validate that the sha1 value obtained from dwim_ref()
against the commit object name e->item->sha1 before placing the
head information in the commit.
The attached patch attempts to fix this problem.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While "git bundle" was a useful way to sneakernet incremental
changes, we did not allow:
$ git bundle create v2.6.20.bndl v2.6.20
to create a bundle that contains the whole history to a
well-known good revision. Such a bundle can be mirrored
everywhere, and people can prime their repository with it to
reduce the load on the repository that serves near the tip of
the development.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make every builtin-*.c file #include "builtin.h".
Also takes care of some declaration/definition mismatches.
Signed-off-by: Peter Hagervall <hager@cs.umu.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch fixes all calls to xread() where the return value is not
stored into an ssize_t. The patch should not have any effect whatsoever,
other than putting better/more appropriate type names on variables.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This fixes a problem reported by Randal Schwartz:
>I finally tracked down all the (albeit inconsequential) errors I was getting
>on both OpenBSD and OSX. It's the warn() function in usage.c. There's
>warn(3) in BSD-style distros. It'd take a "great rename" to change it, but if
>someone with better C skills than I have could do that, my linker and I would
>appreciate it.
It was annoying to me, too, when I was doing some mergetool testing on
Mac OS X, so here's a fix.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: "Randal L. Schwartz" <merlyn@stonehenge.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Now that the run_command family supports all of the redirection
modes needed by builtin-bundle, we can use those functions rather
than the underlying POSIX primitives. This should help to make the
bundle command slightly more portable to other systems, like Windows.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
A use case for git-bundle expected to be quite common is this:
$ git bundle create daily.bundle --since=10.days.ago --all
The expected outcome is _not_ to error out if only a couple of the
refs were not changed during the last 10 days.
This patch complains loudly about refs which are skipped due to the
pack not containing the corresponding objects, but dies only if
no objects would be in the pack _at all_.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Not only does it prevent accidentally losing older bundles, but it
also fixes a subtle bug: when writing into an existing bundle,
git-pack-objects would not truncate the bundle. Therefore,
fetching from the bundle would trigger an error in unpack-objects:
"fatal: pack has junk at the end".
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The earlier patch tried to be nice by just warning, but it seems
more likely that the user wants to adjust the parameters.
Also, it prevents a bundle containing _all_ revisions in the case
when the user only gave one ref, but also rev-list options which
excluded the ref.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The patch to make the packs in a bundle thin forgot the receiving side.
D'oh.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Thin packs are way smaller, but they rely on the receiving end to have the
base objects. However, Git's pack protocol also uses thin packs by
default. So make the packs contained in bundles thin, since bundles are
just another transport.
The patch looks a bit bigger than intended, mainly because --thin
_implies_ that pack-objects should run its own rev-list. Therefore, this
patch removes all the stuff we used to roll rev-list ourselves.
This commit also changes behaviour slightly: since we now know early
enough if a specified ref is _not_ contained in the pack, we can avoid
putting that ref into the pack. So, we don't die() here, but warn()
instead, and skip that ref.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
When saying something like "--since=1.day.ago" or "--max-count=5",
git-bundle finds the boundary commits which are recorded as
prerequisites. However, it failed to tell pack-objects _not_ to
pack the objects which are in these.
Fix that. And add a test for that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Jeff King