git-bundle first appeared in 2e0afafe ("Add git-bundle") in Feb 2007,
and first shipped in Git 1.5.1.
However, OFS_DELTA is an even earlier invention, coming about in
eb32d236 ("introduce delta objects with offset to base") in Sep 2006,
and first shipped in Git 1.4.4.5.
OFS_DELTA is smaller, about 3.2%-5% smaller, and is typically faster
to access than REF_DELTA because the exact location of the delta base
is available after parsing the object header. Since all bundle aware
versions of Git are also OFS_DELTA aware, just make it the default.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
bundle command silently died with no sign of failure if it
could not create the bundle file. (Eg.: its path resovles to a directory,
or the parent dir is sticky while file already exists and is owned
by someone else.)
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the --all-progress flag is used to use force progress display
during the writing object phase even if output goes to stdout which is
primarily the case during a push operation. This has the unfortunate
side effect of forcing progress display even if stderr is not a
terminal.
Let's introduce the --all-progress-implied argument which has the same
intent except for actually forcing the activation of any progress
display. With this, progress display will be automatically inhibited
whenever stderr is not a terminal, or full progress display will be
included otherwise. This should let people use 'git push' within a cron
job without filling their logs with useless percentage displays.
Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Tested-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the logic to read revs from standard input that rev-list knows about
from it to revision machinery, so that all the users of setup_revisions()
can feed the list of revs from the standard input when "--stdin" is used
on the command line.
Allow some users of the revision machinery that want different semantics
from the "--stdin" option to disable it by setting an option in the
rev_info structure.
This also cleans up the kludge made to bundle.c via cut and paste.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There were several unchecked use of fdopen(); replace them with xfdopen()
that checks and dies.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is convention that argv should be terminated with NULL, even if
argc is used to specify the size of argv. setup_revisions() requires
this and may segfault otherwise.
This patch makes sure that all argv (that I can find) is NULL terminated.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git bundle create x master master" used to create a bundle that lists
the same branch (master) twice. Cloning from such a bundle resulted in
a needless warning "warning: Duplicated ref: refs/remotes/origin/master".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With options such as "--all --since=2.weeks.ago", annotated tags used to
be included, when they should have been excluded. The reason is that we
heavily abuse the revision walker to determine what needs to be included
or excluded. And the revision walker does not show tags at all (and
therefore never marks tags as uninteresting).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These were found using gcc 4.3.2-1ubuntu11 with the warning:
warning: format not a string literal and no format arguments
Incorporated suggestions from Brandon Casey <casey@nrlssc.navy.mil>.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This changes the "die_on_error" boolean parameter to a mere "flags", and
changes the existing callers of hold_lock_file_for_update/append()
functions to pass LOCK_DIE_ON_ERROR.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch allows the caller to feed the revision parameters to git-bundle
from its standard input. This way, a script do not have to worry about
limitation of the length of command line.
Documentation/git-bundle.txt says that git-bundle takes arguments acceptable
to git-rev-list. Obviously some arguments that git-rev-list handles don't
make sense for git-bundle (e.g. --bisect) but --stdin is pretty reasonable.
Signed-off-by: Adam Brewster <adambrewster@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Callers of start_command() can set the members .in and .out of struct
child_process to a value > 0 to specify that this descriptor is used as
the stdin or stdout of the child process.
Previously, if start_command() was successful, this descriptor was closed
upon return. Here we now make sure that the descriptor is also closed in
case of failures. All callers are updated not to close the file descriptor
themselves after start_command() was called.
Note that earlier run_gpg_verify() of git-verify-tag set .out = 1, which
worked because start_command() treated this as a special case, but now
this is incorrect because it closes the descriptor. The intent here is to
inherit stdout to the child, which is achieved by .out = 0.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
By setting .in, .out, or .err members of struct child_process to -1, the
callers of start_command() can request that a pipe is allocated that talks
to the child process and one end is returned by replacing -1 with the
file descriptor.
Previously, a flag was set (for .in and .out, but not .err) to signal
finish_command() to close the pipe end that start_command() had handed out,
so it was optional for callers to close the pipe, and many already do so.
Now we make it mandatory to close the pipe.
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A failure in prepare_revision_walk can be caused by
a not parseable object.
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I noticed some unchecked writes. This fixes them.
* bundle.c (create_bundle): Die upon write failure.
* fast-import.c (keep_pack): Die upon write or close failure.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A NUL byte at beginning of file, or just after a newline
would provoke an invalid buf[-1] access in a few places.
* builtin-grep.c (cmd_grep): Don't access buf[-1].
* builtin-pack-objects.c (get_object_list): Likewise.
* builtin-rev-list.c (read_revisions_from_stdin): Likewise.
* bundle.c (read_bundle_header): Likewise.
* server-info.c (read_pack_info_file): Likewise.
* transport.c (insert_packed_refs): Likewise.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The transport specific stuff was moved into libgit.a, and the
bundle specific stuff will not be left behind.
This is a big code move, with one exception: the function
unbundle() no longer outputs the list of refs. You have to call
list_bundle_refs() yourself for that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This eliminates all use of byte-at-a-time reading of data in this
function: as Junio noted, a bundle file is seekable so we can
reset the file position to the first part of the pack-file using lseek
after reading the header.
Signed-off-by: Mark Levedahl <mdl123@verizon.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-bundle create on cygwin was nearly unusable due to 1 character
at a time (unbuffered) reading from an exec'ed process. Fix by using
fdopen to get a buffered stream.
Results for "time git bundle create test.bdl v1.0.3..v1.5.2" are:
before this patch:
cygwin linux
real 1m38.828s 0m3.578s
user 0m12.122s 0m2.896s
sys 1m28.215s 0m0.692s
after this patch:
real 0m3.688s 0m2.835s
user 0m3.075s 0m2.731s
sys 0m1.075s 0m0.149s
Signed-off-by: Mark Levedahl <mdl123@verizon.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Mark Levedahl <mlevedahl@gmail.com> writes:
> Junio C Hamano wrote:
>> While "git bundle" was a useful way to sneakernet incremental
>> changes, we did not allow:
>>
> Thanks - I've been thinking for months I could fix this bug, never
> figured it out and didn't want to nag Dscho one more time. I confirm
> that this allows creation of bundles with arbitrary refs, not just
> those under refs/heads. Yahoo!
Actually, there is another bug nearby.
If you do:
git bundle create v2.6-20-v2.6.22.bndl v2.6.20..v2.6.22
the bundle records that it requires v2.6.20^0 commit (correct)
and gives you tag v2.6.22 (incorrect); the bug is that the
object it lists in fact is the commit v2.6.22^0, not the tag.
This is because the revision range operation .. is always about
set of commits, but the code near where my patch touches does
not validate that the sha1 value obtained from dwim_ref()
against the commit object name e->item->sha1 before placing the
head information in the commit.
The attached patch attempts to fix this problem.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While "git bundle" was a useful way to sneakernet incremental
changes, we did not allow:
$ git bundle create v2.6.20.bndl v2.6.20
to create a bundle that contains the whole history to a
well-known good revision. Such a bundle can be mirrored
everywhere, and people can prime their repository with it to
reduce the load on the repository that serves near the tip of
the development.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make every builtin-*.c file #include "builtin.h".
Also takes care of some declaration/definition mismatches.
Signed-off-by: Peter Hagervall <hager@cs.umu.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch fixes all calls to xread() where the return value is not
stored into an ssize_t. The patch should not have any effect whatsoever,
other than putting better/more appropriate type names on variables.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This fixes a problem reported by Randal Schwartz:
>I finally tracked down all the (albeit inconsequential) errors I was getting
>on both OpenBSD and OSX. It's the warn() function in usage.c. There's
>warn(3) in BSD-style distros. It'd take a "great rename" to change it, but if
>someone with better C skills than I have could do that, my linker and I would
>appreciate it.
It was annoying to me, too, when I was doing some mergetool testing on
Mac OS X, so here's a fix.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: "Randal L. Schwartz" <merlyn@stonehenge.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Now that the run_command family supports all of the redirection
modes needed by builtin-bundle, we can use those functions rather
than the underlying POSIX primitives. This should help to make the
bundle command slightly more portable to other systems, like Windows.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
A use case for git-bundle expected to be quite common is this:
$ git bundle create daily.bundle --since=10.days.ago --all
The expected outcome is _not_ to error out if only a couple of the
refs were not changed during the last 10 days.
This patch complains loudly about refs which are skipped due to the
pack not containing the corresponding objects, but dies only if
no objects would be in the pack _at all_.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Not only does it prevent accidentally losing older bundles, but it
also fixes a subtle bug: when writing into an existing bundle,
git-pack-objects would not truncate the bundle. Therefore,
fetching from the bundle would trigger an error in unpack-objects:
"fatal: pack has junk at the end".
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The earlier patch tried to be nice by just warning, but it seems
more likely that the user wants to adjust the parameters.
Also, it prevents a bundle containing _all_ revisions in the case
when the user only gave one ref, but also rev-list options which
excluded the ref.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The patch to make the packs in a bundle thin forgot the receiving side.
D'oh.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Thin packs are way smaller, but they rely on the receiving end to have the
base objects. However, Git's pack protocol also uses thin packs by
default. So make the packs contained in bundles thin, since bundles are
just another transport.
The patch looks a bit bigger than intended, mainly because --thin
_implies_ that pack-objects should run its own rev-list. Therefore, this
patch removes all the stuff we used to roll rev-list ourselves.
This commit also changes behaviour slightly: since we now know early
enough if a specified ref is _not_ contained in the pack, we can avoid
putting that ref into the pack. So, we don't die() here, but warn()
instead, and skip that ref.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
When saying something like "--since=1.day.ago" or "--max-count=5",
git-bundle finds the boundary commits which are recorded as
prerequisites. However, it failed to tell pack-objects _not_ to
pack the objects which are in these.
Fix that. And add a test for that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If read_header() fails, it returns <0, not 0. Further, an open(/dev/null)
was not checked for errors.
Also, this adds two tests to make sure that the bundle file looks
correct, by checking if it has the header has the expected form, and that
the pack contains the right amount of objects.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
verify_bundle() returned with an error early only when all
prerequisite commits were missing. It should error out much
earlier when some are missing.
When the rev-list is limited in ways other than revision range
(e.g. --max-count or --max-age), create_bundle() listed all
positive refs given from the command line as if they are
available, but resulting pack may not have some of them. Add a
logic to make sure all of them are included, and error out
otherwise.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This removes the flag internally used by revision traversal to
decide which commits are indeed boundaries and renames it to
CHILD_SHOWN. builtin-bundle uses the symbol for its
verification, but I think the logic it uses it is wrong. The
flag is still useful but it is local to the git-bundle, so it is
renamed to PREREQ_MARK.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The handcrafted built-in rev-list lookalike forgot to mark the trees
and blobs contained in the boundary commits uninteresting, resulting
in unnecessary objects in the pack.
Signed-off-by: Junio C Hamano <junkio@cox.net>
We can use the revision walker easily for checking if the
prerequisites are met, instead of fork()ing off a rev-list,
which would list only the first unmet prerequisite.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This patch fixes issues mentioned by Junio, Nico and Simon:
- I forgot to convert the usage string when removing the "--" from
the subcommands,
- a style fix in the bundle_header,
- use xread() instead of read(),
- use write_or_die() instead of write(),
- make the bundle header extensible,
- fail if the whitespace after a sha1 of a reference is missing,
- close() the fds passed to a subprocess,
- in verify_bundle(), do not use "rev-list --stdin", but rather
pass the revs directly (avoiding a fork()),
- fix a corrupted comment in show_object(), and
- fix the size check in index_pack.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Some workflows require use of repositories on machines that cannot be
connected, preventing use of git-fetch / git-push to transport objects and
references between the repositories.
git-bundle provides an alternate transport mechanism, effectively allowing
git-fetch and git-pull to operate using sneakernet transport. `git-bundle
create` allows the user to create a bundle containing one or more branches
or tags, but with specified basis assumed to exist on the target
repository. At the receiving end, git-bundle acts like git-fetch-pack,
allowing the user to invoke git-fetch or git-pull using the bundle file as
the URL. git-fetch and git-ls-remote determine they have a bundle URL by
checking that the URL points to a file, but are otherwise unchanged in
operation with bundles.
The original patch was done by Mark Levedahl <mdl123@verizon.net>.
It was updated to make git-bundle a builtin, and get rid of the tar
format: now, the first line is supposed to say "# v2 git bundle", the next
lines either contain a prerequisite ("-" followed by the hash of the
needed commit), or a ref (the hash of a commit, followed by the name of
the ref), and finally the pack. As a result, the bundle argument can be
"-" now.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Shawn O. Pearce