Browse Source
* js/bundle: bundle: reword missing prerequisite error message git-bundle: record commit summary in the prerequisite data git-bundle: fix 'create --all' git-bundle: avoid fork() in verify_bundle() git-bundle: assorted fixes Add git-bundle: move objects and references by archivemaint
Junio C Hamano
18 years ago
12 changed files with 611 additions and 4 deletions
@ -0,0 +1,139 @@
@@ -0,0 +1,139 @@
|
||||
git-bundle(1) |
||||
============= |
||||
|
||||
NAME |
||||
---- |
||||
git-bundle - Move objects and refs by archive |
||||
|
||||
|
||||
SYNOPSIS |
||||
-------- |
||||
'git-bundle' create <file> [git-rev-list args] |
||||
'git-bundle' verify <file> |
||||
'git-bundle' list-heads <file> [refname...] |
||||
'git-bundle' unbundle <file> [refname...] |
||||
|
||||
DESCRIPTION |
||||
----------- |
||||
|
||||
Some workflows require that one or more branches of development on one |
||||
machine be replicated on another machine, but the two machines cannot |
||||
be directly connected so the interactive git protocols (git, ssh, |
||||
rsync, http) cannot be used. This command provides suport for |
||||
git-fetch and git-pull to operate by packaging objects and references |
||||
in an archive at the originating machine, then importing those into |
||||
another repository using gitlink:git-fetch[1] and gitlink:git-pull[1] |
||||
after moving the archive by some means (i.e., by sneakernet). As no |
||||
direct connection between repositories exists, the user must specify a |
||||
basis for the bundle that is held by the destination repository: the |
||||
bundle assumes that all objects in the basis are already in the |
||||
destination repository. |
||||
|
||||
OPTIONS |
||||
------- |
||||
|
||||
create <file>:: |
||||
Used to create a bundle named 'file'. This requires the |
||||
git-rev-list arguments to define the bundle contents. |
||||
|
||||
verify <file>:: |
||||
Used to check that a bundle file is valid and will apply |
||||
cleanly to the current repository. This includes checks on the |
||||
bundle format itself as well as checking that the prerequisite |
||||
commits exist and are fully linked in the current repository. |
||||
git-bundle prints a list of missing commits, if any, and exits |
||||
with non-zero status. |
||||
|
||||
list-heads <file>:: |
||||
Lists the references defined in the bundle. If followed by a |
||||
list of references, only references matching those given are |
||||
printed out. |
||||
|
||||
unbundle <file>:: |
||||
Passes the objects in the bundle to gitlink:git-index-pack[1] |
||||
for storage in the repository, then prints the names of all |
||||
defined references. If a reflist is given, only references |
||||
matching those in the given list are printed. This command is |
||||
really plumbing, intended to be called only by |
||||
gitlink:git-fetch[1]. |
||||
|
||||
[git-rev-list-args...]:: |
||||
A list of arguments, accepatble to git-rev-parse and |
||||
git-rev-list, that specify the specific objects and references |
||||
to transport. For example, "master~10..master" causes the |
||||
current master reference to be packaged along with all objects |
||||
added since its 10th ancestor commit. There is no explicit |
||||
limit to the number of references and objects that may be |
||||
packaged. |
||||
|
||||
|
||||
[refname...]:: |
||||
A list of references used to limit the references reported as |
||||
available. This is principally of use to git-fetch, which |
||||
expects to recieve only those references asked for and not |
||||
necessarily everything in the pack (in this case, git-bundle is |
||||
acting like gitlink:git-fetch-pack[1]). |
||||
|
||||
SPECIFYING REFERENCES |
||||
--------------------- |
||||
|
||||
git-bundle will only package references that are shown by |
||||
git-show-ref: this includes heads, tags, and remote heads. References |
||||
such as master~1 cannot be packaged, but are perfectly suitable for |
||||
defining the basis. More than one reference may be packaged, and more |
||||
than one basis can be specified. The objects packaged are those not |
||||
contained in the union of the given bases. Each basis can be |
||||
specified explicitly (e.g., ^master~10), or implicitly (e.g., |
||||
master~10..master, master --since=10.days.ago). |
||||
|
||||
It is very important that the basis used be held by the destination. |
||||
It is ok to err on the side of conservatism, causing the bundle file |
||||
to contain objects already in the destination as these are ignored |
||||
when unpacking at the destination. |
||||
|
||||
EXAMPLE |
||||
------- |
||||
|
||||
Assume two repositories exist as R1 on machine A, and R2 on machine B. |
||||
For whatever reason, direct connection between A and B is not allowed, |
||||
but we can move data from A to B via some mechanism (CD, email, etc). |
||||
We want to update R2 with developments made on branch master in R1. |
||||
We set a tag in R1 (lastR2bundle) after the previous such transport, |
||||
and move it afterwards to help build the bundle. |
||||
|
||||
in R1 on A: |
||||
$ git-bundle create mybundle master ^lastR2bundle |
||||
$ git tag -f lastR2bundle master |
||||
|
||||
(move mybundle from A to B by some mechanism) |
||||
|
||||
in R2 on B: |
||||
$ git-bundle verify mybundle |
||||
$ git-fetch mybundle refspec |
||||
|
||||
where refspec is refInBundle:localRef |
||||
|
||||
|
||||
Also, with something like this in your config: |
||||
|
||||
[remote "bundle"] |
||||
url = /home/me/tmp/file.bdl |
||||
fetch = refs/heads/*:refs/remotes/origin/* |
||||
|
||||
You can first sneakernet the bundle file to ~/tmp/file.bdl and |
||||
then these commands: |
||||
|
||||
$ git ls-remote bundle |
||||
$ git fetch bundle |
||||
$ git pull bundle |
||||
|
||||
would treat it as if it is talking with a remote side over the |
||||
network. |
||||
|
||||
Author |
||||
------ |
||||
Written by Mark Levedahl <mdl123@verizon.net> |
||||
|
||||
GIT |
||||
--- |
||||
Part of the gitlink:git[7] suite |
||||
@ -0,0 +1,424 @@
@@ -0,0 +1,424 @@
|
||||
#include "cache.h" |
||||
#include "object.h" |
||||
#include "commit.h" |
||||
#include "diff.h" |
||||
#include "revision.h" |
||||
#include "list-objects.h" |
||||
#include "exec_cmd.h" |
||||
|
||||
/* |
||||
* Basic handler for bundle files to connect repositories via sneakernet. |
||||
* Invocation must include action. |
||||
* This function can create a bundle or provide information on an existing |
||||
* bundle supporting git-fetch, git-pull, and git-ls-remote |
||||
*/ |
||||
|
||||
static const char *bundle_usage="git-bundle (create <bundle> <git-rev-list args> | verify <bundle> | list-heads <bundle> [refname]... | unbundle <bundle> [refname]... )"; |
||||
|
||||
static const char bundle_signature[] = "# v2 git bundle\n"; |
||||
|
||||
struct ref_list { |
||||
unsigned int nr, alloc; |
||||
struct ref_list_entry { |
||||
unsigned char sha1[20]; |
||||
char *name; |
||||
} *list; |
||||
}; |
||||
|
||||
static void add_to_ref_list(const unsigned char *sha1, const char *name, |
||||
struct ref_list *list) |
||||
{ |
||||
if (list->nr + 1 >= list->alloc) { |
||||
list->alloc = alloc_nr(list->nr + 1); |
||||
list->list = xrealloc(list->list, |
||||
list->alloc * sizeof(list->list[0])); |
||||
} |
||||
memcpy(list->list[list->nr].sha1, sha1, 20); |
||||
list->list[list->nr].name = xstrdup(name); |
||||
list->nr++; |
||||
} |
||||
|
||||
struct bundle_header { |
||||
struct ref_list prerequisites; |
||||
struct ref_list references; |
||||
}; |
||||
|
||||
/* this function returns the length of the string */ |
||||
static int read_string(int fd, char *buffer, int size) |
||||
{ |
||||
int i; |
||||
for (i = 0; i < size - 1; i++) { |
||||
int count = xread(fd, buffer + i, 1); |
||||
if (count < 0) |
||||
return error("Read error: %s", strerror(errno)); |
||||
if (count == 0) { |
||||
i--; |
||||
break; |
||||
} |
||||
if (buffer[i] == '\n') |
||||
break; |
||||
} |
||||
buffer[i + 1] = '\0'; |
||||
return i + 1; |
||||
} |
||||
|
||||
/* returns an fd */ |
||||
static int read_header(const char *path, struct bundle_header *header) { |
||||
char buffer[1024]; |
||||
int fd = open(path, O_RDONLY); |
||||
|
||||
if (fd < 0) |
||||
return error("could not open '%s'", path); |
||||
if (read_string(fd, buffer, sizeof(buffer)) < 0 || |
||||
strcmp(buffer, bundle_signature)) { |
||||
close(fd); |
||||
return error("'%s' does not look like a v2 bundle file", path); |
||||
} |
||||
while (read_string(fd, buffer, sizeof(buffer)) > 0 |
||||
&& buffer[0] != '\n') { |
||||
int is_prereq = buffer[0] == '-'; |
||||
int offset = is_prereq ? 1 : 0; |
||||
int len = strlen(buffer); |
||||
unsigned char sha1[20]; |
||||
struct ref_list *list = is_prereq ? &header->prerequisites |
||||
: &header->references; |
||||
char delim; |
||||
|
||||
if (buffer[len - 1] == '\n') |
||||
buffer[len - 1] = '\0'; |
||||
if (get_sha1_hex(buffer + offset, sha1)) { |
||||
warn("unrecognized header: %s", buffer); |
||||
continue; |
||||
} |
||||
delim = buffer[40 + offset]; |
||||
if (!isspace(delim) && (delim != '\0' || !is_prereq)) |
||||
die ("invalid header: %s", buffer); |
||||
add_to_ref_list(sha1, isspace(delim) ? |
||||
buffer + 41 + offset : "", list); |
||||
} |
||||
return fd; |
||||
} |
||||
|
||||
/* if in && *in >= 0, take that as input file descriptor instead */ |
||||
static int fork_with_pipe(const char **argv, int *in, int *out) |
||||
{ |
||||
int needs_in, needs_out; |
||||
int fdin[2], fdout[2], pid; |
||||
|
||||
needs_in = in && *in < 0; |
||||
if (needs_in) { |
||||
if (pipe(fdin) < 0) |
||||
return error("could not setup pipe"); |
||||
*in = fdin[1]; |
||||
} |
||||
|
||||
needs_out = out && *out < 0; |
||||
if (needs_out) { |
||||
if (pipe(fdout) < 0) |
||||
return error("could not setup pipe"); |
||||
*out = fdout[0]; |
||||
} |
||||
|
||||
if ((pid = fork()) < 0) { |
||||
if (needs_in) { |
||||
close(fdin[0]); |
||||
close(fdin[1]); |
||||
} |
||||
if (needs_out) { |
||||
close(fdout[0]); |
||||
close(fdout[1]); |
||||
} |
||||
return error("could not fork"); |
||||
} |
||||
if (!pid) { |
||||
if (needs_in) { |
||||
dup2(fdin[0], 0); |
||||
close(fdin[0]); |
||||
close(fdin[1]); |
||||
} else if (in) { |
||||
dup2(*in, 0); |
||||
close(*in); |
||||
} |
||||
if (needs_out) { |
||||
dup2(fdout[1], 1); |
||||
close(fdout[0]); |
||||
close(fdout[1]); |
||||
} else if (out) { |
||||
dup2(*out, 1); |
||||
close(*out); |
||||
} |
||||
exit(execv_git_cmd(argv)); |
||||
} |
||||
if (needs_in) |
||||
close(fdin[0]); |
||||
else if (in) |
||||
close(*in); |
||||
if (needs_out) |
||||
close(fdout[1]); |
||||
else if (out) |
||||
close(*out); |
||||
return pid; |
||||
} |
||||
|
||||
static int verify_bundle(struct bundle_header *header) |
||||
{ |
||||
/* |
||||
* Do fast check, then if any prereqs are missing then go line by line |
||||
* to be verbose about the errors |
||||
*/ |
||||
struct ref_list *p = &header->prerequisites; |
||||
struct rev_info revs; |
||||
const char *argv[] = {NULL, "--all"}; |
||||
struct object_array refs; |
||||
struct commit *commit; |
||||
int i, ret = 0, req_nr; |
||||
const char *message = "Repository lacks these prerequisite commits:"; |
||||
|
||||
init_revisions(&revs, NULL); |
||||
for (i = 0; i < p->nr; i++) { |
||||
struct ref_list_entry *e = p->list + i; |
||||
struct object *o = parse_object(e->sha1); |
||||
if (o) { |
||||
o->flags |= BOUNDARY_SHOW; |
||||
add_pending_object(&revs, o, e->name); |
||||
continue; |
||||
} |
||||
if (++ret == 1) |
||||
error(message); |
||||
error("%s %s", sha1_to_hex(e->sha1), e->name); |
||||
} |
||||
if (revs.pending.nr == 0) |
||||
return ret; |
||||
req_nr = revs.pending.nr; |
||||
setup_revisions(2, argv, &revs, NULL); |
||||
|
||||
memset(&refs, 0, sizeof(struct object_array)); |
||||
for (i = 0; i < revs.pending.nr; i++) { |
||||
struct object_array_entry *e = revs.pending.objects + i; |
||||
add_object_array(e->item, e->name, &refs); |
||||
} |
||||
|
||||
prepare_revision_walk(&revs); |
||||
|
||||
i = req_nr; |
||||
while (i && (commit = get_revision(&revs))) |
||||
if (commit->object.flags & BOUNDARY_SHOW) |
||||
i--; |
||||
|
||||
for (i = 0; i < req_nr; i++) |
||||
if (!(refs.objects[i].item->flags & SHOWN)) { |
||||
if (++ret == 1) |
||||
error(message); |
||||
error("%s %s", sha1_to_hex(refs.objects[i].item->sha1), |
||||
refs.objects[i].name); |
||||
} |
||||
|
||||
for (i = 0; i < refs.nr; i++) |
||||
clear_commit_marks((struct commit *)refs.objects[i].item, -1); |
||||
|
||||
return ret; |
||||
} |
||||
|
||||
static int list_heads(struct bundle_header *header, int argc, const char **argv) |
||||
{ |
||||
int i; |
||||
struct ref_list *r = &header->references; |
||||
|
||||
for (i = 0; i < r->nr; i++) { |
||||
if (argc > 1) { |
||||
int j; |
||||
for (j = 1; j < argc; j++) |
||||
if (!strcmp(r->list[i].name, argv[j])) |
||||
break; |
||||
if (j == argc) |
||||
continue; |
||||
} |
||||
printf("%s %s\n", sha1_to_hex(r->list[i].sha1), |
||||
r->list[i].name); |
||||
} |
||||
return 0; |
||||
} |
||||
|
||||
static void show_commit(struct commit *commit) |
||||
{ |
||||
write_or_die(1, sha1_to_hex(commit->object.sha1), 40); |
||||
write_or_die(1, "\n", 1); |
||||
if (commit->parents) { |
||||
free_commit_list(commit->parents); |
||||
commit->parents = NULL; |
||||
} |
||||
} |
||||
|
||||
static void show_object(struct object_array_entry *p) |
||||
{ |
||||
/* An object with name "foo\n0000000..." can be used to |
||||
* confuse downstream git-pack-objects very badly. |
||||
*/ |
||||
const char *ep = strchr(p->name, '\n'); |
||||
int len = ep ? ep - p->name : strlen(p->name); |
||||
write_or_die(1, sha1_to_hex(p->item->sha1), 40); |
||||
write_or_die(1, " ", 1); |
||||
if (len) |
||||
write_or_die(1, p->name, len); |
||||
write_or_die(1, "\n", 1); |
||||
} |
||||
|
||||
static int create_bundle(struct bundle_header *header, const char *path, |
||||
int argc, const char **argv) |
||||
{ |
||||
int bundle_fd = -1; |
||||
const char **argv_boundary = xmalloc((argc + 4) * sizeof(const char *)); |
||||
const char **argv_pack = xmalloc(4 * sizeof(const char *)); |
||||
int pid, in, out, i, status; |
||||
char buffer[1024]; |
||||
struct rev_info revs; |
||||
|
||||
bundle_fd = (!strcmp(path, "-") ? 1 : |
||||
open(path, O_CREAT | O_WRONLY, 0666)); |
||||
if (bundle_fd < 0) |
||||
return error("Could not write to '%s'", path); |
||||
|
||||
/* write signature */ |
||||
write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature)); |
||||
|
||||
/* write prerequisites */ |
||||
memcpy(argv_boundary + 3, argv + 1, argc * sizeof(const char *)); |
||||
argv_boundary[0] = "rev-list"; |
||||
argv_boundary[1] = "--boundary"; |
||||
argv_boundary[2] = "--pretty=oneline"; |
||||
argv_boundary[argc + 2] = NULL; |
||||
out = -1; |
||||
pid = fork_with_pipe(argv_boundary, NULL, &out); |
||||
if (pid < 0) |
||||
return -1; |
||||
while ((i = read_string(out, buffer, sizeof(buffer))) > 0) |
||||
if (buffer[0] == '-') |
||||
write_or_die(bundle_fd, buffer, i); |
||||
while ((i = waitpid(pid, &status, 0)) < 0) |
||||
if (errno != EINTR) |
||||
return error("rev-list died"); |
||||
if (!WIFEXITED(status) || WEXITSTATUS(status)) |
||||
return error("rev-list died %d", WEXITSTATUS(status)); |
||||
|
||||
/* write references */ |
||||
save_commit_buffer = 0; |
||||
init_revisions(&revs, NULL); |
||||
revs.tag_objects = 1; |
||||
revs.tree_objects = 1; |
||||
revs.blob_objects = 1; |
||||
argc = setup_revisions(argc, argv, &revs, NULL); |
||||
if (argc > 1) |
||||
return error("unrecognized argument: %s'", argv[1]); |
||||
for (i = 0; i < revs.pending.nr; i++) { |
||||
struct object_array_entry *e = revs.pending.objects + i; |
||||
if (!(e->item->flags & UNINTERESTING)) { |
||||
unsigned char sha1[20]; |
||||
char *ref; |
||||
if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1) |
||||
continue; |
||||
write_or_die(bundle_fd, sha1_to_hex(e->item->sha1), 40); |
||||
write_or_die(bundle_fd, " ", 1); |
||||
write_or_die(bundle_fd, ref, strlen(ref)); |
||||
write_or_die(bundle_fd, "\n", 1); |
||||
free(ref); |
||||
} |
||||
} |
||||
|
||||
/* end header */ |
||||
write_or_die(bundle_fd, "\n", 1); |
||||
|
||||
/* write pack */ |
||||
argv_pack[0] = "pack-objects"; |
||||
argv_pack[1] = "--all-progress"; |
||||
argv_pack[2] = "--stdout"; |
||||
argv_pack[3] = NULL; |
||||
in = -1; |
||||
out = bundle_fd; |
||||
pid = fork_with_pipe(argv_pack, &in, &out); |
||||
if (pid < 0) |
||||
return error("Could not spawn pack-objects"); |
||||
close(1); |
||||
dup2(in, 1); |
||||
close(in); |
||||
prepare_revision_walk(&revs); |
||||
traverse_commit_list(&revs, show_commit, show_object); |
||||
close(1); |
||||
while (waitpid(pid, &status, 0) < 0) |
||||
if (errno != EINTR) |
||||
return -1; |
||||
if (!WIFEXITED(status) || WEXITSTATUS(status)) |
||||
return error ("pack-objects died"); |
||||
return 0; |
||||
} |
||||
|
||||
static int unbundle(struct bundle_header *header, int bundle_fd, |
||||
int argc, const char **argv) |
||||
{ |
||||
const char *argv_index_pack[] = {"index-pack", "--stdin", NULL}; |
||||
int pid, status, dev_null; |
||||
|
||||
if (verify_bundle(header)) |
||||
return -1; |
||||
dev_null = open("/dev/null", O_WRONLY); |
||||
pid = fork_with_pipe(argv_index_pack, &bundle_fd, &dev_null); |
||||
if (pid < 0) |
||||
return error("Could not spawn index-pack"); |
||||
while (waitpid(pid, &status, 0) < 0) |
||||
if (errno != EINTR) |
||||
return error("index-pack died"); |
||||
if (!WIFEXITED(status) || WEXITSTATUS(status)) |
||||
return error("index-pack exited with status %d", |
||||
WEXITSTATUS(status)); |
||||
return list_heads(header, argc, argv); |
||||
} |
||||
|
||||
int cmd_bundle(int argc, const char **argv, const char *prefix) |
||||
{ |
||||
struct bundle_header header; |
||||
int nongit = 0; |
||||
const char *cmd, *bundle_file; |
||||
int bundle_fd = -1; |
||||
char buffer[PATH_MAX]; |
||||
|
||||
if (argc < 3) |
||||
usage(bundle_usage); |
||||
|
||||
cmd = argv[1]; |
||||
bundle_file = argv[2]; |
||||
argc -= 2; |
||||
argv += 2; |
||||
|
||||
prefix = setup_git_directory_gently(&nongit); |
||||
if (prefix && bundle_file[0] != '/') { |
||||
snprintf(buffer, sizeof(buffer), "%s/%s", prefix, bundle_file); |
||||
bundle_file = buffer; |
||||
} |
||||
|
||||
memset(&header, 0, sizeof(header)); |
||||
if (strcmp(cmd, "create") && |
||||
!(bundle_fd = read_header(bundle_file, &header))) |
||||
return 1; |
||||
|
||||
if (!strcmp(cmd, "verify")) { |
||||
close(bundle_fd); |
||||
if (verify_bundle(&header)) |
||||
return 1; |
||||
fprintf(stderr, "%s is okay\n", bundle_file); |
||||
return 0; |
||||
} |
||||
if (!strcmp(cmd, "list-heads")) { |
||||
close(bundle_fd); |
||||
return !!list_heads(&header, argc, argv); |
||||
} |
||||
if (!strcmp(cmd, "create")) { |
||||
if (nongit) |
||||
die("Need a repository to create a bundle."); |
||||
return !!create_bundle(&header, bundle_file, argc, argv); |
||||
} else if (!strcmp(cmd, "unbundle")) { |
||||
if (nongit) |
||||
die("Need a repository to unbundle."); |
||||
return !!unbundle(&header, bundle_fd, argc, argv); |
||||
} else |
||||
usage(bundle_usage); |
||||
} |
||||
|
||||
Loading…
Reference in new issue