* 'master' of git://repo.or.cz/git/fastimport:
Allow fast-import frontends to reload the marks table
Use atomic updates to the fast-import mark file
Preallocate memory earlier in fast-import
I'm giving fast-import a lesson on how to reload the marks table
using the same format it outputs with --export-marks. This way
a frontend can reload the marks table from a prior import, making
incremental imports less painful.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When we allow fast-import frontends to reload a mark file from a
prior session we want to let them use the same file as they exported
the marks to. This makes it very simple for the frontend to save
state across incremental imports.
But we don't want to lose the old marks table if anything goes wrong
while writing our current marks table. So instead of truncating and
overwriting the path specified to --export-marks we use the standard
lockfile code to write the current marks out to a temporary file,
then rename it over the old marks table.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Bill Lear pointed out that it is easy to send out notifications of
changes with the update hook, but successful execution of the update
hook does not necessarily mean that the ref was actually updated.
Lock contention on the ref or being unable to append to the reflog
may prevent the ref from being changed. Sending out notifications
prior to the ref actually changing is very misleading.
To help this situation I am introducing two new hooks to the
receive-pack flow: pre-receive and post-receive. These new hooks
are invoked only once per receive-pack execution and are passed
three arguments per ref (refname, old-sha1, new-sha1).
The new post-receive hook is ideal for sending out notifications,
as it has the complete list of all refnames that were successfully
updated as well as the old and new SHA-1 values. This allows more
interesting notifications to be sent. Multiple ref updates could
be easily summarized into one email, for example.
The new pre-receive hook is ideal for logging update attempts, as it
is run only once for the entire receive-pack operation. It can also
be used to verify multiple updates happen at once, e.g. an update
to the `maint` head must also be accompained by a new annotated tag.
Lots of documentation improvements for receive-pack are included
in this change, as we want to make sure the new hooks are clearly
explained.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I discovered we did not send an ng line in the report-status feedback
if the ref was not updated because the repository has the config
option receive.denyNonFastForwards enabled. I think the reason this
happened is that it is simply too easy to forget to set error_string
when returning back a failure from update()
We now return an ng line for a non-fastforward update, which in
turn will cause send-pack to exit with a non-zero exit status.
Hence the modified test.
This refactoring changes update to return a const char* describing
the error, which execute_commands always loads into error_string.
The result is what I think is cleaner code, and allows us to
initialize the error_string member to NULL when we read_head_info.
I want error_string to be NULL in all commands before we call
execute_commands, so that we can reuse the run_hook function to
execute a new pre-receive hook.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a simple refactoring of run_update_hook to allow the function
to be passed the name of the hook it runs and also to build the
argument list from a list of struct commands, rather than just one
struct command.
The refactoring is to support new pre-receive and post-receive
hooks that will be given the entire list of struct commands,
rather than just one struct command. These new hooks will follow
in another patch.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
There is little point in executing the post-update hook if all refs
had an error and were unable to be updated. In this case nothing
new is reachable within the repository, and there is no state change
for the post-update hook to be interested in.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
As the post-update hook is meant to run after we have completed the
receipt of the pushed changes, and it might actually try to kick off
a `repack -a -d`, we should delay on invoking it until after we have
removed the *.keep file on the uploaded pack (if we kept the pack).
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I'm about to teach fast-import how to reload the marks file created
by a prior session. The general approach that I want to use is to
immediately parse the marks file when the specific argument is found
in argv, thereby allowing the caller to supply multiple marks files,
as the mark space can be sparsely populated.
To make that work out we need to allocate our object tables before
we parse the command line options. Since none of these tables
depend on the command line options, we can easily relocate them.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some systems have sizeof(off_t) == 8 while sizeof(size_t) == 4.
This implies that we are able to access and work on files whose
maximum length is around 2^63-1 bytes, but we can only malloc or
mmap somewhat less than 2^32-1 bytes of memory.
On such a system an implicit conversion of off_t to size_t can cause
the size_t to wrap, resulting in unexpected and exciting behavior.
Right now we are working around all gcc warnings generated by the
-Wshorten-64-to-32 option by passing the off_t through xsize_t().
In the future we should make xsize_t on such problematic platforms
detect the wrapping and die if such a file is accessed.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Always use an off_t value in pack-objects anytime we are dealing
with an offset to some data within a packfile.
Also fixed a minor uintmax_t that was incorrectly defined before.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Not all platforms have declared 'unsigned long' to be a 64 bit value,
but we want to support a 64 bit packfile (or close enough anyway)
in the near future as some projects are getting large enough that
their packed size exceeds 4 GiB.
By using off_t, the POSIX type that is declared to mean an offset
within a file, we support whatever maximum file size the underlying
operating system will handle. For most modern systems this is up
around 2^60 or higher.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
As we technically try to support up to a maximum of 2**32-1 objects
in a single packfile we should act like it and use unsigned 32 bit
integers for all of our object counts and progress output.
This change does not modify everything in pack-objects that probably
needs to change to fully support the maximum of 2**32-1 objects.
I'm intentionally breaking the improvements into slightly smaller
commits to make them easier to follow.
No logic change should be occuring here, with the exception that
some comparsions will now work properly when the number of objects
exceeds 2**31-1.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
As we permit up to 2^32-1 objects in a single packfile we cannot
use a signed int to represent the object offset within a packfile,
after 2^31-1 objects we will start seeing negative indexes and
error out or compute bad addresses within the mmap'd index.
This is a minor cleanup that does not introduce any significant
logic changes. It is roach free.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We shouldn't attempt to assign constant strings into char*, as the
string is not writable at runtime. Likewise we should always be
treating unsigned values as unsigned values, not as signed values.
Most of these are very straightforward. The only exception is the
(unnecessary) xstrdup/free in builtin-branch.c for the detached
head case. Since this is a user-level interactive type program
and that particular code path is executed no more than once, I feel
that the extra xstrdup call is well worth the easy elimination of
this warning.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If an index is corrupt, or is simply too new for us to understand,
we were leaking the mmap that held the entire content of the index.
This could be a considerable size on large projects, given that
the index is at least 24 bytes * nr_objects.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Because we are currently cheating and never supplying the delta base
for an OBJ_OFS_DELTA we get a random SHA-1 in the delta base field.
Instead lets clear the hash out so its at least all 0's. This is
somewhat more obvious that something fishy is going on, like we
don't actually have the SHA-1 of the base handy. :)
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This patch adds support to archimport for remapping the branch
names to match those used in git more closely. This is useful
for projects that migrate to git (as opposed to users that want
to use git on Arch-based projects). For example, one can choose
an Arch branch name and call it "master".
The new command-line syntax works even if there is a colon in
a branch name, since only the part after the last colon is taken
to be the git name (git does not allow colons in branch names).
The new feature is implemented so that archives rotated every
year can also be remapped into a single git archive.
Signed-off-by: Paolo Bonzini <bonzini@gnu.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
They test the behaviour with just a URL in the command line.
Signed-off-by: Santi B,Ai(Bjar <sbejar@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This failure to catch the failure of write_ref_sha1 was noticed
by Bill Lear. The ref will not update if the log file could not
be appended to (due to file permissions problems). Such a failure
should be flagged as a failure to update the ref, so that the client
knows the push did not succeed.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
There is no need to escape HTML tag's attributes in CGI.pm
HTML methods (like CGI::a()), because CGI.pm does attribute
escaping automatically.
$cgi->a({ ... -attribute => atribute_value }, tag_contents)
is translated to
<a ... attribute="attribute_value">tag_contents</a>
The rules for escaping attribute values (which are string contents) are
different. For example you have to take care about escaping embedded '"'
and "'" characters; CGI::a() does that for us automatically.
CGI::a() does not HTML escape tag_contents; we would need to write
<a href="URL">some <b>bold</b> text</a>
for example. So we use esc_html (or esc_path) to escape tag_contents
as needed.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The test was recently broken to expect sed to leave the
incomplete line at the end without newline.
POSIX says that output of the pattern space is to be followed by
a newline, while GNU adds the newline back only when it was
stripped when input. GNU behaviour is arguably more intuitive
and nicer, but we should not depend on it.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Change to use explicitly function call cgi->escapHTML().
This fix the problem on some systems that escapeHTML() is not
functioning, as default CGI is not setting 'escape' parameter.
Signed-off-by: Li Yang <leoli@freescale.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* maint:
git-gui: Make 'make' quieter by default
git-gui: Remove unnecessary /dev/null redirection.
git-gui: Don't create empty (same tree as parent) commits.
git-gui: Add Reset to the Branch menu.
git-gui: Relocate the menu/transport menu code.
* 'master' of git://repo.or.cz/git-gui:
git-gui: Make 'make' quieter by default
git-gui: Remove unnecessary /dev/null redirection.
git-gui: Don't create empty (same tree as parent) commits.
git-gui: Add Reset to the Branch menu.
git-gui: Relocate the menu/transport menu code.
The handcrafted built-in rev-list lookalike forgot to mark the trees
and blobs contained in the boundary commits uninteresting, resulting
in unnecessary objects in the pack.
Signed-off-by: Junio C Hamano <junkio@cox.net>
To fit nicely into the output of the git.git project's own quieter
Makefile, we want to make the git-gui Makefile nice and quiet too.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* 'master-for-junio' of git://repo.or.cz/git/fastimport:
fast-import: Fail if a non-existant commit is used for merge
fast-import: Avoid infinite loop after reset
* maint:
Fix diff-options references in git-diff and git-format-patch
Add definition of <commit-ish> to the main git man page.
Begin SubmittingPatches with a check list
fast-import: Fail if a non-existant commit is used for merge
fast-import: Avoid infinite loop after reset
Most of the git-diff-* documentation used [<common diff options>]
instead of [--diff-options], so make that change in git-diff and
git-format-patch.
In addition, git-format-patch didn't include the meanings of the diff
options.
Signed-off-by: Brian Gernhardt <benji@silverinsanity.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* 'maint-for-junio' of git://repo.or.cz/git/fastimport:
fast-import: Fail if a non-existant commit is used for merge
fast-import: Avoid infinite loop after reset
It seems that some people prefer a short list to a long text. But even for
the latter group, a quick reminder list is useful. So, add a check list to
Documentation/SubmittingPatches of what to do to get your patch accepted.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* maint:
fast-import: Fail if a non-existant commit is used for merge
fast-import: Avoid infinite loop after reset
[sp: Minor evil merge to deal with type_names array moving
to be private in 'master'.]
Johannes Sixt noticed during one of his own imports that fast-import
did not fail if a non-existant commit is referenced by SHA-1 value
as an argument to the 'merge' command. This allowed the user to
unknowingly create commits that would fail in fsck, as the commit
contents would not be completely reachable.
A side effect of this bug was that a frontend process could mark
any SHA-1 object (blob, tree, tag) as a parent of a merge commit.
This should also fail in fsck, as the commit is not a valid commit.
We now use the same rule as the 'from' command. If a commit is
referenced in the 'merge' command by hex formatted SHA-1 then the
SHA-1 must be a commit or a tag that can be peeled back to a commit,
the commit must already exist, and must be readable by the core Git
infrastructure code. This requirement means that the commit must
have existed prior to fast-import starting, or the commit must have
been flushed out by a prior 'checkpoint' command.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Johannes Sixt noticed that a 'reset' command applied to a branch that
is already active in the branch LRU cache can cause fast-import to
relink the same branch into the LRU cache twice. This will cause
the LRU cache to contain a cycle, making unload_one_branch run in an
infinite loop as it tries to select the oldest branch for eviction.
I have trivially fixed the problem by adding an active bit to
each branch object; this bit indicates if the branch is already
in the LRU and allows us to avoid trying to add it a second time.
Converting the pack_id field into a bitfield makes this change take
up no additional memory.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Add git-blame as a candidate to the byte-compilation.
batch-byte-compile is the prefered way to byte-compile files in
batch mode. Use it instead of the interactive function.
Signed-off-by: Xavier Maillard <zedek@gnu.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Update the main git.html page to point at 1.5.0.3 documentation.
Update draft 1.5.1 release notes with what we have so far.
Signed-off-by: Junio C Hamano <junkio@cox.net>
* 'js/fetch-progress' (early part):
Fixup no-progress for fetch & clone
fetch & clone: do not output progress when not on a tty
Conflicts:
git-fetch.sh
* js/symlink:
Tell multi-parent diff about core.symlinks.
Handle core.symlinks=false case in merge-recursive.
Add core.symlinks to mark filesystems that do not support symbolic links.
* maint:
GIT 1.5.0.3
glossary: Add definitions for dangling and unreachable objects
user-manual: more detailed merge discussion
user-manual: how to replace commits older than most recent
user-manual: insert earlier of mention content-addressable architecture
user-manual: ensure generated manual references stylesheet
user-manual: reset to ORIG_HEAD not HEAD to undo merge
Documentation: mention module option to git-cvsimport
Define "dangling" and "unreachable" objects. Modified from original
text proposed by Yasushi Shoji.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Add more details on conflict, including brief discussion of file stages.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
"Modifying" an old commit by checking it out, --amend'ing it, then
rebasing on top of it, is a slightly cumbersome technique, but I've
found it useful frequently enough to make it seem worth documenting.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Junio C Hamano