The current progress code really just counts commits.
This patch makes it count all objects, giving us a "total"
count close to what a repack would show. This is nice when
using "git gc", which will usually have just repacked the
whole repo.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
prune already shows progress meter while pruning. The marking part may
take a few seconds or more, depending on repository size. Show
progress meter during this time too.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The name of the processed object was duplicated for passing it to
add_object(), but that already calls path_name, which allocates a new
string anyway. So the memory allocated by the xstrdup calls just went
nowhere, leaking memory.
This reduces the RSS usage for a "rev-list --all --objects" by about 10% on
the gentoo repo (fully packed) as well as linux-2.6.git:
gentoo:
| old | new
----------------|-------------------------------
RSS | 1537284 | 1388408
VSZ | 1816852 | 1667952
time elapsed | 1:49.62 | 1:48.99
min. page faults| 417178 | 379919
linux-2.6.git:
| old | new
----------------|-------------------------------
RSS | 324452 | 292996
VSZ | 491792 | 460376
time elapsed | 0:14.53 | 0:14.28
min. page faults| 89360 | 81613
Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As these functions are directly called with the result
from lookup_tree/blob, they must handle NULL.
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A failure in prepare_revision_walk can be caused by
a not parseable object.
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This converts the index explicitly on read and write to its on-disk
format, allowing the in-core format to contain more flags, and be
simpler.
In particular, the in-core format is now host-endian (as opposed to the
on-disk one that is network endian in order to be able to be shared
across machines) and as a result we can dispense with all the
htonl/ntohl on accesses to the cache_entry fields.
This will make it easier to make use of various temporary flags that do
not exist in the on-disk format.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
I ran git-prune on a repository and got this:
$ git-prune
error: Object 228f8065b930120e35fc0c154c237487ab02d64a is a blob, not a commit
Segmentation fault (core dumped)
This repository was a strange one in that it was being used to provide
its own submodule. That is, the repository was cloned into a
subdirectory, an independent branch checked out in that subdirectory,
and then it was marked as a submodule. git-prune then failed in the
above manner.
The problem was that git-prune was not submodule aware in two areas.
Linus said:
> So what happens is that something traverses a tree object, looks at each
> entry, sees that it's not a tree, and tries to look it up as a blob. But
> subprojects are commits, not blobs, and then when you look at the object
> more closely, you get the above kind of object type confusion.
and included a patch to add an S_ISGITLINK() test to reachable.c's
process_tree() function. That fixed the first git-prune error, and
stopped it from trying to process the gitlink entries in trees as if
they were pointers to other trees (and of course failing, because
gitlinks _aren't_ trees). That part of this patch is his.
The second area is add_cache_refs(). This is called before starting the
reachability analysis, and was calling lookup_blob() on every object
hash found in the index. However, it is no longer true that every hash
in the index is a pointer to a blob, some of them are gitlinks, and are
not backed by any object at all, they are commits in another repository.
Normally this bug was not causing any problems, but in the case of the
self-referencing repository described above, it meant that the gitlink
hash was being marked as being of type OBJ_BLOB by add_cache_refs() call
to lookup_blob(). Then later, because that hash was also pointed to by
a ref, add_one_ref() would treat it as a commit; lookup_commit() would
return a NULL because that object was already noted as being an
OBJ_BLOB, not an OBJ_COMMIT; and parse_commit_buffer() would SEGFAULT on
that NULL pointer.
The fix made by this patch is to not blindly call lookup_blob() in
reachable.c's add_cache_refs(), and instead skip any index entries that
are S_ISGITLINK().
Signed-off-by: Andy Parkins <andyparkins@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This removes slightly more lines than it adds, but the real reason for
doing this is that future optimizations will require more setup of the
tree descriptor, and so we want to do it in one place.
Also renamed the "desc.buf" field to "desc.buffer" just to trigger
compiler errors for old-style manual initializations, making sure I
didn't miss anything.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Currently, the search for all reflogs depends on the existence of
corresponding refs under the .git/refs/ directory. Let's scan the
.git/logs/ directory directly instead.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It used to ignore the return value of the helper function; now, it
expects it to return 0, and stops iteration upon non-zero return
values; this value is then passed on as the return value of
for_each_reflog_ent().
Further, it makes no sense to force the parsing upon the helper
functions; for_each_reflog_ent() now calls the helper function with
old and new sha1, the email, the timestamp & timezone, and the message.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This moves major part of builtin-prune into a separate file,
reachable.c. It is used to mark the objects that are reachable
from refs, and optionally from reflogs.
The patch looks very large, but if you look at it with diff -C,
which this message is formatted in, most of them are copied
lines and there are very little additions.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is necessary for the next step, because the reason I am
making the connectivity walker into a library is because I want
to use it for cleaning up stale reflog entries.
Signed-off-by: Junio C Hamano <junkio@cox.net>
I want to make the first part of 'git prune' that marks the
reachable objects callable as a library, so this starts the
first step toward the goal by making the callchain to pass
rev_info structure as an argument.
No functionality change should be in this step.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Somehow we forgot to turn save_commit_buffer off while walking
the reachable objects. Releasing the memory for commit object
data that we do not use matters for large projects (for example,
about 90MB is saved while traversing linux-2.6 history).
Signed-off-by: Junio C Hamano <junkio@cox.net>
It passed (const char*) to a function that took a (char *); the
buffer itself was of course writable, so pass the buffer itself.
Signed-off-by: Junio C Hamano <junkio@cox.net>
prune_object() in show_only mode would previously just show the path to the
object that would be deleted. The path the object is stored in shouldn't be
shown to users, they only know about sha1 identifiers so show that instead.
Further, the sha1 alone isn't that useful for examining what is going to be
deleted. This patch also adds the object type to the output, which makes it
easy to pick out, say, the commits and use git-show to display them.
Signed-off-by: Andy Parkins <andyparkins@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Both the git-prune manpage and everday.txt say that git-prune should also prune
unpacked objects that are also found in packs, by running git prune-packed.
Junio thought this was "a regression when prune was rewritten as a built-in."
So modify prune to call prune-packed again.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
This adds a "int *flag" parameter to resolve_ref() and makes
for_each_ref() family to call callback function with an extra
"int flag" parameter. They are used to give two bits of
information (REF_ISSYMREF and REF_ISPACKED) about the ref.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a long overdue fix to the API for for_each_ref() family
of functions. It allows the callers to specify a callback data
pointer, so that the caller does not have to use static
variables to communicate with the callback funciton.
The updated for_each_ref() family takes a function of type
int (*fn)(const char *, const unsigned char *, void *)
and a void pointer as parameters, and calls the function with
the name of the ref and its SHA-1 with the caller-supplied void
pointer as parameters.
The commit updates two callers, builtin-name-rev.c and
builtin-pack-refs.c as an example.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Like xmalloc and xrealloc xstrdup dies with a useful message if
the native strdup() implementation returns NULL rather than a
valid pointer.
I just tried to use xstrdup in new code and found it to be missing.
However I expected it to be present as xmalloc and xrealloc are
already commonly used throughout the code.
[jc: removed the part that deals with last_XXX, which I am
finding more and more dubious these days.]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
[jc: I needed to hand merge the changes to the updated codebase,
so the result needs to be checked.]
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This hopefully finishes the clean-up Ramsay started with recent
commit 15e593e4d3 and commit
8cdf33643d.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This changes the calling convention of built-in commands and
passes the "prefix" (i.e. pathname of $PWD relative to the
project root level) down to them.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Any git command that expects to work in a subdirectory of a project, and
that reads the git config files (which is just about all of them) needs to
make sure that it does the "setup_git_directory()" call before it tries to
read the config file.
This means, among other things, that we need to move the call out of
"init_revisions()", and into the caller.
This does the mostly trivial conversion to do that.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This actually removes the objects to be pruned, unless you specify "-n"
(at which point it will just tell you which files it would prune).
This doesn't do the pack-file pruning that the shell-script used to do,
but if somebody really wants to, they could add it easily enough. I wonder
how useful it is, though, considering that "git repack -a -d" is just a
lot more efficient and generates a better end result.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Jeff King