This fixes the previous implementation which failed to optimize
repositories with tons of lightweight tags. The updated
packed-refs format begins with "# packed-refs with:" line that
lists the kind of extended data the file records. Currently,
there is only one such extension defined, "peeled". This stores
the "peeled tag" on a line that immediately follows a line for a
tag object itself in the format "^<sha-1>".
The header line itself and any extended data are ignored by
older implementation, so packed-refs file generated with this
version can still be used by older git. packed-refs made by
older git can of course be used with this version.
Signed-off-by: Junio C Hamano <junkio@cox.net>
An earlier commit b37a562a added a check to see if the ref
points at a valid object (as a part of 'negative ref' support
which we currently do not use), but did so only while iterating
over both packed and loose refs, and forgot to apply the same
check while iterating over the remaining ones.
We might want to replace the "if null then omit it" check with
"eh --- what business does a 0{40} value have here?" complaint
later since we currently do not use negative refs, but that is
a separate issue.
Signed-off-by: Junio C Hamano <junkio@cox.net>
After deleting a branch l/k, you should be able to create a
branch l. Earlier we added remove_empty_directories() on the
ref creation side to remove leftover .git/refs/l directory but
we also need a matching code to remove .git/logs/refs/l
directory.
Signed-off-by: Junio C Hamano <junkio@cox.net>
It used to mean "create log file for any ref that is updated",
but now it creates new log files only for branch heads.
The old behaviour made this configuration less useful than
otherwise it would be; automatically creating log file for tags
is almost always not useful.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Often I find myself wanting to say 'tip of "next" before I
merged the last three topics'. Now I can say that with:
git log next@{3}..next
Since small integers alone are invalid input strings to
approxidate, there is no fear of confusion.
Signed-off-by: Junio C Hamano <junkio@cox.net>
lock_ref_sha1_basic relies on errno beeing set to EISDIR by the
call to read() in resolve_ref() to detect directories. But calling
read() on a directory under NetBSD returns EPERM, and even succeeds
for local filesystems on FreeBSD.
Signed-off-by: Dennis Stosberg <dennis@stosberg.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
lock_ref_sha1_basic relies on errno beeing set to EISDIR by the
call to read() in resolve_ref() to detect directories. But calling
read() on a directory under NetBSD returns EPERM, and even succeeds
for local filesystems on FreeBSD.
Signed-off-by: Dennis Stosberg <dennis@stosberg.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The function repack_without_ref() passes a lock-file structure
on the stack to hold_lock_file_for_update(), which in turn
registers it to be cleaned up via atexit(). This is a big
no-no.
This is the same bug James Bottomley fixed with commit
31f584c242.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This implements deletion of a packed ref. Since it is a very
rare event to delete a ref compared to looking up, creating and
updating, this opts to remove the ref from the packed-ref file
instead of doing any of the filesystem based "negative ref" trick
to optimize the deletion path.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This cleans up the error path in the function so it does not
die() itself sometimes while signalling an error with NULL some
other times which was inconsistent and confusing.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Once we read packed and loose refs, for_each_ref() and friends
kept using them even after write_ref_sha1() and delete_ref()
changed the refs. This adds invalidate_cached_refs() as a way
to flush the cached information.
Signed-off-by: Junio C Hamano <junkio@cox.net>
It is normal to have .git/refs/heads/foo directory which is
empty after the last branch whose name starts with foo/ is
removed. Make sure we notice this case and allow creation of
branch foo by removing the empty directory.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds -d flag to update-ref to allow safe deletion of ref.
Before deleting it, the command checks if the given <oldvalue>
still matches the value the caller thought the ref contained.
Similarly, it also accepts 0{40} or an empty string as <oldvalue>
to allow safe creation of a new ref.
Signed-off-by: Junio C Hamano <junkio@cox.net>
There is a format string vulnerability introduced with the packed refs
file format.
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds a "int *flag" parameter to resolve_ref() and makes
for_each_ref() family to call callback function with an extra
"int flag" parameter. They are used to give two bits of
information (REF_ISSYMREF and REF_ISPACKED) about the ref.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a long overdue fix to the API for for_each_ref() family
of functions. It allows the callers to specify a callback data
pointer, so that the caller does not have to use static
variables to communicate with the callback funciton.
The updated for_each_ref() family takes a function of type
int (*fn)(const char *, const unsigned char *, void *)
and a void pointer as parameters, and calls the function with
the name of the ref and its SHA-1 with the caller-supplied void
pointer as parameters.
The commit updates two callers, builtin-name-rev.c and
builtin-pack-refs.c as an example.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Current git#next is totally broken wrt. cloning over HTTP, generating refs
at random directories. Of course it's caused by the static get_pathname()
buffer. lock_ref_sha1() stores return value of mkpath()'s get_pathname()
call, then calls lock_ref_sha1_basic() which calls git_path(ref) which
calls get_pathname() at that point returning pointer to the same buffer.
So now you are sprintf()ing a format string into itself, wow! The resulting
pathnames are really cute. (If you've been paying attention, yes, the
mere fact that a format string _could_ write over itself is very wrong
and probably exploitable here. See the other mail I've just sent.)
I've never liked how we use return values of those functions so liberally,
the "allow some random number of get_pathname() return values to work
concurrently" is absolutely horrible pit and we've already fallen in this
before IIRC. I consider it an awful coding practice, you add a call
somewhere and at some other point some distant caller of that breaks since
it reuses the same return values. Not to mention this takes quite some time
to debug.
My gut feeling tells me that there might be more of this. I don't have
time to review the rest of the users of the refs.c functions though.
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This actually "turns on" the packed ref file format, now that the
infrastructure to do so sanely exists (ie notably the change to make the
reference reading logic take refnames rather than pathnames to the loose
objects that no longer necessarily even exist).
In particular, when the ref lookup hits a refname that has no loose file
associated with it, it falls back on the packed-ref information. Also, the
ref-locking code, while still using a loose file for the locking itself
(and _creating_ a loose file for the new ref) no longer requires that the
old ref be in such an unpacked state.
Finally, this does a minimal hack to git-checkout.sh to rather than check
the ref-file directly, do a "git-rev-parse" on the "heads/$refname".
That's not really wonderful - we should rather really have a special
routine to verify the names as proper branch head names, but it is a
workable solution for now.
With this, I can literally do something like
git pack-refs
find .git/refs -type f -print0 | xargs -0 rm -f --
and the end result is a largely working repository (ie I've done two
commits - which creates _one_ unpacked ref file - done things like run
"gitk" and "git log" etc, and it all looks ok).
There are probably things missing, but I'm hoping that the missing things
are now of the "small and obvious" kind, and that somebody else might want
to start looking at this too. Hint hint ;)
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The old code used to totally mix up the notion of a ref-name and the path
that that ref was associated with. That was not only horribly ugly (a
number of users got the path, and then wanted to try to turn it back into
a ref-name again), but it fundamnetally doesn't work at all once we do any
setup where a ref doesn't have a 1:1 relationship with a particular
pathname.
This fixes things up so that we use the ref-name throughout, and only
turn it into a pathname once we actually look it up in the filesystem.
That makes a lot of things much clearer and more straightforward.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
You can remove a ref that is packed two different ways: either simply
repack all the refs without that one, or create a loose ref that has the
magic all-zero SHA1.
This also adds back the test that a ref actually has the object it
points to.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This also adds some very rudimentary support for the notion of packed
refs. HOWEVER! At this point it isn't used to actually look up a ref
yet, only for listing them (ie "for_each_ref()" and friends see the
packed refs, but none of the other single-ref lookup routines).
Note how we keep two separate lists: one for the loose refs, and one for
the packed refs we read. That's so that we can easily keep the two apart,
and read only one set or the other (and still always make sure that the
loose refs take precedence).
[ From this, it's not actually obvious why we'd keep the two separate
lists, but it's important to have the packed refs on their own list
later on, when I add support for looking up a single loose one.
For that case, we will want to read _just_ the packed refs in case the
single-ref lookup fails, yet we may end up needing the other list at
some point in the future, so keeping them separated is important ]
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Like xmalloc and xrealloc xstrdup dies with a useful message if
the native strdup() implementation returns NULL rather than a
valid pointer.
I just tried to use xstrdup in new code and found it to be missing.
However I expected it to be present as xmalloc and xrealloc are
already commonly used throughout the code.
[jc: removed the part that deals with last_XXX, which I am
finding more and more dubious these days.]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This abstracts away the size of the hash values when copying them
from memory location to memory location, much as the introduction
of hashcmp abstracted away hash value comparsion.
A few call sites were using char* rather than unsigned char* so
I added the cast rather than open hashcpy to be void*. This is a
reasonable tradeoff as most call sites already use unsigned char*
and the existing hashcmp is also declared to be unsigned char*.
[jc: Splitted the patch to "master" part, to be followed by a
patch for merge-recursive.c which is not in "master" yet.
Fixed the cast in the latter hunk to combine-diff.c which was
wrong in the original.
Also converted ones left-over in combine-diff.c, diff-lib.c and
upload-pack.c ]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Introduces global inline:
hashcmp(const unsigned char *sha1, const unsigned char *sha2)
Uses memcmp for comparison and returns the result based on the length of
the hash name (a future runtime decision).
Acked-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Most of the callers except the one in refs.c use the function to
update the index file. Among the index writers, everybody
except write-tree dies if they cannot open it for writing.
This gives the function an extra argument, to tell it to die
when it cannot create a new file as the lockfile.
The only caller that does not have to die is write-tree, because
updating the index for the cache-tree part is optional and not
being able to do so does not affect the correctness. I think we
do not have to be so careful and make the failure into die() the
same way as other callers, but that would be a different patch.
Signed-off-by: Junio C Hamano <junkio@cox.net>
As Fredrik points out the current interface of has_extension() is
potentially confusing. Its parameters include both a nul-terminated
string and a length-limited string.
This patch drops the length argument, requiring two nul-terminated
strings; all callsites are updated. I checked that all of them indeed
provide nul-terminated strings. Filenames need to be nul-terminated
anyway if they are to be passed to open() etc. The performance penalty
of the additional strlen() is negligible compared to the system calls
which inevitably surround has_extension() calls.
Additionally, change has_extension() to use size_t inside instead of
int, as that is the exact type strlen() returns and memcmp() expects.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The little helper has_extension() documents through its name what we are
trying to do and makes sure we don't forget the underrun check.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Alex Riesen (raa.lkml@gmail.com) recently observed that git branch
would fail with no error message due to unexpected situations with
regards to refs. For example, if .git/refs/heads/gu is a file but
"git branch -b refs/heads/gu/fixa HEAD" was invoked by the user
it would fail silently due to refs/heads/gu being a file and not
a directory.
This change adds a test for trying to create a ref within a directory
that is actually currently a file, and adds error printing within
the ref locking routine should the resolve operation fail.
The error printing code probably belongs at this level of the library
as other failures within the ref locking, writing and logging code
are also currently at this level of the code.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Apparently calling setup_ident() after git_config causes the
user.name and user.email values read from the config file to be
replaced with the data obtained from the host. This means that
users who have setup their email address in user.email will instead
be writing reflog entries with their hostname.
Moving setup_ident() to before git_config in update-ref resolves
this ordering problem.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
There were a few calls to adjust_shared_perm() that were
missing:
- init-db creates refs, refs/heads, and refs/tags before
reading from templates that could specify sharedrepository in
the config file;
- updating config file created it under user's umask without
adjusting;
- updating refs created it under user's umask without
adjusting;
- switching branches created .git/HEAD under user's umask
without adjusting.
This moves adjust_shared_perm() from sha1_file.c to path.c,
since a few SIMPLE_PROGRAM need to call repository configuration
functions which in turn need to call adjust_shared_perm().
sha1_file.c needs to link with SHA1 computation library which
is usually not linked to SIMPLE_PROGRAM.
Signed-off-by: Junio C Hamano <junkio@cox.net>
A few style fixes to get the code in line with the rest.
- asterisk to make a type a pointer to something goes in front
of the variable, not at the end of the base type.
E.g. a pointer to an integer is "int *ip", not "int* ip".
- open parenthesis for function parameter list, unlike
syntactic constructs, comes immediately after the function
name. E.g. "if (foo) bar();" not "if(foo) bar ();".
- "else" does not come on the same line as the closing brace of
corresponding "if".
The style is mostly a matter of personal taste, and people may
disagree, but consistency is important.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This updates the ref locking code to use creat-rename locking
code we use for the index file, so that it can borrow the code
to clean things up upon signals and program termination.
Signed-off-by: Junio C Hamano <junkio@cox.net>
My earlier attempt at forcing a write for non-existant refs worked;
it forced a write for pretty much all refs. This corrects the
condition to only force a write for refs which don't exist yet.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If a ref is changed by http-fetch, local-fetch or ssh-fetch
record the change and the remote URL/name in the log for the ref.
This requires loading the config file to check logAllRefUpdates.
Also fixed a bug in the ref lock generation; the log file name was
not being produced right due to a bad prefix length.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Normally we try to skip writing a ref if its value hasn't changed
but in the special case that the ref doesn't exist but the new
value is going to be 0{40} then force writing the ref anyway.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Corrected the log starting time displayed in the error message
(as it was always showing the epoch due to a bad input to strtoul).
Improved the log parser so we only scan backwards towards the
'\n' from the end of the prior log; during this scan the last '>'
is remembered to improve performance (rather than scanning forward
to it).
If the log record matched is the last log record in the file only
use its new sha1 value if the date matches exactly; otherwise we
leave the passed in sha1 alone as it already contains the current
value of the ref. This way lookups of dates later than the log
end to stick with the current ref value in case the ref was updated
without logging.
If it looks like someone changed the ref without logging it and we
are going to return the sha1 which should have been valid during
the missing period then warn the user that there might be log data
missing and thus their query result may not be accurate. The check
isn't perfect as its just based on comparing the old and new sha1
values between the two log records but its better than not checking
at all.
Implemented test cases based on git-rev-parse for most of the
boundary conditions.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The log parser was only ever matching the last log record due to
calling strtoul on "> 1136091609" rather than " 1136091609". Also
once a match for '@' has been found after the name of the ref there
is no point in looking for another '@' within the remaining text.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Extended sha1 expressions may now include date specifications
which indicate a point in time within the local repository's
history. If the ref indicated to the left of '@' has a log in
$GIT_DIR/logs/<ref> then the value of the ref at the time indicated
by the specification is obtained from the ref's log.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If config parameter core.logAllRefUpdates is true or the log
file already exists then append a line to ".git/logs/refs/<ref>"
whenever git-update-ref <ref> is executed. Each log line contains
the following information:
oldsha1 <SP> newsha1 <SP> committer <LF>
where committer is the current user, date, time and timezone in
the standard GIT ident format. If the caller is unable to append
to the log file then git-update-ref will fail without updating <ref>.
An optional message may be included in the log line with the -m flag.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Created 'struct ref_lock' to contain the data necessary to perform
a ref update. This change improves writing a ref as the file names
are generated only once (rather than twice) and supports following
symrefs (up to the maximum depth). Further the ref_lock structure
provides room to extend the update API with ref logging.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
"git branch" uses "rev-parse --all" and becomes much too slow when
there are many tags (it scans all refs). Use the new "--branches"
option of rev-parse to speed things up.
Signed-off-by: Sean Estabrooks <seanlkml@sympatico.ca>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Junio C Hamano