This adds an option (--window-memory=N) and configuration variable
(pack.windowMemory = N) to limit the memory size of the pack-objects
delta search window. This works by removing the oldest unpacked objects
whenever the total size goes above the limit. It will always leave
at least one object, though, so as not to completely eliminate the
possibility of computing deltas.
This is an extra limit on top of the normal window size (--window=N);
the window will not dynamically grow above the fixed number of entries
specified to fill the memory limit.
With this, repacking a repository with a mix of large and small objects
is possible even with a very large window.
Cleaner and correct circular buffer handling courtesy of Nicolas Pitre.
Signed-off-by: Brian Downing <bdowning@lavos.net>
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Split out the nnn{k,m,g} parsing code from git_config_int into
git_parse_long, so command-line parameters can enjoy the same
functionality. Also add get_parse_ulong for unsigned values.
Make git_config_int use git_parse_long, and add get_config_ulong
as well.
Signed-off-by: Brian Downing <bdowning@lavos.net>
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Delta indices, at least on 64-bit platforms, tend to be larger than
the actual uncompressed data. As such, keeping track of this storage
is important if you want to successfully limit the memory size of your
pack window.
Squirrel away the total allocation size inside the delta_index struct,
and add an accessor "sizeof_delta_index" to access it.
Signed-off-by: Brian Downing <bdowning@lavos.net>
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a new try_delta heuristic. Don't bother trying to make a delta if
the target object size is much smaller (currently 1/32) than the source,
as it's very likely not going to get a match. Even if it does, you will
have to read at least 32x the size of the new file to reassemble it,
which isn't such a good deal. This leads to a considerable performance
improvement when deltifying a mix of small and large files with a very
large window, because you don't have to wait for the large files to
percolate out of the window before things start going fast again.
Signed-off-by: Brian Downing <bdowning@lavos.net>
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This script was originally posted on the git mailing list by
Randal L. Schwartz <merlyn@stonehenge.com>.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We already apply a bias on the initial delta attempt with max_size being
a function of the base object depth. This has the effect of favoring
shallower deltas even if deeper deltas could be smaller, and therefore
creating a wider delta tree (see commits 4e8da195 and c3b06a69).
This principle should also be applied to all delta attempts for the same
object and not only the first attempt. With this the criteria for the
best delta is not only its size but also its depth, so that a shallower
delta might be selected even if it is larger than a deeper one. Even if
some deltas get larger, they allow for wider delta trees making the
depth limit less quickly reached and therefore better deltas can be
subsequently found, keeping the resulting pack size even smaller.
Runtime access to the pack should also benefit from shallower deltas.
Testing on different repositories showed slighter faster repacks,
smaller resulting packs, and a much nicer curve for delta depth
distribution with no more peak at the maximum depth level.
Improvements are even more significant with smaller depth limits.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* commit 'git-gui/master': (36 commits)
git-gui: Change prior tree SHA-1 verification to use git_read
git-gui: Include a space in Cygwin shortcut command lines
git-gui: Use sh.exe in Cygwin shortcuts
git-gui: Paper bag fix for Cygwin shortcut creation
git-gui: Improve the Windows and Mac OS X shortcut creators
git-gui: Teach console widget to use git_read
git-gui: Perform our own magic shbang detection on Windows
git-gui: Treat `git version` as `git --version`
git-gui: Assume unfound commands are known by git wrapper
git-gui: Correct gitk installation location
git-gui: Always use absolute path to all git executables
git-gui: Show a progress meter for checking out files
git-gui: Change the main window progress bar to use status_bar
git-gui: Extract blame viewer status bar into mega-widget
git-gui: Allow double-click in checkout dialog to start checkout
git-gui: Default selection to first matching ref
git-gui: Unabbreviate commit SHA-1s prior to display
git-gui: Refactor branch switch to support detached head
git-gui: Refactor our ui_status_value update technique
git-gui: Better handling of detached HEAD
...
* maint:
GIT 1.5.2.4
Teach read-tree 2-way merge to ignore intermediate symlinks
git-gui: Work around bad interaction between Tcl and cmd.exe on ^{tree}
git-gui: Don't linewrap within console windows
git-gui: Correct ls-tree buffering problem in browser
git-gui: Skip nicknames when selecting author initials
git-gui: Ensure windows shortcuts always have .bat extension
git-gui: Include a Push action on the left toolbar
git-gui: Bind M1-P to push action
git-gui: Don't bind F5/M1-R in all windows
git-gui: Unlock the index when cancelling merge dialog
git-gui: properly popup error if gitk should be started but is not installed
Earlier in 16a4c61, we taught "read-tree -m -u" not to be
confused when switching from a branch that has a path frotz/filfre
to another branch that has a symlink frotz that points at xyzzy/
directory. The fix was incomplete in that it was still confused
when coming back (i.e. switching from a branch with frotz -> xyzzy/
to another branch with frotz/filfre).
This fix is rather expensive in that for a path that is created
we would need to see if any of the leading component of that
path exists as a symbolic link in the filesystem (in which case,
we know that path itself does not exist, and the fact we already
decided to check it out tells us that in the index we already
know that symbolic link is going away as there is no D/F
conflict).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* 'maint' of git://repo.or.cz/git-gui:
git-gui: Work around bad interaction between Tcl and cmd.exe on ^{tree}
git-gui: Don't linewrap within console windows
git-gui: Correct ls-tree buffering problem in browser
git-gui: Skip nicknames when selecting author initials
git-gui: Ensure windows shortcuts always have .bat extension
git-gui: Include a Push action on the left toolbar
git-gui: Bind M1-P to push action
git-gui: Don't bind F5/M1-R in all windows
git-gui: Unlock the index when cancelling merge dialog
git-gui: properly popup error if gitk should be started but is not installed
This cat-file was done on maint, where we did not have git_read
available to us. But here on master we do, so we should make
use of it.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
From Johannes Sixt <J.Sixt@eudaptics.com>:
> It seems that MSYS's wish does some quoting for Bourne shells,
> in particular, escape the first '{' of the "^{tree}" suffix, but
> then it uses cmd.exe to run "git rev-parse". However, cmd.exe does
> not remove the backslash, so that the resulting rev expression
> ends up in git's guts as unrecognizable garbage: rev-parse fails,
> and git-gui hickups in a way that it must be restarted.
Johannes originally submitted a patch to this section of commit.tcl
to use `git rev-parse $PARENT:`, but not all versions of Git will
accept that format. So I'm just taking the really simple approach
here of scanning the first line of the commit to grab its tree.
About the same cost, but works everywhere.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
An implementation easier to call from builtins. It is designed
to be used from the upcoming builtin-tag.c and builtin-commit.c,
because both need to remove unwanted spaces from messages.
Signed-off-by: Carlos Rica <jasampler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This tool will print vaguely pretty information about a pack. It
expects the output of "git-verify-pack -v" as input on stdin.
$ git-verify-pack -v | packinfo.pl
See the documentation in the script (contrib/stats/packinfo.pl)
for more information.
Signed-off-by: Brian Downing <bdowning@lavos.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch cleans up some complicated code, and replaces it with a
cleaner version, using code from remote.[ch], which got extended a
little in the process. This also enables us to fix two cases:
The earlier "fix" to setup tracking only when the original ref started
with "refs/remotes" is wrong. You are absolutely allowed to use a
separate layout for your tracking branches. The correct fix, of course,
is to set up tracking information only when there is a matching
remote.<nick>.fetch line containing a colon.
Another corner case was not handled properly. If two remotes write to
the original ref, just warn the user and do not set up tracking.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function for_each_remote() does exactly what the name
suggests.
The function remote_find_tracking() was extended to be able to
search remote refs for a given local ref. The caller sets
either src or dst (but not both) in the refspec parameter, and
remote_find_tracking() will fill in the other and return 0.
Both changes are required for the next step: simplification of
git-branch's --track functionality.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These tests check the syntax for the git tag -n option
and its output when one, none or many lines of the
message are requested.
Also this commit adds a missing && in the test
that checks the sorted output of git tag -l.
Signed-off-by: Carlos Rica <jasampler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previous tests only had paragraphs of one line. This commit adds some
tests to check when many consecutive text lines are given.
Also, it adds tests for checking that many lines between paragraphs are
correctly reduced to one when there are tabs and spaces in those lines.
Signed-off-by: Carlos Rica <jasampler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Moved some tests to another test_expect_success block.
Many tests now reuse the same "expect" file. Also replacing
many printf "" >expect with one >expect instruction.
Added missing && which concatenated tests in some
test_expect_success blocks.
Signed-off-by: Carlos Rica <jasampler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If you say --cherry-pick, you do not want to see patches which are
in the upstream. If you specify paths with that, what you usually
expect is that only those parts of the patches are looked at which
actually touch the given paths.
With this patch, that expectation is met.
Noticed by Sam Vilain.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For compatibility reasons, "git init --shared=all" does not write
"all" into the config, but a number. In the shared setup, you
really have to support even older clients on the _same_ repository.
But git_config_perm() did not pick up on it.
Also, "git update-server-info" failed to pick up on the shared
permissions.
This patch fixes both issues, and adds a test to prove it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Tested-by: martin f krafft <madduck@madduck.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows gitweb users to set $projects_list_description_width
in their gitweb.conf to determine how many characters of a project
description are displayed before being truncated with an ellipsis.
Signed-off-by: Michael Hendricks <michael@ndrix.org>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Functions for managing ref lists were named based on their use in
match_refs (for push). For fetch, they will be used for other purposes, so
rename them as a separate patch to make the future code readable.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of open-coding allocation wherever it happens, have a function.
Also, add a function to free a list of refs, which we currently never
actually do.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 334d28ae factored out git.o as an intermediate stage between
git.c and git$X. However:
- It left some no-longer-relevant flags in the rule for git$X.
- It failed to replace git$X with git.o in the list of files that
record GIT_VERSION. This broke incorporation of a changed
GIT_VERSION into git$X because, when GIT_VERSION changes, git.o isn't
remade and git$X is relinked from the git.o that still contains the
old GIT_VERSION.
This patch removes the irrelevant flags and fixes incorporation of a
changed GIT_VERSION into git$X.
Signed-off-by: Matt McCutchen <hashproduct@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Two-file merges were rare enough that they were dropped outside of the
radar. This fix is a trivial change to builtin-rerere.c::find_conflict().
It is still sane to insist that we do not do rerere for symlinks, and
require to have stages #2 and #3, but we can drop the requirement to have
stage #1. rerere does not use information from there anyway.
This fix is from Junio, together with two tests to verify that it works
as expected.
Acked-by: Uwe Kleine-König <ukleinek@informatik.uni-freiburg.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some source material (e.g. Subversion dump files) perform directory
renames without telling us exactly which files in that subdirectory
were moved. This makes it hard for a frontend to convert such data
formats to a fast-import stream, as all the frontend has on hand
is "Rename a/ to b/" with no details about what files are in a/,
unless the frontend also kept track of all files.
The new 'R' subcommand within a commit allows the frontend to
rename either a file or an entire subdirectory, without needing to
know the object's SHA-1 or the specific files contained within it.
The rename is performed as efficiently as possible internally,
making it cheaper than a 'D'/'M' pair for a file rename.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The 'D' subcommand within a commit can also delete a directory
recursively. This wasn't clear in the prior version of the
documentation, leading to a question on the mailing list.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Because we are trying to execute /bin/sh we know it must be a real
Windows executable and thus ends with the standard .exe suffix.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We cannot execute the git directory, it is not a valid Tcl command
name. Instead we just want to pass it as an argument to our sq
proc.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If we get more than 80 characters of text in a single line odds
are it is output from git-fetch or git-push and its showing a
lot of detail off to the right edge that is not so important to
the average user. We still want to make sure we show everything
we need, but we can get away with that information being off to
the side with a horizontal scrollbar.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Our file browser was showing bad output as it did not properly buffer
a partial record when read from `ls-tree -z`. This did not show up on
my Mac OS X system as most trees are small, the pipe buffers generally
big and `ls-tree -z` was generally fast enough that all data was ready
before Tcl started to read. However on my Cygwin system one of my
production repositories had a large enough tree and packfile that it
took a couple of pipe buffers for `ls-tree -z` to complete its dump.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We now embed any GIT_* and SSH_* environment variables as well as
the path to the git wrapper executable into the Mac OS X .app file.
This should allow us to restore the environment properly when
we restart.
We also try to use proper Bourne shell single quoting when we can,
as this avoids any sort of problems that might occur due to a path
containing shell metacharacters.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Now that we are pretty strict about setting up own absolute paths to
any git helper (saving a marginal runtime cost to resolve the tool)
we can do the same in our console widget by making sure all console
execs go through git_read if they are a git subcommand, and if not
make sure they at least try to use the Tcl 2>@1 IO redirection if
possible, as it should be faster than |& cat.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Change "try_delta" so that if it finds a delta that has the same size
but shallower depth than the existing delta, it will prefer the
shallower one. This makes certain delta trees vastly less deep.
Signed-off-by: Brian Downing <bdowning@lavos.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we cannot locate a .exe for a git tool that we want to run than
it may just be a Bourne shell script as these are popular in Git.
In such a case the first line of the file will say "#!/bin/sh" so
a UNIX kernel knows what program to start to parse and run that.
But Windows doesn't support shbang lines, and neither does the Tcl
that comes with Cygwin.
We can pass control off to the git wrapper as that is a real Cygwin
program and can therefore start the Bourne shell script, but that is
at least two fork+exec calls to get the program running. One to do
the fork+exec of the git wrapper and another to start the Bourne shell
script. If the program is run multiple times it is rather expensive
as the magic shbang detection won't be cached across executions.
On MinGW/MSYS we don't have the luxury of such magic detection. The
MSYS team has taught some of this magic to the git wrapper, but again
its slower than it needs to be as the git wrapper must still go and
run the Bourne shell after it is called.
We now attempt to guess the shbang line on Windows by reading the
first line of the file and building our own command line path from
it. Currently we support Bourne shell (sh), Perl and Python. That
is the entire set of shbang lines that appear in git.git today.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We know that the version subcommand of git is special. It does not
currently have an executable link installed into $gitexecdir and we
therefore would never match it with one of our file exists tests.
So we forward any invocations to it directly to the git wrapper, as
it is a builtin within that executable.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When we simplify history due to path limits, the parents list
for a rewritten commit can end up having duplicates. Instead of
filtering them out in the output codepath like earlier commit
88494423 did, remove them much earlier, when the parent
information actually gets rewritten.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we cannot locate a command in $gitexecdir on our own then it may
just be because we are supposed to run it by `git $name` rather than
by `git-$name`. Many commands are now builtins, more are likely to
go in that direction, and we may see the hardlinks in $gitexecdir go
away in future versions of git.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Brian Downing