A changelist that contains only excluded files due to a client spec was
imported as an empty commit. Fix that issue by ignoring these commits.
Add option "git-p4.keepEmptyCommits" to make the previous behavior
available.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Helped-by: Pete Harlan
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When "p4 submit" command fails in P4Submit.applyCommit, the
workspace is left with the changes. We already have code to revert
the changes to the workspace when the user decides to cancel
submission by aborting the editor that edits the change description,
and we should treat the "p4 submit" failure the same way.
Clean the workspace if p4_write_pipe raised SystemExit, so that the
user don't have to do it themselves.
Signed-off-by: GIRARD Etienne <egirard@murex.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
When submitting, git-p4 finds the current branch in
order to know if it is allowed to submit (configuration
"git-p4.allowSubmit").
On a detached head, detecting the branch would fail, and
git-p4 would report a cryptic error.
This change teaches git-p4 to recognise a detached head and
submit successfully.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
Add an optional parameter ignore_error to the git-p4 system()
function. If used, it will return the subshell exit status
rather than throwing an exception.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
The ctypes module is used on windows to calculate free disk space,
so it must be imported. We won't need it on other platforms, but
the module is available in Python 2.5 and newer, so importing it
unconditionally is harmless.
Signed-off-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add example implementation including test cases for the large file
system using Git LFS.
Pushing files to the Git LFS server is not tested.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce repositories can contain large (binary) files. Migrating these
repositories to Git generates very large local clones. External storage
systems such as Git LFS [1], Git Fat [2], Git Media [3], git-annex [4]
try to address this problem.
Add a generic mechanism to detect large files based on extension,
uncompressed size, and/or compressed size.
[1] https://git-lfs.github.com/
[2] https://github.com/jedbrown/git-fat
[3] https://github.com/alebedev/git-media
[4] https://git-annex.branchable.com/
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Conflicts:
Documentation/git-p4.txt
git-p4.py
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 will just halt if there is not enough disk space while
streaming content from P4 to Git. Add a check to ensure at least
4 times (arbitrarily chosen) the size of a streamed file is available.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a file is streamed from P4 to Git then the verbose mode prints
continuously the progress as percentage like this:
//depot/file.bin 20% (10 MB)
Upon completion the progress is overwritten with depot source, local
file and size like this:
//depot/file.bin --> local/file.bin (10 MB)
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a git config reader for integer variables. Please note that the
git config implementation automatically supports k, m, and g suffixes.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The functions "gitConfig" and "gitConfigBool" are almost identical.
Make "gitConfig" more generic by adding an optional type specifier.
Use the type specifier "--bool" with "gitConfig" to implement
"gitConfigBool. This prepares the implementation of other type
specifiers such as "--int".
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A P4 repository can get into a state where it contains a file with
type UTF-16 that does not contain a valid UTF-16 BOM. If git-p4
attempts to retrieve the file then the process crashes with a
"Translation of file content failed" error.
More info here: http://answers.perforce.com/articles/KB/3117
Fix this by detecting this error and retrieving the file as binary
instead. The result in Git is the same.
Known issue: This works only if git-p4 is executed in verbose mode.
In normal mode no exceptions are thrown and git-p4 just exits.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If non UTF-8 characters are detected in paths then replace them with
a placeholder instead of throwing a UnicodeDecodeError
exception. This restores the original (implicit) implementation that
was broken in 00a9403.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a path with non-ASCII characters is detected then print the
encoding and the encoded string in verbose mode.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce keeps the encoding of a path as given by the originating OS.
Git expects paths encoded as UTF-8. Add a config to tell git-p4 what
encoding Perforce had used for the paths. This encoding is used to
transcode the paths to UTF-8. As an example, Perforce on Windows often
uses “cp1252” to encode path names.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With --detect-labels enabled, git-p4 will try to create tags
using git fast-import by writing a "tag" clause to the
fast-import stream.
If the commit that the tag references has not yet actually
been processed by fast-import, then the tag can't be created
and git-p4 fails to import the P4 label.
Teach git-p4 to use fast-import "marks" when creating tags
which reference commits created during the current run of the
program.
Commits created before the current run are still referenced
in the old way using a normal git commit.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If p4 reports a tag for a commit that git-p4 does not know
about (e.g. because it references a P4 changelist that was
imported prior to the point at which the repo was cloned into
git), make sure that the error is correctly caught and handled.
rather than just crashing.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce depot may record paths in mixed cases, e.g. "p4 files" may
show that there are these two paths:
//depot/Path/to/file1
//depot/pATH/to/file2
and with "p4" or "p4v", these end up in the same directory, e.g.
//depot/Path/to/file1
//depot/Path/to/file2
which is the desired outcome on case insensitive systems.
If git-p4 is used with client spec "//depot/Path/...", however, then
all files not matching the case in the client spec are ignored (in
the example above "//depot/pATH/to/file2").
Fix this by using the path case that appears first in lexicographical
order when core.ignorecase is set to true. This behavior is consistent
with "p4" and "p4v".
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The --changes-block-size handling was intended to help when
a user has a limited "maxscanrows" (see "p4 group"). It used
"p4 changes -m $maxchanges" to limit the number of results.
Unfortunately, it turns out that the "maxscanrows" and "maxresults"
limits are actually applied *before* the "-m maxchanges" parameter
is considered (experimentally).
Fix the block-size handling so that it gets blocks of changes
limited by revision number ($Start..$Start+$N, etc). This limits
the number of results early enough that both sets of tests pass.
Note that many other Perforce operations can fail for the same
reason (p4 print, p4 files, etc) and it's probably not possible
to workaround this. In the real world, this is probably not
usually a problem.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fixing bug with UTF-16 files when they are retrieved by git-p4. It
was always getting the tip version of the file and the history of
the file was lost.
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This teaches git-p4 to pass the P4EDITOR variable to the
shell for expansion, so that any command-line arguments are
correctly handled. Without this, git-p4 can only launch the
editor if P4EDITOR is solely the path to the binary, without
any arguments.
This also adjusts t9805, which relied on the previous behaviour.
Suggested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce allows client side file/directory remapping through
the use of the client view definition that is part of the
user's client spec.
To support this functionality while branch detection is
enabled it is important to determine the branch location in
the workspace such that the correct files are patched before
Perforce submission. Perforce provides a command that
facilitates this process: p4 where.
This patch does two things to fix improve file location
detection when git-p4 has branch detection and use of client
spec enabled:
1. Enable usage of "p4 where" when Perforce branches exist
in the git repository, even when client specification is
used. This makes use of the already existing function
p4Where.
2. Allow identifying partial matches of the branch's depot
path while processing the output of "p4 where". For
robustness, paths will only match if ending in "/...".
Signed-off-by: Vitor Antunes <vitor.hda@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Simply running "p4 changes" on a large branch can result in a "too
many rows scanned" error from the Perforce server. It is better to
use a sequence of smaller calls to "p4 changes", using the "-m"
option to limit the size of each call.
Signed-off-by: Lex Spoon <lex@lexspoon.org>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a Perforce server is configured to automatically set +l
(exclusive lock) on add of certain file types, git p4 submit will
fail during getP4OpenedType, as the regex doesn't expect the
trailing '*exclusive*' from p4 opened:
//depot/file.png#1 - add default change (binary+l) *exclusive*
Signed-off-by: Blair Holloway <blair_holloway@playstation.sony.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The clone subcommand has long had support for excluding
subdirectories, but sync has not. This is a nuisance,
since as soon as you do a sync, any changed files that
were initially excluded start showing up.
Move the "exclude" command-line option into the parent
class; the actual behavior was already present there so
it simply had to be exposed.
Signed-off-by: Luke Diamand <luke@diamand.org>
Reviewed-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If you use git-p4 with the "--prepare-p4-only" option, then
it prints the p4 command line to use. However, the command
line was incorrect: the changelist specification must be
supplied on standard input, not as an argument to p4.
Signed-off-by: Luke Diamand <luke@diamand.org>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
b4073bb3 (git-p4: Do not include diff in spec file when just
preparing p4, 2014-05-24) broke git p4 submit, here is a proper
fix, including proper handling for windows end of lines.
Signed-off-by: Maxime Coste <frrrwww@gmail.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The diff information render the spec file unusable as is by p4,
do not include it when run with --prepare-p4-only so that the
given file can be directly passed to p4.
With --prepare-p4-only, git-p4 already tells the user it can use
p4 submit with the generated spec file. This fails because of the
diff being present in the file. Not including the diff fixes that.
Without --prepare-p4-only, keeping the diff makes sense for a
quick review of the patch before submitting it. And does not cause
problems with p4 as we remove it programmatically.
Signed-off-by: Maxime Coste <frrrwww@gmail.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When applying binary patches a full index is required. format-patch
already handles this, but diff-tree needs '--full-index' argument
to always output full index. When git-p4 runs git-apply to test
the patch, git-apply rejects the patch due to abbreviated blob
object names. This is the error message git-apply emits in this
case:
error: cannot apply binary patch to '<filename>' without full index line
error: <filename>: patch does not apply
Signed-off-by: Tolga Ceylan <tolga.ceylan@gmail.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git p4 rebase' fails with the following message if there is a file
named HEAD in the current directory:
fatal: ambiguous argument 'HEAD': both revision and filename
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Take the suggestion above and explicitly state that HEAD should be
treated as a revision.
Signed-off-by: Vlad Dogaru <vdogaru@ixiacom.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When "p4 where" fails, for whatever reason, the error message tries to
show an undefined variable. This minor bug applies only when using a
client spec, and was introduced recently in 9d57c4a (git p4: implement
view spec wildcards with "p4 where", 2013-08-30).
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 9d7d446 (git p4: submit files with wildcards, 2012-04-29)
fixed problems with handling files that had p4 wildcard
characters, like "@" and "*". But it missed one case, that of
RCS keyword scrubbing, which uses "p4 fstat" to extract type
information. Fix it by calling wildcard_encode() on the raw
filename.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Generating the submit template for p4 uses tempfile.mkstemp(),
which by default puts files in /tmp. For a test that fails,
possibly on purpose, this is not cleaned up. Run with TMPDIR
pointing into the trash directory so the temp files go away
with the test results.
To do this required some other minor changes. First, the editor
is launched using system(editor + " " + template_file), using
shell expansion to build the command string. This doesn't work
if editor has a space in it. And is generally unwise as it's
easy to fool the shell into doing extra work. Exec the args
directly, without shell expansion.
Second, without shell expansion, the trick of "P4EDITOR=:" used
in the tests doesn't work. Use a real command, true, as the
non-interactive editor for testing.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Damien Gérard highlights an interesting problem. Some p4
repositories end up with symlinks that have an empty target. It
is not possible to create this with current p4, but they do
indeed exist.
The effect in git p4 is that "p4 print" on the symlink returns an
empty string, confusing the curret symlink-handling code.
Such broken repositories cause problems in p4 as well, even with
no git involved. In p4, syncing to a change that includes a
bogus symlink causes errors:
//depot/empty-symlink - updating /home/me/p4/empty-symlink
rename: /home/me/p4/empty-symlink: No such file or directory
and leaves no symlink.
In git, replicate the p4 behavior by ignoring these bad symlinks.
If, in a later p4 revision, the symlink happens to point to
something non-null, the symlink will be replaced properly.
Add a big test for all this too.
This happens to be a regression introduced by 1292df1 (git-p4:
Fix occasional truncation of symlink contents., 2013-08-08) and
appeared first in 1.8.5. But it shows up only in p4 repositories
of dubious character, so can wait for a proper release.
Tested-by: Damien Gérard <damien@iwi.me>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The output of git format-patch can vary with user preferences. In
particular setting diff.noprefix will break the "git apply" that
is done as part of "git p4 submit".
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Crestez Dan Leonard <cdleonard@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" does not support many of the view wildcards, such as * and
%%n. It only knows the common ... mapping, and exclusions.
Redo the entire wildcard code around the idea of directly querying
the p4 server for the mapping. For each commit, invoke "p4 where"
with committed file paths as args and use the client mapping to
decide where the file goes in git.
This simplifies a lot of code, and adds support for all wildcards
supported by p4. Downside is that there is probably a 20%-ish
slowdown with this approach.
[pw: redo code and tests]
Signed-off-by: Kazuki Saitoh <ksaitoh560@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Symlink contents in p4 print sometimes have a trailing
new line character, but sometimes it doesn't. git-p4
should only remove the last character if that character
is '\n'.
Signed-off-by: Alex Juncu <ajuncu@ixiacom.com>
Signed-off-by: Alex Badea <abadea@ixiacom.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The generic chdir() helper sets the PWD environment
variable, as that is what is used by p4 to know its
current working directory. Normally the shell would
do this, but in git-p4, we must do it by hand.
However, when the path contains a symbolic link,
os.getcwd() will return the physical location. If the
p4 client specification includes symlinks, setting PWD
to the physical location causes p4 to think it is not
inside the client workspace. It complains, e.g.
Path /vol/bar/projects/foo/... is not under client root /p/foo
One workaround is to use AltRoots in the p4 client specification,
but it is cleaner to handle it directly in git-p4.
Other uses of chdir still require setting PWD to an
absolute path so p4 features like P4CONFIG work. See
bf1d68f (git-p4: use absolute directory for PWD env
var, 2011-12-09).
[ pw: tweak patch and commit message ]
Thanks-to: John Keeping <john@keeping.me.uk>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make the intent of "--bool" more obvious by returning a direct True
or False value. Convert a couple non-bool users with obvious bool
intent.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Invoke git rev-list directly, avoiding the shell, in
P4Submit and P4Sync. The overhead of starting extra
processes is significant in cygwin; this speeds things
up on that platform.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The extra quoting and double-% are unneeded, just to work
around the shell. Instead, avoid the shell indirection.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On windows, p4 marks un-edited files as read-only. Not only are
they read-only, but also they cannot be deleted. Remove the
read-only attribute before deleting in both the copy and rename
cases.
This also happens in the RCS cleanup code, where a file is marked
to be deleted, but must first be edited to remove adjust the
keyword lines. Make sure it is editable before patching.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Files of type utf16 are handled with "p4 print" instead of the
normal "p4 -G print" interface due to how the latter does not
produce correct output. See 55aa571 (git-p4: handle utf16
filetype properly, 2011-09-17) for details.
On windows, though, "p4 print" can not be told which line
endings to use, as there is no underlying client, and always
chooses crlf, even for utf16 files. Convert the \r\n into \n
when importing utf16 files.
The fix for this is complex, in that the problem is a property
of the NT version of p4. There are old versions of p4 that
were compiled directly for cygwin that should not be subjected
to text replacement. The right check here, then, is to look
at the p4 version, not the OS version. Note also that on cygwin,
platform.system() is "CYGWIN_NT-5.1" or similar, not "Windows".
Add a function to memoize the p4 version string and use it to
check for "/NT", indicating the Windows build of p4.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replacing \r\n with \n on windows was added in c1f9197 (Replace
\r\n with \n when importing from p4 on Windows, 2007-05-24), to
work around an oddity with "p4 print" on windows. Text files
are printed with "\r\r\n" endings, regardless of whether they
were created on unix or windows, and regardless of the client
LineEnd setting.
As of d2c6dd3 (use p4CmdList() to get file contents in Python
dicts. This is more robust., 2007-05-23), git-p4 uses "p4 -G
print", which generates files in a raw format. As the native
line ending format if p4 is \n, there will be no \r\n in the
raw text.
Actually, it is possible to generate a text file so that the
p4 representation includes embedded \r\n, even though this is not
normal on either windows or unix. In that case the code would
have mistakenly stripped them out, but now they will be left
intact.
More information on how p4 deals with line endings is here:
http://kb.perforce.com/article/63
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Depot paths must start with //. Exit with a better explanation
when a bad depot path is supplied.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lars Schneider