git-p4 was using "git name-rev" to find out the current branch.
That is not safe, since if multiple branches or tags point at
the same revision, the result obtained might not be what is
expected.
Instead use "git symbolic-ref".
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The existing read_pipe() function returns an empty string on
error, but also returns an empty string if the command returns
an empty string.
This leads to ugly constructions trying to detect error cases.
Add read_pipe_text() which just returns None on error.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a9e38359e3 we taught git-p4 a way to re-encode path names from what
was used in Perforce to UTF-8. This path re-encoding worked properly for
"added" paths. "Removed" paths were not re-encoded and therefore
different from the "added" paths. Consequently, these files were not
removed in a git-p4 cloned Git repository because the path names did not
match.
Fix this by moving the re-encoding to a place that affects "added" and
"removed" paths. Add a test to demonstrate the issue.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running git-p4 on Windows, with multiple git-p4.mapUser entries in
git config - no user mappings are applied to the generated repository.
Reproduction Steps:
1. Add multiple git-p4.mapUser entries to git config on a Windows
machine
2. Attempt to clone a p4 repository
None of the user mappings will be applied.
This issue is actually caused by gitConfigList, using split(os.linesep)
to convert the output of git config --get-all into a list. On Windows,
os.linesep is equal to '\r\n' - however git.exe returns configuration
with a line seperator of '\n'.
This leads to the list returned by gitConfigList containing only one
element - which contains the full output of git config --get-all in
string form, which causes problems for the code introduced to
getUserMapFromPerforceServer in 10d08a149d ("git-p4: map a P4 user to
Git author name and email address", 2016-03-01)
This issue should be caught by the test introduced in 10d08a1, however
would require running on Windows to reproduce.
Using splitlines solves this issue, by splitting config on all
typical delimiters ('\n', '\r\n' etc.)
Signed-off-by: George Vanburgh <gvanburgh@bloomberg.net>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 crashes when used with a very old p4 client version
that does not support the '-r <number>' option in its commands.
Allow making git-p4 work with old p4 clients by setting git-p4.retries to 0.
Alternatively git-p4.retries could be made opt-in.
But since only very old, barely maintained p4 versions don't support
the '-r' option, the setting-retries-to-0 workaround would do.
The "-r retries" option is present in Perforce 2012.2 Command Reference,
but absent from Perforce 2012.1 Command Reference.
Signed-off-by: Igor Kushnir <igorkuo@gmail.com>
Acked-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `git lfs track` command generates a .gitattributes file with diff
and merge properties [1]. Set the same .gitattributes format for files
tracked with GitLFS in git-p4.
[1] https://github.com/git-lfs/git-lfs/blob/v1.5.3/commands/command_track.go#L121
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When importing from multiple perforce paths - we may attempt to
import a changelist that contains files from two (or more) of these
depot paths. Currently, this results in multiple git commits - one
containing the changes, and the other(s) as empty commit(s). This
behavior was introduced in commit 1f90a64891 ("git-p4: reduce number
of server queries for fetches", 2015-12-19).
Reproduction Steps:
1. Have a git repo cloned from a perforce repo using multiple
depot paths (e.g. //depot/foo and //depot/bar).
2. Submit a single change to the perforce repo that makes changes
in both //depot/foo and //depot/bar.
3. Run "git p4 sync" to sync the change from #2.
Change is synced as multiple commits, one for each depot path that
was affected.
Using a set, instead of a list inside p4ChangesForPaths() ensures
that each changelist is unique to the returned list, and therefore
only a single commit is generated for each changelist.
Reported-by: James Farwell <jfarwell@vmware.com>
Signed-off-by: George Vanburgh <gvanburgh@bloomberg.net>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When submitting to P4, if git-p4 came across a symlinked
directory, then during the generation of the submit diff, it would
try to open it as a normal file and fail.
Spot symlinks (of any type) and output a description of the symlink
instead.
Add a test case.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 would attempt to find the git directory using
its own specific code, which did not know about git
worktrees.
Rework it to use "git rev-parse --git-dir" instead.
Add test cases for worktree usage and specifying
git directory via --git-dir and $GIT_DIR.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If git-p4 tried to store an empty file in GitLFS then it crashed while
parsing the pointer file:
oid = re.search(r'^oid \w+:(\w+)', pointerFile, re.MULTILINE).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
This happens because GitLFS does not create a pointer file for an empty
file. Teach git-p4 this behavior to fix the problem and add a test case.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
P4 commands can fail due to random network issues. P4 users can counter
these issues by using a retry flag supported by all p4 commands [1].
Add an integer Git config value `git-p4.retries` to define the number of
retries for all p4 invocations. If the config is not defined then set
the default retry count to 3.
[1] https://www.perforce.com/perforce/doc.current/manuals/cmdref/global.options.html
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Adds new option "--update-shelve CHANGELIST" which updates
an existing shelved changelist.
The original changelist must have been created by the current user.
This allows workflow something like:
hack hack hack
git commit
git p4 submit --shelve
$mail interested parties about shelved changelist
make corrections
git commit --amend
git p4 submit --update-shelve $CHANGELIST
$mail interested parties about shelved changelist
etc
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a --shelve command line argument which invokes p4 shelve instead
of submitting changes. After shelving the changes are reverted from the
p4 workspace.
Signed-off-by: Vinicius Kursancew <viniciusalexandre@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git-P4 used to place temporary refs under "git-p4-tmp". Since 3da1f37
Git checks that all refs are placed under "refs". Instruct Git-P4 to
place temporary refs under "refs/git-p4-tmp". There are no backwards
compatibility considerations as these refs are transient.
Use "git show-ref --verify" to check the (non-)existience of the refs
instead of file checks assuming the file-based ref backend.
All refs under "refs" are shared across all worktrees. This is not
desired for temporary Git-P4 refs and will be adressed in a later patch.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Vitor Antunes <vitor.hda@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The logic here was inverted, you got a message saying the file is
ignored for each file that is not ignored.
Signed-off-by: Andrew Oakley <aoakley@roku.com>
Acked-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git LFS 1.2.0 removed a preamble from the output of the 'git lfs pointer'
command [1] which broke the parsing of this output. Adjust the parser
to support the old and the new format.
Please note that this patch slightly changes the second return parameter
from a list of LF terminated strings to a single string that contains
a number of LF characters.
[1] da2935d9a7
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Helped-by: Sebastian Schuberth <sschuberth@gmail.com>
Helped-by: Ben Woosley <ben.woosley@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When migrating from Perforce to git the information about P4 jobs
associated with P4 changelists is lost.
Having these jobs listed on messages of related git commits enables smooth
migration for projects that take advantage of e.g. JIRA integration
(which uses jobs on Perforce side and parses commit messages on git side).
The jobs are added to the message in the same format as is expected when
migrating in the reverse direction.
Signed-off-by: Jan Durovec <jan.durovec@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Map a P4 user to a specific name and email address in Git with the
"git-p4.mapUser" config. The config value must be a string adhering
to the format "p4user = First Lastname <email@address.com>".
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After changing the type of a file in the git repository, it is not possible to
"git p4 publish" the commit to perforce. This is due to the fact that the git
"T" status is not handled in git-p4.py. This can typically occur when replacing
an existing file with a symbolic link.
The "T" modifier is now supported in git-p4.py. When a file type has changed,
inform perforce with the "p4 edit -f auto" command.
Signed-off-by: Romain Picard <romain.picard@oakbits.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When fetching changes from a depot using a full client spec, there
is no need to perform as many queries as there are top-level paths
in the client spec. Instead we query all changes in chronological
order, also getting rid of the need to sort the results and remove
duplicates.
Signed-off-by: Sam Hocevar <sam@hocevar.net>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When submitting from a repository that was cloned using a client spec,
use the full list of paths when ruling out files that are outside the
view. This fixes a bug where only files pertaining to the first path
would be included in the p4 submit.
Signed-off-by: Sam Hocevar <sam@hocevar.net>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A changelist that contains only excluded files due to a client spec was
imported as an empty commit. Fix that issue by ignoring these commits.
Add option "git-p4.keepEmptyCommits" to make the previous behavior
available.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Helped-by: Pete Harlan
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When "p4 submit" command fails in P4Submit.applyCommit, the
workspace is left with the changes. We already have code to revert
the changes to the workspace when the user decides to cancel
submission by aborting the editor that edits the change description,
and we should treat the "p4 submit" failure the same way.
Clean the workspace if p4_write_pipe raised SystemExit, so that the
user don't have to do it themselves.
Signed-off-by: GIRARD Etienne <egirard@murex.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
When submitting, git-p4 finds the current branch in
order to know if it is allowed to submit (configuration
"git-p4.allowSubmit").
On a detached head, detecting the branch would fail, and
git-p4 would report a cryptic error.
This change teaches git-p4 to recognise a detached head and
submit successfully.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
Add an optional parameter ignore_error to the git-p4 system()
function. If used, it will return the subshell exit status
rather than throwing an exception.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
The ctypes module is used on windows to calculate free disk space,
so it must be imported. We won't need it on other platforms, but
the module is available in Python 2.5 and newer, so importing it
unconditionally is harmless.
Signed-off-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add example implementation including test cases for the large file
system using Git LFS.
Pushing files to the Git LFS server is not tested.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce repositories can contain large (binary) files. Migrating these
repositories to Git generates very large local clones. External storage
systems such as Git LFS [1], Git Fat [2], Git Media [3], git-annex [4]
try to address this problem.
Add a generic mechanism to detect large files based on extension,
uncompressed size, and/or compressed size.
[1] https://git-lfs.github.com/
[2] https://github.com/jedbrown/git-fat
[3] https://github.com/alebedev/git-media
[4] https://git-annex.branchable.com/
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Conflicts:
Documentation/git-p4.txt
git-p4.py
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 will just halt if there is not enough disk space while
streaming content from P4 to Git. Add a check to ensure at least
4 times (arbitrarily chosen) the size of a streamed file is available.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a file is streamed from P4 to Git then the verbose mode prints
continuously the progress as percentage like this:
//depot/file.bin 20% (10 MB)
Upon completion the progress is overwritten with depot source, local
file and size like this:
//depot/file.bin --> local/file.bin (10 MB)
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a git config reader for integer variables. Please note that the
git config implementation automatically supports k, m, and g suffixes.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The functions "gitConfig" and "gitConfigBool" are almost identical.
Make "gitConfig" more generic by adding an optional type specifier.
Use the type specifier "--bool" with "gitConfig" to implement
"gitConfigBool. This prepares the implementation of other type
specifiers such as "--int".
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A P4 repository can get into a state where it contains a file with
type UTF-16 that does not contain a valid UTF-16 BOM. If git-p4
attempts to retrieve the file then the process crashes with a
"Translation of file content failed" error.
More info here: http://answers.perforce.com/articles/KB/3117
Fix this by detecting this error and retrieving the file as binary
instead. The result in Git is the same.
Known issue: This works only if git-p4 is executed in verbose mode.
In normal mode no exceptions are thrown and git-p4 just exits.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If non UTF-8 characters are detected in paths then replace them with
a placeholder instead of throwing a UnicodeDecodeError
exception. This restores the original (implicit) implementation that
was broken in 00a9403.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a path with non-ASCII characters is detected then print the
encoding and the encoded string in verbose mode.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce keeps the encoding of a path as given by the originating OS.
Git expects paths encoded as UTF-8. Add a config to tell git-p4 what
encoding Perforce had used for the paths. This encoding is used to
transcode the paths to UTF-8. As an example, Perforce on Windows often
uses “cp1252” to encode path names.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With --detect-labels enabled, git-p4 will try to create tags
using git fast-import by writing a "tag" clause to the
fast-import stream.
If the commit that the tag references has not yet actually
been processed by fast-import, then the tag can't be created
and git-p4 fails to import the P4 label.
Teach git-p4 to use fast-import "marks" when creating tags
which reference commits created during the current run of the
program.
Commits created before the current run are still referenced
in the old way using a normal git commit.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If p4 reports a tag for a commit that git-p4 does not know
about (e.g. because it references a P4 changelist that was
imported prior to the point at which the repo was cloned into
git), make sure that the error is correctly caught and handled.
rather than just crashing.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce depot may record paths in mixed cases, e.g. "p4 files" may
show that there are these two paths:
//depot/Path/to/file1
//depot/pATH/to/file2
and with "p4" or "p4v", these end up in the same directory, e.g.
//depot/Path/to/file1
//depot/Path/to/file2
which is the desired outcome on case insensitive systems.
If git-p4 is used with client spec "//depot/Path/...", however, then
all files not matching the case in the client spec are ignored (in
the example above "//depot/pATH/to/file2").
Fix this by using the path case that appears first in lexicographical
order when core.ignorecase is set to true. This behavior is consistent
with "p4" and "p4v".
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The --changes-block-size handling was intended to help when
a user has a limited "maxscanrows" (see "p4 group"). It used
"p4 changes -m $maxchanges" to limit the number of results.
Unfortunately, it turns out that the "maxscanrows" and "maxresults"
limits are actually applied *before* the "-m maxchanges" parameter
is considered (experimentally).
Fix the block-size handling so that it gets blocks of changes
limited by revision number ($Start..$Start+$N, etc). This limits
the number of results early enough that both sets of tests pass.
Note that many other Perforce operations can fail for the same
reason (p4 print, p4 files, etc) and it's probably not possible
to workaround this. In the real world, this is probably not
usually a problem.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fixing bug with UTF-16 files when they are retrieved by git-p4. It
was always getting the tip version of the file and the history of
the file was lost.
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This teaches git-p4 to pass the P4EDITOR variable to the
shell for expansion, so that any command-line arguments are
correctly handled. Without this, git-p4 can only launch the
editor if P4EDITOR is solely the path to the binary, without
any arguments.
This also adjusts t9805, which relied on the previous behaviour.
Suggested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce allows client side file/directory remapping through
the use of the client view definition that is part of the
user's client spec.
To support this functionality while branch detection is
enabled it is important to determine the branch location in
the workspace such that the correct files are patched before
Perforce submission. Perforce provides a command that
facilitates this process: p4 where.
This patch does two things to fix improve file location
detection when git-p4 has branch detection and use of client
spec enabled:
1. Enable usage of "p4 where" when Perforce branches exist
in the git repository, even when client specification is
used. This makes use of the already existing function
p4Where.
2. Allow identifying partial matches of the branch's depot
path while processing the output of "p4 where". For
robustness, paths will only match if ending in "/...".
Signed-off-by: Vitor Antunes <vitor.hda@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Simply running "p4 changes" on a large branch can result in a "too
many rows scanned" error from the Perforce server. It is better to
use a sequence of smaller calls to "p4 changes", using the "-m"
option to limit the size of each call.
Signed-off-by: Lex Spoon <lex@lexspoon.org>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a Perforce server is configured to automatically set +l
(exclusive lock) on add of certain file types, git p4 submit will
fail during getP4OpenedType, as the regex doesn't expect the
trailing '*exclusive*' from p4 opened:
//depot/file.png#1 - add default change (binary+l) *exclusive*
Signed-off-by: Blair Holloway <blair_holloway@playstation.sony.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The clone subcommand has long had support for excluding
subdirectories, but sync has not. This is a nuisance,
since as soon as you do a sync, any changed files that
were initially excluded start showing up.
Move the "exclude" command-line option into the parent
class; the actual behavior was already present there so
it simply had to be exposed.
Signed-off-by: Luke Diamand <luke@diamand.org>
Reviewed-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If you use git-p4 with the "--prepare-p4-only" option, then
it prints the p4 command line to use. However, the command
line was incorrect: the changelist specification must be
supplied on standard input, not as an argument to p4.
Signed-off-by: Luke Diamand <luke@diamand.org>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
b4073bb3 (git-p4: Do not include diff in spec file when just
preparing p4, 2014-05-24) broke git p4 submit, here is a proper
fix, including proper handling for windows end of lines.
Signed-off-by: Maxime Coste <frrrwww@gmail.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Luke Diamand