kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Junio C Hamano	c06c79667c	diffcore-rename: somewhat optimized. This changes diffcore-rename to reuse statistics information gathered during similarity estimation, and updates the hashtable implementation used to keep track of the statistics to be denser. This seems to give better performance. Signed-off-by: Junio C Hamano <junkio@cox.net>	19 years ago
Junio C Hamano	4d0f39cecf	diffcore-break: similarity estimator fix. This is a companion patch to the previous fix to diffcore-rename. The merging-back process should use a logic similar to what is used there. Signed-off-by: Junio C Hamano <junkio@cox.net>	19 years ago
Junio C Hamano	65416758cd	diffcore-rename: split out the delta counting code. This is to rework diffcore break/rename/copy detection code so that it does not affected when deltifier code gets improved. Signed-off-by: Junio C Hamano <junkio@cox.net>	19 years ago
Junio C Hamano	581845f0b8	Tweak break/merge score to adjust to the new delta generation code. This lowers the default merge threshold score to 75% from earlier 80%. The break threshold stays the same at 50% for now, but we might want to revisit it (and the rename detection limit as well). * break score: this much edit (both insertion of new material and deletion of old material) needs to be there in the file before we consider this _might_ be a rewrite and break the filepair. * merge score: after a filepair is broken by the above criteria and goes through rename detection, if their pieces did not match with other files as rename/copy, we merge them back into one as if nothing happened. If the filepair had at least this much deletion of old material, however, we say this is completely rewritten with dissimilarity index X% when we do so. The updated delta code by Nico is so good that what we earlier thought to be complete rewrite now reuses a lot more from the source material (reducing the counted "delete"), so this adjustment is needed to keep the perceived behaviour similar to what we had earlier. Signed-off-by: Junio C Hamano <junkio@cox.net>	19 years ago
Junio C Hamano	ee3d299e93	diffcore-break/diffcore-rename: integer overflow. While reviewing the end user tutorial rewrite by J. Bruce Fields, I noticed that "git-diff-tree -B -C" did not correctly break the total rewrite of Documentation/tutorial.txt. It turns out that we had integer overflow during the break score computations. Cop out by using floating point. This is not a kernel. Signed-off-by: Junio C Hamano <junkio@cox.net>	19 years ago
Junio C Hamano	8082d8d305	Diff: -l<num> to limit rename/copy detection. When many paths are modified, rename detection takes a lot of time. The new option -l<num> can be used to disable rename detection when more than <num> paths are possibly created as renames. Signed-off-by: Junio C Hamano <junkio@cox.net>	19 years ago
Junio C Hamano	19397b4521	Revert "[PATCH] plug memory leak in diff.c::diff_free_filepair()" This reverts `068eac91ce` commit.	19 years ago
Yasushi SHOJI	068eac91ce	[PATCH] plug memory leak in diff.c::diff_free_filepair() When I run git-diff-tree on big change, it seems the command eats so much memory. so I just put git under valgrind to see what's going on. diff_free_filespec_data() doesn't free diff_filespec itself. [jc: I ended up doing things slightly differently from Yasushi's patch. The original idea was to use free_filespec_data() only to free the data portion and keep useing the filespec itself, but no existing code seems to do things that way, so I just yanked that part out.] Signed-off-by: Yasushi SHOJI <yashi@atmark-techno.com> Signed-off-by: Junio C Hamano <junkio@cox.net>	20 years ago
Junio C Hamano	dc7090efbc	[PATCH] Re-Fix SIGSEGV on unmerged files in git-diff-files -p When an unmerged path was fed via diff_unmerged() into diffcore, it eventually called run_diff() with "one" and "two" parameters with NULL, but run_diff() was not written carefully enough to notice this situation. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Linus Torvalds	f9e7750621	Fix SIGSEGV on unmerged files in git-diff-files -p NULL is not considered a VALID pathspec.	20 years ago
Junio C Hamano	f78c79c5d4	[PATCH] diffcore-break.c: various fixes. This fixes three bugs in the -B heuristics. - Although it was advertised that the initial break criteria used was the same as what diffcore-rename uses, it was using something different. Instead of using smaller of src and dst size to compare with "edit" size, (insertion and deletion), it was using larger of src and dst, unlike the rename/copy detection logic. This caused the parameter to -B to mean something different from the one to -M and -C. To compensate for this change, the default break score is also changed to match that of the default for rename/copy. - The code would have crashed with division by zero when trying to break an originally empty file. - Contrary to what the comment said, the algorithm was breaking small files, only to later merge them together. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	eeaa460314	[PATCH] diff: Update -B heuristics. As Linus pointed out on the mailing list discussion, -B should break a files that has many inserts even if it still keeps enough of the original contents, so that the broken pieces can later be matched with other files by -M or -C. However, if such a broken pair does not get picked up by -M or -C, we would want to apply different criteria; namely, regardless of the amount of new material in the result, the determination of "rewrite" should be done by looking at the amount of original material still left in the result. If you still have the original 97 lines from a 100-line document, it does not matter if you add your own 13 lines to make a 110-line document, or if you add 903 lines to make a 1000-line document. It is not a rewrite but an in-place edit. On the other hand, if you did lose 97 lines from the original, it does not matter if you added 27 lines to make a 30-line document or if you added 997 lines to make a 1000-line document. You did a complete rewrite in either case. This patch introduces a post-processing phase that runs after diffcore-rename matches up broken pairs diffcore-break creates. The purpose of this post-processing is to pick up these broken pieces and merge them back into in-place modifications. For this, the score parameter -B option takes is changed into a pair of numbers, and it takes "-B99/80" format when fully spelled out. The first number is the minimum amount of "edit" (same definition as what diffcore-rename uses, which is "sum of deletion and insertion") that a modification needs to have to be broken, and the second number is the minimum amount of "delete" a surviving broken pair must have to avoid being merged back together. It can be abbreviated to "-B" to use default for both, "-B9" or "-B9/" to use 90% for "edit" but default (80%) for merge avoidance, or "-B/75" to use default (99%) "edit" and 75% for merge avoidance. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	ce24067549	[PATCH] diff: Fix docs and add -O to diff-helper. This patch updates diff documentation and usage strings: - clarify the semantics of -R. It is not "output in reverse"; rather, it is "I will feed diff backwards". Semantically they are different when -C is involved. - describe -O in usage strings of diff-* brothers. It was implemented, documented but not described in usage text. Also it adds -O to diff-helper. Like -S (and unlike -M/-C/-B), this option can work on sanitized diff-raw output produced by the diff-* brothers. While we are at it, the call it makes to diffcore is cleaned up to use the diffcore_std() like everybody else, and the declaration for the low level diffcore routines are moved from diff.h (public) to diffcore.h (private between diff.c and diffcore backends). Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	355e76a4a3	[PATCH] Tweak count-delta interface Make it return copied source and insertion separately, so that later implementation of heuristics can use them more flexibly. This does not change the heuristics implemented in diffcore-rename nor diffcore-break in any way. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	67574c403f	[PATCH] diff: mode bits fixes The core GIT repository has trees that record regular file mode in 0664 instead of normalized 0644 pattern. Comparing such a tree with another tree that records the same file in 0644 pattern without content changes with git-diff-tree causes it to feed otherwise unmodified pairs to the diff_change() routine, which triggers a sanity check routine and barfs. This patch fixes the problem, along with the fix to another caller that uses unnormalized mode bits to call diff_change() routine in a similar way. Without this patch, you will see "fatal error" from diff-tree when you run git-deltafy-script on the core GIT repository itself. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	f345b0a066	[PATCH] Add -B flag to diff-* brothers. A new diffcore transformation, diffcore-break.c, is introduced. When the -B flag is given, a patch that represents a complete rewrite is broken into a deletion followed by a creation. This makes it easier to review such a complete rewrite patch. The -B flag takes the same syntax as the -M and -C flags to specify the minimum amount of non-source material the resulting file needs to have to be considered a complete rewrite, and defaults to 99% if not specified. As the new test t4008-diff-break-rewrite.sh demonstrates, if a file is a complete rewrite, it is broken into a delete/create pair, which can further be subjected to the usual rename detection if -M or -C is used. For example, if file0 gets completely rewritten to make it as if it were rather based on file1 which itself disappeared, the following happens: The original change looks like this: file0 --> file0' (quite different from file0) file1 --> /dev/null After diffcore-break runs, it would become this: file0 --> /dev/null /dev/null --> file0' file1 --> /dev/null Then diffcore-rename matches them up: file1 --> file0' The internal score values are finer grained now. Earlier maximum of 10000 has been raised to 60000; there is no user visible changes but there is no reason to waste available bits. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	01c4e70f63	[PATCH] diff: code clean-up and removal of rename hack. A new macro, DIFF_PAIR_RENAME(), is introduced to distinguish a filepair that is a rename/copy (the definition of which is src and dst are different paths, of course). This removes the hack used in the record_rename_pair() to always put a non-zero value in the score field. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	f0c6b2a2fd	[PATCH] Optimize diff-tree -[CM] --stdin This attempts to optimize "diff-tree -[CM] --stdin", which compares successible tree pairs. This optimization does not make much sense for other commands in the diff-* brothers. When reading from --stdin and using rename/copy detection, the patch makes diff-tree to read the current index file first. This is done to reuse the optimization used by diff-cache in the non-cached case. Similarity estimator can avoid expanding a blob if the index says what is in the work tree has an exact copy of that blob already expanded. Another optimization the patch makes is to check only file sizes first to terminate similarity estimation early. In order for this to work, it needs a way to tell the size of the blob without expanding it. Since an obvious way of doing it, which is to keep all the blobs previously used in the memory, is too costly, it does so by keeping the filesize for each object it has already seen in memory. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	15d061b435	[PATCH] Fix the way diffcore-rename records unremoved source. Earier version of diffcore-rename used to keep unmodified filepair in its output so that the last stage of the processing that tells renames from copies can make all of rename/copy to copies. However this had a bad interaction with other diffcore filters that wanted to run after diffcore-rename, in that such unmodified filepair must be retained for proper distinction between renames and copies to happen. This patch fixes the problem by changing the way diffcore-rename records the information needed to distinguish "all are copies" case and "the last one is a rename" case. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	be020332a1	[PATCH] Remove a function not used anymore. Earlier rename/copy detection left unmodified filepair in the output and forced downstream to keep them even when they are filtering, and the diff_needs_to_stay() function was used for the logic. It is not used anymore, so remove it. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	226406f693	[PATCH] Introduce diff_free_filepair() funcion. This introduces a new function to free a common data structure, and plugs some leaks. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	4130b99571	[PATCH] Diff updates to express type changes With the introduction of type 'T' in the diff-raw output, and the "apply-patch" program Linus has been quietly working on without much advertisement, it started to make sense to emit usable information in the "diff --git" patch output format as well. Earlier built-in diff driver punted and did not say anything about a symbolic link changing into a file or vice versa, but this version represents it as a pair of deletion and creation. It also fixes a minor problem dealing with old archive created with ancient git. The earlier code was reporting file mode change between 100664 and 100644 (we shouldn't). The linux-2.6 git tree has a good example that exposes this problem. A good test case is commit ce1dc02f76432a46db149241e015a4f782974623. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	96716a1976	[PATCH] Fix type-change handling when assigning the status code to filepairs. The interim single-liner '?' fix resulted delete entries that should not have emitted coming out in the output as an unintended side effect; I caught this with the "rename" test in the test suite. This patch instead fixes the code that assigns the status code to each filepair. I verified this does not break the testcase in udev.git tree Kay Sievers gave us, by running git-diff-tree on that tree which showed 21 file to symlink changes. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	25d5ea410f	[PATCH] Redo rename/copy detection logic. Earlier implementation had a major screw-up in the memory management area. Rename/copy logic sometimes borrowed a pointer to a structure without any provision for downstream to determine which pointer is shared and which is not. This resulted in the later clean-up code to sometimes double free such structure, resulting in a segfault. This made -M and -C useless. Another problem the earlier implementation had was that it reordered the patches, and forced the logic to differentiate renames and copies to depend on that particular order. This problem was fixed by teaching rename/copy detection logic not to do any reordering, and rename-copy differentiator not to depend on the order of the patches. The diffs will leave rename/copy detector in the same destination path order as the patch that was fed into it. Some test vectors have been reordered to accommodate this change. It also adds a sanity check logic to the human-readable diff-raw output to detect paths with embedded TAB and LF characters, which cannot be expressed with that format. This idea came up during a discussion with Chris Wedgwood. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	bceafe752c	[PATCH] Fix diff-pruning logic which was running prune too early. For later stages to reorder patches, pruning logic and rename detection logic should not decide which delete to discard (because another entry said it will take over the file as a rename) until the very end. Also fix some tests that were assuming the earlier "last one is rename or keep everything else is copy" semantics of diff-raw format, which no longer is true. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	b6d8f309d9	[PATCH] diff-raw format update take #2 . This changes the diff-raw format again, following the mailing list discussion. The new format explicitly expresses which one is a rename and which one is a copy. The documentation and tests are updated to match this change. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	f7c1512af8	[PATCH] Rename/copy detection fix. The rename/copy detection logic in earlier round was only good enough to show patch output and discussion on the mailing list about the diff-raw format updates revealed many problems with it. This patch fixes all the ones known to me, without making things I want to do later impossible, mostly related to patch reordering. (1) Earlier rename/copy detector determined which one is rename and which one is copy too early, which made it impossible to later introduce diffcore transformers to reorder patches. This patch fixes it by moving that logic to the very end of the processing. (2) Earlier output routine diff_flush() was pruning all the "no-change" entries indiscriminatingly. This was done due to my false assumption that one of the requirements in the diff-raw output was not to show such an entry (which resulted in my incorrect comment about "diff-helper never being able to be equivalent to built-in diff driver"). My special thanks go to Linus for correcting me about this. When we produce diff-raw output, for the downstream to be able to tell renames from copies, sometimes it _is_ necessary to output "no-change" entries, and this patch adds diffcore_prune() function for doing it. (3) Earlier diff_filepair structure was trying to be not too specific about rename/copy operations, but the purpose of the structure was to record one or two paths, which _was_ indeed about rename/copy. This patch discards xfrm_msg field which was trying to be generic for this wrong reason, and introduces a couple of fields (rename_score and rename_rank) that are explicitly specific to rename/copy logic. One thing to note is that the information in a single diff_filepair structure _still_ does not distinguish renames from copies, and it is deliberately so. This is to allow patches to be reordered in later stages. (4) This patch also adds some tests about diff-raw format output and makes sure that necessary "no-change" entries appear on the output. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	6b14d7faf0	[PATCH] Diffcore updates. This moves the path selection logic from individual programs to a new diffcore transformer (diff-tree still needs to have its own for performance reasons). Also the header printing code in diff-tree was tweaked not to produce anything when pickaxe is in effect and there is nothing interesting to report. An interesting example is the following in the GIT archive itself: $ git-whatchanged -p -C -S'or something in a real script' Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	81e50eabf0	[PATCH] The diff-raw format updates. Update the diff-raw format as Linus and I discussed, except that it does not use sequence of underscore '_' letters to express nonexistence. All '0' mode is used for that purpose instead. The new diff-raw format can express rename/copy, and the earlier restriction that -M and -C _must_ be used with the patch format output is no longer necessary. The patch makes -M and -C flags independent of -p flag, so you need to say git-whatchanged -M -p to get the diff/patch format. Updated are both documentations and tests. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	38c6f78059	[PATCH] Prepare diffcore interface for diff-tree header supression. This does not actually supress the extra headers when pickaxe is used, but prepares enough support for diff-tree to implement it. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	52e9578985	[PATCH] Introducing software archaeologist's tool "pickaxe". This steals the "pickaxe" feature from JIT and make it available to the bare Plumbing layer. From the command line, the user gives a string he is intersted in. Using the diff-core infrastructure previously introduced, it filters the differences to limit the output only to the diffs between <src> and <dst> where the string appears only in one but not in the other. For example: $ ./git-rev-list HEAD \| ./git-diff-tree -Sdiff-tree-helper --stdin -M would show the diffs that touch the string "diff-tree-helper". In real software-archaeologist application, you would typically look for a few to several lines of code and see where that code came from. The "pickaxe" module runs after "rename/copy detection" module, so it even crosses the file rename boundary, as the above example demonstrates. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago
Junio C Hamano	427dcb4bca	[PATCH] Diff overhaul, adding half of copy detection. This introduces the diff-core, the layer between the diff-tree family and the external diff interface engine. The calls to the interface diff-tree family uses (diff_change and diff_addremove) have not changed and will not change. The purpose of the diff-core layer is to provide an infrastructure to transform the set of differences sent from the applications, before sending them to the external diff interface. The recently introduced rename detection code has been rewritten to use the diff-core facility. When applications send in separate creates and deletes, matching ones are transformed into a single rename-and-edit diff, and sent out to the external diff interface as such. This patch also enhances the rename detection code further to be able to detect copies. Currently this happens only as long as copy sources appear as part of the modified files, but there already is enough provision for callers to report unmodified files to diff-core, so that they can be also used as copy source candidates. Extending the callers this way will be done in a separate patch. Please see and marvel at how well this works by trying out the newly added t/t4003-diff-rename-1.sh test script. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>	20 years ago

31 Commits (26a8ad25b28a1cb906c88bdf539d840774ca5aeb)