kernel
/
git
mirror of https://git.kernel.org/pub/scm/git/git.git
1
0
Fork 0
Code Releases Activity
Browse Source

filter-branch: Big syntax change; support rewriting multiple refs 

We used to take the first non-option argument as the name for the new
branch.  This syntax is not extensible to support rewriting more than just
HEAD.

Instead, we now have the following syntax:

	git filter-branch [<filter options>...] [<rev-list options>]

All positive refs given in <rev-list options> are rewritten.  Yes,
in-place.  If a ref was changed, the original head is stored in
refs/original/$ref now, for your inspecting pleasure, in addition to the
reflogs (since it is easier to inspect "git show-ref | grep original" than
to inspect all the reflogs).

This commit also adds the --force option to remove .git-rewrite/ and all
refs from refs/original/ before filtering.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
maint
Johannes Schindelin 18 years ago committed by Junio C Hamano
parent
3b38ec16d5
commit
dfd05e38f0
3 changed files with 182 additions and 60 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 51
      Documentation/git-filter-branch.txt
  2. 150
      git-filter-branch.sh
  3. 41
      t/t7003-filter-branch.sh

51
Documentation/git-filter-branch.txt
Unescape View File

@ -12,7 +12,7 @@ SYNOPSIS
[--index-filter <command>] [--parent-filter <command>] [--index-filter <command>] [--parent-filter <command>]
[--msg-filter <command>] [--commit-filter <command>] [--msg-filter <command>] [--commit-filter <command>]
[--tag-name-filter <command>] [--subdirectory-filter <directory>] [--tag-name-filter <command>] [--subdirectory-filter <directory>]
[-d <directory>] <new-branch-name> [<rev-list options>...] [-d <directory>] [-f | --force] [<rev-list options>...]


DESCRIPTION DESCRIPTION
----------- -----------
@ -26,10 +26,9 @@ information) will be preserved.
The command takes the new branch name as a mandatory argument and The command takes the new branch name as a mandatory argument and
the filters as optional arguments. If you specify no filters, the the filters as optional arguments. If you specify no filters, the
commits will be recommitted without any changes, which would normally commits will be recommitted without any changes, which would normally
have no effect and result in the new branch pointing to the same have no effect. Nevertheless, this may be useful in the future for
branch as your current branch. Nevertheless, this may be useful in compensating for some git bugs or such, therefore such a usage is
the future for compensating for some git bugs or such, therefore permitted.
such a usage is permitted.


*WARNING*! The rewritten history will have different object names for all *WARNING*! The rewritten history will have different object names for all
the objects and will not converge with the original branch. You will not the objects and will not converge with the original branch. You will not
@ -38,8 +37,9 @@ original branch. Please do not use this command if you do not know the
full implications, and avoid using it anyway, if a simple single commit full implications, and avoid using it anyway, if a simple single commit
would suffice to fix your problem. would suffice to fix your problem.


Always verify that the rewritten version is correct before disposing Always verify that the rewritten version is correct: The original refs,
the original branch. if different from the rewritten ones, will be stored in the namespace
'refs/original/'.


Note that since this operation is extensively I/O expensive, it might Note that since this operation is extensively I/O expensive, it might
be a good idea to redirect the temporary directory off-disk, e.g. on be a good idea to redirect the temporary directory off-disk, e.g. on
@ -142,6 +142,11 @@ definition impossible to preserve signatures at any rate.)
does this in the '.git-rewrite/' directory but you can override does this in the '.git-rewrite/' directory but you can override
that choice by this parameter. that choice by this parameter.


-f\|--force::
`git filter-branch` refuses to start with an existing temporary
directory or when there are already refs starting with
'refs/original/', unless forced.

<rev-list-options>:: <rev-list-options>::
When options are given after the new branch name, they will When options are given after the new branch name, they will
be passed to gitlink:git-rev-list[1]. Only commits in the resulting be passed to gitlink:git-rev-list[1]. Only commits in the resulting
@ -156,14 +161,14 @@ Suppose you want to remove a file (containing confidential information
or copyright violation) from all commits: or copyright violation) from all commits:


------------------------------------------------------- -------------------------------------------------------
git filter-branch --tree-filter 'rm filename' newbranch git filter-branch --tree-filter 'rm filename' HEAD
------------------------------------------------------- -------------------------------------------------------


A significantly faster version: A significantly faster version:


------------------------------------------------------------------------------- --------------------------------------------------------------------------
git filter-branch --index-filter 'git update-index --remove filename' newbranch git filter-branch --index-filter 'git update-index --remove filename' HEAD
------------------------------------------------------------------------------- --------------------------------------------------------------------------


Now, you will get the rewritten history saved in the branch 'newbranch' Now, you will get the rewritten history saved in the branch 'newbranch'
(your current branch is left untouched). (your current branch is left untouched).
@ -172,25 +177,25 @@ To set a commit (which typically is at the tip of another
history) to be the parent of the current initial commit, in history) to be the parent of the current initial commit, in
order to paste the other history behind the current history: order to paste the other history behind the current history:


------------------------------------------------------------------------ -------------------------------------------------------------------
git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' newbranch git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' HEAD
------------------------------------------------------------------------ -------------------------------------------------------------------


(if the parent string is empty - therefore we are dealing with the (if the parent string is empty - therefore we are dealing with the
initial commit - add graftcommit as a parent). Note that this assumes initial commit - add graftcommit as a parent). Note that this assumes
history with a single root (that is, no merge without common ancestors history with a single root (that is, no merge without common ancestors
happened). If this is not the case, use: happened). If this is not the case, use:


------------------------------------------------------------------------------- --------------------------------------------------------------------------
git filter-branch --parent-filter \ git filter-branch --parent-filter \
'cat; test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>"' newbranch 'cat; test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>"' HEAD
------------------------------------------------------------------------------- --------------------------------------------------------------------------


or even simpler: or even simpler:


----------------------------------------------- -----------------------------------------------
echo "$commit-id $graft-id" >> .git/info/grafts echo "$commit-id $graft-id" >> .git/info/grafts
git filter-branch newbranch $graft-id.. git filter-branch $graft-id..HEAD
----------------------------------------------- -----------------------------------------------


To remove commits authored by "Darl McBribe" from the history: To remove commits authored by "Darl McBribe" from the history:
@ -208,7 +213,7 @@ git filter-branch --commit-filter '
done; done;
else else
git commit-tree "$@"; git commit-tree "$@";
fi' newbranch fi' HEAD
------------------------------------------------------------------------------ ------------------------------------------------------------------------------


The shift magic first throws away the tree id and then the -p The shift magic first throws away the tree id and then the -p
@ -238,14 +243,14 @@ A--B-----C
To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use: To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use:


-------------------------------- --------------------------------
git filter-branch ... new-H C..H git filter-branch ... C..H
-------------------------------- --------------------------------


To rewrite commits E,F,G,H, use one of these: To rewrite commits E,F,G,H, use one of these:


---------------------------------------- ----------------------------------------
git filter-branch ... new-H C..H --not D git filter-branch ... C..H --not D
git filter-branch ... new-H D..H --not C git filter-branch ... D..H --not C
---------------------------------------- ----------------------------------------


To move the whole tree into a subdirectory, or remove it from there: To move the whole tree into a subdirectory, or remove it from there:
@ -255,7 +260,7 @@ git filter-branch --index-filter \
'git ls-files -s | sed "s-\t-&newsubdir/-" | 'git ls-files -s | sed "s-\t-&newsubdir/-" |
GIT_INDEX_FILE=$GIT_INDEX_FILE.new \ GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
git update-index --index-info && git update-index --index-info &&
mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' directorymoved mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD
--------------------------------------------------------------- ---------------------------------------------------------------





150
git-filter-branch.sh
Unescape View File

@ -78,6 +78,8 @@ filter_msg=cat
filter_commit='git commit-tree "$@"' filter_commit='git commit-tree "$@"'
filter_tag_name= filter_tag_name=
filter_subdir= filter_subdir=
orig_namespace=refs/original/
force=
while case "$#" in 0) usage;; esac while case "$#" in 0) usage;; esac
do do
case "$1" in case "$1" in
@ -85,6 +87,11 @@ do
shift shift
break break
;; ;;
--force|-f)
shift
force=t
continue
;;
-*) -*)
;; ;;
*) *)
@ -126,24 +133,43 @@ do
--subdirectory-filter) --subdirectory-filter)
filter_subdir="$OPTARG" filter_subdir="$OPTARG"
;; ;;
--original)
orig_namespace="$OPTARG"
;;
*) *)
usage usage
;; ;;
esac esac
done done


dstbranch="$1" case "$force" in
shift t)
test -n "$dstbranch" || die "missing branch name" rm -rf "$tempdir"
git show-ref "refs/heads/$dstbranch" 2> /dev/null && ;;
die "branch $dstbranch already exists" '')

test -d "$tempdir" &&
test ! -e "$tempdir" || die "$tempdir already exists, please remove it" die "$tempdir already exists, please remove it"
esac
mkdir -p "$tempdir/t" && mkdir -p "$tempdir/t" &&
tempdir="$(cd "$tempdir"; pwd)" &&
cd "$tempdir/t" && cd "$tempdir/t" &&
workdir="$(pwd)" || workdir="$(pwd)" ||
die "" die ""


# Make sure refs/original is empty
git for-each-ref > "$tempdir"/backup-refs
while read sha1 type name
do
case "$force,$name" in
,$orig_namespace*)
die "Namespace $orig_namespace not empty"
;;
t,$orig_namespace*)
git update-ref -d "$name" $sha1
;;
esac
done < "$tempdir"/backup-refs

case "$GIT_DIR" in case "$GIT_DIR" in
/*) /*)
;; ;;
@ -153,6 +179,29 @@ case "$GIT_DIR" in
esac esac
export GIT_DIR GIT_WORK_TREE=. export GIT_DIR GIT_WORK_TREE=.


# These refs should be updated if their heads were rewritten

git rev-parse --revs-only --symbolic "$@" |
while read ref
do
# normalize ref
case "$ref" in
HEAD)
ref="$(git symbolic-ref "$ref")"
;;
refs/*)
;;
*)
ref="$(git for-each-ref --format='%(refname)' |
grep /"$ref")"
esac

git check-ref-format "$ref" && echo "$ref"
done > "$tempdir"/heads

test -s "$tempdir"/heads ||
die "Which ref do you want to rewrite?"

export GIT_INDEX_FILE="$(pwd)/../index" export GIT_INDEX_FILE="$(pwd)/../index"
git read-tree || die "Could not seed the index" git read-tree || die "Could not seed the index"


@ -174,6 +223,8 @@ commits=$(wc -l <../revs | tr -d " ")


test $commits -eq 0 && die "Found nothing to rewrite" test $commits -eq 0 && die "Found nothing to rewrite"


# Rewrite the commits

i=0 i=0
while read commit parents; do while read commit parents; do
i=$(($i+1)) i=$(($i+1))
@ -234,22 +285,75 @@ while read commit parents; do
$(git write-tree) $parentstr < ../message > ../map/$commit $(git write-tree) $parentstr < ../message > ../map/$commit
done <../revs done <../revs


src_head=$(tail -n 1 ../revs | sed -e 's/ .*//') # In case of a subdirectory filter, it is possible that a specified head
target_head=$(head -n 1 ../map/$src_head) # is not in the set of rewritten commits, because it was pruned by the
case "$target_head" in # revision walker. Fix it by mapping these heads to the next rewritten
'') # ancestor(s), i.e. the boundaries in the set of rewritten commits.
echo Nothing rewritten
# NEEDSWORK: we should sort the unmapped refs topologically first
while read ref
do
sha1=$(git rev-parse "$ref"^0)
test -f "$workdir"/../map/$sha1 && continue
# Assign the boundarie(s) in the set of rewritten commits
# as the replacement commit(s).
# (This would look a bit nicer if --not --stdin worked.)
for p in $((cd "$workdir"/../map; ls | sed "s/^/^/") |
git rev-list $ref --boundary --stdin |
sed -n "s/^-//p")
do
map $p >> "$workdir"/../map/$sha1
done
done < "$tempdir"/heads

# Finally update the refs

_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
count=0
echo
while read ref
do
# avoid rewriting a ref twice
test -f "$orig_namespace$ref" && continue

sha1=$(git rev-parse "$ref"^0)
rewritten=$(map $sha1)

test $sha1 = "$rewritten" &&
warn "WARNING: Ref '$ref' is unchanged" &&
continue

case "$rewritten" in
'')
echo "Ref '$ref' was deleted"
git update-ref -m "filter-branch: delete" -d "$ref" $sha1 ||
die "Could not delete $ref"
;; ;;
*) $_x40)
git update-ref refs/heads/"$dstbranch" $target_head || echo "Ref '$ref' was rewritten"
die "Could not update $dstbranch with $target_head" git update-ref -m "filter-branch: rewrite" \
if [ $(wc -l <../map/$src_head) -gt 1 ]; then "$ref" $rewritten $sha1 ||
echo "WARNING: Your commit filter caused the head commit to expand to several rewritten commits. Only the first such commit was recorded as the current $dstbranch head but you will need to resolve the situation now (probably by manually merging the other commits). These are all the commits:" >&2 die "Could not rewrite $ref"
sed 's/^/ /' ../map/$src_head >&2
ret=1
fi
;; ;;
esac *)
# NEEDSWORK: possibly add -Werror, making this an error
warn "WARNING: '$ref' was rewritten into multiple commits:"
warn "$rewritten"
warn "WARNING: Ref '$ref' points to the first one now."
rewritten=$(echo "$rewritten" | head -n 1)
git update-ref -m "filter-branch: rewrite to first" \
"$ref" $rewritten $sha1 ||
die "Could not rewrite $ref"
;;
esac
git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1
count=$(($count+1))
done < "$tempdir"/heads

# TODO: This should possibly go, with the semantics that all positive given
# refs are updated, and their original heads stored in refs/original/
# Filter tags


if [ "$filter_tag_name" ]; then if [ "$filter_tag_name" ]; then
git for-each-ref --format='%(objectname) %(objecttype) %(refname)' refs/tags | git for-each-ref --format='%(objectname) %(objecttype) %(refname)' refs/tags |
@ -286,6 +390,8 @@ fi


cd ../.. cd ../..
rm -rf "$tempdir" rm -rf "$tempdir"
printf "\nRewritten history saved to the $dstbranch branch\n" echo
test $count -gt 0 && echo "These refs were rewritten:"
git show-ref | grep ^"$orig_namespace"


exit $ret exit $ret

41
t/t7003-filter-branch.sh
Unescape View File

@ -30,24 +30,24 @@ test_expect_success 'setup' '
H=$(git rev-parse H) H=$(git rev-parse H)


test_expect_success 'rewrite identically' ' test_expect_success 'rewrite identically' '
git-filter-branch H2 git-filter-branch branch
' '

test_expect_success 'result is really identical' ' test_expect_success 'result is really identical' '
test $H = $(git rev-parse H2) test $H = $(git rev-parse HEAD)
' '


test_expect_success 'rewrite, renaming a specific file' ' test_expect_success 'rewrite, renaming a specific file' '
git-filter-branch --tree-filter "mv d doh || :" H3 git-filter-branch -f --tree-filter "mv d doh || :" HEAD
' '


test_expect_success 'test that the file was renamed' ' test_expect_success 'test that the file was renamed' '
test d = $(git show H3:doh) test d = $(git show HEAD:doh)
' '


git tag oldD H3~4 git tag oldD HEAD~4
test_expect_success 'rewrite one branch, keeping a side branch' ' test_expect_success 'rewrite one branch, keeping a side branch' '
git-filter-branch --tree-filter "mv b boh || :" modD D..oldD git branch modD oldD &&
git-filter-branch -f --tree-filter "mv b boh || :" D..modD
' '


test_expect_success 'common ancestor is still common (unchanged)' ' test_expect_success 'common ancestor is still common (unchanged)' '
@ -69,7 +69,8 @@ test_expect_success 'filter subdirectory only' '
git rm a && git rm a &&
test_tick && test_tick &&
git commit -m "again not subdir" && git commit -m "again not subdir" &&
git-filter-branch --subdirectory-filter subdir sub git branch sub &&
git-filter-branch -f --subdirectory-filter subdir refs/heads/sub
' '


test_expect_success 'subdirectory filter result looks okay' ' test_expect_success 'subdirectory filter result looks okay' '
@ -89,7 +90,8 @@ test_expect_success 'setup and filter history that requires --full-history' '
test_tick && test_tick &&
git commit -m "again subdir on master" && git commit -m "again subdir on master" &&
git merge branch && git merge branch &&
git-filter-branch --subdirectory-filter subdir sub-master git branch sub-master &&
git-filter-branch -f --subdirectory-filter subdir sub-master
' '


test_expect_success 'subdirectory filter result looks okay' ' test_expect_success 'subdirectory filter result looks okay' '
@ -100,7 +102,8 @@ test_expect_success 'subdirectory filter result looks okay' '
' '


test_expect_success 'use index-filter to move into a subdirectory' ' test_expect_success 'use index-filter to move into a subdirectory' '
git-filter-branch --index-filter \ git branch directorymoved &&
git-filter-branch -f --index-filter \
"git ls-files -s | sed \"s-\\t-&newsubdir/-\" | "git ls-files -s | sed \"s-\\t-&newsubdir/-\" |
GIT_INDEX_FILE=\$GIT_INDEX_FILE.new \ GIT_INDEX_FILE=\$GIT_INDEX_FILE.new \
git update-index --index-info && git update-index --index-info &&
@ -108,9 +111,10 @@ test_expect_success 'use index-filter to move into a subdirectory' '
test -z "$(git diff HEAD directorymoved:newsubdir)"' test -z "$(git diff HEAD directorymoved:newsubdir)"'


test_expect_success 'stops when msg filter fails' ' test_expect_success 'stops when msg filter fails' '
! git-filter-branch --msg-filter false nonono && old=$(git rev-parse HEAD) &&
rm -rf .git-rewrite && ! git-filter-branch -f --msg-filter false &&
! git rev-parse nonono test $old = $(git rev-parse HEAD) &&
rm -rf .git-rewrite
' '


test_expect_success 'author information is preserved' ' test_expect_success 'author information is preserved' '
@ -118,7 +122,8 @@ test_expect_success 'author information is preserved' '
git add i && git add i &&
test_tick && test_tick &&
GIT_AUTHOR_NAME="B V Uips" git commit -m bvuips && GIT_AUTHOR_NAME="B V Uips" git commit -m bvuips &&
git-filter-branch --msg-filter "cat; \ git branch preserved-author &&
git-filter-branch -f --msg-filter "cat; \
test \$GIT_COMMIT != $(git rev-parse master) || \ test \$GIT_COMMIT != $(git rev-parse master) || \
echo Hallo" \ echo Hallo" \
preserved-author && preserved-author &&
@ -129,7 +134,8 @@ test_expect_success "remove a certain author's commits" '
echo i > i && echo i > i &&
test_tick && test_tick &&
git commit -m i i && git commit -m i i &&
git-filter-branch --commit-filter "\ git branch removed-author &&
git-filter-branch -f --commit-filter "\
if [ \"\$GIT_AUTHOR_NAME\" = \"B V Uips\" ];\ if [ \"\$GIT_AUTHOR_NAME\" = \"B V Uips\" ];\
then\ then\
shift;\ shift;\
@ -148,4 +154,9 @@ test_expect_success "remove a certain author's commits" '
test 0 = $(git rev-list --author="B V Uips" removed-author | wc -l) test 0 = $(git rev-list --author="B V Uips" removed-author | wc -l)
' '


test_expect_success 'barf on invalid name' '
! git filter-branch -f master xy-problem &&
! git filter-branch -f HEAD^
'

test_done test_done

Write Preview
Loading…
Cancel
Save