Merge branch 'cc/split-index-config'
The experimental "split index" feature has gained a few
configuration variables to make it easier to use.
* cc/split-index-config: (22 commits)
  Documentation/git-update-index: explain splitIndex.*
  Documentation/config: add splitIndex.sharedIndexExpire
  read-cache: use freshen_shared_index() in read_index_from()
  read-cache: refactor read_index_from()
  t1700: test shared index file expiration
  read-cache: unlink old sharedindex files
  config: add git_config_get_expiry() from gc.c
  read-cache: touch shared index files when used
  sha1_file: make check_and_freshen_file() non static
  Documentation/config: add splitIndex.maxPercentChange
  t1700: add tests for splitIndex.maxPercentChange
  read-cache: regenerate shared index if necessary
  config: add git_config_get_max_percent_split_change()
  Documentation/git-update-index: talk about core.splitIndex config var
  Documentation/config: add information for core.splitIndex
  t1700: add tests for core.splitIndex
  update-index: warn in case of split-index incoherency
  read-cache: add and then use tweak_split_index()
  split-index: add {add,remove}_split_index() functions
  config: add git_config_get_split_index()
  ...
			
			
				maint
			
			
		
						commit
						94c9b5af70
					
				|  | @ -334,6 +334,10 @@ core.trustctime:: | |||
| 	crawlers and some backup systems). | ||||
| 	See linkgit:git-update-index[1]. True by default. | ||||
|  | ||||
| core.splitIndex:: | ||||
| 	If true, the split-index feature of the index will be used. | ||||
| 	See linkgit:git-update-index[1]. False by default. | ||||
|  | ||||
| core.untrackedCache:: | ||||
| 	Determines what to do about the untracked cache feature of the | ||||
| 	index. It will be kept, if this variable is unset or set to | ||||
|  | @ -2850,6 +2854,31 @@ showbranch.default:: | |||
| 	The default set of branches for linkgit:git-show-branch[1]. | ||||
| 	See linkgit:git-show-branch[1]. | ||||
|  | ||||
| splitIndex.maxPercentChange:: | ||||
| 	When the split index feature is used, this specifies the | ||||
| 	percent of entries the split index can contain compared to the | ||||
| 	total number of entries in both the split index and the shared | ||||
| 	index before a new shared index is written. | ||||
| 	The value should be between 0 and 100. If the value is 0 then | ||||
| 	a new shared index is always written, if it is 100 a new | ||||
| 	shared index is never written. | ||||
| 	By default the value is 20, so a new shared index is written | ||||
| 	if the number of entries in the split index would be greater | ||||
| 	than 20 percent of the total number of entries. | ||||
| 	See linkgit:git-update-index[1]. | ||||
|  | ||||
| splitIndex.sharedIndexExpire:: | ||||
| 	When the split index feature is used, shared index files that | ||||
| 	were not modified since the time this variable specifies will | ||||
| 	be removed when a new shared index file is created. The value | ||||
| 	"now" expires all entries immediately, and "never" suppresses | ||||
| 	expiration altogether. | ||||
| 	The default value is "2.weeks.ago". | ||||
| 	Note that a shared index file is considered modified (for the | ||||
| 	purpose of expiration) each time a new split-index file is | ||||
| 	either created based on it or read from it. | ||||
| 	See linkgit:git-update-index[1]. | ||||
|  | ||||
| status.relativePaths:: | ||||
| 	By default, linkgit:git-status[1] shows paths relative to the | ||||
| 	current directory. Setting this variable to `false` shows paths | ||||
|  |  | |||
|  | @ -163,14 +163,16 @@ may not support it yet. | |||
|  | ||||
| --split-index:: | ||||
| --no-split-index:: | ||||
| 	Enable or disable split index mode. If enabled, the index is | ||||
| 	split into two files, $GIT_DIR/index and $GIT_DIR/sharedindex.<SHA-1>. | ||||
| 	Changes are accumulated in $GIT_DIR/index while the shared | ||||
| 	index file contains all index entries stays unchanged. If | ||||
| 	split-index mode is already enabled and `--split-index` is | ||||
| 	given again, all changes in $GIT_DIR/index are pushed back to | ||||
| 	the shared index file. This mode is designed for very large | ||||
| 	indexes that take a significant amount of time to read or write. | ||||
| 	Enable or disable split index mode. If split-index mode is | ||||
| 	already enabled and `--split-index` is given again, all | ||||
| 	changes in $GIT_DIR/index are pushed back to the shared index | ||||
| 	file. | ||||
| + | ||||
| These options take effect whatever the value of the `core.splitIndex` | ||||
| configuration variable (see linkgit:git-config[1]). But a warning is | ||||
| emitted when the change goes against the configured value, as the | ||||
| configured value will take effect next time the index is read and this | ||||
| will remove the intended effect of the option. | ||||
|  | ||||
| --untracked-cache:: | ||||
| --no-untracked-cache:: | ||||
|  | @ -388,6 +390,31 @@ Although this bit looks similar to assume-unchanged bit, its goal is | |||
| different from assume-unchanged bit's. Skip-worktree also takes | ||||
| precedence over assume-unchanged bit when both are set. | ||||
|  | ||||
| Split index | ||||
| ----------- | ||||
|  | ||||
| This mode is designed for repositories with very large indexes, and | ||||
| aims at reducing the time it takes to repeatedly write these indexes. | ||||
|  | ||||
| In this mode, the index is split into two files, $GIT_DIR/index and | ||||
| $GIT_DIR/sharedindex.<SHA-1>. Changes are accumulated in | ||||
| $GIT_DIR/index, the split index, while the shared index file contains | ||||
| all index entries and stays unchanged. | ||||
|  | ||||
| All changes in the split index are pushed back to the shared index | ||||
| file when the number of entries in the split index reaches a level | ||||
| specified by the splitIndex.maxPercentChange config variable (see | ||||
| linkgit:git-config[1]). | ||||
|  | ||||
| Each time a new shared index file is created, the old shared index | ||||
| files are deleted if their modification time is older than what is | ||||
| specified by the splitIndex.sharedIndexExpire config variable (see | ||||
| linkgit:git-config[1]). | ||||
|  | ||||
| To avoid deleting a shared index file that is still used, its | ||||
| modification time is updated to the current time everytime a new split | ||||
| index based on the shared index file is either created or read from. | ||||
|  | ||||
| Untracked cache | ||||
| --------------- | ||||
|  | ||||
|  |  | |||
							
								
								
									
										17
									
								
								builtin/gc.c
								
								
								
								
							
							
						
						
									
										17
									
								
								builtin/gc.c
								
								
								
								
							|  | @ -64,17 +64,6 @@ static void report_pack_garbage(unsigned seen_bits, const char *path) | |||
| 		string_list_append(&pack_garbage, path); | ||||
| } | ||||
|  | ||||
| static void git_config_date_string(const char *key, const char **output) | ||||
| { | ||||
| 	if (git_config_get_string_const(key, output)) | ||||
| 		return; | ||||
| 	if (strcmp(*output, "now")) { | ||||
| 		unsigned long now = approxidate("now"); | ||||
| 		if (approxidate(*output) >= now) | ||||
| 			git_die_config(key, _("Invalid %s: '%s'"), key, *output); | ||||
| 	} | ||||
| } | ||||
|  | ||||
| static void process_log_file(void) | ||||
| { | ||||
| 	struct stat st; | ||||
|  | @ -131,9 +120,9 @@ static void gc_config(void) | |||
| 	git_config_get_int("gc.auto", &gc_auto_threshold); | ||||
| 	git_config_get_int("gc.autopacklimit", &gc_auto_pack_limit); | ||||
| 	git_config_get_bool("gc.autodetach", &detach_auto); | ||||
| 	git_config_date_string("gc.pruneexpire", &prune_expire); | ||||
| 	git_config_date_string("gc.worktreepruneexpire", &prune_worktrees_expire); | ||||
| 	git_config_date_string("gc.logexpiry", &gc_log_expire); | ||||
| 	git_config_get_expiry("gc.pruneexpire", &prune_expire); | ||||
| 	git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire); | ||||
| 	git_config_get_expiry("gc.logexpiry", &gc_log_expire); | ||||
|  | ||||
| 	git_config(git_default_config, NULL); | ||||
| } | ||||
|  |  | |||
|  | @ -1099,17 +1099,20 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) | |||
| 	} | ||||
|  | ||||
| 	if (split_index > 0) { | ||||
| 		init_split_index(&the_index); | ||||
| 		if (git_config_get_split_index() == 0) | ||||
| 			warning(_("core.splitIndex is set to false; " | ||||
| 				  "remove or change it, if you really want to " | ||||
| 				  "enable split index")); | ||||
| 		if (the_index.split_index) | ||||
| 			the_index.cache_changed |= SPLIT_INDEX_ORDERED; | ||||
| 	} else if (!split_index && the_index.split_index) { | ||||
| 		/* | ||||
| 		 * can't discard_split_index(&the_index); because that | ||||
| 		 * will destroy split_index->base->cache[], which may | ||||
| 		 * be shared with the_index.cache[]. So yeah we're | ||||
| 		 * leaking a bit here. | ||||
| 		 */ | ||||
| 		the_index.split_index = NULL; | ||||
| 		the_index.cache_changed |= SOMETHING_CHANGED; | ||||
| 		else | ||||
| 			add_split_index(&the_index); | ||||
| 	} else if (!split_index) { | ||||
| 		if (git_config_get_split_index() == 1) | ||||
| 			warning(_("core.splitIndex is set to true; " | ||||
| 				  "remove or change it, if you really want to " | ||||
| 				  "disable split index")); | ||||
| 		remove_split_index(&the_index); | ||||
| 	} | ||||
|  | ||||
| 	switch (untracked_cache) { | ||||
|  |  | |||
							
								
								
									
										8
									
								
								cache.h
								
								
								
								
							
							
						
						
									
										8
									
								
								cache.h
								
								
								
								
							|  | @ -1270,6 +1270,9 @@ extern int has_pack_index(const unsigned char *sha1); | |||
|  | ||||
| extern void assert_sha1_type(const unsigned char *sha1, enum object_type expect); | ||||
|  | ||||
| /* Helper to check and "touch" a file */ | ||||
| extern int check_and_freshen_file(const char *fn, int freshen); | ||||
|  | ||||
| extern const signed char hexval_table[256]; | ||||
| static inline unsigned int hexval(unsigned char c) | ||||
| { | ||||
|  | @ -1956,6 +1959,11 @@ extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest); | |||
| extern int git_config_get_maybe_bool(const char *key, int *dest); | ||||
| extern int git_config_get_pathname(const char *key, const char **dest); | ||||
| extern int git_config_get_untracked_cache(void); | ||||
| extern int git_config_get_split_index(void); | ||||
| extern int git_config_get_max_percent_split_change(void); | ||||
|  | ||||
| /* This dies if the configured or default date is in the future */ | ||||
| extern int git_config_get_expiry(const char *key, const char **output); | ||||
|  | ||||
| /* | ||||
|  * This is a hack for test programs like test-dump-untracked-cache to | ||||
|  |  | |||
							
								
								
									
										42
									
								
								config.c
								
								
								
								
							
							
						
						
									
										42
									
								
								config.c
								
								
								
								
							|  | @ -1803,6 +1803,19 @@ int git_config_get_pathname(const char *key, const char **dest) | |||
| 	return ret; | ||||
| } | ||||
|  | ||||
| int git_config_get_expiry(const char *key, const char **output) | ||||
| { | ||||
| 	int ret = git_config_get_string_const(key, output); | ||||
| 	if (ret) | ||||
| 		return ret; | ||||
| 	if (strcmp(*output, "now")) { | ||||
| 		unsigned long now = approxidate("now"); | ||||
| 		if (approxidate(*output) >= now) | ||||
| 			git_die_config(key, _("Invalid %s: '%s'"), key, *output); | ||||
| 	} | ||||
| 	return ret; | ||||
| } | ||||
|  | ||||
| int git_config_get_untracked_cache(void) | ||||
| { | ||||
| 	int val = -1; | ||||
|  | @ -1819,14 +1832,39 @@ int git_config_get_untracked_cache(void) | |||
| 		if (!strcasecmp(v, "keep")) | ||||
| 			return -1; | ||||
|  | ||||
| 		error("unknown core.untrackedCache value '%s'; " | ||||
| 		      "using 'keep' default value", v); | ||||
| 		error(_("unknown core.untrackedCache value '%s'; " | ||||
| 			"using 'keep' default value"), v); | ||||
| 		return -1; | ||||
| 	} | ||||
|  | ||||
| 	return -1; /* default value */ | ||||
| } | ||||
|  | ||||
| int git_config_get_split_index(void) | ||||
| { | ||||
| 	int val; | ||||
|  | ||||
| 	if (!git_config_get_maybe_bool("core.splitindex", &val)) | ||||
| 		return val; | ||||
|  | ||||
| 	return -1; /* default value */ | ||||
| } | ||||
|  | ||||
| int git_config_get_max_percent_split_change(void) | ||||
| { | ||||
| 	int val = -1; | ||||
|  | ||||
| 	if (!git_config_get_int("splitindex.maxpercentchange", &val)) { | ||||
| 		if (0 <= val && val <= 100) | ||||
| 			return val; | ||||
|  | ||||
| 		return error(_("splitIndex.maxPercentChange value '%d' " | ||||
| 			       "should be between 0 and 100"), val); | ||||
| 	} | ||||
|  | ||||
| 	return -1; /* default value */ | ||||
| } | ||||
|  | ||||
| NORETURN | ||||
| void git_die_config_linenr(const char *key, const char *filename, int linenr) | ||||
| { | ||||
|  |  | |||
							
								
								
									
										157
									
								
								read-cache.c
								
								
								
								
							
							
						
						
									
										157
									
								
								read-cache.c
								
								
								
								
							|  | @ -1558,10 +1558,27 @@ static void tweak_untracked_cache(struct index_state *istate) | |||
| 	} | ||||
| } | ||||
|  | ||||
| static void tweak_split_index(struct index_state *istate) | ||||
| { | ||||
| 	switch (git_config_get_split_index()) { | ||||
| 	case -1: /* unset: do nothing */ | ||||
| 		break; | ||||
| 	case 0: /* false */ | ||||
| 		remove_split_index(istate); | ||||
| 		break; | ||||
| 	case 1: /* true */ | ||||
| 		add_split_index(istate); | ||||
| 		break; | ||||
| 	default: /* unknown value: do nothing */ | ||||
| 		break; | ||||
| 	} | ||||
| } | ||||
|  | ||||
| static void post_read_index_from(struct index_state *istate) | ||||
| { | ||||
| 	check_ce_order(istate); | ||||
| 	tweak_untracked_cache(istate); | ||||
| 	tweak_split_index(istate); | ||||
| } | ||||
|  | ||||
| /* remember to discard_cache() before reading a different cache! */ | ||||
|  | @ -1657,10 +1674,25 @@ unmap: | |||
| 	die("index file corrupt"); | ||||
| } | ||||
|  | ||||
| /* | ||||
|  * Signal that the shared index is used by updating its mtime. | ||||
|  * | ||||
|  * This way, shared index can be removed if they have not been used | ||||
|  * for some time. | ||||
|  */ | ||||
| static void freshen_shared_index(char *base_sha1_hex, int warn) | ||||
| { | ||||
| 	const char *shared_index = git_path("sharedindex.%s", base_sha1_hex); | ||||
| 	if (!check_and_freshen_file(shared_index, 1) && warn) | ||||
| 		warning("could not freshen shared index '%s'", shared_index); | ||||
| } | ||||
|  | ||||
| int read_index_from(struct index_state *istate, const char *path) | ||||
| { | ||||
| 	struct split_index *split_index; | ||||
| 	int ret; | ||||
| 	char *base_sha1_hex; | ||||
| 	const char *base_path; | ||||
|  | ||||
| 	/* istate->initialized covers both .git/index and .git/sharedindex.xxx */ | ||||
| 	if (istate->initialized) | ||||
|  | @ -1678,15 +1710,16 @@ int read_index_from(struct index_state *istate, const char *path) | |||
| 		discard_index(split_index->base); | ||||
| 	else | ||||
| 		split_index->base = xcalloc(1, sizeof(*split_index->base)); | ||||
| 	ret = do_read_index(split_index->base, | ||||
| 			    git_path("sharedindex.%s", | ||||
| 				     sha1_to_hex(split_index->base_sha1)), 1); | ||||
|  | ||||
| 	base_sha1_hex = sha1_to_hex(split_index->base_sha1); | ||||
| 	base_path = git_path("sharedindex.%s", base_sha1_hex); | ||||
| 	ret = do_read_index(split_index->base, base_path, 1); | ||||
| 	if (hashcmp(split_index->base_sha1, split_index->base->sha1)) | ||||
| 		die("broken index, expect %s in %s, got %s", | ||||
| 		    sha1_to_hex(split_index->base_sha1), | ||||
| 		    git_path("sharedindex.%s", | ||||
| 			     sha1_to_hex(split_index->base_sha1)), | ||||
| 		    base_sha1_hex, base_path, | ||||
| 		    sha1_to_hex(split_index->base->sha1)); | ||||
|  | ||||
| 	freshen_shared_index(base_sha1_hex, 0); | ||||
| 	merge_base_index(istate); | ||||
| 	post_read_index_from(istate); | ||||
| 	return ret; | ||||
|  | @ -2169,6 +2202,65 @@ static int write_split_index(struct index_state *istate, | |||
| 	return ret; | ||||
| } | ||||
|  | ||||
| static const char *shared_index_expire = "2.weeks.ago"; | ||||
|  | ||||
| static unsigned long get_shared_index_expire_date(void) | ||||
| { | ||||
| 	static unsigned long shared_index_expire_date; | ||||
| 	static int shared_index_expire_date_prepared; | ||||
|  | ||||
| 	if (!shared_index_expire_date_prepared) { | ||||
| 		git_config_get_expiry("splitindex.sharedindexexpire", | ||||
| 				      &shared_index_expire); | ||||
| 		shared_index_expire_date = approxidate(shared_index_expire); | ||||
| 		shared_index_expire_date_prepared = 1; | ||||
| 	} | ||||
|  | ||||
| 	return shared_index_expire_date; | ||||
| } | ||||
|  | ||||
| static int should_delete_shared_index(const char *shared_index_path) | ||||
| { | ||||
| 	struct stat st; | ||||
| 	unsigned long expiration; | ||||
|  | ||||
| 	/* Check timestamp */ | ||||
| 	expiration = get_shared_index_expire_date(); | ||||
| 	if (!expiration) | ||||
| 		return 0; | ||||
| 	if (stat(shared_index_path, &st)) | ||||
| 		return error_errno(_("could not stat '%s"), shared_index_path); | ||||
| 	if (st.st_mtime > expiration) | ||||
| 		return 0; | ||||
|  | ||||
| 	return 1; | ||||
| } | ||||
|  | ||||
| static int clean_shared_index_files(const char *current_hex) | ||||
| { | ||||
| 	struct dirent *de; | ||||
| 	DIR *dir = opendir(get_git_dir()); | ||||
|  | ||||
| 	if (!dir) | ||||
| 		return error_errno(_("unable to open git dir: %s"), get_git_dir()); | ||||
|  | ||||
| 	while ((de = readdir(dir)) != NULL) { | ||||
| 		const char *sha1_hex; | ||||
| 		const char *shared_index_path; | ||||
| 		if (!skip_prefix(de->d_name, "sharedindex.", &sha1_hex)) | ||||
| 			continue; | ||||
| 		if (!strcmp(sha1_hex, current_hex)) | ||||
| 			continue; | ||||
| 		shared_index_path = git_path("%s", de->d_name); | ||||
| 		if (should_delete_shared_index(shared_index_path) > 0 && | ||||
| 		    unlink(shared_index_path)) | ||||
| 			warning_errno(_("unable to unlink: %s"), shared_index_path); | ||||
| 	} | ||||
| 	closedir(dir); | ||||
|  | ||||
| 	return 0; | ||||
| } | ||||
|  | ||||
| static struct tempfile temporary_sharedindex; | ||||
|  | ||||
| static int write_shared_index(struct index_state *istate, | ||||
|  | @ -2190,14 +2282,48 @@ static int write_shared_index(struct index_state *istate, | |||
| 	} | ||||
| 	ret = rename_tempfile(&temporary_sharedindex, | ||||
| 			      git_path("sharedindex.%s", sha1_to_hex(si->base->sha1))); | ||||
| 	if (!ret) | ||||
| 	if (!ret) { | ||||
| 		hashcpy(si->base_sha1, si->base->sha1); | ||||
| 		clean_shared_index_files(sha1_to_hex(si->base->sha1)); | ||||
| 	} | ||||
|  | ||||
| 	return ret; | ||||
| } | ||||
|  | ||||
| static const int default_max_percent_split_change = 20; | ||||
|  | ||||
| static int too_many_not_shared_entries(struct index_state *istate) | ||||
| { | ||||
| 	int i, not_shared = 0; | ||||
| 	int max_split = git_config_get_max_percent_split_change(); | ||||
|  | ||||
| 	switch (max_split) { | ||||
| 	case -1: | ||||
| 		/* not or badly configured: use the default value */ | ||||
| 		max_split = default_max_percent_split_change; | ||||
| 		break; | ||||
| 	case 0: | ||||
| 		return 1; /* 0% means always write a new shared index */ | ||||
| 	case 100: | ||||
| 		return 0; /* 100% means never write a new shared index */ | ||||
| 	default: | ||||
| 		break; /* just use the configured value */ | ||||
| 	} | ||||
|  | ||||
| 	/* Count not shared entries */ | ||||
| 	for (i = 0; i < istate->cache_nr; i++) { | ||||
| 		struct cache_entry *ce = istate->cache[i]; | ||||
| 		if (!ce->index) | ||||
| 			not_shared++; | ||||
| 	} | ||||
|  | ||||
| 	return (int64_t)istate->cache_nr * max_split < (int64_t)not_shared * 100; | ||||
| } | ||||
|  | ||||
| int write_locked_index(struct index_state *istate, struct lock_file *lock, | ||||
| 		       unsigned flags) | ||||
| { | ||||
| 	int new_shared_index, ret; | ||||
| 	struct split_index *si = istate->split_index; | ||||
|  | ||||
| 	if (!si || alternate_index_output || | ||||
|  | @ -2212,13 +2338,24 @@ int write_locked_index(struct index_state *istate, struct lock_file *lock, | |||
| 		if ((v & 15) < 6) | ||||
| 			istate->cache_changed |= SPLIT_INDEX_ORDERED; | ||||
| 	} | ||||
| 	if (istate->cache_changed & SPLIT_INDEX_ORDERED) { | ||||
| 		int ret = write_shared_index(istate, lock, flags); | ||||
| 	if (too_many_not_shared_entries(istate)) | ||||
| 		istate->cache_changed |= SPLIT_INDEX_ORDERED; | ||||
|  | ||||
| 	new_shared_index = istate->cache_changed & SPLIT_INDEX_ORDERED; | ||||
|  | ||||
| 	if (new_shared_index) { | ||||
| 		ret = write_shared_index(istate, lock, flags); | ||||
| 		if (ret) | ||||
| 			return ret; | ||||
| 	} | ||||
|  | ||||
| 	return write_split_index(istate, lock, flags); | ||||
| 	ret = write_split_index(istate, lock, flags); | ||||
|  | ||||
| 	/* Freshen the shared index only if the split-index was written */ | ||||
| 	if (!ret && !new_shared_index) | ||||
| 		freshen_shared_index(sha1_to_hex(si->base_sha1), 1); | ||||
|  | ||||
| 	return ret; | ||||
| } | ||||
|  | ||||
| /* | ||||
|  |  | |||
|  | @ -667,7 +667,7 @@ static int freshen_file(const char *fn) | |||
|  * either does not exist on disk, or has a stale mtime and may be subject to | ||||
|  * pruning). | ||||
|  */ | ||||
| static int check_and_freshen_file(const char *fn, int freshen) | ||||
| int check_and_freshen_file(const char *fn, int freshen) | ||||
| { | ||||
| 	if (access(fn, F_OK)) | ||||
| 		return 0; | ||||
|  |  | |||
|  | @ -317,3 +317,25 @@ void replace_index_entry_in_base(struct index_state *istate, | |||
| 		istate->split_index->base->cache[new->index - 1] = new; | ||||
| 	} | ||||
| } | ||||
|  | ||||
| void add_split_index(struct index_state *istate) | ||||
| { | ||||
| 	if (!istate->split_index) { | ||||
| 		init_split_index(istate); | ||||
| 		istate->cache_changed |= SPLIT_INDEX_ORDERED; | ||||
| 	} | ||||
| } | ||||
|  | ||||
| void remove_split_index(struct index_state *istate) | ||||
| { | ||||
| 	if (istate->split_index) { | ||||
| 		/* | ||||
| 		 * can't discard_split_index(&the_index); because that | ||||
| 		 * will destroy split_index->base->cache[], which may | ||||
| 		 * be shared with the_index.cache[]. So yeah we're | ||||
| 		 * leaking a bit here. | ||||
| 		 */ | ||||
| 		istate->split_index = NULL; | ||||
| 		istate->cache_changed |= SOMETHING_CHANGED; | ||||
| 	} | ||||
| } | ||||
|  |  | |||
|  | @ -31,5 +31,7 @@ void merge_base_index(struct index_state *istate); | |||
| void prepare_to_write_split_index(struct index_state *istate); | ||||
| void finish_writing_split_index(struct index_state *istate); | ||||
| void discard_split_index(struct index_state *istate); | ||||
| void add_split_index(struct index_state *istate); | ||||
| void remove_split_index(struct index_state *istate); | ||||
|  | ||||
| #endif | ||||
|  |  | |||
|  | @ -8,6 +8,7 @@ test_description='split index mode tests' | |||
| sane_unset GIT_TEST_SPLIT_INDEX | ||||
|  | ||||
| test_expect_success 'enable split index' ' | ||||
| 	git config splitIndex.maxPercentChange 100 && | ||||
| 	git update-index --split-index && | ||||
| 	test-dump-split-index .git/index >actual && | ||||
| 	indexversion=$(test-index-version <.git/index) && | ||||
|  | @ -19,12 +20,12 @@ test_expect_success 'enable split index' ' | |||
| 		own=8299b0bcd1ac364e5f1d7768efb62fa2da79a339 | ||||
| 		base=39d890139ee5356c7ef572216cebcd27aa41f9df | ||||
| 	fi && | ||||
| 	cat >expect <<EOF && | ||||
| own $own | ||||
| base $base | ||||
| replacements: | ||||
| deletions: | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	own $own | ||||
| 	base $base | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
|  | @ -32,51 +33,51 @@ test_expect_success 'add one file' ' | |||
| 	: >one && | ||||
| 	git update-index --add one && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<EOF && | ||||
| base $base | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| replacements: | ||||
| deletions: | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	base $base | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'disable split index' ' | ||||
| 	git update-index --no-split-index && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	BASE=$(test-dump-split-index .git/index | grep "^own" | sed "s/own/base/") && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<EOF && | ||||
| not a split index | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	not a split index | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'enable split index again, "one" now belongs to base index"' ' | ||||
| 	git update-index --split-index && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<EOF && | ||||
| $BASE | ||||
| replacements: | ||||
| deletions: | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
|  | @ -84,18 +85,18 @@ test_expect_success 'modify original file, base index untouched' ' | |||
| 	echo modified >one && | ||||
| 	git update-index one && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0	one | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0	one | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	q_to_tab >expect <<EOF && | ||||
| $BASE | ||||
| 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q | ||||
| replacements: 0 | ||||
| deletions: | ||||
| EOF | ||||
| 	q_to_tab >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q | ||||
| 	replacements: 0 | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
|  | @ -103,54 +104,54 @@ test_expect_success 'add another file, which stays index' ' | |||
| 	: >two && | ||||
| 	git update-index --add two && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0	one | ||||
| 100644 $EMPTY_BLOB 0	two | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0	one | ||||
| 	100644 $EMPTY_BLOB 0	two | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	q_to_tab >expect <<EOF && | ||||
| $BASE | ||||
| 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q | ||||
| 100644 $EMPTY_BLOB 0	two | ||||
| replacements: 0 | ||||
| deletions: | ||||
| EOF | ||||
| 	q_to_tab >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q | ||||
| 	100644 $EMPTY_BLOB 0	two | ||||
| 	replacements: 0 | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'remove file not in base index' ' | ||||
| 	git update-index --force-remove two && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0	one | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0	one | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	q_to_tab >expect <<EOF && | ||||
| $BASE | ||||
| 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q | ||||
| replacements: 0 | ||||
| deletions: | ||||
| EOF | ||||
| 	q_to_tab >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q | ||||
| 	replacements: 0 | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'remove file in base index' ' | ||||
| 	git update-index --force-remove one && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<EOF && | ||||
| $BASE | ||||
| replacements: | ||||
| deletions: 0 | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	replacements: | ||||
| 	deletions: 0 | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
|  | @ -158,18 +159,18 @@ test_expect_success 'add original file back' ' | |||
| 	: >one && | ||||
| 	git update-index --add one && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<EOF && | ||||
| $BASE | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| replacements: | ||||
| deletions: 0 | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	replacements: | ||||
| 	deletions: 0 | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
|  | @ -177,26 +178,26 @@ test_expect_success 'add new file' ' | |||
| 	: >two && | ||||
| 	git update-index --add two && | ||||
| 	git ls-files --stage >actual && | ||||
| 	cat >expect <<EOF && | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| 100644 $EMPTY_BLOB 0	two | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	100644 $EMPTY_BLOB 0	two | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'unify index, two files remain' ' | ||||
| 	git update-index --no-split-index && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<EOF && | ||||
| 100644 $EMPTY_BLOB 0	one | ||||
| 100644 $EMPTY_BLOB 0	two | ||||
| EOF | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 $EMPTY_BLOB 0	one | ||||
| 	100644 $EMPTY_BLOB 0	two | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
|  | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<EOF && | ||||
| not a split index | ||||
| EOF | ||||
| 	cat >expect <<-EOF && | ||||
| 	not a split index | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
|  | @ -216,4 +217,157 @@ test_expect_success 'rev-parse --shared-index-path' ' | |||
| 	) | ||||
| ' | ||||
|  | ||||
| test_expect_success 'set core.splitIndex config variable to true' ' | ||||
| 	git config core.splitIndex true && | ||||
| 	: >three && | ||||
| 	git update-index --add three && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0	one | ||||
| 	100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0	three | ||||
| 	100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0	two | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
| 	BASE=$(test-dump-split-index .git/index | grep "^base") && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'set core.splitIndex config variable to false' ' | ||||
| 	git config core.splitIndex false && | ||||
| 	git update-index --force-remove three && | ||||
| 	git ls-files --stage >ls-files.actual && | ||||
| 	cat >ls-files.expect <<-EOF && | ||||
| 	100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0	one | ||||
| 	100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0	two | ||||
| 	EOF | ||||
| 	test_cmp ls-files.expect ls-files.actual && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	not a split index | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'set core.splitIndex config variable to true' ' | ||||
| 	git config core.splitIndex true && | ||||
| 	: >three && | ||||
| 	git update-index --add three && | ||||
| 	BASE=$(test-dump-split-index .git/index | grep "^base") && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual && | ||||
| 	: >four && | ||||
| 	git update-index --add four && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0	four | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'check behavior with splitIndex.maxPercentChange unset' ' | ||||
| 	git config --unset splitIndex.maxPercentChange && | ||||
| 	: >five && | ||||
| 	git update-index --add five && | ||||
| 	BASE=$(test-dump-split-index .git/index | grep "^base") && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual && | ||||
| 	: >six && | ||||
| 	git update-index --add six && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0	six | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'check splitIndex.maxPercentChange set to 0' ' | ||||
| 	git config splitIndex.maxPercentChange 0 && | ||||
| 	: >seven && | ||||
| 	git update-index --add seven && | ||||
| 	BASE=$(test-dump-split-index .git/index | grep "^base") && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual && | ||||
| 	: >eight && | ||||
| 	git update-index --add eight && | ||||
| 	BASE=$(test-dump-split-index .git/index | grep "^base") && | ||||
| 	test-dump-split-index .git/index | sed "/^own/d" >actual && | ||||
| 	cat >expect <<-EOF && | ||||
| 	$BASE | ||||
| 	replacements: | ||||
| 	deletions: | ||||
| 	EOF | ||||
| 	test_cmp expect actual | ||||
| ' | ||||
|  | ||||
| test_expect_success 'shared index files expire after 2 weeks by default' ' | ||||
| 	: >ten && | ||||
| 	git update-index --add ten && | ||||
| 	test $(ls .git/sharedindex.* | wc -l) -gt 2 && | ||||
| 	just_under_2_weeks_ago=$((5-14*86400)) && | ||||
| 	test-chmtime =$just_under_2_weeks_ago .git/sharedindex.* && | ||||
| 	: >eleven && | ||||
| 	git update-index --add eleven && | ||||
| 	test $(ls .git/sharedindex.* | wc -l) -gt 2 && | ||||
| 	just_over_2_weeks_ago=$((-1-14*86400)) && | ||||
| 	test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* && | ||||
| 	: >twelve && | ||||
| 	git update-index --add twelve && | ||||
| 	test $(ls .git/sharedindex.* | wc -l) -le 2 | ||||
| ' | ||||
|  | ||||
| test_expect_success 'check splitIndex.sharedIndexExpire set to 16 days' ' | ||||
| 	git config splitIndex.sharedIndexExpire "16.days.ago" && | ||||
| 	test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* && | ||||
| 	: >thirteen && | ||||
| 	git update-index --add thirteen && | ||||
| 	test $(ls .git/sharedindex.* | wc -l) -gt 2 && | ||||
| 	just_over_16_days_ago=$((-1-16*86400)) && | ||||
| 	test-chmtime =$just_over_16_days_ago .git/sharedindex.* && | ||||
| 	: >fourteen && | ||||
| 	git update-index --add fourteen && | ||||
| 	test $(ls .git/sharedindex.* | wc -l) -le 2 | ||||
| ' | ||||
|  | ||||
| test_expect_success 'check splitIndex.sharedIndexExpire set to "never" and "now"' ' | ||||
| 	git config splitIndex.sharedIndexExpire never && | ||||
| 	just_10_years_ago=$((-365*10*86400)) && | ||||
| 	test-chmtime =$just_10_years_ago .git/sharedindex.* && | ||||
| 	: >fifteen && | ||||
| 	git update-index --add fifteen && | ||||
| 	test $(ls .git/sharedindex.* | wc -l) -gt 2 && | ||||
| 	git config splitIndex.sharedIndexExpire now && | ||||
| 	just_1_second_ago=-1 && | ||||
| 	test-chmtime =$just_1_second_ago .git/sharedindex.* && | ||||
| 	: >sixteen && | ||||
| 	git update-index --add sixteen && | ||||
| 	test $(ls .git/sharedindex.* | wc -l) -le 2 | ||||
| ' | ||||
|  | ||||
| test_done | ||||
|  |  | |||
		Loading…
	
		Reference in New Issue
	
	 Junio C Hamano
						Junio C Hamano