kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Junio C Hamano	afe8a9070b	tree-wide: apply equals-null.cocci Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Neeraj Singh	f12f3b9807	core.fsyncmethod: correctly camel-case warning message The warning for an unrecognized fsyncMethod was not camel-cased. Reported-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jeff Hostetler	1e0ea5c431	fsmonitor: config settings are repository-specific Move fsmonitor config settings to a new and opaque `struct fsmonitor_settings` structure. Add a lazily-loaded pointer to this into `struct repo_settings` Create an `enum fsmonitor_mode` type in `struct fsmonitor_settings` to represent the state of fsmonitor. This lets us represent which, if any, fsmonitor provider (hook or IPC) is enabled. Create `fsm_settings__get_()` getters to lazily look up fsmonitor- related config settings. Get rid of the `core_fsmonitor` global variable. Move the code to lookup the existing `core.fsmonitor` config value into the fsmonitor settings. Create a hook pathname variable in `struct fsmonitor-settings` and only set it when in hook mode. Extend the definition of `core.fsmonitor` to be either a boolean or a hook pathname. When true, the builtin FSMonitor is used. When false or unset, no FSMonitor (neither builtin nor hook) is used. The existing `core_fsmonitor` global variable was used to store the pathname to the fsmonitor hook and* it was used as a boolean to see if fsmonitor was enabled. This dual usage and global visibility leads to confusion when we add the IPC-based provider. So lets hide the details in fsmonitor-settings.c and let it decide which provider to use in the case of multiple settings. This avoids cluttering up repo-settings.c with these private details. A future commit in builtin-fsmonitor series will add the ability to disqualify worktrees for various reasons, such as being mounted from a remote volume, where fsmonitor should not be started. Having the config settings hidden in fsmonitor-settings.c allows such worktree restrictions to override the config values used. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Patrick Steinhardt	bc22d845c4	core.fsync: new option to harden references When writing both loose and packed references to disk we first create a lockfile, write the updated values into that lockfile, and on commit we rename the file into place. According to filesystem developers, this behaviour is broken because applications should always sync data to disk before doing the final rename to ensure data consistency [1][2][3]. If applications fail to do this correctly, a hard crash of the machine can easily result in corrupted on-disk data. This kind of corruption can in fact be easily observed with Git when the machine hard-resets shortly after writing references to disk. On machines with ext4, this will likely lead to the "empty files" problem: the file has been renamed, but its data has not been synced to disk. The result is that the reference is corrupt, and in the worst case this can lead to data loss. Implement a new option to harden references so that users and admins can avoid this scenario by syncing locked loose and packed references to disk before we rename them into place. [1]: https://thunk.org/tytso/blog/2009/03/15/dont-fear-the-fsync/ [2]: https://btrfs.wiki.kernel.org/index.php/FAQ (What are the crash guarantees of overwrite-by-rename) [3]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/ext4.rst (see auto_da_alloc) Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Neeraj Singh	b9f5d0358d	core.fsync: documentation and user-friendly aggregate options This commit adds aggregate options for the core.fsync setting that are more user-friendly. These options are specified in terms of 'levels of safety', indicating which Git operations are considered to be sync points for durability. The new documentation is also included here in its entirety for ease of review. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Neeraj Singh	ba95e96d4c	core.fsync: new option to harden the index This commit introduces the new ability for the user to harden the index. In the event of a system crash, the index must be durable for the user to actually find a file that has been added to the repo and then deleted from the working tree. We use the presence of the COMMIT_LOCK flag and absence of the alternate_index_output as a proxy for determining whether we're updating the persistent index of the repo or some temporary index. We don't sync these temporary indexes. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Neeraj Singh	844a8ad4f8	core.fsync: add configuration parsing This change introduces code to parse the core.fsync setting and configure the fsync_components variable. core.fsync is configured as a comma-separated list of component names to sync. Each time a core.fsync variable is encountered in the configuration heirarchy, we start off with a clean state with the platform default value. Passing 'none' resets the value to indicate nothing will be synced. We gather all negative and positive entries from the comma separated list and then compute the new value by removing all the negative entries and adding all of the positive entries. We issue a warning for components that are not recognized so that the configuration code is compatible with configs from future versions of Git with more repo components. Complete documentation for the new setting is included in a later patch in the series so that it can be reviewed once in final form. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Neeraj Singh	abf38abec2	core.fsyncmethod: add writeout-only mode This commit introduces the `core.fsyncMethod` configuration knob, which can currently be set to `fsync` or `writeout-only`. The new writeout-only mode attempts to tell the operating system to flush its in-memory page cache to the storage hardware without issuing a CACHE_FLUSH command to the storage controller. Writeout-only fsync is significantly faster than a vanilla fsync on common hardware, since data is written to a disk-side cache rather than all the way to a durable medium. Later changes in this patch series will take advantage of this primitive to implement batching of hardware flushes. When git_fsync is called with FSYNC_WRITEOUT_ONLY, it may fail and the caller is expected to do an ordinary fsync as needed. On Apple platforms, the fsync system call does not issue a CACHE_FLUSH directive to the storage controller. This change updates fsync to do fcntl(F_FULLFSYNC) to make fsync actually durable. We maintain parity with existing behavior on Apple platforms by setting the default value of the new core.fsyncMethod option. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Elijah Newren	ecc7c8841d	repo_read_index: add config to expect files outside sparse patterns Typically with sparse checkouts, we expect files outside the sparsity patterns to be marked as SKIP_WORKTREE and be missing from the working tree. Sometimes this expectation would be violated however; including in cases such as: * users grabbing files from elsewhere and writing them to the worktree (perhaps by editing a cached copy in an editor, copying/renaming, or even untarring) * various git commands having incomplete or no support for the SKIP_WORKTREE bit[1,2] * users attempting to "abort" a sparse-checkout operation with a not-so-early Ctrl+C (updating $GIT_DIR/info/sparse-checkout and the working tree is not atomic)[3]. When the SKIP_WORKTREE bit in the index did not reflect the presence of the file in the working tree, it traditionally caused confusion and was difficult to detect and recover from. So, in a sparse checkout, since `af6a51875a` (repo_read_index: clear SKIP_WORKTREE bit from files present in worktree, 2022-01-14), Git automatically clears the SKIP_WORKTREE bit at index read time for entries corresponding to files that are present in the working tree. There is another workflow, however, where it is expected that paths outside the sparsity patterns appear to exist in the working tree and that they do not lose the SKIP_WORKTREE bit, at least until they get modified. A Git-aware virtual file system[4] takes advantage of its position as a file system driver to expose all files in the working tree, fetch them on demand using partial clone on access, and tell Git to pay attention to them on demand by updating the sparse checkout pattern on writes. This means that commands like "git status" only have to examine files that have potentially been modified, whereas commands like "ls" are able to show the entire codebase without requiring manual updates to the sparse checkout pattern. Thus since `af6a51875a`, Git with such Git-aware virtual file systems unsets the SKIP_WORKTREE bit for all files and commands like "git status" have to fetch and examine them all. Introduce a configuration setting sparse.expectFilesOutsideOfPatterns to allow limiting the tracked set of files to a small set once again. A Git-aware virtual file system or other application that wants to maintain files outside of the sparse checkout can set this in a repository to instruct Git not to check for the presence of SKIP_WORKTREE files. The setting defaults to false, so most users of sparse checkout will still get the benefit of an automatically updating index to recover from the variety of difficult issues detailed in `af6a51875a` for paths with SKIP_WORKTREE set despite the path being present. [1] https://lore.kernel.org/git/xmqqbmb1a7ga.fsf@gitster-ct.c.googlers.com/ [2] The three long paragraphs in the middle of https://lore.kernel.org/git/CABPp-BH9tju7WVm=QZDOvaMDdZbpNXrVWQdN-jmfN8wC6YVhmw@mail.gmail.com/ [3] https://lore.kernel.org/git/CABPp-BFnFpzwGC11TLoLs8YK5yiisA5D5-fFjXnJsbESVDwZsA@mail.gmail.com/ [4] such as the vfsd described in https://lore.kernel.org/git/20220207190320.2960362-1-jonathantanmy@google.com/ Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Elijah Newren <newren@gmail.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	88c7b4c3c8	date API: create a date.h, split from cache.h Move the declaration of the date.c functions from cache.h, and adjust the relevant users to include the new date.h header. The show_ident_date() function belonged in pretty.h (it's defined in pretty.c), its two users outside of pretty.c didn't strictly need to include pretty.h, as they get it indirectly, but let's add it to them anyway. Similarly, the change to "builtin/{fast-import,show-branch,tag}.c" isn't needed as far as the compiler is concerned, but since they all use the "DATE_MODE()" macro we now define in date.h, let's have them include it. We could simply include this new header in "cache.h", but as this change shows these functions weren't common enough to warrant including in it in the first place. By moving them out of cache.h changes to this API will no longer cause a (mostly) full re-build of the project when "make" is run. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Derrick Stolee	3ce1138272	config: make git_configset_get_string_tmp() private This method was created in `f1de981e8` (config: fix leaks from git_config_get_string_const(), 2020-08-14) but its only use was in the repo_config_get_string_tmp() method, also declared in config.h and implemented in config.c. Since this is otherwise unused and is a very similar implementation to git_configset_get_value(), let's remove this declaration. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Derrick Stolee	fe18733927	config: add repo_config_set_worktree_gently() Some config settings, such as those for sparse-checkout, are likely intended to only apply to one worktree at a time. To make this write easier, add a new config API method, repo_config_set_worktree_gently(). This method will attempt to write to the worktree-specific config, but will instead write to the common config file if worktree config is not enabled. The next change will introduce a consumer of this method. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jonathan Tan	399b198489	config: include file if remote URL matches a glob This is a feature that supports config file inclusion conditional on whether the repo has a remote with a URL that matches a glob. Similar to my previous work on remote-suggested hooks [1], the main motivation is to allow remote repo administrators to provide recommended configs in a way that can be consumed more easily (e.g. through a package installable by a package manager - it could, for example, contain a file to be included conditionally and a post-install script that adds the include directive to the system-wide config file). In order to do this, Git reruns the config parsing mechanism upon noticing the first URL-conditional include in order to find all remote URLs, and these remote URLs are then used to determine if that first and all subsequent includes are executed. Remote URLs are not allowed to be configued in any URL-conditionally-included file. [1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/ Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Acked-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jonathan Tan	ed69e11b89	config: make git_config_include() static It is not used from outside the file in which it is declared. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Acked-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Josh Steadmon	44f14a9d24	config: require lowercase for branch..autosetupmerge Although we only documented that branch..autosetupmerge would accept "always" as a value, the actual implementation would accept any combination of upper- or lower-case. Fix this to be consistent with documentation and with other values of this config variable. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Josh Steadmon	d3115660b4	branch: add flags and config to inherit tracking It can be helpful when creating a new branch to use the existing tracking configuration from the branch point. However, there is currently not a method to automatically do so. Teach git-{branch,checkout,switch} an "inherit" argument to the "--track" option. When this is set, creating a new branch will cause the tracking configuration to default to the configuration of the branch point, if set. For example, if branch "main" tracks "origin/main", and we run `git checkout --track=inherit -b feature main`, then branch "feature" will track "origin/main". Thus, `git status` will show us how far ahead/behind we are from origin, and `git pull` will pull from origin. This is particularly useful when creating branches across many submodules, such as with `git submodule foreach ...` (or if running with a patch such as [1], which we use at $job), as it avoids having to manually set tracking info for each submodule. Since we've added an argument to "--track", also add "--track=direct" as another way to explicitly get the original "--track" behavior ("--track" without an argument still works as well). Finally, teach branch.autoSetupMerge a new "inherit" option. When this is set, "--track=inherit" becomes the default behavior. [1]: https://lore.kernel.org/git/20180927221603.148025-1-sbeller@google.com/ Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	f5c39c3268	config API: use get_error_routine(), not vreportf() Change the git_die_config() function added in `5a80e97c82` (config: add `git_die_config()` to the config-set API, 2014-08-07) to use the public callbacks in the usage.[ch] API instead of the the underlying vreportf() function. In preceding commits the rest of the vreportf() users outside of usage.c was migrated to die_message(), so we can now make it "static". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	25ad722126	config.c: don't leak memory in handle_path_include() Fix a memory leak in the error() path in handle_path_include(), this allows us to run t1305-config-include.sh under SANITIZE=leak, previously 4 tests there would fail. This fixes up a leak in `9b25a0b52e` (config: add include directive, 2012-02-06). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	73c5f67071	config.c: remove unused git_config_key_is_valid() The git_config_key_is_valid() function got left behind in a refactoring in `a9bcf6586d` (alias: use the early config machinery to expand aliases, 2017-06-14), It previously had two users when it was added in `9e9de18f1a` (config: silence warnings for command names with invalid keys, 2015-08-24), and after `6a1e1bc0a1` (pager: use callbacks instead of configset, 2016-09-12) only one remained. By removing it we can get rid of the "quiet" branches in this function, as well as cases where "store_key" is NULL, for which there are no other users. Out of the 5 callers of git_config_parse_key() only one needs to pass a non-NULL "size_t *baselen_", so we could remove the third parameter from the public interface. I did not find that potential simplification to be worthwhile. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
René Scharfe	8f0f110156	compression: drop write-only core_compression_* variables Since `8de7eeb54b` (compression: unify pack.compression configuration parsing, 2016-11-15) the variables core_compression_level and core_compression_seen are only set, but never read. Remove them. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jonathan Tan	e3e8bf046e	submodule-config: pass repo upon blob config read When reading the config of a submodule, if reading from a blob, read using an explicitly specified repository instead of by adding the submodule's ODB as an alternate and then reading an object from the_repository. This makes the "grep --recurse-submodules with submodules without .gitmodules in the working tree" test in t7814 work when GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB is true. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Johannes Schindelin	a03b097d63	Use a better name for the function interpolating paths It is not immediately clear what `expand_user_path()` means, so let's rename it to `interpolate_path()`. This also opens the path for interpolating more than just a home directory. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	bc40dfb10a	string-list.h users: change to use *_{nodup,dup}() Change all in-tree users of the string_list_init(LIST, BOOL) API to use string_list_init_{nodup,dup}(LIST) instead. As noted in the preceding commit let's leave the now-unused string_list_init() wrapper in-place for any in-flight users, it can be removed at some later date. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Eric Wong	dc05929411	xmmap: inform Linux users of tuning knobs on ENOMEM Linux users may benefit from additional information on how to avoid ENOMEM from mmap despite the system having enough RAM to accomodate them. We can't reliably unmap pack windows to work around the issue since malloc and other library routines may mmap without our knowledge. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Johannes Schindelin	e355307692	config: normalize the path of the system gitconfig Git for Windows is compiled with a runtime prefix, and that runtime prefix is typically `C:/Program Files/Git/mingw64`. As we want the system gitconfig to live in the sibling directory `etc`, we define the relative path as `../etc/gitconfig`. However, as reported by Philip Oakley, the output of `git config --show-origin --system -l` looks rather ugly, as it shows the path as `file:C:/Program Files/Git/mingw64/../etc/gitconfig`, i.e. with the `mingw64/../` part. By normalizing the path, we get a prettier path. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Derrick Stolee	46a237f42f	*: fix typos These typos were found while searching the codebase for gendered pronouns. In the case of t9300-fast-import.sh, remove a confusing comment that is unnecessary to the understanding of the test. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	4179b4897f	config: allow overriding of global and system configuration In order to have git run in a fully controlled environment without any misconfiguration, it may be desirable for users or scripts to override global- and system-level configuration files. We already have a way of doing this, which is to unset both HOME and XDG_CONFIG_HOME environment variables and to set `GIT_CONFIG_NOGLOBAL=true`. This is quite kludgy, and unsetting the first two variables likely has an impact on other executables spawned by such a script. The obvious way to fix this would be to introduce `GIT_CONFIG_NOGLOBAL` as an equivalent to `GIT_CONFIG_NOSYSTEM`. But in the past, it has turned out that this design is inflexible: we cannot test system-level parsing of the git configuration in our test harness because there is no way to change its location, so all tests run with `GIT_CONFIG_NOSYSTEM` set. Instead of doing the same mistake with `GIT_CONFIG_NOGLOBAL`, introduce two new variables `GIT_CONFIG_GLOBAL` and `GIT_CONFIG_SYSTEM`: - If unset, git continues to use the usual locations. - If set to a specific path, we skip reading the normal configuration files and instead take the path. By setting the path to `/dev/null`, no configuration will be loaded for the respective level. This implements the usecase where we want to execute code in a sanitized environment without any potential misconfigurations via `/dev/null`, but is more flexible and allows for more usecases than simply adding `GIT_CONFIG_NOGLOBAL`. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	1e06eb9b5d	config: unify code paths to get global config paths There's two callsites which assemble global config paths, once in the config loading code and once in the git-config(1) builtin. We're about to implement a way to override global config paths via an environment variable which would require us to adjust both sites. Unify both code paths into a single `git_global_config()` function which returns both paths for `~/.gitconfig` and the XDG config file. This will make the subsequent patch which introduces the new envvar easier to implement. No functional changes are expected from this patch. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	c62a999c6e	config: rename `git_etc_config()` The `git_etc_gitconfig()` function retrieves the system-level path of the configuration file. We're about to introduce a way to override it via an environment variable, at which point the name of this function would start to become misleading. Rename the function to `git_system_config()` as a preparatory step. While at it, the function is also refactored to pass memory ownership to the caller. This is done to better match semantics of `git_global_config()`, which is going to be introduced in the next commit. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	39e12650d7	config.c: remove last remnant of GIT_TEST_GETTEXT_POISON Remove a use of GIT_TEST_GETTEXT_POISON added in `f276e2a469` (config: improve error message for boolean config, 2021-02-11). This was simultaneously in-flight with my `d162b25f95` (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) which removed the rest of the GIT_TEST_GETTEXT_POISON code. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
René Scharfe	ca56dadb4b	use CALLOC_ARRAY Add and apply a semantic patch for converting code that open-codes CALLOC_ARRAY to use it instead. It shortens the code and infers the element size automatically. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Andrew Klotz	f276e2a469	config: improve error message for boolean config Currently invalid boolean config values return messages about 'bad numeric', which is slightly misleading when the error was due to a boolean value. We can improve the developer experience by returning a boolean error message when we know the value is neither a bool text or int. before with an invalid boolean value of `non-boolean`, its unclear what numeric is referring to: fatal: bad numeric config value 'non-boolean' for 'commit.gpgsign': invalid unit now the error message mentions `non-boolean` is a bad boolean value: fatal: bad boolean config value 'non-boolean' for 'commit.gpgsign' Signed-off-by: Andrew Klotz <agc.klotz@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Ævar Arnfjörð Bjarmason	d162b25f95	tests: remove support for GIT_TEST_GETTEXT_POISON This removes the ability to inject "poison" gettext() messages via the GIT_TEST_GETTEXT_POISON special test setup. I initially added this as a compile-time option in `bb946bba76` (i18n: add GETTEXT_POISON to simulate unfriendly translator, 2011-02-22), and most recently modified to be toggleable at runtime in `6cdccfce1e` (i18n: make GETTEXT_POISON a runtime option, 2018-11-08).. The reason for its removal is that the trade-off of maintaining it v.s. what it's getting us has long since flipped. When gettext was integrated in `5e9637c629` (i18n: add infrastructure for translating Git with gettext, 2011-11-18) there was understandable concern on the Git ML that in marking messages for translation en-masse we'd inadvertently mark plumbing messages. The GETTEXT_POISON facility was a way to smoke those out via our test suite. Nowadays however we're done (or almost entirely done) with any marking of messages for translation. New messages are usually marked by their authors, who'll know whether it makes sense to translate them or not. If not any errors in marking the messages are much more likely to be spotted in review than in the the initial deluge of i18n patches in the 2011-2012 era. So let's just remove this. This leaves the test suite in a state where we still have a lot of test_i18n, C_LOCALE_OUTPUT etc. uses. Subsequent commits will remove those too. The change to t/lib-rebase.sh is a selective revert of the relevant part of `f2d17068fd` (i18n: rebase-interactive: mark comments of squash for translation, 2016-06-17), and the comment in t/t3406-rebase-message.sh is from `c7108bf9ed` (i18n: rebase: mark messages for translation, 2012-07-25). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	d8d77153ea	config: allow specifying config entries via envvar pairs While we currently have the `GIT_CONFIG_PARAMETERS` environment variable which can be used to pass runtime configuration data to git processes, it's an internal implementation detail and not supposed to be used by end users. Next to being for internal use only, this way of passing config entries has a major downside: the config keys need to be parsed as they contain both key and value in a single variable. As such, it is left to the user to escape any potentially harmful characters in the value, which is quite hard to do if values are controlled by a third party. This commit thus adds a new way of adding config entries via the environment which gets rid of this shortcoming. If the user passes the `GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each `i` in `[0,n)`. While the same can be achieved with `git -c <name>=<value>`, one may wish to not do so for potentially sensitive information. E.g. if one wants to set `http.extraHeader` to contain an authentication token, doing so via `-c` would trivially leak those credentials via e.g. ps(1), which typically also shows command arguments. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	1ff21c05ba	config: store "git -c" variables using more robust format The previous commit added a new format for $GIT_CONFIG_PARAMETERS which is able to robustly handle subsections with "=" in them. Let's start writing the new format. Unfortunately, this does much less than you'd hope, because "git -c" itself has the same ambiguity problem! But it's still worth doing: - we've now pushed the problem from the inter-process communication into the "-c" command-line parser. This would free us up to later add an unambiguous format there (e.g., separate arguments like "git --config key value", etc). - for --config-env, the parser already disallows "=" in the environment variable name. So: git --config-env section.with=equals.key=ENVVAR will robustly set section.with=equals.key to the contents of $ENVVAR. The new test shows the improvement for --config-env. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Jeff King	f9dbb64fad	config: parse more robust format in GIT_CONFIG_PARAMETERS When we stuff config options into GIT_CONFIG_PARAMETERS, we shell-quote each one as a single unit, like: 'section.one=value1' 'section.two=value2' On the reading side, we de-quote to get the individual strings, and then parse them by splitting on the first "=" we find. This format is ambiguous, because an "=" may appear in a subsection. So the config represented in a file by both: [section "subsection=with=equals"] key = value and: [section] subsection = with=equals.key=value ends up in this flattened format like: 'section.subsection=with=equals.key=value' and we can't tell which was desired. We have traditionally resolved this by taking the first "=" we see starting from the left, meaning that we allowed arbitrary content in the value, but not in the subsection. Let's make our environment format a bit more robust by separately quoting the key and value. That turns those examples into: 'section.subsection=with=equals.key'='value' and: 'section.subsection'='with=equals.key=value' respectively, and we can tell the difference between them. We can detect which format is in use for any given element of the list based on the presence of the unquoted "=". That means we can continue to allow the old format to work to support any callers which manually used the old format, and we can even intermingle the two formats. The old format wasn't documented, and nobody was supposed to be using it. But it's likely that such callers exist in the wild, so it's nice if we can avoid breaking them. Likewise, it may be possible to trigger an older version of "git -c" that runs a script that calls into a newer version of "git -c"; that new version would see the intermingled format. This does create one complication, which is that the obvious format in the new scheme for [section] some-bool is: 'section.some-bool' with no equals. We'd mistake that for an old-style variable. And it even has the same meaning in the old style, but: [section "with=equals"] some-bool does not. It would be: 'section.with=equals=some-bool' which we'd take to mean: [section] with = equals=some-bool in the old, ambiguous style. Likewise, we can't use: 'section.some-bool'='' because that's ambiguous with an actual empty string. Instead, we'll again use the shell-quoting to give us a hint, and use: 'section.some-bool'= to show that we have no value. Note that this commit just expands the reading side. We'll start writing the new format via "git -c" in a future patch. In the meantime, the existing "git -c" tests will make sure we didn't break reading the old format. But we'll also add some explicit coverage of the two formats to make sure we continue to handle the old one after we move the writing side over. And one final note: since we're now using the shell-quoting as a semantically meaningful hint, this closes the door to us ever allowing arbitrary shell quoting, like: 'a'shell'would'be'ok'with'this'.key=value But we have never supported that (only what sq_quote() would produce), and we are probably better off keeping things simple, robust, and backwards-compatible, than trying to make it easier for humans. We'll continue not to advertise the format of the variable to users, and instead keep "git -c" as the recommended mechanism for setting config (even if we are trying to be kind not to break users who may be relying on the current undocumented format). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	b342ae61b3	config: extract function to parse config pairs The function `git_config_parse_parameter` is responsible for parsing a `foo.bar=baz`-formatted configuration key, sanitizing the key and then processing it via the given callback function. Given that we're about to add a second user which is going to process keys which already has keys and values separated, this commit extracts a function `config_parse_pair` which only does the sanitization and processing part as a preparatory step. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Patrick Steinhardt	ce81b1da23	config: add new way to pass config via `--config-env` While it's already possible to pass runtime configuration via `git -c <key>=<value>`, it may be undesirable to use when the value contains sensitive information. E.g. if one wants to set `http.extraHeader` to contain an authentication token, doing so via `-c` would trivially leak those credentials via e.g. ps(1), which typically also shows command arguments. To enable this usecase without leaking credentials, this commit introduces a new switch `--config-env=<key>=<envvar>`. Instead of directly passing a value for the given key, it instead allows the user to specify the name of an environment variable. The value of that variable will then be used as value of the key. Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Eric Wong	a9ecaa06a7	core.abbrev=no disables abbreviations This allows users to write hash-agnostic scripts and configs by disabling abbreviations. Using "-c core.abbrev=40" will be insufficient with SHA-256, and "-c core.abbrev=64" won't work with SHA-1 repos today. Signed-off-by: Eric Wong <e@80x24.org> [jc: tweaked implementation, added doc and a test] Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Derrick Stolee	c90702a1f6	config: plumb --fixed-value into config API The git_config_set_multivar_in_file_gently() and related methods now take a 'flags' bitfield, so add a new bit representing the --fixed-value option from 'git config'. This alters the purpose of the value_pattern parameter to be an exact string match. This requires some initialization changes in git_config_set_multivar_in_file_gently() and a new strcmp() call in the matches() method. The new CONFIG_FLAGS_FIXED_VALUE flag is initialized in builtin/config.c based on the --fixed-value option, and that needs to be updated in several callers. This patch only affects some of the modes of 'git config', and the rest will be completed in the next change. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Derrick Stolee	247e2f822e	config: replace 'value_regex' with 'value_pattern' The 'value_regex' argument in the 'git config' builtin is poorly named, especially related to an upcoming change that allows exact string matches instead of ERE pattern matches. Perform a mostly mechanical change of every instance of 'value_regex' to 'value_pattern' in the codebase. This is only critical for documentation and error messages, but it is best to be consistent inside the codebase, too. For documentation, use 'value-pattern' which is better punctuation. This affects Documentation/git-config.txt and the usage in builtin/config.c, which was already mixed between 'value_regex' and 'value-regex'. I gave some thought to leaving the value_regex variables inside config.c that are regex_t pointers. However, it is probably best to keep the name consistent with the rest of the variables. This does not update the translations inside the po/ directory, as that creates conflicts with ongoing work. The input strings should automatically update through automation, and a few of the output strings currently use "[value_regex]" directly. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Derrick Stolee	504ee1290e	config: convert multi_replace to flags We will extend the flexibility of the config API. Before doing so, let's take an existing 'int multi_replace' parameter and replace it with a new 'unsigned flags' parameter that can take multiple options as a bit field. Update all callers that specified multi_replace to now specify the CONFIG_FLAGS_MULTI_REPLACE flag. To add more clarity, extend the documentation of git_config_set_multivar_in_file() including a clear labeling of its arguments. Other config API methods in config.h require only a change of the final parameter from 'int' to 'unsigned'. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Elijah Newren	6da1a25814	hashmap: provide deallocation function names hashmap_free(), hashmap_free_entries(), and hashmap_free_() have existed for a while, but aren't necessarily the clearest names, especially with hashmap_partial_clear() being added to the mix and lazy-initialization now being supported. Peff suggested we adopt the following names[1]: - hashmap_clear() - remove all entries and de-allocate any hashmap-specific data, but be ready for reuse - hashmap_clear_and_free() - ditto, but free the entries themselves - hashmap_partial_clear() - remove all entries but don't deallocate table - hashmap_partial_clear_and_free() - ditto, but free the entries This patch provides the new names and converts all existing callers over to the new naming scheme. [1] https://lore.kernel.org/git/20201030125059.GA3277724@coredump.intra.peff.net/ Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	4 years ago
Jeff King	1c890016a1	config: fix leak in git_config_get_expiry_in_days() We use git_config_get_string() to retrieve the expiry value in a newly allocated string. But after parsing it, we never free it, leaking the memory. We could fix this with a free() obviously, but there's an even better solution: we can use the non-allocating "tmp" variant of the function; we only need it to be valid for the lifetime of our parse function. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jeff King	9a53219f69	config: drop git_config_get_string_const() As evidenced by the leak fixes in the previous commit, the "const" in git_config_get_string_const() clearly misleads people into thinking that it does not allocate a copy of the string. We can fix this by renaming it, but it's easier still to just drop it. Of the four remaining callers: - The one in git_config_parse_expiry() still needs to allocate, since that's what its callers expect. We can just use the non-const version and cast our pointer. Slightly ugly, but the damage is contained in one spot. - The two in apply are writing to global "const char " variables, and need to continue allocating. We often mark these as const because we assign default string literals to them. But in this case we don't do that, so we can just declare them as real "char " pointers and use the non-const version. - The call in checkout doesn't actually need a copy; it can just use the non-allocating "tmp" version of the function. The function is also mentioned in the MyFirstContribution document. We can swap that call out for the non-allocating "tmp" variant, which fits well in the example given. We'll drop the "configset" and "repo" variants, as well (which are unused). Note that this frees up the "const" name, so we could rename the "tmp" variant back to that. But let's give some time for topics in flight to adapt to the new code before doing so (if we do it too soon, the function semantics will change but the compiler won't alert us). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jeff King	f1de981e8b	config: fix leaks from git_config_get_string_const() There are two functions to get a single config string: - git_config_get_string() - git_config_get_string_const() One might naively think that the first one allocates a new string and the second one just points us to the internal configset storage. But in fact they both allocate a new copy; the second one exists only to avoid having to cast when using it with a const global which we never intend to free. The documentation for the function explains that clearly, but it seems I'm not alone in being surprised by this. Of 17 calls to the function, 13 of them leak the resulting value. We could obviously fix these by adding the appropriate free(). But it would be simpler still if we actually had a non-allocating way to get the string. There's git_config_get_value() but that doesn't quite do what we want. If the config key is present but is a boolean with no value (e.g., "[foo]bar" in the file), then we'll get NULL (whereas the string versions will print an error and die). So let's introduce a new variant, git_config_get_string_tmp(), that behaves as these callers expect. We need a new name because we have new semantics but the same function signature (so even if we converted the four remaining callers, topics in flight might be surprised). The "tmp" is because this value should only be held onto for a short time. In practice it's rare for us to clear and refresh the configset, invalidating the pointer, but hopefully the "tmp" makes callers think about the lifetime. In each of the converted cases here the value only needs to last within the local function or its immediate caller. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jeff King	aec0bba106	config: work around gcc-10 -Wstringop-overflow warning Compiling with gcc-10, -O2, and -fsanitize=undefined results in a compiler warning: config.c: In function ‘git_config_copy_or_rename_section_in_file’: config.c:3170:17: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=] 3170 \| output[0] = '\t'; \| ~~~~~~~~~~^~~~~~ config.c:3076:7: note: at offset -1 to object ‘buf’ with size 1024 declared here 3076 \| char buf[1024]; \| ^~~ This is a false positive. The interesting lines of code are: int i; char output = buf; ... for (i = 0; buf[i] && isspace(buf[i]); i++) ; / do nothing / ... int offset; offset = section_name_match(&buf[i], old_name); if (offset > 0) { ... output += offset + i; if (strlen(output) > 0) { / * More content means there's * a declaration to put on the * next line; indent with a * tab */ output -= 1; output[0] = '\t'; } } So we do assign output to buf initially. Later we increment it based on "offset" and "i" and then subtract "1" from it. That latter step is what the compiler is complaining about; it could lead to going off the left side of the array if "output == buf" at the moment of the subtraction. For that to be the case, then "offset + i" would have to be 0. But that can't happen: - we know that "offset" is at least 1, since we're in a conditional block that checks that - we know that "i" is not negative, since it started at 0 and only incremented over whitespace So the sum must be at least 1, and therefore it's OK to subtract one from "output". But that's not quite the whole story. Since "i" is an int, it could in theory be possible to overflow to negative (when counting whitespace on a very large string). But we know that's impossible because we're counting the 1024-byte buffer we just fed to fgets(), so it can never be larger than that. Switching the type of "i" to "unsigned" makes the warning go away, so let's do that. Arguably size_t is an even better type (for this and for the other length fields), but switching to it produces a similar but distinct warning: config.c: In function ‘git_config_copy_or_rename_section_in_file’: config.c:3170:13: error: array subscript -1 is outside array bounds of ‘char[1024]’ [-Werror=array-bounds] 3170 \| output[0] = '\t'; \| ~~~~~~^~~ config.c:3076:7: note: while referencing ‘buf’ 3076 \| char buf[1024]; \| ^~~ If we were to ever switch off of fgets() to strbuf_getline() or similar, we'd probably need to use size_t to avoid other overflow problems. But for now we know we're safe because of the small fixed size of our buffer. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jeff King	348482dede	config: reject parsing of files over INT_MAX While the last few commits have made it possible for the config parser to handle config files up to the limits of size_t, the rest of the code isn't really ready for this. In particular, we often feed the keys as strings into printf "%s" format specifiers. And because the printf family of functions must return an int to specify the result, they complain. Here are two concrete examples (using glibc; we're in uncharted territory here so results may vary): Generate a gigantic .gitmodules file like this: git submodule add /some/other/repo foo { printf '[submodule "' perl -e 'print "a" x 2**31' echo '"]path = foo' } >.gitmodules git commit -m 'huge gitmodule' then try this: $ git show BUG: strbuf.c:397: your vsnprintf is broken (returned -1) The problem is that we end up calling: strbuf_addf(&sb, "submodule.%s.ignore", submodule_name); which relies on vsnprintf(), and that function has no way to report back a size larger than INT_MAX. Taking that same file, try this: git config --file=.gitmodules --list --name-only On my system it produces an output with exactly 4GB of spaces. I confirmed in a debugger that we reach the config callback with the key intact: it's 2147483663 bytes and full of a's. But when we print it with this call: printf("%s%c", key_, term); we just get the spaces. So given the fact that these are insane cases which we have no need to support, the weird behavior from feeding the results to printf even if the code is careful, and the possibility of uncareful code introducing its own integer truncation issues, let's just declare INT_MAX as a limit for parsing config files. We'll enforce the limit in get_next_char(), which generalizes over all sources (blobs, files, etc) and covers any element we're parsing (whether section, key, value, etc). For simplicity, the limit is over the length of the _whole_ file, so you couldn't have two 1GB values in the same file. This should be perfectly fine, as the expected size for config files is generally kilobytes at most. With this patch both cases above will yield: fatal: bad config line 1 in file .gitmodules That's not an amazing error message, but the parser isn't set up to provide specific messages (it just breaks out of the parsing loop and gives that generic error even if see a syntactic issue). And we really wouldn't expect to see this case outside of somebody maliciously probing the limits of the config system. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jeff King	6a9c235eb4	config: use size_t to store parsed variable baselen Most of the config parsing infrastructure is limited in what it can parse only by the size of memory, because it parses character by character, building up strbufs for keys, values, etc. One exception is the "baselen" value we keep in git_parse_source(), which is an int. That stores the length of the section.subsection base, to which we can then append individual key names (by truncating back to the baselen with strbuf_setlen(), and then appending characters for the key name). But because it's an int, if we see an absurdly long section or subsection, we may overflow the integer, wrapping negative. That negative value is then implicitly cast to a size_t when we pass it to strbuf_setlen(), creating a very large value and triggering a BUG. For example: $ { printf '[foo "' perl -e 'print "a" x 231' echo '"]bar = value' } >huge $ git config --file=huge --list fatal: BUG: strbuf_setlen() beyond buffer While this is obviously a silly case that we don't care about supporting, it's worth fixing it by switching to a size_t for a few reasons: - we should try to avoid hitting BUG assertions at all - avoiding integer truncation or overflow sets a good example and makes it easier to audit the code for more important issues - the BUG outcome is what happens in _this_ instance, because we wrap negative. If we used a 232 subsection, we'd wrap to a small positive value and actually generate wrong output (the subsection of our key would be truncated). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago
Jeff King	f011a9654d	git_config_parse_key(): return baselen as size_t As with the recent change to parse_config_key(), the best type to return a string length is a size_t, as it won't cause integer truncation for a gigantic key. And as with that change, this is mostly a clarity / hygiene issue for now, as our config parser would choke on such a large key anyway. There are a few ripple effects within the config code, as callers switch to using size_t. I also adjusted a few related variables that iterate over strings. The most unexpected change is that a call to strbuf_addf() had to switch to strbuf_add(). We can't use a size_t with "%.*s", because printf precisions must have type "int" (we could cast, of course, but that would miss the point of using size_t in the first place). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	5 years ago

1 2 3 4 5 ...

755 Commits (08dccc8fc1f248c8de7e87ac6e435edac4f77ebe)