kernel/git - git - PowerEL Git System

Commit Graph

Author	SHA1	Message	Date
Elia Pinto	96ddfecc5b	Makefile: add a prerequisite to the coverage-report target Directly invoking make coverage-report as a target results in an error because its prerequisites are missing, This patch adds the compile-test prerequisite, which is run only once each time the compile-report target is invoked. In practice, the developer may decide to review the coverage-report results without necessarily rerunning for this coverage-test, if it has already been run. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Phillip Wood	067109a5e7	tests: make SANITIZE=address imply TEST_NO_MALLOC_CHECK As the address sanitizer checks for a superset of the issues detected by setting MALLOC_CHECK_ (which tries to detect things like double frees and off-by-one errors) there is no need to set the latter when compiling with -fsanitize=address. This fixes a regression introduced by `131b94a10a` ("test-lib.sh: Use GLIBC_TUNABLES instead of MALLOC_CHECK_ on glibc >= 2.34", 2022-03-04) which causes all the tests to fail with the message ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. when git is compiled with SANITIZE=address on systems with glibc >= 2.34. I have tested SANITIZE=leak and SANITIZE=undefined and they do not suffer from this regression so the fix in this patch should be sufficient. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Carlo Marcelo Arenas Belón	5b52d9f15e	git-compat-util: really support openssl as a source of entropy `05cd988dce` (wrapper: add a helper to generate numbers from a CSPRNG, 2022-01-17), configure openssl as the source for entropy in NON-STOP but doesn't add the needed header or link options. Since the only system that is configured to use openssl as a source of entropy is NON-STOP, add the header unconditionally, and -lcrypto to the list of external libraries. An additional change is required to make sure a NO_OPENSSL=1 build will be able to work as well (tested on Linux with a modified value of CSPRNG_METHOD = openssl), and the more complex logic that allows for compatibility with APPLE_COMMON_CRYPTO or allowing for simpler ways to link (without libssl) has been punted for now. Reported-by: Randall Becker <rsbecker@nexbridge.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jeff Hostetler	148405fb27	t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon Create an IPC client to send query and flush commands to the daemon. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jeff Hostetler	62c7367133	compat/fsmonitor/fsm-listen-win32: stub in backend for Windows Stub in empty filesystem listener backend for fsmonitor--daemon on Windows. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jeff Hostetler	16d9d6175b	fsmonitor--daemon: add a built-in fsmonitor daemon Create a built-in file system monitoring daemon that can be used by the existing `fsmonitor` feature (protocol API and index extension) to improve the performance of various Git commands, such as `status`. The `fsmonitor--daemon` feature builds upon the `Simple IPC` API and provides an alternative to hook access to existing fsmonitors such as `watchman`. This commit merely adds the new command without any functionality. Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jeff Hostetler	1e0ea5c431	fsmonitor: config settings are repository-specific Move fsmonitor config settings to a new and opaque `struct fsmonitor_settings` structure. Add a lazily-loaded pointer to this into `struct repo_settings` Create an `enum fsmonitor_mode` type in `struct fsmonitor_settings` to represent the state of fsmonitor. This lets us represent which, if any, fsmonitor provider (hook or IPC) is enabled. Create `fsm_settings__get_()` getters to lazily look up fsmonitor- related config settings. Get rid of the `core_fsmonitor` global variable. Move the code to lookup the existing `core.fsmonitor` config value into the fsmonitor settings. Create a hook pathname variable in `struct fsmonitor-settings` and only set it when in hook mode. Extend the definition of `core.fsmonitor` to be either a boolean or a hook pathname. When true, the builtin FSMonitor is used. When false or unset, no FSMonitor (neither builtin nor hook) is used. The existing `core_fsmonitor` global variable was used to store the pathname to the fsmonitor hook and* it was used as a boolean to see if fsmonitor was enabled. This dual usage and global visibility leads to confusion when we add the IPC-based provider. So lets hide the details in fsmonitor-settings.c and let it decide which provider to use in the case of multiple settings. This avoids cluttering up repo-settings.c with these private details. A future commit in builtin-fsmonitor series will add the ability to disqualify worktrees for various reasons, such as being mounted from a remote volume, where fsmonitor should not be started. Having the config settings hidden in fsmonitor-settings.c allows such worktree restrictions to override the config values used. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Jeff Hostetler	d2bd862e7a	fsmonitor-ipc: create client routines for git-fsmonitor--daemon Create fsmonitor_ipc__() client routines to spawn the built-in file system monitor daemon and send it an IPC request using the `Simple IPC` API. Stub in empty fsmonitor_ipc__() functions for unsupported platforms. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Neeraj Singh	abf38abec2	core.fsyncmethod: add writeout-only mode This commit introduces the `core.fsyncMethod` configuration knob, which can currently be set to `fsync` or `writeout-only`. The new writeout-only mode attempts to tell the operating system to flush its in-memory page cache to the storage hardware without issuing a CACHE_FLUSH command to the storage controller. Writeout-only fsync is significantly faster than a vanilla fsync on common hardware, since data is written to a disk-side cache rather than all the way to a durable medium. Later changes in this patch series will take advantage of this primitive to implement batching of hardware flushes. When git_fsync is called with FSYNC_WRITEOUT_ONLY, it may fail and the caller is expected to do an ordinary fsync as needed. On Apple platforms, the fsync system call does not issue a CACHE_FLUSH directive to the storage controller. This change updates fsync to do fcntl(F_FULLFSYNC) to make fsync actually durable. We maintain parity with existing behavior on Apple platforms by setting the default value of the new core.fsyncMethod option. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	0b6d0bc924	Makefiles: add and use wildcard "mkdir -p" template Add a template to do the "mkdir -p" of $(@D) (the parent dir of $@) for us, and use it for the "make lint-docs" targets I added in `8650c6298c` (doc lint: make "lint-docs" non-.PHONY, 2021-10-15). As seen in `4c64fb5aad` (Documentation/Makefile: fix lint-docs mkdir dependency, 2021-10-26) maintaining these manual lists of parent directory dependencies is fragile, in addition to being obviously verbose. I used this pattern at the time because I couldn't find another method than "order-only" prerequisites to avoid doing a "mkdir -p $(@D)" for every file being created, which as noted in [1] would be significantly slower. But as it turns out we can use this neat trick of only doing a "mkdir -p" if the $(wildcard) macro tells us the path doesn't exist. A re-run of a performance test similar to that noted downthread of [1] in [2] shows that this is faster, in addition to being less verbose and more reliable (this uses my "git-hyperfine" thin wrapper for "hyperfine"[3]): $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make -C Documentation lint-docs' -p 'rm -rf Documentation/.build' 'make -C Documentation -j1 lint-docs' Benchmark 1: make -C Documentation -j1 lint-docs' in 'HEAD~1 Time (mean ± σ): 2.914 s ± 0.062 s [User: 2.449 s, System: 0.489 s] Range (min … max): 2.834 s … 3.020 s 10 runs Benchmark 2: make -C Documentation -j1 lint-docs' in 'HEAD~0 Time (mean ± σ): 2.315 s ± 0.062 s [User: 1.950 s, System: 0.386 s] Range (min … max): 2.229 s … 2.397 s 10 runs Summary 'make -C Documentation -j1 lint-docs' in 'HEAD~0' ran 1.26 ± 0.04 times faster than 'make -C Documentation -j1 lint-docs' in 'HEAD~1' So let's use that pattern both for the "lint-docs" target, and a few miscellaneous other targets. This method of creating parent directories is explicitly racy in that we don't know if we're going to say always create a "foo" followed by a "foo/bar" under parallelism, or skip the "foo" because we created "foo/bar" first. In this case it doesn't matter for anything except that we aren't guaranteed to get the same number of rules firing when running make in parallel. 1. https://lore.kernel.org/git/211028.861r45y3pt.gmgdl@evledraar.gmail.com/ 2. https://lore.kernel.org/git/211028.86o879vvtp.gmgdl@evledraar.gmail.com/ 3. https://gitlab.com/avar/git-hyperfine/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	a9fda017f4	Makefile: add "$(QUIET)" boilerplate to shared.mak The $(QUIET) variables we define are largely duplicated between our various Makefiles, let's define them in the new "shared.mak" instead. Since we're not using the environment to pass these around we don't need to export the "QUIET_GEN" and "QUIET_BUILT_IN" variables anymore. The "QUIET_GEN" variable is used in "git-gui/Makefile" and "gitweb/Makefile", but they've got their own definition for those. The "QUIET_BUILT_IN" variable is only used in the top-level "Makefile". We still need to export the "V" variable. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	fd15f8a5fa	Makefile: move $(comma), $(empty) and $(space) to shared.mak Move these variables over to the shared.mak, we'll make use of them in a subsequent commit. Note that there's reason for these to be "simply expanded variables", i.e. to use ":=" assignments instead of lazily expanded "=" assignments. We could use "=", but let's leave this as-is for now for ease of review. See `425ca6710b` (Makefile: allow combining UBSan with other sanitizers, 2017-07-15) for the commit that introduced these. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	dad9cd7d51	Makefile: move ".SUFFIXES" rule to shared.mak This was added in `30248886ce` (Makefile: disable default implicit rules, 2010-01-26), let's move it to the top of "shared.mak" so it'll apply to all our Makefiles. This doesn't benefit the main Makefile at all, since it already had the rule, but since we're including shared.mak in other Makefiles starts to benefit them. E.g. running the 'man" target is now faster: $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make -C Documentation man' 'make -C Documentation -j1 man' Benchmark 1: make -C Documentation -j1 man' in 'HEAD~1 Time (mean ± σ): 121.7 ms ± 8.8 ms [User: 105.8 ms, System: 18.6 ms] Range (min … max): 112.8 ms … 148.4 ms 26 runs Benchmark 2: make -C Documentation -j1 man' in 'HEAD~0 Time (mean ± σ): 97.5 ms ± 8.0 ms [User: 80.1 ms, System: 20.1 ms] Range (min … max): 89.8 ms … 111.8 ms 32 runs Summary 'make -C Documentation -j1 man' in 'HEAD~0' ran 1.25 ± 0.14 times faster than 'make -C Documentation -j1 man' in 'HEAD~1' The reason for that can be seen when comparing that run with "--debug=a". Without this change making a target like "git-status.1" will cause "make" to consider not only "git-status.txt", but "git-status.txt.o", as well as numerous other implicit suffixes such as ".c", ".cc", ".cpp" etc. See [1] for a more detailed before/after example. So this is causing us to omit a bunch of work we didn't need to do. For making "git-status.1" the "--debug=a" output is reduced from ~140k lines to ~6k. 1. https://lore.kernel.org/git/220222.86bkyz875k.gmgdl@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	f4c6a526a1	Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES) Combine the definitions of $(FIND_SOURCE_FILES) and $(LIB_H) to speed up the Makefile, as these are the two main expensive $(shell) commands that we execute unconditionally. When see what was in $(FOUND_SOURCE_FILES) that wasn't in $(LIB_H) via the ad-hoc test of: $(error $(filter-out $(LIB_H),$(filter %.h,$(ALL_SOURCE_FILES)))) $(error $(filter-out $(ALL_SOURCE_FILES),$(filter %.h,$(LIB_H)))) We'll get, respectively: Makefile:850: * t/helper/test-tool.h. Stop. Makefile:850: * . Stop. I.e. we only had a discrepancy when it came to t/helper/test-tool.h. In terms of correctness this was broken before, but now works: $ make t/helper/test-tool.hco HDR t/helper/test-tool.h This speeds things up a lot: $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make NO_TCLTK=Y' 'make -j1 NO_TCLTK=Y' --warmup 10 -M 10 Benchmark 1: make -j1 NO_TCLTK=Y' in 'HEAD~1 Time (mean ± σ): 159.9 ms ± 6.8 ms [User: 137.2 ms, System: 28.0 ms] Range (min … max): 154.6 ms … 175.9 ms 10 runs Benchmark 2: make -j1 NO_TCLTK=Y' in 'HEAD~0 Time (mean ± σ): 100.0 ms ± 1.3 ms [User: 84.2 ms, System: 20.2 ms] Range (min … max): 98.8 ms … 102.8 ms 10 runs Summary 'make -j1 NO_TCLTK=Y' in 'HEAD~0' ran 1.60 ± 0.07 times faster than 'make -j1 NO_TCLTK=Y' in 'HEAD~1' Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	8df786d298	Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it We have various behavior that's shared across our Makefiles, or that really should be (e.g. via defined templates). Let's create a top-level "shared.mak" to house those sorts of things, and start by adding the ".DELETE_ON_ERROR" flag to it. See my own `7b76d6bf22` (Makefile: add and use the ".DELETE_ON_ERROR" flag, 2021-06-29) and `db10fc6c09` (doc: simplify Makefile using .DELETE_ON_ERROR, 2021-05-21) for the addition and use of the ".DELETE_ON_ERROR" flag. I.e. this changes the behavior of existing rules in the altered Makefiles (except "Makefile" & "Documentation/Makefile"). I'm confident that this is safe having read the relevant rules in those Makfiles, and as the GNU make manual notes that it isn't the default behavior is out of an abundance of backwards compatibility caution. From edition 0.75 of its manual, covering GNU make 4.3: [Enabling '.DELETE_ON_ERROR' is] almost always what you want 'make' to do, but it is not historical practice; so for compatibility, you must explicitly request it. This doesn't introduce a bug by e.g. having this ".DELETE_ON_ERROR" flag only apply to this new shared.mak, Makefiles have no such scoping semantics. It does increase the danger that any Makefile without an explicit "The default target of this Makefile is..." snippet to define the default target as "all" could have its default rule changed if our new shared.mak ever defines a "real" rule. In subsequent commits we'll be careful not to do that, and such breakage would be obvious e.g. in the case of "make -C t". We might want to make that less fragile still (e.g. by using ".DEFAULT_GOAL" as noted in the preceding commit), but for now let's simply include "shared.mak" without adding that boilerplate to all the Makefiles that don't have it already. Most of those are already exposed to that potential caveat e.g. due to including "config.mak*". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
John Cai	7d3d226e70	reflog: libify delete reflog function and helpers Currently stash shells out to reflog in order to delete refs. In an effort to reduce how much we shell out to a subprocess, libify the functionality that stash needs into reflog.c. Add a reflog_delete function that is pretty much the logic in the while loop in builtin/reflog.c cmd_reflog_delete(). This is a function that builtin/reflog.c and builtin/stash.c can both call. Also move functions needed by reflog_delete and export them. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: John Cai <johncai86@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	07564773c2	compat: auto-detect if zlib has uncompress2() We have a copy of uncompress2() implementation in compat/ so that we can build with an older version of zlib that lack the function, and the build procedure selects if it is used via the NO_UNCOMPRESS2 $(MAKE) variable. This is yet another "annoying" knob the porters need to tweak on platforms that are not common enough to have the default set in the config.mak.uname file. Attempt to instead ask the system header <zlib.h> to decide if we need the compatibility implementation. This is a deviation from the way we have been handling the "compatiblity" features so far, and if it can be done cleanly enough, it could work as a model for features that need compatibility definition we discover in the future. With that goal in mind, avoid expedient but ugly hacks, like shoving the code that is conditionally compiled into an unrelated .c file, which may not work in future cases---instead, take an approach that uses a file that is independently compiled and stands on its own. Compile and link compat/zlib-uncompress2.c file unconditionally, but conditionally hide the implementation behind #if/#endif when zlib version is 1.2.9 or newer, and unconditionally archive the resulting object file in the libgit.a to be picked up by the linker. There are a few things to note in the shape of the code base after this change: - We no longer use NO_UNCOMPRESS2 knob; if the system header <zlib.h> claims a version that is more cent than the library actually is, this would break, but it is easy to add it back when we find such a system. - The object file compat/zlib-uncompress2.o is always compiled and archived in libgit.a, just like a few other compat/ object files already are. - The inclusion of <zlib.h> is done in <git-compat-util.h>; we used to do so from <cache.h> which includes <git-compat-util.h> as the first thing it does, so from the *.c codes, there is no practical change. - Until objects in libgit.a that is already used gains a reference to the function, the reftable code will be the only one that wants it, so libgit.a on the linker command line needs to appear once more at the end to satisify the mutual dependency. - Beat found a trick used by OpenSSL to avoid making the conditionally-compiled object truly empty (apparently because they had to deal with compilers that do not want to see an effectively empty input file). Our compat/zlib-uncompress2.c file borrows the same trick for portabilty. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: Beat Bolli <dev+git@drbeat.li> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
brian m. carlson	05cd988dce	wrapper: add a helper to generate numbers from a CSPRNG There are many situations in which having access to a cryptographically secure pseudorandom number generator (CSPRNG) is helpful. In the future, we'll encounter one of these when dealing with temporary files. To make this possible, let's add a function which reads from a system CSPRNG and returns some bytes. We know that all systems will have such an interface. A CSPRNG is required for a secure TLS or SSH implementation and a Git implementation which provided neither would be of little practical use. In addition, POSIX is set to standardize getentropy(2) in the next version, so in the (potentially distant) future we can rely on that. For systems which lack one of the other interfaces, we provide the ability to use OpenSSL's CSPRNG. OpenSSL is highly portable and functions on practically every known OS, and we know it will have access to some source of cryptographically secure randomness. We also provide support for the arc4random in libbsd for folks who would prefer to use that. Because this is a security sensitive interface, we take some precautions. We either succeed by filling the buffer completely as we requested, or we fail. We don't return partial data because the caller will almost never find that to be a useful behavior. Specify a makefile knob which users can use to specify one or more suitable CSPRNGs, and turn the multiple string options into a set of defines, since we cannot match on strings in the preprocessor. We allow multiple options to make the job of handling this in autoconf easier. The order of options is important here. On systems with arc4random, which is most of the BSDs, we use that, since, except on MirBSD and macOS, it uses ChaCha20, which is extremely fast, and sits entirely in userspace, avoiding a system call. We then prefer getrandom over getentropy, because the former has been available longer on Linux, and then OpenSSL. Finally, if none of those are available, we use /dev/urandom, because most Unix-like operating systems provide that API. We prefer options that don't involve device files when possible because those work in some restricted environments where device files may not be available. Set the configuration variables appropriately for Linux and the BSDs, including macOS, as well as Windows and NonStop. We specifically only consider versions which receive publicly available security support here. For the same reason, we don't specify getrandom(2) on Linux, because CentOS 7 doesn't support it in glibc (although its kernel does) and we don't want to resort to making syscalls. Finally, add a test helper to allow this to be tested by hand and in tests. We don't add any tests, since invoking the CSPRNG is not likely to produce interesting, reproducible results. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Emily Shaffer	96e7225b31	hook: add 'run' subcommand In order to enable hooks to be run as an external process, by a standalone Git command, or by tools which wrap Git, provide an external means to run all configured hook commands for a given hook event. Most of our hooks require more complex functionality than this, but let's start with the bare minimum required to support our simplest hooks. In terms of implementation the usage_with_options() and "goto usage" pattern here mirrors that of builtin/{commit-graph,multi-pack-index}.c. Some of the implementation here, such as a function being named run_hooks_opt() when it's tasked with running one hook, to using the run_processes_parallel_tr2() API to run with jobs=1 is somewhere between a bit odd and and an overkill for the current features of this "hook run" command and the hook.[ch] API. This code will eventually be able to run multiple hooks declared in config in parallel, by starting out with these names and APIs we reduce the later churn of renaming functions, switching from the run_command() to run_processes_parallel_tr2() API etc. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	2f12b31b74	Makefile: don't invoke msgfmt with --statistics Remove the --statistics flag that I added in `5e9637c629` (i18n: add infrastructure for translating Git with gettext, 2011-11-18). Our Makefile output is good about reducing verbosity by default, except in this case: $ rm -rf po/build/locale/e; time make -j $(nproc) all SUBDIR templates MKDIR -p po/build/locale/el/LC_MESSAGES MSGFMT po/build/locale/el/LC_MESSAGES/git.mo MKDIR -p po/build/locale/es/LC_MESSAGES MSGFMT po/build/locale/es/LC_MESSAGES/git.mo 1038 translated messages, 3325 untranslated messages. 5230 translated messages. I didn't have any good reason for using --statistics at the time other than ad-hoc eyeballing of the output. We don't need to spew out exactly how many messages we've got translated every time. Now we'll instead emit: $ rm -rf po/build/locale/e; time make -j $(nproc) all SUBDIR templates MKDIR -p po/build/locale/el/LC_MESSAGES MSGFMT po/build/locale/el/LC_MESSAGES/git.mo MKDIR -p po/build/locale/es/LC_MESSAGES MSGFMT po/build/locale/es/LC_MESSAGES/git.mo Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	451a7dbe28	Makefile: move -DPAGER_ENV from BASIC_CFLAGS to EXTRA_CPPFLAGS Remove -DPAGER_ENV from the BASIC_CFLAGS and instead have it passed via the EXTRA_CPPFLAGS passed when compiling pager.c. This doesn't change anything except to make it clear that only pager.c needs this, as it's the only user of this define. See `995bc22d7f` (pager: move pager-specific setup into the build, 2016-08-04) for the commit that originally added this. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	d3fd1a6667	Makefile: correct the dependency graph of hook-list.h Fix an issue in my `cfe853e66b` (hook-list.h: add a generated list of hooks, like config-list.h, 2021-09-26), the builtin/help.c was inadvertently made to depend on hook-list.h, but it's used by builtin/bugreport.c. The hook.c also does not depend on hook-list.h. It did in an earlier version of the greater series `cfe853e66b` was extracted from, but not anymore. We might end up needing that line again, but let's remove it for now. Reported-by: Mike Hommey <mh@glandium.org> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Johannes Schindelin	0a43fb2202	scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Junio C Hamano	44ba10d671	revision: use C99 declaration of variable in for() loop There are certain C99 features that might be nice to use in our code base, but we've hesitated to do so in order to avoid breaking compatibility with older compilers. But we don't actually know if people are even using pre-C99 compilers these days. One way to figure that out is to introduce a very small use of a feature, and see if anybody complains, and we've done so to probe the portability for a few features like "trailing comma in enum declaration", "designated initializer for struct", and "designated initializer for array". A few years ago, we tried to use a handy for (int i = 0; i < n; i++) use(i); to introduce a new variable valid only in the loop, but found that some compilers we cared about didn't like it back then. Two years is a long-enough time, so let's try it again. If this patch can survive a few releases without complaint, then we can feel more confident that variable declaration in for() loop is supported by the compilers our user base use. And if we do get complaints, then we'll have gained some data and we can easily revert this patch. Helped-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
brian m. carlson	7bc341e21b	git-compat-util: add a test balloon for C99 support The C99 standard was released in January 1999, now 22 years ago. It provides a variety of useful features, including variadic arguments for macros, declarations after statements, designated initializers, and a wide variety of other useful features, many of which we already use. We'd like to take advantage of these features, but we want to be cautious. As far as we know, all major compilers now support C99 or a later C standard, such as C11 or C17. POSIX has required C99 support as a requirement for the 2001 revision, so we can safely assume any POSIX system which we are interested in supporting has C99. Even MSVC, long a holdout against modern C, now supports both C11 and C17 with an appropriate update. Moreover, even if people are using an older version of MSVC on these systems, they will generally need some implementation of the standard Unix utilities for the testsuite, and GNU coreutils, the most common option, has required C99 since 2009. Therefore, we can safely assume that a suitable version of GCC or clang is available to users even if their version of MSVC is not sufficiently capable. Let's add a test balloon to git-compat-util.h to see if anyone is using an older compiler. We'll add a comment telling people how to enable this functionality on GCC and Clang, even though modern versions of both will automatically do the right thing, and ask people still experiencing a problem to report that to us on the list. Note that C89 compilers don't provide the __STDC_VERSION__ macro, so we use a well-known hack of using "- 0". On compilers with this macro, it doesn't change the value, and on C89 compilers, the macro will be replaced with nothing, and our value will be 0. For sparse, we explicitly request the gnu99 style because we've traditionally taken advantage of some GCC- and clang-specific extensions when available and we'd like to retain the ability to do that. sparse also defaults to C89 without it, so things will fail for us if we don't. Update the cmake configuration to require C11 for MSVC. We do this because this will make MSVC to use C11, since it does not explicitly support C99. We do this with a compiler options because setting the C_STANDARD option does not work in our CI on MSVC and at the moment, we don't want to require C11 for Unix compilers. In the Makefile, don't set any compiler flags for the compiler itself, since on some systems, such as FreeBSD, we actually need C11, and asking for C99 causes things to fail to compile. The error message should make it obvious what's going wrong and allow a user to set the appropriate option when building in the event they're using a Unix compiler that doesn't support it by default. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	8a7a90bc3d	Makefile: remove redundant GIT-CFLAGS dependency from "sparse" The "sparse" target needed the GIT-CFLAGS dependency before my `c234e8a0ec` (Makefile: make the "sparse" target non-.PHONY, 2021-09-23), but since then it depends on the corresponding *.o files, which in turn depend on the correct header files, as well as on GIT-CFLAGS. There's no need to re-state this dependency here. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	ebeb39faad	git-sh-setup: remove "sane_grep", it's not needed anymore Remove the sane_grep() shell function in git-sh-setup. The two reasons for why it existed don't apply anymore: 1. It was added due to GNU grep supporting GREP_OPTIONS. See `e1622bfcba` (Protect scripted Porcelains from GREP_OPTIONS insanity, 2009-11-23). Newer versions of GNU grep ignore that, but even on older versions its existence won't matter, none of these sane_grep() uses care about grep's output, they're merely using it to check if a string exists in a file or stream. We also don't care about the "LC_ALL=C" that "sane_grep" was using, these greps for fixed or ASCII strings will behave the same under any locale. 2. The SANE_TEXT_GREP added in `71b401032b` (sane_grep: pass "-a" if grep accepts it, 2016-03-08) isn't needed either, none of these grep uses deal with binary data. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	c7c9692897	Makefile: remove $(NO_CURL) from $(SCRIPT_DEFINES) Stop including $(NO_CURL) in $(SCRIPT_DEFINES). The "@NO_CURL@" replacement added in `6c5c62f340` (Print an error if cloning a http repo and NO_CURL is set, 2006-02-15) has not been referenced by anything in-tree since `49eb8d39c7` (Remove contrib/examples/, 2018-03-25). That commit removed the reference from contrib/examples/, but this @@NO_CURL@@ hasn't been used since git-pull.sh was the primary entry point for "git pull". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	ab77294a2a	Makefile: remove $(GIT_VERSION) from $(SCRIPT_DEFINES) Remove the $(GIT_VERSION) from $(SCRIPT_DEFINES). Now every time HEAD changes in a development copy we don't need to re-build the scripts and script libraries. This has not been needed since `2b9391bc67` (Makefile: do not replace @@GIT_VERSION@@ in shell scripts, 2012-06-20). On my setup this changes the re-making of 44 targets in a development copy where moved HEAD to 27. The $(GIT_VERSION) was seemingly left here by mistake or omission. We didn't need it since `2b9391bc67`, but in the later `e4dd89ab98` (Makefile: update scripts when build-time parameters change, 2012-06-20) it was added to SCRIPT_DEFINES. The two were part of the same series of patches, and given the summary in [1] and [2] it looks like this was probably a case of some earlier version of a later patch being combined with an updated earlier patch. 1. https://lore.kernel.org/git/20120619232231.GA6328@sigill.intra.peff.net/ 2. https://lore.kernel.org/git/20120619232453.GB6496@sigill.intra.peff.net/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	7212f2887a	Makefile: move git-SCRIPT-DEFINES adjacent to $(SCRIPT_DEFINES) When "GIT-SCRIPT-DEFINES" was added in `e4dd89ab98` (Makefile: update scripts when build-time parameters change, 2012-06-20) the rules for generating the scripts themselves were moved further away from the "cmd_munge_script" added in `46bac90458` (Do not install shell libraries executable, 2010-01-31). Let's move these around so that the variables and defines needed by given targets immediately precede them. This is not needed for any subsequent changes to work, but makes the code consistent with how GIT-PERL-DEFINES is structured. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	d860c86ba5	Add "test-tool dump-reftable" command. This command dumps individual tables or a stack of of tables. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	e48d427268	reftable: implement stack, a mutable database of reftable files. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	acb533440f	reftable: implement refname validation The packed/loose format has restrictions on refnames: a and a/b cannot coexist. This limitation does not apply to reftable per se, but must be maintained for interoperability. This code adds validation routines to abort transactions that are trying to add invalid names. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	1ae2b8cda8	reftable: add merged table view This adds an abstract, read-only interface to the ref database. This primitive is used to construct the read view of the ref database (the read view is constructed by merging several *.ref files). It also provides the mechanism to provide a unified view of the refs in the main repository and the per-worktree refs. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	3b34f636df	reftable: add a heap-based priority queue for reftable records This is needed to create a merged view multiple reftables Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	ffc97f1a9e	reftable: reftable file level tests With support for reading and writing files in place, we can construct files (in memory) and attempt to read them back. Because some sections of the format are optional (eg. indices, log entries), we have to exercise this code using multiple sizes of input data Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	46bc0e731a	reftable: read reftable files This supports reading a single reftable file. The commit introduces an abstract iterator type, which captures the usecases both of reading individual refs, and iterating over a segment of the ref namespace. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	17df8dbeba	reftable: generic interface to tables Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	f14bd71934	reftable: write reftable files Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	35425d1034	reftable: a generic binary tree implementation The reftable format includes support for an (OID => ref) map. This map can speed up visibility and reachability checks. In particular, various operations along the fetch/push path within Gerrit have ben sped up by using this structure. The map is constructed with help of a binary tree. Object IDs are hashes, so they are uniformly distributed. Hence, the tree does not attempt forced rebalancing. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	e581fd7231	reftable: reading/writing blocks The reftable format is structured as a sequence of block. Within a block, records are prefix compressed, with an index of offsets for fully expand keys to enable binary search within blocks. This commit provides the logic to read and write these blocks. Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	a322920d0b	Provide zlib's uncompress2 from compat/zlib-compat.c This will be needed for reading reflog blocks in reftable. Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	e303bf22f9	reftable: (de)serialization for the polymorphic record type. The reftable format is structured as a sequence of blocks, and each block contains a sequence of prefix-compressed key-value records. There are 4 types of records, and they have similarities in how they must be handled. This is achieved by introducing a polymorphic 'record' type that encapsulates ref, log, index and object records. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	1214aa841b	reftable: add blocksource, an abstraction for random access reads The reftable format is usually used with files for storage. However, we abstract away this using the blocksource data structure. This has two advantages: * log blocks are zlib compressed, and handling them is simplified if we can discard byte segments from within the block layer. * for unittests, it is useful to read and write in-memory. The blocksource allows us to abstract the data away from on-disk files. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Han-Wen Nienhuys	ef8a6c6268	reftable: utility functions This commit provides basic utility classes for the reftable library. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	dd20e4a6db	Makefile: pass -Wno-pendantic under GENERATE_COMPILATION_DATABASE=yes The same bug fixed in the "COMPUTE_HEADER_DEPENDENCIES=auto" mode in the preceding commit was also present with "GENERATE_COMPILATION_DATABASE=yes". Let's fix it so it works again with "DEVOPTS=1". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	cfe853e66b	hook-list.h: add a generated list of hooks, like config-list.h Make githooks(5) the source of truth for what hooks git supports, and punt out early on hooks we don't know about in find_hook(). This ensures that the documentation and the C code's idea about existing hooks doesn't diverge. We still have Perl and Python code running its own hooks, but that'll be addressed by Emily Shaffer's upcoming "git hook run" command. This resolves a long-standing TODO item in bugreport.c of there being no centralized listing of hooks, and fixes a bug with the bugreport listing only knowing about 1/4 of the p4 hooks. It didn't know about the recent "reference-transaction" hook either. We could make the find_hook() function die() or BUG() out if the new known_hook() returned 0, but let's make it return NULL just as it does when it can't find a hook of a known type. Making it die() is overly anal, and unlikely to be what we need in catching stupid typos in the name of some new hook hardcoded in git.git's sources. By making this be tolerant of unknown hook names, changes in a later series to make "git hook run" run arbitrary user-configured hook names will be easier to implement. I have not been able to directly test the CMake change being made here. Since `4c2c38e800` (ci: modification of main.yml to use cmake for vs-build job, 2020-06-26) some of the Windows CI has a hard dependency on CMake, this change works there, and is to my eyes an obviously correct use of a pattern established in previous CMake changes, namely: - `061c2240b1` (Introduce CMake support for configuring Git, 2020-06-12) - `709df95b78` (help: move list_config_help to builtin/help, 2020-04-16) - `976aaedca0` (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29) The LC_ALL=C is needed because at least in my locale the dash ("-") is ignored for the purposes of sorting, which results in a different order. I'm not aware of anything in git that has a hard dependency on the order, but e.g. the bugreport output would end up using whatever locale was in effect when git was compiled. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	5e3aba33da	hook.[ch]: move find_hook() from run-command.c to hook.c Move the find_hook() function from run-command.c to a new hook.c library. This change establishes a stub library that's pretty pointless right now, but will see much wider use with Emily Shaffer's upcoming "configuration-based hooks" series. Eventually all the hook related code will live in hook.[ch]. Let's start that process by moving the simple find_hook() function over as-is. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	f53df0bdf6	Makefile: remove an out-of-date comment This comment added in `dfea575017` (Makefile: lazily compute header dependencies, 2010-01-26) has been out of date since `92b88eba9f` (Makefile: use `git ls-files` to list header files, if possible, 2019-03-04), when we did exactly what it tells us not to do and added $(GENERATED_H) to $(OBJECTS) dependencies. The rest of it was also somewhere between inaccurate and outdated, since as of `b8ba629264` (Makefile: fold MISC_H into LIB_H, 2012-06-20) it's not followed by a list of header files, that got moved earlier in the file into LIB_H in `60d24dd255` (Makefile: fold XDIFF_H and VCSSVN_H into LIB_H, 2012-07-06). Let's just remove it entirely, to the extent that we have anything useful to say here the comment on the "USE_COMPUTED_HEADER_DEPENDENCIES" variable a few lines above this change does the job for us. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago
Ævar Arnfjörð Bjarmason	7c81295382	Makefile: don't perform "mv $@+ $@" dance for $(GENERATED_H) Change the "cmd.sh > $@+ && mv $@+ $@" pattern used for generating the config-list.h and command-list.h to just "cmd.sh >$@". This was needed as a guard to ensure that we don't have an empty file if the script failed, but since `7b76d6bf22` (Makefile: add and use the ".DELETE_ON_ERROR" flag, 2021-06-29) GNU make ensures that doesn't happen. There's still a lot of other places in the Makefile where we needlessly use this pattern, but I'm just changing these because I'm about to add a new $(GENERATED_H) target, let's have them all look and act the same way. Even with ".DELETE_ON_ERROR" there is still a point to using the "mv $@+ $@" pattern in some cases, e.g. to ensure that you have a working binary during recompilation (see [1] for the start of a long discussion about that), but that doesn't apply here. Nothing external uses $(GENERATED_H) directly, it's only ever used in the context of the Makefile's own dependency (re-)generation. 1. https://lore.kernel.org/git/8735t93h0u.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>	3 years ago

1 2 3 4 5 ...

2782 Commits (2088a0c0cd61ab6c98064e68619e1d931a4619e2)