git/contrib
Chungmin Lee cbbcd33957 regexec: work around macOS TRE leak on invalid UTF-8
On macOS, the system regex engine leaks an internal buffer when
regexec() encounters an invalid multibyte sequence in a UTF-8 locale.
The line-by-line path can call regexec_buf() for each pattern on every
line, so "git grep" can leak repeatedly on a file containing invalid
UTF-8.  The total leak grows with the number of calls, and the per-call
allocation grows with the pattern's automaton.  In one case, grepping a
repository containing PDFs exhausted memory and caused the machine to
restart.

ce025ae4f6 (grep: disable lookahead on error, 2024-10-20) made "git
grep" fall back to line-by-line matching when regexec() reports an error
on invalid UTF-8.  That fallback cannot prevent this leak: the allocation
has already leaked when regexec() returns REG_ILLSEQ.

Avoid the leaking path by providing a Darwin-specific regexec_buf().
Walk the input with mbrtowc(), split it at bytes that cannot form a
complete multibyte character, and search each valid segment separately.
This preserves matches in valid text on either side of an invalid byte.

Search each segment with REG_STARTEND so match offsets remain relative to
the original buffer.  Set REG_NOTBOL and REG_NOTEOL for internal segment
boundaries so "^" and "$" do not match there.  Keep the flags clear at
the true beginning and end of the buffer.

Use the normal regexec_buf() path in single-byte locales, where no byte
can form an invalid multibyte sequence.  Use the bundled regex
implementation unchanged when NO_REGEX is enabled.

Declare the Darwin override in compat/darwin.h and map regexec_buf() to
darwin_regexec_buf().  This follows the platform override pattern used
by the other compatibility headers and leaves the common inline
implementation as the default.

There is no reliable way to detect a future macOS version in which the
system regex implementation has been fixed.  Even after a fix, Git will
need the workaround while it supports affected macOS releases, so treat
it as an indefinite compatibility workaround.

Add tests for matches before, after, and between invalid bytes, including
an offset check after an invalid byte.  Also check incomplete trailing
input and anchors at true and internal line boundaries.

Signed-off-by: Chungmin Lee <chungmin@chungminlee.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-28 15:16:37 -07:00
..
buildsystems regexec: work around macOS TRE leak on invalid UTF-8 2026-07-28 15:16:37 -07:00
completion bash-completions: add --max-count-oldest 2026-06-11 11:54:40 -07:00
contacts meson: make GIT_HTML_PATH configurable 2025-11-06 09:58:56 -08:00
credential osxkeychain: define build targets in the top-level Makefile. 2026-02-20 08:10:59 -08:00
diff-highlight diff-highlight: fetch all config with one process 2026-03-23 07:42:27 -07:00
fast-import
git-jump git-jump: pick a mode automatically when invoked without arguments 2026-05-21 23:01:04 +09:00
git-shell-commands
libgit-rs libgit: add higher-level libgit crate 2025-01-29 15:06:50 -08:00
libgit-sys libgit: add higher-level libgit crate 2025-01-29 15:06:50 -08:00
long-running-filter contrib/long-running-filter: *.txt -> *.adoc fixes 2025-03-03 13:49:22 -08:00
stats contrib: remove some scripts in "stats" directory 2025-05-12 10:55:47 -07:00
subtree docs: fix typos and grammar 2026-05-30 07:07:27 +09:00
vscode mingw: drop the build-system plumbing for nedmalloc 2026-05-09 11:19:23 +09:00
Makefile test: optionally test contrib in CI 2026-02-05 09:26:18 -08:00
README doc: fix some typos, grammar and wording issues 2023-10-05 12:55:38 -07:00
meson.build contrib: move "coccinelle/" directory into "tools/" 2026-03-19 06:40:08 -07:00
rerere-train.sh contrib/rerere-train: avoid useless gpg sign in training 2022-07-19 11:24:08 -07:00

README

Contributed Software

Although these pieces are available as part of the official git
source tree, they are in somewhat different status.  The
intention is to keep interesting tools around git here, maybe
even experimental ones, to give users an easier access to them,
and to give tools wider exposure, so that they can be improved
faster.

I am not expecting to touch these myself that much.  As far as
my day-to-day operation is concerned, these subdirectories are
owned by their respective primary authors.  I am willing to help
if users of these components and the contrib/ subtree "owners"
have technical/design issues to resolve, but the initiative to
fix and/or enhance things _must_ be on the side of the subtree
owners.  IOW, I won't be actively looking for bugs and rooms for
enhancements in them as the git maintainer -- I may only do so
just as one of the users when I want to scratch my own itch.  If
you have patches to things in contrib/ area, the patch should be
first sent to the primary author, and then the primary author
should ack and forward it to me (git pull request is nicer).
This is the same way as how I have been treating gitk, and to a
lesser degree various foreign SCM interfaces, so you know the
drill.

I expect things that start their life in the contrib/ area
to graduate out of contrib/ once they mature, either by becoming
projects on their own, or moving to the toplevel directory.  On
the other hand, I expect I'll be proposing removal of disused
and inactive ones from time to time.

If you have new things to add to this area, please first propose
it on the git mailing list, and after a list discussion proves
there is general interest (it does not have to be a
list-wide consensus for a tool targeted to a relatively narrow
audience -- for example I do not work with projects whose
upstream is svn, so I have no use for git-svn myself, but it is
of general interest for people who need to interoperate with SVN
repositories in a way git-svn works better than git-svnimport),
submit a patch to create a subdirectory of contrib/ and put your
stuff there.

-jc