You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
55 lines
3.0 KiB
55 lines
3.0 KiB
From 8b9a0af8ca012217bf90d1dc0694f85b49ae09da Mon Sep 17 00:00:00 2001 |
|
From: Noah Goldstein <goldstein.w.n@gmail.com> |
|
Date: Tue, 18 Jul 2023 10:27:59 -0500 |
|
Subject: [PATCH] [PATCH v1] x86: Use `3/4*sizeof(per-thread-L3)` as low bound |
|
for NT threshold. |
|
Content-type: text/plain; charset=UTF-8 |
|
|
|
On some machines we end up with incomplete cache information. This can |
|
make the new calculation of `sizeof(total-L3)/custom-divisor` end up |
|
lower than intended (and lower than the prior value). So reintroduce |
|
the old bound as a lower bound to avoid potentially regressing code |
|
where we don't have complete information to make the decision. |
|
Reviewed-by: DJ Delorie <dj@redhat.com> |
|
--- |
|
sysdeps/x86/dl-cacheinfo.h | 15 ++++++++++++--- |
|
1 file changed, 12 insertions(+), 3 deletions(-) |
|
|
|
[diff rebased by DJ] |
|
diff -rup a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h |
|
--- a/sysdeps/x86/dl-cacheinfo.h 2023-07-25 00:38:43.343986368 -0400 |
|
+++ b/sysdeps/x86/dl-cacheinfo.h 2023-07-25 00:38:44.336025100 -0400 |
|
@@ -751,8 +751,8 @@ dl_init_cacheinfo (struct cpu_features * |
|
|
|
/* The default setting for the non_temporal threshold is [1/8, 1/2] of size |
|
of the chip's cache (depending on `cachesize_non_temporal_divisor` which |
|
- is microarch specific. The defeault is 1/4). For most Intel and AMD |
|
- processors with an initial release date between 2017 and 2023, a thread's |
|
+ is microarch specific. The default is 1/4). For most Intel processors |
|
+ with an initial release date between 2017 and 2023, a thread's |
|
typical share of the cache is from 18-64MB. Using a reasonable size |
|
fraction of L3 is meant to estimate the point where non-temporal stores |
|
begin out-competing REP MOVSB. As well the point where the fact that |
|
@@ -763,12 +763,21 @@ dl_init_cacheinfo (struct cpu_features * |
|
the maximum thrashing capped at 1/associativity. */ |
|
unsigned long int non_temporal_threshold |
|
= shared / cachesize_non_temporal_divisor; |
|
+ |
|
+ /* If the computed non_temporal_threshold <= 3/4 * per-thread L3, we most |
|
+ likely have incorrect/incomplete cache info in which case, default to |
|
+ 3/4 * per-thread L3 to avoid regressions. */ |
|
+ unsigned long int non_temporal_threshold_lowbound |
|
+ = shared_per_thread * 3 / 4; |
|
+ if (non_temporal_threshold < non_temporal_threshold_lowbound) |
|
+ non_temporal_threshold = non_temporal_threshold_lowbound; |
|
+ |
|
/* If no ERMS, we use the per-thread L3 chunking. Normal cacheable stores run |
|
a higher risk of actually thrashing the cache as they don't have a HW LRU |
|
hint. As well, their performance in highly parallel situations is |
|
noticeably worse. */ |
|
if (!CPU_FEATURE_USABLE_P (cpu_features, ERMS)) |
|
- non_temporal_threshold = shared_per_thread * 3 / 4; |
|
+ non_temporal_threshold = non_temporal_threshold_lowbound; |
|
/* SIZE_MAX >> 4 because memmove-vec-unaligned-erms right-shifts the value of |
|
'x86_non_temporal_threshold' by `LOG_4X_MEMCPY_THRESH` (4) and it is best |
|
if that operation cannot overflow. Minimum of 0x4040 (16448) because the
|
|
|