diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2023-04-17 00:06:58 +0300 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2023-04-19 03:05:28 +0300 |
| commit | 8c9b6a88b7e2f33c656cd667a081bfd4dc8f5005 (patch) | |
| tree | 4472b64ceefb67bc6d4115d1af4bc7ebb7201741 /arch/x86/include/asm/uaccess_64.h | |
| parent | 577e6a7fd50d519c201d20968b6a027a6563dc4c (diff) | |
| download | linux-8c9b6a88b7e2f33c656cd667a081bfd4dc8f5005.tar.xz | |
x86: improve on the non-rep 'clear_user' function
The old version was oddly written to have the repeat count in multiple
registers. So instead of taking advantage of %rax being zero, it had
some sub-counts in it. All just for a "single word clearing" loop,
which isn't even efficient to begin with.
So get rid of those games, and just keep all the state in the same
registers we got it in (and that we should return things in). That not
only makes this act much more like 'rep stos' (which this function is
replacing), but makes it much easier to actually do the obvious loop
unrolling.
Also rename the function from the now nonsensical 'clear_user_original'
to what it now clearly is: 'rep_stos_alternative'.
End result: if we don't have a fast 'rep stosb', at least we can have a
fast fallback for it.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'arch/x86/include/asm/uaccess_64.h')
| -rw-r--r-- | arch/x86/include/asm/uaccess_64.h | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index 8cc918acbabc..a0533e672496 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -83,7 +83,7 @@ __copy_from_user_flushcache(void *dst, const void __user *src, unsigned size) */ __must_check unsigned long -clear_user_original(void __user *addr, unsigned long len); +rep_stos_alternative(void __user *addr, unsigned long len); static __always_inline __must_check unsigned long __clear_user(void __user *addr, unsigned long size) { @@ -97,7 +97,7 @@ static __always_inline __must_check unsigned long __clear_user(void __user *addr asm volatile( "1:\n\t" ALTERNATIVE("rep stosb", - "call clear_user_original", ALT_NOT(X86_FEATURE_FSRS)) + "call rep_stos_alternative", ALT_NOT(X86_FEATURE_FSRS)) "2:\n" _ASM_EXTABLE_UA(1b, 2b) : "+c" (size), "+D" (addr), ASM_CALL_CONSTRAINT |