linux/arch/arm64
Ryan Roberts 6b1e4efb6f arm64/mm: implement new [get_and_]clear_full_ptes() batch APIs
Optimize the contpte implementation to fix some of the
exit/munmap/dontneed performance regression introduced by the initial
contpte commit.  Subsequent patches will solve it entirely.

During exit(), munmap() or madvise(MADV_DONTNEED), mappings must be
cleared.  Previously this was done 1 PTE at a time.  But the core-mm
supports batched clear via the new [get_and_]clear_full_ptes() APIs.  So
let's implement those APIs and for fully covered contpte mappings, we no
longer need to unfold the contpte.  This significantly reduces unfolding
operations, reducing the number of tlbis that must be issued.

Link: https://lkml.kernel.org/r/20240215103205.2607016-15-ryan.roberts@arm.com
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Tested-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: Barry Song <21cnbao@gmail.com>
Cc: Borislav Petkov (AMD) <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: James Morse <james.morse@arm.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will@kernel.org>
Cc: Yang Shi <shy828301@gmail.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-02-22 15:27:18 -08:00
..
boot Samsung fixes for v6.8 2024-01-25 18:23:10 +01:00
configs arm64: defconfig reorder config lines 2024-01-11 09:34:42 +01:00
crypto crypto: arm64/sm4 - Remove cfb(sm4) 2023-12-08 11:59:45 +08:00
hyperv
include arm64/mm: implement new [get_and_]clear_full_ptes() batch APIs 2024-02-22 15:27:18 -08:00
kernel arm64/mm: new ptep layer to manage contig bit 2024-02-22 15:27:18 -08:00
kvm arm64/mm: new ptep layer to manage contig bit 2024-02-22 15:27:18 -08:00
lib arm64: Get rid of ARM64_HAS_NO_HW_PREFETCH 2023-12-05 12:02:52 +00:00
mm arm64/mm: implement new [get_and_]clear_full_ptes() batch APIs 2024-02-22 15:27:18 -08:00
net bpf: Use arch_bpf_trampoline_size 2023-12-06 17:17:20 -08:00
tools arm64: Rename ARM64_WORKAROUND_2966298 2024-01-12 12:51:33 +00:00
xen
Kbuild
Kconfig arm64/mm: wire up PTE_CONT for user mappings 2024-02-22 15:27:18 -08:00
Kconfig.debug
Kconfig.platforms arm64: Add config for AMD Pensando SoC platforms 2023-09-28 09:45:23 +02:00
Makefile arm64: vdso32: Remove unused vdso32-offsets.h 2024-01-30 11:59:17 +00:00