linux/arch/x86/mm
Rik van Riel 2ff6ddf19c x86/mm/tlb: Leave lazy TLB mode at page table free time
Andy discovered that speculative memory accesses while in lazy
TLB mode can crash a system, when a CPU tries to dereference a
speculative access using memory contents that used to be valid
page table memory, but have since been reused for something else
and point into la-la land.

The latter problem can be prevented in two ways. The first is to
always send a TLB shootdown IPI to CPUs in lazy TLB mode, while
the second one is to only send the TLB shootdown at page table
freeing time.

The second should result in fewer IPIs, since operationgs like
mprotect and madvise are very common with some workloads, but
do not involve page table freeing. Also, on munmap, batching
of page table freeing covers much larger ranges of virtual
memory than the batching of unmapped user pages.

Tested-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Rik van Riel <riel@surriel.com>
Acked-by: Dave Hansen <dave.hansen@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: efault@gmx.de
Cc: kernel-team@fb.com
Cc: luto@kernel.org
Link: http://lkml.kernel.org/r/20180716190337.26133-3-riel@surriel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-07-17 09:35:31 +02:00
..
amdtopology.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cpu_entry_area.c x86/pti: Enable global pages for shared areas 2018-04-12 09:05:59 +02:00
debug_pagetables.c x86/efi: Replace efi_pgd with efi_mm.pgd 2018-03-12 11:05:05 +01:00
dump_pagetables.c x86/mm: Stop pretending pgtable_l5_enabled is a variable 2018-05-19 11:56:57 +02:00
extable.c x86/extable: Mark exception handler functions visible 2018-01-14 20:04:16 +01:00
fault.c x86/mm: Clean up the printk()s in show_fault_oops() 2018-06-27 14:08:11 +02:00
highmem_32.c x86/mm: Audit and remove any unnecessary uses of module.h 2016-07-14 13:04:20 +02:00
hugetlbpage.c x86/mm: Prevent non-MAP_FIXED mapping across DEFAULT_MAP_WINDOW border 2017-11-16 11:43:11 +01:00
ident_map.c x86/mm: Stop pretending pgtable_l5_enabled is a variable 2018-05-19 11:56:57 +02:00
init_32.c treewide: use PHYS_ADDR_MAX to avoid type casting ULLONG_MAX 2018-06-15 07:55:25 +09:00
init_64.c Merge branch 'linus' into x86/urgent 2018-06-22 21:20:35 +02:00
init.c mm: fix devmem_is_allowed() for sub-page System RAM intersections 2018-06-15 07:55:23 +09:00
iomap_32.c x86/mm: Do not auto-massage page protections 2018-04-12 09:04:22 +02:00
ioremap.c x86/mm: Do not auto-massage page protections 2018-04-12 09:04:22 +02:00
kasan_init_64.c x86/mm: Stop pretending pgtable_l5_enabled is a variable 2018-05-19 11:56:57 +02:00
kaslr.c x86/mm: Stop pretending pgtable_l5_enabled is a variable 2018-05-19 11:56:57 +02:00
kmmio.c x86/mm: Rename flush_tlb_single() and flush_tlb_one() to __flush_tlb_one_[user|kernel]() 2018-02-15 01:15:52 +01:00
Makefile x86/mm/sme: Disable stack protection for mem_encrypt_identity.c 2018-02-28 15:24:12 +01:00
mem_encrypt_boot.S x86/mm/sme, objtool: Annotate indirect call in sme_encrypt_execute() 2018-02-21 09:05:04 +01:00
mem_encrypt_identity.c x86/mm: Decouple dynamic __PHYSICAL_MASK from AMD SME 2018-06-06 13:38:01 +02:00
mem_encrypt.c dma/direct: Handle force decryption for DMA coherent buffers in common code 2018-03-20 10:01:59 +01:00
mm_internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mmap.c exec: pass stack rlimit into mm layout functions 2018-04-11 10:28:37 -07:00
mmio-mod.c x86: do not use print_symbol() 2018-01-05 15:23:01 +01:00
mpx.c Merge branch 'x86/mpx' into x86/asm, to pick up dependent commits 2017-11-08 10:55:48 +01:00
numa_32.c x86/mm/32: Remove unused node_memmap_size_bytes() & CONFIG_NEED_NODE_MEMMAP_SIZE logic 2018-03-27 08:45:02 +02:00
numa_64.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
numa_emulation.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
numa_internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
numa.c x86: Remove pr_fmt duplicate logging prefixes 2018-05-13 21:25:18 +02:00
pageattr-test.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pageattr.c x86/pti: Fix boot warning from Global-bit setting 2018-04-25 11:02:50 +02:00
pat_internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pat_rbtree.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pat.c x86/mm: add a function to check if a pfn is UC/UC-/WC 2018-01-16 16:40:09 +01:00
pf_in.c x86/mm: Audit and remove any unnecessary uses of module.h 2016-07-14 13:04:20 +02:00
pf_in.h
pgtable_32.c x86/mm: Rename flush_tlb_single() and flush_tlb_one() to __flush_tlb_one_[user|kernel]() 2018-02-15 01:15:52 +01:00
pgtable.c x86/mm: Add TLB purge to free pmd/pte page interfaces 2018-07-04 21:37:09 +02:00
physaddr.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
physaddr.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pkeys.c x86/pkeys: Override pkey when moving away from PROT_EXEC 2018-05-14 11:14:45 +02:00
pti.c x86/pti: Disallow global kernel text with RANDSTRUCT 2018-04-25 11:02:51 +02:00
setup_nx.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
srat.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
testmmiotrace.c Annotate hardware config module parameters in arch/x86/mm/ 2017-04-04 16:54:21 +01:00
tlb.c x86/mm/tlb: Leave lazy TLB mode at page table free time 2018-07-17 09:35:31 +02:00