linux/arch/powerpc/mm
Nicholas Piggin d4748276ae powerpc/64s: Improve local TLB flush for boot and MCE on POWER9
There are several cases outside the normal address space management
where a CPU's entire local TLB is to be flushed:

  1. Booting the kernel, in case something has left stale entries in
     the TLB (e.g., kexec).

  2. Machine check, to clean corrupted TLB entries.

One other place where the TLB is flushed, is waking from deep idle
states. The flush is a side-effect of calling ->cpu_restore with the
intention of re-setting various SPRs. The flush itself is unnecessary
because in the first case, the TLB should not acquire new corrupted
TLB entries as part of sleep/wake (though they may be lost).

This type of TLB flush is coded inflexibly, several times for each CPU
type, and they have a number of problems with ISA v3.0B:

- The current radix mode of the MMU is not taken into account, it is
  always done as a hash flushn For IS=2 (LPID-matching flush from host)
  and IS=3 with HV=0 (guest kernel flush), tlbie(l) is undefined if
  the R field does not match the current radix mode.

- ISA v3.0B hash must flush the partition and process table caches as
  well.

- ISA v3.0B radix must flush partition and process scoped translations,
  partition and process table caches, and also the page walk cache.

So consolidate the flushing code and implement it in C and inline asm
under the mm/ directory with the rest of the flush code. Add ISA v3.0B
cases for radix and hash, and use the radix flush in radix environment.

Provide a way for IS=2 (LPID flush) to specify the radix mode of the
partition. Have KVM pass in the radix mode of the guest.

Take out the flushes from early cputable/dt_cpu_ftrs detection hooks,
and move it later in the boot process after, the MMU registers are set
up and before relocation is first turned on.

The TLB flush is no longer called when restoring from deep idle states.
This was not be done as a separate step because booting secondaries
uses the same cpu_restore as idle restore, which needs the TLB flush.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-18 00:40:31 +11:00
..
8xx_mmu.c powerpc/8xx: Use L1 entry APG to handle _PAGE_ACCESSED for CONFIG_SWAP 2018-01-16 23:47:15 +11:00
40x_mmu.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
44x_mmu.c powerpc: Delete __cpuinit usage from all users 2013-07-01 11:10:36 +10:00
copro_fault.c powerpc/mm: Update PROTFAULT handling in the page fault path 2017-02-15 20:02:39 +11:00
dma-noncoherent.c powerpc/mm: Rename map_page() to map_kernel_page() on 32-bit 2017-06-05 19:59:03 +10:00
drmem.c powerpc/drmem: Add support for ibm, dynamic-memory-v2 property 2018-01-16 23:26:29 +11:00
dump_hashpagetable.c powerpc/64s: Replace CONFIG_PPC_STD_MMU_64 with CONFIG_PPC_BOOK3S_64 2017-11-06 16:48:14 +11:00
dump_linuxpagetables.c powerpc/mm: Introduce _PAGE_NA 2018-01-16 23:47:14 +11:00
fault.c powerpc: Use the TRAP macro whenever comparing a trap number 2018-01-16 23:51:43 +11:00
fsl_booke_mmu.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
hash64_4k.c powerpc: use helper functions to get and set hash slots 2017-12-20 18:57:25 +11:00
hash64_64k.c powerpc: Free up four 64K PTE bits in 64K backed HPTE pages 2017-12-20 18:57:22 +11:00
hash_low_32.S powerpc: fix location of two EXPORT_SYMBOL 2017-09-01 16:42:45 +10:00
hash_native_64.c powerpc/64s: Improve local TLB flush for boot and MCE on POWER9 2018-01-18 00:40:31 +11:00
hash_utils_64.c powerpc/64s: Improve local TLB flush for boot and MCE on POWER9 2018-01-18 00:40:31 +11:00
highmem.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugepage-hash64.c powerpc/mm: Move hash table ops to a separate structure 2016-07-21 18:59:09 +10:00
hugetlbpage-book3e.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugetlbpage-hash64.c powerpc: Free up four 64K PTE bits in 64K backed HPTE pages 2017-12-20 18:57:22 +11:00
hugetlbpage-radix.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
hugetlbpage.c powerpc/8xx: Remove _PAGE_USER and handle user access at PMD level 2018-01-16 23:47:14 +11:00
init_32.c powerpc/mm/hugetlb: Add support for reserving gigantic huge pages via kernel command line 2017-08-16 14:56:12 +10:00
init_64.c powerpc: Use pr_warn instead of pr_warning 2017-12-04 11:54:34 +11:00
init-common.c Merge branch 'topic/ppc-kvm' into next 2017-02-14 17:18:29 +11:00
Makefile powerpc/mm: Separate ibm, dynamic-memory data from DT format 2018-01-16 23:26:27 +11:00
mem.c powerpc: Use pr_warn instead of pr_warning 2017-12-04 11:54:34 +11:00
mmap.c powerpc/64s: mm_context.addr_limit is only used on hash 2017-11-13 23:35:43 +11:00
mmu_context_book3s64.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
mmu_context_hash32.c powerpc: Remove power3 from comments 2014-07-28 14:10:26 +10:00
mmu_context_iommu.c Merge branch 'topic/ppc-kvm' into next 2017-04-28 20:19:37 +10:00
mmu_context_nohash.c powerpc/mm/nohash: MM_SLICE is only used by book3s 64 2017-03-31 23:09:47 +11:00
mmu_context.c cxl: Enable global TLBIs for cxl contexts 2017-09-28 17:09:16 +10:00
mmu_decl.h powerpc/8xx: Getting rid of remaining use of CONFIG_8xx 2017-08-10 23:32:12 +10:00
numa.c powerpc/numa: Update numa code use walk_drmem_lmbs 2018-01-16 23:26:28 +11:00
pgtable_32.c powerpc/mm: extend _PAGE_PRIVILEGED to all CPUs 2018-01-16 23:47:13 +11:00
pgtable_64.c powerpc/mm: extend _PAGE_PRIVILEGED to all CPUs 2018-01-16 23:47:13 +11:00
pgtable-book3e.c powerpc/mm: Make page table size a variable 2016-05-01 18:32:48 +10:00
pgtable-book3s64.c powerpc/mm/cxl: Add the fault handling cpu to mm cpumask 2017-08-17 23:31:52 +10:00
pgtable-hash64.c powerpc/mm: Use mm_is_thread_local() instread of open-coding 2017-08-23 22:27:45 +10:00
pgtable-radix.c powerpc/64s: Improve local TLB flush for boot and MCE on POWER9 2018-01-18 00:40:31 +11:00
pgtable.c powerpc/mm: extend _PAGE_PRIVILEGED to all CPUs 2018-01-16 23:47:13 +11:00
ppc_mmu_32.c powerpc32: refactor x_mapped_by_bats() and x_mapped_by_tlbcam() together 2016-03-11 17:18:02 -06:00
slb_low.S powerpc/64s: mm_context.addr_limit is only used on hash 2017-11-13 23:35:43 +11:00
slb.c powerpc/64s: Rename slb_allocate_realmode() to slb_allocate() 2017-06-21 16:18:33 +10:00
slice.c powerpc/64s/slice: Use addr limit when computing slice mask 2017-11-20 19:28:25 +11:00
subpage-prot.c powerpc/mm/hash: Free the subpage_prot_table correctly 2017-07-27 13:05:50 +10:00
tlb_hash32.c powerpc/mm: remove flush_tlb_page_nohash 2016-08-01 11:15:13 +10:00
tlb_hash64.c powerpc/mm: Use mm_is_thread_local() instread of open-coding 2017-08-23 22:27:45 +10:00
tlb_low_64e.S powerpc: Fix misspellings in comments. 2016-03-01 19:27:20 +11:00
tlb_nohash_low.S powerpc/8xx: Getting rid of remaining use of CONFIG_8xx 2017-08-10 23:32:12 +10:00
tlb_nohash.c powerpc/nohash: Fix use of mmu_has_feature() in setup_initial_memory_limit() 2017-04-11 07:46:04 +10:00
tlb-radix.c powerpc/64s: Improve local TLB flush for boot and MCE on POWER9 2018-01-18 00:40:31 +11:00
vphn.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vphn.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00