linux/arch/arm64/include/asm
James Morse 1401bef703 arm64: mm: Always update TCR_EL1 from __cpu_set_tcr_t0sz()
Because only the idmap sets a non-standard T0SZ, __cpu_set_tcr_t0sz()
can check for platforms that need to do this using
__cpu_uses_extended_idmap() before doing its work.

The idmap is only built with enough levels, (and T0SZ bits) to map
its single page.

To allow hibernate, and then kexec to idmap their single page copy
routines, __cpu_set_tcr_t0sz() needs to consider additional users,
who may need a different number of levels/T0SZ-bits to the idmap.
(i.e. VA_BITS may be enough for the idmap, but not hibernate/kexec)

Always read TCR_EL1, and check whether any work needs doing for
this request. __cpu_uses_extended_idmap() remains as it is used
by KVM, whose idmap is also part of the kernel image.

This mostly affects the cpuidle path, where we now get an extra
system register read .

CC: Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
CC: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Pavel Tatashin <pasha.tatashin@soleen.com>
Link: https://lore.kernel.org/r/20210125191923.1060122-8-pasha.tatashin@soleen.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-01-27 15:41:12 +00:00
..
vdso vdso/treewide: Add vdso_data pointer argument to __arch_get_hw_counter() 2020-08-06 10:57:30 +02:00
xen xen/arm: do not setup the runstate info page if kpti is enabled 2020-10-04 18:41:33 -05:00
acenv.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
acpi.h arm64/acpi: disallow AML memory opregions to access kernel memory 2020-07-14 18:02:03 +01:00
alternative-macros.h arm64: alternatives: Split up alternative.h 2020-11-09 21:49:34 +00:00
alternative.h arm64: alternatives: Split up alternative.h 2020-11-09 21:49:34 +00:00
arch_gicv3.h irqchip/gic-v3: Support pseudo-NMIs when SCR_EL3.FIQ == 0 2020-09-13 17:52:04 +01:00
arch_timer.h arm64: arch_timer: Allow an workaround descriptor to disable compat vdso 2020-07-08 21:57:51 +01:00
archrandom.h arm64: random: Remove no longer needed prototypes 2020-10-07 14:17:20 +01:00
arm_dsu_pmu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
arm-cci.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
asm_pointer_auth.h arm64: simplify ptrauth initialization 2020-04-28 11:23:21 +01:00
asm-bug.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
asm-offsets.h
asm-prototypes.h arm64: asm-prototypes: Fix fat-fingered typo in comment 2019-01-10 11:11:46 +00:00
asm-uaccess.h Merge branch 'for-next/misc' into for-next/core 2020-12-09 18:04:48 +00:00
assembler.h kasan, arm64: expand CONFIG_KASAN checks 2020-12-22 12:55:08 -08:00
atomic_ll_sc.h arm64: Move the LSE gas support detection to Kconfig 2020-01-15 12:50:48 +00:00
atomic_lse.h arm64: lse: fix LSE atomics with LLVM's integrated assembler 2020-01-16 17:25:10 +00:00
atomic.h arm64: make atomic helpers __always_inline 2021-01-13 15:09:06 +00:00
barrier.h iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer 2020-09-28 22:57:43 +01:00
bitops.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
bitrev.h
boot.h arm64: get rid of TEXT_OFFSET 2020-09-07 15:00:52 +01:00
brk-imm.h arm64: kprobes: Use BRK instead of single-step when executing instructions out-of-line 2020-11-03 14:03:38 +00:00
bug.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
cache.h arm64: kasan: align allocations for HW_TAGS 2020-12-22 12:55:08 -08:00
cacheflush.h arch, mm: make kernel_page_present() always available 2020-12-15 12:13:43 -08:00
checksum.h arm64: csum: Fix handling of bad packets 2020-07-30 17:01:38 +01:00
clocksource.h arm64: Introduce asm/vdso/clocksource.h 2020-03-21 15:23:55 +01:00
cmpxchg.h arm64: fix unreachable code issue with cmpxchg 2019-09-17 12:11:50 +01:00
compat.h compat: lift compat_s64 and compat_u64 to <asm-generic/compat.h> 2020-09-17 13:00:46 -04:00
compiler.h arm64: use a common .arch preamble for inline assembly 2020-08-28 11:15:15 +01:00
cpu_ops.h arm64: fix some spelling mistakes in the comments by codespell 2020-09-07 14:18:50 +01:00
cpu.h Revert "arm64: initialize per-cpu offsets earlier" 2020-10-09 11:24:17 +01:00
cpucaps.h Merge remote-tracking branch 'origin/kvm-arm64/psci-relay' into kvmarm-master/next 2020-12-09 10:00:24 +00:00
cpufeature.h ARM: 2020-12-20 10:44:05 -08:00
cpuidle.h
cputype.h arm64: Add MIDR value for KRYO2XX gold/silver CPU cores 2020-11-13 09:47:08 +00:00
current.h
daifflags.h arm64: entry: fix non-NMI kernel<->kernel transitions 2020-11-30 12:11:38 +00:00
dcc.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 284 2019-06-05 17:36:37 +02:00
debug-monitors.h arm64: kprobes: Use BRK instead of single-step when executing instructions out-of-line 2020-11-03 14:03:38 +00:00
device.h arm64: Remove dev->archdata.iommu pointer 2020-06-30 11:59:49 +02:00
dmi.h
efi.h efi: stub: get rid of efi_get_max_fdt_addr() 2020-12-09 08:37:27 +01:00
el2_setup.h arm64: Extract parts of el2_setup into a macro 2020-12-04 10:07:12 +00:00
elf.h Split the old READ_IMPLIES_EXEC workaround from executable PT_GNU_STACK 2020-06-05 13:45:21 -07:00
esr.h arm64: mte: add in-kernel MTE helpers 2020-12-22 12:55:07 -08:00
exception.h arm64: entry: suppress W=1 prototype warnings 2020-12-15 12:38:02 +00:00
exec.h arm64: uaccess: remove set_fs() 2020-12-02 19:49:11 +00:00
extable.h arm64: Improve diagnostics when trapping BRK with FAULT_BRK_IMM 2020-09-18 16:35:54 +01:00
fb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
fixmap.h arm64/fixmap: make notes of fixed_addresses more precisely 2020-08-08 19:25:04 +01:00
fpsimd.h arm64/sve: Implement a helper to load SVE registers from FPSIMD state 2020-09-21 18:06:33 +01:00
fpsimdmacros.h arm64/sve: Implement a helper to flush SVE registers 2020-09-21 18:06:33 +01:00
ftrace.h arm64: implement ftrace with regs 2019-11-06 14:17:35 +00:00
futex.h arm64: uaccess: rename privileged uaccess routines 2020-12-02 19:49:10 +00:00
hardirq.h arm64: irqstat: Get rid of duplicated declaration 2020-11-23 10:31:05 +01:00
hugetlb.h arm64/hugetlb: Reserve CMA areas for gigantic pages on 16K and 64K configs 2020-07-15 13:38:03 +01:00
hw_breakpoint.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
hwcap.h Merge branch 'for-next/mte' into for-next/core 2020-10-02 12:16:11 +01:00
hyp_image.h kvm: arm64: Move nVHE hyp namespace macros to hyp_image.h 2020-09-30 08:33:52 +01:00
hypervisor.h
image.h docs: arm64: convert docs to ReST and rename to .rst 2019-06-14 14:20:27 -06:00
insn.h arm64: alternatives: Split up alternative.h 2020-11-09 21:49:34 +00:00
io.h Add and use a generic version of devmem_is_allowed() 2020-12-11 12:30:26 -08:00
irq_work.h arm64: Kill __smp_cross_call and co 2020-09-17 16:37:28 +01:00
irq.h
irqflags.h arm64: Implement arch_irqs_disabled() 2020-08-26 12:41:55 +02:00
jump_label.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kasan.h kasan, arm64: move initialization message 2020-12-22 12:55:07 -08:00
Kbuild local64.h: make <asm/local64.h> mandatory 2020-12-29 15:36:49 -08:00
kernel-pgtable.h arm64: consistently use reserved_pg_dir 2020-11-10 17:30:40 +00:00
kexec.h arm64: kexec: make dtb_mem always enabled 2021-01-27 15:41:11 +00:00
kgdb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kprobes.h arm64: kprobes: Remove redundant kprobe_step_ctx 2020-11-10 17:12:30 +00:00
kvm_arm.h KVM: arm64: Trap host SMCs in protected mode 2020-12-04 10:08:36 +00:00
kvm_asm.h ARM: 2020-12-20 10:44:05 -08:00
kvm_emulate.h Merge branch 'kvm-master' into kvm-next 2021-01-07 18:06:52 -05:00
kvm_host.h KVM: arm64: Declutter host PSCI 0.1 handling 2020-12-22 12:56:44 +00:00
kvm_hyp.h KVM: arm64: Bootstrap PSCI SMC handler in nVHE EL2 2020-12-04 10:08:34 +00:00
kvm_mmu.h Merge remote-tracking branch 'origin/kvm-arm64/psci-relay' into kvmarm-master/next 2020-12-09 10:00:24 +00:00
kvm_pgtable.h KVM: arm64: Add support for relaxing stage-2 perms in generic page-table code 2020-09-11 15:51:15 +01:00
kvm_ptrauth.h KVM: arm64: Update context references from host to hyp 2020-09-15 18:39:02 +01:00
kvm_ras.h arm64: KVM/mm: Move SEA handling behind a single 'claim' interface 2019-02-07 23:10:45 +01:00
kvm_types.h KVM: arm64: Use common KVM implementation of MMU memory caches 2020-07-09 13:29:43 -04:00
linkage.h arm64: Don't insert a BTI instruction at inner labels 2020-06-24 14:24:29 +01:00
lse.h arm64: lse: Fix LSE atomics with LLVM 2020-02-18 18:10:49 +00:00
memory.h kasan, arm64: implement HW_TAGS runtime 2020-12-22 12:55:08 -08:00
mman.h arm64: mte: Validate the PROT_MTE request via arch_validate_flags() 2020-09-04 12:46:07 +01:00
mmu_context.h arm64: mm: Always update TCR_EL1 from __cpu_set_tcr_t0sz() 2021-01-27 15:41:12 +00:00
mmu.h KVM: arm64: Move BP hardening helpers into spectre.h 2020-11-16 10:40:18 +00:00
mmzone.h
module.h arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h> 2020-04-23 10:50:26 +09:00
module.lds.h kbuild: preprocess module linker script 2020-09-25 00:36:41 +09:00
mte-def.h arm64: mte: switch GCR_EL1 in kernel entry and exit 2020-12-22 12:55:07 -08:00
mte-kasan.h arm64: mte: switch GCR_EL1 in kernel entry and exit 2020-12-22 12:55:07 -08:00
mte.h arm64: mte: switch GCR_EL1 in kernel entry and exit 2020-12-22 12:55:07 -08:00
neon-intrinsics.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
neon.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
numa.h arm64/mm: return cpu_all_mask when node is NUMA_NO_NODE 2020-09-21 22:05:01 +01:00
page-def.h arm64/mm: Unify CONT_PTE_SHIFT 2020-09-11 16:33:43 +01:00
page.h arm64: mte: Add PROT_MTE support to mmap() and mprotect() 2020-09-04 12:46:07 +01:00
paravirt.h arm64: Retrieve stolen time as paravirtualized guest 2019-10-21 19:20:31 +01:00
pci.h arm64: Enable PCI write-combine resources under sysfs 2020-09-18 14:05:13 +01:00
percpu.h KVM: arm64: Support per_cpu_ptr in nVHE hyp code 2020-12-04 10:08:34 +00:00
perf_event.h arm64: perf: Add support caps under sysfs 2020-09-28 14:53:45 +01:00
pgalloc.h asm-generic: pgalloc: provide generic pgd_free() 2020-08-07 11:33:26 -07:00
pgtable-hwdef.h kasan: arm64: set TCR_EL1.TBID1 when enabled 2020-11-25 16:53:19 +00:00
pgtable-prot.h ARM: 2020-10-23 11:17:56 -07:00
pgtable-types.h arm64: add support for folded p4d page tables 2020-06-04 19:06:21 -07:00
pgtable.h Merge branch 'akpm' (patches from Andrew) 2020-12-15 12:53:37 -08:00
pointer_auth.h arm64: sync kernel APIAKey when installing 2020-04-21 15:52:56 +01:00
preempt.h sched/rt, arm64: Use CONFIG_PREEMPTION 2019-12-08 14:37:32 +01:00
probes.h arm64/fpsimd: add <asm/insn.h> to <asm/kprobes.h> to fix fpsimd build 2020-11-23 10:59:00 +00:00
proc-fns.h arm64: mm: convert cpu_do_switch_mm() to C 2020-02-27 14:30:50 +00:00
processor.h arm64: Remove arm64_dma32_phys_limit and its uses 2021-01-12 17:49:25 +00:00
ptdump.h arm64: mm: convert mm/dump.c to use walk_page_range() 2020-02-04 03:05:25 +00:00
ptrace.h Merge branch 'kvm-master' into kvm-next 2021-01-07 18:06:52 -05:00
pvclock-abi.h KVM: arm64: Implement PV_TIME_FEATURES call 2019-10-21 19:20:27 +01:00
rwonce.h arm64: lto: Strengthen READ_ONCE() to acquire when CONFIG_LTO=y 2020-11-09 21:49:34 +00:00
scs.h scs: Move scs_overflow_check() out of architecture code 2020-05-18 17:47:40 +01:00
sdei.h arm64: Use the correct style for SPDX License Identifier 2019-04-16 16:28:01 +01:00
seccomp.h arm64: Enable seccomp architecture tracking 2020-11-20 11:16:34 -08:00
sections.h KVM: arm64: Add .hyp.data..ro_after_init ELF section 2020-12-04 10:08:33 +00:00
shmparam.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
signal32.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
signal.h arm64: expose FAR_EL1 tag bits in siginfo 2020-11-23 18:17:39 +00:00
simd.h arm64: fpsimd: Make sure SVE setup is complete before SIMD is used 2020-01-14 17:11:21 +00:00
smp_plat.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
smp.h arm64: Make cpu_logical_map() take unsigned int 2020-12-04 08:44:24 +00:00
sparsemem.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
spectre.h Merge remote-tracking branch 'origin/kvm-arm64/csv3' into kvmarm-master/queue 2020-12-03 19:12:24 +00:00
spinlock_types.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
spinlock.h arm64/spinlock: fix a -Wunused-function warning 2020-02-10 11:29:24 +00:00
stack_pointer.h
stackprotector.h arm64: initialize ptrauth keys for kernel booting task 2020-03-18 09:50:20 +00:00
stacktrace.h arm64: stacktrace: Make stack walk callback consistent with generic code 2020-09-18 14:24:16 +01:00
stage2_pgtable.h KVM: arm64: Remove unused page-table code 2020-09-11 15:51:15 +01:00
stat.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
string.h kasan, arm64: expand CONFIG_KASAN checks 2020-12-22 12:55:08 -08:00
suspend.h arm64: Preserve register x18 when CPU is suspended 2020-05-15 16:35:50 +01:00
sync_bitops.h arm64, locking/atomics: Use instrumented atomics 2018-11-01 11:01:40 +01:00
syscall_wrapper.h arm64: simplify syscall wrapper ifdeffery 2019-10-14 10:55:00 +01:00
syscall.h arm64: compat: Ensure upper 32 bits of x0 are zero on syscall return 2020-07-16 11:41:31 +01:00
sysreg.h Merge branch 'kvm-master' into kvm-next 2021-01-07 18:06:52 -05:00
system_misc.h arm64: expose FAR_EL1 tag bits in siginfo 2020-11-23 18:17:39 +00:00
thread_info.h arm64: make _TIF_WORK_MASK bits contiguous 2020-12-16 14:08:47 -08:00
timex.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
tlb.h arm64: tlb: Set the TTL field in flush_tlb_range 2020-07-07 11:23:47 +01:00
tlbflush.h arm64: use a common .arch preamble for inline assembly 2020-08-28 11:15:15 +01:00
topology.h arm64: wrap and generalise counter read functions 2020-11-13 20:05:10 +00:00
trans_pgd.h arm64: trans_pgd: pass allocator trans_pgd_create_copy 2021-01-27 15:41:12 +00:00
traps.h arm64: expose FAR_EL1 tag bits in siginfo 2020-11-23 18:17:39 +00:00
uaccess.h arm64: Move PSTATE.TCO setting to separate functions 2021-01-07 11:39:01 +00:00
unistd32.h epoll: fix compat syscall wire up of epoll_pwait2 2020-12-20 10:01:38 -08:00
unistd.h epoll: wire up syscall epoll_pwait2 2020-12-19 11:18:38 -08:00
uprobes.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
vdso.h arm64/vdso: Add time namespace page 2020-07-24 13:15:20 +01:00
vermagic.h arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h> 2020-04-23 10:50:26 +09:00
virt.h KVM: arm64: Fix EL2 mode availability checks 2020-12-04 10:08:36 +00:00
vmalloc.h mm/vmalloc: Add empty <asm/vmalloc.h> headers and use them from <linux/vmalloc.h> 2019-12-10 10:12:55 +01:00
vmap_stack.h mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
word-at-a-time.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
xor.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00