linux/arch/arm/include/asm
Christoffer Dall 2d0e63e030 KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
We can finally get completely rid of any calls to the VGICv3
save/restore functions when the AP lists are empty on VHE systems.  This
requires carefully factoring out trap configuration from saving and
restoring state, and carefully choosing what to do on the VHE and
non-VHE path.

One of the challenges is that we cannot save/restore the VMCR lazily
because we can only write the VMCR when ICC_SRE_EL1.SRE is cleared when
emulating a GICv2-on-GICv3, since otherwise all Group-0 interrupts end
up being delivered as FIQ.

To solve this problem, and still provide fast performance in the fast
path of exiting a VM when no interrupts are pending (which also
optimized the latency for actually delivering virtual interrupts coming
from physical interrupts), we orchestrate a dance of only doing the
activate/deactivate traps in vgic load/put for VHE systems (which can
have ICC_SRE_EL1.SRE cleared when running in the host), and doing the
configuration on every round-trip on non-VHE systems.

Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2018-03-19 10:53:21 +00:00
..
hardware Merge branches 'fixes', 'misc', 'sa1111' and 'sa1100-for-next' into for-next 2018-01-21 15:38:10 +00:00
mach ARM/PCI: Convert PCI scan API to pci_scan_root_bus_bridge() 2017-06-28 15:13:55 -05:00
xen License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
arch_gicv3.h irqchip/gic-v3: Add support for Range Selector (RS) feature 2017-10-19 11:22:34 +01:00
arch_timer.h arm64 updates for 4.15 2017-11-15 10:56:56 -08:00
arm-cci.h
asm-offsets.h
assembler.h ARM: BUG if jumping to usermode address in kernel mode 2017-11-26 15:41:39 +00:00
atomic.h locking/atomic: Remove linux/atomic.h:atomic_fetch_or() 2016-06-16 10:48:32 +02:00
auxvec.h
barrier.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bitops.h lib: optimize cpumask_next_and() 2018-02-06 18:32:44 -08:00
bitrev.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bL_switcher.h
bug.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bugs.h
cache.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cacheflush.h randstruct: opt-out externally exposed function pointer structs 2017-06-30 12:00:52 -07:00
cachetype.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
checksum.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
clocksource.h arm/arm64: arch_timer: Use archdata to indicate vdso suitability 2016-09-23 17:19:25 +01:00
cmpxchg.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
compiler.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cp15.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cpu.h
cpufeature.h ARM: 8663/1: wire up HWCAP/HWCAP2 feature bits to the CPU modalias 2017-03-30 19:45:52 +01:00
cpuidle.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cputype.h Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm 2017-11-16 12:50:35 -08:00
cti.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dcc.h
delay.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
device.h ARM: dma-mapping: Don't tear down third-party mappings 2017-05-30 11:31:33 +02:00
div64.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dma-contiguous.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dma-direct.h dma-mapping: move swiotlb arch helpers to a new header 2018-01-10 16:40:54 +01:00
dma-iommu.h kmemcheck: remove annotations 2017-11-15 18:21:04 -08:00
dma-mapping.h dma-direct: rename dma_noop to dma_direct 2018-01-15 09:35:06 +01:00
dma.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dmi.h efi/arm: Enable DMI/SMBIOS 2017-06-05 17:50:44 +02:00
domain.h ARM: 8573/1: domain: move {set,get}_domain under config guard 2016-05-05 19:03:02 +01:00
ecard.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
edac.h
efi.h efi/arm-stub: Correct FDT and initrd allocation rules for arm64 2017-04-05 12:27:23 +02:00
elf.h Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm 2017-11-16 12:50:35 -08:00
entry-macro-multi.S License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
exception.h ARM: probes: avoid adding kprobes to sensitive kernel-entry/exit code 2017-12-17 22:14:21 +00:00
fb.h
fiq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
firmware.h
fixmap.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
flat.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
floppy.h ARM: 8584/1: floppy: avoid gcc-6 warning 2016-07-02 11:02:13 +01:00
fncpy.h
fpstate.h
ftrace.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
futex.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
glue-cache.h ARM: 8725/1: Add Broadcom Brahma-B15 readahead cache support 2017-12-17 22:15:35 +00:00
glue-df.h
glue-pf.h
glue-proc.h
glue.h
gpio.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hardirq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
highmem.h Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm 2017-11-16 12:50:35 -08:00
hugetlb-3level.h
hugetlb.h
hw_breakpoint.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hw_irq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hwcap.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hypervisor.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ide.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
idmap.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
insn.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
io.h ARM: Implement pci_remap_cfgspace() interface 2017-04-24 13:53:13 -05:00
irq_work.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irqflags.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
jump_label.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Kbuild arch: Remove clkdev.h asm-generic from Kbuild 2018-01-03 09:02:11 -08:00
kexec.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
kgdb.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
kmap_types.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
kprobes.h kprobes: move kprobe declarations to asm-generic/kprobes.h 2017-02-27 18:43:45 -08:00
kvm_arm.h arm: KVM: Fix VTTBR_BADDR_MASK BUG_ON off-by-one 2017-11-29 16:46:18 +01:00
kvm_asm.h KVM: arm64: Introduce VHE-specific kvm_vcpu_run 2018-03-19 10:53:13 +00:00
kvm_coproc.h KVM: arm: plug potential guest hardware debug leakage 2017-05-15 14:29:19 +02:00
kvm_emulate.h KVM: arm/arm64: Prepare to handle deferred save/restore of SPSR_EL1 2018-03-19 10:53:17 +00:00
kvm_host.h KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs 2018-03-19 10:53:11 +00:00
kvm_hyp.h KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs 2018-03-19 10:53:21 +00:00
kvm_mmio.h KVM: arm/arm64: Export mmio_read/write_bus 2016-05-20 15:39:42 +02:00
kvm_mmu.h KVM changes for 4.16 2018-02-10 13:16:35 -08:00
limits.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
linkage.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mc146818rtc.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mcpm.h
mcs_spinlock.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
memblock.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
memory.h ARM: 8739/1: NOMMU: Setup VBAR/Hivecs for secondaries cores 2018-01-21 15:32:23 +00:00
mmu_context.h sched/headers: Prepare to remove the <linux/mm_types.h> dependency from <linux/sched.h> 2017-03-02 08:42:37 +01:00
mmu.h Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm 2017-11-16 12:50:35 -08:00
module.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mpu.h Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm 2017-11-16 12:50:35 -08:00
mtd-xip.h
neon.h
nwflash.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
opcodes-sec.h
opcodes-virt.h
opcodes.h
outercache.h ARM: move heavy barrier support out of line 2015-07-25 15:28:05 +01:00
page-nommu.h ARM: 8684/1: NOMMU: Remove unused KTHREAD_SIZE definition 2017-06-27 14:58:10 +01:00
page.h
paravirt.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
patch.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pci.h PCI: Remove PCI_REASSIGN_ALL_RSRC use on arm and arm64 2017-12-18 23:07:43 -06:00
percpu.h
perf_event.h
pgalloc.h kmemcheck: stop using GFP_NOTRACK and SLAB_NOTRACK 2017-11-15 18:21:04 -08:00
pgtable-2level-hwdef.h ARM: 8612/1: LPAE: initialize cache policy correctly 2016-09-12 12:12:30 +01:00
pgtable-2level-types.h
pgtable-2level.h ARM: 8578/1: mm: ensure pmd_present only checks the valid bit 2016-06-09 17:51:47 +01:00
pgtable-3level-hwdef.h ARM: 8612/1: LPAE: initialize cache policy correctly 2016-09-12 12:12:30 +01:00
pgtable-3level-types.h
pgtable-3level.h arm/mm: provide pmdp_establish() helper 2018-01-31 17:18:37 -08:00
pgtable-hwdef.h
pgtable-nommu.h ARM: 8676/1: NOMMU: provide pgprot_device() macro 2017-06-05 10:29:40 +01:00
pgtable.h KVM: arm/arm64: Limit icache invalidation to prefetch aborts 2018-01-08 15:20:45 +01:00
probes.h
proc-fns.h ARM: soft-reboot into same mode that we entered the kernel 2017-04-09 07:49:24 -07:00
processor.h arm: Implement thread_struct whitelist for hardened usercopy 2018-01-15 12:08:06 -08:00
procinfo.h
prom.h
psci.h ARM: SoC cleanups for v4.5 2016-01-20 17:55:20 -08:00
ptdump.h ARM: 8737/1: mm: dump: add checking for writable and executable 2018-01-21 15:32:20 +00:00
ptrace.h linux/compiler.h: Split into compiler.h and compiler_types.h 2017-10-24 13:17:32 +02:00
sections.h ARM: probes: avoid adding kprobes to sensitive kernel-entry/exit code 2017-12-17 22:14:21 +00:00
set_memory.h treewide: move set_memory_* functions away from cacheflush.h 2017-05-08 17:15:13 -07:00
setup.h ARM: 8495/1: ATAGS: move save_atags() to arch/arm/include/asm/setup.h 2016-01-04 11:26:00 +00:00
shmparam.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
signal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
smp_plat.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
smp_scu.h ARM: SoC platform updates for 4.15 2017-11-16 14:05:12 -08:00
smp_twd.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
smp.h ARM: 8708/1: NOMMU: Rework MPU to be mostly done in C 2017-10-23 16:58:59 +01:00
sparsemem.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
spinlock_types.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
spinlock.h Merge branch 'linus' into locking/core, to resolve conflicts 2017-11-07 10:32:44 +01:00
stackprotector.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stacktrace.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stage2_pgtable.h kvm-arm: arm32: Introduce stage2 page table helpers 2016-04-21 14:57:26 +02:00
string.h ARM: 8745/1: get rid of __memzero() 2018-01-21 15:37:56 +00:00
suspend.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
swab.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
switch_to.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sync_bitops.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
syscall.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
system_info.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
system_misc.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
tcm.h
therm.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
thread_info.h Construct init thread stack in the linker script rather than by union 2018-01-09 23:21:02 +00:00
thread_notify.h
timex.h
tlb.h mm: fix MADV_[FREE|DONTNEED] TLB flush miss problem 2017-08-10 15:54:07 -07:00
tlbflush.h sched/headers: Prepare to remove the <linux/mm_types.h> dependency from <linux/sched.h> 2017-03-02 08:42:37 +01:00
tls.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
topology.h Merge branch 'pm-cpufreq' 2017-11-13 01:34:49 +01:00
traps.h ARM: probes: avoid adding kprobes to sensitive kernel-entry/exit code 2017-12-17 22:14:21 +00:00
trusted_foundations.h treewide: remove redundant #include <linux/kconfig.h> 2016-10-11 15:06:33 -07:00
uaccess.h Revert "arm/syscalls: Check address limit on user-mode return" 2017-09-17 19:45:33 +02:00
ucontext.h Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm 2017-11-16 12:50:35 -08:00
unaligned.h ARM: 8715/1: add a private asm/unaligned.h 2017-10-24 10:33:23 +01:00
unified.h ARM: 8723/2: always assume the "unified" syntax for assembly code 2017-12-17 22:14:21 +00:00
unistd.h ARM: wire up new pkey syscalls 2016-10-19 10:18:47 +01:00
unwind.h
uprobes.h
user.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
v7m.h Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm 2017-11-16 12:50:35 -08:00
vdso_datapage.h
vdso.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vfp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vfpmacros.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vga.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
virt.h ARM: hyp-stub/KVM: Kill __hyp_get_vectors 2017-04-09 07:49:34 -07:00
word-at-a-time.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xor.h