Deepak Gupta be97d0db5f riscv: VMAP_STACK overflow detection thread-safe ... commit 31da94c25a ("riscv: add VMAP_STACK overflow detection") added support for CONFIG_VMAP_STACK. If overflow is detected, CPU switches to `shadow_stack` temporarily before switching finally to per-cpu `overflow_stack`. If two CPUs/harts are racing and end up in over flowing kernel stack, one or both will end up corrupting each other state because `shadow_stack` is not per-cpu. This patch optimizes per-cpu overflow stack switch by directly picking per-cpu `overflow_stack` and gets rid of `shadow_stack`. Following are the changes in this patch - Defines an asm macro to obtain per-cpu symbols in destination register. - In entry.S, when overflow is detected, per-cpu overflow stack is located using per-cpu asm macro. Computing per-cpu symbol requires a temporary register. x31 is saved away into CSR_SCRATCH (CSR_SCRATCH is anyways zero since we're in kernel). Please see Links for additional relevant disccussion and alternative solution. Tested by `echo EXHAUST_STACK > /sys/kernel/debug/provoke-crash/DIRECT` Kernel crash log below Insufficient stack space to handle exception!/debug/provoke-crash/DIRECT Task stack: [0xff20000010a98000..0xff20000010a9c000] Overflow stack: [0xff600001f7d98370..0xff600001f7d99370] CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34 Hardware name: riscv-virtio,qemu (DT) epc : __memset+0x60/0xfc ra : recursive_loop+0x48/0xc6 [lkdtm] epc : ffffffff808de0e4 ra : ffffffff0163a752 sp : ff20000010a97e80 gp : ffffffff815c0330 tp : ff600000820ea280 t0 : ff20000010a97e88 t1 : 000000000000002e t2 : 3233206874706564 s0 : ff20000010a982b0 s1 : 0000000000000012 a0 : ff20000010a97e88 a1 : 0000000000000000 a2 : 0000000000000400 a3 : ff20000010a98288 a4 : 0000000000000000 a5 : 0000000000000000 a6 : fffffffffffe43f0 a7 : 00007fffffffffff s2 : ff20000010a97e88 s3 : ffffffff01644680 s4 : ff20000010a9be90 s5 : ff600000842ba6c0 s6 : 00aaaaaac29e42b0 s7 : 00fffffff0aa3684 s8 : 00aaaaaac2978040 s9 : 0000000000000065 s10: 00ffffff8a7cad10 s11: 00ffffff8a76a4e0 t3 : ffffffff815dbaf4 t4 : ffffffff815dbaf4 t5 : ffffffff815dbab8 t6 : ff20000010a9bb48 status: 0000000200000120 badaddr: ff20000010a97e88 cause: 000000000000000f Kernel panic - not syncing: Kernel stack overflow CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34 Hardware name: riscv-virtio,qemu (DT) Call Trace: [<ffffffff80006754>] dump_backtrace+0x30/0x38 [<ffffffff808de798>] show_stack+0x40/0x4c [<ffffffff808ea2a8>] dump_stack_lvl+0x44/0x5c [<ffffffff808ea2d8>] dump_stack+0x18/0x20 [<ffffffff808dec06>] panic+0x126/0x2fe [<ffffffff800065ea>] walk_stackframe+0x0/0xf0 [<ffffffff0163a752>] recursive_loop+0x48/0xc6 [lkdtm] SMP: stopping secondary CPUs ---[ end Kernel panic - not syncing: Kernel stack overflow ]--- Cc: Guo Ren <guoren@kernel.org> Cc: Jisheng Zhang <jszhang@kernel.org> Link: https://lore.kernel.org/linux-riscv/Y347B0x4VUNOd6V7@xhacker/T/#t Link: https://lore.kernel.org/lkml/20221124094845.1907443-1-debug@rivosinc.com/ Signed-off-by: Deepak Gupta <debug@rivosinc.com> Co-developed-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Acked-by: Guo Ren <guoren@kernel.org> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-9-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>		2023-10-27 14:43:05 -07:00
..
boot	RISC-V Devicetrees for v6.6 Part 2	2023-08-21 21:47:55 -04:00
configs	Kbuild updates for v6.6	2023-09-05 11:01:47 -07:00
errata	Merge patch series "Add non-coherent DMA support for AX45MP"	2023-09-08 11:24:34 -07:00
include	riscv: VMAP_STACK overflow detection thread-safe	2023-10-27 14:43:05 -07:00
kernel	riscv: VMAP_STACK overflow detection thread-safe	2023-10-27 14:43:05 -07:00
kvm	KVM/riscv changes for 6.6	2023-08-31 13:25:55 -04:00
lib	riscv: uaccess: Return the number of bytes effectively not copied	2023-08-16 07:30:06 -07:00
mm	Merge patch series "riscv: Introduce KASLR"	2023-09-08 11:25:13 -07:00
net	bpf, riscv: use prog pack allocator in the BPF JIT	2023-09-06 06:26:07 -07:00
purgatory	riscv/purgatory: Disable CFI	2023-08-23 14:16:40 -07:00
tools	riscv: Check relocations at compile time	2023-04-19 07:46:32 -07:00
Kbuild	kexec: rename ARCH_HAS_KEXEC_PURGATORY	2023-08-18 10:18:54 -07:00
Kconfig	riscv: Kconfig: Select DMA_DIRECT_REMAP only if MMU is enabled	2023-09-08 11:25:26 -07:00
Kconfig.debug	RISC-V: Remove EARLY_PRINTK support	2018-12-17 10:23:46 -08:00
Kconfig.errata	riscv: Kconfig.errata: Add dependency for RISCV_SBI in ERRATA_ANDES config	2023-09-08 11:25:28 -07:00
Kconfig.socs	RISC-V: make ARCH_THEAD preclude XIP_KERNEL	2023-07-05 22:21:23 +02:00
Makefile	riscv: Require FRAME_POINTER for some configurations	2023-08-23 09:07:26 -07:00
Makefile.postlink	riscv: Use --emit-relocs in order to move .rela.dyn in init	2023-04-19 07:46:33 -07:00