This series is based on the alternatives changes done in my svpbmt
series and thus also depends on Atish's isa-extension parsing series.
It implements using the cache-management instructions from the Zicbom-
extension to handle cache flush, etc actions on platforms needing them.
SoCs using cpu cores from T-Head like the Allwinne D1 implement a
different set of cache instructions. But while they are different,
instructions they provide the same functionality, so a variant can easly
hook into the existing alternatives mechanism on those.
[Palmer: Some minor fixups, including a RISCV_ISA_ZICBOM dependency on
MMU that's probably not strictly necessary. The Zicbom support will
trip up sparse for users that have new toolchains, I just sent a patch.]
Link: https://lore.kernel.org/all/20220706231536.2041855-1-heiko@sntech.de/
Link: https://lore.kernel.org/linux-sparse/20220811033138.20676-1-palmer@rivosinc.com/T/#u
* palmer/riscv-zicbom:
riscv: implement cache-management errata for T-Head SoCs
riscv: Add support for non-coherent devices using zicbom extension
dt-bindings: riscv: document cbom-block-size
of: also handle dma-noncoherent in of_dma_is_coherent()
* Enabling the FPU is now a static_key.
* Improvements to the Svpbmt support.
* CPU topology bindings for a handful of systems.
* Support for systems with 64-bit hart IDs.
* Many settings have been enabled in the defconfig, including both
support for the StarFive systems and many of the Docker requirements.
There are also a handful of cleanups and improvements, like usual.
-----BEGIN PGP SIGNATURE-----
iQJHBAABCAAxFiEEAM520YNJYN/OiG3470yhUCzLq0EFAmLtolMTHHBhbG1lckBk
YWJiZWx0LmNvbQAKCRDvTKFQLMurQcqLD/48RwM9IjNKanYgA0JWMSzrtQKLpv6R
+vD6/9RVse+8HL5pMacJdUKKMLUFkIxJIpzmTviGHMc5j4MnVql7CNHAPQQX0L60
+eV6ogVQI54CzC/lYgfiDolS7ZIVVSatM6YgBVf8045bHqgmEqx9H4W87fsM42XW
IDmRmW9bQu6a5I+gegsHvKwG4ex5SWeesrIOi0RaTwMR7lZuiy3shFTfpHsqalSp
N3oPyC0AvngsZcl/iqMB4AbEFgWct7bL6EipuDP3q4VvT4v9eMOkl+j6wIKNlfBJ
1igdmNNTLQQHCVSeTx93i1ZsJDh1yvAx3+sbv34WiMIVEDrusTFIxGzV4uX3kVO/
2pR31ItH7SFNjOMp6j/ouaGvtAqJVYSiya82jijUpMtxt0Pajua+cxN5xrX3RY2f
L13VRW5XIgBykmfJLhLz0rzPj/Sfk55RhcV8CIoOPhzugz39oF6z4XaTH2lvqykM
mZ1s0Ab4Ontf26MDnj73gUHzRadDzHbIptvdDGVPlxn8+Ki/Yed6tT7nQrAvxZ5b
93uHJAjGTL9DrorYCEhfp4SAnQPTtqXnmSvw84dAU5QrsLvDk/JIS+MNMPm9VCcS
GRUmI6iCE1D+KERT2mED7NOmRqbd4NCZ6T1UXO64MVv1PuqM9f5+bVohU31PJEU1
uAXelP/sl0+mYw==
=v9hn
-----END PGP SIGNATURE-----
Merge tag 'riscv-for-linus-5.20-mw0' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux
Pull RISC-V updates from Palmer Dabbelt:
- Enabling the FPU is now a static_key
- Improvements to the Svpbmt support
- CPU topology bindings for a handful of systems
- Support for systems with 64-bit hart IDs
- Many settings have been enabled in the defconfig, including both
support for the StarFive systems and many of the Docker requirements
There are also a handful of cleanups and improvements, as usual.
* tag 'riscv-for-linus-5.20-mw0' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux: (28 commits)
riscv: enable Docker requirements in defconfig
riscv: convert the t-head pbmt errata to use the __nops macro
riscv: introduce nops and __nops macros for NOP sequences
RISC-V: Add fast call path of crash_kexec()
riscv: mmap with PROT_WRITE but no PROT_READ is invalid
riscv/efi_stub: Add 64bit boot-hartid support on RV64
riscv: cpu: Add 64bit hartid support on RV64
riscv: smp: Add 64bit hartid support on RV64
riscv: spinwait: Fix hartid variable type
riscv: cpu_ops_sbi: Add 64bit hartid support on RV64
riscv: dts: sifive: "fix" pmic watchdog node name
riscv: dts: canaan: Add k210 topology information
riscv: dts: sifive: Add fu740 topology information
riscv: dts: sifive: Add fu540 topology information
riscv: dts: starfive: Add JH7100 CPU topology
RISC-V: Add CONFIG_{NON,}PORTABLE
riscv: config: enable SOC_STARFIVE in defconfig
riscv: dts: microchip: Add mpfs' topology information
riscv: Kconfig.socs: Add comments
riscv: Kconfig.erratas: Add comments
...
The T-Head C906 and C910 implement a scheme for handling
cache operations different from the generic Zicbom extension.
Add an errata for it next to the generic dma coherency ops.
Reviewed-by: Samuel Holland <samuel@sholland.org>
Tested-by: Samuel Holland <samuel@sholland.org>
Reviewed-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://lore.kernel.org/r/20220706231536.2041855-5-heiko@sntech.de
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
The Zicbom ISA-extension was ratified in november 2021
and introduces instructions for dcache invalidate, clean
and flush operations.
Implement cache management operations for non-coherent devices
based on them.
Of course not all cores will support this, so implement an
alternative-based mechanism that replaces empty instructions
with ones done around Zicbom instructions.
As discussed in previous versions, assume the platform
being coherent by default so that non-coherent devices need
to get marked accordingly by firmware.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Guo Ren <guoren@kernel.org>
Link: https://lore.kernel.org/r/20220706231536.2041855-4-heiko@sntech.de
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Instead of manually inserting the list of nops, use the recently
introduced __nops(n) macro to make everything more readable.
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
After commit a35707c3d8 ("riscv: add memory-type errata for T-Head"),
builds with LLVM's integrated assembler fail like:
In file included from arch/riscv/kernel/asm-offsets.c:10:
In file included from ./include/linux/mm.h:29:
In file included from ./include/linux/pgtable.h:6:
In file included from ./arch/riscv/include/asm/pgtable.h:114:
./arch/riscv/include/asm/pgtable-64.h:210:2: error: invalid input constraint '0' in asm
ALT_THEAD_PMA(prot_val);
^
./arch/riscv/include/asm/errata_list.h:88:4: note: expanded from macro 'ALT_THEAD_PMA'
: "0"(_val), \
^
This was reported upstream to LLVM where Jessica pointed out a couple of
issues with the existing implementation of ALT_THEAD_PMA:
* t3 is modified but not listed in the clobbers list.
* "+r"(_val) marks _val as both an input and output of the asm but then
"0"(_val) marks _val as an input matching constraint, which does not
make much sense in this situation, as %1 is not actually used in the
asm and matching constraints are designed to be used for different
inputs that need to use the same register.
Drop the matching contraint and shift all the operands by one, as %1 is
unused, and mark t3 as clobbered. This resolves the build error and goes
not cause any problems with GNU as.
Fixes: a35707c3d8 ("riscv: add memory-type errata for T-Head")
Link: https://github.com/ClangBuiltLinux/linux/issues/1641
Link: https://github.com/llvm/llvm-project/issues/55514
Link: https://gcc.gnu.org/onlinedocs/gcc/Simple-Constraints.html
Suggested-by: Jessica Clarke <jrtc27@jrtc27.com>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Heiko Stuebner <heiko@sntech.de>
Tested-by: Heiko Stuebner <heiko@sntech.de>
Link: https://lore.kernel.org/r/20220518184529.454008-1-nathan@kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Some current cpus based on T-Head cores implement memory-types
way different than described in the svpbmt spec even going
so far as using PTE bits marked as reserved.
Add the T-Head vendor-id and necessary errata code to
replace the affected instructions.
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Tested-by: Samuel Holland <samuel@sholland.org>
Link: https://lore.kernel.org/r/20220511192921.2223629-13-heiko@sntech.de
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Svpbmt (the S should be capitalized) is the
"Supervisor-mode: page-based memory types" extension
that specifies attributes for cacheability, idempotency
and ordering.
The relevant settings are done in special bits in PTEs:
Here is the svpbmt PTE format:
| 63 | 62-61 | 60-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
N MT RSW D A G U X W R V
^
Of the Reserved bits [63:54] in a leaf PTE, the high bit is already
allocated (as the N bit), so bits [62:61] are used as the MT (aka
MemType) field. This field specifies one of three memory types that
are close equivalents (or equivalent in effect) to the three main x86
and ARMv8 memory types - as shown in the following table.
RISC-V
Encoding &
MemType RISC-V Description
---------- ------------------------------------------------
00 - PMA Normal Cacheable, No change to implied PMA memory type
01 - NC Non-cacheable, idempotent, weakly-ordered Main Memory
10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory
11 - Rsvd Reserved for future standard use
As the extension will not be present on all implementations,
implement a method to handle cpufeatures via alternatives
to not incur runtime penalties on cpu variants not supporting
specific extensions and patch relevant code parts at runtime.
Co-developed-by: Wei Fu <wefu@redhat.com>
Signed-off-by: Wei Fu <wefu@redhat.com>
Co-developed-by: Liu Shaohua <liush@allwinnertech.com>
Signed-off-by: Liu Shaohua <liush@allwinnertech.com>
Co-developed-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
[moved to use the alternatives mechanism]
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Link: https://lore.kernel.org/r/20220511192921.2223629-10-heiko@sntech.de
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
For certain SiFive CPUs, "sfence.vma addr" cannot exactly flush addr
from TLB in the particular cases. The details could be found here:
https://sifive.cdn.prismic.io/sifive/167a1a56-03f4-4615-a79e-b2a86153148f_FU740_errata_20210205.pdf
In order to ensure the functionality, this patch uses the Alternative
scheme to replace all "sfence.vma addr" with "sfence.vma" at runtime.
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
Add sign extension to the $badaddr before addressing the instruction page
fault and instruction access fault to workaround the issue "cip-453".
To avoid affecting the existing code sequence, this patch will creates two
trampolines to add sign extension to the $badaddr. By the "alternative"
mechanism, these two trampolines will replace the original exception
handler of instruction page fault and instruction access fault in the
excp_vect_table. In this case, only the specific SiFive CPU core jumps to
the do_page_fault and do_trap_insn_fault through these two trampolines.
Other CPUs are not affected.
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
