switch_mm is an expensive operations that has two users.
flush_icache_deferred is only called within switch_mm and can be moved
together. The function is expected to be more complicated when ASID
support is added, so clean up eagerly.
By moving them to a separate file we also removes some excessive
dependency of tlbflush.h and cacheflush.h.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Currently, flush_icache_all is macro-expanded into a SBI call, yet no
asm/sbi.h is included in asm/cacheflush.h. This could be moved to
mm/cacheflush.c instead (SBI call will dominate performance-wise and
there is no worry to not have it inlined.
Currently, flush_icache_mm stays in kernel/smp.c, which looks like a
hack to prevent it from being compiled when CONFIG_SMP=n. It should
also be in mm/cacheflush.c.
Signed-off-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
We should prefer accessing CSRs using their CSR numbers because:
1. It compiles fine with older toolchains.
2. We can use latest CSR names in #define macro names of CSR numbers
as-per RISC-V spec.
3. We can access newly added CSRs even if toolchain does not recognize
newly addes CSRs by name.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
The riscv version of free_initmem() differs from the generic one only in
that it sets the freed memory to zero.
Make ricsv use the generic version and poison the freed memory.
Link: http://lkml.kernel.org/r/1550515285-17446-5-git-send-email-rppt@linux.ibm.com
Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Richard Kuo <rkuo@codeaurora.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The Maximum Physical Memory 2GiB option for 64bit systems is currently
broken because kernel hangs at boot-time when this option is enabled
and the underlying system has more than 2GiB memory.
This issue can be easily reproduced on SiFive Unleashed board where
we have 8GiB of memory.
This patch fixes above issue by removing unusable memory region in
setup_bootmem().
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
The Linux RISC-V 32bit kernel is broken after we moved setup_vm() from
kernel/setup.c to mm/init.c because Linux RISC-V 32bit kernel by default
uses cmodel=medlow which results in a non-position-independent setup_vm().
This patch fixes Linux RISC-V 32bit kernel booting by:
1. Forcing cmodel=medany for mm/init.c
2. Moving remaing MM-related stuff va_pa_offset, pfn_base and
empty_zero_page from kernel/setup.c to mm/init.c
Further, the setup_vm() cannot handle GCC instrumentation for FTRACE so
we disable it for mm/init.c by not using "-pg" compiler flag.
Fixes: 6f1e9e946f ("RISC-V: Move setup_vm() to mm/init.c")
Suggested-by: Christoph Hellwig <hch@lst.de>
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
We should free-up initrd memory in free_initrd_mem() instead
of doing nothing.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
This patch implements compile-time virtual to physical mappings. These
compile-time fixed mappings can be used by earlycon, ACPI, and early
ioremap for creating fixed mappings when FIX_EARLYCON_MEM=y.
To start with, we have enabled compile-time fixed mappings for earlycon.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
The setup_vm() is responsible for setting up initial page table hence
should be placed in mm/init.c.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
The setup_bootmem() mainly populates memblocks and does early memory
reservations. The right location for this function is mm/init.c. It
calls setup_initrd() so we move that as well.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
max_low_pfn should be pfn_size not byte_size.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Signed-off-by: Mao Han <mao_han@c-sky.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Move remaining definitions and declarations from include/linux/bootmem.h
into include/linux/memblock.h and remove the redundant header.
The includes were replaced with the semantic patch below and then
semi-automated removal of duplicated '#include <linux/memblock.h>
@@
@@
- #include <linux/bootmem.h>
+ #include <linux/memblock.h>
[sfr@canb.auug.org.au: dma-direct: fix up for the removal of linux/bootmem.h]
Link: http://lkml.kernel.org/r/20181002185342.133d1680@canb.auug.org.au
[sfr@canb.auug.org.au: powerpc: fix up for removal of linux/bootmem.h]
Link: http://lkml.kernel.org/r/20181005161406.73ef8727@canb.auug.org.au
[sfr@canb.auug.org.au: x86/kaslr, ACPI/NUMA: fix for linux/bootmem.h removal]
Link: http://lkml.kernel.org/r/20181008190341.5e396491@canb.auug.org.au
Link: http://lkml.kernel.org/r/1536927045-23536-30-git-send-email-rppt@linux.vnet.ibm.com
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Chris Zankel <chris@zankel.net>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Greentime Hu <green.hu@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Guan Xuetao <gxt@pku.edu.cn>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
Cc: Jonas Bonn <jonas@southpole.se>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Ley Foon Tan <lftan@altera.com>
Cc: Mark Salter <msalter@redhat.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Richard Kuo <rkuo@codeaurora.org>
Cc: Richard Weinberger <richard@nod.at>
Cc: Rich Felker <dalias@libc.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Serge Semin <fancer.lancer@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
For 32bit, the upper 32-bit of phys_addr_t will be flushed to zero
after AND with PAGE_MASK because the data type of PAGE_MASK is
unsigned long. To fix this problem, the page alignment is done by
subtracting the page offset instead of AND with PAGE_MASK.
Signed-off-by: Vincent Chen <vincentc@andestech.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Use new return type vm_fault_t for fault handler. For now, this is just
documenting that the function returns a VM_FAULT value rather than an
errno. Once all instances are converted, vm_fault_t will become a
distinct type.
Ref-> commit 1c8f422059 ("mm: change return type to vm_fault_t")
In this patch all the caller of handle_mm_fault() are changed to return
vm_fault_t type.
Link: http://lkml.kernel.org/r/20180617084810.GA6730@jordon-HP-15-Notebook-PC
Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Richard Kuo <rkuo@codeaurora.org>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Michal Simek <monstr@monstr.eu>
Cc: James Hogan <jhogan@kernel.org>
Cc: Ley Foon Tan <lftan@altera.com>
Cc: Jonas Bonn <jonas@southpole.se>
Cc: James E.J. Bottomley <jejb@parisc-linux.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: David S. Miller <davem@davemloft.net>
Cc: Richard Weinberger <richard@nod.at>
Cc: Guan Xuetao <gxt@pku.edu.cn>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: "Levin, Alexander (Sasha Levin)" <alexander.levin@verizon.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The DMA32 is for 64-bit usage.
Signed-off-by: Zong Li <zong@andestech.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
This tag contains the fixes we'd like to target for the 4.16 merge
window. It's not as much as I was originally hoping to do but between
glibc, the chip, and FOSDEM there just wasn't enough time to get
everything put together. As such, this merge window is essentially just
going to be small changes. This includes mostly cleanups:
* A build fix failure to the audit test cases. RISC-V doesn't have
renameat because the generic syscall ABI moved to renameat2 by the
time of our port. The syscall audit test cases don't understand this,
so I added a trivial fix. This went through mailing list review
during the 4.15 merge window, but nobody has picked it up so I think
it's best to just do this here.
* The removal of our command-line argument processing code. The
"mem_end" stuff was broken and the rest duplicated generic device tree
code. The generic code was already being called.
* Some unused/redundant code has been removed, including
__ARCH_HAVE_MMU, current_pgdir, and the initialization of init_mm.pgd.
* SUM is disabled upon taking a trap, which means that user memory is
protected during traps taking inside copy_{to,from}_user().
* The sptbr CSR has been renamed to satp in C code. We haven't changed
the assembly code in order to maintain compatibility with binutils
2.29, which doesn't understand the new name.
Additionally, we're adding some new features:
* Basic ftrace support, thanks to Alan Kao!
* Support for ZONE_DMA32. This is necessary for all the normal reasons,
but also to deal with a deficiency in the Xilinx PCIe controller we're
using on our FPGA-based systems. While the ZONE_DMA32 addition should
be sufficient for most uses, it doesn't complete the fix for the
Xilinx controller.
* TLB shootdowns now only target the harts where they're necessary,
instead of applying to all harts in the system.
These patches have all been sitting on our linux-next branch for a while
now. Due to time constraints this is all I feel comfortable submitting
during the 4.16 merge window, hopefully we'll do better next time!
-----BEGIN PGP SIGNATURE-----
iQJHBAABCAAxFiEEAM520YNJYN/OiG3470yhUCzLq0EFAlp7N2gTHHBhbG1lckBk
YWJiZWx0LmNvbQAKCRDvTKFQLMurQX8kD/4xxw6TuuESmDXxAQPQ+S8J98uKRfAF
9kMMzJJARcW5sT1vo3pKpE8+Ss0Hy2fIcaYsw5Je/Yl7vdAy/Dk7X3/mx7mxf5BP
8m2cSd7DFLLLhntZTbr1Y5fJ6awFLtzI46zn/SzTdTatLWKXNLS5wmPKE33ddq/C
iTi4k/as8E/vuNtuPy1GsOF0gICpZ2xB4YoMwTgWfpxTekBkUktO3EOHmZTwQEEM
U1muB+4WoqusbBt6cP3Q7cUF3b6aMVSevWnywZGkD+yWOGRXTVzMgT7R4YlKEOre
OQypZocYUbRmZQMZACKpgHIcOZpePaSTIQ2zzhXEPVGB0XAHtMRnAaVtwPxwG6c4
EThDCN9ldShutKqT4XilHrh5gf0sy7qG0PIidPhMmXH9LCeTSAU4VdISJP1jkq19
chiMHlf6+/DhikyiH0+lK/MX8vQMt6UJL1SlRKO/c2FxxKAZKnENJ+tuAlkAlwoC
gnvZsE5BUYw1ptRHXR0d5C4m8M2M9LPZfpWYcg+1mRO9EA+kt0XCupL7RsrdFuoa
FCVEhP/JMaiX0JtmAHfVIU0yNGjH3b5xi3FoGk2Aoj/c8O3F5YcwT5C5nO+jpv32
n9vyMR20/721+yA2dFIlq4DnelwdZczOTqrcDYJrLxXzk8OXUFFffbe4kbDCxp34
WniBxwnY9BF25g==
=cNRH
-----END PGP SIGNATURE-----
Merge tag 'riscv-for-linus-4.16-merge_window' of git://git.kernel.org/pub/scm/linux/kernel/git/palmer/riscv-linux
Pull RISC-V updates from Palmer Dabbelt:
"This contains the fixes we'd like to target for the 4.16 merge window.
It's not as much as I was originally hoping to do but between glibc,
the chip, and FOSDEM there just wasn't enough time to get everything
put together. As such, this merge window is essentially just going to
be small changes. This includes mostly cleanups:
- A build fix failure to the audit test cases.
RISC-V doesn't have renameat because the generic syscall ABI moved
to renameat2 by the time of our port. The syscall audit test cases
don't understand this, so I added a trivial fix. This went through
mailing list review during the 4.15 merge window, but nobody has
picked it up so I think it's best to just do this here.
- The removal of our command-line argument processing code. The
"mem_end" stuff was broken and the rest duplicated generic device
tree code. The generic code was already being called.
- Some unused/redundant code has been removed, including
__ARCH_HAVE_MMU, current_pgdir, and the initialization of
init_mm.pgd.
- SUM is disabled upon taking a trap, which means that user memory is
protected during traps taking inside copy_{to,from}_user().
- The sptbr CSR has been renamed to satp in C code. We haven't
changed the assembly code in order to maintain compatibility with
binutils 2.29, which doesn't understand the new name.
Additionally, we're adding some new features:
- Basic ftrace support, thanks to Alan Kao!
- Support for ZONE_DMA32.
This is necessary for all the normal reasons, but also to deal with
a deficiency in the Xilinx PCIe controller we're using on our
FPGA-based systems. While the ZONE_DMA32 addition should be
sufficient for most uses, it doesn't complete the fix for the
Xilinx controller.
- TLB shootdowns now only target the harts where they're necessary,
instead of applying to all harts in the system.
These patches have all been sitting on our linux-next branch for a
while now. Due to time constraints this is all I feel comfortable
submitting during the 4.16 merge window, hopefully we'll do better
next time!"
[ Note to self: "harts" is RISC-V speak for "hardware threads". I had
to look that up. - Linus ]
* tag 'riscv-for-linus-4.16-merge_window' of git://git.kernel.org/pub/scm/linux/kernel/git/palmer/riscv-linux:
riscv: inline set_pgdir into its only caller
riscv: rename sptbr to satp
riscv: don't read back satp in paging_init
riscv: remove the unused current_pgdir function
riscv: add ZONE_DMA32
RISC-V: Limit the scope of TLB shootdowns
riscv: disable SUM in the exception handler
riscv: remove redundant unlikely()
riscv: remove unused __ARCH_HAVE_MMU define
riscv/ftrace: Add basic support
RISC-V: Remove mem_end command line processing
RISC-V: Remove duplicate command-line parsing logic
audit: Avoid build failures on systems without renameat
Pull asm/uaccess.h whack-a-mole from Al Viro:
"It's linux/uaccess.h, damnit... Oh, well - eventually they'll stop
cropping up..."
* 'work.whack-a-mole' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs:
asm-prototypes.h: use linux/uaccess.h, not asm/uaccess.h
riscv: use linux/uaccess.h, not asm/uaccess.h...
ppc: for put_user() pull linux/uaccess.h, not asm/uaccess.h
satp is the name used by the current privileged spec 1.10, use it
instead of the old name. The most recent release binutils release
(2.29) doesn't know about the satp name yet, so stick to the name from
the previous privileged ISA release and comment on the fact.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
init_mm.pgd (aka swapped_pgd) gets relocated like all other kernel
symbols by the elf loader, so there is no need to reload it from satp.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
This patch allows devices that require memory that can be addressed
using 32-bit addresses to work easily on RISC-V systems. The newly
improved dma-direct ops will tap into this pool automatically for
32-bit addressing.
Based on an earlier patch from Wesley W. Terpstra.
CC: Wesley W. Terpstra <terpstra@sifive.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Olaf said: Here's a short series of patches that produces a working
allmodconfig. Would be nice to see them go in so we can add build
coverage.
I've dropped patches 8 and 10 from the original set:
* [PATCH 08/10] (RISC-V: Set __ARCH_WANT_RENAMEAT to pick up generic
version) has a better fix that I've sent out for review, we don't want
renameat.
* [PATCH 10/10] (input: joystick: riscv has get_cycles) has already been
taken into Dmitry Torokhov's tree.
The RISC-V ISA allows for instruction caches that are not coherent WRT
stores, even on a single hart. As a result, we need to explicitly flush
the instruction cache whenever marking a dirty page as executable in
order to preserve the correct system behavior.
Local instruction caches aren't that scary (our implementations actually
flush the cache, but RISC-V is defined to allow higher-performance
implementations to exist), but RISC-V defines no way to perform an
instruction cache shootdown. When explicitly asked to do so we can
shoot down remote instruction caches via an IPI, but this is a bit on
the slow side.
Instead of requiring an IPI to all harts whenever marking a page as
executable, we simply flush the currently running harts. In order to
maintain correct behavior, we additionally mark every other hart as
needing a deferred instruction cache which will be taken before anything
runs on it.
Signed-off-by: Andrew Waterman <andrew@sifive.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
include <linux/types.h> for __iomem definition. Also, add volatile to
iounmap() like other architectures have it to avoid "discarding
volatile" warnings from some drivers.
Finally, explicitly promote the base address for INB/OUTB functions to
avoid some old legacy drivers complaining about int-to-ptr promotions.
The drivers are unlikely to work but they're included in allmodconfig
so the warnings are noisy.
Fixes, among other warnings, these with allmodconfig:
../arch/riscv/include/asm/io.h:24:21: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token
extern void __iomem *ioremap(phys_addr_t offset, unsigned long size);
sound/pci/echoaudio/echoaudio.c: In function 'snd_echo_free':
sound/pci/echoaudio/echoaudio.c:1879:10: warning: passing argument 1 of 'iounmap' discards 'volatile' qualifier from pointer target type [-Wdiscarded-qualifiers]
Signed-off-by: Olof Johansson <olof@lixom.net>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
This patch contains all the build infrastructure that actually enables
the RISC-V port. This includes Makefiles, linker scripts, and Kconfig
files. It also contains the only top-level change, which adds RISC-V to
the list of architectures that need a sed run to produce the ARCH
variable when building locally.
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
This patch contains code to manage the RISC-V MMU, including definitions
of the page tables and the page walking code.
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
This patch contains code that interfaces with devices that are mandated
by the RISC-V supervisor specification and that don't have explicit
drivers anywhere else in the tree. This includes the staticly defined
interrupts, the CSR-mapped timer, and virtualized SBI devices.
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
This patch contains the code that interfaces with ELF objects on RISC-V
systems, the vast majority of which is present to load kernel modules.
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
This contains the various __init C functions, the initial assembly
kernel entry point, and the code to reset the system. When a file was
init-related this patch contains the entire file.
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
