The first issue fixed in this patch is that pending rwlock write locks
could lock out new readers; this could cause a deadlock if a read lock was
held on cpu 1, a write lock was then attempted on cpu 2 and was pending,
and cpu 1 was interrupted and attempted to re-acquire a read lock.
The write lock code was modified to not lock out new readers.
The second issue fixed is that there was a narrow race window where a tns
instruction had been issued (setting the lock value to "1") and the store
instruction to reset the lock value correctly had not yet been issued.
In this case, if an interrupt occurred and the same cpu then tried to
manipulate the lock, it would find the lock value set to "1" and spin
forever, assuming some other cpu was partway through updating it. The fix
is to enforce an interrupt critical section around the tns/store pair.
In addition, this change now arranges to always validate that after
a readlock we have not wrapped around the count of readers, which
is only eight bits.
Since these changes make the rwlock "fast path" code heavier weight,
I decided to move all the rwlock code all out of line, leaving only the
conventional spinlock code with fastpath inlines. Since the read_lock
and read_trylock implementations ended up very similar, I just expressed
read_lock in terms of read_trylock.
As part of this change I also eliminate support for the now-obsolete
tns_atomic mode.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
The Tilera architecture traditionally supports 64KB page sizes
to improve TLB utilization and improve performance when the
hardware is being used primarily to run a single application.
For more generic server scenarios, it can be beneficial to run
with 4KB page sizes, so this commit allows that to be specified
(by modifying the arch/tile/include/hv/pagesize.h header).
As part of this change, we also re-worked the PTE management
slightly so that PTE writes all go through a __set_pte() function
where we can do some additional validation. The set_pte_order()
function was eliminated since the "order" argument wasn't being used.
One bug uncovered was in the PCI DMA code, which wasn't properly
flushing the specified range. This was benign with 64KB pages,
but with 4KB pages we were getting some larger flushes wrong.
The per-cpu memory reservation code also needed updating to
conform with the newer percpu stuff; before it always chose 64KB,
and that was always correct, but with 4KB granularity we now have
to pay closer attention and reserve the amount of memory that will
be requested when the percpu code starts allocating.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
This is a grab bag of changes with no actual change to generated code.
This includes whitespace and comment typos, plus a couple of stale
comments being removed.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
It now takes an additional argument so it can be used to
flush-and-invalidate pages that are cached using hash-for-home
as well those that are cached with coherence point on a single cpu.
This allows it to be used more widely for changing the coherence
point of arbitrary pages when necessary.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
This avoids having to maintain an additional separate assembly
file, and of course the inline is slightly more efficient as well.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
The current implementations of __ndelay and __udelay call a hypervisor
service to delay, but the hypervisor service isn't actually implemented
very well, and the consensus is that Linux should handle figuring this
out natively and not use a hypervisor service.
By converting nanoseconds to cycles, and then spinning until the
cycle counter reaches the desired cycle, we get several benefits:
first, we are sensitive to the actual clock speed; second, we use
less power by issuing a slow SPR read once every six cycles while
we delay; and third, we properly handle the case of an interrupt by
exiting at the target time rather than after some number of cycles.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
The convention changed to, e.g., ".data..page_aligned". This commit
fixes the places in the tile architecture that were still using the
old convention. One tile-specific section (.init.page) was dropped
in favor of just using an "aligned" attribute.
Sam Ravnborg <sam@ravnborg.org> pointed out __PAGE_ALIGNED_BSS, etc.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
This change fixes a bug that memchr() will read the first word
of the source even if the length is zero. Ironically, the code
was originally written with a test to avoid exactly this problem,
but to make the code conform to Linux coding standards with all
declarations preceding all statements, the first load from memory
was moved up above that test as the initial value for a variable.
The change just moves all the variable declarations to the top
of the file, with no initializers, so that the test can also be
at the top of the file.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
This avoids a deadlock in the IGMP code where one core gets a read
lock, another core starts trying to get a write lock (thus blocking
new readers), and then the first core tries to recursively re-acquire
the read lock.
We still try to preserve some degree of balance by giving priority
to additional write lockers that come along while the lock is held
for write, so they can all complete quickly and return the lock to
the readers.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
This change makes KM_TYPE_NR independent of the actual deprecated
list of km_type values, which are no longer used in tile code anywhere.
For now we leave it set to 8, allowing that many nested mappings,
and thus reserving 32MB of address space.
A few remaining places using KM_* values were cleaned up as well.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Our internal process shares memcpy, memset, etc., with libc, and
we did some minor tweaking as part of moving from uclibc to glibc,
which is now reflected in the kernel versions of these files.
There are no semantic changes in this commit, just whitespace
(memcpy_32.S now properly uses tabs), naming (memmove.c instead
of memmove_32.c, since TILE-Gx shares the file with TILEPro),
and a couple of other minor tweaks.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Some BUG_ON checks can be detected at compile time rather than
at runtime.
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
This change rolls up random cleanups not representing any actual bugs.
- Remove a stale CONFIG_ value from the default tile_defconfig
- Remove unused tns_atomic_xxx() family of methods from <asm/atomic.h>
- Optimize get_order() using Tile's "clz" instruction
- Fix a bad hypervisor upcall name (not currently used in Linux anyway)
- Use __copy_in_user_inatomic() name for consistency, and export it
- Export some additional hypervisor driver I/O upcalls and some homecache calls
- Remove the obfuscating MEMCPY_TEST_WH64 support code
- Other stray comment cleanups, #if 0 removal, etc.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
This commit is primarily changes caused by reviewing "sparse"
and "checkpatch" output on our sources, so is somewhat noisy, since
things like "printk() -> pr_err()" (or whatever) throughout the
codebase tend to get tedious to read. Rather than trying to tease
apart precisely which things changed due to which type of code
review, this commit includes various cleanups in the code:
- sparse: Add declarations in headers for globals.
- sparse: Fix __user annotations.
- sparse: Using gfp_t consistently instead of int.
- sparse: removing functions not actually used.
- checkpatch: Clean up printk() warnings by using pr_info(), etc.;
also avoid partial-line printks except in bootup code.
- checkpatch: Use exposed structs rather than typedefs.
- checkpatch: Change some C99 comments to C89 comments.
In addition, a couple of minor other changes are rolled in
to this commit:
- Add support for a "raise" instruction to cause SIGFPE, etc., to be raised.
- Remove some compat code that is unnecessary when we fully eliminate
some of the deprecated syscalls from the generic syscall ABI.
- Update the tile_defconfig to reflect current config contents.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
This code is used in other places in our system than in Linux, so
to share it we now implement it as an inline function in our low-level
<arch> headers, and instantiate it in one file in Linux's arch/tile/lib.
The file is now cacheflush.c and is C code rather than the strangely-named
and assembler-implemented __invalidate_icache.S.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
This wasn't properly tested until the perf-event subsystem started
to get brought up under the tile architecture.
The bug caused bogus atomic64_cmpxchg() values to be returned,
among other things.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
This change is the core kernel support for TILEPro and TILE64 chips.
No driver support (except the console driver) is included yet.
This includes the relevant Linux headers in asm/; the low-level
low-level "Tile architecture" headers in arch/, which are
shared with the hypervisor, etc., and are build-system agnostic;
and the relevant hypervisor headers in hv/.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Reviewed-by: Paul Mundt <lethal@linux-sh.org>
