korg/linux
0
0
Fork 0
mirror of https://mirrors.bfsu.edu.cn/git/linux.git synced 2024-11-27 06:04:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

s390/ftrace: Avoid calling unwinder in ftrace_return_address()

Browse Source 
ftrace_return_address() is called extremely often from
performance-critical code paths when debugging features like
CONFIG_TRACE_IRQFLAGS are enabled. For example, with debug_defconfig,
ftrace selftests on my LPAR currently execute ftrace_return_address()
as follows:

ftrace_return_address(0) - 0 times (common code uses __builtin_return_address(0) instead)
ftrace_return_address(1) - 2,986,805,401 times (with this patch applied)
ftrace_return_address(2) - 140 times
ftrace_return_address(>2) - 0 times

The use of __builtin_return_address(n) was replaced by return_address()
with an unwinder call by commit cae74ba8c2 ("s390/ftrace:
Use unwinder instead of __builtin_return_address()") because
__builtin_return_address(n) simply walks the stack backchain and doesn't
check for reaching the stack top. For shallow stacks with fewer than
"n" frames, this results in reads at low addresses and random
memory accesses.

While calling the fully functional unwinder "works", it is very slow
for this purpose. Moreover, potentially following stack switches and
walking past IRQ context is simply wrong thing to do for
ftrace_return_address().

Reimplement return_address() to essentially be __builtin_return_address(n)
with checks for reaching the stack top. Since the ftrace_return_address(n)
argument is always a constant, keep the implementation in the header,
allowing both GCC and Clang to unroll the loop and optimize it to the
bare minimum.

Fixes: cae74ba8c2 ("s390/ftrace: Use unwinder instead of __builtin_return_address()")
Cc: stable@vger.kernel.org
Reported-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This commit is contained in:
Vasily Gorbik 2024-08-24 02:14:04 +02:00
parent 57216cc985
commit a84dd0d8ae
2 changed files with 16 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
arch/s390/include/asm/ftrace.h
View File

@ -6,8 +6,23 @@
#define MCOUNT_INSN_SIZE 6
#ifndef __ASSEMBLY__
#include <asm/stacktrace.h>
unsigned long return_address(unsigned int n);
static __always_inline unsigned long return_address(unsigned int n)
{
struct stack_frame *sf;
if (!n)
return (unsigned long)__builtin_return_address(0);
sf = (struct stack_frame *)current_frame_address();
do {
sf = (struct stack_frame *)sf->back_chain;
if (!sf)
return 0;
} while (--n);
return sf->gprs[8];
}
#define ftrace_return_address(n) return_address(n)
void ftrace_caller(void);

19
arch/s390/kernel/stacktrace.c
View File

@ -162,22 +162,3 @@ void arch_stack_walk_user(stack_trace_consume_fn consume_entry, void *cookie,
{
arch_stack_walk_user_common(consume_entry, cookie, NULL, regs, false);
}
unsigned long return_address(unsigned int n)
{
struct unwind_state state;
unsigned long addr;
/* Increment to skip current stack entry */
n++;
unwind_for_each_frame(&state, NULL, NULL, 0) {
addr = unwind_get_return_address(&state);
if (!addr)
break;
if (!n--)
return addr;
}
return 0;
}
EXPORT_SYMBOL_GPL(return_address);