linux/tools/perf
Thomas Richter d8d85ce86d perf lock contention: Fix core dump related to not finding the "__sched_text_end" symbol on s/390
The test case perf lock contention dumps core on s390. Run the following
commands:

  # ./perf lock record -- ./perf bench sched messaging
  # Running 'sched/messaging' benchmark:
  # 20 sender and receiver processes per group
  # 10 groups == 400 processes run

      Total time: 2.799 [sec]
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.073 MB perf.data (100 samples) ]
  #
  # ./perf lock contention
  Segmentation fault (core dumped)
  #

The function call stack is lengthy, here are the top 5 functions:

  # gdb ./perf core.24048
  GNU gdb (GDB) Fedora Linux 12.1-6.fc37
  Core was generated by `./perf lock contention'.
  Program terminated with signal SIGSEGV, Segmentation fault.
  #0  0x00000000011dd25c in machine__is_lock_function (machine=0x3029e28, addr=1789230) at util/machine.c:3356
         3356 machine->sched.text_end = kmap->unmap_ip(kmap, sym->start);

 (gdb) where
  #0  0x00000000011dd25c in machine__is_lock_function (machine=0x3029e28, addr=1789230) at util/machine.c:3356
  #1  0x000000000109f244 in callchain_id (evsel=0x30313e0, sample=0x3ffea4f77d0) at builtin-lock.c:957
  #2  0x000000000109e094 in get_key_by_aggr_mode (key=0x3ffea4f7290, addr=27758136, evsel=0x30313e0, sample=0x3ffea4f77d0) at builtin-lock.c:586
  #3  0x000000000109f4d0 in report_lock_contention_begin_event (evsel=0x30313e0, sample=0x3ffea4f77d0) at builtin-lock.c:1004
  #4  0x00000000010a00ae in evsel__process_contention_begin (evsel=0x30313e0, sample=0x3ffea4f77d0) at builtin-lock.c:1254
  #5  0x00000000010a0e14 in process_sample_event (tool=0x3ffea4f8480, event=0x3ff85601ef8, sample=0x3ffea4f77d0, evsel=0x30313e0, machine=0x3029e28) at builtin-lock.c:1464
  .....

The issue is in function machine__is_lock_function() in file
./util/machine.c lines 3355:

   /* should not fail from here */
   sym = machine__find_kernel_symbol_by_name(machine, "__sched_text_end", &kmap);
   machine->sched.text_end = kmap->unmap_ip(kmap, sym->start)

On s390 the symbol __sched_text_end is *NOT* in the symbol list and the
resulting pointer sym is set to NULL. The sym->start is then a NULL pointer
access and generates the core dump.

The reason why __sched_text_end is not in the symbol list on s390 is
simple:

When the symbol list is created at perf start up with function calls

  dso__load
  +--> dso__load_vmlinux_path
       +--> dso__load_vmlinux
            +--> dso__load_sym
	         +--> dso__load_sym_internal (reads kernel symbols)
		 +--> symbols__fixup_end
		 +--> symbols__fixup_duplicate

The issue is in function symbols__fixup_duplicate(). It deletes all
symbols with have the same address. On s390:

  # nm -g  ~/linux/vmlinux| fgrep c68390
  0000000000c68390 T __cpuidle_text_start
  0000000000c68390 T __sched_text_end
  #

two symbols have identical addresses and __sched_text_end is considered
duplicate (in ascending sort order) and removed from the symbol list.
Therefore it is missing and an invalid pointer reference occurs.  The
code checks for symbol __sched_text_start and when it exists assumes
symbol __sched_text_end is also in the symbol table. However this is not
the case on s390.

Same situation exists for symbol __lock_text_start:

0000000000c68770 T __cpuidle_text_end
0000000000c68770 T __lock_text_start

This symbol is also removed from the symbol table but used in function
machine__is_lock_function().

To fix this and keep duplicate symbols in the symbol table, set
symbol_conf.allow_aliases to true. This prevents the removal of
duplicate symbols in function symbols__fixup_duplicate().

Output After:

 # ./perf lock contention
 contended total wait  max wait  avg wait    type   caller

        48   124.39 ms 123.99 ms   2.59 ms rwsem:W unlink_anon_vmas+0x24a
        47    83.68 ms  83.26 ms   1.78 ms rwsem:W free_pgtables+0x132
         5    41.22 us  10.55 us   8.24 us rwsem:W free_pgtables+0x140
         4    40.12 us  20.55 us  10.03 us rwsem:W copy_process+0x1ac8
 #

Fixes: 0d2997f750 ("perf lock: Look up callchain for the contended locks")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Sumanth Korikkar <sumanthk@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Link: https://lore.kernel.org/r/20221230102627.2410847-1-tmricht@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2023-01-03 15:48:23 -03:00
..
arch perf tools riscv: Fix build error on riscv due to missing header for 'struct perf_sample' 2023-01-02 11:47:22 -03:00
bench parisc architecture fixes for kernel v6.2-rc1: 2022-12-20 08:43:53 -06:00
dlfilters perf tools: Fix usage of the verbose variable 2022-12-20 15:16:33 -03:00
Documentation perf tools: Don't include signature in version strings 2023-01-02 12:34:06 -03:00
examples/bpf perf trace: Remove unused bpf map 'syscalls' 2022-11-23 10:30:00 -03:00
include/perf perf bpf: Remove now unused BPF headers 2022-11-04 11:41:48 -03:00
jvmti
pmu-events perf vendor events amd: Add Zen 4 mapping 2022-12-21 14:52:42 -03:00
python perf python: Convert tracepoint.py example to python3 2022-04-01 16:19:35 -03:00
scripts perf script: task-analyzer add csv support 2022-12-14 11:24:31 -03:00
tests perf test record_probe_libc_inet_pton: Fix failure due to extra inet_pton() backtrace in glibc >= 2.35 2023-01-03 10:47:43 -03:00
trace perf tools: Use "grep -E" instead of "egrep" 2022-12-14 15:28:19 -03:00
ui perf tools: Fix "kernel lock contention analysis" test by not printing warnings in quiet mode 2022-10-27 16:37:26 -03:00
util perf tools: Fix segfault when trying to process tracepoints in perf.data and not linked with libtraceevent 2023-01-02 13:45:19 -03:00
.gitignore perf build: Install libsymbol locally when building 2022-11-16 12:17:15 -03:00
Build perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-annotate.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-bench.c perf bench: Add breakpoint benchmarks 2022-05-13 11:00:38 -03:00
builtin-buildid-cache.c perf record: Disable debuginfod by default 2022-01-15 17:41:25 -03:00
builtin-buildid-list.c perf buildid-list: Add a "-m" option to show kernel and modules build-ids 2022-07-18 16:35:34 -03:00
builtin-c2c.c perf tools: Use dedicated non-atomic clear/set bit helpers 2022-12-05 09:29:06 -03:00
builtin-config.c
builtin-daemon.c perf daemon: Use sig_atomic_t to avoid UB 2022-11-03 09:35:44 -03:00
builtin-data.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-diff.c perf tools: Make quiet mode consistent between tools 2022-10-27 16:37:26 -03:00
builtin-evlist.c
builtin-ftrace.c perf ftrace: Use sig_atomic_t to avoid UB 2022-11-03 09:36:09 -03:00
builtin-help.c
builtin-inject.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-kallsyms.c
builtin-kmem.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-kvm.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-kwork.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-list.c perf list: JSON escape encoding improvements 2022-11-24 10:03:03 -03:00
builtin-lock.c perf lock contention: Fix core dump related to not finding the "__sched_text_end" symbol on s/390 2023-01-03 15:48:23 -03:00
builtin-mem.c perf tools: Move 'struct perf_sample' to a separate header file to disentangle headers 2022-10-31 11:06:41 -03:00
builtin-probe.c perf probe: Check -v and -q options in the right place 2022-12-20 15:16:33 -03:00
builtin-record.c perf tools: Fix usage of the verbose variable 2022-12-20 15:16:33 -03:00
builtin-report.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-sched.c perf tools: Use dedicated non-atomic clear/set bit helpers 2022-12-05 09:29:06 -03:00
builtin-script.c perf tools: Fix usage of the verbose variable 2022-12-20 15:16:33 -03:00
builtin-stat.c perf tools: Fix usage of the verbose variable 2022-12-20 15:16:33 -03:00
builtin-timechart.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin-top.c perf evlist: Remove group option. 2022-12-14 15:28:18 -03:00
builtin-trace.c perf tools: Use "grep -E" instead of "egrep" 2022-12-14 15:28:19 -03:00
builtin-version.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
builtin.h perf kwork: New tool to trace time properties of kernel work (such as softirq, and workqueue) 2022-07-26 16:01:24 -03:00
check-headers.sh tools headers: Update the copy of x86's memcpy_64.S used in 'perf bench' 2022-10-25 17:40:48 -03:00
command-list.txt perf help: Use HAVE_LIBTRACEEVENT to filter out unsupported commands 2023-01-02 11:51:53 -03:00
CREDITS
design.txt perf design.txt: Synchronize the definition of enum perf_hw_id with code 2021-11-13 18:11:50 -03:00
Makefile perf tools: Use "grep -E" instead of "egrep" 2022-12-14 15:28:19 -03:00
Makefile.config perf build: Remove explicit reference to python 2.x devel files 2022-12-21 14:52:42 -03:00
Makefile.perf perf build: Don't propagate subdir to submakes for install_headers 2023-01-03 15:35:01 -03:00
MANIFEST tools lib traceevent: Remove libtraceevent 2022-12-14 11:16:12 -03:00
perf-archive.sh
perf-completion.sh
perf-iostat.sh
perf-read-vdso.c
perf-sys.h
perf.c perf build: Use libtraceevent from the system 2022-12-14 11:16:12 -03:00
perf.h