Powerpc has Global Entry Point and Local Entry Point for functions. LEP
catches call from both the GEP and the LEP. Symbol table of ELF contains
GEP and Offset from which we can calculate LEP, but debuginfo does not
have LEP info.
Currently, perf prioritize symbol table over dwarf to probe on LEP for
ppc64le. But when user tries to probe with function parameter, we fall
back to using dwarf(i.e. GEP) and when function called via LEP, probe
will never hit.
For example:
$ objdump -d vmlinux
...
do_sys_open():
c0000000002eb4a0: e8 00 4c 3c addis r2,r12,232
c0000000002eb4a4: 60 00 42 38 addi r2,r2,96
c0000000002eb4a8: a6 02 08 7c mflr r0
c0000000002eb4ac: d0 ff 41 fb std r26,-48(r1)
$ sudo ./perf probe do_sys_open
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/do_sys_open _text+3060904
$ sudo ./perf probe 'do_sys_open filename:string'
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/do_sys_open _text+3060896 filename_string=+0(%gpr4):string
For second case, perf probed on GEP. So when function will be called via
LEP, probe won't hit.
$ sudo ./perf record -a -e probe:do_sys_open ls
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.195 MB perf.data ]
To resolve this issue, let's not prioritize symbol table, let perf
decide what it wants to use. Perf is already converting GEP to LEP when
it uses symbol table. When perf uses debuginfo, let it find LEP offset
form symbol table. This way we fall back to probe on LEP for all cases.
After patch:
$ sudo ./perf probe 'do_sys_open filename:string'
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/do_sys_open _text+3060904 filename_string=+0(%gpr4):string
$ sudo ./perf record -a -e probe:do_sys_open ls
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.197 MB perf.data (11 samples) ]
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1470723805-5081-2-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Instead of inline code, introduce function to post process kernel
probe trace events.
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1470723805-5081-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If module is "module" then dso->short_name is "[module]". Substring
comparing is't enough: "raid10" matches to "[raid1]". This patch also
checks terminating zero in module name.
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Link: http://lkml.kernel.org/r/147039975648.715620.12985971832789032159.stgit@buzz
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Adjust map->reloc offset for the unmapped address when finding
alternative symbol address from map, because KASLR can relocate the
kernel symbol address.
The same adjustment has been done when finding appropriate kernel symbol
address from map which was introduced by commit f90acac757 ("perf
probe: Find given address from offline dwarf")
Reported-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu@linaro.org>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20160806192948.e366f3fbc4b194de600f8326@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Warn unmatched function filter correctly instead of warning
"symbol-loading error", since that can be a filter issue.
From the technical point of view, this adds a filter chech in map__load
and if there is a filter, it returns -2 (filter-out), instead of -1
(error), and perf-probe checks it and change message.
E.g. without this fix:
# perf probe -F rt_sp*
no symbols found in [kernel.kallsyms], maybe install a debug package?
Failed to load symbols in kernel
With this fix:
# perf probe -F rt_sp*
no symbols passed the given filter.
Failed to find symbols matched to "rt_sp*"
Error: Failed to show functions.
Reported-and-Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146885835596.16106.2293540792775552481.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Support a special SDT probe format which can omit the '%' prefix only if
the SDT group name starts with "sdt_". So, for example both of
"%sdt_libc:setjump" and "sdt_libc:setjump" are acceptable for perf probe
--add.
E.g. without this:
# perf probe -a sdt_libc:setjmp
Semantic error :There is non-digit char in line number.
...
With this:
# perf probe -a sdt_libc:setjmp
Added new event:
sdt_libc:setjmp (on %setjmp in /usr/lib64/libc-2.20.so)
You can now use it in all perf tools, such as:
perf record -e sdt_libc:setjmp -aR sleep 1
Suggested-by: Brendan Gregg <brendan.d.gregg@gmail.com>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831794674.17065.13359473252168740430.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Support @BUILDID or @FILE suffix for SDT events. This allows perf to add
probes on SDTs/pre-cached events on given FILE or the file which has
given BUILDID (also, this complements BUILDID.)
For example, both gcc and libstdc++ has same SDTs as below. If you
would like to add a probe on sdt_libstdcxx:catch on gcc, you can do as
below.
----
# perf list sdt | tail -n 6
sdt_libstdcxx:catch@/usr/bin/gcc(0cc207fc4b27) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.20(91c7a88fdf49)
sdt_libstdcxx:rethrow@/usr/bin/gcc(0cc207fc4b27) [SDT event]
sdt_libstdcxx:rethrow@/usr/lib64/libstdc++.so.6.0.20(91c7a88fdf49)
sdt_libstdcxx:throw@/usr/bin/gcc(0cc207fc4b27) [SDT event]
sdt_libstdcxx:throw@/usr/lib64/libstdc++.so.6.0.20(91c7a88fdf49)
# perf probe -a %sdt_libstdcxx:catch@0cc
Added new event:
sdt_libstdcxx:catch (on %catch in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
----
Committer note:
Doing the full sequence of steps to get the results above:
With a clean build-id cache:
[root@jouet ~]# rm -rf ~/.debug/
[root@jouet ~]# perf list sdt
List of pre-defined events (to be used in -e):
[root@jouet ~]#
No events whatsoever, then, we can add all events in gcc to the build-id
cache, doing a --add + --dry-run:
[root@jouet ~]# perf probe --dry-run --cache -x /usr/bin/gcc --add %sdt_libstdcxx:\*
Added new events:
sdt_libstdcxx:throw (on %* in /usr/bin/gcc)
sdt_libstdcxx:rethrow (on %* in /usr/bin/gcc)
sdt_libstdcxx:catch (on %* in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
[root@jouet ~]#
It really didn't add any events, it just cached them:
[root@jouet ~]# perf probe -l
[root@jouet ~]#
We can see that it was cached as:
[root@jouet ~]# ls -la ~/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/
total 976
drwxr-xr-x. 2 root root 4096 Jul 13 21:47 .
drwxr-xr-x. 3 root root 4096 Jul 13 21:47 ..
-rwxr-xr-x. 4 root root 985912 Jun 22 18:52 elf
-rw-r--r--. 1 root root 303 Jul 13 21:47 probes
[root@jouet ~]# file ~/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/elf
/root/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/elf: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2, stripped
[root@jouet ~]# cat ~/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/probes
%sdt_libstdcxx:throw=throw
p:sdt_libstdcxx/throw /usr/bin/gcc:0x71ffd
%sdt_libstdcxx:rethrow=rethrow
p:sdt_libstdcxx/rethrow /usr/bin/gcc:0x720b8
%sdt_libstdcxx:catch=catch
p:sdt_libstdcxx/catch /usr/bin/gcc:0x7307f
%sdt_libgcc:unwind=unwind
p:sdt_libgcc/unwind /usr/bin/gcc:0x7eec0
#sdt_libstdcxx:*=%*
[root@jouet ~]#
Ok, now we can use 'perf probe' to refer to those cached entries as:
Humm, nope, doing as above we end up with:
[root@jouet ~]# perf probe -a %sdt_libstdcxx:catch
Semantic error :* is bad for event name -it must follow C symbol-naming rule.
Error: Failed to add events.
[root@jouet ~]#
But it worked at some point, lets try not using --dry-run:
Resetting everything:
# rm -rf ~/.debug/
# perf probe -d *:*
# perf probe -l
# perf list sdt
List of pre-defined events (to be used in -e):
#
Ok, now it cached everything, even things we haven't asked it to
(sdt_libgcc:unwind):
[root@jouet ~]# perf probe -x /usr/bin/gcc --add %sdt_libstdcxx:\*
Added new events:
sdt_libstdcxx:throw (on %* in /usr/bin/gcc)
sdt_libstdcxx:rethrow (on %* in /usr/bin/gcc)
sdt_libstdcxx:catch (on %* in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
[root@jouet ~]# perf list sdt
List of pre-defined events (to be used in -e):
sdt_libgcc:unwind [SDT event]
sdt_libstdcxx:catch [SDT event]
sdt_libstdcxx:rethrow [SDT event]
sdt_libstdcxx:throw [SDT event]
[root@jouet ~]#
And we have the events in place:
[root@jouet ~]# perf probe -l
sdt_libstdcxx:catch (on execute_cfa_program+1551@../../../libgcc/unwind-dw2.c in /usr/bin/gcc)
sdt_libstdcxx:rethrow (on d_print_subexpr+280@libsupc++/cp-demangle.c in /usr/bin/gcc)
sdt_libstdcxx:throw (on d_print_subexpr+93@libsupc++/cp-demangle.c in /usr/bin/gcc)
[root@jouet ~]#
And trying to use them at least has 'perf trace --event sdt*:*' working.
Then, if we try to add the ones in libstdc++:
[root@jouet ~]# perf probe -x /usr/lib64/libstdc++.so.6 -a %sdt_libstdcxx:\*
Error: event "catch" already exists.
Hint: Remove existing event by 'perf probe -d'
or force duplicates by 'perf probe -f'
or set 'force=yes' in BPF source.
Error: Failed to add events.
[root@jouet ~]#
Doesn't work, dups, but at least this served to, unbeknownst to the user, add
the SDT probes in /usr/lib64/libstdc++.so.6!
[root@jouet ~]# perf list sdt
List of pre-defined events (to be used in -e):
sdt_libgcc:unwind [SDT event]
sdt_libstdcxx:catch@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
sdt_libstdcxx:rethrow@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:rethrow@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
sdt_libstdcxx:throw@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:throw@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
[root@jouet ~]#
Now we should be able to get to the original cset comment, if we remove all
SDTs events in place, not from the cache, from the kernel, where it was set up as:
[root@jouet ~]# ls -la /sys/kernel/debug/tracing/events/sdt_libstdcxx/
total 0
drwxr-xr-x. 5 root root 0 Jul 13 22:00 .
drwxr-xr-x. 80 root root 0 Jul 13 21:56 ..
drwxr-xr-x. 2 root root 0 Jul 13 22:00 catch
-rw-r--r--. 1 root root 0 Jul 13 22:00 enable
-rw-r--r--. 1 root root 0 Jul 13 22:00 filter
drwxr-xr-x. 2 root root 0 Jul 13 22:00 rethrow
drwxr-xr-x. 2 root root 0 Jul 13 22:00 throw
[root@jouet ~]#
[root@jouet ~]# head -2 /sys/kernel/debug/tracing/events/sdt_libstdcxx/throw/format
name: throw
ID: 2059
[root@jouet ~]#
Now to remove it:
[root@jouet ~]# perf probe -d sdt_libstdc*:*
Removed event: sdt_libstdcxx:catch
Removed event: sdt_libstdcxx:rethrow
Removed event: sdt_libstdcxx:throw
[root@jouet ~]#
Which caused:
[root@jouet ~]# ls -la /sys/kernel/debug/tracing/events/sdt_libstdcxx/
ls: cannot access '/sys/kernel/debug/tracing/events/sdt_libstdcxx/': No such file or directory
[root@jouet ~]#
Ok, now we can do:
[root@jouet ~]# perf list sdt_libstdcxx:catch
List of pre-defined events (to be used in -e):
sdt_libstdcxx:catch@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
[root@jouet ~]#
So, these are not really 'pre-defined events', i.e. we can't use them with
'perf record --event':
[root@jouet ~]# perf record --event sdt_libstdcxx:catch*
event syntax error: 'sdt_libstdcxx:catch*'
\___ unknown tracepoint
Error: File /sys/kernel/debug/tracing/events/sdt_libstdcxx/catch* not found.
Hint: Perhaps this kernel misses some CONFIG_ setting to enable this feature?.
<SNIP>
[root@jouet ~]#
To have it really pre-defined we must use perf probe to get its definition from
the cache and set it up in the kernel, creating the tracepoint to _then_ use it
with 'perf record --event':
[root@jouet ~]# perf probe -a sdt_libstdcxx:catch
Semantic error :There is non-digit char in line number.
<SNIP>
Oops, there is another gotcha here, we need that pesky '%' character:
[root@jouet ~]# perf probe -a %sdt_libstdcxx:catch
Added new events:
sdt_libstdcxx:catch (on %catch in /usr/bin/gcc)
sdt_libstdcxx:catch_1 (on %catch in /usr/lib64/libstdc++.so.6.0.22)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch_1 -aR sleep 1
[root@jouet ~]#
But then we added _two_ events, one with the name we expected, the other one
with a _ added, when doing the analysis we need to pay attention to who maps to
who.
And here is where we get to the point of this patch, which is to be able to
disambiguate those definitions for 'catch' in the build-id cache, but first we need
remove those events we just added:
[root@jouet ~]# perf probe -d %sdt_libstdcxx:catch
Oops, that didn't remove anything, we need to _remove_ that % char in this case:
[root@jouet ~]# perf probe -d sdt_libstdcxx:catch
Removed event: sdt_libstdcxx:catch
And we need to remove the other event added, i.e. I forgot to add a * at the end:
[root@jouet ~]# perf probe -d sdt_libstdcxx:catch*
Removed event: sdt_libstdcxx:catch_1
[root@jouet ~]#
Ok, disambiguating it using what is in this patch:
[root@jouet ~]# perf list sdt_libstdcxx:catch
List of pre-defined events (to be used in -e):
sdt_libstdcxx:catch@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
[root@jouet ~]#
[root@jouet ~]# perf probe -a %sdt_libstdcxx:catch@9a07
Added new event:
sdt_libstdcxx:catch (on %catch in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
[root@jouet ~]# perf probe -l
sdt_libstdcxx:catch (on execute_cfa_program+1551@../../../libgcc/unwind-dw2.c in /usr/bin/gcc)
[root@jouet ~]#
Yeah, it works! But we need to try and simplify this :-)
Update: Some aspects of this simplification take place in the following
patches.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831793746.17065.13065062753978236612.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Search SDT/cached event from all probe caches if user doesn't pass any
binary. With this, we don't have to specify target binary for SDT and
named cached events (which start with %).
E.g. without this, a target binary must be passed with -x.
# perf probe -x /usr/lib64/libc-2.20.so -a %sdt_libc:\*
With this change, we don't need it anymore.
# perf probe -a %sdt_libc:\*
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831792812.17065.2353705982669445313.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allo glob wildcard for reusing cached/SDT events. E.g.
# perf probe -x /usr/lib64/libc-2.20.so -a %sdt_libc:\*
This example adds probes for all SDT in libc.
Note that the SDTs must have been scanned by perf buildid-cache.
Committer note:
Using it to check what of those SDT probes would take place when doing
a cargo run (rust):
# trace --no-sys --event sdt_libc:* cargo run
0.000 sdt_libc:setjmp:(7f326b69c4d1))
28.423 sdt_libc:setjmp:(7f4b0a5364d1))
29.000 sdt_libc:setjmp:(7f4b0a5364d1))
88.597 sdt_libc:setjmp:(7fc01fd414d1))
89.220 sdt_libc:setjmp:(7fc01fd414d1))
95.501 sdt_libc:setjmp:(7f326b69c4d1))
Running `target/debug/hello_world`
97.110 sdt_libc:setjmp:(7f95e09234d1))
Hello, world!
#
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831791813.17065.17846564230840594888.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To improve usability, support %[PROVIDER:]SDTEVENT format to add new
probes on SDT and cached events.
e.g.
----
# perf probe -x /lib/libc-2.17.so %lll_lock_wait_private
Added new event:
sdt_libc:lll_lock_wait_private (on %lll_lock_wait_private in /usr/lib/libc-2.17.so)
You can now use it in all perf tools, such as:
perf record -e sdt_libc:lll_lock_wait_private -aR sleep 1
# perf probe -l | more
sdt_libc:lll_lock_wait_private (on __lll_lock_wait_private+21 in /usr/lib/libc-2.17.so)
----
Note that this is not only for SDT events, but also normal
events with event-name.
e.g. define "myevent" on cache (-n doesn't add the real probe)
----
# perf probe -x ./perf --cache -n --add 'myevent=dso__load $params'
----
Reuse the "myevent" from cache as below.
----
# perf probe -x ./perf %myevent
----
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831788372.17065.3645054540325909346.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix to show correct error messages for $vars and $params because
those special variables requires debug information to find the
real variables or function parameters.
E.g. without this fix;
----
# perf probe -x /lib64/libc-2.23.so getaddrinfo \$params
Failed to write event: Invalid argument
Please upgrade your kernel to at least 3.14 to have access to feature $params
Error: Failed to add events.
----
Perf ends up with an error, but the message is not correct. With this
fix, perf shows correct error message as below.
----
# perf probe -x /lib64/libc-2.23.so getaddrinfo \$params
The /usr/lib64/libc-2.23.so file has no debug information.
Rebuild with -g, or install an appropriate debuginfo package.
Error: Failed to add events.
----
Reported-and-Tested-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831787438.17065.6152436996780110699.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The tools so far have been using the strerror_r() GNU variant, that
returns a string, be it the buffer passed or something else.
But that, besides being tricky in cases where we expect that the
function using strerror_r() returns the error formatted in a provided
buffer (we have to check if it returned something else and copy that
instead), breaks the build on systems not using glibc, like Alpine
Linux, where musl libc is used.
So, introduce yet another wrapper, str_error_r(), that has the GNU
interface, but uses the portable XSI variant of strerror_r(), so that
users rest asured that the provided buffer is used and it is what is
returned.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-d4t42fnf48ytlk8rjxs822tf@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allow user to set group name for adding new event. Note that user must
ensure that the group name doesn't conflict with existing group name
carefully.
E.g. Existing group name can conflict with other events. Especially,
using the group name reserved for kernel modules can hide kernel
embedded events when loading modules.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146736024091.27797.9471545190066268995.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Before analyzing debuginfo, try to find a corresponding entry from probe
cache always. This does not depend on --cache, the --cache enables to
store/update cache, but looking up the cache is always enabled.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146736019226.27797.16366402884098398857.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To match the semantics for list.h in the kernel, that are the
interface we use in them.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Taeung Song <treeze.taeung@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-mdp1heu9xjjc12zebh91232l@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To match the semantics for list.h in the kernel, that are the
interface we use in them.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Taeung Song <treeze.taeung@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-0b5i2ki9c3di6706fxpticsb@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
And do nothing, just like free(), to avoid having to test it in callers,
usually in error paths.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-q42gj3b3znhho9z1mrbo4jce@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add --cache option to cache the probe definitions. This just saves the
result of the dwarf analysis to probe cache.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160615032840.31330.44412.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Uncomment and export synthesize_perf_probe_point() which had once
introduced but has been disabled for a long time. This renews the code
and re-enable it.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160608092949.3116.21958.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add perf_probe_event__copy() to copy perf_probe_event data structure and
sub data structures under given source perf_probe_event.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160608092940.3116.18034.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Check the return value of strbuf APIs in perf-probe
related code, so that it can handle errors in strbuf.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160510054707.6158.69861.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
ppc64le functions have a Global Entry Point (GEP) and a Local Entry
Point (LEP). While placing a probe, we always prefer the LEP since it
catches function calls through both the GEP and the LEP. In order to do
this, we fixup the function entry points during elf symbol table lookup
to point to the LEPs. This works, but breaks 'perf test kallsyms' since
the symbols loaded from the symbol table (pointing to the LEP) do not
match the symbols in kallsyms.
To fix this, we do not adjust all the symbols during symbol table load.
Instead, we note down st_other in a newly introduced arch-specific
member of perf symbol structure, and later use this to adjust the probe
trace point.
Reported-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Acked-by: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Acked-by: Balbir Singh <bsingharora@gmail.com>
Cc: Mark Wielaard <mjw@redhat.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Thiago Jung Bauermann <bauerman@linux.vnet.ibm.com>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/r/6be7c2b17e370100c2f79dd444509df7929bdd3e.1460451721.git.naveen.n.rao@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Perf is not able to register probe in kernel module when dwarf supprt
is not there(and so it goes for symtab). Perf passes full path of
module where only module name is required which is causing the problem.
This patch fixes this issue.
Before applying patch:
$ dpkg -s libdw-dev
dpkg-query: package 'libdw-dev' is not installed and no information is...
$ sudo ./perf probe -m /linux/samples/kprobes/kprobe_example.ko kprobe_init
Added new event:
probe:kprobe_init (on kprobe_init in /linux/samples/kprobes/kprobe_example.ko)
You can now use it in all perf tools, such as:
perf record -e probe:kprobe_init -aR sleep 1
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/kprobe_init /linux/samples/kprobes/kprobe_example.ko:kprobe_init
$ sudo ./perf record -a -e probe:kprobe_init
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.105 MB perf.data ]
$ sudo ./perf script # No output here
After applying patch:
$ sudo ./perf probe -m /linux/samples/kprobes/kprobe_example.ko kprobe_init
Added new event:
probe:kprobe_init (on kprobe_init in kprobe_example)
You can now use it in all perf tools, such as:
perf record -e probe:kprobe_init -aR sleep 1
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/kprobe_init kprobe_example:kprobe_init
$ sudo ./perf record -a -e probe:kprobe_init
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.105 MB perf.data (2 samples) ]
$ sudo ./perf script
insmod 13990 [002] 5961.216833: probe:kprobe_init: ...
insmod 13995 [002] 5962.889384: probe:kprobe_init: ...
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1461680741-12517-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Perf can add a probe on kernel module which has not been loaded yet.
The current implementation finds the module name from path. But if the
filename is different from the actual module name then perf fails to
register a probe while loading module because of mismatch in the names.
For example, samples/kobject/kobject-example.ko is loaded as
kobject_example.
Before applying patch:
$ sudo ./perf probe -m /linux/samples/kobject/kobject-example.ko foo_show
Added new event:
probe:foo_show (on foo_show in kobject-example)
You can now use it in all perf tools, such as:
perf record -e probe:foo_show -aR sleep 1
$ cat /sys/kernel/debug/tracing/kprobe_events
p:probe/foo_show kobject-example:foo_show
$ insmod kobject-example.ko
$ lsmod
Module Size Used by
kobject_example 16384 0
Generate read to /sys/kernel/kobject_example/foo while recording data
with below command
$ sudo ./perf record -e probe:foo_show -a
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.093 MB perf.data ]
$./perf report --stdio -F overhead,comm,dso,sym
Error:
The perf.data.old file has no samples!
After applying patch:
$ sudo ./perf probe -m /linux/samples/kobject/kobject-example.ko foo_show
Added new event:
probe:foo_show (on foo_show in kobject_example)
You can now use it in all perf tools, such as:
perf record -e probe:foo_show -aR sleep 1
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/foo_show kobject_example:foo_show
$ insmod kobject-example.ko
$ lsmod
Module Size Used by
kobject_example 16384 0
Generate read to /sys/kernel/kobject_example/foo while recording data
with below command
$ sudo ./perf record -e probe:foo_show -a
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.097 MB perf.data (8 samples) ]
$ sudo ./perf report --stdio -F overhead,comm,dso,sym
...
# Samples: 8 of event 'probe:foo_show'
# Event count (approx.): 8
#
# Overhead Command Shared Object Symbol
# ........ ....... ................. ............
#
100.00% cat [kobject_example] [k] foo_show
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1461680741-12517-2-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Set kprobe group name as "probe" if it is not given.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160426090413.11891.95640.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix a bug to close target elf file in get_text_start_address().
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160426064737.1443.44093.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We have addch() for chars, add() for fixed size data, and addstr() for
variable length strings, use them.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-0ap02fn2xtvpduj2j6b2o1j4@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
By extending the syntax of BPF object section names, this patch allows users to
config probing options like what they can do in 'perf probe'.
The error message in 'perf probe' is also updated.
Test result:
For following BPF file test_probe_glob.c:
# cat test_probe_glob.c
__attribute__((section("inlines=no;func=SyS_dup?"), used))
int func(void *ctx)
{
return 1;
}
char _license[] __attribute__((section("license"), used)) = "GPL";
int _version __attribute__((section("version"), used)) = 0x40300;
#
# ./perf record -e ./test_probe_glob.c ls /
...
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.013 MB perf.data ]
# ./perf evlist
perf_bpf_probe:func_1
perf_bpf_probe:func
After changing "inlines=no" to "inlines=yes":
# ./perf record -e ./test_probe_glob.c ls /
...
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.013 MB perf.data ]
# ./perf evlist
perf_bpf_probe:func_3
perf_bpf_probe:func_2
perf_bpf_probe:func_1
perf_bpf_probe:func
Then test 'force':
Use following program:
# cat test_probe_force.c
__attribute__((section("func=sys_write"), used))
int funca(void *ctx)
{
return 1;
}
__attribute__((section("force=yes;func=sys_write"), used))
int funcb(void *ctx)
{
return 1;
}
char _license[] __attribute__((section("license"), used)) = "GPL";
int _version __attribute__((section("version"), used)) = 0x40300;
#
# perf record -e ./test_probe_force.c usleep 1
Error: event "func" already exists.
Hint: Remove existing event by 'perf probe -d'
or force duplicates by 'perf probe -f'
or set 'force=yes' in BPF source.
event syntax error: './test_probe_force.c'
\___ Probe point exist. Try 'perf probe -d "*"' and set 'force=yes'
(add -v to see detail)
...
Then replace 'force=no' to 'force=yes':
# vim test_probe_force.c
# perf record -e ./test_probe_force.c usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.017 MB perf.data ]
# perf evlist
perf_bpf_probe:func_1
perf_bpf_probe:func
#
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1447675815-166222-7-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In find_perf_probe_point_from_map(), the 'ret' variable is initialized
with -ENOENT but overwritten by the return code of
kernel_get_symbol_address_by_name(), and after that it is re-initialized
with -ENOENT again.
Setting ret=-ENOENT twice looks a bit redundant. This avoids the
overwriting and just returns -ENOENT if some error happens to simplify
the code.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: Zefan Li <lizefan@huawei.com>
Link: http://lkml.kernel.org/n/tip-ufp1zgbktzmttcputozneomd@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It is possible that find_perf_probe_point_from_map() fails to find a
symbol but still returns 0 because of an small error when coding:
find_perf_probe_point_from_map() set 'ret' to error code at first, but
also use it to hold return value of kernel_get_symbol_address_by_name().
This patch resets 'ret' to error even kernel_get_symbol_address_by_name()
success, so if !sym, the whole function returns error correctly.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1446729565-27592-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allow probing on kernel modules when 'perf' is built without debuginfo
support.
Currently perf-probe --module requires linking with libdw, but this
doesn't make sense.
E.g.
----
# make NO_DWARF=1
# ./perf probe -m pcspkr pcspkr_event%return
Error: unknown switch `m'
----
With this patch
----
# ./perf probe -m pcspkr pcspkr_event%return
Added new event:
probe:pcspkr_event (on pcspkr_event%return in pcspkr)
You can now use it in all perf tools, such as:
perf record -e probe:pcspkr_event -aR sleep 1
----
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20151002125832.18617.78721.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
perf probe --list will get a segfault if the first kprobe event is on a
module and the second or latter one is on the kernel.
e.g.
----
# ./perf probe -q -m pcspkr pcspkr_event
# ./perf probe -q vfs_read
# ./perf probe -l
Segmentation fault (core dumped)
----
This is because the debuginfo_cache fails to handle NULL module name,
which causes segfault on strcmp. (Note that strcmp("something", NULL)
always causes segfault)
To fix this debuginfo_cache__open always translates the NULL module name
to "kernel" (this is correct, because NULL module name means opening the
debuginfo for the kernel)
----
# ./perf probe -l
probe:pcspkr_event (on pcspkr_event@drivers/input/misc/pcspkr.c
in pcspkr)
probe:vfs_read (on vfs_read@ksrc/linux-3/fs/read_write.c)
----
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20150930164135.3733.23993.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Perf probe always failed to find appropriate line numbers because of
failing to find .text start address offset from debuginfo.
e.g.
----
# ./perf probe -m pcspkr pcspkr_event:5
Added new events:
probe:pcspkr_event (on pcspkr_event:5 in pcspkr)
probe:pcspkr_event_1 (on pcspkr_event:5 in pcspkr)
You can now use it in all perf tools, such as:
perf record -e probe:pcspkr_event_1 -aR sleep 1
# ./perf probe -l
Failed to find debug information for address ffffffffa031f006
Failed to find debug information for address ffffffffa031f016
probe:pcspkr_event (on pcspkr_event+6 in pcspkr)
probe:pcspkr_event_1 (on pcspkr_event+22 in pcspkr)
----
This fixes the above issue as below.
1. Get the relative address of the symbol in .text by using
map->start.
2. Adjust the address by adding the offset of .text section
in the kernel module binary.
With this fix, perf probe -l shows lines correctly.
----
# ./perf probe -l
probe:pcspkr_event (on pcspkr_event:5@drivers/input/misc/pcspkr.c in pcspkr)
probe:pcspkr_event_1 (on pcspkr_event:5@drivers/input/misc/pcspkr.c in pcspkr)
----
Reported-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20150930164132.3733.24643.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix to remove dot suffix (e.g. .const, .isra) from the second or latter
events which has suffix numbers.
Since the previous commit 35a23ff928 ("perf probe: Cut off the gcc
optimization postfixes from function name") didn't care about the suffix
numbered events, therefore we'll have an error when we add additional
events on the same dot suffix functions.
e.g.
----
# ./perf probe -f -a get_sigframe.isra.2.constprop.3 \
-a get_sigframe.isra.2.constprop.3
Failed to write event: Invalid argument
Error: Failed to add events.
----
This fixes above issue as below:
----
# ./perf probe -f -a get_sigframe.isra.2.constprop.3 \
-a get_sigframe.isra.2.constprop.3
Added new events:
probe:get_sigframe (on get_sigframe.isra.2.constprop.3)
probe:get_sigframe_1 (on get_sigframe.isra.2.constprop.3)
You can now use it in all perf tools, such as:
perf record -e probe:get_sigframe_1 -aR sleep 1
----
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20150930164130.3733.26573.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
And it is also a step in the direction of killing the separation of data
and text maps in map_groups.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-rrds86kb3wx5wk8v38v56gw8@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In places where we were using its open coded equivalent.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-khkdugcdoqy3tkszm3jdxgbe@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We have map_groups__find_by_name() to look at the list of modules that
are in place for a given machine, so use it instead of traversing the
machine dso list, which also includes DSOs for userspace.
When merging the user and kernel DSO lists a bug was introduced where
'perf probe' stopped being able to add probes to modules using its short
name:
# perf probe -m usbnet --add usbnet_start_xmit
usbnet_start_xmit is out of .text, skip it.
Error: Failed to add events.
#
With this fix it works again:
# perf probe -m usbnet --add usbnet_start_xmit
Added new event:
probe:usbnet_start_xmit (on usbnet_start_xmit in usbnet)
You can now use it in all perf tools, such as:
perf record -e probe:usbnet_start_xmit -aR sleep 1
#
Reported-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Stephane Eranian <eranian@google.com>
Fixes: 3d39ac5386 ("perf machine: No need to have two DSOs lists")
Link: http://lkml.kernel.org/r/20150924015008.GE1897@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The init/exit_symbols_maps() functions are to setup and cleanup
necessary info for probe events. But they need to be called from out of
the probe code now, so this patch exports them.
However the names are too generic, so change them to have 'probe'. :)
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1441852026-28974-2-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The cleanup_perf_probe_events() frees all resources related to a perf
probe event. However it only freed resources in trace probe events, not
perf probe events. So call clear_perf_probe_event() too.
Reported-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1441852026-28974-1-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Switching to the fs.c related filesystem framework.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Raphael Beamonte <raphael.beamonte@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/1441180605-24737-14-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Showing actual trace event when deleteing perf events is only needed in
perf probe command. But the add functionality itself can be used by
other places. So move the printing code into the cmd_probe().
The output is not changed.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1441368963-11565-5-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Showing actual trace event when adding perf events is only needed in
perf probe command. But the add functionality itself can be used by
other places. So move the printing code into the cmd_probe().
Also it combines the output if more than one event is added.
Before:
$ sudo perf probe -a do_fork -a do_exit
Added new event:
probe:do_fork (on do_fork)
You can now use it in all perf tools, such as:
perf record -e probe:do_fork -aR sleep 1
Added new events:
probe:do_exit (on do_exit)
probe:do_exit_1 (on do_exit)
You can now use it in all perf tools, such as:
perf record -e probe:do_exit_1 -aR sleep 1
After:
$ sudo perf probe -a do_fork -a do_exit
Added new events:
probe:do_fork (on do_fork)
probe:do_exit (on do_exit)
probe:do_exit_1 (on do_exit)
You can now use it in all perf tools, such as:
perf record -e probe:do_exit_1 -aR sleep 1
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1441368963-11565-3-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch drops struct __event_package structure. Instead, it adds a
'struct trace_probe_event' pointer to 'struct perf_probe_event'.
The trace_probe_event information gives further patches a chance to
access actual probe points and actual arguments.
Using them, 'perf probe' can get the whole list of added probes and
print them at once.
Other users like the upcoming bpf_loader will be able to attach one bpf
program to different probing points of an inline function (which has
multiple probing points) and glob functions.
Moreover, by reading the arguments information, bpf code for reading
those arguments can be generated.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1441368963-11565-2-git-send-email-namhyung@kernel.org
[namhyung: extract necessary part from the existing patch]
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The add_perf_probe_events() does 3 things:
1. convert all perf events to trace events
2. add all trace events to kernel
3. cleanup all trace events
But sometimes we need to do something with the trace events. So split
the funtion into three, so that it can access intermediate trace events
via struct __event_package if needed.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1441368963-11565-1-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It should be useful to allow 'perf probe' probe at absolute offset of a
target. For example, when (u)probing at a instruction of a shared object
in a embedded system where debuginfo is not avaliable but we know the
offset of that instruction by manually digging.
This patch enables following perf probe command syntax:
# perf probe 0xffffffff811e6615
And
# perf probe /lib/x86_64-linux-gnu/libc-2.19.so 0xeb860
In the above example, we don't need a anchor symbol, so it is possible
to compute absolute addresses using other methods and then use 'perf
probe' to create the probing points.
v1 -> v2:
Drop the leading '+' in cmdline;
Allow uprobing at offset 0x0;
Improve 'perf probe -l' result when uprobe at area without debuginfo.
v2 -> v3:
Split bugfix to a separated patch.
Test result:
# perf probe 0xffffffff8119d175 %ax
# perf probe sys_write %ax
# perf probe /lib64/libc-2.18.so 0x0 %ax
# perf probe /lib64/libc-2.18.so 0x5 %ax
# perf probe /lib64/libc-2.18.so 0xd8e40 %ax
# perf probe /lib64/libc-2.18.so __write %ax
# perf probe /lib64/libc-2.18.so 0xd8e49 %ax
# cat /sys/kernel/debug/tracing/uprobe_events
p:probe_libc/abs_0 /lib64/libc-2.18.so:0x (null) arg1=%ax
p:probe_libc/abs_5 /lib64/libc-2.18.so:0x0000000000000005 arg1=%ax
p:probe_libc/abs_d8e40 /lib64/libc-2.18.so:0x00000000000d8e40 arg1=%ax
p:probe_libc/__write /lib64/libc-2.18.so:0x00000000000d8e40 arg1=%ax
p:probe_libc/abs_d8e49 /lib64/libc-2.18.so:0x00000000000d8e49 arg1=%ax
# cat /sys/kernel/debug/tracing/kprobe_events
p:probe/abs_ffffffff8119d175 0xffffffff8119d175 arg1=%ax
p:probe/sys_write _text+1692016 arg1=%ax
# perf probe -l
Failed to find debug information for address 5
probe:abs_ffffffff8119d175 (on sys_write+5 with arg1)
probe:sys_write (on sys_write with arg1)
probe_libc:__write (on @unix/syscall-template.S:81 in /lib64/libc-2.18.so with arg1)
probe_libc:abs_0 (on 0x0 in /lib64/libc-2.18.so with arg1)
probe_libc:abs_5 (on 0x5 in /lib64/libc-2.18.so with arg1)
probe_libc:abs_d8e40 (on @unix/syscall-template.S:81 in /lib64/libc-2.18.so with arg1)
probe_libc:abs_d8e49 (on __GI___libc_write+9 in /lib64/libc-2.18.so with arg1)
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1440586666-235233-7-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch fixes a bug that, when offset is provided but function is
lost, parse_perf_probe_point() will give a "" string as function name,
so the checking code at the end of parse_perf_probe_point() become
useless. For example:
# perf probe +0x1234
Failed to find symbol in kernel
Error: Failed to add events.
After this patch:
# perf probe +0x1234
Semantic error :Offset requires an entry function.
Error: Command Parse Error.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1440586666-235233-6-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
