Commit Graph

624 Commits

Author SHA1 Message Date
Arnaldo Carvalho de Melo
27702bcfe8 perf trace: Support syscall name globbing
So now we can use:

  # perf trace -e pkey_*
   532.784 ( 0.006 ms): pkey/16018 pkey_alloc(init_val: DISABLE_WRITE) = -1 EINVAL Invalid argument
   532.795 ( 0.004 ms): pkey/16018 pkey_mprotect(start: 0x7f380d0a6000, len: 4096, prot: READ|WRITE, pkey: -1) = 0
   532.801 ( 0.002 ms): pkey/16018 pkey_free(pkey: -1                ) = -1 EINVAL Invalid argument
  ^C[root@jouet ~]#

Or '-e epoll*', '-e *msg*', etc.

Combining syscall names with perf events, tracepoints, etc, continues to
be valid, i.e. this is possible:

  # perf probe -L sys_nanosleep
  <SyS_nanosleep@/home/acme/git/linux/kernel/time/hrtimer.c:0>
      0  SYSCALL_DEFINE2(nanosleep, struct timespec __user *, rqtp,
                        struct timespec __user *, rmtp)
         {
                struct timespec64 tu;

      5         if (get_timespec64(&tu, rqtp))
      6                 return -EFAULT;

                if (!timespec64_valid(&tu))
      9                 return -EINVAL;

     11         current->restart_block.nanosleep.type = rmtp ? TT_NATIVE : TT_NONE;
     12         current->restart_block.nanosleep.rmtp = rmtp;
     13         return hrtimer_nanosleep(&tu, HRTIMER_MODE_REL, CLOCK_MONOTONIC);
         }

  # perf probe my_probe="sys_nanosleep:12 rmtp"
  Added new event:
    probe:my_probe       (on sys_nanosleep:12 with rmtp)

  You can now use it in all perf tools, such as:

	perf record -e probe:my_probe -aR sleep 1

  #
  # perf trace -e probe:my_probe/max-stack=5/,*sleep sleep 1
     0.427 ( 0.003 ms): sleep/16690 nanosleep(rqtp: 0x7ffefc245090) ...
     0.430 (         ): probe:my_probe:(ffffffffbd112923) rmtp=0)
                                       sys_nanosleep ([kernel.kallsyms])
                                       do_syscall_64 ([kernel.kallsyms])
                                       return_from_SYSCALL_64 ([kernel.kallsyms])
                                       __nanosleep_nocancel (/usr/lib64/libc-2.25.so)
     0.427 (1000.208 ms): sleep/16690  ... [continued]: nanosleep()) = 0
  #

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-elycoi8wy6y0w9dkj7ox1mzz@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-09-01 14:45:58 -03:00
Jack Henschel
726647d052 perf stat: Fix path to PMU formats in documentation
As defined in tools/perf/util/pmu.c, the EVENT_SOURCE_DEVICE_PATH is
/sys/bus/event_source/devices/ (no traling 's' in event_source)

This patch corrects the path in the perf stat documentation

Signed-off-by: Jack Henschel <jackdev@mailbox.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jack Henschel <jackdev@mailbox.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: trivial@kernel.org
Link: http://lkml.kernel.org/r/20170824132022.10934-1-jackdev@mailbox.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-28 11:05:09 -03:00
Konstantin Khlebnikov
2826478a66 perf tools: Really install manpages via 'make install-man'
Target install-man builds them but forget to install.

Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Borislav Petkov <borislav.petkov@amd.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Fixes: af3df2cf17 ("perf tools: Try to build Documentation when installing")
Link: http://lkml.kernel.org/r/150322915300.129715.13645857235229756834.stgit@buzz
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-22 13:24:53 -03:00
Taeung Song
01c85629f5 perf annotate: Document --show-total-period option
When the --show-total-period option was introduced we forgot to add an
entry in the man page, fix it.

Signed-off-by: Taeung Song <treeze.taeung@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Martin Liška <mliska@suse.cz>
Fixes: 0c4a5bcea4 ("perf annotate: Display total number of samples with --show-total-period")
Link: http://lkml.kernel.org/r/1503046013-5555-1-git-send-email-treeze.taeung@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-18 10:34:08 -03:00
Taeung Song
1ac39372e0 perf annotate stdio: Support --show-nr-samples option
Add --show-nr-samples option to "perf annotate" so that it matches "perf
report".

Committer note:

Note that it can't be used together with --show-total-period, which
seems like a silly limitation, that can be lifted at some point.

Made it bail out if not on --stdio.

Signed-off-by: Taeung Song <treeze.taeung@gmail.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/1503046008-5511-1-git-send-email-treeze.taeung@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-18 10:31:53 -03:00
Adrian Hunter
69e6e410f1 perf script python: Rename call-graph-from-postgresql.py to call-graph-from-sql.py
Rename call-graph-from-postgresql.py to call-graph-from-sql.py in
preparation for adding support to it for SQLite 3.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Link: http://lkml.kernel.org/r/1501749090-20357-5-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-15 16:38:06 -03:00
Adrian Hunter
564b9527d1 perf script python: Add support for exporting to sqlite3
Add support for exporting to SQLite 3 the same data as the PostgreSQL
export.

Committer note:

Tested on RHEL 7.4 using the 1.2.2-4el python-pyside packages from EPEL.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/r/1501749090-20357-4-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-15 16:37:55 -03:00
Krister Johansen
868a832918 perf top: Support lookup of symbols in other mount namespaces.
The perf top command needs to unshare its fs from the helper threads in
order to successfully setns(2) during its symbol lookup.  It also needs
to impelement a force flag to ignore ownership of perf-<pid>.map files.

Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas-Mich Richter <tmricht@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/1499305693-1599-6-git-send-email-kjlx@templeofstupid.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-07-25 22:43:16 -03:00
Jin Yao
60f83fa634 perf record: Create a new option save_type in --branch-filter
The option indicates the kernel to save branch type during sampling.

One example:

  perf record -g --branch-filter any,save_type <command>

Signed-off-by: Yao Jin <yao.jin@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Kan Liang <kan.liang@intel.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1500379995-6449-4-git-send-email-yao.jin@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-07-18 23:14:39 -03:00
David Carrillo-Cisneros
e9def1b2e7 perf tools: Add feature header record to pipe-mode
Add header record types to pipe-mode, reusing the functions
used in file-mode and leveraging the new struct feat_fd.

For alignment, check that synthesized events don't exceed
pagesize.

Add the perf_event__synthesize_feature event call back to
process the new header records.

Before this patch:

  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

After this patch:
  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  # ========
  # captured on: Mon May 22 16:33:43 2017
  # ========
  #
  # hostname : my_hostname
  # os release : 4.11.0-dbx-up_perf
  # perf version : 4.11.rc6.g6277c80
  # arch : x86_64
  # nrcpus online : 72
  # nrcpus avail : 72
  # cpudesc : Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz
  # cpuid : GenuineIntel,6,63,2
  # total memory : 263457192 kB
  # cmdline : /root/perf record -o - -e cycles -c 100000 sleep 1
  # HEADER_CPU_TOPOLOGY info available, use -I to display
  # HEADER_NUMA_TOPOLOGY info available, use -I to display
  # pmu mappings: intel_bts = 6, uncore_imc_4 = 22, uncore_sbox_1 = 47, uncore_cbox_5 = 33, uncore_ha_0 = 16, uncore_cbox
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

Support added for the subcommands: report, inject, annotate and script.

Signed-off-by: David Carrillo-Cisneros <davidcc@google.com>
Acked-by: David Ahern <dsahern@gmail.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Simon Que <sque@chromium.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20170718042549.145161-16-davidcc@google.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-07-18 23:14:36 -03:00
Krister Johansen
f045b8c4b3 perf buildid-cache: Support binary objects from other namespaces
Teach buildid-cache how to add, remove, and update binary objects from
other mount namespaces.  Allow probe events tracing binaries in
different namespaces to add their objects to the probe and build-id
caches too.  As a handy side effect, this also lets us access SDT probes
in binaries from alternate mount namespaces.

Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
Tested-by: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas-Mich Richter <tmricht@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/1499305693-1599-5-git-send-email-kjlx@templeofstupid.com
[ Add util/namespaces.c to tools/perf/util/python-ext-sources, to fix the python binding 'perf test' ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-07-18 23:14:11 -03:00
Krister Johansen
544abd44c7 perf probe: Allow placing uprobes in alternate namespaces.
Teaches perf how to place a uprobe on a file that's in a different mount
namespace.  The user must add the probe using the --target-ns argument
to perf probe.  Once it has been placed, it may be recorded against
without further namespace-specific commands.

Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
[ PPC build fixed by Ravi: ]
Link: http://lkml.kernel.org/r/1500287542-6219-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Cc: Thomas-Mich Richter <tmricht@linux.vnet.ibm.com>
[ Fix !HAVE_DWARF_SUPPORT build ]
Link: http://lkml.kernel.org/r/1499305693-1599-4-git-send-email-kjlx@templeofstupid.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-07-18 23:14:10 -03:00
Adrian Hunter
ead2bfdb85 perf intel-pt: Update documentation to include new ptwrite and power events
Update documentation to include new ptwrite and power events.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1495786658-18063-36-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-30 11:50:54 -03:00
Adrian Hunter
47e780848e perf script: Add 'synth' field for synthesized event payloads
Add a field to display the content the raw_data of a synthesized event.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1495786658-18063-22-git-send-email-adrian.hunter@intel.com
[ Resolved conflict with 106dacd86f ("perf script: Support -F brstackoff,dso") ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-27 12:19:10 -03:00
Adrian Hunter
70d110d775 perf auxtrace: Add itrace option to output power events
Add itrace option to output power events.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1495786658-18063-25-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-27 12:09:58 -03:00
Adrian Hunter
3bdafdffa9 perf auxtrace: Add itrace option to output ptwrite events
Add itrace option to output ptwrite events.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1495786658-18063-24-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-27 12:09:20 -03:00
Adrian Hunter
2bc60ffd66 perf intel-pt: Add documentation for new config terms
Add documentation for new config terms.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1495786658-18063-13-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-21 11:35:49 -03:00
Kan Liang
daefd0bc0b perf stat: Add support to measure SMI cost
Implementing a new --smi-cost mode in perf stat to measure SMI cost.

During the measurement, the /sys/device/cpu/freeze_on_smi will be set.

The measurement can be done with one counter (unhalted core cycles), and
two free running MSR counters (IA32_APERF and SMI_COUNT).

In practice, the percentages of SMI core cycles should be more useful
than absolute value. So the output will be the percentage of SMI core
cycles and SMI#. metric_only will be set by default.

SMI cycles% = (aperf - unhalted core cycles) / aperf

Here is an example output.

 Performance counter stats for 'sudo echo ':

SMI cycles%          SMI#
    0.1%              1

       0.010858678 seconds time elapsed

Users who wants to get the actual value can apply additional
--no-metric-only.

Signed-off-by: Kan Liang <Kan.liang@intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Kan Liang <kan.liang@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Robert Elliott <elliott@hpe.com>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1495825538-5230-3-git-send-email-kan.liang@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-21 11:35:35 -03:00
Namhyung Kim
1096c35aa8 perf ftrace: Add -D option for depth filter
The -D/--graph-depth option is to set max graph depth.  The following
example traces max 2-depth of page fault handler.

  $ sudo perf ftrace -G __do_page_fault -D 2 -- hello
   ...
   0)               |  __do_page_fault() {
   0)   0.063 us    |    down_read_trylock();
   0)   0.251 us    |    find_vma();
   0)   5.374 us    |    handle_mm_fault();
   0)   0.054 us    |    up_read();
   0)   7.463 us    |  }
   ...

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170618142302.25390-4-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-19 22:05:54 -03:00
Namhyung Kim
78b83e8b12 perf ftrace: Add option for function filtering
The -T/--trace-funcs and -N/--notrace-funcs options are to specify
functions to enable/disable tracing dynamically.

The -G/--graph-funcs and -g/--nograph-funcs options are to set filters
for function graph tracer.

For example, to trace fault handling functions only:

  $ sudo perf ftrace -T *fault hello
   0)               |  __do_page_fault() {
   0)               |    handle_mm_fault() {
   0)   2.117 us    |      __handle_mm_fault();
   0)   3.627 us    |    }
   0)   7.811 us    |  }
   0)               |  __do_page_fault() {
   0)               |    handle_mm_fault() {
   0)   2.014 us    |      __handle_mm_fault();
   0)   2.424 us    |    }
   0)   2.951 us    |  }
   ...

To trace all functions executed in __do_page_fault:

  $ sudo perf ftrace -G __do_page_fault hello
   2)               |  __do_page_fault() {
   3)   0.060 us    |    down_read_trylock();
   3)               |    find_vma() {
   3)   0.075 us    |      vmacache_find();
   3)   0.053 us    |      vmacache_update();
   3)   1.246 us    |    }
   3)               |    handle_mm_fault() {
   3)   0.063 us    |      __rcu_read_lock();
   3)   0.056 us    |      mem_cgroup_from_task();
   3)   0.057 us    |      __rcu_read_unlock();
   3)               |      __handle_mm_fault() {
   3)               |        filemap_map_pages() {
   3)   0.058 us    |          __rcu_read_lock();
   3)               |          alloc_set_pte() {
   ...

But don't want to show details in handle_mm_fault:

  $ sudo perf ftrace -G __do_page_fault -g handle_mm_fault hello
   3)               |  __do_page_fault() {
   3)   0.049 us    |    down_read_trylock();
   3)               |    find_vma() {
   3)   0.048 us    |      vmacache_find();
   3)   0.041 us    |      vmacache_update();
   3)   0.680 us    |    }
   3)   0.036 us    |    up_read();
   3)   4.547 us    |  } /* __do_page_fault */
   ...

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170618142302.25390-3-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-19 22:05:53 -03:00
Mark Santaniello
106dacd86f perf script: Support -F brstackoff,dso
The idea here is to make AutoFDO easier in cloud environment with ASLR.
It's easiest to show how this is useful by example. I built a small test
akin to "while(1) { do_nothing(); }" where the do_nothing function is
loaded from a dso:

  $ cat burncpu.cpp
  #include <dlfcn.h>

  int main() {
    void* handle = dlopen("./dso.so", RTLD_LAZY);
    if (!handle) return -1;

    typedef void (*fp)();
    fp do_nothing = (fp) dlsym(handle, "do_nothing");

    while(1) {
      do_nothing();
    }
  }

  $ cat dso.cpp
  extern "C" void do_nothing() {}

  $ cat build.sh
  #!/bin/bash
  g++ -shared dso.cpp -o dso.so
  g++ burncpu.cpp -o burncpu -ldl

I sampled the execution of this program with perf record -b.

Using the existing "brstack,dso", we get absolute addresses that are
affected by ASLR, and could be different on different hosts. The address
does not uniquely identify a branch/target in the binary:

  $ perf script -F brstack,dso | sed 's/\/0 /\/0\n/g' | grep burncpu | grep dso.so | head -n 1
  0x7f967139b6aa(/tmp/burncpu/dso.so)/0x4006b1(/tmp/burncpu/exe)/P/-/-/0

Using the existing "brstacksym,dso" is a little better, because the
symbol plus offset and dso name *does* uniquely identify a branch/target
in the binary.  Ultimately, however, AutoFDO wants a simple offset into
the binary, so we'd have to undo all the work perf did to symbolize in
the first place:

  $ perf script -F brstacksym,dso | sed 's/\/0 /\/0\n/g' | grep burncpu | grep dso.so | head -n 1
  do_nothing+0x5(/tmp/burncpu/dso.so)/main+0x44(/tmp/burncpu/exe)/P/-/-/0

With the new "brstackoff,dso" we get what we need: a simple offset into a
specific dso/binary that uniquely identifies a branch/target:
  $ perf script -F brstackoff,dso | sed 's/\/0 /\/0\n/g' | grep burncpu | grep dso.so | head -n 1
  0x6aa(/tmp/burncpu/dso.so)/0x4006b1(/tmp/burncpu/exe)/P/-/-/0

Signed-off-by: Mark Santaniello <marksan@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20170619163825.2012979-2-marksan@fb.com
[ Updated documentation about 'brstackoff' using text from above ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-19 22:05:46 -03:00
Andi Kleen
36ce565114 perf script: Allow adding and removing fields
With 'perf script' it is common that we just want to add or remove a field.

Currently this requires figuring out the long list of default fields and
specifying them first, and then adding/removing the new field.

This patch adds a new + - syntax to merely add or remove fields,
that allows more succint and clearer command lines

For example to remove the comm field from PMU samples:

Previously

  $ perf script -F tid,cpu,time,event,sym,ip,dso,period | head -1
  swapper  0 [000] 504345.383126:          1 cycles:  ffffffff90060c66 native_write_msr ([kernel.kallsyms])

with the new syntax

  perf script -F -comm | head -1
  0 [000] 504345.383126:          1 cycles:  ffffffff90060c66 native_write_msr ([kernel.kallsyms])

The new syntax cannot be mixed with normal overriding.

v2: Fix example in description. Use tid vs pid. No functional changes.
v3: Don't skip initialization when user specified explicit type.
v4: Rebase. Remove empty line.

Committer testing:

  # perf record -a usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 1.748 MB perf.data (14 samples) ]

Without a explicit field list specified via -F, defaults to:

  # perf script | head -2
      perf 6338 [000] 18467.058607: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
   swapper    0 [001] 18467.058617: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
  #

Which is equivalent to:

  # perf script -F comm,tid,cpu,time,period,event,ip,sym,dso | head -2
      perf 6338 [000] 18467.058607: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
   swapper    0 [001] 18467.058617: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
  #

So if we want to remove the comm, as in your original example, we would have to
figure out the default field list and remove ' comm' from it:

  # perf script -F tid,cpu,time,period,event,ip,sym,dso | head -2
   6338 [000] 18467.058607: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
      0 [001] 18467.058617: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
  #

With your patch this becomes simpler, one can remove fields by prefixing them
with '-':

  # perf script -F -comm | head -2
  6338 [000] 18467.058607: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
     0 [001] 18467.058617: 1 cycles: ffffffff89060c36 native_write_msr (/lib/modules/4.11.0-rc8+/build/vmlinux)
  #

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Milian Wolff <milian.wolff@kdab.com>
Link: http://lkml.kernel.org/r/20170602154810.15875-1-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-19 15:14:58 -03:00
SeongJae Park
14fc42fa1b perf script python: Remove dups in documentation examples
Few shell command examples in perf-script-python.txt has few nitpicks
include:

- tools/perf/scripts/python directory listing command is unnecessarily
  repeated.
- few examples contain additional information in command prompt
  unnecessarily and inconsistently.

This commit fixes them to enhance readability of the document.

Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tom Zanussi <tzanussi@gmail.com>
Fixes: cff68e5822 ("perf/scripts: Add perf-trace-python Documentation")
Link: http://lkml.kernel.org/r/20170530111827.21732-4-sj38.park@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-07 20:36:12 -03:00
SeongJae Park
1bf8d5a4a5 perf script python: Updated trace_unhandled() signature
Default function signature of trace_unhandled() got changed to include a
field dict, but its documentation, perf-script-python.txt has not been
updated.  Fix it.

Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Pierre Tardy <tardyp@gmail.com>
Fixes: c02514850d ("perf scripts python: Give field dict to unhandled callback")
Link: http://lkml.kernel.org/r/20170530111827.21732-6-sj38.park@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-07 20:27:32 -03:00
SeongJae Park
26ddb8722d perf script python: Fix wrong code snippets in documentation
This commit fixes wrong code snippets for trace_begin() and trace_end()
function example definition.

Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tom Zanussi <tzanussi@gmail.com>
Fixes: cff68e5822 ("perf/scripts: Add perf-trace-python Documentation")
Link: http://lkml.kernel.org/r/20170530111827.21732-5-sj38.park@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-07 20:27:26 -03:00
SeongJae Park
34d4453dac perf script: Fix documentation errors
This commit fixes two errors in documents for perf-script-python and
perf-script-perl as below:

- /sys/kernel/debug/tracing events -> /sys/kernel/debug/tracing/events/
- trace_handled -> trace_unhandled

Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tom Zanussi <tzanussi@gmail.com>
Fixes: cff68e5822 ("perf/scripts: Add perf-trace-python Documentation")
Link: http://lkml.kernel.org/r/20170530111827.21732-3-sj38.park@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-07 20:27:20 -03:00
SeongJae Park
d89269a89e perf probe: Fix examples section of documentation
An example in perf-probe documentation for pattern of function name
based probe addition is not providing example command for that case.

This commit fixes the example to give appropriate example command.

Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Taeung Song <treeze.taeung@gmail.com>
Fixes: ee391de876 ("perf probe: Update perf probe document")
Link: http://lkml.kernel.org/r/20170507103642.30560-1-sj38.park@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-07 20:23:11 -03:00
Namhyung Kim
325fbff51f perf script: Add --inline option for debugging
The --inline option is to show inlined functions in callchains.

For example:

  $ perf script
  a.out  5644 11611.467597:     309961 cycles:u:
                     790 main (/home/namhyung/tmp/perf/a.out)
                   20511 __libc_start_main (/usr/lib/libc-2.25.so)
                     8ba _start (/home/namhyung/tmp/perf/a.out)
  ...

  $ perf script --inline
  a.out  5644 11611.467597:     309961 cycles:u:
                     790 main (/home/namhyung/tmp/perf/a.out)
                         std::__detail::_Adaptor<std::linear_congruential_engine<unsigned long, 16807ul, 0ul, 2147483647ul>, double>::operator()
                         std::uniform_real_distribution<double>::operator()<std::linear_congruential_engine<unsigned long, 16807ul, 0ul, 2147483647ul> >
                         std::uniform_real_distribution<double>::operator()<std::linear_congruential_engine<unsigned long, 16807ul, 0ul, 2147483647ul> >
                         main
                   20511 __libc_start_main (/usr/lib/libc-2.25.so)
                     8ba _start (/home/namhyung/tmp/perf/a.out)
  ...

Reviewed-and-tested-by: Milian Wolff <milian.wolff@kdab.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jin Yao <yao.jin@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170524062129.32529-5-namhyung@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-05-24 08:41:48 +02:00
Kim Phillips
1291927a49 perf tools: Fix spelling mistakes
Mostly in the documentation.

Signed-off-by: Kim Phillips <kim.phillips@arm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20170503131350.cebeecd8bd0f2968417626ab@arm.com
[ Fix spelling of "parameter" in one of the spell-checked lines ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-05-04 09:59:53 -03:00
Ravi Bangoria
739cf30551 perf trace: Add usage of --no-syscalls in man page
perf trace supports --no-syscalls option but it's not listed in the man
page. (Though, I see an example using --no-syscalls in EXAMPLES
section.)

Committer note:

The --no-syscalls option tells 'perf trace' not to automagically ask for
raw_syscalls:sys_{enter,exit} to then format it in a strace like way.

This become more used as 'perf trace' got support for arbitrary events,
such as tracepoints, so more and more we use:

  # perf trace --no-syscalls -e nmi:*
     0.000 nmi:nmi_handler:perf_event_nmi_handler() delta_ns: 36649 handled: 1)
     0.019 nmi:nmi_handler:nmi_cpu_backtrace_handler() delta_ns: 2907 handled: 0)
     0.676 nmi:nmi_handler:perf_event_nmi_handler() delta_ns: 9401 handled: 1)
     0.680 nmi:nmi_handler:nmi_cpu_backtrace_handler() delta_ns: 288 handled: 0)
     0.701 nmi:nmi_handler:perf_event_nmi_handler() delta_ns: 4977 handled: 1)
     0.703 nmi:nmi_handler:nmi_cpu_backtrace_handler() delta_ns: 67 handled: 0)
     0.736 nmi:nmi_handler:perf_event_nmi_handler() delta_ns: 8549 handled: 1)
  ^C#

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexis Berlemont <alexis.berlemont@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1492063332-5745-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-04-13 10:54:04 -03:00
David Carrillo-Cisneros
6d13491e2d perf tools: Describe pipe mode in perf.data-file-fomat.txt
Add a minimal description of pipe's data format.

Signed-off-by: David Carrillo-Cisneros <davidcc@google.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Simon Que <sque@chromium.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20170410201432.24807-4-davidcc@google.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-04-11 15:23:41 -03:00
Milian Wolff
5dfa210e40 perf report: Enable sorting by srcline as key
Often it is interesting to know how costly a given source line is in
total. Previously, one had to build these sums manually based on all
addresses that pointed to the same source line. This patch introduces
srcline as a sort key, which will do the aggregation for us.

Paired with the recent addition of showing inline frames, this makes
perf report much more useful for many C++ work loads.

The following shows the new feature in action. First, let's show the
status quo output when we sort by address. The result contains many hist
entries that generate the same output:

  ~~~~~~~~~~~~~~~~
  $ perf report --stdio --inline -g address
  # Children      Self  Command       Shared Object        Symbol
  # ........  ........  ............  ...................  .........................................
  #
      99.89%    35.34%  cpp-inlining  cpp-inlining         [.] main
            |
            |--64.55%--main complex:655
            |          /home/milian/projects/kdab/rnd/hotspot/tests/test-clients/cpp-inlining/main.cpp:39 (inline)
            |          /usr/include/c++/6.3.1/complex:664 (inline)
            |          |
            |          |--60.31%--hypot +20
            |          |          |
            |          |          |--8.52%--__hypot_finite +273
            |          |          |
            |          |          |--7.32%--__hypot_finite +411
...
             --35.34%--_start +4194346
                       __libc_start_main +241
                       |
                       |--6.65%--main random.tcc:3326
                       |          /home/milian/projects/kdab/rnd/hotspot/tests/test-clients/cpp-inlining/main.cpp:39 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1809 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1818 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:185 (inline)
                       |
                       |--2.70%--main random.tcc:3326
                       |          /home/milian/projects/kdab/rnd/hotspot/tests/test-clients/cpp-inlining/main.cpp:39 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1809 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1818 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:185 (inline)
                       |
                       |--1.69%--main random.tcc:3326
                       |          /home/milian/projects/kdab/rnd/hotspot/tests/test-clients/cpp-inlining/main.cpp:39 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1809 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1818 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:185 (inline)
  ...
  ~~~~~~~~~~~~~~~~

With this patch and `-g srcline` we instead get the following output:

  ~~~~~~~~~~~~~~~~
  $ perf report --stdio --inline -g srcline
  # Children      Self  Command       Shared Object        Symbol
  # ........  ........  ............  ...................  .........................................
  #
      99.89%    35.34%  cpp-inlining  cpp-inlining         [.] main
            |
            |--64.55%--main complex:655
            |          /home/milian/projects/kdab/rnd/hotspot/tests/test-clients/cpp-inlining/main.cpp:39 (inline)
            |          /usr/include/c++/6.3.1/complex:664 (inline)
            |          |
            |          |--64.02%--hypot
            |          |          |
            |          |           --59.81%--__hypot_finite
            |          |
            |           --0.53%--cabs
            |
             --35.34%--_start
                       __libc_start_main
                       |
                       |--12.48%--main random.tcc:3326
                       |          /home/milian/projects/kdab/rnd/hotspot/tests/test-clients/cpp-inlining/main.cpp:39 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1809 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:1818 (inline)
                       |          /usr/include/c++/6.3.1/bits/random.h:185 (inline)
  ...
  ~~~~~~~~~~~~~~~~

Signed-off-by: Milian Wolff <milian.wolff@kdab.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Yao Jin <yao.jin@linux.intel.com>
Link: http://lkml.kernel.org/r/20170318214928.9047-1-milian.wolff@kdab.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-27 12:13:28 -03:00
Jin Yao
f3a60646cc perf report: Introduce --inline option
It takes some time to look for inline stack for callgraph addresses.  So
it provides new option "--inline" to let user decide if enable this
feature.

  --inline:

  If a callgraph address belongs to an inlined function, the inline stack
  will be printed. Each entry is the inline function name or file/line.

Signed-off-by: Yao Jin <yao.jin@linux.intel.com>
Tested-by: Milian Wolff <milian.wolff@kdab.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@intel.com>
Link: http://lkml.kernel.org/r/1490474069-15823-4-git-send-email-yao.jin@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-27 12:01:46 -03:00
Ravi Bangoria
6963d3c387 perf list sdt: Show option in man page
Commit 40218daea1 ("perf list: Show SDT and pre-cached events") added
sdt support in perf list, but it missed to update documentation.

Show sdt option in man perf-list.

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20170327025538.1753-1-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-27 11:58:09 -03:00
Andi Kleen
bf874fcf9f perf list: Move extra details printing to new option
Move the printing of perf expressions and internal events to a new
clearer --details flag, instead of lumping it together with other debug
options in --debug. This makes it clearer to use.

Before

  perf list --debug
  ...
  unc_m_power_critical_throttle_cycles
         [Cycles all ranks are in critical thermal throttle. Unit: uncore_imc]
          uncore_imc_2/event=0x86/  MetricName: power_critical_throttle_cycles % MetricExpr: (unc_m_power_critical_throttle_cycles / unc_m_clockticks) * 100.

after

  perf list --details
  ...
  unc_m_power_critical_throttle_cycles
         [Cycles all ranks are in critical thermal throttle. Unit: uncore_imc]
          uncore_imc_2/event=0x86/  MetricName: power_critical_throttle_cycles % MetricExpr: (unc_m_power_critical_throttle_cycles / unc_m_clockticks) * 100.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/r/20170320201711.14142-14-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-23 11:42:31 -03:00
Andi Kleen
430daf2dc7 perf stat: Collapse identically named events
The uncore PMU has a lot of duplicated PMUs for different subsystems.
When expanding an uncore alias we usually end up with a large
number of identically named aliases, which makes perf stat
output difficult to read.

Automatically sum them up in perf stat, unless --no-merge is specified.

This can be default because only the uncores generally have duplicated
aliases. Other PMUs have unique names.

Before:

  % perf stat --no-merge -a -e unc_c_llc_lookup.any sleep 1

  Performance counter stats for 'system wide':

           694,976 Bytes unc_c_llc_lookup.any
           706,304 Bytes unc_c_llc_lookup.any
           956,608 Bytes unc_c_llc_lookup.any
           782,720 Bytes unc_c_llc_lookup.any
           605,696 Bytes unc_c_llc_lookup.any
           442,816 Bytes unc_c_llc_lookup.any
           659,328 Bytes unc_c_llc_lookup.any
           509,312 Bytes unc_c_llc_lookup.any
           263,936 Bytes unc_c_llc_lookup.any
           592,448 Bytes unc_c_llc_lookup.any
           672,448 Bytes unc_c_llc_lookup.any
           608,640 Bytes unc_c_llc_lookup.any
           641,024 Bytes unc_c_llc_lookup.any
           856,896 Bytes unc_c_llc_lookup.any
           808,832 Bytes unc_c_llc_lookup.any
           684,864 Bytes unc_c_llc_lookup.any
           710,464 Bytes unc_c_llc_lookup.any
           538,304 Bytes unc_c_llc_lookup.any

       1.002577660 seconds time elapsed

After:

  % perf stat -a -e unc_c_llc_lookup.any sleep 1

  Performance counter stats for 'system wide':

         2,685,120 Bytes unc_c_llc_lookup.any

       1.002648032 seconds time elapsed

v2: Split collect_aliases. Rename alias flag.
v3: Make sure unsupported/not counted is always printed.
v4: Factor out callback change into separate patch.
v5: Move check for bad results here
    Move merged check into collect_data

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: http://lkml.kernel.org/r/20170320201711.14142-3-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-21 16:04:11 -03:00
Ravi Bangoria
efc9c05681 perf stat: Correct --no-aggr description
Description of --no-aggr in perf-stat man page is outdated. --no-aggr
can also be used while profiling specific set of cpus. For ex,

  $ perf stat -e cycles,instructions -C 1-2 --no-aggr -- sleep 1

    Performance counter stats for 'CPU(s) 1-2':

    CPU1   5,94,92,795   cycles
    CPU2   2,69,72,403   cycles
    CPU1   2,02,08,327   instructions   # 0.34 insn per cycle
    CPU2     73,17,123   instructions   # 0.12 insn per cycle

    1.000989132 seconds time elapsed

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1490013438-5713-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-20 15:01:31 -03:00
Andi Kleen
48d02a1d5c perf script: Add 'brstackinsn' for branch stacks
Implement printing instruction sequences as hex dump for branch stacks.

This relies on the x86 instruction decoder used by the PT decoder to
find the lengths of instructions to dump them individually.

This is good enough for pattern matching.

This allows to study hot paths for individual samples, together with
branch misprediction and cycle count / IPC information if available (on
Skylake systems).

  % perf record -b ...
  % perf script -F brstackinsn
  ...
    read_hpet+67:
          ffffffff9905b843        insn: 74 ea                     # PRED
          ffffffff9905b82f        insn: 85 c9
          ffffffff9905b831        insn: 74 12
          ffffffff9905b833        insn: f3 90
          ffffffff9905b835        insn: 48 8b 0f
          ffffffff9905b838        insn: 48 89 ca
          ffffffff9905b83b        insn: 48 c1 ea 20
          ffffffff9905b83f        insn: 39 f2
          ffffffff9905b841        insn: 89 d0
          ffffffff9905b843        insn: 74 ea                     # PRED

Only works when no special branch filters are specified.

Occasionally the path does not reach up to the sample IP, as the LBRs
may be frozen before executing a final jump. In this case we print a
special message.

The instruction dumper piggy backs on the existing infrastructure from
the IP PT decoder.

An earlier iteration of this patch relied on a disassembler, but this
version only uses the existing instruction decoder.

Committer note:

Added hint about how to get suitable perf.data files for use with
'-F brstackinsm':

  $ perf record usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.018 MB perf.data (8 samples) ]
  $
  $ perf script -F brstackinsn
  Display of branch stack assembler requested, but non all-branch filter set
  Hint: run 'perf record -b ...'
  $

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Link: http://lkml.kernel.org/r/20170223234634.583-1-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-16 09:24:35 -03:00
Brendan Gregg
292c4a8f98 perf sched timehist: Add --next option
The --next option shows the next task for each context switch, providing
more context for the sequence of scheduler events.

  $ perf sched timehist --next | head
  Samples do not have callchains.
       time  cpu task name  waittime schdelay run time
                 [tid/pid]     (msec) (msec) (msec)
  ---------- --- ---------- --------- ------ -----
  374.793792 [0] <idle>         0.000  0.000 0.000 next: rngd[1524]
  374.793801 [0] rngd[1524]     0.000  0.000 0.009 next: swapper/0[0]
  374.794048 [7] <idle>         0.000  0.000 0.000 next: yes[30884]
  374.794066 [7] yes[30884]     0.000  0.000 0.018 next: swapper/7[0]
  374.794126 [2] <idle>         0.000  0.000 0.000 next: rngd[1524]
  374.794140 [2] rngd[1524]     0.325  0.006 0.013 next: swapper/2[0]
  374.794281 [3] <idle>         0.000  0.000 0.000 next: perf[31070]

Signed-off-by: Brendan Gregg <bgregg@netflix.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1489456589-32555-1-git-send-email-bgregg@netflix.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-14 15:17:38 -03:00
Hari Bathini
d890a98c92 perf tools: Add 'cgroup_id' sort order keyword
This patch introduces a cgroup identifier entry field in perf report to
identify or distinguish data of different cgroups. It uses the device
number and inode number of cgroup namespace, included in perf data with
the new PERF_RECORD_NAMESPACES event, as cgroup identifier.

With the assumption that each container is created with it's own cgroup
namespace,  this allows assessment/analysis of multiple containers at
once.

A simple test for this would be to clone a few processes passing
SIGCHILD & CLONE_NEWCROUP flags to each of them, execute shell and run
different workloads  on each of those contexts,  while running perf
record command with --namespaces option.

Shown below is the output of perf report, sorted with cgroup identifier,
on perf.data generated with the above test scenario, clearly indicating
one context's considerable use of kernel memory in comparison with
others:

	$ perf report -s cgroup_id,sample --stdio
	#
	# Total Lost Samples: 0
	#
	# Samples: 5K of event 'kmem:kmalloc'
	# Event count (approx.): 5965
	#
	# Overhead  cgroup id (dev/inode)       Samples
	# ........  .....................  ............
	#
	    81.27%  3/0xeffffffb                   4848
	    16.24%  3/0xf00000d0                    969
	     1.16%  3/0xf00000ce                     69
	     0.82%  3/0xf00000cf                     49
	     0.50%  0/0x0                            30

While this is a start, there is further scope of improving this. For
example, instead of cgroup namespace's device and inode numbers, dev
and inode numbers of some or all namespaces may be used to distinguish
which processes are running in a given container context.

Also, scripts to map device and inode info to containers sounds
plausible for better tracing of containers.

Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sargun Dhillon <sargun@sargun.me>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/148891933338.25309.756882900782042645.stgit@hbathini.in.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-14 15:17:37 -03:00
Hari Bathini
96a44bbccd perf script: Add script print support for namespace events
Introduce a new option to display events of type PERF_RECORD_NAMESPACES
and update perf-script documentation accordingly.

Shown below is output (trimmed) of perf script command with the newly
introduced option, on perf.data generated with perf record command using
--namespaces option.

  $ perf script --show-namespace-events
      swapper   0 [000]     0.000000: PERF_RECORD_NAMESPACES 1/1 - nr_namespaces: 7
                [0/net: 3/0xf000001c, 1/uts: 3/0xeffffffe, 2/ipc: 3/0xefffffff, 3/pid: 3/0xeffffffc,
                 4/user: 3/0xeffffffd, 5/mnt: 3/0xf0000000, 6/cgroup: 3/0xeffffffb]
      swapper   0 [000]     0.000000: PERF_RECORD_NAMESPACES 2/2 - nr_namespaces: 7
                [0/net: 3/0xf000001c, 1/uts: 3/0xeffffffe, 2/ipc: 3/0xefffffff, 3/pid: 3/0xeffffffc,
                 4/user: 3/0xeffffffd, 5/mnt: 3/0xf0000000, 6/cgroup: 3/0xeffffffb]

Commiter notes:

Testing it:

Investigating that double PERF_RECORD_NAMESPACES for the 19155
pid/tid... Its more than that, there are two PERF_RECORD_COMM as well,
and with zeroed timestamps, so probably a synthesizing artifact...

  # perf script --show-task --show-namespace
  <SNIP>
      perf     0 [000]     0.000000: PERF_RECORD_COMM: perf:19154/19154
      perf     0 [000]     0.000000: PERF_RECORD_FORK(19155:19155):(19154:19154)
      perf     0 [000]     0.000000: PERF_RECORD_NAMESPACES 19155/19155 - nr_namespaces: 7
          [0/net: 3/0xf0000081, 1/uts: 3/0xeffffffe, 2/ipc: 3/0xefffffff, 3/pid: 3/0xeffffffc,
           4/user: 3/0xeffffffd, 5/mnt: 3/0xf0000000, 6/cgroup: 3/0xeffffffb]
      perf     0 [000]     0.000000: PERF_RECORD_COMM: perf:19155/19155
      perf     0 [000]     0.000000: PERF_RECORD_COMM: perf:19155/19155
      perf     0 [000]     0.000000: PERF_RECORD_NAMESPACES 19155/19155 - nr_namespaces: 7
          [0/net: 3/0xf0000081, 1/uts: 3/0xeffffffe, 2/ipc: 3/0xefffffff, 3/pid: 3/0xeffffffc,
           4/user: 3/0xeffffffd, 5/mnt: 3/0xf0000000, 6/cgroup: 3/0xeffffffb]
   swapper     0 [000]  3110.881834:          1 cycles:  ffffffffa7060bf6 native_write_msr (/lib/modules/4.11.0-rc1+/build/vmlinux)

  <SNIP>

Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sargun Dhillon <sargun@sargun.me>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/148891932627.25309.1941587059154176221.stgit@hbathini.in.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-14 15:17:36 -03:00
Hari Bathini
f3b3614a28 perf tools: Add PERF_RECORD_NAMESPACES to include namespaces related info
Introduce a new option to record PERF_RECORD_NAMESPACES events emitted
by the kernel when fork, clone, setns or unshare are invoked. And update
perf-record documentation with the new option to record namespace
events.

Committer notes:

Combined it with a later patch to allow printing it via 'perf report -D'
and be able to test the feature introduced in this patch. Had to move
here also perf_ns__name(), that was introduced in another later patch.

Also used PRIu64 and PRIx64 to fix the build in some enfironments wrt:

  util/event.c:1129:39: error: format '%lx' expects argument of type 'long unsigned int', but argument 6 has type 'long long unsigned int' [-Werror=format=]
     ret  += fprintf(fp, "%u/%s: %lu/0x%lx%s", idx
                                         ^
Testing it:

  # perf record --namespaces -a
  ^C[ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 1.083 MB perf.data (423 samples) ]
  #
  # perf report -D
  <SNIP>
  3 2028902078892 0x115140 [0xa0]: PERF_RECORD_NAMESPACES 14783/14783 - nr_namespaces: 7
                [0/net: 3/0xf0000081, 1/uts: 3/0xeffffffe, 2/ipc: 3/0xefffffff, 3/pid: 3/0xeffffffc,
                 4/user: 3/0xeffffffd, 5/mnt: 3/0xf0000000, 6/cgroup: 3/0xeffffffb]

  0x1151e0 [0x30]: event: 9
  .
  . ... raw event: size 48 bytes
  .  0000:  09 00 00 00 02 00 30 00 c4 71 82 68 0c 7f 00 00  ......0..q.h....
  .  0010:  a9 39 00 00 a9 39 00 00 94 28 fe 63 d8 01 00 00  .9...9...(.c....
  .  0020:  03 00 00 00 00 00 00 00 ce c4 02 00 00 00 00 00  ................
  <SNIP>
        NAMESPACES events:          1
  <SNIP>
  #

Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sargun Dhillon <sargun@sargun.me>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/148891930386.25309.18412039920746995488.stgit@hbathini.in.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-14 11:38:23 -03:00
Changbin Du
d35fa1e75f perf report: Document +field style argument support for --field option
Commit 2f3f9bcf00 ("perf tools: Add +field argument support for
--field option") by Jiri Olsa <jolsa@kernel.org> introduced +field style
argument support for --field option.

This is useful but not updated documentation.  This add a little
description there.

Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20170313083252.23644-1-changbin.du@intel.com
[ Slightly improved the phrase structure ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-13 11:44:44 -03:00
Namhyung Kim
dc23103278 perf ftrace: Add support for -a and -C option
The -a/--all-cpus and -C/--cpu option is for controlling tracing cpus.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170224011251.14946-3-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-03 19:07:17 -03:00
Namhyung Kim
a9af6be5bc perf ftrace: Add support for --pid option
The -p (--pid) option enables to trace existing process by its pid.

Committer notes:

Testing it:

Using the function_graph tracer on a process that is just waiting for user
input and thus will make 'perf ftrace' sit there waiting for that, then press
any key on that mutt session and see what happens:

  # perf ftrace -t function_graph -p `pidof mutt` | head -40
  2)   1.038 us    |  switch_mm_irqs_off();
  ------------------------------------------
  2)    <idle>-0    =>   mutt-3595
  ------------------------------------------

  2)               |              finish_task_switch() {
  2)               |                smp_irq_work_interrupt() {
  2)               |                  irq_enter() {
  2)   0.180 us    |                    rcu_irq_enter();
  2)   1.248 us    |                  }
  2)               |                  __wake_up() {
  2)   0.126 us    |                    _raw_spin_lock_irqsave();
  2)               |                    __wake_up_common() {
  2)               |                      pollwake() {
  2)               |                        default_wake_function() {
  2)               |                          try_to_wake_up() {
  2)   0.662 us    |                            _raw_spin_lock_irqsave();
  2)               |                            select_task_rq_fair() {
  2)   1.719 us    |                              effective_load.isra.41();
  2)   1.343 us    |                              effective_load.isra.41();
  2)               |                              select_idle_sibling() {
  2)   0.331 us    |                                idle_cpu();
  2)   1.458 us    |                              }
  2)   8.350 us    |                            }
  2)   0.200 us    |                            _raw_spin_lock();
  2)               |                            ttwu_do_activate() {
  2)               |                              activate_task() {
  2)   0.136 us    |                                update_rq_clock.part.77();
  2)               |                                enqueue_task_fair() {
  2)               |                                  enqueue_entity() {
  2)   0.146 us    |                                    update_curr();
  2)   0.330 us    |                                    account_entity_enqueue();
  2)   0.280 us    |                                    update_cfs_shares();
  2)   0.321 us    |                                    place_entity();
  2)   0.206 us    |                                    __enqueue_entity();
  2)   6.926 us    |                                  }
  2)               |                                  enqueue_entity() {
  2)   0.105 us    |                                    update_curr();
  2)   0.175 us    |                                    account_entity_enqueue();
  2)   0.531 us    |                                    update_cfs_shares();
 #

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170224011251.14946-1-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-03 19:07:16 -03:00
Charles Baylis
7768f8dada perf tools: Allow sorting by symbol size
Add new sort key 'symbol_size' to allow user to sort by symbol size, or
(more usefully) display the symbol size using --fields=...,symbol_size.

Committer note:

Testing it together with the recently added -q, to remove the headers,
and using the '+' sign with -s, to add the symbol_size sort order to
the default, which is '-s/--sort comm,dso,symbol':

  # perf report -q -s +symbol_size | head -10
  10.39%  swapper       [kernel.vmlinux] [k] intel_idle               270
   3.45%  swapper       [kernel.vmlinux] [k] update_blocked_averages 1546
   2.61%  swapper       [kernel.vmlinux] [k] update_load_avg         1292
   2.36%  swapper       [kernel.vmlinux] [k] update_cfs_shares        240
   1.83%  swapper       [kernel.vmlinux] [k] __hrtimer_run_queues     606
   1.74%  swapper       [kernel.vmlinux] [k] update_cfs_rq_load_avg. 1187
   1.66%  swapper       [kernel.vmlinux] [k] apic_timer_interrupt     152
   1.60%  CPU 0/KVM     [kvm]            [k] kvm_set_msr_common      3046
   1.60%  gnome-shell   libglib-2.0.so.0 [.] g_slist_find              37
   1.46%  gnome-termina libglib-2.0.so.0 [.] g_hash_table_lookup      370
  #

Signed-off-by: Charles Baylis <charles.baylis@linaro.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1487943176-13840-1-git-send-email-charles.baylis@linaro.org
[ Use symbol__size(), remove needless %lld + (long long) casting ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-03 19:07:16 -03:00
Linus Torvalds
3f26b0c876 Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf fixes from Ingo Molnar:
 "Misc fixes on the kernel and tooling side - nothing in particular
  stands out"

* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (21 commits)
  perf/core: Fix the perf_cpu_time_max_percent check
  perf/core: Fix perf_event_enable_on_exec() timekeeping (again)
  perf/core: Remove confusing comment and move put_ctx()
  perf record: Honor --quiet option properly
  perf annotate: Add -q/--quiet option
  perf diff: Add -q/--quiet option
  perf report: Add -q/--quiet option
  perf utils: Check verbose flag properly
  perf utils: Add perf_quiet_option()
  perf record: Add -a as default target
  perf stat: Add -a as default target
  perf tools: Fail on using multiple bits long terms without value
  perf tools: Move new_term arguments into struct parse_events_term template
  perf build: Add special fixdep cleaning rule
  perf tools: Replace _SC_NPROCESSORS_CONF with max_present_cpu in cpu_topology_map
  perf header: Make build_cpu_topology skip offline/absent CPUs
  perf cpumap: Add cpu__max_present_cpu()
  perf session: Fix DEBUG=1 build with clang
  tools lib traceevent: It's preempt not prempt
  perf python: Filter out -specs=/a/b/c from the python binding cc options
  ...
2017-02-28 11:38:18 -08:00
Masahiro Yamada
9332ef9dbd scripts/spelling.txt: add "an user" pattern and fix typo instances
Fix typos and add the following to the scripts/spelling.txt:

  an user||a user
  an userspace||a userspace

I also added "userspace" to the list since it is a common word in Linux.
I found some instances for "an userfaultfd", but I did not add it to the
list.  I felt it is endless to find words that start with "user" such as
"userland" etc., so must draw a line somewhere.

Link: http://lkml.kernel.org/r/1481573103-11329-4-git-send-email-yamada.masahiro@socionext.com
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-02-27 18:43:46 -08:00
Namhyung Kim
eddaef8896 perf annotate: Add -q/--quiet option
The -q/--quiet option is to suppress any message.  Sometimes users just
want to see the numbers and it can be used for that case.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Suggested-and-Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170217081742.17417-6-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-02-20 11:47:18 -03:00
Namhyung Kim
63b42fce86 perf diff: Add -q/--quiet option
The -q/--quiet option is to suppress any message.  Sometimes users just
want to see the numbers and it can be used for that case.

Committer notes:

Before:

  # perf diff | head -10
  Failed to open /tmp/perf-6678.map, continuing without symbols
  Failed to open /tmp/perf-6678.map, continuing without symbols
  Failed to open /tmp/perf-2646.map, continuing without symbols
  # Event 'cycles'
  #
  # Baseline  Delta Abs  Shared Object               Symbol
  # ........  .........  ..........................  ............................................
  #
       5.36%     -1.76%  [kernel.vmlinux]            [k] intel_idle
       2.80%     +1.48%  firefox                     [.] 0x00000000000101fe
      57.12%     -1.25%  libxul.so                   [.] 0x00000000009bea92
       1.36%     -1.11%  [kernel.vmlinux]            [k] __schedule
       4.26%     -1.00%  perf-6678.map               [.] 0x00007fac4b0e9320

After:

  # perf diff -q | head -10
       5.36%     -1.76%  [kernel.vmlinux]            [k] intel_idle
       2.80%     +1.48%  firefox                     [.] 0x00000000000101fe
      57.12%     -1.25%  libxul.so                   [.] 0x00000000009bea92
       1.36%     -1.11%  [kernel.vmlinux]            [k] __schedule
       4.26%     -1.00%  perf-6678.map               [.] 0x00007fac4b0e9320
       1.86%     +0.95%  [kernel.vmlinux]            [k] update_blocked_averages
       0.80%     -0.70%  [kernel.vmlinux]            [k] native_sched_clock
       0.74%     -0.58%  [kernel.vmlinux]            [k] native_write_msr
       0.76%     -0.56%  qemu-system-x86_64          [.] 0x00000000002395c0
                 +0.54%  libpulsecommon-10.0.so      [.] 0x000000000002d91b
  #

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Suggested-and-Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170217081742.17417-5-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-02-20 11:47:08 -03:00