linux/tools
Andrii Nakryiko fa9bb590c2 selftests/bpf: support stats ordering in comparison mode in veristat
Introduce the concept of "stat variant", by which it's possible to
specify whether to use the value from A (baseline) side, B (comparison
or control) side, the absolute difference value or relative (percentage)
difference value.

To support specifying this, veristat recognizes `_a`, `_b`, `_diff`,
`_pct` suffixes, which can be appended to stat name(s). In
non-comparison mode variants are ignored (there is only `_a` variant
effectively), if no variant suffix is provided, `_b` is assumed, as
control group is of primary interest in comparison mode.

These stat variants can be flexibly combined with asc/desc orders.

Here's an example of ordering results first by verdict match/mismatch (or n/a
if one of the sides is missing; n/a is always considered to be the lowest
value), and within each match/mismatch/n/a group further sort by number of
instructions in B side. In this case we don't have MISMATCH cases, but N/A are
split from MATCH, demonstrating this custom ordering.

  $ ./veristat -e file,prog,verdict,insns -s verdict_diff,insns_b_ -C ~/base.csv ~/comp.csv
  File                Program                         Verdict (A)  Verdict (B)  Verdict (DIFF)  Insns (A)  Insns (B)  Insns   (DIFF)
  ------------------  ------------------------------  -----------  -----------  --------------  ---------  ---------  --------------
  bpf_xdp.o           tail_lb_ipv6                    N/A          success      N/A                   N/A     151895             N/A
  bpf_xdp.o           tail_nodeport_nat_egress_ipv4   N/A          success      N/A                   N/A      15619             N/A
  bpf_xdp.o           tail_nodeport_ipv6_dsr          N/A          success      N/A                   N/A       1206             N/A
  bpf_xdp.o           tail_nodeport_ipv4_dsr          N/A          success      N/A                   N/A       1162             N/A
  bpf_alignchecker.o  tail_icmp6_send_echo_reply      N/A          failure      N/A                   N/A         74             N/A
  bpf_alignchecker.o  __send_drop_notify              success      N/A          N/A                    53        N/A             N/A
  bpf_host.o          __send_drop_notify              success      N/A          N/A                    53        N/A             N/A
  bpf_host.o          cil_from_host                   success      N/A          N/A                   762        N/A             N/A
  bpf_xdp.o           tail_lb_ipv4                    success      success      MATCH               71736      73430  +1694 (+2.36%)
  bpf_xdp.o           tail_handle_nat_fwd_ipv4        success      success      MATCH               21547      20920   -627 (-2.91%)
  bpf_xdp.o           tail_rev_nodeport_lb6           success      success      MATCH               17954      17905    -49 (-0.27%)
  bpf_xdp.o           tail_handle_nat_fwd_ipv6        success      success      MATCH               16974      17039    +65 (+0.38%)
  bpf_xdp.o           tail_nodeport_nat_ingress_ipv4  success      success      MATCH                7658       7713    +55 (+0.72%)
  bpf_xdp.o           tail_rev_nodeport_lb4           success      success      MATCH                7126       6934   -192 (-2.69%)
  bpf_xdp.o           tail_nodeport_nat_ingress_ipv6  success      success      MATCH                6405       6397     -8 (-0.12%)
  bpf_xdp.o           tail_nodeport_nat_ipv6_egress   failure      failure      MATCH                 752        752     +0 (+0.00%)
  bpf_xdp.o           cil_xdp_entry                   success      success      MATCH                 423        423     +0 (+0.00%)
  bpf_xdp.o           __send_drop_notify              success      success      MATCH                 151        151     +0 (+0.00%)
  bpf_alignchecker.o  tail_icmp6_handle_ns            failure      failure      MATCH                  33         33     +0 (+0.00%)
  ------------------  ------------------------------  -----------  -----------  --------------  ---------  ---------  --------------

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/r/20221103055304.2904589-10-andrii@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2022-11-03 21:54:14 -07:00
..
accounting
arch tools headers cpufeatures: Sync with the kernel sources 2022-10-25 17:40:48 -03:00
bootconfig
bpf bpftool: Fix spelling mistake "disasembler" -> "disassembler" 2022-10-26 18:20:22 -07:00
build perf bpf: Fix build with libbpf 0.7.0 by checking if bpf_program__set_insns() is available 2022-10-25 17:40:48 -03:00
certs
cgroup iocost_monitor: reorder BlkgIterator 2022-09-23 16:57:10 -10:00
counter
debugging tools: Add new "test" taint to kernel-chktaint 2022-09-07 14:51:12 -06:00
edid
firewire
firmware
gpio
hv tools: hv: kvp: remove unnecessary (void*) conversions 2022-09-05 16:55:20 +00:00
iio tools: iio: iio_utils: fix digit calculation 2022-10-17 08:51:26 +01:00
include Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2022-11-03 13:21:54 -07:00
io_uring
kvm/kvm_stat
laptop
leds
lib bpf-next-for-netdev 2022-11-02 08:18:27 -07:00
memory-model tools/memory-model: Clarify LKMM's limitations in litmus-tests.txt 2022-08-31 05:15:31 -07:00
objtool - Yu Zhao's Multi-Gen LRU patches are here. They've been under test in 2022-10-10 17:53:04 -07:00
pci
pcmcia
perf perf vendor events arm64: Fix incorrect Hisi hip08 L3 metrics 2022-10-26 11:01:56 -03:00
power pm-graph v5.10 2022-10-25 17:46:15 +02:00
rcu
scripts
spi
testing selftests/bpf: support stats ordering in comparison mode in veristat 2022-11-03 21:54:14 -07:00
thermal tools/thermal: Fix possible path truncations 2022-08-03 19:28:46 +02:00
time
tracing rtla: Consolidate and show all necessary libraries that failed for building 2022-08-10 12:03:02 -04:00
usb tools: usb: testusb: Add super-plus speed reporting 2022-07-08 14:54:49 +02:00
verification rv/dot2c: Make automaton definition static 2022-10-20 16:02:45 -04:00
virtio virtio_test: fixup for vq reset 2022-09-27 18:30:49 -04:00
vm tools/vm/page_owner_sort: fix -f option 2022-09-11 20:25:51 -07:00
wmi
Makefile