linux/tools/testing/selftests
Andrii Nakryiko 537c3f66ea selftests/bpf: add generic BPF program tester-loader
It's become a common pattern to have a collection of small BPF programs
in one BPF object file, each representing one test case. On user-space
side of such tests we maintain a table of program names and expected
failure or success, along with optional expected verifier log message.

This works, but each set of tests reimplement this mundane code over and
over again, which is a waste of time for anyone trying to add a new set
of tests. Furthermore, it's quite error prone as it's way too easy to miss
some entries in these manually maintained test tables (as evidences by
dynptr_fail tests, in which ringbuf_release_uninit_dynptr subtest was
accidentally missed; this is fixed in next patch).

So this patch implements generic test_loader, which accepts skeleton
name and handles the rest of details: opens and loads BPF object file,
making sure each program is tested in isolation. Optionally each test
case can specify expected BPF verifier log message. In case of failure,
tester makes sure to report verifier log, but it also reports verifier
log in verbose mode unconditionally.

Now, the interesting deviation from existing custom implementations is
the use of btf_decl_tag attribute to specify expected-to-fail vs
expected-to-succeed markers and, optionally, expected log message
directly next to BPF program source code, eliminating the need to
manually create and update table of tests.

We define few macros wrapping btf_decl_tag with a convention that all
values of btf_decl_tag start with "comment:" prefix, and then utilizing
a very simple "just_some_text_tag" or "some_key_name=<value>" pattern to
define things like expected success/failure, expected verifier message,
extra verifier log level (if necessary). This approach is demonstrated
by next patch in which two existing sets of failure tests are converted.

Tester supports both expected-to-fail and expected-to-succeed programs,
though this patch set didn't convert any existing expected-to-succeed
programs yet, as existing tests couple BPF program loading with their
further execution through attach or test_prog_run. One way to allow
testing scenarios like this would be ability to specify custom callback,
executed for each successfully loaded BPF program. This is left for
follow up patches, after some more analysis of existing test cases.

This test_loader is, hopefully, a start of a test_verifier-like runner,
but integrated into test_progs infrastructure. It will allow much better
"user experience" of defining low-level verification tests that can take
advantage of all the libbpf-provided nicety features on BPF side: global
variables, declarative maps, etc.  All while having a choice of defining
it in C or as BPF assembly (through __attribute__((naked)) functions and
using embedded asm), depending on what makes most sense in each
particular case. This will be explored in follow up patches as well.

Acked-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/r/20221207201648.2990661-1-andrii@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2022-12-07 17:01:21 -08:00
..
alsa selftests: alsa: Handle pkg-config failure more gracefully 2022-05-31 18:02:18 +02:00
amd-pstate cpufreq: amd-pstate: Add explanation for X86_AMD_PSTATE_UT 2022-10-05 11:05:18 -06:00
arm64 Merge branch 'for-next/kselftest' into for-next/core 2022-09-30 09:18:11 +01:00
bpf selftests/bpf: add generic BPF program tester-loader 2022-12-07 17:01:21 -08:00
breakpoints
capabilities
cgroup - Yu Zhao's Multi-Gen LRU patches are here. They've been under test in 2022-10-10 17:53:04 -07:00
clone3 selftests: fixup build warnings in pidfd / clone3 tests 2022-02-04 13:07:07 -07:00
core
cpu-hotplug selftests/cpu-hotplug: Add log info when test success 2022-10-05 11:05:18 -06:00
cpufreq selftests: cpufreq: Write test output to stdout as well 2022-01-25 13:20:03 -07:00
damon selftest/damon: add a test for duplicate context dirs creation 2022-10-03 14:03:06 -07:00
dma selftests dma: fix compile error for dma_map_benchmark 2022-06-16 14:03:21 -06:00
dmabuf-heaps
drivers selftests: bonding: up/down delay w/ slave link flapping 2022-11-23 20:14:48 -08:00
efivarfs
exec linux-kselftest-next-5.18-rc1 2022-03-23 12:53:00 -07:00
filesystems Updates to various subsystems which I help look after. lib, ocfs2, 2022-08-07 10:03:24 -07:00
firmware selftests: firmware: Add firmware upload selftests 2022-04-29 16:49:36 +02:00
fpu
ftrace selftests/ftrace: fix dynamic_events dependency check 2022-10-18 14:27:23 -06:00
futex selftests/futex: fix build for clang 2022-10-18 14:13:11 -06:00
gpio selftests: gpio: fix include path to kernel headers for out of tree builds 2022-07-20 14:35:18 +02:00
ia64
intel_pstate selftests/intel_pstate: fix build for ARCH=x86_64 2022-10-18 14:13:19 -06:00
ipc
ir kselftests/ir : Improve readability of modprobe error message 2022-05-16 13:34:19 -06:00
kcmp selftests/kcmp: Make the test output consistent and clear 2022-07-08 10:55:43 -06:00
kexec selftests/kexec: fix build for ARCH=x86_64 2022-10-18 14:13:25 -06:00
kmod
kselftest selftests/kselftest/runner.sh: Pass optional command parameters in environment 2022-02-23 17:14:36 -07:00
kvm KVM: selftests: add svm part to triple_fault_test 2022-11-17 11:40:00 -05:00
landlock selftests/landlock: Build without static libraries 2022-10-19 22:10:56 +02:00
lib
livepatch Merge branch 'for-6.1/sysfs-patched-object' into for-linus 2022-10-05 13:00:03 +02:00
lkdtm lkdtm: Update tests for memcpy() run-time warnings 2022-09-07 16:37:27 -07:00
locking
media_tests
membarrier
memfd selftests/memfd: clean up mapping in mfd_fail_write 2022-02-26 09:51:17 -08:00
memory-hotplug selftests/memory-hotplug: Remove the redundant warning information 2022-10-18 14:21:18 -06:00
mincore selftests: skip mincore.check_file_mmap when fs lacks needed support 2022-01-27 16:40:45 -07:00
mount
mount_setattr tests: fix idmapped mount_setattr test 2022-02-11 09:08:20 +01:00
move_mount_set_group
mqueue selftests: mqueue: drop duplicate min definition 2022-04-19 19:28:47 -06:00
nci NFC: nci: Extend virtual NCI deinit test 2022-11-21 10:49:58 +00:00
net selftests/net: add csum offload test 2022-11-29 21:24:32 -08:00
netfilter selftests: netfilter: Fix and review rpath.sh 2022-11-09 10:29:57 +01:00
nolibc selftests/nolibc: Avoid generated files being committed 2022-08-31 05:17:45 -07:00
nsfs
ntb
openat2 selftests: openat2: Skip testcases that fail with EOPNOTSUPP 2022-01-27 16:31:07 -07:00
perf_events selftests/perf_events: Add a SIGTRAP stress test with disables 2022-10-17 16:32:06 +02:00
pid_namespace selftests: fix header dependency for pid_namespace selftests 2022-04-04 13:32:31 -06:00
pidfd selftests/pidfd_test: Remove the erroneous ',' 2022-11-02 03:09:57 -06:00
powerpc selftests/powerpc: Update bhrb filter sampling test for multiple branch filters 2022-09-28 19:22:13 +10:00
prctl
proc proc: test how it holds up with mapping'less process 2022-10-11 18:51:11 -07:00
pstore
ptp testptp: add option to shift clock by nanoseconds 2022-02-22 17:03:40 -08:00
ptrace
rcutorture torture: Create kvm-check-branches.sh output in proper location 2022-06-21 15:57:04 -07:00
resctrl selftests/resctrl: Fix null pointer dereference on open failed 2022-04-26 09:20:00 -06:00
rlimits
rseq selftests/rseq: check if libc rseq support is registered 2022-06-28 09:08:28 +02:00
rtc selftests/rtc: continuously read RTC in a loop for 30s 2022-02-25 17:00:51 -07:00
safesetid LSM: SafeSetID: add setgroups() testing to selftest 2022-07-15 18:24:42 +00:00
sched
seccomp selftests/seccomp: Fix compile warning when CC=clang 2022-07-27 12:12:16 -07:00
sgx selftests/sgx: Ignore OpenSSL 3.0 deprecated functions warning 2022-08-15 16:50:07 -06:00
sigaltstack
size
sparc64
splice
static_keys
sync remove CONFIG_ANDROID 2022-07-01 10:41:09 +02:00
syscall_user_dispatch
sysctl selftests/sysctl: add sysctl macro test 2022-05-03 10:15:07 +02:00
tc-testing selftests: tc-testing: Add matchJSON to tdc 2022-10-26 20:22:33 -07:00
timens Revert "selftests/timens: add a test for vfork+exit" 2022-09-13 10:38:43 -07:00
timers selftests: timers: clocksource-switch: adapt to kselftest framework 2022-07-14 14:36:52 -06:00
tmpfs
tpm2 selftest: tpm2: Add Client.__del__() to close /dev/tpm* handle 2022-10-05 00:25:56 +03:00
uevent
user
user_events tracing/user_events: Use bits vs bytes for enabled status page data 2022-09-29 10:17:37 -04:00
vDSO selftests/vDSO: fix array_size.cocci warning 2022-04-04 13:27:11 -06:00
vm - Alistair Popple has a series which addresses a race which causes page 2022-10-14 12:28:43 -07:00
watchdog
wireguard wireguard: selftests: do not install headers on UML 2022-09-20 11:26:14 -07:00
x86 selftests/x86/corrupt_xstate_header: Use provided __cpuid_count() macro 2022-04-25 15:13:03 -06:00
zram selftests/zram: Adapt the situation that /dev/zram0 is being used 2022-01-27 10:59:02 -07:00
.gitignore selftests: add kselftest_install to .gitignore 2022-03-03 15:16:57 -07:00
gen_kselftest_tar.sh
kselftest_deps.sh selftests: Make the usage formatting consistent in kselftest_deps.sh 2022-06-27 14:14:27 -06:00
kselftest_harness.h selftests/harness: Pass variant to teardown 2022-04-04 13:37:48 -06:00
kselftest_install.sh
kselftest_module.h selftest: Taint kernel when test module loaded 2022-07-11 16:58:11 -06:00
kselftest.h selftests: Provide local define of __cpuid_count() 2022-04-25 15:12:36 -06:00
lib.mk selftests: net: Fix cross-tree inclusion of scripts 2022-10-20 21:09:22 -07:00
Makefile selftests: add openvswitch selftest suite 2022-10-27 12:31:24 +02:00
run_kselftest.sh