linux-kselftest-next-5.10-rc1

This kselftest update for Linux 5.10-rc1 consists of enhancements to
 
 -- speed up headers_install done during selftest build
 -- add generic make nesting support
 -- add support to select individual tests:
    - Selftests build/install generates run_kselftest.sh script to run
      selftests on a target system. Currently the script doesn't have
      support for selecting individual tests. Add support for it.
 
      With this enhancement, user can select test collections (or tests)
      individually. e.g:
 
      run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep
 
      Additionally adds a way to list all known tests with "-l", usage
      with "-h", and perform a dry run without running tests with "-n".
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAl+IpLgACgkQCwJExA0N
 QxygsBAAvE07jW7UDErVbpaKWafxmY2gYZCWr0/e2vDGE3BdPfJZjcbEUADIlk8o
 prN2fO17fVE66B5K1ppqRSmLR7GBs/iVVpODW034FF1b0cRSlGFTxsmivxpdCOCy
 ox7In1I5fXyXZx1MD3IoWWuSLP3UtP5O3MMX0q+3nqk/FTV5Qe9XBCNDDsb0aBuV
 /tuQ+TcYec+mnvIlCtRJr01i+NnfDZyHflcyJy4i6GQ8s5O/bweb9iqiyIeEj5YG
 P1GaUdQg7DLKkIpplS8jJvjmDD1uH+qiLKtUvEZ2TB7NSB/NUyW3oEFQmJz2b5j0
 aAwyLh1ApTYZA2GPRBmd1/eS5VhEe/XRQ/cqzxSMYg9adFt6VvxwuRWgtE1GA/Jy
 mRogJjFW+UaTXdFkjyJw3V3d03+YxfmsVVUYqp1kf0rrRkVB7gV3wiTzoJEVao48
 y3xZuni93RGHGdYFBtEmM9hkFgwopMnEv6e2g1ndwdG1Fifw4R/KEDG+/htCIhwG
 iiKCy4BCCDsBM+3fZvQG4wqgBIXIyn5W40+9l/GGKSjB1vZRob3VUva6Cwq4XdPp
 iNICEmFwbfn5RanQFht+8Xkbsh/H+p7SVtCvG3cQbduJKxCyfTrXUKjyKYULG1Pv
 uMtuZxJC9X/SritwTo2K/i+iCYDXfM0YsOfSNMF40BvW+Rtrss4=
 =pf0+
 -----END PGP SIGNATURE-----

Merge tag 'linux-kselftest-next-5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

Pull kselftest updates from Shuah Khan:

 - speed up headers_install done during selftest build

 - add generic make nesting support

 - add support to select individual tests:

   Selftests build/install generates run_kselftest.sh script to run
   selftests on a target system. Currently the script doesn't have
   support for selecting individual tests. Add support for it.

   With this enhancement, user can select test collections (or tests)
   individually. e.g:

      run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep

   Additionally adds a way to list all known tests with "-l", usage with
   "-h", and perform a dry run without running tests with "-n".

* tag 'linux-kselftest-next-5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
  doc: dev-tools: kselftest.rst: Update examples and paths
  selftests/run_kselftest.sh: Make each test individually selectable
  selftests: Extract run_kselftest.sh and generate stand-alone test list
  selftests: Add missing gitignore entries
  selftests: more general make nesting support
  selftests: use "$(MAKE)" instead of "make" for headers_install
This commit is contained in:
Linus Torvalds 2020-10-15 15:14:32 -07:00
commit 0674324b16
7 changed files with 135 additions and 41 deletions

View File

@ -125,32 +125,41 @@ Note that some tests will require root privileges.
Install selftests
=================
You can use the kselftest_install.sh tool to install selftests in the
default location, which is tools/testing/selftests/kselftest, or in a
user specified location.
You can use the "install" target of "make" (which calls the `kselftest_install.sh`
tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
or in a user specified location via the `INSTALL_PATH` "make" variable.
To install selftests in default location::
$ cd tools/testing/selftests
$ ./kselftest_install.sh
$ make -C tools/testing/selftests install
To install selftests in a user specified location::
$ cd tools/testing/selftests
$ ./kselftest_install.sh install_dir
$ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
Running installed selftests
===========================
Kselftest install as well as the Kselftest tarball provide a script
named "run_kselftest.sh" to run the tests.
Found in the install directory, as well as in the Kselftest tarball,
is a script named `run_kselftest.sh` to run the tests.
You can simply do the following to run the installed Kselftests. Please
note some tests will require root privileges::
$ cd kselftest
$ cd kselftest_install
$ ./run_kselftest.sh
To see the list of available tests, the `-l` option can be used::
$ ./run_kselftest.sh -l
The `-c` option can be used to run all the tests from a test collection, or
the `-t` option for specific single tests. Either can be used multiple times::
$ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
For other features see the script usage output, seen with the `-h` option.
Packaging selftests
===================
@ -160,9 +169,9 @@ different system. To package selftests, run::
$ make -C tools/testing/selftests gen_tar
This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
default, `.gz` format is used. The tar format can be overridden by specifying
a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ option
is supported, such as::
default, `.gz` format is used. The tar compression format can be overridden by
specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
option is supported, such as::
$ make -C tools/testing/selftests gen_tar FORMAT=.xz

View File

@ -88,10 +88,10 @@ endif
# of the targets gets built.
FORCE_TARGETS ?=
# Clear LDFLAGS and MAKEFLAGS if called from main
# Makefile to avoid test build failures when test
# Makefile doesn't have explicit build rules.
ifeq (1,$(MAKELEVEL))
# Clear LDFLAGS and MAKEFLAGS when implicit rules are missing. This provides
# implicit rules to sub-test Makefiles which avoids build failures in test
# Makefile that don't have explicit build rules.
ifeq (,$(LINK.c))
override LDFLAGS =
override MAKEFLAGS =
endif
@ -206,6 +206,7 @@ KSFT_INSTALL_PATH := $(abspath $(KSFT_INSTALL_PATH))
# Avoid changing the rest of the logic here and lib.mk.
INSTALL_PATH := $(KSFT_INSTALL_PATH)
ALL_SCRIPT := $(INSTALL_PATH)/run_kselftest.sh
TEST_LIST := $(INSTALL_PATH)/kselftest-list.txt
install: all
ifdef INSTALL_PATH
@ -214,6 +215,8 @@ ifdef INSTALL_PATH
install -m 744 kselftest/module.sh $(INSTALL_PATH)/kselftest/
install -m 744 kselftest/runner.sh $(INSTALL_PATH)/kselftest/
install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/
install -m 744 run_kselftest.sh $(INSTALL_PATH)/
rm -f $(TEST_LIST)
@ret=1; \
for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
@ -222,33 +225,18 @@ ifdef INSTALL_PATH
ret=$$((ret * $$?)); \
done; exit $$ret;
@# Ask all targets to emit their test scripts
echo "#!/bin/sh" > $(ALL_SCRIPT)
echo "BASE_DIR=\$$(realpath \$$(dirname \$$0))" >> $(ALL_SCRIPT)
echo "cd \$$BASE_DIR" >> $(ALL_SCRIPT)
echo ". ./kselftest/runner.sh" >> $(ALL_SCRIPT)
echo "ROOT=\$$PWD" >> $(ALL_SCRIPT)
echo "if [ \"\$$1\" = \"--summary\" ]; then" >> $(ALL_SCRIPT)
echo " logfile=\$$BASE_DIR/output.log" >> $(ALL_SCRIPT)
echo " cat /dev/null > \$$logfile" >> $(ALL_SCRIPT)
echo "fi" >> $(ALL_SCRIPT)
@# While building run_kselftest.sh skip also non-existent TARGET dirs:
@# Ask all targets to emit their test scripts
@# While building kselftest-list.text skip also non-existent TARGET dirs:
@# they could be the result of a build failure and should NOT be
@# included in the generated runlist.
for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
[ ! -d $(INSTALL_PATH)/$$TARGET ] && echo "Skipping non-existent dir: $$TARGET" && continue; \
echo "[ -w /dev/kmsg ] && echo \"kselftest: Running tests in $$TARGET\" >> /dev/kmsg" >> $(ALL_SCRIPT); \
echo "cd $$TARGET" >> $(ALL_SCRIPT); \
echo -n "run_many" >> $(ALL_SCRIPT); \
echo -n "Emit Tests for $$TARGET\n"; \
$(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET -C $$TARGET emit_tests >> $(ALL_SCRIPT); \
echo "" >> $(ALL_SCRIPT); \
echo "cd \$$ROOT" >> $(ALL_SCRIPT); \
$(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET COLLECTION=$$TARGET \
-C $$TARGET emit_tests >> $(TEST_LIST); \
done;
chmod u+x $(ALL_SCRIPT)
else
$(error Error: set INSTALL_PATH to use install)
endif

View File

@ -0,0 +1,2 @@
# SPDX-License-Identifier: GPL-2.0-only
fw_namespace

View File

@ -47,9 +47,9 @@ ARCH ?= $(SUBARCH)
khdr:
ifndef KSFT_KHDR_INSTALL_DONE
ifeq (1,$(DEFAULT_INSTALL_HDR_PATH))
make --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
$(MAKE) --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
else
make --no-builtin-rules INSTALL_HDR_PATH=$$OUTPUT/usr \
$(MAKE) --no-builtin-rules INSTALL_HDR_PATH=$$OUTPUT/usr \
ARCH=$(ARCH) -C $(top_srcdir) headers_install
endif
endif
@ -107,9 +107,8 @@ endif
emit_tests:
for TEST in $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(TEST_PROGS); do \
BASENAME_TEST=`basename $$TEST`; \
echo " \\"; \
echo -n " \"$$BASENAME_TEST\""; \
done; \
echo "$(COLLECTION):$$BASENAME_TEST"; \
done
# define if isn't already. It is undefined in make O= case.
ifeq ($(RM),)

View File

@ -0,0 +1,2 @@
# SPDX-License-Identifier: GPL-2.0-only
nf-queue

View File

@ -1,3 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
get_syscall_info
peeksiginfo
vmaccess

View File

@ -0,0 +1,93 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
#
# Run installed kselftest tests.
#
BASE_DIR=$(realpath $(dirname $0))
cd $BASE_DIR
TESTS="$BASE_DIR"/kselftest-list.txt
if [ ! -r "$TESTS" ] ; then
echo "$0: Could not find list of tests to run ($TESTS)" >&2
available=""
else
available="$(cat "$TESTS")"
fi
. ./kselftest/runner.sh
ROOT=$PWD
usage()
{
cat <<EOF
Usage: $0 [OPTIONS]
-s | --summary Print summary with detailed log in output.log
-t | --test COLLECTION:TEST Run TEST from COLLECTION
-c | --collection COLLECTION Run all tests from COLLECTION
-l | --list List the available collection:test entries
-d | --dry-run Don't actually run any tests
-h | --help Show this usage info
EOF
exit $1
}
COLLECTIONS=""
TESTS=""
dryrun=""
while true; do
case "$1" in
-s | --summary)
logfile="$BASE_DIR"/output.log
cat /dev/null > $logfile
shift ;;
-t | --test)
TESTS="$TESTS $2"
shift 2 ;;
-c | --collection)
COLLECTIONS="$COLLECTIONS $2"
shift 2 ;;
-l | --list)
echo "$available"
exit 0 ;;
-n | --dry-run)
dryrun="echo"
shift ;;
-h | --help)
usage 0 ;;
"")
break ;;
*)
usage 1 ;;
esac
done
# Add all selected collections to the explicit test list.
if [ -n "$COLLECTIONS" ]; then
for collection in $COLLECTIONS ; do
found="$(echo "$available" | grep "^$collection:")"
if [ -z "$found" ] ; then
echo "No such collection '$collection'" >&2
exit 1
fi
TESTS="$TESTS $found"
done
fi
# Replace available test list with explicitly selected tests.
if [ -n "$TESTS" ]; then
valid=""
for test in $TESTS ; do
found="$(echo "$available" | grep "^${test}$")"
if [ -z "$found" ] ; then
echo "No such test '$test'" >&2
exit 1
fi
valid="$valid $found"
done
available="$(echo "$valid" | sed -e 's/ /\n/g')"
fi
collections=$(echo "$available" | cut -d: -f1 | uniq)
for collection in $collections ; do
[ -w /dev/kmsg ] && echo "kselftest: Running tests in $collection" >> /dev/kmsg
tests=$(echo "$available" | grep "^$collection:" | cut -d: -f2)
($dryrun cd "$collection" && $dryrun run_many $tests)
done