linux/tools/testing/kunit
David Gow acd8e8407b kunit: Print test statistics on failure
When a number of tests fail, it can be useful to get higher-level
statistics of how many tests are failing (or how many parameters are
failing in parameterised tests), and in what cases or suites. This is
already done by some non-KUnit tests, so add support for automatically
generating these for KUnit tests.

This change adds a 'kunit.stats_enabled' switch which has three values:
- 0: No stats are printed (current behaviour)
- 1: Stats are printed only for tests/suites with more than one
     subtest (new default)
- 2: Always print test statistics

For parameterised tests, the summary line looks as follows:
"    # inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16"
For test suites, there are two lines looking like this:
"# ext4_inode_test: pass:1 fail:0 skip:0 total:1"
"# Totals: pass:16 fail:0 skip:0 total:16"

The first line gives the number of direct subtests, the second "Totals"
line is the accumulated sum of all tests and test parameters.

This format is based on the one used by kselftest[1].

[1]: https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests/kselftest.h#L109

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-08-13 13:38:31 -06:00
..
configs kunit: Remove the unused all_tests.config 2021-06-23 17:49:41 -06:00
qemu_configs kunit: tool: add support for QEMU 2021-06-11 16:10:23 -06:00
test_data kunit: tool: Fix error messages for cases of no tests and wrong TAP header 2021-07-12 13:53:05 -06:00
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
kunit_config.py kunit: tool: add support for QEMU 2021-06-11 16:10:23 -06:00
kunit_json.py kunit: tool: surface and address more typing issues 2021-01-15 17:49:34 -07:00
kunit_kernel.py kunit: tool: remove unnecessary "annotations" import 2021-07-12 13:54:29 -06:00
kunit_parser.py kunit: Print test statistics on failure 2021-08-13 13:38:31 -06:00
kunit_tool_test.py kunit: tool: make --raw_output support only showing kunit output 2021-08-13 13:32:01 -06:00
kunit.py kunit: tool: make --raw_output support only showing kunit output 2021-08-13 13:32:01 -06:00
qemu_config.py kunit: tool: add support for QEMU 2021-06-11 16:10:23 -06:00