linux/tools/testing/selftests/tc-testing
Linus Torvalds f86d1fbbe7 Networking changes for 6.0.
Core
 ----
 
  - Refactor the forward memory allocation to better cope with memory
    pressure with many open sockets, moving from a per socket cache to
    a per-CPU one
 
  - Replace rwlocks with RCU for better fairness in ping, raw sockets
    and IP multicast router.
 
  - Network-side support for IO uring zero-copy send.
 
  - A few skb drop reason improvements, including codegen the source file
    with string mapping instead of using macro magic.
 
  - Rename reference tracking helpers to a more consistent
    netdev_* schema.
 
  - Adapt u64_stats_t type to address load/store tearing issues.
 
  - Refine debug helper usage to reduce the log noise caused by bots.
 
 BPF
 ---
  - Improve socket map performance, avoiding skb cloning on read
    operation.
 
  - Add support for 64 bits enum, to match types exposed by kernel.
 
  - Introduce support for sleepable uprobes program.
 
  - Introduce support for enum textual representation in libbpf.
 
  - New helpers to implement synproxy with eBPF/XDP.
 
  - Improve loop performances, inlining indirect calls when
    possible.
 
  - Removed all the deprecated libbpf APIs.
 
  - Implement new eBPF-based LSM flavor.
 
  - Add type match support, which allow accurate queries to the
    eBPF used types.
 
  - A few TCP congetsion control framework usability improvements.
 
  - Add new infrastructure to manipulate CT entries via eBPF programs.
 
  - Allow for livepatch (KLP) and BPF trampolines to attach to the same
    kernel function.
 
 Protocols
 ---------
 
  - Introduce per network namespace lookup tables for unix sockets,
    increasing scalability and reducing contention.
 
  - Preparation work for Wi-Fi 7 Multi-Link Operation (MLO) support.
 
  - Add support to forciby close TIME_WAIT TCP sockets via user-space
    tools.
 
  - Significant performance improvement for the TLS 1.3 receive path,
    both for zero-copy and not-zero-copy.
 
  - Support for changing the initial MTPCP subflow priority/backup
    status
 
  - Introduce virtually contingus buffers for sockets over RDMA,
    to cope better with memory pressure.
 
  - Extend CAN ethtool support with timestamping capabilities
 
  - Refactor CAN build infrastructure to allow building only the needed
    features.
 
 Driver API
 ----------
 
  - Remove devlink mutex to allow parallel commands on multiple links.
 
  - Add support for pause stats in distributed switch.
 
  - Implement devlink helpers to query and flash line cards.
 
  - New helper for phy mode to register conversion.
 
 New hardware / drivers
 ----------------------
 
  - Ethernet DSA driver for the rockchip mt7531 on BPI-R2 Pro.
 
  - Ethernet DSA driver for the Renesas RZ/N1 A5PSW switch.
 
  - Ethernet DSA driver for the Microchip LAN937x switch.
 
  - Ethernet PHY driver for the Aquantia AQR113C EPHY.
 
  - CAN driver for the OBD-II ELM327 interface.
 
  - CAN driver for RZ/N1 SJA1000 CAN controller.
 
  - Bluetooth: Infineon CYW55572 Wi-Fi plus Bluetooth combo device.
 
 Drivers
 -------
 
  - Intel Ethernet NICs:
    - i40e: add support for vlan pruning
    - i40e: add support for XDP framented packets
    - ice: improved vlan offload support
    - ice: add support for PPPoE offload
 
  - Mellanox Ethernet (mlx5)
    - refactor packet steering offload for performance and scalability
    - extend support for TC offload
    - refactor devlink code to clean-up the locking schema
    - support stacked vlans for bridge offloads
    - use TLS objects pool to improve connection rate
 
  - Netronome Ethernet NICs (nfp):
    - extend support for IPv6 fields mangling offload
    - add support for vepa mode in HW bridge
    - better support for virtio data path acceleration (VDPA)
    - enable TSO by default
 
  - Microsoft vNIC driver (mana)
    - add support for XDP redirect
 
  - Others Ethernet drivers:
    - bonding: add per-port priority support
    - microchip lan743x: extend phy support
    - Fungible funeth: support UDP segmentation offload and XDP xmit
    - Solarflare EF100: add support for virtual function representors
    - MediaTek SoC: add XDP support
 
  - Mellanox Ethernet/IB switch (mlxsw):
    - dropped support for unreleased H/W (XM router).
    - improved stats accuracy
    - unified bridge model coversion improving scalability
      (parts 1-6)
    - support for PTP in Spectrum-2 asics
 
  - Broadcom PHYs
    - add PTP support for BCM54210E
    - add support for the BCM53128 internal PHY
 
  - Marvell Ethernet switches (prestera):
    - implement support for multicast forwarding offload
 
  - Embedded Ethernet switches:
    - refactor OcteonTx MAC filter for better scalability
    - improve TC H/W offload for the Felix driver
    - refactor the Microchip ksz8 and ksz9477 drivers to share
      the probe code (parts 1, 2), add support for phylink
      mac configuration
 
  - Other WiFi:
    - Microchip wilc1000: diable WEP support and enable WPA3
    - Atheros ath10k: encapsulation offload support
 
 Old code removal:
 
  - Neterion vxge ethernet driver: this is untouched since more than
    10 years.
 
 Signed-off-by: Paolo Abeni <pabeni@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEg1AjqC77wbdLX2LbKSR5jcyPE6QFAmLqN+oSHHBhYmVuaUBy
 ZWRoYXQuY29tAAoJECkkeY3MjxOkB9kQAI9VqW0c3SfiTJnkVBEIovZ6Tnh5stD2
 UYFkh1BdchLsYxi7W4XMpVPSzRztiTP87mIx5c/KvIzj+QNeWL1XWRJSPdI9HhTD
 pTAA/tM2OG7bqrbyQiKDNfpQdNl7+kk1RwnYd+f9RFl1QVuIJaYhmjVwrsN5xF/+
 jUsotpROarM2dGFWiFwJbKhP2zMDT+6qEEahM8pEPggKhv8wRLYjany2cZVEe4e0
 WGUpbINAS8gEKm0Ob922WaDfDrcK/N1Z0jNz/kMaENkK18Vvc7F6bCO0DzAawKX9
 QZMMwm6mHp3EThflJAMAzCGIYiIcwLhykgdyj8rrjPhFrWbMD2Sdsbo21HOXU/8j
 u4aAhVl+d+h7emmbgBoJ8sycVJ7BQlXz7lX20sTgADv9xI4/dPhQ17CMRuwX6fXX
 JSrn6P6e1LTV5CEg6vrlSPnKPY6uhFn/cPw47FxCjRwJ9phVnp+8uZWQmf9Pz3yf
 Ok/tcj+juFbsmuOshHy2cbRkuNZNS0oRWlSTBo5795ZwOLSakMonR3L+ev2aOvzz
 DVrFp2Y/iIVwMSFdCbouYdYnhArPRhOAtCmZc2afY8aBN7aaMgrdTy3+mzUoHy3I
 FG3K+VuKpfi0vY4zn6ZoLZDIpyXIoJJ93RcSGltD32t3Dp1RaQMVEI4s45k05PVm
 1nYpXKHA8qML
 =hxEG
 -----END PGP SIGNATURE-----

Merge tag 'net-next-6.0' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

Pull networking changes from Paolo Abeni:
 "Core:

   - Refactor the forward memory allocation to better cope with memory
     pressure with many open sockets, moving from a per socket cache to
     a per-CPU one

   - Replace rwlocks with RCU for better fairness in ping, raw sockets
     and IP multicast router.

   - Network-side support for IO uring zero-copy send.

   - A few skb drop reason improvements, including codegen the source
     file with string mapping instead of using macro magic.

   - Rename reference tracking helpers to a more consistent netdev_*
     schema.

   - Adapt u64_stats_t type to address load/store tearing issues.

   - Refine debug helper usage to reduce the log noise caused by bots.

  BPF:

   - Improve socket map performance, avoiding skb cloning on read
     operation.

   - Add support for 64 bits enum, to match types exposed by kernel.

   - Introduce support for sleepable uprobes program.

   - Introduce support for enum textual representation in libbpf.

   - New helpers to implement synproxy with eBPF/XDP.

   - Improve loop performances, inlining indirect calls when possible.

   - Removed all the deprecated libbpf APIs.

   - Implement new eBPF-based LSM flavor.

   - Add type match support, which allow accurate queries to the eBPF
     used types.

   - A few TCP congetsion control framework usability improvements.

   - Add new infrastructure to manipulate CT entries via eBPF programs.

   - Allow for livepatch (KLP) and BPF trampolines to attach to the same
     kernel function.

  Protocols:

   - Introduce per network namespace lookup tables for unix sockets,
     increasing scalability and reducing contention.

   - Preparation work for Wi-Fi 7 Multi-Link Operation (MLO) support.

   - Add support to forciby close TIME_WAIT TCP sockets via user-space
     tools.

   - Significant performance improvement for the TLS 1.3 receive path,
     both for zero-copy and not-zero-copy.

   - Support for changing the initial MTPCP subflow priority/backup
     status

   - Introduce virtually contingus buffers for sockets over RDMA, to
     cope better with memory pressure.

   - Extend CAN ethtool support with timestamping capabilities

   - Refactor CAN build infrastructure to allow building only the needed
     features.

  Driver API:

   - Remove devlink mutex to allow parallel commands on multiple links.

   - Add support for pause stats in distributed switch.

   - Implement devlink helpers to query and flash line cards.

   - New helper for phy mode to register conversion.

  New hardware / drivers:

   - Ethernet DSA driver for the rockchip mt7531 on BPI-R2 Pro.

   - Ethernet DSA driver for the Renesas RZ/N1 A5PSW switch.

   - Ethernet DSA driver for the Microchip LAN937x switch.

   - Ethernet PHY driver for the Aquantia AQR113C EPHY.

   - CAN driver for the OBD-II ELM327 interface.

   - CAN driver for RZ/N1 SJA1000 CAN controller.

   - Bluetooth: Infineon CYW55572 Wi-Fi plus Bluetooth combo device.

  Drivers:

   - Intel Ethernet NICs:
      - i40e: add support for vlan pruning
      - i40e: add support for XDP framented packets
      - ice: improved vlan offload support
      - ice: add support for PPPoE offload

   - Mellanox Ethernet (mlx5)
      - refactor packet steering offload for performance and scalability
      - extend support for TC offload
      - refactor devlink code to clean-up the locking schema
      - support stacked vlans for bridge offloads
      - use TLS objects pool to improve connection rate

   - Netronome Ethernet NICs (nfp):
      - extend support for IPv6 fields mangling offload
      - add support for vepa mode in HW bridge
      - better support for virtio data path acceleration (VDPA)
      - enable TSO by default

   - Microsoft vNIC driver (mana)
      - add support for XDP redirect

   - Others Ethernet drivers:
      - bonding: add per-port priority support
      - microchip lan743x: extend phy support
      - Fungible funeth: support UDP segmentation offload and XDP xmit
      - Solarflare EF100: add support for virtual function representors
      - MediaTek SoC: add XDP support

   - Mellanox Ethernet/IB switch (mlxsw):
      - dropped support for unreleased H/W (XM router).
      - improved stats accuracy
      - unified bridge model coversion improving scalability (parts 1-6)
      - support for PTP in Spectrum-2 asics

   - Broadcom PHYs
      - add PTP support for BCM54210E
      - add support for the BCM53128 internal PHY

   - Marvell Ethernet switches (prestera):
      - implement support for multicast forwarding offload

   - Embedded Ethernet switches:
      - refactor OcteonTx MAC filter for better scalability
      - improve TC H/W offload for the Felix driver
      - refactor the Microchip ksz8 and ksz9477 drivers to share the
        probe code (parts 1, 2), add support for phylink mac
        configuration

   - Other WiFi:
      - Microchip wilc1000: diable WEP support and enable WPA3
      - Atheros ath10k: encapsulation offload support

  Old code removal:

   - Neterion vxge ethernet driver: this is untouched since more than 10 years"

* tag 'net-next-6.0' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1890 commits)
  doc: sfp-phylink: Fix a broken reference
  wireguard: selftests: support UML
  wireguard: allowedips: don't corrupt stack when detecting overflow
  wireguard: selftests: update config fragments
  wireguard: ratelimiter: use hrtimer in selftest
  net/mlx5e: xsk: Discard unaligned XSK frames on striding RQ
  net: usb: ax88179_178a: Bind only to vendor-specific interface
  selftests: net: fix IOAM test skip return code
  net: usb: make USB_RTL8153_ECM non user configurable
  net: marvell: prestera: remove reduntant code
  octeontx2-pf: Reduce minimum mtu size to 60
  net: devlink: Fix missing mutex_unlock() call
  net/tls: Remove redundant workqueue flush before destroy
  net: txgbe: Fix an error handling path in txgbe_probe()
  net: dsa: Fix spelling mistakes and cleanup code
  Documentation: devlink: add add devlink-selftests to the table of contents
  dccp: put dccp_qpolicy_full() and dccp_qpolicy_push() in the same lock
  net: ionic: fix error check for vlan flags in ionic_set_nic_features()
  net: ice: fix error NETIF_F_HW_VLAN_CTAG_FILTER check in ice_vsi_sync_fltr()
  nfp: flower: add support for tunnel offload without key ID
  ...
2022-08-03 16:29:08 -07:00
..
creating-plugins tools: tc-testing: Introduce plugin architecture 2018-02-15 15:38:33 -05:00
creating-testcases tc-testing: introduce scapyPlugin for basic traffic 2019-07-09 14:07:20 -07:00
plugin-lib tc-testing: add support for sending various scapy packets 2021-06-22 10:52:39 -07:00
plugins tools: tc-testing: Introduce plugin architecture 2018-02-15 15:38:33 -05:00
tc-tests selftests: tc-testing: Add testcases to test new flush behaviour 2022-06-27 21:51:28 -07:00
.gitignore tc-testing: gitignore, delete plugins directory 2022-06-24 16:23:47 -07:00
action.c tc-testing: Add tdc to kselftests 2020-07-20 18:29:37 -07:00
config selftests/tc-testing: Fix cannot create /sys/bus/netdevsim/new_device: Directory nonexistent 2021-12-03 11:46:41 +00:00
Makefile selftests: stop using KSFT_KHDR_INSTALL 2022-07-11 17:23:30 -06:00
README tc-testing: Clarify the use of tdc's -d option 2019-07-31 18:54:34 -04:00
tdc_batch.py tweewide: Fix most Shebang lines 2020-12-08 23:30:04 +09:00
tdc_config_local_template.py tc-testing: split config file 2017-11-01 11:04:47 +09:00
tdc_config.py selftests: tc-testing: Increase timeout in tdc config file 2022-03-14 10:27:52 +00:00
tdc_helper.py tc-testing: Restore original behaviour for namespaces in tdc 2019-06-25 12:57:29 -07:00
tdc_multibatch.py tweewide: Fix most Shebang lines 2020-12-08 23:30:04 +09:00
tdc.py selftests/tc-testing: add exit code 2021-12-03 11:46:41 +00:00
tdc.sh selftests/tc-testing: Fix cannot create /sys/bus/netdevsim/new_device: Directory nonexistent 2021-12-03 11:46:41 +00:00
TdcPlugin.py tc-testing: Allow tdc plugins to see test case data 2019-07-09 14:07:20 -07:00
TdcResults.py tc-testing: Add new TdcResults module 2018-12-07 16:39:03 -08:00
TODO.txt tools: tc-testing: Update README and TODO 2018-02-15 15:38:33 -05:00

tdc - Linux Traffic Control (tc) unit testing suite

Author: Lucas Bates - lucasb@mojatatu.com

tdc is a Python script to load tc unit tests from a separate JSON file and
execute them inside a network namespace dedicated to the task.


REQUIREMENTS
------------

*  Minimum Python version of 3.4. Earlier 3.X versions may work but are not
   guaranteed.

*  The kernel must have network namespace support if using nsPlugin

*  The kernel must have veth support available, as a veth pair is created
   prior to running the tests when using nsPlugin.

*  The kernel must have the appropriate infrastructure enabled to run all tdc
   unit tests. See the config file in this directory for minimum required
   features. As new tests will be added, config options list will be updated.

*  All tc-related features being tested must be built in or available as
   modules.  To check what is required in current setup run:
   ./tdc.py -c

   Note:
   In the current release, tdc run will abort due to a failure in setup or
   teardown commands - which includes not being able to run a test simply
   because the kernel did not support a specific feature. (This will be
   handled in a future version - the current workaround is to run the tests
   on specific test categories that your kernel supports)


BEFORE YOU RUN
--------------

The path to the tc executable that will be most commonly tested can be defined
in the tdc_config.py file. Find the 'TC' entry in the NAMES dictionary and
define the path.

If you need to test a different tc executable on the fly, you can do so by
using the -p option when running tdc:
	./tdc.py -p /path/to/tc


RUNNING TDC
-----------

To use tdc, root privileges are required.  This is because the
commands being tested must be run as root.  The code that enforces
execution by root uid has been moved into a plugin (see PLUGIN
ARCHITECTURE, below).

Tests that use a network device should have nsPlugin.py listed as a
requirement for that test. nsPlugin executes all commands within a
network namespace and creates a veth pair which may be used in those test
cases. To disable execution within the namespace, pass the -N option
to tdc when starting a test run; the veth pair will still be created
by the plugin.

Running tdc without any arguments will run all tests. Refer to the section
on command line arguments for more information, or run:
	./tdc.py -h

tdc will list the test names as they are being run, and print a summary in
TAP (Test Anything Protocol) format when they are done. If tests fail,
output captured from the failing test will be printed immediately following
the failed test in the TAP output.


OVERVIEW OF TDC EXECUTION
-------------------------

One run of tests is considered a "test suite" (this will be refined in the
future).  A test suite has one or more test cases in it.

A test case has four stages:

  - setup
  - execute
  - verify
  - teardown

The setup and teardown stages can run zero or more commands.  The setup
stage does some setup if the test needs it.  The teardown stage undoes
the setup and returns the system to a "neutral" state so any other test
can be run next.  These two stages require any commands run to return
success, but do not otherwise verify the results.

The execute and verify stages each run one command.  The execute stage
tests the return code against one or more acceptable values.  The
verify stage checks the return code for success, and also compares
the stdout with a regular expression.

Each of the commands in any stage will run in a shell instance.


USER-DEFINED CONSTANTS
----------------------

The tdc_config.py file contains multiple values that can be altered to suit
your needs. Any value in the NAMES dictionary can be altered without affecting
the tests to be run. These values are used in the tc commands that will be
executed as part of the test. More will be added as test cases require.

Example:
	$TC qdisc add dev $DEV1 ingress

The NAMES values are used to substitute into the commands in the test cases.


COMMAND LINE ARGUMENTS
----------------------

Run tdc.py -h to see the full list of available arguments.

usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]]
              [-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v] [-N]
              [-d DEVICE] [-P] [-n] [-V]

Linux TC unit tests

optional arguments:
  -h, --help            show this help message and exit
  -p PATH, --path PATH  The full path to the tc executable to use
  -v, --verbose         Show the commands that are being run
  -N, --notap           Suppress tap results for command under test
  -d DEVICE, --device DEVICE
                        Execute test cases that use a physical device, where
                        DEVICE is its name. (If not defined, tests that require
                        a physical device will be skipped)
  -P, --pause           Pause execution just before post-suite stage

selection:
  select which test cases: files plus directories; filtered by categories
  plus testids

  -D DIR [DIR ...], --directory DIR [DIR ...]
                        Collect tests from the specified directory(ies)
                        (default [tc-tests])
  -f FILE [FILE ...], --file FILE [FILE ...]
                        Run tests from the specified file(s)
  -c [CATG [CATG ...]], --category [CATG [CATG ...]]
                        Run tests only from the specified category/ies, or if
                        no category/ies is/are specified, list known
                        categories.
  -e ID [ID ...], --execute ID [ID ...]
                        Execute the specified test cases with specified IDs

action:
  select action to perform on selected test cases

  -l, --list            List all test cases, or those only within the
                        specified category
  -s, --show            Display the selected test cases
  -i, --id              Generate ID numbers for new test cases

netns:
  options for nsPlugin (run commands in net namespace)

  -N, --no-namespace
                        Do not run commands in a network namespace.

valgrind:
  options for valgrindPlugin (run command under test under Valgrind)

  -V, --valgrind        Run commands under valgrind


PLUGIN ARCHITECTURE
-------------------

There is now a plugin architecture, and some of the functionality that
was in the tdc.py script has been moved into the plugins.

The plugins are in the directory plugin-lib.  The are executed from
directory plugins.  Put symbolic links from plugins to plugin-lib,
and name them according to the order you want them to run. This is not
necessary if a test case being run requires a specific plugin to work.

Example:

bjb@bee:~/work/tc-testing$ ls -l plugins
total 4
lrwxrwxrwx  1 bjb  bjb    27 Oct  4 16:12 10-rootPlugin.py -> ../plugin-lib/rootPlugin.py
lrwxrwxrwx  1 bjb  bjb    25 Oct 12 17:55 20-nsPlugin.py -> ../plugin-lib/nsPlugin.py
-rwxr-xr-x  1 bjb  bjb     0 Sep 29 15:56 __init__.py

The plugins are a subclass of TdcPlugin, defined in TdcPlugin.py and
must be called "SubPlugin" so tdc can find them.  They are
distinguished from each other in the python program by their module
name.

This base class supplies "hooks" to run extra functions.  These hooks are as follows:

pre- and post-suite
pre- and post-case
pre- and post-execute stage
adjust-command (runs in all stages and receives the stage name)

The pre-suite hook receives the number of tests and an array of test ids.
This allows you to dump out the list of skipped tests in the event of a
failure during setup or teardown stage.

The pre-case hook receives the ordinal number and test id of the current test.

The adjust-command hook receives the stage id (see list below) and the
full command to be executed.  This allows for last-minute adjustment
of the command.

The stages are identified by the following strings:

  - pre  (pre-suite)
  - setup
  - command
  - verify
  - teardown
  - post (post-suite)


To write a plugin, you need to inherit from TdcPlugin in
TdcPlugin.py.  To use the plugin, you have to put the
implementation file in plugin-lib, and add a symbolic link to it from
plugins.  It will be detected at run time and invoked at the
appropriate times.  There are a few examples in the plugin-lib
directory:

  - rootPlugin.py:
      implements the enforcement of running as root
  - nsPlugin.py:
      sets up a network namespace and runs all commands in that namespace,
      while also setting up dummy devices to be used in testing.
  - valgrindPlugin.py
      runs each command in the execute stage under valgrind,
      and checks for leaks.
      This plugin will output an extra test for each test in the test file,
      one is the existing output as to whether the test passed or failed,
      and the other is a test whether the command leaked memory or not.
      (This one is a preliminary version, it may not work quite right yet,
      but the overall template is there and it should only need tweaks.)
  - buildebpfPlugin.py:
      builds all programs in $EBPFDIR.


ACKNOWLEDGEMENTS
----------------

Thanks to:

Jamal Hadi Salim, for providing valuable test cases
Keara Leibovitz, who wrote the CLI test driver that I used as a base for the
   first version of the tc testing suite. This work was presented at
   Netdev 1.2 Tokyo in October 2016.
Samir Hussain, for providing help while I dove into Python for the first time
    and being a second eye for this code.