More complex tests often have to spawn a background process,
like a server which will respond to requests or tcpdump.
Add support for creating such processes using the with keyword:
with bkg("my-daemon", ..):
# my-daemon is alive in this block
My initial thought was to add this support to cmd() directly
but it runs the command in the constructor, so by the time
we __enter__ it's too late to make sure we used "background=True".
Second useful helper transplanted from net_helper.sh is
wait_port_listen().
The test itself uses socat, which insists on v6 addresses
being wrapped in [], it's not the only command which requires
this format, so add the wrapped address to env. The hope
is to save test code from checking if address is v6.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-7-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
While writing tests with a lot more cases I got tired of having
to jump back and forth to add the name of the test to the ksft_run()
list. Most unittest frameworks do some name matching, e.g. assume
that functions with names starting with test_ are test cases.
Support similar flow in ksft_run(). Let the author list the desired
prefixes. globals() need to be passed explicitly, IDK how to work
around that.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-6-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a very simple test for testing with a remote system.
Both IPv4 and IPv6 connectivity is optional, later change
will add checks to skip tests based on available addresses.
Using netdevsim:
$ ./run_kselftest.sh -t drivers/net:ping.py
TAP version 13
1..1
# timeout set to 45
# selftests: drivers/net: ping.py
# KTAP version 1
# 1..2
# ok 1 ping.test_v4
# ok 2 ping.test_v6
# # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 1 selftests: drivers/net: ping.py
Command line SSH:
$ NETIF=virbr0 REMOTE_TYPE=ssh REMOTE_ARGS=root@192.168.122.123 \
LOCAL_V4=192.168.122.1 REMOTE_V4=192.168.122.123 \
./tools/testing/selftests/drivers/net/ping.py
KTAP version 1
1..2
ok 1 ping.test_v4
ok 2 ping.test_v6 # SKIP Test requires IPv6 connectivity
# Totals: pass:1 fail:0 xfail:1 xpass:0 skip:0 error:0
Existing devices placed in netns (and using net.config):
$ cat drivers/net/net.config
NETIF=veth0
REMOTE_TYPE=netns
REMOTE_ARGS=red
LOCAL_V4="192.168.1.1"
REMOTE_V4="192.168.1.2"
$ ./run_kselftest.sh -t drivers/net:ping.py
TAP version 13
1..1
# timeout set to 45
# selftests: drivers/net: ping.py
# KTAP version 1
# 1..2
# ok 1 ping.test_v4
# ok 2 ping.test_v6 # SKIP Test requires IPv6 connectivity
# # Totals: pass:1 fail:0 xfail:1 xpass:0 skip:0 error:0
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Define the remote endpoint "model". To execute most meaningful device
driver tests we need to be able to communicate with a remote system,
and have it send traffic to the device under test.
Various test environments will have different requirements.
0) "Local" netdevsim-based testing can simply use net namespaces.
netdevsim supports connecting two devices now, to form a veth-like
construct.
1) Similarly on hosts with multiple NICs, the NICs may be connected
together with a loopback cable or internal device loopback.
One interface may be placed into separate netns, and tests
would proceed much like in the netdevsim case. Note that
the loopback config or the moving of one interface
into a netns is not expected to be part of selftest code.
2) Some systems may need to communicate with the remote endpoint
via SSH.
3) Last but not least environment may have its own custom communication
method.
Fundamentally we only need two operations:
- run a command remotely
- deploy a binary (if some tool we need is built as part of kselftests)
Wrap these two in a class. Use dynamic loading to load the Remote
class. This will allow very easy definition of other communication
methods without bothering upstream code base.
Stick to the "simple" / "no unnecessary abstractions" model for
referring to the remote endpoints. The host / remote object are
passed as an argument to the usual cmd() or ip() invocation.
For example:
ip("link show", json=True, host=remote)
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a "--multi <do-op> <json>" command line to ynl that makes it
possible to add several operations to a single netlink request payload.
The --multi command line option is repeated for each operation.
This is used by the nftables family for transaction batches. For
example:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi batch-begin '{"res-id": 10}' \
--multi newtable '{"name": "test", "nfgen-family": 1}' \
--multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--multi batch-end '{"res-id": 10}'
[None, None, None, None]
It can also be used for bundling get requests:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi gettable '{"name": "test", "nfgen-family": 1}' \
--multi getchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--output-json
[{"name": "test", "use": 1, "handle": 1, "flags": [],
"nfgen-family": 1, "version": 0, "res-id": 2},
{"table": "test", "name": "chain", "handle": 1, "use": 0,
"nfgen-family": 1, "version": 0, "res-id": 2}]
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240418104737.77914-4-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This testcase doesn't work if auditd is running, audit_logread will not
receive any data in that case.
Add a nftables feature test for the reset keyword and skip this test
if that fails.
While at it, do a few minor shellcheck cleanups.
Signed-off-by: Florian Westphal <fw@strlen.de>
Link: https://lore.kernel.org/r/20240418152744.15105-12-fw@strlen.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Intentional changes:
- Use socat instead of netcat
- Use a temporary file instead of pipe, else packets do not match
"-m string" rules, multiple writes to the pipe cause multiple packets,
but this needs only one to work.
Signed-off-by: Florian Westphal <fw@strlen.de>
Link: https://lore.kernel.org/r/20240418152744.15105-6-fw@strlen.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Also do shellcheck cleanups here, no functional changes intended.
When running tests via vng tool, the packetpath insertion test fails:
dd: failed to open '/dev/stdout': Device or resource busy
Just omit 'of=' and this will work as intended.
Signed-off-by: Florian Westphal <fw@strlen.de>
Link: https://lore.kernel.org/r/20240418152744.15105-5-fw@strlen.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
- switch to socat, like other tests
- use buswait helper to test once listener netns is ready
- do not generate multiple input test files, only generate
one and use cleanup hook to remove it, like other temporary files.
Signed-off-by: Florian Westphal <fw@strlen.de>
Link: https://lore.kernel.org/r/20240418152744.15105-2-fw@strlen.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Test cases need to exit with non-zero status if they failed,
we currently don't do that:
# KTAP version 1
# 1..3
# # At /root/ksft-net-drv/drivers/net/./ping.py line 18:
# # Check failed 1 != 2
# not ok 1 ping.test_v4
# ok 2 ping.test_v6
# ok 3 ping.test_tcp
# # Totals: pass:2 fail:1 xfail:0 xpass:0 skip:0 error:0
ok 1 selftests: drivers/net: ping.py
^^^^
It's a bit tempting to make the exit part of ksft_run(),
but that only works well for very trivial setups. We can
revisit this later, if people forget to call ksft_exit().
Link: https://lore.kernel.org/r/20240417231146.2435572-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Totals currently only pay attention to exceptions, if check fails
(say ksft_eq()) the test case will be counted as pass:
# At /ksft/drivers/net/./ping.py line 18:
# Check failed 1 != 2
not ok 1 ping.test_v4
ok 2 ping.test_v6
ok 3 ping.test_tcp
# Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
^^^^^^^^^^^^^
Pay attention to the result.
Fixes: b86761ff63 ("selftests: net: add scaffolding for Netlink tests in Python")
Link: https://lore.kernel.org/r/20240417231146.2435572-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Pull networking fixes from Jakub Kicinski:
"A little calmer than usual, probably just the timing of sub-tree PRs.
Including fixes from netfilter.
Current release - regressions:
- inet: bring NLM_DONE out to a separate recv() again, fix user space
which assumes multiple recv()s will happen and gets blocked forever
- drv: mlx5:
- restore mistakenly dropped parts in register devlink flow
- use channel mdev reference instead of global mdev instance for
coalescing
- acquire RTNL lock before RQs/SQs activation/deactivation
Previous releases - regressions:
- net: change maximum number of UDP segments to 128, fix virtio
compatibility with Windows peers
- usb: ax88179_178a: avoid writing the mac address before first
reading
Previous releases - always broken:
- sched: fix mirred deadlock on device recursion
- netfilter:
- br_netfilter: skip conntrack input hook for promisc packets
- fixes removal of duplicate elements in the pipapo set backend
- various fixes for abort paths and error handling
- af_unix: don't peek OOB data without MSG_OOB
- drv: flower: fix fragment flags handling in multiple drivers
- drv: ravb: fix jumbo frames and packet stats accounting
Misc:
- kselftest_harness: fix Clang warning about zero-length format
- tun: limit printing rate when illegal packet received by tun dev"
* tag 'net-6.9-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (46 commits)
net: ethernet: ti: am65-cpsw-nuss: cleanup DMA Channels before using them
net: usb: ax88179_178a: avoid writing the mac address before first reading
net: ravb: Fix RX byte accounting for jumbo packets
net: ravb: Fix GbEth jumbo packet RX checksum handling
net: ravb: Allow RX loop to move past DMA mapping errors
net: ravb: Count packets instead of descriptors in R-Car RX path
net: ethernet: mtk_eth_soc: fix WED + wifi reset
net:usb:qmi_wwan: support Rolling modules
selftests: kselftest_harness: fix Clang warning about zero-length format
net/sched: Fix mirred deadlock on device recursion
netfilter: nf_tables: fix memleak in map from abort path
netfilter: nf_tables: restore set elements when delete set fails
netfilter: nf_tables: missing iterator type in lookup walk
s390/ism: Properly fix receive message buffer allocation
net: dsa: mt7530: fix port mirroring for MT7988 SoC switch
net: dsa: mt7530: fix mirroring frames received on local port
tun: limit printing rate when illegal packet received by tun dev
ice: Fix checking for unsupported keys on non-tunnel device
ice: tc: allow zero flags in parsing tc flower
ice: tc: check src_vsi in case of traffic from VF
...
Character sequences starting with `\` are interpreted by python as
escaped Unicode characters. However, they have other meaning in
regular expressions (e.g: "\d").
It seems Python >= 3.12 starts emitting a SyntaxWarning when these
escaped sequences are not recognized as valid Unicode characters.
An example of these warnings:
tools/testing/selftests/net/openvswitch/ovs-dpctl.py:505:
SyntaxWarning: invalid escape sequence '\d'
Fix all the warnings by flagging literals as raw strings.
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Reviewed-by: Aaron Conole <aconole@redhat.com>
Link: https://lore.kernel.org/r/20240416090913.2028475-1-amorenoz@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
selftest build is fairly noisy, it's easy to miss warnings.
It's standard practice to add alternative messages in
the Makefile. I was grepping for existing solutions,
and found that bpf already has the right knobs.
Move them to lib.mk and adopt in net.
Convert the basic rules in lib.mk.
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/r/20240411190534.444918-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Netfilter tests have been moved to a subdir under selftests/net by
patch series [1]. Fix the path in selftests/Makefile accordingly.
This helps fix the following error:
tools/testing/selftests$ make
...
make[1]: Entering directory 'tools/testing/selftests'
make[1]: *** netfilter: No such file or directory. Stop.
make[1]: Leaving directory 'tools/testing/selftests'
[1] https://lore.kernel.org/all/20240411233624.8129-1-fw@strlen.de/
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Yujie Liu <yujie.liu@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
On my new laptop with packages from nixos-unstable, gcc 12.3.0 produces
> lib/setup.c: In function ‘__test_msg’:
> lib/setup.c:20:9: error: format not a string literal and no format arguments [-Werror=format-security]
> 20 | ksft_print_msg(buf);
> | ^~~~~~~~~~~~~~
> lib/setup.c: In function ‘__test_ok’:
> lib/setup.c:26:9: error: format not a string literal and no format arguments [-Werror=format-security]
> 26 | ksft_test_result_pass(buf);
> | ^~~~~~~~~~~~~~~~~~~~~
> lib/setup.c: In function ‘__test_fail’:
> lib/setup.c:32:9: error: format not a string literal and no format arguments [-Werror=format-security]
> 32 | ksft_test_result_fail(buf);
> | ^~~~~~~~~~~~~~~~~~~~~
> lib/setup.c: In function ‘__test_xfail’:
> lib/setup.c:38:9: error: format not a string literal and no format arguments [-Werror=format-security]
> 38 | ksft_test_result_xfail(buf);
> | ^~~~~~~~~~~~~~~~~~~~~~
> lib/setup.c: In function ‘__test_error’:
> lib/setup.c:44:9: error: format not a string literal and no format arguments [-Werror=format-security]
> 44 | ksft_test_result_error(buf);
> | ^~~~~~~~~~~~~~~~~~~~~~
> lib/setup.c: In function ‘__test_skip’:
> lib/setup.c:50:9: error: format not a string literal and no format arguments [-Werror=format-security]
> 50 | ksft_test_result_skip(buf);
> | ^~~~~~~~~~~~~~~~~~~~~
> cc1: some warnings being treated as errors
As the buffer was already pre-printed into, print it as a string
rather than a format-string.
Fixes: cfbab37b3d ("selftests/net: Add TCP-AO library")
Signed-off-by: Dmitry Safonov <0x7f454c46@gmail.com>
Reported-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
On my new laptop with packages from nixos-unstable, gcc 12.3.0 produces:
> lib/proc.c: In function ‘netstat_read_type’:
> lib/proc.c:89:9: error: format not a string literal and no format arguments [-Werror=format-security]
> 89 | if (fscanf(fnetstat, type->header_name) == EOF)
> | ^~
> cc1: some warnings being treated as errors
Here the selftests lib parses header name, while expectes non-space word
ending with a column.
Fixes: cfbab37b3d ("selftests/net: Add TCP-AO library")
Signed-off-by: Dmitry Safonov <0x7f454c46@gmail.com>
Reported-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
The structure is on the stack and has to be zero-initialized as
the kernel checks for:
> if (in.reserved != 0 || in.reserved2 != 0)
> return -EINVAL;
Fixes: b26660531c ("selftests/net: Add test for TCP-AO add setsockopt() command")
Signed-off-by: Dmitry Safonov <0x7f454c46@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Currently, "active reset" cases are flaky, because select() is called
for 3 sockets, while only 2 are expected to receive RST.
The idea of the third socket was to get into request_sock_queue,
but the test mistakenly attempted to connect() after the listener
socket was shut down.
Repair this test, it's important to check the different kernel
code-paths for signing RST TCP-AO segments.
Fixes: c6df7b2361 ("selftests/net: Add TCP-AO RST test")
Reported-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Dmitry Safonov <0x7f454c46@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
This test lacks a topology diagram, making the setup not obvious.
Add one.
Signed-off-by: Petr Machata <petrm@nvidia.com>
Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
This test lacks a topology diagram, making the setup not obvious.
Add one.
Signed-off-by: Petr Machata <petrm@nvidia.com>
Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
The ethtool dump includes the lanes parameter only when the port is up.
Therefore, the ethtool_lanes.sh test waits for ports to come before testing
the lanes parameter.
In some cases, the test considers the port as up, but the lanes parameter
is not yet dumped although assumed to be, resulting in ethtool_lanes.sh
test failure.
To avoid that, ensure that the lanes parameter is indeed dumped by waiting
for it explicitly, before preforming the test cases.
Signed-off-by: Danielle Ratson <danieller@nvidia.com>
Reviewed-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
The tests use the constant TC_HIT_TIMEOUT when waiting on the counter
values. However it does not include tc_common.sh where the counter is
specified. The test has been robust in our testing, which means the counter
is bumped quickly enough that the updated value is available already on the
first iteration. Nevertheless it's not correct. Include tc_common.sh as
appropriate.
Signed-off-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Some log_test calls are done in a loop, and lead to the same log output.
This might prove tricky to deduplicate for automated tools. Instead, roll
the unique information from log_info to log_test, and drop the log_info.
This also leads to more compact and clearer output.
This change prompts rewording the messages so that they are not excessively
long.
Some check_err messages do not indicate what the issue actually is, so
reword them to say it's a "ping with", like is the case in some other
instances in this test.
Signed-off-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
When rx-pktsNtoM reports a range that involves very low-valued range, such
as 0-64, the calculated length of the packet will be -4, because FCS is
subtracted from the value. mausezahn then confuses the value for an option
and bails out. As a result, the test dumps many mausezahn error messages.
Instead, cap the value at 0. mausezahn will use an appropriate minimum
packet length.
Cc: Vladimir Oltean <vladimir.oltean@nxp.com>
Cc: Tobias Waldekranz <tobias@waldekranz.com>
Signed-off-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
$ksft_skip is used to mark selftests that have tooling issues. The fact
that LLDPad is running, but shouldn't, is one such issue. Therefore have
bail_on_lldpad() bail with $ksft_skip.
Signed-off-by: Petr Machata <petrm@nvidia.com>
Reviewed-by: Benjamin Poirier <bpoirier@nvidia.com>
Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
The variable should contain at least NUM_NETIFS interfaces, stored
as keys named "p$i", for i in `seq $NUM_NETIFS`.
Signed-off-by: Petr Machata <petrm@nvidia.com>
Reviewed-by: Benjamin Poirier <bpoirier@nvidia.com>
Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Bodies of busywait() and slowwait() functions are almost identical. Extract
the common code into a helper, loopy_wait, and convert busywait() and
slowwait() into trivial wrappers.
Moreover, the fact that slowwait() uses seconds for units is really not
intuitive, and the comment does not help much. Instead make the unit part
of the name of the argument to further clarify what units are expected.
Cc: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Petr Machata <petrm@nvidia.com>
Reviewed-by: Benjamin Poirier <bpoirier@nvidia.com>
Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Developing Python tests is a bit annoying because when test fails
we only print the fail message and no info about which exact check
led to it. Print the location (the first line of this example is new):
# At /root/ksft-net-drv/./net/nl_netdev.py line 38:
# Check failed 0 != 10
not ok 3 nl_netdev.page_pool_check
Reviewed-by: Petr Machata <petrm@nvidia.com>
Link: https://lore.kernel.org/r/20240412141436.828666-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
