Richard Gobert 4b0ebbca3e
net: gro: move L3 flush checks to tcp_gro_receive and udp_gro_receive_segment ...
{inet,ipv6}_gro_receive functions perform flush checks (ttl, flags,
iph->id, ...) against all packets in a loop. These flush checks are used in
all merging UDP and TCP flows.
These checks need to be done only once and only against the found p skb,
since they only affect flush and not same_flow.
This patch leverages correct network header offsets from the cb for both
outer and inner network headers - allowing these checks to be done only
once, in tcp_gro_receive and udp_gro_receive_segment. As a result,
NAPI_GRO_CB(p)->flush is not used at all. In addition, flush_id checks are
more declarative and contained in inet_gro_flush, thus removing the need
for flush_id in napi_gro_cb.
This results in less parsing code for non-loop flush tests for TCP and UDP
flows.
To make sure results are not within noise range - I've made netfilter drop
all TCP packets, and measured CPU performance in GRO (in this case GRO is
responsible for about 50% of the CPU utilization).
perf top while replaying 64 parallel IP/TCP streams merging in GRO:
(gro_receive_network_flush is compiled inline to tcp_gro_receive)
net-next:
6.94% [kernel] [k] inet_gro_receive
3.02% [kernel] [k] tcp_gro_receive
patch applied:
4.27% [kernel] [k] tcp_gro_receive
4.22% [kernel] [k] inet_gro_receive
perf top while replaying 64 parallel IP/IP/TCP streams merging in GRO (same
results for any encapsulation, in this case inet_gro_receive is top
offender in net-next)
net-next:
10.09% [kernel] [k] inet_gro_receive
2.08% [kernel] [k] tcp_gro_receive
patch applied:
6.97% [kernel] [k] inet_gro_receive
3.68% [kernel] [k] tcp_gro_receive
Signed-off-by: Richard Gobert <richardbgobert@gmail.com >
Reviewed-by: Willem de Bruijn <willemb@google.com >
Link: https://lore.kernel.org/r/20240509190819.2985-3-richardbgobert@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-05-13 14:44:06 -07:00
2023-03-27 02:33:48 +00:00
2024-04-24 16:26:11 -04:00
2023-08-10 18:24:48 -07:00
2024-03-13 09:23:48 +01:00
2024-04-24 11:06:25 -07:00
2024-04-11 07:38:28 -07:00
2024-05-10 11:13:45 +02:00
2023-12-15 11:01:27 +00:00
2024-03-05 11:21:18 -08:00
2024-04-24 11:06:25 -07:00
2023-06-15 22:33:26 -07:00
2024-02-05 11:08:06 +00:00
2024-01-02 12:41:16 +00:00
2024-02-02 10:57:55 -08:00
2024-04-09 17:09:05 -07:00
2023-12-24 15:22:50 +00:00
2024-04-25 12:41:37 -07:00
2023-12-15 10:37:35 +00:00
2022-07-22 12:53:22 +01:00
2023-03-21 21:32:18 -07:00
2024-05-03 13:29:43 +01:00
2022-08-09 22:14:02 -07:00
2021-12-13 12:34:09 +00:00
2024-02-06 13:17:54 +01:00
2022-11-02 20:38:13 -07:00
2024-02-06 13:17:54 +01:00
2024-02-06 13:17:54 +01:00
2024-02-09 10:01:09 -08:00
2022-11-16 11:31:47 +02:00
2024-05-03 10:18:03 +02:00
2024-03-06 21:01:26 +01:00
2023-01-27 11:19:46 +00:00
2021-12-22 15:03:47 -08:00
2022-07-22 12:53:22 +01:00
2023-07-14 20:39:29 -07:00
2022-07-24 18:39:17 -06:00
2023-07-27 17:17:32 -07:00
2022-07-22 12:53:22 +01:00
2023-01-20 09:33:22 +00:00
2024-04-22 13:05:19 -07:00
2024-02-28 10:39:21 +00:00
2023-08-14 08:01:06 +01:00
2023-10-24 13:08:14 -07:00
2024-05-08 10:35:10 +01:00
2024-05-08 10:35:09 +01:00
2024-05-08 18:49:51 -07:00
2024-04-01 10:49:28 +01:00
2023-08-22 13:19:02 +02:00
2024-02-06 11:45:53 +01:00
2024-03-05 19:21:17 -08:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2024-04-01 21:27:08 -07:00
2022-07-22 12:53:22 +01:00
2021-12-06 16:06:02 -08:00
2024-01-03 16:42:48 -08:00
2022-07-28 22:21:54 -07:00
2024-04-01 10:49:28 +01:00
2024-04-15 10:33:14 +01:00
2023-11-02 09:31:02 +01:00
2023-04-12 16:40:39 -07:00
2022-12-01 15:09:10 +01:00
2023-08-08 15:58:23 -07:00
2022-07-22 12:53:22 +01:00
2021-10-18 12:54:41 +01:00
2024-04-01 21:44:34 -07:00
2022-10-31 10:43:04 +00:00
2024-04-01 10:49:28 +01:00
2024-05-13 14:44:06 -07:00
2023-06-10 00:11:41 -07:00
2024-05-07 01:35:55 +02:00
2022-07-22 12:53:22 +01:00
2023-07-28 14:07:59 -07:00
2024-04-30 18:46:52 -07:00
2022-07-22 12:53:22 +01:00
2021-06-28 14:29:45 -07:00
2024-05-08 10:35:09 +01:00
2023-10-12 15:14:27 +03:00
2023-11-20 11:43:03 +01:00
2024-02-26 18:40:34 -08:00
2022-07-22 12:53:22 +01:00
2023-08-15 13:57:51 -07:00
2023-08-14 07:06:13 +01:00
2024-04-02 18:21:13 -07:00
2022-02-07 20:12:45 -08:00
2021-10-15 11:33:08 +01:00
2023-07-14 20:39:29 -07:00
2023-12-22 22:15:35 +00:00
2024-03-06 12:37:06 +00:00
2024-04-01 21:27:58 -07:00
2024-02-28 11:19:41 +00:00
2021-11-16 13:16:54 +00:00
2024-04-29 13:32:01 +01:00
2024-04-29 13:32:01 +01:00
2024-04-01 10:49:28 +01:00
2024-02-26 11:46:12 +00:00
2024-04-11 14:23:47 -07:00
2023-04-22 01:39:41 +02:00
2024-04-30 18:32:38 -07:00
2022-09-29 07:18:00 +02:00
2022-07-22 12:53:22 +01:00
2022-10-31 20:14:27 -07:00
2023-10-30 14:36:57 -07:00
2024-03-01 08:42:31 +00:00
2023-08-22 21:40:40 +02:00
2023-06-12 21:13:23 -07:00
2023-08-04 15:33:17 -07:00
2023-08-04 15:33:17 -07:00
2022-07-22 12:53:22 +01:00
2023-07-20 10:46:28 +02:00
2021-10-13 09:40:46 -07:00
2024-01-19 21:30:09 -08:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2022-07-28 11:29:36 +02:00
2021-11-07 19:25:29 +00:00
2023-08-18 16:05:26 +02:00
2024-05-03 10:18:19 +02:00
2022-10-12 12:57:19 +02:00
2024-04-25 08:20:54 -07:00
2024-02-29 14:24:56 -08:00
2021-10-29 13:23:51 +01:00
2022-07-22 12:53:22 +01:00
2023-08-22 17:31:18 -07:00
2022-11-18 12:14:55 +00:00
2022-07-22 12:53:22 +01:00
2023-08-07 08:53:55 +01:00
2023-11-28 12:18:29 +01:00
2022-05-11 12:43:10 +01:00
2024-02-07 18:55:10 -08:00
2021-12-10 06:38:26 -08:00
2024-05-05 14:35:33 +01:00
2023-12-04 18:04:05 -08:00
2022-07-22 12:53:22 +01:00
2023-11-20 10:15:16 -08:00
2024-02-07 12:38:30 +00:00
2024-04-06 18:20:14 +01:00
2024-02-20 09:22:58 +01:00
2022-07-22 12:53:22 +01:00
2024-04-25 15:24:23 -07:00
2023-11-20 11:43:45 +01:00
2023-07-14 20:39:29 -07:00
2023-08-04 15:33:50 -07:00
2024-04-01 10:49:28 +01:00
2023-07-14 20:39:30 -07:00
2023-06-02 09:55:22 +01:00
2024-03-29 09:46:39 +00:00
2024-02-02 10:57:55 -08:00
2022-07-22 12:53:22 +01:00
2024-04-30 18:46:52 -07:00
2024-03-07 21:12:43 -08:00
2021-08-09 15:34:21 -07:00
2022-07-22 12:53:22 +01:00
2023-04-06 12:01:20 -07:00
2023-03-17 08:56:37 +00:00
2024-04-19 11:34:07 +01:00
2023-09-18 09:44:05 +02:00
2024-04-26 15:34:00 +02:00
2022-08-01 11:59:23 -07:00
2024-04-30 18:32:38 -07:00
2023-06-19 11:32:58 -07:00
2024-04-01 11:28:32 +01:00
2023-07-14 20:39:30 -07:00
2024-04-26 15:34:00 +02:00
2024-02-26 11:46:12 +00:00
2024-04-26 10:46:41 +02:00
2024-05-10 18:52:45 -07:00
2022-07-22 12:53:22 +01:00
2022-01-04 12:17:35 +00:00
2021-04-28 14:06:45 -07:00
2024-04-30 13:24:48 +02:00
2022-10-25 11:35:16 +02:00
2024-04-30 18:46:52 -07:00
2023-10-04 11:49:20 -07:00
2022-07-22 12:53:22 +01:00
2022-07-18 11:24:10 +01:00
2024-02-16 09:36:37 +00:00
2024-02-14 10:49:37 +01:00
2024-01-04 18:06:46 -08:00
2023-12-04 14:45:26 -08:00
2024-05-06 11:54:04 +02:00
2023-10-23 15:01:53 -07:00
2024-05-09 20:25:55 -07:00
2023-07-28 14:07:59 -07:00
2024-04-25 12:41:37 -07:00
2023-08-01 15:06:27 -07:00
2022-12-12 15:04:39 -08:00
2022-07-22 12:53:22 +01:00
2024-04-01 10:49:28 +01:00
2023-08-07 08:53:55 +01:00
2023-09-14 16:16:36 +02:00
2023-11-16 22:33:31 +00:00
2023-03-18 12:23:34 +00:00
2022-07-22 12:53:22 +01:00
2024-01-24 16:24:06 -08:00
2024-03-19 13:47:29 +01:00
2023-12-14 16:38:59 +01:00
2024-05-09 10:01:01 -07:00
2023-12-13 16:16:40 -08:00
4b0ebbca3e