Jakub Kicinski 301f227fc8
net: piggy back on the memory barrier in bql when waking queues ...
Drivers call netdev_tx_completed_queue() right before
netif_txq_maybe_wake(). If BQL is enabled netdev_tx_completed_queue()
should issue a memory barrier, so we can depend on that separating
the stop check from the consumer index update, instead of adding
another barrier in netif_txq_maybe_wake().
This matters more than the barriers on the xmit path, because
the wake condition is almost always true. So we issue the
consumer side barrier often.
Wrap netdev_tx_completed_queue() in a local helper to issue
the barrier even if BQL is disabled. Keep the same semantics
as netdev_tx_completed_queue() (barrier only if bytes != 0)
to make it clear that the barrier is conditional.
Plus since macro gets pkt/byte counts as arguments now -
we can skip waking if there were no packets completed.
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2023-04-10 17:56:18 -07:00
2022-12-06 07:30:55 +09:00
2023-03-22 16:05:55 -07:00
2021-07-01 13:19:48 -07:00
2021-11-19 11:52:25 +00:00
2023-03-17 07:58:18 +00:00
2023-03-22 21:48:59 +01:00
2023-02-22 21:25:23 -08:00
2021-11-17 20:17:05 -08:00
2022-07-22 12:53:22 +01:00
2023-03-09 11:31:44 +01:00
2023-02-16 10:39:28 +01:00
2023-02-20 16:46:10 -08:00
2023-03-17 08:56:37 +00:00
2023-01-06 09:43:32 +00:00
2023-03-18 12:23:33 +00:00
2023-03-29 08:19:38 +01:00
2022-07-22 12:53:22 +01:00
2023-03-21 21:32:18 -07:00
2023-03-18 12:23:34 +00:00
2022-08-09 22:14:02 -07:00
2021-12-13 12:34:09 +00:00
2022-09-16 14:34:01 +01:00
2022-11-02 20:38:13 -07:00
2022-07-22 12:53:22 +01:00
2022-10-27 12:45:38 +02:00
2021-03-26 17:43:55 +01:00
2022-08-24 13:46:58 +01:00
2022-11-16 11:31:47 +02:00
2023-03-24 11:45:51 +01:00
2023-01-28 13:51:22 +01:00
2023-01-27 11:19:46 +00:00
2021-12-22 15:03:47 -08:00
2022-07-22 12:53:22 +01:00
2021-12-22 15:03:51 -08:00
2022-07-24 18:39:17 -06:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2023-01-20 09:33:22 +00:00
2023-02-13 09:49:14 +00:00
2023-02-20 08:54:23 +00:00
2023-04-09 15:35:49 +01:00
2023-04-03 10:04:27 +01:00
2021-11-29 19:50:45 -08:00
2022-12-05 21:58:27 -08:00
2023-01-13 20:59:14 -08:00
2023-03-28 18:52:28 -07:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2021-12-06 16:06:02 -08:00
2021-12-16 07:18:35 -08:00
2022-07-28 22:21:54 -07:00
2022-09-20 09:13:38 +02:00
2023-02-20 16:46:10 -08:00
2023-01-27 11:19:46 +00:00
2022-12-01 15:09:10 +01:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2021-10-18 12:54:41 +01:00
2022-11-21 13:09:07 +00:00
2022-10-31 10:43:04 +00:00
2021-01-29 20:39:14 -08:00
2022-08-25 16:07:42 -07:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2021-06-28 14:29:45 -07:00
2023-03-07 20:11:36 +01:00
2023-01-28 13:55:10 +01:00
2022-04-06 22:09:39 -07:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2022-07-29 11:58:54 +01:00
2022-07-22 12:53:22 +01:00
2022-08-24 19:30:07 -07:00
2022-02-07 20:12:45 -08:00
2021-10-15 11:33:08 +01:00
2022-10-31 20:14:27 -07:00
2022-12-30 07:25:52 +00:00
2023-03-17 08:56:37 +00:00
2022-12-30 07:25:52 +00:00
2022-02-17 07:00:39 -06:00
2021-11-16 13:16:54 +00:00
2023-03-29 08:22:52 +01:00
2023-03-28 18:52:22 -07:00
2022-04-25 11:40:45 +01:00
2022-04-11 17:37:50 -07:00
2023-03-17 22:41:16 -07:00
2023-02-02 14:02:01 +01:00
2023-01-25 22:45:00 -08:00
2022-09-29 07:18:00 +02:00
2022-07-22 12:53:22 +01:00
2022-10-31 20:14:27 -07:00
2022-09-02 20:34:32 -07:00
2023-02-13 19:55:32 -08:00
2021-03-23 14:14:50 -07:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2021-12-07 20:44:59 -08:00
2021-10-13 09:40:46 -07:00
2021-07-27 13:05:56 +01:00
2022-07-22 12:53:22 +01:00
2022-07-22 12:53:22 +01:00
2022-07-28 11:29:36 +02:00
2021-11-07 19:25:29 +00:00
2021-08-30 01:51:36 +02:00
2023-03-22 13:54:57 +01:00
2022-10-12 12:57:19 +02:00
2022-09-23 06:56:08 -07:00
2022-04-01 12:04:15 +01:00
2021-10-29 13:23:51 +01:00
2021-03-26 15:14:56 -07:00
2022-07-22 12:53:22 +01:00
2022-12-12 13:11:24 -08:00
2022-11-18 12:14:55 +00:00
2022-07-22 12:53:22 +01:00
2023-03-21 21:32:18 -07:00
2023-03-21 21:32:18 -07:00
2022-05-11 12:43:10 +01:00
2022-10-24 11:04:43 +01:00
2021-12-10 06:38:26 -08:00
2023-04-10 17:56:18 -07:00
2022-07-22 12:53:22 +01:00
2023-01-28 00:33:51 -08:00
2022-07-22 12:53:22 +01:00
2023-03-21 21:32:18 -07:00
2023-01-28 13:51:22 +01:00
2022-07-22 12:53:22 +01:00
2023-02-21 09:15:39 -08:00
2022-12-01 12:42:46 +01:00
2023-02-20 16:46:10 -08:00
2023-03-09 23:24:14 -08:00
2022-07-22 12:53:22 +01:00
2022-07-15 18:50:35 -07:00
2021-08-09 15:34:21 -07:00
2022-07-22 12:53:22 +01:00
2023-04-06 12:01:20 -07:00
2023-03-17 08:56:37 +00:00
2022-10-11 17:42:58 -06:00
2022-07-22 12:53:22 +01:00
2022-02-02 14:45:18 +00:00
2022-08-01 11:59:23 -07:00
2023-03-28 18:52:22 -07:00
2022-10-31 18:10:21 -07:00
2023-02-20 16:46:10 -08:00
2023-03-15 08:20:12 +00:00
2022-07-22 12:53:22 +01:00
2022-01-04 12:17:35 +00:00
2021-04-28 14:06:45 -07:00
2023-03-15 08:18:35 +00:00
2022-10-25 11:35:16 +02:00
2023-03-28 18:52:28 -07:00
2021-08-03 13:05:26 +01:00
2022-07-22 12:53:22 +01:00
2022-07-18 11:24:10 +01:00
2022-11-09 19:06:13 -08:00
2023-02-16 09:27:07 +01:00
2023-03-18 12:23:34 +00:00
2022-09-22 17:27:41 -07:00
2022-10-24 09:40:39 +01:00
2022-12-12 15:04:39 -08:00
2022-07-22 12:53:22 +01:00
2022-11-08 16:42:28 +00:00
2022-10-24 10:52:50 +01:00
2022-10-12 17:50:37 -07:00
2023-03-17 22:41:16 -07:00
2023-03-18 12:23:34 +00:00
2022-07-22 12:53:22 +01:00
2022-09-30 07:55:46 -07:00
2023-03-07 09:33:43 -08:00
2023-03-10 21:33:47 -08:00
2023-03-20 11:29:33 +02:00
2023-01-23 09:58:23 -08:00
301f227fc8