Steven Rostedt (Google)
86087383ec
tracing/hist: Call hist functions directly via a switch statement
Due to retpolines, indirect calls are much more expensive than direct
calls. The histograms have a select set of functions it uses for the
histograms, instead of using function pointers to call them, create a
hist_fn_call() function that uses a switch statement to call the histogram
functions directly. This gives a 13% speedup to the histogram logic.
Using the histogram benchmark:
Before:
# event histogram
#
# trigger info: hist:keys=delta:vals=hitcount:sort=delta:size=2048 if delta > 0 [active]
#
{ delta: 129 } hitcount: 2213
{ delta: 130 } hitcount: 285965
{ delta: 131 } hitcount: 1146545
{ delta: 132 } hitcount: 5185432
{ delta: 133 } hitcount: 19896215
{ delta: 134 } hitcount: 53118616
{ delta: 135 } hitcount: 83816709
{ delta: 136 } hitcount: 68329562
{ delta: 137 } hitcount: 41859349
{ delta: 138 } hitcount: 46257797
{ delta: 139 } hitcount: 54400831
{ delta: 140 } hitcount: 72875007
{ delta: 141 } hitcount: 76193272
{ delta: 142 } hitcount: 49504263
{ delta: 143 } hitcount: 38821072
{ delta: 144 } hitcount: 47702679
{ delta: 145 } hitcount: 41357297
{ delta: 146 } hitcount: 22058238
{ delta: 147 } hitcount: 9720002
{ delta: 148 } hitcount: 3193542
{ delta: 149 } hitcount: 927030
{ delta: 150 } hitcount: 850772
{ delta: 151 } hitcount: 1477380
{ delta: 152 } hitcount: 2687977
{ delta: 153 } hitcount: 2865985
{ delta: 154 } hitcount: 1977492
{ delta: 155 } hitcount: 2475607
{ delta: 156 } hitcount: 3403612
After:
# event histogram
#
# trigger info: hist:keys=delta:vals=hitcount:sort=delta:size=2048 if delta > 0 [active]
#
{ delta: 113 } hitcount: 272
{ delta: 114 } hitcount: 840
{ delta: 118 } hitcount: 344
{ delta: 119 } hitcount: 25428
{ delta: 120 } hitcount: 350590
{ delta: 121 } hitcount: 1892484
{ delta: 122 } hitcount: 6205004
{ delta: 123 } hitcount: 11583521
{ delta: 124 } hitcount: 37590979
{ delta: 125 } hitcount: 108308504
{ delta: 126 } hitcount: 131672461
{ delta: 127 } hitcount: 88700598
{ delta: 128 } hitcount: 65939870
{ delta: 129 } hitcount: 45055004
{ delta: 130 } hitcount: 33174464
{ delta: 131 } hitcount: 31813493
{ delta: 132 } hitcount: 29011676
{ delta: 133 } hitcount: 22798782
{ delta: 134 } hitcount: 22072486
{ delta: 135 } hitcount: 17034113
{ delta: 136 } hitcount: 8982490
{ delta: 137 } hitcount: 2865908
{ delta: 138 } hitcount: 980382
{ delta: 139 } hitcount: 1651944
{ delta: 140 } hitcount: 4112073
{ delta: 141 } hitcount: 3963269
{ delta: 142 } hitcount: 1712508
{ delta: 143 } hitcount: 575941
{ delta: 144 } hitcount: 351427
{ delta: 145 } hitcount: 218077
{ delta: 146 } hitcount: 167297
{ delta: 147 } hitcount: 146198
{ delta: 148 } hitcount: 116122
{ delta: 149 } hitcount: 58993
{ delta: 150 } hitcount: 40228
The delta above is in nanoseconds. It brings the fastest time down from
129ns to 113ns, and the peak from 141ns to 126ns.
Link: https://lkml.kernel.org/r/20220906225529.411545333@goodmis.org
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Tom Zanussi <zanussi@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-09-26 13:01:10 -04:00
..
2022-08-26 12:19:09 +01:00
2022-09-23 07:18:45 -10:00
2022-08-14 09:28:54 -07:00
2022-05-26 17:13:43 -07:00
2022-09-07 10:38:28 +02:00
2022-07-05 13:32:59 -07:00
2022-08-06 17:28:12 -07:00
2022-05-25 16:18:27 -07:00
2021-12-02 17:25:21 +09:00
2022-07-28 12:36:35 +02:00
2022-07-22 09:22:59 -06:00
2022-06-02 08:55:01 -07:00
2022-08-02 19:12:45 -07:00
2022-08-15 13:51:07 -06:00
2022-08-04 11:05:48 -07:00
2022-07-15 10:52:11 +02:00
2022-08-05 16:32:45 -07:00
2022-09-09 15:08:40 -04:00
2022-08-09 20:02:13 +02:00
2022-09-26 13:01:10 -04:00
2022-04-06 13:43:44 -07:00
2022-02-03 11:20:34 -08:00
2022-08-22 18:50:06 -04:00
2022-04-25 14:37:28 +02:00
2022-04-25 14:37:12 +02:00
2022-06-13 14:08:57 -04:00
2022-02-22 13:51:40 -05:00
2022-01-11 13:08:21 -08:00
2022-08-25 17:25:08 -04:00
2022-03-09 10:32:06 -08:00
2022-07-05 13:32:59 -07:00
2022-07-05 13:33:00 -07:00
2022-07-05 13:32:59 -07:00
2022-05-23 17:51:12 -07:00
2022-08-28 14:02:44 -07:00
2022-03-15 10:32:44 +01:00
2022-06-01 15:55:25 -07:00
2022-08-03 10:44:54 +02:00
2022-07-05 13:32:59 -07:00
2022-09-20 08:38:55 -07:00
2022-05-08 03:16:59 +09:00
2022-07-15 18:21:49 +00:00
2022-07-29 18:12:35 -07:00
2022-04-15 14:49:55 -07:00
2022-06-24 09:48:55 +02:00
2022-07-17 17:31:39 -07:00
2022-08-07 10:03:24 -07:00
2022-03-31 10:36:55 +02:00
2022-05-25 13:05:42 -07:00
2022-07-15 12:21:16 -04:00
2022-08-07 10:03:24 -07:00
2022-09-08 17:08:43 -04:00
2022-03-23 19:00:33 -07:00
2022-06-16 19:11:30 -07:00
2022-04-21 11:40:59 -07:00
2022-08-01 07:42:56 +02:00
2022-05-19 19:30:30 +02:00
2022-09-13 10:38:43 -07:00
2022-01-31 11:21:46 +11:00
2022-08-02 19:34:45 -07:00
2021-12-28 11:26:18 +01:00
2022-04-29 14:38:00 -07:00
2021-10-14 13:29:18 +02:00
2022-07-29 18:12:36 -07:00
2022-07-09 11:06:19 -07:00
2022-06-23 19:11:28 +02:00
2022-05-12 20:38:37 -07:00
2022-07-21 17:19:25 -07:00
2022-08-01 15:21:42 +02:00
2022-04-11 17:07:29 -07:00
2022-03-24 19:06:48 -07:00
2022-05-03 14:11:58 -07:00
2022-07-07 09:53:43 -07:00
2022-07-19 11:40:00 -07:00
2022-04-12 14:13:01 +02:00
2022-07-05 13:32:59 -07:00
2022-05-08 01:33:09 -07:00
2022-02-25 09:36:06 +01:00
2022-04-05 09:59:38 +02:00
2022-04-05 09:59:38 +02:00
2022-05-24 11:11:13 -07:00
2022-08-20 15:17:45 -07:00
2022-04-22 18:50:54 +01:00
2022-08-08 09:01:36 -07:00
2022-04-30 08:39:32 -06:00
2022-04-29 14:38:03 -07:00
2022-02-01 17:24:39 -08:00
2022-09-06 22:26:00 -04:00
2022-04-29 14:38:03 -07:00
2022-02-17 09:11:57 -06:00
2022-05-06 14:49:44 -05:00
2022-02-25 10:40:14 -06:00
2022-05-19 23:25:47 -04:00
2022-08-02 19:24:24 -07:00
2022-06-23 18:41:40 +02:00
2022-08-06 16:38:17 -07:00
2022-08-16 06:27:35 -10:00