Charlie Marsh
b845e81c4a
Use memchr for computing line indexes (#21838)
## Summary
Some benchmarks with Claude's help:
| File | Size | Baseline | Optimized | Speedup |
|---------------------|-------|----------------------|----------------------|---------|
| numpy/globals.py | 3 KB | 1.48 µs (1.95 GiB/s) | 740 ns (3.89 GiB/s) |
2.0x |
| unicode/pypinyin.py | 4 KB | 2.04 µs (2.01 GiB/s) | 1.18 µs (3.49
GiB/s) | 1.7x |
| pydantic/types.py | 26 KB | 13.1 µs (1.90 GiB/s) | 5.88 µs (4.23
GiB/s) | 2.2x |
| numpy/ctypeslib.py | 17 KB | 8.45 µs (1.92 GiB/s) | 3.94 µs (4.13
GiB/s) | 2.1x |
| large/dataset.py | 41 KB | 21.6 µs (1.84 GiB/s) | 11.2 µs (3.55 GiB/s)
| 1.9x |
I think that I originally thought we _had_ to iterate
character-by-character here because we needed to do the ASCII check, but
the ASCII check can be vectorized by LLVM (and the "search for newlines"
can be done with `memchr`).
2025-12-08 08:50:51 -05:00
..
2025-12-04 09:45:53 -05:00
2025-11-19 10:02:58 +01:00
2025-12-02 20:10:46 +01:00
2025-05-16 13:25:28 +02:00
2025-12-04 08:12:04 +01:00
2025-11-01 01:50:58 +00:00
2025-11-29 15:41:54 +01:00
2025-11-03 10:38:20 -05:00
2025-12-03 15:04:36 -05:00
2025-06-28 20:24:00 +02:00
2025-12-07 11:40:46 +01:00
2025-10-23 14:48:41 -04:00
2025-12-03 12:16:18 +01:00
2025-11-13 18:58:08 +01:00
2025-08-07 18:21:50 +02:00
2025-12-03 08:15:17 +00:00
2025-12-03 08:15:17 +00:00
2025-12-02 20:10:46 +01:00
2025-12-02 20:10:46 +01:00
2025-12-02 20:10:46 +01:00
2025-12-02 20:10:46 +01:00
2025-05-16 13:25:28 +02:00
2025-12-03 12:05:15 -05:00
2025-11-26 09:31:22 +01:00
2025-10-03 09:45:32 -04:00
2025-09-25 14:33:37 -05:00
2025-09-25 14:33:37 -05:00
2025-11-21 17:42:35 +01:00
2025-12-08 08:50:51 -05:00
2025-11-21 15:07:37 -05:00
2025-12-04 09:45:53 -05:00
2025-11-16 12:30:24 +00:00
2025-12-03 15:04:36 -05:00
2025-08-22 11:13:47 -07:00
2025-12-04 17:37:37 -05:00
2025-12-05 19:12:13 +01:00
2025-12-03 12:16:18 +01:00
2025-12-08 14:36:13 +01:00
2025-12-04 17:37:37 -05:00
2025-10-03 13:56:06 +00:00
2025-12-08 11:44:20 +01:00
2025-12-05 08:57:21 -05:00
2025-11-29 15:41:54 +01:00