Add troubleshooting section to benchmarks guide (#1485)

## Summary

It took me a while to figure out why the benchmarks are extremelly flaky
(and slow) on my machine.
I documented my finding in the benchmarks.md file in case someone else
runs into the same problems as I
This commit is contained in:
Micha Reiser 2024-02-16 15:27:32 +01:00 committed by GitHub
parent 3280562e3a
commit 58bc069420
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 7 additions and 0 deletions

View File

@ -104,3 +104,10 @@ You need to install the [Roboto Font](https://fonts.google.com/specimen/Roboto)
The inclusion of this `BENCHMARKS.md` file was inspired by the excellent benchmarking documentation
in [Orogene](https://github.com/orogene/orogene/blob/472e481b4fc6e97c2b57e69240bf8fe995dfab83/BENCHMARKS.md).
## Troubleshooting
### Flaky benchmarks
If you're seeing high variance when running the cold benchmarks, then it's likely that you're running into throttling or DDoS prevention from your ISP. In that case, ISPs forcefully terminate TCP connections with a TCP reset. We believe this is due to the benchmarks making the exact same requests in a very short time (especially true for `uv`). A possible workaround is to connect to VPN to bypass your ISPs filtering mechanism.