diff --git a/README.md b/README.md
index e5803ea3a2..a02d24c776 100644
--- a/README.md
+++ b/README.md
@@ -10,9 +10,9 @@ An extremely fast Python linter, written in Rust.
-
-
-
+
+
+
@@ -1642,57 +1642,28 @@ which makes it a good target for benchmarking.
git clone --branch 3.10 https://github.com/python/cpython.git resources/test/cpython
```
-Add this `pyproject.toml` to the CPython directory:
-
-```toml
-[tool.ruff]
-line-length = 88
-extend-exclude = [
- "Lib/lib2to3/tests/data/bom.py",
- "Lib/lib2to3/tests/data/crlf.py",
- "Lib/lib2to3/tests/data/different_encoding.py",
- "Lib/lib2to3/tests/data/false_encoding.py",
- "Lib/lib2to3/tests/data/py2_test_grammar.py",
- "Lib/test/bad_coding2.py",
- "Lib/test/badsyntax_3131.py",
- "Lib/test/badsyntax_pep3120.py",
- "Lib/test/encoded_modules/module_iso_8859_1.py",
- "Lib/test/encoded_modules/module_koi8_r.py",
- "Lib/test/test_fstring.py",
- "Lib/test/test_grammar.py",
- "Lib/test/test_importlib/test_util.py",
- "Lib/test/test_named_expressions.py",
- "Lib/test/test_patma.py",
- "Lib/test/test_source_encoding.py",
- "Tools/c-analyzer/c_parser/parser/_delim.py",
- "Tools/i18n/pygettext.py",
- "Tools/test2to3/maintest.py",
- "Tools/test2to3/setup.py",
- "Tools/test2to3/test/test_foo.py",
- "Tools/test2to3/test2to3/hello.py",
-]
-```
-
-Next, to benchmark the release build:
+To benchmark the release build:
```shell
-cargo build --release
-
-hyperfine --ignore-failure --warmup 10 --runs 100 \
+cargo build --release && hyperfine --ignore-failure --warmup 10 \
"./target/release/ruff ./resources/test/cpython/ --no-cache" \
"./target/release/ruff ./resources/test/cpython/"
Benchmark 1: ./target/release/ruff ./resources/test/cpython/ --no-cache
- Time (mean ± σ): 297.4 ms ± 4.9 ms [User: 2460.0 ms, System: 67.2 ms]
- Range (min … max): 287.7 ms … 312.1 ms 100 runs
+ Time (mean ± σ): 293.8 ms ± 3.2 ms [User: 2384.6 ms, System: 90.3 ms]
+ Range (min … max): 289.9 ms … 301.6 ms 10 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: ./target/release/ruff ./resources/test/cpython/
- Time (mean ± σ): 79.6 ms ± 7.3 ms [User: 59.7 ms, System: 356.1 ms]
- Range (min … max): 62.4 ms … 111.2 ms 100 runs
+ Time (mean ± σ): 48.0 ms ± 3.1 ms [User: 65.2 ms, System: 124.7 ms]
+ Range (min … max): 45.0 ms … 66.7 ms 62 runs
Warning: Ignoring non-zero exit code.
+
+Summary
+ './target/release/ruff ./resources/test/cpython/' ran
+ 6.12 ± 0.41 times faster than './target/release/ruff ./resources/test/cpython/ --no-cache'
```
To benchmark against the ecosystem's existing tools:
@@ -1700,75 +1671,80 @@ To benchmark against the ecosystem's existing tools:
```shell
hyperfine --ignore-failure --warmup 5 \
"./target/release/ruff ./resources/test/cpython/ --no-cache" \
- "pylint --recursive=y resources/test/cpython/" \
"pyflakes resources/test/cpython" \
"autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython" \
"pycodestyle resources/test/cpython" \
- "flake8 resources/test/cpython" \
- "python -m scripts.run_flake8 resources/test/cpython"
-```
+ "flake8 resources/test/cpython"
-In order, these evaluate:
-
-- Ruff
-- Pylint
-- Pyflakes
-- autoflake
-- pycodestyle
-- Flake8
-- Flake8, with a hack to enable multiprocessing on macOS
-
-(You can `poetry install` from `./scripts` to create a working environment for the above.)
-
-```shell
Benchmark 1: ./target/release/ruff ./resources/test/cpython/ --no-cache
- Time (mean ± σ): 297.9 ms ± 7.0 ms [User: 2436.6 ms, System: 65.9 ms]
- Range (min … max): 289.9 ms … 314.6 ms 10 runs
+ Time (mean ± σ): 294.3 ms ± 3.3 ms [User: 2467.5 ms, System: 89.6 ms]
+ Range (min … max): 291.1 ms … 302.8 ms 10 runs
Warning: Ignoring non-zero exit code.
-Benchmark 2: pylint --recursive=y resources/test/cpython/
- Time (mean ± σ): 37.634 s ± 0.225 s [User: 36.728 s, System: 0.853 s]
- Range (min … max): 37.201 s … 38.106 s 10 runs
+Benchmark 2: pyflakes resources/test/cpython
+ Time (mean ± σ): 15.786 s ± 0.143 s [User: 15.560 s, System: 0.214 s]
+ Range (min … max): 15.640 s … 16.157 s 10 runs
Warning: Ignoring non-zero exit code.
-Benchmark 3: pyflakes resources/test/cpython
- Time (mean ± σ): 40.950 s ± 0.449 s [User: 40.688 s, System: 0.229 s]
- Range (min … max): 40.348 s … 41.671 s 10 runs
+Benchmark 3: autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython
+ Time (mean ± σ): 6.175 s ± 0.169 s [User: 54.102 s, System: 1.057 s]
+ Range (min … max): 5.950 s … 6.391 s 10 runs
+
+Benchmark 4: pycodestyle resources/test/cpython
+ Time (mean ± σ): 46.921 s ± 0.508 s [User: 46.699 s, System: 0.202 s]
+ Range (min … max): 46.171 s … 47.863 s 10 runs
Warning: Ignoring non-zero exit code.
-Benchmark 4: autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython
- Time (mean ± σ): 11.562 s ± 0.160 s [User: 107.022 s, System: 1.143 s]
- Range (min … max): 11.417 s … 11.917 s 10 runs
-
-Benchmark 5: pycodestyle resources/test/cpython
- Time (mean ± σ): 67.428 s ± 0.985 s [User: 67.199 s, System: 0.203 s]
- Range (min … max): 65.313 s … 68.496 s 10 runs
+Benchmark 5: flake8 resources/test/cpython
+ Time (mean ± σ): 12.260 s ± 0.321 s [User: 102.934 s, System: 1.230 s]
+ Range (min … max): 11.848 s … 12.933 s 10 runs
Warning: Ignoring non-zero exit code.
-Benchmark 6: flake8 resources/test/cpython
- Time (mean ± σ): 116.099 s ± 1.178 s [User: 115.217 s, System: 0.845 s]
- Range (min … max): 114.180 s … 117.724 s 10 runs
-
- Warning: Ignoring non-zero exit code.
-
-Benchmark 7: python -m scripts.run_flake8 resources/test/cpython
- Time (mean ± σ): 20.477 s ± 0.349 s [User: 142.372 s, System: 1.504 s]
- Range (min … max): 20.107 s … 21.183 s 10 runs
-
Summary
'./target/release/ruff ./resources/test/cpython/ --no-cache' ran
- 38.81 ± 1.05 times faster than 'autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython'
- 68.74 ± 1.99 times faster than 'python -m scripts.run_flake8 resources/test/cpython'
- 126.33 ± 3.05 times faster than 'pylint --recursive=y resources/test/cpython/'
- 137.46 ± 3.55 times faster than 'pyflakes resources/test/cpython'
- 226.35 ± 6.23 times faster than 'pycodestyle resources/test/cpython'
- 389.73 ± 9.92 times faster than 'flake8 resources/test/cpython'
+ 20.98 ± 0.62 times faster than 'autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython'
+ 41.66 ± 1.18 times faster than 'flake8 resources/test/cpython'
+ 53.64 ± 0.77 times faster than 'pyflakes resources/test/cpython'
+ 159.43 ± 2.48 times faster than 'pycodestyle resources/test/cpython'
```
+You can run `poetry install` from `./scripts` to create a working environment for the above. All
+reported benchmarks were computed using the versions specified by `./scripts/pyproject.toml`
+on Python 3.11.
+
+To benchmark Pylint, remove the following files from the CPython repository:
+
+```shell
+rm Lib/test/bad_coding.py \
+ Lib/test/bad_coding2.py \
+ Lib/test/bad_getattr.py \
+ Lib/test/bad_getattr2.py \
+ Lib/test/bad_getattr3.py \
+ Lib/test/badcert.pem \
+ Lib/test/badkey.pem \
+ Lib/test/badsyntax_3131.py \
+ Lib/test/badsyntax_future10.py \
+ Lib/test/badsyntax_future3.py \
+ Lib/test/badsyntax_future4.py \
+ Lib/test/badsyntax_future5.py \
+ Lib/test/badsyntax_future6.py \
+ Lib/test/badsyntax_future7.py \
+ Lib/test/badsyntax_future8.py \
+ Lib/test/badsyntax_future9.py \
+ Lib/test/badsyntax_pep3120.py \
+ Lib/test/test_asyncio/test_runners.py \
+ Lib/test/test_copy.py \
+ Lib/test/test_inspect.py \
+ Lib/test/test_typing.py
+```
+
+Then, from `resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This
+will execute Pylint with maximum parallelism and only report errors.
+
## Reference
### Options
diff --git a/scripts/benchmarks/README.md b/scripts/benchmarks/README.md
new file mode 100644
index 0000000000..4b71cd2ae3
--- /dev/null
+++ b/scripts/benchmarks/README.md
@@ -0,0 +1,25 @@
+# benchmarks
+
+Utilities for benchmarking Ruff.
+
+## Getting Started
+
+Run `./scripts/benchmarks/run.sh` to clone the benchmarking target (CPython).
+
+If you're looking to benchmark Ruff against other tools, you'll also need to run `poetry
+install` to create a virtual environment with the required dependencies.
+
+## Running Benchmarks
+
+Run `./scripts/benchmarks/run.sh` to run Ruff over the target repo (CPython). The
+`./scripts/benchmarks` folder contains a few other benchmarks (e.g., `scripts/benchmarks/run_comparisons.sh`
+compares Ruff to a variety of other tools).
+
+## Generating Plots
+
+The Vega specification for the benchmark plot depicted in the root README can be found at
+`scripts/benchmarks/graph-spec.json`. You can render this JSON spec in the [Vega Editor](https://vega.github.io/editor/#/edited).
+
+The images seen in the README are generated by exporting the rendered Vega spec as SVG (at around
+688px wide) and manually bolding the Ruff title and benchmark time. The dark mode variant is
+generated by changing the fill from `fill="#333333"` to `fill="#C9D1D9"`.
diff --git a/scripts/benchmarks/dark.svg b/scripts/benchmarks/dark.svg
new file mode 100644
index 0000000000..82e965a9e6
--- /dev/null
+++ b/scripts/benchmarks/dark.svg
@@ -0,0 +1 @@
+0s 20s 40s 60s Ruff Autoflake Flake8 Pyflakes Pycodestyle Pylint 0.29s 6.18s 12.26s 15.79s 46.92s > 60s
diff --git a/scripts/benchmarks/graph-spec.json b/scripts/benchmarks/graph-spec.json
new file mode 100644
index 0000000000..e6025e4a17
--- /dev/null
+++ b/scripts/benchmarks/graph-spec.json
@@ -0,0 +1,209 @@
+{
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {
+ "values": [
+ {
+ "tool": "Ruff",
+ "time": 0.2943,
+ "timeFormat": "0.29s"
+ },
+ {
+ "tool": "Autoflake",
+ "time": 6.175,
+ "timeFormat": "6.18s"
+ },
+ {
+ "tool": "Flake8",
+ "time": 12.26,
+ "timeFormat": "12.26s"
+ },
+ {
+ "tool": "Pyflakes",
+ "time": 15.786,
+ "timeFormat": "15.79s"
+ },
+ {
+ "tool": "Pycodestyle",
+ "time": 46.921,
+ "timeFormat": "46.92s"
+ },
+ {
+ "tool": "Pylint",
+ "time": 62.0,
+ "timeFormat": "> 60s"
+ }
+ ]
+ },
+ "config": {
+ "params": [
+ {
+ "name": "defaultFont",
+ "value": "-apple-system,BlinkMacSystemFont,\"Segoe UI\",Helvetica,Arial,sans-serif,\"Apple Color Emoji\",\"Segoe UI Emoji\""
+ },
+ {
+ "name": "titleColor",
+ "value": "#333333"
+ },
+ {
+ "name": "labelColor",
+ "value": "#333333"
+ }
+ ],
+ "header": {
+ "labelFont": {
+ "expr": "defaultFont"
+ },
+ "titleFont": {
+ "expr": "defaultFont"
+ },
+ "titleFontWeight": 500
+ },
+ "text": {
+ "font": {
+ "expr": "defaultFont"
+ },
+ "color": {
+ "expr": "labelColor"
+ }
+ },
+ "mark": {
+ "font": {
+ "expr": "defaultFont"
+ },
+ "color": {
+ "expr": "labelColor"
+ }
+ },
+ "title": {
+ "font": {
+ "expr": "defaultFont"
+ },
+ "subtitleFont": {
+ "expr": "defaultFont"
+ },
+ "fontWeight": 500
+ },
+ "axis": {
+ "labelColor": {
+ "expr": "labelColor"
+ },
+ "labelFont": {
+ "expr": "defaultFont"
+ },
+ "titleFont": {
+ "expr": "defaultFont"
+ },
+ "titleFontWeight": 500,
+ "titleColor": {
+ "expr": "titleColor"
+ },
+ "titleFontSize": 12
+ },
+ "legend": {
+ "titleFontWeight": 500,
+ "titleColor": {
+ "expr": "titleColor"
+ },
+ "titleFontSize": 12,
+ "labelColor": {
+ "expr": "labelColor"
+ },
+ "labelFont": {
+ "expr": "defaultFont"
+ },
+ "titleFont": {
+ "expr": "defaultFont"
+ }
+ },
+ "view": {
+ "stroke": null
+ },
+ "background": "transparent"
+ },
+ "background": "transparent",
+ "encoding": {
+ "y": {
+ "field": "tool",
+ "type": "nominal",
+ "axis": {
+ "grid": false,
+ "title": null,
+ "labelFontSize": 12,
+ "ticks": false,
+ "labelPadding": 10,
+ "domain": false
+ },
+ "sort": null
+ },
+ "x": {
+ "field": "time",
+ "type": "quantitative",
+ "axis": {
+ "title": null,
+ "labelExpr": "datum.value + 's'",
+ "tickCount": 3,
+ "tickSize": 0,
+ "labelPadding": 6,
+ "labelAlign": "center",
+ "labelFontSize": 12,
+ "tickColor": "rgba(127,127,127,0.25)",
+ "gridColor": "rgba(127,127,127,0.25)",
+ "domain": false
+ }
+ }
+ },
+ "height": 140,
+ "width": "container",
+ "layer": [
+ {
+ "mark": "bar",
+ "encoding": {
+ "size": {
+ "value": 13
+ },
+ "color": {
+ "value": "#E15759"
+ }
+ }
+ },
+ {
+ "transform": [
+ {
+ "filter": "datum.tool !== 'ruff'"
+ }
+ ],
+ "mark": {
+ "type": "text",
+ "align": "left",
+ "baseline": "middle",
+ "dx": 6,
+ "fontSize": 12
+ },
+ "encoding": {
+ "text": {
+ "field": "timeFormat"
+ }
+ }
+ },
+ {
+ "transform": [
+ {
+ "filter": "datum.tool === 'ruff'"
+ }
+ ],
+ "mark": {
+ "type": "text",
+ "align": "left",
+ "baseline": "middle",
+ "dx": 6,
+ "fontSize": 12,
+ "fontWeight": "bold"
+ },
+ "encoding": {
+ "text": {
+ "field": "timeFormat"
+ }
+ }
+ }
+ ]
+}
diff --git a/scripts/benchmarks/light.svg b/scripts/benchmarks/light.svg
new file mode 100644
index 0000000000..4c81d8bc85
--- /dev/null
+++ b/scripts/benchmarks/light.svg
@@ -0,0 +1 @@
+0s 20s 40s 60s Ruff Autoflake Flake8 Pyflakes Pycodestyle Pylint 0.29s 6.18s 12.26s 15.79s 46.92s > 60s
diff --git a/scripts/poetry.lock b/scripts/benchmarks/poetry.lock
similarity index 55%
rename from scripts/poetry.lock
rename to scripts/benchmarks/poetry.lock
index 286168b328..5e6fcccebe 100644
--- a/scripts/poetry.lock
+++ b/scripts/benchmarks/poetry.lock
@@ -2,36 +2,90 @@
[[package]]
name = "astroid"
-version = "2.12.13"
+version = "2.13.2"
description = "An abstract syntax tree for Python with inference support."
category = "main"
optional = false
python-versions = ">=3.7.2"
files = [
- {file = "astroid-2.12.13-py3-none-any.whl", hash = "sha256:10e0ad5f7b79c435179d0d0f0df69998c4eef4597534aae44910db060baeb907"},
- {file = "astroid-2.12.13.tar.gz", hash = "sha256:1493fe8bd3dfd73dc35bd53c9d5b6e49ead98497c47b2307662556a5692d29d7"},
+ {file = "astroid-2.13.2-py3-none-any.whl", hash = "sha256:8f6a8d40c4ad161d6fc419545ae4b2f275ed86d1c989c97825772120842ee0d2"},
+ {file = "astroid-2.13.2.tar.gz", hash = "sha256:3bc7834720e1a24ca797fd785d77efb14f7a28ee8e635ef040b6e2d80ccb3303"},
]
[package.dependencies]
lazy-object-proxy = ">=1.4.0"
-wrapt = {version = ">=1.11,<2", markers = "python_version < \"3.11\""}
+typing-extensions = ">=4.0.0"
+wrapt = [
+ {version = ">=1.11,<2", markers = "python_version < \"3.11\""},
+ {version = ">=1.14,<2", markers = "python_version >= \"3.11\""},
+]
[[package]]
name = "autoflake"
-version = "1.7.8"
+version = "2.0.0"
description = "Removes unused imports and unused variables"
category = "main"
optional = false
python-versions = ">=3.7"
files = [
- {file = "autoflake-1.7.8-py3-none-any.whl", hash = "sha256:46373ef69b6714f5064c923bb28bd797c4f8a9497f557d87fc36665c6d956b39"},
- {file = "autoflake-1.7.8.tar.gz", hash = "sha256:e7e46372dee46fa1c97acf310d99d922b63d369718a270809d7c278d34a194cf"},
+ {file = "autoflake-2.0.0-py3-none-any.whl", hash = "sha256:d58ed4187c6b4f623a942b9a90c43ff84bf6a266f3682f407b42ca52073c9678"},
+ {file = "autoflake-2.0.0.tar.gz", hash = "sha256:7185b596e70d8970c6d4106c112ef41921e472bd26abf3613db99eca88cc8c2a"},
]
[package.dependencies]
-pyflakes = ">=1.1.0,<3"
+pyflakes = ">=3.0.0"
tomli = {version = ">=2.0.1", markers = "python_version < \"3.11\""}
+[[package]]
+name = "black"
+version = "22.12.0"
+description = "The uncompromising code formatter."
+category = "main"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "black-22.12.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9eedd20838bd5d75b80c9f5487dbcb06836a43833a37846cf1d8c1cc01cef59d"},
+ {file = "black-22.12.0-cp310-cp310-win_amd64.whl", hash = "sha256:159a46a4947f73387b4d83e87ea006dbb2337eab6c879620a3ba52699b1f4351"},
+ {file = "black-22.12.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d30b212bffeb1e252b31dd269dfae69dd17e06d92b87ad26e23890f3efea366f"},
+ {file = "black-22.12.0-cp311-cp311-win_amd64.whl", hash = "sha256:7412e75863aa5c5411886804678b7d083c7c28421210180d67dfd8cf1221e1f4"},
+ {file = "black-22.12.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c116eed0efb9ff870ded8b62fe9f28dd61ef6e9ddd28d83d7d264a38417dcee2"},
+ {file = "black-22.12.0-cp37-cp37m-win_amd64.whl", hash = "sha256:1f58cbe16dfe8c12b7434e50ff889fa479072096d79f0a7f25e4ab8e94cd8350"},
+ {file = "black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:77d86c9f3db9b1bf6761244bc0b3572a546f5fe37917a044e02f3166d5aafa7d"},
+ {file = "black-22.12.0-cp38-cp38-win_amd64.whl", hash = "sha256:82d9fe8fee3401e02e79767016b4907820a7dc28d70d137eb397b92ef3cc5bfc"},
+ {file = "black-22.12.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:101c69b23df9b44247bd88e1d7e90154336ac4992502d4197bdac35dd7ee3320"},
+ {file = "black-22.12.0-cp39-cp39-win_amd64.whl", hash = "sha256:559c7a1ba9a006226f09e4916060982fd27334ae1998e7a38b3f33a37f7a2148"},
+ {file = "black-22.12.0-py3-none-any.whl", hash = "sha256:436cc9167dd28040ad90d3b404aec22cedf24a6e4d7de221bec2730ec0c97bcf"},
+ {file = "black-22.12.0.tar.gz", hash = "sha256:229351e5a18ca30f447bf724d007f890f97e13af070bb6ad4c0a441cd7596a2f"},
+]
+
+[package.dependencies]
+click = ">=8.0.0"
+mypy-extensions = ">=0.4.3"
+pathspec = ">=0.9.0"
+platformdirs = ">=2"
+tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""}
+
+[package.extras]
+colorama = ["colorama (>=0.4.3)"]
+d = ["aiohttp (>=3.7.4)"]
+jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
+uvloop = ["uvloop (>=0.15.2)"]
+
+[[package]]
+name = "click"
+version = "8.1.3"
+description = "Composable command line interface toolkit"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "click-8.1.3-py3-none-any.whl", hash = "sha256:bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48"},
+ {file = "click-8.1.3.tar.gz", hash = "sha256:7682dc8afb30297001674575ea00d1814d808d6a36af415a82bd481d37ba7b8e"},
+]
+
+[package.dependencies]
+colorama = {version = "*", markers = "platform_system == \"Windows\""}
+
[[package]]
name = "colorama"
version = "0.4.6"
@@ -61,20 +115,20 @@ graph = ["objgraph (>=1.7.2)"]
[[package]]
name = "flake8"
-version = "5.0.4"
+version = "6.0.0"
description = "the modular source code checker: pep8 pyflakes and co"
category = "main"
optional = false
-python-versions = ">=3.6.1"
+python-versions = ">=3.8.1"
files = [
- {file = "flake8-5.0.4-py2.py3-none-any.whl", hash = "sha256:7a1cf6b73744f5806ab95e526f6f0d8c01c66d7bbe349562d22dfca20610b248"},
- {file = "flake8-5.0.4.tar.gz", hash = "sha256:6fbe320aad8d6b95cec8b8e47bc933004678dc63095be98528b7bdd2a9f510db"},
+ {file = "flake8-6.0.0-py2.py3-none-any.whl", hash = "sha256:3833794e27ff64ea4e9cf5d410082a8b97ff1a06c16aa3d2027339cd0f1195c7"},
+ {file = "flake8-6.0.0.tar.gz", hash = "sha256:c61007e76655af75e6785a931f452915b371dc48f56efd765247c8fe68f2b181"},
]
[package.dependencies]
mccabe = ">=0.7.0,<0.8.0"
-pycodestyle = ">=2.9.0,<2.10.0"
-pyflakes = ">=2.5.0,<2.6.0"
+pycodestyle = ">=2.10.0,<2.11.0"
+pyflakes = ">=3.0.0,<3.1.0"
[[package]]
name = "isort"
@@ -96,31 +150,48 @@ requirements-deprecated-finder = ["pip-api", "pipreqs"]
[[package]]
name = "lazy-object-proxy"
-version = "1.8.0"
+version = "1.9.0"
description = "A fast and thorough lazy object proxy."
category = "main"
optional = false
python-versions = ">=3.7"
files = [
- {file = "lazy-object-proxy-1.8.0.tar.gz", hash = "sha256:c219a00245af0f6fa4e95901ed28044544f50152840c5b6a3e7b2568db34d156"},
- {file = "lazy_object_proxy-1.8.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:4fd031589121ad46e293629b39604031d354043bb5cdf83da4e93c2d7f3389fe"},
- {file = "lazy_object_proxy-1.8.0-cp310-cp310-win32.whl", hash = "sha256:b70d6e7a332eb0217e7872a73926ad4fdc14f846e85ad6749ad111084e76df25"},
- {file = "lazy_object_proxy-1.8.0-cp310-cp310-win_amd64.whl", hash = "sha256:eb329f8d8145379bf5dbe722182410fe8863d186e51bf034d2075eb8d85ee25b"},
- {file = "lazy_object_proxy-1.8.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:4e2d9f764f1befd8bdc97673261b8bb888764dfdbd7a4d8f55e4fbcabb8c3fb7"},
- {file = "lazy_object_proxy-1.8.0-cp311-cp311-win32.whl", hash = "sha256:e20bfa6db17a39c706d24f82df8352488d2943a3b7ce7d4c22579cb89ca8896e"},
- {file = "lazy_object_proxy-1.8.0-cp311-cp311-win_amd64.whl", hash = "sha256:14010b49a2f56ec4943b6cf925f597b534ee2fe1f0738c84b3bce0c1a11ff10d"},
- {file = "lazy_object_proxy-1.8.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:6850e4aeca6d0df35bb06e05c8b934ff7c533734eb51d0ceb2d63696f1e6030c"},
- {file = "lazy_object_proxy-1.8.0-cp37-cp37m-win32.whl", hash = "sha256:5b51d6f3bfeb289dfd4e95de2ecd464cd51982fe6f00e2be1d0bf94864d58acd"},
- {file = "lazy_object_proxy-1.8.0-cp37-cp37m-win_amd64.whl", hash = "sha256:6f593f26c470a379cf7f5bc6db6b5f1722353e7bf937b8d0d0b3fba911998858"},
- {file = "lazy_object_proxy-1.8.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:0c1c7c0433154bb7c54185714c6929acc0ba04ee1b167314a779b9025517eada"},
- {file = "lazy_object_proxy-1.8.0-cp38-cp38-win32.whl", hash = "sha256:d176f392dbbdaacccf15919c77f526edf11a34aece58b55ab58539807b85436f"},
- {file = "lazy_object_proxy-1.8.0-cp38-cp38-win_amd64.whl", hash = "sha256:afcaa24e48bb23b3be31e329deb3f1858f1f1df86aea3d70cb5c8578bfe5261c"},
- {file = "lazy_object_proxy-1.8.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:71d9ae8a82203511a6f60ca5a1b9f8ad201cac0fc75038b2dc5fa519589c9288"},
- {file = "lazy_object_proxy-1.8.0-cp39-cp39-win32.whl", hash = "sha256:8f6ce2118a90efa7f62dd38c7dbfffd42f468b180287b748626293bf12ed468f"},
- {file = "lazy_object_proxy-1.8.0-cp39-cp39-win_amd64.whl", hash = "sha256:eac3a9a5ef13b332c059772fd40b4b1c3d45a3a2b05e33a361dee48e54a4dad0"},
- {file = "lazy_object_proxy-1.8.0-pp37-pypy37_pp73-any.whl", hash = "sha256:ae032743794fba4d171b5b67310d69176287b5bf82a21f588282406a79498891"},
- {file = "lazy_object_proxy-1.8.0-pp38-pypy38_pp73-any.whl", hash = "sha256:7e1561626c49cb394268edd00501b289053a652ed762c58e1081224c8d881cec"},
- {file = "lazy_object_proxy-1.8.0-pp39-pypy39_pp73-any.whl", hash = "sha256:ce58b2b3734c73e68f0e30e4e725264d4d6be95818ec0a0be4bb6bf9a7e79aa8"},
+ {file = "lazy-object-proxy-1.9.0.tar.gz", hash = "sha256:659fb5809fa4629b8a1ac5106f669cfc7bef26fbb389dda53b3e010d1ac4ebae"},
+ {file = "lazy_object_proxy-1.9.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:b40387277b0ed2d0602b8293b94d7257e17d1479e257b4de114ea11a8cb7f2d7"},
+ {file = "lazy_object_proxy-1.9.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e8c6cfb338b133fbdbc5cfaa10fe3c6aeea827db80c978dbd13bc9dd8526b7d4"},
+ {file = "lazy_object_proxy-1.9.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:721532711daa7db0d8b779b0bb0318fa87af1c10d7fe5e52ef30f8eff254d0cd"},
+ {file = "lazy_object_proxy-1.9.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:66a3de4a3ec06cd8af3f61b8e1ec67614fbb7c995d02fa224813cb7afefee701"},
+ {file = "lazy_object_proxy-1.9.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:1aa3de4088c89a1b69f8ec0dcc169aa725b0ff017899ac568fe44ddc1396df46"},
+ {file = "lazy_object_proxy-1.9.0-cp310-cp310-win32.whl", hash = "sha256:f0705c376533ed2a9e5e97aacdbfe04cecd71e0aa84c7c0595d02ef93b6e4455"},
+ {file = "lazy_object_proxy-1.9.0-cp310-cp310-win_amd64.whl", hash = "sha256:ea806fd4c37bf7e7ad82537b0757999264d5f70c45468447bb2b91afdbe73a6e"},
+ {file = "lazy_object_proxy-1.9.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:946d27deaff6cf8452ed0dba83ba38839a87f4f7a9732e8f9fd4107b21e6ff07"},
+ {file = "lazy_object_proxy-1.9.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:79a31b086e7e68b24b99b23d57723ef7e2c6d81ed21007b6281ebcd1688acb0a"},
+ {file = "lazy_object_proxy-1.9.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f699ac1c768270c9e384e4cbd268d6e67aebcfae6cd623b4d7c3bfde5a35db59"},
+ {file = "lazy_object_proxy-1.9.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:bfb38f9ffb53b942f2b5954e0f610f1e721ccebe9cce9025a38c8ccf4a5183a4"},
+ {file = "lazy_object_proxy-1.9.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:189bbd5d41ae7a498397287c408617fe5c48633e7755287b21d741f7db2706a9"},
+ {file = "lazy_object_proxy-1.9.0-cp311-cp311-win32.whl", hash = "sha256:81fc4d08b062b535d95c9ea70dbe8a335c45c04029878e62d744bdced5141586"},
+ {file = "lazy_object_proxy-1.9.0-cp311-cp311-win_amd64.whl", hash = "sha256:f2457189d8257dd41ae9b434ba33298aec198e30adf2dcdaaa3a28b9994f6adb"},
+ {file = "lazy_object_proxy-1.9.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:d9e25ef10a39e8afe59a5c348a4dbf29b4868ab76269f81ce1674494e2565a6e"},
+ {file = "lazy_object_proxy-1.9.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cbf9b082426036e19c6924a9ce90c740a9861e2bdc27a4834fd0a910742ac1e8"},
+ {file = "lazy_object_proxy-1.9.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9f5fa4a61ce2438267163891961cfd5e32ec97a2c444e5b842d574251ade27d2"},
+ {file = "lazy_object_proxy-1.9.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:8fa02eaab317b1e9e03f69aab1f91e120e7899b392c4fc19807a8278a07a97e8"},
+ {file = "lazy_object_proxy-1.9.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:e7c21c95cae3c05c14aafffe2865bbd5e377cfc1348c4f7751d9dc9a48ca4bda"},
+ {file = "lazy_object_proxy-1.9.0-cp37-cp37m-win32.whl", hash = "sha256:f12ad7126ae0c98d601a7ee504c1122bcef553d1d5e0c3bfa77b16b3968d2734"},
+ {file = "lazy_object_proxy-1.9.0-cp37-cp37m-win_amd64.whl", hash = "sha256:edd20c5a55acb67c7ed471fa2b5fb66cb17f61430b7a6b9c3b4a1e40293b1671"},
+ {file = "lazy_object_proxy-1.9.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:2d0daa332786cf3bb49e10dc6a17a52f6a8f9601b4cf5c295a4f85854d61de63"},
+ {file = "lazy_object_proxy-1.9.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9cd077f3d04a58e83d04b20e334f678c2b0ff9879b9375ed107d5d07ff160171"},
+ {file = "lazy_object_proxy-1.9.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:660c94ea760b3ce47d1855a30984c78327500493d396eac4dfd8bd82041b22be"},
+ {file = "lazy_object_proxy-1.9.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:212774e4dfa851e74d393a2370871e174d7ff0ebc980907723bb67d25c8a7c30"},
+ {file = "lazy_object_proxy-1.9.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:f0117049dd1d5635bbff65444496c90e0baa48ea405125c088e93d9cf4525b11"},
+ {file = "lazy_object_proxy-1.9.0-cp38-cp38-win32.whl", hash = "sha256:0a891e4e41b54fd5b8313b96399f8b0e173bbbfc03c7631f01efbe29bb0bcf82"},
+ {file = "lazy_object_proxy-1.9.0-cp38-cp38-win_amd64.whl", hash = "sha256:9990d8e71b9f6488e91ad25f322898c136b008d87bf852ff65391b004da5e17b"},
+ {file = "lazy_object_proxy-1.9.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9e7551208b2aded9c1447453ee366f1c4070602b3d932ace044715d89666899b"},
+ {file = "lazy_object_proxy-1.9.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5f83ac4d83ef0ab017683d715ed356e30dd48a93746309c8f3517e1287523ef4"},
+ {file = "lazy_object_proxy-1.9.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7322c3d6f1766d4ef1e51a465f47955f1e8123caee67dd641e67d539a534d006"},
+ {file = "lazy_object_proxy-1.9.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:18b78ec83edbbeb69efdc0e9c1cb41a3b1b1ed11ddd8ded602464c3fc6020494"},
+ {file = "lazy_object_proxy-1.9.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:09763491ce220c0299688940f8dc2c5d05fd1f45af1e42e636b2e8b2303e4382"},
+ {file = "lazy_object_proxy-1.9.0-cp39-cp39-win32.whl", hash = "sha256:9090d8e53235aa280fc9239a86ae3ea8ac58eff66a705fa6aa2ec4968b95c821"},
+ {file = "lazy_object_proxy-1.9.0-cp39-cp39-win_amd64.whl", hash = "sha256:db1c1722726f47e10e0b5fdbf15ac3b8adb58c091d12b3ab713965795036985f"},
]
[[package]]
@@ -135,6 +206,30 @@ files = [
{file = "mccabe-0.7.0.tar.gz", hash = "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325"},
]
+[[package]]
+name = "mypy-extensions"
+version = "0.4.3"
+description = "Experimental type system extensions for programs checked with the mypy typechecker."
+category = "main"
+optional = false
+python-versions = "*"
+files = [
+ {file = "mypy_extensions-0.4.3-py2.py3-none-any.whl", hash = "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d"},
+ {file = "mypy_extensions-0.4.3.tar.gz", hash = "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"},
+]
+
+[[package]]
+name = "pathspec"
+version = "0.10.3"
+description = "Utility library for gitignore style pattern matching of file paths."
+category = "main"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "pathspec-0.10.3-py3-none-any.whl", hash = "sha256:3c95343af8b756205e2aba76e843ba9520a24dd84f68c22b9f93251507509dd6"},
+ {file = "pathspec-0.10.3.tar.gz", hash = "sha256:56200de4077d9d0791465aa9095a01d421861e405b5096955051deefd697d6f6"},
+]
+
[[package]]
name = "platformdirs"
version = "2.6.2"
@@ -153,44 +248,47 @@ test = ["appdirs (==1.4.4)", "covdefaults (>=2.2.2)", "pytest (>=7.2)", "pytest-
[[package]]
name = "pycodestyle"
-version = "2.9.1"
+version = "2.10.0"
description = "Python style guide checker"
category = "main"
optional = false
python-versions = ">=3.6"
files = [
- {file = "pycodestyle-2.9.1-py2.py3-none-any.whl", hash = "sha256:d1735fc58b418fd7c5f658d28d943854f8a849b01a5d0a1e6f3f3fdd0166804b"},
- {file = "pycodestyle-2.9.1.tar.gz", hash = "sha256:2c9607871d58c76354b697b42f5d57e1ada7d261c261efac224b664affdc5785"},
+ {file = "pycodestyle-2.10.0-py2.py3-none-any.whl", hash = "sha256:8a4eaf0d0495c7395bdab3589ac2db602797d76207242c17d470186815706610"},
+ {file = "pycodestyle-2.10.0.tar.gz", hash = "sha256:347187bdb476329d98f695c213d7295a846d1152ff4fe9bacb8a9590b8ee7053"},
]
[[package]]
name = "pyflakes"
-version = "2.5.0"
+version = "3.0.1"
description = "passive checker of Python programs"
category = "main"
optional = false
python-versions = ">=3.6"
files = [
- {file = "pyflakes-2.5.0-py2.py3-none-any.whl", hash = "sha256:4579f67d887f804e67edb544428f264b7b24f435b263c4614f384135cea553d2"},
- {file = "pyflakes-2.5.0.tar.gz", hash = "sha256:491feb020dca48ccc562a8c0cbe8df07ee13078df59813b83959cbdada312ea3"},
+ {file = "pyflakes-3.0.1-py2.py3-none-any.whl", hash = "sha256:ec55bf7fe21fff7f1ad2f7da62363d749e2a470500eab1b555334b67aa1ef8cf"},
+ {file = "pyflakes-3.0.1.tar.gz", hash = "sha256:ec8b276a6b60bd80defed25add7e439881c19e64850afd9b346283d4165fd0fd"},
]
[[package]]
name = "pylint"
-version = "2.15.9"
+version = "2.15.10"
description = "python code static checker"
category = "main"
optional = false
python-versions = ">=3.7.2"
files = [
- {file = "pylint-2.15.9-py3-none-any.whl", hash = "sha256:349c8cd36aede4d50a0754a8c0218b43323d13d5d88f4b2952ddfe3e169681eb"},
- {file = "pylint-2.15.9.tar.gz", hash = "sha256:18783cca3cfee5b83c6c5d10b3cdb66c6594520ffae61890858fe8d932e1c6b4"},
+ {file = "pylint-2.15.10-py3-none-any.whl", hash = "sha256:9df0d07e8948a1c3ffa3b6e2d7e6e63d9fb457c5da5b961ed63106594780cc7e"},
+ {file = "pylint-2.15.10.tar.gz", hash = "sha256:b3dc5ef7d33858f297ac0d06cc73862f01e4f2e74025ec3eff347ce0bc60baf5"},
]
[package.dependencies]
astroid = ">=2.12.13,<=2.14.0-dev0"
colorama = {version = ">=0.4.5", markers = "sys_platform == \"win32\""}
-dill = {version = ">=0.2", markers = "python_version < \"3.11\""}
+dill = [
+ {version = ">=0.2", markers = "python_version < \"3.11\""},
+ {version = ">=0.3.6", markers = "python_version >= \"3.11\""},
+]
isort = ">=4.2.5,<6"
mccabe = ">=0.6,<0.8"
platformdirs = ">=2.2.0"
@@ -225,6 +323,18 @@ files = [
{file = "tomlkit-0.11.6.tar.gz", hash = "sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73"},
]
+[[package]]
+name = "typing-extensions"
+version = "4.4.0"
+description = "Backported and Experimental Type Hints for Python 3.7+"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "typing_extensions-4.4.0-py3-none-any.whl", hash = "sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e"},
+ {file = "typing_extensions-4.4.0.tar.gz", hash = "sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa"},
+]
+
[[package]]
name = "wrapt"
version = "1.14.1"
@@ -301,5 +411,5 @@ files = [
[metadata]
lock-version = "2.0"
-python-versions = ">=3.10,<3.11"
-content-hash = "959633dfe6335ab3f943a95c5fdd12ff1bb66cd03c5917704f10ae38d3a5009c"
+python-versions = ">=3.10,<3.12"
+content-hash = "382b5873c3c0e9f46b85c501cacfce4978c68bd40bedf7a41740f3ed464a1e22"
diff --git a/scripts/pyproject.toml b/scripts/benchmarks/pyproject.toml
similarity index 50%
rename from scripts/pyproject.toml
rename to scripts/benchmarks/pyproject.toml
index 6fffb97a96..232da580c7 100644
--- a/scripts/pyproject.toml
+++ b/scripts/benchmarks/pyproject.toml
@@ -1,11 +1,3 @@
-[tool.black]
-line-length = 120
-
-[tool.ruff]
-line-length = 120
-select = ["E", "F", "W", "I", "C", "RET", "ANN", "UP"]
-target-version = "py310"
-
[tool.poetry]
name = "scripts"
version = "0.1.0"
@@ -13,12 +5,14 @@ description = ""
authors = ["Charles Marsh "]
[tool.poetry.dependencies]
-python = ">=3.10,<3.11"
-autoflake = "^1.4"
-flake8 = "^5.0.4"
-pycodestyle = "^2.9.1"
-pyflakes = "^2.5.0"
-pylint = "^2.15.0"
+python = ">=3.10,<3.12"
+autoflake = "^2.0.0"
+flake8 = "^6.0.0"
+pycodestyle = "^2.10.0"
+pyflakes = "^3.0.1"
+pylint = "^2.15.10"
+black = "^22.12.0"
+isort = "^5.11.4"
[tool.poetry.dev-dependencies]
diff --git a/scripts/benchmarks/run.sh b/scripts/benchmarks/run.sh
new file mode 100755
index 0000000000..6fe9ebb307
--- /dev/null
+++ b/scripts/benchmarks/run.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/env sh
+
+###
+# Benchmark Ruff on the CPython codebase.
+###
+
+cargo build --release && hyperfine --ignore-failure --warmup 10 \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache"
diff --git a/scripts/benchmarks/run_comparisons.sh b/scripts/benchmarks/run_comparisons.sh
new file mode 100755
index 0000000000..0101c3e646
--- /dev/null
+++ b/scripts/benchmarks/run_comparisons.sh
@@ -0,0 +1,43 @@
+#!/usr/bin/env sh
+
+###
+# Benchmark the incremental performance of each subsequent plugin.
+###
+
+cargo build --release && hyperfine --ignore-failure --warmup 10 \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select C90" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select I" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select D" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select UP" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select N" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select YTT" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ANN" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select S" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select BLE" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select FBT" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select B" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select A" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select C4" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select T10" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select EM" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ISC" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ICN" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select T20" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PT" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select Q" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select RET" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select SIM" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select TID" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ARG" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select DTZ" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ERA" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PD" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PGH" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLC" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLE" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLR" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLW" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PIE" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select COM" \
+ "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select RUF"
diff --git a/scripts/benchmarks/run_plugins.sh b/scripts/benchmarks/run_plugins.sh
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/scripts/benchmarks/setup.sh b/scripts/benchmarks/setup.sh
new file mode 100755
index 0000000000..3ede4df228
--- /dev/null
+++ b/scripts/benchmarks/setup.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env sh
+
+###
+# Setup the CPython repository to enable benchmarking.
+###
+
+git clone --branch 3.10 https://github.com/python/cpython.git resources/test/cpython
diff --git a/scripts/run_flake8.py b/scripts/run_flake8.py
deleted file mode 100755
index d3bbb501a8..0000000000
--- a/scripts/run_flake8.py
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/usr/bin/env python3
-"""Wrapper around Flake8 to enable multiprocessing on all operating systems.
-
-As of Python 3.8, macOS's default "start method" for multiprocessing is `spawn`. Flake8
-requires a "start method" of `fork`, and disables multiprocessing if it detects `spawn`
-or some other "start method". This script enables the `fork` start method before passing
-along any command-line arguments to `flake8`.
-
-This has never caused me any problems, but note that they disabled this for a reason:
-Flake8's plugin interface doesn't work with `spawn`, and the maintainer says that `fork`
-is "pretty broken" on macOS.
-
-See:
-
-- https://github.com/pycqa/flake8/issues/955
-- https://github.com/PyCQA/flake8/issues/1337
-- https://github.com/PyCQA/flake8/issues/342
-- https://github.com/PyCQA/flake8/pull/1621
-
-Example usage: python -m run_flake8 --select=E501 .
-"""
-import multiprocessing
-import sys
-
-from flake8.main import cli
-
-if __name__ == "__main__":
- multiprocessing.set_start_method("fork", force=True)
- cli.main(sys.argv[1:])