From 42031b857405de4a232fa8488881d717cfecb7cb Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Mon, 16 Jan 2023 01:38:58 -0500 Subject: [PATCH] Re-run benchmark and update documentation (#1907) Closes #269. --- README.md | 156 ++++++++---------- scripts/benchmarks/README.md | 25 +++ scripts/benchmarks/dark.svg | 1 + scripts/benchmarks/graph-spec.json | 209 ++++++++++++++++++++++++ scripts/benchmarks/light.svg | 1 + scripts/{ => benchmarks}/poetry.lock | 202 +++++++++++++++++------ scripts/{ => benchmarks}/pyproject.toml | 22 +-- scripts/benchmarks/run.sh | 8 + scripts/benchmarks/run_comparisons.sh | 43 +++++ scripts/benchmarks/run_plugins.sh | 0 scripts/benchmarks/setup.sh | 7 + scripts/run_flake8.py | 29 ---- 12 files changed, 524 insertions(+), 179 deletions(-) create mode 100644 scripts/benchmarks/README.md create mode 100644 scripts/benchmarks/dark.svg create mode 100644 scripts/benchmarks/graph-spec.json create mode 100644 scripts/benchmarks/light.svg rename scripts/{ => benchmarks}/poetry.lock (55%) rename scripts/{ => benchmarks}/pyproject.toml (50%) create mode 100755 scripts/benchmarks/run.sh create mode 100755 scripts/benchmarks/run_comparisons.sh create mode 100644 scripts/benchmarks/run_plugins.sh create mode 100755 scripts/benchmarks/setup.sh delete mode 100755 scripts/run_flake8.py diff --git a/README.md b/README.md index e5803ea3a2..a02d24c776 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,9 @@ An extremely fast Python linter, written in Rust.

- - - Shows a bar chart with benchmark results. + + + Shows a bar chart with benchmark results.

@@ -1642,57 +1642,28 @@ which makes it a good target for benchmarking. git clone --branch 3.10 https://github.com/python/cpython.git resources/test/cpython ``` -Add this `pyproject.toml` to the CPython directory: - -```toml -[tool.ruff] -line-length = 88 -extend-exclude = [ - "Lib/lib2to3/tests/data/bom.py", - "Lib/lib2to3/tests/data/crlf.py", - "Lib/lib2to3/tests/data/different_encoding.py", - "Lib/lib2to3/tests/data/false_encoding.py", - "Lib/lib2to3/tests/data/py2_test_grammar.py", - "Lib/test/bad_coding2.py", - "Lib/test/badsyntax_3131.py", - "Lib/test/badsyntax_pep3120.py", - "Lib/test/encoded_modules/module_iso_8859_1.py", - "Lib/test/encoded_modules/module_koi8_r.py", - "Lib/test/test_fstring.py", - "Lib/test/test_grammar.py", - "Lib/test/test_importlib/test_util.py", - "Lib/test/test_named_expressions.py", - "Lib/test/test_patma.py", - "Lib/test/test_source_encoding.py", - "Tools/c-analyzer/c_parser/parser/_delim.py", - "Tools/i18n/pygettext.py", - "Tools/test2to3/maintest.py", - "Tools/test2to3/setup.py", - "Tools/test2to3/test/test_foo.py", - "Tools/test2to3/test2to3/hello.py", -] -``` - -Next, to benchmark the release build: +To benchmark the release build: ```shell -cargo build --release - -hyperfine --ignore-failure --warmup 10 --runs 100 \ +cargo build --release && hyperfine --ignore-failure --warmup 10 \ "./target/release/ruff ./resources/test/cpython/ --no-cache" \ "./target/release/ruff ./resources/test/cpython/" Benchmark 1: ./target/release/ruff ./resources/test/cpython/ --no-cache - Time (mean ± σ): 297.4 ms ± 4.9 ms [User: 2460.0 ms, System: 67.2 ms] - Range (min … max): 287.7 ms … 312.1 ms 100 runs + Time (mean ± σ): 293.8 ms ± 3.2 ms [User: 2384.6 ms, System: 90.3 ms] + Range (min … max): 289.9 ms … 301.6 ms 10 runs Warning: Ignoring non-zero exit code. Benchmark 2: ./target/release/ruff ./resources/test/cpython/ - Time (mean ± σ): 79.6 ms ± 7.3 ms [User: 59.7 ms, System: 356.1 ms] - Range (min … max): 62.4 ms … 111.2 ms 100 runs + Time (mean ± σ): 48.0 ms ± 3.1 ms [User: 65.2 ms, System: 124.7 ms] + Range (min … max): 45.0 ms … 66.7 ms 62 runs Warning: Ignoring non-zero exit code. + +Summary + './target/release/ruff ./resources/test/cpython/' ran + 6.12 ± 0.41 times faster than './target/release/ruff ./resources/test/cpython/ --no-cache' ``` To benchmark against the ecosystem's existing tools: @@ -1700,75 +1671,80 @@ To benchmark against the ecosystem's existing tools: ```shell hyperfine --ignore-failure --warmup 5 \ "./target/release/ruff ./resources/test/cpython/ --no-cache" \ - "pylint --recursive=y resources/test/cpython/" \ "pyflakes resources/test/cpython" \ "autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython" \ "pycodestyle resources/test/cpython" \ - "flake8 resources/test/cpython" \ - "python -m scripts.run_flake8 resources/test/cpython" -``` + "flake8 resources/test/cpython" -In order, these evaluate: - -- Ruff -- Pylint -- Pyflakes -- autoflake -- pycodestyle -- Flake8 -- Flake8, with a hack to enable multiprocessing on macOS - -(You can `poetry install` from `./scripts` to create a working environment for the above.) - -```shell Benchmark 1: ./target/release/ruff ./resources/test/cpython/ --no-cache - Time (mean ± σ): 297.9 ms ± 7.0 ms [User: 2436.6 ms, System: 65.9 ms] - Range (min … max): 289.9 ms … 314.6 ms 10 runs + Time (mean ± σ): 294.3 ms ± 3.3 ms [User: 2467.5 ms, System: 89.6 ms] + Range (min … max): 291.1 ms … 302.8 ms 10 runs Warning: Ignoring non-zero exit code. -Benchmark 2: pylint --recursive=y resources/test/cpython/ - Time (mean ± σ): 37.634 s ± 0.225 s [User: 36.728 s, System: 0.853 s] - Range (min … max): 37.201 s … 38.106 s 10 runs +Benchmark 2: pyflakes resources/test/cpython + Time (mean ± σ): 15.786 s ± 0.143 s [User: 15.560 s, System: 0.214 s] + Range (min … max): 15.640 s … 16.157 s 10 runs Warning: Ignoring non-zero exit code. -Benchmark 3: pyflakes resources/test/cpython - Time (mean ± σ): 40.950 s ± 0.449 s [User: 40.688 s, System: 0.229 s] - Range (min … max): 40.348 s … 41.671 s 10 runs +Benchmark 3: autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython + Time (mean ± σ): 6.175 s ± 0.169 s [User: 54.102 s, System: 1.057 s] + Range (min … max): 5.950 s … 6.391 s 10 runs + +Benchmark 4: pycodestyle resources/test/cpython + Time (mean ± σ): 46.921 s ± 0.508 s [User: 46.699 s, System: 0.202 s] + Range (min … max): 46.171 s … 47.863 s 10 runs Warning: Ignoring non-zero exit code. -Benchmark 4: autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython - Time (mean ± σ): 11.562 s ± 0.160 s [User: 107.022 s, System: 1.143 s] - Range (min … max): 11.417 s … 11.917 s 10 runs - -Benchmark 5: pycodestyle resources/test/cpython - Time (mean ± σ): 67.428 s ± 0.985 s [User: 67.199 s, System: 0.203 s] - Range (min … max): 65.313 s … 68.496 s 10 runs +Benchmark 5: flake8 resources/test/cpython + Time (mean ± σ): 12.260 s ± 0.321 s [User: 102.934 s, System: 1.230 s] + Range (min … max): 11.848 s … 12.933 s 10 runs Warning: Ignoring non-zero exit code. -Benchmark 6: flake8 resources/test/cpython - Time (mean ± σ): 116.099 s ± 1.178 s [User: 115.217 s, System: 0.845 s] - Range (min … max): 114.180 s … 117.724 s 10 runs - - Warning: Ignoring non-zero exit code. - -Benchmark 7: python -m scripts.run_flake8 resources/test/cpython - Time (mean ± σ): 20.477 s ± 0.349 s [User: 142.372 s, System: 1.504 s] - Range (min … max): 20.107 s … 21.183 s 10 runs - Summary './target/release/ruff ./resources/test/cpython/ --no-cache' ran - 38.81 ± 1.05 times faster than 'autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython' - 68.74 ± 1.99 times faster than 'python -m scripts.run_flake8 resources/test/cpython' - 126.33 ± 3.05 times faster than 'pylint --recursive=y resources/test/cpython/' - 137.46 ± 3.55 times faster than 'pyflakes resources/test/cpython' - 226.35 ± 6.23 times faster than 'pycodestyle resources/test/cpython' - 389.73 ± 9.92 times faster than 'flake8 resources/test/cpython' + 20.98 ± 0.62 times faster than 'autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython' + 41.66 ± 1.18 times faster than 'flake8 resources/test/cpython' + 53.64 ± 0.77 times faster than 'pyflakes resources/test/cpython' + 159.43 ± 2.48 times faster than 'pycodestyle resources/test/cpython' ``` +You can run `poetry install` from `./scripts` to create a working environment for the above. All +reported benchmarks were computed using the versions specified by `./scripts/pyproject.toml` +on Python 3.11. + +To benchmark Pylint, remove the following files from the CPython repository: + +```shell +rm Lib/test/bad_coding.py \ + Lib/test/bad_coding2.py \ + Lib/test/bad_getattr.py \ + Lib/test/bad_getattr2.py \ + Lib/test/bad_getattr3.py \ + Lib/test/badcert.pem \ + Lib/test/badkey.pem \ + Lib/test/badsyntax_3131.py \ + Lib/test/badsyntax_future10.py \ + Lib/test/badsyntax_future3.py \ + Lib/test/badsyntax_future4.py \ + Lib/test/badsyntax_future5.py \ + Lib/test/badsyntax_future6.py \ + Lib/test/badsyntax_future7.py \ + Lib/test/badsyntax_future8.py \ + Lib/test/badsyntax_future9.py \ + Lib/test/badsyntax_pep3120.py \ + Lib/test/test_asyncio/test_runners.py \ + Lib/test/test_copy.py \ + Lib/test/test_inspect.py \ + Lib/test/test_typing.py +``` + +Then, from `resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This +will execute Pylint with maximum parallelism and only report errors. + ## Reference ### Options diff --git a/scripts/benchmarks/README.md b/scripts/benchmarks/README.md new file mode 100644 index 0000000000..4b71cd2ae3 --- /dev/null +++ b/scripts/benchmarks/README.md @@ -0,0 +1,25 @@ +# benchmarks + +Utilities for benchmarking Ruff. + +## Getting Started + +Run `./scripts/benchmarks/run.sh` to clone the benchmarking target (CPython). + +If you're looking to benchmark Ruff against other tools, you'll also need to run `poetry +install` to create a virtual environment with the required dependencies. + +## Running Benchmarks + +Run `./scripts/benchmarks/run.sh` to run Ruff over the target repo (CPython). The +`./scripts/benchmarks` folder contains a few other benchmarks (e.g., `scripts/benchmarks/run_comparisons.sh` +compares Ruff to a variety of other tools). + +## Generating Plots + +The Vega specification for the benchmark plot depicted in the root README can be found at +`scripts/benchmarks/graph-spec.json`. You can render this JSON spec in the [Vega Editor](https://vega.github.io/editor/#/edited). + +The images seen in the README are generated by exporting the rendered Vega spec as SVG (at around +688px wide) and manually bolding the Ruff title and benchmark time. The dark mode variant is +generated by changing the fill from `fill="#333333"` to `fill="#C9D1D9"`. diff --git a/scripts/benchmarks/dark.svg b/scripts/benchmarks/dark.svg new file mode 100644 index 0000000000..82e965a9e6 --- /dev/null +++ b/scripts/benchmarks/dark.svg @@ -0,0 +1 @@ +0s20s40s60sRuffAutoflakeFlake8PyflakesPycodestylePylint0.29s6.18s12.26s15.79s46.92s> 60s diff --git a/scripts/benchmarks/graph-spec.json b/scripts/benchmarks/graph-spec.json new file mode 100644 index 0000000000..e6025e4a17 --- /dev/null +++ b/scripts/benchmarks/graph-spec.json @@ -0,0 +1,209 @@ +{ + "$schema": "https://vega.github.io/schema/vega-lite/v5.json", + "data": { + "values": [ + { + "tool": "Ruff", + "time": 0.2943, + "timeFormat": "0.29s" + }, + { + "tool": "Autoflake", + "time": 6.175, + "timeFormat": "6.18s" + }, + { + "tool": "Flake8", + "time": 12.26, + "timeFormat": "12.26s" + }, + { + "tool": "Pyflakes", + "time": 15.786, + "timeFormat": "15.79s" + }, + { + "tool": "Pycodestyle", + "time": 46.921, + "timeFormat": "46.92s" + }, + { + "tool": "Pylint", + "time": 62.0, + "timeFormat": "> 60s" + } + ] + }, + "config": { + "params": [ + { + "name": "defaultFont", + "value": "-apple-system,BlinkMacSystemFont,\"Segoe UI\",Helvetica,Arial,sans-serif,\"Apple Color Emoji\",\"Segoe UI Emoji\"" + }, + { + "name": "titleColor", + "value": "#333333" + }, + { + "name": "labelColor", + "value": "#333333" + } + ], + "header": { + "labelFont": { + "expr": "defaultFont" + }, + "titleFont": { + "expr": "defaultFont" + }, + "titleFontWeight": 500 + }, + "text": { + "font": { + "expr": "defaultFont" + }, + "color": { + "expr": "labelColor" + } + }, + "mark": { + "font": { + "expr": "defaultFont" + }, + "color": { + "expr": "labelColor" + } + }, + "title": { + "font": { + "expr": "defaultFont" + }, + "subtitleFont": { + "expr": "defaultFont" + }, + "fontWeight": 500 + }, + "axis": { + "labelColor": { + "expr": "labelColor" + }, + "labelFont": { + "expr": "defaultFont" + }, + "titleFont": { + "expr": "defaultFont" + }, + "titleFontWeight": 500, + "titleColor": { + "expr": "titleColor" + }, + "titleFontSize": 12 + }, + "legend": { + "titleFontWeight": 500, + "titleColor": { + "expr": "titleColor" + }, + "titleFontSize": 12, + "labelColor": { + "expr": "labelColor" + }, + "labelFont": { + "expr": "defaultFont" + }, + "titleFont": { + "expr": "defaultFont" + } + }, + "view": { + "stroke": null + }, + "background": "transparent" + }, + "background": "transparent", + "encoding": { + "y": { + "field": "tool", + "type": "nominal", + "axis": { + "grid": false, + "title": null, + "labelFontSize": 12, + "ticks": false, + "labelPadding": 10, + "domain": false + }, + "sort": null + }, + "x": { + "field": "time", + "type": "quantitative", + "axis": { + "title": null, + "labelExpr": "datum.value + 's'", + "tickCount": 3, + "tickSize": 0, + "labelPadding": 6, + "labelAlign": "center", + "labelFontSize": 12, + "tickColor": "rgba(127,127,127,0.25)", + "gridColor": "rgba(127,127,127,0.25)", + "domain": false + } + } + }, + "height": 140, + "width": "container", + "layer": [ + { + "mark": "bar", + "encoding": { + "size": { + "value": 13 + }, + "color": { + "value": "#E15759" + } + } + }, + { + "transform": [ + { + "filter": "datum.tool !== 'ruff'" + } + ], + "mark": { + "type": "text", + "align": "left", + "baseline": "middle", + "dx": 6, + "fontSize": 12 + }, + "encoding": { + "text": { + "field": "timeFormat" + } + } + }, + { + "transform": [ + { + "filter": "datum.tool === 'ruff'" + } + ], + "mark": { + "type": "text", + "align": "left", + "baseline": "middle", + "dx": 6, + "fontSize": 12, + "fontWeight": "bold" + }, + "encoding": { + "text": { + "field": "timeFormat" + } + } + } + ] +} diff --git a/scripts/benchmarks/light.svg b/scripts/benchmarks/light.svg new file mode 100644 index 0000000000..4c81d8bc85 --- /dev/null +++ b/scripts/benchmarks/light.svg @@ -0,0 +1 @@ +0s20s40s60sRuffAutoflakeFlake8PyflakesPycodestylePylint0.29s6.18s12.26s15.79s46.92s> 60s diff --git a/scripts/poetry.lock b/scripts/benchmarks/poetry.lock similarity index 55% rename from scripts/poetry.lock rename to scripts/benchmarks/poetry.lock index 286168b328..5e6fcccebe 100644 --- a/scripts/poetry.lock +++ b/scripts/benchmarks/poetry.lock @@ -2,36 +2,90 @@ [[package]] name = "astroid" -version = "2.12.13" +version = "2.13.2" description = "An abstract syntax tree for Python with inference support." category = "main" optional = false python-versions = ">=3.7.2" files = [ - {file = "astroid-2.12.13-py3-none-any.whl", hash = "sha256:10e0ad5f7b79c435179d0d0f0df69998c4eef4597534aae44910db060baeb907"}, - {file = "astroid-2.12.13.tar.gz", hash = "sha256:1493fe8bd3dfd73dc35bd53c9d5b6e49ead98497c47b2307662556a5692d29d7"}, + {file = "astroid-2.13.2-py3-none-any.whl", hash = "sha256:8f6a8d40c4ad161d6fc419545ae4b2f275ed86d1c989c97825772120842ee0d2"}, + {file = "astroid-2.13.2.tar.gz", hash = "sha256:3bc7834720e1a24ca797fd785d77efb14f7a28ee8e635ef040b6e2d80ccb3303"}, ] [package.dependencies] lazy-object-proxy = ">=1.4.0" -wrapt = {version = ">=1.11,<2", markers = "python_version < \"3.11\""} +typing-extensions = ">=4.0.0" +wrapt = [ + {version = ">=1.11,<2", markers = "python_version < \"3.11\""}, + {version = ">=1.14,<2", markers = "python_version >= \"3.11\""}, +] [[package]] name = "autoflake" -version = "1.7.8" +version = "2.0.0" description = "Removes unused imports and unused variables" category = "main" optional = false python-versions = ">=3.7" files = [ - {file = "autoflake-1.7.8-py3-none-any.whl", hash = "sha256:46373ef69b6714f5064c923bb28bd797c4f8a9497f557d87fc36665c6d956b39"}, - {file = "autoflake-1.7.8.tar.gz", hash = "sha256:e7e46372dee46fa1c97acf310d99d922b63d369718a270809d7c278d34a194cf"}, + {file = "autoflake-2.0.0-py3-none-any.whl", hash = "sha256:d58ed4187c6b4f623a942b9a90c43ff84bf6a266f3682f407b42ca52073c9678"}, + {file = "autoflake-2.0.0.tar.gz", hash = "sha256:7185b596e70d8970c6d4106c112ef41921e472bd26abf3613db99eca88cc8c2a"}, ] [package.dependencies] -pyflakes = ">=1.1.0,<3" +pyflakes = ">=3.0.0" tomli = {version = ">=2.0.1", markers = "python_version < \"3.11\""} +[[package]] +name = "black" +version = "22.12.0" +description = "The uncompromising code formatter." +category = "main" +optional = false +python-versions = ">=3.7" +files = [ + {file = "black-22.12.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9eedd20838bd5d75b80c9f5487dbcb06836a43833a37846cf1d8c1cc01cef59d"}, + {file = "black-22.12.0-cp310-cp310-win_amd64.whl", hash = "sha256:159a46a4947f73387b4d83e87ea006dbb2337eab6c879620a3ba52699b1f4351"}, + {file = "black-22.12.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d30b212bffeb1e252b31dd269dfae69dd17e06d92b87ad26e23890f3efea366f"}, + {file = "black-22.12.0-cp311-cp311-win_amd64.whl", hash = "sha256:7412e75863aa5c5411886804678b7d083c7c28421210180d67dfd8cf1221e1f4"}, + {file = "black-22.12.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c116eed0efb9ff870ded8b62fe9f28dd61ef6e9ddd28d83d7d264a38417dcee2"}, + {file = "black-22.12.0-cp37-cp37m-win_amd64.whl", hash = "sha256:1f58cbe16dfe8c12b7434e50ff889fa479072096d79f0a7f25e4ab8e94cd8350"}, + {file = "black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:77d86c9f3db9b1bf6761244bc0b3572a546f5fe37917a044e02f3166d5aafa7d"}, + {file = "black-22.12.0-cp38-cp38-win_amd64.whl", hash = "sha256:82d9fe8fee3401e02e79767016b4907820a7dc28d70d137eb397b92ef3cc5bfc"}, + {file = "black-22.12.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:101c69b23df9b44247bd88e1d7e90154336ac4992502d4197bdac35dd7ee3320"}, + {file = "black-22.12.0-cp39-cp39-win_amd64.whl", hash = "sha256:559c7a1ba9a006226f09e4916060982fd27334ae1998e7a38b3f33a37f7a2148"}, + {file = "black-22.12.0-py3-none-any.whl", hash = "sha256:436cc9167dd28040ad90d3b404aec22cedf24a6e4d7de221bec2730ec0c97bcf"}, + {file = "black-22.12.0.tar.gz", hash = "sha256:229351e5a18ca30f447bf724d007f890f97e13af070bb6ad4c0a441cd7596a2f"}, +] + +[package.dependencies] +click = ">=8.0.0" +mypy-extensions = ">=0.4.3" +pathspec = ">=0.9.0" +platformdirs = ">=2" +tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""} + +[package.extras] +colorama = ["colorama (>=0.4.3)"] +d = ["aiohttp (>=3.7.4)"] +jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"] +uvloop = ["uvloop (>=0.15.2)"] + +[[package]] +name = "click" +version = "8.1.3" +description = "Composable command line interface toolkit" +category = "main" +optional = false +python-versions = ">=3.7" +files = [ + {file = "click-8.1.3-py3-none-any.whl", hash = "sha256:bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48"}, + {file = "click-8.1.3.tar.gz", hash = "sha256:7682dc8afb30297001674575ea00d1814d808d6a36af415a82bd481d37ba7b8e"}, +] + +[package.dependencies] +colorama = {version = "*", markers = "platform_system == \"Windows\""} + [[package]] name = "colorama" version = "0.4.6" @@ -61,20 +115,20 @@ graph = ["objgraph (>=1.7.2)"] [[package]] name = "flake8" -version = "5.0.4" +version = "6.0.0" description = "the modular source code checker: pep8 pyflakes and co" category = "main" optional = false -python-versions = ">=3.6.1" +python-versions = ">=3.8.1" files = [ - {file = "flake8-5.0.4-py2.py3-none-any.whl", hash = "sha256:7a1cf6b73744f5806ab95e526f6f0d8c01c66d7bbe349562d22dfca20610b248"}, - {file = "flake8-5.0.4.tar.gz", hash = "sha256:6fbe320aad8d6b95cec8b8e47bc933004678dc63095be98528b7bdd2a9f510db"}, + {file = "flake8-6.0.0-py2.py3-none-any.whl", hash = "sha256:3833794e27ff64ea4e9cf5d410082a8b97ff1a06c16aa3d2027339cd0f1195c7"}, + {file = "flake8-6.0.0.tar.gz", hash = "sha256:c61007e76655af75e6785a931f452915b371dc48f56efd765247c8fe68f2b181"}, ] [package.dependencies] mccabe = ">=0.7.0,<0.8.0" -pycodestyle = ">=2.9.0,<2.10.0" -pyflakes = ">=2.5.0,<2.6.0" +pycodestyle = ">=2.10.0,<2.11.0" +pyflakes = ">=3.0.0,<3.1.0" [[package]] name = "isort" @@ -96,31 +150,48 @@ requirements-deprecated-finder = ["pip-api", "pipreqs"] [[package]] name = "lazy-object-proxy" -version = "1.8.0" +version = "1.9.0" description = "A fast and thorough lazy object proxy." category = "main" optional = false python-versions = ">=3.7" files = [ - {file = "lazy-object-proxy-1.8.0.tar.gz", hash = "sha256:c219a00245af0f6fa4e95901ed28044544f50152840c5b6a3e7b2568db34d156"}, - {file = "lazy_object_proxy-1.8.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:4fd031589121ad46e293629b39604031d354043bb5cdf83da4e93c2d7f3389fe"}, - {file = "lazy_object_proxy-1.8.0-cp310-cp310-win32.whl", hash = "sha256:b70d6e7a332eb0217e7872a73926ad4fdc14f846e85ad6749ad111084e76df25"}, - {file = "lazy_object_proxy-1.8.0-cp310-cp310-win_amd64.whl", hash = "sha256:eb329f8d8145379bf5dbe722182410fe8863d186e51bf034d2075eb8d85ee25b"}, - {file = "lazy_object_proxy-1.8.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:4e2d9f764f1befd8bdc97673261b8bb888764dfdbd7a4d8f55e4fbcabb8c3fb7"}, - {file = "lazy_object_proxy-1.8.0-cp311-cp311-win32.whl", hash = "sha256:e20bfa6db17a39c706d24f82df8352488d2943a3b7ce7d4c22579cb89ca8896e"}, - {file = "lazy_object_proxy-1.8.0-cp311-cp311-win_amd64.whl", hash = "sha256:14010b49a2f56ec4943b6cf925f597b534ee2fe1f0738c84b3bce0c1a11ff10d"}, - {file = "lazy_object_proxy-1.8.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:6850e4aeca6d0df35bb06e05c8b934ff7c533734eb51d0ceb2d63696f1e6030c"}, - {file = "lazy_object_proxy-1.8.0-cp37-cp37m-win32.whl", hash = "sha256:5b51d6f3bfeb289dfd4e95de2ecd464cd51982fe6f00e2be1d0bf94864d58acd"}, - {file = "lazy_object_proxy-1.8.0-cp37-cp37m-win_amd64.whl", hash = "sha256:6f593f26c470a379cf7f5bc6db6b5f1722353e7bf937b8d0d0b3fba911998858"}, - {file = "lazy_object_proxy-1.8.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:0c1c7c0433154bb7c54185714c6929acc0ba04ee1b167314a779b9025517eada"}, - {file = "lazy_object_proxy-1.8.0-cp38-cp38-win32.whl", hash = "sha256:d176f392dbbdaacccf15919c77f526edf11a34aece58b55ab58539807b85436f"}, - {file = "lazy_object_proxy-1.8.0-cp38-cp38-win_amd64.whl", hash = "sha256:afcaa24e48bb23b3be31e329deb3f1858f1f1df86aea3d70cb5c8578bfe5261c"}, - {file = "lazy_object_proxy-1.8.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:71d9ae8a82203511a6f60ca5a1b9f8ad201cac0fc75038b2dc5fa519589c9288"}, - {file = "lazy_object_proxy-1.8.0-cp39-cp39-win32.whl", hash = "sha256:8f6ce2118a90efa7f62dd38c7dbfffd42f468b180287b748626293bf12ed468f"}, - {file = "lazy_object_proxy-1.8.0-cp39-cp39-win_amd64.whl", hash = "sha256:eac3a9a5ef13b332c059772fd40b4b1c3d45a3a2b05e33a361dee48e54a4dad0"}, - {file = "lazy_object_proxy-1.8.0-pp37-pypy37_pp73-any.whl", hash = "sha256:ae032743794fba4d171b5b67310d69176287b5bf82a21f588282406a79498891"}, - {file = "lazy_object_proxy-1.8.0-pp38-pypy38_pp73-any.whl", hash = "sha256:7e1561626c49cb394268edd00501b289053a652ed762c58e1081224c8d881cec"}, - {file = "lazy_object_proxy-1.8.0-pp39-pypy39_pp73-any.whl", hash = "sha256:ce58b2b3734c73e68f0e30e4e725264d4d6be95818ec0a0be4bb6bf9a7e79aa8"}, + {file = "lazy-object-proxy-1.9.0.tar.gz", hash = "sha256:659fb5809fa4629b8a1ac5106f669cfc7bef26fbb389dda53b3e010d1ac4ebae"}, + {file = "lazy_object_proxy-1.9.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:b40387277b0ed2d0602b8293b94d7257e17d1479e257b4de114ea11a8cb7f2d7"}, + {file = "lazy_object_proxy-1.9.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e8c6cfb338b133fbdbc5cfaa10fe3c6aeea827db80c978dbd13bc9dd8526b7d4"}, + {file = "lazy_object_proxy-1.9.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:721532711daa7db0d8b779b0bb0318fa87af1c10d7fe5e52ef30f8eff254d0cd"}, + {file = "lazy_object_proxy-1.9.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:66a3de4a3ec06cd8af3f61b8e1ec67614fbb7c995d02fa224813cb7afefee701"}, + {file = "lazy_object_proxy-1.9.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:1aa3de4088c89a1b69f8ec0dcc169aa725b0ff017899ac568fe44ddc1396df46"}, + {file = "lazy_object_proxy-1.9.0-cp310-cp310-win32.whl", hash = "sha256:f0705c376533ed2a9e5e97aacdbfe04cecd71e0aa84c7c0595d02ef93b6e4455"}, + {file = "lazy_object_proxy-1.9.0-cp310-cp310-win_amd64.whl", hash = "sha256:ea806fd4c37bf7e7ad82537b0757999264d5f70c45468447bb2b91afdbe73a6e"}, + {file = "lazy_object_proxy-1.9.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:946d27deaff6cf8452ed0dba83ba38839a87f4f7a9732e8f9fd4107b21e6ff07"}, + {file = "lazy_object_proxy-1.9.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:79a31b086e7e68b24b99b23d57723ef7e2c6d81ed21007b6281ebcd1688acb0a"}, + {file = "lazy_object_proxy-1.9.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f699ac1c768270c9e384e4cbd268d6e67aebcfae6cd623b4d7c3bfde5a35db59"}, + {file = "lazy_object_proxy-1.9.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:bfb38f9ffb53b942f2b5954e0f610f1e721ccebe9cce9025a38c8ccf4a5183a4"}, + {file = "lazy_object_proxy-1.9.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:189bbd5d41ae7a498397287c408617fe5c48633e7755287b21d741f7db2706a9"}, + {file = "lazy_object_proxy-1.9.0-cp311-cp311-win32.whl", hash = "sha256:81fc4d08b062b535d95c9ea70dbe8a335c45c04029878e62d744bdced5141586"}, + {file = "lazy_object_proxy-1.9.0-cp311-cp311-win_amd64.whl", hash = "sha256:f2457189d8257dd41ae9b434ba33298aec198e30adf2dcdaaa3a28b9994f6adb"}, + {file = "lazy_object_proxy-1.9.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:d9e25ef10a39e8afe59a5c348a4dbf29b4868ab76269f81ce1674494e2565a6e"}, + {file = "lazy_object_proxy-1.9.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cbf9b082426036e19c6924a9ce90c740a9861e2bdc27a4834fd0a910742ac1e8"}, + {file = "lazy_object_proxy-1.9.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9f5fa4a61ce2438267163891961cfd5e32ec97a2c444e5b842d574251ade27d2"}, + {file = "lazy_object_proxy-1.9.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:8fa02eaab317b1e9e03f69aab1f91e120e7899b392c4fc19807a8278a07a97e8"}, + {file = "lazy_object_proxy-1.9.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:e7c21c95cae3c05c14aafffe2865bbd5e377cfc1348c4f7751d9dc9a48ca4bda"}, + {file = "lazy_object_proxy-1.9.0-cp37-cp37m-win32.whl", hash = "sha256:f12ad7126ae0c98d601a7ee504c1122bcef553d1d5e0c3bfa77b16b3968d2734"}, + {file = "lazy_object_proxy-1.9.0-cp37-cp37m-win_amd64.whl", hash = "sha256:edd20c5a55acb67c7ed471fa2b5fb66cb17f61430b7a6b9c3b4a1e40293b1671"}, + {file = "lazy_object_proxy-1.9.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:2d0daa332786cf3bb49e10dc6a17a52f6a8f9601b4cf5c295a4f85854d61de63"}, + {file = "lazy_object_proxy-1.9.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9cd077f3d04a58e83d04b20e334f678c2b0ff9879b9375ed107d5d07ff160171"}, + {file = "lazy_object_proxy-1.9.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:660c94ea760b3ce47d1855a30984c78327500493d396eac4dfd8bd82041b22be"}, + {file = "lazy_object_proxy-1.9.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:212774e4dfa851e74d393a2370871e174d7ff0ebc980907723bb67d25c8a7c30"}, + {file = "lazy_object_proxy-1.9.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:f0117049dd1d5635bbff65444496c90e0baa48ea405125c088e93d9cf4525b11"}, + {file = "lazy_object_proxy-1.9.0-cp38-cp38-win32.whl", hash = "sha256:0a891e4e41b54fd5b8313b96399f8b0e173bbbfc03c7631f01efbe29bb0bcf82"}, + {file = "lazy_object_proxy-1.9.0-cp38-cp38-win_amd64.whl", hash = "sha256:9990d8e71b9f6488e91ad25f322898c136b008d87bf852ff65391b004da5e17b"}, + {file = "lazy_object_proxy-1.9.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9e7551208b2aded9c1447453ee366f1c4070602b3d932ace044715d89666899b"}, + {file = "lazy_object_proxy-1.9.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5f83ac4d83ef0ab017683d715ed356e30dd48a93746309c8f3517e1287523ef4"}, + {file = "lazy_object_proxy-1.9.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7322c3d6f1766d4ef1e51a465f47955f1e8123caee67dd641e67d539a534d006"}, + {file = "lazy_object_proxy-1.9.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:18b78ec83edbbeb69efdc0e9c1cb41a3b1b1ed11ddd8ded602464c3fc6020494"}, + {file = "lazy_object_proxy-1.9.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:09763491ce220c0299688940f8dc2c5d05fd1f45af1e42e636b2e8b2303e4382"}, + {file = "lazy_object_proxy-1.9.0-cp39-cp39-win32.whl", hash = "sha256:9090d8e53235aa280fc9239a86ae3ea8ac58eff66a705fa6aa2ec4968b95c821"}, + {file = "lazy_object_proxy-1.9.0-cp39-cp39-win_amd64.whl", hash = "sha256:db1c1722726f47e10e0b5fdbf15ac3b8adb58c091d12b3ab713965795036985f"}, ] [[package]] @@ -135,6 +206,30 @@ files = [ {file = "mccabe-0.7.0.tar.gz", hash = "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325"}, ] +[[package]] +name = "mypy-extensions" +version = "0.4.3" +description = "Experimental type system extensions for programs checked with the mypy typechecker." +category = "main" +optional = false +python-versions = "*" +files = [ + {file = "mypy_extensions-0.4.3-py2.py3-none-any.whl", hash = "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d"}, + {file = "mypy_extensions-0.4.3.tar.gz", hash = "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"}, +] + +[[package]] +name = "pathspec" +version = "0.10.3" +description = "Utility library for gitignore style pattern matching of file paths." +category = "main" +optional = false +python-versions = ">=3.7" +files = [ + {file = "pathspec-0.10.3-py3-none-any.whl", hash = "sha256:3c95343af8b756205e2aba76e843ba9520a24dd84f68c22b9f93251507509dd6"}, + {file = "pathspec-0.10.3.tar.gz", hash = "sha256:56200de4077d9d0791465aa9095a01d421861e405b5096955051deefd697d6f6"}, +] + [[package]] name = "platformdirs" version = "2.6.2" @@ -153,44 +248,47 @@ test = ["appdirs (==1.4.4)", "covdefaults (>=2.2.2)", "pytest (>=7.2)", "pytest- [[package]] name = "pycodestyle" -version = "2.9.1" +version = "2.10.0" description = "Python style guide checker" category = "main" optional = false python-versions = ">=3.6" files = [ - {file = "pycodestyle-2.9.1-py2.py3-none-any.whl", hash = "sha256:d1735fc58b418fd7c5f658d28d943854f8a849b01a5d0a1e6f3f3fdd0166804b"}, - {file = "pycodestyle-2.9.1.tar.gz", hash = "sha256:2c9607871d58c76354b697b42f5d57e1ada7d261c261efac224b664affdc5785"}, + {file = "pycodestyle-2.10.0-py2.py3-none-any.whl", hash = "sha256:8a4eaf0d0495c7395bdab3589ac2db602797d76207242c17d470186815706610"}, + {file = "pycodestyle-2.10.0.tar.gz", hash = "sha256:347187bdb476329d98f695c213d7295a846d1152ff4fe9bacb8a9590b8ee7053"}, ] [[package]] name = "pyflakes" -version = "2.5.0" +version = "3.0.1" description = "passive checker of Python programs" category = "main" optional = false python-versions = ">=3.6" files = [ - {file = "pyflakes-2.5.0-py2.py3-none-any.whl", hash = "sha256:4579f67d887f804e67edb544428f264b7b24f435b263c4614f384135cea553d2"}, - {file = "pyflakes-2.5.0.tar.gz", hash = "sha256:491feb020dca48ccc562a8c0cbe8df07ee13078df59813b83959cbdada312ea3"}, + {file = "pyflakes-3.0.1-py2.py3-none-any.whl", hash = "sha256:ec55bf7fe21fff7f1ad2f7da62363d749e2a470500eab1b555334b67aa1ef8cf"}, + {file = "pyflakes-3.0.1.tar.gz", hash = "sha256:ec8b276a6b60bd80defed25add7e439881c19e64850afd9b346283d4165fd0fd"}, ] [[package]] name = "pylint" -version = "2.15.9" +version = "2.15.10" description = "python code static checker" category = "main" optional = false python-versions = ">=3.7.2" files = [ - {file = "pylint-2.15.9-py3-none-any.whl", hash = "sha256:349c8cd36aede4d50a0754a8c0218b43323d13d5d88f4b2952ddfe3e169681eb"}, - {file = "pylint-2.15.9.tar.gz", hash = "sha256:18783cca3cfee5b83c6c5d10b3cdb66c6594520ffae61890858fe8d932e1c6b4"}, + {file = "pylint-2.15.10-py3-none-any.whl", hash = "sha256:9df0d07e8948a1c3ffa3b6e2d7e6e63d9fb457c5da5b961ed63106594780cc7e"}, + {file = "pylint-2.15.10.tar.gz", hash = "sha256:b3dc5ef7d33858f297ac0d06cc73862f01e4f2e74025ec3eff347ce0bc60baf5"}, ] [package.dependencies] astroid = ">=2.12.13,<=2.14.0-dev0" colorama = {version = ">=0.4.5", markers = "sys_platform == \"win32\""} -dill = {version = ">=0.2", markers = "python_version < \"3.11\""} +dill = [ + {version = ">=0.2", markers = "python_version < \"3.11\""}, + {version = ">=0.3.6", markers = "python_version >= \"3.11\""}, +] isort = ">=4.2.5,<6" mccabe = ">=0.6,<0.8" platformdirs = ">=2.2.0" @@ -225,6 +323,18 @@ files = [ {file = "tomlkit-0.11.6.tar.gz", hash = "sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73"}, ] +[[package]] +name = "typing-extensions" +version = "4.4.0" +description = "Backported and Experimental Type Hints for Python 3.7+" +category = "main" +optional = false +python-versions = ">=3.7" +files = [ + {file = "typing_extensions-4.4.0-py3-none-any.whl", hash = "sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e"}, + {file = "typing_extensions-4.4.0.tar.gz", hash = "sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa"}, +] + [[package]] name = "wrapt" version = "1.14.1" @@ -301,5 +411,5 @@ files = [ [metadata] lock-version = "2.0" -python-versions = ">=3.10,<3.11" -content-hash = "959633dfe6335ab3f943a95c5fdd12ff1bb66cd03c5917704f10ae38d3a5009c" +python-versions = ">=3.10,<3.12" +content-hash = "382b5873c3c0e9f46b85c501cacfce4978c68bd40bedf7a41740f3ed464a1e22" diff --git a/scripts/pyproject.toml b/scripts/benchmarks/pyproject.toml similarity index 50% rename from scripts/pyproject.toml rename to scripts/benchmarks/pyproject.toml index 6fffb97a96..232da580c7 100644 --- a/scripts/pyproject.toml +++ b/scripts/benchmarks/pyproject.toml @@ -1,11 +1,3 @@ -[tool.black] -line-length = 120 - -[tool.ruff] -line-length = 120 -select = ["E", "F", "W", "I", "C", "RET", "ANN", "UP"] -target-version = "py310" - [tool.poetry] name = "scripts" version = "0.1.0" @@ -13,12 +5,14 @@ description = "" authors = ["Charles Marsh "] [tool.poetry.dependencies] -python = ">=3.10,<3.11" -autoflake = "^1.4" -flake8 = "^5.0.4" -pycodestyle = "^2.9.1" -pyflakes = "^2.5.0" -pylint = "^2.15.0" +python = ">=3.10,<3.12" +autoflake = "^2.0.0" +flake8 = "^6.0.0" +pycodestyle = "^2.10.0" +pyflakes = "^3.0.1" +pylint = "^2.15.10" +black = "^22.12.0" +isort = "^5.11.4" [tool.poetry.dev-dependencies] diff --git a/scripts/benchmarks/run.sh b/scripts/benchmarks/run.sh new file mode 100755 index 0000000000..6fe9ebb307 --- /dev/null +++ b/scripts/benchmarks/run.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env sh + +### +# Benchmark Ruff on the CPython codebase. +### + +cargo build --release && hyperfine --ignore-failure --warmup 10 \ + "./target/release/ruff ./resources/test/cpython/ --no-cache" diff --git a/scripts/benchmarks/run_comparisons.sh b/scripts/benchmarks/run_comparisons.sh new file mode 100755 index 0000000000..0101c3e646 --- /dev/null +++ b/scripts/benchmarks/run_comparisons.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env sh + +### +# Benchmark the incremental performance of each subsequent plugin. +### + +cargo build --release && hyperfine --ignore-failure --warmup 10 \ + "./target/release/ruff ./resources/test/cpython/ --no-cache" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select C90" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select I" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select D" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select UP" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select N" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select YTT" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ANN" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select S" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select BLE" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select FBT" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select B" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select A" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select C4" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select T10" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select EM" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ISC" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ICN" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select T20" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PT" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select Q" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select RET" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select SIM" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select TID" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ARG" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select DTZ" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select ERA" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PD" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PGH" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLC" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLE" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLR" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PLW" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select PIE" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select COM" \ + "./target/release/ruff ./resources/test/cpython/ --no-cache --extend-select RUF" diff --git a/scripts/benchmarks/run_plugins.sh b/scripts/benchmarks/run_plugins.sh new file mode 100644 index 0000000000..e69de29bb2 diff --git a/scripts/benchmarks/setup.sh b/scripts/benchmarks/setup.sh new file mode 100755 index 0000000000..3ede4df228 --- /dev/null +++ b/scripts/benchmarks/setup.sh @@ -0,0 +1,7 @@ +#!/usr/bin/env sh + +### +# Setup the CPython repository to enable benchmarking. +### + +git clone --branch 3.10 https://github.com/python/cpython.git resources/test/cpython diff --git a/scripts/run_flake8.py b/scripts/run_flake8.py deleted file mode 100755 index d3bbb501a8..0000000000 --- a/scripts/run_flake8.py +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env python3 -"""Wrapper around Flake8 to enable multiprocessing on all operating systems. - -As of Python 3.8, macOS's default "start method" for multiprocessing is `spawn`. Flake8 -requires a "start method" of `fork`, and disables multiprocessing if it detects `spawn` -or some other "start method". This script enables the `fork` start method before passing -along any command-line arguments to `flake8`. - -This has never caused me any problems, but note that they disabled this for a reason: -Flake8's plugin interface doesn't work with `spawn`, and the maintainer says that `fork` -is "pretty broken" on macOS. - -See: - -- https://github.com/pycqa/flake8/issues/955 -- https://github.com/PyCQA/flake8/issues/1337 -- https://github.com/PyCQA/flake8/issues/342 -- https://github.com/PyCQA/flake8/pull/1621 - -Example usage: python -m run_flake8 --select=E501 . -""" -import multiprocessing -import sys - -from flake8.main import cli - -if __name__ == "__main__": - multiprocessing.set_start_method("fork", force=True) - cli.main(sys.argv[1:])