2f7e2a8de3
Previously, the ecosystem checks formatted with the baseline then formatted again with `--diff` to get the changed files. Now, the ecosystem checks support a new mode where we: - Format with the baseline - Commit the changes - Reset to the target ref - Format again - Check the diff from the baseline commit This effectively tests Ruff changes on unformatted code rather than changes in previously formatted code (unless, of course, the project is already using Ruff). While this mode is the new default, I've retained the old one for local checks. The mode can be toggled with `--format-comparison <type>`. Includes some more aggressive resetting of the GitHub repositories when cached. Here, I've also stubbed comparison modes in which `black` is used as the baseline. While these do nothing here, #8419 adds support. I tested this with the commit from #8216 and ecosystem changes appear https://gist.github.com/zanieb/a982ec8c392939043613267474471a6e
ruff-ecosystem
Compare lint and format results for two different ruff versions (e.g. main and a PR) on real world projects.
Installation
From the Ruff project root, install with pip:
pip install -e ./python/ruff-ecosystem
Usage
ruff-ecosystem <check | format> <baseline executable> <comparison executable>
Note executable paths may be absolute, relative to the current working directory, or will be looked up in the current Python environment and PATH.
Run ruff check ecosystem checks comparing your debug build to your system Ruff:
ruff-ecosystem check ruff "./target/debug/ruff"
Run ruff format ecosystem checks comparing your debug build to your system Ruff:
ruff-ecosystem format ruff "./target/debug/ruff"
Run ruff format ecosystem checks comparing with changes to code that is already formatted:
ruff-ecosystem format ruff "./target/debug/ruff" --format-comparison ruff-then-ruff
The default output format is markdown, which includes nice summaries of the changes. You can use --output-format json to display the raw data — this is
particularly useful when making changes to the ecosystem checks.
Development
When developing, it can be useful to set the --pdb flag to drop into a debugger on failure:
ruff-ecosystem check ruff "./target/debug/ruff" --pdb
You can also provide a path to cache checkouts to speed up repeated runs:
ruff-ecosystem check ruff "./target/debug/ruff" --cache ./repos