mirror of
https://github.com/astral-sh/ruff
synced 2026-01-21 13:30:49 -05:00
fc94857a20
Closes #7239 - Refactors `scripts/check_ecosystem.py` into a new Python project at `python/ruff-ecosystem` - Includes [documentation](https://github.com/astral-sh/ruff/blob/zanie/ecosystem-format/python/ruff-ecosystem/README.md) now - Provides a `ruff-ecosystem` CLI - Fixes bug where `ruff check` report included "fixable" summary line - Adds truncation to `ruff check` reports - Otherwise we often won't see the `ruff format` reports - The truncation uses some very simple heuristics and could be improved in the future - Identifies diagnostic changes that occur just because a violation's fix available changes - We still show the diff for the line because it's could matter _where_ this changes, but we could improve this - Similarly, we could improve detection of diagnostic changes where just the message changes - Adds support for JSON ecosystem check output - I added this primarily for development purposes - If there are no changes, only errors while processing projects, we display a different summary message - When caching repositories, we now checkout the requested ref - Adds `ruff format` reports, which format with the baseline then the use `format --diff` to generate a report - Runs all CI jobs when the CI workflow is changed ## Known problems - Since we must format the project to get a baseline, the permalink line numbers do not exactly correspond to the correct range - This looks... hard. I tried using `git diff` and some wonky hunk matching to recover the original line numbers but it doesn't seem worth it. I think we should probably commit the formatted changes to a fork or something if we want great results here. Consequently, I've just used the start line instead of a range for now. - I don't love the comment structure — it'd be nice, perhaps, to have separate headings for the linter and formatter. - However, the `pr-comment` workflow is an absolute pain to change because it runs _separately_ from this pull request so I if I want to make edits to it I can only test it via manual workflow dispatch. - Lines are not printed "as we go" which means they're all held in memory, presumably this would be a problem for large-scale ecosystem checks - We are encountering a hard limit with the maximum comment length supported by GitHub. We will need to move the bulk of the report elsewhere. ## Future work - Update `ruff-ecosystem` to support non-default projects and `check_ecosystem_all.py` behavior - Remove existing ecosystem check scripts - Add preview mode toggle (#8076) - Add a toggle for truncation - Add hints for quick reproduction of runs locally - Consider parsing JSON output of Ruff instead of using regex to parse the text output - Links to project repositories should use the commit hash we checked against - When caching repositories, we should pull the latest changes for the ref - Sort check diffs by path and rule code only (changes in messages should not change order) - Update check diffs to distinguish between new violations and changes in messages - Add "fix" diffs - Remove existing formatter similarity reports - On release pull request, compare to the previous tag instead --------- Co-authored-by: konsti <konstin@mailbox.org>
47 lines
1.1 KiB
Markdown
47 lines
1.1 KiB
Markdown
# ruff-ecosystem
|
|
|
|
Compare lint and format results for two different ruff versions (e.g. main and a PR) on real world projects.
|
|
|
|
## Installation
|
|
|
|
From the Ruff project root, install with `pip`:
|
|
|
|
```shell
|
|
pip install -e ./python/ruff-ecosystem
|
|
```
|
|
|
|
## Usage
|
|
|
|
```shell
|
|
ruff-ecosystem <check | format> <baseline executable> <comparison executable>
|
|
```
|
|
|
|
Note executable paths may be absolute, relative to the current working directory, or will be looked up in the
|
|
current Python environment and PATH.
|
|
|
|
Run `ruff check` ecosystem checks comparing your debug build to your system Ruff:
|
|
|
|
```shell
|
|
ruff-ecosystem check ruff "./target/debug/ruff"
|
|
```
|
|
|
|
Run `ruff format` ecosystem checks comparing your debug build to your system Ruff:
|
|
|
|
```shell
|
|
ruff-ecosystem format ruff "./target/debug/ruff"
|
|
```
|
|
|
|
## Development
|
|
|
|
When developing, it can be useful to set the `--pdb` flag to drop into a debugger on failure:
|
|
|
|
```shell
|
|
ruff-ecosystem check ruff "./target/debug/ruff" --pdb
|
|
```
|
|
|
|
You can also provide a path to cache checkouts to speed up repeated runs:
|
|
|
|
```shell
|
|
ruff-ecosystem check ruff "./target/debug/ruff" --cache ./repos
|
|
```
|