Commit Graph

3921 Commits

Author SHA1 Message Date
Micha Reiser
f9129e435a Normalize '\r' in string literals to '\n'
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR normalizes line endings inside of strings to `\n` as required by the printer.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

I added a new test using `\r\n` and ran the ecosystem check. There are no remaining end of line panics. 


https://gist.github.com/MichaReiser/8f36b1391ca7b48475b3a4f592d74ff4

<!-- How was it tested? -->
2023-06-30 10:13:23 +02:00
Micha Reiser
dc65007fe9 Use rayon to parallelize the stability check
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR uses rayon to parallelize the stability check by scheduling each project as its own task.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

I ran the ecosystem check. It now makes use of all cores (except at the end, there are some large projects). 

## Performance

The check now completes in minutes where it took about 30 minutes before.

<!-- How was it tested? -->
2023-06-30 10:05:25 +02:00
Micha Reiser
9c2a75284b Preserve parentheses around left side of binary expression
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR fixes an issue where the binary expression formatting removed parentheses around the left hand side of an expression.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

I added a new regression test and re-ran the ecosystem check. It brings down the `check-formatter-stability` output from a 3.4MB file down to 900KB. 

<!-- How was it tested? -->
2023-06-30 09:52:14 +02:00
Micha Reiser
ae25638b0b Update Black tests (#5438) 2023-06-30 06:32:50 +00:00
Micha Reiser
f7969cf23c ecosystem: Run git command with no human interaction flag (#5435) 2023-06-29 09:19:11 +02:00
Micha Reiser
955e9ef821 Fix invalid syntax for binary expression in unary op (#5370) 2023-06-29 08:09:26 +02:00
Micha Reiser
38189ed913 Fix invalid printer IR error (#5422) 2023-06-29 08:09:13 +02:00
David Szotten
ca5e10b5ea format StmtTryStar (#5418) 2023-06-29 08:07:33 +02:00
Charlie Marsh
a973019358 Rewrite a variety of .contains() calls as matches! statements (#5432)
## Summary

These have the potential to be much more efficient, as we've seen in the
past.
2023-06-28 22:42:27 -04:00
Charlie Marsh
aa887d5a1d Use "manual" fixability for E731 in shadowed context (#5430)
## Summary

This PR makes E731 a "manual" fix in one other context: when the lambda
is shadowing another variable in the scope. Function declarations (with
shadowing) cause issues for type checkers, and so rewriting an
annotation, e.g., in branches of an `if` statement can lead to failures.

Closes https://github.com/astral-sh/ruff/issues/5421.
2023-06-28 22:00:06 -04:00
Charlie Marsh
72f7f11bac Use matches! for reserved attribute lookup (#5431) 2023-06-29 01:52:11 +00:00
Tom Kuson
5aa2a90e17 Add documentation to flake8-logging-format rules (#5417)
## Summary

Completes the documentation for the `flake8-logging-format` rules.
Related to #2646.

I included both the `flake8-logging-format` recommendation to use the
`extra` keyword and the Pylint recommendation to pass format values as
parameters so that formatting is done lazily, as #970 suggests the
Pylint logging rules are covered by this ruleset. Using lazy formatting
via parameters is probably more common than avoiding formatting entirely
in favour of the `extra` argument, regardless.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-06-29 01:30:11 +00:00
Charlie Marsh
0e89c94947 Run shadowed-variable analyses in deferred handlers (#5181)
## Summary

This PR extracts a bunch of complex logic from `add_binding`, instead
running the the shadowing rules in the deferred handler, thereby
decoupling the binding phase (during which we build up the semantic
model) from the analysis phase, and generally making `add_binding` much
more focused.

This was made possible by improving the semantic model to better handle
deletions -- previously, we'd "lose track" of bindings if they were
deleted, which made this kind of refactor impossible.

## Test Plan

We have good automated coverage for this, but I want to benchmark it
separately.
2023-06-29 00:08:18 +00:00
Eric H
139a9f757b Update default configuration.md to mention C901 rule (#5397) 2023-06-28 21:22:16 +00:00
Charlie Marsh
c5e20505f8 Remove an unsafe access in the resolver (#5428) 2023-06-28 19:08:10 +00:00
Charlie Marsh
69c4b7fa11 Add dedicated struct for implicit imports (#5427)
## Summary

This was some feedback on a prior PR that I decided to act on
separately.
2023-06-28 18:55:43 +00:00
Charlie Marsh
0e12eb3071 Add a snapshot test for native module resolution (#5423) 2023-06-28 18:16:39 +00:00
Charlie Marsh
864f50a3a4 Remove all unwrap calls from the resolver (#5426) 2023-06-28 18:06:17 +00:00
Charlie Marsh
4d90a5a9bc Move resolver tests out to top-level (#5424)
## Summary

These are really tests for the entire crate.
2023-06-28 13:25:37 -04:00
Charlie Marsh
1d2d015bc5 Make standard input detection robust to invalid arguments (#5393)
## Summary

This PR fixes a silent failure that manifested itself in
https://github.com/astral-sh/ruff-vscode/issues/238. In short, if the
user provided invalid arguments to Ruff in the VS Code extension (like
`"ruff.args": ["a"]`), then we generated something like the following
command:

```console
/path/to/ruff --force-exclude --no-cache --no-fix --format json - --fix a --stdin-filename /path/to/file.py
```

Since this contains both `-` and `a` as the "input files", Ruff would
treat this as if we're linting the files names `-` and `a`, rather than
linting standard input.

This PR modifies out standard input detection to force standard input
when `--stdin-filename` is present, or at least one file is `-`. (We
then warn and ignore the others.)
2023-06-28 14:52:23 +00:00
Charlie Marsh
ea7bb199bc Fill-in missing implementation for is_native_module_file_name (#5410)
## Summary

This was just an oversight -- the last remaining `todo!()` that I never
filled in. We clearly don't have any test coverage for it yet, but this
mimics the Pyright implementation.
2023-06-28 14:50:54 +00:00
Charlie Marsh
979049b2a6 Make lib iteration platform-specific (#5406) 2023-06-28 13:52:20 +00:00
Charlie Marsh
6587fb844a Add snapshot tests for resolver (#5404)
## Summary

This PR adds some snapshot tests for the resolver based on executing
resolutions within a "mock" of the Airflow repo (that is: a folder that
contains a subset of the repo's files, but all empty, and with an
only-partially-complete virtual environment). It's intended to act as a
lightweight integration test, to enable us to test resolutions on a
"real" project without adding a dependency on Airflow itself.
2023-06-28 13:38:51 +00:00
Dhruv Manilawala
a68a86e18b fixup! Consider Jupyter index for code frames (--show-source) (#5402) (#5414) 2023-06-28 10:25:05 +00:00
Christian Clauss
b42d76494c types.rs: fnmatch url should point to current Python docs (#5413)
Like #5412
2023-06-28 15:54:13 +05:30
David Szotten
c7adb9117f format StmtAsyncWith (#5376)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-06-28 10:21:44 +00:00
David Szotten
1979103ec0 Format StmtTry (#5222)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-06-28 10:02:15 +00:00
Christian Clauss
9e2fd0c620 ruff rule SLOT uses URL to current Python docs (#5412)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Currently the URL at the bottom of the `ruff rule SLOT00x` output points
to Python 3.7 docs.
Given that Python 3.7 is now end-of-life (as of yesterday), let's
instead point users to the current Python docs.

## Test Plan

<!-- How was it tested? -->
2023-06-28 09:48:52 +00:00
Charlie Marsh
366edc5a3f Fix string annotation in docs (#5411) 2023-06-28 03:29:56 +00:00
Dhruv Manilawala
2aecaf5060 Consider Jupyter index for code frames (--show-source) (#5402)
## Summary

Consider Jupyter index for code frames (`--show-source`).

This solves two problems as mentioned in the linked issue:

> Omit any contents from adjoining cells

If the Jupyter index is present, we'll use that to check if the
surrounding
lines belong to the same cell as the content line. If not, we'll skip
that line
until we either reach the one which does or we reach the content line.

> code frame line number

If the Jupyter index is present, we'll use that to get the actual start
line in
corresponding to the computed start index.

## Test Plan

`cargo run --bin ruff -- check --no-cache --isolated --select=ALL --show-source /path/to/notebook.ipynb`

fixes: #5395
2023-06-28 08:54:51 +05:30
Dhruv Manilawala
d19324df69 Add Jupyter integration to the docs (#5403)
## Summary

Add Jupyter integration to the docs, specifically the Configuration and
FAQ sections.

## Test Plan

`mkdocs serve` and check that the new sections are visible and
functional.

fixes: #5396
2023-06-28 00:27:24 +00:00
Marti Raudsepp
2c99b268c6 Exclude docstrings from PYI053 (#5405)
## Summary

The `Y053` rule of `flake8-pyi` ignores docstrings, it only triggers on
other string literals.

The separate `Y021/PYI021` rule exists to disallow docstrings.

## Test Plan

Added some `# OK` test cases to `PYI053.py(i)` files.
2023-06-28 00:19:20 +00:00
Charlie Marsh
56f73de0cb Misc. clean-up for import resolver (#5401)
## Summary

Renaming functions, adding documentation, refactoring the test
infrastructure a bit.
2023-06-27 19:27:12 +00:00
Tom Kuson
a0a93a636f Implement Pylint single-string-used-for-slots (C0205) as single-string-slots (PLC0205) (#5399)
## Summary

Implement Pylint rule `single-string-used-for-slots` (`C0205`) as
`single-string-slots` (`PLC0205`). This rule checks for single strings
being assigned to `__slots__`. For example

```python
class Foo:
    __slots__: str = "bar"

    def __init__(self, bar: str) -> None:
        self.bar = bar
```

should be

```python
class Foo:
    __slots__: tuple[str, ...] = ("bar",)

    def __init__(self, bar: str) -> None:
        self.bar = bar
```

Related to #970. Includes documentation.

## Test Plan

`cargo test`
2023-06-27 18:33:58 +00:00
Tom Kuson
035f8993f4 Complete documentation for pydocstyle rules (#5387)
## Summary

Completes the documentation for the `pydocstyle` ruleset. Related to
#2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-06-27 18:12:21 +00:00
Charlie Marsh
032b967b05 Enable --watch for Jupyter notebooks (#5394)
## Summary

The list of extensions that support watching is hard-coded
(unfortunately); this PR adds `.ipynb` to the list.
2023-06-27 12:53:47 -04:00
Dhruv Manilawala
962479d943 Replace same length equal line with dash line in D407 (#5383)
## Summary

Replace same length equal line with dash line in D407

Do we want to update the message and autofix title to reflect this
change?

## Test Plan

Added test cases for:
- Equal line length == dash line length
- Equal line length != dash line length

fixes: #5378
2023-06-27 16:50:20 +00:00
Evan Rittenhouse
ff0d0ab7a0 Add applicability to pydocstyle (#5390) 2023-06-27 12:40:19 -04:00
Evan Rittenhouse
0585e14d3b Add applicability to flake8_pytest_style (#5389) 2023-06-27 12:39:56 -04:00
Charlie Marsh
1ed227a1e0 Port Pyright's import resolver to Rust (#5381)
## Summary

This PR contains the first step towards enabling robust first-party,
third-party, and standard library import resolution in Ruff (including
support for `typeshed`, stub files, native modules, etc.) by porting
Pyright's import resolver to Rust.

The strategy taken here was to start with a more-or-less direct port of
the Pyright's TypeScript resolver. The code is intentionally similar,
and the test suite is effectively a superset of Pyright's test suite for
its own resolver. Due to the nature of the port, the code is very, very
non-idiomatic for Rust. The code is also entirely unused outside of the
test suite, and no effort has been made to integrate it with the rest of
the codebase.

Future work will include:

- Refactoring the code (now that it works) to match Rust and Ruff
idioms.
- Further testing, in practice, to ensure that the resolver can resolve
imports in a complex project, when provided with a virtual environment
path.
- Caching, to minimize filesystem lookups and redundant resolutions.
- Integration into Ruff itself (use Ruff's existing settings, find rules
that can make use of robust resolution, etc.)
2023-06-27 16:15:07 +00:00
Charlie Marsh
502e15585d Ignore unpacking in iteration-over-set (#5392)
Closes #5386.
2023-06-27 15:33:42 +00:00
konstin
520f4f33c3 Fix ruff_dev repeat by removing short argument (#5388)
ruff_dev repeat recently broke (i think with the cargo update?):

> thread 'main' panicked at 'Command repeat: Short option names must be
unique for each argument, but '-n' is in use by both 'no_cache' and
'repeat''

This fixes this by removing the short argument.
2023-06-27 13:29:20 +00:00
konstin
7f6cb9dfb5 Format call expressions (without call chaining) (#5341)
## Summary

This formats call expressions with magic trailing comma and parentheses
behaviour but without call chaining

## Test Plan

Lots of new test fixtures, including some that don't work yet
2023-06-27 09:29:40 +00:00
David Szotten
50a7769d69 magic trailing comma for ExprList (#5365) 2023-06-26 21:59:01 +02:00
Evan Rittenhouse
190bed124f [perflint] Implement try-except-in-loop (PERF203) (#5166)
## Summary

Implements PERF203 from #4789, which throws if a `try/except` block is
inside of a loop. Not sure if we want to extend the diagnostic to the
`except` as well, but I thought that that may get a little messy. We may
also want to just throw on the word `try` - open to suggestions though.

## Test Plan
`cargo test`
2023-06-26 17:34:37 +00:00
Charlie Marsh
d53b986fd4 Fix autofix capabilities in playground (#5375)
## Summary

These had just bitrotted over time -- we were no longer passing along
the row-and-column indices, etc.

## Test Plan

![Screen Shot 2023-06-26 at 12 03 41
PM](https://github.com/astral-sh/ruff/assets/1309177/6791330d-010b-45d3-91ef-531d4745193f)
2023-06-26 16:40:28 +00:00
Charlie Marsh
8a1bb7a5af Fix version number in playground (#5372)
## Summary

`v0.0.275` in the top-right was showing `v0.0.0` at all times.

## Test Plan

![Screen Shot 2023-06-26 at 11 31 16
AM](https://github.com/astral-sh/ruff/assets/1309177/e6cd0e19-6a5f-4b46-a060-54f492524737)
2023-06-26 15:56:12 +00:00
Dhruv Manilawala
2fc38d81e6 Experimental release for Jupyter notebook integration (#5363)
## Summary

Experimental release for Jupyter Notebook integration.

Currently, this requires a user to explicitly opt-in using the
[include](https://beta.ruff.rs/docs/settings/#include) configuration:

```toml
[tool.ruff]
include = ["*.py", "*.pyi", "**/pyproject.toml", "*.ipynb"]
```

Or, a user can pass in the file directly:

```sh
ruff check path/to/notebook.ipynb
```

For known limitations, please refer #5188 

## Test Plan

Following command should work without the `--all-features` flag:

```sh
cargo dev round-trip /path/to/notebook.ipynb
```

Following command should work with the above config file along with
`select = ["ALL"]`:

```sh
cargo run --bin ruff -- check --no-cache --config=../test-repos/openai-cookbook/pyproject.toml --fix ../test-repos/openai-cookbook/
```

Passing the Jupyter notebook directly:

```sh
cargo run --bin ruff -- check --no-cache --isolated --select=ALL --fix ../test-repos/openai-cookbook/examples/Classification_using_embeddings.ipynb
```
2023-06-26 21:22:42 +05:30
Charlie Marsh
fa1b85b3da Remove prelude from ruff_python_ast (#5369)
## Summary

Per @MichaReiser, this is causing more confusion than it is helpful.
2023-06-26 11:43:49 -04:00
Tom Kuson
baa7264ca4 Add documentation for flake8-2020 (#5366)
## Summary

Completes the documentation for the `flake8-2020` ruleset. Related to
#2646 .

## Test Plan

`python scripts/check_docs_formatted.py`
2023-06-26 15:24:42 +00:00