Commit Graph

2017 Commits

Author SHA1 Message Date
Harutaka Kawamura 9c3fbcdf4a
Add `PT011` and `PT012` docs (#6362) 2023-08-06 21:28:24 -04:00
Konrad Listwan-Ciesielski 61532e8aad
Add `DTZ003` and `DTZ004` docs (#6223)
Changes:
- Fixes typo and repeated phrase in `DTZ002`
- Adds docs for `DTZ003`
- Adds docs for `DTZ004`
- Adds example for <=Python3.10 in `DTZ001`

Related to: https://github.com/astral-sh/ruff/issues/2646
2023-08-07 01:21:14 +00:00
Charlie Marsh 9171e97d15
Avoid allocation in no-signature (#6375) 2023-08-06 15:27:56 +00:00
Charlie Marsh a5a29bb8d6
Revert change to `require_git(false)` in `WalkBuilder` (#6368)
## Summary

This was changed to fix https://github.com/astral-sh/ruff/issues/5930
(respect `.gitignore` for unzipped source repositories), but led to
undesirable behavior whereby `.gitignore` files in parent directories
are respected regardless of whether you're working in a child git
repository (see: https://github.com/astral-sh/ruff/issues/6335). The
latter is a bigger problem than the former is an important use-case to
support, so pragmatically erring on the side of a revert.

Closes https://github.com/astral-sh/ruff/issues/6335.
2023-08-05 19:45:50 +00:00
Zixuan Li be657f5e7e
Respect typing_extensions imports of Annotated for B006. (#6361)
`typing_extensions.Annotated` should be treated the same way as
`typing.Annotated`.
2023-08-05 17:39:52 +00:00
Charlie Marsh 76148ddb76
Store call paths rather than stringified names (#6102)
## Summary

Historically, we've stored "qualified names" on our
`BindingKind::Import`, `BindingKind::SubmoduleImport`, and
`BindingKind::ImportFrom` structs. In Ruff, a "qualified name" is a
dot-separated path to a symbol. For example, given `import foo.bar`, the
"qualified name" would be `"foo.bar"`; and given `from foo.bar import
baz`, the "qualified name" would be `foo.bar.baz`.

This PR modifies the `BindingKind` structs to instead store _call paths_
rather than qualified names. So in the examples above, we'd store
`["foo", "bar"]` and `["foo", "bar", "baz"]`. It turns out that this
more efficient given our data access patterns. Namely, we frequently
need to convert the qualified name to a call path (whenever we call
`resolve_call_path`), and it turns out that we do this operation enough
that those conversations show up on benchmarks.

There are a few other advantages to using call paths, rather than
qualified names:

1. The size of `BindingKind` is reduced from 32 to 24 bytes, since we no
longer need to store a `String` (only a boxed slice).
2. All three import types are more consistent, since they now all store
a boxed slice, rather than some storing an `&str` and some storing a
`String` (for `BindingKind::ImportFrom`, we needed to allocate a
`String` to create the qualified name, but the call path is a slice of
static elements that don't require that allocation).
3. A lot of code gets simpler, in part because we now do call path
resolution "earlier". Most notably, for relative imports (`from .foo
import bar`), we store the _resolved_ call path rather than the relative
call path, so the semantic model doesn't have to deal with that
resolution. (See that `resolve_call_path` is simpler, fewer branches,
etc.)

In my testing, this change improves the all-rules benchmark by another
4-5% on top of the improvements mentioned in #6047.
2023-08-05 15:21:50 +00:00
Harutaka Kawamura 501f537cb8
Avoid auto-fixing UP031 if there are comments within the right-hand side (#6364) 2023-08-05 11:14:29 -04:00
Dhruv Manilawala 1ac2699b5e
Update `F841` autofix to not remove line magic expr (#6141)
## Summary

Update `F841` autofix to not remove line magic expr

## Test Plan

Added test case for assignment statement with and without type
annotation

fixes: #6116
2023-08-05 00:45:01 +00:00
Dhruv Manilawala 32fa05765a
Use `Jupyter` mode while parsing Notebook files (#5552)
## Summary

Enable using the new `Mode::Jupyter` for the tokenizer/parser to parse
Jupyter line magic tokens.

The individual call to the lexer i.e., `lex_starts_at` done by various
rules should consider the context of the source code (is this content
from a Jupyter Notebook?). Thus, a new field `source_type` (of type
`PySourceType`) is added to `Checker` which is being passed around as an
argument to the relevant functions. This is then used to determine the
`Mode` for the lexer.

## Test Plan

Add new test cases to make sure that the magic statement is considered
while generating the diagnostic and autofix:
* For `I001`, if there's a magic statement in between two import blocks,
they should be sorted independently

fixes: #6090
2023-08-05 00:32:07 +00:00
Charlie Marsh d788957ec4
Allow capitalized names for logger candidate heuristic match (#6356)
Closes https://github.com/astral-sh/ruff/issues/6353.
2023-08-04 23:25:34 +00:00
Victor Hugo Gomes 78a370303b
[`flake8-pyi`] Add tests cases for bad imports from PYI027 to PYI022 (UP035) (#6354)
## Summary
As of version
[23.1.0](2a86db8271/CHANGELOG.md?plain=1#L158-L160),
`flake8-pyi` remove the rule `Y027`.

The errors that resulted in `PYI027` are now being emitted by `PYI022`
(`UP035`).

ref: #848 

## Test Plan

Add new tests cases.
2023-08-04 19:00:33 -04:00
Charlie Marsh 5e73345a1c
Avoid panic with positional-only arguments in `PYI019` (#6350)
## Summary

Previously, failed on methods like:

```python
@classmethod
def bad_posonly_class_method(cls: type[_S], /) -> _S: ...  # PYI019
```

Since we check if there are any positional-only or non-positional
arguments, but then do an unsafe access on `parameters.args`.

Closes https://github.com/astral-sh/ruff/issues/6349.

## Test Plan

`cargo test` (verified that `main` panics on the new fixtures)
2023-08-04 18:37:07 +00:00
Charlie Marsh b8fd69311c
Remove `ruff_python_ast` prefix in fixes.rs (#6346) 2023-08-04 16:48:20 +00:00
Charlie Marsh fa5c9cced9
Ignore same-line docstrings for lines-before and lines-after rules (#6344)
These rules assume that the docstring is on its own line. pydocstyle
treats them inconsistently, so I'm just going to disable them in this
case.

Closes https://github.com/astral-sh/ruff/issues/6329.
2023-08-04 16:08:36 +00:00
Harutaka Kawamura 08dd87e04d
Avoid auto-fixing UP032 if comments are present around format call arguments (#6342) 2023-08-04 15:37:23 +00:00
konsti 9bb21283ca
More similarity index digits (#6343)
**Summary** We were at similarity index 0.998 for django, we need more
decimal places, now we're at 0.99779.

**Test Plan** n/a
2023-08-04 17:12:33 +02:00
Charlie Marsh 4d47dfd6c0
Tweak breaking groups for comprehensions (#6321)
## Summary

Fixes some comprehension formatting by avoiding creating the group for
the comprehension itself (so that if it breaks, all parts break on their
own lines, e.g. the `for` and the `if` clauses).

Closes https://github.com/astral-sh/ruff/issues/6063.

## Test Plan

Bunch of new fixtures.
2023-08-04 14:00:54 +00:00
konsti 99baad12d8
Call chain formatting in fluent style (#6151)
Implement fluent style/call chains. See the `call_chains.py` formatting
for examples.

This isn't fully like black because in `raise A from B` they allow `A`
breaking can influence the formatting of `B` even if it is already
multiline.

Similarity index:

| project      | main  | PR    |
|--------------|-------|-------|
| build        | ???   | 0.753 |
| django       | 0.991 | 0.998 |
| transformers | 0.993 | 0.994 |
| typeshed     | 0.723 | 0.723 |
| warehouse    | 0.978 | 0.994 |
| zulip        | 0.992 | 0.994 |

Call chain formatting is affected by
https://github.com/astral-sh/ruff/issues/627, but i'm cutting scope
here.

Closes #5343

**Test Plan**:
 * Added a dedicated call chains test file
 * The ecosystem checks found some bugs
 * I manually check django and zulip formatting

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-04 13:58:01 +00:00
Charlie Marsh 35bdbe43a8
Flag `comparison-with-itself` on builtin calls (#6324)
## Summary

Extends `comparison-with-itself` to cover simple function calls on
known-pure functions, like `id`. For example, we now flag `id(x) ==
id(x)`.

Closes https://github.com/astral-sh/ruff/issues/6276.

## Test Plan

`cargo test`
2023-08-04 09:51:41 -04:00
Charlie Marsh 3a985dd71e
Rename `CommentPlacement#then_with` to `or_else` (#6341)
Per nits in the PR.
2023-08-04 13:50:57 +00:00
Charlie Marsh 1e3fe67ca5
Refactor and rename `skip_trailing_trivia` (#6312)
Based on feedback here:
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282747964.
2023-08-04 13:30:53 +00:00
Charlie Marsh 38a96c88c1
Add missing enable check for bad-string-format-character (#6340) 2023-08-04 13:27:53 +00:00
Micha Reiser f4831d5a26
Formatter comment handling nits (#6339) 2023-08-04 13:22:16 +00:00
konsti 1031bb6550
Formatter: Add SourceType to context to enable special formatting for stub files (#6331)
**Summary** This adds the information whether we're in a .py python
source file or in a .pyi stub file to enable people working on #5822 and
related issues.

I'm not completely happy with `Default` for something that depends on
the input.

**Test Plan** None, this is currently unused, i'm leaving this to first
implementation of stub file specific formatting.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-04 11:52:26 +00:00
David Szotten fe97a2a302
Fix panic with empty attribute inner comment (#6332)
Fixes https://github.com/astral-sh/ruff/issues/6181
2023-08-04 11:59:55 +02:00
konsti a48d16e025
Replace `Formatter<PyFormatContext<'_>>` with `PyFormatter` (#6330)
This is a refactoring to use the type alias in more places. In the
process, I had to fix and run generate.py. There are no functional
changes.
2023-08-04 10:48:58 +02:00
Charlie Marsh 8a5bc93fdd
Make the `Nodes` vector generic on node type (#6328) 2023-08-04 03:57:15 +00:00
Charlie Marsh 6da527170f
Match left-hand side `types()` call in `types-comparison` (#6326)
Follow-up to https://github.com/astral-sh/ruff/pull/6325, to avoid false
positives in cases like:

```python
if x == int:
    ...
```

Which is valid, since we don't know that we're comparing the type _of_
something -- we're comparing the type objects directly.
2023-08-03 23:01:23 -04:00
Charlie Marsh 8cddb6c08d
Include comparisons to builtin types in `type-comparison` rule (#6325)
## Summary

Extends `type-comparison` to flag:

```python
if type(obj) is int:
    pass
```

In addition to the existing cases, like:

```python
if type(obj) is type(1):
    pass
```

Closes https://github.com/astral-sh/ruff/issues/6260.
2023-08-04 02:25:19 +00:00
Victor Hugo Gomes b8ca220eeb
[`flake8-pyi`] Implement PYI055 (#6316) 2023-08-04 01:36:00 +00:00
Charlie Marsh 1d8759d5df
Generalize comment-after-bracket handling to lists, sets, etc. (#6320)
## Summary

We already support preserving the end-of-line comment in calls and type
parameters, as in:

```python
foo(  # comment
    bar,
)
```

This PR adds the same behavior for lists, sets, comprehensions, etc.,
such that we preserve:

```python
[  # comment
    1,
    2,
    3,
]
```

And related cases.
2023-08-04 01:28:05 +00:00
Charlie Marsh d3aa8b4ee0
Add API to chain comment placement operations (#6319)
## Summary

This PR adds an API for chaining comment placement methods based on the
[`then_with`](https://doc.rust-lang.org/std/cmp/enum.Ordering.html#method.then_with)
from `Ordering` in the standard library.

For example, you can now do:

```rust
try_some_case(comment).then_with(|comment| try_some_other_case_if_still_default(comment))
```

This lets us avoid this kind of pattern, which I've seen in
`placement.rs` and used myself before:

```rust
let comment = match handle_own_line_comment_between_branches(comment, preceding, locator) {
    CommentPlacement::Default(comment) => comment,
    placement => return placement,
};
```
2023-08-03 21:08:50 -04:00
Charlie Marsh 5f225b18ab
Generalize bracketed end-of-line comment handling (#6315)
Micha suggested this in
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282774151, and
it allows us to unify the implementations for arguments and type params.
2023-08-03 20:51:03 +00:00
Charlie Marsh 1705fcef36
Mark trailing comments in parenthesized tests (#6287)
## Summary

This ensures that we treat `# comment` as parenthesized in contexts
like:

```python
while (
    True
    # comment
):
    pass
```

The same logic applies equally to `for`, `async for`, `if`, `with`, and
`async with`. The general pattern is that you have an expression which
precedes a colon-separated suite.
2023-08-03 20:45:03 +00:00
konsti 51ff98f9e9
Make formatter ecosystem check failure output better understandable (#6300)
**Summary** Prompted by
https://github.com/astral-sh/ruff/pull/6257#issuecomment-1661308410, it
tried to make the ecosystem script output on failure better
understandable. All log messages are now written to a file, which is
printed on error. Running locally progress is still shown.

Looking through the log output i saw that we currently log syntax errors
in input, which is confusing because they aren't actual errors, but we
don't check that these files don't change due to parser regressions or
improvements. I added `--files-with-errors` to catch that.

**Test Plan** CI
2023-08-03 20:23:25 +02:00
Charlie Marsh b3f3529499
Improve comments around `Arguments` handling in classes (#6310)
## Summary

Based on the confusion here:
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282754515.

I looked into moving this logic into `placement.rs`, but I think it's
trickier than it may appear.
2023-08-03 12:34:03 -04:00
Charlie Marsh 2fa508793f
Return a slice in `StmtClassDef#bases` (#6311)
Slices are strictly more flexible, since you can always convert to an
iterator, etc., but not the other way around. Suggested in
https://github.com/astral-sh/ruff/pull/6259#discussion_r1282730994.
2023-08-03 16:21:55 +00:00
Zanie Blue 718e3945e3
Add rule to upgrade type alias annotations to keyword (UP040) (#6289)
Adds rule to convert type aliases defined with annotations i.e. `x:
TypeAlias = int` to the new PEP-695 syntax e.g. `type x = int`.

Does not support using new generic syntax for type variables, will be
addressed in a follow-up.
Added as part of pyupgrade — ~the code 100 as chosen to avoid collision
with real pyupgrade codes~.

Part of #4617 
Builds on #5062
2023-08-03 16:13:06 +00:00
Charlie Marsh c75e8a8dab
Move `ExprCall`'s `NeedsParentheses` impl into `expr_call.rs` (#6309)
Accidental move.
2023-08-03 16:01:01 +00:00
Harutaka Kawamura 74e734e962
More precise invalid expression check for `UP032` (#6308) 2023-08-03 15:49:02 +00:00
Zanie Blue 0e18abcf95
Add `is_` and `is_not` to excluded functions for `FBT003` (#6307)
These methods are commonly used in SQLAlchemy.

See https://github.com/astral-sh/ruff/discussions/6302
2023-08-03 10:41:45 -05:00
Anders Kaseorg 7c8bcede5b
Broaden appropriate flake8-pyi rules to check non-stub code too (#6297)
Of the rules that flake8-pyi enforces for `.pyi` type stubs, many of
them equally make sense to check in normal runtime code with type
annotations. Broaden these rules to check all files:

PYI013 ellipsis-in-non-empty-class-body
PYI016 duplicate-union-member
PYI018 unused-private-type-var
PYI019 custom-type-var-return-type
PYI024 collections-named-tuple
PYI025 unaliased-collections-abc-set-import
PYI030 unnecessary-literal-union
PYI032 any-eq-ne-annotation
PYI034 non-self-return-type
PYI036 bad-exit-annotation
PYI041 redundant-numeric-union
PYI042 snake-case-type-alias
PYI043 t-suffixed-type-alias
PYI045 iter-method-return-iterable
PYI046 unused-private-protocol
PYI047 unused-private-type-alias
PYI049 unused-private-typed-dict
PYI050 no-return-argument-annotation-in-stub (Python ≥ 3.11)
PYI051 redundant-literal-union
PYI056 unsupported-method-call-on-all

The other rules are stub-specific and remain enabled only in `.pyi`
files.

PYI001 unprefixed-type-param
PYI002 complex-if-statement-in-stub
PYI003 unrecognized-version-info-check
PYI004 patch-version-comparison
PYI005 wrong-tuple-length-version-comparison (could make sense to
broaden, see
https://github.com/astral-sh/ruff/pull/6297#issuecomment-1663314807)
PYI006 bad-version-info-comparison (same)
PYI007 unrecognized-platform-check
PYI008 unrecognized-platform-name
PYI009 pass-statement-stub-body
PYI010 non-empty-stub-body
PYI011 typed-argument-default-in-stub
PYI012 pass-in-class-body
PYI014 argument-default-in-stub
PYI015 assignment-default-in-stub
PYI017 complex-assignment-in-stub
PYI020 quoted-annotation-in-stub
PYI021 docstring-in-stub
PYI026 type-alias-without-annotation (could make sense to broaden, but
gives many false positives on runtime code as currently implemented)
PYI029 str-or-repr-defined-in-stub
PYI033 type-comment-in-stub
PYI035 unassigned-special-variable-in-stub
PYI044 future-annotations-in-stub
PYI048 stub-body-multiple-statements
PYI052 unannotated-assignment-in-stub
PYI053 string-or-bytes-too-long
PYI054 numeric-literal-too-long

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-08-03 11:40:42 -04:00
Harutaka Kawamura 30c2e9430e
Update `UP032` to support `await` expressions (#6304)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

In Python >= 3.7, `await` can be included in f-strings. 

https://bugs.python.org/issue28942

## Test Plan

<!-- How was it tested? -->

Existing tests
2023-08-03 09:53:36 -05:00
Harutaka Kawamura b6f0316d55
Add PT013 and PT015 docs (#6303)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

#2646

## Test Plan

<!-- How was it tested? -->
2023-08-03 09:51:52 -05:00
Charlie Marsh 9f3567dea6
Use `range: _` in lieu of `range: _range` (#6296)
## Summary

`range: _range` is slightly inconvenient because you can't use it
multiple times within a single match, unlike `_`.
2023-08-02 22:11:13 -04:00
qdegraaf d40597a266
[`flake8-pyi`] Implement `custom_type_var_return_type` (`PYI019`) (#6204)
## Summary

Implements `Y019` from
[flake8-pyi](https://github.com/PyCQA/flake8-pyi).

The rule checks if

-  instance methods that return `self` 
-  class methods that return an instance of `cls`
- `__new__` methods

Return a custom `TypeVar` instead of `typing.Self` and raises a
violation if this is the case. The rule also covers
[PEP-695](https://peps.python.org/pep-0695/) syntax as introduced in
upstream in https://github.com/PyCQA/flake8-pyi/pull/402

## Test Plan

Added fixtures with test cases from upstream implementation (plus
additional one for an excluded edge case, mentioned in upstream
implementation)
2023-08-03 00:42:42 +00:00
Silvano Cerza 82410524d9
[`pylint`] Implement Pylint `bad-format-character` (`E1300`) (#6171)
## Summary

Relates to #970.

Add new `bad-format-character` Pylint rule.

I had to make a change in `crates/ruff_python_literal/src/format.rs` to
get a more detailed error in case the format character is not correct. I
chose to do this since most of the format spec parsing functions are
private. It would have required me reimplementing most of the parsing
logic just to know if the format char was correct.

This PR also doesn't reflect current Pylint functionality in two ways.

It supports new format strings correctly, Pylint as of now doesn't. See
pylint-dev/pylint#6085.

In case there are multiple adjacent string literals delimited by
whitespace the index of the wrong format char will relative to the
single string. Pylint will instead reported it relative to the
concatenated string.

Given this:
```
"%s" "%z" % ("hello", "world")
```

Ruff will report this:
```Unsupported format character 'z' (0x7a) at index 1```

Pylint instead:
```Unsupported format character 'z' (0x7a) at index 3```

I believe it's more sensible to report the index relative to the
individual string.

## Test Plan

Added new snapshot and a small test in
`crates/ruff_python_literal/src/format.rs`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-02 21:32:43 +00:00
Zanie Blue 5b2e973fa5
Add formatting of type alias statements (#6162)
Part of #5062 
Extends https://github.com/astral-sh/ruff/pull/6161
Closes #5929
2023-08-02 20:40:32 +00:00
Zanie Blue 1a60d1e3c6
Add formatting of type parameters in class and function definitions (#6161)
Part of #5062 
Closes https://github.com/astral-sh/ruff/issues/5931

Implements formatting of a sequence of type parameters in a dedicated
struct for reuse by classes, functions, and type aliases (preparing for
#5929). Adds formatting of type parameters in class and function
definitions — previously, they were just elided.
2023-08-02 20:29:28 +00:00
Charlie Marsh 9425ed72a0
Break global and nonlocal statements over continuation lines (#6172)
## Summary

Builds on #6170 to break `global` and `nonlocal` statements, such that
we get:

```python
def f():
    global \
        analyze_featuremap_layer, \
        analyze_featuremapcompression_layer, \
        analyze_latencies_post, \
        analyze_motions_layer, \
        analyze_size_model
```

Instead of:

```python
def f():
    global analyze_featuremap_layer, analyze_featuremapcompression_layer, analyze_latencies_post, analyze_motions_layer, analyze_size_model
```

Notably, we avoid applying this formatting if the statement ends in a
comment. Otherwise, the comment would _need_ to be placed after the last
item, like:

```python
def f():
    global \
        analyze_featuremap_layer, \
        analyze_featuremapcompression_layer, \
        analyze_latencies_post, \
        analyze_motions_layer, \
        analyze_size_model  # noqa
```

To me, this seems wrong (and would break the `# noqa` comment). Ideally,
the items would be parenthesized, and the comment would be on the inner
parenthesis, like:

```python
def f():
    global (  # noqa
        analyze_featuremap_layer,
        analyze_featuremapcompression_layer,
        analyze_latencies_post,
        analyze_motions_layer,
        analyze_size_model
    )
```

But that's not valid syntax.
2023-08-02 19:55:00 +00:00
Victor Hugo Gomes 9f38dbd06e
[`flake8-pyi`] Implement PYI051 (#6215)
## Summary
Checks for the presence of redundant `Literal` types and builtin super
types in an union. See [original
source](2a86db8271/pyi.py (L1261)).

This implementation has a couple of differences from the original. The
first one is, we support the `complex` and `float` builtin types. The
second is, when reporting diagnostic for a `Literal` with multiple
members of the same type, we print the entire `Literal` while `flak8`
only prints the `Literal` with its first member.
For example:
```python
from typing import Literal

x: Literal[1, 2] | int
```  
Ruff will show `Literal[1, 2]` while flake8 only shows `Literal[1]`.

```shell
$ ruff crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: PYI051 `Literal[b"bar", b"foo"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: PYI051 `Literal[5]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: PYI051 `Literal[b"str_bytes"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: PYI051 `Literal[42]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:31: PYI051 `Literal[1J]` is redundant in an union with `complex`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:53: PYI051 `Literal[3.14]` is redundant in an union with `float`
Found 8 errors.
```

```shell
$ flake8 crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: Y051 "Literal[b'bar']" is redundant in a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: Y051 "Literal[5]" is redundant in a unionwith "int"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: Y051 "Literal[b'str_bytes']" is redundantin a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: Y051 "Literal[42]" is redundant in a union with "int"
```

While implementing this rule, I found a bug in the `is_unchecked_union`
check. This is the new check.


1ab86bad35/crates/ruff/src/checkers/ast/analyze/expression.rs (L85-L102)

The purpose of the check was to prevent rules from navigating through
nested `Union`s, as they already handle nested `Union`s. The way it was
implemented, this was not happening, the rules were getting executed
more than one time and sometimes were receiving expressions that were
not `Union`. For example, with the following code:
 ```python
  typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
 ```

The rules were receiving the expressions in the following order:
- `typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]`
     - `Literal[5]`
     - `typing.Union[Literal["foo"], str]]`

This was causing `PYI030` to report redundant information, for example:
 ```python
typing.Union[Literal[5], int, typing.Union[Literal["foo"],
Literal["bar"]]]
 ```
This is the `PYI030` output for this code:
```shell
PYI030 Multiple literal members in a union. Use a single literal, e.g. `Literal[5, "foo", "bar"]`
YI030 Multiple literal members in a union. Use a single literal, e.g.`Literal[5, "foo"]`
```

If I haven't misinterpreted the rule, that looks incorrect. I didn't
have the time to check the `PYI016` rule.

The last thing is, I couldn't find a reason for the "Why is this bad?"
section for `PYI051`.

Ref: #848 

## Test Plan

Snapshots and manual runs of flake8.
\
2023-08-02 15:37:40 -04:00
Victor Hugo Gomes 7c5791fb77
Fix formatting of `lambda` star arguments (#6257)
## Summary
Previously, the ruff formatter was removing the star argument of
`lambda` expressions when formatting.

Given the following code snippet
```python
lambda *a: ()
lambda **b: ()
```
it would be formatted to
```python
lambda: ()
lambda: ()
```

We fix this by checking for the presence of `args`, `vararg` or `kwarg`
in the `lambda` expression, before we were only checking for the
presence of `args`.

Fixes #5894

## Test Plan

Add new tests cases.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-02 19:31:20 +00:00
Harutaka Kawamura c362ea7fd4
Add `PT025` and `PT026` docs (#6264) 2023-08-02 19:00:03 +00:00
Harutaka Kawamura ec8fad5b02
Extend `UP032` to support implicitly concatenated strings (#6263) 2023-08-02 18:56:24 +00:00
Harutaka Kawamura bcc41ba062
Extend `UP032` to support repeated format fields (#6266) 2023-08-02 14:23:25 -04:00
Charlie Marsh 556abf4bd3
Avoid `PTH206` with `maxsplit` (#6283)
## Summary

Avoid suggesting `Path.parts` when a `maxsplit` is specified, since
these behavior differently.

## Test Plan

`cargo test`
2023-08-02 18:16:57 +00:00
Charlie Marsh 23b8fc4366
Move `includes_arg_name` onto `Parameters` (#6282)
## Summary

Like #6279, no reason for this to be a standalone method.
2023-08-02 18:05:26 +00:00
Charlie Marsh fd40864924
Move `find_keyword` helpers onto `Arguments` struct (#6280)
## Summary

Similar to #6279, moving some helpers onto the struct in the name of
reducing the number of random undiscoverable utilities we have in
`helpers.rs`.

Most of the churn is migrating rules to take `ast::ExprCall` instead of
the spread call arguments.

## Test Plan

`cargo test`
2023-08-02 13:54:48 -04:00
Charlie Marsh 041946fb64
Remove `CallArguments` abstraction (#6279)
## Summary

This PR removes a now-unnecessary abstraction from `helper.rs`
(`CallArguments`), in favor of adding methods to `Arguments` directly,
which helps with discoverability.
2023-08-02 13:25:43 -04:00
Charlie Marsh 8a0f844642
Box type params and arguments fields on the class definition node (#6275)
## Summary

This PR boxes the `TypeParams` and `Arguments` fields on the class
definition node. These fields are optional and often emitted, and given
that class definition is our largest enum variant, we pay the cost of
including them for every statement in the AST. Boxing these types
reduces the statement size by 40 bytes, which seems like a good tradeoff
given how infrequently these are accessed.

## Test Plan

Need to benchmark, but no behavior changes.
2023-08-02 16:47:06 +00:00
Charlie Marsh 8c40886f87
Use `Arguments` node to power `remove_argument` method (#6278)
## Summary

Internal refactor to take advantage of the new `Arguments` node, to
power our `remove_argument` fix action.

## Test Plan

`cargo test`
2023-08-02 12:38:43 -04:00
Charlie Marsh 4c53bfe896
Add formatter support for call and class definition `Arguments` (#6274)
## Summary

This PR leverages the `Arguments` AST node introduced in #6259 in the
formatter, which ensures that we correctly handle trailing comments in
calls, like:

```python
f(
  1,
  # comment
)

pass
```

(Previously, this was treated as a leading comment on `pass`.)

This also allows us to unify the argument handling across calls and
class definitions.

## Test Plan

A bunch of new fixture tests, plus improved Black compatibility.
2023-08-02 11:54:22 -04:00
Charlie Marsh b095b7204b
Add a `TypeParams` node to the AST (#6261)
## Summary

Similar to #6259, this PR adds a `TypeParams` node to the AST, to
capture the list of type parameters with their surrounding brackets.

If a statement lacks type parameters, the `type_params` field will be
`None`.
2023-08-02 14:12:45 +00:00
Charlie Marsh 981e64f82b
Introduce an `Arguments` AST node for function calls and class definitions (#6259)
## Summary

This PR adds a new `Arguments` AST node, which we can use for function
calls and class definitions.

The `Arguments` node spans from the left (open) to right (close)
parentheses inclusive.

In the case of classes, the `Arguments` is an option, to differentiate
between:

```python
# None
class C: ...

# Some, with empty vectors
class C(): ...
```

In this PR, we don't really leverage this change (except that a few
rules get much simpler, since we don't need to lex to find the start and
end ranges of the parentheses, e.g.,
`crates/ruff/src/rules/pyupgrade/rules/lru_cache_without_parameters.rs`,
`crates/ruff/src/rules/pyupgrade/rules/unnecessary_class_parentheses.rs`).

In future PRs, this will be especially helpful for the formatter, since
we can track comments enclosed on the node itself.

## Test Plan

`cargo test`
2023-08-02 10:01:13 -04:00
Ran Benita 0d62ad2480
Permit `ClassVar` and `Final` without subscript in RUF012 (#6273)
Fix #6267.
2023-08-02 12:58:44 +00:00
Harutaka Kawamura b4f224ecea
Fix links in docs (#6265)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Before:

<img width="1031" alt="Screen Shot 2023-08-02 at 15 57 10"
src="https://github.com/astral-sh/ruff/assets/17039389/171a21d5-01a5-4aa5-8079-4e7f8a59ade8">

After:

<img width="1031" alt="Screen Shot 2023-08-02 at 15 58 03"
src="https://github.com/astral-sh/ruff/assets/17039389/afd1b9b7-89e0-4e38-a4a6-e3255b62f021">


## Test Plan

<!-- How was it tested? -->

Manual inspection
2023-08-02 09:42:25 +02:00
Charlie Marsh 7842c82a0a
Preserve end-of-line comments on import-from statements (#6216)
## Summary

Ensures that we keep comments at the end-of-line in cases like:

```python
from foo import (  # comment
  bar,
)
```

Closes https://github.com/astral-sh/ruff/issues/6067.
2023-08-01 18:58:05 +00:00
Charlie Marsh 9c708d8fc1
Rename `Parameter#arg` and `ParameterWithDefault#def` fields (#6255)
## Summary

This PR renames...

- `Parameter#arg` to `Parameter#name`
- `ParameterWithDefault#def` to `ParameterWithDefault#parameter` (such
that `ParameterWithDefault` has a `default` and a `parameter`)

## Test Plan

`cargo test`
2023-08-01 14:28:34 -04:00
Charlie Marsh adc8bb7821
Rename `Arguments` to `Parameters` in the AST (#6253)
## Summary

This PR renames a few AST nodes for clarity:

- `Arguments` is now `Parameters`
- `Arg` is now `Parameter`
- `ArgWithDefault` is now `ParameterWithDefault`

For now, the attribute names that reference `Parameters` directly are
changed (e.g., on `StmtFunctionDef`), but the attributes on `Parameters`
itself are not (e.g., `vararg`). We may revisit that decision in the
future.

For context, the AST node formerly known as `Arguments` is used in
function definitions. Formally (outside of the Python context),
"arguments" typically refers to "the values passed to a function", while
"parameters" typically refers to "the variables used in a function
definition". E.g., if you Google "arguments vs parameters", you'll get
some explanation like:

> A parameter is a variable in a function definition. It is a
placeholder and hence does not have a concrete value. An argument is a
value passed during function invocation.

We're thus deviating from Python's nomenclature in favor of a scheme
that we find to be more precise.
2023-08-01 13:53:28 -04:00
Charlie Marsh a82eb9544c
Implement Black's rules around newlines before and after class docstrings (#6209)
## Summary

Black allows up to one blank line _before_ a class docstring, and
enforces one blank line _after_ a class docstring. This PR implements
that handling. The cases in
`crates/ruff_python_formatter/resources/test/fixtures/ruff/statement/class_definition.py`
match Black identically.
2023-08-01 13:33:01 -04:00
konsti 1df7e9831b
Replace `.map_or(false, $closure)` with `.is_some_and(closure)` (#6244)
**Summary**
[Option::is_some_and](https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.is_some_and)
and
[Result::is_ok_and](https://doc.rust-lang.org/std/result/enum.Result.html#method.is_ok_and)
are new methods is rust 1.70. I find them way more readable than
`.map_or(false, ...)`.

The changes are `s/.map_or(false,/.is_some_and(/g`, then manually
switching to `is_ok_and` where the value is a Result rather than an
Option.

**Test Plan** n/a^
2023-08-01 19:29:42 +02:00
Micha Reiser debfca3a11
Remove `Parse` trait (#6235) 2023-08-01 18:35:03 +02:00
Charlie Marsh 83fe103d6e
Allow generic tuple and list calls in __all__ (#6247)
## Summary

Allows, e.g., `__all__ = list[str]()`.

Closes https://github.com/astral-sh/ruff/issues/6226.
2023-08-01 12:01:48 -04:00
Charlie Marsh 928ab63a64
Add empty lines before nested functions and classes (#6206)
## Summary

This PR ensures that if a function or class is the first statement in a
nested suite that _isn't_ a function or class body, we insert a leading
newline.

For example, given:

```python
def f():
    if True:

        def register_type():
            pass
```

We _want_ to preserve the newline, whereas today, we remove it.

Note that this only applies when the function or class doesn't have any
leading comments.

Closes https://github.com/astral-sh/ruff/issues/6066.
2023-08-01 15:30:59 +00:00
Charlie Marsh 1a85953129
Don't require docstrings in `.pyi` files (#6239)
Closes https://github.com/astral-sh/ruff/issues/6224.
2023-08-01 10:02:57 -04:00
Charlie Marsh 743118ae9a
Bump version to 0.0.282 (#6241) 2023-08-01 13:21:33 +00:00
Charlie Marsh 0753017cf1
Revert "Expand scope of `quoted-annotation` rule (#5766)" (#6237)
This is causing some problems, so we'll just revert for now.

Closes https://github.com/astral-sh/ruff/issues/6189.
2023-08-01 09:03:02 -04:00
Charlie Marsh 29fb655e04
Fix `logger-objects` documentation (#6238)
Closes https://github.com/astral-sh/ruff/issues/6234.
2023-08-01 12:57:56 +00:00
Micha Reiser f45e8645d7
Remove unused parser modes
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the `Interactive` and `FunctionType` parser modes that are unused by ruff

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-08-01 13:10:07 +02:00
Micha Reiser 7c7231db2e
Remove unsupported `type_comment` field
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the `type_comment` field which our parser doesn't support.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-08-01 12:53:13 +02:00
Micha Reiser 4ad5903ef6
Delete type-ignore node
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the type ignore node from the AST because our parser doesn't support it, and just having it around is confusing.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo build`

<!-- How was it tested? -->
2023-08-01 12:34:50 +02:00
konsti c6986ac95d
Consistent `CommentPlacement` conversion signatures (#6231)
**Summary** Allow passing any node to `CommentPlacement::{leading,
trailing, dangling}` without manually converting. Conversely, Restrict
the comment to the only type we actually pass.

**Test Plan** No changes.
2023-08-01 12:01:17 +02:00
Micha Reiser ecfdd8d58b
Add static assertions to nodes (#6228) 2023-08-01 11:54:49 +02:00
David Szotten 07468f8be9
format ExprJoinedStr (#5932) 2023-08-01 08:26:30 +02:00
David Szotten ba990b676f
add `DebugText` for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
Harutaka Kawamura 44a8d1c644
Add `PT021`, `PT022` and `PT023` docs (#6143) 2023-08-01 00:41:54 -04:00
Charlie Marsh 88b984e885
Avoid detecting continuations at non-start-of-line (#6219)
## Summary

Previously, given:

```python
a = \
  5;
```

When detecting continuations starting at the offset of the `;`, we'd
flag the previous line as a continuation. We should only flag a
continuation if there isn't leading content prior to the offset.

Closes https://github.com/astral-sh/ruff/issues/6214
2023-08-01 00:20:29 -04:00
Charlie Marsh bf584c6d74
Remove use of `SmallVec` in `unnecessary-literal-union` (#6221)
I prefer to use this on an as-needed basis.
2023-08-01 04:03:58 +00:00
Konrad Listwan-Ciesielski 6ea3c178fd
Add DTZ002 documentation (#6146)
## Summary

Adds documentation for DTZ002. Related to
https://github.com/astral-sh/ruff/issues/2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-08-01 04:00:50 +00:00
Charlie Marsh 764d35667f
Avoid PERF401 false positive on list access in loop (#6220)
Closes https://github.com/astral-sh/ruff/issues/6210.
2023-08-01 03:56:53 +00:00
Charlie Marsh ff9ebbaa5f
Skip trivia when searching for named exception (#6218)
Closes https://github.com/astral-sh/ruff/issues/6213.
2023-08-01 03:42:30 +00:00
Micha Reiser 38b5726948
formatter: `WithNodeLevel` helper (#6212) 2023-07-31 21:22:17 +00:00
Charlie Marsh 615337a54d
Remove newline-insertion logic from `JoinNodesBuilder` (#6205)
## Summary

This PR moves the "insert empty lines" behavior out of
`JoinNodesBuilder` and into the `Suite` formatter. I find it a little
confusing that the logic is split between those two formatters right
now, and since this is _only_ used in that one place, IMO it is a bit
simpler to just inline it and use a single approach to tracking state
(right now, both are stateful).

The only other place this was used was for decorators. As a side effect,
we now remove blank lines in both of these cases, which is a known but
intentional deviation from Black (which preserves the empty line before
the comment in the first case):

```python
@foo

# Hello
@bar
def baz():
    pass

@foo

@bar
def baz():
    pass
```
2023-07-31 16:58:15 -04:00
Charlie Marsh 6ee5cb37c0
Reset model state when exiting deferred visitors (#6208)
## Summary

Very subtle bug related to the AST traversal. Given:

```python
from __future__ import annotations

from logging import getLogger

__all__ = ("getLogger",)


def foo() -> None:
    pass
```

We end up visiting the `-> None` annotation, then reusing the state
snapshot when we go to visit the `__all__` exports, so when we visit
`"getLogger"`, we think we're inside of a deferred type annotation.

This PR changes all the deferred visitors to snapshot and restore the
state, which is a lot safer -- that way, the visitors avoid modifying
the current visitor state. (Previously, they implicitly left the visitor
state set to the state of the _last_ thing they visited.)

Closes https://github.com/astral-sh/ruff/issues/6207.
2023-07-31 19:46:52 +00:00
konsti 0fddb31235
Use tracing for format_dev (#6177)
## Summary

[tracing](https://github.com/tokio-rs/tracing) is library for logging,
tracing and related features that has a large ecosystem. Using
[tracing-subscriber](https://docs.rs/tracing-subscriber) and
[tracing-indicatif](https://github.com/emersonford/tracing-indicatif),
we get a nice logging output that you can configure with `RUST_LOG`
(e.g. `RUST_LOG=debug`) and a live look into the formatter progress.

Default:
![Screenshot from 2023-07-30
13-59-53](https://github.com/astral-sh/ruff/assets/6826232/6432f835-9ff1-4771-955b-398e54c406dc)

`RUST_LOG=debug`:
![Screenshot from 2023-07-30
14-01-32](https://github.com/astral-sh/ruff/assets/6826232/5f2c87da-0867-4159-82e7-b5757eebb8eb)

It's easy to see in this output which files take a disproportionate
amount of time.

[Peek 2023-07-30
14-35.webm](https://github.com/astral-sh/ruff/assets/6826232/2c92db5c-1354-465b-a6bc-ddfb281d6f9d)

It opens up further integration with the tracing ecosystem,
[tracing-timing](https://docs.rs/tracing-timing/latest/tracing_timing/)
and [tokio-console](https://github.com/tokio-rs/console) can e.g. show
histograms and the json output allows us building better pipelines than
grepping a log file.

One caveat is using `parent: None` for the logging statements because
tracing subscriber does not allow deactivating the span without
reimplementing all the other log message formatting, too, and we don't
need span information, esp. since it would currently show the progress
bar span.

## Test Plan

n/a
2023-07-31 19:14:01 +00:00
konsti a7aa3caaae
Rename formatter_progress to formatter_ecosystem_checks (#6194)
Rename the `scripts/formatter_progress.sh` to
`formatter/formatter_ecosysytem_checks.sh` since it fits the actual task
better.
2023-07-31 18:33:12 +00:00
konsti e52b636da0
Log configuration in ruff_dev (#6193)
**Summary** This includes two changes:
 * Allow setting `-v` in `ruff_dev`, using the `ruff_cli` implementation
 * `debug!` which ruff configuration strategy was used

This is a byproduct of debugging #6187.

**Test Plan** n/a
2023-07-31 17:52:38 +00:00
konsti 9063f4524d
Fix formatting of trailing unescaped quotes in raw triple quoted strings (#6202)
**Summary** This prevents us from turning `r'''\""'''` into
`r"""\"""""`, which is invalid syntax.

This PR fixes CI, which is currently broken on main (in a way that still
passes on linter PRs and allows merging formatter PRs, but it's bad to
have a job be red). Once merged, i'll make the formatted ecosystem
checks a required check.

**Test Plan** Added a regression test.
2023-07-31 19:25:16 +02:00
Charlie Marsh dbd60b2cf5
Bump version to 0.0.281 (#6195) 2023-07-31 13:21:43 -04:00
Charlie Marsh 7eb2ba47cc
Add empty line after `import` block (#6200)
## Summary

Ensures that, given:

```python
import os
x = 1
```

We format like:

```python
import os

x = 1
```
2023-07-31 12:01:45 -04:00