Commit Graph

224 Commits

Author SHA1 Message Date
Charlie Marsh 1a65e544c5
Allow `flake8-type-checking` rules to automatically quote runtime-evaluated references (#6001)
## Summary

This allows us to fix usages like:

```python
from pandas import DataFrame

def baz() -> DataFrame:
    ...
```

By quoting the `DataFrame` in `-> DataFrame`. Without quotes, moving
`from pandas import DataFrame` into an `if TYPE_CHECKING:` block will
fail at runtime, since Python tries to evaluate the annotation to add it
to the function's `__annotations__`.

Unfortunately, this does require us to split our "annotation kind" flags
into three categories, rather than two:

- `typing-only`: The annotation is only evaluated at type-checking-time.
- `runtime-evaluated`: Python will evaluate the annotation at runtime
(like above) -- but we're willing to quote it.
- `runtime-required`: Python will evaluate the annotation at runtime
(like above), and some library (like Pydantic) needs it to be available
at runtime, so we _can't_ quote it.

This functionality is gated behind a setting
(`flake8-type-checking.quote-annotations`).

Closes https://github.com/astral-sh/ruff/issues/5559.
2023-12-13 03:12:38 +00:00
Charlie Marsh 4d2ee5bf98
Add named expression handling to `find_assigned_value` (#9109) 2023-12-12 20:07:33 -05:00
qdegraaf 8314c8bb05
[`typing`] Add `find_assigned_value` helper func to `typing.rs` to retrieve value of a given variable `id` (#8583)
## Summary

Adds `find_assigned_value` a function which gets the `&Expr` assigned to
a given `id` if one exists in the semantic model.

Open TODOs:

- [ ] Handle `binding.kind.is_unpacked_assignment()`: I am bit confused
by this one. The snippet from its documentation does not appear to be
counted as an unpacked assignment and the only ones I could find for
which that was true were invalid Python like:
```python
x, y = 1 
```
- [ ] How to handle AugAssign. Can we combine statements like:
```python
(a, b) = [(1, 2, 3), (4,)]
a += (6, 7)
```
to get the full value for a? Code currently just returns `None` for
these assign types

- [ ] Multi target assigns
```python
m_c = (m_d, m_e) = (0, 0)
trio.sleep(m_c)  # OK
trio.sleep(m_d)  # TRIO115
trio.sleep(m_e)  # TRIO115
```

## Test Plan

Used the function in two rules:

- `TRIO115`
- `PERF101`

Expanded both their fixtures for explicit multi target check
2023-12-13 00:24:47 +00:00
T-256 cb201bc4a5
`PIE804`: Prevent keyword arguments duplication (#8450) 2023-12-12 23:19:55 +00:00
Shantanu cb8eea64a8
[`pylint`] Add fix for `subprocess-run-without-check` (`PLW1510`) (#6708) 2023-12-12 05:08:17 +00:00
Tuomas Siipola a53d59f6bd
Support floating-point base in FURB163 (#9100)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Check floating-point numbers similarly to integers in FURB163. For
example, both `math.log(x, 10)` and `math.log(x, 10.0)` should be
changed to `math.log10(x)`.

## Test Plan

<!-- How was it tested? -->

Added couple of test cases.
2023-12-11 15:47:37 -05:00
Samuel Cormier-Iijima 1026ece946
E274: allow tab indentation before keyword (#9099)
## Summary

E274 currently flags any keyword at the start of a line indented with
tabs. This turns out to be due to a bug in `Whitespace::trailing` that
never considers any whitespace containing a tab as indentation.

## Test Plan

Added a simple test case.
2023-12-11 15:25:31 -05:00
Charlie Marsh f452bf8cad
Allow `matplotlib.use` calls to intersperse imports (#9094)
This PR allows `matplotlib.use` calls to intersperse imports without
triggering `E402`. This is a pragmatic choice as it's common to require
`matplotlib.use` calls prior to importing from within `matplotlib`
itself.

Closes https://github.com/astral-sh/ruff/issues/9091.
2023-12-11 17:06:25 +00:00
Simon Brugman 6e36dcfefe
[`refurb`] Implement `hashlib-digest-hex` (`FURB181`) (#9077)
## Summary

Implementation of  Refurb FURB181
Part of https://github.com/astral-sh/ruff/issues/1348

## Test Plan

Test cases from Refurb
2023-12-10 02:00:11 +00:00
asafamr-mm 6c2613b44e
Detect `unused-asyncio-dangling-task` (`RUF006`) on unused assignments (#9060)
## Summary

Fixes #8863 : Detect asyncio-dangling-task (RUF006) when discarding
return value

## Test Plan

added new two testcases, changed result of an old one that was made more
specific
2023-12-09 21:10:38 +00:00
Charlie Marsh cb8a2f5615
Add fix for comment-related whitespace rules (#9075)
Closes https://github.com/astral-sh/ruff/issues/9067.

Closes https://github.com/astral-sh/ruff/issues/9068.

Closes https://github.com/astral-sh/ruff/issues/8119.
2023-12-09 15:18:07 -05:00
Charlie Marsh 20e33bf514
Allow class names when `apps.get_model` is a non-string (#9065)
See:
https://github.com/astral-sh/ruff/issues/7675#issuecomment-1848206022
2023-12-08 22:59:05 -05:00
Dhruv Manilawala b7dd2b5941
Allow `EM` fixes even if `msg` variable is defined (#9059)
This PR updates the `EM` rules to generate the auto-fix even if the
`msg` variable is defined in the current scope.

As discussed in https://github.com/astral-sh/ruff/issues/9052.
2023-12-08 15:16:15 -06:00
Charlie Marsh e043bd46b5
Make `math-constant` rule more targeted (#9054)
## Summary

We now only flag `math.pi` if the value is in `[3.14, 3.15)`, and apply
similar rules to the other constants.

Closes https://github.com/astral-sh/ruff/issues/9049.
2023-12-08 12:42:18 -05:00
Charlie Marsh b021ede481
Allow `sys.path` modifications between imports (#9047)
## Summary

It's common to interleave a `sys.path` modification between imports at
the top of a file. This is a frequent cause of `# noqa: E402` false
positives, as seen in the ecosystem checks. This PR modifies E402 to
omit such modifications when determining the "import boundary".

(We could consider linting against `sys.path` modifications, but that
should be a separate rule.)

Closes: https://github.com/astral-sh/ruff/issues/5557.
2023-12-07 13:35:55 -05:00
Dhruv Manilawala cdac90ef68
New AST nodes for f-string elements (#8835)
Rebase of #6365 authored by @davidszotten.

## Summary

This PR updates the AST structure for an f-string elements.

The main **motivation** behind this change is to have a dedicated node
for the string part of an f-string. Previously, the existing
`ExprStringLiteral` node was used for this purpose which isn't exactly
correct. The `ExprStringLiteral` node should include the quotes as well
in the range but the f-string literal element doesn't include the quote
as it's a specific part within an f-string. For example,

```python
f"foo {x}"
# ^^^^
# This is the literal part of an f-string
```

The introduction of `FStringElement` enum is helpful which represent
either the literal part or the expression part of an f-string.

### Rule Updates

This means that there'll be two nodes representing a string depending on
the context. One for a normal string literal while the other is a string
literal within an f-string. The AST checker is updated to accommodate
this change. The rules which work on string literal are updated to check
on the literal part of f-string as well.

#### Notes

1. The `Expr::is_literal_expr` method would check for
`ExprStringLiteral` and return true if so. But now that we don't
represent the literal part of an f-string using that node, this improves
the method's behavior and confines to the actual expression. We do have
the `FStringElement::is_literal` method.
2. We avoid checking if we're in a f-string context before adding to
`string_type_definitions` because the f-string literal is now a
dedicated node and not part of `Expr`.
3. Annotations cannot use f-string so we avoid changing any rules which
work on annotation and checks for `ExprStringLiteral`.

## Test Plan

- All references of `Expr::StringLiteral` were checked to see if any of
the rules require updating to account for the f-string literal element
node.
- New test cases are added for rules which check against the literal
part of an f-string.
- Check the ecosystem results and ensure it remains unchanged.

## Performance

There's a performance penalty in the parser. The reason for this remains
unknown as it seems that the generated assembly code is now different
for the `__reduce154` function. The reduce function body is just popping
the `ParenthesizedExpr` on top of the stack and pushing it with the new
location.

- The size of `FStringElement` enum is the same as `Expr` which is what
it replaces in `FString::format_spec`
- The size of `FStringExpressionElement` is the same as
`ExprFormattedValue` which is what it replaces

I tried reducing the `Expr` enum from 80 bytes to 72 bytes but it hardly
resulted in any performance gain. The difference can be seen here:
- Original profile: https://share.firefox.dev/3Taa7ES
- Profile after boxing some node fields:
https://share.firefox.dev/3GsNXpD

### Backtracking

I tried backtracking the changes to see if any of the isolated change
produced this regression. The problem here is that the overall change is
so small that there's only a single checkpoint where I can backtrack and
that checkpoint results in the same regression. This checkpoint is to
revert using `Expr` to the `FString::format_spec` field. After this
point, the change would revert back to the original implementation.

## Review process

The review process is similar to #7927. The first set of commits update
the node structure, parser, and related AST files. Then, further commits
update the linter and formatter part to account for the AST change.

---------

Co-authored-by: David Szotten <davidszotten@gmail.com>
2023-12-07 10:28:05 -06:00
Charlie Marsh ebc7ac31cb
Avoid invalid combination of `force-sort-within-types` and `lines-between-types` (#9041)
Closes https://github.com/astral-sh/ruff/issues/8792.
2023-12-06 23:56:14 -05:00
Charlie Marsh 946b308197
Ensure that from-style imports are always ordered first in `__future__` (#9039)
Closes https://github.com/astral-sh/ruff/issues/8823.
2023-12-06 22:56:23 -05:00
Charlie Marsh acab5f3cf2
Enable `printf-string-formatting` fix with comments on right-hand side (#9037)
## Summary

This was added in https://github.com/astral-sh/ruff/pull/6364 (as a
follow-on to https://github.com/astral-sh/ruff/pull/6342), but I don't
think it applies in the same way, because we don't _remove_ the
right-hand side when converting from `%`-style formatting to `.format`
calls.

Closes https://github.com/astral-sh/ruff/issues/8107.
2023-12-06 22:43:21 -05:00
Dhruv Manilawala 9361e22fe9
Avoid `ANN2xx` autofix for abstract methods with empty body (#9034)
## Summary

This PR updates the `ANN201`, `ANN202`, `ANN205`, and `ANN206` rules to
not create a fix for the return type when it's an abstract method and
the function body is empty i.e., it only contains either a pass
statement, docstring or an ellipsis literal.

fixes: #9004

## Test Plan

Add the following test cases:
- Abstract method with pass statement
- Abstract method with docstring
- Abstract method with ellipsis literal
- Abstract method with possible return type
2023-12-06 20:47:36 -06:00
Charlie Marsh b918647927
Avoid removing parentheses on ctypes.WinError (#9027)
Re-resolves https://github.com/astral-sh/ruff/issues/6730.
2023-12-06 17:05:34 +00:00
Charlie Marsh 958702ded0
Respect trailing comma in unnecessary-dict-kwargs (#9015)
Closes https://github.com/astral-sh/ruff/issues/9014.
2023-12-05 21:30:29 +00:00
Philipp A b90027d037
[`pylint`] Implement `too-many-positional` (`PLR0917`) (#8995)
## Summary

Adds a rule that bans too many positional (i.e. not keyword-only)
parameters in function definitions.

Fixes https://github.com/astral-sh/ruff/issues/8946

Rule ID code taken from https://github.com/pylint-dev/pylint/pull/9278

## Test Plan
1. fixtures file checking multiple OKs/fails
2. parametrized test file
2023-12-04 18:03:09 +00:00
Charlie Marsh bfae1f1412
Convert over-indentation rule to use number of characters (#8983)
Closes https://github.com/astral-sh/ruff/issues/8978.
2023-12-03 20:45:30 +00:00
Charlie Marsh b358cbf398
Fix start >= end error in over-indentation (#8982)
Closes https://github.com/astral-sh/ruff/issues/8977.
2023-12-03 20:19:43 +00:00
Charlie Marsh 17c8817695
Avoid off-by-one error in stripping noqa following multi-byte char (#8979)
Closes https://github.com/astral-sh/ruff/issues/8976.
2023-12-03 11:01:58 -05:00
Charlie Marsh 1dda669f9a
Avoid syntax error via invalid ur string prefix (#8971)
## Summary

If a string has a Unicode prefix, we can't add the `r` prefix on top of
that -- we need to remove and replace it. (The Unicode prefix is
redundant anyway in Python 3.)

Closes https://github.com/astral-sh/ruff/issues/8967.
2023-12-02 18:37:49 +00:00
Tom Kuson 3fbabfe126
[`flake8-pyi`] Check PEP 695 type aliases for `snake-case-type-alias` and `t-suffixed-type-alias` (#8966)
## Summary

Check PEP 695 type alias definitions for `snake-case-type-alias`
(`PYI042`) and `t-suffixed-type-alias` (`PYI043`)

Related to #8771.

## Test Plan

`cargo test`
2023-12-02 13:26:43 -05:00
Charlie Marsh 20ab14e354
Avoid unnecessary index diagnostics when value is modified (#8970)
Closes https://github.com/astral-sh/ruff/issues/8969.
2023-12-02 18:17:17 +00:00
Charlie Marsh 22d8a989d4
Avoid underflow in `get_model` matching (#8965)
Closes https://github.com/astral-sh/ruff/issues/8962.
2023-12-02 13:56:57 +00:00
Charlie Marsh 277cd80175
Add erroneous for-loop test case for CFG (#8957) 2023-12-01 23:11:42 +00:00
Charlie Marsh 20a40771a5
Consider more wildcards in control flow graph matches (#8956) 2023-12-01 17:58:32 -05:00
qdegraaf 64c2535e28
[`pylint`] Add `add_argument` utility and autofix for `PLW1514` (#8928)
## Summary

- Adds `add_argument` similar to existing `remove_argument` utility to
safely add arguments to functions.
- Adds autofix for `PLW1514` as per specs requested in
https://github.com/astral-sh/ruff/issues/8883 as a test

## Test Plan

Checks on existing fixtures as well as additional test and fixture for
Python 3.9 and lower fix

## Issue Link

Closes: https://github.com/astral-sh/ruff/issues/8883
2023-12-01 18:23:56 +00:00
Charlie Marsh 5510a6131e
Ignore `@overload` and `@override` methods for too-many-arguments checks (#8954)
Closes https://github.com/astral-sh/ruff/issues/8945.
2023-12-01 18:22:53 +00:00
Charlie Marsh e5db72459e
Detect implicit returns in auto-return-types (#8952)
## Summary

Adds detection for branches without a `return` or `raise`, so that we
can properly `Optional` the return types. I'd like to remove this and
replace it with our code graph analysis from the `unreachable.rs` rule,
but it at least fixes the worst offenders.

Closes #8942.
2023-12-01 12:35:01 -05:00
Tom Kuson d66063bb33
[`flake8-pyi`] Check for kwarg and vararg `NoReturn` type annotations (#8948)
## Summary

Triggers `no-return-argument-annotation-in-stub` (`PYI050`) for vararg
and kwarg `NoReturn` type annotations.

Related to #8771.

## Test Plan

`cargo test`
2023-12-01 12:18:52 -05:00
Steve C cb1d3df085
[`pylint`] Implement `unnecessary-dict-index-lookup` (`PLR1733`) (#8036)
## Summary

Add
[R1733](https://pylint.readthedocs.io/en/latest/user_guide/messages/refactor/unnecessary-dict-index-lookup.html)
and autofix!

See #970 

## Test Plan

`cargo test` and manually
2023-12-01 05:09:50 +00:00
Charlie Marsh 46a174a22e
Use full arguments range for zero-sleep-call (#8936) 2023-12-01 03:09:18 +00:00
Charlie Marsh 912c39ce2a
Add support for `@functools.singledispatch` (#8934)
## Summary

When a function uses `@functools.singledispatch`, we need to treat the
first argument of any implementations as runtime-required.

Closes https://github.com/astral-sh/ruff/issues/6849.
2023-12-01 03:04:58 +00:00
Steve C 70febb1862
[pylint] - add `unnecessary-list-index-lookup` (`PLR1736`) + autofix (#7999)
## Summary

Add
[R1736](https://pylint.readthedocs.io/en/latest/user_guide/messages/refactor/unnecessary-list-index-lookup.html)
along with the autofix

See #970 

## Test Plan

`cargo test` and manually

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2023-11-30 17:45:12 -06:00
Steve C 4212b41796
[pylint] - implement R0202 and R0203 with autofixes (#8335)
## Summary

Implements
[`no-classmethod-decorator`/`R0202`](https://pylint.readthedocs.io/en/latest/user_guide/messages/refactor/no-classmethod-decorator.html)
and
[`no-staticmethod-decorator`/`R0203`](https://pylint.readthedocs.io/en/latest/user_guide/messages/refactor/no-staticmethod-decorator.html)
with autofixes.

They're similar enough that all code is reusable for both.

See: #970 

## Test Plan

`cargo test`
2023-11-30 16:18:09 -06:00
Steve C bbad4b4c93
Add autofix for `PYI030` (#7934)
## Summary

Part 2 of implementing the reverted autofix for `PYI030`

Also handles `typing.Union` and `typing_extensions.Literal` etc, uses
the first subscript name it finds for each offensive line.

## Test Plan

<!-- How was it tested? -->

`cargo test` and manually

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2023-11-30 22:16:57 +00:00
Charlie Marsh d674e7946d
Ignore underlines when determining docstring logical lines (#8929) 2023-11-30 14:27:04 -05:00
Charlie Marsh c324cb6202
[`pep8-naming`] Allow Django model loads in `non-lowercase-variable-in-function` (`N806`) (#8917)
## Summary

Allows assignments of the form, e.g., `Attachment =
apps.get_model("zerver", "Attachment")`, for better compatibility with
Django.

Closes https://github.com/astral-sh/ruff/issues/7675.

## Test Plan

`cargo test`
2023-11-29 20:43:40 -05:00
Charlie Marsh 6435e4e4aa
Enable auto-return-type involving `Optional` and `Union` annotations (#8885)
## Summary

Previously, this was only supported for Python 3.10 and later, since we
always use the PEP 604-style unions.
2023-11-28 18:35:55 -08:00
Dhruv Manilawala b28556d739
Update `E402` to work at cell level for notebooks (#8872)
## Summary

This PR updates the `E402` rule to work at cell level for Jupyter
notebooks. This is enabled only in preview to gather feedback.

The implementation basically resets the import boundary flag on the
semantic model when we encounter the first statement in a cell.

Another potential solution is to introduce `E403` rule that is
specifically for notebooks that works at cell level while `E402` will be
disabled for notebooks.

## Test Plan

Add a notebook with imports in multiple cells and verify that the rule
works as expected.

resolves: #8669
2023-11-29 00:32:35 +00:00
Charlie Marsh 5d554edace
Allow booleans in `@override` methods (#8882)
Closes #8867.
2023-11-28 13:42:31 -08:00
Charlie Marsh 412688826c
Avoid filtering out un-representable types in return annotation (#8881)
## Summary

Given `Union[Dict, None]` (in our internal representation), we were
filtering out `Dict` since we treat it as un-representable (i.e., we
can't convert it to an expression), returning just `None` as the type
annotation. We should require that all members of the union are
representable.

Closes https://github.com/astral-sh/ruff/issues/8879.
2023-11-28 21:10:42 +00:00
Joffrey Bluthé 578ddf1bb1
[`isort`] Add support for length-sort settings (#8841)
## Summary

Closes #1567.

Add both `length-sort` and `length-sort-straight` settings for isort.

Here are a few notable points:
- The length is determined using the
[`unicode_width`](https://crates.io/crates/unicode-width) crate, i.e. we
are talking about displayed length (this is explicitly mentioned in the
description of the setting)
- The dots are taken into account in the length to be compatible with
the original isort
- I had to reorder a few fields of the module key struct for it all to
make sense (notably the `force_to_top` field is now the first one)

## Test Plan

I added tests for the following cases:
- Basic tests for length-sort with ASCII characters only
- Tests with non-ASCII characters
- Tests with relative imports
- Tests for length-sort-straight
2023-11-28 06:00:37 +00:00
Charlie Marsh ed14fd9163
[`pydocstyle`] Avoid non-character breaks in `over-indentation` (`D208`) (#8866)
Closes https://github.com/astral-sh/ruff/issues/8844.
2023-11-27 21:47:35 -08:00