Commit Graph

3740 Commits

Author SHA1 Message Date
Guilherme Vasconcelos a8e50a7f40
[RUF008] Make it clearer that a mutable default in a dataclass is only valid if it is typed as a ClassVar (#10395)
## Summary

The previous documentation sounded as if typing a mutable default as a
`ClassVar` were optional. However, it is not, as not doing so causes a
`ValueError`. The snippet below was tested in Python's interactive
shell:

```
>>> from dataclasses import dataclass
>>> @dataclass
... class A:
...     mutable_default: list[int] = []
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.11/dataclasses.py", line 1230, in dataclass
    return wrap(cls)
           ^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 1220, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 958, in _process_class
    cls_fields.append(_get_field(cls, name, type, kw_only))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 815, in _get_field
    raise ValueError(f'mutable default {type(f.default)} for field '
ValueError: mutable default <class 'list'> for field mutable_default is not allowed: use default_factory
>>>
```

This behavior is also documented in Python's docs, see
[here](https://docs.python.org/3/library/dataclasses.html#mutable-default-values):

> [...] the
[dataclass()](https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass)
decorator will raise a
[ValueError](https://docs.python.org/3/library/exceptions.html#ValueError)
if it detects an unhashable default parameter. The assumption is that if
a value is unhashable, it is mutable. This is a partial solution, but it
does protect against many common errors.

And
[here](https://docs.python.org/3/library/dataclasses.html#class-variables)
it is documented why it works if it is typed as a `ClassVar`:

> One of the few places where
[dataclass()](https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass)
actually inspects the type of a field is to determine if a field is a
class variable as defined in [PEP
526](https://peps.python.org/pep-0526/). It does this by checking if the
type of the field is typing.ClassVar. If a field is a ClassVar, it is
excluded from consideration as a field and is ignored by the dataclass
mechanisms. Such ClassVar pseudo-fields are not returned by the
module-level
[fields()](https://docs.python.org/3/library/dataclasses.html#dataclasses.fields)
function.

In this PR I have changed the documentation to make it a little bit
clearer that not using `ClassVar` makes the code invalid.
2024-03-14 23:18:03 +00:00
Hoël Bagard e944c16c46
[`pycodestyle`] Do not ignore lines before the first logical line in blank lines rules (#10382)
## Summary

Ignoring all lines until the first logical line does not match the
behavior from pycodestyle. This PR therefore removes the `if
state.is_not_first_logical_line` skipping the line check before the
first logical line, and applies it only to `E302`.

For example, in the snippet below a rule violation should be detected on
the second comment and on the import.

```python
# first comment




# second comment




import foo
```

Fixes #10374

## Test Plan

Add test cases, update the snapshots and verify the ecosystem check output
2024-03-14 14:05:24 +05:30
Dhruv Manilawala 5f40371ffc
Use `ExprFString` for `StringLike::FString` variant (#10311)
## Summary

This PR updates the `StringLike::FString` variant to use `ExprFString`
instead of `FStringLiteralElement`.

For context, the reason it used `FStringLiteralElement` is that the node
is actually the string part of an f-string ("foo" in `f"foo{x}"`). But,
this is inconsistent with other variants where the captured value is the
_entire_ string.

This is also problematic w.r.t. implicitly concatenated strings. Any
rules which work with `StringLike::FString` doesn't account for the
string part in an implicitly concatenated f-strings. For example, we
don't flag confusable character in the first part of `"𝐁ad" f"𝐁ad
string"`, but only the second part
(https://play.ruff.rs/16071c4c-a1dd-4920-b56f-e2ce2f69c843).

### Update `PYI053`

_This is included in this PR because otherwise it requires a temporary
workaround to be compatible with the old logic._

This PR also updates the `PYI053` (`string-or-bytes-too-long`) rule for
f-string to consider _all_ the visible characters in a f-string,
including the ones which are implicitly concatenated. This is consistent
with implicitly concatenated strings and bytes.

For example,

```python
def foo(
	# We count all the characters here
    arg1: str = '51 character ' 'stringgggggggggggggggggggggggggggggggg',
	# But not here because of the `{x}` replacement field which _breaks_ them up into two chunks
    arg2: str = f'51 character {x} stringgggggggggggggggggggggggggggggggggggggggggggg',
) -> None: ...
```

This PR fixes it to consider all _visible_ characters inside an f-string
which includes expressions as well.

fixes: #10310 
fixes: #10307 

## Test Plan

Add new test cases and update the snapshots.

## Review

To facilitate the review process, the change have been split into two
commits: one which has the code change while the other has the test
cases and updated snapshots.
2024-03-14 13:30:22 +05:30
boolean f7802ad5de
[`pylint`] Extend docs and test in `invalid-str-return-type` (`E307`) (#10400)
## Summary

Added some docs, and a little of test cases in
`invalid-str-return-type`, mentioned in
https://github.com/astral-sh/ruff/pull/10377#pullrequestreview-1934295027

## Test Plan

On `invalid_return_type_str.py`.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-03-14 04:38:30 +00:00
Jane Lewis e832327a56
Require --preview for `ruff server` (#10368)
## Summary

Fixes #10367.

While the server is still in an unstable state, requiring a `--preview`
flag would be a good way to indicate this to end users.
2024-03-13 23:52:44 +00:00
Charlie Marsh 324390607c
[`pylint`] Include builtin warnings in useless-exception-statement (`PLW0133`) (#10394)
## Summary

Closes https://github.com/astral-sh/ruff/issues/10392.
2024-03-13 15:26:11 -04:00
Tri Ho 4db5c29f19
Indicated Successful Check (#8631)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Adds a successful check message after no errors were found 
Implements #8553 

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

Ran a check on a test file with `cargo run -p ruff_cli -- check test.py
--no-cache` and outputted as expected.

Ran the same check with `cargo run -p ruff_cli -- check test.py
--no-cache --silent` and the command was gone as expected.

<!-- How was it tested? -->

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-03-13 19:07:11 +00:00
Zanie Blue 7b3ee2daff
Remove `F401` fix for `__init__` imports by default and allow opt-in to unsafe fix (#10365)
Re-implementation of https://github.com/astral-sh/ruff/pull/5845 but
instead of deprecating the option I toggle the default. Now users can
_opt-in_ via the setting which will give them an unsafe fix to remove
the import. Otherwise, we raise violations but do not offer a fix. The
setting is a bit of a misnomer in either case, maybe we'll want to
remove it still someday.

As discussed there, I think the safe fix should be to import it as an
alias. I'm not sure. We need support for offering multiple fixes for
ideal behavior though? I think we should remove the fix entirely and
consider it separately.

Closes https://github.com/astral-sh/ruff/issues/5697
Supersedes https://github.com/astral-sh/ruff/pull/5845

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-03-13 12:58:25 -05:00
Alex Waygood c2e15f38ee
Unify enums used for internal representation of quoting style (#10383) 2024-03-13 17:19:17 +00:00
Charlie Marsh d59433b12e
Avoid removing shadowed imports that point to different symbols (#10387)
This ensures that we don't have incorrect, automated fixes for shadowed
names that actually point to different imports.

See: https://github.com/astral-sh/ruff/issues/10384.
2024-03-13 15:44:28 +00:00
boolean c269c1a706
[`pylint`] Implement `invalid-bool-return-type` (`E304`) (#10377)
## Summary

Implement `E304` in the issue #970. Throws an error when the returning value
of `__bool__` method is not boolean.

Reference: https://pylint.readthedocs.io/en/stable/user_guide/messages/error/invalid-bool-returned.html

## Test Plan

Add test cases and run `cargo test`
2024-03-13 19:43:45 +05:30
Dhruv Manilawala 32d6f84e3d
Add methods to iter over f-string elements (#10309)
## Summary

This PR adds methods on `FString` to iterate over the two different kind
of elements it can have - literals and expressions. This is similar to
the methods we have on `ExprFString`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-03-13 08:46:55 +00:00
Auguste Lalande 93d582d734
Avoid `TRIO115` if the argument is a variable (#10376)
## Summary

Fix "TRIO115 false positive with with sleep(var) where var starts as 0"
#9935 based on the discussion in the issue.

## Test Plan

Issue code added to fixture
2024-03-13 13:09:18 +05:30
Auguste Lalande 3ed707f245
Spellcheck & grammar (#10375)
## Summary

I used `codespell` and `gramma` to identify mispellings and grammar
errors throughout the codebase and fixed them. I tried not to make any
controversial changes, but feel free to revert as you see fit.
2024-03-13 02:34:23 +00:00
Charlie Marsh c56fb6e15a
Sort hash maps in `Settings` display (#10370)
## Summary

We had a report of a test failure on a specific architecture, and
looking into it, I think the test assumes that the hash keys are
iterated in a specific order. This PR thus adds a variant to our
settings display macro specifically for maps and sets. Like `CacheKey`,
it sorts the keys when printing.

Closes https://github.com/astral-sh/ruff/issues/10359.
2024-03-12 15:59:38 -04:00
Charlie Marsh dbf82233b8
Gate f-string struct size test for Rustc < 1.76 (#10371)
Closes https://github.com/astral-sh/ruff/issues/10319.
2024-03-12 15:46:36 -04:00
Zanie Blue 87afe36c87
Add test case for `F401` in `__init__` files (#10364)
In preparation for https://github.com/astral-sh/ruff/pull/5845
2024-03-12 13:30:17 -05:00
Alex Waygood 704fefc7ab
F821: Fix false negatives in `.py` files when `from __future__ import annotations` is active (#10362) 2024-03-12 17:07:44 +00:00
Auguste Lalande dacec7377c
Fix Indexer fails to identify continuation preceded by newline #10351 (#10354)
## Summary

Fixes #10351

It seems the bug was caused by this section of code

b669306c87/crates/ruff_python_index/src/indexer.rs (L55-L58)

It's true that newline tokens cannot be immediately followed by line
continuations, but only outside parentheses. e.g. the exception
```
(
    1
    \
    + 2)
```

But why was this check put there in the first place? Is it guarding
against something else?



## Test Plan

New test was added to indexer
2024-03-12 00:35:41 -04:00
Anuraag (Rag) Agrawal b669306c87
Fix typo in docs snippt -> snippet (#10353) 2024-03-11 22:33:40 -04:00
Auguste Lalande b117f33075
[`pycodestyle`] Implement `blank-line-at-end-of-file` (`W391`) (#10243)
## Summary

Implements the [blank line at end of
file](https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes)
rule (W391) from pycodestyle. Renamed to TooManyNewlinesAtEndOfFile for
clarity.

## Test Plan

New fixtures have been added

Part of #2402
2024-03-11 22:07:59 -04:00
Auguste Lalande c746912b9e
[`pycodestyle`] Implement `redundant-backslash` (`E502`) (#10292)
## Summary

Implements the
[redundant-backslash](https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes)
rule (E502) from pycodestyle.

## Test Plan

New fixture has been added

Part of #2402
2024-03-11 21:15:06 -04:00
Mathieu Kniewallner fc7139d9a5
[`flake8-bandit`]: Implement `S610` rule (#10316)
Part of https://github.com/astral-sh/ruff/issues/1646.

## Summary

Implement `S610` rule from `flake8-bandit`. 

Upstream references:
- Implementation:
https://github.com/PyCQA/bandit/blob/1.7.8/bandit/plugins/django_sql_injection.py#L20-L97
- Test cases:
https://github.com/PyCQA/bandit/blob/1.7.8/examples/django_sql_injection_extra.py
- Test assertion:
https://github.com/PyCQA/bandit/blob/1.7.8/tests/functional/test_functional.py#L517-L524

The implementation in `bandit` targets additional arguments (`params`,
`order_by` and `select_params`) but doesn't seem to do anything with
them in the end, so I did not include them in the implementation.

Note that this rule could be prone to false positives, as ideally we
would want to check if `extra()` is tied to a [Django
queryset](https://docs.djangoproject.com/en/5.0/ref/models/querysets/),
but AFAIK Ruff is not able to resolve classes outside of the current
module.

## Test Plan

Snapshot tests
2024-03-11 20:22:02 -04:00
Charlie Marsh f8f56186b3
[`pylint`] Avoid false-positive slot non-assignment for `__dict__` (`PLE0237`) (#10348)
Closes https://github.com/astral-sh/ruff/issues/10306.
2024-03-11 18:48:56 -04:00
Charlie Marsh 02fc521369
Wrap expressions in parentheses when negating (#10346)
## Summary

When negating an expression like `a or b`, we need to wrap it in
parentheses, e.g., `not (a or b)` instead of `not a or b`, due to
operator precedence.

Closes https://github.com/astral-sh/ruff/issues/10335.

## Test Plan

`cargo test`
2024-03-11 18:20:55 -04:00
Alex Waygood 4b0666919b
F821, F822: fix false positive for `.pyi` files; add more test coverage for `.pyi` files (#10341)
This PR fixes the following false positive in a `.pyi` stub file:

```py
x: int
y = x  # F821 currently emitted here, but shouldn't be in a stub file
```

In a `.py` file, this is invalid regardless of whether `from __future__ import annotations` is enabled or not. In a `.pyi` stub file, however, it's always valid, as an annotation counts as a binding in a stub file even if no value is assigned to the variable.

I also added more test coverage for `.pyi` stub files in various edge cases where ruff's behaviour is currently correct, but where `.pyi` stub files do slightly different things to `.py` files.
2024-03-11 22:15:24 +00:00
Hoël Bagard 8d73866f70
[`pycodestyle`] Do not trigger `E225` and `E275` when the next token is a ')' (#10315)
## Summary

Fixes #10295.

`E225` (`Missing whitespace around operator`) and `E275` (`Missing
whitespace after keyword`) try to add a white space even when the next
character is a `)` (which is a syntax error in most cases, the
exceptions already being handled). This causes `E202` (`Whitespace
before close bracket`) to try to remove the added whitespace, resulting
in an infinite loop when `E225`/`E275` re-add it.
This PR adds an exception in `E225` and `E275` to not trigger in case
the next token is a `)`. It is a bit simplistic, but it solves the
example given in the issue without introducing a change in behavior
(according to the fixtures).

## Test Plan

`cargo test` and the `ruff-ecosystem` check were used to check that the
PR's changes do not have side-effects.
A new fixture was added to check that running the 3 rules on the example
given in the issue does not cause ruff to fail to converge.
2024-03-11 21:23:18 +00:00
Mathieu Kniewallner bc693ea13a
[`flake8-bandit`] Implement upstream updates for `S311`, `S324` and `S605` (#10313)
## Summary

Pick up updates made in latest
[releases](https://github.com/PyCQA/bandit/releases) of `bandit`:
- `S311`: https://github.com/PyCQA/bandit/pull/940 and
https://github.com/PyCQA/bandit/pull/1096
- `S324`: https://github.com/PyCQA/bandit/pull/1018
- `S605`: https://github.com/PyCQA/bandit/pull/1116

## Test Plan

Snapshot tests
2024-03-11 21:07:58 +00:00
Jane Lewis 0c84fbb6db
`ruff server` - A new built-in LSP for Ruff, written in Rust (#10158)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR introduces the `ruff_server` crate and a new `ruff server`
command. `ruff_server` is a re-implementation of
[`ruff-lsp`](https://github.com/astral-sh/ruff-lsp), written entirely in
Rust. It brings significant performance improvements, much tighter
integration with Ruff, a foundation for supporting entirely new language
server features, and more!

This PR is an early version of `ruff_lsp` that we're calling the
**pre-release** version. Anyone is more than welcome to use it and
submit bug reports for any issues they encounter - we'll have some
documentation on how to set it up with a few common editors, and we'll
also provide a pre-release VSCode extension for those interested.

This pre-release version supports:
- **Diagnostics for `.py` files**
- **Quick fixes**
- **Full-file formatting**
- **Range formatting**
- **Multiple workspace folders**
- **Automatic linter/formatter configuration** - taken from any
`pyproject.toml` files in the workspace.

Many thanks to @MichaReiser for his [proof-of-concept
work](https://github.com/astral-sh/ruff/pull/7262), which was important
groundwork for making this PR possible.

## Architectural Decisions

I've made an executive choice to go with `lsp-server` as a base
framework for the LSP, in favor of `tower-lsp`. There were several
reasons for this:

1. I would like to avoid `async` in our implementation. LSPs are mostly
computationally bound rather than I/O bound, and `async` adds a lot of
complexity to the API, while also making harder to reason about
execution order. This leads into the second reason, which is...
2. Any handlers that mutate state should be blocking and run in the
event loop, and the state should be lock-free. This is the approach that
`rust-analyzer` uses (also with the `lsp-server`/`lsp-types` crates as a
framework), and it gives us assurances about data mutation and execution
order. `tower-lsp` doesn't support this, which has caused some
[issues](https://github.com/ebkalderon/tower-lsp/issues/284) around data
races and out-of-order handler execution.
3. In general, I think it makes sense to have tight control over
scheduling and the specifics of our implementation, in exchange for a
slightly higher up-front cost of writing it ourselves. We'll be able to
fine-tune it to our needs and support future LSP features without
depending on an upstream maintainer.

## Test Plan

The pre-release of `ruff_server` will have snapshot tests for common
document editing scenarios. An expanded test suite is on the roadmap for
future version of `ruff_server`.
2024-03-08 20:57:23 -08:00
Charlie Marsh a892fc755d
Bump version to v0.3.2 (#10304) 2024-03-09 00:24:22 +00:00
Gautier Moin a067d87ccc
Fix incorrect `Parameter` range for `*args` and `**kwargs` (#10283)
## Summary

Fix #10282 

This PR updates the Python grammar to include the `*` character in
`*args` `**kwargs` in the range of the `Parameter`
```
def f(*args, **kwargs): pass
#      ~~~~    ~~~~~~    <-- range before the PR
#     ^^^^^  ^^^^^^^^    <-- range after
```

The invalid syntax `def f(*, **kwargs): ...` is also now correctly
reported.

## Test Plan

Test cases were added to `function.rs`.
2024-03-08 18:57:49 -05:00
Micha Reiser b64f2ea401
Formatter: Improve single-with item formatting for Python 3.8 or older (#10276)
## Summary

This PR changes how we format `with` statements with a single with item
for Python 3.8 or older. This change is not compatible with Black.

This is how we format a single-item with statement today 

```python
def run(data_path, model_uri):
    with pyspark.sql.SparkSession.builder.config(
        key="spark.python.worker.reuse", value=True
    ).config(key="spark.ui.enabled", value=False).master(
        "local-cluster[2, 1, 1024]"
    ).getOrCreate():
        # ignore spark log output
        spark.sparkContext.setLogLevel("OFF")
        print(score_model(spark, data_path, model_uri))
```

This is different than how we would format the same expression if it is
inside any other clause header (`while`, `if`, ...):

```python
def run(data_path, model_uri):
    while (
        pyspark.sql.SparkSession.builder.config(
            key="spark.python.worker.reuse", value=True
        )
        .config(key="spark.ui.enabled", value=False)
        .master("local-cluster[2, 1, 1024]")
        .getOrCreate()
    ):
        # ignore spark log output
        spark.sparkContext.setLogLevel("OFF")
        print(score_model(spark, data_path, model_uri))

```

Which seems inconsistent to me. 

This PR changes the formatting of the single-item with Python 3.8 or
older to match that of other clause headers.

```python
def run(data_path, model_uri):
    with (
        pyspark.sql.SparkSession.builder.config(
            key="spark.python.worker.reuse", value=True
        )
        .config(key="spark.ui.enabled", value=False)
        .master("local-cluster[2, 1, 1024]")
        .getOrCreate()
    ):
        # ignore spark log output
        spark.sparkContext.setLogLevel("OFF")
        print(score_model(spark, data_path, model_uri))
```

According to our versioning policy, this style change is gated behind a
preview flag.

## Test Plan

See added tests.

Added
2024-03-08 23:56:02 +00:00
Micha Reiser 4bce801065
Fix unstable with-items formatting (#10274)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/10267

The issue with the current formatting is that the formatter flips
between the `SingleParenthesizedContextManager` and
`ParenthesizeIfExpands` or `SingleWithTarget` because the layouts use
incompatible formatting ( `SingleParenthesizedContextManager`:
`maybe_parenthesize_expression(context)` vs `ParenthesizeIfExpands`:
`parenthesize_if_expands(item)`, `SingleWithTarget`:
`optional_parentheses(item)`.

The fix is to ensure that the layouts between which the formatter flips
when adding or removing parentheses are the same. I do this by
introducing a new `SingleWithoutTarget` layout that uses the same
formatting as `SingleParenthesizedContextManager` if it has no target
and prefer `SingleWithoutTarget` over using `ParenthesizeIfExpands` or
`SingleWithTarget`.

## Formatting change

The downside is that we now use `maybe_parenthesize_expression` over
`parenthesize_if_expands` for expressions where
`can_omit_optional_parentheses` returns `false`. This can lead to stable
formatting changes. I only found one formatting change in our ecosystem
check and, unfortunately, this is necessary to fix the instability (and
instability fixes are okay to have as part of minor changes according to
our versioning policy)

The benefit of the change is that `with` items with a single context
manager and without a target are now formatted identically to how the
same expression would be formatted in other clause headers.

## Test Plan

I ran the ecosystem check locally
2024-03-08 23:48:47 +00:00
Micha Reiser a56d42f183
Refactor with statement formatting to have explicit layouts (#10296)
## Summary

This PR refactors the with item formatting to use more explicit layouts
to make it easier to understand the different formatting cases.

The benefit of the explicit layout is that it makes it easier to reasons
about layout transition between format runs. For example, today it's
possible that `SingleWithTarget` or `ParenthesizeIfExpands` add
parentheses around the with items for `with aaaaaaaaaa + bbbbbbbbbbbb:
pass`, resulting in `with (aaaaaaaaaa + bbbbbbbbbbbb): pass`. The
problem with this is that the next formatting pass uses the
`SingleParenthesizedContextExpression` layout that uses
`maybe_parenthesize_expression` which is different from
`parenthesize_if_expands(&expr)` or `optional_parentheses(&expr)`.

## Test Plan

`cargo test`

I ran the ecosystem checks locally and there are no changes.
2024-03-08 18:40:39 -05:00
Alex Waygood 1d97f27335
Start tracking quoting style in the AST (#10298)
This PR modifies our AST so that nodes for string literals, bytes literals and f-strings all retain the following information:
- The quoting style used (double or single quotes)
- Whether the string is triple-quoted or not
- Whether the string is raw or not

This PR is a followup to #10256. Like with that PR, this PR does not, in itself, fix any bugs. However, it means that we will have the necessary information to preserve quoting style and rawness of strings in the `ExprGenerator` in a followup PR, which will allow us to provide a fix for https://github.com/astral-sh/ruff/issues/7799.

The information is recorded on the AST nodes using a bitflag field on each node, similarly to how we recorded the information on `Tok::String`, `Tok::FStringStart` and `Tok::FStringMiddle` tokens in #10298. Rather than reusing the bitflag I used for the tokens, however, I decided to create a custom bitflag for each AST node.

Using different bitflags for each node allows us to make invalid states unrepresentable: it is valid to set a `u` prefix on a string literal, but not on a bytes literal or an f-string. It also allows us to have better debug representations for each AST node modified in this PR.
2024-03-08 19:11:47 +00:00
Micha Reiser 965adbed4b
Fix trailing kwargs end of line comment after slash (#10297)
## Summary

Fixes the handling end of line comments that belong to `**kwargs` when
the `**kwargs` come after a slash.

The issue was that we missed to include the `**kwargs` start position
when determining the start of the next node coming after the `/`.

Fixes https://github.com/astral-sh/ruff/issues/10281

## Test Plan

Added test
2024-03-08 14:45:26 +00:00
Alex Waygood c504d7ab11
Track quoting style in the tokenizer (#10256) 2024-03-08 08:40:06 +00:00
Tom Kuson 72c9f7e4c9
Include actual conditions in E712 diagnostics (#10254)
## Summary

Changes the generic recommendation to replace

```python
if foo == True: ...
```

with `if cond:` to `if foo:`.

Still uses a generic message for compound comparisons as a specific
message starts to become confusing. For example,

```python
if foo == True != False: ...
```

produces two recommendations, one of which would recommend `if True:`,
which is confusing.

Resolves [recommendation in a previous
PR](https://github.com/astral-sh/ruff/pull/8613/files#r1514915070).

## Test Plan

`cargo nextest run`
2024-03-08 01:20:56 +00:00
Charlie Marsh 57be3fce90
Treat `typing.Annotated` subscripts as type definitions (#10285)
## Summary

I think this code has existed since the start of `typing.Annotated`
support (https://github.com/astral-sh/ruff/pull/333), and was then
overlooked over a series of refactors.

Closes https://github.com/astral-sh/ruff/issues/10279.
2024-03-07 19:51:54 -05:00
Samuel Cormier-Iijima 7b4a73d421
Fix E203 false positive for slices in format strings (#10280)
## Summary

The code later in this file that checks for slices relies on the stack
of brackets to determine the position. I'm not sure why format strings
were being excluded from this, but the tests still pass with these match
guards removed.

Closes #10278

## Test Plan

~Still needs a test.~ Test case added for this example.
2024-03-07 17:09:05 -05:00
Charlie Marsh 91af5a4b74
[`pyupgrade`] Allow fixes for f-string rule regardless of line length (`UP032`) (#10263)
## Summary

This is a follow-up to https://github.com/astral-sh/ruff/pull/10238 to
offer fixes for the f-string rule regardless of the line length of the
resulting fix. To quote Alex in the linked PR:

> Yes, from the user's perspective I'd rather have a fix that may lead
to line length issues than have to fix them myself :-) Cleaning up line
lengths is easier than changing from `"".format()` to `f""`

I agree with this position, which is that if we're going to offer a
diagnostic, we should really be offering the user the ability to fix it
-- otherwise, we're just inconveniencing them.
2024-03-07 08:59:29 -05:00
Charlie Marsh 461cdad53a
Avoid repeating function calls in f-string conversions (#10265)
## Summary

Given a format string like `"{x} {x}".format(x=foo())`, we should avoid
converting to an f-string, since doing so would require repeating the
function call (`f"{foo()} {foo()}"`), which could introduce side
effects.

Closes https://github.com/astral-sh/ruff/issues/10258.
2024-03-06 23:33:19 -05:00
Charlie Marsh ea79f616bc
Bump version to v0.3.1 (#10252) 2024-03-06 19:59:04 +00:00
Tom Kuson f999b1b617
Tweak `E712` docs (#8613)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-03-06 17:54:29 +00:00
Eero Vaher cbd927f346
Make rule PT012 example clearer (#10248) 2024-03-06 15:47:14 +00:00
Micha Reiser 6159a8e532
[`pyupgrade`] Generate diagnostic for all valid f-string conversions regardless of line-length (`UP032`) (#10238)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/10235

This PR changes `UP032` to flag all `"".format` calls that can
technically be rewritten to an f-string, even if rewritting it to an
fstring, at least automatically, exceeds the line length (or increases
the amount by which it goes over the line length).

I looked at the Git history to understand whether the check prevents
some false positives (reported by an issue), but i haven't found a
compelling reason to limit the rule to only flag format calls that stay
in the line length limit:

* https://github.com/astral-sh/ruff/pull/7818 Changed the heuristic to
determine if the fix fits to address
https://github.com/astral-sh/ruff/discussions/7810
* https://github.com/astral-sh/ruff/pull/1905 first version of the rule 


I did take a look at pyupgrade and couldn't find a similar check, at
least not in the rule code (maybe it's checked somewhere else?)
https://github.com/asottile/pyupgrade/blob/main/pyupgrade/_plugins/fstrings.py


## Breaking Change?

This could be seen as a breaking change according to ruff's [versioning
policy](https://docs.astral.sh/ruff/versioning/):

> The behavior of a stable rule is changed
  
  * The scope of a stable rule is significantly increased
  * The intent of the rule changes
* Does not include bug fixes that follow the original intent of the rule

It does increase the scope of the rule, but it is in the original intent
of the rule (so it's not).

## Test Plan

See changed test output
2024-03-06 09:58:20 +01:00
Micha Reiser 8ea5b08700
refactor: Use `QualifiedName` for `Imported::call_path` (#10214)
## Summary

When you try to remove an internal representation leaking into another
type and end up rewriting a simple version of `smallvec`.

The goal of this PR is to replace the `Box<[&'a str]>` with
`Box<QualifiedName>` to avoid that the internal `QualifiedName`
representation leaks (and it gives us a nicer API too). However, doing
this when `QualifiedName` uses `SmallVec` internally gives us all sort
of funny lifetime errors. I was lost but @BurntSushi came to rescue me.
He figured out that `smallvec` has a variance problem which is already
tracked in https://github.com/servo/rust-smallvec/issues/146

To fix the variants problem, I could use the smallvec-2-alpha-4 or
implement our own smallvec. I went with implementing our own small vec
for this specific problem. It obviously isn't as sophisticated as
smallvec (only uses safe code), e.g. it doesn't perform any size
optimizations, but it does its job.

Other changes:

* Removed `Imported::qualified_name` (the version that returns a
`String`). This can be replaced by calling `ToString` on the qualified
name.
* Renamed `Imported::call_path` to `qualified_name` and changed its
return type to `&QualifiedName`.
* Renamed `QualifiedName::imported` to `user_defined` which is the more
common term when talking about builtins vs the rest/user defined
functions.


## Test plan

`cargo test`
2024-03-06 09:55:59 +01:00
Micha Reiser af6ea2f5e4
[`pycodestyle`]: Make blank lines in typing stub files optional (`E3*`) (#10098)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/10039

The [recommendation for typing stub
files](https://typing.readthedocs.io/en/latest/source/stubs.html#blank-lines)
is to use **one** blank line to group related definitions and
otherwise omit blank lines. 

The newly added blank line rules (`E3*`) didn't account for typing stub
files and enforced two empty lines at the top level and one empty line
otherwise, making it impossible to group related definitions.

This PR implements the `E3*` rules to:

* Not enforce blank lines. The use of blank lines in typing definitions
is entirely up to the user.
* Allow at most one empty line, including between top level statements. 

## Test Plan

Added unit tests (It may look odd that many snapshots are empty but the
point is that the rule should no longer emit diagnostics)
2024-03-05 12:48:50 +01:00
Micha Reiser 46ab9dec18
[`pycodestyle`] Respect `isort` settings in blank line rules (`E3*`) (#10096)
## Summary

This PR changes the `E3*` rules to respect the `isort`
`lines-after-imports` and `lines-between-types` settings. Specifically,
the following rules required changing

* `TooManyBlannkLines` : Respects both settings.
* `BlankLinesTopLevel`: Respects `lines-after-imports`. Doesn't need to
respect `lines-between-types` because it only applies to classes and
functions


The downside of this approach is that `isort` and the blank line rules
emit a diagnostic when there are too many blank lines. The fixes aren't
identical, the blank line is less opinionated, but blank lines accepts
the fix of `isort`.

<details>
	<summary>Outdated approach</summary>
Fixes
https://github.com/astral-sh/ruff/issues/10077#issuecomment-1961266981

This PR changes the blank line rules to not enforce the number of blank
lines after imports (top-level) if isort is enabled and leave it to
isort to enforce the right number of lines (depends on the
`isort.lines-after-imports` and `isort.lines-between-types` settings).

The reason to give `isort` precedence over the blank line rules is that
they are configurable. Users that always want to blank lines after
imports can use `isort.lines-after-imports=2` to enforce that
(specifically for imports).

This PR does not fix the incompatibility with the formatter in pyi files
that only uses 0 to 1 blank lines. I'll address this separately.

</details>

## Review
The first commit is a small refactor that simplified implementing the
fix (and makes it easier to reason about what's mutable and what's not).


## Test Plan

I added a new test and verified that it fails with an error that the fix
never converges. I verified the snapshot output after implementing the
fix.

---------

Co-authored-by: Hoël Bagard <34478245+hoel-bagard@users.noreply.github.com>
2024-03-05 10:09:15 +00:00
ooo oo 72599dafb6
docs: fix a rustdoc typo in C409 rule (#10233) 2024-03-05 02:33:08 +00:00
Charlie Marsh 7eaec300dd
Move shell expansion into `--config` lookup (#10219)
## Summary

When users provide configurations via `--config`, we use `shellexpand`
to ensure that we expand signifiers like `~` and environment variables.

In https://github.com/astral-sh/ruff/pull/9599, we modified `--config`
to accept either a path or an arbitrary setting. However, the detection
(to determine whether the value is a path or a setting) was lacking the
`shellexpand` behavior -- it was downstream. So we were always treating
paths like `~/ruff.toml` as values, not paths.

Closes https://github.com/astral-sh/ruff-vscode/issues/413.
2024-03-04 12:45:50 -05:00
Micha Reiser 184241f99a
Remove `Expr` postfix from `ExprNamed`, `ExprIf`, and `ExprGenerator` (#10229)
The expression types in our AST are called `ExprYield`, `ExprAwait`,
`ExprStringLiteral` etc, except `ExprNamedExpr`, `ExprIfExpr` and
`ExprGenratorExpr`. This seems to align with [Python AST's
naming](https://docs.python.org/3/library/ast.html) but feels
inconsistent and excessive.

This PR removes the `Expr` postfix from `ExprNamedExpr`, `ExprIfExpr`,
and `ExprGeneratorExpr`.
2024-03-04 12:55:01 +01:00
Alex Waygood 8b749e1d4d
Make `--config` and `--isolated` global flags (#10150) 2024-03-04 11:19:40 +00:00
Steve C 8dde81a905
[`pylint`] - add fix for unary expressions in `PLC2801` (#9587)
## Summary

Closes #9572

Don't go easy on this review!

## Test Plan

`cargo test`
2024-03-04 11:25:17 +01:00
Dominik Spicher 00300c0d9d
cache: tweak generated .gitignore (#10226)
- Add a notice that this file was generated by ruff

 - Add a trailing newline
2024-03-04 10:49:51 +01:00
Micha Reiser a6d892b1f4
Split `CallPath` into `QualifiedName` and `UnqualifiedName` (#10210)
## Summary

Charlie can probably explain this better than I but it turns out,
`CallPath` is used for two different things:

* To represent unqualified names like `version` where `version` can be a
local variable or imported (e.g. `from sys import version` where the
full qualified name is `sys.version`)
* To represent resolved, full qualified names

This PR splits `CallPath` into two types to make this destinction clear.

> Note: I haven't renamed all `call_path` variables to `qualified_name`
or `unqualified_name`. I can do that if that's welcomed but I first want
to get feedback on the approach and naming overall.

## Test Plan

`cargo test`
2024-03-04 09:06:51 +00:00
Micha Reiser 64f66cd8fe
Refine SemanticModel lifetime bounds (#10221)
## Summary

Corrects/refines some semantic model and related lifetime bounds.

## Test Plan

`cargo check`
2024-03-04 09:21:13 +01:00
Gautier Moin 4eac9baf43
[`pep8_naming`] Add fixes `N804` and `N805` (#10215)
## Summary

This PR fixes for `invalid-first-argument` rules.
The fixes rename the first argument of methods and class methods to the
valid one. References to this argument are also renamed.
Fixes are skipped when another argument is named as the valid first
argument.
The fix is marked as unsafe due

The functions for the `N804` and `N805` rules are now merged, as they
only differ by the name of the valid first argument.
The rules were moved from the AST iteration to the deferred scopes to be
in the function scope while creating the fix.

## Test Plan

`cargo test`
2024-03-04 02:22:54 +00:00
Charlie Marsh 737fcfd79e
Remove trailing space from `CapWords` message (#10220) 2024-03-04 01:54:53 +00:00
Charlie Marsh 84bf333031
Accept a PEP 440 version specifier for required-version (#10216)
## Summary

Allows `required-version` to be set with a version specifier, like
`>=0.3.1`.

If a single version is provided, falls back to assuming `==0.3.1`, for
backwards compatibility.

Closes https://github.com/astral-sh/ruff/issues/10192.
2024-03-03 18:43:49 -05:00
Micha Reiser db25a563f7
Remove unneeded lifetime bounds (#10213)
## Summary

This PR removes the unneeded lifetime `'b` from many of our `Visitor`
implementations.

The lifetime is unneeded because it is only constraint by `'a`, so we
can use `'a` directly.

## Test Plan

`cargo build`
2024-03-03 18:12:11 +00:00
Micha Reiser e725b6fdaf
CallPath newtype wrapper (#10201)
## Summary

This PR changes the `CallPath` type alias to a newtype wrapper. 

A newtype wrapper allows us to limit the API and to experiment with
alternative ways to implement matching on `CallPath`s.



## Test Plan

`cargo test`
2024-03-03 16:54:24 +01:00
Charlie Marsh ba7f6783e9
Avoid false-positives for parens-on-raise with futures.exception() (#10206)
## Summary

As a heuristic, we now ignore function calls that "look like" method
calls (e.g., `future.exception()`).

Closes https://github.com/astral-sh/ruff/issues/10205.
2024-03-03 00:28:51 +00:00
Charlie Marsh 7515196245
Respect external codes in file-level exemptions (#10203)
We shouldn't warn when an "external" code is used in a file-level
exemption.

Closes https://github.com/astral-sh/ruff/issues/10202.
2024-03-03 00:20:36 +00:00
Jeremy Hiatt c007b175ba
Check for use of `debugpy` and `ptvsd` debug modules (#10177) (#10194)
## Summary

This addresses https://github.com/astral-sh/ruff/issues/10177.

## Test Plan

I added additional lines to the existing test file for T100.
2024-03-01 23:02:44 -05:00
trag1c 0cd3b07efa
Removed unused variable in `TRY300`'s example (#10190)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Removes the unnecessary `exc` variable in `TRY300`'s docs example.

## Test Plan
```
python scripts/generate_mkdocs.py; mkdocs serve -f mkdocs.public.yml
```
2024-03-01 18:50:52 -05:00
Meheret c59d82a22e
CLI: Color entire line in Diffs (#10183) 2024-03-01 13:53:45 +01:00
Greenstar 8b5daaec7d
Fix broken documentation links affected by namespace changes in lint rules (#10182) 2024-03-01 12:35:29 +01:00
Hoël Bagard b82e87790e
Fix E301 not triggering on decorated methods. (#10117)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-03-01 09:30:53 +00:00
Meheret 56d445add9
Colorize the output of ruff format --diff (#10110)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-03-01 08:55:30 +00:00
Jane Lewis 8ecdf5369a
Fix RUF028 not allowing `# fmt: skip` on match cases (#10178)
## Summary

Fixes #10174 by allowing match cases to be enclosing nodes for
suppression comments. `else/elif` clauses are now also allowed to be
enclosing nodes.

## Test Plan
I've added the offending code from the original issue to the `RUF028`
snapshot test, and I've also expanded it to test the allowed `else/elif`
clause.
2024-03-01 00:36:23 -08:00
Michael Merickel c9931a548f
Implement isort's `default-section` setting (#10149)
## Summary

This fixes https://github.com/astral-sh/ruff/issues/7868.

Support isort's `default-section` feature which allows any imports that
match sections that are not in `section-order` to be mapped to a
specifically named section.


https://pycqa.github.io/isort/docs/configuration/options.html#default-section

This has a few implications:

- It is no longer required that all known sections are defined in
`section-order`.
- This is technically a bw-incompat change because currently if folks
define custom groups, and do not define a `section-order`, the code used
to add all known sections to `section-order` while emitting warnings.
**However, when this happened, users would be seeing warnings so I do
not think it should count as a bw-incompat change.**

## Test Plan

- Added a new test.
- Did not break any existing tests.

Finally, I ran the following config against Pyramid's complex codebase
that was previously using isort and this change worked there.

### pyramid's previous isort config


5f7e286b06/pyproject.toml (L22-L37)

```toml
[tool.isort]
profile = "black"
multi_line_output = 3
src_paths = ["src", "tests"]
skip_glob = ["docs/*"]
include_trailing_comma = true
force_grid_wrap = false
combine_as_imports = true
line_length = 79
force_sort_within_sections = true
no_lines_before = "THIRDPARTY"
sections = "FUTURE,THIRDPARTY,FIRSTPARTY,LOCALFOLDER"
default_section = "THIRDPARTY"
known_first_party = "pyramid"
```

### tested with ruff isort config

```toml
[tool.ruff.lint.isort]
case-sensitive = true
combine-as-imports = true
force-sort-within-sections = true
section-order = [
    "future",
    "third-party",
    "first-party",
    "local-folder",
]
default-section = "third-party"
known-first-party = [
    "pyramid",
]
```
2024-03-01 03:32:03 +00:00
Mikko Leppänen 8e0a70cfa3
[`pylint`] Implement `useless-exception-statement` (`W0133`) (#10176)
## Summary

This review contains a new rule for handling `useless exception
statements` (`PLW0133`). Is it based on the following pylint's rule:
[W0133](https://pylint.readthedocs.io/en/latest/user_guide/messages/warning/pointless-exception-statement.html)


Note: this rule does not cover the case if an error is a custom
exception class.

See: [Rule request](https://github.com/astral-sh/ruff/issues/10145)

## Test Plan

```bash
cargo test & manually
```
2024-02-29 21:37:16 -05:00
Shaygan Hooshyari cbafae022d
[`pylint`] Implement `singledispatch-method` (`E1519`) (#10140)
Implementing the rule 

https://pylint.readthedocs.io/en/stable/user_guide/messages/error/singledispatch-method.html#singledispatch-method-e1519

Implementation simply checks the function type and name of the
decorators.
2024-03-01 02:22:30 +00:00
Micha Reiser b53118ed00
Bump version to v0.3.0 (#10151)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-02-29 16:05:20 +01:00
Micha Reiser 52f4c1e41b
Remove deprecated CLI option `--format` (#10170)
Co-authored-by: Tibor Reiss <tibor.reiss@gmail.com>
2024-02-29 13:59:08 +00:00
Micha Reiser eceffe74a0
Deprecate `ruff <path>` `ruff --explain`, `ruff --clean` and `ruff --generate-shell-completion` (#10169) 2024-02-29 14:50:01 +01:00
Justin Sexton c73c497477
[`pydocstyle`] Trim whitespace when removing blank lines after section (`D413`) (#10162)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-02-29 13:29:40 +00:00
Micha Reiser dcc92f50cf
Update black tests (#10166) 2024-02-29 10:00:51 +01:00
Micha Reiser a6f32ddc5e
Ruff 2024.2 style (#9639) 2024-02-29 09:30:54 +01:00
Jane Lewis 0293908b71
Implement RUF028 to detect useless formatter suppression comments (#9899)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Fixes #6611

## Summary

This lint rule spots comments that are _intended_ to suppress or enable
the formatter, but will be ignored by the Ruff formatter.

We borrow some functions the formatter uses for determining comment
placement / putting them in context within an AST.

The analysis function uses an AST visitor to visit each comment and
attach it to the AST. It then uses that context to check:
1. Is this comment in an expression?
2. Does this comment have bad placement? (e.g. a `# fmt: skip` above a
function instead of at the end of a line)
3. Is this comment redundant?
4. Does this comment actually suppress any code?
5. Does this comment have ambiguous placement? (e.g. a `# fmt: off`
above an `else:` block)

If any of these are true, a violation is thrown. The reported reason
depends on the order of the above check-list: in other words, a `# fmt:
skip` comment on its own line within a list expression will be reported
as being in an expression, since that reason takes priority.

The lint suggests removing the comment as an unsafe fix, regardless of
the reason.

## Test Plan

A snapshot test has been created.
2024-02-28 19:21:06 +00:00
Philipp Thiel 36bc725eaa
[`flake8-bugbear`] Avoid adding default initializers to stubs (`B006`) (#10152)
## Summary

Adapts the fix for rule B006 to no longer modify the body of function
stubs, while retaining the change in method signature.

## Test Plan

The existing tests for B006 were adapted to reflect this change in
behavior.

## Relevant issue

https://github.com/astral-sh/ruff/issues/10083
2024-02-28 18:19:36 +00:00
Charlie Marsh a1905172a8
[`flake8-bandit`] Remove `suspicious-lxml-import` (`S410`) (#10154)
## Summary

The `lxml` library has been modified to address known vulnerabilities
and unsafe defaults. As such, the `defusedxml`
library is no longer necessary, `defusedxml` has deprecated its `lxml`
module.

Closes https://github.com/astral-sh/ruff/issues/10030.
2024-02-28 12:38:55 -05:00
Micha Reiser 1791e7d73b
Limit `isort.lines-after-imports` to 1 for stub files (#9971) 2024-02-28 17:36:51 +01:00
Charlie Marsh 317d2e4c75
Remove `build` from the default exclusion list (#10093)
## Summary

This is a not-unpopular directory name, and it's led to tons of issues
and user confusion (most recently:
https://github.com/astral-sh/ruff-pre-commit/issues/69). I've wanted to
remove it for a long time, but we need to do so as part of a minor
release.
2024-02-28 16:30:38 +00:00
Micha Reiser 8044c24c7e
Remove "Beta" Label from formatter documentation (#10144) 2024-02-28 12:47:37 +00:00
Robin Caloudis a1e8784207
[`ruff`] Expand rule for `list(iterable).pop(0)` idiom (`RUF015`) (#10148)
## Summary

Currently, rule `RUF015` is not able to detect the usage of
`list(iterable).pop(0)` falling under the category of an _unnecessary
iterable allocation for accessing the first element_. This PR wants to
change that. See the underlying issue for more details.

* Provide extension to detect `list(iterable).pop(0)`, but not
`list(iterable).pop(i)` where i > 1
* Update corresponding doc

## Test Plan

* `RUF015.py` and the corresponding snap file were extended such that
their correspond to the new behaviour

Closes https://github.com/astral-sh/ruff/issues/9190

--- 

PS: I've only been working on this ticket as I haven't seen any activity
from issue assignee @rmad17, neither in this repo nor in a fork. I hope
I interpreted his inactivity correctly. Didn't mean to steal his chance.
Since I stumbled across the underlying problem myself, I wanted to offer
a solution as soon as possible.
2024-02-28 00:24:28 +00:00
Micha Reiser 8dc22d5793
Perf: Skip string normalization when possible (#10116) 2024-02-26 17:35:29 +00:00
Micha Reiser 15b87ea8be
E203: Don't warn about single whitespace before tuple , (#10094) 2024-02-26 18:22:35 +01:00
Alex Waygood c25f1cd12a
Explicitly ban overriding `extend` as part of a --config flag (#10135) 2024-02-26 16:07:28 +00:00
Micha Reiser 77c5561646
Add `parenthesized` flag to `ExprTuple` and `ExprGenerator` (#9614) 2024-02-26 15:35:20 +00:00
Arkin Modi ab4bd71755
docs: fix pycodestyle.max-line-length link (#10136) 2024-02-26 14:58:13 +01:00
Alex Waygood 14fa1c5b52
[Minor] Improve the style of some tests in `crates/ruff/tests/format.rs` (#10132) 2024-02-26 11:02:09 +00:00
Micha Reiser 761d4d42f1
Add cold attribute to less likely printer queue branches (#10121) 2024-02-26 08:19:40 +01:00
Robin Caloudis fc8738f52a
[`ruff`] Avoid f-string false positives in `gettext` calls (`RUF027`) (#10118)
## Summary

It is a convention to use the `_()` alias for `gettext()`. We want to
avoid
statement expressions and assignments related to aliases of the gettext
API.
See https://docs.python.org/3/library/gettext.html for details. When one
uses `_() to mark a string for translation, the tools look for these
markers
and replace the original string with its translated counterpart. If the
string contains variable placeholders or formatting, it can complicate
the
translation process, lead to errors or incorrect translations.

## Test Plan

* Test file `RUF027_1.py` was extended such that the test reproduces the
false-positive

Closes https://github.com/astral-sh/ruff/issues/10023.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-02-25 18:17:56 -05:00
Micha Reiser 1711bca4a0
FString formatting: remove fstring handling in `normalize_string` (#10119) 2024-02-25 18:28:46 +01:00
Micha Reiser a284c711bf
Refactor trailing comma rule into explicit check and state update code (#10100) 2024-02-23 17:56:05 +01:00
Charlie Marsh 946028e358
Respect runtime-required decorators for function signatures (#10091)
## Summary

The original implementation of this applied the runtime-required context
to definitions _within_ the function, but not the signature itself. (We
had test coverage; the snapshot was just correctly showing the wrong
outcome.)

Closes https://github.com/astral-sh/ruff/issues/10089.
2024-02-23 03:33:08 +00:00
Charlie Marsh 6fe15e7289
Allow © in copyright notices (#10065)
Closes https://github.com/astral-sh/ruff/issues/10061.
2024-02-22 12:44:22 -05:00
Seo Sanghyeon 7d9ce5049a
PLR0203: Delete entire statement, including semicolons (#10074)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-02-22 16:03:00 +00:00
Arjun Munji 175c266de3
Omit repeated equality comparison for sys (#10054)
## Summary
Update PLR1714 to ignore `sys.platform` and `sys.version` checks. 
I'm not sure if these checks or if we need to add more. Please advise.

Fixes #10017

## Test Plan
Added a new test case and ran `cargo nextest run`
2024-02-20 19:03:32 +00:00
Charlie Marsh 4997c681f1
[`pycodestyle`] Allow `os.environ` modifications between imports (`E402`) (#10066)
## Summary

Allows, e.g.:

```python
import os

os.environ["WORLD_SIZE"] = "1"
os.putenv("CUDA_VISIBLE_DEVICES", "4")

import torch
```

For now, this is only allowed in preview.

Closes https://github.com/astral-sh/ruff/issues/10059
2024-02-20 13:24:27 -05:00
Seo Sanghyeon 7eafba2a4d
[`pyupgrade`] Detect literals with unary operators (`UP018`) (#10060)
Fix #10029.
2024-02-20 18:21:06 +00:00
Ottavio Hartman 0f70c99c42
feat(ERA001): detect single-line code for try:, except:, etc. (#10057)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-02-20 18:40:18 +01:00
Ottavio Hartman 0d363ab239
fix(ERA001): detect commented out `case` statements, add more one-line support (#10055)
## Summary

Closes #10031 

- Detect commented out `case` statements. Playground repro:
https://play.ruff.rs/5a305aa9-6e5c-4fa4-999a-8fc427ab9a23
- Add more support for one-line commented out code.

## Test Plan

Unit tested and tested with
```sh
cargo run -p ruff -- check crates/ruff_linter/resources/test/fixtures/eradicate/ERA001.py --no-cache --preview --select ERA001
```

TODO:
- [x] `cargo insta test`
2024-02-19 22:56:42 -05:00
Daniël van Noord 68b8abf9c6
[`pylint`] Add PLE1141 `DictIterMissingItems` (#9845)
## Summary

References https://github.com/astral-sh/ruff/issues/970.

Implements
[`dict-iter-missing-items`](https://pylint.readthedocs.io/en/latest/user_guide/messages/error/dict-iter-missing-items.html).

Took the tests from "upstream"
[here](https://github.com/DanielNoord/pylint/blob/main/tests/functional/d/dict_iter_missing_items.py).

~I wasn't able to implement code for one false positive, but it is
pretty estoric: https://github.com/pylint-dev/pylint/issues/3283. I
would personally argue that adding this check as preview rule without
supporting this specific use case is fine. I did add a "test" for it.~
This was implemented.

## Test Plan

Followed the Contributing guide to create tests, hopefully I didn't miss
any.
Also ran CI on my own fork and seemed to be all okay 😄 

~Edit: the ecosystem check seems a bit all over the place? 😅~ All good.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-02-19 19:56:55 +05:30
Seo Sanghyeon 1c8851e5fb
Do multiline string test for W293 too (#10049) 2024-02-19 11:58:56 +00:00
Charlie Marsh e1928be36e
Allow boolean positionals in `__post_init__` (#10027)
Closes https://github.com/astral-sh/ruff/issues/10011.
2024-02-18 15:03:17 +00:00
Charlie Marsh 235cfb7976
Bump version to v0.2.2 (#10018) 2024-02-17 22:15:04 +00:00
Dhruv Manilawala 91ae81b565
Move `RUF001`, `RUF002` to AST checker (#9993)
## Summary

Part of #7595 

This PR moves the `RUF001` and `RUF002` rules to the AST checker. This
removes the use of docstring detection from these rules.

## Test Plan

As this is just a refactor, make sure existing test cases pass.
2024-02-17 17:01:31 +00:00
Adam Kuhn d46c5d8ac8
docs: Formatter compatibility warning for D207 and D300 (#10007)
- Update docs to mention formatter compatibility interactions for
under-indentation (D207) and triple-single-quotes (D300)
- Changes verified locally with mkdocs
- Closes: https://github.com/astral-sh/ruff/issues/9675
2024-02-17 07:37:38 -05:00
Jane Lewis 20217e9bbd
Fix panic on RUF027 (#9990)
## Summary

Fixes #9895 

The cause for this panic came from an offset error in the code. When
analyzing a hypothetical f-string, we attempt to re-parse it as an
f-string, and use the AST data to determine, among other things, whether
the format specifiers are correct. To determine the 'correctness' of a
format specifier, we actually have to re-parse the format specifier, and
this is where the issue lies. To get the source text for the specifier,
we were taking a slice from the original file source text... even though
the AST data for the specifier belongs to the standalone parsed f-string
expression, meaning that the ranges are going to be way off. In a file
with Unicode, this can cause panics if the slice is inside a char
boundary.

To fix this, we now slice from the temporary source we created earlier
to parse the literal as an f-string.

## Test Plan

The RUF027 snapshot test was amended to include a string with format
specifiers which we _should_ be calling out. This is to ensure we do
slice format specifiers from the source text correctly.
2024-02-16 20:04:39 +00:00
Dhruv Manilawala 72bf1c2880
Preview minimal f-string formatting (#9642)
## Summary

_This is preview only feature and is available using the `--preview`
command-line flag._

With the implementation of [PEP 701] in Python 3.12, f-strings can now
be broken into multiple lines, can contain comments, and can re-use the
same quote character. Currently, no other Python formatter formats the
f-strings so there's some discussion which needs to happen in defining
the style used for f-string formatting. Relevant discussion:
https://github.com/astral-sh/ruff/discussions/9785

The goal for this PR is to add minimal support for f-string formatting.
This would be to format expression within the replacement field without
introducing any major style changes.

### Newlines

The heuristics for adding newline is similar to that of
[Prettier](https://prettier.io/docs/en/next/rationale.html#template-literals)
where the formatter would only split an expression in the replacement
field across multiple lines if there was already a line break within the
replacement field.

In other words, the formatter would not add any newlines unless they
were already present i.e., they were added by the user. This makes
breaking any expression inside an f-string optional and in control of
the user. For example,

```python
# We wouldn't break this
aaaaaaaaaaa = f"asaaaaaaaaaaaaaaaa { aaaaaaaaaaaa + bbbbbbbbbbbb + ccccccccccccccc } cccccccccc"

# But, we would break the following as there's already a newline
aaaaaaaaaaa = f"asaaaaaaaaaaaaaaaa {
	aaaaaaaaaaaa + bbbbbbbbbbbb + ccccccccccccccc } cccccccccc"
```


If there are comments in any of the replacement field of the f-string,
then it will always be a multi-line f-string in which case the formatter
would prefer to break expressions i.e., introduce newlines. For example,

```python
x = f"{ # comment
    a }"
```

### Quotes

The logic for formatting quotes remains unchanged. The existing logic is
used to determine the necessary quote char and is used accordingly.

Now, if the expression inside an f-string is itself a string like, then
we need to make sure to preserve the existing quote and not change it to
the preferred quote unless it's 3.12. For example,

```python
f"outer {'inner'} outer"

# For pre 3.12, preserve the single quote
f"outer {'inner'} outer"

# While for 3.12 and later, the quotes can be changed
f"outer {"inner"} outer"
```

But, for triple-quoted strings, we can re-use the same quote char unless
the inner string is itself a triple-quoted string.

```python
f"""outer {"inner"} outer"""  # valid
f"""outer {'''inner'''} outer"""  # preserve the single quote char for the inner string
```

### Debug expressions

If debug expressions are present in the replacement field of a f-string,
then the whitespace needs to be preserved as they will be rendered as it
is (for example, `f"{ x = }"`. If there are any nested f-strings, then
the whitespace in them needs to be preserved as well which means that
we'll stop formatting the f-string as soon as we encounter a debug
expression.

```python
f"outer {   x =  !s  :.3f}"
#                  ^^
#                  We can remove these whitespaces
```

Now, the whitespace doesn't need to be preserved around conversion spec
and format specifiers, so we'll format them as usual but we won't be
formatting any nested f-string within the format specifier.

### Miscellaneous

- The
[`hug_parens_with_braces_and_square_brackets`](https://github.com/astral-sh/ruff/issues/8279)
preview style isn't implemented w.r.t. the f-string curly braces.
- The
[indentation](https://github.com/astral-sh/ruff/discussions/9785#discussioncomment-8470590)
is always relative to the f-string containing statement

## Test Plan

* Add new test cases
* Review existing snapshot changes
* Review the ecosystem changes

[PEP 701]: https://peps.python.org/pep-0701/
2024-02-16 20:28:11 +05:30
Adrien Ball c3bba54b6b
Fix SIM113 false positive with async for loops (#9996)
## Summary
Ignore `async for` loops when checking the SIM113 rule.

Closes #9995 

## Test Plan
A new test case was added to SIM113.py with an async for loop.
2024-02-15 22:40:01 -05:00
Micha Reiser fe79798c12
split string module (#9987) 2024-02-14 18:54:55 +01:00
Micha Reiser bb8d2034e2
Use atomic write when persisting cache (#9981) 2024-02-14 15:09:21 +01:00
Charlie Marsh f40e012b4e
Use name directly in RUF006 (#9979) 2024-02-14 00:00:47 +00:00
Asger Hautop Drewsen 3e9d761b13
Expand `asyncio-dangling-task` (`RUF006`) to include `new_event_loop` (#9976)
## Summary

Fixes #9974

## Test Plan

I added some new test cases.
2024-02-13 18:28:06 +00:00
Dhruv Manilawala 6f9c128d77
Separate `StringNormalizer` from `StringPart` (#9954)
## Summary

This PR is a small refactor to extract out the logic for normalizing
string in the formatter from the `StringPart` struct. It also separates
the quote selection into a separate method on the new
`StringNormalizer`. Both of these will help in the f-string formatting
to use `StringPart` and `choose_quotes` irrespective of normalization.

The reason for having separate quote selection and normalization step is
so that the f-string formatting can perform quote selection on its own.

Unlike string and byte literals, the f-string formatting would require
that the normalization happens only for the literal elements of it i.e.,
the "foo" and "bar" in `f"foo {x + y} bar"`. This will automatically be
handled by the already separate `normalize_string` function.

Another use-case in the f-string formatting is to extract out the
relevant information from the `StringPart` like quotes and prefix which
is to be passed as context while formatting each element of an f-string.

## Test Plan

Ensure that clippy is happy and all tests pass.
2024-02-13 18:14:56 +05:30
Micha Reiser 6380c90031
Run isort CRLF tests (#9970) 2024-02-13 09:25:22 +01:00
Charlie Marsh d96a0dbe57
Respect tuple assignments in typing analyzer (#9969)
## Summary

Just addressing some discrepancies between the analyzers like `is_dict`
and the logic that's matured in `find_binding_value`.
2024-02-13 05:02:52 +00:00
Dhruv Manilawala 180920fdd9
Make semantic model aware of docstring (#9960)
## Summary

This PR introduces a new semantic model flag `DOCSTRING` which suggests
that the model is currently in a module / class / function docstring.
This is the first step in eliminating the docstring detection state
machine which is prone to bugs as stated in #7595.

## Test Plan

~TODO: Is there a way to add a test case for this?~

I tested this using the following code snippet and adding a print
statement in the `string_like` analyzer to print if we're currently in a
docstring or not.

<details><summary>Test code snippet:</summary>
<p>

```python
"Docstring" ", still a docstring"
"Not a docstring"


def foo():
    "Docstring"
    "Not a docstring"
    if foo:
        "Not a docstring"
        pass


class Foo:
    "Docstring"
    "Not a docstring"

    foo: int
    "Unofficial variable docstring"

    def method():
        "Docstring"
        "Not a docstring"
        pass


def bar():
    "Not a docstring".strip()


def baz():
    _something_else = 1
    """Not a docstring"""
```

</p>
</details>
2024-02-13 04:26:08 +00:00
Aleksei Latyshev dd0ba16a79
[`refurb`] Implement `readlines_in_for` lint (FURB129) (#9880)
## Summary
Implement [implicit readlines
(FURB129)](https://github.com/dosisod/refurb/blob/master/refurb/checks/iterable/implicit_readlines.py)
lint.

## Notes
I need a help/an opinion about suggested implementations.

This implementation differs from the original one from `refurb` in the
following way. This implementation checks syntactically the call of the
method with the name `readlines()` inside `for` {loop|generator
expression}. The implementation from refurb also
[checks](https://github.com/dosisod/refurb/blob/master/refurb/checks/iterable/implicit_readlines.py#L43)
that callee is a variable with a type `io.TextIOWrapper` or
`io.BufferedReader`.

- I do not see a simple way to implement the same logic.
- The best I can have is something like
```rust
checker.semantic().binding(checker.semantic().resolve_name(attr_expr.value.as_name_expr()?)?).statement(checker.semantic())
```
and analyze cases. But this will be not about types, but about guessing
the type by assignment (or with) expression.
- Also this logic has several false negatives, when the callee is not a
variable, but the result of function call (e.g. `open(...)`).
- On the other side, maybe it is good to lint this on other things,
where this suggestion is not safe, and push the developers to change
their interfaces to be less surprising, comparing with the standard
library.
- Anyway while the current implementation has false-positives (I
mentioned some of them in the test) I marked the fixes to be unsafe.
2024-02-12 22:28:35 -05:00
Charlie Marsh 609d0a9a65
Remove symbol from type-matching API (#9968)
## Summary

These should be no-op refactors to remove some redundant data from the
type analysis APIs.
2024-02-12 20:57:19 -05:00
Auguste Lalande 8fba97f72f
`PLR2004`: Accept 0.0 and 1.0 as common magic values (#9964)
## Summary

Accept 0.0 and 1.0 as common magic values. This is in line with the
pylint behaviour, and I think makes sense conceptually.


## Test Plan

Test cases were added to
`crates/ruff_linter/resources/test/fixtures/pylint/magic_value_comparison.py`
2024-02-13 01:21:06 +00:00
Charlie Marsh 5bc0d9c324
Add a binding kind for comprehension targets (#9967)
## Summary

I was surprised to learn that we treat `x` in `[_ for x in y]` as an
"assignment" binding kind, rather than a dedicated comprehension
variable.
2024-02-12 20:09:39 -05:00
Hashem cf77eeb913
unused_imports/F401: Explain when imports are preserved (#9963)
The docs previously mentioned an irrelevant config option, but were
missing a link to the relevant `ignore-init-module-imports` config
option which _is_ actually used.

Additionally, this commit adds a link to the documentation to explain
the conventions around a module interface which includes using a
redundant import alias to preserve an unused import.

(noticed this while filing  #9962)
2024-02-12 19:07:20 -05:00
Dhruv Manilawala 3f4dd01e7a
Rename semantic model flag to `MODULE_DOCSTRING_BOUNDARY` (#9959)
## Summary

This PR renames the semantic model flag `MODULE_DOCSTRING` to
`MODULE_DOCSTRING_BOUNDARY`. The main reason is for readability and for
the new semantic model flag `DOCSTRING` which tracks that the model is
in a module / class / function docstring.

I got confused earlier with the name until I looked at the use case and
it seems that the `_BOUNDARY` prefix is more appropriate for the
use-case and is consistent with other flags.
2024-02-13 00:47:12 +05:30
Micha Reiser edfe8421ec
Disable top-level docstring formatting for notebooks (#9957) 2024-02-12 18:14:02 +00:00
Charlie Marsh ab2253db03
[`pylint`] Avoid suggesting set rewrites for non-hashable types (#9956)
## Summary

Ensures that `x in [y, z]` does not trigger in `x`, `y`, or `z` are
known _not_ to be hashable.

Closes https://github.com/astral-sh/ruff/issues/9928.
2024-02-12 13:05:54 -05:00
Dhruv Manilawala 33ac2867b7
Use non-parenthesized range for `DebugText` (#9953)
## Summary

This PR fixes the `DebugText` implementation to use the expression range
instead of the parenthesized range.

Taking the following code snippet as an example:
```python
x = 1
print(f"{  ( x  ) = }")
```

The output of running it would be:
```
  ( x  ) = 1
```

Notice that the whitespace between the parentheses and the expression is
preserved as is.

Currently, we don't preserve this information in the AST which defeats
the purpose of `DebugText` as the main purpose of the struct is to
preserve whitespaces _around_ the expression.

This is also problematic when generating the code from the AST node as
then the generator has no information about the parentheses the
whitespaces between them and the expression which would lead to the
removal of the parentheses in the generated code.

I noticed this while working on the f-string formatting where the debug
text would be used to preserve the text surrounding the expression in
the presence of debug expression. The parentheses were being dropped
then which made me realize that the problem is instead in the parser.

## Test Plan

1. Add a test case for the parser
2. Add a test case for the generator
2024-02-12 23:00:02 +05:30
Charlie Marsh 0304623878
[`perflint`] Catch a wider range of mutations in `PERF101` (#9955)
## Summary

This PR ensures that if a list `x` is modified within a `for` loop, we
avoid flagging `list(x)` as unnecessary. Previously, we only detected
calls to exactly `.append`, and they couldn't be nested within other
statements.

Closes https://github.com/astral-sh/ruff/issues/9925.
2024-02-12 12:17:55 -05:00
Charlie Marsh e2785f3fb6
[`flake8-pyi`] Ignore 'unused' private type dicts in class scopes (#9952)
## Summary

If these are defined within class scopes, they're actually attributes of
the class, and can be accessed through the class itself.

(We preserve our existing behavior for `.pyi` files.)

Closes https://github.com/astral-sh/ruff/issues/9948.
2024-02-12 17:06:20 +00:00
Micha Reiser 8657a392ff
Docstring formatting: Preserve tab indentation when using `indent-style=tabs` (#9915) 2024-02-12 16:09:13 +01:00
Micha Reiser 4946a1876f
Stabilize quote-style `preserve` (#9922) 2024-02-12 09:30:07 +00:00
Micha Reiser 341c2698a7
Run doctests as part of CI pipeline (#9939) 2024-02-12 10:18:58 +01:00
Alex Waygood 8ec56277e9
Allow arbitrary configuration options to be overridden via the CLI (#9599)
Fixes #8368
Fixes https://github.com/astral-sh/ruff/issues/9186

## Summary

Arbitrary TOML strings can be provided via the command-line to override
configuration options in `pyproject.toml` or `ruff.toml`. As an example:
to run over typeshed and respect typeshed's `pyproject.toml`, but
override a specific isort setting and enable an additional pep8-naming
setting:

```
cargo run -- check ../typeshed --no-cache --config ../typeshed/pyproject.toml --config "lint.isort.combine-as-imports=false" --config "lint.extend-select=['N801']"
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-02-09 21:56:37 +00:00
Alex Waygood d387d0ba82
RUF022, RUF023: Ensure closing parentheses for multiline sequences are always on their own line (#9793)
## Summary

Currently these rules apply the heuristic that if the original sequence
doesn't have a newline in between the final sequence item and the
closing parenthesis, the autofix won't add one for you. The feedback
from @ThiefMaster, however, was that this was producing slightly unusual
formatting -- things like this:

```py
__all__ = [
    "b", "c",
    "a", "d"]
```

were being autofixed to this:

```py
__all__ = [
    "a",
    "b",
    "c",
    "d"]
```

When, if it was _going_ to be exploded anyway, they'd prefer something
like this (with the closing parenthesis on its own line, and a trailing comma added):

```py
__all__ = [
    "a",
    "b",
    "c",
    "d",
]
```

I'm still pretty skeptical that we'll be able to please everybody here
with the formatting choices we make; _but_, on the other hand, this
_specific_ change is pretty easy to make.

## Test Plan

`cargo test`. I also ran the autofixes for RUF022 and RUF023 on CPython
to check how they looked; they looked fine to me.
2024-02-09 21:27:44 +00:00
Charlie Marsh 6f0e4ad332
Remove unnecessary string cloning from the parser (#9884)
Closes https://github.com/astral-sh/ruff/issues/9869.
2024-02-09 16:03:27 -05:00
trag1c 7ca515c0aa
Corrected PTH203–PTH205 rule descriptions (#9914)
## Summary
Closes #9898.

## Test Plan
```sh
python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml
```
2024-02-09 15:47:07 -05:00
Micha Reiser 1ce07d65bd
Use `usize` instead of `TextSize` for `indent_len` (#9903) 2024-02-09 20:41:36 +00:00
Charlie Marsh 52ebfc9718
Respect duplicates when rewriting type aliases (#9905)
## Summary

If a generic appears multiple times on the right-hand side, we should
only include it once on the left-hand side when rewriting.

Closes https://github.com/astral-sh/ruff/issues/9904.
2024-02-09 14:02:41 +00:00
Hoël Bagard 12a91f4e90
Fix `E30X` panics on blank lines with trailing white spaces (#9907) 2024-02-09 14:00:26 +00:00
Mikko Leppänen b4f2882b72
[`pydocstyle-D405`] Allow using `parameters` as a sub-section header (#9894)
## Summary

This review contains a fix for
[D405](https://docs.astral.sh/ruff/rules/capitalize-section-name/)
(capitalize-section-name)
The problem is that Ruff considers the sub-section header as a normal
section if it has the same name as some section name. For instance, a
function/method has an argument named "parameters". This only applies if
you use Numpy style docstring.

See: [ISSUE](https://github.com/astral-sh/ruff/issues/9806)

The following will not raise D405 after the fix:
```python  
def some_function(parameters: list[str]):
    """A function with a parameters parameter

    Parameters
    ----------

    parameters:
        A list of string parameters
    """
    ...
```


## Test Plan

```bash
cargo test
```

---------

Co-authored-by: Mikko Leppänen <mikko.leppanen@vaisala.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-02-08 21:54:32 -05:00
Charlie Marsh 49fe1b85f2
Reduce size of `Expr` from 80 to 64 bytes (#9900)
## Summary

This PR reduces the size of `Expr` from 80 to 64 bytes, by reducing the
sizes of...

- `ExprCall` from 72 to 56 bytes, by using boxed slices for `Arguments`.
- `ExprCompare` from 64 to 48 bytes, by using boxed slices for its
various vectors.

In testing, the parser gets a bit faster, and the linter benchmarks
improve quite a bit.
2024-02-09 02:53:13 +00:00
Micha Reiser bd8123c0d8
Fix clippy unused variable warning (#9902) 2024-02-08 22:13:31 +00:00
Micha Reiser 49c5e715f9
Filter out test rules in `RuleSelector` JSON schema (#9901) 2024-02-08 21:06:51 +00:00
Micha Reiser fe7d965334
Reduce `Result<Tok, LexicalError>` size by using `Box<str>` instead of `String` (#9885) 2024-02-08 20:36:22 +00:00
Hoël Bagard 9027169125
[`pycodestyle`] Add blank line(s) rules (`E301`, `E302`, `E303`, `E304`, `E305`, `E306`) (#9266)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-02-08 18:35:08 +00:00
Micha Reiser 688177ff6a
Use Rust 1.76 (#9897) 2024-02-08 18:20:08 +00:00
trag1c eb2784c495
Corrected Path symlink method name (PTH114) (#9896)
## Summary
Corrects mentions of `Path.is_link` to `Path.is_symlink` (the former
doesn't exist).

## Test Plan
```sh
python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml
```
2024-02-08 13:09:28 -05:00
Charlie Marsh 6fffde72e7
Use `memchr` for string lexing (#9888)
## Summary

On `main`, string lexing consists of walking through the string
character-by-character to search for the closing quote (with some
nuance: we also need to skip escaped characters, and error if we see
newlines in non-triple-quoted strings). This PR rewrites `lex_string` to
instead use `memchr` to search for the closing quote, which is
significantly faster. On my machine, at least, the `globals.py`
benchmark (which contains a lot of docstrings) gets 40% faster...

```text
lexer/numpy/globals.py  time:   [3.6410 µs 3.6496 µs 3.6585 µs]
                        thrpt:  [806.53 MiB/s 808.49 MiB/s 810.41 MiB/s]
                 change:
                        time:   [-40.413% -40.185% -39.984%] (p = 0.00 < 0.05)
                        thrpt:  [+66.623% +67.181% +67.822%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
lexer/unicode/pypinyin.py
                        time:   [12.422 µs 12.445 µs 12.467 µs]
                        thrpt:  [337.03 MiB/s 337.65 MiB/s 338.27 MiB/s]
                 change:
                        time:   [-9.4213% -9.1930% -8.9586%] (p = 0.00 < 0.05)
                        thrpt:  [+9.8401% +10.124% +10.401%]
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
lexer/pydantic/types.py time:   [107.45 µs 107.50 µs 107.56 µs]
                        thrpt:  [237.11 MiB/s 237.24 MiB/s 237.35 MiB/s]
                 change:
                        time:   [-4.0108% -3.7005% -3.3787%] (p = 0.00 < 0.05)
                        thrpt:  [+3.4968% +3.8427% +4.1784%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
lexer/numpy/ctypeslib.py
                        time:   [46.123 µs 46.165 µs 46.208 µs]
                        thrpt:  [360.36 MiB/s 360.69 MiB/s 361.01 MiB/s]
                 change:
                        time:   [-19.313% -18.996% -18.710%] (p = 0.00 < 0.05)
                        thrpt:  [+23.016% +23.451% +23.935%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) low mild
  1 (1.00%) high mild
  4 (4.00%) high severe
lexer/large/dataset.py  time:   [231.07 µs 231.19 µs 231.33 µs]
                        thrpt:  [175.87 MiB/s 175.97 MiB/s 176.06 MiB/s]
                 change:
                        time:   [-2.0437% -1.7663% -1.4922%] (p = 0.00 < 0.05)
                        thrpt:  [+1.5148% +1.7981% +2.0864%]
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe
```
2024-02-08 17:23:06 +00:00
Jane Lewis ad313b9089
RUF027 no longer has false negatives with string literals inside of method calls (#9865)
Fixes #9857.

## Summary

Statements like `logging.info("Today it is: {day}")` will no longer be
ignored by RUF027. As before, statements like `"Today it is:
{day}".format(day="Tuesday")` will continue to be ignored.

## Test Plan

The snapshot tests were expanded to include new cases. Additionally, the
snapshot tests have been split in two to separate positive cases from
negative cases.
2024-02-08 10:00:20 -05:00
Charlie Marsh f76a3e8502
Detect `mark_safe` usages in decorators (#9887)
## Summary

Django's `mark_safe` can also be used as a decorator, so we should
detect usages of `@mark_safe` for the purpose of the relevant Bandit
rule.

Closes https://github.com/astral-sh/ruff/issues/9780.
2024-02-07 23:10:46 -05:00
Tom Kuson ed07fa08bd
Fix list formatting in documention (#9886)
## Summary

Adds a blank line to render the list correctly.

## Test Plan

Ocular inspection
2024-02-07 20:01:21 -05:00
Charlie Marsh 45937426c7
Fix blank-line docstring rules for module-level docstrings (#9878)
## Summary

Given:

```python
"""Make a summary line.

Note:
----
  Per the code comment the next two lines are blank. "// The first blank line is the line containing the closing
      triple quotes, so we need at least two."

"""
```

It turns out we excluded the line ending in `"""`, because it's empty
(unlike for functions, where it consists of the indent). This PR changes
the `following_lines` iterator to always include the trailing newline,
which gives us correct and consistent handling between function and
module-level docstrings.

Closes https://github.com/astral-sh/ruff/issues/9877.
2024-02-07 16:48:28 -05:00
Charlie Marsh 533dcfb114
Add a note regarding ignore-without-code (#9879)
Closes https://github.com/astral-sh/ruff/issues/9863.
2024-02-07 21:20:18 +00:00
Hugo van Kemenade bc023f47a1
Fix typo in option name: `output_format` -> `output-format` (#9874) 2024-02-07 16:17:58 +00:00
Jack McIvor aa38307415
Add more NPY002 violations (#9862) 2024-02-07 09:54:11 -05:00
Charlie Marsh e9ddd4819a
Make show-settings filters directory-agnostic (#9866)
Closes https://github.com/astral-sh/ruff/issues/9864.
2024-02-07 03:20:27 +00:00
Micha Reiser fdb5eefb33
Improve trailing comma rule performance (#9867) 2024-02-06 23:04:36 +00:00
Charlie Marsh daae28efc7
Respect `async with` in `timeout-without-await` (#9859)
Closes https://github.com/astral-sh/ruff/issues/9855.
2024-02-06 12:04:24 -05:00
Charlie Marsh c34908f5ad
Use `memchr` for tab-indentation detection (#9853)
## Summary

The benchmarks show a pretty consistent 1% speedup here for all-rules,
though not enough to trigger our threshold of course:

![Screenshot 2024-02-05 at 11 55
59 PM](https://github.com/astral-sh/ruff/assets/1309177/317dca3f-f25f-46f5-8ea8-894a1747d006)
2024-02-06 09:44:56 -05:00
Charlie Marsh a662c2447c
Ignore builtins when detecting missing f-strings (#9849)
## Summary

Reported on Discord: if the name maps to a builtin, it's not bound
locally, so is very unlikely to be intended as an f-string expression.
2024-02-05 23:49:56 -05:00
Seo Sanghyeon df7fb95cbc
Index multiline f-strings (#9837)
Fix #9777.
2024-02-05 21:25:33 -05:00
Eero Vaher cd5bcd815d
Mention a related setting in C408 description (#9839)
#2977 added the `allow-dict-calls-with-keyword-arguments` configuration
option for the `unnecessary-collection-call (C408)` rule, but it did not
update the rule description.
2024-02-06 03:57:53 +05:30
Charlie Marsh 0ccca4083a
Bump version to v0.2.1 (#9843) 2024-02-05 15:31:05 -05:00
Charlie Marsh 041ce1e166
Respect generic `Protocol` in ellipsis removal (#9841)
Closes https://github.com/astral-sh/ruff/issues/9840.
2024-02-05 19:36:16 +00:00
Dhruv Manilawala 36b752876e
Implement `AnyNode`/`AnyNodeRef` for `FStringFormatSpec` (#9836)
## Summary

This PR adds the `AnyNode` and `AnyNodeRef` implementation for
`FStringFormatSpec` node which will be required in the f-string
formatting.

The main usage for this is so that we can pass in the node directly to
`suppressed_node` in case debug expression is used to format is as
verbatim text.
2024-02-05 19:23:43 +00:00
Micha Reiser b3dc565473
Add `--range` option to `ruff format` (#9733)
Co-authored-by: T-256 <132141463+T-256@users.noreply.github.com>
2024-02-05 19:21:45 +00:00
Thomas M Kehrenberg e708c08b64
Fix default for `max-positional-args` (#9838)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary
`max-positional-args` defaults to `max-args` if it's not specified and
the default to `max-args` is 5, so saying that the default is 3 is
definitely wrong. Ideally, we wouldn't specify a default at all for this
config option, but I don't think that's possible?

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->
Not sure.
2024-02-05 16:58:14 +00:00
Charlie Marsh 9781563ef6
Add fast-path for comment detection (#9808)
## Summary

When we fall through to parsing, the comment-detection rule is a
significant portion of lint time. This PR adds an additional fast
heuristic whereby we abort if a comment contains two consecutive name
tokens (via the zero-allocation lexer). For the `ctypeslib.py`, which
has a few cases that are now caught by this, it's a 2.5x speedup for the
rule (and a 20% speedup for token-based rules).
2024-02-05 11:00:18 -05:00
Zanie Blue 84aea7f0c8
Drop `__get__` and `__set__` from `unnecessary-dunder-call` (#9791)
These are for descriptors which affects the behavior of the object _as a
property_; I do not think they should be called directly but there is no
alternative when working with the object directly.

Closes https://github.com/astral-sh/ruff/issues/9789
2024-02-05 10:54:29 -05:00
Shaygan Hooshyari b47f85eb69
Preview Style: Format module level docstring (#9725)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-02-05 15:03:34 +00:00
Micha Reiser 80fc02e7d5
Don't trim last empty line in docstrings (#9813) 2024-02-05 13:29:24 +00:00
Charlie Marsh 602f8b8250
Remove CST-based fixer for `C408` (#9822)
## Summary

We have to keep the fixer for a specific case: `dict` calls that include
keyword-argument members.
2024-02-04 22:26:51 -05:00
Charlie Marsh a6bc4b2e48
Remove CST-based fixers for `C405` and `C409` (#9821) 2024-02-05 02:17:34 +00:00
Charlie Marsh c5fa0ccffb
Remove CST-based fixers for `C400`, `C401`, `C410`, and `C418` (#9819) 2024-02-04 21:00:11 -05:00
Charlie Marsh dd77d29d0e
Remove LibCST-based fixer for `C403` (#9818)
## Summary

Experimenting with rewriting one of the comprehension fixes _without_
LibCST.
2024-02-04 20:08:19 -05:00
Charlie Marsh ad0121660e
Run dunder method rule on methods directly (#9815)
This stood out in the flamegraph and I realized it requires us to
traverse over all statements in the class (unnecessarily).
2024-02-04 14:24:57 -05:00
Charlie Marsh 5c99967c4d
Short-circuit typing matches based on imports (#9800) 2024-02-04 14:06:44 -05:00
Charlie Marsh c53aae0b6f
Add our own ignored-names abstractions (#9802)
## Summary

These run over nearly every identifier. It's rare to override them, so
when not provided, we can just use a match against the hardcoded default
set.
2024-02-03 09:48:07 -05:00
Charlie Marsh 2352de2277
Slight speed-up for lowercase and uppercase identifier checks (#9798)
It turns out that for ASCII identifiers, this is nearly 2x faster:

```
Parser/before     time:   [15.388 ns 15.395 ns 15.406 ns]
Parser/after      time:   [8.3786 ns 8.5821 ns 8.7715 ns]
```
2024-02-03 14:40:41 +00:00
Jane Lewis e0a6034cbb
Implement RUF027: `Missing F-String Syntax` lint (#9728)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Fixes #8151

This PR implements a new rule, `RUF027`.

## What it does
Checks for strings that contain f-string syntax but are not f-strings.

### Why is this bad?
An f-string missing an `f` at the beginning won't format anything, and
instead treat the interpolation syntax as literal.

### Example

```python
name = "Sarah"
dayofweek = "Tuesday"
msg = "Hello {name}! It is {dayofweek} today!"
```

It should instead be:
```python
name = "Sarah"
dayofweek = "Tuesday"
msg = f"Hello {name}! It is {dayofweek} today!"
```

## Heuristics
Since there are many possible string literals which contain syntax
similar to f-strings yet are not intended to be,
this lint will disqualify any literal that satisfies any of the
following conditions:
1. The string literal is a standalone expression. For example, a
docstring.
2. The literal is part of a function call with keyword arguments that
match at least one variable (for example: `format("Message: {value}",
value = "Hello World")`)
3. The literal (or a parent expression of the literal) has a direct
method call on it (for example: `"{value}".format(...)`)
4. The string has no `{...}` expression sections, or uses invalid
f-string syntax.
5. The string references variables that are not in scope, or it doesn't
capture variables at all.
6. Any format specifiers in the potential f-string are invalid.

## Test Plan

I created a new test file, `RUF027.py`, which is both an example of what
the lint should catch and a way to test edge cases that may trigger
false positives.
2024-02-03 00:21:03 +00:00
Emil Telstad 25d93053da
Update max-pos-args example to max-positional-args. (#9797) 2024-02-02 20:29:13 +00:00
Charlie Marsh ee5b07d4ca
Skip empty lines when determining base indentation (#9795)
## Summary

It turns out we saw a panic in cases when dedenting blocks like the `def
wrapper` here:

```python
def instrument_url(f: UrlFuncT) -> UrlFuncT:
    # TODO: Type this with ParamSpec to preserve the function signature.
    if not INSTRUMENTING:  # nocoverage -- option is always enabled; should we remove?
        return f
    else:

        def wrapper(
            self: "ZulipTestCase", url: str, info: object = {}, **kwargs: Union[bool, str]
        ) -> HttpResponseBase:
```

Since we relied on the first line to determine the indentation, instead
of the first non-empty line.

## Test Plan

`cargo test`
2024-02-02 19:42:47 +00:00
Charlie Marsh e50603caf6
Track top-level module imports in the semantic model (#9775)
## Summary

This is a simple idea to avoid unnecessary work in the linter,
especially for rules that run on all name and/or all attribute nodes.
Imagine a rule like the NumPy deprecation check. If the user never
imported `numpy`, we should be able to skip that rule entirely --
whereas today, we do a `resolve_call_path` check on _every_ name in the
file. It turns out that there's basically a finite set of modules that
we care about, so we now track imports on those modules as explicit
flags on the semantic model. In rules that can _only_ ever trigger if
those modules were imported, we add a dedicated and extremely cheap
check to the top of the rule.

We could consider generalizing this to all modules, but I would expect
that not to be much faster than `resolve_call_path`, which is just a
hash map lookup on `TextSize` anyway.

It would also be nice to make this declarative, such that rules could
declare the modules they care about, the analyzers could call the rules
as appropriate. But, I don't think such a design should block merging
this.
2024-02-02 14:37:20 -05:00
Charlie Marsh c3ca34543f
Skip LibCST parsing for standard dedent adjustments (#9769)
## Summary

Often, when fixing, we need to dedent a block of code (e.g., if we
remove an `if` and dedent its body). Today, we use LibCST to parse and
adjust the indentation, which is really expensive -- but this is only
really necessary if the block contains a multiline string, since naively
adjusting the indentation for such a string can change the whitespace
_within_ the string.

This PR uses a simple dedent implementation for cases in which the block
doesn't intersect with a multi-line string (or an f-string, since we
don't support tracking multi-line strings for f-strings right now).

We could improve this even further by using the ranges to guide the
dedent function, such that we don't apply the dedent if the line starts
within a multiline string. But that would also need to take f-strings
into account, which is a little tricky.

## Test Plan

`cargo test`
2024-02-02 18:13:46 +00:00
Micha Reiser 4f7fb566f0
Range formatting: Fix invalid syntax after parenthesizing expression (#9751) 2024-02-02 17:56:25 +01:00
Charlie Marsh ea1c089652
Use `AhoCorasick` to speed up quote match (#9773)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

When I was looking at the v0.2.0 release, this method showed up in a
CodSpeed regression (we were calling it more), so I decided to quickly
look at speeding it up. @BurntSushi suggested using Aho-Corasick, and it
looks like it's about 7 or 8x faster:

```text
Parser/AhoCorasick      time:   [8.5646 ns 8.5914 ns 8.6191 ns]
Parser/Iterator         time:   [64.992 ns 65.124 ns 65.271 ns]
```

## Test Plan

`cargo test`
2024-02-02 09:57:39 -05:00
Mikael Arguedas b947dde8ad
[flake8-bugbear][B006] remove outdated comment (#9776)
I noticed that the comment doesn't match the behavior:
- zip function is not used anymore
- parameters are not scanned in reverse

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

No need

Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
2024-02-02 09:32:46 -05:00
Alex Gaynor 467c091382
Fixed example code in weak_cryptographic_key.rs (#9774)
The proper way to use these APIs is to instantiate the curve classes
2024-02-01 22:42:31 -05:00
Charlie Marsh 66d2c1e1c4
Move `adjust_indentation` to a shared home (#9768)
Now that this method is used in multiple linters, it should be moved out
of the `pyupgrade` module.
2024-02-02 00:53:59 +00:00
Charlie Marsh ded8c7629f
Invert order of checks in `zero-sleep-call` (#9766)
The other conditions are cheaper and should eliminate the vast majority
of these checks.
2024-02-01 23:30:03 +00:00
Zanie Blue 1fadefa67b
Bump version to 0.2.0 (#9762)
Follows https://github.com/astral-sh/ruff/pull/9680
2024-02-01 17:10:33 -06:00
Charlie Marsh 06ad687efd
Deduplicate deprecation warnings for v0.2.0 release (#9764)
## Summary

Adds an additional warning macro (we should consolidate these later)
that shows a warning once based on the content of the warning itself.
This is less efficient than `warn_user_once!` and `warn_user_by_id!`,
but this is so expensive that it doesn't matter at all.

Applies this macro to the various warnings for the v0.2.0 release, and
also includes the filename in said warnings, so the FastAPI case is now:

```text
warning: The top-level linter settings are deprecated in favour of their counterparts in the `lint` section. Please update the following options in /Users/crmarsh/workspace/fastapi/pyproject.toml:
  - 'ignore' -> 'lint.ignore'
  - 'select' -> 'lint.select'
  - 'isort' -> 'lint.isort'
  - 'pyupgrade' -> 'lint.pyupgrade'
  - 'per-file-ignores' -> 'lint.per-file-ignores'
```

---------

Co-authored-by: Zanie <contact@zanie.dev>
2024-02-01 17:10:24 -06:00
Jane Lewis 148b64ead3
Fix issue where output format mode would not change to `full` if preview mode was set in configuration file (#9763)
## Summary

This was causing build failures for #9599. We were referencing the
command line overrides instead of the merged configuration data, hence
the issue.

## Test Plan

A snapshot test was added.
2024-02-01 16:07:21 -06:00
Charlie Marsh 99eddbd2a0 Remove stale preview documentation from stabilized rule behaviors (#9759)
These behaviors were stabilized, so the docs referring to them as
preview-only are incorrect.
2024-02-01 13:35:02 -06:00
Zanie Blue 836d2eaa01 Restore RUF011 documentation (#9758)
For consistency with other redirected rules as in
https://github.com/astral-sh/ruff/pull/9755

Follow-up to #9428
2024-02-01 13:35:02 -06:00
Zanie Blue 994514d686 Redirect `PHG001` to `S307` and `PGH002` to `G010` (#9756)
Follow-up to #9754 and #9689. Alternative to #9714.
Replaces #7506 and #7507
Same ideas as #9755
Part of #8931
2024-02-01 13:35:02 -06:00