Commit Graph

5955 Commits

Author SHA1 Message Date
InSync 19f3424a1a
[`pylint`] Do not report calls when object type and argument type mismatch, remove custom escape handling logic (`PLE1310`) (#15984)
## Summary

Resolves #15968.

Previously, these would be considered violations:

```python
b''.strip('//')
''.lstrip('//', foo = "bar")
```

...while these are not:

```python
b''.strip(b'//')
''.strip('\\b\\x08')
```

Ruff will now not report when the types of the object and that of the
argument mismatch, or when there are extra arguments.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-07 14:31:07 -06:00
Brent Westbrook d4a5772d96
[`flake8-builtins`] Match upstream module name comparison (`A005`) (#16006)
See #15951 for the original discussion and reviews. This is just the
first half of that PR (reaching parity with `flake8-builtins` without
adding any new configuration options) split out for nicer changelog
entries.

For posterity, here's a script for generating the module structure that
was useful for interactive testing and creating the table
[here](https://github.com/astral-sh/ruff/pull/15951#issuecomment-2640662041).
The results for this branch are the same as the `Strict` column there,
as expected.

```shell
mkdir abc collections foobar urlparse

for i in */
do
	touch $i/__init__.py
done	

cp -r abc foobar collections/.
cp -r abc collections foobar/.

touch ruff.toml

touch foobar/logging.py
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-07 13:55:56 -05:00
Alex Waygood efa8a3ddcc
[`pyupgrade`] Don't introduce invalid syntax when upgrading old-style type aliases with parenthesized multiline values (`UP040`) (#16026) 2025-02-07 17:05:17 +00:00
Dylan 46fe17767d
Pass `Checker` by immutable reference to lint rules (#16012)
This very large PR changes the field `.diagnostics` in the `Checker`
from a `Vec<Diagnostic>` to a `RefCell<Vec<Diagnostic>>`, adds methods
to push new diagnostics to this cell, and then removes unnecessary
mutability throughout all of our lint rule implementations.

Consequently, the compiler may now enforce what was, till now, the
_convention_ that the only changes to the `Checker` that can happen
during a lint are the addition of diagnostics[^1].

The PR is best reviewed commit-by-commit. I have tried to keep the large
commits limited to "bulk actions that you can easily see are performing
the same find/replace on a large number of files", and separate anything
ad-hoc or with larger diffs. Please let me know if there's anything else
I can do to make this easier to review!

Many thanks to [`ast-grep`](https://github.com/ast-grep/ast-grep),
[`helix`](https://github.com/helix-editor/helix), and good ol'
fashioned`git` magic, without which this PR would have taken the rest of
my natural life.

[^1]: And randomly also the seen variables violating `flake8-bugbear`?
2025-02-07 09:05:50 -06:00
David Peter 1f7a29d347
[red-knot] Unpacker: Make invariant explicit and directly return a Type (#16018)
## Summary

- Do not return `Option<Type<…>>` from `Unpacker::get`, but just `Type`.
Panic otherwise.
- Rename `Unpacker::get` to `Unpacker::expression_type`
2025-02-07 12:00:04 +00:00
Wei Lee 618bfaf884
[`airflow`] Add `external_task.{ExternalTaskMarker, ExternalTaskSensor}` for `AIR302` (#16014)
## Summary

Apply suggestions similar to
https://github.com/astral-sh/ruff/pull/15922#discussion_r1940697704


## Test Plan

a test fixture has been updated
2025-02-07 16:38:34 +05:30
Alex Waygood b1c61cb2ee
[`ruff`] Fix invalid annotation in docs example (#16016) 2025-02-07 10:45:51 +00:00
David Peter 97e6fc3793
[red-knot] Unpacking and for loop assignments to attributes (#16004)
## Summary

* Support assignments to attributes in more cases:
    - assignments in `for` loops
    - in unpacking assignments
* Add test for multi-target assignments
* Add tests for all other possible assignments to attributes that could
   possibly occur (in decreasing order of likeliness):
    - augmented attribute assignments
    - attribute assignments in `with` statements
    - attribute assignments in comprehensions
- Note: assignments to attributes in named expressions are not
   syntactically allowed

closes #15962

## Test Plan

New Markdown tests
2025-02-07 11:30:51 +01:00
Micha Reiser 38351e00ee
[red-knot] Partial revert of relative import handling for files in the root of a search path (#16001)
## Summary

This PR reverts the behavior changes from
https://github.com/astral-sh/ruff/pull/15990

But it isn't just a revert, it also:

* Adds a test covering this specific behavior
* Preserves the improvement to use `saturating_sub` in the package case
to avoid overflows in the case of invalid syntax
* Use `ancestors` instead of a `for` loop

## Test Plan

Added test
2025-02-07 11:04:09 +01:00
Micha Reiser 26c37b1e0e
Add knot.toml schema (#15735)
## Summary

Adds a JSON schema generation step for Red Knot. This PR doesn't yet add
a publishing step because it's still a bit early for that


## Test plan

I tested the schema in Zed, VS Code and PyCharm:

* PyCharm: You have to manually add a schema mapping (settings JSON
Schema Mappings)
* Zed and VS code support the inline schema specification

```toml
#:schema /Users/micha/astral/ruff/knot.schema.json


[environment]
extra-paths = []


[rules]
call-possibly-unbound-method = "error"
unknown-rule = "error"

# duplicate-base = "error"
```

```json
{
    "$schema": "file:///Users/micha/astral/ruff/knot.schema.json",

    "environment": {
        "python-version": "3.13",
        "python-platform": "linux2"
    },

    "rules": {
        "unknown-rule": "error"
    }
}
```


https://github.com/user-attachments/assets/a18fcd96-7cbe-4110-985b-9f1935584411


The Schema overall works but all editors have their own quirks:

* PyCharm: Hovering a name always shows the section description instead
of the description of the specific setting. But it's the same for other
settings in `pyproject.toml` files 🤷
* VS Code (JSON): Using the generated schema in a JSON file gives
exactly the experience I want
* VS Code (TOML): 
* Properties with multiple possible values are repeated during
auto-completion without giving any hint how they're different. ![Screen
Shot 2025-02-06 at 14 05 35
PM](https://github.com/user-attachments/assets/d7f3c2a9-2351-4226-9fc1-b91aa192a237)
* The property description mushes together the description of the
property and the value, which looks sort of ridiculous. ![Screen Shot
2025-02-06 at 14 04 40
PM](https://github.com/user-attachments/assets/8b72f04a-c62a-49b5-810f-7ddd472884d0)
* Autocompletion and documentation hovering works (except the
limitations mentioned above)
* Zed:
* Very similar to VS Code with the exception that it uses the
description attribute to distinguish settings with multiple possible
values ![Screen Shot 2025-02-06 at 14 08 19
PM](https://github.com/user-attachments/assets/78a7f849-ff4e-44ff-8317-708eaf02dc1f)


I don't think there's much we can do here other than hope (or help)
editors improve their auto completion. The same short comings also apply
to ruff, so this isn't something new. For now, I think this is good
enough
2025-02-07 10:59:40 +01:00
InSync 7db5a924af
[`flake8-comprehensions`] Detect overshadowed `list`/`set`/`dict`, ignore variadics and named expressions (`C417`) (#15955)
## Summary

Part of #15809 and #15876.

This change brings several bugfixes:

* The nested `map()` call in `list(map(lambda x: x, []))` where `list`
is overshadowed is now correctly reported.
* The call will no longer reported if:
	* Any arguments given to `map()` are variadic.
	* Any of the iterables contain a named expression.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-07 08:58:05 +00:00
Junhson Jean-Baptiste 349f93389e
[flake8-simplify] Only trigger SIM401 on known dictionaries (SIM401) (#15995)
## Summary

This change resolves #15814 to ensure that `SIM401` is only triggered on
known dictionary types. Before, the rule was getting triggered even on
types that _resemble_ a dictionary but are not actually a dictionary.

I did this using the `is_known_to_be_of_type_dict(...)` functionality.
The logic for this function was duplicated in a few spots, so I moved
the code to a central location, removed redundant definitions, and
updated existing calls to use the single definition of the function!

## Test Plan

Since this PR only modifies an existing rule, I made changes to the
existing test instead of adding new ones. I made sure that `SIM401` is
triggered on types that are clearly dictionaries and that it's not
triggered on a simple custom dictionary-like type (using a modified
version of [the code in the issue](#15814))

The additional changes to de-duplicate `is_known_to_be_of_type_dict`
don't break any existing tests -- I think this should be fine since the
logic remains the same (please let me know if you think otherwise, I'm
excited to get feedback and work towards a good fix 🙂).

---------

Co-authored-by: Junhson Jean-Baptiste <junhsonjb@naan.mynetworksettings.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-07 08:25:20 +00:00
InSync bb979e05ac
[`flake8-pie`] Remove following comma correctly when the unpacked dictionary is empty (`PIE800`) (#16008)
## Summary

Resolves #15997.

Ruff used to introduce syntax errors while fixing these cases, but no
longer will:

```python
{"a": [], **{},}
#         ^^^^ Removed, leaving two contiguous commas

{"a": [], **({})}
#         ^^^^^ Removed, leaving a stray closing parentheses
```

Previously, the function would take a shortcut if the unpacked
dictionary is empty; now, both cases are handled using the same logic
introduced in #15394. This change slightly modifies that logic to also
remove the first comma following the dictionary, if and only if it is
empty.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-07 08:52:10 +01:00
Dylan 10d3e64ccd
Bump version to 0.9.5 (#16002) 2025-02-06 13:24:45 -06:00
InSync 84ceddcbd9
[`ruff`] Classes with mixed type variable style (`RUF053`) (#15841) 2025-02-06 18:35:51 +00:00
Ayush Baweja ba2f0e998d
[flake8-pyi] Add autofix for unused-private-type-var (PYI018) (#15999)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-06 18:08:36 +00:00
Alex Waygood 18b497a913
[red-knot] Fixup a couple of nits in the `red_knot_test` README (#15996) 2025-02-06 15:04:27 +00:00
Micha Reiser 7cac0da44d
Workaround Even Better TOML crash related to `allOf` (#15992)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/15978

Even Better TOML doesn't support `allOf` well. In fact, it just crashes.

This PR works around this limitation by avoid using `allOf` in the
automatically
derived schema for the docstring formatting setting. 

### Alternatives

schemars introduces `allOf` whenver it sees a `$ref` alongside other
object properties
because this is no longer valid according to Draft 7. We could replace
the
visitor performing the rewrite but I prefer not to because replacing
`allOf` with `oneOf`
is only valid for objects that don't have any other `oneOf` or `anyOf`
schema.

## Test Plan


https://github.com/user-attachments/assets/25d73b2a-fee1-4ba6-9ffe-869b2c3bc64e
2025-02-06 16:00:50 +01:00
David Peter e345307260
[red-knot] Fix diagnostic range for non-iterable unpacking assignments (#15994)
## Summary

I noticed that the diagnostic range in specific unpacking assignments is
wrong. For this example

```py
a, b = 1
```

we previously got (see first commit):

```
error: lint:not-iterable
 --> /src/mdtest_snippet.py:1:1
  |
1 | a, b = 1
  | ^^^^ Object of type `Literal[1]` is not iterable
  |
```

and with this change, we get:

```
error: lint:not-iterable
 --> /src/mdtest_snippet.py:1:8
  |
1 | a, b = 1
  |        ^ Object of type `Literal[1]` is not iterable
  |
```

## Test Plan

New snapshot tests.
2025-02-06 15:36:22 +01:00
Micha Reiser 5588c75d65
[red-knot] Fix relative imports in `src.root` (#15990)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/15989

Red Knot failed to resolve relative imports if the importing module is
located at a search path root.

The issue was that the module resolver returned an `Err(TooManyDots)` as
soon as the parent of the current module is `None` (which is the case
for a module at the search path root).
However, this is incorrect if a `tail` (a module name) exists.
2025-02-06 14:08:20 +00:00
Raymond Berger 9d2105b863
add instance variable examples to RUF012 (#15982)
## Summary

Closes #15804 

Add more examples to the documentation.

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-06 14:01:09 +00:00
David Salvisberg 8fcac0ff36
Recognize all symbols named `TYPE_CHECKING` for `in_type_checking_block` (#15719)
Closes #15681

## Summary

This changes `analyze::typing::is_type_checking_block` to recognize all
symbols named "TYPE_CHECKING".
This matches the current behavior of mypy and pyright as well as
`flake8-type-checking`.

It also drops support for detecting `if False:` and `if 0:` as type
checking blocks. This used to be an option for
providing backwards compatibility with Python versions that did not have
a `typing` module, but has since
been removed from the typing spec and is no longer supported by any of
the mainstream type checkers.

## Test Plan

`cargo nextest run`

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-06 14:45:12 +01:00
Vasco Schiavo 81059d05fc
[`pep8-naming`] Consider any number of leading underscore for `N801` (#15988)
## Summary

The PR addresses the issue #15939 

Let me know if you think there are other test cases I should add ;-)
2025-02-06 14:08:27 +05:30
Vasco Schiavo 24bab7e82e
[pycodestyle] Exempt `sys.path += ...` calls (E402) (#15980)
## Summary

The PR addresses issue #15886 .
2025-02-06 08:51:51 +01:00
David Peter d0555f7b5c
[red-knot] Litate tests: minor follow-up (#15987)
## Summary

- Minor wording update
- Code improvement (thanks Alex)
- Removed all unnecessary filenames throughout our Markdown tests (two
new ones were added in the meantime)
- Minor rewording of the statically-known-branches introduction
2025-02-06 07:15:26 +00:00
Douglas Creager 0906554357
[red-knot] Combine terminal statement support with statically known branches (#15817)
This example from @sharkdp shows how terminal statements can appear in
statically known branches:
https://github.com/astral-sh/ruff/pull/15676#issuecomment-2618809716

```py
def _(cond: bool):
    x = "a"
    if cond:
        x = "b"
        if True:
            return

    reveal_type(x)  # revealed: "a", "b"; should be "a"
```

We now use visibility constraints to track reachability, which allows us
to model this correctly. There are two related changes as a result:

- New bindings are not assumed to be visible; they inherit the current
"scope start" visibility, which effectively means that new bindings are
visible if/when the current flow is reachable

- When simplifying visibility constraints after branching control flow,
we only simplify if none of the intervening branches included a terminal
statement. That is, earlier unaffected bindings are only _actually_
unaffected if all branches make it to the merge point.
2025-02-05 17:47:49 -05:00
David Peter d296f602e7
[red-knot] Merge Markdown code blocks inside a single section (#15950)
## Summary

Allow for literate style in Markdown tests and merge multiple (unnamed)
code blocks into a single embedded file.

closes #15941

## Test Plan

- Interactively made sure that error-lines were reported correctly in
  multi-snippet sections.
2025-02-05 22:26:15 +01:00
Andrew Gallant d47088c8f8
[red-knot] fix unresolvable import range (#15976)
This causes the diagnostic to highlight the actual unresovable import
instead of the entire `from ... import ...` statement.

While we're here, we expand the test coverage to cover all of the
possible ways that an `import` or a `from ... import` can fail.

Some considerations:

* The first commit in this PR adds a regression test for the current
behavior.
* This creates a new `mdtest/diagnostics` directory. Are folks cool
with this? I guess the idea is to put tests more devoted to diagnostics
than semantics in this directory. (Although I'm guessing there will
be some overlap.)

Fixes #15866
2025-02-05 14:01:58 -05:00
David Peter 1f0ad675d3
[red-knot] Initial set of descriptor protocol tests (#15972)
## Summary

This is a first step towards creating a test suite for
[descriptors](https://docs.python.org/3/howto/descriptor.html). It does
not (yet) aim to be exhaustive.

relevant ticket: #15966 

## Test Plan

Compared desired behavior with the runtime behavior and the behavior of
existing type checkers.

---------

Co-authored-by: Mike Perlov <mishamsk@gmail.com>
2025-02-05 19:47:43 +01:00
Andrew Gallant a84b27e679 red_knot_test: add support for diagnostic snapshotting
This ties together everything from the previous commits.
Some interesting bits here are how the snapshot is generated
(where we include relevant info to make it easier to review
the snapshots) and also a tweak to how inline assertions are
processed.

This commit also includes some example snapshots just to get
a sense of what they look like. Follow-up work should add
more of these I think.
2025-02-05 13:02:54 -05:00
Andrew Gallant 8d4679b3ae red_knot_test: update README with section on diagnostic snapshotting
I split this out into a separate commit and put it here
so that reviewers can get a conceptual model of what the
code is doing before seeing the code. (Hopefully that helps.)
2025-02-05 13:02:54 -05:00
Andrew Gallant b40a7cce15 red_knot_test: add snapshot path
This makes it possible for callers to set where snapshots
should be stored. In general, I think we expect this to
always be set, since otherwise snapshots will end up in
`red_knot_test`, which is where the tests are actually run.
But that's overall counter-intuitive. This permits us to
store snapshots from mdtests alongside the mdtests themselves.
2025-02-05 13:02:54 -05:00
Andrew Gallant 54b3849dfb ruff_db: add more `dyn Diagnostic` impls
I found it useful to have the `&dyn Diagnostic` trait impl
specifically. I added `Arc<dyn Diagnostic>` for completeness.

(I do kind of wonder if we should be preferring `Arc<dyn ...>`
over something like `Box<dyn ...>` more generally, especially
for things with immutable APIs. It would make cloning cheap.)
2025-02-05 13:02:54 -05:00
Andrew Gallant ffd94e9ace red_knot_test: generate names for unnamed files using more local reasoning
This change was done to reduce snapshot churn. Previously,
if one added a new section to an Markdown test suite, then
the snapshots of all sections with unnamed files below it would
necessarily change because of the unnamed file count being
global to the test suite.

Instead, we track counts based on section. While adding new
unnamed files within a section will still change unnamed
files below it, I believe this will be less "churn" because
the snapshot will need to change anyway. Some churn is still
possible, e.g., if code blocks are re-ordered. But I think this
is an acceptable trade-off.
2025-02-05 13:02:54 -05:00
Alex Waygood c816542704
[red-knot] Fix some instance-attribute TODOs around `ModuleType` (#15974) 2025-02-05 15:33:37 +00:00
Alex Waygood 2ebb5e8d4b
[red-knot] Make `Symbol::or_fall_back_to()` lazy (#15943) 2025-02-05 14:51:02 +00:00
Dylan c69b19fe1d
[`flake8-comprehensions`] Handle trailing comma in fixes for `unnecessary-generator-list/set` (`C400`,`C401`) (#15929)
The unsafe fixes for the rules [unnecessary-generator-list
(C400)](https://docs.astral.sh/ruff/rules/unnecessary-generator-list/#unnecessary-generator-list-c400)
and [unnecessary-generator-set
(C401)](https://docs.astral.sh/ruff/rules/unnecessary-generator-set/#unnecessary-generator-set-c401)
used to introduce syntax errors if the argument to `list` or `set` had a
trailing comma, because the fix would retain the comma after
transforming the function call to a comprehension.

This PR accounts for the trailing comma when replacing the end of the
call with a `]` or `}`.

Closes #15852
2025-02-05 07:38:03 -06:00
Brent Westbrook 076d35fb93
[minor] Mention UP049 in UP046 and UP047, add `See also` section to UP040 (#15956)
## Summary

Minor docs follow-up to #15862 to mention UP049 in the UP046 and UP047
`See also` sections. I wanted to mention it in UP040 too but realized it
didn't have a `See also` section, so I also added that, adapted from the
other two rules.

## Test Plan

cargo test
2025-02-05 08:34:47 -05:00
Dylan 16f2a93fca
[`ruff`] Analyze deferred annotations before enforcing `mutable-(data)class-default` and `function-call-in-dataclass-default-argument` (`RUF008`,`RUF009`,`RUF012`) (#15921) 2025-02-05 06:44:19 -06:00
David Peter eb08345fd5
[red-knot] Extend instance/class attribute tests (#15959)
## Summary

In preparation for creating some (sub) issues for
https://github.com/astral-sh/ruff/issues/14164, I'm trying to document
the current behavior (and a bug) a bit better.
2025-02-05 12:45:00 +01:00
Alex Waygood 7ca778f492
[`refurb`] Minor nits regarding `for-loop-writes` and `for-loop-set-mutations` (#15958) 2025-02-05 10:21:36 +00:00
Vasco Schiavo 827a076a2f
[pylint] Fix PL1730: min/max auto-fix and suggestion (#15930)
## Summary

The PR addresses the issue #15887 

For two objects `a` and `b`, we ensure that the auto-fix and the
suggestion is of the form `a = min(a, b)` (or `a = max(a, b)`). This is
because we want to be consistent with the python implementation of the
methods: `min` and `max`. See the above issue for more details.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-05 09:29:10 +00:00
InSync 4855e0b288
[`refurb`] Handle unparenthesized tuples correctly (`FURB122`, `FURB142`) (#15953)
## Summary

Resolves #15936.

The fixes will now attempt to preserve the original iterable's format
and quote it if necessary. For `FURB142`, comments within the fix range
will make it unsafe as well.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-05 10:16:54 +01:00
InSync 44ddd98d7e
[`pyupgrade`] Better messages and diagnostic range (`UP015`) (#15872)
## Summary

Resolves #15863.

In preview, diagnostic ranges will now be limited to that of the
argument. Rule documentation, variable names, error messages and fix
titles have all been modified to use "argument" consistently.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-05 09:44:26 +01:00
InSync 82cb8675dd
[`pep8-naming`] Ignore `@override` methods (`N803`) (#15954)
## Summary

Resolves #15925.

`N803` now checks for functions instead of parameters. In preview mode,
if a method is decorated with `@override` and the current scope is that
of a class, it will be ignored.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-05 09:35:57 +01:00
InSync 5852217198
[`refurb`] Also report non-name expressions (`FURB169`) (#15905)
## Summary

Follow-up to #15779.

Prior to this change, non-name expressions are not reported at all:

```python
type(a.b) is type(None)  # no error
```

This change enhances the rule so that such cases are also reported in
preview. Additionally:

* The fix will now be marked as unsafe if there are any comments within
its range.
* Error messages are slightly modified.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-05 08:46:37 +01:00
Dylan 700e969c56
Config error only when `flake8-import-conventions` alias conflicts with `isort.required-imports` bound name (#15918)
Previously an error was emitted any time the configuration required both
an import of a module and an alias for that module. However, required
imports could themselves contain an alias, which may or may not agree
with the required alias.

To wit: requiring `import pandas as pd` does not conflict with the
`flake8-import-conventions.alias` config `{"pandas":"pd"}`.

This PR refines the check before throwing an error.

Closes #15911
2025-02-04 17:05:35 -06:00
InSync 4c15d7a559
Fix a typo in `non_pep695_generic_class.rs` (#15946)
(Accidentally introduced in #15904.)
2025-02-04 22:16:18 +00:00
Mike Perlov e15419396c
[red-knot] Fix Stack overflow in Type::bool (#15843)
## Summary

This PR adds `Type::call_bound` method for calls that should follow
descriptor protocol calling convention. The PR is intentionally shallow
in scope and only fixes #15672

Couple of obvious things that weren't done:

* Switch to `call_bound` everywhere it should be used
* Address the fact, that red_knot resolves `__bool__ = bool` as a Union,
which includes `Type::Dynamic` and hence fails to infer that the
truthiness is always false for such a class (I've added a todo comment
in mdtests)
* Doesn't try to invent a new type for descriptors, although I have a
gut feeling it may be more convenient in the end, instead of doing
method lookup each time like I did in `call_bound`

## Test Plan

* extended mdtests with 2 examples from the issue
* cargo neatest run
2025-02-04 12:40:07 -08:00
Douglas Creager 444b055cec
[red-knot] Use ternary decision diagrams (TDDs) for visibility constraints (#15861)
We now use ternary decision diagrams (TDDs) to represent visibility
constraints. A TDD is just like a BDD ([_binary_ decision
diagram](https://en.wikipedia.org/wiki/Binary_decision_diagram)), but
with "ambiguous" as an additional allowed value. Unlike the previous
representation, TDDs are strongly normalizing, so equivalent ternary
formulas are represented by exactly the same graph node, and can be
compared for equality in constant time.

We currently have a slight 1-3% performance regression with this in
place, according to local testing. However, we also have a _5× increase_
in performance for pathological cases, since we can now remove the
recursion limit when we evaluate visibility constraints.

As follow-on work, we are now closer to being able to remove the
`simplify_visibility_constraint` calls in the semantic index builder. In
the vast majority of cases, we now see (for instance) that the
visibility constraint after an `if` statement, for bindings of symbols
that weren't rebound in any branch, simplifies back to `true`. But there
are still some cases we generate constraints that are cyclic. With
fixed-point cycle support in salsa, or with some careful analysis of the
still-failing cases, we might be able to remove those.
2025-02-04 14:32:11 -05:00
Brent Westbrook 6bb32355ef
[`pyupgrade`] Rename private type parameters in PEP 695 generics (`UP049`) (#15862)
## Summary

This is a new rule to implement the renaming of PEP 695 type parameters
with leading underscores after they have (presumably) been converted
from standalone type variables by either UP046 or UP047. Part of #15642.

I'm not 100% sure the fix is always safe, but I haven't come up with any
counterexamples yet. `Renamer` seems pretty precise, so I don't think
the usual issues with comments apply.

I initially tried writing this as a rule that receives a `Stmt` rather
than a `Binding`, but in that case the
`checker.semantic().current_scope()` was the global scope, rather than
the scope of the type parameters as I needed. Most of the other rules
using `Renamer` also used `Binding`s, but it does have the downside of
offering separate diagnostics for each parameter to rename.

## Test Plan

New snapshot tests for UP049 alone and the combination of UP046, UP049,
and PYI018.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-04 13:22:57 -05:00
Alex Waygood cb71393332
Simplify the `StringFlags` trait (#15944) 2025-02-04 18:14:28 +00:00
Alex Waygood 64e64d2681
[`flake8-pyi`] Make `PYI019` autofixable for `.py` files in preview mode as well as stubs (#15889) 2025-02-04 16:41:22 +00:00
Alex Waygood 5bf0e2e95e
[`flake8-pyi`] Make PEP-695 functions with multiple type parameters fixable by PYI019 again (#15938) 2025-02-04 14:38:22 +00:00
David Peter 24c1cf71cb
[red-knot] Use unambiguous invalid-syntax-construct for suppression comment test (#15933)
## Summary

I experimented with [not trimming trailing newlines in code
snippets](https://github.com/astral-sh/ruff/pull/15926#discussion_r1940992090),
but since came to the conclusion that the current behavior is better
because otherwise, there is no way to write snippets without a trailing
newline at all. And when you copy the code from a Markdown snippet in
GitHub, you also don't get a trailing newline.

I was surprised to see some test failures when I played with this
though, and decided to make this test independent from this
implementation detail.
2025-02-04 15:24:50 +01:00
Alex Waygood f23802e219
Make `Binding::range()` point to the range of a type parameter's name, not the full type parameter (#15935)
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2025-02-04 14:14:21 +00:00
David Peter cc60701b59
[red-knot] MDTest: Fix line numbers in error messages (#15932)
## Summary

Fix line number reporting in MDTest error messages.

## Test Plan

Introduced an error in a Markdown test and made sure that the line in
the error message matches.
2025-02-04 13:44:05 +00:00
Brent Westbrook b5e5271adf
Preserve triple quotes and prefixes for strings (#15818)
## Summary

This is a follow-up to #15726, #15778, and #15794 to preserve the triple
quote and prefix flags in plain strings, bytestrings, and f-strings.

I also added a `StringLiteralFlags::without_triple_quotes` method to
avoid passing along triple quotes in rules like SIM905 where it might
not make sense, as discussed
[here](https://github.com/astral-sh/ruff/pull/15726#discussion_r1930532426).

## Test Plan

Existing tests, plus many new cases in the `generator::tests::quote`
test that should cover all combinations of quotes and prefixes, at least
for simple string bodies.

Closes #7799 when combined with #15694, #15726, #15778, and #15794.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-04 08:41:06 -05:00
David Peter 9a33924a65
[red-knot] Hand-written MDTest parser (#15926)
## Summary

Replaces our existing Markdown test parser with a fully hand-written
parser. I tried to fix this bug using the old approach and kept running
into problems. Eventually this seemed like the easier way. It's more
code (+50 lines, excluding the new test), but I hope it's relatively
straightforward to understand, compared to the complex interplay between
the byte-stream-manipulation and regex-parsing that we had before.

I did not really focus on performance, as the parsing time does not
dominate the test execution time, but this seems to be slightly faster
than what we had before (executing all MD tests; debug):

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| this branch | 2.775 ± 0.072 | 2.690 | 2.877 | 1.00 |
| `main` | 2.921 ± 0.034 | 2.865 | 2.967 | 1.05 ± 0.03 |

closes #15923

## Test Plan

One new regression test.
2025-02-04 14:01:53 +01:00
Mike Perlov 15dd3b5ebd
[`pylint`] Fix missing parens in unsafe fix for `unnecessary-dunder-call` (`PLC2801`) (#15762)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-04 09:54:01 +00:00
Alexander Nordin b848afeae8
nit: docs for ignore & select (#15883) 2025-02-04 10:05:41 +01:00
Wei Lee de4d9979eb
[airflow] `BashOperator` has been moved to `airflow.providers.standard.operators.bash.BashOperator` (AIR302) (#15922)
## Summary

Extend AIR302 with 

* `airflow.operators.bash.BashOperator →
airflow.providers.standard.operators.bash.BashOperator`
* change existing rules `airflow.operators.bash_operator.BashOperator →
airflow.operators.bash.BashOperator` to
`airflow.operators.bash_operator.BashOperator →
airflow.providers.standard.operators.bash.BashOperator`

## Test Plan

a test fixture has been updated
2025-02-04 14:28:00 +05:30
InSync ba02294af3
[`flake8-logging`] `.exception()` and `exc_info=` outside exception handlers (`LOG004`, `LOG014`) (#15799) 2025-02-04 09:52:12 +01:00
InSync 11cfe2ea8a
[red-knot] Enforce specifying paths for mdtest code blocks in a separate preceding line (#15890)
## Summary

Resolves #15695, rework of #15704.

This change modifies the Mdtests framework so that:

* Paths must now be specified in a separate preceding line:

	`````markdown
	`a.py`:

	```py
	x = 1
	```
	`````

If the path of a file conflicts with its `lang`, an error will be
thrown.

* Configs are no longer accepted. The pattern still take them into
account, however, to avoid "Unterminated code block" errors.
* Unnamed files are now assigned unique, `lang`-respecting paths
automatically.

Additionally, all legacy usages have been updated.

## Test Plan

Unit tests and Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-02-04 08:27:17 +01:00
Douglas Creager 0529ad67d7
[red-knot] Internal refactoring of visibility constraints API (#15913)
This extracts some pure refactoring noise from
https://github.com/astral-sh/ruff/pull/15861. This changes the API for
creating and evaluating visibility constraints, but does not change how
they are respresented internally. There should be no behavioral or
performance changes in this PR.

Changes:

- Hide the internal representation isn't changed, so that we can make
changes to it in #15861.
- Add a separate builder type for visibility constraints. (With TDDs, we
will have some additional builder state that we can throw away once
we're done constructing.)
- Remove a layer of helper methods from `UseDefMapBuilder`, making
`SemanticIndexBuilder` responsible for constructing whatever visibility
constraints it needs.
2025-02-03 15:13:09 -05:00
David Peter 102c2eec12
[red-knot] Implicit instance attributes (#15811)
## Summary

Add support for implicitly-defined instance attributes, i.e. support
type inference for cases like this:
```py
class C:
    def __init__(self) -> None:
        self.x: int = 1
        self.y = None

reveal_type(C().x)  # int
reveal_type(C().y)  # Unknown | None
```

## Benchmarks

Codspeed reports no change in a cold-cache benchmark, and a -1%
regression in the incremental benchmark. On `black`'s `src` folder, I
don't see a statistically significant difference between the branches:

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./red_knot_main check --project /home/shark/black/src` | 133.7 ± 9.5 | 126.7 | 164.7 | 1.01 ± 0.08 |
| `./red_knot_feature check --project /home/shark/black/src` | 132.2 ± 5.1 | 118.1 | 140.9 | 1.00 |

## Test Plan

Updated and new Markdown tests
2025-02-03 19:34:23 +01:00
Justin Bramley dc5e922221
[`flake8-comprehensions`] Handle extraneous parentheses around list comprehension (`C403`) (#15877)
## Summary

Given the following code:

```python
set(([x for x in range(5)]))
```

the current implementation of C403 results in

```python
{(x for x in range(5))}
```

which is a set containing a generator rather than the result of the
generator.

This change removes the extraneous parentheses so that the resulting
code is:

```python
{x for x in range(5)}
```


## Test Plan

`cargo nextest run` and `cargo insta test`
2025-02-03 13:26:03 -05:00
Alex Waygood 62075afe4f
[flake8-pyi] Significantly improve accuracy of `PYI019` if preview mode is enabled (#15888) 2025-02-03 15:45:10 +00:00
InSync dfe1b849d0
Convert `.md` links in rule documentation to full URLs (#15904) 2025-02-03 15:33:03 +01:00
Alex Waygood 9c64d65552
[`flake8-pyi`] Rename `PYI019` and improve its diagnostic message (#15885) 2025-02-03 14:23:58 +00:00
Vasco Schiavo 83243de93d
Improve Docs: Pylint subcategories' codes (#15909) 2025-02-03 13:53:36 +01:00
renovate[bot] 638186afbd
Update Rust crate rand to 0.9.0 (#15899)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-03 12:25:57 +01:00
Dhruv Manilawala d082c1b202
[red-knot] Add missing imports in mdtests (#15869)
## Summary

Related to #15848, this PR adds the imports explicitly as we'll now flag
these symbols as undefined.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-03 09:27:29 +00:00
InSync 30d5e9a2af
[red-knot] Support `--exit-zero` and `--error-on-warning` (#15746)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-03 07:35:30 +00:00
Alex Waygood b08ce5fb18
[`flake8-pyi`] Minor cosmetic changes to `PYI019` (#15881) 2025-02-02 19:20:05 +00:00
Alex Waygood 418aa35041
[`flake8-pyi`] Avoid an unnecessary `.unwrap()` call in `PYI019` autofix (#15880) 2025-02-02 19:04:41 +00:00
Tom Kuson 813a76e9e2
[red-knot] Add version command (#15823)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-02 18:56:51 +00:00
InSync 3c09100484
[`flake8-pyi`] Fix more complex cases (`PYI019`) (#15821)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-02 18:38:49 +00:00
Micha Reiser 770b7f3439
Vendor benchmark test files (#15878)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-02 18:16:07 +00:00
Alex Waygood d9a1034db0
Add convenience helper methods for AST nodes representing function parameters (#15871) 2025-02-01 17:16:32 +00:00
Alex Waygood bcdb3f9840
Use `Diagnostic::try_set_fix` in `bad-generator-return-type` (#15873) 2025-02-01 15:44:42 +00:00
github-actions[bot] 942d7f395a
Sync vendored typeshed stubs (#15864)
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
2025-02-01 01:01:58 +00:00
Andrew Gallant b58f2c399e
[red-knot] ruff_db: make diagnostic rendering prettier (#15856)
This change does a simple swap of the existing renderer for one that
uses our vendored copy of `annotate-snippets`. We don't change anything
about the diagnostic data model, but this alone already makes
diagnostics look a lot nicer!
2025-01-31 16:37:02 -05:00
Douglas Creager fab86de3ef
[red-knot] Should A ∧ !A always be false? (#15839)
This mimics a simplification we have on the OR side, where we simplify
`A ∨ !A` to true. This requires changes to how we add `while` statements
to the semantic index, since we now need distinct
`VisibilityConstraint`s if we need to model evaluating a `Constraint`
multiple times at different points in the execution of the program.
2025-01-31 14:06:52 -05:00
Alex Waygood c5c0b724fb
[`flake8-pyi`] Minor simplification for `PYI019` (#15855) 2025-01-31 16:54:38 +00:00
Alex Waygood 0d191a13c1
[`flake8-pyi`] Fix incorrect behaviour of `custom-typevar-return-type` preview-mode autofix if `typing` was already imported (`PYI019`) (#15853) 2025-01-31 16:46:31 +00:00
InSync b2cb757fa8
[`flake8-pyi`] Remove type parameter correctly when it is the last (`PYI019`) (#15854) 2025-01-31 16:22:54 +00:00
Carl Meyer ce769f6ae2
[red-knot] gather type prevalence statistics (#15834)
Something Alex and I threw together during our 1:1 this morning. Allows
us to collect statistics on the prevalence of various types in a file,
most usefully TODO types or other dynamic types.
2025-01-31 07:10:00 -08:00
Alex Waygood 44ac17b3ba
[`flake8-pyi`] Fix several correctness issues with `custom-type-var-return-type` (`PYI019`) (#15851) 2025-01-31 14:19:35 +00:00
Brent Westbrook f1418be81c
[`pyupgrade`] Reuse replacement logic from `UP046` and `UP047` (`UP040`) (#15840)
## Summary

This is a follow-up to #15565, tracked in #15642, to reuse the string
replacement logic from the other PEP 695 rules instead of the
`Generator`, which has the benefit of preserving more comments. However,
comments in some places are still dropped, so I added a check for this
and update the fix safety accordingly. I also added a `## Fix safety`
section to the docs to reflect this and the existing `isinstance`
caveat.

## Test Plan

Existing UP040 tests, plus some new cases.
2025-01-31 08:10:53 -05:00
InSync 59be5f5278
[`refurb`] Avoid `None | None` as well as better detection and fix (`FURB168`) (#15779) 2025-01-31 11:34:57 +00:00
InSync 172f62d8f4
[`refurb`] Mark fix as unsafe if there are comments (`FURB171`) (#15832)
## Summary

Resolves #10063 and follow-up to #15521.

The fix is now marked as unsafe if there are any comments within its
range. Tests are adapted from that of #15521.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-30 17:21:07 -06:00
Dylan 071862af5a
[`flake8-comprehensions`] Skip when `TypeError` present from too many (kw)args for `C410`,`C411`, and `C418` (#15838)
Both `list` and `dict` expect only a single positional argument. Giving
more positional arguments, or a keyword argument, is a `TypeError` and
neither the lint rule nor its fix make sense in that context.

Closes #15810
2025-01-30 17:10:43 -06:00
Brent Westbrook fe516e24f5
[`pyflakes`] Visit forward annotations in `TypeAliasType` as types (`F401`) (#15829)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/15812 by visiting the
second argument as a type definition.

## Test Plan

New F401 tests based on the report.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-30 18:06:38 -05:00
Dylan 4f2aea8d50
[`flake8-comprehensions`] Handle builtins at top of file correctly for `unnecessary-dict-comprehension-for-iterable` (`C420`) (#15837)
Builtin bindings are given a range of `0..0`, which causes strange
behavior when range checks are made at the top of the file. In this
case, the logic of the rule demands that the value of the dict
comprehension is not self-referential (i.e. it does not contain
definitions for any of the variables used within it). This logic was
confused by builtins which looked like they were defined "in the
comprehension", if the comprehension appeared at the top of the file.

Closes #15830
2025-01-30 15:49:13 -06:00
Dylan 854ab03078
Bump version to 0.9.4 (#15831)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-30 11:29:22 -06:00
Leo Gaskin b0b8b06241
Remove semicolon after TypeScript interface definition (#15827)
## Summary

This PR removes a trailing semicolon after an interface definition in
the custom TypeScript section of `ruff_wasm`. Currently, this semicolon
triggers the error "TS1036: Statements are not allowed in ambient
contexts" when including the file and compiling with e.g `tsc`.

## Test Plan

I made the change, ran `wasm-pack` and copied the generated directory
manually to my `node_modules` folder. I then compiled a file importing
`@astral-sh/ruff-wasm-web` again and confirmed that the compilation
error was gone.
2025-01-30 20:40:16 +05:30
David Peter 451f251a31
[red-knot] Clarify behavior when redeclaring base class attributes (#15826)
# Summary

Clarify the behavior regarding re-declaration of attributes from base
classes following [this
discussion](https://github.com/astral-sh/ruff/pull/15808#discussion_r1934236095)
2025-01-30 14:49:23 +01:00
Dylan 13cf3e65f1
[`flake8-comprehensions`] Parenthesize `sorted` when needed for `unnecessary-call-around-sorted` (`C413`) (#15825)
If there is any `ParenthesizedWhitespace` (in the sense of LibCST) after
the function name `sorted` and before the arguments, then we must wrap
`sorted` with parentheses after removing the surrounding function.

Closes #15789
2025-01-30 07:10:56 -06:00
Dylan 56f956a238
[`pyupgrade`] Handle end-of-line comments for `quoted-annotation` (`UP037`) (#15824)
This PR uses the tokens of the parsed annotation available in the
`Checker`, instead of re-lexing (using `SimpleTokenizer`) the
annotation. This avoids some limitations of the `SimpleTokenizer`, such
as not being able to handle number and string literals.

Closes #15816 .
2025-01-30 00:03:05 -06:00
Tom Kuson 7a10a40b0d
[`flake8-bandit`] Permit suspicious imports within stub files (`S4`) (#15822)
## Summary

Permits suspicious imports (the `S4` namespaced diagnostics) from stub
files.

Closes #15207.

## Test Plan

Added tests and ran `cargo nextest run`. The test files are copied from
the `.py` variants.
2025-01-29 23:42:56 -06:00
Alex Waygood 3125332ec1
[red-knot] Format mdtest snippets with the latest version of black (#15819) 2025-01-29 23:05:43 +00:00
Douglas Creager 15d886a502
[red-knot] Consider all definitions after terminal statements unreachable (#15676)
`FlowSnapshot` now tracks a `reachable` bool, which indicates whether we
have encountered a terminal statement on that control flow path. When
merging flow states together, we skip any that have been marked
unreachable. This ensures that bindings that can only be reached through
unreachable paths are not considered visible.

## Test Plan

The new mdtests failed (with incorrect `reveal_type` results, and
spurious `possibly-unresolved-reference` errors) before adding the new
visibility constraints.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-01-29 14:06:57 -05:00
InSync e1c9d10863
[`flake8-comprehensions`] Do not emit `unnecessary-map` diagnostic when lambda has different arity (`C417`) (#15802) 2025-01-29 18:45:55 +00:00
Brent Westbrook 23c98849fc
Preserve quotes in generated f-strings (#15794)
## Summary

This is another follow-up to #15726 and #15778, extending the
quote-preserving behavior to f-strings and deleting the now-unused
`Generator::quote` field.

## Details
I also made one unrelated change to `rules/flynt/helpers.rs` to remove a
`to_string` call for making a `Box<str>` and tweaked some arguments to
some of the `Generator::unparse_f_string` methods to make the code
easier to follow, in my opinion. Happy to revert especially the latter
of these if needed.

Unfortunately this still does not fix the issue in #9660, which appears
to be more of an escaping issue than a quote-preservation issue. After
#15726, the result is now `a = f'# {"".join([])}' if 1 else ""` instead
of `a = f"# {''.join([])}" if 1 else ""` (single quotes on the outside
now), but we still don't have the desired behavior of double quotes
everywhere on Python 3.12+. I added a test for this but split it off
into another branch since it ended up being unaddressed here, but my
`dbg!` statements showed the correct preferred quotes going into
[`UnicodeEscape::with_preferred_quote`](https://github.com/astral-sh/ruff/blob/main/crates/ruff_python_literal/src/escape.rs#L54).

## Test Plan

Existing rule and `Generator` tests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-29 13:28:22 -05:00
InSync d151ca85d3
[`pyupgrade`] Ignore `is_typeddict` and `TypedDict` for `deprecated-import` (`UP035`) (#15800) 2025-01-29 18:05:46 +00:00
Garrett Reynolds 6c1e19592e
[`ruff`] Add support for more `re` patterns (`RUF055`) (#15764)
## Summary
Implements some of #14738, by adding support for 6 new patterns:
```py
re.search("abc", s) is None       # ⇒ "abc" not in s
re.search("abc", s) is not None   # ⇒ "abc" in s

re.match("abc", s) is None       # ⇒ not s.startswith("abc")  
re.match("abc", s) is not None   # ⇒ s.startswith("abc")

re.fullmatch("abc", s) is None       # ⇒ s != "abc"
re.fullmatch("abc", s) is not None   # ⇒ s == "abc"
```


## Test Plan

```shell
cargo nextest run
cargo insta review
```

And ran the fix on my startup's repo.


## Note

One minor limitation here:

```py
if not re.match('abc', s) is None:
    pass
```

will get fixed to this (technically correct, just not nice):
```py
if not not s.startswith('abc'):
    pass
```

This seems fine given that Ruff has this covered: the initial code
should be caught by
[E714](https://docs.astral.sh/ruff/rules/not-is-test/) and the fixed
code should be caught by
[SIM208](https://docs.astral.sh/ruff/rules/double-negation/).
2025-01-29 10:14:44 -05:00
David Peter 0f1035b930
[red-knot] Extend instance-attribute tests (#15808)
## Summary

When we discussed the plan on how to proceed with instance attributes,
we said that we should first extend our research into the behavior of
existing type checkers. The result of this research is summarized in the
newly added / modified tests in this PR. The TODO comments align with
existing behavior of other type checkers. If we deviate from the
behavior, it is described in a comment.
2025-01-29 14:06:32 +01:00
Marek Hanuš 2c3d889dbb
Fix formatter warning message for `flake8-quotes` option (#15788)
## Summary

Fix wrong option name in warning message about docstring quotes
incompatibility.

## Test Plan

Only in CI. No manual testing.
2025-01-29 16:30:28 +05:30
InSync 4bec8ba731
[`flake8-bugbear`] Exempt `NewType` calls where the original type is immutable (`B008`) (#15765)
## Summary

Resolves #12717.

This change incorporates the logic added in #15588.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2025-01-29 10:26:17 +00:00
Mike Perlov 6090408f65
Add missing config docstrings (#15803)
## Summary

As promised in #15603 - the **highly** sophisticated change - adding
missing config docstrings that are used in command completions.

## Test Plan

I actually made a local change to emit all empty items and verified
there are none now, before opening the PR.
2025-01-29 09:02:05 +05:30
InSync 72a4d343ff
[`refurb`] Do not emit diagnostic when loop variables are used outside loop body (`FURB122`) (#15757)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-01-28 19:16:21 +00:00
Brent Westbrook 786099a872
[`ruff`] Check for shadowed `map` before suggesting fix (`RUF058`) (#15790)
## Summary

Fixes #15786 by not suggesting a fix if `map` doesn't have its builtin
binding.

## Test Plan

New test taken from the report in #15786.
2025-01-28 14:15:37 -05:00
David Peter ca53eefa6f
[red-knot] Do not use explicit `knot_extensions.Unknown` declaration (#15787)
## Summary

Do not use an explict `knot_extensions.Unknown` declaration, as per
[this
comment](https://github.com/astral-sh/ruff/pull/15766#discussion_r1930997592).
Instead, use an undefined name to achieve the same effect.
2025-01-28 17:18:22 +01:00
Brent Westbrook 98d20a8219
Preserve quotes in generated byte strings (#15778)
## Summary

This is a very closely related follow-up to #15726, adding the same
quote-preserving behavior to bytestrings. Only one rule (UP018) was
affected this time, and it was easy to mirror the plain string changes.

## Test Plan

Existing tests
2025-01-28 08:19:40 -05:00
Alex Waygood 9c938442e5
[minor] Simplify some `ExprStringLiteral` creation logic (#15775) 2025-01-27 18:51:13 +00:00
Brent Westbrook 9bf138c45a
Preserve quote style in generated code (#15726)
## Summary

This is a first step toward fixing #7799 by using the quoting style
stored in the `flags` field on `ast::StringLiteral`s to select a quoting
style. This PR does not include support for f-strings or byte strings.

Several rules also needed small updates to pass along existing quoting
styles instead of using `StringLiteralFlags::default()`. The remaining
snapshot changes are intentional and should preserve the quotes from the
input strings.

## Test Plan

Existing tests with some accepted updates, plus a few new RUF055 tests
for raw strings.

---------

Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-01-27 13:41:03 -05:00
Dhruv Manilawala e994970538
Rename internal helper functions (#15771)
Refer:
https://github.com/astral-sh/ruff/pull/15713#discussion_r1930700717
2025-01-27 15:25:45 +00:00
Wei Lee c161e4fb12
[`airflow`] Extend airflow context parameter check for `BaseOperator.execute` (`AIR302`) (#15713)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
* feat
* add is_execute_method_inherits_from_airflow_operator for checking the
removed context key in the execute method
* refactor: rename
    * is_airflow_task as is_airflow_task_function_def
    * in_airflow_task as in_airflow_task_function_def
    * removed_in_3 as airflow_3_removal_expr
    * removed_in_3_function_def as airflow_3_removal_function_def
* test:
    * reorganize test cases

## Test Plan

a test fixture has been updated

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2025-01-27 20:48:18 +05:30
Mike Perlov 646f1942aa
Implement tab autocomplete for `ruff config` (#15603)
## Summary

Not the most important feature, but hey... was marked as the good first
issue ;-) fixes #4551

Unfortunately, looks like clap only generates proper completions for
zsh, so this would not make any difference for bash/fish.

## Test Plan

- cargo nextest run
- manual test by sourcing completions and then triggering autocomplete:
 
```shell
misha@PandaBook ruff % source <(target/debug/ruff generate-shell-completion zsh)
misha@PandaBook ruff % target/debug/ruff config lin
line-length                                                         -- The line length to use when enforcing long-lines violations
lint                                                                -- Configures how Ruff checks your code.
lint.allowed-confusables                                            -- A list of allowed 'confusable' Unicode characters to ignore
lint.dummy-variable-rgx                                             -- A regular expression used to identify 'dummy' variables, or
lint.exclude                                                        -- A list of file patterns to exclude from linting in addition
lint.explicit-preview-rules                                         -- Whether to require exact codes to select preview rules. Whe
lint.extend-fixable                                                 -- A list of rule codes or prefixes to consider fixable, in ad
lint.extend-ignore                                                  -- A list of rule codes or prefixes to ignore, in addition to
lint.extend-per-file-ignores                                        -- A list of mappings from file pattern to rule codes or prefi
lint.extend-safe-fixes                                              -- A list of rule codes or prefixes for which unsafe fixes sho
lint.extend-select                                                  -- A list of rule codes or prefixes to enable, in addition to
lint.extend-unsafe-fixes                                            -- A list of rule codes or prefixes for which safe fixes shoul
lint.external                                                       -- A list of rule codes or prefixes that are unsupported by Ru
lint.fixable                                                        -- A list of rule codes or prefixes to consider fixable. By de
lint.flake8-annotations                                             -- Print a list of available options
lint.flake8-annotations.allow-star-arg-any                          -- Whether to suppress `ANN401` for dynamically typed `*args`

...
```

- check command help
```shell
❯ target/debug/ruff config -h
List or describe the available configuration options

Usage: ruff config [OPTIONS] [OPTION]

Arguments:
  [OPTION]  Config key to show

Options:
      --output-format <OUTPUT_FORMAT>  Output format [default: text] [possible values: text, json]
  -h, --help                           Print help

Log levels:
  -v, --verbose  Enable verbose logging
  -q, --quiet    Print diagnostics, but nothing else
  -s, --silent   Disable all logging (but still exit with status code "1" upon detecting diagnostics)

Global options:
      --config <CONFIG_OPTION>  Either a path to a TOML configuration file (`pyproject.toml` or `ruff.toml`), or a TOML `<KEY> =
                                <VALUE>` pair (such as you might find in a `ruff.toml` configuration file) overriding a specific
                                configuration option. Overrides of individual settings using this option always take precedence over
                                all configuration files, including configuration files that were also specified using `--config`
      --isolated                Ignore all configuration files
```

- running original command
```shell
❯ target/debug/ruff config
cache-dir
extend
output-format
fix
unsafe-fixes
fix-only
show-fixes
required-version
preview
exclude
extend-exclude
extend-include
force-exclude
include
respect-gitignore
builtins
namespace-packages
target-version
src
line-length
indent-width
lint
format
analyze
```
2025-01-27 20:39:04 +05:30
David Peter 2ef94e5f3e
[red-knot] Document public symbol type inferece (#15766)
## Summary

Adds a slightly more comprehensive documentation of our behavior
regarding type inference for public uses of symbols. In particular:

- What public type do we infer for `x: int = any()`?
- What public type do we infer for `x: Unknown = 1`?
2025-01-27 10:52:13 +01:00
Marcus Näslund d0709093fe
Fix docstring in ruff_annotate_snippets (#15748)
## Summary

Found a comment that looks to be intended as docstring but accidentally
is just a normal comment.

Didn't create an issue as the readme said it's not neccessary for
trivial changes.

## Test Plan

<!-- How was it tested? -->
Can be tested by regenerating the docs.

Co-authored-by: Marcus Näslund <vidaochmarcus@gmail.com>
2025-01-26 22:25:29 -05:00
Charlie Marsh 37925ac442
Add references to `trio.run_process` and `anyio.run_process` (#15761)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14806.
2025-01-27 01:52:03 +00:00
InSync cb3361e682
[`ruff`] Do not emit diagnostic when all arguments to `zip()` are variadic (`RUF058`) (#15744) 2025-01-25 18:42:28 +00:00
Alex Waygood c824140fa8
[red-knot] Ensure differently ordered unions are considered equivalent when they appear inside tuples inside top-level intersections (#15743) 2025-01-25 18:19:28 +00:00
Alex Waygood f85ea1bf46
[red-knot] Ensure differently ordered unions and intersections are understood as equivalent even inside arbitrarily nested tuples (#15740)
## Summary

On `main`, red-knot:
- Considers `P | Q` equivalent to `Q | P`
- Considered `tuple[P | Q]` equivalent to `tuple[Q | P]`
- Considers `tuple[P | tuple[P | Q]]` equivalent to `tuple[tuple[Q | P]
| P]`
- ‼️ Does _not_ consider `tuple[tuple[P | Q]]` equivalent to
`tuple[tuple[Q | P]]`

The key difference for the last one of these is that the union appears
inside a tuple that is directly nested inside another tuple.

This PR fixes this so that differently ordered unions are considered
equivalent even when they appear inside arbitrarily nested tuple types.

## Test Plan

- Added mdtests that fails on `main`
- Checked that all property tests continue to pass with this PR
2025-01-25 16:39:07 +00:00
Alex Waygood a77a32b7d4
[red-knot] Promote the `all_type_pairs_are_assignable_to_their_union` property test to stable (#15739) 2025-01-25 16:26:37 +00:00
Jelle Zijlstra d8c2d20325
[`pylint`] Do not trigger `PLR6201` on empty collections (#15732)
Fixes #15729.
2025-01-24 20:42:49 -06:00
Zanie Blue fcd0f349f9
Improve the file watching failure error message (#15728)
I really misunderstood this in
https://github.com/astral-sh/ruff/pull/15664#issuecomment-2613079710
2025-01-24 15:28:30 -06:00
Douglas Creager 5a9d71a5f1
Speed symbol state merging back up (#15731)
This is a follow-up to #15702 that hopefully claws back the 1%
performance regression. Assuming it works, the trick is to iterate over
the constraints vectors via mut reference (aka a single pointer), so
that we're not copying `BitSet`s into and out of the zip tuples as we
iterate. We use `std::mem::take` as a poor-man's move constructor only
at the very end, when we're ready to emplace it into the result. (C++
idioms intended! 😄)

With local testing via hyperfine, I'm seeing this be 1-3% faster than
`main` most of the time — though a small number of runs (1 in 10,
maybe?) are a wash or have `main` faster. Codspeed reports a 2%
gain.
2025-01-24 16:07:31 -05:00
Micha Reiser 9353482a5a
Add `check` command (#15692) 2025-01-24 17:00:30 +01:00
Douglas Creager 716b246cf3
[red-knot] Use itertools to clean up `SymbolState::merge` (#15702)
[`merge_join_by`](https://docs.rs/itertools/latest/itertools/trait.Itertools.html#method.merge_join_by)
handles the "merge two sorted iterators" bit, and `zip` handles
iterating through the bindings/definitions along with their associated
constraints.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-24 10:21:29 -05:00
Micha Reiser 4e3982cf95
[red-knot] Add `--ignore`, `--warn`, and `--error` CLI arguments (#15689) 2025-01-24 16:20:15 +01:00
David Peter 1feb3cf41a
[red-knot] Use `Unknown | T_inferred` for undeclared public symbols (#15674)
## Summary

Use `Unknown | T_inferred` as the type for *undeclared* public symbols.

## Test Plan

- Updated existing tests
- New test for external `__slots__` modifications.
- New tests for external modifications of public symbols.
2025-01-24 12:47:48 +01:00
Dylan 7778d1d646
[`ruff`] Parenthesize fix when argument spans multiple lines for `unnecessary-round` (`RUF057`) (#15703) 2025-01-24 04:34:56 -06:00
David Peter fb58a9b610
[red-knot] Rename `TestDbBuilder::typeshed` to `.custom_typeshed` (#15712)
## Summary

Correcting a small oversight by me
(https://github.com/astral-sh/ruff/pull/15683#discussion_r1926830914).
2025-01-24 10:25:23 +00:00
Mike Perlov 17a8a55f08
Honor banned top level imports by TID253 in PLC0415. (#15628)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-24 11:07:21 +01:00
Dhruv Manilawala 99d8ec6769
Apply `AIR302`-context check only in `@task` function (#15711)
This PR updates `AIR302` to only apply the context keys check in `@task`
decorated function.

Ref: https://github.com/astral-sh/ruff/pull/15144
2025-01-24 07:30:35 +00:00
Ankit Chaurasia 34cc3cab98
[`airflow`] Update `AIR302` to check for deprecated context keys (#15144)
**Summary**

Airflow 3.0 removes a set of deprecated context variables that were
phased out in 2.x. This PR introduces lint rules to detect usage of
these removed variables in various patterns, helping identify
incompatibilities. The removed context variables include:

```
conf
execution_date
next_ds
next_ds_nodash
next_execution_date
prev_ds
prev_ds_nodash
prev_execution_date
prev_execution_date_success
tomorrow_ds
yesterday_ds
yesterday_ds_nodash
```

**Detected Patterns and Examples**

The linter now flags the use of removed context variables in the
following scenarios:

1. **Direct Subscript Access**  
   ```python
   execution_date = context["execution_date"]  # Flagged
   ```
   
2. **`.get("key")` Method Calls**  
   ```python
   print(context.get("execution_date"))  # Flagged
   ```
   
3. **Variables Assigned from `get_current_context()`**  
If a variable is assigned from `get_current_context()` and then used to
access a removed key:
   ```python
   c = get_current_context()
   print(c.get("execution_date"))  # Flagged
   ```
   
4. **Function Parameters in `@task`-Decorated Functions**  
Parameters named after removed context variables in functions decorated
with `@task` are flagged:
   ```python
   from airflow.decorators import task
   
   @task
def my_task(execution_date, **kwargs): # Parameter 'execution_date'
flagged
       pass
   ```
   
5. **Removed Keys in Task Decorator `kwargs` and Other Scenarios**  
Other similar patterns where removed context variables appear (e.g., as
part of `kwargs` in a `@task` function) are also detected.
```
from airflow.decorators import task

@task
def process_with_execution_date(**context):
    execution_date = lambda: context["execution_date"]  # flagged
    print(execution_date)

@task(kwargs={"execution_date": "2021-01-01"})   # flagged
def task_with_kwargs(**context):  
    pass
```

**Test Plan**

Test fixtures covering various patterns of deprecated context usage are
included in this PR. For example:

```python
from airflow.decorators import task, dag, get_current_context
from airflow.models import DAG
from airflow.operators.dummy import DummyOperator
import pendulum
from datetime import datetime

@task
def access_invalid_key_task(**context):
    print(context.get("conf"))  # 'conf' flagged

@task
def print_config(**context):
    execution_date = context["execution_date"]  # Flagged
    prev_ds = context["prev_ds"]                # Flagged

@task
def from_current_context():
    context = get_current_context()
    print(context["execution_date"])            # Flagged

# Usage outside of a task decorated function
c = get_current_context()
print(c.get("execution_date"))                 # Flagged

@task
def some_task(execution_date, **kwargs):
    print("execution date", execution_date)     # Parameter flagged

@dag(
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC")
)
def my_dag():
    task1 = DummyOperator(
        task_id="task1",
        params={
            "execution_date": "{{ execution_date }}",  # Flagged in template context
        },
    )

    access_invalid_key_task()
    print_config()
    from_current_context()
    
dag = my_dag()

class CustomOperator(BaseOperator):
    def execute(self, context):
        execution_date = context.get("execution_date")                      # Flagged
        next_ds = context.get("next_ds")                                               # Flagged
        next_execution_date = context["next_execution_date"]          # Flagged
```

Ruff will emit `AIR302` diagnostics for each deprecated usage, with
suggestions when applicable, aiding in code migration to Airflow 3.0.

related: https://github.com/apache/airflow/issues/44409,
https://github.com/apache/airflow/issues/41641

---------

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
2025-01-24 11:25:05 +05:30
Dhruv Manilawala 9384ba4b91
Remove test rules from JSON schema (#15627)
Closes: #15707
2025-01-24 10:17:59 +05:30
Dylan b5ffb404de
Bump version to 0.9.3 (#15698) 2025-01-23 12:43:56 -06:00
Brent Westbrook cffd1866ce
Preserve raw string prefix and escapes (#15694)
## Summary

Fixes #9663 and also improves the fixes for
[RUF055](https://docs.astral.sh/ruff/rules/unnecessary-regular-expression/)
since regular expressions are often written as raw strings.

This doesn't include raw f-strings.

## Test Plan

Existing snapshots for RUF055 and PT009, plus a new `Generator` test and
a regression test for the reported `PIE810` issue.
2025-01-23 12:12:10 -05:00
InSync 569060f46c
[`flake8-pytest-style`] Rewrite references to `.exception` (`PT027`) (#15680) 2025-01-23 17:50:40 +01:00
David Peter 15394a8028
[red-knot] MDTests: Do not depend on precise public-symbol type inference (#15691)
## Summary

Another small PR to focus #15674 solely on the relevant changes. This
makes our Markdown tests less dependent on precise types of public
symbols, without actually changing anything semantically in these tests.

Best reviewed using ignore-whitespace-mode.

## Test Plan

Tested these changes on `main` and on the branch from #15674.
2025-01-23 13:51:33 +00:00
David Peter fc2ebea736
[red-knot] Make `infer.rs` unit tests independent of public symbol inference (#15690)
## Summary

Make the remaining `infer.rs` unit tests independent from public symbol
type inference decisions (see upcoming change in #15674).

## Test Plan

- Made sure that the unit tests actually fail if one of the
  `assert_type` assertions is changed.
2025-01-23 14:30:18 +01:00
Micha Reiser 43160b4c3e
Tidy knot CLI tests (#15685) 2025-01-23 14:06:07 +01:00
David Peter 0173738eef
[red-knot] Port comprehension tests to Markdown (#15688)
## Summary

Port comprehension tests from Rust to Markdown

I don' think the remaining tests in `infer.rs` should be ported to
Markdown, maybe except for the incremental-checking tests when (if ever)
we have support for that in the MD tests.


closes #13696
2025-01-23 12:49:30 +00:00
Micha Reiser 05ea77b1d4
Create Unknown rule diagnostics with a source range (#15648) 2025-01-23 12:50:43 +01:00
David Peter 1e790d3885
[red-knot] Port 'deferred annotations' unit tests to Markdown (#15686)
## Summary

- Port "deferred annotations" unit tests to Markdown
- Port `implicit_global_in_function` unit test to Markdown
- Removed `resolve_method` and `local_inference` unit tests. These seem
  like relics from a time where type inference was in it's early stages.
  There is no way that these tests would fail today without lots of other
  things going wrong as well.

part of #13696
based on #15683 

## Test Plan

New MD tests for existing Rust unit tests.
2025-01-23 11:45:05 +00:00
David Peter 7855f03735
[red-knot] Support custom typeshed Markdown tests (#15683)
## Summary

- Add feature to specify a custom typeshed from within Markdown-based
  tests
- Port "builtins" unit tests from `infer.rs` to Markdown tests, part of
  #13696

## Test Plan

- Tests for the custom typeshed feature
- New Markdown tests for deleted Rust unit tests
2025-01-23 12:36:38 +01:00
Micha Reiser 7b17c9c445
Add `rules` table to configuration (#15645) 2025-01-23 10:56:58 +01:00
Micha Reiser 23c222368e
[red-knot] Make `Diagnostic::file` optional (#15640) 2025-01-23 10:43:14 +01:00
David Peter 1ecd97855e
[red-knot] Add test for nested attribute access (#15684)
## Summary

Add a new test for attribute accesses in case of nested modules /
classes. Resolves this comment:

https://github.com/astral-sh/ruff/pull/15613#discussion_r1925637561

## Test Plan

New MD test.
2025-01-23 10:26:34 +01:00
Micha Reiser 39e2df7ada
[red-knot] Anchor relative paths in configurations (#15634) 2025-01-23 10:14:01 +01:00
Brent Westbrook ce8110332c
[`pyupgrade`] Handle multiple base classes for PEP 695 generics (`UP046`) (#15659)
## Summary

Addresses the second follow up to #15565 in #15642. This was easier than
expected by using this cool destructuring syntax I hadn't used before,
and by assuming
[PYI059](https://docs.astral.sh/ruff/rules/generic-not-last-base-class/)
(`generic-not-last-base-class`).

## Test Plan

Using an existing test, plus two new tests combining multiple base
classes and multiple generics. It looks like I deleted a relevant test,
which I did, but I meant to rename this in #15565. It looks like instead
I copied it and renamed the copy.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-22 20:19:13 -05:00
Alex Waygood 555b3a6a2c
[`pyflakes`] Treat arguments passed to the `default=` parameter of `TypeVar` as type expressions (`F821`) (#15679) 2025-01-22 23:04:20 +00:00
Brent Westbrook bb6fb4686d
[`pyupgrade`] Add rules to use PEP 695 generics in classes and functions (`UP046`, `UP047`) (#15565)
## Summary

This PR extends our [PEP 695](https://peps.python.org/pep-0695) handling
from the type aliases handled by `UP040` to generic function and class
parameters, as suggested in the latter two examples from #4617:

```python
# Input
T = TypeVar("T", bound=float)
class A(Generic[T]):
    ...

def f(t: T):
    ...

# Output
class A[T: float]:
    ...

def f[T: float](t: T):
    ...
```

I first implemented this as part of `UP040`, but based on a brief
discussion during a very helpful pairing session with @AlexWaygood, I
opted to split them into rules separate from `UP040` and then also
separate from each other. From a quick look, and based on [this
issue](https://github.com/asottile/pyupgrade/issues/836), I'm pretty
sure neither of these rules is currently in pyupgrade, so I just took
the next available codes, `UP046` and `UP047`.

The last main TODO, noted in the rule file and in the fixture, is to
handle generic method parameters not included in the class itself, `S`
in this case:

```python
T = TypeVar("T")
S = TypeVar("S")

class Foo(Generic[T]):
    def bar(self, x: T, y: S) -> S: ...
```

but Alex mentioned that that might be okay to leave for a follow-up PR.

I also left a TODO about handling multiple subclasses instead of bailing
out when more than one is present. I'm not sure how common that would
be, but I can still handle it here, or follow up on that too.

I think this is unrelated to the PR, but when I ran `cargo dev
generate-all`, it removed the rule code `PLW0101` from
`ruff.schema.json`. It seemed unrelated, so I left that out, but I
wanted to mention it just in case.

## Test Plan

New test fixture, `cargo nextest run`

Closes #4617, closes #12542

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-22 11:35:21 -05:00
Alex Waygood b4877f1661
[red-knot] Ensure a gradual type can always be assigned to itself (#15675) 2025-01-22 16:01:13 +00:00
David Peter 3235cd8019
[red-knot] Fix possible TOCTOU mistake in mdtest runner (#15673)
## Summary

Somehow, I managed to crash the `mdtest` runner today. I struggled to
reproduce this again to see if it's actually fixed (even with an
artificial `sleep` between the two `cargo test` invocations), but the
original backtrace clearly showed that this is where the problem
originated from. And it seems like a clear TOCTOU problem.
2025-01-22 15:24:25 +00:00
David Peter 13e7afca42
[red-knot] Improved error message for attribute-assignments (#15668)
## Summary

Slightly improved error message for attribute assignments.
2025-01-22 11:04:38 +00:00
David Peter f349dab4fc
[red-knot] Invalid assignments to attributes (#15613)
## Summary

Raise "invalid-assignment" diagnostics for incorrect assignments to
attributes, for example:

```py
class C:
    var: str = "a"

C.var = 1  # error: "Object of type `Literal[1]` is not assignable to `str`"
```

closes #15456 

## Test Plan

- Updated test assertions
- New test for assignments to module-attributes
2025-01-22 10:42:47 +01:00
Dhruv Manilawala 043ff61a0b
Consider `unsafe-fixes` settings for code actions (#15666)
## Summary

Closes: #13960 

## Test Plan

Using the example from
https://github.com/astral-sh/ruff-vscode/issues/672:


https://github.com/user-attachments/assets/7bdb01ef-8752-4cb7-9b5d-8a0d131984da
2025-01-22 13:44:13 +05:30
David Peter 792f9e357e
[red-knot] Rename *_ty functions (#15617)
## Summary

General rules:

* Change the `_ty` suffix of all functions to `_type`.
* `_type_and_qualifiers` suffixes seem too long, so we ignore the
existence of qualifiers and still speak of "types"
* Functions only have a `_type` suffix if they return either `Type`,
`Option<Type>`, or `TypeAndQualifiers`

Free functions:

* `binding_ty` => `binding_type`
* `declaration_ty` => `declaration_type`
* `definition_expression_ty` => `definition_expression_type`

Methods:

* `CallDunderResult::return_ty` => `return_type`
* `NotCallableError::return_ty` => `return_type`
* `NotCallableError::called_ty` => `called_type`
* `TypeAndQualifiers::inner_ty` => `inner_type`
* `TypeAliasType::value_ty` => `value_type`
* `TypeInference::expression_ty` => `expression_type`
* `TypeInference::try_expression_ty` => `try_expression_type`
* `TypeInference::binding_ty` => `binding_type`
* `TypeInference::declaration_ty` => `declaration_type` 
* `TypeInferenceBuilder::expression_ty` => `expression_type`
* `TypeInferenceBuilder::file_expression_ty` => `file_expression_type`
* `TypeInferenceBuilder::module_ty_from_name` => `module_type_from_name`
* `ClassBase::try_from_ty` => `try_from_type`
* `Parameter::annotated_ty` => `annotated_type`
* `Parameter::default_ty` => `default_type`
* `CallOutcome::return_ty` => `return_type`
* `CallOutcome::return_ty_result` => `return_type_result`
* `CallBinding::from_return_ty` => `from_return_type`
* `CallBinding::set_return_ty` => `set_return_type`
* `CallBinding::return_ty` => `return_type`
* `CallBinding::parameter_tys` => `parameter_types`
* `CallBinding::one_parameter_ty` => `one_parameter_type`
* `CallBinding::two_parameter_tys` => `two_parameter_types`
* `Unpacker::tuple_ty_elements` => `tuple_type_elements`
* `StringPartsCollector::ty` => `string_type`

Traits

* `HasTy` => `HasType`
* `HasTy::ty` => `inferred_type`

Test functions:

* `assert_public_ty` => `assert_public_type`
* `assert_scope_ty` => `assert_scope_type`

closes #15569

## Test Plan

—
2025-01-22 09:06:56 +01:00
InSync f54b82147e
[`flake8-bandit`] Add missing single-line/dotall regex flag (`S608`) (#15654)
## Summary

Resolves #15653.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-22 10:20:22 +05:30
Wei Lee 1e053531b6
[`airflow`] Argument `fail_stop` in DAG has been renamed as `fail_fast` (`AIR302`) (#15633)
## Summary

argument `fail_stop` in DAG has been renamed as `fail_fast` (AIR302)

## Test Plan

a test fixture has been updated
2025-01-22 09:18:57 +05:30
Alex Waygood fbb06fe0ac
[red-knot] Small simplifications to `Type::is_subtype_of` and `Type::is_disjoint_from` (#15622)
## Summary

This PR generalizes some of the logic we have in `Type::is_subtype_of`
and `Type::is_disjoint_from` so that we fallback to the instance type of
the metaclass more often in `Type::ClassLiteral` and `Type::SubclassOf`
branches. This simplifies the code (we end up with one less branch in
`is_subtype_of`, and we can remove a helper method that's no longer
used), makes the code more robust (any fixes made to subtyping or
disjointness of instance types will automatically improve our
understanding of subtyping/disjointness for class-literal types and
`type[]` types) and more elegantly expresses the type-system invariants
encoded in these branches.

## Test Plan

No new tests added (it's a pure refactor, adding no new functionality).
All existing tests pass, however, including the property tests.
2025-01-22 00:31:55 +00:00
Douglas Creager ef85c682bd
Remove customizable reference enum names (#15647)
The AST generator creates a reference enum for each syntax group — an
enum where each variant contains a reference to the relevant syntax
node. Previously you could customize the name of the reference enum for
a group — primarily because there was an existing `ExpressionRef` type
that wouldn't have lined up with the auto-derived name `ExprRef`. This
follow-up PR is a simple search/replace to switch over to the
auto-derived name, so that we can remove this customization point.
2025-01-21 13:46:31 -05:00
Douglas Creager fa546b20a6
Separate grouped and ungrouped nodes more clearly in AST generator (#15646)
This is a minor cleanup to the AST generation script to make a clearer
separation between nodes that do appear in a group enum, and those that
don't. There are some types and methods that we create for every syntax
node, and others that refer to the group that the syntax node belongs
to, and which therefore don't make sense for ungrouped nodes. This new
separation makes it clearer which category each definition is in, since
you're either inside of a `for group in ast.groups` loop, or a `for node
in ast.all_nodes` loop.
2025-01-21 13:37:18 -05:00
InSync fce4adfd41
[`flake8-simplify`] Mark fixes as unsafe (`SIM201`, `SIM202`) (#15626) 2025-01-21 18:17:48 +01:00
David Peter 13a6b5600b
[red-knot] mdtest runner: include stderr for crashing tests (#15644)
## Summary

Test executables usually write failure messages (including panics) to
stdout, but I just managed to make a mdtest crash with
```
thread 'mdtest__unary_not' has overflowed its stack
fatal runtime error: stack overflow
```
which is printed to stderr. This test simply appends stderr to stdout
(`stderr=subprocess.STDOUT` can not be used with `capture_output`)

## Test Plan

Make sure that the error message is now visible in the output of `uv -q
run crates/red_knot_python_semantic/mdtest.py`
2025-01-21 14:59:36 +00:00
Micha Reiser 067c6de465
Change `EnvironmentOptions::venv-path` to `Option<SystemPathBuf>` (#15631)
## Summary

The `Options` struct is intended to capture the user's configuration
options but
`EnvironmentOptions::venv_path` supports both a `SitePackages::Known`
and `SitePackages::Derived`.

Users should only be able to provide `SitePackages::Derived`—they
specify a path to a venv, and Red Knot derives the path to the
site-packages directory. We'll only use the `Known` variant once we
automatically discover the Python installation.

That's why this PR changes `EnvironmentOptions::venv_path` from
`Option<SitePackages>` to `Option<SystemPathBuf>`.

This requires making some changes to the file watcher test, and I
decided to use `extra_paths` over venv path
because our venv validation is annoyingly correct -- making mocking a
venv rather involved.

## Test Plan

`cargo test`
2025-01-21 14:10:41 +00:00
David Salvisberg 4366473d9b
[`flake8-type-checking`] Fix some safe fixes being labeled unsafe (#15638)
## Summary

We were mistakenly using `CommentRanges::has_comments` to determine
whether our edits
were safe, which sometimes expands the checked range to the end of a
line. But in order to
determine safety we need to check exactly the range we're replacing.

This bug affected the rules `runtime-cast-value` (`TC006`) and
`quoted-type-alias` (`TC008`)
although it was very unlikely to be hit for `TC006` and for `TC008` we
never hit it because we
were checking the wrong expression.

## Test Plan

`cargo nextest run`
2025-01-21 15:08:46 +01:00
Chandra Kiran G cff9c13c42
feat: Update RUF055 to do var == value (#15605)
This commit fixes RUF055 rule to format `re.fullmatch(pattern, var)` to
`var == pattern` instead of the current `pattern == var` behaviour. This
is more idiomatic and easy to understand.

## Summary

This changes the current formatting behaviour of `re.fullmatch(pattern,
var)` to format it to `var == pattern` instead of `pattern == var`.

## Test Plan

I used a code file locally to see the updated formatting behaviour.

Fixes https://github.com/astral-sh/ruff/issues/14733
2025-01-21 08:47:06 -05:00
David Peter 4656e3c90f
[red-knot] Markdown test runner (#15632)
## Summary

As more and more tests move to Markdown, running the mdtest suite
becomes one of the most common tasks for developers working on Red Knot.
There are a few pain points when doing so, however:

- The `build.rs` script enforces recompilation (~five seconds) whenever
something changes in the `resource/mdtest` folder. This is strictly
necessary, because whenever files are added or removed, the test harness
needs to be updated. But this is very rarely the case! The most common
scenario is that a Markdown file has *changed*, and in this case, no
recompilation is necessary. It is currently not possible to distinguish
these two cases using `cargo::rerun-if-changed`. One can work around
this by running the test executable manually, but it requires finding
the path to the correct `mdtest-<random-hash>` executable.
- All Markdown tests are run by default. This is needed whenever Rust
code changes, but while working on the tests themselves, it is often
much more convenient to only run the tests for a single file. This can
be done by using a `mdtest__path_to_file` filter, but this needs to be
manually spelled out or copied from the test output.
- `cargo`s test output for a failing Markdown test is often
unnecessarily verbose. Unless there is an *actual* panic somewhere in
the code, mdtests usually fail with the explicit *"Some tests failed"*
panic in the mdtest suite. But in those cases, we are not interested in
the pointer to the source of this panic, but only in the mdtest suite
output.

This PR adds a Markdown test runner tool that attempts to make the
developer experience better.

Once it is started using
```bash
uv run -q crates/red_knot_python_semantic/mdtest.py
```
it will first recompile the tests once (if cargo requires it), find the
path to the `mdtest` executable, and then enter into a mode where it
watches for changes in the `red_knot_python_semantic` crate. Whenever …
* … a Markdown file changes, it will rerun the mdtest for this specific
  file automatically (no recompilation!).
* … a Markdown file is added, it will recompile the tests and then run
  the mdtest for the new file
* … Rust code is changed, it will recompile the tests and run all of
  them

The tool also trims down `cargo test` output and only shows the actual
mdtest errors.

The tool will certainly require a few more iterations before it becomes
mature, but I'm curious to hear if there is any interest for something
like this.

## Test Plan

- Tested the new runner under various scenarios.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-21 14:06:35 +01:00
Alex Waygood 187a358d7a
[red-knot] Heterogeneous tuple types with differently ordered (but equivalent) unions at the same index should be considered equivalent (#15637) 2025-01-21 12:51:20 +00:00
Calum Young 023c52d82b
Standardise ruff config (#15558) 2025-01-21 12:09:11 +01:00
InSync c616650dfa
[`ruff`] Needless `else` clause (`RUF047`) (#15051)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-21 08:21:19 +00:00
InSync 4cfa355519
[`ruff`] Exempt `NewType` calls where the original type is immutable (`RUF009`) (#15588)
## Summary

Resolves #6447.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-20 20:14:47 +05:30
David Peter 134fefa945
[red-knot] Rename `bindings_ty`, `declarations_ty` (#15618)
## Summary

Rename two functions with outdated names (they used to return `Type`s):

* `bindings_ty` => `symbol_from_bindings` (returns `Symbol`)
* `declarations_ty` => `symbol_from_declarations` (returns a
`SymbolAndQualifiers` result)

I chose `symbol_from_*` instead of `*_symbol` as I found the previous
name quite confusing. Especially since `binding_ty` and `declaration_ty`
also exist (singular).

## Test Plan

—

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-20 15:23:33 +01:00
Micha Reiser 73798327c6
Flatten `red_knot_project` import paths (#15616) 2025-01-20 14:57:57 +01:00
wooly18 f82ef32e53
[`red-knot`] No `cyclic-class-def` diagnostics for subclasses of cyclic classes (#15561)
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-01-20 13:35:29 +00:00
Micha Reiser d70d959612
Rename `red_knot_workspace` to `red_knot_project` (#15615) 2025-01-20 14:02:36 +01:00
Micha Reiser 80345e72c4
show-settings: Properly filter out backslashes on windows (#15612)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->
2025-01-20 10:57:21 +01:00
Wei Lee cbf9b66fc1
[`airflow`] Extend `AIR303` with more symbols (#15611)
## Summary

Extend `AIR303` with the following rules

* `airflow.operators.datetime.*` → `airflow.providers.standard.time.operators.datetime.*`
* `airflow.operators.weekday.*` → `airflow.providers.standard.time.operators.weekday.*`
* `airflow.sensors.date_time.*` → `airflow.providers.standard.time.sensors.date_time.*`
* `airflow.sensors.time_sensor.*` → `airflow.providers.standard.time.sensors.time.*`
* `airflow.sensors.time_delta.*` → `airflow.providers.standard.time.sensors.time_delta.*`
* `airflow.sensors.weekday.*` → `airflow.providers.standard.time.sensors.weekday.*`
* `airflow.hooks.filesystem.*` → `airflow.providers.standard.hooks.filesystem.*`
* `airflow.hooks.package_index.*` → `airflow.providers.standard.hooks.package_index.*`
* `airflow.hooks.subprocess.*` → `airflow.providers.standard.hooks.subprocess.*`
* `airflow.triggers.external_task.*` → `airflow.providers.standard.triggers.external_task.*`
* `airflow.triggers.file.*` → `airflow.providers.standard.triggers.file.*`
* `airflow.triggers.temporal.*` → `airflow.providers.standard.triggers.temporal.*`
* `airflow.sensors.filesystem.FileSensor` → `airflow.providers.standard.sensors.filesystem.FileSensor`
* `airflow.operators.trigger_dagrun.TriggerDagRunOperator` → `airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunOperator`
* `airflow.sensors.external_task.ExternalTaskMarker` → `airflow.providers.standard.sensors.external_task.ExternalTaskMarker`
* `airflow.sensors.external_task.ExternalTaskSensor` → `airflow.providers.standard.sensors.external_task.ExternalTaskSensor`

## Test Plan

a test fixture has been updated
2025-01-20 15:00:26 +05:30
Micha Reiser 248f0ec6b2
Isolate `show_settings` test from Ruff's `pyproject.toml (#15610)
## Summary

In preperation for https://github.com/astral-sh/ruff/pull/15558

Isolate the `show_settings` test instead of reading Ruff's
`pyproject.toml` for better test isolation.

## Test Plan

`cargo test`
2025-01-20 09:28:01 +00:00
Dhruv Manilawala d8cabf62b1
Avoid large closure to make `rustfmt` work (#15609)
## Summary

I noticed this while reviewing
https://github.com/astral-sh/ruff/pull/15541 that the code inside the
large closure cannot be formatted by the Rust formatter. This PR
extracts the qualified name and inlines the match expression.

## Test Plan

`cargo clippy` and `cargo insta`
2025-01-20 09:15:48 +00:00
David Peter 912247635d
[red-knot] Move `Ty` enum to property tests (#15608)
## Summary

Move the `Ty` enum into the `property_tests` module, as it was only used
in a single place in `types.rs`.
2025-01-20 10:15:31 +01:00
InSync 5cd1f79864
[`flake8-bandit`] Report all references to suspicious functions (`S3`) (#15541)
## Summary

Resolves #15522.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2025-01-20 09:02:53 +00:00
David Peter 4eb465ee95
[red-knot] Move `type_alias_types` test to Markdown (#15607)
## Summary

Move `type_alias_types` test to Markdown

## Test Plan

New MD test
2025-01-20 09:55:54 +01:00
David Peter 9725a2d476
[red-knot] More exhaustive disjointness tests (#15606)
## Summary

Mostly just brings the structure/format of the disjointness-tests closer
to what we have for `is_subtype_of` etc.
2025-01-20 09:47:51 +01:00
InSync 975d1457c5
[red-knot] Migrate `is_disjoint_from` unit tests to Markdown tests (#15580)
## Summary

Part of and resolves #15397, built on top of #15579.

## Test Plan

Markdown tests.
2025-01-20 08:42:22 +01:00
Charlie Marsh 98fccec2e7
Avoid removing too many imports in `redefined-while-unused` (#15585)
## Summary

Closes https://github.com/astral-sh/ruff/issues/15583.
2025-01-19 13:28:08 -05:00
InSync 444f799f5e
[red-knot] Two gradual equivalent fully static types are also equivalent (#15579) 2025-01-19 16:37:22 +00:00
Alex Waygood 2b24b3b316
[red-knot] Ensure differently ordered unions and intersections are considered equivalent (#15516) 2025-01-19 16:10:42 +00:00
Charlie Marsh b8e5b95423
Avoid quadratic membership check in import fixes (#15576)
## Summary

This leads to an explosion in runtime for (admittedly absurd) cases with
tens of thousands of imports.
2025-01-18 23:01:26 +00:00
Charlie Marsh 6004c8c003
Apply redefinition fixes by source code order (#15575)
## Summary

Right now, these are being applied in random order, since if we have two
`RedefinitionWhileUnused`, it just takes the first-generated (whereas
the next comparator in the sort here orders by location)... Which means
we frequently have to re-run!
2025-01-18 17:44:10 -05:00
Charlie Marsh 1344c8a4e2
Group redefinition fixes by source statement (#15574)
## Summary

Like unused imports, we should create a single fix for all redefined
members in a single statement.

Closes https://github.com/astral-sh/ruff/issues/15182.
2025-01-18 17:31:58 -05:00
Dylan 8a50f3f361
[`isort`] Omit trailing whitespace in `unsorted-imports` (`I001`) (#15518)
## Summary
The fix range for sorting imports accounts for trailing whitespace, but
we should only show the trimmed range to the user when displaying the
diagnostic. So this PR changes the diagnostic range.

Closes #15504 

## Test Plan

Reviewed snapshot changes
2025-01-18 11:08:58 -06:00
InSync 001e5adec5
[`flake8-simplify`] Avoid double negations (`SIM103`) (#15562)
## Summary

Related to [this
comment](https://github.com/astral-sh/ruff/issues/6184#issuecomment-2578673788)
at #6184.

---------

Co-authored-by: Dylan <53534755+dylwil3@users.noreply.github.com>
2025-01-18 17:06:46 +00:00
Will Lachance 38adc7f702
TRY300: Add some extra notes on not catching exceptions you didn't expect (#15036)
## Summary

Added some extra notes on why you should have focused try...except
blocks to
[TRY300](https://docs.astral.sh/ruff/rules/try-consider-else/).

When fixing a violation of this rule, a co-worker of mine (very
understandably) asked why this was better. The current docs just say
putting the return in the else is "more explicit", but if you look at
the [linked reference in the python
documentation](https://docs.python.org/3/tutorial/errors.html) they are
more clear on why violations like this is bad:

> The use of the else clause is better than adding additional code to
the [try](https://docs.python.org/3/reference/compound_stmts.html#try)
clause because it avoids accidentally catching an exception that wasn’t
raised by the code being protected by the try … except statement.

This is my attempt at adding more context to the docs on this. Open to
suggestions for wording!

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-01-18 10:23:43 -06:00