Python
/
ruff
mirror of https://github.com/astral-sh/ruff
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,549 Commits 503 Branches 393 Tags 281 MiB
Commit Graph

6656 Commits

Author SHA1 Message Date
Andrew Gallant a4d3c4bf8b red_knot_python_semantic: make the guards implement DerefMut 
This is the payoff from removing a bit of indirection. The types still
exist, but now callers don't need to do builder -> reporter ->
diagnostic. They can just conceptually think of it as builder ->
diagnostic.
 2025-04-11 12:36:36 -04:00
Andrew Gallant f3cc287a40 red_knot_python_semantic: tweak naming 
We're going to make the guards deref to `Diagnostic` in order to remove
a layer of indirection in the reporter API. (Well, technically the layer
is not removed since the types still exist, but in actual _usage_ the
layer will be removed. We'll see how it shakes out in the next commit.)
 2025-04-11 12:36:36 -04:00
Andrew Gallant bd5b8a9ec6 ruff_db: tweak APIs accepting a diagnostic message 
This expands the set of types accepted for diagnostic messages. Instead
of only `std::fmt::Display` impls, we now *also* accept a concrete
`DiagnosticMessage`.

This will be useful to avoid unnecessary copies of every single
diagnostic message created via `InferContext::report_lint`. (I'll call
out how this helps in a subsequent commit.)
 2025-04-11 12:36:36 -04:00
David Peter 47956db567
[red-knot] Specialize `str.startswith` for string literals (#17351) 
## Summary

Infer precise Boolean literal types for `str.startswith` calls where the
instance and the prefix are both string literals. This allows us to
understand `sys.platform.startswith(…)` branches.

## Test Plan

New Markdown tests
 2025-04-11 16:26:45 +02:00
Brent Westbrook ffef71d106
[syntax-errors] `yield`, `yield from`, and `await` outside functions (#17298) 
Summary
--

This PR reimplements [yield-outside-function
(F704)](https://docs.astral.sh/ruff/rules/yield-outside-function/) as a
semantic syntax error. Despite the name, this rule covers `yield from`
and `await` in addition to `yield`.

Test Plan
--

New linter tests, along with the existing F704 test.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
 2025-04-11 10:16:23 -04:00
Micha Reiser 7e571791c0
[red-knot] Refresh diagnostics when changing related files (#17350) 2025-04-11 15:50:28 +02:00
Brent Westbrook 8e11c53310
Add `Checker::import_from_typing` (#17340) 
Summary
--

This PR replaces uses of version-dependent imports from `typing` or
`typing_extensions` with a centralized `Checker::import_from_typing`
method.

The idea here is to make the fix for #9761 (whatever it ends up being)
applicable to all of the rules performing similar checks.

Test Plan
--

Existing tests for the affected rules.
 2025-04-11 13:37:55 +00:00
Max Mynter 1aad180aae
Don't add chaperone space after escaped quote in triple quote (#17216) 
Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-04-11 10:21:47 +02:00
David Peter 1a3b73720c
[red-knot] Silence errors in unreachable type annotations / class bases (#17342) 
## Summary

For silencing `invalid-type-form` diagnostics in unreachable code, we
use the same approach that we use before and check the reachability that
we already record.

For silencing `invalid-bases`, we simply check if the type of the base
is `Never`. If so, we silence the diagnostic with the argument that the
class construction would never happen.

## Test Plan

Updated Markdown tests.
 2025-04-10 22:47:47 +02:00
David Peter 8b2727cf67
[red-knot] Silence `unresolved-import` in unreachable code (#17336) 
## Summary

Similar to what we did for `unresolved-reference` and
`unresolved-attribute`, we now also silence `unresolved-import`
diagnostics if the corresponding `import` statement is unreachable.

This addresses the (already closed) issue #17049.

## Test Plan

Adapted Markdown tests.
 2025-04-10 21:13:28 +02:00
Andrew Gallant 410aa4b8fd red_knot_python_semantic: move TODO comment 
It fits with reporting lint diagnostics much better than reporting
general diagnostics.
 2025-04-10 13:21:00 -04:00
Andrew Gallant bd58135b0d red_knot_python_semantic: rename `lint()` and `report()` 
... to `report_lint()` and `report_diagnostic()`. And rename the old
`report_lint()` to `report_lint_old()`.
 2025-04-10 13:21:00 -04:00
Andrew Gallant 81045758d3 ruff_db: use `Annotation::get_message` in more places 
I added this accessor because tests want it, but we can also use it in
other places internally. It's a little nicer because it does the
`as_deref()` for you.
 2025-04-10 13:21:00 -04:00
Andrew Gallant 0b2a5b5cd3 red_knot_python_semantic: tweak docs on building reporter builders 2025-04-10 13:21:00 -04:00
Andrew Gallant 7d958a9ee5 red_knot_python_semantic: remove the "old" secondary message type 
This finally completes the deletion of all old diagnostic types.

We do this by migrating the second (and last) use of secondary
diagnostic messages: to highlight the return type of a function
definition when its return value is inconsistent with the type.

Like the last diagnostic, we do actually change the message here a bit.
We don't need a sub-diagnostic here, and we can instead just add a
secondary annotation to highlight the return type.
 2025-04-10 13:21:00 -04:00
Andrew Gallant 7e2eb591bc red_knot_python_semantic: replace one use of "old" secondary diagnostic messages 
This is the first use of the new `lint()` reporter.

I somewhat skipped a step here and also modified the actual diagnostic
message itself. The snapshots should tell the story.

We couldn't do this before because we had no way of differentiating
between "message for the diagnostic as a whole" and "message for a
specific code annotation." Now we can, so we can write more precise
messages based on the assumption that users are also seeing the code
snippet.

The downside here is that the actual message text can become quite vague
in the absence of the code snippet. This occurs, for example, with
concise diagnostic formatting. It's unclear if we should do anything
about it. I don't really see a way to make it better that doesn't
involve creating diagnostics with messages for each mode, which I think
would be a major PITA.

The upside is that this code gets a bit simpler, and we very
specifically avoid doing extra work if this specific lint is disabled.
 2025-04-10 13:21:00 -04:00
Andrew Gallant ba408f4231 red_knot_python_semantic: update revealed type snapshots 
This required a bit of surgery in the diagnostic matching and more
faffing about using a "concise" message from a diagnostic instead of
only printing the "primary" message.
 2025-04-10 13:21:00 -04:00
Andrew Gallant 28b64064f5 ruff_db: tweak how the revealed type diagnostic is rendered 
In the new diagnostic data model, we really should have a main
diagnostic message *and* a primary span (with an optional message
attached to it) for every diagnostic.

In this commit, I try to make this true for the "revealed type"
diagnostic. Instead of the annotation saying both "revealed type is"
and also the revealed type itself, the annotation is now just the
revealed type and the main diagnostic message is "Revealed type."

I expect this may be controversial. I'm open to doing something
different. I tried to avoid redundancy, but maybe this is a special case
where we want the redundancy. I'm honestly not sure. I do *like* how it
looks with this commit, but I'm not working with Red Knot's type
checking daily, so my opinion doesn't count for much.

This did also require some tweaking to concise diagnostic formatting in
order to preserve the essential information.

This commit doesn't update every relevant snapshot. Just a few. I split
the rest out into the next commit.
 2025-04-10 13:21:00 -04:00
Andrew Gallant 75b15ea2d0 red_knot: add explicit test for concise `reveal_type` diagnostic 
This test reflects the status quo before we change things.
 2025-04-10 13:21:00 -04:00
Andrew Gallant e7e86b8584 red_knot_python_semantic: remove `InferContext::report_diagnostic` 
... and replace it with use of `report()`.

Interestingly, this is the only instance of `report_diagnostic` used
directly, and thus anticipated to be the only instance of using
`report()`. If this ends up being a true single use method, we could
make it less generic and tailored specifically to "reveal type."

Two other things to note:

I left the "primary message" as empty. This avoids changing snapshots.
I address this in a subsequent commit.

The creation of a diagnostic here is a bit verbose/annoying. Certainly
more so than it was. This is somewhat expected since our diagnostic
model is more expressive and because we don't have a proc macro. I
avoided creating helpers for this case since there's only one use of
`report()`. But I expect to create helpers for the `lint()` case.
 2025-04-10 13:21:00 -04:00
Andrew Gallant f84bc07d56 red_knot_python_semantic: add "reporter" API 
This is a surgical change that adds new `report()` and `lint()`
APIs to `InferContext`. These are intended to replace the existing
`report_*` APIs.

The comments should explain what these reporters are meant to do. For
the most part, this is "just" shuffling some code around. The actual
logic for determining whether a lint *should* be reported or not remains
unchanged and we don't make any changes to how a `Diagnostic` is
actually constructed (yet).

I initially tried to just use `LintReporter` and `DiagnosticReporter`
without the builder types, since I perceive the builder types to be an
annoying additional layer. But I found it also exceedingly annoying to
have to construct and provide the diagnostic message before you even
know if you are going to build the diagnostic. I also felt like this
could result in potentially unnecessary and costly querying in some
cases, although this is somewhat hand wavy. So I overall felt like the
builder route was the way to go. If the builders end up being super
annoying, we can probably add convenience APIs for common patterns to
paper over them.
 2025-04-10 13:21:00 -04:00
Dylan 7186d5e9ad
Bump 0.11.5 (#17337) 2025-04-10 11:57:44 -05:00
David Peter 5b6e94981d
[red-knot] Silence `unresolved-attribute` in unreachable code (#17305) 
## Summary

Basically just repeat the same thing that we did for
`unresolved-reference`, but now for attribute expressions.

We now also handle the case where the unresolved attribute (or the
unresolved reference) diagnostic originates from a stringified type
annotation.

And I made the evaluation of reachability constraints lazy (will only be
evaluated right before we are about to emit a diagnostic).

## Test Plan

New Markdown tests for stringified annotations.
 2025-04-10 17:15:47 +02:00
Carl Meyer ec74f2d522
Revert "[red-knot] Type narrowing for assertions (#17149)" (#17335) 
I merged #17149 without checking the ecosystem results, and it still
caused a cycle panic in pybind11. Reverting for now until I fix that, so
we don't lose the ecosystem signal on other PRs.
 2025-04-10 11:06:25 -04:00
Matthew Mckee 907b6ed7b5
[red-knot] Type narrowing for assertions (#17149) 
## Summary

Fixes #17147 

## Test Plan

Add new narrow/assert.md test file

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-04-10 10:15:52 -04:00
Carl Meyer fd9882a1f4
[red-knot] avoid unnecessary evaluation of visibility constraint on definitely-unbound symbol (#17326) 
This causes spurious query cycles.

This PR also includes an update to Salsa, which gives us db events on
cycle iteration, so we can write tests asserting the absence of a cycle.
 2025-04-10 13:59:38 +00:00
David Peter 4d50ee6f52
[red-knot] Track reachability of scopes (#17332) 
## Summary

Track the reachability of nested scopes within their parent scopes. We
use this as an additional requirement for emitting
`unresolved-reference` diagnostics (and in the future,
`unresolved-attribute` and `unresolved-import`). This means that we only
emit `unresolved-reference` for a given use of a symbol if the use
itself is reachable (within its own scope), *and if the scope itself is
reachable*. For example, no diagnostic should be emitted for the use of
`x` here:

```py
if False:
    x = 1

    def f():
        print(x)  # this use of `x` is reachable inside the `f` scope,
                  # but the whole `f` scope is not reachable.
```

There are probably more fine-grained ways of solving this problem, but
they require a more sophisticated understanding of nested scopes (see
#15777, in particular
https://github.com/astral-sh/ruff/issues/15777#issuecomment-2788950267).
But it doesn't seem completely unreasonable to silence *this specific
kind of error* in unreachable scopes.

## Test Plan

Observed changes in reachability tests and ecosystem.
 2025-04-10 11:56:40 +00:00
Matthew Mckee 10e44124e6
[red-knot] Add inlay type hints (#17214) 
Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-04-10 09:21:40 +00:00
Micha Reiser 9f6913c488
[red-knot] Update salsa (#17320) 
## Summary

Update Salsa to pull in https://github.com/salsa-rs/salsa/pull/788 which
fixes the, by now, famous *access to field whilst the value is being
initialized*.

This PR also re-enables all tests that previously triggered the panic.

## Test Plan

`cargo test`
 2025-04-09 16:22:02 -04:00
David Peter 5fef4d4572
Use python.typing.org for typing documentation links (#17323) 
## Summary

There is a new official URL for the typing documentation:
https://typing.python.org/

Change all https://typing.readthedocs.io/ links to use the new sub
domain, which is slightly shorter and looks more official.

## Test Plan

Tested to see if each and every new URL is accessible. I noticed that
some links go to https://typing.python.org/en/latest/source/stubs.html
which seems to be outdated, but that is a separate issue. The same page
shows up for the old URL.
 2025-04-09 20:38:20 +02:00
Brent Westbrook 144484d46c
Refactor semantic syntax error scope handling (#17314) 
## Summary

Based on the discussion in
https://github.com/astral-sh/ruff/pull/17298#discussion_r2033975460, we
decided to move the scope handling out of the `SemanticSyntaxChecker`
and into the `SemanticSyntaxContext` trait. This PR implements that
refactor by:

- Reverting all of the `Checkpoint` and `in_async_context` code in the
`SemanticSyntaxChecker`
- Adding four new methods to the `SemanticSyntaxContext` trait
- `in_async_context`: matches `SemanticModel::in_async_context` and only
detects the nearest enclosing function
- `in_sync_comprehension`: uses the new `is_async` tracking on
`Generator` scopes to detect any enclosing sync comprehension
  - `in_module_scope`: reports whether we're at the top-level scope
  - `in_notebook`: reports whether we're in a Jupyter notebook
- In-lining the `TestContext` directly into the
`SemanticSyntaxCheckerVisitor`
- This allows modifying the context as the visitor traverses the AST,
which wasn't possible before

One potential question here is "why not add a single method returning a
`Scope` or `Scopes` to the context?" The main reason is that the `Scope`
type is defined in the `ruff_python_semantic` crate, which is not
currently a dependency of the parser. It also doesn't appear to be used
in red-knot. So it seemed best to use these more granular methods
instead of trying to access `Scope` in `ruff_python_parser` (and
red-knot).

## Test Plan

Existing parser and linter tests.
 2025-04-09 14:23:29 -04:00
Wei Lee c87e3ccb2f
[`airflow`] Add missing `AIR302` attribute check (#17115) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

attribute check was missing in the previous implementation

e.g.

```python
from airflow.api.auth.backend import basic_auth

basic_auth.auth_current_user
```

This PR adds this kind of check.

## Test Plan

<!-- How was it tested? -->

The test case has been added to the button of the existing test
fixtures, confirmed to be correct and later reorgnaized
 2025-04-09 13:58:41 -04:00
Alex Waygood 781b653511
[red-knot] Fix false positives on `types.UnionType` instances in type expressions (#17297) 2025-04-09 18:33:16 +01:00
Brent Westbrook 2fbc4d577e
[syntax-errors] Document behavior of `global` declarations in `try` nodes before 3.13 (#17285) 
Summary
--

This PR extends the documentation of the `LoadBeforeGlobalDeclaration`
check to specify the behavior on versions of Python before 3.13. Namely,
on Python 3.12, the `else` clause of a `try` statement is visited before
the `except` handlers:

```pycon
Python 3.12.9 (main, Feb 12 2025, 14:50:50) [Clang 19.1.6 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = 10
>>> def g():
...     try:
...         1 / 0
...     except:
...         a = 1
...     else:
...         global a
...
>>> def f():
...     try:
...         pass
...     except:
...         global a
...     else:
...         print(a)
...
  File "<stdin>", line 5
SyntaxError: name 'a' is used prior to global declaration

```

The order is swapped on 3.13 (see
[CPython#111123](https://github.com/python/cpython/issues/111123)):

```pycon
Python 3.13.2 (main, Feb  5 2025, 08:05:21) [GCC 14.2.1 20250128] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = 10
... def g():
...     try:
...         1 / 0
...     except:
...         a = 1
...     else:
...         global a
...
  File "<python-input-0>", line 8
    global a
    ^^^^^^^^
SyntaxError: name 'a' is assigned to before global declaration
>>> def f():
...     try:
...         pass
...     except:
...         global a
...     else:
...         print(a)
...
>>>
```

The current implementation of PLE0118 is correct for 3.13 but not 3.12:
[playground](https://play.ruff.rs/d7467ea6-f546-4a76-828f-8e6b800694c9)
(it flags the first case regardless of Python version).

We decided to maintain this incorrect diagnostic for Python versions
before 3.13 because the pre-3.13 behavior is very unintuitive and
confirmed to be a bug, although the bug fix was not backported to
earlier versions. This can lead to false positives and false negatives
for pre-3.13 code, but we also expect that to be very rare, as
demonstrated by the ecosystem check (before the version-dependent check
was reverted here).

Test Plan
--

N/a
 2025-04-09 12:54:21 -04:00
Alex Waygood 73399029b2
[red-knot] Optimise visibility constraints for `*`-import definitions (#17317) 2025-04-09 16:53:26 +00:00
Douglas Creager ff376fc262
[red-knot] Allow explicit specialization of generic classes (#17023) 
This PR lets you explicitly specialize a generic class using a subscript
expression. It introduces three new Rust types for representing classes:

- `NonGenericClass`
- `GenericClass` (not specialized)
- `GenericAlias` (specialized)

and two enum wrappers:

- `ClassType` (a non-generic class or generic alias, represents a class
_type_ at runtime)
- `ClassLiteralType` (a non-generic class or generic class, represents a
class body in the AST)

We also add internal support for specializing callables, in particular
function literals. (That is, the internal `Type` representation now
attaches an optional specialization to a function literal.) This is used
in this PR for the methods of a generic class, but should also give us
most of what we need for specializing generic _functions_ (which this PR
does not yet tackle).

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-04-09 11:18:46 -04:00
Wei Lee 7c81408c54
[`airflow`] Refactor `AIR301` logic and fix typos (`AIR301`) (#17293) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

* Simplify match conditions in AIR301
* Fix
* `airflow.datasets.manager.DatasetManager` →
`airflow.assets.manager.AssetManager`
* `airflow.www.auth.has_access_dataset` →
`airflow.www.auth.has_access_dataset`

## Test Plan

<!-- How was it tested? -->
The test fixture has been updated accordingly
 2025-04-09 10:46:17 -04:00
Wei Lee 7207c86971
[`airflow`] Extract `AIR312` from `AIR302` rules (`AIR302`, `AIR312`) (#17152) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

As discussed in
https://github.com/astral-sh/ruff/issues/14626#issuecomment-2766146129,
we're to separate suggested changes from required changes.

The following symbols has been moved to AIR312 from AIR302. They still
work in Airflow 3.0, but they're suggested to be changed as they're
expected to be removed in future version

```python
from airflow.hooks.filesystem import FSHook
from airflow.hooks.package_index import PackageIndexHook
from airflow.hooks.subprocess import (SubprocessHook, SubprocessResult, working_directory)
from airflow.operators.bash import BashOperator
from airflow.operators.datetime import BranchDateTimeOperator, target_times_as_dates
from airflow.operators.trigger_dagrun import TriggerDagRunLink, TriggerDagRunOperator
from airflow.operators.empty import EmptyOperator
from airflow.operators.latest_only import LatestOnlyOperator
from airflow.operators.python import (BranchPythonOperator, PythonOperator, PythonVirtualenvOperator, ShortCircuitOperator)
from airflow.operators.weekday import BranchDayOfWeekOperator
from airflow.sensors.date_time import DateTimeSensor, DateTimeSensorAsync
from airflow.sensors.external_task import ExternalTaskMarker, ExternalTaskSensor, ExternalTaskSensorLink
from airflow.sensors.filesystem import FileSensor
from airflow.sensors.time_sensor import TimeSensor, TimeSensorAsync
from airflow.sensors.time_delta import TimeDeltaSensor, TimeDeltaSensorAsync, WaitSensor
from airflow.sensors.weekday import DayOfWeekSensor
from airflow.triggers.external_task import DagStateTrigger, WorkflowTrigger
from airflow.triggers.file import FileTrigger
from airflow.triggers.temporal import DateTimeTrigger, TimeDeltaTrigger
```

## Test Plan

<!-- How was it tested? -->

The test fixture has been updated acccordingly

---------

Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
 2025-04-09 10:43:07 -04:00
Alex Waygood 6ec4c6a97e
[red-knot] Improve handling of visibility constraints in external modules when resolving `*` imports (#17286) 2025-04-09 14:36:52 +00:00
Alex Waygood f1ba596f22
[red-knot] Add more tests for `*` imports (#17315) 2025-04-09 15:10:30 +01:00
Micha Reiser 8249a72412
[red-knot] Default `python-platform` to current platform (#17183) 
## Summary

As discussed in https://github.com/astral-sh/ruff/issues/16983 and
"mitigate" said issue for the alpha.

This PR changes the default for `PythonPlatform` to be the current
platform rather than `all`.

I'm not sure if we should be as sophisticated as supporting `ios` and
`android` as defaults but it was easy...

## Test Plan

Updated Markdown tests.

---------

Co-authored-by: David Peter <mail@david-peter.de>
 2025-04-09 12:05:18 +02:00
David Peter 00e00b9ad6
[red-knot] Add new 'unreachable code' test case (#17306) 
## Summary

This is a new test case that I don't know how to handle yet. It leads to
many false positives in `rich/tests/test_win32_console.py`, which does
something like:

```py
if sys.platform == "win32":
    from windows_only_module import some_symbol

    some_other_symbol = 1

    def some_test_case():
        use(some_symbol)  # Red Knot: unresolved-reference
        use(some_other_symbol)  # Red Knot: unresolved-reference
```

Also adds a test for using unreachable symbols in type annotations or as
class bases.
 2025-04-09 11:45:42 +02:00
David Peter 2cee86d807
[red-knot] Add custom `__setattr__` support (#16748) 
## Summary

Add support for classes with a custom `__setattr__` method.

## Test Plan

New Markdown tests, ecosystem checks.
 2025-04-09 08:04:11 +02:00
Mike Perlov fab7d820bd
[red-knot] Add `__init__` arguments check when doing `try_call` on a class literal (#16512) 
## Summary

* Addresses #16511 for simple cases where only `__init__` method is
bound on class or doesn't exist at all.
* fixes a bug with argument counting in bound method diagnostics

Caveats:
* No handling of `__new__` or modified `__call__` on metaclass.
* This leads to a couple of false positive errors in tests

## Test Plan

- A couple new cases in mdtests
- cargo nextest run -p red_knot_python_semantic --no-fail-fast

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
 2025-04-08 17:26:20 -04:00
Denys Kyslytsyn ed14dbb1a2
[`flake8-pie`] Avoid false positive for multiple assignment with `auto()` (`PIE796`) (#17274) 
This fix closes #16868 

I noticed the issue is assigned, but the assignee appears to be actively
working on another pull request. I hope that’s okay!

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

As of Python 3.11.1, `enum.auto()` can be used in multiple assignments.
This pattern should not trigger non-unique-enums check.
Reference: [Python docs on
enum.auto()](https://docs.python.org/3/library/enum.html#enum.auto)

This fix updates the check logic to skip enum variant statements where
the right-hand side is a tuple containing a call to `enum.auto()`.

## Test Plan

<!-- How was it tested? -->

The added test case uses the example from the original issue. It
previously triggered a false positive, but now passes successfully.
 2025-04-08 15:53:27 -04:00
Brent Westbrook 058439d5d3
[syntax-errors] Async comprehension in sync comprehension (#17177) 
Summary
--

Detect async comprehensions nested in sync comprehensions in async
functions before Python 3.11, when this was [changed].

The actual logic of this rule is very straightforward, but properly
tracking the async scopes took a bit of work. An alternative to the
current approach is to offload the `in_async_context` check into the
`SemanticSyntaxContext` trait, but that actually required much more
extensive changes to the `TestContext` and also to ruff's semantic
model, as you can see in the changes up to
31554b473507034735bd410760fde6341d54a050. This version has the benefit
of mostly centralizing the state tracking in `SemanticSyntaxChecker`,
although there was some subtlety around deferred function body traversal
that made the changes to `Checker` more intrusive too (hence the new
linter test).

The `Checkpoint` struct/system is obviously overkill for now since it's
only tracking a single `bool`, but I thought it might be more useful
later.

[changed]: https://github.com/python/cpython/issues/77527

Test Plan
--

New inline tests and a new linter integration test.
 2025-04-08 12:50:52 -04:00
Wei Lee dc02732d4d
[`airflow`] Expand module path check to individual symbols (`AIR302`) (#17278) 
## Summary

### Improvement
Expand the following moved module into individual symbols.

* airflow.triggers.temporal
* airflow.triggers.file
* airflow.triggers.external_task
* airflow.hooks.subprocess
* airflow.hooks.package_index
* airflow.hooks.filesystem
* airflow.sensors.weekday
* airflow.sensors.time_delta
* airflow.sensors.time_sensor
* airflow.sensors.date_time
* airflow.operators.weekday
* airflow.operators.datetime
* airflow.operators.bash 

This removes `Replacement::ImportPathMoved`.

## Fix
During the expansion, the following paths were also fixed

* airflow.sensors.s3_key_sensor.S3KeySensor →
airflow.providers.amazon.aws.sensors.S3KeySensor
* airflow.operators.sql.SQLThresholdCheckOperator →
airflow.providers.common.sql.operators.sql.SQLThresholdCheckOperator
* airflow.hooks.druid_hook.DruidDbApiHook →
airflow.providers.apache.druid.hooks.druid.DruidDbApiHook
* airflow.hooks.druid_hook.DruidHook →
airflow.providers.apache.druid.hooks.druid.DruidHook
* airflow.kubernetes.pod_generator.extend_object_field →
airflow.providers.cncf.kubernetes.pod_generator.extend_object_field
* airflow.kubernetes.pod_launcher.PodLauncher →
airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodLauncher
* airflow.kubernetes.pod_launcher.PodStatus →
airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodStatus
* airflow.kubernetes.pod_generator.PodDefaults →
airflow.providers.cncf.kubernetes.pod_generator.PodDefaults
* airflow.kubernetes.pod_launcher_deprecated.PodDefaults →
airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodDefaults

### Refactor
As many symbols are moved into the same module,
`SourceModuleMovedToProvider` is introduced for grouping similar logic

## Test Plan
 2025-04-08 09:03:27 -04:00
Brent Westbrook 0891689d2f
[syntax-errors] Check annotations in annotated assignments (#17283) 
Summary
--

This PR extends the checks in #17101 and #17282 to annotated assignments
after Python 3.13.

Currently stacked on #17282 to include `await`.

Test Plan
--

New inline tests. These are simpler than the other cases because there's
no place to put generics.
 2025-04-08 08:56:25 -04:00
Brent Westbrook 127a45622f
[syntax-errors] Extend annotation checks to `await` (#17282) 
Summary
--

This PR extends the changes in #17101 to include `await` in the same
positions.

I also renamed the `valid_annotation_function` test to include `_py313`
and explicitly passed a Python version to contrast it with the `_py314`
version.

Test Plan
--

New test cases added to existing files.
 2025-04-08 08:55:43 -04:00
David Peter b662c3ff7e
[red-knot] Add support for `assert_never` (#17287) 
## Summary

We already have partial "support" for `assert_never`, because it is
annotated as
```pyi
def assert_never(arg: Never, /) -> Never: ...
```
in typeshed. So we already emit a `invalid-argument-type` diagnostic if
the argument type to `assert_never` is not assignable to `Never`.

That is not enough, however. Gradual types like `Any`, `Unknown`,
`@Todo(…)` or `Any & int` can be assignable to `Never`. Which means that
we didn't issue any diagnostic in those cases.

Also, it seems like `assert_never` deserves a dedicated diagnostic
message, not just a generic "invalid argument type" error.

## Test Plan

New Markdown tests.
 2025-04-08 09:31:49 +02:00
Denys Kyslytsyn 97dd6d120c
[`flake8-pytest-style`] Avoid false positive for legacy form of `pytest.raises` (`PT011`) (#17231) 
This fix closes #17026 

## Summary

The check for the `PytestRaisesTooBroad` rule is now skipped if there is
a second positional argument present, which means `pytest.raises` is
used as a function.

## Test Plan

Tested on the example from the issue, which now passes the check.
```Python3
pytest.raises(Exception, func, *func_args, **func_kwargs).match("error message")
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-04-08 09:24:47 +02:00
InSync 34e06f2d17
[red-knot] Do not show types for literal expressions on hover (#17290) 
## Summary

Resolves #17289.

After this change, Red Knot will no longer show types on hover for
`None`, `...`, `True`, `False`, numbers, strings (but not f-strings),
and bytes literals.

## Test Plan

Unit tests.
 2025-04-08 09:05:51 +02:00
David Peter a388c73752
[red-knot] Fix dead-code clippy warning (#17291) 
## Summary

Failed run on main:
https://github.com/astral-sh/ruff/actions/runs/14326812317
 2025-04-08 08:56:59 +02:00
David Peter 60f2e67454
[red-knot] Reachability analysis (#17199) 
## Summary

This implements a new approach to silencing `unresolved-reference`
diagnostics by keeping track of the reachability of each use of a
symbol. The changes merged in
https://github.com/astral-sh/ruff/pull/17169 are still needed for the
"Use of variable in nested function" test case, but that could also be
solved in another way eventually (see
https://github.com/astral-sh/ruff/issues/15777). We can use the same
technique to silence `unresolved-import` and `unresolved-attribute`
false-positives, but I think this could be merged in isolation.

## Test Plan

New Markdown tests, ecosystem tests
 2025-04-08 08:37:20 +02:00
Micha Reiser cb7f56fb20
[red-knot] Don't use latency-sensitive for handlers (#17227) 
## Summary

The priority latency-sensitive is reserved for actions that need to run
immediately because they would otherwise block the user's action. An
example of this is a format request. VS code blocks the editor until the
save action is complete. That's why formatting a document is very
sensitive to delays and it's important that we always have a worker
thread available to run a format request *immediately*. Another example
are code completions, where it's important that they appear immediately
when the user types.

On the other hand, showing diagnostics, hover, or inlay hints has high
priority but users are used that the editor takes a few ms to compute
the overlay.
Computing this information can also be expensive (e.g. find all
references), blocking the worker for quiet some time (a few 100ms).
That's why it's important
that those requests don't clog the sensitive worker threads.
 2025-04-08 08:33:30 +02:00
David Peter 3657f798c9
[red-knot] Add `--python-platform` CLI option (#17284) 
## Summary

Add a new `--python-platform` command-line option, in analogy to
`--python-version`.

## Test Plan

Added new integration test.
 2025-04-07 21:04:44 +02:00
Matthew Mckee 4a4a376f02
[red-knot] Allow ellipsis default params in stub functions (#17243) 
## Summary

Fixes #17234

## Test Plan

Add tests to functions/paremeters.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-04-07 17:34:59 +00:00
Max Mynter f1d4ba32cc
Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061) 
Closes #17042

## Summary
This PR fixes the issue outlined in #17042 where RUF100 (unused-noqa)
fails to detect unused file-level noqa directives (`# ruff: noqa` or `#
ruff: noqa: {code}`).

The issue stems from two underlying causes:

1. For blanket file-level directives (`# ruff: noqa`), there's a
circular dependency: the directive exempts all rules including RUF100
itself, which prevents checking for usage. This isn't changed by this
PR. I would argue it is intendend behavior - a blanket `# ruff: noqa`
directive should exempt all rules including RUF100 itself.

2. For code-specific file-level directives (e.g. `# ruff: noqa: F841`),
the handling was missing in the `check_noqa` function. This is added in
this PR.

## Notes
- For file-level directives, the `matches` array is pre-populated with
the specified codes during parsing, unlike line-level directives which
only populate their `matches` array when actually suppressing
diagnostics. This difference requires the somewhat clunky handling of
both cases. I would appreciate guidance on a cleaner design :)

- A more fundamental solution would be to change how file-level
directives initialize the `matches` array in
`FileNoqaDirectives::extract()`, but that requires more substantial
changes as it breaks existing functionality. I suspect discussions in
#16483 are relevant for this.

## Test Plan
- Local verification
- Added a test case and fixture
 2025-04-07 09:21:52 -05:00
Wei Lee 1e9e423362
[`airflow`] Update oudated `AIR301`, `AIR302` rules (#17123) 
## Summary

Some of the migration rules has been changed during Airflow 3
development. The following are new AIR302 rules. Corresponding AIR301
has also been removed.

* airflow.sensors.external_task_sensor.ExternalTaskMarker →
airflow.providers.standard.sensors.external_task.ExternalTaskMarker
* airflow.sensors.external_task_sensor.ExternalTaskSensor →
airflow.providers.standard.sensors.external_task.ExternalTaskSensor
* airflow.sensors.external_task_sensor.ExternalTaskSensorLink →
airflow.providers.standard.sensors.external_task.ExternalTaskSensorLink
* airflow.sensors.time_delta_sensor.TimeDeltaSensor →
airflow.providers.standard.sensors.time_delta.TimeDeltaSensor
* airflow.operators.dagrun_operator.TriggerDagRunLink →
airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunLink
* airflow.operators.dagrun_operator.TriggerDagRunOperator →
airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunOperator
* airflow.operators.python_operator.BranchPythonOperator →
airflow.providers.standard.operators.python.BranchPythonOperator
* airflow.operators.python_operator.PythonOperator →
airflow.providers.standard.operators.python.PythonOperator
* airflow.operators.python_operator.PythonVirtualenvOperator →
airflow.providers.standard.operators.python.PythonVirtualenvOperator
* airflow.operators.python_operator.ShortCircuitOperator →
airflow.providers.standard.operators.python.ShortCircuitOperator
* airflow.operators.latest_only_operator.LatestOnlyOperator →
airflow.providers.standard.operators.latest_only.LatestOnlyOperator
* airflow.sensors.date_time_sensor.DateTimeSensor →
airflow.providers.standard.sensors.DateTimeSensor
* airflow.operators.email_operator.EmailOperator →
airflow.providers.smtp.operators.smtp.EmailOperator
* airflow.operators.email.EmailOperator →
airflow.providers.smtp.operators.smtp.EmailOperator
* airflow.operators.bash.BashOperator →
airflow.providers.standard.operators.bash.BashOperator
* airflow.operators.EmptyOperator →
airflow.providers.standard.operators.empty.EmptyOperator

closes: https://github.com/astral-sh/ruff/issues/17103

## Test Plan

The test fixture has been updated and checked after each change and
later reorganized in the latest commit
 2025-04-07 09:45:56 -04:00
Micha Reiser 6cc2d02dfa
[red-knot] Support stub packages (#17204) 
## Summary

This PR adds support for stub packages, except for partial stub packages
(a stub package is always considered non-partial).

I read the specification at
[typing.python.org/en/latest/spec/distributing.html#stub-only-packages](https://typing.python.org/en/latest/spec/distributing.html#stub-only-packages)
but I found it lacking some details, especially on how to handle
namespace packages or when the regular and stub packages disagree on
whether they're namespace packages. I tried to document my decisions in
the mdtests where the specification isn't clear and compared the
behavior to Pyright.

Mypy seems to only support stub packages in the venv folder. At least,
it never picked up my stub packages otherwise. I decided not to spend
too much time fighting mypyp, which is why I focused the comparison
around Pyright

Closes https://github.com/astral-sh/ruff/issues/16612

## Test plan

Added mdtests
 2025-04-07 14:40:50 +02:00
Andrew Gallant c12c76e9c8 ruff_annotate_snippets: address unused code warnings 
Fixes #17230
 2025-04-07 08:24:08 -04:00
Alex Waygood 81cf860dc8
[red-knot] Add a couple more tests for `*` imports (#17270) 
## Summary

Some more edge cases that I thought of while working on integrating
knowledge of statically known branches into the `*`-import machinery

## Test Plan

`cargo test -p red_knot_python_semantic`
 2025-04-07 11:12:28 +00:00
Micha Reiser 3150812ac4
[red-knot] Add 'Format document' to playground (#17217) 
## Summary
This is more "because we can" than something we need. 

But since we're already building an "almost IDE" 

## Test Plan



https://github.com/user-attachments/assets/3a4bdad1-ba32-455a-9909-cfeb8caa1b28
 2025-04-07 09:26:03 +02:00
Micha Reiser 67f8c30b53
[red-knot] Remove global `dead_code` from red-knot-server (#17229) 
## Summary

Use more local `expect(dead_code)` suppressions instead of a global
`allow(dead_code)` in `lib.rs`.

Remove some methods that are either easy to add later, are less likely
to be needed for red knot, or it's unclear if we'd add it the same way
as in ruff.
 2025-04-06 22:09:24 +01:00
Alex Waygood ac5d220d75
[red-knot] Fix `python` setting in mdtests, and rewrite a `site-packages` test as an mdtest (#17222) 
## Summary

This PR does the following things:
- Fixes the `python` configuration setting for mdtest (added in
https://github.com/astral-sh/ruff/pull/17221) so that it expects a path
pointing to a venv's `sys.prefix` variable rather than the a path
pointing to the venv's `site-packages` subdirectory. This brings the
`python` setting in mdtest in sync with our CLI `--python` flag.
- Tweaks mdtest so that it automatically creates a valid `pyvenv.cfg`
file for you if you don't specify one. This makes it much more ergonomic
to write an mdtest with a custom `python` setting: red-knot will reject
a `python` setting that points to a directory that doesn't have a
`pyvenv.cfg` file in it
- Tweaks mdtest so that it doesn't check a custom `pyvenv.cfg` as Python
source code if you _do_ add a custom `pyvenv.cfg` file for your mock
virtual environment in an mdtest. (You get a lot of diagnostics about
Python syntax errors in the `pyvenv.cfg` file, otherwise!)
- Rewrites the test added in
https://github.com/astral-sh/ruff/pull/17178 as an mdtest, and deletes
the original test that was added in that PR

## Test Plan

I verified that the new mdtest fails if I revert the changes to
`resolver.rs` that were added in
https://github.com/astral-sh/ruff/pull/17178
 2025-04-06 18:24:32 +01:00
Matthew Mckee 73a9974d8a
Fix `CallableTypeOf` display signature (#17235) 2025-04-06 18:12:52 +01:00
Micha Reiser 1c652e6b98
[red-knot] Allow setting `python` in mdtests (#17221) 
## Summary

This PR extends the mdtest options to allow setting the
`environment.python` option.

## Test Plan

I let @AlexWaygood write a test and he'll tell me if it works 😆
 2025-04-05 16:43:31 +02:00
cake-monotone 7e6d3838bd
[red-knot] Add cycle handling to narrow constraints queries (#17209) 
## Summary

This PR fixes the cycle issue that was causing problems in the `support
super` PR.

### Affected queries
- `all_narrowing_constraints_for_expression`
- `all_negative_narrowing_constraints_for_expression`


--

Additionally, `bidict` and `werkzeug` have been added to the
project-selection list in `mypy_primer`.
This PR also addresses the panics that occurred while analyzing those
packages:

- `bidict`: panic triggered by
`all_narrowing_constraints_for_expression`
- `werkzeug`: panic triggered by
`all_negative_narrowing_constraints_for_expression`

I think the mypy-primer results for this PR can serve as sufficient test
:)
 2025-04-04 22:26:20 -07:00
David Peter 1a6a10b30f
[red-knot] Empty tuple is always-falsy (#17213) 
## Summary

Fix assignability of `tuple[()]` to `AlwaysFalsy`.

closes #17202 

## Test Plan

Ran the property tests for a while
 2025-04-04 22:00:28 +02:00
Brent Westbrook 95d6ed40cc
Bump 0.11.4 (#17212) 2025-04-04 14:09:10 -04:00
Brent Westbrook acc5662e8b
[syntax-errors] Allow `yield` in base classes and annotations (#17206) 
Summary
--

This PR fixes the issue pointed out by @JelleZijlstra in
https://github.com/astral-sh/ruff/pull/17101#issuecomment-2777480204.
Namely, I conflated two very different errors from CPython:

```pycon
>>> def m[T](x: (yield from 1)): ...
  File "<python-input-310>", line 1
    def m[T](x: (yield from 1)): ...
                 ^^^^^^^^^^^^
SyntaxError: yield expression cannot be used within the definition of a generic
>>> def m(x: (yield from 1)): ...
  File "<python-input-311>", line 1
    def m(x: (yield from 1)): ...
              ^^^^^^^^^^^^
SyntaxError: 'yield from' outside function
>>> def outer():
...     def m(x: (yield from 1)): ...
...
>>>
```

I thought the second error was the same as the first, but `yield` (and
`yield from`) is actually valid in this position when inside a function
scope. The same is true for base classes, as pointed out in the original
comment.

We don't currently raise an error for `yield` outside of a function, but
that should be handled separately.

On the upside, this had the benefit of removing the
`InvalidExpressionPosition::BaseClass` variant and the
`allow_named_expr` field from the visitor because they were both no
longer used.

Test Plan
--

Updated inline tests.
 2025-04-04 13:48:28 -04:00
David Salvisberg 33a56f198b
Don't skip visiting non-tuple slice in `typing.Annotated` subscripts (#17201) 
Fixes: #17196

## Summary

Skipping these nodes for malformed type expressions would lead to
incorrect semantic state, which can in turn mean we emit false positives
for rules like `unused-variable`(`F841`)

## Test Plan

`cargo nextest run`
 2025-04-04 18:47:40 +05:30
Micha Reiser ffa824e037
[red-knot] Add `Type.definition` method (#17153) 
## Summary

This is a follow up to the goto type definition PR. Specifically, that
we want to avoid exposing too many semantic model internals publicly.

I want to get some feedback on the approach taken. I think it goes into
the right direction but I'm not super happy with it.
The basic idea is that we add a `Type::definition` method which does the
"goto type definition". The parts that I think make it awkward:

* We can't directly return `Definition` because we don't create a
`Definition` for modules (but we could?). Although I think it makes
sense to possibly have a more public wrapper type anyway?
* It doesn't handle unions and intersections. Mainly because not all
elements in an intersection may have a definition and we only want to
show a navigation target for intersections if there's only a single
positive element (besides maybe `Unknown`).


An alternative design or an addition to this design is to introduce a
`SemanticAnalysis(Db)` struct that has methods like
`type_definition(&self, type)` which explicitly exposes the methods we
want. I don't feel comfortable design this API yet because it's unclear
how fine granular it has to be (and if it is very fine granular,
directly using `Type` might be better after all)


## Test Plan

`cargo test`
 2025-04-04 10:56:20 +02:00
Max Mynter 98b95c9c38
Implement `Invalid rule provided` as rule RUF102 with `--fix` (#17138) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

Closes #17084

## Summary
This PR adds a new rule (RUF102) to detect and fix invalid rule codes in
`noqa` comments.
Invalid rule codes in `noqa` directives serve no purpose and may
indicate outdated code suppressions.

This extends the previous behaviour originating from
`crates/ruff_linter/src/noqa.rs` which would only emit a warnigs.
With this rule a `--fix` is available.

The rule:
1. Analyzes all `noqa` directives to identify invalid rule codes
2. Provides autofix functionality to:
   - Remove the entire comment if all codes are invalid
   - Remove only the invalid codes when mixed with valid codes
3. Preserves original comment formatting and whitespace where possible

Example cases:
- `# noqa: XYZ111` → Remove entire comment (keep empty line)
- `# noqa: XYZ222, XYZ333` → Remove entire comment (keep empty line)
-  `# noqa: F401, INVALID123` → Keep only valid codes (`# noqa: F401`)

## Test Plan
- Added tests in
`crates/ruff_linter/resources/test/fixtures/ruff/RUF102.py` covering
different example cases.

<!-- How was it tested? -->

## Notes 
- This does not handle cases where parsing fails. E.g. `# noqa:
NON_EXISTENT, ANOTHER_INVALID` causes a `LexicalError` and the
diagnostic is not propagated and we cannot handle the diagnostic. I am
also unsure what proper `fix` handling would be and making the user
aware we don't understand the codes is probably the best bet.
- The rule is added to the Preview rule group as it's a new addition

## Questions
- Should we remove the warnings, now that we have a rule?
- Is the current fix behavior appropriate for all cases, particularly
the handling of whitespace and line deletions?
- I'm new to the codebase; let me know if there are rule utilities which
could have used but didn't.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-04-04 08:05:59 +00:00
Micha Reiser a4ba10ff0a
[red-knot] Add basic on-hover to playground and LSP (#17057) 
## Summary

Implement a very basic hover in the playground and LSP.

It's basic, because it only shows the type on-hover. Most other LSPs
also show:

* The signature of the symbol beneath the cursor. E.g. `class
Test(a:int, b:int)` (we want something like
54f7da25f9/packages/pyright-internal/src/analyzer/typeEvaluator.ts (L21929-L22129))
* The symbols' documentation
* Do more fancy markdown rendering

I decided to defer these features for now because it requires new
semantic APIs (similar to *goto definition*), and investing in fancy
rendering only makes sense once we have the relevant data.

Closes [#16826](https://github.com/astral-sh/ruff/issues/16826)

## Test Plan



https://github.com/user-attachments/assets/044aeee4-58ad-4d4e-9e26-ac2a712026be


https://github.com/user-attachments/assets/4a1f4004-2982-4cf2-9dfd-cb8b84ff2ecb
 2025-04-04 08:13:43 +02:00
Carl Meyer bf0306887a
[red-knot] don't remove negations when simplifying constrained typevars (#17189) 
For two non-disjoint types `P` and `Q`, the simplification of `(P | Q) &
~Q` is not `P`, but `P & ~Q`. In other words, the non-empty set `P & Q`
is also excluded from the type.

The same applies for a constrained typevar `[T: (P, Q)]`: `T & ~Q`
should simplify to `P & ~Q`, not just `P`.

Implementing this is actually purely a matter of removing code from the
constrained typevar simplification logic; we just need to not bother
removing the negations. If the negations are actually redundant (because
the constraint types are disjoint), normal intersection simplification
will already eliminate them (as shown in the added test.)
 2025-04-03 16:30:57 -07:00
Brent Westbrook 4f924bb975
[minor] Fix extra semicolon for clippy (#17188) 2025-04-03 18:17:00 -04:00
Brent Westbrook c2b2e42ad3
[syntax-errors] Invalid syntax in annotations (#17101) 
Summary
--

This PR detects the use of invalid syntax in annotation scopes,
including
`yield` and `yield from` expressions and named expressions. I combined a
few
different types of CPython errors here, but I think the resulting error
messages
still make sense and are even preferable to what CPython gives. For
example, we
report `yield expression cannot be used in a type annotation` for both
of these:

```pycon
>>> def f[T](x: (yield 1)): ...
  File "<python-input-26>", line 1
    def f[T](x: (yield 1)): ...
                 ^^^^^^^
SyntaxError: yield expression cannot be used within the definition of a generic
>>> def foo() -> (yield x): ...
  File "<python-input-28>", line 1
    def foo() -> (yield x): ...
                  ^^^^^^^
SyntaxError: 'yield' outside function
```

Fixes https://github.com/astral-sh/ruff/issues/11118.

Test Plan
--

New inline tests, along with some updates to existing tests.
 2025-04-03 17:56:55 -04:00
Brent Westbrook 24b1b1d52c
[syntax-errors] Duplicate attributes in match class pattern (#17186) 
Summary
--

Detects duplicate attributes in a `match` class pattern:

```python
match x:
    case Class(x=1, x=2): ...
```

which are more analogous to the similar check for mapping patterns than
to the
multiple assignments rule.

I also realized that both this and the mapping check would only work on
top-level patterns, despite the possibility that they can be nested
inside other
patterns:

```python
match x:
    case [{"x": 1, "x": 2}]: ...  # false negative in the old version
```

and moved these checks into the recursive pattern visitor instead.

I also tidied up some of the names like the `multiple_case_assignment`
function
and the `MultipleCaseAssignmentVisitor`, which are now doing more than
checking
for multiple assignments.

Test Plan
--

New inline tests for both classes and mappings.
 2025-04-03 17:55:37 -04:00
Brent Westbrook 6a07dd227d
[syntax-errors] Fix multiple assignment for class keyword argument (#17184) 
Summary
--

Fixes #17181. The cases being tested with multiple *keys* being equal
are actually a slightly different error, more like the error for
`MatchMapping` than like the other multiple assignment errors:

```pycon
>>> match x:
...     case Class(x=x, x=x): ...
...
  File "<python-input-249>", line 2
    case Class(x=x, x=x): ...
                      ^
SyntaxError: attribute name repeated in class pattern: x
>>> match x:
...     case {"x": 1, "x": 2}: ...
...
  File "<python-input-251>", line 2
    case {"x": 1, "x": 2}: ...
         ^^^^^^^^^^^^^^^^
SyntaxError: mapping pattern checks duplicate key ('x')
>>> match x:
...     case [x, x]: ...
...
  File "<python-input-252>", line 2
    case [x, x]: ...
             ^
SyntaxError: multiple assignments to name 'x' in pattern
```

This PR just stops the false positive reported in the issue, but I will
quickly follow it up with a new rule (or possibly combined with the
mapping rule) catching the repeated attributes separately.

Test Plan
--

New inline `test_ok` and updating the `test_err` cases to have duplicate
values instead of keys.
 2025-04-03 17:32:39 -04:00
Micha Reiser e7684d3493
Enable overindented docs lint (#17182) 
## Summary

It turns out that `a.` isn't a list format supported by rustdoc. I
changed the documentation to use `1.`, `2.` instead.

## Test Plan

`cargo clippy`
 2025-04-03 20:27:11 +01:00
Douglas Creager 64e7e1aa64
[red-knot] Add `Type::TypeVar` variant (#17102) 
This adds a new `Type` variant for holding an instance of a typevar
inside of a generic function or class. We don't handle specializing the
typevars yet, but this should implement most of the typing rules for
inside the generic function/class, where we don't know yet which
specific type the typevar will be specialized to.

This PR does _not_ yet handle the constraint that multiple occurrences
of the typevar must be specialized to the _same_ time. (There is an
existing test case for this in `generics/functions.md` which is still
marked as TODO.)

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-04-03 14:36:29 -04:00
Micha Reiser 8a4158c5f8
Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

I decided to disable the new
[`needless_continue`](https://rust-lang.github.io/rust-clippy/master/index.html#needless_continue)
rule because I often found the explicit `continue` more readable over an
empty block or having to invert the condition of an other branch.


## Test Plan

`cargo test`

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
 2025-04-03 15:59:44 +00:00
David Peter fedd982fd5
[red-knot] Avoid unresolved-reference in unreachable code (#17169) 
## Summary

This PR changes the inferred type for symbols in unreachable sections of
code to `Never` (instead of reporting them as unbound), in order to
silence false positive diagnostics. See the lengthy comment in the code
for further details.

## Test Plan

- Updated Markdown tests.
- Manually verified a couple of ecosystem diagnostic changes.
 2025-04-03 16:52:11 +02:00
Alex Waygood a1eb834a5f
Fix relative import resolution in `site-packages` packages when the `site-packages` search path is a subdirectory of the first-party search path (#17178) 
## Summary

If a package in `site-packages` had this directory structure:

```py
# bar/__init__.py
from .a import A

# bar/a.py
class A: ...
```

then we would fail to resolve the `from .a import A` import _if_ (as is
usually the case!) the `site-packages` search path was located inside a
`.venv` directory that was a subdirectory of the project's first-party
search path. The reason for this is a bug in `file_to_module` in the
module resolver. In this loop, we would identify that
`/project_root/.venv/lib/python3.13/site-packages/foo/__init__.py` can
be turned into a path relative to the first-party search path
(`/project_root`):


6e2b8f9696/crates/red_knot_python_semantic/src/module_resolver/resolver.rs (L101-L110)

but we'd then try to turn the relative path
(.venv/lib/python3.13/site-packages/foo/__init__.py`) into a module
path, realise that it wasn't a valid module path... and therefore
immediately `break` out of the loop before trying any other search paths
(such as the `site-packages` search path).

This bug was originally reported on Discord by @MatthewMckee4.

## Test Plan

I added a unit test for `file_to_module` in `resolver.rs`, and an
integration test that shows we can now resolve the import correctly in
`infer.rs`.
 2025-04-03 15:48:05 +01:00
Brent Westbrook 6e2b8f9696
[syntax-errors] Detect duplicate keys in `match` mapping patterns (#17129) 
Summary
--

Detects duplicate literals in `match` mapping keys.

This PR also adds a `source` method to `SemanticSyntaxContext` to
display the duplicated key in the error message by slicing out its
range.

Test Plan
--

New inline tests.
 2025-04-03 10:22:37 -04:00
Alex Waygood ca0cce3f9c
[red-knot] Fix more [redundant-cast] false positives (#17170) 
Fixes #17164. Simply checking whether one type is gradually equivalent
to another is too simplistic here: `Any` is gradually equivalent to
`Todo`, but we should permit users to cast from `Todo` or `Unknown` to
`Any` without complaining about it. This changes our logic so that we
only complain about redundant casts if:
- the two types are exactly equal (when normalized) OR they are
equivalent (we'll still complain about `Any -> Any` casts, and about
`Any | str | int` -> `str | int | Any` casts, since their normalized
forms are exactly equal, even though the type is not fully static -- and
therefore does not participate in equivalence relations)
- AND the casted type does not contain `Todo`
 2025-04-03 15:00:00 +01:00
David Peter 3f00010a7a
[red-knot] Three-argument type-calls take 'str' as the first argument (#17168) 
## Summary

Similar to #17163, a minor fix in the signature of `type(…)`.

## Test Plan

New MD tests
 2025-04-03 15:45:08 +02:00
Dylan d401a5440e
Control flow: `return` and `raise` (#17121) 
We add support for `return` and `raise` statements in the control flow
graph: we simply add an edge to the terminal block, push the statements
to the current block, and proceed.

This implementation will have to be modified somewhat once we add
support for `try` statements - then we will need to check whether to
_defer_ the jump. But for now this will do!

Also in this PR: We fix the `unreachable` diagnostic range so that it
lumps together consecutive unreachable blocks.
 2025-04-03 08:30:29 -05:00
Brent Westbrook 755ece0c36
Bump 0.11.3 (#17173) 2025-04-03 09:05:40 -04:00
Alex Waygood 62f8d855d2
[red-knot] Improve `Debug` implementation for `semantic_index::SymbolTable` (#17172) 
## Summary

`dbg!`ing a `SymbolTable` is currently very noisy due to the
`symbols_by_name` field, which doesn't tell you very much at all. The
noisiness makes debugging difficult. This PR removes the
`symbols_by_name` field from the `Debug` implementation.

## Test Plan

`dbg!` output before of the `builtins.pyi` global-scope symbol table:

<details>

```
[crates/red_knot_python_semantic/src/symbol.rs:633:5] symbol_table(db, scope) = SymbolTable {
    symbols: [
        Symbol {
            name: Name("_ast"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_sitebuiltins"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_typeshed"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("sys"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("types"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("dict_items"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("dict_keys"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("dict_values"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("AnyStr_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ConvertibleToFloat"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ConvertibleToInt"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("FileDescriptorOrPath"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("OpenBinaryMode"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("OpenBinaryModeReading"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("OpenBinaryModeUpdating"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("OpenBinaryModeWriting"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("OpenTextMode"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ReadableBuffer"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsAdd"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsAiter"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsAnext"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsDivMod"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsFlush"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsIter"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsKeysAndGetItem"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsLenAndGetItem"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsNext"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsRAdd"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsRDivMod"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsRichComparison"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsRichComparisonT"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsWrite"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Awaitable"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Callable"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Iterable"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Iterator"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("MutableSet"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Reversible"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("AbstractSet"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Sized"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BufferedRandom"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BufferedReader"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BufferedWriter"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("FileIO"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TextIOWrapper"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("CellType"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("CodeType"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TracebackType"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("IO"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Any"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BinaryIO"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ClassVar"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Generic"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Mapping"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("MutableMapping"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("MutableSequence"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Protocol"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Sequence"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsAbs"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsBytes"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsComplex"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsFloat"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SupportsIndex"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TypeVar"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("final"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("overload"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("type_check_only"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Concatenate"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Literal"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("LiteralString"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ParamSpec"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Self"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TypeAlias"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TypeGuard"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TypeIs"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TypeVarTuple"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("deprecated"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("GenericAlias"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_T"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("int"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_I"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_T_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_T_contra"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_R_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_KT"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_VT"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_S"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_T1"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_T2"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_T3"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_T4"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_T5"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_SupportsNextT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_SupportsAnextT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_AwaitableT"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_AwaitableT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_P"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_StartT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_StopT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_StepT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("object"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("staticmethod"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("classmethod"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("type"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("super"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_PositiveInteger"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_NegativeInteger"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_LiteralInteger"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("float"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("complex"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_FormatMapMapping"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_TranslateTable"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("str"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("bytes"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("bytearray"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_IntegerFormats"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("memoryview"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("bool"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("slice"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("tuple"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("function"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("list"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("dict"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("set"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("frozenset"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("enumerate"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("range"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("property"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_NotImplementedType"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("NotImplemented"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("abs"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("all"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("any"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ascii"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("bin"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("breakpoint"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("callable"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("chr"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_PathLike"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("aiter"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsSynchronousAnext"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("anext"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("compile"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("copyright"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("credits"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("delattr"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("dir"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("divmod"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("eval"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("exec"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("exit"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("filter"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("format"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("getattr"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("globals"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("hasattr"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("hash"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("help"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("hex"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("id"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("input"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_GetItemIterable"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("iter"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_ClassInfo"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("isinstance"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("issubclass"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("len"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("license"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("locals"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("map"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("max"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("min"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("next"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("oct"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_Opener"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("open"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ord"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsWriteAndFlush"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("print"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_E_contra"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_M_contra"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_SupportsPow2"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsPow3NoneOnly"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsPow3"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsSomeKindOfPow"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("pow"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("quit"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("reversed"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("repr"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsRound1"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsRound2"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("round"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("setattr"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("sorted"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_AddableT1"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_AddableT2"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_SupportsSumWithNoDefaultGiven"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_SupportsSumNoDefaultT"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("sum"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("vars"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("zip"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("__import__"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("__build_class__"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("EllipsisType"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ellipsis"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Ellipsis"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BaseException"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("GeneratorExit"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("KeyboardInterrupt"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SystemExit"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Exception"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("StopIteration"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("OSError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("EnvironmentError"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("IOError"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("WindowsError"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("ArithmeticError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("AssertionError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("AttributeError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BufferError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("EOFError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ImportError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("LookupError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("MemoryError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("NameError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ReferenceError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("RuntimeError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("StopAsyncIteration"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SyntaxError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SystemError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TypeError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ValueError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("FloatingPointError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("OverflowError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ZeroDivisionError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ModuleNotFoundError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("IndexError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("KeyError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("UnboundLocalError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BlockingIOError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ChildProcessError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ConnectionError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BrokenPipeError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ConnectionAbortedError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ConnectionRefusedError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ConnectionResetError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("FileExistsError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("FileNotFoundError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("InterruptedError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("IsADirectoryError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("NotADirectoryError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("PermissionError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ProcessLookupError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TimeoutError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("NotImplementedError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("RecursionError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("IndentationError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("TabError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("UnicodeError"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("UnicodeDecodeError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("UnicodeEncodeError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("UnicodeTranslateError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("Warning"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("UserWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("DeprecationWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("SyntaxWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("RuntimeWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("FutureWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("PendingDeprecationWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ImportWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("UnicodeWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("BytesWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ResourceWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("EncodingWarning"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("_BaseExceptionT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_BaseExceptionT"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_ExceptionT_co"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND,
            ),
        },
        Symbol {
            name: Name("_ExceptionT"),
            flags: SymbolFlags(
                IS_BOUND,
            ),
        },
        Symbol {
            name: Name("BaseExceptionGroup"),
            flags: SymbolFlags(
                IS_USED | IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("ExceptionGroup"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
        Symbol {
            name: Name("PythonFinalizationError"),
            flags: SymbolFlags(
                IS_BOUND | IS_DECLARED,
            ),
        },
    ],
    symbols_by_name: {
        ScopedSymbolId(
            235,
        ): (),
        ScopedSymbolId(
            267,
        ): (),
        ScopedSymbolId(
            86,
        ): (),
        ScopedSymbolId(
            22,
        ): (),
        ScopedSymbolId(
            182,
        ): (),
        ScopedSymbolId(
            213,
        ): (),
        ScopedSymbolId(
            252,
        ): (),
        ScopedSymbolId(
            177,
        ): (),
        ScopedSymbolId(
            173,
        ): (),
        ScopedSymbolId(
            108,
        ): (),
        ScopedSymbolId(
            122,
        ): (),
        ScopedSymbolId(
            118,
        ): (),
        ScopedSymbolId(
            197,
        ): (),
        ScopedSymbolId(
            69,
        ): (),
        ScopedSymbolId(
            134,
        ): (),
        ScopedSymbolId(
            52,
        ): (),
        ScopedSymbolId(
            248,
        ): (),
        ScopedSymbolId(
            168,
        ): (),
        ScopedSymbolId(
            2,
        ): (),
        ScopedSymbolId(
            129,
        ): (),
        ScopedSymbolId(
            5,
        ): (),
        ScopedSymbolId(
            18,
        ): (),
        ScopedSymbolId(
            150,
        ): (),
        ScopedSymbolId(
            9,
        ): (),
        ScopedSymbolId(
            16,
        ): (),
        ScopedSymbolId(
            205,
        ): (),
        ScopedSymbolId(
            246,
        ): (),
        ScopedSymbolId(
            68,
        ): (),
        ScopedSymbolId(
            93,
        ): (),
        ScopedSymbolId(
            189,
        ): (),
        ScopedSymbolId(
            161,
        ): (),
        ScopedSymbolId(
            64,
        ): (),
        ScopedSymbolId(
            124,
        ): (),
        ScopedSymbolId(
            229,
        ): (),
        ScopedSymbolId(
            94,
        ): (),
        ScopedSymbolId(
            202,
        ): (),
        ScopedSymbolId(
            67,
        ): (),
        ScopedSymbolId(
            120,
        ): (),
        ScopedSymbolId(
            219,
        ): (),
        ScopedSymbolId(
            12,
        ): (),
        ScopedSymbolId(
            20,
        ): (),
        ScopedSymbolId(
            79,
        ): (),
        ScopedSymbolId(
            11,
        ): (),
        ScopedSymbolId(
            157,
        ): (),
        ScopedSymbolId(
            216,
        ): (),
        ScopedSymbolId(
            231,
        ): (),
        ScopedSymbolId(
            239,
        ): (),
        ScopedSymbolId(
            140,
        ): (),
        ScopedSymbolId(
            36,
        ): (),
        ScopedSymbolId(
            13,
        ): (),
        ScopedSymbolId(
            184,
        ): (),
        ScopedSymbolId(
            204,
        ): (),
        ScopedSymbolId(
            70,
        ): (),
        ScopedSymbolId(
            259,
        ): (),
        ScopedSymbolId(
            96,
        ): (),
        ScopedSymbolId(
            111,
        ): (),
        ScopedSymbolId(
            72,
        ): (),
        ScopedSymbolId(
            247,
        ): (),
        ScopedSymbolId(
            101,
        ): (),
        ScopedSymbolId(
            242,
        ): (),
        ScopedSymbolId(
            261,
        ): (),
        ScopedSymbolId(
            263,
        ): (),
        ScopedSymbolId(
            214,
        ): (),
        ScopedSymbolId(
            62,
        ): (),
        ScopedSymbolId(
            166,
        ): (),
        ScopedSymbolId(
            244,
        ): (),
        ScopedSymbolId(
            257,
        ): (),
        ScopedSymbolId(
            133,
        ): (),
        ScopedSymbolId(
            112,
        ): (),
        ScopedSymbolId(
            87,
        ): (),
        ScopedSymbolId(
            90,
        ): (),
        ScopedSymbolId(
            117,
        ): (),
        ScopedSymbolId(
            158,
        ): (),
        ScopedSymbolId(
            162,
        ): (),
        ScopedSymbolId(
            230,
        ): (),
        ScopedSymbolId(
            154,
        ): (),
        ScopedSymbolId(
            255,
        ): (),
        ScopedSymbolId(
            35,
        ): (),
        ScopedSymbolId(
            39,
        ): (),
        ScopedSymbolId(
            138,
        ): (),
        ScopedSymbolId(
            190,
        ): (),
        ScopedSymbolId(
            21,
        ): (),
        ScopedSymbolId(
            66,
        ): (),
        ScopedSymbolId(
            181,
        ): (),
        ScopedSymbolId(
            7,
        ): (),
        ScopedSymbolId(
            236,
        ): (),
        ScopedSymbolId(
            251,
        ): (),
        ScopedSymbolId(
            152,
        ): (),
        ScopedSymbolId(
            227,
        ): (),
        ScopedSymbolId(
            78,
        ): (),
        ScopedSymbolId(
            55,
        ): (),
        ScopedSymbolId(
            61,
        ): (),
        ScopedSymbolId(
            253,
        ): (),
        ScopedSymbolId(
            47,
        ): (),
        ScopedSymbolId(
            65,
        ): (),
        ScopedSymbolId(
            153,
        ): (),
        ScopedSymbolId(
            104,
        ): (),
        ScopedSymbolId(
            74,
        ): (),
        ScopedSymbolId(
            107,
        ): (),
        ScopedSymbolId(
            149,
        ): (),
        ScopedSymbolId(
            98,
        ): (),
        ScopedSymbolId(
            155,
        ): (),
        ScopedSymbolId(
            169,
        ): (),
        ScopedSymbolId(
            180,
        ): (),
        ScopedSymbolId(
            237,
        ): (),
        ScopedSymbolId(
            146,
        ): (),
        ScopedSymbolId(
            15,
        ): (),
        ScopedSymbolId(
            243,
        ): (),
        ScopedSymbolId(
            17,
        ): (),
        ScopedSymbolId(
            136,
        ): (),
        ScopedSymbolId(
            80,
        ): (),
        ScopedSymbolId(
            44,
        ): (),
        ScopedSymbolId(
            228,
        ): (),
        ScopedSymbolId(
            60,
        ): (),
        ScopedSymbolId(
            245,
        ): (),
        ScopedSymbolId(
            193,
        ): (),
        ScopedSymbolId(
            264,
        ): (),
        ScopedSymbolId(
            268,
        ): (),
        ScopedSymbolId(
            58,
        ): (),
        ScopedSymbolId(
            258,
        ): (),
        ScopedSymbolId(
            279,
        ): (),
        ScopedSymbolId(
            113,
        ): (),
        ScopedSymbolId(
            135,
        ): (),
        ScopedSymbolId(
            240,
        ): (),
        ScopedSymbolId(
            85,
        ): (),
        ScopedSymbolId(
            186,
        ): (),
        ScopedSymbolId(
            100,
        ): (),
        ScopedSymbolId(
            187,
        ): (),
        ScopedSymbolId(
            106,
        ): (),
        ScopedSymbolId(
            73,
        ): (),
        ScopedSymbolId(
            223,
        ): (),
        ScopedSymbolId(
            49,
        ): (),
        ScopedSymbolId(
            83,
        ): (),
        ScopedSymbolId(
            77,
        ): (),
        ScopedSymbolId(
            43,
        ): (),
        ScopedSymbolId(
            274,
        ): (),
        ScopedSymbolId(
            46,
        ): (),
        ScopedSymbolId(
            151,
        ): (),
        ScopedSymbolId(
            217,
        ): (),
        ScopedSymbolId(
            178,
        ): (),
        ScopedSymbolId(
            278,
        ): (),
        ScopedSymbolId(
            132,
        ): (),
        ScopedSymbolId(
            6,
        ): (),
        ScopedSymbolId(
            174,
        ): (),
        ScopedSymbolId(
            99,
        ): (),
        ScopedSymbolId(
            196,
        ): (),
        ScopedSymbolId(
            109,
        ): (),
        ScopedSymbolId(
            128,
        ): (),
        ScopedSymbolId(
            123,
        ): (),
        ScopedSymbolId(
            102,
        ): (),
        ScopedSymbolId(
            116,
        ): (),
        ScopedSymbolId(
            172,
        ): (),
        ScopedSymbolId(
            32,
        ): (),
        ScopedSymbolId(
            105,
        ): (),
        ScopedSymbolId(
            241,
        ): (),
        ScopedSymbolId(
            95,
        ): (),
        ScopedSymbolId(
            206,
        ): (),
        ScopedSymbolId(
            209,
        ): (),
        ScopedSymbolId(
            198,
        ): (),
        ScopedSymbolId(
            81,
        ): (),
        ScopedSymbolId(
            170,
        ): (),
        ScopedSymbolId(
            171,
        ): (),
        ScopedSymbolId(
            8,
        ): (),
        ScopedSymbolId(
            276,
        ): (),
        ScopedSymbolId(
            1,
        ): (),
        ScopedSymbolId(
            212,
        ): (),
        ScopedSymbolId(
            222,
        ): (),
        ScopedSymbolId(
            33,
        ): (),
        ScopedSymbolId(
            144,
        ): (),
        ScopedSymbolId(
            194,
        ): (),
        ScopedSymbolId(
            125,
        ): (),
        ScopedSymbolId(
            89,
        ): (),
        ScopedSymbolId(
            38,
        ): (),
        ScopedSymbolId(
            51,
        ): (),
        ScopedSymbolId(
            19,
        ): (),
        ScopedSymbolId(
            163,
        ): (),
        ScopedSymbolId(
            0,
        ): (),
        ScopedSymbolId(
            211,
        ): (),
        ScopedSymbolId(
            3,
        ): (),
        ScopedSymbolId(
            226,
        ): (),
        ScopedSymbolId(
            121,
        ): (),
        ScopedSymbolId(
            148,
        ): (),
        ScopedSymbolId(
            232,
        ): (),
        ScopedSymbolId(
            262,
        ): (),
        ScopedSymbolId(
            260,
        ): (),
        ScopedSymbolId(
            91,
        ): (),
        ScopedSymbolId(
            270,
        ): (),
        ScopedSymbolId(
            269,
        ): (),
        ScopedSymbolId(
            225,
        ): (),
        ScopedSymbolId(
            191,
        ): (),
        ScopedSymbolId(
            115,
        ): (),
        ScopedSymbolId(
            28,
        ): (),
        ScopedSymbolId(
            220,
        ): (),
        ScopedSymbolId(
            164,
        ): (),
        ScopedSymbolId(
            250,
        ): (),
        ScopedSymbolId(
            137,
        ): (),
        ScopedSymbolId(
            141,
        ): (),
        ScopedSymbolId(
            24,
        ): (),
        ScopedSymbolId(
            54,
        ): (),
        ScopedSymbolId(
            45,
        ): (),
        ScopedSymbolId(
            188,
        ): (),
        ScopedSymbolId(
            75,
        ): (),
        ScopedSymbolId(
            40,
        ): (),
        ScopedSymbolId(
            234,
        ): (),
        ScopedSymbolId(
            30,
        ): (),
        ScopedSymbolId(
            41,
        ): (),
        ScopedSymbolId(
            127,
        ): (),
        ScopedSymbolId(
            185,
        ): (),
        ScopedSymbolId(
            145,
        ): (),
        ScopedSymbolId(
            23,
        ): (),
        ScopedSymbolId(
            238,
        ): (),
        ScopedSymbolId(
            143,
        ): (),
        ScopedSymbolId(
            167,
        ): (),
        ScopedSymbolId(
            110,
        ): (),
        ScopedSymbolId(
            25,
        ): (),
        ScopedSymbolId(
            31,
        ): (),
        ScopedSymbolId(
            57,
        ): (),
        ScopedSymbolId(
            195,
        ): (),
        ScopedSymbolId(
            221,
        ): (),
        ScopedSymbolId(
            218,
        ): (),
        ScopedSymbolId(
            37,
        ): (),
        ScopedSymbolId(
            71,
        ): (),
        ScopedSymbolId(
            50,
        ): (),
        ScopedSymbolId(
            176,
        ): (),
        ScopedSymbolId(
            179,
        ): (),
        ScopedSymbolId(
            200,
        ): (),
        ScopedSymbolId(
            266,
        ): (),
        ScopedSymbolId(
            277,
        ): (),
        ScopedSymbolId(
            119,
        ): (),
        ScopedSymbolId(
            84,
        ): (),
        ScopedSymbolId(
            114,
        ): (),
        ScopedSymbolId(
            165,
        ): (),
        ScopedSymbolId(
            271,
        ): (),
        ScopedSymbolId(
            280,
        ): (),
        ScopedSymbolId(
            256,
        ): (),
        ScopedSymbolId(
            249,
        ): (),
        ScopedSymbolId(
            88,
        ): (),
        ScopedSymbolId(
            27,
        ): (),
        ScopedSymbolId(
            139,
        ): (),
        ScopedSymbolId(
            265,
        ): (),
        ScopedSymbolId(
            4,
        ): (),
        ScopedSymbolId(
            53,
        ): (),
        ScopedSymbolId(
            29,
        ): (),
        ScopedSymbolId(
            130,
        ): (),
        ScopedSymbolId(
            42,
        ): (),
        ScopedSymbolId(
            76,
        ): (),
        ScopedSymbolId(
            147,
        ): (),
        ScopedSymbolId(
            156,
        ): (),
        ScopedSymbolId(
            208,
        ): (),
        ScopedSymbolId(
            273,
        ): (),
        ScopedSymbolId(
            183,
        ): (),
        ScopedSymbolId(
            224,
        ): (),
        ScopedSymbolId(
            97,
        ): (),
        ScopedSymbolId(
            233,
        ): (),
        ScopedSymbolId(
            82,
        ): (),
        ScopedSymbolId(
            142,
        ): (),
        ScopedSymbolId(
            254,
        ): (),
        ScopedSymbolId(
            131,
        ): (),
        ScopedSymbolId(
            63,
        ): (),
        ScopedSymbolId(
            48,
        ): (),
        ScopedSymbolId(
            215,
        ): (),
        ScopedSymbolId(
            103,
        ): (),
        ScopedSymbolId(
            14,
        ): (),
        ScopedSymbolId(
            92,
        ): (),
        ScopedSymbolId(
            207,
        ): (),
        ScopedSymbolId(
            275,
        ): (),
        ScopedSymbolId(
            160,
        ): (),
        ScopedSymbolId(
            26,
        ): (),
        ScopedSymbolId(
            56,
        ): (),
        ScopedSymbolId(
            34,
        ): (),
        ScopedSymbolId(
            272,
        ): (),
        ScopedSymbolId(
            59,
        ): (),
        ScopedSymbolId(
            126,
        ): (),
        ScopedSymbolId(
            159,
        ): (),
        ScopedSymbolId(
            199,
        ): (),
        ScopedSymbolId(
            175,
        ): (),
        ScopedSymbolId(
            192,
        ): (),
        ScopedSymbolId(
            201,
        ): (),
        ScopedSymbolId(
            203,
        ): (),
        ScopedSymbolId(
            210,
        ): (),
        ScopedSymbolId(
            10,
        ): (),
    },
}
```

</details>


I checked that with this PR, the second field is gone from the debug
output (I'd paste it in, but it goes over the GitHub comment length
maximum).
 2025-04-03 13:37:29 +01:00
David Peter 130339f3d8
[red-knot] Fix `str(…)` calls (#17163) 
## Summary

The existing signature for `str` calls had various problems, one of
which I noticed while looking at some ecosystem projects (`scrapy`,
added as a project to mypy_primer in this PR).

## Test Plan

- New tests for `str(…)` calls.
- Observed reduction of false positives in ecosystem checks
 2025-04-03 13:26:32 +02:00
Eric Mark Martin e50fc049ab
[red-knot] visibility_constraint analysis for match cases (#17077) 
## Summary

Add visibility constraint analysis for pattern predicate kinds
`Singleton`, `Or`, and `Class`.

## Test Plan

update conditional/match.md
 2025-04-03 11:15:33 +00:00
Dhruv Manilawala 177afabe18
[red-knot] Callable types are disjoint from literals (#17160) 
## Summary

A callable type is disjoint from other literal types. For example,
`Type::StringLiteral` must be an instance of exactly `str`, not a
subclass of `str`, and `str` is not callable. The same applies to other
literal types.

This should hopefully fix #17144, I couldn't produce any failures after
running property tests multiple times.

## Test Plan

Add test cases for disjointness check between callable and other literal
types.

Run property tests multiple times.
 2025-04-03 03:38:13 +05:30
Alex Waygood 28c68934a4
[red-knot] Fix inference for `pow` between two literal integers (#17161) 
## Summary

Python `**` works differently to Rust `**`!

## Test Plan

Added an mdtest for various edge cases, and checked in the Python REPL
that we infer the correct type in all the new cases tested.
 2025-04-02 21:25:57 +00:00
Alex Waygood 195bb433db
[red-knot] Add GitHub PR annotations when mdtests fail in CI (#17150) 
## Summary

This PR adds a CI job that causes GitHub to add annotations to a PR diff
when mdtest assertions fail. For example:

<details>
<summary>Screenshot</summary>


![image](https://github.com/user-attachments/assets/bb2a649b-46ab-429d-a576-b36545940eaf)

</details>

## Motivation

Debugging mdtest failures locally is currently a really nice experience:
- Errors are displayed with pretty colours, which makes them much more
readable
- If you run the test from inside an IDE, you can CTRL-click on a path
and jump directly to the line that had the failing assertion
- If you use
[`mdtest.py`](https://github.com/astral-sh/ruff/blob/main/crates/red_knot_python_semantic/mdtest.py),
you don't even need to recompile anything after changing an assertion in
an mdtest, amd the test results instantly live-update with each change
to the MarkDown file

Debugging mdtest failures in CI is much more unpleasant, however.
Sometimes an error message is just

> [static-assert-error] Argument evaluates to `False`

...which doesn't tell you very much unless you navigate to the line in
question that has the failing mdtest assertion. The line in question
might not even be touched by the PR, and even if it is, it can be hard
to find the line if the PR touches many files. Unlike locally, you can't
click on the error and jump straight to the line that contains the
failing assertion. You also don't get colourised output in CI
(https://github.com/astral-sh/ruff/issues/13939).

GitHub PR annotations should make it really easy to debug why mdtests
are failing on PRs, making PR review much easier.

## Test Plan

I opened a PR to my fork
[here](https://github.com/AlexWaygood/ruff/pull/11/files) with some
bogus changes to an mdtest to show what it looks like when there are
failures in CI and this job has been added. Scroll down to
`crates/red_knot_python_semantic/resources/mdtest/type_properties/is_equivalent_to.md`
on the "files changed" tab for that PR to see the annotations.
 2025-04-02 21:51:52 +01:00
Alex Waygood c2bb5d5250
[red-knot] Fix equivalence of differently ordered unions that contain `Callable` types (#17145) 
## Summary

Fixes https://github.com/astral-sh/ruff/issues/17058.

Equivalent callable types were not understood as equivalent when they
appeared nested inside unions and intersections. This PR fixes that by
ensuring that `Callable` elements nested inside unions, intersections
and tuples have their representations normalized before one union type
is compared with another for equivalence, or before one intersection
type is compared with another for equivalence.

The normalizations applied to a `Callable` type are:
- the type of the default value is stripped from all parameters (only
whether the parameter _has_ a default value is relevant to whether one
`Callable` type is equivalent to another)
- The names of the parameters are stripped from positional-only
parameters, variadic parameters and keyword-variadic parameters
- Unions and intersections that are present (top-level or nested) inside
parameter annotations or return annotations are normalized.

Adding a `CallableType::normalized()` method also allows us to simplify
the implementation of `CallableType::is_equivalent_to()`.

### Should these normalizations be done eagerly as part of a
`CallableType` constructor?

I considered this. It's something that we could still consider doing in
the future; this PR doesn't rule it out as a possibility. However, I
didn't pursue it for now, for several reasons:
1. Our current `Display` implementation doesn't handle well the
possibility that a parameter might not have a name or an annotated type.
Callable types with parameters like this would be displayed as follows:
   ```py
   (, ,) -> None: ...
   ```

That's fixable! It could easily become something like `(Unknown,
Unknown) -> None: ...`. But it also illustrates that we probably want to
retain the parameter names when displaying the signature of a `lambda`
function if you're hovering over a reference to the lambda in an IDE.
Currently we don't have a `LambdaType` struct for representing `lambda`
functions; if we wanted to eagerly normalize signatures when creating
`CallableType`s, we'd probably have to add a `LambdaType` struct so that
we would retain the full signature of a `lambda` function, rather than
representing it as an eagerly simplified `CallableType`.
2. In order to ensure that it's impossible to create `CallableType`s
without the parameters being normalized, I'd either have to create an
alternative `SimplifiedSignature` struct (which would duplicate a lot of
code), or move `CallableType` to a new module so that the only way of
constructing a `CallableType` instance would be via a constructor method
that performs the normalizations eagerly on the callable's signature.
Again, this isn't a dealbreaker, and I think it's still an option, but
it would be a lot of churn, and it didn't seem necessary for now. Doing
it this way, at least to start with, felt like it would create a diff
that's easier to review and felt like it would create fewer merge
conflicts for others.

## Test Plan

- Added a regression mdtest for
https://github.com/astral-sh/ruff/issues/17058
- Ran `QUICKCHECK_TESTS=1000000 cargo test --release -p
red_knot_python_semantic -- --ignored types::property_tests::stable`
 2025-04-02 17:43:34 +00:00
David Peter cb7dae1e96
[red-knot] Add initial set of tests for unreachable code (#17159) 
## Summary

Add an initial set of tests that will eventually document our behavior
around unreachable code. In the last section of this suite, I argue why
we should never type check unreachable sections and never emit any
diagnostics in these sections.
 2025-04-02 19:39:44 +02:00
Wei Lee 8833484b10
[`airflow`] Move `AIR302` to `AIR301` and `AIR303` to `AIR302` (#17151) 
## Summary

Following up the discussion in
https://github.com/astral-sh/ruff/issues/14626#issuecomment-2766548545,
we're to reorganize airflow rules. Before this discussion happens, we
combine required changes and suggested changes in to one single error
code.

This PR first rename the original error code to the new error code as we
discussed. We will gradually extract suggested changes out of AIR301 and
AIR302 to AIR311 and AIR312 in the following PRs

## Test Plan

Except for file, error code rename, the test case should work as it used
to be.
 2025-04-02 23:01:31 +05:30
Andrew Gallant adeba3dca7 ruff_db: simplify lifetimes on `DiagnosticDisplay` 
I initially split the lifetime out into three distinct lifetimes on
near-instinct because I moved the struct into the public API. But
because they are all shared borrows, and because there are no other APIs
on `DisplayDiagnostic` to access individual fields (and probably never
will be), it's probably fine to just specify one lifetime. Because of
subtyping, the one lifetime will be the shorter of the three.

There's also the point that `ruff_db` isn't _really_ a public API, since
it isn't a library that others depend on. So my instinct is probably a
bit off there.
 2025-04-02 12:47:02 -04:00
David Peter af988bf866
[red-knot] Detect division-by-zero in unions and intersections (#17157) 
## Summary

With this PR, we emit a diagnostic for this case where
previously didn't:
```py
from typing import Literal

def f(m: int, n: Literal[-1, 0, 1]):
    # error: [division-by-zero] "Cannot divide object of type `int` by zero"
    return m / n
```

## Test Plan

New Markdown test
 2025-04-02 18:21:27 +02:00
Wei Lee f989c2c3af
[`airflow`] Add autofix infrastructure to `AIR302` name checks (#16965) 
## Summary

Add autofix infrastructure to `AIR302` name checks and use this logic to 
fix`"airflow", "api_connexion", "security", "requires_access_dataset"`, `"airflow", "Dataset"` and `"airflow",
"datasets", "Dataset"`

## Test Plan

The existing test fixture reflects the update
 2025-04-02 15:27:51 +00:00
trag1c c2512b4c50
[`flake8-bandit`] Mark `str` and `list[str]` literals as trusted input (`S603`) (#17136) 
## Summary

Closes #17112. Allows passing in string and list-of-strings literals
into `subprocess.run` (and related) calls without marking them as
untrusted input:
```py
import subprocess

subprocess.run("true")

# "instant" named expressions are also allowed
subprocess.run(c := "ls")
```

## Test Plan

Added test cases covering new behavior, passed with `cargo nextest run`.
 2025-04-02 11:22:37 -04:00
Wei Lee 6bc2b04c49
[`airflow`] Add autofix for `AIR302` attribute checks (#16977) 
## Summary

Add autofix logic to AIR302 check_method

## Test Plan

test fixtures have been updated accordingly
 2025-04-02 15:18:24 +00:00
Wei Lee fc2a0950eb
[`airflow`] Extend `AIR302` with additional symbols (#17085) 
## Summary

* ``airflow.auth.managers.base_auth_manager.is_authorized_dataset`` has
been moved to
``airflow.api_fastapi.auth.managers.base_auth_manager.is_authorized_asset``
in Airflow 3.0
* ``airflow.auth.managers.models.resource_details.DatasetDetails`` has
been moved to
``airflow.api_fastapi.auth.managers.models.resource_details.AssetDetails``
in Airflow 3.0
* Dag arguments `default_view` and `orientation` has been removed in
Airflow 3.0
* `airflow.models.baseoperatorlink.BaseOperatorLink` has been moved to
`airflow.sdk.definitions.baseoperatorlink.BaseOperatorLink` in Airflow
3.0
* ``airflow.notifications.basenotifier.BaseNotifier`` has been moved to
``airflow.sdk.BaseNotifier`` in Airflow 3.0
* ``airflow.utils.log.secrets_masker`` has been moved to
``airflow.sdk.execution_time.secrets_masker`` in Airflow 3.0
* ``airflow...DAG.allow_future_exec_dates`` has been removed in Airflow
3.0
* `airflow.utils.db.create_session` has een removed in Airflow 3.0
* `airflow.sensors.base_sensor_operator.BaseSensorOperator` has been
moved to `airflow.sdk.bases.sensor.BaseSensorOperator` removed Airflow
3.0
* `airflow.utils.file.TemporaryDirectory` has been removed in Airflow
3.0 and can be replaced by `tempfile.TemporaryDirectory`
* `airflow.utils.file.mkdirs` has been removed in Airflow 3.0 and can be
replaced by `pathlib.Path({path}).mkdir`

## Test Plan

Test fixture has been added for these changes
 2025-04-02 20:38:52 +05:30
Wei Lee 33bd08f49b
[`airflow`] Move `AIR301` to `AIR002` (#16978) 
## Summary

Unlike other AIR3XX rules, this best practice can be applied to Airflow
1 and Airflow 2 as well. Thus, we think it might make sense for use to
move it to AIR002 so that the first number of the error align to Airflow
version as possible to reduce confusion

## Test Plan

the test fixture has been updated
 2025-04-02 20:37:35 +05:30
Wei Lee 5d57788328
[`airflow`] Add autofix for `AIR302` method checks (#16976) 
## Summary

Add autofix logic to `AIR302` method checks

## Test Plan

Test fixtures have been updated accordingly
 2025-04-02 20:35:49 +05:30
Andrew Gallant 718b0cadf4 ruff_db: switch diagnostic rendering over to `std::fmt::Display` 
It was already using this approach internally, so this is "just" a
matter of rejiggering the public API of `Diagnostic`.

We were previously writing directly to a `std::io::Write` since it was
thought that this worked better with the linear typing fakery. Namely,
it increased confidence that the diagnostic rendering was actually
written somewhere useful, instead of just being converted to a string
that could potentially get lost.

For reasons discussed in #17130, the linear type fakery was removed.
And so there is less of a reason to require a `std::io::Write`
implementation for diagnostic rendering. Indeed, this would sometimes
result in `unwrap()` calls when one wants to convert to a `String`.
 2025-04-02 11:01:16 -04:00
Micha Reiser 24498e383d
[red-knot] Add 'Goto type definition' to the playground (#17055) 
## Summary

This PR adds Goto type definition to the playground, using the same
infrastructure as the LSP.


The main *challenge* with implementing this feature was that the editor
can now participate in which tab is open.

## Known limitations

The same as for the LSP. Most notably, navigating to types defined in
typeshed isn't supported.

## Test Plan


https://github.com/user-attachments/assets/22dad7c8-7ac7-463f-b066-5d5b2c45d1fe
 2025-04-02 16:35:31 +02:00
Andrew Gallant 28c7e724e3 red_knot_ide: update snapshots 
This just adds an extra blank line. I think these tests were written
against the new renderer before it was used by Red Knot's `main`
function. Once I did that, I saw that it was missing a blank line, and
so I added it to match the status quo. But that means these snapshots
have become stale. So this commit updates them.
 2025-04-02 10:10:01 -04:00
Andrew Gallant 185bcfef1a red_knot_python_semantic: remove comment about `TypeCheckDiagnostic` 
I put this in its own commit in case all of the information removed here
was controversial. But it *looks* stale to me. At the very least,
`TypeCheckDiagnostic` no longer exists, so that would need to be fixed.
And it doesn't really make sense to me (at this point) to make
`Diagnostic` a Salsa struct, particularly since we are keen on using it
in Ruff (at some point).
 2025-04-02 10:10:01 -04:00
Andrew Gallant c30e80a3f4 ruff_db: delete most of the old diagnostic code 
We do keep around `OldSecondaryDiagnosticMessage`, since that's part of
the Red Knot `InferContext` API. But it's a rather simple type, and
we'll be able to delete it entirely once `InferContext` exposes the new
`Diagnostic` type directly.

Since we aren't consuming `OldSecondaryDiagnosticMessage` any more, we
can now accept a slice instead of a vec. (Thanks Clippy.)
 2025-04-02 10:10:01 -04:00
Andrew Gallant 4e169e5f6c red_knot: use `Diagnostic` inside of red knot 
This replaces things like `TypeCheckDiagnostic` with the new Diagnostic`
type.

This is a "surgical" replacement where we retain the existing API of
of diagnostic reporting such that _most_ of Red Knot doesn't need to be
changed to support this update. But it will enable us to start using the
new diagnostic renderer and to delete the old renderer. It also paves
the path for exposing the new `Diagnostic` data model to the broader Red
Knot codebase.
 2025-04-02 10:10:01 -04:00
Andrew Gallant 883b8e3870 ruff_db: port concise diagnostic rendering to new renderer 
Previously, this was only available in the old renderer.
To avoid regressions, we just copy it to the new renderer.
We don't bother with DRY because the old renderer will be
deleted very soon.
 2025-04-02 10:10:01 -04:00
Andrew Gallant 2ca2f73ba8 ruff_db: tweak line terminators emitted by diagnostic rendering 
This change just brings diagnostic rendering into parity
with the status quo.
 2025-04-02 10:10:01 -04:00
Andrew Gallant 90f0766210 ruff_db: make `Diagnostic::print` use a non-mutable borrow 
Now that we don't need to update the `printed` flag, this can just be an
immutable borrow.

(Arguably this should have been an immutable borrow even initially, but
I didn't want to introduce interior mutability without a more compelling
justification.)
 2025-04-02 10:10:01 -04:00
Andrew Gallant a9527edbbe ruff_db: switch `Diagnostic` to use `Arc`, drop linear type fakery 
The switch to `Arc` was done because Salsa sometimes requires cloning a
`Diagnostic` (or something that contains a `Diagnostic`). And so it
probably makes sense to make this cheap.

Since `Diagnostic` exposes a mutable API, we adopt "clone on write"
semantics. Although, it's more like, "clone on write when the `Arc` has
more than one reference." In the common case of creating a `Diagnostic`
and then immediately mutating it, no additional copies should be made
over the status quo.

We also drop the linear type fakery. Its interaction with Salsa is
somewhat awkward, and it has been suggested that there will be points
where diagnostics will be dropped unceremoniously without an opportunity
to tag them as having been ignored. Moreover, this machinery was added
out of "good sense" and isn't actually motivated by real world problems
with accidentally ignoring diagnostics. So that makes it easier, I
think, to just kick this out entirely instead of trying to find a way to
make it work.
 2025-04-02 10:10:01 -04:00
Andrew Gallant 57be814acb ruff_db: add method to create sub-diagnostics from old secondary messages 
This is temporary to scaffold the refactor.

The main idea is that we want to take the `InferContext` API,
*as it is*, and migrate that to the new diagnostic data model
*internally*. Then we can rip out the old stuff and iterate
on the API.
 2025-04-02 10:10:01 -04:00
Andrew Gallant a8a18e7171 red_knot_python_semantic: remove `WithDiagnostic` trait 
I did this mostly because it wasn't buying us much, and I'm
trying to simplify the public API of the types I'd like to
refactor in order to make the refactor simpler.

If we really want something like this, we can re-add it
later.
 2025-04-02 10:10:01 -04:00
Andrew Gallant a953373892 red_knot_python_semantic: remove `Deref` impl on `TypeCheckDiagnostics` 
I removed this to see how much code was depending internally on the
`&[Arc<TypeCheckDiagnostic>]` representation. Thankfully, it was just
one place. So I just removed the `Deref` impl in favor of adding an
explicit `iter` method.

In general, I think using `Deref` for things like this is _somewhat_ of
an abuse. The tip-off is if there are `&self` or `&mut self` methods on
the type, then it's probably not a good candidate for `Deref`.
 2025-04-02 10:10:01 -04:00
Brent Westbrook d382065f8a
[syntax-errors] Reimplement PLE0118 (#17135) 
Summary
--

This PR reimplements
[load-before-global-declaration
(PLE0118)](https://docs.astral.sh/ruff/rules/load-before-global-declaration/)
as a semantic syntax error.

I added a `global` method to the `SemanticSyntaxContext` trait to make
this very easy, at least in ruff. Does red-knot have something similar?

If this approach will also work in red-knot, I think some of the other
PLE rules are also compile-time errors in CPython, PLE0117 in
particular. 0115 and 0116 also mention `SyntaxError`s in their docs, but
I haven't confirmed them in the REPL yet.

Test Plan
--

Existing linter tests for PLE0118. I think this actually can't be tested
very easily in an inline test because the `TestContext` doesn't have a
real way to track globals.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-04-02 13:03:44 +00:00
Brent Westbrook d45593288f
[syntax-errors] Starred expressions in return, yield, and for (#17134) 
Summary
--

Fixes https://github.com/astral-sh/ruff/issues/16520 by flagging single,
starred expressions in `return`, `yield`, and
`for` statements.

I thought `yield from` would also be included here, but that error is
emitted by
the CPython parser:

```pycon
>>> ast.parse("def f(): yield from *x")
Traceback (most recent call last):
  File "<python-input-214>", line 1, in <module>
    ast.parse("def f(): yield from *x")
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/ast.py", line 54, in parse
    return compile(source, filename, mode, flags,
                   _feature_version=feature_version, optimize=optimize)
  File "<unknown>", line 1
    def f(): yield from *x
                        ^
SyntaxError: invalid syntax
```

And we also already catch it in our parser.

Test Plan
--

New inline tests and updates to existing tests.
 2025-04-02 08:38:25 -04:00
Micha Reiser 2ae39edccf
[red-knot] Goto type definition (#16901) 
## Summary

Implement basic *Goto type definition* support for Red Knot's LSP.

This PR also builds the foundation for other LSP operations. E.g., Goto
definition, hover, etc., should be able to reuse some, if not most,
logic introduced in this PR.

The basic steps of resolving the type definitions are:

1. Find the closest token for the cursor offset. This is a bit more
subtle than I first anticipated because the cursor could be positioned
right between the callee and the `(` in `call(test)`, in which case we
want to resolve the type for `call`.
2. Find the node with the minimal range that fully encloses the token
found in 1. I somewhat suspect that 1 and 2 could be done at the same
time but it complicated things because we also need to compute the spine
(ancestor chain) for the node and there's no guarantee that the found
nodes have the same ancestors
3. Reduce the node found in 2. to a node that is a valid goto target.
This may require traversing upwards to e.g. find the closest expression.
4. Resolve the type for the goto target
5. Resolve the location for the type, return it to the LSP

## Design decisions

The current implementation navigates to the inferred type. I think this
is what we want because it means that it correctly accounts for
narrowing (in which case we want to go to the narrowed type because
that's the value's type at the given position). However, it does have
the downside that Goto type definition doesn't work whenever we infer `T
& Unknown` because intersection types aren't supported. I'm not sure
what to do about this specific case, other than maybe ignoring `Unkown`
in Goto type definition if the type is an intersection?

## Known limitations

* Types defined in the vendored typeshed aren't supported because the
client can't open files from the red knot binary (we can either
implement our own file protocol and handler OR extract the typeshed
files and point there). See
https://github.com/astral-sh/ruff/issues/17041
* Red Knot only exposes an API to get types for expressions and
definitions. However, there are many other nodes with identifiers that
can have a type (e.g. go to type of a globals statement, match patterns,
...). We can add support for those in separate PRs (after we figure out
how to query the types from the semantic model). See
https://github.com/astral-sh/ruff/issues/17113
* We should have a higher-level API for the LSP that doesn't directly
call semantic queries. I intentionally decided not to design that API
just yet.


## Test plan


https://github.com/user-attachments/assets/fa077297-a42d-4ec8-b71f-90c0802b4edb

Goto type definition on a union

<img width="1215" alt="Screenshot 2025-04-01 at 13 02 55"
src="https://github.com/user-attachments/assets/689cabcc-4a86-4a18-b14a-c56f56868085"
/>



Note: I recorded this using a custom typeshed path so that navigating to
builtins works.
 2025-04-02 12:12:48 +00:00
cake-monotone 7e97910704
[red-knot] Fix `_NotImplementedType` check for Python >=3.10 (#17143) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

from https://github.com/astral-sh/ruff/pull/17034#discussion_r2024222525

This is a simple PR to fix the invalid behavior of `NotImplemented` on
Python >=3.10.


## Test Plan

I think it would be better if we could run mdtest across multiple Python
versions in GitHub Actions.

<!-- How was it tested? -->

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
 2025-04-02 10:02:59 +00:00
David Peter ae2cf91a36
[red-knot] Decorators and properties (#17017) 
## Summary

Add support for decorators on function as well as support
for properties by adding special handling for `@property` and `@<name of
property>.setter`/`.getter` decorators.

closes https://github.com/astral-sh/ruff/issues/16987

## Ecosystem results

- ✔️ A lot of false positives are fixed by our new
understanding of properties
- 🔴 A bunch of new false positives (typically
`possibly-unbound-attribute` or `invalid-argument-type`) occur because
we currently do not perform type narrowing on attributes. And with the
new understanding of properties, this becomes even more relevant. In
many cases, the narrowing occurs through an assertion, so this is also
something that we need to implement to get rid of these false positives.
- 🔴 A few new false positives occur because we do not
understand generics, and therefore some calls to custom setters fail.
- 🔴 Similarly, some false positives occur because we do not
understand protocols yet.
- ✔️ Seems like a true positive to me. [The
setter](e624d8edfa/src/packaging/specifiers.py (L752-L754))
only accepts `bools`, but `None` is assigned in [this
line](e624d8edfa/tests/test_specifiers.py (L688)).
  ```
+ error[lint:invalid-assignment]
/tmp/mypy_primer/projects/packaging/tests/test_specifiers.py:688:9:
Invalid assignment to data descriptor attribute `prereleases` on type
`SpecifierSet` with custom `__set__` method
  ```
- ✔️ This is arguable also a true positive. The setter
[here](0c6c75644f/rich/table.py (L359-L363))
returns `Table`, but typeshed wants [setters to return
`None`](bf8d2a9912/stdlib/builtins.pyi (L1298)).
  ```
+ error[lint:invalid-argument-type]
/tmp/mypy_primer/projects/rich/rich/table.py:359:5: Object of type
`Literal[padding]` cannot be assigned to parameter 2 (`fset`) of bound
method `setter`; expected type `(Any, Any, /) -> None`
  ```  

## Follow ups

- Fix the `@no_type_check` regression
- Implement class decorators

## Test Plan

New Markdown test suites for decorators and properties.
 2025-04-02 09:27:46 +02:00
Mohammad Amin Ghasemi e1b5b0de71
[flake8-import-conventions] Add import `numpy.typing as npt` to default `flake8-import-conventions.aliases` (#17133) 
## Summary
Adds import `numpy.typing as npt` to `default in
flake8-import-conventions.aliases`
Resolves #17028

## Test Plan
Manually ran local ruff on the altered fixture and also ran `cargo test`
 2025-04-02 09:25:46 +02:00
Dhruv Manilawala f63024843c
[red-knot] Move tuple containing `Never` tests (#17137) 
Refer
https://github.com/astral-sh/ruff/pull/17094#discussion_r2023682840
 2025-04-02 02:46:18 +00:00
Matthew Mckee eb3e176309
[red-knot] Add callable subtyping for callable instances and bound methods (#17105) 
## Summary

Trying to improve #17005
Partially fixes #16953

## Test Plan

Update is_subtype_of.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-04-01 23:40:44 +00:00
Dhruv Manilawala d38f6fcc55
[red-knot] Add property tests for callable types (#17006) 
## Summary

Part of #15382, this PR adds property tests for callable types.

Specifically, this PR updates the property tests to generate an
arbitrary signature for a general callable type which includes:
* Arbitrary combination of parameter kinds in the correct order
* Arbitrary number of parameters
* Arbitrary optional types for annotation and return type
* Arbitrary parameter names (no duplicate names), optional for
positional-only parameters

## Test Plan

```
QUICKCHECK_TESTS=100000 cargo test -p red_knot_python_semantic -- --ignored types::property_tests::stable
```

Also, the commands in CI:


d72b4100a3/.github/workflows/daily_property_tests.yaml (L47-L52)
 2025-04-02 01:07:42 +05:30
Dhruv Manilawala 6be0a5057d
[red-knot] Disjointness for callable types (#17094) 
## Summary

Part of #15382, this PR adds support for disjointness between two
callable types. They are never disjoint because there exists a callable
type that's a subtype of all other callable types:
```py
(*args: object, **kwargs: object) -> Never
```

The `Never` is a subtype of every fully static type thus a callable type
that has the return type of `Never` means that it is a subtype of every
return type.

## Test Plan

Add test cases related to mixed parameter kinds, gradual form (`...`)
and `Never` type.
 2025-04-01 19:00:27 +00:00
Alex Waygood d6dcc377f7
[red-knot] Flatten `Type::Callable` into four `Type` variants (#17126) 
## Summary

Currently our `Type::Callable` wraps a four-variant `CallableType` enum.
But as time has gone on, I think we've found that the four variants in
`CallableType` are really more different to each other than they are
similar to each other:
- `GeneralCallableType` is a structural type describing all callable
types with a certain signature, but the other three types are "literal
types", more similar to the `FunctionLiteral` variant
- `GeneralCallableType` is not a singleton or a single-valued type, but
the other three are all single-valued types
(`WrapperDescriptorDunderGet` is even a singleton type)
- `GeneralCallableType` has (or should have) ambiguous truthiness, but
all possible inhabitants of the other three types are always truthy.
- As a structural type, `GeneralCallableType` can contain inner unions
and intersections that must be sorted in some contexts in our internal
model, but this is not true for the other three variants.

This PR flattens `Type::Callable` into four distinct `Type::` variants.
In the process, it fixes a number of latent bugs that were concealed by
the current architecture but are laid bare by the refactor. Unit tests
for these bugs are included in the PR.
 2025-04-01 19:30:06 +01:00
Alex Waygood a43b683d08
mdtest.py: do a full mdtest run immediately when the script is executed (#17128) 
## Summary

Currently if I run `uv run crates/red_knot_python_semantic/mdtest.py`
from the Ruff repo root, I get this output:

```
~/dev/ruff (main)⚡ % uv run crates/red_knot_python_semantic/mdtest.py
Ready to watch for changes...
```

...And I then have to make some spurious whitespace changes or something
to a test file in order to get the script to actually run mdtest. This
PR changes mdtest.py so that it does an initial run of all mdtests when
you invoke the script, and _then_ starts watching for changes in test
files/Rust code.
 2025-04-01 19:27:55 +01:00
Dhruv Manilawala d29d4956de
[red-knot] Fix callable subtyping for standard parameters (#17125) 
## Summary

This PR fixes a bug in callable subtyping to consider both the
positional and keyword form of the standard parameter in the supertype
when matching against variadic, keyword-only and keyword-variadic
parameter in the subtype.

This is done by collecting the unmatched standard parameters and then
checking them against the keyword-only / keyword-variadic parameters
after the positional loop.

## Test Plan

Add test cases.
 2025-04-01 23:37:35 +05:30
Alex Waygood c74ba00219
[red-knot] Fix more `redundant-cast` false positives (#17119) 
## Summary

There are quite a few places we infer `Todo` types currently, and some
of them are nested somewhat deeply in type expressions. These can cause
spurious issues for the new `redundant-cast` diagnostics. We fixed all
the false positives we saw in the mypy_primer report before merging
https://github.com/astral-sh/ruff/pull/17100, but I think there are
still lots of places where we'd emit false positives due to this check
-- we currently don't run on that many projects at all in our
mypy_primer check:


d0c8eaa092/.github/workflows/mypy_primer.yaml (L71)

This PR fixes some more false positives from this diagnostic by making
the `Type::contains_todo()` method more expansive.

## Test Plan

I added a regression test which causes us to emit a spurious diagnostic
on `main`, but does not with this PR.
 2025-04-01 19:03:42 +01:00
github-actions[bot] a15404a5c1
Sync vendored typeshed stubs (#17106) 
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
 2025-04-01 17:44:27 +01:00
Carl Meyer 3f63c08728
[red-knot] support Any as a class in typeshed (#17107) 
## Summary

In https://github.com/python/typeshed/pull/13520 the typeshed definition
of `typing.Any` was changed from `Any = object()` to `class Any: ...`.
Our automated typeshed updater pulled down this change in
https://github.com/astral-sh/ruff/pull/17106, with the consequence that
we no longer understand `Any`, which is... not good.

This PR gives us the ability to understand `Any` defined as a class
instead of `object()`. It doesn't remove our ability to understand the
old form. Perhaps at some point we'll want to remove it, but for now we
may as well support both old and new typeshed?

This also directly patches typeshed to use the new form of `Any`; this
is purely to work around our tests that no known class is inferred as
`Unknown`, which otherwise fail with the old typeshed and the changes in
this PR. (All other tests pass.) This patch to typeshed will shortly be
subsumed by https://github.com/astral-sh/ruff/pull/17106 anyway.

## Test Plan

Without the typeshed change in this PR, all tests pass except for the
two `known_class_doesnt_fallback_to_unknown_unexpectedly_*` tests (so we
still support the old form of defining `Any`). With the typeshed change
in this PR, all tests pass, so we now support the new form in a way that
is indistinguishable to our test suite from the old form. And
indistinguishable to the ecosystem check: after rebasing
https://github.com/astral-sh/ruff/pull/17106 on this PR, there's zero
ecosystem impact.
 2025-04-01 16:38:23 +00:00
Micha Reiser 5a876ed25e
Visit `Identifier` node as part of the `SourceOrderVisitor` (#17110) 
## Summary

I don't remember exactly when we made `Identifier` a node but it is now
considered a node (it implements `AnyNodeRef`, it has a range). However,
we never updated
the `SourceOrderVisitor` to visit identifiers because we never had a use
case for it and visiting new nodes can change how the formatter
associates comments (breaking change!).
This PR updates the `SourceOrderVisitor` to visit identifiers and
changes the formatter comment visitor to skip identifiers (updating the
visitor might be desired because it could help simplifying some comment
placement logic but this is out of scope for this PR).

## Test Plan

Tests, updated snapshot tests
 2025-04-01 16:58:09 +02:00
Alex Waygood 49c25993eb
[red-knot] Don't infer Todo for quite so many tuple type expressions (#17116) 
## Summary

I noticed we were inferring `Todo` as the declared type for annotations
such as `x: tuple[list[int], list[int]]`. This PR reworks our annotation
parsing so that we instead infer `tuple[Todo, Todo]` for this
annotation, which is quite a bit more precise.

## Test Plan

Existing mdtest updated.
 2025-04-01 15:44:02 +01:00
Max Mynter d0c8eaa092
Error instead of `panic!` when running Ruff from a deleted directory (#16903) (#17054) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Closes #16903

## Summary
Check if the current working directory exist. If not, provide an error
instead of panicking.

Fixed a stale comment in `resolve_default_files`.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
I added a test to the `resolve_files.rs`. 

Manual testing follows steps of #16903 :
- Terminal 1
```bash
mkdir tmp
cd tmp
```
- Terminal 2
```bash
rm -rf tmp
```
- Terminal 1
```bash
ruff check
```

## Open Issues / Questions to Reviewer
All tests pass when executed with `cargo nextest run`.

However, with `cargo test` the parallelization makes the other tests
fail as we change the `pwd`.

Serial execution with `cargo test` seems to require [another dependency
or some
workarounds](https://stackoverflow.com/questions/51694017/how-can-i-avoid-running-some-tests-in-parallel).

Do you think an additional dependency or test complexity is worth
testing this small edge case, do you have another implementation idea,
or should i rather remove the test?
  
---
P.S.: I'm currently participating in a batch at the [Recurse
Center](https://www.recurse.com/) and would love to contribute more for
the next six weeks to improve my Rust. Let me know if you're open to
mentoring/reviewing and/or if you have specific areas where help would
be most valued.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-04-01 14:17:07 +02:00
Dylan aa93005d8d
Control flow graph: setup (#17064) 
This PR contains the scaffolding for a new control flow graph
implementation, along with its application to the `unreachable` rule. At
the moment, the implementation is a maximal over-approximation: no
control flow is modeled and all statements are counted as reachable.
With each additional statement type we support, this approximation will
improve.

So this PR just contains:
- A `ControlFlowGraph` struct and builder
- Support for printing the flow graph as a Mermaid graph
- Snapshot tests for the actual graphs
- (a very bad!) reimplementation of `unreachable` using the new structs
- Snapshot tests for `unreachable`

# Instructions for Viewing Mermaid snapshots
Unfortunately I don't know how to convince GitHub to render the Mermaid
graphs in the snapshots. However, you can view these locally in VSCode
if you install an extension that supports Mermaid graphs in Markdown,
and then add this to your `settings.json`:

```json
  "files.associations": {
"*.md.snap": "markdown",
  }
  ```
 2025-04-01 05:53:42 -05:00
Micha Reiser 0073fd4945
[red-knot] Playground improvements (#17109) 
## Summary

A few smaller editor improvements that felt worth pulling out of my
other feature PRs:

* Load the `Editor` lazily: This allows splitting the entire monaco
javascript into a separate async bundle, drastically reducing the size
of the `index.js`
* Fix the name of `to_range` and `text_range` to the more idiomatic js
names `toRange` and `textRange`
* Use one indexed values for `Position::line` and `Position::column`,
which is the same as monaco (reduces the need for `+1` and `-1`
operations spread all over the place)
* Preserve the editor state when navigating between tabs. This ensures
that selections are preserved even when switching between tabs.
* Stop the default handling of the `Enter` key press event when renaming
a file because it resulted in adding a newline in the editor
 2025-04-01 10:04:51 +02:00
Micha Reiser b57c62e6b3
[red-knot] IDE crate (#17045) 
## Summary

This PR adds a new but so far empty and unused `red_knot_ide` crate. 

This new crate's purpose is to implement IDE-specific functionality,
such as go to definition, hover, completion, etc., which are used by
both the LSP and the playground.

The crate itself doesn't depend on `lsptypes`. The idea is that the
facade crates (e.g., `red_knot_server`) convert external to internal
types.
Not only allows this to share the logic between server and playground,
it also ensures that the core functionality is easier to test because it
can be tested without needing a full LSP.



## Test Plan

`cargo build`
 2025-04-01 09:36:00 +02:00
Trevor Manz 53cfaaebc4
[red-knot] Add redundant-cast error (#17100) 
## Summary

Following up from earlier discussion on Discord, this PR adds logic to
flag casts as redundant when the inferred type of the expression is the
same as the target type. It should follow the semantics from
[mypy](https://github.com/python/mypy/pull/1705).

Example:

```python
def f() -> int:
    return 10

# error: [redundant-cast] "Value is already of type `int`"
cast(int, f())
```
 2025-04-01 00:37:25 +00:00
Matthew Mckee 3ad123bc23
[red-knot] Narrowing on `in tuple[...]` and `in str` (#17059) 
## Summary

Part of #13694

Seems there a bit more to cover regarding `in` and other types, but i
can cover them in different PRs

## Test Plan
Add `in.md` file in narrowing conditionals folder

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-31 23:38:09 +00:00
Micha Reiser a1535fbdbd
[red-knot] Change venv discovery (#17099) 
## Summary

Rewrites the virtual env discovery to:

* Only use of `System` APIs, this ensures that the discovery will also
work when using a memory file system (testing or WASM)
* Don't traverse ancestor directories. We're not convinced that this is
necessary. Let's wait until someone shows us a use case where it is
needed
* Start from the project root and not from the current working
directory. This ensures that Red Knot picks up the right venv even when
using `knot --project ../other-dir`

## Test Plan

Existing tests, @ntBre tested that the `file_watching` tests no longer
pick up his virtual env in a parent directory
 2025-03-31 19:39:05 +02:00
Matthew Mckee 4a6fa5fc27
[red-knot] Add assignability of function literals to callables (#17095) 
## Summary

Part  of #16953

## Test Plan

Update is_assignable_to.md

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
 2025-03-31 15:42:42 +00:00
Aarni Koskela 491a51960e
[`ruff`] Support slices in `RUF005` (#17078) 
## Summary

Teaches `RUF005` to also consider slices for concatenation. Other
indexing (`foo[0] + [7, 8, 9] + bar[1]`) is explicitly not considered.

```diff
 foo = [4, 5, 6]
-bar = [1, 2, 3] + foo
-slicing1 = foo[:1] + [7, 8, 9]
-slicing2 = [7, 8, 9] + bar[1:]
-slicing3 = foo[:1] + [7, 8, 9] + bar[1:]
+bar = [1, 2, 3, *foo]
+slicing1 = [*foo[:1], 7, 8, 9]
+slicing2 = [7, 8, 9, *bar[1:]]
+slicing3 = [*foo[:1], 7, 8, 9, *bar[1:]]
```

## Test Plan

Manually tested (diff above from `ruff check --diff`), snapshot updated.
 2025-03-31 09:09:39 -04:00
David Peter 2d7f118f52
[red-knot] Binary operator inference: generalize code for non-instances (#17081) 
## Summary

Generalize the rich-comparison fallback code for binary operator
inference. This gets rid of one `todo_type!(…)` and implements the last
remaining failing case from
https://github.com/astral-sh/ruff/issues/14200.

closes https://github.com/astral-sh/ruff/issues/14200

## Test Plan

New Markdown tests.
 2025-03-31 13:01:25 +02:00
David Peter 3d1e5676fb
[red-knot] Add `ParamSpecArgs` and `ParamSpecKwargs` as `KnownClass` (#17086) 
## Summary

In preparation for #17017, where we will need them to suppress new false
positives (once we understand the `ParamSpec.args`/`ParamSpec.kwargs`
properties).

## Test Plan

Tested on branch #17017
 2025-03-31 10:52:23 +00:00
Wei Lee 0b1ab8fd5a
[airflow] Combine AIR302 matches (#17080) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

* Combine AIR302 matches
* Found a few errors. Will be fixed in another PR

## Test Plan

<!-- How was it tested? -->

This PR does not change anything. The existing testing fixture should
work as it used to be
 2025-03-31 12:17:18 +02:00
David Peter d9616e61b0
[red-knot] Disallow `todo_type!` without custom message (#17083) 
## Summary

Disallow empty `todo_type!()`s without a custom message. They can lead
to spurious diffs in `mypy_primer` where the only thing that's changed
is the file/line information.
 2025-03-31 10:49:19 +02:00
Wei Lee fa80e10aac
[airflow] fix typos in AIR302 implementation and test cases (#17082) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

* The following paths are wrong 
* `airflow.providers.amazon.auth_manager.avp.entities` should be
`airflow.providers.amazon.aws.auth_manager.avp.entities`
* `["airflow", "datasets", "manager", "dataset_manager"]` should be
fixed as `airflow.assets.manager` but not
`airflow.assets.manager.asset_manager`
* `["airflow", "datasets.manager", "DatasetManager"]` should be `
["airflow", "datasets", "manager", "DatasetManager"]` instead

## Test Plan

<!-- How was it tested? -->

the test fixture is updated accordingly
 2025-03-31 10:42:04 +02:00
Wei Lee 30a5f69913
[airflow] fix missing or wrong test cases (AIR302) (#16968) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Improve AIR302 test cases

## Test Plan

<!-- How was it tested? -->
test fixtures have been updated accordingly
 2025-03-31 10:22:45 +02:00
renovate[bot] a192d96880
Update pre-commit dependencies (#17073) 
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[abravalheri/validate-pyproject](https://redirect.github.com/abravalheri/validate-pyproject)
| repository | patch | `v0.24` -> `v0.24.1` |
|
[astral-sh/ruff-pre-commit](https://redirect.github.com/astral-sh/ruff-pre-commit)
| repository | patch | `v0.11.0` -> `v0.11.2` |
| [crate-ci/typos](https://redirect.github.com/crate-ci/typos) |
repository | minor | `v1.30.2` -> `v1.31.0` |
|
[python-jsonschema/check-jsonschema](https://redirect.github.com/python-jsonschema/check-jsonschema)
| repository | minor | `0.31.3` -> `0.32.1` |
|
[woodruffw/zizmor-pre-commit](https://redirect.github.com/woodruffw/zizmor-pre-commit)
| repository | patch | `v1.5.1` -> `v1.5.2` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

Note: The `pre-commit` manager in Renovate is not supported by the
`pre-commit` maintainers or community. Please do not report any problems
there, instead [create a Discussion in the Renovate
repository](https://redirect.github.com/renovatebot/renovate/discussions/new)
if you have any questions.

---

### Release Notes

<details>
<summary>abravalheri/validate-pyproject
(abravalheri/validate-pyproject)</summary>

###
[`v0.24.1`](https://redirect.github.com/abravalheri/validate-pyproject/releases/tag/v0.24.1)

[Compare
Source](https://redirect.github.com/abravalheri/validate-pyproject/compare/v0.24...v0.24.1)

#### What's Changed

- Fixed multi plugin id was read from the wrong place by
[@&#8203;henryiii](https://redirect.github.com/henryiii) in
[https://github.com/abravalheri/validate-pyproject/pull/240](https://redirect.github.com/abravalheri/validate-pyproject/pull/240)
- Implemented alternative plugin sorting,
[https://github.com/abravalheri/validate-pyproject/pull/243](https://redirect.github.com/abravalheri/validate-pyproject/pull/243)

**Full Changelog**:
https://github.com/abravalheri/validate-pyproject/compare/v0.24...v0.24.1

</details>

<details>
<summary>astral-sh/ruff-pre-commit (astral-sh/ruff-pre-commit)</summary>

###
[`v0.11.2`](https://redirect.github.com/astral-sh/ruff-pre-commit/releases/tag/v0.11.2)

[Compare
Source](https://redirect.github.com/astral-sh/ruff-pre-commit/compare/v0.11.1...v0.11.2)

See: https://github.com/astral-sh/ruff/releases/tag/0.11.2

###
[`v0.11.1`](https://redirect.github.com/astral-sh/ruff-pre-commit/releases/tag/v0.11.1)

[Compare
Source](https://redirect.github.com/astral-sh/ruff-pre-commit/compare/v0.11.0...v0.11.1)

See: https://github.com/astral-sh/ruff/releases/tag/0.11.1

</details>

<details>
<summary>crate-ci/typos (crate-ci/typos)</summary>

###
[`v1.31.0`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.31.0)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.30.3...v1.31.0)

#### \[1.31.0] - 2025-03-28

##### Features

- Updated the dictionary with the [March
2025](https://redirect.github.com/crate-ci/typos/issues/1266) changes

###
[`v1.30.3`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.30.3)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.30.2...v1.30.3)

#### \[1.30.3] - 2025-03-24

##### Features

-   Support detecting `go.work` and `go.work.sum` files

</details>

<details>
<summary>python-jsonschema/check-jsonschema
(python-jsonschema/check-jsonschema)</summary>

###
[`v0.32.1`](https://redirect.github.com/python-jsonschema/check-jsonschema/blob/HEAD/CHANGELOG.rst#0321)

[Compare
Source](https://redirect.github.com/python-jsonschema/check-jsonschema/compare/0.32.0...0.32.1)

-   Fix the `check-meltano` hook to use `types_or`. Thanks
    :user:`edgarrmondragon`! (:pr:`543`)

###
[`v0.32.0`](https://redirect.github.com/python-jsonschema/check-jsonschema/blob/HEAD/CHANGELOG.rst#0320)

[Compare
Source](https://redirect.github.com/python-jsonschema/check-jsonschema/compare/0.31.3...0.32.0)

- Update vendored schemas: circle-ci, compose-spec, dependabot,
github-workflows,
    gitlab-ci, mergify, renovate, taskfile (2025-03-25)
- Add Meltano schema and pre-commit hook. Thanks
:user:`edgarrmondragon`! (:issue:`540`)
- Add Snapcraft schema and pre-commit hook. Thanks :user:`fabolhak`!
(:issue:`535`)

</details>

<details>
<summary>woodruffw/zizmor-pre-commit
(woodruffw/zizmor-pre-commit)</summary>

###
[`v1.5.2`](https://redirect.github.com/woodruffw/zizmor-pre-commit/releases/tag/v1.5.2)

[Compare
Source](https://redirect.github.com/woodruffw/zizmor-pre-commit/compare/v1.5.1...v1.5.2)

See: https://github.com/woodruffw/zizmor/releases/tag/v1.5.2

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4yMDcuMSIsInVwZGF0ZWRJblZlciI6IjM5LjIwNy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-31 07:42:15 +00:00
Mike Perlov 5c1fab0661
Fix member lookup for unions & intersections ignoring policy (#17066) 
## Summary

A quick fix for how union/intersection member search ins performed in
Knot.

## Test Plan

* Added a dunder method call test for Union, which exhibits the error
* Also added an intersection error, but it is not triggering currently
due to `call` logic not being fully implemented for intersections.

---------

Co-authored-by: David Peter <mail@david-peter.de>
 2025-03-31 09:40:47 +02:00
cake-monotone c6efa93cf7
[red-knot] Handle special case returning `NotImplemented` (#17034) 
## Summary

Closes #16661

This PR includes two changes:

- `NotImplementedType` is now a member of `KnownClass`
- We skip `is_assignable_to` checks for `NotImplemented` when checking
return types

### Limitation

```py
def f(cond: bool) -> int:
    return 1 if cond else NotImplemented
```

The implementation covers cases where `NotImplemented` appears inside a
`Union`.
However, for more complex types (ex. `Intersection`) it will not worked.
In my opinion, supporting such complexity is unnecessary at this point.

## Test Plan

Two `mdtest` files were updated:

- `mdtest/function/return_type.md`
- `mdtest/type_properties/is_singleton.md`

To test `KnownClass`, run:
```bash
cargo test -p red_knot_python_semantic -- types::class::
```
 2025-03-30 11:06:12 -07:00
Brent Westbrook ab1011ce70
[syntax-errors] Single starred assignment target (#17024) 
Summary
--

Detects starred assignment targets outside of tuples and lists like `*a
= (1,)`.

This PR only considers assignment statements. I also checked annotated
assigment statements, but these give a separate error that we already
catch, so I think they're okay not to consider:

```pycon
>>> *a: list[int] = []
  File "<python-input-72>", line 1
    *a: list[int] = []
      ^
SyntaxError: invalid syntax
```

Fixes #13759

Test Plan
--

New inline tests, plus a new `SemanticSyntaxError` for an existing
parser test. I also removed a now-invalid case from an otherwise-valid
test fixture.

The new semantic error leads to two errors for the case below:

```python
*foo() = 42
```

but this matches [pyright] too.

[pyright]: https://pyright-play.net/?code=FQMw9mAUCUAEC8sAsAmAUEA
 2025-03-29 12:35:47 -04:00
Brent Westbrook a0819f0c51
[syntax-errors] Store to or delete `__debug__` (#16984) 
Summary
--

Detect setting or deleting `__debug__`. Assigning to `__debug__` was a
`SyntaxError` on the earliest version I tested (3.8). Deleting
`__debug__` was made a `SyntaxError` in [BPO 45000], which said it was
resolved in Python 3.10. However, `del __debug__` was also a runtime
error (`NameError`) when I tested in Python 3.9.6, so I thought it was
worth including 3.9 in this check.

I don't think it was ever a *good* idea to try `del __debug__`, so I
think there's also an argument for not making this version-dependent at
all. That would only simplify the implementation very slightly, though.

[BPO 45000]: https://github.com/python/cpython/issues/89163

Test Plan
--

New inline tests. This also required adding a `PythonVersion` field to
the `TestContext` that could be taken from the inline `ParseOptions` and
making the version field on the options accessible.
 2025-03-29 12:07:20 -04:00
Alex Waygood 93052331b0
[red-knot] Allow `CallableTypeFromFunction` to display the signatures of callable types that are not function literals (#17047) 
I found this helpful for understanding some of the stuff that was going
on in https://github.com/astral-sh/ruff/pull/17005.
 2025-03-28 20:23:04 +00:00
Micha Reiser e07741e553
Add `as_group` methods to `AnyNodeRef` (#17048) 
## Summary

This PR adds `as_<group>` methods to `AnyNodeRef` to e.g. convert an
`AnyNodeRef` to an `ExprRef`.

I need this for go to definition where the fallback is to test if
`AnyNodeRef` is an expression and then call `inferred_type` (listing
this mapping at every call site where we need to convert `AnyNodeRef` to
an `ExprRef` is a bit painful ;))

Split out from https://github.com/astral-sh/ruff/pull/16901

## Test Plan

`cargo test`
 2025-03-28 19:42:45 +00:00
Micha Reiser 050f332771
Rename `visit_preorder` to `visit_source_order` (#17046) 
## Summary

We renamed the `PreorderVisitor` to `SourceOrderVisitor` a long time ago
but it seems that we missed to rename the `visit_preorder` functions to
`visit_source_order`.
This PR renames `visit_preorder` to `visit_source_order`

## Test Plan

`cargo test`
 2025-03-28 19:40:26 +00:00
Micha Reiser 6b02c39321
[red-knot] Incorporate recent ruff server improvements into red knot's LSP (#17044) 2025-03-28 18:39:18 +00:00
Eric Mark Martin 78b0b5a3ab
[red-knot] Factor out shared unpacking logic (#16595) 
## Summary

This PR refactors the common logic for unpacking in assignment, for loops, and with items.

## Test Plan

Make sure existing tests pass.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
 2025-03-28 23:52:51 +05:30
Matthew Mckee 0e48940ea4
[red-knot] Discover local venv folder in cli (#16917) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Fixes #16744 

Code from 

bbf4f830b5/crates/uv-python/src/virtualenv.rs (L124-L144)

## Test Plan

Manual testing

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-28 17:59:49 +00:00
Shunsuke Shibayama aca6254e82
[red-knot] fix eager nested scopes handling (#16916) 
## Summary

From #16861, and the continuation of #16915.

This PR fixes the incorrect behavior of
`TypeInferenceBuilder::infer_name_load` in eager nested scopes.

And this PR closes #16341.

## Test Plan

New test cases are added in `annotations/deferred.md`.
 2025-03-28 11:11:56 -04:00
Eric Mark Martin 64171744dc
[red-knot] support narrowing on or patterns in matches (#17030) 
## Summary

Part of #13694

Narrow in or-patterns by taking the type union of the type constraints
in each disjunct pattern.

## Test Plan

Add new tests to narrow/match.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-28 14:27:09 +00:00
Eric Mark Martin 3acf4e716d
[red-knot] support narrowing on constants in matches (#16974) 
## Summary

Part of #13694

The implementation here was suspiciously straightforward so please lmk
if I missed something

Also some drive-by changes to DRY things up a bit

## Test Plan

Add new tests to narrow/match.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-28 02:36:51 +00:00
Alex Waygood 992a1af4c2
[red-knot] Reduce false positives on `super()` and enum-class attribute accesses (#17004) 
## Summary

This PR adds some branches so that we infer `Todo` types for attribute
access on instances of `super()` and subtypes of `type[Enum]`. It reduces
false positives in the short term until we implement full support for
these features.

## Test Plan

New mdtests added + mypy_primer report
 2025-03-27 17:30:56 -04:00
Micha Reiser 4067a7e50c
[red-knot] Don't check non-python files (#17021) 
## Summary

Fixes https://github.com/astral-sh/ruff/issues/17018

## Test Plan

I renamed a python file to `knot.toml` and verified that there are no
diagnostics. Renaming back the file to `*.py` brings back the
diagnostics
 2025-03-27 19:45:04 +00:00
InSync 6ef522159d
Check `pyproject.toml` correctly when it is passed via stdin (#16971) 
## Summary

Resolves #16950 and [a 1.5-year-old TODO
comment](8d16a5c8c9/crates/ruff/src/diagnostics.rs (L380)).

After this change, a `pyproject.toml` will be linted the same as any
Python files would when passed via stdin.

## Test Plan

Integration tests.
 2025-03-27 16:01:45 +00:00
Matthew Mckee b9a7328789
[red-knot] Make every type a subtype of object (#16960) 
## Summary

Mainly for partially fixing #16953

## Test Plan

Update is_subtype tests. And should maybe do these checks for many other
types (is subtype of object but object is not subtype)

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
 2025-03-27 14:24:18 +00:00
Brent Westbrook d70a3e6753
[syntax-errors] Multiple assignments in `case` pattern (#16957) 
Summary
--

This PR detects multiple assignments to the same name in `case` patterns
by recursively visiting each pattern.

Test Plan
--

New inline tests.
 2025-03-26 13:02:42 -04:00
Brent Westbrook 5697d21fca
[syntax-errors] Irrefutable case pattern before final case (#16905) 
Summary
--

Detects irrefutable `match` cases before the final case using a modified
version
of the existing `Pattern::is_irrefutable` method from the AST crate. The
modified method helps to retrieve a more precise diagnostic range to
match what
Python 3.13 shows in the REPL.

Test Plan
--

New inline tests, as well as some updates to existing tests that had
irrefutable
patterns before the last block.
 2025-03-26 12:27:16 -04:00
Wei Lee 58350ec93b
[`airflow`] refactor: remove unnecessary `Some` in `check_method`, `check_class_attribute` (`AIR302`) (#16975) 
## Summary

remove unnecessary `Some`

## Test Plan

It's a refactoring change. Existing test cases won't be affected
 2025-03-26 12:17:34 -04:00
Matthew Mckee aae4d0f3eb
[red-knot] A `FunctionType` can be a subtype of `Callable` (but never the other way around) (#16970) 
## Summary

Partially fixes #16953

## Test Plan

Update is_subtype_of.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-25 22:04:34 +00:00
Micha Reiser 8d16a5c8c9
[red-knot] Use `web-time` instead of `FileTime::now` (#16967) 
## Summary

`std::time::now` isn't available on `wasm32-unknown-unknown` but it is
used by `FileTime::now`.

This PR replaces the usages of `FileTime::now` with a target specific
helper function that we already had in the memory file system.
Fixes https://github.com/astral-sh/ruff/issues/16966

## Test Plan

Tested that the playground no longer crash when adding an extra-path
 2025-03-25 13:03:30 +00:00
Alex Waygood 4975c2f027
[red-knot] Fix panic on cyclic `*` imports (#16958) 
## Summary

Further work towards https://github.com/astral-sh/ruff/issues/14169.

We currently panic on encountering cyclic `*` imports. This is easily
fixed using fixpoint iteration.

## Test Plan

Added a test that panics on `main`, but passes with this PR
 2025-03-24 18:23:02 +00:00
Dhruv Manilawala dd5b02aaa2
[red-knot] Fix gradual equivalence for callable types (#16887) 
## Summary

As mentioned in
https://github.com/astral-sh/ruff/pull/16698#discussion_r2004920075,
part of #15382, this PR updates the `is_gradual_equivalent_to`
implementation between callable types to be similar to
`is_equivalent_to` and checks other attributes of parameters like name,
optionality, and parameter kind.

## Test Plan

Expand the existing test cases to consider other properties but not all
similar to how the tests are structured for subtyping and assignability.
 2025-03-24 23:46:06 +05:30
Aleksei Latyshev 68ea2b8b5b
[red-knot] simplify "removing" in UnionBuilder::add (#16947) 
## Summary

Simplify "removing" in UnionBuilder::add
It's now O(m) instead of O(n + m) and easier to read.

## Test Plan

cargo test (incl. mdtest)
 2025-03-24 14:04:03 -04:00
Alex Waygood e87fee4b3b
[red-knot] Add initial support for `*` imports (#16923) 
## Summary

This PR adds initial support for `*` imports to red-knot. The approach
is to implement a standalone query, called from semantic indexing, that
visits the module referenced by the `*` import and collects all
global-scope public names that will be imported by the `*` import. The
`SemanticIndexBuilder` then adds separate definitions for each of these
names, all keyed to the same `ast::Alias` node that represents the `*`
import.

There are many pieces of `*`-import semantics that are still yet to be
done, even with this PR:
- This PR does not attempt to implement any of the semantics to do with
`__all__`. (If a module defines `__all__`, then only the symbols
included in `__all__` are imported, _not_ all public global-scope
symbols.
- With the logic implemented in this PR as it currently stands, we
sometimes incorrectly consider a symbol bound even though it is defined
in a branch that is statically known to be dead code, e.g. (assuming the
target Python version is set to 3.11):

  ```py
  # a.py

  import sys

  if sys.version_info < (3, 10):
      class Foo: ...

  ```

  ```py
  # b.py

  from a import *

  print(Foo)  # this is unbound at runtime on 3.11,
# but we currently consider it bound with the logic in this PR
  ```

Implementing these features is important, but is for now deferred to
followup PRs.

Many thanks to @ntBre, who contributed to this PR in a pairing session
on Friday!

## Test Plan

Assertions in existing mdtests are adjusted, and several new ones are
added.
 2025-03-24 17:15:58 +00:00
Alex Waygood 66d0cf2a72
[red-knot] Add more tests for `*` imports (#16955) 
## Summary

This PR separates out the entirely new tests from
https://github.com/astral-sh/ruff/pull/16923 into a standalone PR. I'll
rebase https://github.com/astral-sh/ruff/pull/16923 on top of this
branch.

The reasons for separating it out are:
- It should make it clearer to see in
<https://github.com/astral-sh/ruff/pull/16923> exactly how the
functionality is changing (we can see the assertions in the tests
_change_, which isn't so obvious if the tests are entirely new)
- The diff on <https://github.com/astral-sh/ruff/pull/16923> is getting
pretty big; this should reduce the diff on that PR somewhat
- These tests seem useful in and of themselves, so even if we need to do
a wholesale revert of <https://github.com/astral-sh/ruff/pull/16923> for
whatever reason, it'll be nice to keep the tests

## Test Plan

`cargo test -p red_knot_python_semantic`
 2025-03-24 16:39:16 +00:00
Alex Waygood 888a910925
[red-knot] Demote the `negation_reverses_subtype_order` test back to flaky (#16951) 
Fixes #16913. See my analysis in the issue for the rationale
 2025-03-24 11:37:03 -04:00
Wei Lee 581b7005dc
[airflow] refactor: combine similar case condition (AIR302) (#16944) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

combine similar case condition in AIR302 

## Test Plan

<!-- How was it tested? -->

nothing should be changed. existing test case should already cover it
 2025-03-24 15:36:33 +00:00
Brent Westbrook 2711e08eb8
[syntax-errors] Fix false positive for parenthesized tuple index (#16948) 
Summary
--

Fixes #16943 by checking if the tuple is not parenthesized before
emitting an error.

Test Plan
--

New inline test based on the initial report
 2025-03-24 10:34:38 -04:00
Micha Reiser f5cdf23545
[red-knot] Add settings support to playground (#16929) 
## Summary

This PR extends the Red Knot playground by adding configuration support
by adding a `knot.json` file.

<img width="1679" alt="Screenshot 2025-03-23 at 21 12 16"
src="https://github.com/user-attachments/assets/81ff1588-a07a-4847-97d8-61250aa2feda"
/>
 2025-03-24 01:38:48 +00:00
John Stilley 3899f7156f
Fixing more spelling errors (#16926) 
## Summary

Here I fix the last English spelling errors I could find in the repo.

Again, I am trying not to touch variable/function names, or anything
that might be misspelled in the API. The goal is to make this PR safe
and easy to merge.

## Test Plan

I have run all the unit tests. Though, again, all of the changes I make
here are to docs and docstrings. I make no code changes, which I believe
should greatly mitigate the testing concerns.
 2025-03-23 10:55:14 -07:00
InSync 902d86e79e
[red-knot] Do not emit `invalid-return-type` for abstract functions (#16900) 
## Summary

Resolves #16895.

`abstractmethod` is now a `KnownFunction`. When a function is decorated
by `abstractmethod` or when the parent class inherits directly from
`Protocol`, `invalid-return-type` won't be emitted for that function.

## Test Plan

Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@oddbird.net>
 2025-03-23 17:51:10 +00:00
Daniel Wilton 9fe89ddfba
[`refurb`] Document why `UserDict`, `UserList`, `UserString` are preferred over `dict`, `list`, `str` (`FURB189`) (#16927) 
## Summary

This PR addresses docs issue
https://github.com/astral-sh/ruff/issues/14328.
 2025-03-23 13:24:39 -04:00
Matthew Mckee 08a0995108
[red-knot] Disambiguate display for intersection types (#16914) 
## Summary

Fixes #16912 

Create a new type `DisplayMaybeParenthesizedType` that is now used in
Union and Intersection display

## Test Plan

Update callable annotations
 2025-03-23 07:18:30 -07:00
InSync 2d892bc9f7
Fix typos (#16908) 
## Summary

The noun is spelled "descend<strong><em>a</em></strong>nt" and the
adjective "descend<strong><em>e</em></strong>nt".

## Test Plan

[From the English
Wiktionary](https://en.wiktionary.org/wiki/descendent#Usage_notes):

> The adjective, "descending from a biological ancestor", may be spelt
either with an <i>[a](https://en.wiktionary.org/wiki/-ant)</i> or with
an <i>[e](https://en.wiktionary.org/wiki/-ent)</i> in the final syllable
(see [descendant](https://en.wiktionary.org/wiki/descendant)). However,
the noun <i>descendant</i>, "one who is the progeny of someone", may be
spelt only with an <i>[a](https://en.wiktionary.org/wiki/-ant)</i>.
Compare also
<i>[dependent](https://en.wiktionary.org/wiki/dependent#English)</i> and
<i>[dependant](https://en.wiktionary.org/wiki/dependant#English)</i>.
 2025-03-23 07:15:56 -07:00
Shunsuke Shibayama ee51c2a389
[red-knot] fix ordering of `ClassDef` semantic index building (#16915) 
## Summary

From #16861

This PR fixes the incorrect `ClassDef` handling of
`SemanticIndexBuilder::visit_stmt`, which fixes some of the incorrect
behavior of referencing the class itself in the class scope (a complete
fix requires a different fix, which will be done in the another PR).

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-23 13:23:12 +00:00
Vasco Schiavo bb07ccd783
[`pylint`] Fix typo in documentation of PLC1802 (#16920) 2025-03-23 06:17:33 -05:00
John Stilley c35f2bfe32
Fixing various spelling errors (#16924) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This is a cleanup PR. I am fixing various English language spelling
errors. This is mostly in docs and docstrings.

## Test Plan

The usual CI tests were run. I tried to build the docs (though I had
some troubles there). The testing needs here are, I trust, very low
impact. (Though I would happily test more.)
 2025-03-23 08:08:40 +00:00
Micha Reiser 7fb765d9b6
[red-knot] Log sys-prefix origin for easier debugging (#16921) 
## Summary

Log the origin of the sys path prefix. This should help with debugging
if someone doesn't understand
why Red Knot picks up a certain venv.

## Test Plan

Ran the CLI and tested that it logs the origin
 2025-03-23 08:06:04 +00:00
Dhruv Manilawala 0360c6b219
[red-knot] Support calling a `typing.Callable` (#16888) 
## Summary

Part of #15382, this PR adds support for calling a variable that's
annotated with `typing.Callable`.

## Test Plan

Add test cases in a new `call/annotation.md` file.
 2025-03-23 02:39:33 +05:30
Dhruv Manilawala 1cffb323bc
[red-knot] Check assignability for two callable types (#16845) 
## Summary

Part of #15382

This PR adds support for checking the assignability of two general
callable types.

This is built on top of #16804 by including the gradual parameters check
and accepting a function that performs the check between the two types.

## Test Plan

Update `is_assignable_to.md` with callable types section.
 2025-03-23 02:28:44 +05:30
Matthew Mckee 92028efe3d
[red-knot] Fix disambiguate display for union types (#16907) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

When callables are displayed in unions, like:
```py
from typing import Callable


def foo(x: Callable[[], int] | None):
    # red-knot: Revealed type is `() -> int | None` [revealed-type]
    reveal_type(x)
```

This leaves the type rather ambiguous, to fix this we can add
parenthesis to callable type in union

Fixes #16893

## Test Plan

Update callable annotations tests

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-22 13:08:51 +01:00
Matthew Mckee 7b86f54c4c
[red-knot] Add line number to mdtest panic message about language tag mismatch (#16906) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Fixes #16898 

## Test Plan

Update test for lang mismatch panic
 2025-03-22 13:05:31 +01:00
Brent Westbrook e4f5fe8cf7
[syntax-errors] Duplicate type parameter names (#16858) 
Summary
--

Detects duplicate type parameter names in function definitions, class
definitions, and type alias statements.

I also boxed the `type_params` field on `StmtTypeAlias` to make it
easier to
`match` with functions and classes. (That's the reason for the red-knot
code
owner review requests, sorry!)

Test Plan
--

New `ruff_python_syntax_errors` unit tests.

Fixes #11119.
 2025-03-21 15:06:22 -04:00
Brent Westbrook 2baaedda6c
[syntax-errors] Start detecting compile-time syntax errors (#16106) 
## Summary

This PR implements the "greeter" approach for checking the AST for
syntax errors emitted by the CPython compiler. It introduces two main
infrastructural changes to support all of the compile-time errors:
1. Adds a new `semantic_errors` module to the parser crate with public
`SemanticSyntaxChecker` and `SemanticSyntaxError` types
2. Embeds a `SemanticSyntaxChecker` in the `ruff_linter::Checker` for
checking these errors in ruff

As a proof of concept, it also implements detection of two syntax
errors:
1. A reimplementation of
[`late-future-import`](https://docs.astral.sh/ruff/rules/late-future-import/)
(`F404`)
2. Detection of rebound comprehension iteration variables
(https://github.com/astral-sh/ruff/issues/14395)

## Test plan
Existing F404 tests, new inline tests in the `ruff_python_parser` crate,
and a linter CLI test showing an example of the `Message` output.

I also tested in VS Code, where `preview = false` and turning off syntax
errors both disable the new errors:


![image](https://github.com/user-attachments/assets/cf453d95-04f7-484b-8440-cb812f29d45e)

And on the playground, where `preview = false` also disables the errors:


![image](https://github.com/user-attachments/assets/a97570c4-1efa-439f-9d99-a54487dd6064)


Fixes #14395

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-21 14:45:25 -04:00
Ash Berlin-Taylor b1deab83d9
Update replacement paths for AIR302 (#16876) 
I am one of the core developers of Airflow and working on the
"airflow.sdk"
package, and this updates the recommended replacments to the correct
user-facing imports.[^1]

cc @Lee-W @uranusjr 

[^1]:
33f0f1d639/task-sdk/src/airflow/sdk/__init__.py (L68-L93)

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

Hope and pray? 😉 

I'm sure there are some snapshot files I'm supposed to fix first.


<!-- How was it tested? -->
 2025-03-21 18:46:56 +01:00
Alex Waygood d21d639ee0
[red-knot] Avoid false-positive diagnostics on `*` import statements (#16899) 
## Summary

This PR removes false-positive diagnostics for `*` imports. Currently we
always emit a diagnostic for these statements unless the module we're
importing from has a symbol named `"*"` in its symbol table for the
global scope. (And if we were doing everything correctly, no module ever
would have a symbol named `"*"` in its global scope!)

The fix here is sort-of hacky and won't be what we'll want to do
long-term. However, I think it's useful to do this as a first step
since:
- It significantly reduces false positives when running on code that
uses `*` imports
- It "resets" the tests to a cleaner state with many fewer TODOs, making
it easier to see what the hard work is that's still to be done.

## Test Plan

`cargo test -p red_knot_python_semantic`
 2025-03-21 14:41:49 +00:00
Alex Waygood 14eb4cac88
[red-knot] Add failing tests for `*` imports (#16873) 
## Summary

This PR adds a suite of tests for wildcard (`*`) imports. The tests
nearly all fail for now, and those that don't, ahem, pass for the wrong
reasons...

I've tried to add TODO comments in all instances for places where we are
currently inferring the incorrect thing, incorrectly emitting a
diagnostic, or emitting a diagnostic with a bad error message.

## Test Plan

`cargo test -p red_knot_python_semantic`
 2025-03-21 14:17:15 +00:00
Douglas Creager c03c28d199
[red-knot] Break up call binding into two phases (#16546) 
This breaks up call binding into two phases:

- **_Matching parameters_** just looks at the names and kinds
(positional/keyword) of each formal and actual parameters, and matches
them up. Most of the current call binding errors happen during this
phase.

- Once we have matched up formal and actual parameters, we can **_infer
types_** of each actual parameter, and **_check_** that each one is
assignable to the corresponding formal parameter type.

As part of this, we add information to each formal parameter about
whether it is a type form or not. Once [PEP
747](https://peps.python.org/pep-0747/) is finalized, we can hook that
up to this internal type form representation. This replaces the
`ParameterExpectations` type, which did the same thing in a more ad hoc
way.

While we're here, we add a new fluent API for building `Parameter`s,
which makes our signature constructors a bit nicer to read. We also
eliminate a TODO where we were consuming types from the argument list
instead of the bound parameter list when evaluating our special-case
known functions.

Closes #15460

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-21 09:38:11 -04:00
Brent Westbrook 4773878ee7
Bump 0.11.2 (#16896) 
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
 2025-03-21 09:17:07 -04:00
Junhson Jean-Baptiste 2a4d835132
Use the common `OperatorPrecedence` for the parser (#16747) 
## Summary

This change continues to resolve #16071 (and continues the work started
in #16162). Specifically, this PR changes the code in the parser so that
it uses the `OperatorPrecedence` struct from `ruff_python_ast` instead
of its own version. This is part of an effort to get rid of the
redundant definitions of `OperatorPrecedence` throughout the codebase.

Note that this PR only makes this change for `ruff_python_parser` -- we
still want to make a similar change for the formatter (namely the
`OperatorPrecedence` defined in the expression part of the formatter,
the pattern one is different). I separated the work to keep the PRs
small and easily reviewable.

## Test Plan

Because this is an internal change, I didn't add any additional tests.
Existing tests do pass.
 2025-03-21 09:40:37 +05:30
Dhruv Manilawala 04a8756379
[red-knot] Check subtype relation between callable types (#16804) 
## Summary

Part of #15382

This PR adds support for checking the subtype relationship between the
two callable types.

The main source of reference used for implementation is
https://typing.python.org/en/latest/spec/callables.html#assignability-rules-for-callables.

The implementation is split into two phases:
1. Check all the positional parameters which includes positional-only,
standard (positional or keyword) and variadic kind
2. Collect all the keywords in a `HashMap` to do the keyword parameters
check via name lookup

For (1), there's a helper struct which is similar to `.zip_longest`
(from `itertools`) except that it allows control over one of the
iterator as that's required when processing a variadic parameter. This
is required because positional parameters needs to be checked as per
their position between the two callable types. The struct also keeps
track of the current iteration element because when the loop is exited
(to move on to the phase 2) the current iteration element would be
carried over to the phase 2 check.

This struct is internal to the `is_subtype_of` method as I don't think
it makes sense to expose it outside. It also allows me to use "self" and
"other" suffixed field names as that's only relevant in that context.

## Test Plan

Add extensive tests in markdown.

Converted all of the code snippets from
https://typing.python.org/en/latest/spec/callables.html#assignability-rules-for-callables
to use `knot_extensions.is_subtype_of` and verified the result.
 2025-03-21 03:27:22 +00:00
Dhruv Manilawala 193c38199e
[red-knot] Check whether two callable types are equivalent (#16698) 
## Summary

This PR checks whether two callable types are equivalent or not.

This is required because for an equivalence relationship, the default
value does not necessarily need to be the same but if the parameter in
one of the callable has a default value then the corresponding parameter
in the other callable should also have a default value. This is the main
reason a manual implementation is required.

And, as per https://typing.python.org/en/latest/spec/callables.html#id4,
the default _type_ doesn't participate in a subtype relationship, only
the optionality (required or not) participates. This means that the
following two callable types are equivalent:

```py
def f1(a: int = 1) -> None: ...
def f2(a: int = 2) -> None: ...
```

Additionally, the name of positional-only, variadic and keyword-variadic
are not required to be the same for an equivalence relation.

A potential solution to avoid the manual implementation would be to only
store whether a parameter has a default value or not but the type is
currently required to check for assignability.

## Test plan

Add tests for callable types in `is_equivalent_to.md`
 2025-03-21 03:19:07 +00:00
Matthew Mckee 63e78b41cd
[red-knot] Ban most `Type::Instance` types in type expressions (#16872) 
## Summary

Catch some Instances, but raise type error for the rest of them
Fixes #16851 

## Test Plan

Extend invalid.md in annotations

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
 2025-03-20 15:19:56 -07:00
Alex Waygood 296d67a496
Special-case value-expression inference of special form subscriptions (#16877) 
## Summary

Currently for something like `X = typing.Tuple[str, str]`, we infer the
value of `X` as `object`. That's because `Tuple` (like many of the
symbols in the typing module) is annotated as a `_SpecialForm` instance
in typeshed's stubs:


23382f5f8c/crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi (L215)

and we don't understand implicit type aliases yet, and the stub for
`_SpecialForm.__getitem__` says it always returns `object`:


23382f5f8c/crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi (L198-L200)

We have existing false positives in our test suite due to this:


23382f5f8c/crates/red_knot_python_semantic/resources/mdtest/annotations/annotated.md?plain=1#L76-L78

and it's causing _many_ new false positives in #16872, which tries to
make our annotation-expression parsing stricter in some ways.

This PR therefore adds some small special casing for `KnownInstanceType`
variants that fallback to `_SpecialForm`, so that these false positives
can be avoided.

## Test Plan

Existing mdtest altered.

Cc. @MatthewMckee4
 2025-03-20 21:46:02 +00:00
Brent Westbrook 42cbce538b
[syntax-errors] Fix star annotation before Python 3.11 (#16878) 
Summary
--

Fixes #16874. I previously emitted a syntax error when starred
annotations were _allowed_ rather than when they were actually used.
This caused false positives for any starred parameter name because these
are allowed to have starred annotations but not required to. The fix is
to check if the annotation is actually starred after parsing it.

Test Plan
--

New inline parser tests derived from the initial report and more
examples from the comments, although I think the first case should cover
them all.
 2025-03-20 17:44:52 -04:00
Shunsuke Shibayama 23382f5f8c
[red-knot] add test cases result in false positive errors (#16856) 
## Summary

From #16641

The previous PR attempted to fix the errors presented in this PR, but as
discussed in the conversation, it was concluded that the approach was
undesirable and that further work would be needed to fix the errors with
a correct general solution.

In this PR, I instead add the test cases from the previous PR as TODOs,
as a starting point for future work.

## Test Plan

---------

Co-authored-by: Carl Meyer <carl@oddbird.net>
 2025-03-20 17:17:54 +00:00
Dylan c1971fdde2
Bump 0.11.1 (#16871) 2025-03-20 09:50:46 -05:00
Matthew Mckee cdafd8e32b
Allow discovery of venv in VIRTUAL_ENV env variable (#16853) 
## Summary

Fixes #16744 

Allows the cli to find a virtual environment from the VIRTUAL_ENV
environment variable if no `--python` is set

## Test Plan

Manual testing, of:
- Virtual environments explicitly activated using `source .venv/bin/activate`
- Virtual environments implicilty activated via `uv run`
- Broken virtual environments with no `pyvenv.cfg` file
 2025-03-20 13:55:35 +00:00
Junhson Jean-Baptiste 47c4ccff5d
Separate `BitXorOr` into `BitXor` and `BitOr` precedence (#16844) 
## Summary

This change follows up on the bug-fix requested in #16747 --
`ruff_python_ast::OperatorPrecedence` had an enum variant, `BitXorOr`,
which which gave the same precedence to the `|` and `^` operators. This
goes against [Python's documentation for operator
precedence](https://docs.python.org/3/reference/expressions.html#operator-precedence),
so this PR changes the code so that it's correct.

This is part of the overall effort to unify redundant definitions of
`OperatorPrecedence` throughout the codebase (#16071)

## Test Plan

Because this is an internal change, I only ran existing tests to ensure
nothing was broken.
 2025-03-20 16:13:47 +05:30
Dylan 74f64d3f96
Server: Allow `FixAll` action in presence of version-specific syntax errors (#16848) 
The single flag `has_syntax_error` on `LinterResult` is replaced with
two (private) flags: `has_valid_syntax` and
`has_no_unsupported_syntax_errors`, which record whether there are
`ParseError`s or `UnsupportedSyntaxError`s, respectively. Only the
former is used to prevent a `FixAll` action.

An attempt has been made to make consistent the usage of the phrases
"valid syntax" (which seems to be used to refer only to _parser_ errors)
and "syntax error" (which refers to both _parser_ errors and
version-specific syntax errors).

Closes #16841
 2025-03-20 05:09:14 -05:00
Vasco Schiavo 999fd4f885
[`refurb`] Fix starred expressions fix (`FURB161`) (#16550) 
The PR partially solves issue #16457

Specifically, it solves the following problem:

```text
$ cat >furb161_1.py <<'# EOF'
print(bin(*[123]).count("1"))
# EOF

$ python furb161_1.py
6

$ ruff --isolated check --target-version py310 --preview --select FURB161 furb161_1.py --diff 2>&1 | grep error:
error: Fix introduced a syntax error. Reverting all changes.
```

Now starred expressions are corrected handled.
 2025-03-19 17:43:58 -04:00
Dylan 433a342656
[`flake8-executable`] Add pytest and uv run to help message for `shebang-missing-python` (`EXE003`) (#16855) 
Followup to #16849 per
https://github.com/astral-sh/ruff/pull/16849#issuecomment-2737316564
 2025-03-19 13:12:32 -05:00
Matthew Mckee 4ed93b4311
Show more precise messages in invalid type expressions (#16850) 
## Summary

Some error messages were not very specific; this PR improves them

## Test Plan

New mdtests added; existing mdtests tweaked
 2025-03-19 17:00:30 +00:00
Dylan 98fdc0ebae
[`flake8-executables`] Allow `uv run` in shebang line for `shebang-missing-python` (`EXE003`) (#16849) 
Skip the lint for [shebang-missing-python
(EXE003)](https://docs.astral.sh/ruff/rules/shebang-missing-python/#shebang-missing-python-exe003)
if we find `uv run` on the shebang line.

Closes #13021
 2025-03-19 10:35:07 -05:00
Josh Cannon 861931795c
Add `--exit-non-zero-on-format` (#16009) 
## Summary

Fixes #8191 by introducing `--exit-non-zero-on-format` to `ruff format`
which pretty much does what it says on the tin.

## Test Plan

Added a new test!

---------

Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
 2025-03-19 10:55:05 -04:00
Alex Waygood a3f3d734a1
[red-knot] Ban list literals in most contexts in type expressions (#16847) 
## Summary

This PR reworks `TypeInferenceBuilder::infer_type_expression()` so that
we emit diagnostics when encountering a list literal in a type
expression. The only place where a list literal is allowed in a type
expression is if it appears as the first argument to `Callable[]`, and
`Callable` is already heavily special-cased in our type-expression
parsing.

In order to ensure that list literals are _always_ allowed as the
_first_ argument to `Callabler` (but never allowed as the second, third,
etc. argument), I had to do some refactoring of our type-expression
parsing for `Callable` annotations.

## Test Plan

New mdtests added, and existing ones updated
 2025-03-19 14:42:42 +00:00
Matthew Mckee 3a5f1d46c0
[red-knot] Make' Type::in_type_expression()' exhaustive for Type::KnownInstance (#16836) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

fixes #15048 
We want to handle more types from Type::KnownInstance 

## Test Plan

Add tests for each type added explicitly in the match

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
 2025-03-19 07:36:28 -07:00
Alex Waygood f3f3e55d97
[red-knot] Minor cleanup to `infer_parameterized_known_instance_type_expression` (#16846) 
## Summary

These are just cosmetic changes, but I'm separating them out into a
standalone PR to make a branch I have stacked on top of this easier to
review

## Test Plan

Existing tests all pass
 2025-03-19 14:19:13 +00:00
Brent Westbrook 22de00de16 [internal] Return `Message`s from `check_path` (#16837) 
Summary
--

This PR updates `check_path` in the `ruff_linter` crate to return a
`Vec<Message>` instead of a `Vec<Diagnostic>`. The main motivation for
this is to make it easier to convert semantic syntax errors directly
into `Message`s rather than `Diagnostic`s in #16106. However, this also
has the benefit of keeping the preview check on unsupported syntax
errors in `check_path`, as suggested in
https://github.com/astral-sh/ruff/pull/16429#discussion_r1974748024.

All of the interesting changes are in the first commit. The second
commit just renames variables like `diagnostics` to `messages`, and the
third commit is a tiny import fix.

I also updated the `ExpandedMessage::location` field name, which caused
a few extra commits tidying up the playground code. I thought it was
nicely symmetric with `end_location`, but I'm happy to revert that too.

Test Plan
--

Existing tests. I also tested the playground and server manually.
 2025-03-19 10:08:07 -04:00
Dhruv Manilawala a69f6240cc
[red-knot] Infer `lambda` return type as `Unknown` (#16695) 
## Summary

Part of #15382

This PR infers the return type `lambda` expression as `Unknown`. In the
future, it would be more useful to infer the expression type considering
the surrounding context (#16696).

## Test Plan

Update existing test cases from `@todo` to the (verified) return type.
 2025-03-18 22:48:10 +05:30
Dhruv Manilawala c3d429ddd8
[red-knot] Move `name` field on parameter kind (#16830) 
## Summary

Previously, the `name` field was on `Parameter` which required it to be
always optional regardless of the parameter kind because a
`typing.Callable` signature does not have name for the parameters. This
is the case for positional-only parameters. This wasn't enforced at the
type level which meant that downstream usages would have to unwrap on
`name` even though it's guaranteed to be present.

This commit moves the `name` field from `Parameter` to the
`ParameterKind` variants and makes it optional only for
`ParameterKind::PositionalOnly` variant while required for all other
variants.

One change that's now required is that a `Callable` form using a gradual
form for parameter types (`...`) would have a default `args` and
`kwargs` name used for variadic and keyword-variadic parameter kind
respectively. This is also the case for invalid `Callable` type forms. I
think this is fine as names are not relevant in this context but happy
to make it optional even in variadic variants.

## Test Plan

No new tests; make sure existing tests are passing.
 2025-03-18 22:47:44 +05:30
Matthew Mckee ab3ec4de6a
[red-knot] Emit errors for more AST nodes that are invalid (or only valid in specific contexts) in type expressions (#16822) 
## Summary

Add error messages for invalid nodes in type expressions

Fixes #16816 

## Test Plan

Extend annotations/invalid.md to handle these invalid AST nodes error
messages
 2025-03-18 17:16:50 +00:00
Micha Reiser c027979851
Red Knot Playground (#12681) 
## Summary

This PR adds a playground for Red Knot

[Screencast from 2024-08-14
10-33-54.webm](https://github.com/user-attachments/assets/ae81d85f-74a3-4ba6-bb61-4a871b622f05)

Sharing does work 😆 I just forgot to start wrangler. 


It supports:

* Multiple files
* Showing the AST
* Showing the tokens
* Sharing
* Persistence to local storage

Future extensions:

* Configuration support: The `pyproject.toml` would *just* be another
file.
* Showing type information on hover

## Blockers

~~Salsa uses `catch_unwind` to break cycles, which Red Knot uses
extensively when inferring types in the standard library.
However, WASM (at least `wasm32-unknown-unknown`) doesn't support
`catch_unwind` today, so the playground always crashes when the type
inference encounters a cycle.~~

~~I created a discussion in the [salsa
zulip](https://salsa.zulipchat.com/#narrow/stream/333573-salsa-3.2E0/topic/WASM.20support)
to see if it would be possible to **not** use catch unwind to break
cycles.~~

~~[Rust tracking issue for WASM catch unwind
support](https://github.com/rust-lang/rust/issues/118168)~~

~~I tried to build the WASM with the nightly compiler option but ran
into problems because wasm-bindgen doesn't support WASM-exceptions. We
could try to write the binding code by hand.~~

~~Another alternative is to use `wasm32-unknown-emscripten` but it's
rather painful to build~~
 2025-03-18 17:17:11 +01:00
Brent Westbrook dcf31c9348
[syntax-errors] PEP 701 f-strings before Python 3.12 (#16543) 
## Summary

This PR detects the use of PEP 701 f-strings before 3.12. This one
sounded difficult and ended up being pretty easy, so I think there's a
good chance I've over-simplified things. However, from experimenting in
the Python REPL and checking with [pyright], I think this is correct.
pyright actually doesn't even flag the comment case, but Python does.

I also checked pyright's implementation for
[quotes](98dc4469cc/packages/pyright-internal/src/analyzer/checker.ts (L1379-L1398))
and
[escapes](98dc4469cc/packages/pyright-internal/src/analyzer/checker.ts (L1365-L1377))
and think I've approximated how they do it.

Python's error messages also point to the simple approach of these
characters simply not being allowed:

```pycon
Python 3.11.11 (main, Feb 12 2025, 14:51:05) [Clang 19.1.6 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f'''multiline {
... expression # comment
... }'''
  File "<stdin>", line 3
    }'''
        ^
SyntaxError: f-string expression part cannot include '#'
>>> f'''{not a line \
... continuation}'''
  File "<stdin>", line 2
    continuation}'''
                    ^
SyntaxError: f-string expression part cannot include a backslash
>>> f'hello {'world'}'
  File "<stdin>", line 1
    f'hello {'world'}'
              ^^^^^
SyntaxError: f-string: expecting '}'
```

And since escapes aren't allowed, I don't think there are any tricky
cases where nested quotes or comments can sneak in.

It's also slightly annoying that the error is repeated for every nested
quote character, but that also mirrors pyright, although they highlight
the whole nested string, which is a little nicer. However, their check
is in the analysis phase, so I don't think we have such easy access to
the quoted range, at least without adding another mini visitor.

## Test Plan

New inline tests

[pyright]:
https://pyright-play.net/?pythonVersion=3.11&strict=true&code=EYQw5gBAvBAmCWBjALgCgO4gHaygRgEoAoEaCAIgBpyiiBiCLAUwGdknYIBHAVwHt2LIgDMA5AFlwSCJhwAuCAG8IoMAG1Rs2KIC6EAL6iIxosbPmLlq5foRWiEAAcmERAAsQAJxAomnltY2wuSKogA6WKIAdABWfPBYqCAE%2BuSBVqbpWVm2iHwAtvlMWMgB2ekiolUAgq4FjgA2TAAeEMieSADWCsoV5qoaqrrGDJ5MiDz%2B8ABuLqosAIREhlXlaybrmyYMXsDw7V4AnoysyAmQ5SIhwYo3d9cheADUeKlv5O%2BpQA
 2025-03-18 11:12:15 -04:00
cake-monotone 4ab529803f
[red-knot] Refactor `property_tests.rs` into `property_tests` module structure (#16827) 
## Summary

For now, `property_tests.rs` has grown larger and larger, making the
file difficult to read and maintain.

Although the code has been split, the test paths and full names remain
unchanged. There are no changes affecting test execution.
 2025-03-18 12:59:14 +00:00
Alex Waygood 23b7df9b29
[red-knot] Simplify `IterationError` and `ContextManagerError` (#16820) 
## Summary

This PR simplifies `IterationError` and `ContextManagerError` so that
they no longer "remember" what type it was that was (respectively) not
iterable or not valid as a context manager. Instead, the type that was
iterated over (or was used as a context manager) is passed back in when
calling the error struct's `report_diagnostic` method.

The motivations for this are:
- It significantly simplifies the code
- It reduces the size of these types on the stack

## Test Plan

`cargo test -p red_knot_python_semantic`
 2025-03-18 11:30:41 +00:00
cake-monotone 3e2cf5d7c4
[red-knot] Improve property test performance by cloning db instead of holding `MutexGuard` (#16823) 
## Summary

This PR brings an optimization.

- `get_cached_db` no longer returns a `MutexGuard`; instead, it returns
a cloned database.

### `get_cached_db`

Previously, the `MutexGuard` was held inside the property test function
(defined in the macro), which prevented multiple property tests from
running in parallel. More specifically, the program could only test one
random test case at a time, which likely caused a significant
bottleneck.

On my local machine, running:

```
QUICKCHECK_TESTS=100000 cargo test --release -p red_knot_python_semantic -- --ignored stable
```

showed about **a 75% speedup** (from \~60s to \~15s).
 2025-03-18 09:09:57 +01:00
Peter Hill 433879d852
[ruff] Fix `--statistics` reporting for unsafe fixes (#16756) 
Fixes #16751

## Summary

Previously, unsafe fixes were counted as "fixable" in
`Printer::write_statistics`, in contrast to the behaviour in
`Printer::write_once`. This changes the behaviour to align with
`write_once`, including them only if `--unsafe-fixes` is set.

We now also reuse `Printer::write_summary` to avoid duplicating the
logic for whether or not to report if there are hidden fixes.

## Test Plan

Existing tests modified to use an unsafe-fixable rule, and new ones
added to cover the case with `--unsafe-fixes`
 2025-03-18 08:03:14 +01:00
Kaxil Naik b7d232cf89
[`airflow`] Add `chain`, `chain_linear` and `cross_downstream` for `AIR302` (#16647) 
## Summary

Similar to https://github.com/astral-sh/ruff/pull/16014. PR on Airflow
side: https://github.com/apache/airflow/pull/47639

## Test Plan

A test fixture has been updated
 2025-03-18 11:08:45 +05:30
Andrew Gallant bd9eab059f red_knot: update diagnostic output snapshots 
These should all be minor cosmetic changes. To summarize:

* In many cases, `-` was replaced with `^` for primary annotations.
This is because, previously, whether `-` or `^` was used depended
on the severity. But in the new data model, it's based on whether
the annotation is "primary" or not. We could of course change this
in whatever way we want, but I think we should roll with this for now.

* The "secondary messages" in the old API are rendered as
sub-diagnostics. This in turn results in a small change in the output
format, since previously, the secondary messages were represented as
just another snippet. We use sub-diagnostics because that's the intended
way to enforce relative ordering between messages within a diagnostic.

* The "info:" prefix used in some annotation messages has been dropped.
We could re-add this, but I think I like it better without this prefix.

I believe those 3 cover all of the snapshot changes here.
 2025-03-17 12:46:49 -04:00
Andrew Gallant 6883c1dde7 ruff_db: delete old diagnostic renderer 
... and switch to the new one.

We do this switch by converting the old diagnostics to a
`Diagnostic`, and then rendering that.

This does not quite emit identical output. There are some
changes. They *could* be fixed to remain the same, but the
changes aren't obviously worse to me and I think the right
way to *improve* them is to move Red Knot to the new `Diagnostic`
API.

The next commit will have the snapshot changes.
 2025-03-17 12:46:49 -04:00
Andrew Gallant 9291074ba6 ruff_db: tweak main diagnostic message 
In our existing diagnostics, our message is just the diagnostic
ID, and the message goes to the annotation. In reality, the
diagnostic can have its own message distinct from the optional
messages associated with an annotation.

In order to make the outputs match, we do a small tweak here:
when the main diagnostic message is empty, we drop the colon
after the diagnostic ID.

I expect that we'll want to rejigger this output format more
in the future, but for now this was a very simple change to
preserve the status quo.
 2025-03-17 12:46:49 -04:00
Andrew Gallant 602a27c4e3 ruff_db: tweak number of line terminators emitted in new diagnostic renderer 
When moving over to the new renderer, I noticed that it
was emitting an extra line terminator compared to the status
quo. This removes it by turning the line terminator into a
line delimiter between diagnostics.
 2025-03-17 12:46:49 -04:00
Andrew Gallant ff548b1272 ruff_db: clarify the error conditions of `Diagnostic::print` 2025-03-17 12:46:49 -04:00
Brent Westbrook b2e0ae6416
[`flake8-gettext`] Swap `format-` and `printf-in-get-text-func-call` examples (`INT002`, `INT003`) (#16769) 
Summary
--
Fixes #16735. I also checked `INT001`, and it correctly has an f-string
example.

Test Plan
--
None
 2025-03-17 14:37:38 +00:00
Douglas Creager 23ccb52fa6
[red-knot] Handle unions of callables better (#16716) 
This cleans up how we handle calling unions of types. #16568 adding a
three-level structure for callable signatures (`Signatures`,
`CallableSignature`, and `Signature`) to handle unions and overloads.

This PR updates the bindings side to mimic that structure. What used to
be called `CallOutcome` is now `Bindings`, and represents the result of
binding actual arguments against a possible union of callables.
`CallableBinding` is the result of binding a single, possibly
overloaded, callable type. `Binding` is the result of binding a single
overload.

While we're here, this also cleans up `CallError` greatly. It was
previously extracting error information from the bindings and storing it
in the error result. It is now a simple enum, carrying no data, that's
used as a status code to talk about whether the overall binding was
successful or not. We are now more consistent about walking the binding
itself to get detailed information about _how_ the binding was
unsucessful.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-17 10:35:52 -04:00
Dhruv Manilawala 3ccc8dbbf9
[red-knot] Fix fully static check for callable type (#16803) 
## Summary

This PR fixes a bug in the check for fully static callable type where we
would skip unannotated parameter type.

## Test Plan

Add tests using the new `CallableTypeFromFunction` special form.
 2025-03-17 20:01:30 +05:30
Brent Westbrook 75a562d313
[syntax-errors] Parenthesized context managers before Python 3.9 (#16523) 
Summary
--

I thought this was very complicated based on the comment here:
https://github.com/astral-sh/ruff/pull/16106#issuecomment-2653505671 and
on some of the discussion in the CPython issue here:
https://github.com/python/cpython/issues/56991. However, after a little
bit of experimentation, I think it boils down to this example:

```python
with (x as y): ...
```

The issue is parentheses around a `with` item with an `optional_var`, as
we (and
[Python](https://docs.python.org/3/library/ast.html#ast.withitem)) call
the trailing variable name (`y` in this case). It's not actually about
line breaks after all, except that line breaks are allowed in
parenthesized expressions, which explains the validity of cases like


```pycon
>>> with (
...     x,
...     y
... ) as foo:
...     pass
... 
```

even on Python 3.8.

I followed [pyright]'s example again here on the diagnostic range (just
the opening paren) and the wording of the error.


Test Plan
--
Inline tests

[pyright]:
https://pyright-play.net/?pythonVersion=3.7&strict=true&code=FAdwlgLgFgBAFAewA4FMB2cBEAzBCB0EAHhJgJQwCGAzjLgmQFwz6tA
 2025-03-17 08:54:55 -04:00
Alex Waygood 50b66dc025
[red-knot] Stabilize `negation_reverses_subtype_order` property test (#16801) 
## Summary

This is a re-creation of https://github.com/astral-sh/ruff/pull/16764 by
@mtshiba, which I closed meaning to immediately reopen (GitHub wasn't
updating the PR with the latest pushed changes), and which GitHub will
not allow me to reopen for some reason. Pasting the summary from that PR
below:

> From https://github.com/astral-sh/ruff/pull/16641
> 
> As stated in this comment
(https://github.com/astral-sh/ruff/pull/16641#discussion_r1996153702),
the current ordering implementation for intersection types is incorrect.
So, I will introduce lexicographic ordering for intersection types.

## Test Plan

One property test stabilised (tested locally with
`QUICKCHECK_TESTS=2000000 cargo test --release -p
red_knot_python_semantic -- --ignored
types::property_tests::stable::negation_reverses_subtype_order`), and
existing mdtests that previously failed now pass.

Primarily-authored-by:
[mtshiba](https://github.com/astral-sh/ruff/commits?author=mtshiba)

---------

Co-authored-by: Shunsuke Shibayama <sbym1346@gmail.com>
 2025-03-17 12:33:38 +00:00
Matthew Mckee 24707777af
[red-knot] Emit error if int/float/complex/bytes/boolean literals appear in type expressions outside `typing.Literal[]` (#16765) 
## Summary
Fixes https://github.com/astral-sh/ruff/issues/16532

## Test Plan

New mdtest assertions added

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
 2025-03-17 11:56:16 +00:00
Alex Waygood 38bfda94ce
[syntax-errors] Improve error message and range for pre-PEP-614 decorator syntax errors (#16581) 
## Summary

A small followup to https://github.com/astral-sh/ruff/pull/16386. We now
tell the user exactly what it was about their decorator that constituted
invalid syntax on Python <3.9, and the range now highlights the specific
sub-expression that is invalid rather than highlighting the whole
decorator

## Test Plan

Inline snapshots are updated, and new ones are added.
 2025-03-17 11:17:27 +00:00
Mauro Fontana 4da6936ec4
[`flake8-bandit`] Allow raw strings in `suspicious-mark-safe-usage` (`S308`) #16702 (#16770) 
## Summary
Stop flagging each invocation of `django.utils.safestring.mark_safe`
(also available at, `django.utils.html.mark_safe`) as an error.

Instead, allow string literals as valid uses for `mark_safe`.

Also, update the documentation, pointing at
`django.utils.html.format_html` for dynamic content generation use
cases.

Closes #16702 

## Test Plan
I verified several possible uses, but string literals, are still
flagged.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-17 11:29:07 +01:00
Dylan 238ec39c56
[`refurb`] Avoid panicking `unwrap` in `verbose-decimal-constructor` (`FURB157`) (#16777) 2025-03-17 05:09:07 -05:00
Micha Reiser b04103fa1d
[red-knot] Add `--color` CLI option (#16758) 
## Summary

This PR adds a new `--color` CLI option that controls whether the output
should be colorized or not.

This is implements part of
https://github.com/astral-sh/ruff/issues/16727 except that it doesn't
implement the persistent configuration support as initially proposed in
the CLI document. I realized, that having this as a persistent
configuration is somewhat awkward because we may end up writing tracing
logs **before** we loaded and resolved the settings. Arguably, it's
probably fine to color the output up to that point, but it feels like a
somewhat broken experience. That's why I decided not to add the
persistent configuration option for now.


## Test Plan

I tested this change manually by running Red Knot with `--color=always`,
`--color=never`, and `--color=auto` (or no argument) and verified that:

* The diagnostics are or aren't colored
* The tracing output is or isn't colored.

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
 2025-03-17 10:06:34 +00:00
Micha Reiser c100d519e9
[internal]: Upgrade salsa (#16794) 
## Summary

Another salsa upgrade. 

The main motivation is to stay on a recent salsa version because there
are still a lot of breaking changes happening.
The most significant changes in this update:

* Salsa no longer derives `Debug` by default. It now requires
`interned(debug)` (or similar)
* This version ships the foundation for garbage collecting interned
values. However, this comes at the cost that queries now track which
interned values they created (or read). The micro benchmarks in the
salsa repo showed a significant perf regression. Will see if this also
visible in our benchmarks.

## Test Plan

`cargo test`
 2025-03-17 11:05:54 +01:00
Micha Reiser 6f5a68608e
[ci]: Fixup codspeed upgrade (#16790) 
## Summary

Benchmark isn't a required build step. That's why
https://github.com/astral-sh/ruff/pull/16784/ got merged with the step
failing.

This PR fixes up the benchmarking step
 2025-03-17 09:14:22 +01:00
Carl Meyer 2de8455e43
[red-knot] LSP: only emit WARN logs from non-red-knot sources (#16760) 
Currently the red-knot LSP server emits any log messages of level `INFO`
or higher from non-red-knot crates. This makes its output quite verbose,
because Salsa emits an `INFO` level message every time it executes a
query. I use red-knot as LSP with neovim, and this spams the log file
quite a lot.

It seems like a better default to only emit `WARN` or higher messages
from non-red-knot sources.

I confirmed that this fixes the nvim LSP log spam.
 2025-03-15 08:47:50 -07:00
github-actions[bot] 1fab292ec1
Sync vendored typeshed stubs (#16762) 
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
 2025-03-15 00:38:58 +00:00
David Peter ebcad6e641
[red-knot] Use `try_call_dunder` for augmented assignment (#16717) 
## Summary

Uses the `try_call_dunder` infrastructure for augmented assignment and
fixes the logic to work for types other than `Type::Instance(…)`. This
allows us to infer the correct type here:
```py
x = (1, 2)
x += (3, 4)
reveal_type(x)  # revealed: tuple[Literal[1], Literal[2], Literal[3], Literal[4]]
```
Or in this (extremely weird) scenario:
```py
class Meta(type):
    def __iadd__(cls, other: int) -> str:
        return ""

class C(metaclass=Meta): ...

cls = C
cls += 1

reveal_type(cls)  # revealed: str
```

Union and intersection handling could also be improved here, but I made
no attempt to do so in this PR.

## Test Plan

New MD tests
 2025-03-14 20:36:09 +01:00
David Peter fe275725e0
[red-knot] Document current state of attribute assignment diagnostics (#16746) 
## Summary

A follow-up to https://github.com/astral-sh/ruff/pull/16705 which
documents various kinds of diagnostics that can appear when assigning to
an attribute.

## Test Plan

New snapshot tests.
 2025-03-14 20:34:43 +01:00
Micha Reiser a467e7c8d3
[red-knot] Case sensitive module resolver (#16521) 
## Summary

This PR implements the first part of
https://github.com/astral-sh/ruff/discussions/16440. It ensures that Red
Knot's module resolver is case sensitive on all systems.

This PR combines a few approaches:

1. It uses `canonicalize` on non-case-sensitive systems to get the real
casing of a path. This works for as long as no symlinks or mapped
network drives (the windows `E:\` is mapped to `\\server\share` thingy).
This is the same as what Pyright does
2. If 1. fails, fall back to recursively list the parent directory and
test if the path's file name matches the casing exactly as listed in by
list dir. This is the same approach as CPython takes in its module
resolver. The main downside is that it requires more syscalls because,
unlike CPython, we Red Knot needs to invalidate its caches if a file
name gets renamed (CPython assumes that the folders are immutable).

It's worth noting that the file watching test that I added that renames
`lib.py` to `Lib.py` currently doesn't pass on case-insensitive systems.
Making it pass requires some more involved changes to `Files`. I plan to
work on this next. There's the argument that landing this PR on its own
isn't worth it without this issue being addressed. I think it's still a
good step in the right direction even when some of the details on how
and where the path case sensitive comparison is implemented.

## Test plan

I added multiple integration tests (including a failing one). I tested
that the `case-sensitivity` detection works as expected on Windows,
MacOS and Linux and that the fast-paths are taken accordingly.
 2025-03-14 19:16:44 +00:00
Micha Reiser a128ca761f
[red-knot] Very minor simplification of the render tests (#16759) 2025-03-14 19:13:07 +00:00
Brent Westbrook 3a32e56445
[syntax-errors] Unparenthesized assignment expressions in sets and indexes (#16404) 
## Summary
This PR detects unparenthesized assignment expressions used in set
literals and comprehensions and in sequence indexes. The link to the
release notes in https://github.com/astral-sh/ruff/issues/6591 just has
this entry:
> * Assignment expressions can now be used unparenthesized within set
literals and set comprehensions, as well as in sequence indexes (but not
slices).

with no other information, so hopefully the test cases I came up with
cover all of the changes. I also tested these out in the Python REPL and
they actually worked in Python 3.9 too. I'm guessing this may be another
case that was "formally made part of the language spec in Python 3.10,
but usable -- and commonly used -- in Python >=3.9" as @AlexWaygood
added to the body of #6591 for context managers. So we may want to
change the version cutoff, but I've gone along with the release notes
for now.

## Test Plan

New inline parser tests and linter CLI tests.
 2025-03-14 15:06:42 -04:00
Andrew Gallant b9d7c36a23 ruff_db: add a new diagnostic renderer 
We don't actually hook this up to anything in this PR, but we do
go to some trouble to granularly unit test it. The unit tests caught
plenty of bugs after I initially wrote down the implementation, so they
were very much worth it.

Closes #16506
 2025-03-14 14:59:33 -04:00
Andrew Gallant ef9a825827 ruff_db: add `context` configuration 
Instead of hard-coding a specific context window,
it seemed prudent to make this configurable. That
makes it easier to test different context window
sizes as well.

I am not totally convinced that this is the right
place for this configuration. I could see the context
window size being a property of `Diagnostic` instead,
since we might want to change the context window
size based not just on some end user configuration,
but perhaps also the specific diagnostic.

But for now, I think it's fine for it to live here,
and all of the rendering logic doesn't care where
it lives. So it should be relatively easy to change
in the future.
 2025-03-14 14:59:33 -04:00
Andrew Gallant 2bcd2b4147 red_knot: plumb through `DiagnosticFormat` to the CLI 
The CLI calls this `OutputFormat`, and so does the type where the CLI is
defined. But it's called `DiagnosticFormat` in `ruff_db` to be
consistent with `DisplayDiagnosticConfig`.

Ref https://github.com/astral-sh/ruff/issues/15697#issuecomment-2706477278
 2025-03-14 14:46:17 -04:00
Andrew Gallant eb6871d209 ruff_db: add concise diagnostic mode 
This adds a new configuration knob to diagnostic rendering that, when
enabled, will make diagnostic rendering much more terse. Specifically,
it will guarantee that each diagnostic will only use one line.

This doesn't actually hook the concise output option up to anything.
We'll do that plumbing in the next commit.
 2025-03-14 14:46:17 -04:00
Brent Westbrook 6311412373
[syntax-errors] Star annotations before Python 3.11 (#16545) 
Summary
--

This is closely related to (and stacked on)
https://github.com/astral-sh/ruff/pull/16544 and detects star
annotations in function definitions.

I initially called the variant `StarExpressionInAnnotation` to mirror
`StarExpressionInIndex`, but I realized it's not really a "star
expression" in this position and renamed it. `StarAnnotation` seems in
line with the PEP.

Test Plan
--

Two new inline tests. It looked like there was pretty good existing
coverage of this syntax, so I just added simple examples to test the
version cutoff.
 2025-03-14 15:20:44 +00:00
Brent Westbrook 4f2851982d
[syntax-errors] Star expression in index before Python 3.11 (#16544) 
Summary
--

This PR detects tuple unpacking expressions in index/subscript
expressions before Python 3.11.

Test Plan
--

New inline tests
 2025-03-14 14:51:34 +00:00
Micha Reiser 2cd25ef641
Ruff 0.11.0 (#16723) 
## Summary

Follow-up release for Ruff v0.10 that now includes the following two
changes that we intended to ship but slipped:

* Changes to how the Python version is inferred when a `target-version`
is not specified (#16319)
* `blanket-noqa` (`PGH004`): Also detect blanked file-level noqa
comments (and not just line level comments).

## Test plan

I verified that the binary built on this branch respects the
`requires-python` setting
([logs](https://www.diffchecker.com/qyJWYi6W/), left: v0.10, right:
v0.11)
 2025-03-14 13:57:56 +01:00
David Peter a22d206db2
[red-knot] Preliminary tests for typing.Final (#15917) 
## Summary

WIP.

Adds some preliminary tests for `typing.Final`.

## Test Plan

New MD tests
 2025-03-14 12:30:13 +01:00
cake-monotone 270318c2e0
[red-knot] fix: improve type inference for binary ops on tuples (#16725) 
## Summary

This PR includes minor improvements to binary operation inference,
specifically for tuple concatenation.

### Before

```py
reveal_type((1, 2) + (3, 4))  # revealed: @Todo(return type of decorated function)
# If TODO is ignored, the revealed type would be `tuple[1|2|3|4, ...]`
```

The `builtins.tuple` type stub defines `__add__`, but it appears to only
work for homogeneous tuples. However, I think this limitation is not
ideal for many use cases.

### After

```py
reveal_type((1, 2) + (3, 4))  # revealed: tuple[Literal[1], Literal[2], Literal[3], Literal[4]]
```

## Test Plan

### Added
- `mdtest/binary/tuples.md`

### Affected
- `mdtest/slots.md` (a test have been moved out of the `False-Negative`
block.)
 2025-03-14 12:29:57 +01:00
David Peter d03b12e711
[red-knot] Assignments to attributes (#16705) 
## Summary

This changeset adds proper support for assignments to attributes:
```py
obj.attr = value
```

In particular, the following new features are now available:

* We previously didn't raise any errors if you tried to assign to a
non-existing attribute `attr`. This is now fixed.
* If `type(obj).attr` is a data descriptor, we now call its `__set__`
method instead of trying to assign to the load-context type of
`obj.attr`, which can be different for data descriptors.
* An initial attempt was made to support unions and intersections, as
well as possibly-unbound situations. There are some remaining TODOs in
tests, but they only affect edge cases. Having nested diagnostics would
be one way that could help solve the remaining cases, I believe.

## Follow ups

The following things are planned as follow-ups:

- Write a test suite with snapshot diagnostics for various attribute
assignment errors
- Improve the diagnostics. An easy improvement would be to highlight the
right hand side of the assignment as a secondary span (with the rhs type
as additional information). Some other ideas are mentioned in TODO
comments in this PR.
- Improve the union/intersection/possible-unboundness handling
- Add support for calling custom `__setattr__` methods (see new false
positive in the ecosystem results)

## Ecosystem changes

Some changes are related to assignments on attributes with a custom
`__setattr__` method (see above). Since we didn't notice missing
attributes at all in store context previously, these are new.

The other changes are related to properties. We previously used their
read-context type to test the assignment. That results in weird error
messages, as we often see assignments to `self.property` and then we
think that those are instance attributes *and* descriptors, leading to
union types. Now we properly look them up on the meta type, see the
decorated function, and try to overwrite it with the new value (as we
don't understand decorators yet). Long story short: the errors are still
weird, we need to understand decorators to make them go away.

## Test Plan

New Markdown tests
 2025-03-14 12:15:41 +01:00
Micha Reiser 14c5ed5d7d
[`pygrep-hooks`]: Detect file-level suppressions comments without rul… (#16720) 
## Summary

I accidentially dropped this commit from the Ruff 0.10 release. See
https://github.com/astral-sh/ruff/pull/16699
 2025-03-14 09:37:16 +01:00
Micha Reiser 595565015b
Fallback to requires-python in certain cases when target-version is not found (#16721) 
## Summary

Restores https://github.com/astral-sh/ruff/pull/16319 after it got
dropped from the 0.10 release branch :(

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
 2025-03-14 09:36:51 +01:00
Brent Westbrook 2382fe1f25
[syntax-errors] Tuple unpacking in `for` statement iterator clause before Python 3.9 (#16558) 
Summary
--

This PR reuses a slightly modified version of the
`check_tuple_unpacking` method added for detecting unpacking in `return`
and `yield` statements to detect the same issue in the iterator clause
of `for` loops.

I ran into the same issue with a bare `for x in *rest: ...` example
(invalid even on Python 3.13) and added it as a comment on
https://github.com/astral-sh/ruff/issues/16520.

I considered just making this an additional `StarTupleKind` variant as
well, but this change was in a different version of Python, so I kept it
separate.

Test Plan
--

New inline tests.
 2025-03-13 15:55:17 -04:00
Micha Reiser 27e9d1fe3e
Ruff v0.10 Release (#16708) 
Co-authored-by: dylwil3 <dylwil3@gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
 2025-03-13 13:53:11 -04:00
Dylan b9b256209b
describe requires-python fallback in docs (#16704) 
Adds description of `requires-python` fallback to documentation for
configuration file discovery.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
 2025-03-13 11:48:12 -05:00
Carl Meyer abaa18993b
[red-knot] handle cycles in MRO/bases resolution (#16693) 
There can be semi-cyclic inheritance patterns (e.g. recursive generics)
that are not technically inheritance cycles, but that can cause us to
hit Salsa query cycles in evaluating a type's MRO. Add fixed-point
handling to these MRO-related queries so we don't panic on these cycles.

The details of what queries we hit in what order in this case will
change as we implement support for generics, but ultimately we will
probably need cycle handling for all queries that can re-enter type
inference, otherwise we are susceptible to small changes in query
execution order causing panics.

Fixes #14333
Further reduces the panicking set of seeds in #14737
 2025-03-13 08:16:03 -07:00
Shaygan Hooshyari 360ba095ff
[red-knot] Auto generate statement nodes (#16645) 
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Part of #15655 

Replaced statement nodes with autogenerated ones. Reused the stuff we
introduced in #16285. Nothing except for copying the nodes to new
format.

## Test Plan

Tests run without any changes. Also moved the test that checks size of
AST nodes to `generated.rs` since all of the structs that it tests are
now there.
<!-- How was it tested? -->
 2025-03-13 15:43:48 +01:00
Micha Reiser d8159e816f [`pylint`] Better inference for `str.strip` (`PLE310`) (#16671) 
## Summary
This PR stabilizes the behavior introduced in
https://github.com/astral-sh/ruff/pull/15985

The new behavior improves the inference of `str.strip` calls:

* before: The rule only considered calls on string or byte literals
(`"abcd".strip`)
* now: The rule also catches calls to `strip` on object where the type
is known to be a `str` or `bytes` (e.g. `a = "abc"; a.strip("//")`)


The new behavior shipped as part of Ruff 0.9.6 on the 10th of Feb which
is a little more than a month ago.
There have been now new issues or PRs related to the new behavior.
 2025-03-13 15:37:37 +01:00
Micha Reiser 04ad562afd [`pylint`] Improve `repeated-equality-comparison` fix to use a `set` when all elements are hashable (`PLR1714`) (#16685) 
## Summary

This PR promotes the fix improvements for `PLR1714` that were introduced
in https://github.com/astral-sh/ruff/pull/14372/ to stable.

The improvement is that the fix now proposes to use a set if all
elements are hashable:

```
foo == "bar" or foo == "baz" or foo == "qux"
```

Gets fixed to 

```py
foo in {"bar", "baz", "qux"}
```

where it previously always got fixed to a tuple.

The new fix was first released in ruff 0.8.0 (Nov last year). This is
not a breaking change. The change was preview gated only to get some
extra test coverage.


There are no open issues or PRs related to this changed fix behavior.
 2025-03-13 15:37:37 +01:00
Micha Reiser 91674718c4 [`pylint`/`pep8-naming`] Check `__new__` argument name in `bad-staticmethod-argument` and not `invalid-first-argument-name-for-class-method` (`PLW0211`/`N804`) (#16676) 
## Summary

This PR stabilizes the behavior changes introduced by
https://github.com/astral-sh/ruff/pull/13305 that were gated behind
preview.
The change is that `__new__` methods are now no longer flagged by
`invalid-first-argument-name-for-class-method` (`N804`) but instead by
`bad-staticmethod-argument` (`PLW0211`)

> __new__ methods are technically static methods, with cls as their
first argument. However, Ruff currently classifies them as classmethod,
which causes two issues:

## Test Plan

There have been no new issues or PRs related to `N804` or `PLW0211`
since the behavior change was released in Ruff 0.9.7 (about 3 weeks
ago).
This is a somewhat recent change but I don't think it's necessary to
leave this in preview for another 2 months. The main reason why it was
in preview
is that it is breaking, not because it is a risky change.
 2025-03-13 15:37:37 +01:00
Micha Reiser 348815d6d6 [`flake8-pyi`] Stabilize fix for `unused-private-type-var` (`PYI018`) (#16682) 
## Summary

This PR stabilizes the fix for `PYI018` introduced in
https://github.com/astral-sh/ruff/pull/15999/ (first released with Ruff
0.9.5 early February)

There are no known issues with the fix or open PRs.
 2025-03-13 15:37:37 +01:00
Micha Reiser 1326d55c29 [`flake8-bandit`] Deprecate `suspicious-xmle-tree-usage` (`S320`) (#16680) 
## Summary
Deprecate `S320` because defusedxml has deprecated there `lxml` module
and `lxml` has been hardened since.

flake8-bandit has removed their implementation as well
(https://github.com/PyCQA/bandit/pull/1212).

Addresses https://github.com/astral-sh/ruff/issues/13707


## Test Plan

I verified that selecting `S320` prints a warning and fails if the
preview mode is enabled.
 2025-03-13 15:37:37 +01:00
Micha Reiser c19cd58670 [`flake8-simplify`] Avoid double negation in fixes (`SIM103`) (#16684) 
## Summary

This PR stabilizes the fixes improvements made in
https://github.com/astral-sh/ruff/pull/15562 (released with ruff 0.9.3
in mid January).

There's no open issue or PR related to the changed fix behavior.

This is not a breaking change. The fix was only gated behind preview to
get some more test coverage before releasing.
 2025-03-13 15:37:37 +01:00
Micha Reiser 8155197549 [`pyupgrade`]: Improve diagnostic range for `redundant-open-mode` (`UP015`) (#16672) 
## Summary

This PR stabilizes the behavior change introduced in
https://github.com/astral-sh/ruff/pull/15872/

The diagnostic range is now the range of the redundant `mode` argument
where it previously was the range of the entire `open` call:

Before:

```
UP015.py:2:1: UP015 [*] Unnecessary mode argument
  |
1 | open("foo", "U")
2 | open("foo", "Ur")
  | ^^^^^^^^^^^^^^^^^ UP015
3 | open("foo", "Ub")
4 | open("foo", "rUb")
  |
  = help: Remove mode argument
```


Now:

```
UP015.py:2:13: UP015 [*] Unnecessary mode argument
  |
1 | open("foo", "U")
2 | open("foo", "Ur")
  |             ^^^^ UP015
3 | open("foo", "Ub")
4 | open("foo", "rUb")
  |
  = help: Remove mode argument
```

This is a breaking change because it may require moving a `noqa` comment
onto a different line, e.g if you have

```py
open(
    "foo",
    "Ur",
) # noqa: UP015
```

Needs to be rewritten to 

```py
open(
    "foo",
    "Ur", # noqa: UP015
)
```

There have been now new issues or PRs since the new preview behavior was
implemented. It first was released as part of Ruff 0.9.5 on the 5th of
Feb (a little more than a month ago)

## Test Plan

I reviewed the snapshot tests
 2025-03-13 15:37:37 +01:00
Micha Reiser 92193a3254 Consider all `TYPE_CHECKING` symbols for type-checking blocks (#16669) 
## Summary

This PR stabilizes the preview behavior introduced in
https://github.com/astral-sh/ruff/pull/15719 to recognize all symbols
named `TYPE_CHECKING` as type-checking
checks in `if TYPE_CHECKING` conditions. This ensures compatibility with
mypy and pyright.

This PR also stabilizes the new behavior that removes `if 0:` and `if
False` to be no longer considered type checking blocks.
Since then, this syntax has been removed from the typing spec and was
only used for Python modules that don't have a `typing` module
([comment](https://github.com/astral-sh/ruff/pull/15719#issuecomment-2612787793)).

The preview behavior was first released with Ruff 0.9.5 (6th of
February), which was about a month ago. There are no open issues or PRs
for the changed behavior


## Test Plan

The snapshots for `SIM108` change because `SIM108` ignored type checking
blocks but it can no
simplify `if 0` or `if False` blocks again because they're no longer
considered type checking blocks.

The changes in the `TC005` snapshot or only due to that `if 0` and `if
False` are no longer recognized as type checking blocks

<!-- How was it tested? -->
 2025-03-13 15:37:37 +01:00
Micha Reiser 3d2f2a2f8d [`pep8-naming`]: Ignore methods decorated with `@typing.override` (`invalid-argument-name`) (#16667) 
## Summary

This PR stabilizes the preview behavior for `invalid-argument-name`
(`N803`)
to ignore argument names of functions decorated with `typing.override`
because
these methods are *out of the authors* control. 

This behavior was introduced in
https://github.com/astral-sh/ruff/pull/15954
and released as part of Ruff 0.9.5 (6th of February). 

There have been no new issues or PRs since this behavior change
(preview) was introduced.
 2025-03-13 15:37:37 +01:00
Micha Reiser 0aded52c40 Stabilize FURB169 preview behavior (#16666) 
## Summary

This PR stabilizes the preview behavior introduced in
https://github.com/astral-sh/ruff/pull/15905

The behavior change is that the rule now also recognizes `type(expr) is
type(None)` comparisons where `expr` isn't a name expression.
For example, the rule now detects `type(a.b) is type(None)` and suggests
rewriting the comparison to `a.b is None`.

The new behavior was introduced with Ruff 0.9.5 (6th of February), about
a month ago. There are no open issues or PRs related to this rule (or
behavior change).
 2025-03-13 15:37:37 +01:00
Micha Reiser 7af5b98606 [`pylint`] Detect invalid default value type for `os.environ.get` (`PLW1508`) (#16674) 
## Summary
This PR stabilizes the new behavior introduced in
https://github.com/astral-sh/ruff/pull/14512 to also detect defalut
value arguemnts to `os.environ.get` that have an invalid type (not
`str`).
There's an upstream issue for this behavior change
https://github.com/pylint-dev/pylint/issues/10092 that was accepted and
a PR, but it hasn't been merged yet.

This behavior change was first shipped with Ruff 0.8.1 (Nov 22). 

There has only be one PR since the new behavior was introduced but it
was unrelated to the scope increase
(https://github.com/astral-sh/ruff/pull/14841).
 2025-03-13 15:37:37 +01:00
Micha Reiser b9ed3e3876 [`flake8-pytest-style`] Allow for loops with empty bodies (`PT012`, `PT031`) (#16678) 
## Summary

This PR stabilizes the behavior change introduced in
https://github.com/astral-sh/ruff/pull/15542 to allow
for statements with an empty body in `pytest.raises` and `pytest.warns`
with statements.

This raised an error before but is now allowed:

```py
with pytest.raises(KeyError, match='unknown'):
    async for _ in gpt.generate(gpt_request):
        pass
```

The same applies to 

```py
with pytest.raises(KeyError, match='unknown'):
    async for _ in gpt.generate(gpt_request):
        ...
```


There have been now new issues or PRs related to PT012 or PT031 since
this behavior change was introduced in ruff 0.9.3 (January 23rd).
 2025-03-13 15:37:37 +01:00
Micha Reiser e740286bbd [`pyupgrade`]: Deprecate `non-pep604-isinstance` (`UP038`) (#16681) 
## Summary

This PR deprecates UP038. Using PEP 604 syntax in `isinstance` and
`issubclass` calls isn't a recommended pattern (or community agreed best
practice)
and it negatively impacts performance. 

Resolves https://github.com/astral-sh/ruff/issues/7871

## Test Plan

I tested that selecting `UP038` results in a warning in no-preview mode
and an error in preview mode
 2025-03-13 15:37:37 +01:00
Brent Westbrook 776a401703 [`flake8-type-checking`] Stabilize `runtime-cast-value` (`TC006`) (#16637) 
Summary
--

Stabilizes TC006. The test was already in the right place.

Test Plan
--

No open issues or PRs. The last related [issue] was closed on
2025-02-09.

[issue]: https://github.com/astral-sh/ruff/issues/16037
 2025-03-13 15:37:37 +01:00
Brent Westbrook 5c3d555950 [`flake8-bandit`] Stabilize `unsafe-markup-use` (`S704`) (#16643) 
Summary
--

Stabilizes S704, which is also being recoded from RUF035 in 0.10.

Test Plan
--
Existing tests with `PreviewMode` removed from the settings.

There was one issue closed on 2024-12-20 calling the rule noisy and
asking for a config option, but the option was added and then there were
no more issues or PRs.
 2025-03-13 15:37:37 +01:00
Brent Westbrook ba37c7cdba [`flake8-datetimez`] Stabilize `datetime-min-max` (`DTZ901`) (#16635) 
Summary
--

Stabilizes DTZ901, renames the rule function to match the rule name,
removes the `preview_rules` test, and handles some nits in the docs
(mention `min` first to match the rule name too).

Test Plan
--

1 closed issue on 2024-11-12, 4 days after the rule was added. No issues
since
 2025-03-13 15:37:37 +01:00
InSync c605ce6fe2 Use inline snapshots in `# noqa` unit tests (#16687) 
## Summary

Follow-up to #16677.

This change converts all unit tests (69 of them) in `noqa.rs` to use
inline snapshots instead. It extends the file by more than 1000 lines,
but the tests are now much easier to read and reason about.

## Test Plan

`cargo insta test`.
 2025-03-13 15:37:37 +01:00
Brent Westbrook 16b0902e28 [`ruff`] Stabilize `unnecessary-nested-literal` (`RUF041`) (#16648) 
Summary
--

Stabilizes RUF041. The tests are already in the right place, and the
docs look good.

Test Plan
--

0 issues, 1 [PR] fixing nested literals and unions the day after the
rule was added. No changes since then

I wonder if the fix in that PR could be relevant for
https://github.com/astral-sh/ruff/pull/16639, where I noticed a
potential issue with `Union`. It could be unrelated, though.

[PR]: https://github.com/astral-sh/ruff/pull/14641
 2025-03-13 15:37:37 +01:00
Brent Westbrook 5bcc0c460b [`flake8-use-pathlib`] Stabilize `invalid-pathlib-with-suffix` (`PTH210`) (#16656) 
Summary
--

Stabilizes PTH210. Tests and docs looked good.

Test Plan
--

Mentioned in 1 open issue around Python 3.14 support (`"."` becomes a
valid suffix in 3.14). Otherwise no issues or PRs since 2024-12-12, 6
days after the rule was added.
 2025-03-13 15:37:37 +01:00
InSync 62e9c9e506 Add missing unit tests for `# noqa: A`-like cases (#16677) 
## Summary

Follow-up to #16659.

This change adds tests for these three cases, which are (also) not
covered by existing tests:

* `# noqa: A` (lone incomplete code)
* `# noqa: A123, B` (complete codes, last one incomplete)
* `# noqa: A123B` (squashed codes, last one incomplete)
 2025-03-13 15:37:37 +01:00
Brent Westbrook a6d4e5aae4 [`ruff`] Stabilize `if-key-in-dict-del` (`RUF051`) (#16658) 
Summary
--

Stabilizes RUF051. The tests and docs looked good.

Test Plan
--

1 closed documentation issue from 4 days after the rule was added and 1
typo fix from the same day it was added, but no other issues or PRs.
 2025-03-13 15:37:37 +01:00
Brent Westbrook 692b651776 [`flake8-bugbear`] Stabilize `batched-without-explicit-strict` (`B911`) (#16655) 
Summary
--

Stabilizes B911. Tests and docs looked good.

Test Plan
--

0 issues or PRs, open or closed
 2025-03-13 15:37:37 +01:00
Brent Westbrook 2f2d72dc95 [`flake8-logging`] Stabilize `root-logger-call` (`LOG015`) (#16654) 
Summary
--

Stabilizes LOG015. The tests and docs looked good.

Test Plan
--

1 closed documentation issue from 4 days after the rule was added, but
no other issues or PRs.
 2025-03-13 15:37:37 +01:00
Brent Westbrook d15641faea [`ruff`] Stabilize `map-int-version-parsing` (`RUF048`) (#16653) 
Summary
--

Stabilizes RUF048 and moves its test to the right place. The docs look
good.

Test Plan
--

0 closed or open issues. There was 1 [PR] related to an extension to the
rule, but it was closed without comment.

[PR]: https://github.com/astral-sh/ruff/pull/14701
 2025-03-13 15:37:37 +01:00
Brent Westbrook 9b1b3c1859 [`ruff`] Stabilize `unnecessary-cast-to-int` (`RUF046`) (#16649) 
Summary
--

Stabilizes RUF046 and moves its test to the right place. The docs look
good.

Test Plan
--

2 closed newline/whitespace issues from early January and 1 closed issue
about really being multiple rules, but otherwise no recent issues or
PRs.
 2025-03-13 15:37:37 +01:00
Brent Westbrook e4b46913d2 [`ruff`] Stabilize `invalid-assert-message-literal-argument` (`RUF040`) (#16646) 
Summary
--

Stabilizes RUF040 and fixes a very minor typo in the docs. The tests are
already in the right place.

Test Plan
--

0 issues or PRs
 2025-03-13 15:37:37 +01:00
Brent Westbrook f48bc3aea1 [`flake8-use-pathlib`] Stabilize `os-listdir` (`PTH208`) (#16642) 
Summary
--

Stabilizes PTH208. The test was already in the right place, and the docs
look good.

Test Plan
--

0 issues and PRs, open or closed
 2025-03-13 15:37:37 +01:00
Brent Westbrook 64b248ccfc [`flake8-type-checking`] Stabilize `unquoted-type-alias` (`TC007`) (#16638) 
Summary
--

Stabilizes TC007. The test was already in the right place.

Test Plan
--

No open issues or PRs.
 2025-03-13 15:37:37 +01:00
InSync c236be4320 Add missing unit tests for `# noqa:`-like cases (#16659) 2025-03-13 15:37:37 +01:00
Dylan 8bd140c99d Make noqa parsing consistent and more robust (#16483) 
# Summary
The goal of this PR is to address various issues around parsing
suppression comments by

1. Unifying the logic used to parse in-line (`# noqa`) and file-level
(`# ruff: noqa`) noqa comments
2. Recovering from certain errors and surfacing warnings in these cases

Closes #15682 
Supersedes #12811 
Addresses
https://github.com/astral-sh/ruff/pull/14229#discussion_r1835481018
Related: #14229 , #12809
 2025-03-13 15:37:37 +01:00
Brent Westbrook a04347b7a3 [`flake8-builtins`] Default to non-strict checking (`A005`) (#16125) 
## Summary

This PR changes the default value of
`lint.flake8-builtins.builtins-strict-checking` added in
https://github.com/astral-sh/ruff/pull/15951 from `true` to `false`.
This also allows simplifying the default option logic and removes the
dependence on preview mode.

https://github.com/astral-sh/ruff/issues/15399 was already closed by
#15951, but this change will finalize the behavior mentioned in
https://github.com/astral-sh/ruff/issues/15399#issuecomment-2587017147.

As an example, strict checking flags modules based on their last
component, so `utils/logging.py` triggers A005. Non-strict checking
checks the path to the module, so `utils/logging.py` is allowed (this is
the example and desired behavior from #15399 exactly) but a top-level
`logging.py` or `logging/__init__.py` is still disallowed.

## Test Plan

Existing tests from #15951 and #16006, with the snapshot updated in
`a005_module_shadowing_strict_default` to reflect the new default.
 2025-03-13 15:37:37 +01:00
Brent Westbrook 958e1177ce [`pyupgrade`] Stabilize `non-pep646-unpack` (`UP044`) (#16632) 
Summary
--

Stabilizes UP044, renames the module to match the rule name, and removes
the `PreviewMode` from the test settings.

Test Plan
--

2 closed issues in November, just after the rule was added, otherwise no
issues
 2025-03-13 15:37:37 +01:00
Brent Westbrook fce5d892c1 [`flake8-simplify`] Stabilize `split-static-string` (`SIM905`) (#16631) 
Summary
--

Stabilizes SIM905 and adds a small addition to the docs. The test was
already in the right place.

Test Plan
--

No issues except 2 recent, general issues about whitespace
normalization.
 2025-03-13 15:37:37 +01:00
Alex Waygood 66cae0a3ec [ruff-0.10] [`flake8-pyi`] Stabilize preview-mode behaviours for `custom-type-var-for-self`(`PYI019`) (#16607) 
## Summary

This PR stabilizes several preview-only behaviours for
`custom-typevar-for-self` (`PYI019`). Namely:
- A new, more accurate technique is now employed for detecting custom
TypeVars that are replaceable with `Self`. See
https://github.com/astral-sh/ruff/pull/15888 for details.
- The range of the diagnostic is now the full function header rather
than just the return annotation. (Previously, the rule only applied to
methods with return annotations, but this is no longer true due to the
changes in the first bullet point.)
- The fix is now available even when preview mode is not enabled.

## Test Plan

- Existing snapshots that do not have preview mode enabled are updated
- Preview-specific snapshots are removed
- I'll check the ecosystem report on this PR to verify everything's as
expected
 2025-03-13 15:37:37 +01:00
Brent Westbrook bbcddf7e79 [`pylint`] Stabilize `len-test` (`PLC1802`) (#16626) 
Summary
--

Stabilizes PLC1802. The tests were already in the right place, and I
just tidied the docs a little bit.

Test Plan
--

1 issue closed 4 days after the rule was added, no other issues
 2025-03-13 15:37:37 +01:00
Brent Westbrook c387a51cad [`pylint`] Stabilize `shallow-copy-environ` (`PLW1507`) (#16627) 
Summary
--

Stabilizes PLW1507. The tests were already in the right place, and I
just tidied the docs a little bit.

Test Plan
--

1 issue from 2 weeks ago but just suggesting to mark the fix unsafe. The
shallow vs deep copy *does* change the program behavior, just usually in
a preferable way.
 2025-03-13 15:37:37 +01:00
Brent Westbrook 5285e3fcbc [`FastAPI`] Stabilize `fast-api-unused-path-parameter` (`FAST003`) (#16625) 
## Summary

Stabilizes FAST003, completing the group with FAST001 and FAST002.

## Test Plan

Last bug fix (false positive) was fixed on 2025-01-13, almost 2 months
ago.

The test case was already in the right place.
 2025-03-13 15:37:37 +01:00
Brent Westbrook ed4152dec6 [`flake8-comprehensions`] Stabilize `unnecessary-dict-comprehension-for-iterable` (`C420`) (#16624) 
## Summary

Stabilizes C420 for the 0.10 release.

## Test Plan

No open issues or PRs (except a general issue about [string
normalization](https://github.com/astral-sh/ruff/issues/16579)). The
last (and only) false-negative bug fix was over a month ago.

The tests for this rule were already not on the `preview_rules` test, so
I just changed the `RuleGroup`. The documentation looked okay to me.
 2025-03-13 15:37:37 +01:00
InSync 24ec94562c [`flake8-builtins`] Remove `builtins-` prefix from option names (#16092) 
## Summary

Resolves #15368.

The following options have been renamed:

* `builtins-allowed-modules` &rarr; `allowed-modules`
* `builtins-ignorelist` &rarr; `ignorelist`
* `builtins-strict-checking` &rarr; `strict-checking`

To preserve compatibility, the old names are kept as Serde aliases.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-13 15:37:37 +01:00
David Salvisberg c0b1413ecd [`flake8-bandit`] Move `unsafe-markup-use` from `RUF035` to `S704` (#15957) 
## Summary

`RUF035` has been backported into bandit as `S704` in this
[PR](https://github.com/PyCQA/bandit/pull/1225)

This moves the rule and its corresponding setting to the `flake8-bandit`
category

## Test Plan

`cargo nextest run`

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
 2025-03-13 15:37:37 +01:00
Dhruv Manilawala 798fa47c2e Server: Remove log notification for `printDebugInformation` command (#16617) 
## Summary

For context, the initial implementation started out by sending a log
notification to the client to include this information in the client
channel. This is a bit ineffective because it doesn't allow the client
to display this information in a more obvious way. In addition to that,
it isn't obvious from a users perspective as to where the information is
being printed unless they actually open the output channel.

The change was to actually return this formatted string that contains
the information and let the client handle how it should display this
information. For example, in the Ruff VS Code extension we open a split
window and show this information which is similar to what rust-analyzer
does.

The notification request was kept as a precaution in case there are
users who are actually utilizing this way. If they exists, it should a
minority as it requires the user to actually dive into the code to
understand how to hook into this notification. With 0.10, we're removing
the old way as it only clobbers the output channel with a long message.

fixes: #16225

## Test Plan

Tested it out locally that the information is not being logged to the
output channel of VS Code.
 2025-03-13 15:37:37 +01:00
Micha Reiser a4b7c4ef70 [formatter] Stabilize fix for single-with-item formatting with trailing comment (#16603) 
## Summary

This PR stabilizies the fix for
https://github.com/astral-sh/ruff/issues/14001

We try to only make breaking formatting changes once a year. However,
the plan was to release this fix as part of Ruff 0.9 but I somehow
missed it when promoting all other formatter changes.
I think it's worth making an exception here considering that this is a
bug fix, it improves readability, and it should be rare
(very few files in a single project). Our version policy explicitly
allows breaking formatter changes in any minor release and the idea of
only making breaking formatter changes once a year is mainly to avoid
multiple releases throughout the year that introduce large formatter
changes

Closes https://github.com/astral-sh/ruff/issues/14001

## Test Plan

Updated snapshot
 2025-03-13 15:37:37 +01:00
Peter Hill a90cf9d59c
[ruff] Fix `last_tag`/`commits_since_last_tag` for `version` command (#16686) 
## Summary

Since Ruff changed to GitHub releases, tags are no longer annotated and
`git describe` no longer picks them up. Instead, it's necessary to also
search lightweight tags.

This changes fixes the `version` command to give more accurate
`last_tag`/`commits_since_last_tag` information. This only affects
development builds, as this information is not present in releases.

## Test Plan

Testing is a little tricky because this information changes on every
commit. Running manually on current `main` and my branch:

`main`:

```
# cargo run --bin ruff -- version --output-format=text
ruff 0.9.10+2547 (dd2313ab0 2025-03-12)

# cargo run --bin ruff -- version --output-format=json
{
  "version": "0.9.10",
  "commit_info": {
    "short_commit_hash": "dd2313ab0",
    "commit_hash": "dd2313ab0faea90abf66a75f1b5c388e728d9d0a",
    "commit_date": "2025-03-12",
    "last_tag": "v0.4.10",
    "commits_since_last_tag": 2547
  }
}
```

This PR:

```
# cargo run --bin ruff -- version --output-format=text
ruff 0.9.10+46 (11f39f616 2025-03-12)

# cargo run --bin ruff -- version --output-format=json
{
  "version": "0.9.10",
  "commit_info": {
    "short_commit_hash": "11f39f616",
    "commit_hash": "11f39f6166c3d7a521725b938a166659f64abb59",
    "commit_date": "2025-03-12",
    "last_tag": "0.9.10",
    "commits_since_last_tag": 46
  }
}
```
 2025-03-13 11:59:54 +00:00
Dhruv Manilawala 58d5fe982e
[red-knot] Check gradual equivalence between callable types (#16634) 2025-03-13 08:16:51 +05:30
Dhruv Manilawala 08fa9b4a90
[red-knot] Add `CallableTypeFromFunction` special form (#16683) 
## Summary

This PR adds a new `CallableTypeFromFunction` special form to allow
extracting the abstract signature of a function literal i.e., convert a
`Type::Function` into a `Type::Callable` (`CallableType::General`).

This is done to support testing the `is_gradual_equivalent_to` type
relation specifically the case we want to make sure that a function that
has parameters with no annotations and does not have a return type
annotation is gradual equivalent to `Callable[[Any, Any, ...], Any]`
where the number of parameters should match between the function literal
and callable type.

Refer
https://github.com/astral-sh/ruff/pull/16634#discussion_r1989976692

### Bikeshedding

The name `CallableTypeFromFunction` is a bit too verbose. A possibly
alternative from Carl is `CallableTypeOf` but that would be similar to
`TypeOf` albeit with a limitation that the former only accepts function
literal types and errors on other types.

Some other alternatives:
* `FunctionSignature`
* `SignatureOf` (similar issues as `TypeOf`?)
* ...

## Test Plan

Update `type_api.md` with a new section that tests this special form,
both invalid and valid forms.
 2025-03-13 07:49:34 +05:30
David Peter dd2313ab0f
[red-knot] Add mypy_primer usage documentation (#16679) 
## Summary

Add documentation on how to run mypy_primer locally.
 2025-03-12 16:47:10 +01:00
David Peter 083df0cf84
[red-knot] Support custom `__getattr__` methods (#16668) 
## Summary

Add support for calling custom `__getattr__` methods in case an
attribute is not otherwise found. This allows us to get rid of many
ecosystem false positives where we previously emitted errors when
accessing attributes on `argparse.Namespace`.

closes #16614

## Test Plan

* New Markdown tests
* Observed expected ecosystem changes (the changes for `arrow` also look
fine, since the `Arrow` class has a custom [`__getattr__`
here](1d70d00919/arrow/arrow.py (L802-L815)))
 2025-03-12 13:44:11 +01:00
Carl Meyer a176c1ac80
[red-knot] use fixpoint iteration for all cycles (#14029) 
Pulls in the latest Salsa main branch, which supports fixpoint
iteration, and uses it to handle all query cycles.

With this, we no longer need to skip any corpus files to avoid panics.

Latest perf results show a 6% incremental and 1% cold-check regression.
This is not a "no cycles" regression, as tomllib and typeshed do trigger
some definition cycles (previously handled by our old
`infer_definition_types` fallback to `Unknown`). We don't currently have
a benchmark we can use to measure the pure no-cycles regression, though
I expect there would still be some regression; the fixpoint iteration
feature in Salsa does add some overhead even for non-cyclic queries.

I think this regression is within the reasonable range for this feature.
We can do further optimization work later, but I don't think it's the
top priority right now. So going ahead and acknowledging the regression
on CodSpeed.

Mypy primer is happy, so this doesn't regress anything on our
currently-checked projects. I expect it probably unlocks adding a number
of new projects to our ecosystem check that previously would have
panicked.

Fixes #13792
Fixes #14672
 2025-03-12 12:41:40 +00:00
David Peter a6572a57c4
[red-knot] Attribute access on intersection types (#16665) 
## Summary

Implements attribute access on intersection types, which didn't
previously work. For example:

```py
from typing import Any

class P: ...
class Q: ...

class A:
    x: P = P()

class B:
    x: Any = Q()

def _(obj: A):
    if isinstance(obj, B):
        reveal_type(obj.x)  # revealed: P & Any
```

Refers to [this comment].

[this comment]:
https://github.com/astral-sh/ruff/pull/16416#discussion_r1985040363

## Test Plan

New Markdown tests
 2025-03-12 13:20:17 +01:00
Joey Bar b250304ad3
[red-knot] Improve is_disjoint for two intersections (#16636) 
## Summary

Background - as a follow up to #16611 I noticed that there's a lot of
code duplicated between the `is_assignable_to` and `is_subtype_of`
functions and considered trying to merge them.

[A subtype and an assignable type are pretty much the
same](https://typing.python.org/en/latest/spec/concepts.html#the-assignable-to-or-consistent-subtyping-relation),
except that subtypes are by definition fully static, so I think we can
replace the whole of `is_subtype_of` with:

```
if !self.is_fully_static(db) || !target.is_fully_static(db) {
    return false;
}
return self.is_assignable_to(target)
```

if we move all of the logic to is_assignable_to and delete duplicate
code. Then we can discuss if it even makes sense to have a separate
is_subtype_of function (I think the answer is yes since it's used by a
bunch of other places, but we may be able to basically rip out the
concept).

Anyways while playing with combining the functions I noticed is that the
handling of Intersections in `is_subtype_of` has a special case for two
intersections, which I didn't include in the last PR - rather I first
handled right hand intersections before left hand, which should properly
handle double intersections (hand-wavy explanation I can justify if
needed - (A & B & C) is assignable to (A & B) because the left is
assignable to both A and B, but none of A, B, or C is assignable to (A &
B)).

I took a look at what breaks if I remove the handling for double
intersections, and the reason it is needed is because is_disjoint does
not properly handle intersections with negative conditions (so instead
`is_subtype_of` basically implements the check correctly).

This PR adds support to is_disjoint for properly checking negative
branches, which also lets us simplify `is_subtype_of`, bringing it in
line with `is_assignable_to`

## Test Plan

Added a bunch of tests, most of which failed before this fix

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-12 12:13:04 +00:00
David Peter 11b5cbcd2f
[red-knot] Restructure attribute-access and descriptor-protocol test suites. (#16664) 
## Summary

This is a pure restructuring of the `attributes.md` and
`descriptor_protocol.md` test suites. They have grown organically and I
didn't want to make major structural changes in my recent PR to keep the
diff clean.
 2025-03-12 09:52:21 +01:00
David Peter 3228545598
[red-knot] Minor optimization/cleanup in member lookup (#16663) 
## Summary

A follow up to address [this comment]:

> Similarly here, it might be a little more performant to have a single
`Type::instance()` branch with an inner match over `class.known()`
rather than having multiple branches with `if class.is_known()` guards

[this comment]:
https://github.com/astral-sh/ruff/pull/16416#discussion_r1985159037
 2025-03-12 09:11:05 +01:00
David Peter 860b95a318
[red-knot] Binary operator inference for union types (#16601) 
## Summary

Properly handle binary operator inference for union types.

This fixes a bug I noticed while looking at ecosystem results. The MRE
version of it is this:

```py
def sub(x: float, y: float):
    # Red Knot: Operator `-` is unsupported between objects of type `int | float` and `int | float`
    return x - y
```

## Test Plan

- New Markdown tests.
- Expected diff in the ecosystem checks
 2025-03-12 08:21:54 +01:00
Dhruv Manilawala 6de2b2873b
[red-knot] Check if callable type is fully static (#16633) 
## Summary

Part of #15382 

This PR adds the check for whether a callable type is fully static or
not.

A callable type is fully static if all of the parameter types are fully
static _and_ the return type is fully static _and_ if it does not use
the gradual form (`...`) for its parameters.

## Test Plan

Update `is_fully_static.md` with callable types.

It seems that currently this test is grouped into either fully static or
not, I think it would be useful to split them up in groups like
callable, etc. I intentionally avoided that in this PR but I'll put up a
PR for an appropriate split.

Note: I've an explicit goal of updating the property tests with the new
callable types once all relations are implemented.
 2025-03-12 12:13:22 +05:30
Dhruv Manilawala 6b84253679
[red-knot] Callable member lookup, meta type impl (#16618) 
## Summary

This PR is a follow-up to https://github.com/astral-sh/ruff/pull/16493
that implements member lookup for the general callable type.

Based on the discussion around [member lookup
here](https://github.com/astral-sh/ruff/pull/16493#discussion_r1982041180)
and [`.to_meta_type()`
here](https://github.com/astral-sh/ruff/pull/16493#discussion_r1985104664).

## Test Plan

Add a new test cases.
 2025-03-12 12:01:38 +05:30
Carl Meyer 0340e23395
[red-knot] remove redundant sentence in test (#16660) 
Removes a redundant sentence I accidentally left in the test suite from
in #16540 (my mistake).
 2025-03-12 04:20:31 +00:00
Shunsuke Shibayama 78b5f0b165
[red-knot] detect invalid return type (#16540) 
## Summary

This PR closes #16248.

If the return type of the function isn't assignable to the one
specified, an `invalid-return-type` error occurs.
I thought it would be better to report this as a different kind of error
than the `invalid-assignment` error, so I defined this as a new error.

## Test Plan

All type inconsistencies in the test cases have been replaced with
appropriate ones.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-12 01:58:59 +00:00
Douglas Creager e17cd350b6
[red-knot] Support multiple overloads when binding parameters at call sites (#16568) 
This updates the `Signature` and `CallBinding` machinery to support
multiple overloads for a callable. This is currently only used for
`KnownFunction`s that we special-case in our type inference code. It
does **_not_** yet update the semantic index builder to handle
`@overload` decorators and construct a multi-signature `Overloads`
instance for real Python functions.

While I was here, I updated many of the `try_call` special cases to use
signatures (possibly overloaded ones now) and `bind_call` to check
parameter lists. We still need some of the mutator methods on
`OverloadBinding` for the special cases where we need to update return
types based on some Rust code.
 2025-03-11 15:08:17 -04:00
Alex Waygood c16237ddc0
[red-knot] Rework `Type::to_instance()` to return `Option<Type>` (#16428) 
## Summary

This PR fixes https://github.com/astral-sh/ruff/issues/16302.

The PR reworks `Type::to_instance()` to return `Option<Type>` rather
than `Type`. This reflects more accurately the fact that some variants
cannot be "turned into an instance", since they _already_ represent
instances of some kind. On `main`, we silently fallback to `Unknown` for
these variants, but this implicit behaviour can be somewhat surprising
and lead to unexpected bugs.

Returning `Option<Type>` rather than `Type` means that each callsite has
to account for the possibility that the type might already represent an
instance, and decide what to do about it.
In general, I think this increases the robustness of the code. Working
on this PR revealed two latent bugs in the code:
- One which has already been fixed by
https://github.com/astral-sh/ruff/pull/16427
- One which is fixed as part of https://github.com/astral-sh/ruff/pull/16608

I added special handling to `KnownClass::to_instance()`: If we fail to find one of these classes and the `test` feature is
_not_ enabled, we log a warning to the terminal saying that we failed to
find the class in typeshed and that we will be falling back to
`Type::Unknown`. A cache is maintained so that we record all classes
that we have already logged a warning for; we only log a warning for
failing to lookup a `KnownClass` if we know that it's the first time
we're looking it up.

## Test Plan

- All existing tests pass
- I ran the property tests via `QUICKCHECK_TESTS=1000000 cargo test
--release -p red_knot_python_semantic -- --ignored
types::property_tests::stable`

I also manually checked that warnings are appropriately printed to the
terminal when `KnownClass::to_instance()` falls back to `Unknown` and
the `test` feature is not enabled. To do this, I applied this diff to
the PR branch:

<details>
<summary>Patch deleting `int` and `str` from buitins</summary>

```diff
diff --git a/crates/red_knot_vendored/vendor/typeshed/stdlib/builtins.pyi b/crates/red_knot_vendored/vendor/typeshed/stdlib/builtins.pyi
index 0a6dc57b0..86636a05b 100644
--- a/crates/red_knot_vendored/vendor/typeshed/stdlib/builtins.pyi
+++ b/crates/red_knot_vendored/vendor/typeshed/stdlib/builtins.pyi
@@ -228,111 +228,6 @@ _PositiveInteger: TypeAlias = Literal[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,
 _NegativeInteger: TypeAlias = Literal[-1, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15, -16, -17, -18, -19, -20]
 _LiteralInteger = _PositiveInteger | _NegativeInteger | Literal[0]  # noqa: Y026  # TODO: Use TypeAlias once mypy bugs are fixed
 
-class int:
-    @overload
-    def __new__(cls, x: ConvertibleToInt = ..., /) -> Self: ...
-    @overload
-    def __new__(cls, x: str | bytes | bytearray, /, base: SupportsIndex) -> Self: ...
-    def as_integer_ratio(self) -> tuple[int, Literal[1]]: ...
-    @property
-    def real(self) -> int: ...
-    @property
-    def imag(self) -> Literal[0]: ...
-    @property
-    def numerator(self) -> int: ...
-    @property
-    def denominator(self) -> Literal[1]: ...
-    def conjugate(self) -> int: ...
-    def bit_length(self) -> int: ...
-    if sys.version_info >= (3, 10):
-        def bit_count(self) -> int: ...
-
-    if sys.version_info >= (3, 11):
-        def to_bytes(
-            self, length: SupportsIndex = 1, byteorder: Literal["little", "big"] = "big", *, signed: bool = False
-        ) -> bytes: ...
-        @classmethod
-        def from_bytes(
-            cls,
-            bytes: Iterable[SupportsIndex] | SupportsBytes | ReadableBuffer,
-            byteorder: Literal["little", "big"] = "big",
-            *,
-            signed: bool = False,
-        ) -> Self: ...
-    else:
-        def to_bytes(self, length: SupportsIndex, byteorder: Literal["little", "big"], *, signed: bool = False) -> bytes: ...
-        @classmethod
-        def from_bytes(
-            cls,
-            bytes: Iterable[SupportsIndex] | SupportsBytes | ReadableBuffer,
-            byteorder: Literal["little", "big"],
-            *,
-            signed: bool = False,
-        ) -> Self: ...
-
-    if sys.version_info >= (3, 12):
-        def is_integer(self) -> Literal[True]: ...
-
-    def __add__(self, value: int, /) -> int: ...
-    def __sub__(self, value: int, /) -> int: ...
-    def __mul__(self, value: int, /) -> int: ...
-    def __floordiv__(self, value: int, /) -> int: ...
-    def __truediv__(self, value: int, /) -> float: ...
-    def __mod__(self, value: int, /) -> int: ...
-    def __divmod__(self, value: int, /) -> tuple[int, int]: ...
-    def __radd__(self, value: int, /) -> int: ...
-    def __rsub__(self, value: int, /) -> int: ...
-    def __rmul__(self, value: int, /) -> int: ...
-    def __rfloordiv__(self, value: int, /) -> int: ...
-    def __rtruediv__(self, value: int, /) -> float: ...
-    def __rmod__(self, value: int, /) -> int: ...
-    def __rdivmod__(self, value: int, /) -> tuple[int, int]: ...
-    @overload
-    def __pow__(self, x: Literal[0], /) -> Literal[1]: ...
-    @overload
-    def __pow__(self, value: Literal[0], mod: None, /) -> Literal[1]: ...
-    @overload
-    def __pow__(self, value: _PositiveInteger, mod: None = None, /) -> int: ...
-    @overload
-    def __pow__(self, value: _NegativeInteger, mod: None = None, /) -> float: ...
-    # positive __value -> int; negative __value -> float
-    # return type must be Any as `int | float` causes too many false-positive errors
-    @overload
-    def __pow__(self, value: int, mod: None = None, /) -> Any: ...
-    @overload
-    def __pow__(self, value: int, mod: int, /) -> int: ...
-    def __rpow__(self, value: int, mod: int | None = None, /) -> Any: ...
-    def __and__(self, value: int, /) -> int: ...
-    def __or__(self, value: int, /) -> int: ...
-    def __xor__(self, value: int, /) -> int: ...
-    def __lshift__(self, value: int, /) -> int: ...
-    def __rshift__(self, value: int, /) -> int: ...
-    def __rand__(self, value: int, /) -> int: ...
-    def __ror__(self, value: int, /) -> int: ...
-    def __rxor__(self, value: int, /) -> int: ...
-    def __rlshift__(self, value: int, /) -> int: ...
-    def __rrshift__(self, value: int, /) -> int: ...
-    def __neg__(self) -> int: ...
-    def __pos__(self) -> int: ...
-    def __invert__(self) -> int: ...
-    def __trunc__(self) -> int: ...
-    def __ceil__(self) -> int: ...
-    def __floor__(self) -> int: ...
-    def __round__(self, ndigits: SupportsIndex = ..., /) -> int: ...
-    def __getnewargs__(self) -> tuple[int]: ...
-    def __eq__(self, value: object, /) -> bool: ...
-    def __ne__(self, value: object, /) -> bool: ...
-    def __lt__(self, value: int, /) -> bool: ...
-    def __le__(self, value: int, /) -> bool: ...
-    def __gt__(self, value: int, /) -> bool: ...
-    def __ge__(self, value: int, /) -> bool: ...
-    def __float__(self) -> float: ...
-    def __int__(self) -> int: ...
-    def __abs__(self) -> int: ...
-    def __hash__(self) -> int: ...
-    def __bool__(self) -> bool: ...
-    def __index__(self) -> int: ...
-
 class float:
     def __new__(cls, x: ConvertibleToFloat = ..., /) -> Self: ...
     def as_integer_ratio(self) -> tuple[int, int]: ...
@@ -437,190 +332,6 @@ class _FormatMapMapping(Protocol):
 class _TranslateTable(Protocol):
     def __getitem__(self, key: int, /) -> str | int | None: ...
 
-class str(Sequence[str]):
-    @overload
-    def __new__(cls, object: object = ...) -> Self: ...
-    @overload
-    def __new__(cls, object: ReadableBuffer, encoding: str = ..., errors: str = ...) -> Self: ...
-    @overload
-    def capitalize(self: LiteralString) -> LiteralString: ...
-    @overload
-    def capitalize(self) -> str: ...  # type: ignore[misc]
-    @overload
-    def casefold(self: LiteralString) -> LiteralString: ...
-    @overload
-    def casefold(self) -> str: ...  # type: ignore[misc]
-    @overload
-    def center(self: LiteralString, width: SupportsIndex, fillchar: LiteralString = " ", /) -> LiteralString: ...
-    @overload
-    def center(self, width: SupportsIndex, fillchar: str = " ", /) -> str: ...  # type: ignore[misc]
-    def count(self, sub: str, start: SupportsIndex | None = ..., end: SupportsIndex | None = ..., /) -> int: ...
-    def encode(self, encoding: str = "utf-8", errors: str = "strict") -> bytes: ...
-    def endswith(
-        self, suffix: str | tuple[str, ...], start: SupportsIndex | None = ..., end: SupportsIndex | None = ..., /
-    ) -> bool: ...
-    @overload
-    def expandtabs(self: LiteralString, tabsize: SupportsIndex = 8) -> LiteralString: ...
-    @overload
-    def expandtabs(self, tabsize: SupportsIndex = 8) -> str: ...  # type: ignore[misc]
-    def find(self, sub: str, start: SupportsIndex | None = ..., end: SupportsIndex | None = ..., /) -> int: ...
-    @overload
-    def format(self: LiteralString, *args: LiteralString, **kwargs: LiteralString) -> LiteralString: ...
-    @overload
-    def format(self, *args: object, **kwargs: object) -> str: ...
-    def format_map(self, mapping: _FormatMapMapping, /) -> str: ...
-    def index(self, sub: str, start: SupportsIndex | None = ..., end: SupportsIndex | None = ..., /) -> int: ...
-    def isalnum(self) -> bool: ...
-    def isalpha(self) -> bool: ...
-    def isascii(self) -> bool: ...
-    def isdecimal(self) -> bool: ...
-    def isdigit(self) -> bool: ...
-    def isidentifier(self) -> bool: ...
-    def islower(self) -> bool: ...
-    def isnumeric(self) -> bool: ...
-    def isprintable(self) -> bool: ...
-    def isspace(self) -> bool: ...
-    def istitle(self) -> bool: ...
-    def isupper(self) -> bool: ...
-    @overload
-    def join(self: LiteralString, iterable: Iterable[LiteralString], /) -> LiteralString: ...
-    @overload
-    def join(self, iterable: Iterable[str], /) -> str: ...  # type: ignore[misc]
-    @overload
-    def ljust(self: LiteralString, width: SupportsIndex, fillchar: LiteralString = " ", /) -> LiteralString: ...
-    @overload
-    def ljust(self, width: SupportsIndex, fillchar: str = " ", /) -> str: ...  # type: ignore[misc]
-    @overload
-    def lower(self: LiteralString) -> LiteralString: ...
-    @overload
-    def lower(self) -> str: ...  # type: ignore[misc]
-    @overload
-    def lstrip(self: LiteralString, chars: LiteralString | None = None, /) -> LiteralString: ...
-    @overload
-    def lstrip(self, chars: str | None = None, /) -> str: ...  # type: ignore[misc]
-    @overload
-    def partition(self: LiteralString, sep: LiteralString, /) -> tuple[LiteralString, LiteralString, LiteralString]: ...
-    @overload
-    def partition(self, sep: str, /) -> tuple[str, str, str]: ...  # type: ignore[misc]
-    if sys.version_info >= (3, 13):
-        @overload
-        def replace(
-            self: LiteralString, old: LiteralString, new: LiteralString, /, count: SupportsIndex = -1
-        ) -> LiteralString: ...
-        @overload
-        def replace(self, old: str, new: str, /, count: SupportsIndex = -1) -> str: ...  # type: ignore[misc]
-    else:
-        @overload
-        def replace(
-            self: LiteralString, old: LiteralString, new: LiteralString, count: SupportsIndex = -1, /
-        ) -> LiteralString: ...
-        @overload
-        def replace(self, old: str, new: str, count: SupportsIndex = -1, /) -> str: ...  # type: ignore[misc]
-    if sys.version_info >= (3, 9):
-        @overload
-        def removeprefix(self: LiteralString, prefix: LiteralString, /) -> LiteralString: ...
-        @overload
-        def removeprefix(self, prefix: str, /) -> str: ...  # type: ignore[misc]
-        @overload
-        def removesuffix(self: LiteralString, suffix: LiteralString, /) -> LiteralString: ...
-        @overload
-        def removesuffix(self, suffix: str, /) -> str: ...  # type: ignore[misc]
-
-    def rfind(self, sub: str, start: SupportsIndex | None = ..., end: SupportsIndex | None = ..., /) -> int: ...
-    def rindex(self, sub: str, start: SupportsIndex | None = ..., end: SupportsIndex | None = ..., /) -> int: ...
-    @overload
-    def rjust(self: LiteralString, width: SupportsIndex, fillchar: LiteralString = " ", /) -> LiteralString: ...
-    @overload
-    def rjust(self, width: SupportsIndex, fillchar: str = " ", /) -> str: ...  # type: ignore[misc]
-    @overload
-    def rpartition(self: LiteralString, sep: LiteralString, /) -> tuple[LiteralString, LiteralString, LiteralString]: ...
-    @overload
-    def rpartition(self, sep: str, /) -> tuple[str, str, str]: ...  # type: ignore[misc]
-    @overload
-    def rsplit(self: LiteralString, sep: LiteralString | None = None, maxsplit: SupportsIndex = -1) -> list[LiteralString]: ...
-    @overload
-    def rsplit(self, sep: str | None = None, maxsplit: SupportsIndex = -1) -> list[str]: ...  # type: ignore[misc]
-    @overload
-    def rstrip(self: LiteralString, chars: LiteralString | None = None, /) -> LiteralString: ...
-    @overload
-    def rstrip(self, chars: str | None = None, /) -> str: ...  # type: ignore[misc]
-    @overload
-    def split(self: LiteralString, sep: LiteralString | None = None, maxsplit: SupportsIndex = -1) -> list[LiteralString]: ...
-    @overload
-    def split(self, sep: str | None = None, maxsplit: SupportsIndex = -1) -> list[str]: ...  # type: ignore[misc]
-    @overload
-    def splitlines(self: LiteralString, keepends: bool = False) -> list[LiteralString]: ...
-    @overload
-    def splitlines(self, keepends: bool = False) -> list[str]: ...  # type: ignore[misc]
-    def startswith(
-        self, prefix: str | tuple[str, ...], start: SupportsIndex | None = ..., end: SupportsIndex | None = ..., /
-    ) -> bool: ...
-    @overload
-    def strip(self: LiteralString, chars: LiteralString | None = None, /) -> LiteralString: ...
-    @overload
-    def strip(self, chars: str | None = None, /) -> str: ...  # type: ignore[misc]
-    @overload
-    def swapcase(self: LiteralString) -> LiteralString: ...
-    @overload
-    def swapcase(self) -> str: ...  # type: ignore[misc]
-    @overload
-    def title(self: LiteralString) -> LiteralString: ...
-    @overload
-    def title(self) -> str: ...  # type: ignore[misc]
-    def translate(self, table: _TranslateTable, /) -> str: ...
-    @overload
-    def upper(self: LiteralString) -> LiteralString: ...
-    @overload
-    def upper(self) -> str: ...  # type: ignore[misc]
-    @overload
-    def zfill(self: LiteralString, width: SupportsIndex, /) -> LiteralString: ...
-    @overload
-    def zfill(self, width: SupportsIndex, /) -> str: ...  # type: ignore[misc]
-    @staticmethod
-    @overload
-    def maketrans(x: dict[int, _T] | dict[str, _T] | dict[str | int, _T], /) -> dict[int, _T]: ...
-    @staticmethod
-    @overload
-    def maketrans(x: str, y: str, /) -> dict[int, int]: ...
-    @staticmethod
-    @overload
-    def maketrans(x: str, y: str, z: str, /) -> dict[int, int | None]: ...
-    @overload
-    def __add__(self: LiteralString, value: LiteralString, /) -> LiteralString: ...
-    @overload
-    def __add__(self, value: str, /) -> str: ...  # type: ignore[misc]
-    # Incompatible with Sequence.__contains__
-    def __contains__(self, key: str, /) -> bool: ...  # type: ignore[override]
-    def __eq__(self, value: object, /) -> bool: ...
-    def __ge__(self, value: str, /) -> bool: ...
-    @overload
-    def __getitem__(self: LiteralString, key: SupportsIndex | slice, /) -> LiteralString: ...
-    @overload
-    def __getitem__(self, key: SupportsIndex | slice, /) -> str: ...  # type: ignore[misc]
-    def __gt__(self, value: str, /) -> bool: ...
-    def __hash__(self) -> int: ...
-    @overload
-    def __iter__(self: LiteralString) -> Iterator[LiteralString]: ...
-    @overload
-    def __iter__(self) -> Iterator[str]: ...  # type: ignore[misc]
-    def __le__(self, value: str, /) -> bool: ...
-    def __len__(self) -> int: ...
-    def __lt__(self, value: str, /) -> bool: ...
-    @overload
-    def __mod__(self: LiteralString, value: LiteralString | tuple[LiteralString, ...], /) -> LiteralString: ...
-    @overload
-    def __mod__(self, value: Any, /) -> str: ...
-    @overload
-    def __mul__(self: LiteralString, value: SupportsIndex, /) -> LiteralString: ...
-    @overload
-    def __mul__(self, value: SupportsIndex, /) -> str: ...  # type: ignore[misc]
-    def __ne__(self, value: object, /) -> bool: ...
-    @overload
-    def __rmul__(self: LiteralString, value: SupportsIndex, /) -> LiteralString: ...
-    @overload
-    def __rmul__(self, value: SupportsIndex, /) -> str: ...  # type: ignore[misc]
-    def __getnewargs__(self) -> tuple[str]: ...
-
 class bytes(Sequence[int]):
```

</details>

And then ran red-knot on my
[typeshed-stats](https://github.com/AlexWaygood/typeshed-stats) project
using the command

```
cargo run -p red_knot -- check --project ../typeshed-stats --python-version="3.12" --verbose
```

I observed that the following logs were printed to the terminal, but
that each warning was only printed once (the desired behaviour):

```
INFO Python version: Python 3.12, platform: all
INFO Indexed 15 file(s)
INFO Could not find class `builtins.int` in typeshed on Python 3.12. Falling back to `Unknown` for the symbol instead.
INFO Could not find class `builtins.str` in typeshed on Python 3.12. Falling back to `Unknown` for the symbol instead.
```
 2025-03-11 16:42:44 +00:00
Alex Waygood 989075dc16
[red-knot] Add tests asserting that `KnownClass::to_instance()` doesn't unexpectedly fallback to `Type::Unknown` with full typeshed stubs (#16608) 
## Summary

One of the motivations in https://github.com/astral-sh/ruff/pull/16428
for panicking when the `test` or `debug_assertions` features are enabled
and a lookup of a `KnownClass` fails is that we've had some latent bugs
in our code where certain variants have been silently falling back to
`Unknown` in every typeshed lookup without us realising. But that in
itself isn't a great motivation for panicking in
`KnownClass::to_instance()`, since we can fairly easily add some tests
that assert that we don't unexpectedly fallback to `Unknown` for any
`KnownClass` variant. This PR adds those tests.

## Test Plan

`cargo test -p red_knot_python_semantic`
 2025-03-11 16:12:44 +00:00
Joey Bar e8e24310fb
[red-knot] Handle gradual intersection types in assignability (#16611) 
## Summary

This mostly fixes #14899

My motivation was similar to the last comment by @sharkdp there. I ran
red_knot on a codebase and the most common error was patterns like this
failing:

```
def foo(x: str): ...

x: Any = ...
if isinstance(x, str):
    foo(x) # Object of type `Any & str` cannot be assigned to parameter 1 (`x`) of function `foo`; expected type `str`
```

The desired behavior is pretty much to ignore Any/Unknown when resolving
intersection assignability - `Any & str` should be assignable to `str`,
and `str` should be assignable to `str & Any`
 
The fix is actually very similar to the existing code in
`is_subtype_of`, we need to correctly handle intersections on either
side, while being careful to handle dynamic types as desired.

This does not fix the second test case from that issue:

```
static_assert(is_assignable_to(Intersection[Unrelated, Any], Not[tuple[Unrelated, Any]]))
```

but that's misleading because the root cause there has nothing to do
with gradual types. I added a simpler test case that also fails:

```
static_assert(is_assignable_to(Unrelated, Not[tuple[Unrelated]]))
```
This is because we don't determine that Unrelated does not subclass from
tuple so we can't rule out this relation. If that logic is improved then
this fix should also handle the case of the intersection

## Test Plan

Added a bunch of is_assignable_to tests, most of which failed before
this fix.
 2025-03-11 07:58:56 -07:00
Dhruv Manilawala da069aa00c
[red-knot] Infer `lambda` expression (#16547) 
## Summary

Part of https://github.com/astral-sh/ruff/issues/15382

This PR adds support for inferring the `lambda` expression and return
the `CallableType`.

Currently, this is only limited to inferring the parameters and a todo
type for the return type.

For posterity, I tried using the `file_expression_type` to infer the
return type of lambda but it would always lead to cycle. The main reason
is that in `infer_parameter_definition`, the default expression is being
inferred using `file_expression_type`, which is correct, but it then

Take the following source code as an example:
```py
lambda x=1: x
```

Here's how the code will flow:
* `infer_scope_types` for the global scope
* `infer_lambda_expression`
* `infer_expression` for the default value `1`
* `file_expression_type` for the return type using the body expression.
This is because the body creates it's own scope
* `infer_scope_types` (lambda body scope)
* `infer_name_load` for the symbol `x` whose visible binding is the
lambda parameter `x`
* `infer_parameter_definition` for parameter `x`
* `file_expression_type` for the default value `1`
* `infer_scope_types` for the global scope because of the default
expression

This will then reach to `infer_definition` for the parameter `x` again
which then creates the cycle.

## Test Plan

Add tests around `lambda` expression inference.
 2025-03-11 11:25:20 +05:30
David Peter c60e8a037a
[red-knot] Add support for calling `type[…]` (#16597) 
## Summary

This fixes the non-diagnostics part of #15948.

## Test Plan

New Markdown tests.

Negative diff on the ecosystem checks:

```diff
zipp (https://github.com/jaraco/zipp)
- error: lint:call-non-callable
-    --> /tmp/mypy_primer/projects/zipp/zipp/__init__.py:393:16
-     |
- 392 |     def _next(self, at):
- 393 |         return self.__class__(self.root, at)
-     |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Object of type `type[Unknown]` is not callable
- 394 |
- 395 |     def is_dir(self):
-     |
- 
- Found 9 diagnostics
+ Found 8 diagnostics

arrow (https://github.com/arrow-py/arrow)
+     |
+     |
+ warning: lint:unused-ignore-comment
+    --> /tmp/mypy_primer/projects/arrow/arrow/arrow.py:576:66
+ 574 |                 values.append(1)
+ 575 |
+ 576 |             floor = self.__class__(*values, tzinfo=self.tzinfo)  # type: ignore[misc]
+     |                                                                  -------------------- Unused blanket `type: ignore` directive
+ 577 |
+ 578 |             if frame_absolute == "week":
- error: lint:call-non-callable
-     --> /tmp/mypy_primer/projects/arrow/arrow/arrow.py:1080:16
-      |
- 1078 |           dt = self._datetime.astimezone(tz)
- 1079 |
- 1080 |           return self.__class__(
-      |  ________________^
- 1081 | |             dt.year,
- 1082 | |             dt.month,
- 1083 | |             dt.day,
- 1084 | |             dt.hour,
- 1085 | |             dt.minute,
- 1086 | |             dt.second,
- 1087 | |             dt.microsecond,
- 1088 | |             dt.tzinfo,
- 1089 | |             fold=getattr(dt, "fold", 0),
- 1090 | |         )
-      | |_________^ Object of type `type[Unknown]` is not callable
- 1091 |
- 1092 |       # string output and formatting
-      |

black (https://github.com/psf/black)
- 
-     |
-     |
- error: lint:call-non-callable
-    --> /tmp/mypy_primer/projects/black/src/blib2to3/pgen2/grammar.py:135:15
- 133 |         Copy the grammar.
- 134 |         """
- 135 |         new = self.__class__()
-     |               ^^^^^^^^^^^^^^^^ Object of type `type[@Todo]` is not callable
- 136 |         for dict_attr in (
- 137 |             "symbol2number",
- Found 328 diagnostics
+ Found 327 diagnostics
```
 2025-03-10 13:24:13 +01:00
David Peter ca974706dd
[red-knot] Do not ignore typeshed stubs for 'venv' module (#16596) 
## Summary

We currently fail to add the stubs for the `venv` stdlib module because
there is a `venv/` ignore pattern in the top-level `.gitignore` file.

## Test Plan

Ran the typeshed sync workflow manually once to see if the `venv/`
folder is now correctly added.
 2025-03-10 09:07:48 +01:00
Alex Waygood b6c7ba4f8e
[red-knot] Reduce Salsa lookups in `Type::find_name_in_mro` (#16582) 
## Summary

Theoretically this should be slightly more performant, since the
`class.is_known()` calls each do a separate Salsa lookup, which we can
avoid if we do a single `match` on the value of `class.known()`. It also
ends up being two lines less code overall!

## Test Plan

`cargo test -p red_knot_python_semantic`
 2025-03-10 07:55:22 +01:00
Alex Waygood c970b794d0
Fix broken red-knot property tests (#16574) 
## Summary

Fixes #16566, fixes #16575

The semantics of `Type::class_member` changed in
https://github.com/astral-sh/ruff/pull/16416, but the property-test
infrastructure was not updated. That means that the property tests were
panicking on the second `expect_type` call here:


0361021863/crates/red_knot_python_semantic/src/types/property_tests.rs (L151-L158)

With the somewhat unhelpful message:

```
Expected a (possibly unbound) type, not an unbound symbol
```

Applying this patch, and then running `QUICKCHECK_TESTS=1000000 cargo
test --release -p red_knot_python_semantic -- --ignored
types::property_tests::stable::equivalent_to_is_reflexive` showed
clearly that it was no longer able to find _any_ methods on _any_
classes due to the change in semantics of `Type::class_member`:

```diff
--- a/crates/red_knot_python_semantic/src/types/property_tests.rs
+++ b/crates/red_knot_python_semantic/src/types/property_tests.rs
@@ -27,7 +27,7 @@
 use std::sync::{Arc, Mutex, MutexGuard, OnceLock};
 
 use crate::db::tests::{setup_db, TestDb};
-use crate::symbol::{builtins_symbol, known_module_symbol};
+use crate::symbol::{builtins_symbol, known_module_symbol, Symbol};
 use crate::types::{
     BoundMethodType, CallableType, IntersectionBuilder, KnownClass, KnownInstanceType,
     SubclassOfType, TupleType, Type, UnionType,
@@ -150,10 +150,11 @@ impl Ty {
             Ty::BuiltinsFunction(name) => builtins_symbol(db, name).symbol.expect_type(),
             Ty::BuiltinsBoundMethod { class, method } => {
                 let builtins_class = builtins_symbol(db, class).symbol.expect_type();
-                let function = builtins_class
-                    .class_member(db, method.into())
-                    .symbol
-                    .expect_type();
+                let Symbol::Type(function, ..) =
+                    builtins_class.class_member(db, method.into()).symbol
+                else {
+                    panic!("no method `{method}` on class `{class}`");
+                };
 
                 create_bound_method(db, function, builtins_class)
             }
```

This PR updates the property-test infrastructure to use `Type::member`
rather than `Type::class_member`.

## Test Plan

- Ran `QUICKCHECK_TESTS=1000000 cargo test --release -p
red_knot_python_semantic -- --ignored types::property_tests::stable`
successfully
- Checked that there were no remaining uses of `Type::class_member` in
`property_tests.rs`
 2025-03-09 17:40:08 +00:00
Alex Waygood 335b264fe2
[red-knot] Consistent spelling of "metaclass" and "meta-type" (#16576) 
## Summary

Fixes a small nit of mine -- we are currently inconsistent in our
spelling between "metaclass" and "meta class", and between "meta type"
and "meta-type". This PR means that we consistently use "metaclass" and
"meta-type".

## Test Plan

`uvx pre-commit run -a`
 2025-03-09 12:30:32 +00:00
Dhruv Manilawala 0361021863
[red-knot] Understand `typing.Callable` (#16493) 
## Summary

Part of https://github.com/astral-sh/ruff/issues/15382

This PR implements a general callable type that wraps around a
`Signature` and it uses that new type to represent `typing.Callable`.

It also implements `Display` support for `Callable`. The format is as:
```
([<arg name>][: <arg type>][ = <default type>], ...) -> <return type>
```

The `/` and `*` separators are added at the correct boundary for
positional-only and keyword-only parameters. Now, as `typing.Callable`
only has positional-only parameters, the rendered signature would be:

```py
Callable[[int, str], None]
# (int, str, /) -> None
```

The `/` separator represents that all the arguments are positional-only.

The relationship methods that check assignability, subtype relationship,
etc. are not yet implemented and will be done so as a follow-up.

## Test Plan

Add test cases for display support for `Signature` and various mdtest
for `typing.Callable`.
 2025-03-08 03:58:52 +00:00
Eric Mark Martin 24c8b1242e
[red-knot] Support unpacking `with` target (#16469) 
## Summary

Resolves #16365

Add support for unpacking `with` statement targets.

## Test Plan

Added some test cases, alike the ones added by #15058.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
 2025-03-08 02:36:35 +00:00
David Peter 820a31af5d
[red-knot] Attribute access and the descriptor protocol (#16416) 
## Summary

* Attributes/method are now properly looked up on metaclasses, when
called on class objects
* We properly distinguish between data descriptors and non-data
descriptors (but we do not yet support them in store-context, i.e.
`obj.data_descr = …`)
* The descriptor protocol is now implemented in a single unified place
for instances, classes and dunder-calls. Unions and possibly-unbound
symbols are supported in all possible stages of the process by creating
union types as results.
* In general, the handling of "possibly-unbound" symbols has been
improved in a lot of places: meta-class attributes, attributes,
descriptors with possibly-unbound `__get__` methods, instance
attributes, …
* We keep track of type qualifiers in a lot more places. I anticipate
that this will be useful if we import e.g. `Final` symbols from other
modules (see relevant change to typing spec:
https://github.com/python/typing/pull/1937).
* Detection and special-casing of the `typing.Protocol` special form in
order to avoid lots of changes in the test suite due to new `@Todo`
types when looking up attributes on builtin types which have `Protocol`
in their MRO. We previously
looked up attributes in a wrong way, which is why this didn't come up
before.

closes #16367
closes #15966

## Context

The way attribute lookup in `Type::member` worked before was simply
wrong (mostly my own fault). The whole instance-attribute lookup should
probably never have been integrated into `Type::member`. And the
`Type::static_member` function that I introduced in my last descriptor
PR was the wrong abstraction. It's kind of fascinating how far this
approach took us, but I am pretty confident that the new approach
proposed here is what we need to model this correctly.

There are three key pieces that are required to implement attribute
lookups:

- **`Type::class_member`**/**`Type::find_in_mro`**: The
`Type::find_in_mro` method that can look up attributes on class bodies
(and corresponding bases). This is a partial function on types, as it
can not be called on instance types like`Type::Instance(…)` or
`Type::IntLiteral(…)`. For this reason, we usually call it through
`Type::class_member`, which is essentially just
`type.to_meta_type().find_in_mro(…)` plus union/intersection handling.
- **`Type::instance_member`**: This new function is basically the
type-level equivalent to `obj.__dict__[name]` when called on
`Type::Instance(…)`. We use this to discover instance attributes such as
those that we see as declarations on class bodies or as (annotated)
assignments to `self.attr` in methods of a class.
- The implementation of the descriptor protocol. It works slightly
different for instances and for class objects, but it can be described
by the general framework:
- Call `type.class_member("attribute")` to look up "attribute" in the
MRO of the meta type of `type`. Call the resulting `Symbol` `meta_attr`
(even if it's unbound).
- Use `meta_attr.class_member("__get__")` to look up `__get__` on the
*meta type* of `meta_attr`. Call it with `__get__(meta_attr, self,
self.to_meta_type())`. If this fails (either the lookup or the call),
just proceed with `meta_attr`. Otherwise, replace `meta_attr` in the
following with the return type of `__get__`. In this step, we also probe
if a `__set__` or `__delete__` method exists and store it in
`meta_attr_kind` (can be either "data descriptor" or "normal attribute
or non-data descriptor").
  - Compute a `fallback` type.
    - For instances, we use `self.instance_member("attribute")`
- For class objects, we use `class_attr =
self.find_in_mro("attribute")`, and then try to invoke the descriptor
protocol on `class_attr`, i.e. we look up `__get__` on the meta type of
`class_attr` and call it with `__get__(class_attr, None, self)`. This
additional invocation of the descriptor protocol on the fallback type is
one major asymmetry in the otherwise universal descriptor protocol
implementation.
- Finally, we look at `meta_attr`, `meta_attr_kind` and `fallback`, and
handle various cases of (possible) unboundness of these symbols.
- If `meta_attr` is bound and a data descriptor, just return `meta_attr`
- If `meta_attr` is not a data descriptor, and `fallback` is bound, just
return `fallback`
- If `meta_attr` is not a data descriptor, and `fallback` is unbound,
return `meta_attr`
- Return unions of these three possibilities for partially-bound
symbols.

This allows us to handle class objects and instances within the same
framework. There is a minor additional detail where for instances, we do
not allow the fallback type (the instance attribute) to completely
shadow the non-data descriptor. We do this because we (currently) don't
want to pretend that we can statically infer that an instance attribute
is always set.

Dunder method calls can also be embedded into this framework. The only
thing that changes is that *there is no fallback type*. If a dunder
method is called on an instance, we do not fall back to instance
variables. If a dunder method is called on a class object, we only look
it up on the meta class, never on the class itself.

## Test Plan

New Markdown tests.
 2025-03-07 22:03:28 +01:00
InSync a18d8bfa7d
[`pep8-naming`] Add links to `ignore-names` options in various rules' documentation (#16557) 
## Summary

Resolves #16551.

All rules using
[`lint.pep8-naming.ignore-names`](https://docs.astral.sh/ruff/settings/#lint_pep8-naming_ignore-names)
and
[`lint.pep8-naming.extend-ignore-names`](https://docs.astral.sh/ruff/settings/#lint_pep8-naming_extend-ignore-names)
now have their documentation linked to these two options.

## Test Plan

None.
 2025-03-07 14:49:08 -05:00
Shunsuke Shibayama 348c196cb3
[red-knot] avoid inferring types if unpacking fails (#16530) 
## Summary

This PR closes #15199.

The change I just made is to set all variables to type `Unknown` if
unpacking fails, but in some cases this may be excessive.
For example:

```py
a, b, c = "ab"
reveal_type(a)  # Unknown, but it would be reasonable to think of it as LiteralString
reveal_type(c)  # Unknown
```

```py
# Failed to unpack before the starred expression
(a, b, *c, d, e) = (1,)
reveal_type(a)  # Unknown
reveal_type(b)  # Unknown
...
# Failed to unpack after the starred expression
(a, b, *c, d, e) = (1, 2, 3)
reveal_type(a)  # Unknown, but should it be Literal[1]?
reveal_type(b)  # Unknown, but should it be Literal[2]?
reveal_type(c)  # Todo
reveal_type(d)  # Unknown
reveal_type(e)  # Unknown
```

I will modify it if you think it would be better to make it a different
type than just `Unknown`.

## Test Plan

I have made appropriate modifications to the test cases affected by this
change, and also added some more test cases.
 2025-03-07 11:04:44 -08:00
Vasco Schiavo 6d6e524b90
[flake8-bandit] Fix mixed-case hash algorithm names (S324) (#16552) 
The PR solves issue #16525
 2025-03-07 15:21:07 +00:00
Dylan 0dfa810e9a
Bump 0.9.10 (#16556) 2025-03-07 09:00:08 -06:00