Commit Graph

6544 Commits

Author SHA1 Message Date
Dhruv Manilawala f11d9cb509
[red-knot] Support overloads for callable equivalence (#17698)
## Summary

Part of #15383, this PR adds `is_equivalent_to` support for overloaded
callables.

This is mainly done by delegating it to the subtyping check in that two
types A and B are considered equivalent if A is a subtype of B and B is
a subtype of A.

## Test Plan

Add test cases for overloaded callables in `is_equivalent_to.md`
2025-04-30 02:53:59 +05:30
Victor Hugo Gomes 8c68d30c3a
[`flake8-use-pathlib`] Fix `PTH123` false positive when `open` is passed a file descriptor from a function call (#17705)
## Summary
Includes minor changes to the semantic type inference to help detect the
return type of function call.

Fixes #17691

## Test Plan

Snapshot tests
2025-04-29 16:51:38 -04:00
Alex Waygood 93d6a3567b
[red-knot] mdtest.py: Watch for changes in `red_knot_vendored` and `red_knot_test` as well as in `red_knot_python_semantic` (#17718) 2025-04-29 18:27:49 +00:00
Micha Reiser 1d788981cd
[red-knot] Capture backtrace in "check-failed" diagnostic (#17641)
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2025-04-29 16:58:58 +00:00
Hans 7d46579808
[docs] fix duplicated 'are' in comment for PTH123 rule (#17714) 2025-04-29 17:58:39 +02:00
Alex Waygood c9a6b1a9d0
[red-knot] Make `Type::signatures()` exhaustive (#17706) 2025-04-29 15:14:08 +01:00
Hans 9b9d16c3ba
[red-knot] colorize concise output diagnostics (#17232) (#17479)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Andrew Gallant <andrew@astral.sh>
2025-04-29 16:07:16 +02:00
David Peter 79f8473e51
[red-knot] Assignability of class literals to Callables (#17704)
## Summary

Subtyping was already modeled, but assignability also needs an explicit
branch. Removes 921 ecosystem false positives.

## Test Plan

New Markdown tests.
2025-04-29 15:04:22 +02:00
Douglas Creager ca4fdf452d
Create `TypeVarInstance` type for legacy typevars (#16538)
We are currently representing type variables using a `KnownInstance`
variant, which wraps a `TypeVarInstance` that contains the information
about the typevar (name, bounds, constraints, default type). We were
previously only constructing that type for PEP 695 typevars. This PR
constructs that type for legacy typevars as well.

It also detects functions that are generic because they use legacy
typevars in their parameter list. With the existing logic for inferring
specializations of function calls (#17301), that means that we are
correctly detecting that the definition of `reveal_type` in the typeshed
is generic, and inferring the correct specialization of `_T` for each
call site.

This does not yet handle legacy generic classes; that will come in a
follow-on PR.
2025-04-29 09:03:06 -04:00
Dylan 3c460a7b9a
Make syntax error for unparenthesized except tuples version specific to before 3.14 (#17660)
What it says on the tin 😄
2025-04-29 07:55:30 -05:00
Alex Waygood 31e6576971
[red-knot] micro-optimise `ClassLiteral::is_protocol` (#17703) 2025-04-29 12:35:53 +00:00
Micha Reiser c953e7d143
[red-knot] Improve log message for default python platform (#17700) 2025-04-29 08:26:41 +00:00
Vasco Schiavo 5096824793
[`ruff`] add fix safety section (`RUF017`) (#17480)
The PR add the `fix safety` section for rule `RUF017` (#15584 )
2025-04-28 22:07:22 +00:00
Dylan ae7691b026
Add Python 3.14 to configuration options (#17647)
A small PR that just updates the various settings/configurations to
allow Python 3.14. At the moment selecting that target version will
have no impact compared to Python 3.13 - except that a warning
is emitted if the user does so with `preview` disabled.
2025-04-28 16:29:00 -05:00
Wei Lee 504fa20057
[`airflow`] Apply auto fixes to cases where the names have changed in Airflow 3 (`AIR302`) (#17553)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Apply auto fixes to cases where the names have changed in Airflow 3 in
AIR302 and split the huge test cases into different test cases based on
proivder

## Test Plan

<!-- How was it tested? -->

the test cases has been split into multiple for easier checking
2025-04-28 16:35:17 -04:00
Brent Westbrook 01a31c08f5
Add config option to disable `typing_extensions` imports (#17611)
Summary
--

This PR resolves https://github.com/astral-sh/ruff/issues/9761 by adding
a linter configuration option to disable
`typing_extensions` imports. As mentioned [here], it would be ideal if
we could
detect whether or not `typing_extensions` is available as a dependency
automatically, but this seems like a much easier fix in the meantime.

The default for the new option, `typing-extensions`, is `true`,
preserving the current behavior. Setting it to `false` will bail out of
the new
`Checker::typing_importer` method, which has been refactored from the 
`Checker::import_from_typing` method in
https://github.com/astral-sh/ruff/pull/17340),
with `None`, which is then handled specially by each rule that calls it.

I considered some alternatives to a config option, such as checking if
`typing_extensions` has been imported or checking for a `TYPE_CHECKING`
block we could use, but I think defaulting to allowing
`typing_extensions` imports and allowing the user to disable this with
an option is both simple to implement and pretty intuitive.

[here]:
https://github.com/astral-sh/ruff/issues/9761#issuecomment-2790492853

Test Plan
--

New linter tests exercising several combinations of Python versions and
the new config option for PYI019. I also added tests for the other
affected rules, but only in the case where the new config option is
enabled. The rules' existing tests also cover the default case.
2025-04-28 14:57:36 -04:00
Andrew Gallant 405878a128 ruff_db: render file paths in diagnostics as relative paths if possible
This is done in what appears to be the same way as Ruff: we get the CWD,
strip the prefix from the path if possible, and use that. If stripping
the prefix fails, then we print the full path as-is.

Fixes #17233
2025-04-28 14:32:34 -04:00
Andrew Gallant 9a8f3cf247 red_knot_python_semantic: improve `not-iterable` diagnostic
This cleans up one particular TODO by splitting the "because" part of
the `not-iterable` diagnostic out into an info sub-diagnostic.
2025-04-28 11:03:41 -04:00
David Peter 07718f4788
[red-knot] Allow all callables to be assignable to @Todo-signatures (#17680)
## Summary

Removes ~850 diagnostics related to assignability of callable types,
where the callable-being-assigned-to has a "Todo signature", which
should probably accept any left hand side callable/signature.
2025-04-28 16:40:35 +02:00
Dylan 1e8881f9af
[`refurb`] Mark fix as safe for `readlines-in-for` (`FURB129`) (#17644)
This PR promotes the fix applicability of [readlines-in-for
(FURB129)](https://docs.astral.sh/ruff/rules/readlines-in-for/#readlines-in-for-furb129)
to always safe.

In the original PR (https://github.com/astral-sh/ruff/pull/9880), the
author marked the rule as unsafe because Ruff's type inference couldn't
quite guarantee that we had an `IOBase` object in hand. Some false
positives were recorded in the test fixture. However, before the PR was
merged, Charlie added the necessary type inference and the false
positives went away.

According to the [Python
documentation](https://docs.python.org/3/library/io.html#io.IOBase), I
believe this fix is safe for any proper implementation of `IOBase`:

>[IOBase](https://docs.python.org/3/library/io.html#io.IOBase) (and its
subclasses) supports the iterator protocol, meaning that an
[IOBase](https://docs.python.org/3/library/io.html#io.IOBase) object can
be iterated over yielding the lines in a stream. Lines are defined
slightly differently depending on whether the stream is a binary stream
(yielding bytes), or a text stream (yielding character strings). See
[readline()](https://docs.python.org/3/library/io.html#io.IOBase.readline)
below.

and then in the [documentation for
`readlines`](https://docs.python.org/3/library/io.html#io.IOBase.readlines):

>Read and return a list of lines from the stream. hint can be specified
to control the number of lines read: no more lines will be read if the
total size (in bytes/characters) of all lines so far exceeds hint. [...]
>Note that it’s already possible to iterate on file objects using for
line in file: ... without calling file.readlines().

I believe that a careful reading of our [versioning
policy](https://docs.astral.sh/ruff/versioning/#version-changes)
requires that this change be deferred to a minor release - but please
correct me if I'm wrong!
2025-04-28 09:39:55 -05:00
Dylan 152a0b6585
Collect preview lint behaviors in separate module (#17646)
This PR collects all behavior gated under preview into a new module
`ruff_linter::preview` that exposes functions like
`is_my_new_feature_enabled` - just as is done in the formatter crate.
2025-04-28 09:12:24 -05:00
David Peter 92f95ff494
[red-knot] TypedDict: No errors for introspection dunder attributes (#17677)
## Summary

Do not emit errors when accessing introspection dunder attributes such
as `__required_keys__` on `TypedDict`s.
2025-04-28 13:28:43 +02:00
Victor Hugo Gomes ceb2bf1168
[`flake8-pyi`] Ensure `Literal[None,] | Literal[None,]` is not autofixed to `None | None` (`PYI061`) (#17659)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-04-28 12:23:29 +01:00
David Peter f521358033
[red-knot] No errors for definitions of `TypedDict`s (#17674)
## Summary

Do not emit errors when defining `TypedDict`s:

```py
from typing_extensions import TypedDict

# No error here
class Person(TypedDict):
    name: str
    age: int | None

# No error for this alternative syntax
Message = TypedDict("Message", {"id": int, "content": str})
```

## Ecosystem analysis

* Removes ~ 450 false positives for `TypedDict` definitions.
* Changes a few diagnostic messages.
* Adds a few (< 10) false positives, for example:
  ```diff
+ error[lint:unresolved-attribute]
/tmp/mypy_primer/projects/hydra-zen/src/hydra_zen/structured_configs/_utils.py:262:5:
Type `Literal[DataclassOptions]` has no attribute `__required_keys__`
+ error[lint:unresolved-attribute]
/tmp/mypy_primer/projects/hydra-zen/src/hydra_zen/structured_configs/_utils.py:262:42:
Type `Literal[DataclassOptions]` has no attribute `__optional_keys__`
  ```
* New true positive

4f8263cd7f/corporate/lib/remote_billing_util.py (L155-L157)
  ```diff
+ error[lint:invalid-assignment]
/tmp/mypy_primer/projects/zulip/corporate/lib/remote_billing_util.py:155:5:
Object of type `RemoteBillingIdentityDict | LegacyServerIdentityDict |
None` is not assignable to `LegacyServerIdentityDict | None`
  ```

## Test Plan

New Markdown tests
2025-04-28 13:13:28 +02:00
Micha Reiser dbc137c951
[red-knot] Use 101 exit code when there's at least one diagnostic with severity 'fatal' (#17640) 2025-04-28 10:03:14 +02:00
Victor Hugo Gomes 826b2c9ff3
[`pycodestyle`] Fix duplicated diagnostic in `E712` (#17651) 2025-04-28 08:31:16 +01:00
jie211 a3e55cfd8f
[airflow] fix typos `AIR312` (#17673) 2025-04-28 08:31:41 +02:00
justin d2246278e6
[red-knot] Don't ignore hidden files by default (#17655) 2025-04-28 08:21:11 +02:00
Micha Reiser 1bdb22c139
[red-knot] Fix offset handling in playground for 2-code-point UTF16 characters (#17520) 2025-04-27 11:44:55 +01:00
Micha Reiser 1c65e0ad25
Split `SourceLocation` into `LineColumn` and `SourceLocation` (#17587) 2025-04-27 11:27:33 +01:00
justin 4443f6653c
[red-knot] Add --respect-ignore-files flag (#17645)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-04-27 10:55:41 +01:00
Vasco Schiavo b0d475f353
[`ruff`] add fix safety section (`RUF027`) (#17485)
The PR add the `fix safety` section for rule `RUF027` (#15584 ).

Actually, I have an example of a false positive. Should I include it in
the` fix safety` section?

---------

Co-authored-by: Dylan <dylwil3@gmail.com>
2025-04-26 16:43:53 -05:00
Vasco Schiavo b578a828ef
[`ruff`] add fix safety section (`RUF005`) (#17484)
The PR add the `fix safety` section for rule `RUF005` (#15584 ).

---------

Co-authored-by: Dylan <dylwil3@gmail.com>
2025-04-26 16:43:02 -05:00
Vasco Schiavo 64ba39a385
[`flynt`] add fix safety section (`FLY002`) (#17496)
The PR add the fix safety section for rule `FLY002` (#15584 )

The motivation for the content of the fix safety section is given by the
following example

```python
foo = 1
bar = [2, 3]

try:
    result_join = " ".join((foo, bar))
    print(f"Join result: {result_join}")
except TypeError as e:
    print(f"Join error: {e}")
```

which print `Join error: sequence item 0: expected str instance, int
found`

But after the fix is applied, we have

```python
foo = 1
bar = [2, 3]

try:
    result_join = f"{foo} {bar}"
    print(f"Join result: {result_join}")
except TypeError as e:
    print(f"Join error: {e}")
```

which print `Join result: 1 [2, 3]`

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-04-26 16:00:01 +00:00
Hans a4e225ee8a
[`flake8-async`] Add fix safety section (`ASYNC116`) (#17497)
## Summary

This PR add the `fix safety` section for rule `ASYNC116` in
`long_sleep_not_forever.rs` for #15584

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-04-26 10:40:51 -05:00
Vasco Schiavo 45d0634b01
[`pydocstyle`] add fix safety section (`D200`) (#17502)
The PR add the fix safety section for rule `D200` (#15584 )
2025-04-26 08:59:05 -05:00
Vasco Schiavo 4bcf1778fa
[`ruff`] add fix safety section (`RUF057`) (#17483)
The PR add the `fix safety` section for rule `RUF057` (#15584 )
2025-04-26 06:58:52 -05:00
Micha Reiser 6044f04137
Revert "[red-knot] Add --respect-ignore-files flag (#17569)" (#17642) 2025-04-26 10:30:50 +00:00
justin 2e95475f57
[red-knot] Add --respect-ignore-files flag (#17569)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-04-26 10:02:03 +00:00
Micha Reiser cfa1505068
[red-knot] Fix CLI hang when a dependent query panics (#17631) 2025-04-26 06:28:45 +00:00
Dhruv Manilawala 0251679f87
[red-knot] Add new property tests for subtyping with "bottom" callable (#17635)
## Summary

I remember we discussed about adding this as a property tests so here I
am.

## Test Plan

```console
❯ QUICKCHECK_TESTS=10000000 cargo test --locked --release --package red_knot_python_semantic -- --ignored types::property_tests::stable::bottom_callable_is_subtype_of_all_fully_static_callable
    Finished `release` profile [optimized] target(s) in 0.10s
     Running unittests src/lib.rs (target/release/deps/red_knot_python_semantic-e41596ca2dbd0e98)
running 1 test
test types::property_tests::stable::bottom_callable_is_subtype_of_all_fully_static_callable ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 233 filtered out; finished in 30.91s
```
2025-04-26 03:58:13 +05:30
Douglas Creager 6ab32a7746
[red-knot] Create generic context for generic classes lazily (#17617)
As discussed today, this is needed to handle legacy generic classes
without having to infer the types of the class's explicit bases eagerly
at class construction time. Pulling this out into a separate PR so
there's a smaller diff to review.

This also makes our representation of generic classes and functions more
consistent — before, we had separate Rust types and enum variants for
generic/non-generic classes, but a single type for generic functions.
Now we each a single (respective) type for each.

There were very few places we were differentiation between generic and
non-generic _class literals_, and these are handled now by calling the
(salsa cached) `generic_context` _accessor function_.

Note that _`ClassType`_ is still an enum with distinct variants for
non-generic classes and specialized generic classes.
2025-04-25 14:10:03 -04:00
Andrew Gallant bc0a5aa409 ruff_db: add tests for annotations with no ranges
... and fix the case where an annotation with a `Span` but no
`TextRange` or message gets completely dropped.
2025-04-25 13:25:20 -04:00
Wei Lee aba21a5d47
[`airflow`] Extend `AIR301` rule (#17598)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Add "airflow.operators.python.get_current_context" →
"airflow.sdk.get_current_context" rule

## Test Plan

<!-- How was it tested? -->

the test fixture has been updated accordingly
2025-04-25 12:49:32 -04:00
Wei Lee b6281a8805
[`airflow`] update existing `AIR302` rules with better suggestions (#17542)
## Summary

Even though the original suggestion works, they've been removed in later
version and is no longer the best practices.

e.g., many sql realted operators have been removed and are now suggested
to use SQLExecuteQueryOperator instead

## Test Plan

The existing test fixtures have been updated
2025-04-25 12:44:28 -04:00
Andrew Gallant 049280a3bc red_knot_project: sort diagnostics from checking files
Previously, we could iterate over files in an unspecified order (via
`HashSet` iteration) and we could accumulate diagnostics from files in
an unspecified order (via parallelism).

Here, we change the status quo so that diagnostics collected from files
are sorted after checking is complete. For now, we sort by severity
(with higher severity diagnostics appearing first) and then by
diagnostic ID to give a stable ordering.

I'm not sure if this is the best ordering.
2025-04-25 12:38:31 -04:00
Carl Meyer fa88989ef0
[red-knot] fix detecting a metaclass on a not-explicitly-specialized generic base (#17621)
## Summary

After https://github.com/astral-sh/ruff/pull/17620 (which this PR is
based on), I was looking at other call sites of `Type::into_class_type`,
and I began to feel that _all_ of them were currently buggy due to
silently skipping unspecialized generic class literal types (though in
some cases the bug hadn't shown up yet because we don't understand
legacy generic classes from typeshed), and in every case they would be
better off if an unspecialized generic class literal were implicitly
specialized with the default specialization (which is the usual Python
typing semantics for an unspecialized reference to a generic class),
instead of silently skipped.

So I changed the method to implicitly apply the default specialization,
and added a test that previously failed for detecting metaclasses on an
unspecialized generic base.

I also renamed the method to `to_class_type`, because I feel we have a
strong naming convention where `Type::into_foo` is always a trivial
`const fn` that simply returns `Some()` if the type is of variant `Foo`
and `None` otherwise. Even the existing method (with it handling both
`GenericAlias` and `ClassLiteral`, and distinguishing kinds of
`ClassLiteral`) was stretching this convention, and the new version
definitely breaks that envelope.

## Test Plan

Added a test that failed before this PR.
2025-04-25 06:55:54 -07:00
Carl Meyer 4c3f389598
[red-knot] fix inheritance-cycle detection for generic classes (#17620)
## Summary

The `ClassLiteralType::inheritance_cycle` method is intended to detect
inheritance cycles that would result in cyclic MROs, emit a diagnostic,
and skip actually trying to create the cyclic MRO, falling back to an
"error" MRO instead with just `Unknown` and `object`.

This method didn't work properly for generic classes. It used
`fully_static_explicit_bases`, which filter-maps `explicit_bases` over
`Type::into_class_type`, which returns `None` for an unspecialized
generic class literal. So in a case like `class C[T](C): ...`, because
the explicit base is an unspecialized generic, we just skipped it, and
failed to detect the class as cyclically defined.

Instead, iterate directly over all `explicit_bases`, and explicitly
handle both the specialized (`GenericAlias`) and unspecialized
(`ClassLiteral`) cases, so that we check all bases and correctly detect
cyclic inheritance.

## Test Plan

Added mdtests.
2025-04-25 06:55:00 -07:00
Brent Westbrook 6d3b1d13d6
[`pylint`] Detect `global` declarations in module scope (`PLE0118`) (#17411)
Summary
--

While going through the syntax errors in [this comment], I was surprised
to see the error `name 'x' is assigned to before global declaration`,
which corresponds to [load-before-global-declaration (PLE0118)] and has
also been reimplemented as a syntax error (#17135). However, it looks
like neither of the implementations consider `global` declarations in
the top-level module scope, which is a syntax error in CPython:

```python
# try.py
x = None
global x
```

```shell
> python -m compileall -f try.py
Compiling 'try.py'...
***   File "try.py", line 2
    global x
    ^^^^^^^^
SyntaxError: name 'x' is assigned to before global declaration
```

I'm not sure this is the best or most elegant solution, but it was a
quick fix that passed all of our tests.

Test Plan
--

New PLE0118 test case.

[this comment]:
https://github.com/astral-sh/ruff/issues/7633#issuecomment-1740424031
[load-before-global-declaration (PLE0118)]:
https://docs.astral.sh/ruff/rules/load-before-global-declaration/#load-before-global-declaration-ple0118
2025-04-25 08:37:16 -04:00
Max Mynter 3f84e75e20
Add Semantic Error Test for LateFutureImport (#17612)
Adresses a question in #17526.

## Summary
Adds a syntax error test for `__future__` import not at top of file. 

## Question: 
Is this a redundant with
8d2c79276d/crates/ruff_linter/resources/test/fixtures/pyflakes/F404_0.py (L1-L8)
and
8d2c79276d/crates/ruff_linter/resources/test/fixtures/pyflakes/F404_1.py (L1-L5)

which test pyflake `F404`?
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
This is a test
<!-- How was it tested? -->
2025-04-25 08:32:57 -04:00