## Summary
The [`DateType`](https://github.com/glyph/DateType) library has some
very large protocols in it. Currently we type-check it quite quickly,
but the current version of https://github.com/astral-sh/ruff/pull/18659
makes our execution time on this library pathologically slow. That PR
doesn't seem to have a big impact on any of our current benchmarks,
however, so it seems we have some missing coverage in this area; I
therefore propose that we add `DateType` as a benchmark.
Currently the benchmark runs pretty quickly (about half the runtime of
attrs, which is our fastest real-world benchmark currently), and the
library has 0 third-party dependencies, so the benchmark is quick to
setup.
## Test Plan
`cargo bench -p ruff_benchmark --bench=ty`
## Summary
`ty` does not understand that calls to functions which have been
annotated as having a return type of `Never` / `NoReturn` are terminal.
This PR fixes that, by adding new reachability constraints when call
expressions are seen. If the call expression evaluates to `Never`, the
code following it will be considered to be unreachable. Note that, for
adding these constraints, we only consider call expressions at the
statement level, and that too only inside function scopes. This is
because otherwise, the number of such constraints becomes too high, and
evaluating them later on during type inference results in a major
performance degradation.
Fixes https://github.com/astral-sh/ty/issues/180
## Test Plan
New mdtests.
## Ecosystem changes
This PR removes the following false-positives:
- "Function can implicitly return `None`, which is not assignable to
...".
- "Name `foo` used when possibly not defind" - because the branch in
which it is not defined has a `NoReturn` call, or when `foo` was
imported in a `try`, and the except had a `NoReturn` call.
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
Per @ntBre in https://github.com/astral-sh/ruff/pull/19111, it would be
a good idea to make the tests no longer have these syntax errors, so
this PR updates the tests and snapshots.
`B031` gave me a lot of trouble since the ending test of declaring a
function named `groupby` makes it so that inside other functions, it's
unclear which `groupby` is referred to since it depends on when the
function is called. To fix it I made each function have it's own `from
itertools import groupby` so there's no more ambiguity.
## Summary
From me and @ntBre's discussion in #19111.
This PR makes these two examples into valid code, since they previously
had `F701`-`F707` syntax errors. `SIM110` was already fixed in a
different PR, I just forgot to pull.
## Summary
Was just playing around with this, there's definitely more to do with
this function, but it seems like maybe a better option than having so
many arms in has_relation_to for (_, Callable).
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
This PR is a collaboration with @AlexWaygood from our pairing session
last Friday.
The main goal here is removing `ruff_linter::message::OldDiagnostic` in
favor of
using `ruff_db::diagnostic::Diagnostic` directly. This involved a few
major steps:
- Transferring the fields
- Transferring the methods and trait implementations, where possible
- Converting some constructor methods to free functions
- Moving the `SecondaryCode` struct
- Updating the method names
I'm hoping that some of the methods, especially those in the
`expect_ruff_*`
family, won't be necessary long-term, but I avoided trying to replace
them
entirely for now to keep the already-large diff a bit smaller.
### Related refactors
Alex and I noticed a few refactoring opportunities while looking at the
code,
specifically the very similar implementations for
`create_parse_diagnostic`,
`create_unsupported_syntax_diagnostic`, and
`create_semantic_syntax_diagnostic`.
We combined these into a single generic function, which I then copied
into
`ruff_linter::message` with some small changes and a TODO to combine
them in the
future.
I also deleted the `DisplayParseErrorType` and `TruncateAtNewline` types
for
reporting parse errors. These were added in #4124, I believe to work
around the
error messages from LALRPOP. Removing these didn't affect any tests, so
I think
they were unnecessary now that we fully control the error messages from
the
parser.
On a more minor note, I factored out some calls to the
`OldDiagnostic::filename`
(now `Diagnostic::expect_ruff_filename`) function to avoid repeatedly
allocating
`String`s in some places.
### Snapshot changes
The `show_statistics_syntax_errors` integration test changed because the
`OldDiagnostic::name` method used `syntax-error` instead of
`invalid-syntax`
like in ty. I think this (`--statistics`) is one of the only places we
actually
use this name for syntax errors, so I hope this is okay. An alternative
is to
use `syntax-error` in ty too.
The other snapshot changes are from removing this code, as discussed on
[Discord](https://discord.com/channels/1039017663004942429/1228460843033821285/1388252408848847069):
34052a1185/crates/ruff_linter/src/message/mod.rs (L128-L135)
I think both of these are technically breaking changes, but they only
affect
syntax errors and are very narrow in scope, while also pretty
substantially
simplifying the refactor, so I hope they're okay to include in a patch
release.
## Test plan
Existing tests, with the adjustments mentioned above
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This implements filtering of private symbols from stub files based on
type information as discussed in
https://github.com/astral-sh/ruff/pull/19102. It extends the previous
implementation to apply to all stub files, instead of just the
`builtins` module, and uses type information to retain private names
that are may be relevant at runtime.
Summary
--
Closes#19014 by identifying more `field` functions from `attrs`. We
already detected these when imported from `attrs` but not the `attr`
module from the same package. These functions are identical to the
`attrs` versions:
```pycon
>>> import attrs, attr
>>> attrs.field is attr.field
True
>>> attrs.Factory is attr.Factory
True
>>>
```
Test Plan
--
Regression tests based on the issue
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [pytest-incorrect-mark-parentheses-style
(PT023)](https://docs.astral.sh/ruff/rules/pytest-incorrect-mark-parentheses-style/#pytest-incorrect-mark-parentheses-style-pt023)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/48989153-6d4a-493a-a287-07f330f270bc)
```py
import pytest
@pytest.mark.foo
def test_something(): ...
```
[New example](https://play.ruff.rs/741f4d19-4607-4777-a77e-4ea6c62845e1)
```py
import pytest
@pytest.mark.foo()
def test_something(): ...
```
This just swaps the parenthesis in the "Example" and "Use instead"
sections since the default configuration is no parenthesis
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [pytest-warns-too-broad
(PT030)](https://docs.astral.sh/ruff/rules/pytest-warns-too-broad/#pytest-warns-too-broad-pt030)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/2296ae7e-c775-427a-a020-6fb25321f3f7)
```py
import pytest
def test_foo():
with pytest.warns(RuntimeWarning):
...
# empty string is also an error
with pytest.warns(RuntimeWarning, match=""):
...
```
[New example](https://play.ruff.rs/af35a482-1c2f-47ee-aff3-ff1e9fa447de)
```py
import pytest
def test_foo():
with pytest.warns(Warning):
...
# empty string is also an error
with pytest.warns(Warning, match=""):
...
```
`RuntimeWarning` is not in the default
[warns-require-match-for](https://docs.astral.sh/ruff/settings/#lint_flake8-pytest-style_warns-require-match-for)
list, while `Warning` is. The "Use instead" section was also updated
similarly
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [avoidable-escaped-quote
(Q003)](https://docs.astral.sh/ruff/rules/avoidable-escaped-quote/#avoidable-escaped-quote-q003)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/fb319d0f-8016-46a1-b6bb-42b1b054feea)
```py
foo = 'bar\'s'
```
[New example](https://play.ruff.rs/d9626561-0646-448f-9282-3f0691b90831)
```py
foo = "bar\"s"
```
The original example got overwritten by `Q000`, since double quotes is
the default config. The quotes were also switched in the "Use instead"
section.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [enumerate-for-loop
(SIM113)](https://docs.astral.sh/ruff/rules/enumerate-for-loop/#enumerate-for-loop-sim113)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/a6ef6fec-eb6b-477c-a962-616f0b8e1491)
```py
fruits = ["apple", "banana", "cherry"]
for fruit in fruits:
print(f"{i + 1}. {fruit}")
i += 1
```
[New example](https://play.ruff.rs/1811d608-1aa0-45d8-96dc-18105e74b8cc)
```py
fruits = ["apple", "banana", "cherry"]
i = 0
for fruit in fruits:
print(f"{i + 1}. {fruit}")
i += 1
```
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [enumerate-for-loop [if-else-block-instead-of-dict-get
(SIM401)](https://docs.astral.sh/ruff/rules/if-else-block-instead-of-dict-get/#if-else-block-instead-of-dict-get-sim401)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/635629eb-7146-45a8-9e0c-4a0aa9446ded)
```py
if "bar" in foo:
value = foo["bar"]
else:
value = 0
```
[New example](https://play.ruff.rs/a1227ec9-05c2-4a22-800d-c76cb7abe249)
```py
foo = {}
if "bar" in foo:
value = foo["bar"]
else:
value = 0
```
The "Use instead" section was also updated similarly.
The docs for `SIM401` also has another section on the preview ternary
version, but it does not seem to check that the variable is a dict
(bug?) https://play.ruff.rs/c0feada8-a7fe-43f7-b57e-c10520fdcdca
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [reimplemented-builtin
(SIM110)](https://docs.astral.sh/ruff/rules/reimplemented-builtin/#reimplemented-builtin-sim110)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/1c192e8b-13f8-4f07-8c35-9dcd516a4a02)
```py
for item in iterable:
if predicate(item):
return True
return False
```
[New example](https://play.ruff.rs/f77393ad-20b1-436f-a872-d3bccec7c829)
```py
def foo():
for item in iterable:
if predicate(item):
return True
return False
```
The "Use instead" section was also updated to reflect the change.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
The benchmark is currently very noisy (± 10%). This leads to codspeed
reports on PRs, because we often exceed the trigger threshold. This is
confusing to ty contributors who are not aware about the flakiness.
Let's disable it for now.
## Summary
This PR adds initial support for workspace diagnostics in the ty server.
Reference spec:
https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_diagnostic
This is currently implemented via the **pull diagnostics method** which
was added in the current version (3.17) and the server advertises it via
the `diagnosticProvider.workspaceDiagnostics` server capability.
**Note:** This might be a bit confusing but a workspace diagnostics is
not for a single workspace but for all the workspaces that the server
handles. These are the ones that the server received during
initialization. Currently, the ty server doesn't support multiple
workspaces so this capability is also limited to provide diagnostics
only for a single workspace (the first one if the client provided
multiple).
A new `ty.diagnosticMode` server setting is added which can be either
`workspace` (for workspace diagnostics) or `openFilesOnly` (for checking
only open files) (default). This is same as
`python.analysis.diagnosticMode` that Pyright / Pylance utilizes. In the
future, we could use the value under `python.*` namespace as fallback to
improve the experience on user side to avoid setting the value multiple
times.
Part of: astral-sh/ty#81
## Test Plan
This capability was introduced in the current LSP version (~3 years) and
the way it's implemented by various clients are a bit different. I've
provided notes on what I've noticed and what would need to be done on
our side to further improve the experience.
### VS Code
VS Code sends the `workspace/diagnostic` requests every ~2 second:
```
[Trace - 12:12:32 PM] Sending request 'workspace/diagnostic - (403)'.
[Trace - 12:12:32 PM] Received response 'workspace/diagnostic - (403)' in 2ms.
[Trace - 12:12:34 PM] Sending request 'workspace/diagnostic - (404)'.
[Trace - 12:12:34 PM] Received response 'workspace/diagnostic - (404)' in 2ms.
[Trace - 12:12:36 PM] Sending request 'workspace/diagnostic - (405)'.
[Trace - 12:12:36 PM] Received response 'workspace/diagnostic - (405)' in 2ms.
[Trace - 12:12:38 PM] Sending request 'workspace/diagnostic - (406)'.
[Trace - 12:12:38 PM] Received response 'workspace/diagnostic - (406)' in 3ms.
[Trace - 12:12:40 PM] Sending request 'workspace/diagnostic - (407)'.
[Trace - 12:12:40 PM] Received response 'workspace/diagnostic - (407)' in 2ms.
...
```
I couldn't really find any resource that explains this behavior. But,
this does mean that we'd need to implement the caching layer via the
previous result ids sooner. This will allow the server to avoid sending
all the diagnostics on every request and instead just send a response
stating that the diagnostics hasn't changed yet. This could possibly be
achieved by using the salsa ID.
If we switch from workspace diagnostics to open-files diagnostics, the
server would send the diagnostics only via the `textDocument/diagnostic`
endpoint. Here, when a document containing the diagnostic is closed, the
server would send a publish diagnostics notification with an empty list
of diagnostics to clear the diagnostics from that document. The issue is
the VS Code doesn't seem to be clearing the diagnostics in this case
even though it receives the notification. (I'm going to open an issue on
VS Code side for this today.)
https://github.com/user-attachments/assets/b0c0833d-386c-49f5-8a15-0ac9133e15ed
### Zed
Zed's implementation works by refreshing the workspace diagnostics
whenever the content of the documents are changed. This seems like a
very reasonable behavior and I was a bit surprised that VS Code didn't
use this heuristic.
https://github.com/user-attachments/assets/71c7b546-7970-434a-9ba0-4fa620647f6c
### Neovim
Neovim only recently added support for workspace diagnostics
(https://github.com/neovim/neovim/pull/34262, merged ~3 weeks ago) so
it's only available on nightly versions.
The initial support is limited and requires fetching the workspace
diagnostics manually as demonstrated in the video. It doesn't support
refreshing the workspace diagnostics either, so that would need to be
done manually as well. I'm assuming that these are just a temporary
limitation and will be implemented before the stable release.
https://github.com/user-attachments/assets/25b4a0e5-9833-4877-88ad-279904fffaf9
## Summary
This PR adds a new trait to support running a request in the background.
Currently, there exists a `BackgroundDocumentRequestHandler` trait which
is similar but is scoped to a specific document (file in an editor
context). The new trait `BackgroundRequestHandler` is not tied to a
specific document nor a specific project but it's for the entire
workspace.
This is added to support running workspace wide requests like computing
the [workspace
diagnostics](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_diagnostic)
or [workspace
symbols](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_symbol).
**Note:** There's a slight difference with what a "workspace" means
between the server and ty. Currently, there's a 1-1 relationship between
a workspace in an editor and the project database corresponding to that
workspace in ty but this could change in the future when Micha adds
support for multiple workspaces or multi-root workspaces.
The data that would be required by the request handler (based on
implementing workspace diagnostics) is the list of databases
(`ProjectDatabse`) corresponding to the projects in the workspace and
the index (`Index`) that contains the open documents. The
`WorkspaceSnapshot` represents this and is passed to the handler similar
to `DocumentSnapshot`.
## Test Plan
This is used in implementing the workspace diagnostics which is where
this is tested.
## Summary
Part of https://github.com/astral-sh/ty/issues/129
There were previously some false positives here.
## Test Plan
Updated `is_subtype_of.md` and `is_assignable_to.md`
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [snake-case-type-alias
(PYI042)](https://docs.astral.sh/ruff/rules/snake-case-type-alias/#snake-case-type-alias-pyi042)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/8fafec81-2228-4ffe-81e8-1989b724cb47)
```py
type_alias_name: TypeAlias = int
```
[New example](https://play.ruff.rs/b396746c-e6d2-423c-bc13-01a533bb0747)
```py
from typing import TypeAlias
type_alias_name: TypeAlias = int
```
Imports were also added to the "use instead" section.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This fixes the docs for [expressions-in-star-assignment
(F621)](https://docs.astral.sh/ruff/rules/expressions-in-star-assignment/#expressions-in-star-assignment-f621)
having a backslash `\` before the left shifts `<<`. I'm not sure why
this happened in the first place, as the docstring looks fine, but
putting the `<<` inside a code block fixes it. I was not able to track
down the source of the issue either. The only other rule with a `<<` is
[missing-whitespace-around-bitwise-or-shift-operator
(E227)](https://docs.astral.sh/ruff/rules/missing-whitespace-around-bitwise-or-shift-operator/#missing-whitespace-around-bitwise-or-shift-operator-e227),
which already has it in a code block.
Old docs page:
> In Python 3, no more than 1 \\<< 8 assignments are allowed before a
starred expression, and no more than 1 \\<< 24 expressions are allowed
after a starred expression.
New docs page:
> In Python 3, no more than `1 << 8` assignments are allowed before a
starred expression, and no more than `1 << 24` expressions are allowed
after a starred expression.
## Test Plan
<!-- How was it tested? -->
N/A, no tests/functionality affected.
## Summary
Allow declared-only class-level attributes to be accessed on the class:
```py
class C:
attr: int
C.attr # this is now allowed
```
closes https://github.com/astral-sh/ty/issues/384
closes https://github.com/astral-sh/ty/issues/553
## Ecosystem analysis
* We see many removed `unresolved-attribute` false-positives for code
that makes use of sqlalchemy, as expected (see changes for `prefect`)
* We see many removed `call-non-callable` false-positives for uses of
`pytest.skip` and similar, as expected
* Most new diagnostics seem to be related to cases like the following,
where we previously inferred `int` for `Derived().x`, but now we infer
`int | None`. I think this should be a
conflicting-declarations/bad-override error anyway? The new behavior may
even be preferred here?
```py
class Base:
x: int | None
class Derived(Base):
def __init__(self):
self.x: int = 1
```
## Summary
The motivation of `ScopedExpressionId` was that we have an expression
identifier that's local to a scope and, therefore, unlikely to change if
a user makes changes in another scope. A local identifier like this has
the advantage that query results may remain unchanged even if other
parts of the file change, which in turn allows Salsa to short-circuit
dependent queries.
However, I noticed that we aren't using `ScopedExpressionId` in a place
where it's important that the identifier is local. It's main use is
inside `infer` which we always run for the entire file. The one
exception to this is `Unpack` but unpack runs as part of `infer`.
Edit: The above isn't entirely correct. We used ScopedExpressionId in
TypeInference which is a query result. Now using ExpressionNodeKey does
mean that a change to the AST invalidates most if not all TypeInference
results of a single file. Salsa then has to run all dependent queries to
see if they're affected by this change even if the change was local to
another scope.
If this locality proves to be important I suggest that we create two
queries on top of TypeInference: one that returns the expression map
which is mainly used in the linter and type inference and a second that
returns all remaining fields. This should give us a similar optimization
at a much lower cost
I also considered remove `ScopedUseId` but I believe that one is still
useful because using `ExpressionNodeKey` for it instead would mean that
all `UseDefMap` change when a single AST node changes. Whether this is
important is something difficult to assess. I'm simply not familiar
enough with the `UseDefMap`. If the locality doesn't matter for the
`UseDefMap`, then a similar change could be made and `bindings_by_use`
could be changed to an `FxHashMap<UseId, Bindings>` where `UseId` is a
thin wrapper around `NodeKey`.
Closes https://github.com/astral-sh/ty/issues/721
## Summary
Extracts the vendored typeshed stubs lazily and caches them on the local
filesystem to support go-to in the LSP.
Resolves https://github.com/astral-sh/ty/issues/77.
## Summary
This PR makes the necessary changes to the server that it can request
configurations from the client using the `configuration` request.
This PR doesn't make use of the request yet. It only sets up the
foundation (mainly the coordination between client and server)
so that future PRs could pull specific settings.
I plan to use this for pulling the Python environment from the Python
extension.
Deno does something very similar to this.
## Test Plan
Tested that diagnostics are still shown.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [duplicate-literal-member
(PYI062)](https://docs.astral.sh/ruff/rules/duplicate-literal-member/#duplicate-literal-member-pyi062)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/6b00b41c-c1c5-4421-873d-fc2a143e7337)
```py
foo: Literal["a", "b", "a"]
```
[New example](https://play.ruff.rs/1aea839b-9ae8-4848-bb83-2637e1a68ce4)
```py
from typing import Literal
foo: Literal["a", "b", "a"]
```
Imports were also added to the "use instead" section.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
Follow-up to https://github.com/astral-sh/ruff/pull/19083, also log the
target names like `ty_python_semantic::module_resolver::resolver` in
`2025-07-02 10:12:20.188697000 DEBUG
ty_python_semantic::module_resolver::resolver: Adding first-party search
path '/Users/dhruv/playground/ty_server'` at trace level.
## Summary
I hoped this might fix the latest stack overflows on
https://github.com/astral-sh/ruff/pull/18659... it doesn't look like it
does, but these changes seem like they're probably correct anyway...?
## Test Plan
<!-- How was it tested? -->
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This PR fixes#19047 / the [isinstance-type-none
(FURB168)](https://docs.astral.sh/ruff/rules/isinstance-type-none/#isinstance-type-none-furb168)
tuple false positive by adding a check if the tuple is empty to the
code. I also noticed there was another false positive with the other
tuple check in the same function, so I fixed it the same way.
`Union[()]` is invalid at runtime with `TypeError: Cannot take a Union
of no types.`, but it is accepted by `basedpyright`
[playground](https://basedpyright.com/?pythonVersion=3.8&typeCheckingMode=all&code=GYJw9gtgBALgngBwJYDsDmUkQWEMoCqKSYKAsAFAgCmAbtQIYA2A%2BvAtQBREkoDanAJQBdQUA)
and is equivalent to `Never`, so I fixed it anyways. I'm getting on a
side tangent here, but it looks like MyPy doesn't accept it, and ty
[playground](https://play.ty.dev/c2c468b6-38e4-4dd9-a9fa-0276e843e395)
gives `@Todo`.
## Test Plan
<!-- How was it tested? -->
Added two test cases for the two false positives.
[playground](https://play.ruff.rs/a53afc21-9a1d-4b9b-9346-abfbeabeb449)
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [datetime-min-max
(DTZ901)](https://docs.astral.sh/ruff/rules/datetime-min-max/#datetime-min-max-dtz901)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/c1202727-1a18-4d3f-92a4-334ede07ed3e)
```py
datetime.max
```
[New example](https://play.ruff.rs/af2c76aa-9beb-46bc-8e27-faf53ecdbe8c)
```py
import datetime
datetime.datetime.max
```
I also added imports to the problem demonstration and use instead.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
## Summary
Remove a hack in control flow modeling that was treating `return`
statements at the end of function bodies in a special way (basically
considering the state *just before* the `return` statement as the
end-of-scope state). This is not needed anymore now that #18750 has been
merged.
In order to make this work, we now use *all reachable bindings* for
purposes of finding implicit instance attribute assignments as well as
for deferred lookups of symbols. Both would otherwise be affected by
this change:
```py
def C:
def f(self):
self.x = 1 # a reachable binding that is not visible at the end of the scope
return
```
```py
def f():
class X: ... # a reachable binding that is not visible at the end of the scope
x: "X" = X() # deferred use of `X`
return
```
Implicit instance attributes also required another change. We previously
kept track of possibly-unbound instance attributes in some cases, but we
now give up on that completely and always consider *implicit* instance
attributes to be bound if we see a reachable binding in a reachable
method. The previous behavior was somewhat inconsistent anyway because
we also do not consider attributes possibly-unbound in other scenarios:
we do not (and can not) keep track of whether or not methods are called
that define these attributes.
closes https://github.com/astral-sh/ty/issues/711
## Ecosystem analysis
I think this looks very positive!
* We see an unsurprising drop in `possibly-unbound-attribute`
diagnostics (599), mostly for classes that define attributes in `try …
except` blocks, `for` loops, or `if … else: raise …` constructs. There
might obviously also be true positives that got removed, but the vast
majority should be false positives.
* There is also a drop in `possibly-unresolved-reference` /
`unresolved-reference` diagnostics (279+13) from the change to deferred
lookups.
* Some `invalid-type-form` false positives got resolved (13), because we
can now properly look up the names in the annotations.
* There are some new *true* positives in `attrs`, since we understand
the `Attribute` annotation that was previously inferred as `Unknown`
because of a re-assignment after the class definition.
## Test Plan
The existing attributes.md test suite has sufficient coverage here.
## Summary
Temporarily modify `UseDefMapBuilder::reachability` for star imports in
order for new definitions to pick up the right reachability. This was
already working for `UseDefMapBuilder::place_states`, but not for
`UseDefMapBuilder::reachable_definitions`.
closes https://github.com/astral-sh/ty/issues/728
## Test Plan
Regression test
## Summary
Evaluate `TYPE_CHECKING` to `ALWAYS_TRUE` and `not TYPE_CHECKING` to
`ALWAYS_FALSE` during semantic index building. This is a follow-up to
https://github.com/astral-sh/ruff/pull/18998 and is in principle just a
performance optimization. We see some (favorable) ecosystem changes
because we can eliminate definitely-unreachable branches early now and
retain narrowing constraints without solving
https://github.com/astral-sh/ty/issues/690 first.
Gates all uses of `get-size2` behind the feature `get-size` in the crate
`ruff_python_ast`. Also requires that `ruff_text_size` is pulled in with
the feature `get-size` enabled if we enable the same-named feature for
`ruff_python_ast`.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Make `UP045` ignore `Optional[NamedTuple]` as `NamedTuple` is a function
(not a proper type). Rewriting it to `NamedTuple | None` breaks at
runtime. While type checkers currently accept `NamedTuple` as a type,
they arguably shouldn't. Therefore, we outright ignore it and don't
touch or lint on it.
For a more detailed discussion, see the linked issue.
## Test Plan
Added examples to the existing tests.
## Related Issues
Fixes: https://github.com/astral-sh/ruff/issues/18619
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [call-date-today
(DTZ011)](https://docs.astral.sh/ruff/rules/call-date-today/#call-date-today-dtz011)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/b42d6aef-7777-4b3b-9f96-19132000b765)
```py
import datetime
datetime.datetime.today()
```
[New example](https://play.ruff.rs/8577c3c1-cfa8-425b-b1e1-4c53b2a48375)
```py
import datetime
datetime.date.today()
```
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [no-explicit-stacklevel
(B028)](https://docs.astral.sh/ruff/rules/no-explicit-stacklevel/#no-explicit-stacklevel-b028)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/1ee80aec-2d6e-4a3f-8e98-da82b6a9f544)
```py
warnings.warn("This is a warning")
```
[New example](https://play.ruff.rs/343593aa-38a0-4d76-a32b-5abd0a4306cc)
```py
import warnings
warnings.warn("This is a warning")
```
Imports were also added to the "use instead" section
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [batched-without-explicit-strict
(B911)](https://docs.astral.sh/ruff/rules/batched-without-explicit-strict/#batched-without-explicit-strict-b911)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/a897d96b-0749-4291-8a62-dfd4caf290a0)
```py
itertools.batched(iterable, n)
```
[New example](https://play.ruff.rs/1c1e0ab7-014c-4dc2-abed-c2cb6cd01f70)
```py
import itertools
itertools.batched(iterable, n)
```
Imports were also added to the "use instead" sections
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
## Summary
This just replaces one temporary solution to recursive protocols (the
`SelfReference` mechanism) with another one (track seen types when
recursively descending in `normalize` and replace recursive references
with `Any`). But this temporary solution can handle mutually-recursive
types, not just self-referential ones, and it's sufficient for the
primer ecosystem and some other projects we are testing on to no longer
stack overflow.
The follow-up here will be to properly handle these self-references
instead of replacing them with `Any`.
We will also eventually need cycle detection on more recursive-descent
type transformations and tests.
## Test Plan
Existing tests (including recursive-protocol tests) and primer.
Added mdtest for mutually-recursive protocols that stack-overflowed
before this PR.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
This PR fixes rule C420's fix. The fix replaces `{...}` with
`dict....(...)`. Therefore, if there is any identifier or such right
before the fix, the fix will fuse that previous token with `dict...`.
The example in the issue is
```python
0 or{x: None for x in "x"}
# gets "fixed" to
0 ordict.fromkeys(iterable)
```
## Related Issues
Fixes: https://github.com/astral-sh/ruff/issues/18599
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Fixes#18908
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [airflow3-moved-to-provider
(AIR302)](https://docs.astral.sh/ruff/rules/airflow3-moved-to-provider/#airflow3-moved-to-provider-air302)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/1026c008-57bc-4330-93b9-141444f2a611)
```py
from airflow.auth.managers.fab.fab_auth_manage import FabAuthManager
```
[New example](https://play.ruff.rs/b690e809-a81d-4265-9fde-1494caa0b7fd)
```py
from airflow.auth.managers.fab.fab_auth_manager import FabAuthManager
fab_auth_manager_app = FabAuthManager().get_fastapi_app()
```
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Mark `UP008`'s fix safe if it won't delete comments.
## Relevant Issues
Fixes: https://github.com/astral-sh/ruff/issues/18533
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [flask-debug-true
(S201)](https://docs.astral.sh/ruff/rules/flask-debug-true/#flask-debug-true-s201)'s
example error out-of-the-box
[Old example](https://play.ruff.rs/d5e1a013-1107-4223-9094-0e8393ad3c64)
```py
import flask
app = Flask()
app.run(debug=True)
```
[New example](https://play.ruff.rs/c4aebd2c-0448-4471-8bad-3e38ace68367)
```py
from flask import Flask
app = Flask()
app.run(debug=True)
```
Imports were also added to the `Use instead:` section to make it valid
code out-of-the-box.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
## Summary
Simplifies literal `True` and `False` conditions to `ALWAYS_TRUE` /
`ALWAYS_FALSE` during semantic index building. This allows us to eagerly
evaluate more constraints, which should help with performance (looks
like there is a tiny 1% improvement in instrumented benchmarks), but
also allows us to eliminate definitely-unreachable branches in
control-flow merging. This can lead to better type inference in some
cases because it allows us to retain narrowing constraints without
solving https://github.com/astral-sh/ty/issues/690 first:
```py
def _(c: int | None):
if c is None:
assert False
reveal_type(c) # int, previously: int | None
```
closes https://github.com/astral-sh/ty/issues/713
## Test Plan
* Regression test for https://github.com/astral-sh/ty/issues/713
* Made sure that all ecosystem diffs trace back to removed false
positives
## Summary
This PR adds diagnostic for invalid binary operators in type
expressions. It should close https://github.com/astral-sh/ty/issues/706
if merged.
Please feel free to suggest better wordings for the diagnostic message.
## Test Plan
I modified `mdtest/annotations/invalid.md` and added a test for each
binary operator, and fixed tests that was broken by the new diagnostic.
## Summary
Print the [new salsa memory usage
dumps](https://github.com/astral-sh/ruff/pull/18928) in mypy primer CI
runs to help us catch memory regressions. The numbers are rounded to the
nearest power of 1.1 (about a 5% threshold between buckets) to avoid overly sensitive diffs.
This PR extracts a lot of the complex logic in the `match_parameters`
and `check_types` methods of our call binding machinery into separate
helper types. This is setup for #18996, which will update this logic to
handle variadic arguments. To do so, it is helpful to have the
per-argument logic extracted into a method that we can call repeatedly
for each _element_ of a variadic argument.
This should be a pure refactoring, with no behavioral changes.
This PR updates our unpacking assignment logic to use the new tuple
machinery. As a result, we can now unpack variable-length tuples
correctly.
As part of this, the `TupleSpec` classes have been renamed to `Tuple`,
and can now contain any element (Rust) type, not just `Type<'db>`. The
unpacker uses a tuple of `UnionBuilder`s to maintain the types that will
be assigned to each target, as we iterate through potentially many union
elements on the rhs. We also add a new consuming iterator for tuples,
and update the `all_elements` methods to wrap the result in an enum
(similar to `itertools::Position`) letting you know which part of the
tuple each element appears in. I also added a new
`UnionBuilder::try_build`, which lets you specify a different fallback
type if the union contains no elements.
## Summary
Ensure that we correctly infer calls such as `tuple((1, 2))`,
`tuple(range(42))`, etc. Ensure that we emit errors on invalid calls
such as `tuple[int, str]()`.
## Test Plan
Mdtests
## Summary
Under preview 🧪 I've expanded rule `PYI016` to also flag type
union duplicates containing `None` and `Optional`.
## Test Plan
Examples/tests have been added. I've made sure that the existing
examples did not change unless preview is enabled.
## Relevant Issues
* https://github.com/astral-sh/ruff/issues/18508 (discussing
introducing/extending a rule to flag `Optional[None]`)
* https://github.com/astral-sh/ruff/issues/18546 (where I discussed this
addition with @AlexWaygood)
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
I think this should be the last step before combining `OldDiagnostic`
and `ruff_db::Diagnostic`. We can't store a `NoqaCode` on
`ruff_db::Diagnostic`, so I converted the `noqa_code` field to an
`Option<String>` and then propagated this change to all of the callers.
I tried to use `&str` everywhere it was possible, so I think the
remaining `to_string` calls are necessary. I spent some time trying to
convert _everything_ to `&str` but ran into lifetime issues, especially
in the `FixTable`. Maybe we can take another look at that if it causes a
performance regression, but hopefully these paths aren't too hot. We
also avoid some `to_string` calls, so it might even out a bit too.
## Test Plan
Existing tests
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Most of the work here was doing some light refactoring to facilitate
sensible testing. That is, we don't want to list every builtin included
in most tests, so we add some structure to the completion type returned.
Tests can now filter based on whether a completion is a builtin or not.
Otherwise, builtins are found using the existing infrastructure for
`object.attr` completions (where we hard-code the module name
`builtins`).
I did consider changing the sort order based on whether a completion
suggestion was a builtin or not. In particular, it seemed like it might
be a good idea to sort builtins after other scope based completions,
but before the dunder and sunder attributes. Namely, it seems likely
that there is an inverse correlation between the size of a scope and
the likelihood of an item in that scope being used at any given point.
So it *might* be a good idea to prioritize the likelier candidates in
the completions returned.
Additionally, the number of items introduced by adding builtins is quite
large. So I wondered whether mixing them in with everything else would
become too noisy.
However, it's not totally clear to me that this is the right thing to
do. Right now, I feel like there is a very obvious lexicographic
ordering that makes "finding" the right suggestion to activate
potentially easier than if the ranking mechanism is less clear.
(Technically, the dunder and sunder attributes are not sorted
lexicographically, but I'd put forward that most folks don't have an
intuitive understanding of where `_` ranks lexicographically with
respect to "regular" letters. Moreover, since dunder and sunder
attributes are all grouped together, I think the ordering here ends up
being very obvious after even a quick glance.)
## Summary
Setting `TY_MEMORY_REPORT=full` will generate and print a memory usage
report to the CLI after a `ty check` run:
```
=======SALSA STRUCTS=======
`Definition` metadata=7.24MB fields=17.38MB count=181062
`Expression` metadata=4.45MB fields=5.94MB count=92804
`member_lookup_with_policy_::interned_arguments` metadata=1.97MB fields=2.25MB count=35176
...
=======SALSA QUERIES=======
`File -> ty_python_semantic::semantic_index::SemanticIndex`
metadata=11.46MB fields=88.86MB count=1638
`Definition -> ty_python_semantic::types::infer::TypeInference`
metadata=24.52MB fields=86.68MB count=146018
`File -> ruff_db::parsed::ParsedModule`
metadata=0.12MB fields=69.06MB count=1642
...
=======SALSA SUMMARY=======
TOTAL MEMORY USAGE: 577.61MB
struct metadata = 29.00MB
struct fields = 35.68MB
memo metadata = 103.87MB
memo fields = 409.06MB
```
Eventually, we should integrate these numbers into CI in some form. The
one limitation currently is that heap allocations in salsa structs (e.g.
interned values) are not tracked, but memoized values should have full
coverage. We may also want a peak memory usage counter (that accounts
for non-salsa memory), but that is relatively simple to profile manually
(e.g. `time -v ty check`) and would require a compile-time option to
avoid runtime overhead.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
This PR also supresses the fix if the assignment expression target
shadows one of the lambda's parameters.
Fixes#18675
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Add regression tests.
<!-- How was it tested? -->
## Summary
Part of #15584
This PR adds a fix safety section to [fast-api-non-annotated-dependency
(FAST002)](https://docs.astral.sh/ruff/rules/fast-api-non-annotated-dependency/#fast-api-non-annotated-dependency-fast002).
It also re-words the availability section since I found it confusing.
The lint/fix was added in #11579 as always unsafe.
No reasoning is given in the original PR/code as to why this was chosen.
Example of why the fix is unsafe:
https://play.ruff.rs/3bd0566e-1ef6-4cec-ae34-3b07cd308155
```py
from fastapi import Depends, FastAPI, Query
app = FastAPI()
# Fix will remove the parameter default value
@app.get("/items/")
async def read_items(commons: dict = Depends(common_parameters)):
return commons
# Fix will delete comment and change default parameter value
@app.get("/items/")
async def read_items_1(q: str = Query( # This comment will be deleted
default="rick")):
return q
```
After fixing both instances of `FAST002`:
```py
from fastapi import Depends, FastAPI, Query
from typing import Annotated
app = FastAPI()
# Fix will remove the parameter default value
@app.get("/items/")
async def read_items(commons: Annotated[dict, Depends(common_parameters)]):
return commons
# Fix will delete comment and change default parameter value
@app.get("/items/")
async def read_items_1(q: Annotated[str, Query()] = "rick"):
return q
```
It turns out that astral-sh/ty#18692 also fixedastral-sh/ty#203. This
PR adds a regression test for it. (Locally, I "unfixed" the bug and
confirmed that this is actually a regression test.)
Fixesastral-sh/ty#203
It turns out that `annotate-snippets` doesn't do a great job of
consistently handling tabs. The intent of the implementation is clearly
to expand tabs into 4 ASCII whitespace characters. But there are a few
places where the column computation wasn't taking this expansion into
account. In particular, the `unicode-width` crate returns `None` for a
`\t` input, and `annotate-snippets` would in turn treat this as either
zero columns or one column. Both are wrong.
In patching this, it caused one of the existing `annotate-snippets`
tests to fail. I spent a fair bit of time on it trying to fix it before
coming to the conclusion that the test itself was wrong. In particular,
the annotation ranges are 4 bytes off. However, when the range was
wrong, the buggy code was rendering the example as intended since `\t`
characters were treated as taking up zero columns of space. Now that
they are correctly computed as taking up 4 columns of space, the offsets
of the test needed to be adjusted.
Fixes#670
## Summary
Adds a new micro-benchmark as a regression test for
https://github.com/astral-sh/ty/issues/627.
## Test Plan
Ran the benchmark on the parent commit of
89d915a1e3,
and verified that it took > 1s, while it takes ~10 ms after the fix.
## Summary
Format conflicting declared types as
```
`str`, `int` and `bytes`
```
Thanks to @AlexWaygood for the initial draft.
@dcreager, looking forward to your one-character follow-up PR.
## Summary
This PR includes a behavioral change to how we infer types for public
uses of symbols within a module. Where we would previously use the type
that a use at the end of the scope would see, we now consider all
reachable bindings and union the results:
```py
x = None
def f():
reveal_type(x) # previously `Unknown | Literal[1]`, now `Unknown | None | Literal[1]`
f()
x = 1
f()
```
This helps especially in cases where the the end of the scope is not
reachable:
```py
def outer(x: int):
def inner():
reveal_type(x) # previously `Unknown`, now `int`
raise ValueError
```
This PR also proposes to skip the boundness analysis of public uses.
This is consistent with the "all reachable bindings" strategy, because
the implicit `x = <unbound>` binding is also always reachable, and we
would have to emit "possibly-unresolved" diagnostics for every public
use otherwise. Changing this behavior allows common use-cases like the
following to type check without any errors:
```py
def outer(flag: bool):
if flag:
x = 1
def inner():
print(x) # previously: possibly-unresolved-reference, now: no error
```
closes https://github.com/astral-sh/ty/issues/210
closes https://github.com/astral-sh/ty/issues/607
closes https://github.com/astral-sh/ty/issues/699
## Follow up
It is now possible to resolve the following TODO, but I would like to do
that as a follow-up, because it requires some changes to how we treat
implicit attribute assignments, which could result in ecosystem changes
that I'd like to see separately.
315fb0f3da/crates/ty_python_semantic/src/semantic_index/builder.rs (L1095-L1117)
## Ecosystem analysis
[**Full report**](https://shark.fish/diff-public-types.html)
* This change obviously removes a lot of `possibly-unresolved-reference`
diagnostics (7818) because we do not analyze boundness for public uses
of symbols inside modules anymore.
* As the primary goal here, this change also removes a lot of
false-positive `unresolved-reference` diagnostics (231) in scenarios
like this:
```py
def _(flag: bool):
if flag:
x = 1
def inner():
x
raise
```
* This change also introduces some new false positives for cases like:
```py
def _():
x = None
x = "test"
def inner():
x.upper() # Attribute `upper` on type `Unknown | None | Literal["test"]`
is possibly unbound
```
We have test cases for these situations and it's plausible that we can
improve this in a follow-up.
## Test Plan
New Markdown tests
## Summary
This function is huge, and hugely indented. This PR breaks most of it
out into two helper functions: `KnownFunction::check_call()` and
`KnownClass::check_call`.
My immediate motivation is that we need to add yet more special cases to
this function in order to properly handle `tuple` instantiations and
instantiations of tuple subclasses. But I really don't relish the
thought of doing that with the function's current structure 😆
## Test Plan
Existing tests all pass. No new ones are added; this is a pure refactor
that should have no functional change.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Here's the part that was split out of #18906. I wanted to move these
into the rule files since the rest of the rules in
`deferred_scope`/`statement` have that same structure of implementations
being in the rule definition file. It also resolves the dilemma of where
to put the comment, at least for these rules.
## Test Plan
<!-- How was it tested? -->
N/A, no test/functionality affected
Summary
--
Closes#18849 by adding a `## Known issues` section describing the
potential performance issues when fixing nested iterables. I also
deleted the comment check since the fix is already unsafe and added a
note to the `## Fix safety` docs.
Test Plan
--
Existing tests, updated to allow a fix when comments are present since
the fix is already unsafe.
Summary
--
This PR resolves the easiest part of
https://github.com/astral-sh/ruff/issues/18502 by adding an autofix that
just adds
`from __future__ import annotations` at the top of the file, in the same
way
as FA102, which already has an identical unsafe fix.
Test Plan
--
Existing snapshots, updated to add the fixes.
## Summary
Add type narrowing inside comprehensions:
```py
def _(xs: list[int | None]):
[reveal_type(x) for x in xs if x is not None] # revealed: int
```
closes https://github.com/astral-sh/ty/issues/680
## Test Plan
* New Markdown tests
* Made sure the example from https://github.com/astral-sh/ty/issues/680
now checks without errors
* Made sure that all removed ecosystem diagnostics were actually false
positives
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
From @ntBre
https://github.com/astral-sh/ruff/pull/18906#discussion_r2162843366 :
> This could be a good target for a follow-up PR, but we could fold
these `if checker.is_rule_enabled { checker.report_diagnostic` checks
into calls to `checker.report_diagnostic_if_enabled`. I didn't notice
these when adding that method.
>
> Also, the docs on `Checker::report_diagnostic_if_enabled` and
`LintContext::report_diagnostic_if_enabled` are outdated now that the
`Rule` conversion is basically free 😅
>
> No pressure to take on this refactor, just an idea if you're
interested!
This PR folds those calls. I also updated the doc comments by copying
from `report_diagnostic`.
Note: It seems odd to me that the doc comment for `Checker` says
`Diagnostic` while `LintContext` says `OldDiagnostic`, not sure if that
needs a bigger docs change to fix the inconsistency.
<details>
<summary>Python script to do the changes</summary>
This script assumes it is placed in the top level `ruff` directory (ie
next to `.git`/`crates`/`README.md`)
```py
import re
from copy import copy
from pathlib import Path
ruff_crates = Path(__file__).parent / "crates"
for path in ruff_crates.rglob("**/*.rs"):
with path.open(encoding="utf-8", newline="") as f:
original_content = f.read()
if "is_rule_enabled" not in original_content or "report_diagnostic" not in original_content:
continue
original_content_position = 0
changed_content = ""
for match in re.finditer(r"(?m)(?:^[ \n]*|(?<=(?P<else>else )))if[ \n]+checker[ \n]*\.is_rule_enabled\([ \n]*Rule::\w+[ \n]*\)[ \n]*{[ \n]*checker\.report_diagnostic\(", original_content):
# Content between last match and start of this one is unchanged
changed_content += original_content[original_content_position:match.start()]
# If this was an else if, a { needs to be added at the start
if match.group("else"):
changed_content += "{"
# This will result in bad formatting, but the precommit cargo format will handle it
changed_content += "checker.report_diagnostic_if_enabled("
# Depth tracking would fail if a string/comment included a { or }, but unlikely given the context
depth = 1
position = match.end()
while depth > 0:
if original_content[position] == "{":
depth += 1
if original_content[position] == "}":
depth -= 1
position += 1
# pos - 1 is the closing }
changed_content += original_content[match.end():position - 1]
# If this was an else if, a } needs to be added at the end
if match.group("else"):
changed_content += "}"
# Skip the closing }
original_content_position = position
if original_content[original_content_position] == "\n":
# If the } is followed by a \n, also skip it for better formatting
original_content_position += 1
# Add remaining content between last match and file end
changed_content += original_content[original_content_position:]
with path.open("w", encoding="utf-8", newline="") as f:
f.write(changed_content)
```
</details>
## Test Plan
<!-- How was it tested? -->
N/A, no tests/functionality affected.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
While making some of my other changes, I noticed some of the lints were
missing comments with their lint code/had the wrong numbered lint code.
These comments are super useful since they allow for very easily and
quickly finding the source code of a lint, so I decided to try and
normalize them.
Most of them were fairly straightforward, just adding a doc
comment/comment in the appropriate place.
I decided to make all of the `Pylint` rules have the `PL` prefix.
Previously it was split between no prefix and having prefix, but I
decided to normalize to with prefix since that's what's in the docs, and
the with prefix will show up on no prefix searches, while the reverse is
not true.
I also ran into a lot of rules with implementations in "non-standard"
places (where "standard" means inside a file matching the glob
`crates/ruff_linter/rules/*/rules/**/*.rs` and/or the same rule file
where the rule `struct`/`ViolationMetadata` is defined).
I decided to move all the implementations out of
`crates/ruff_linter/src/checkers/ast/analyze/deferred_scopes.rs` and
into their own files, since that is what the rest of the rules in
`deferred_scopes.rs` did, and those were just the outliers.
There were several rules which I did not end up moving, which you can
see as the extra paths I had to add to my python code besides the
"standard" glob. These rules are generally the error-type rules that
just wrap an error from the parser, and have very small
implementations/are very tightly linked to the module they are in, and
generally every rule of that type was implemented in module instead of
in the "standard" place.
Resolving that requires answering a question I don't think I'm equipped
to handle: Is the point of these comments to give quick access to the
rule definition/docs, or the rule implementation? For all the rules with
implementations in the "standard" location this isn't a problem, as they
are the same, but it is an issue for all of these error type rules. In
the end I chose to leave the implementations where they were, but I'm
not sure if that was the right choice.
<details>
<summary>Python script I wrote to find missing comments</summary>
This script assumes it is placed in the top level `ruff` directory (ie
next to `.git`/`crates`/`README.md`)
```py
import re
from copy import copy
from pathlib import Path
linter_to_code_prefix = {
"Airflow": "AIR",
"Eradicate": "ERA",
"FastApi": "FAST",
"Flake82020": "YTT",
"Flake8Annotations": "ANN",
"Flake8Async": "ASYNC",
"Flake8Bandit": "S",
"Flake8BlindExcept": "BLE",
"Flake8BooleanTrap": "FBT",
"Flake8Bugbear": "B",
"Flake8Builtins": "A",
"Flake8Commas": "COM",
"Flake8Comprehensions": "C4",
"Flake8Copyright": "CPY",
"Flake8Datetimez": "DTZ",
"Flake8Debugger": "T10",
"Flake8Django": "DJ",
"Flake8ErrMsg": "EM",
"Flake8Executable": "EXE",
"Flake8Fixme": "FIX",
"Flake8FutureAnnotations": "FA",
"Flake8GetText": "INT",
"Flake8ImplicitStrConcat": "ISC",
"Flake8ImportConventions": "ICN",
"Flake8Logging": "LOG",
"Flake8LoggingFormat": "G",
"Flake8NoPep420": "INP",
"Flake8Pie": "PIE",
"Flake8Print": "T20",
"Flake8Pyi": "PYI",
"Flake8PytestStyle": "PT",
"Flake8Quotes": "Q",
"Flake8Raise": "RSE",
"Flake8Return": "RET",
"Flake8Self": "SLF",
"Flake8Simplify": "SIM",
"Flake8Slots": "SLOT",
"Flake8TidyImports": "TID",
"Flake8Todos": "TD",
"Flake8TypeChecking": "TC",
"Flake8UnusedArguments": "ARG",
"Flake8UsePathlib": "PTH",
"Flynt": "FLY",
"Isort": "I",
"McCabe": "C90",
"Numpy": "NPY",
"PandasVet": "PD",
"PEP8Naming": "N",
"Perflint": "PERF",
"Pycodestyle": "",
"Pydoclint": "DOC",
"Pydocstyle": "D",
"Pyflakes": "F",
"PygrepHooks": "PGH",
"Pylint": "PL",
"Pyupgrade": "UP",
"Refurb": "FURB",
"Ruff": "RUF",
"Tryceratops": "TRY",
}
ruff = Path(__file__).parent / "crates"
ruff_linter = ruff / "ruff_linter" / "src"
code_to_rule_name = {}
with open(ruff_linter / "codes.rs") as codes_file:
for linter, code, rule_name in re.findall(
# The (?<! skips ruff test rules
# Only Preview|Stable rules are checked
r"(?<!#\[cfg\(any\(feature = \"test-rules\", test\)\)\]\n) \((\w+), \"(\w+)\"\) => \(RuleGroup::(?:Preview|Stable), [\w:]+::(\w+)\)",
codes_file.read(),
):
code_to_rule_name[linter_to_code_prefix[linter] + code] = (rule_name, [])
ruff_linter_rules = ruff_linter / "rules"
for rule_file_path in [
*ruff_linter_rules.rglob("*/rules/**/*.rs"),
ruff / "ruff_python_parser" / "src" / "semantic_errors.rs",
ruff_linter / "pyproject_toml.rs",
ruff_linter / "checkers" / "noqa.rs",
ruff_linter / "checkers" / "ast" / "mod.rs",
ruff_linter / "checkers" / "ast" / "analyze" / "unresolved_references.rs",
ruff_linter / "checkers" / "ast" / "analyze" / "expression.rs",
ruff_linter / "checkers" / "ast" / "analyze" / "statement.rs",
]:
with open(rule_file_path, encoding="utf-8") as f:
rule_file_content = f.read()
for code, (rule, _) in copy(code_to_rule_name).items():
if rule in rule_file_content:
if f"// {code}" in rule_file_content or f", {code}" in rule_file_content:
del code_to_rule_name[code]
else:
code_to_rule_name[code][1].append(rule_file_path)
for code, rule in code_to_rule_name.items():
print(code, rule[0])
for path in rule[1]:
print(path)
```
</details>
## Test Plan
<!-- How was it tested? -->
N/A, no tests/functionality affected.
## Summary
Having a recursive type method to check whether a type is fully static
is inefficient, unnecessary, and makes us overly strict about subtyping
relations.
It's inefficient because we end up re-walking the same types many times
to check for fully-static-ness.
It's unnecessary because we can check relations involving the dynamic
type appropriately, depending whether the relation is subtyping or
assignability.
We use the subtyping relation to simplify unions and intersections. We
can usefully consider that `S <: T` for gradual types also, as long as
it remains true that `S | T` is equivalent to `T` and `S & T` is
equivalent to `S`.
One conservative definition (implemented here) that satisfies this
requirement is that we consider `S <: T` if, for every possible pair of
materializations `S'` and `T'`, `S' <: T'`. Or put differently the top
materialization of `S` (`S+` -- the union of all possible
materializations of `S`) is a subtype of the bottom materialization of
`T` (`T-` -- the intersection of all possible materializations of `T`).
In the most basic cases we can usefully say that `Any <: object` and
that `Never <: Any`, and we can handle more complex cases inductively
from there.
This definition of subtyping for gradual subtypes is not reflexive
(`Any` is not a subtype of `Any`).
As a corollary, we also remove `is_gradual_equivalent_to` --
`is_equivalent_to` now has the meaning that `is_gradual_equivalent_to`
used to have. If necessary, we could restore an
`is_fully_static_equivalent_to` or similar (which would not do an
`is_fully_static` pre-check of the types, but would instead pass a
relation-kind enum down through a recursive equivalence check, similar
to `has_relation_to`), but so far this doesn't appear to be necessary.
Credit to @JelleZijlstra for the observation that `is_fully_static` is
unnecessary and overly restrictive on subtyping.
There is another possible definition of gradual subtyping: instead of
requiring that `S+ <: T-`, we could instead require that `S+ <: T+` and
`S- <: T-`. In other words, instead of requiring all materializations of
`S` to be a subtype of every materialization of `T`, we just require
that every materialization of `S` be a subtype of _some_ materialization
of `T`, and that every materialization of `T` be a supertype of some
materialization of `S`. This definition also preserves the core
invariant that `S <: T` implies that `S | T = T` and `S & T = S`, and it
restores reflexivity: under this definition, `Any` is a subtype of
`Any`, and for any equivalent types `S` and `T`, `S <: T` and `T <: S`.
But unfortunately, this definition breaks transitivity of subtyping,
because nominal subclasses in Python use assignability ("consistent
subtyping") to define acceptable overrides. This means that we may have
a class `A` with `def method(self) -> Any` and a subtype `B(A)` with
`def method(self) -> int`, since `int` is assignable to `Any`. This
means that if we have a protocol `P` with `def method(self) -> Any`, we
would have `B <: A` (from nominal subtyping) and `A <: P` (`Any` is a
subtype of `Any`), but not `B <: P` (`int` is not a subtype of `Any`).
Breaking transitivity of subtyping is not tenable, so we don't use this
definition of subtyping.
## Test Plan
Existing tests (modified in some cases to account for updated
semantics.)
Stable property tests pass at a million iterations:
`QUICKCHECK_TESTS=1000000 cargo test -p ty_python_semantic -- --ignored
types::property_tests::stable`
### Changes to property test type generation
Since we no longer have a method of categorizing built types as
fully-static or not-fully-static, I had to add a previously-discussed
feature to the property tests so that some tests can build types that
are known by construction to be fully static, because there are still
properties that only apply to fully-static types (for example,
reflexiveness of subtyping.)
## Changes to handling of `*args, **kwargs` signatures
This PR "discovered" that, once we allow non-fully-static types to
participate in subtyping under the above definitions, `(*args: Any,
**kwargs: Any) -> Any` is now a subtype of `() -> object`. This is true,
if we take a literal interpretation of the former signature: all
materializations of the parameters `*args: Any, **kwargs: Any` can
accept zero arguments, making the former signature a subtype of the
latter. But the spec actually says that `*args: Any, **kwargs: Any`
should be interpreted as equivalent to `...`, and that makes a
difference here: `(...) -> Any` is not a subtype of `() -> object`,
because (unlike a literal reading of `(*args: Any, **kwargs: Any)`),
`...` can materialize to _any_ signature, including a signature with
required positional arguments.
This matters for this PR because it makes the "any two types are both
assignable to their union" property test fail if we don't implement the
equivalence to `...`. Because `FunctionType.__call__` has the signature
`(*args: Any, **kwargs: Any) -> Any`, and if we take that at face value
it's a subtype of `() -> object`, making `FunctionType` a subtype of `()
-> object)` -- but then a function with a required argument is also a
subtype of `FunctionType`, but not a subtype of `() -> object`. So I
went ahead and implemented the equivalence to `...` in this PR.
## Ecosystem analysis
* Most of the ecosystem report are cases of improved union/intersection
simplification. For example, we can now simplify a union like `bool |
(bool & Unknown) | Unknown` to simply `bool | Unknown`, because we can
now observe that every possible materialization of `bool & Unknown` is
still a subtype of `bool` (whereas before we would set aside `bool &
Unknown` as a not-fully-static type.) This is clearly an improvement.
* The `possibly-unresolved-reference` errors in sockeye, pymongo,
ignite, scrapy and others are true positives for conditional imports
that were formerly silenced by bogus conflicting-declarations (which we
currently don't issue a diagnostic for), because we considered two
different declarations of `Unknown` to be conflicting (we used
`is_equivalent_to` not `is_gradual_equivalent_to`). In this PR that
distinction disappears and all equivalence is gradual, so a declaration
of `Unknown` no longer conflicts with a declaration of `Unknown`, which
then results in us surfacing the possibly-unbound error.
* We will now issue "redundant cast" for casting from a typevar with a
gradual bound to the same typevar (the hydra-zen diagnostic). This seems
like an improvement.
* The new diagnostics in bandersnatch are interesting. For some reason
primer in CI seems to be checking bandersnatch on Python 3.10 (not yet
sure why; this doesn't happen when I run it locally). But bandersnatch
uses `enum.StrEnum`, which doesn't exist on 3.10. That makes the `class
SimpleDigest(StrEnum)` a class that inherits from `Unknown` (and
bypasses our current TODO handling for accessing attributes on enum
classes, since we don't recognize it as an enum class at all). This PR
improves our understanding of assignability to classes that inherit from
`Any` / `Unknown`, and we now recognize that a string literal is not
assignable to a class inheriting `Any` or `Unknown`.
Add property test generators for the new variable-length tuples. This
covers homogeneous tuples as well.
The property tests did their job! This identified several fixes we
needed to make to various type property methods.
cf https://github.com/astral-sh/ruff/pull/18600#issuecomment-2993764471
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
This PR expands PGH005 to also check for AsyncMock methods in the same
vein. E.g., currently `assert mock.not_called` is linted. This PR adds
the corresponding async assertions `assert mock.not_awaited()`.
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
/closes #2331
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
update snapshots
<!-- How was it tested? -->
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
This commit does a small refactor to combine the file and
cursor offset into a single type. I think this makes it
clearer that even if there are multiple files in the cursor
test, this one in particular corresponds to the file that
contains the `<CURSOR>` marker.
This doesn't change any functionality of the cursor tests, but does
re-arrange the code a bit. Firstly, it's now in a builder. And secondly,
there's an API to add multiple files to the test (but exactly one must
have a `<CURSOR>` marker).
We achieve this by setting the "sort text" field of every completion.
Since we are trying to be smart about the order, we want the client to
respect our order.
Prior to this change, VS Code was re-sorting completions in
lexicographic order. This in turn resulted in dunder attributes
appearing before "normal" attributes.
## Summary
This PR removes the last two places we were using `NoqaCode::rule` in
`linter.rs` (see
https://github.com/astral-sh/ruff/pull/18391#discussion_r2154637329 and
https://github.com/astral-sh/ruff/pull/18391#discussion_r2154649726) by
checking whether fixes are actually desired before adding them to a
`DiagnosticGuard`. I implemented this by storing a `Violation`'s `Rule`
on the `DiagnosticGuard` so that we could check if it was enabled in the
embedded `LinterSettings` when trying to set a fix.
All of the corresponding `set_fix` methods on `OldDiagnostic` were now
unused (except in tests where I just set `.fix` directly), so I moved
these to the guard instead of keeping both sets.
The very last place where we were using `NoqaCode::rule` was in the
cache. I just reverted this to parsing the `Rule` from the name. I had
forgotten to update the comment there anyway. Hopefully this doesn't
cause too much of a perf hit.
In terms of binary size, we're back down almost to where `main` was two
days ago
(https://github.com/astral-sh/ruff/pull/18391#discussion_r2155034320):
```
41,559,344 bytes for main 2 days ago
41,669,840 bytes for #18391
41,653,760 bytes for main now (after #18391 merged)
41,602,224 bytes for this branch
```
Only 43 kb up, but that shouldn't all be me this time :)
## Test Plan
Existing tests and benchmarks on this PR
## Summary
Ref:
https://github.com/astral-sh/ruff/issues/14820#issuecomment-2996690681
This PR fixes a bug where virtual paths or any paths that doesn't exists
on the file system weren't being considered for checking inclusion /
exclusion. This was because the logic used `file_path` which returns
`None` for those path. This PR fixes that by using the
`virtual_file_path` method that returns a `Path` corresponding to the
actual file on disk or any kind of virtual path.
This should ideally just fix the above linked issue by way of excluding
the documents representing the interactive window because they aren't in
the inclusion set. It failed only on Windows previously because the file
path construction would fail and then Ruff would default to including
all the files.
## Test Plan
On my machine, the `.interactive` paths are always excluded so I'm using
the inclusion set instead:
```json
{
"ruff.nativeServer": "on",
"ruff.path": ["/Users/dhruv/work/astral/ruff/target/debug/ruff"],
"ruff.configuration": {
"extend-include": ["*.interactive"]
}
}
```
The diagnostics are shown for both the file paths and the interactive
window:
<img width="1727" alt="Screenshot 2025-06-24 at 14 56 40"
src="https://github.com/user-attachments/assets/d36af96a-777e-4367-8acf-4d9c9014d025"
/>
And, the logs:
```
2025-06-24 14:56:26.478275000 DEBUG notification{method="notebookDocument/didChange"}: Included path via `extend-include`: /Interactive-1.interactive
```
And, when using `ruff.exclude` via:
```json
{
"ruff.exclude": ["*.interactive"]
}
```
With logs:
```
2025-06-24 14:58:41.117743000 DEBUG notification{method="notebookDocument/didChange"}: Ignored path via `exclude`: /Interactive-1.interactive
```
## Summary
Previously, the checks for implicit attribute assignments didn't
properly account for method decorators. This PR fixes that by:
- Adding a decorator check in `implicit_instance_attribute`. This allows
it to filter out methods with mismatching decorators when analyzing
attribute assignments.
- Adding attribute search for implicit class attributes: if an attribute
can't be found directly in the class body, the
`ClassLiteral::own_class_member` function will now search in
classmethods.
- Adding `staticmethod`: it has been added into `KnownClass` and
together with the new decorator check, it will no longer expose
attributes when the assignment target name is the same as the first
method name.
If accepted, it should fix https://github.com/astral-sh/ty/issues/205
and https://github.com/astral-sh/ty/issues/207.
## Test Plan
This is tested with existing mdtest suites and is able to get most of
the TODO marks for implicit assignments in classmethods and
staticmethods removed.
However, there's one specific test case I failed to figure out how to
correctly resolve:
b279508bdc/crates/ty_python_semantic/resources/mdtest/attributes.md?plain=1#L754-L755
I tried to add `instance_member().is_unbound()` check in this [else
branch](b279508bdc/crates/ty_python_semantic/src/types/infer.rs (L3299-L3301))
but it causes tests with class attributes defined in class body to fail.
While it's possible to implicitly add `ClassVar` to qualifiers to make
this assignment fail and keep everything else passing, it doesn't feel
like the right solution.
## Summary
This PR fixesastral-sh/ty#185 by avoiding to infer the value expression
for an unpacking.
This is done simply by only inferring the value expression in a
non-unpacking branch for assignment statement, for statement, with
statement and comprehensions.
This is a simpler alternative to
https://github.com/astral-sh/ruff/pull/18890 which I only realized in
hindsight! Ideally, the solution would to consider the "unpack" as it's
own region and do all of the inference of every expressions involved in
an unpacking inside the unpack query and then merge the results in the
outer query. This would require access to the `Unpack` ingredient which
is stored on the `Definition`. And, this would require create the said
`Definition`s for all attributes and subscript expressions. It does
simplify the target inference logic by streamlining it into a single
`infer_target` method instead of the `infer_target`/`infer_target_impl`
split.
Additionally, #18890 also solves a couple of TODOs around raising errors
around attribute / subscript assignment.
## Test Plan
Update the existing test, go through a couple of ecosystem diagnostic.
## Summary
Resolves#18165
Added pattern `["sys", "version_info", "major"]` to the existing matches
for `sys.version_info` to ensure consistent handling of both the base
object and its major version attribute.
## Test Plan
`cargo nextest run` and `cargo insta test`
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
/closes #17424
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
## Summary
Note this modifies the diagnostics a bit. Previously performing
subscript access on something like `NotSubscriptable1 |
NotSubscriptable2` would report the full type as not being
subscriptable:
```
[non-subscriptable] "Cannot subscript object of type `NotSubscriptable1 | NotSubscriptable2` with no `__getitem__` method"
```
Now each erroneous constituent has a separate error:
```
[non-subscriptable] "Cannot subscript object of type `NotSubscriptable2` with no `__getitem__` method"
[non-subscriptable] "Cannot subscript object of type `NotSubscriptable1` with no `__getitem__` method"
```
Closes https://github.com/astral-sh/ty/issues/625
## Test Plan
mdtest
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
There were two main challenges in this PR.
The first was mostly just figuring out how to get the symbols
corresponding to `module`. It turns out that we do this in a couple
of places in ty already, but through different means. In one approach,
we use [`exported_names`]. In another approach, we get a `Type`
corresponding to the module. We take the latter approach here, which is
consistent with how we do completions elsewhere. (I looked into
factoring this logic out into its own function, but it ended up being
pretty constrained. e.g., There's only one other place where we want to
go from `ast::StmtImportFrom` to a module `Type`, and that code also
wants the module name.)
The second challenge was recognizing the `from module import <CURSOR>`
pattern in the code. I initially started with some fixed token patterns
to get a proof of concept working. But I ended up switching to mini
state machine over tokens. I looked at the parser for `StmtImportFrom`
to determine what kinds of tokens we can expect.
[`exported_names`]:
23a3b6ef23/crates/ty_python_semantic/src/semantic_index/re_exports.rs (L47)
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
The fix would create a syntax error if there wasn't a space between the
`in` keyword and the following expression.
For example:
```python
for country, stars in(zip)(flag_stars.keys(), flag_stars.values()):...
```
I also noticed that the tests for `SIM911` were note being run, so I
fixed that.
Fixes#18776
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Add regression test
<!-- How was it tested? -->
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
This PR fixes `PLC2801` autofix creating a syntax error due to lack of
padding if it is directly after a keyword.
Fixes https://github.com/astral-sh/ruff/issues/18813
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Add regression test
<!-- How was it tested? -->
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #15584
This adds a `Fix safety` section to [useless-object-inheritance
(UP004)](https://docs.astral.sh/ruff/rules/useless-object-inheritance/#useless-object-inheritance-up004)
I could not track down the original PR as this rule is so old it has
gone through several large ruff refactors.
No reasoning is given on the unsafety in the PR/code.
The unsafety is determined here:
f24e650dfd/crates/ruff_linter/src/rules/pyupgrade/rules/useless_class_metaclass_type.rs (L76-L80)
Unsafe fix demonstration:
[playground](https://play.ruff.rs/12b24eb4-d7a5-4ae0-93bb-492d64967ae3)
```py
class A( # will be deleted
object
):
...
```
## Test Plan
<!-- How was it tested? -->
N/A, no tests/functionality affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #15584
This adds a `Fix safety` section to [unnecessary-future-import
(UP010)](https://docs.astral.sh/ruff/rules/unnecessary-future-import/#unnecessary-future-import-up010)
The unsafety is determined here:
d9266284df/crates/ruff_linter/src/rules/pyupgrade/rules/unnecessary_future_import.rs (L128-L132)
Unsafe code example:
[playground](https://play.ruff.rs/c07d8c41-9ab8-4b86-805b-8cf482d450d9)
```py
from __future__ import (print_function,# ...
__annotations__) # ...
```
Edit: It looks like there was already a PR for this, #17490, but I
missed it since they said `UP029` instead of `UP010` :/
## Test Plan
<!-- How was it tested? -->
N/A, no tests/functionality affected
## Summary
Fixes `analyze.direction` to use kebab-case for the variant names.
Fixes https://github.com/astral-sh/ruff/issues/18887
## Test Plan
Created a `ruff.toml` and tested that both `dependents` and `Dependents`
were accepted
## Summary
As far as I can tell, the two existing tests did the exact same thing.
Remove the redundant test, and add tests for all combinations of
declared/not-declared and local/"public" use of the name.
Proposing this as a separate PR before the behavior might change via
https://github.com/astral-sh/ruff/pull/18750
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
I've also found another bug while fixing this, where the diagnostic
would not trigger if the `len` call argument variable was shadowed. This
fixed a few false negatives in the test cases.
Example:
```python
fruits = []
fruits = []
if len(fruits): # comment
...
```
Fixes#18811Fixes#18812
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Add regression test
<!-- How was it tested? -->
---------
Co-authored-by: Charlie Marsh <crmarsh416@gmail.com>
## Summary
The code in the `Variable` branch of
`VariableLengthTupleSpec::has_relation_to` made the incorrect assumption
that if you zip two possibly-different-length iterators together and
iterate over the resulting zip iterator, the original two iterators will
only have their common elements consumed. But in fact, the zip iterator
detects that it is done when it receives a `None` from one iterator and
`Some()` element from the other iterator, which means that it consumes
one additional element from the longer iterator. This meant that we
failed to detect mismatched types on this extra consumed element,
because we never compared it to the variable type of the other tuple.
Use `zip_longest` from itertools as an alternative, which allows us to
combine all the handling into just two `zip_longest`, one for prefixes
and one for suffixes.
Marking this PR internal since it fixes a bug in a commit that wasn't
released yet.
## Test Plan
Added mdtests that failed before this fix and pass after it.
## Summary
Fixes https://github.com/astral-sh/ty/issues/640. If a user passes
`--python=<some-virtual-environment>/bin/python`, we must avoid
canonicalizing the path until we've traversed upwards to find the
`sys.prefix` directory (`<some-virtual-environment>`). On Unix systems,
`<sys.prefix>/bin/python` is often a symlink to a system interpreter; if
we resolve the symlink too easily then we'll add the system
interpreter's `site-packages` directory as a search path rather than the
virtual environment's directory.
## Test Plan
I added an integration test to
`crates/ty/tests/cli/python_environment.rs` which fails on `main`. I
also manually tested locally that running `cargo run -p ty check foo.py
--python=.venv/bin/python -vv` now prints this log to the terminal
```
2025-06-20 18:35:24.57702 DEBUG Resolved site-packages directories for this virtual environment are: SitePackagesPaths({"/Users/alexw/dev/ruff/.venv/lib/python3.13/site-packages"})
```
Whereas it previously resolved `site-packages` to my system
intallation's `site-packages` directory
We already had support for homogeneous tuples (`tuple[int, ...]`). This
PR extends this to also support mixed tuples (`tuple[str, str,
*tuple[int, ...], str str]`).
A mixed tuple consists of a fixed-length (possibly empty) prefix and
suffix, and a variable-length portion in the middle. Every element of
the variable-length portion must be of the same type. A homogeneous
tuple is then just a mixed tuple with an empty prefix and suffix.
The new data representation uses different Rust types for a fixed-length
(aka heterogeneous) tuple. Another option would have been to use the
`VariableLengthTuple` representation for all tuples, and to wrap the
"variable + suffix" portion in an `Option`. I don't think that would
simplify the method implementations much, though, since we would still
have a 2×2 case analysis for most of them.
One wrinkle is that the definition of the `tuple` class in the typeshed
has a single typevar, and canonically represents a homogeneous tuple.
When getting the class of a tuple instance, that means that we have to
summarize our detailed mixed tuple type information into its
"homogeneous supertype". (We were already doing this for heterogeneous
types.)
A similar thing happens when concatenating two mixed tuples: the
variable-length portion and suffix of the LHS, and the prefix and
variable-length portion of the RHS, all get unioned into the
variable-length portion of the result. The LHS prefix and RHS suffix
carry through unchanged.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
A little bit of cleanup for consistency's sake: we move all the helpers
modules to a consistent location, and update the import paths when
needed. In the case of `refurb` there were two helpers modules, so we
just merged them.
Happy to revert the last commit if people are okay with `super::super` I
just thought it looked a little silly.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Fix `PYI041`'s fix turning `None | int | None | float` into `None | None
| float`, which raises a `TypeError` when executed.
The fix consists of making sure that the merged super-type is inserted
where the first type that is merged was before.
## Test Plan
Tests have been expanded with examples from the issue.
## Related Issue
Fixes https://github.com/astral-sh/ruff/issues/18298
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes https://github.com/astral-sh/ruff/issues/18726 by also checking if
its a literal and not only that it is truthy. See also the first comment
in the issue.
It would have been nice to check for inheritance of BaseException but I
figured that is not possible yet...
## Test Plan
I added a few tests for valid input to exc_info
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
I noticed this since my code for finding missing safety fix sections
flagged it, there is a missing `/` causing part of the new changes to be
a normal comment instead of a doc comment
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
## Summary
Ignore `__init__.py` files in `useless-import-alias` (PLC0414).
See discussion in #18365 and #6294: we want to allow redundant aliases
in `__init__.py` files, as they're almost always intentional explicit
re-exports.
Closes#18365Closes#6294
---------
Co-authored-by: Dylan <dylwil3@gmail.com>
## Summary
This PR avoids one of the three calls to `NoqaCode::rule` from
https://github.com/astral-sh/ruff/pull/18391 by applying per-file
ignores in the `LintContext`. To help with this, it also replaces all
direct uses of `LinterSettings.rules.enabled` with a
`LintContext::enabled` (or `Checker::enabled`, which defers to its
context) method. There are still some direct accesses to
`settings.rules`, but as far as I can tell these are not in a part of
the code where we can really access a `LintContext`. I believe all of
the code reachable from `check_path`, where the replaced per-file ignore
code was, should be converted to the new methods.
## Test Plan
Existing tests, with a single snapshot updated for RUF100, which I think
actually shows a more accurate diagnostic message now.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
I also noticed that the tests for SIM911 were note being run, so I fixed
that.
Fixes#18777
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Add regression test
<!-- How was it tested? -->
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
While reading the docs I noticed this paragraph on `PERF401`. It was
added in the same PR that the bug with `:=` was fixed, #15050, but don't
know why it was added. The fix should already take care of adding the
parenthesis, so having this paragraph in the docs is just confusing
since it sounds like the user has to do something.
## Test Plan
<!-- How was it tested? -->
N/A, no tests/functionality affected
## Summary
Fixes false positives (and incorrect autofixes) in `nested-min-max`
(`PLW3301`) when the outer `min`/`max` call only has a single argument.
Previously the rule would flatten:
```python
min(min([2, 3], [4, 1]))
```
into `min([2, 3], [4, 1])`, changing the semantics. The rule now skips
any nested call when the outer call has only one positional argument.
The pylint fixture and snapshot were updated accordingly.
## Test Plan
Ran Ruff against the updated `nested_min_max.py` fixture:
```shell
cargo run -p ruff -- check crates/ruff_linter/resources/test/fixtures/pylint/nested_min_max.py --no-cache --select=PLW3301 --preview
```
to verify that `min(min([2, 3], [4, 1]))` and `max(max([2, 4], [3, 1]))`
are no longer flagged. Updated the fixture and snapshot; all other
existing warnings remain unchanged. The code compiles and the unit tests
pass.
---
This PR was generated by an AI system in collaboration with maintainers:
@carljm, @ntBre
Fixes#16163
---------
Signed-off-by: Gene Parmesan Thomas <201852096+gopoto@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
Added `cls.__dict__.get('__annotations__')` check for Python 3.10+ and
Python < 3.10 with `typing-extensions` enabled.
Closes#17853
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Added `cls.__dict__.get('__annotations__')` check for Python 3.10+ and
Python < 3.10 with `typing-extensions` enabled.
## Test Plan
`cargo test`
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Add support for `@staticmethod`s. Overall, the changes are very similar
to #16305.
#18587 will be dependent on this PR for a potential fix of
https://github.com/astral-sh/ty/issues/207.
mypy_primer will look bad since the new code allows ty to check more
code.
## Test Plan
Added new markdown tests. Please comment if there's any missing tests
that I should add in, thank you.
## Summary
This PR resolves the way diagnostics are reported for an invalid call to
an overloaded function.
If any of the steps in the overload call evaluation algorithm yields a
matching overload but it's type checking that failed, the
`no-matching-overload` diagnostic is incorrect because there is a
matching overload, it's the arguments passed that are invalid as per the
signature. So, this PR improves that by surfacing the diagnostics on the
matching overload directly.
It also provides additional context, specifically the matching overload
where this error occurred and other non-matching overloads. Consider the
following example:
```py
from typing import overload
@overload
def f() -> None: ...
@overload
def f(x: int) -> int: ...
@overload
def f(x: int, y: int) -> int: ...
def f(x: int | None = None, y: int | None = None) -> int | None:
return None
f("a")
```
We get:
<img width="857" alt="Screenshot 2025-06-18 at 11 07 10"
src="https://github.com/user-attachments/assets/8dbcaf13-2a74-4661-aa94-1225c9402ea6"
/>
## Test Plan
Update test cases, resolve existing todos and validate the updated
snapshots.
## Summary
Part of [#111](https://github.com/astral-sh/ty/issues/111).
After this change, dataclasses with two or more `KW_ONLY` field will be
reported as invalid. The duplicate fields will simply be ignored when
computing `__init__`'s signature.
## Test Plan
Markdown tests.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #15584
This PR adds a fix safety section to `PIE794`
I could not track down when this rule was initially implemented/made
unsafe due how old it could be + multiple large refactors to `ruff`.
There is no comment/reasoning in the code given for the unsafety.
Here is a code example demonstrating why it should be unsafe, since
removing any of the assignments would change program behavior
[playground](https://play.ruff.rs/01004644-4259-4449-a581-5007cd59846a)
```py
class A:
x = 1
x = 2
print(x)
class B:
x = print(3)
x = print(4)
class C:
x = [1,2,3]
y = x
x = y[1]
```
## Test Plan
<!-- How was it tested? -->
N/A, no tests affected.
---------
Co-authored-by: Dylan <dylwil3@gmail.com>
Essentially this PR ensures that when we do fixes like this:
```diff
- t"{set(f(x) for x in foo)}"
+ t"{ {f(x) for x in foo} }"
```
we are correctly adding whitespace around the braces.
This logic is already in place for f-strings and just needed to be
generalized to interpolated strings.
Summary
--
This PR unifies the remaining differences between `OldDiagnostic` and
`Message` (`OldDiagnostic` was only missing an optional `noqa_offset`
field) and
replaces `Message` with `OldDiagnostic`.
The biggest functional difference is that the combined `OldDiagnostic`
kind no
longer implements `AsRule` for an infallible conversion to `Rule`. This
was
pretty easy to work around with `is_some_and` and `is_none_or` in the
few places
it was needed. In `LintContext::report_diagnostic_if_enabled` we can
just use
the new `Violation::rule` method, which takes care of most cases.
Most of the interesting changes are in [this
range](8156992540)
before I started renaming.
Test Plan
--
Existing tests
Future Work
--
I think it's time to start shifting some of these fields to the new
`Diagnostic`
kind. I believe we want `Fix` for sure, but I'm less sure about the
others. We
may want to keep a thin wrapper type here anyway to implement a `rule`
method,
so we could leave some of these fields on that too.
## Summary
This PR avoids the `Vec::retain` call in `check_tokens` by checking if
rules are enabled as their diagnostics are constructed.
2a425e43fd/crates/ruff_linter/src/checkers/tokens.rs (L174-L176)
Since `LintContext::report_diagnostic_if_enabled` required a
`LinterSettings`, I added a `settings` field to the context itself
instead of trying to pass it everywhere. This also turned
`LogicalLinesContext` into a trivial wrapper around `LintContext`, so I
just removed it in favor of using `LintContext` directly too.
The diff is a bit smaller with whitespace hidden since many blocks got
moved into something like this:
```rust
if let Some(mut diagnostic) = context.report_diagnostic.enabled(...) {
// old code
}
```
## Test Plan
Existing tests
When I try to grep CPython with `__super__` I get 0 results:
```
(.venv) ~/Desktop/cpython main ✔
» ag __super__ .
```
That's how we can understand that the naming is not the best.
This involved slightly more code changes than usual for a stabilization
- so maybe worth double-checking the logic!
I did verify by hand that the new stable behavior on the test fixture
matches the old preview behavior, even after the internal refactor.
Summary
--
Deprecates PD901 as part of #7710. I don't feel particularly strongly
about this one, though I have certainly used `df` as a dataframe name in
the past, just going through the open issues in the 0.12 milestone.
Test Plan
--
N/a
## Summary
- Stabilizes RUF058 (starmap-zip) rule by changing it from Preview to
Stable
- Migrates test cases from preview_rules to main rules function
- Updates snapshots accordingly and removes old preview snapshots
## Test plan
- ✅ Migrated tests from preview to main test function
- ✅ `make check` passes
- ✅ `make test` passes
- ✅ `make citest` passes (no leftover snapshots)
## Rule Documentation
- [Test
file](https://github.com/astral-sh/ruff/blob/main/crates/ruff_linter/src/rules/ruff/mod.rs#L103-L104)
- [Rule documentation](https://docs.astral.sh/ruff/rules/starmap-zip/)
## Summary
Stabilizes the UP049 rule (private-type-parameter) by moving it from
Preview to Stable.
UP049 detects and fixes the use of private type parameters (those with
leading underscores) in PEP 695 generic classes and functions.
## Test plan
- Verified that UP049 tests pass:
`crates/ruff_linter/src/rules/pyupgrade/mod.rs`
- Ran full test suite with `make test`
- Confirmed that no test migration was needed as UP049 was already in
the main `rules` test function
## Rule documentation
https://docs.astral.sh/ruff/rules/private-type-parameter/
Note that the preview behavior was not documented (shame on us!) so the
documentation was not modified.
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
This PR stabilizes the FURB162 rule by moving it from preview to stable
status for the 0.12.0 release.
## Summary
- **Rule**: FURB162 (`fromisoformat-replace-z`)
- **Purpose**: Detects unnecessary timezone replacement operations when
calling `datetime.fromisoformat()`
- **Change**: Move from `RuleGroup::Preview` to `RuleGroup::Stable` in
`codes.rs`
## Verification Links
- **Tests**:
[refurb/mod.rs](https://github.com/astral-sh/ruff/blob/main/crates/ruff_linter/src/rules/refurb/mod.rs#L54)
- Confirms FURB162 has only standard tests, no preview-specific test
cases
- **Documentation**:
https://docs.astral.sh/ruff/rules/fromisoformat-replace-z/ - Current
documentation shows preview status that will be automatically updated
This PR stabilizes the RUF053 rule by moving it from preview to stable
status for the 0.12.0 release.
## Summary
- **Rule**: RUF053 (`class-with-mixed-type-vars`)
- **Purpose**: Detects classes that have both PEP 695 type parameter
lists while also inheriting from `typing.Generic`
- **Change**: Move from `RuleGroup::Preview` to `RuleGroup::Stable` in
`codes.rs` and migrate preview tests to stable tests
## Verification Links
- **Tests**:
[ruff/mod.rs](https://github.com/astral-sh/ruff/blob/main/crates/ruff_linter/src/rules/ruff/mod.rs#L98)
- Shows RUF053 moved from preview_rules to main rules test function
- **Documentation**:
https://docs.astral.sh/ruff/rules/class-with-mixed-type-vars/ - Current
documentation shows preview status that will be automatically updated
Note that the preview behavior was not documented (shame on us!) so the
documentation was not modified.
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
Closes: astral-sh/ty#552
This PR adds support for step 5 of the overload call evaluation
algorithm which specifies:
> For all arguments, determine whether all possible materializations of
the argument’s type are
> assignable to the corresponding parameter type for each of the
remaining overloads. If so,
> eliminate all of the subsequent remaining overloads.
The algorithm works in two parts:
1. Find out the participating parameter indexes. These are the
parameters that aren't gradual equivalent to one or more parameter types
at the same index in other overloads.
2. Loop over each overload and check whether that would be the _final_
overload for the argument types i.e., the remaining overloads will never
be matched against these argument types
For step 1, the participating parameter indexes are computed by just
comparing whether all the parameter types at the corresponding index for
all the overloads are **gradual equivalent**.
The step 2 of the algorithm used is described in [this
comment](https://github.com/astral-sh/ty/issues/552#issuecomment-2969165421).
## Test Plan
Update the overload call tests.
## Summary
This PR closesastral-sh/ty#164.
This PR introduces a basic type narrowing mechanism for
attribute/subscript expressions.
Member accesses, int literal subscripts, string literal subscripts are
supported (same as mypy and pyright).
## Test Plan
New test cases are added to `mdtest/narrow/complex_target.md`.
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
PR adding support for it in the VS Code extension:
https://github.com/astral-sh/ty-vscode/pull/36
This PR adds support for `python.ty.disableLanguageServices` to the ty
language server by accepting this as server setting.
This has the same issue as https://github.com/astral-sh/ty/issues/282 in
that it only works when configured globally. Fixing that requires
support for multiple workspaces in the server itself.
I also went ahead and did a similar refactor as the Ruff server to use
"Options" and "Settings" to keep the code consistent although the
combine functionality doesn't exists yet because workspace settings
isn't supported in the ty server.
## Test Plan
Refer to https://github.com/astral-sh/ty-vscode/pull/36 for the test
demo.
## Summary
* Completely removes the concept of visibility constraints. Reachability
constraints are now used to model the static visibility of bindings and
declarations. Reachability constraints are *much* easier to reason about
/ work with, since they are applied at the beginning of a branch, and
not applied retroactively. Removing the duplication between visibility
and reachability constraints also leads to major code simplifications
[^1]. For an overview of how the new constraint system works, see the
updated doc comment in `reachability_constraints.rs`.
* Fixes a [control-flow modeling bug
(panic)](https://github.com/astral-sh/ty/issues/365) involving `break`
statements in loops
* Fixes a [bug where](https://github.com/astral-sh/ty/issues/624) where
`elif` branches would have wrong reachability constraints
* Fixes a [bug where](https://github.com/astral-sh/ty/issues/648) code
after infinite loops would not be considered unreachble
* Fixes a panic on the `pywin32` ecosystem project, which we should be
able to move to `good.txt` once this has been merged.
* Removes some false positives in unreachable code because we infer
`Never` more often, due to the fact that reachability constraints now
apply retroactively to *all* active bindings, not just to bindings
inside a branch.
* As one example, this removes the `division-by-zero` diagnostic from
https://github.com/astral-sh/ty/issues/443 because we now infer `Never`
for the divisor.
* Supersedes and includes similar test changes as
https://github.com/astral-sh/ruff/pull/18392
closes https://github.com/astral-sh/ty/issues/365
closes https://github.com/astral-sh/ty/issues/624
closes https://github.com/astral-sh/ty/issues/642
closes https://github.com/astral-sh/ty/issues/648
## Benchmarks
Benchmarks on black, pandas, and sympy showed that this is neither a
performance improvement, nor a regression.
## Test Plan
Regression tests for:
- [x] https://github.com/astral-sh/ty/issues/365
- [x] https://github.com/astral-sh/ty/issues/624
- [x] https://github.com/astral-sh/ty/issues/642
- [x] https://github.com/astral-sh/ty/issues/648
[^1]: I'm afraid this is something that @carljm advocated for since the
beginning, and I'm not sure anymore why we have never seriously tried
this before. So I suggest we do *not* attempt to do a historical deep
dive to find out exactly why this ever became so complicated, and just
enjoy the fact that we eventually arrived here.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes#18684
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Add regression test
<!-- How was it tested? -->
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
/closes #18639
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
update snapshots
<!-- How was it tested? -->
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
This PR aims to close#16605.
## Summary
This PR introduces a new rule (`RUF061`) that detects non-contextmanager
usage of `pytest.raises`, `pytest.warns`, and `pytest.deprecated_call`.
This pattern is discouraged and [was proposed in
flake8-pytest-style](https://github.com/m-burst/flake8-pytest-style/pull/332),
but the corresponding PR has been open for over a month without
activity.
Additionally, this PR provides an unsafe fix for simple cases where the
non-contextmanager form can be transformed into the context manager
form. Examples of supported patterns are listed in `RUF061_raises.py`,
`RUF061_warns.py`, and `RUF061_deprecated_call.py` test files.
The more complex case from the original issue (involving two separate
statements):
```python
excinfo = pytest.raises(ValueError, int, "hello")
assert excinfo.match("^invalid literal")
```
is getting fixed like this:
```python
with pytest.raises(ValueError) as excinfo:
int("hello")
assert excinfo.match("^invalid literal")
```
Putting match in the raises call requires multi-statement
transformation, which I am not sure how to implement.
## Test Plan
<!-- How was it tested? -->
New test files were added to cover various usages of the
non-contextmanager form of pytest.raises, warns, and deprecated_call.
Closes#18671
Note that while this has, I believe, always been invalid syntax, it was
reported as a different syntax error until Python 3.12:
Python 3.11:
```pycon
>>> x = 1
>>> f"{x! s}"
File "<stdin>", line 1
f"{x! s}"
^
SyntaxError: f-string: invalid conversion character: expected 's', 'r', or 'a'
```
Python 3.12:
```pycon
>>> x = 1
>>> f"{x! s}"
File "<stdin>", line 1
f"{x! s}"
^^^
SyntaxError: f-string: conversion type must come right after the exclamanation mark
```
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Solves #18257
## Test Plan
<!-- How was it tested? -->
Snapshots updated with some cases (negative, positive, mixed
annotations).
Summary
--
As @AlexWaygood noted on the 0.12 release blog post draft, the existing
example is a bit confusing. Either `**/*.py` or just `*.py`, as I went
with here, makes more sense, although the old version (`scripts/**.py`)
also worked when I tested it. However, this probably shouldn't be relied
upon since the [globset](https://docs.rs/globset/latest/globset/#syntax)
docs say:
> Using ** anywhere else is illegal
where "anywhere else" comes after the listing of the three valid
positions:
1. At the start of a pattern (`**/`)
2. At the end of a pattern (`/**`)
3. Or directly between two slashes (`/**/`)
I think the current version is luckily treated the same as a single `*`,
and the default globbing settings allow it to match subdirectories such
that the new example pattern will apply to the whole `scripts` tree in a
project like this:
```
.
├── README.md
├── pyproject.toml
├── scripts
│ ├── matching.py
│ └── sub
│ └── nested.py
└── src
└── main.py
```
Test Plan
--
Local testing of the new pattern, but the specifics of the pattern
aren't as important as having a more intuitive-looking/correct example.
Specifically, this PR reverts "Make completions an opt-in LSP feature
(#17921)",
corresponding to commit 51e2effd2d.
In practice, this means you don't need to opt into completions working
by enabling experimental features. i.e., I was able to remove this from
my LSP configuration:
```
"experimental": {
"completions": {
"enable": true
}
},
```
There's still a lot of work left to do to make completions awesome, but
I think it's in a state where it would be useful to get real user
feedback. It's also meaningfully using ty to provide completions that
use type information.
Ref astral-sh/ty#86
## Summary
Fixes https://github.com/astral-sh/ruff/issues/18628 by avoiding a fix
if there are "unknown" arguments, including any keyword arguments and
more than the expected 2 positional arguments.
I'm a bit on the fence here because it also seems reasonable to avoid a
diagnostic at all. Especially in the final test case I added (`not
my_dict.get(default=False)`), the hint suggesting to remove
`default=False` seems pretty misleading. At the same time, I guess the
diagnostic at least calls attention to the call site, which could help
to fix the missing argument bug too.
As I commented on the issue, I double-checked that keyword arguments are
invalid as far back as Python 3.8, even though the positional-only
marker was only added to the
[docs](https://docs.python.org/3.11/library/stdtypes.html#dict.get) in
3.12 (link is to 3.11, showing its absence).
## Test Plan
New tests derived from the bug report
## Stabilization
This was planned to be stabilized in 0.12, and the bug is less severe
than some others, but if there's nobody opposed, I will plan **not to
stabilize** this one for now.
## Summary
Part of [#117](https://github.com/astral-sh/ty/issues/117).
`TypeIs[]` is a special form that allows users to define their own
narrowing functions. Despite the syntax, `TypeIs` is not a generic and,
on its own, it is meaningless as a type.
[Officially](https://typing.python.org/en/latest/spec/narrowing.html#typeis),
a function annotated as returning a `TypeIs[T]` is a <i>type narrowing
function</i>, where `T` is called the <i>`TypeIs` return type</i>.
A `TypeIs[T]` may or may not be bound to a symbol. Only bound types have
narrowing effect:
```python
def f(v: object = object()) -> TypeIs[int]: ...
a: str = returns_str()
if reveal_type(f()): # Unbound: TypeIs[int]
reveal_type(a) # str
if reveal_type(f(a)): # Bound: TypeIs[a, int]
reveal_type(a) # str & int
```
Delayed usages of a bound type has no effect, however:
```python
b = f(a)
if b:
reveal_type(a) # str
```
A `TypeIs[T]` type:
* Is fully static when `T` is fully static.
* Is a singleton/single-valued when it is bound.
* Has exactly two runtime inhabitants when it is unbound: `True` and
`False`.
In other words, an unbound type have ambiguous truthiness.
It is possible to infer more precise truthiness for bound types;
however, that is not part of this change.
`TypeIs[T]` is a subtype of or otherwise assignable to `bool`. `TypeIs`
is invariant with respect to the `TypeIs` return type: `TypeIs[int]` is
neither a subtype nor a supertype of `TypeIs[bool]`. When ty sees a
function marked as returning `TypeIs[T]`, its `return`s will be checked
against `bool` instead. ty will also report such functions if they don't
accept a positional argument. Addtionally, a type narrowing function
call with no positional arguments (e.g., `f()` in the example above)
will be considered invalid.
## Test Plan
Markdown tests.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Consider the following example, which leads to a excessively large
runtime on `main`. The reason for this is the following. When inferring
types for `self.a`, we look up the `a` attribute on `C`. While looking
for implicit instance attributes, we go through every method and check
for `self.a = …` assignments. There are no such assignments here, but we
always have an implicit `self.a = <unbound>` binding at the beginning
over every method. This binding accumulates a complex visibility
constraint in `C.f`, due to the `isinstance` checks. While evaluating
that constraint, we need to infer the type of `self.b`. There's no
binding for `self.b` either, but there's also an implicit `self.b =
<unbound>` binding with the same complex visibility constraint
(involving `self.b` recursively). This leads to a combinatorial
explosion:
```py
class C:
def f(self: "C"):
if isinstance(self.a, str):
return
if isinstance(self.b, str):
return
if isinstance(self.b, str):
return
if isinstance(self.b, str):
return
# repeat 20 times
```
(note that the `self` parameter here is annotated explicitly because we
currently still infer `Unknown` for `self` otherwise)
The fix proposed here is rather simple: when there are no `self.name =
…` attribute assignments in a given method, we skip evaluating the
visibility constraint of the implicit `self.name = <unbound>` binding.
This should also generally help with performance, because that's a very
common case.
This is *not* a fix for cases where there *are* actual bindings in the
method. When we add `self.a = 1; self.b = 1` to that example above, we
still see that combinatorial explosion of runtime. I still think it's
worth to make this optimization, as it fixes the problems with `pandas`
and `sqlalchemy` reported by users. I will open a ticket to track that
separately.
closes https://github.com/astral-sh/ty/issues/627
closes https://github.com/astral-sh/ty/issues/641
## Test Plan
* Made sure that `ty` finishes quickly on the MREs in
https://github.com/astral-sh/ty/issues/627
* Made sure that `ty` finishes quickly on `pandas`
* Made sure that `ty` finishes quickly on `sqlalchemy`
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes false positive in B909 (`loop-iterator-mutation`) where mutations
inside return/break statements were incorrectly flagged as violations.
The fix adds tracking for when mutations occur within return/break
statements and excludes them from violation detection, as they don't
cause the iteration issues B909 is designed to prevent.
## Test Plan
- Added test cases covering the reported false positive scenarios to
`B909.py`
- Verified existing B909 tests continue to pass (no regressions)
- Ran `cargo test -p ruff_linter --lib flake8_bugbear` successfully
Fixes#18399
## Summary
Garbage collect ASTs once we are done checking a given file. Queries
with a cross-file dependency on the AST will reparse the file on demand.
This reduces ty's peak memory usage by ~20-30%.
The primary change of this PR is adding a `node_index` field to every
AST node, that is assigned by the parser. `ParsedModule` can use this to
create a flat index of AST nodes any time the file is parsed (or
reparsed). This allows `AstNodeRef` to simply index into the current
instance of the `ParsedModule`, instead of storing a pointer directly.
The indices are somewhat hackily (using an atomic integer) assigned by
the `parsed_module` query instead of by the parser directly. Assigning
the indices in source-order in the (recursive) parser turns out to be
difficult, and collecting the nodes during semantic indexing is
impossible as `SemanticIndex` does not hold onto a specific
`ParsedModuleRef`, which the pointers in the flat AST are tied to. This
means that we have to do an extra AST traversal to assign and collect
the nodes into a flat index, but the small performance impact (~3% on
cold runs) seems worth it for the memory savings.
Part of https://github.com/astral-sh/ty/issues/214.
## Summary
This PR closes https://github.com/astral-sh/ty/issues/238.
Since `DefinitionState::Deleted` was introduced in #18041, support for
the `del` statement (and deletion of except handler names) is
straightforward.
However, it is difficult to determine whether references to attributes
or subscripts are unresolved after they are deleted. This PR only
invalidates narrowing by assignment if the attribute or subscript is
deleted.
## Test Plan
`mdtest/del.md` is added.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Fixes https://github.com/astral-sh/ruff/issues/18612 by:
- Bailing out without a fix in the case of `*args`, which I don't think
we can fix reliably
- Using an `Edit::deletion` from `remove_argument` instead of an
`Edit::range_replacement` in the presence of unrecognized keyword
arguments
I thought we could always switch to the `Edit::deletion` approach
initially, but it caused problems when `maxlen` was passed positionally,
which we didn't have any existing tests for.
The replacement fix can easily delete comments, so I also marked the fix
unsafe in these cases and updated the docs accordingly.
## Test Plan
New test cases derived from the issue.
## Stabilization
These are pretty significant changes, much like those to PYI059 in
https://github.com/astral-sh/ruff/pull/18611 (and based a bit on the
implementation there!), so I think it probably makes sense to
un-stabilize this for the 0.12 release, but I'm open to other thoughts
there.
## Summary
This is to support https://github.com/astral-sh/ruff/pull/18607.
This PR adds support for generating the top materialization (or upper
bound materialization) and the bottom materialization (or lower bound
materialization) of a type. This is the most general and the most
specific form of the type which is fully static, respectively.
More concretely, `T'`, the top materialization of `T`, is the type `T`
with all occurrences
of dynamic type (`Any`, `Unknown`, `@Todo`) replaced as follows:
- In covariant position, it's replaced with `object`
- In contravariant position, it's replaced with `Never`
- In invariant position, it's replaced with an unresolved type variable
(For an invariant position, it should actually be replaced with an
existential type, but this is not currently representable in our type
system, so we use an unresolved type variable for now instead.)
The bottom materialization is implemented in the same way, except we
start out in "contravariant" position.
## Test Plan
Add test cases for various types.
## Summary
Minor documentation update to make `mypy_primer` instructions a bit more
verbose/helpful for running against a local branch
## Test Plan
N/A
This makes it work for a number of additional cases, like nested
attribute access and things like `[].<CURSOR>`.
The basic idea is that instead of selecting a covering node closest to a
leaf that contains the cursor, we walk up the tree as much as we can.
This lets us access the correct `ExprAttribute` node when performing
nested access.
This routine lets us climb up the AST tree when we find
a contiguous sequence of nodes that satisfy our predicate.
This will be useful for making things like `a.b.<CURSOR>`
work. That is, we don't want the `ExprAttribute` closest
to a leaf. We also don't always want the `ExprAttribute`
closest to the root. Rather, (I think) we want the
`ExprAttribute` closest to the root that has an unbroken
chain to the `ExprAttribute` closest to the leaf.
This commit doesn't change any functionality, but instead changes the
representation of `CoveringNode` to make the implementation simpler (as
well as planned future additions). By putting the found node last in the
list of ancestors (now just generically called `nodes`), we reduce the
amount of special case handling we need.
The downside is that the representation now allows invalid states (a
`CoveringNode` with no elements). But I think this is well mitigated by
encapsulation.
## Summary
Fixes https://github.com/astral-sh/ty/issues/557
## Test Plan
Stable property tests succeed with a million iterations. Added mdtests.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Summary
--
Updates the rule docs to explicitly state how cases like
`Decimal("0.1")` are handled (not affected) because the discussion of
"float casts" referring to values like `nan` and `inf` is otherwise a
bit confusing.
These changes are based on suggestions from @AlexWaygood on Notion, with
a slight adjustment to use 0.1 instead of 0.5 since it causes a more
immediate issue in the REPL:
```pycon
>>> from decimal import Decimal
>>> Decimal(0.5) == Decimal("0.5")
True
>>> Decimal(0.1) == Decimal("0.1")
False
```
Test plan
--
N/a
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Summary
--
This PR updates the docs for PLW1641 to place less emphasis on the
example of inheriting a parent class's `__hash__` implementation by both
reducing the length of the example and warning that it may be unsound in
general, as @AlexWaygood pointed out on Notion.
Test plan
--
Existing tests
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Fixes https://github.com/astral-sh/ruff/issues/18602 by:
1. Avoiding a fix when `*args` are present
2. Inserting the `Generic` base class right before the first keyword
argument, if one is present
In an intermediate commit, I also had special handling to avoid a fix in
the `**kwargs` case, but this is treated (roughly) as a normal keyword,
and I believe handling it properly falls out of the other keyword fix.
I also updated the `add_argument` utility function to insert new
arguments right before the keyword argument list instead of at the very
end of the argument list. This changed a couple of snapshots unrelated
to `PYI059`, but there shouldn't be any functional changes to other
rules because all other calls to `add_argument` were adding a keyword
argument anyway.
## Test Plan
Existing PYI059 cases, plus new tests based on the issue
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Closes https://github.com/astral-sh/ty/issues/577. Make global
`__debug__` a `bool` constant.
## Test Plan
Mdtest `global-constants.md` was created to check if resolved type was
`bool`.
---------
Co-authored-by: David Peter <mail@david-peter.de>
Summary
--
Fixes#18590 by adding parentheses around lambdas and if expressions in
`for` loop iterators for FURB122 and FURB142. I also updated the docs on
the helper function to reflect the part actually being parenthesized and
the new checks.
The `lambda` case actually causes a `TypeError` at runtime, but I think
it's still worth handling to avoid causing a syntax error.
```pycon
>>> s = set()
... for x in (1,) if True else (2,):
... s.add(-x)
... for x in lambda: 0:
... s.discard(-x)
...
Traceback (most recent call last):
File "<python-input-0>", line 4, in <module>
for x in lambda: 0:
^^^^^^^^^
TypeError: 'function' object is not iterable
```
Test Plan
--
New test cases based on the bug report
---------
Co-authored-by: Dylan <dylwil3@gmail.com>
## Summary
As the title says, this PR removes the `Message::to_rule` method by
replacing related uses of `Rule` with `NoqaCode` (or the rule's name in
the case of the cache). Where it seemed a `Rule` was really needed, we
convert back to the `Rule` by parsing either the rule name (with
`str::parse`) or the `NoqaCode` (with `Rule::from_code`).
I thought this was kind of like cheating and that it might not resolve
this part of Micha's
[comment](https://github.com/astral-sh/ruff/pull/18391#issuecomment-2933764275):
> because we can't add Rule to Diagnostic or **have it anywhere in our
shared rendering logic**
but after looking again, the only remaining `Rule` conversion in
rendering code is for the SARIF output format. The other two non-test
`Rule` conversions are for caching and writing a fix summary, which I
don't think fall into the shared rendering logic. That leaves the SARIF
format as the only real problem, but maybe we can delay that for now.
The motivation here is that we won't be able to store a `Rule` on the
new `Diagnostic` type, but we should be able to store a `NoqaCode`,
likely as a string.
## Test Plan
Existing tests
##
[Benchmarks](https://codspeed.io/astral-sh/ruff/branches/brent%2Fremove-to-rule)
Almost no perf regression, only -1% on
`linter/default-rules[large/dataset.py]`.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
https://github.com/astral-sh/ty/issues/214 will require a couple
invasive changes that I would like to get merged even before garbage
collection is fully implemented (to avoid rebasing):
- `ParsedModule` can no longer be dereferenced directly. Instead you
need to load a `ParsedModuleRef` to access the AST, which requires a
reference to the salsa database (as it may require re-parsing the AST if
it was collected).
- `AstNodeRef` can only be dereferenced with the `node` method, which
takes a reference to the `ParsedModuleRef`. This allows us to encode the
fact that ASTs do not live as long as the database and may be collected
as soon a given instance of a `ParsedModuleRef` is dropped. There are a
number of places where we currently merge the `'db` and `'ast`
lifetimes, so this requires giving some types/functions two separate
lifetime parameters.
## Summary
This is a spin-off from
https://github.com/astral-sh/ruff/pull/18447#discussion_r2125844669 to
avoid using `Message::noqa_code` to differentiate between lints and
syntax errors. I went through all of the calls on `main` and on the
branch from #18447, and the instance in `ruff_server` noted in the
linked comment was actually the primary place where this was being done.
Other calls to `noqa_code` are typically some variation of
`message.noqa_code().map_or(String::new, format!(...))`, with the major
exception of the gitlab output format:
a120610b5b/crates/ruff_linter/src/message/gitlab.rs (L93-L105)
which obviously assumes that `None` means syntax error. A simple fix
here would be to use `message.name()` for `check_name` instead of the
noqa code, but I'm not sure how breaking that would be. This could just
be:
```rust
let description = message.body();
let description = description.strip_prefix("SyntaxError: ").unwrap_or(description).to_string();
let check_name = message.name();
```
In that case. This sounds reasonable based on the [Code Quality report
format](https://docs.gitlab.com/ci/testing/code_quality/#code-quality-report-format)
docs:
> | Name | Type | Description|
> |-----|-----|----|
> |`check_name` | String | A unique name representing the check, or
rule, associated with this violation. |
## Test Plan
Existing tests
## Summary
Fixes https://github.com/astral-sh/ty/issues/556.
On Windows, system installations have different layouts to virtual
environments. In Windows virtual environments, the Python executable is
found at `<sys.prefix>/Scripts/python.exe`. But in Windows system
installations, the Python executable is found at
`<sys.prefix>/python.exe`. That means that Windows users were able to
point to Python executables inside virtual environments with the
`--python` flag, but they weren't able to point to Python executables
inside system installations.
This PR fixes that issue. It also makes a couple of other changes:
- Nearly all `sys.prefix` resolution is moved inside `site_packages.rs`.
That was the original design of the `site-packages` resolution logic,
but features implemented since the initial implementation have added
some resolution and validation to `resolver.rs` inside the module
resolver. That means that we've ended up with a somewhat confusing code
structure and a situation where several checks are unnecessarily
duplicated between the two modules.
- I noticed that we had quite bad error messages if you e.g. pointed to
a path that didn't exist on disk with `--python` (we just gave a
somewhat impenetrable message saying that we "failed to canonicalize"
the path). I improved the error messages here and added CLI tests for
`--python` and the `environment.python` configuration setting.
## Test Plan
- Existing tests pass
- Added new CLI tests
- I manually checked that virtual-environment discovery still works if
no configuration is given
- Micha did some manual testing to check that pointing `--python` to a
system-installation executable now works on Windows
## Summary
This PR partially solves https://github.com/astral-sh/ty/issues/164
(derived from #17643).
Currently, the definitions we manage are limited to those for simple
name (symbol) targets, but we expand this to track definitions for
attribute and subscript targets as well.
This was originally planned as part of the work in #17643, but the
changes are significant, so I made it a separate PR.
After merging this PR, I will reflect this changes in #17643.
There is still some incomplete work remaining, but the basic features
have been implemented, so I am publishing it as a draft PR.
Here is the TODO list (there may be more to come):
* [x] Complete rewrite and refactoring of documentation (removing
`Symbol` and replacing it with `Place`)
* [x] More thorough testing
* [x] Consolidation of duplicated code (maybe we can consolidate the
handling related to name, attribute, and subscript)
This PR replaces the current `Symbol` API with the `Place` API, which is
a concept that includes attributes and subscripts (the term is borrowed
from Rust).
## Test Plan
`mdtest/narrow/assignment.md` is added.
---------
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
This optimizes some of the logic added in
https://github.com/astral-sh/ruff/pull/18444. In general, we only
calculate information for subdiagnostics if we know we'll actually emit
the diagnostic. The check to see whether we'll emit the diagnostic is
work we'll definitely have to do whereas the the work to gather
information for a subdiagnostic isn't work we necessarily have to do if
the diagnostic isn't going to be emitted at all.
This PR makes us lazier about gathering the information we need for the
subdiagnostic, and moves all the subdiagnostic logic into one function
rather than having some `unresolved-reference` subdiagnostic logic in
`infer.rs` and some in `diagnostic.rs`.
## Test Plan
`cargo test -p ty_python_semantic`
## Summary
As well as excluding a hardcoded set of special attributes, CPython at
runtime also excludes any attributes or declarations starting with
`_abc_` from the set of members that make up a protocol interface. I
missed this in my initial implementation.
This is a bit of a CPython implementation detail, but I do think it's
important that we try to model the runtime as best we can here. The
closer we are to the runtime behaviour, the closer we come to sound
behaviour when narrowing types from `isinstance()` checks against
runtime-checkable protocols (for example)
## Test Plan
Extended an existing mdtest
## Summary
Closes https://github.com/astral-sh/ty/issues/502.
In the following example:
```py
class Foo:
x: int
def method(self):
y = x
```
The user may intended to use `y = self.x` in `method`.
This is now added as a subdiagnostic in the following form :
`info: An attribute with the same name as 'x' is defined, consider using
'self.x'`
## Test Plan
Added mdtest with snapshot diagnostics.
## Summary
Previously, all symbols where provided as possible completions. In an
example like the following, both `foo` and `f` were suggested as
completions, because `f` itself is a symbol.
```py
foo = 1
f<CURSOR>
```
Similarly, in the following example, `hidden_symbol` was suggested, even
though it is not statically visible:
```py
if 1 + 2 != 3:
hidden_symbol = 1
hidden_<CURSOR>
```
With the change suggested here, we only use statically visible
declarations and bindings as a source for completions.
## Test Plan
- Updated snapshot tests
- New test for statically hidden definitions
- Added test for star import
## Summary
Implement a hotfix for the playground/LSP crashes related to missing
`expression_scope_id`s.
relates to: https://github.com/astral-sh/ty/issues/572
## Test Plan
* Regression tests from https://github.com/astral-sh/ruff/pull/18441
* Ran the playground locally to check if panics occur / completions
still work.
---------
Co-authored-by: Andrew Gallant <andrew@astral.sh>
## Summary
Part of astral-sh/ty#104, closes: astral-sh/ty#468
This PR implements the argument type expansion which is step 3 of the
overload call evaluation algorithm.
Specifically, this step needs to be taken if type checking resolves to
no matching overload and there are argument types that can be expanded.
## Test Plan
Add new test cases.
## Ecosystem analysis
This PR removes 174 `no-matching-overload` false positives -- I looked
at a lot of them and they all are false positives.
One thing that I'm not able to understand is that in
2b7e3adf27/sphinx/ext/autodoc/preserve_defaults.py (L179)
the inferred type of `value` is `str | None` by ty and Pyright, which is
correct, but it's only ty that raises `invalid-argument-type` error
while Pyright doesn't. The constructor method of `DefaultValue` has
declared type of `str` which is invalid.
There are few cases of false positives resulting due to the fact that ty
doesn't implement narrowing on attribute expressions.
## Summary
An issue seen here https://github.com/astral-sh/ty/issues/500
The `__init__` method of dataclasses had no inherited generic context,
so we could not infer the type of an instance from a constructor call
with generics
## Test Plan
Add tests to classes.md` in generics folder
## Summary
Part of https://github.com/astral-sh/ty/issues/111
Using `dataclass` as a function, instead of as a decorator did not work
as expected prior to this.
Fix that by modifying the dataclass overload's return type.
## Test Plan
New mdtests, fixing the existing TODO.
This updates our representation of functions to more closely match our
representation of classes.
The new `OverloadLiteral` and `FunctionLiteral` classes represent a
function definition in the AST. If a function is generic, this is
unspecialized. `FunctionType` has been updated to represent a function
type, which is specialized if the function is generic. (These names are
chosen to match `ClassLiteral` and `ClassType` on the class side.)
This PR does not add a separate `Type` variant for `FunctionLiteral`.
Maybe we should? Possibly as a follow-on PR?
Part of https://github.com/astral-sh/ty/issues/462
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Follow-up from #18401, I was looking at whether that would fix the issue
at https://github.com/astral-sh/ty/issues/247#issuecomment-2917656676
and it didn't, which made me realize that the PR only inferred `list[T]`
when the value type was tuple but it could be other types as well.
This PR fixes the actual issue by inferring `list[T]` for the non-tuple
type case.
## Test Plan
Add test cases for starred expression involved with non-tuple type. I
also added a few test cases for list type and list literal.
I also verified that the example in the linked issue comment works:
```py
def _(line: str):
a, b, *c = line.split(maxsplit=2)
c.pop()
```
## Summary
Came across this while debugging some ecosystem changes in
https://github.com/astral-sh/ruff/pull/18347. I think the meta-type of a
typevar-annotated variable should be equal to `type`, not `<class
'object'>`.
## Test Plan
New Markdown tests.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
/closes #18387
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
update snapshots
<!-- How was it tested? -->
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
https://github.com/astral-sh/ruff/issues/18387#issuecomment-2923039331
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
update snapshots
<!-- How was it tested? -->
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Mark `FURB180`'s fix as unsafe if the class already has base classes.
This is because the base classes might validate the other base classes
(like `typing.Protocol` does) or otherwise alter runtime behavior if
more base classes are added.
## Test Plan
The existing snapshot test covers this case already.
## References
Partially addresses https://github.com/astral-sh/ruff/issues/13307 (left
out way to permit certain exceptions)
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Closes#17226.
This PR updates the `FAST003` rule to correctly handle [FastAPI class
dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/classes-as-dependencies/).
Specifically, if a path parameter is declared in either:
- a `pydantic.BaseModel` used as a dependency, or
- the `__init__` method of a class used as a dependency,
then `FAST003` will no longer incorrectly report it as unused.
FastAPI allows a shortcut when using annotated class dependencies -
`Depends` can be called without arguments, e.g.:
```python
class MyParams(BaseModel):
my_id: int
@router.get("/{my_id}")
def get_id(params: Annotated[MyParams, Depends()]): ...
```
This PR ensures that such usage is properly supported by the linter.
Note: Support for dataclasses is not included in this PR. Let me know if
you’d like it to be added.
## Test Plan
Added relevant test cases to the `FAST003.py` fixture.
This PR implements template strings (t-strings) in the parser and
formatter for Ruff.
Minimal changes necessary to compile were made in other parts of the code (e.g. ty, the linter, etc.). These will be covered properly in follow-up PRs.
## Summary
Allow a typevar to be callable if it is bound to a callable type, or
constrained to callable types.
I spent some time digging into why this support didn't fall out
naturally, and ultimately the reason is that we look up `__call__` on
the meta type (since its a dunder), and our implementation of
`Type::to_meta_type` for `Type::Callable` does not return a type with
`__call__`.
A more general solution here would be to have `Type::to_meta_type` for
`Type::Callable` synthesize a protocol with `__call__` and return an
intersection with that protocol (since for a type to be callable, we
know its meta-type must have `__call__`). That solution could in
principle also replace the special-case handling of `Type::Callable`
itself, here in `Type::bindings`. But that more general approach would
also be slower, and our protocol support isn't quite ready for that yet,
and handling this directly in `Type::bindings` is really not bad.
Fixes https://github.com/astral-sh/ty/issues/480
## Test Plan
Added mdtests.
This PR adds initial support for listing all attributes of
an object. It is exposed through a new `all_members`
routine in `ty_extensions`, which is in turn used to test
the functionality.
The purpose of listing all members is for code
completion. That is, given a `object.<CURSOR>`, we
would like to list all available attributes on
`object`.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Follow up on https://github.com/astral-sh/ruff/pull/18093 and apply it
to AIR312
## Test Plan
<!-- How was it tested? -->
The existing test fixtures have been updated
Summary
--
This is the last main difference between the `OldDiagnostic` and
`Message`
types, so attaching a `SourceFile` to `OldDiagnostic` should make
combining the
two types almost trivial.
Initially I updated the remaining rules without access to a `Checker` to
take a
`&SourceFile` directly, but after Micha's suggestion in
https://github.com/astral-sh/ruff/pull/18356#discussion_r2113281552, I
updated all of these calls to take a
`LintContext` instead. This new type is a thin wrapper around a
`RefCell<Vec<OldDiagnostic>>`
and a `SourceFile` and now has the `report_diagnostic` method returning
a `DiagnosticGuard` instead of `Checker`.
This allows the same `Drop`-based implementation to be used in cases
without a `Checker` and also avoids a lot of intermediate allocations of
`Vec<OldDiagnostic>`s.
`Checker` now also contains a `LintContext`, which it defers to for its
`report_diagnostic` methods, which I preserved for convenience.
Test Plan
--
Existing tests
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Follow up on https://github.com/astral-sh/ruff/pull/18093 and apply it
to AIR311
---
Rules fixed
* `airflow.models.datasets.expand_alias_to_datasets` →
`airflow.models.asset.expand_alias_to_assets`
* `airflow.models.baseoperatorlink.BaseOperatorLink` →
`airflow.sdk.BaseOperatorLink`
## Test Plan
<!-- How was it tested? -->
The existing test fixtures have been updated
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Follow up on https://github.com/astral-sh/ruff/pull/18093 and apply it
to AIR301
## Test Plan
<!-- How was it tested? -->
The existing test fixtures have been updated
## Summary
- Convert tests demonstrating our resilience to malformed/absent
`version` fields in `pyvenf.cfg` files to mdtests. Also make them more
expansive.
- Convert the regression test I added in
https://github.com/astral-sh/ruff/pull/18157 to an mdtest
- Add comments next to unit tests that cannot be converted to mdtests
(but where it's not obvious why they can't) so I don't have to do this
exercise again 😄
- In `site_packages.rs`, factor out the logic for figuring out where we
expect the system-installation `site-packages` to be. Currently we have
the same logic twice.
## Test Plan
`cargo test -p ty_python_semantic`
## Summary
This change was based on a mis-reading of a comment in typeshed, and a
wrong assumption about what was causing a test failure in a prior PR.
Reverting it doesn't cause any tests to fail.
## Test Plan
Existing tests.
## Summary
Resolves [#513](https://github.com/astral-sh/ty/issues/513).
Callable types are now considered to be disjoint from nominal instance
types where:
* The class is `@final`, and
* Its `__call__` either does not exist or is not assignable to `(...) ->
Unknown`.
## Test Plan
Markdown tests.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Partially implement https://github.com/astral-sh/ty/issues/538,
```py
from pathlib import Path
def setup_test_project(registry_name: str, registry_url: str, project_dir: str) -> Path:
pyproject_file = Path(project_dir) / "pyproject.toml"
pyproject_file.write_text("...", encoding="utf-8")
```
As no return statement is defined in the function `setup_test_project`
with annotated return type `Path`, we provide the following diagnosis :
- error[invalid-return-type]: Function **always** implicitly returns
`None`, which is not assignable to return type `Path`
with a subdiagnostic :
- note: Consider changing your return annotation to `-> None` or adding a `return` statement
## Test Plan
mdtests with snapshots to capture the subdiagnostic. I have to mention
that existing snapshots were modified since they now fall in this
category.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Add utility functions `generate_import_edit` and
`generate_remove_and_runtime_import_edit` to generate the fix needed for
the airflow rules.
1. `generate_import_edit` is for the cases where the member name has
changed. (e.g., `airflow.datasts.Dataset` to `airflow.sdk.Asset`) It's
just extracted from the original logic
2. `generate_remove_and_runtime_import_edit` is for cases where the
member name has not changed. (e.g.,
`airflow.operators.pig_operator.PigOperator` to
`airflow.providers.apache.pig.hooks.pig.PigCliHook`) This is newly
introduced. As it introduced runtime import, I mark it as an unsafe fix.
Under the hook, it tried to find the original import statement, remove
it, and add a new import fix
---
* rules fix
* `airflow.sensors.external_task_sensor.ExternalTaskSensorLink` →
`airflow.providers.standard.sensors.external_task.ExternalDagLink`
## Test Plan
<!-- How was it tested? -->
The existing test fixtures have been updated
Summary
--
It's a bit late in the refactoring process, but I think there are still
a couple of PRs left before getting rid of this type entirely, so I
thought it would still be worth doing.
This PR is just a quick rename with no other changes.
Test Plan
--
Existing tests
## Summary
Adds coverage of using set(...) in addition to `{...} in
SingleItemMembershipTest.
Fixes#15792
(and replaces the old PR #15793)
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Updated unit test and snapshot.
Steps to reproduce are in the issue linked above.
<!-- How was it tested? -->
Previously, completions were based on just returning every identifier
parsed in the current Python file. In this commit, we change it to
identify an expression under the cursor and then return all symbols
available to the scope containing that expression.
This is still returning too much, and also, in some cases, not enough.
Namely, it doesn't really take the specific context into account other
than scope. But this does improve on the status quo. For example:
def foo(): ...
def bar():
def fast(): ...
def foofoo(): ...
f<CURSOR>
When asking for completions here, the LSP will no longer include `fast`
as a possible completion in this context.
Ref https://github.com/astral-sh/ty/issues/86
This is analogous to the existing `Tokens::after` method. Its
implementation is almost identical.
We plan to use this for looking at the tokens immediately before the
cursor when fetching completions.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes#18231
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Snapshot tests
<!-- How was it tested? -->
## Summary
Implements `use-maxsplit-arg` (`PLC0207`)
https://pylint.readthedocs.io/en/latest/user_guide/messages/convention/use-maxsplit-arg.html
> Emitted when accessing only the first or last element of str.split().
The first and last element can be accessed by using str.split(sep,
maxsplit=1)[0] or str.rsplit(sep, maxsplit=1)[-1] instead.
This is part of https://github.com/astral-sh/ruff/issues/970
## Test Plan
`cargo test`
Additionally compared Ruff output to Pylint:
```
pylint --disable=all --enable=use-maxsplit-arg crates/ruff_linter/resources/test/fixtures/pylint/missing_maxsplit_arg.py
cargo run -p ruff -- check crates/ruff_linter/resources/test/fixtures/pylint/missing_maxsplit_arg.py --no-cache --select PLC0207
```
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
Allow classes with `__init__` to be subtypes of `Callable`
Fixes https://github.com/astral-sh/ty/issues/358
## Test Plan
Update is_subtype_of.md
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
This PR add the `fix safety` section for rule `B006` in
`mutable_argument_default.rs` for #15584
When applying this rule for fixes, certain changes may alter the
original logical behavior. For example:
before:
```python
def cache(x, storage=[]):
storage.append(x)
return storage
print(cache(1)) # [1]
print(cache(2)) # [1, 2]
```
after:
```python
def cache(x, storage=[]):
storage.append(x)
return storage
print(cache(1)) # [1]
print(cache(2)) # [2]
```
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes#18353
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
Snapshot tests
There were many fields in `Signature` and friends that really had more
to do with how a signature was being _used_ — how it was looked up,
details about an individual call site, etc. Those fields more properly
belong in `Bindings` and friends.
This is a pure refactoring, and should not affect any tests or ecosystem
projects.
I started on this journey in support of
https://github.com/astral-sh/ty/issues/462. It seemed worth pulling out
as a separate PR.
One major concrete benefit of this refactoring is that we can now use
`CallableSignature` directly in `CallableType`. (We can't use
`CallableSignature` directly in that `Type` variant because signatures
are not currently interned.)
Summary
--
This PR adds a `DiagnosticGuard` type to ruff that is adapted from the
`DiagnosticGuard` and `LintDiagnosticGuard` types from ty. This guard is
returned by `Checker::report_diagnostic` and derefs to a
`ruff_diagnostics::Diagnostic` (`OldDiagnostic`), allowing methods like
`OldDiagnostic::set_fix` to be called on the result. On `Drop` the
`DiagnosticGuard` pushes its contained `OldDiagnostic` to the `Checker`.
The main motivation for this is to make a following PR adding a
`SourceFile` to each diagnostic easier. For every rule where a `Checker`
is available, this will now only require modifying
`Checker::report_diagnostic` rather than all the rules.
In the few cases where we need to create a diagnostic before we know if
we actually want to emit it, there is a `DiagnosticGuard::defuse`
method, which consumes the guard without emitting the diagnostic. I was
able to restructure about half of the rules that naively called this to
avoid calling it, but a handful of rules still need it.
One of the fairly common patterns where `defuse` was needed initially
was something like
```rust
let diagnostic = Diagnostic::new(DiagnosticKind, range);
if !checker.enabled(diagnostic.rule()) {
return;
}
```
So I also added a `Checker::checked_report_diagnostic` method that
handles this check internally. That helped to avoid some additional
`defuse` calls. The name is a bit repetitive, so I'm definitely open to
suggestions there. I included a warning against using it in the docs
since, as we've seen, the conversion from a diagnostic to a rule is
actually pretty expensive.
Test Plan
--
Existing tests
## Summary
We create `Callable` types for synthesized functions like the `__init__`
method of a dataclass. These generated functions are real functions
though, with descriptor-like behavior. That is, they can bind `self`
when accessed on an instance. This was modeled incorrectly so far.
## Test Plan
Updated tests
## Summary
I don't think we're ever going to add any `KnownInstanceType` variants
that evaluate to `False` in a boolean context; the
`KnownInstanceType::bool()` method just seems like unnecessary
complexity.
## Test Plan
`cargo test -p ty_python_semantic`
# Summary
Adds a subdiagnostic hint in the following scenario where a
synchronous `with` is used with an async context manager:
```py
class Manager:
async def __aenter__(self): ...
async def __aexit__(self, *args): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because it does not implement `__enter__` and `__exit__`"
# note: Objects of type `Manager` *can* be used as async context managers
# note: Consider using `async with` here
with Manager():
...
```
closes https://github.com/astral-sh/ty/issues/508
## Test Plan
New MD snapshot tests
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
fixesastral-sh/ty#366
## Test Plan
* Added panic corpus regression tests
* I also wrote a hover regression test (see below), but decided not to
include it. The corpus tests are much more "effective" at finding these
types of errors, since they exhaustively check all expressions for
types.
<details>
```rs
#[test]
fn hover_regression_test_366() {
let test = cursor_test(
r#"
from ty_extensions import Intersection
class A: ...
class B: ...
def _(x: Intersection[A,<CURSOR> B]):
pass
"#,
);
assert_snapshot!(test.hover(), @r"
A & B
---------------------------------------------
```text
A & B
```
---------------------------------------------
info[hover]: Hovered content is
--> main.py:7:31
|
5 | class B: ...
6 |
7 | def _(x: Intersection[A, B]):
| ^^-^
| | |
| | Cursor offset
| source
8 | pass
|
");
}
```
</details>
## Summary
The previous `try_call_dunder_with_policy` API was a bit of a footgun
since you needed to pass `NO_INSTANCE_FALLBACK` in *addition* to other
policies that you wanted for the member lookup. Implicit calls to dunder
methods never access instance members though, so we can do this
implicitly in `try_call_dunder_with_policy`.
No functional changes.
## Summary
`Type::member_lookup_with_policy` now falls back to calling
`__getattribute__` when a member cannot be found as a second fallback
after `__getattr__`.
closes https://github.com/astral-sh/ty/issues/441
## Test Plan
Added markdown tests.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
This should address a problem that came up while working on
https://github.com/astral-sh/ruff/pull/18280. When looking up an
attribute (typically a dunder method) with the `MRO_NO_OBJECT_FALLBACK`
policy, the attribute is first looked up on the meta type. If the meta
type happens to be `type`, we go through the following branch in
`find_name_in_mro_with_policy`:
97ff015c88/crates/ty_python_semantic/src/types.rs (L2565-L2573)
The problem is that we now look up the attribute on `object` *directly*
(instead of just having `object` in the MRO). In this case,
`MRO_NO_OBJECT_FALLBACK` has no effect in `class_member_from_mro`:
c3feb8ce27/crates/ty_python_semantic/src/types/class.rs (L1081-L1082)
So instead, we need to explicitly respect the `MRO_NO_OBJECT_FALLBACK`
policy here by returning `Symbol::Unbound`.
## Test Plan
Added new Markdown tests that explain the ecosystem changes that we
observe.
## Summary
Fix a bug that involved writes to attributes on union/intersection types
that included modules as elements.
This is a prerequisite to avoid some ecosystem false positives in
https://github.com/astral-sh/ruff/pull/18312
## Test Plan
Added regression test
## Summary
This PR moves the diagnostics API for the language server out from the
request handler module to the diagnostics API module.
This is in preparation to add support for publishing diagnostics.
## Summary
Resolves https://github.com/astral-sh/ty/issues/485.
`infer_binary_intersection_type_comparison()` now checks for all
positive members before concluding that an operation is unsupported for
a given intersection type.
## Test Plan
Markdown tests.
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
This is a practice I followed on previous projects. Should hopefully
further help developers who want to update the documentation.
The big downside is that it's annoying to see this *as a user of the
documentation* if you don't open the Markdown file in the browser. But
I'd argue that those files don't really follow the original Markdown
spirit anyway with all the inline HTML.
## Summary
This is something I wrote a few months ago, and continued to update from
time to time. It was mostly written for my own education. I found a few
bugs while writing it at the time (there are still one or two TODOs in
the test assertions that are probably bugs). Our other tests are fairly
comprehensive, but they are usually structured around a certain
functionality or operation (subtyping, assignability, narrowing). The
idea here was to focus on individual *types and their properties*.
closes#197 (added `JustFloat` and `JustComplex` to `ty_extensions`).
## Summary
Fix remaining `knot.toml` reference and replace it with `ty.toml`. This
change was probably still in flight while we renamed things.
## Test Plan
Added a second assertion which ensures that the config file has any
effect.
## Summary
It doesn't seem to be necessary for our generics implementation to carry
the `GenericContext` in the `ClassBase` variants. Removing it simplifies
the code, fixes many TODOs about `Generic` or `Protocol` appearing
multiple times in MROs when each should only appear at most once, and
allows us to more accurately detect runtime errors that occur due to
`Generic` or `Protocol` appearing multiple times in a class's bases.
In order to remove the `GenericContext` from the `ClassBase` variant, it
turns out to be necessary to emulate
`typing._GenericAlias.__mro_entries__`, or we end up with a large number
of false-positive `inconsistent-mro` errors. This PR therefore also does
that.
Lastly, this PR fixes the inferred MROs of PEP-695 generic classes,
which implicitly inherit from `Generic` even if they have no explicit
bases.
## Test Plan
mdtests
## Summary
Fix some issues with subtying/assignability for instances vs callables.
We need to look up dunders on the class, not the instance, and we should
limit our logic here to delegating to the type of `__call__`, so it
doesn't get out of sync with the calls we allow.
Also, we were just entirely missing assignability handling for
`__call__` implemented as anything other than a normal bound method
(though we had it for subtyping.)
A first step towards considering what else we want to change in
https://github.com/astral-sh/ty/issues/491
## Test Plan
mdtests
---------
Co-authored-by: med <medioqrity@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Division works differently in Python than in Rust. If the result is
negative and there is a remainder, the division rounds down (instead of
towards zero). The remainder needs to be adjusted to compensate so that
`(lhs // rhs) * rhs + (lhs % rhs) == lhs`.
Fixesastral-sh/ty#481.
## Summary
https://github.com/astral-sh/ty/issues/111
This PR adds support for `frozen` dataclasses. It will emit a diagnostic
with a similar message to mypy
Note: This does not include emitting a diagnostic if `__setattr__` or
`__delattr__` are defined on the object as per the
[spec](https://docs.python.org/3/library/dataclasses.html#module-contents)
## Test Plan
mdtest
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Make sure that the following definitions all lead to the same outcome
(bug originally noticed by @AlexWaygood)
```py
from typing import ClassVar
class Descriptor:
def __get__(self, instance, owner) -> int:
return 42
class C:
a: ClassVar[Descriptor]
b: Descriptor = Descriptor()
c: ClassVar[Descriptor] = Descriptor()
reveal_type(C().a) # revealed: int (previously: int | Descriptor)
reveal_type(C().b) # revealed: int
reveal_type(C().c) # revealed: int
```
## Test Plan
New Markdown tests
## Summary
I think `division-by-zero` is a low-value diagnostic in general; most
real division-by-zero errors (especially those that are less obvious to
the human eye) will occur on values typed as `int`, in which case we
don't issue the diagnostic anyway. Mypy and pyright do not emit this
diagnostic.
Currently the diagnostic is prone to false positives because a) we do
not silence it in unreachable code, and b) we do not implement narrowing
of literals from inequality checks. We will probably fix (a) regardless,
but (b) is low priority apart from division-by-zero.
I think we have many more important things to do and should not allow
false positives on a low-value diagnostic to be a distraction. Not
opposed to re-enabling this diagnostic in future when we can prioritize
reducing its false positives.
References https://github.com/astral-sh/ty/issues/443
## Test Plan
Existing tests.
## Summary
This PR updates the language server to avoid panicking when there are
multiple workspace folders passed during initialization. The server
currently picks up the first workspace folder and provides a warning and
a log message.
## Test Plan
<img width="1724" alt="Screenshot 2025-05-17 at 11 43 09"
src="https://github.com/user-attachments/assets/1a7ddbc3-198d-4191-a28f-9b69321e8f99"
/>
## Summary
Resolves [#461](https://github.com/astral-sh/ty/issues/461).
ty was hardcoded to infer `BytesLiteral` types for integer indexing into
`BytesLiteral`. It will now infer `IntLiteral` types instead.
## Test Plan
Markdown tests.
Summary
--
I thought that emitting multiple diagnostics at once would be difficult
to port to a diagnostic construction model closer to ty's
`InferContext::report_lint`, so as a first step toward that, this PR
removes `Checker::report_diagnostics`.
In many cases I was able to do some related refactoring to avoid
allocating a `Vec<Diagnostic>` at all, often by adding a `Checker` field
to a `Visitor` or by passing a `Checker` instead of a `&mut
Vec<Diagnostic>`.
In other cases, I had to fall back on something like
```rust
for diagnostic in diagnostics {
checker.report_diagnostic(diagnostic);
}
```
which I guess is a bit worse than the `extend` call in
`report_diagnostics`, but hopefully it won't make too much of a
difference.
I'm still not quite sure what to do with the remaining loop cases. The
two main use cases for collecting a sequence of diagnostics before
emitting any of them are:
1. Applying a single `Fix` to a group of diagnostics
2. Avoiding an earlier diagnostic if something goes wrong later
I was hoping we could get away with just a `DiagnosticGuard` that
reported a `Diagnostic` on drop, but I guess we will still need a
`DiagnosticGuardBuilder` that can be collected in these cases and
produce a `DiagnosticGuard` once we know we actually want the
diagnostics.
Test Plan
--
Existing tests
Closes https://github.com/astral-sh/ty/issues/453.
## Summary
Add an additional info diagnostic to `unresolved-import` check to hint
to users that they should make sure their Python environment is properly
configured for ty, linking them to the corresponding doc. This
diagnostic is only shown when an import is not relative, e.g., `import
maturin` not `import .maturin`.
## Test Plan
Updated snapshots with new info message and reran tests.
The PR add the `fix safety` section for rule `SIM110` (#15584 )
### Unsafe Fix Example
```python
def predicate(item):
global called
called += 1
if called == 1:
# after first call we change the method
def new_predicate(_): return False
globals()['predicate'] = new_predicate
return True
def foo():
for item in range(10):
if predicate(item):
return True
return False
def foo_gen():
return any(predicate(item) for item in range(10))
called = 0
print(foo()) # true – returns immediately on first call
called = 0
print(foo_gen()) # false – second call uses new `predicate`
```
### Note
I notice that
[here](46be305ad2/crates/ruff_linter/src/rules/flake8_simplify/rules/reimplemented_builtin.rs (L60))
we have two rules, `SIM110` & `SIM111`. The second one seems not anymore
active. Should I delete `SIM111`?
This implements the stopgap approach described in
https://github.com/astral-sh/ty/issues/336#issuecomment-2880532213 for
handling literal types in generic class specializations.
With this approach, we will promote any literal to its instance type,
but _only_ when inferring a generic class specialization from a
constructor call:
```py
class C[T]:
def __init__(self, x: T) -> None: ...
reveal_type(C("string")) # revealed: C[str]
```
If you specialize the class explicitly, we still use whatever type you
provide, even if it's a literal:
```py
from typing import Literal
reveal_type(C[Literal[5]](5)) # revealed: C[Literal[5]]
```
And this doesn't apply at all to generic functions:
```py
def f[T](x: T) -> T:
return x
reveal_type(f(5)) # revealed: Literal[5]
```
---
As part of making this happen, we also generalize the `TypeMapping`
machinery. This provides a way to apply a function to type, returning a
new type. Complicating matters is that for function literals, we have to
apply the mapping lazily, since the function's signature is not created
until (and if) someone calls its `signature` method. That means we have
to stash away the mappings that we want to apply to the signatures
parameter/return annotations once we do create it. This requires some
minor `Cow` shenanigans to continue working for partial specializations.
This is a follow-on to #18155. For the example raised in
https://github.com/astral-sh/ty/issues/370:
```py
import tempfile
with tempfile.TemporaryDirectory() as tmp: ...
```
the new logic would notice that both overloads of `TemporaryDirectory`
match, and combine their specializations, resulting in an inferred type
of `str | bytes`.
This PR updates the logic to match our other handling of other calls,
where we only keep the _first_ matching overload. The result for this
example then becomes `str`, matching the runtime behavior. (We still do
not implement the full [overload resolution
algorithm](https://typing.python.org/en/latest/spec/overload.html#overload-call-evaluation)
from the spec.)
## Summary
Add a new diagnostic hint if you try to use PEP 604 `X | Y` union syntax
in a non-type-expression before 3.10.
closes https://github.com/astral-sh/ty/issues/437
## Test Plan
New snapshot test
## Summary
This PR unifies the ruff `Message` enum variants for syntax errors and
rule violations into a single `Message` struct consisting of a shared
`db::Diagnostic` and some additional, optional fields used for some rule
violations.
This version of `Message` is nearly a drop-in replacement for
`ruff_diagnostics::Diagnostic`, which is the next step I have in mind
for the refactor.
I think this is also a useful checkpoint because we could possibly add
some of these optional fields to the new `Diagnostic` type. I think
we've previously discussed wanting support for `Fix`es, but the other
fields seem less relevant, so we may just need to preserve the `Message`
wrapper for a bit longer.
## Test plan
Existing tests
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
* Remove the following rules
* name
* `airflow.auth.managers.base_auth_manager.is_authorized_dataset` →
`airflow.api_fastapi.auth.managers.base_auth_manager.is_authorized_asset`
*
`airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_dataset`
→
`airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset`
* Update the following rules
* name
* `airflow.models.baseoperatorlink.BaseOperatorLink` →
`airflow.sdk.BaseOperatorLink`
* `airflow.api_connexion.security.requires_access` → "Use
`airflow.api_fastapi.core_api.security.requires_access_*` instead`"
* `airflow.api_connexion.security.requires_access_dataset`→
`airflow.api_fastapi.core_api.security.requires_access_asset`
* `airflow.notifications.basenotifier.BaseNotifier` →
`airflow.sdk.bases.notifier.BaseNotifier`
* `airflow.www.auth.has_access` → None
* `airflow.www.auth.has_access_dataset` → None
* `airflow.www.utils.get_sensitive_variables_fields`→ None
* `airflow.www.utils.should_hide_value_for_key`→ None
* class attribute
* `airflow..sensors.weekday.DayOfWeekSensor`
* `use_task_execution_day` removed
*
`airflow.providers.amazon.aws.auth_manager.aws_auth_manager.AwsAuthManager`
* `is_authorized_dataset`
* Add the following rules
* class attribute
* `airflow.auth.managers.base_auth_manager.BaseAuthManager` |
`airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager`
* name
* `airflow.auth.managers.base_auth_manager.BaseAuthManager` →
`airflow.api_fastapi.auth.managers.base_auth_manager.BaseAuthManager` *
`is_authorized_dataset` → `is_authorized_asset`
* refactor
* simplify unnecessary match with if else
* rename Replacement::Name as Replacement::AttrName
## Test Plan
<!-- How was it tested? -->
The test fixtures have been revised and updated.
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
In the later development of Airflow 3.0, backward compatibility was not
added for some cases. Thus, the following rules are moved back to AIR302
* airflow.hooks.subprocess.SubprocessResult →
airflow.providers.standard.hooks.subprocess.SubprocessResult
* airflow.hooks.subprocess.working_directory →
airflow.providers.standard.hooks.subprocess.working_directory
* airflow.operators.datetime.target_times_as_dates →
airflow.providers.standard.operators.datetime.target_times_as_dates
* airflow.operators.trigger_dagrun.TriggerDagRunLink →
airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunLink
* airflow.sensors.external_task.ExternalTaskSensorLink →
airflow.providers.standard.sensors.external_task.ExternalDagLink (**This
one contains a minor change**)
* airflow.sensors.time_delta.WaitSensor →
airflow.providers.standard.sensors.time_delta.WaitSensor
## Test Plan
<!-- How was it tested? -->
This primarily comes up with annotated `self` parameters in
constructors:
```py
class C[T]:
def __init__(self: C[int]): ...
```
Here, we want infer a specialization of `{T = int}` for a call that hits
this overload.
Normally when inferring a specialization of a function call, typevars
appear in the parameter annotations, and not in the argument types. In
this case, this is reversed: we need to verify that the `self` argument
(`C[T]`, as we have not yet completed specialization inference) is
assignable to the parameter type `C[int]`.
To do this, we simply look for a typevar/type in both directions when
performing inference, and apply the inferred specialization to argument
types as well as parameter types before verifying assignability.
As a wrinkle, this exposed that we were not checking
subtyping/assignability for function literals correctly. Our function
literal representation includes an optional specialization that should
be applied to the signature. Before, function literals were considered
subtypes of (assignable to) each other only if they were identical Salsa
objects. Two function literals with different specializations should
still be considered subtypes of (assignable to) each other if those
specializations result in the same function signature (typically because
the function doesn't use the typevars in the specialization).
Closes https://github.com/astral-sh/ty/issues/370
Closes https://github.com/astral-sh/ty/issues/100
Closes https://github.com/astral-sh/ty/issues/258
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
My editor runs `rustfmt` on save to format Rust code, not `cargo fmt`.
With our recent bump to the Rust 2024 edition, the formatting that
`rustfmt`/`cargo fmt` applies changed. Unfortunately, `rustfmt` and
`cargo fmt` have different behaviors for determining which edition to
use when formatting: `cargo fmt` looks for the Rust edition in
`Cargo.toml`, whereas `rustfmt` looks for it in `rustfmt.toml`. As a
result, whenever I save, I have to remember to manually run `cargo fmt`
before committing/pushing.
There is an open issue asking for `rustfmt` to also look at `Cargo.toml`
when it's present (https://github.com/rust-lang/rust.vim/issues/368),
but it seems like they "closed" that issue just by bumping the default
edition (six years ago, from 2015 to 2018).
In the meantime, this PR adds a `rustfmt.toml` file with our current
Rust edition so that both invocation have the same behavior. I don't
love that this duplicates information in `Cargo.toml`, but I've added a
reminder comment there to hopefully ensure that we bump the edition in
both places three years from now.
## Summary
Support direct uses of `typing.TypeAliasType`, as in:
```py
from typing import TypeAliasType
IntOrStr = TypeAliasType("IntOrStr", int | str)
def f(x: IntOrStr) -> None:
reveal_type(x) # revealed: int | str
```
closes https://github.com/astral-sh/ty/issues/392
## Ecosystem
The new false positive here:
```diff
+ error[invalid-type-form] altair/utils/core.py:49:53: The first argument to `Callable` must be either a list of types, ParamSpec, Concatenate, or `...`
```
comes from the fact that we infer the second argument as a type
expression now. We silence false positives for PEP695 `ParamSpec`s, but
not for `P = ParamSpec("P")` inside `Callable[P, ...]`.
## Test Plan
New Markdown tests
renovate[bot]
Alex Waygood
Abhijeet Prasad Bodas
GiGaGon
Carl Meyer
David Peter
Matthew Mckee
Brent Westbrook
Zanie Blue
Leander Cain Slotosch
Avasam
Dhruv Manilawala
Aria Desires
Micha Reiser
Ibraheem Ahmed
Илья Любавский
github-actions[bot]
Dylan
Robsdedude
Dan Parizher
हिमांशु
med1844
InSync
Shunsuke Shibayama
Douglas Creager
Andrew Gallant
Jordy Williams
Yair Peretz
Victor Hugo Gomes
Nikolas Hearp
Josiah Kane
chiri
K
Igor Drokin
Suneet Tipirneni
Vasco Schiavo
Charlie Marsh
Hmvp
Gideon
Yunchi Pang
Gene Parmesan Thomas
Deric Crago
Andrej
Frazer McLean
sobolevn
Alperen Keleş
Felix Scherz
Denys Kyslytsyn
Juriah
𝕂
Jia Chen
justin
DetachHead
Ben Bar-Or
Victorien
lipefree
Robsdedude
Wei Lee
Marcus Näslund
vjurczenia
Hans
Viktor Merkurev
Max Mynter
Maddy Guthridge
Vasanth
David Sherret
Jo
Brandt Bucher
Sumana Harihareswara
Ramil Aleskerov
Adam Aaronson
Emily B. Zhang
Dragon