The diagram is written in the Dot language, which can
be converted to SVG (or any other image) by GraphViz.
I thought it was a good idea to write this down in
preparation for adding routines that list modules.
Code reuse is likely to be difficult and I wanted to
be sure I understood how it worked.
I mostly just did this because the long string literals were annoying
me. And these can make rustfmt give up on formatting.
I also re-flowed some long comment lines while I was here.
I'm not sure if this used to be used elsewhere, but it no longer is.
And it looks like an internal-only helper function, so just un-export
it.
And note that `ModuleNameIngredient` is also un-exported, so this
function isn't really usable outside of its defining module anyway.
Summary
--
I noticed while reviewing #19390 that in `check_tokens` we were still
passing
around an extra `LinterSettings`, despite all of the same functions also
receiving a `LintContext` with its own settings.
This PR adds the `LintContext::settings` method and calls that instead
of using
the separate `LinterSettings`.
Test Plan
--
Existing tests
## Summary
Resolves#19531
I've implemented a check to determine whether the for_stmt target is
declared as global or nonlocal. I believe we should skip the rule in all
such cases, since variables declared this way are intended for use
outside the loop scope, making value changes expected behavior.
## Test Plan
Added two test cases for global and nonlocal variable to snapshot.
This PR improves the "signature help" language server feature in two
ways:
1. It adds support for the recently-introduced "stub mapper" which maps
symbol declarations within stubs to their implementation counterparts.
This allows the signature help to display docstrings from the original
implementation.
2. It incorporates a more robust fix to a bug that was addressed in a
[previous PR](https://github.com/astral-sh/ruff/pull/19542). It also
adds more comprehensive tests to cover this case.
Co-authored-by: UnboundVariable <unbound@gmail.com>
This eliminates the panic reported in
https://github.com/astral-sh/ty/issues/909, though it doesn't address
the underlying cause, which is that we aren't yet checking the types of
the fields of a protocol when checking whether a class implements the
protocol. And in particular, if a class explictly opts out of iteration
via
```py
class NotIterable:
__iter__ = None
```
we currently treat that as "having an `__iter__`" member, and therefore
implementing `Iterable`.
Note that the assumption that was in the comment before is still
correct: call binding will have already checked that the argument
satisfies `Iterable`, and so it shouldn't be an error to iterate over
said argument. But arguably, the new logic in this PR is a better way to
discharge that assumption — instead of panicking if we happen to be
wrong, fall back on an unknown iteration result.
## Summary
Fixes#18844
I'm not too sure if the solution is as simple as the way I implemented
it, but I'm curious to see if we are covering all cases correctly here.
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
As a follow-up to #18949 (suggested
[here](https://github.com/astral-sh/ruff/pull/18949#pullrequestreview-2998417889)),
this PR implements auto-fix logic for `PLC0207`.
## Test Plan
<!-- How was it tested? -->
Existing tests pass, with updates to the snapshot so that it expects the
new output that comes along with the auto-fix.
## Summary
Split the "Generator functions" tests into two parts. The first part
(synchronous) refers to a function called `i` from a function `i2`. But
`i` is later redeclared in the asynchronous part, which was probably not
intended.
As of [this cpython PR](https://github.com/python/cpython/pull/135996),
it is not allowed to concatenate t-strings with non-t-strings,
implicitly or explicitly. Expressions such as `"foo" t"{bar}"` are now
syntax errors.
This PR updates some AST nodes and parsing to reflect this change.
The structural change is that `TStringPart` is no longer needed, since,
as in the case of `BytesStringLiteral`, the only possibilities are that
we have a single `TString` or a vector of such (representing an implicit
concatenation of t-strings). This removes a level of nesting from many
AST expressions (which is what all the snapshot changes reflect), and
simplifies some logic in the implementation of visitors, for example.
The other change of note is in the parser. When we meet an implicit
concatenation of string-like literals, we now count the number of
t-string literals. If these do not exhaust the total number of
implicitly concatenated pieces, then we emit a syntax error. To recover
from this syntax error, we encode any t-string pieces as _invalid_
string literals (which means we flag them as invalid, record their
range, and record the value as `""`). Note that if at least one of the
pieces is an f-string we prefer to parse the entire string as an
f-string; otherwise we parse it as a string.
This logic is exactly the same as how we currently treat
`BytesStringLiteral` parsing and error recovery - and carries with it
the same pros and cons.
Finally, note that I have not implemented any changes in the
implementation of the formatter. As far as I can tell, none are needed.
I did change a few of the fixtures so that we are always concatenating
t-strings with t-strings.
This PR adds support for the "selection range" language server feature.
This feature was recently requested by a ty user in [this feature
request](https://github.com/astral-sh/ty/issues/882).
This feature allows a client to implement "smart selection expansion"
based on the structure of the parse tree. For example, if you type
"shift-ctrl-right-arrow" in VS Code, the current selection will be
expanded to include the parent AST node. Conversely,
"shift-ctrl-left-arrow" shrinks the selection.
We will probably need to tune the granularity of selection expansion
based on user feedback. The initial implementation includes most AST
nodes, but users may find this to be too fine-grained. We have the
option of skipping some AST nodes that are not as meaningful when
editing code.
Co-authored-by: UnboundVariable <unbound@gmail.com>
We now correctly exclude legacy typevars from enclosing scopes when
constructing the generic context for a generic function.
more detail:
A function is generic if it refers to legacy typevars in its signature:
```py
from typing import TypeVar
T = TypeVar("T")
def f(t: T) -> T:
return t
```
Generic functions are allowed to appear inside of other generic
contexts. When they do, they can refer to the typevars of those
enclosing generic contexts, and that should not rebind the typevar:
```py
from typing import TypeVar, Generic
T = TypeVar("T")
U = TypeVar("U")
class C(Generic[T]):
@staticmethod
def method(t: T, u: U) -> None: ...
# revealed: def method(t: int, u: U) -> None
reveal_type(C[int].method)
```
This substitution was already being performed correctly, but we were
also still including the enclosing legacy typevars in the method's own
generic context, which can be seen via `ty_extensions.generic_context`
(which has been updated to work on generic functions and methods):
```py
from ty_extensions import generic_context
# before: tuple[T, U]
# after: tuple[U]
reveal_type(generic_context(C[int].method))
```
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Changing `BLE001` (blind-except) so that it does not flag `except`
clauses which include `logging.critical(..., exc_info=True)`.
## Test Plan
It passes the following (whereas the `main` branch does not):
```sh
$ cargo run -p ruff -- check somefile.py --no-cache --select=BLE001
```
```python
# somefile.py
import logging
try:
print("Hello world!")
except Exception:
logging.critical("Did not run.", exc_info=True)
```
Related: https://github.com/astral-sh/ruff/issues/19519
Small rewording to indicate that core development is done but that we
may add breaking changes.
Feel free to bikeshed!
Test:
```console
❯ echo "t''" | cargo run -p ruff -- check --no-cache --isolated --target-version py314 -
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
Running `target/debug/ruff check --no-cache --isolated --target-version py314 -`
warning: Support for Python 3.14 is in preview and may undergo breaking changes. Enable `preview` to remove this warning.
All checks passed!
```
This PR adds support for "document symbols" and "workspace symbols"
language server features. Most of the logic to implement these features
is shared.
The "document symbols" feature returns a list of all symbols within a
specified source file. Clients can specify whether they want a flat or
hierarchical list. Document symbols are typically presented by a client
in an "outline" form. Here's what this looks like in VS Code, for
example.
<img width="240" height="249" alt="image"
src="https://github.com/user-attachments/assets/82b11f4f-32ec-4165-ba01-d6496ad13bdf"
/>
The "workspace symbols" feature returns a list of all symbols across the
entire workspace that match some user-supplied query string. This allows
the user to quickly find and navigate to any symbol within their code.
<img width="450" height="134" alt="image"
src="https://github.com/user-attachments/assets/aac131e0-9464-4adf-8a6c-829da028c759"
/>
---------
Co-authored-by: UnboundVariable <unbound@gmail.com>
Summary
--
I looked at other uses of `TextEmitter`, and I think this should be the
only one affected by this. The other integration tests must work
properly since they're run with `assert_cmd_snapshot!`, which I assume
triggers the `SHOULD_COLORIZE` case, and the `cfg!(test)` check will
work for uses in `ruff_linter`.
4a4dc38b5b/crates/ruff_linter/src/message/text.rs (L36-L44)
Alternatively, we could probably move this to a CLI test instead.
Test Plan
--
`cargo test -p ruff`, which was failing on `main` with color codes in
the output before this
## Summary
We currently infer a `@Todo` type whenever we access an attribute on an
intersection type with negative components. This can happen very
naturally. Consequently, this `@Todo` type is rather pervasive and hides
a lot of true positives that ty could otherwise detect:
```py
class Foo:
attr: int = 1
def _(f: Foo | None):
if f:
reveal_type(f) # Foo & ~AlwaysFalsy
reveal_type(f.attr) # now: int, previously: @Todo
```
The changeset here proposes to handle member access on these
intersection types by simply ignoring all negative contributions. This
is not always ideal: a negative contribution like `~<Protocol with
members 'attr'>` could be a hint that `.attr` should not be accessible
on the full intersection type. The behavior can certainly be improved in
the future, but this seems like a reasonable initial step to get rid of
this unnecessary `@Todo` type.
## Ecosystem analysis
There are quite a few changes here. I spot-checked them and found one
bug where attribute access on pure negation types (`~P == object & ~P`)
would not allow attributes on `object` to be accessed. After that was
fixed, I only see true positives and known problems. The fact that a lot
of `unused-ignore-comment` diagnostics go away are also evidence for the
fact that this touches a sensitive area, where static analysis clashes
with dynamically adding attributes to objects:
```py
… # type: ignore # Runtime attribute access
```
## Test Plan
Updated tests.
## Summary
Add basic support for `dataclasses.field`:
* remove fields with `init=False` from the signature of the synthesized
`__init__` method
* infer correct default value types from `default` or `default_factory`
arguments
```py
from dataclasses import dataclass, field
def default_roles() -> list[str]:
return ["user"]
@dataclass
class Member:
name: str
roles: list[str] = field(default_factory=default_roles)
tag: str | None = field(default=None, init=False)
# revealed: (self: Member, name: str, roles: list[str] = list[str]) -> None
reveal_type(Member.__init__)
```
Support for `kw_only` has **not** been added.
part of https://github.com/astral-sh/ty/issues/111
## Test Plan
New Markdown tests
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Micha Reiser <micha@reiser.io>
This PR fixes bug [#879](https://github.com/astral-sh/ty/issues/879)
where the signature help popup remains visible after typing the closing
paren in a call expression.
Co-authored-by: UnboundVariable <unbound@gmail.com>
This PR adds support for the "document highlights" language server
feature.
This feature allows a client to highlight all instances of a selected
name within a document. Without this feature, editors perform
highlighting based on a simple text match. This adds semantic knowledge.
The implementation of this feature largely overlaps that of the
recently-added "references" feature. This PR refactors the existing
"references.rs" module, separating out the functionality and tests that
are specific to the other language feature into a "goto_references.rs"
module. The "references.rs" module now contains the functionality that
is common to "goto references", "document highlights" and "rename"
(which is not yet implemented).
As part of this PR, I also created a new `ReferenceTarget` type which is
similar to the existing `NavigationTarget` type but better suited for
references. This idea was suggested by @MichaReiser in [this code review
feedback](https://github.com/astral-sh/ruff/pull/19475#discussion_r2224061006)
from a previous PR. Notably, this new type contains a field that
specifies the "kind" of the reference (read, write or other). This
"kind" is needed for the document highlights feature.
Before: all textual instances of `foo` are highlighted
<img width="156" height="126" alt="Screenshot 2025-07-23 at 12 51 09 PM"
src="https://github.com/user-attachments/assets/37ccdb2f-d48a-473d-89d5-8e89cb6c394e"
/>
After: only semantic matches are highlighted
<img width="164" height="157" alt="Screenshot 2025-07-23 at 12 52 05 PM"
src="https://github.com/user-attachments/assets/2efadadd-4691-4815-af04-b031e74c81b7"
/>
---------
Co-authored-by: UnboundVariable <unbound@gmail.com>
## Summary
Expand cases in which ruff can offer a fix for `RUF039` (some of which
are unsafe).
While turning `"\n"` (== `\n`) into `r"\n"` (== `\\n`) is not equivalent
at run-time, it's still functionally equivalent to do so in the context
of [regex
patterns](https://docs.python.org/3/library/re.html#regular-expression-syntax)
as they themselves interpret the escape sequence. Therefore, an unsafe
fix can be offered.
Further, this PR also makes ruff offer fixes for byte string literals,
not only strings literals as before.
## Test Plan
Tests for all escape sequences have been added.
## Related
Closes: https://github.com/astral-sh/ruff/issues/16713
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
## Summary
I saw that this creates a lot of false positives in the ecosystem, and
it seemed to be relatively easy to add basic support for this.
Some preliminary work on this was done by @InSyncWithFoo — thank you.
part of https://github.com/astral-sh/ty/issues/111
## Ecosystem analysis
The results look good.
## Test Plan
New Markdown tests
---------
Co-authored-by: InSync <insyncwithfoo@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
The generated fix for `RUF033` would cause a syntax error for named
expressions as parameter defaults.
```python
from dataclasses import InitVar, dataclass
@dataclass
class Foo:
def __post_init__(self, bar: int = (x := 1)) -> None:
pass
```
would be turned into
```python
from dataclasses import InitVar, dataclass
@dataclass
class Foo:
x: InitVar[int] = x := 1
def __post_init__(self, bar: int = (x := 1)) -> None:
pass
```
instead of the syntactically correct
```python
# ...
x: InitVar[int] = (x := 1)
# ...
```
## Test Plan
Test reproducer (plus some extra tests) have been added to the test
suite.
## Related
Fixes: https://github.com/astral-sh/ruff/issues/18950
This PR updates our iterator protocol machinery to return a tuple spec
describing the elements that are returned, instead of a type. That
allows us to track heterogeneous iterators more precisely, and
consolidates the logic in unpacking and splatting, which are the two
places where we can take advantage of that more precise information.
(Other iterator consumers, like `for` loops, have to collapse the
iterated elements down to a single type regardless, and we provide a new
helper method on `TupleSpec` to perform that summarization.)
## Summary
Implements proper reachability analysis and — in effect — exhaustiveness
checking for `match` statements. This allows us to check the following
code without any errors (leads to *"can implicitly return `None`"* on
`main`):
```py
from enum import Enum, auto
class Color(Enum):
RED = auto()
GREEN = auto()
BLUE = auto()
def hex(color: Color) -> str:
match color:
case Color.RED:
return "#ff0000"
case Color.GREEN:
return "#00ff00"
case Color.BLUE:
return "#0000ff"
```
Note that code like this already worked fine if there was a
`assert_never(color)` statement in a catch-all case, because we would
then consider that `assert_never` call terminal. But now this also works
without the wildcard case. Adding a member to the enum would still lead
to an error here, if that case would not be handled in `hex`.
What needed to happen to support this is a new way of evaluating match
pattern constraints. Previously, we would simply compare the type of the
subject expression against the patterns. For the last case here, the
subject type would still be `Color` and the value type would be
`Literal[Color.BLUE]`, so we would infer an ambiguous truthiness.
Now, before we compare the subject type against the pattern, we first
generate a union type that corresponds to the set of all values that
would have *definitely been matched* by previous patterns. Then, we
build a "narrowed" subject type by computing `subject_type &
~already_matched_type`, and compare *that* against the pattern type. For
the example here, `already_matched_type = Literal[Color.RED] |
Literal[Color.GREEN]`, and so we have a narrowed subject type of `Color
& ~(Literal[Color.RED] | Literal[Color.GREEN]) = Literal[Color.BLUE]`,
which allows us to infer a reachability of `AlwaysTrue`.
<details>
<summary>A note on negated reachability constraints</summary>
It might seem that we now perform duplicate work, because we also record
*negated* reachability constraints. But that is still important for
cases like the following (and possibly also for more realistic
scenarios):
```py
from typing import Literal
def _(x: int | str):
match x:
case None:
pass # never reachable
case _:
y = 1
y
```
</details>
closes https://github.com/astral-sh/ty/issues/99
## Test Plan
* I verified that this solves all examples from the linked ticket (the
first example needs a PEP 695 type alias, because we don't support
legacy type aliases yet)
* Verified that the ecosystem changes are all because of removed false
positives
* Updated tests
## Summary
I noticed that our type narrowing and reachability analysis was
incorrect for class patterns that are not irrefutable. The test cases
below compare the old and the new behavior:
```py
from dataclasses import dataclass
@dataclass
class Point:
x: int
y: int
class Other: ...
def _(target: Point):
y = 1
match target:
case Point(0, 0):
y = 2
case Point(x=0, y=1):
y = 3
case Point(x=1, y=0):
y = 4
reveal_type(y) # revealed: Literal[1, 2, 3, 4] (previously: Literal[2])
def _(target: Point | Other):
match target:
case Point(0, 0):
reveal_type(target) # revealed: Point
case Point(x=0, y=1):
reveal_type(target) # revealed: Point (previously: Never)
case Point(x=1, y=0):
reveal_type(target) # revealed: Point (previously: Never)
case Other():
reveal_type(target) # revealed: Other (previously: Other & ~Point)
```
## Test Plan
New Markdown test
## Summary
closes#19204
## Test Plan
1. test case is added in dedicated file
2. locally tested the code manually
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: CodeMan62 <sharmahimanshu150082007@gmail.com>
## Summary
This PR moves most of the work of rendering concise diagnostics in Ruff
into `ruff_db`, where the code is shared with ty. To accomplish this
without breaking backwards compatibility in Ruff, there are two main
changes on the `ruff_db`/ty side:
- Added the logic from Ruff for remapping notebook line numbers to cells
- Reordered the fields in the diagnostic to match Ruff and rustc
```text
# old
error[invalid-assignment] try.py:3:1: Object of type `Literal[1]` is not
assignable to `str`
# new
try.py:3:1: error[invalid-assignment]: Object of type `Literal[1]` is
not assignable to `str`
```
I don't think the notebook change failed any tests on its own, and only
a handful of snaphots changed in ty after reordering the fields, but
this will obviously affect any other uses of the concise format, outside
of tests, too.
The other big change should only affect Ruff:
- Added three new `DisplayDiagnosticConfig` options
Micha and I hoped that we could get by with one option
(`hide_severity`), but Ruff also toggles `show_fix_status` itself,
independently (there are cases where we want neither severity nor the
fix status), and during the implementation I realized we also needed
access to an `Applicability`. The main goal here is to suppress the
severity (`error` above) because ruff only uses the `error` severity and
to use the secondary/noqa code instead of the line name
(`invalid-assignment` above).
```text
# ty - same as "new" above
try.py:3:1: error[invalid-assignment]: Object of type `Literal[1]` is
not assignable to `str`
# ruff
try.py:3:1: RUF123 [*] Object of type `Literal[1]` is not assignable to
`str`
```
This part of the concise diagnostic is actually shared with the `full`
output format in Ruff, but with the settings above, there are no
snapshot changes to either format.
## Test Plan
Existing tests with the handful of updates mentioned above, as well as
some new tests in the `concise` module.
Also this PR. Swapping the fields might have broken mypy_primer, unless
it occasionally times out on its own.
I also ran this script in the root of my Ruff checkout, which also has
CPython in it:
```shell
flags=(--isolated --no-cache --no-respect-gitignore --output-format concise .)
diff <(target/release/ruff check ${flags[@]} 2> /dev/null) \
<(ruff check ${flags[@]} 2> /dev/null)
```
This yielded an expected diff due to some t-string error changes on main
since 0.12.4:
```diff
33622c33622
< crates/ruff_python_parser/resources/inline/err/f_string_lambda_without_parentheses.py:1:15: SyntaxError: Expected an element of or the end of the f-string
---
> crates/ruff_python_parser/resources/inline/err/f_string_lambda_without_parentheses.py:1:15: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
33742c33742
< crates/ruff_python_parser/resources/inline/err/implicitly_concatenated_unterminated_string_multiline.py:4:1: SyntaxError: Expected an element of or the end of the f-string
---
> crates/ruff_python_parser/resources/inline/err/implicitly_concatenated_unterminated_string_multiline.py:4:1: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
34131c34131
< crates/ruff_python_parser/resources/inline/err/t_string_lambda_without_parentheses.py:2:15: SyntaxError: Expected an element of or the end of the t-string
---
> crates/ruff_python_parser/resources/inline/err/t_string_lambda_without_parentheses.py:2:15: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
```
So modulo color, the results are identical on 38,186 errors in our test
suite and CPython 3.10.
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
We previously didn't recognize `Literal[Color.RED]` as single-valued, if
the enum also derived from `str` or `int`:
```py
from enum import Enum
class Color(str, Enum):
RED = "red"
GREEN = "green"
BLUE = "blue"
def _(color: Color):
if color == Color.RED:
reveal_type(color) # previously: Color, now: Literal[Color.RED]
```
The reason for that was that `int` and `str` have "custom" `__eq__` and
`__ne__` implementations that return `bool`. We do not treat enum
literals from classes with custom `__eq__` and `__ne__` implementations
as single-valued, but of course we know that `int.__eq__` and
`str.__eq__` are well-behaved.
## Test Plan
New Markdown tests.
This makes caching of submodules independent of whether `Module`
is itself a Salsa ingredient. In fact, this makes the work done in
the prior commit superfluous. But we're possibly keeping it as an
ingredient for now since it's a bit of a tedious change and we might
need it in the near future.
Ref https://github.com/astral-sh/ruff/pull/19495#pullrequestreview-3045736715
## Summary
Add more precise type inference for a limited set of `isinstance(…)`
calls, i.e. return `Literal[True]` if we can be sure that this is the
correct result. This improves exhaustiveness checking / reachability
analysis for if-elif-else chains with `isinstance` checks. For example:
```py
def is_number(x: int | str) -> bool: # no "can implicitly return `None` error here anymore
if isinstance(x, int):
return True
elif isinstance(x, str):
return False
# code here is now detected as being unreachable
```
This PR also adds a new test suite for exhaustiveness checking.
## Test Plan
New Markdown tests
### Ecosystem analysis
The removed diagnostics look good. There's [one
case](f52c4f1afd/torchvision/io/video_reader.py (L125-L143))
where a "true positive" is removed in unreachable code. `src` is
annotated as being of type `str`, but there is an `elif isinstance(src,
bytes)` branch, which we now detect as unreachable. And so the
diagnostic inside that branch is silenced. I don't think this is a
problem, especially once we have a "graying out" feature, or a lint that
warns about unreachable code.
## Summary
Fixes https://github.com/astral-sh/ty/issues/874
Labeling this as `internal`, since we haven't released the
enum-expansion feature.
## Test Plan
New Markdown tests
## Summary
Closes: astral-sh/ty#88
This PR implements an initial version of a mock language server that can
be used to write e2e tests using the real server running in the
background.
The way it works is that you'd use the `TestServerBuilder` to help
construct the `TestServer` with the setup data. This could be the
workspace folders, populating the file and it's content in the memory
file system, setting the right client capabilities to make the server
respond correctly, etc. This can be expanded as we write more test
cases.
There are still a few things to follow-up on:
- ~In the `Drop` implementation, we should assert that there are no
pending notification, request and responses from the server that the
test code hasn't handled yet~ Implemented in [`afd1f82`
(#19391)](afd1f82bde)
- Reduce the setup boilerplate in any way we can
- Improve the final assertion, currently I'm just snapshotting the final
output
## Test Plan
Written a few test cases.
## Summary
This PR implements the following section from the [typing spec on
enums](https://typing.python.org/en/latest/spec/enums.html#enum-definition):
> Enum classes can also be defined using a subclass of `enum.Enum` **or
any class that uses `enum.EnumType` (or a subclass thereof) as a
metaclass**. Note that `enum.EnumType` was named `enum.EnumMeta` prior
to Python 3.11.
part of https://github.com/astral-sh/ty/issues/183
## Test Plan
New Markdown tests
This PR updates our call binding logic to handle splatted arguments.
Complicating matters is that we have separated call bind analysis into
two phases: parameter matching and type checking. Parameter matching
looks at the arity of the function signature and call site, and assigns
arguments to parameters. Importantly, we don't yet know the type of each
argument! This is needed so that we can decide whether to infer the type
of each argument as a type form or value form, depending on the
requirements of the parameter that the argument was matched to.
This is an issue when splatting an argument, since we need to know how
many elements the splatted argument contains to know how many positional
parameters to match it against. And to know how many elements the
splatted argument has, we need to know its type.
To get around this, we now make the assumption that splatted arguments
can only be used with value-form parameters. (If you end up splatting an
argument into a type-form parameter, we will silently pass in its
value-form type instead.) That allows us to preemptively infer the
(value-form) type of any splatted argument, so that we have its arity
available during parameter matching. We defer inference of non-splatted
arguments until after parameter matching has finished, as before.
We reuse a lot of the new tuple machinery to make this happen — in
particular resizing the tuple spec representing the number of arguments
passed in with the tuple length representing the number of parameters
the splat was matched with.
This work also shows that we might need to change how we are performing
argument expansion during overload resolution. At the moment, when we
expand parameters, we assume that each argument will still be matched to
the same parameters as before, and only retry the type-checking phase.
With splatted arguments, this is no longer the case, since the inferred
arity of each union element might be different than the arity of the
union as a whole, which can affect how many parameters the splatted
argument is matched to. See the regression test case in
`mdtest/call/function.md` for more details.
## Summary
Infer the correct type in a scenario like this:
```py
class Color(Enum):
RED = 1
GREEN = 2
BLUE = 3
for color in Color:
reveal_type(color) # revealed: Color
```
We should eventually support this out-of-the-box when
https://github.com/astral-sh/ty/issues/501 is implemented. For this
reason, @AlexWaygood would prefer to keep things as they are (we
currently infer `Unknown`, so false positives seem unlikely). But it
seemed relatively easy to support, so I'm opening this for discussion.
part of https://github.com/astral-sh/ty/issues/183
## Test Plan
Adapted existing test.
## Ecosystem analysis
```diff
- warning[unused-ignore-comment] rotkehlchen/chain/aggregator.py:591:82: Unused blanket `type: ignore` directive
```
This `unused-ignore-comment` goes away due to a new true positive.
## Summary
Fixes pull-types panics for illegal annotations like
`Literal[object[index]]`.
Originally reported by @AlexWaygood
## Test Plan
* Verified that this caused panics in the playground, when typing (and
potentially hovering over) `x: Literal[obj[0]]`.
* Added a regression test
Summary
--
This PR tweaks Ruff's internal usage of the new diagnostic model to more
closely
match the intended use, as I understand it. Specifically, it moves the
fix/help
suggestion from the primary annotation's message to a subdiagnostic. In
turn, it
adds the secondary/noqa code as the new primary annotation message. As
shown in
the new `ruff_db` tests, this more closely mirrors Ruff's current
diagnostic
output.
I also added `Severity::Help` to render the fix suggestion with a
`help:` prefix
instead of `info:`.
These changes don't have any external impact now but should help a bit
with #19415.
Test Plan
--
New full output format tests in `ruff_db`
Rendered Diagnostics
--
Full diagnostic output from `annotate-snippets` in this PR:
```
error[unused-import]: `os` imported but unused
--> fib.py:1:8
|
1 | import os
| ^^
|
help: Remove unused import: `os`
```
Current Ruff output for the same code:
```
fib.py:1:8: F401 [*] `os` imported but unused
|
1 | import os
| ^^ F401
|
= help: Remove unused import: `os`
```
Proposed final output after #19415:
```
F401 [*] `os` imported but unused
--> fib.py:1:8
|
1 | import os
| ^^
|
help: Remove unused import: `os`
```
These are slightly updated from
https://github.com/astral-sh/ruff/pull/19464#issuecomment-3097377634
below to remove the extra noqa codes in the primary annotation messages
for the first and third cases.
This implements mapping of definitions in stubs to definitions in the
"real" implementation using the approach described in
https://github.com/astral-sh/ty/issues/788#issuecomment-3097000287
I've tested this with goto-definition in vscode with code that uses
`colorama` and `types-colorama`.
Notably this implementation does not add support for stub-mapping stdlib
modules, which can be done as an essentially orthogonal followup in the
implementation of `resolve_real_module`.
Part of https://github.com/astral-sh/ty/issues/788
## Summary
It was faster to implement this then to write the ticket: Disallow
`ClassVar` annotations almost everywhere outside of class body scopes.
## Test Plan
New Markdown tests
## Summary
Disallow `Final` in function parameter- and return-type annotations.
[Typing
spec](https://typing.python.org/en/latest/spec/qualifiers.html#uppercase-final):
> `Final` may only be used in assignments or variable annotations. Using
it in any other position is an error. In particular, `Final` can’t be
used in annotations for function arguments
## Test Plan
Updated MD test
I noticed that the semantic token implementation was not handling
identifiers in a few cases. This adds support for identifiers that
appear in `except`, `case`, `nonlocal`, and `global` statements.
Co-authored-by: UnboundVariable <unbound@gmail.com>
This is a follow-on to #19410 that further reduces the memory usage of
our reachability constraints. When finishing the building of a use-def
map, we walk through all of the "final" states and mark only those
reachability constraints as "used". We then throw away the interior TDD
nodes of any reachability constraints that weren't marked as used.
(This helps because we build up quite a few intermediate TDD nodes when
constructing complex reachability constraints. These nodes can never be
accessed if they were _only_ used as an intermediate TDD node. The
marking step ensures that we keep any nodes that ended up being referred
to in some accessible use-def map state.)
## Summary
Adds proper type inference for implicit instance attributes that are
declared with a "bare" `Final` and adds `invalid-assignment` diagnostics
for all implicit instance attributes that are declared `Final` or
`Final[…]`.
## Test Plan
New and updated MD tests.
## Ecosystem analysis
```diff
pytest (https://github.com/pytest-dev/pytest)
+ error[invalid-return-type] src/_pytest/fixtures.py:1662:24: Return type does not match returned value: expected `Scope`, found `Scope | (Unknown & ~None & ~((...) -> object) & ~str) | (((str, Config, /) -> Unknown) & ~((...) -> object) & ~str) | (Unknown & ~str)
```
The definition of the `scope` attribute is [here](
5f99385635/src/_pytest/fixtures.py (L1020-L1028)).
Looks like this is a new false positive due to missing `TypeAlias`
support that is surfaced here because we now infer a more precise type
for `FixtureDef._scope`.
## Summary
Implement expansion of enums into unions of enum literals (and the
reverse operation). For the enum below, this allows us to understand
that `Color = Literal[Color.RED, Color.GREEN, Color.BLUE]`, or that
`Color & ~Literal[Color.RED] = Literal[Color.GREEN, Color.BLUE]`. This
helps in exhaustiveness checking, which is why we see some removed
`assert_never` false positives. And since exhaustiveness checking also
helps with understanding terminal control flow, we also see a few
removed `invalid-return-type` and `possibly-unresolved-reference` false
positives. This PR also adds expansion of enums in overload resolution
and type narrowing constructs.
```py
from enum import Enum
from typing_extensions import Literal, assert_never
from ty_extensions import Intersection, Not, static_assert, is_equivalent_to
class Color(Enum):
RED = 1
GREEN = 2
BLUE = 3
type Red = Literal[Color.RED]
type Green = Literal[Color.GREEN]
type Blue = Literal[Color.BLUE]
static_assert(is_equivalent_to(Red | Green | Blue, Color))
static_assert(is_equivalent_to(Intersection[Color, Not[Red]], Green | Blue))
def color_name(color: Color) -> str: # no error here (we detect that this can not implicitly return None)
if color is Color.RED:
return "Red"
elif color is Color.GREEN:
return "Green"
elif color is Color.BLUE:
return "Blue"
else:
assert_never(color) # no error here
```
## Performance
I avoided an initial regression here for large enums, but the
`UnionBuilder` and `IntersectionBuilder` parts can certainly still be
optimized. We might want to use the same technique that we also use for
unions of other literals. I didn't see any problems in our benchmarks so
far, so this is not included yet.
## Test Plan
Many new Markdown tests
## Summary
Emit errors for the following assignments:
```py
class C:
CLASS_LEVEL_CONSTANT: Final[int] = 1
C.CLASS_LEVEL_CONSTANT = 2
C().CLASS_LEVEL_CONSTANT = 2
```
## Test Plan
Updated and new MD tests
Parsing the (invalid) expression `f"{\t"i}"` caused a panic because the
`TStringMiddle` character was "unreachable" due the way the parser
recovered from the line continuation (it ate the t-string start).
The cause of the issue is as follows:
The parser begins parsing the f-string and expects to see a list of
objects, essentially alternating between _interpolated elements_ and
ordinary strings. It is happy to see the first left brace, but then
there is a lexical error caused by the line-continuation character. So
instead of the parser seeing a list of elements with just one member, it
sees a list that starts like this:
- Interpolated element with an invalid token, stored as a `Name`
- Something else built from tokens beginning with `TStringStart` and
`TStringMiddle`
When it sees the `TStringStart` error recovery says "that's a list
element I don't know what to do with, let's skip it". When it sees
`TStringMiddle` it says "oh, that looks like the middle of _some
interpolated string_ so let's try to parse it as one of the literal
elements of my `FString`". Unfortunately, the function being used to
parse individual list elements thinks (arguably correctly) that it's not
possible to have a `TStringMiddle` sitting in your `FString`, and hits
`unreachable`.
Two potential ways (among many) to solve this issue are:
1. Allow a `TStringMiddle` as a valid "literal" part of an f-string
during parsing (with the hope/understanding that this would only occur
in an invalid context)
2. Skip the `TStringMiddle` as an "unexpected/invalid list item" in the
same way that we skipped `TStringStart`.
I have opted for the second approach since it seems somehow more morally
correct, even though it loses more information. To implement this, the
recovery context needs to know whether we are in an f-string or t-string
- hence the changes to that enum. As a bonus we get slightly more
specific error messages in some cases.
Closes#18860
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Closes#18739
## Test Plan
<!-- How was it tested? -->
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Andrew Gallant
Dan Parizher
हिमांशु
justin
Alex Waygood
Brent Westbrook
Charlie Marsh
github-actions[bot]
Igor Drokin
UnboundVariable
Douglas Creager
Junhson Jean-Baptiste
David Peter
Micha Reiser
Dylan
Elliot Simpson
Shunsuke Shibayama
Dhruv Manilawala
Carl Meyer
Robsdedude
chiri
frank
Jack O'Connor
Aria Desires
David Peter