Commit Graph

105 Commits

Author SHA1 Message Date
Shunsuke Shibayama 83a00c0ac8
[ty] follow the breaking API changes made in salsa-rs/salsa#1015 (#21117) 2025-10-29 14:56:12 +00:00
Micha Reiser 76a55314e4
Fix rare multithreaded related hang (#21038) 2025-10-23 09:25:16 +02:00
Douglas Creager 766ed5b5f3
[ty] Some more simplifications when rendering constraint sets (#21009)
This PR adds another useful simplification when rendering constraint
sets: `T = int` instead of `T = int ∧ T ≠ str`. (The "smaller"
constraint `T = int` implies the "larger" constraint `T ≠ str`.
Constraint set clauses are intersections, and if one constraint in a
clause implies another, we can throw away the "larger" constraint.)

While we're here, we also normalize the bounds of a constraint, so that
we equate e.g. `T ≤ int | str` with `T ≤ str | int`, and change the
ordering of BDD variables so that all constraints with the same typevar
are ordered adjacent to each other.

Lastly, we also add a new `display_graph` helper method that prints out
the full graph structure of a BDD.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-10-22 13:38:44 -04:00
Bhuminjay Soni 3dd78e711e
[syntax-errors] Name is parameter and global (#20426)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
This PR implements a new semantic syntax error where name is parameter &
global.

## Test Plan

<!-- How was it tested? -->
I have written inline test as directed in #17412

---------

Signed-off-by: 11happy <soni5happy@gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2025-10-21 16:51:16 +00:00
Shunsuke Shibayama a7c38eb122
[ty] display variance on hover over type variables (#20900) 2025-10-20 17:28:36 +00:00
Alex Waygood c7e2bfd759
[ty] `continue` and `break` statements outside loops are syntax errors (#20944)
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2025-10-17 17:13:40 +00:00
Aria Desires 7155a62e5c
[ty] Add version hint for failed stdlib attribute accesses (#20909)
This is the ultra-minimal implementation of

* https://github.com/astral-sh/ty/issues/296

that was previously discussed as a good starting point. In particular we
don't actually bother trying to figure out the exact python versions,
but we still mention "hey btw for No Reason At All... you're on python
3.10" when you try to access something that has a definition rooted in
the stdlib that we believe exists sometimes.
2025-10-16 14:07:33 +00:00
Shunsuke Shibayama 9de34e7ac1
[ty] refactor `Place` (#20871)
## Summary

Part of astral-sh/ty#1341

The following changes will be made to `Place`.

* Introduce `TypeOrigin`
* `Place::Type` -> `Place::Defined`
* `Place::Unbound` -> `Place::Undefined`
* `Boundness` -> `Definedness`

`TypeOrigin::Declared`+`Definedness::PossiblyUndefined` are patterns
that weren't considered before, but this PR doesn't address them yet,
only refactors.

## Test Plan

Refactoring
2025-10-15 20:19:19 +02:00
David Peter f73bb45be6
[ty] Rename Type unwrapping methods (#20857)
## Summary

Rename "unwrapping" methods on `Type` from e.g.
`Type::into_class_literal` to `Type::as_class_literal`. I personally
find that name more intuitive, since no transformation of any kind is
happening. We are just unwrapping from certain enum variants. An
alternative would be `try_as_class_literal`, which would follow the
[`strum` naming
scheme](https://docs.rs/strum/latest/strum/derive.EnumTryAs.html), but
is slightly longer.

Also rename `Type::into_callable` to `Type::try_upcast_to_callable`.
Note that I intentionally kept names like
`FunctionType::into_callable_type`, because those return `CallableType`,
not `Option<Type<…>>`.

## Test Plan

Pure refactoring
2025-10-14 09:53:29 +02:00
Bhuminjay Soni 2b729b4d52
[syntax-errors]: break outside loop F701 (#20556)
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
  requests.)
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

This PR implements https://docs.astral.sh/ruff/rules/break-outside-loop/
(F701) as a semantic syntax error.

## Test Plan

<!-- How was it tested? -->

---------

Signed-off-by: 11happy <soni5happy@gmail.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2025-10-13 20:00:59 +00:00
Carl Meyer 8248193ed9
[ty] defer inference of legacy TypeVar bound/constraints/defaults (#20598)
## Summary

This allows us to handle self-referential bounds/constraints/defaults
without panicking.

Handles more cases from https://github.com/astral-sh/ty/issues/256

This also changes the way we infer the types of legacy TypeVars. Rather
than understanding a constructor call to `typing[_extension].TypeVar`
inside of any (arbitrarily nested) expression, and having to use a
special `assigned_to` field of the semantic index to try to best-effort
figure out what name the typevar was assigned to, we instead understand
the creation of a legacy `TypeVar` only in the supported syntactic
position (RHS of a simple un-annotated assignment with one target). In
any other position, we just infer it as creating an opaque instance of
`typing.TypeVar`. (This behavior matches all other type checkers.)

So we now special-case TypeVar creation in `TypeInferenceBuilder`, as a
special case of an assignment definition, rather than deeper inside call
binding. This does mean we re-implement slightly more of
argument-parsing, but in practice this is minimal and easy to handle
correctly.

This is easier to implement if we also make the RHS of a simple (no
unpacking) one-target assignment statement no longer a standalone
expression. Which is fine to do, because simple one-target assignments
don't need to infer the RHS more than once. This is a bonus performance
(0-3% across various projects) and significant memory-usage win, since
most assignment statements are simple one-target assignment statements,
meaning we now create many fewer standalone-expression salsa
ingredients.

This change does mean that inference of manually-constructed
`TypeAliasType` instances can no longer find its Definition in
`assigned_to`, which regresses go-to-definition for these aliases. In a
future PR, `TypeAliasType` will receive the same treatment that
`TypeVar` did in this PR (moving its special-case inference into
`TypeInferenceBuilder` and supporting it only in the correct syntactic
position, and lazily inferring its value type to support recursion),
which will also fix the go-to-definition regression. (I decided a
temporary edge-case regression is better in this case than doubling the
size of this PR.)

This PR also tightens up and fixes various aspects of the validation of
`TypeVar` creation, as seen in the tests.

We still (for now) treat all typevars as instances of `typing.TypeVar`,
even if they were created using `typing_extensions.TypeVar`. This means
we'll wrongly error on e.g. `T.__default__` on Python 3.11, even if `T`
is a `typing_extensions.TypeVar` instance at runtime. We share this
wrong behavior with both mypy and pyrefly. It will be easier to fix
after we pull in https://github.com/python/typeshed/pull/14840.

There are some issues that showed up here with typevar identity and
`MarkTypeVarsInferable`; the fix here (using the new `original` field
and `is_identical_to` methods on `BoundTypeVarInstance` and
`TypeVarInstance`) is a bit kludgy, but it can go away when we eliminate
`MarkTypeVarsInferable`.

## Test Plan

Added and updated mdtests.

### Conformance suite impact

The impact here is all positive:

* We now correctly error on a legacy TypeVar with exactly one constraint
type given.
* We now correctly error on a legacy TypeVar with both an upper bound
and constraints specified.

### Ecosystem impact

Basically none; in the setuptools case we just issue slightly different
errors on an invalid TypeVar definition, due to the modified validation
code.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-10-09 21:08:37 +00:00
Takayuki Maeda 666f53331f
[`ruff`] Fix minor typos in doc comments (#20623) 2025-09-29 08:56:23 +02:00
Micha Reiser 3ffe56d19d
[ty] Remove unnecessary `FileScopeId` to `ScopeId` conversion (#20481) 2025-09-20 11:20:10 +00:00
Andrew Gallant cf16fc4aa4 [ty] Export some stuff from `ty_python_semantic`
We're going to want to use this outside of `ty_python_semantic`.
Specifically, in `ty_ide`.
2025-09-17 13:59:28 -04:00
Ibraheem Ahmed 36888198a6
[ty] Integrate type context for bidirectional inference (#20337)
## Summary

Adds the infrastructure necessary to perform bidirectional type
inference (https://github.com/astral-sh/ty/issues/168) without any
typing changes.
2025-09-11 15:19:12 -04:00
Douglas Creager abb705aa4e
[ty] Add dedicated variant for `NominalInstance(object)` (#20340)
Previously, `Type::object` would find the definition of the `object`
class in typeshed, load that in (to produce a `ClassLiteral` and
`ClassType`), and then create a `NominalInstance` of that class.

It's possible that we are using a typeshed that doesn't define `object`.
We will not be able to do much useful work with that kind of typeshed,
but it's still a possibility that we have to support at least without
panicking. Previously, we would handle this situation by falling back on
`Unknown`.

In most cases, that's a perfectly fine fallback! But `object` is also
our top type — the type of all values. `Unknown` is _not_ an acceptable
stand-in for the top type.

This PR adds a new `NominalInstance` variant for "instances of
`object`". Unlike other nominal instances, we do not need to load in
`object`'s `ClassType` to instantiate this variant. We will use this new
variant even when the current typeshed does not define an `object`
class, ensuring that we have a fully static representation of our top
type at all times.

There are several operations that need access to a nominal instance's
class, and for this new `object` variant we load it lazily only when
it's needed. That means this operation is now fallible, since this is
where the "typeshed doesn't define `object`" failure shows up.

This new approach also has the benefit of avoiding some salsa cycles
that were cropping up while I was debugging #20093, since the new
constraint set representation was trying to instantiate `Type::object`
while in the middle of processing its definition in typeshed. Cycle
handling was kicking in correctly and returning the `Unknown` fallback
mentioned above. But the constraint set implementation depends on
`Type::object` being a distinct and fully static type, highlighting that
this is a correctness fix, not just an optimization fix.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-09-11 13:02:58 -04:00
Shunsuke Shibayama 8770b95509
[ty] introduce `DivergentType` (#20312)
## Summary

From #17371

In #17371, `DivergentType` was introduced to prevent type inference for
recursive functions from diverging and causing panics. This turned out
to be useful for other divergent type inferences
(https://github.com/astral-sh/ruff/pull/17371#discussion_r2329337965),
so I extracted the introduction part of `DivergentType` into this PR so
that we can use it without waiting for the merge of #17371.

Note that this PR only introduces `DivergentType` and does not actually
address divergent type inference yet. Please refer to
https://github.com/astral-sh/ruff/pull/17371/files#diff-20b910c6e20faa962bb1642e111db1cbad8e66ace089bdd966ac9d7f9fa99ff2R542-R622
etc. when implementing handling of divergence suppression using this
type.

## Test Plan

---------

Co-authored-by: Carl Meyer <carl@oddbird.net>
2025-09-10 08:32:26 -07:00
Shunsuke Shibayama 08a561fc05
[ty] more precise lazy scope place lookup (#19932)
## Summary

This is a follow-up to https://github.com/astral-sh/ruff/pull/19321.

Now lazy snapshots are updated to take into account new bindings on
every symbol reassignment.

```python
def outer(x: A | None):
    if x is None:
        x = A()

    reveal_type(x)  # revealed: A

    def inner() -> None:
        # lazy snapshot: {x: A}
        reveal_type(x)  # revealed: A
    inner()

def outer() -> None:
    x = None

    x = 1

    def inner() -> None:
        # lazy snapshot: {x: Literal[1]} -> {x: Literal[1, 2]}
        reveal_type(x)  # revealed: Literal[1, 2]
    inner()

    x = 2
```

Closes astral-sh/ty#559.

## Test Plan

Some TODOs in `public_types.md` now work properly.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-09-08 21:08:35 +00:00
Alex Waygood 5d52902e18
[ty] Implement the legacy PEP-484 convention for indicating positional-only parameters (#20248)
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-09-05 17:56:06 +01:00
David Peter 9e45bfa9fd
[ty] Cover full range of annotated assignments (#20261)
## Summary

An annotated assignment `name: annotation` without a right-hand side was
previously not covered by the range returned from
`DefinitionKind::full_range`, because we did expand the range to include
the right-hand side (if there was one), but failed to include the
annotation.

## Test Plan

Updated snapshot tests
2025-09-05 10:12:40 +02:00
David Peter b3c4005289
[ty] No boundness analysis for implicit instance attributes (#20128)
## Summary

With this PR, we stop performing boundness analysis for implicit
instance attributes:

```py
class C:
    def __init__(self):
        if False:   
            self.x = 1

C().x  # would previously show an error, with this PR we pretend the attribute exists
```

This PR is potentially just a temporary measure until we find a better
fix. But I have already invested a lot of time trying to find the root
cause of https://github.com/astral-sh/ty/issues/758 (and [this
example](https://github.com/astral-sh/ty/issues/758#issuecomment-3206108262),
which I'm not entirely sure is related) and I still don't understand
what is going on. This PR fixes the performance problems in both of
these problems (in a rather crude way).

The impact of the proposed change on the ecosystem is small, and the
three new diagnostics are arguably true positives (previously hidden
because we considered the code unreachable, based on e.g. `assert`ions
that depended on implicit instance attributes). So this seems like a
reasonable fix for now.

Note that we still support cases like these:

```py
class D:
    if False:  # or any other expression that statically evaluates to `False`
        x: int = 1

D().x  # still an error


class E:
    if False:  # or any other expression that statically evaluates to `False`
        def f(self):
            self.x = 1

E().x  # still an error
```

closes https://github.com/astral-sh/ty/issues/758

## Test Plan

Updated tests, benchmark results
2025-08-28 16:25:07 +02:00
Shaygan Hooshyari d9aaacd01f
[ty] Evaluate reachability of non-definitely-bound to Ambiguous (#19579)
## Summary

closes https://github.com/astral-sh/ty/issues/692

If the expression (or any child expressions) is not definitely bound the
reachability constraint evaluation is determined as ambiguous.

This fixes the infinite cycles panic in the following code:

```py
from typing import Literal

class Toggle:
    def __init__(self: "Toggle"):
        if not self.x:
            self.x: Literal[True] = True
```

Credit of this solution is for David.

## Test Plan

- Added a test case with too many cycle iterations panic.
- Previous tests.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-08-28 14:34:49 +02:00
David Peter 4b80f5fa4f
[ty] Optimize TDD atom ordering (#20098)
## Summary

While looking at some logging output that I added to
`ReachabilityConstraintBuilder::add_and_constraint` in order to debug
https://github.com/astral-sh/ty/issues/1091, I noticed that it seemed to
suggest that the TDD was built in an imbalanced way for code like the
following, where we have a sequence of non-nested `if` conditions:

```py
def f(t1, t2, t3, t4, …):
    x = 0
    if t1:
        x = 1
    if t2:
        x = 2
    if t3:
        x = 3
    if t4:
        x = 4
    …
```

To understand this a bit better, I added some code to the
`ReachabilityConstraintBuilder` to render the resulting TDD. On `main`,
we get a tree that looks like the following, where you can see a pattern
of N sub-trees that grow linearly with N (number of `if` statements).
This results in an overall tree structure that has N² nodes (see graph
below):

<img alt="normal order"
src="https://github.com/user-attachments/assets/aab40ce9-e82a-4fcd-823a-811f05f15f66"
/>

If we zoom in to one of these subgraphs, we can see what the problem is.
When we add new constraints that represent combinations like `t1 AND ~t2
AND ~t3 AND t4 AND …`, they start with the evaluation of "early"
conditions (`t1`, `t2`, …). This means that we have to create new
subgraphs for each new `if` condition because there is little sharing
with the previous structure. We evaluate the Boolean condition in a
right-associative way: `t1 AND (~t2 AND (~t3 AND t4)))`:

<img width="500" align="center"
src="https://github.com/user-attachments/assets/31ea7182-9e00-4975-83df-d980464f545d"
/>

If we change the ordering of TDD atoms, we can change that to a
left-associative evaluation: `(((t1 AND ~t2) AND ~t3) AND t4) …`. This
means that we can re-use previous subgraphs `(t1 AND ~t2)`, which
results in a much more compact graph structure overall (note how "late"
conditions are now at the top, and "early" conditions are further down
in the graph):

<img alt="reverse order"
src="https://github.com/user-attachments/assets/96a6b7c1-3d35-4192-a917-0b2d24c6b144"
/>

If we count the number of TDD nodes for a growing number if `if`
statements, we can see that this change results in a slower growth. It's
worth noting that the growth is still superlinear, though:

<img width="800" height="600" alt="plot"
src="https://github.com/user-attachments/assets/22e8394f-e74e-4a9e-9687-0d41f94f2303"
/>

On the actual code from the referenced ticket (the `t_main.py` file
reduced to its main function, with the main function limited to 2000
lines instead of 11000 to allow the version on `main` to run to
completion), the effect is much more dramatic. Instead of 26 million TDD
nodes (`main`), we now only create 250 thousand (this branch), which is
slightly less than 1%.

The change in this PR allows us to build the semantic index and
type-check the problematic `t_main.py` file in
https://github.com/astral-sh/ty/issues/1091 in 9 seconds. This is still
not great, but an obvious improvement compared to running out of memory
after *minutes* of execution.

An open question remains whether this change is beneficial for all kinds
of code patterns, or just this linear sequence of `if` statements. It
does not seem unreasonable to think that referring to "earlier"
conditions is generally a good idea, but I learned from Doug that it's
generally not possible to find a TDD-construction heuristic that is
non-pathological for all kinds of inputs. Fortunately, it seems like
this change here results in performance improvements across *all of our
benchmarks*, which should increase the confidence in this change:

| Benchmark           | Improvement |
|---------------------|-------------------------|
| hydra-zen           | +13%                    |
| DateType            | +5%                     |
| sympy (walltime)    | +4%                     |
| attrs               | +4%                     |
| pydantic (walltime) | +2%                     |
| pandas (walltime)   | +2%                     |
| altair (walltime)   | +2%                     |
| static-frame        | +2%                     |
| anyio               | +1%                     |
| freqtrade           | +1%                     |
| colour-science      | +1%                     |
| tanjun              | +1%                     |

closes https://github.com/astral-sh/ty/issues/1091

---------

Co-authored-by: Douglas Creager <dcreager@dcreager.net>
2025-08-27 20:42:09 +02:00
Alex Waygood 656fc335f2
[ty] Strict validation of protocol members (#17750) 2025-08-19 22:45:41 +00:00
Shunsuke Shibayama 0e5577ab56
[ty] fix lazy snapshot sweeping in nested scopes (#19908)
## Summary

This PR closes astral-sh/ty#955.

## Test Plan

New test cases in `narrowing/conditionals/nested.md`.
2025-08-14 17:52:52 -07:00
Carl Meyer 5a570c8e6d
[ty] fix deferred name loading in PEP695 generic classes/functions (#19888)
## Summary

For PEP 695 generic functions and classes, there is an extra "type
params scope" (a child of the outer scope, and wrapping the body scope)
in which the type parameters are defined; class bases and function
parameter/return annotations are resolved in that type-params scope.

This PR fixes some longstanding bugs in how we resolve name loads from
inside these PEP 695 type parameter scopes, and also defers type
inference of PEP 695 typevar bounds/constraints/default, so we can
handle cycles without panicking.

We were previously treating these type-param scopes as lazy nested
scopes, which is wrong. In fact they are eager nested scopes; the class
`C` here inherits `int`, not `str`, and previously we got that wrong:

```py
Base = int

class C[T](Base): ...

Base = str
```

But certain syntactic positions within type param scopes (typevar
bounds/constraints/defaults) are lazy at runtime, and we should use
deferred name resolution for them. This also means they can have cycles;
in order to handle that without panicking in type inference, we need to
actually defer their type inference until after we have constructed the
`TypeVarInstance`.

PEP 695 does specify that typevar bounds and constraints cannot be
generic, and that typevar defaults can only reference prior typevars,
not later ones. This reduces the scope of (valid from the type-system
perspective) cycles somewhat, although cycles are still possible (e.g.
`class C[T: list[C]]`). And this is a type-system-only restriction; from
the runtime perspective an "invalid" case like `class C[T: T]` actually
works fine.

I debated whether to implement the PEP 695 restrictions as a way to
avoid some cycles up-front, but I ended up deciding against that; I'd
rather model the runtime name-resolution semantics accurately, and
implement the PEP 695 restrictions as a separate diagnostic on top.
(This PR doesn't yet implement those diagnostics, thus some `# TODO:
error` in the added tests.)

Introducing the possibility of cyclic typevars made typevar display
potentially stack overflow. For now I've handled this by simply removing
typevar details (bounds/constraints/default) from typevar display. This
impacts display of two kinds of types. If you `reveal_type(T)` on an
unbound `T` you now get just `typing.TypeVar` instead of
`typing.TypeVar("T", ...)` where `...` is the bound/constraints/default.
This matches pyright and mypy; pyrefly uses `type[TypeVar[T]]` which
seems a bit confusing, but does include the name. (We could easily
include the name without cycle issues, if there's a syntax we like for
that.)

It also means that displaying a generic function type like `def f[T:
int](x: T) -> T: ...` now displays as `f[T](x: T) -> T` instead of `f[T:
int](x: T) -> T`. This matches pyright and pyrefly; mypy does include
bound/constraints/defaults of typevars in function/callable type
display. If we wanted to add this, we would either need to thread a
visitor through all the type display code, or add a `decycle` type
transformation that replaced recursive reoccurrence of a type with a
marker.

## Test Plan

Added mdtests and modified existing tests to improve their correctness.

After this PR, there's only a single remaining py-fuzzer seed in the
0-500 range that panics! (Before this PR, there were 10; the fuzzer
likes to generate cyclic PEP 695 syntax.)

## Ecosystem report

It's all just the changes to `TypeVar` display.
2025-08-13 15:51:59 -07:00
Aria Desires f0b03c3e86
[ty] resolve docstrings for modules (#19898)
This also reintroduces the `ResolvedDefinition::Module` variant because
reverse-engineering it in several places is a bit confusing. In an ideal
world we wouldn't have `ResolvedDefinition::FileWithRange` as it kinda
kills the ability to do richer analysis, so I want to chip away at its
scope wherever I can (currently it's used to point at asname parts of
import statements when doing `ImportAliasResolution::PreserveAliases`,
and also keyword arguments).

This also makes a kind of odd change to allow a hover to *only* produce
a docstring. This works around an oddity where hovering over a module
name in an import fails to resolve to a `ty` even though hovering over
uses of that imported name *does*.

The two fixed tests reflect the two interesting cases here.
2025-08-13 12:24:01 -04:00
Ibraheem Ahmed f34b65b7a0
[ty] Track heap usage of salsa structs (#19790)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-08-12 13:28:44 +02:00
Micha Reiser 2abd683376
[ty] Short circuit `ReachabilityConstraints::analyze_single` for dynamic types (#19867) 2025-08-11 21:58:34 +02:00
Jack O'Connor 827456f977 [ty] more cases for the class body global fallback 2025-08-07 17:30:27 -07:00
Andrew Gallant d5e1b7983e
[ty] Fix static assertion size check (#19814)
A `Segment` has a `Box` in it, which has a platform dependent size.
Restrict the check to only 64-bit targets.
2025-08-07 13:38:16 -05:00
Micha Reiser 7dfde3b929
Update Rust toolchain to 1.89 (#19807) 2025-08-07 18:21:50 +02:00
Micha Reiser b96aa4605b
[ty] Reduce size of member table (#19572) 2025-08-07 11:16:04 +02:00
Ibraheem Ahmed 21ac16db85
[ty] Avoid overcounting shared memory usage (#19773)
## Summary

Use a global tracker to avoid double counting `Arc` instances.
2025-08-06 15:32:02 -04:00
Shunsuke Shibayama 351121c5c5
[ty] fix incorrect lazy scope narrowing (#19744)
## Summary

This is a follow-up to #19321.

Narrowing constraints introduced in a class scope were not applied even
when they can be applied in lazy nested scopes. This PR fixes so that
they are now applied.
Conversely, there were cases where narrowing constraints were being
applied in places where they should not, so it is also fixed.

## Test Plan

Some TODOs in `narrow/conditionals/nested.md` are now work correctly.
2025-08-04 20:32:08 -07:00
David Peter 739c94f95a
[ty] Support as-patterns in reachability analysis (#19728)
## Summary

Support `as` patterns in reachability analysis:

```py
from typing import assert_never


def f(subject: str | int):
    match subject:
        case int() as x:
            pass
        case str():
            pass
        case _:
            assert_never(subject)  # would previously emit an error
```

Note that we still don't support inferring correct types for the bound
name (`x`).

Closes https://github.com/astral-sh/ty/issues/928

## Test Plan

New Markdown tests
2025-08-04 20:13:50 +02:00
Douglas Creager 06cd249a9b
[ty] Track different uses of legacy typevars, including context when rendering typevars (#19604)
This PR introduces a few related changes:

- We now keep track of each time a legacy typevar is bound in a
different generic context (e.g. class, function), and internally create
a new `TypeVarInstance` for each usage. This means the rest of the code
can now assume that salsa-equivalent `TypeVarInstance`s refer to the
same typevar, even taking into account that legacy typevars can be used
more than once.

- We also go ahead and track the binding context of PEP 695 typevars.
That's _much_ easier to track since we have the binding context right
there during type inference.

- With that in place, we can now include the name of the binding context
when rendering typevars (e.g. `T@f` instead of `T`)
2025-08-01 12:20:32 -04:00
Ibraheem Ahmed 8f8c39c435
Simplify `get_size2` usage (#19643)
## Summary

These were added in the 0.5.0 release.
2025-07-30 15:31:37 -04:00
David Peter eb02aa5676
[ty] Async for loops and async iterables (#19634)
## Summary

Add support for `async for` loops and async iterables.

part of https://github.com/astral-sh/ty/issues/151

## Ecosystem impact

```diff
- boostedblob/listing.py:445:54: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
```

This is correct. We now find a true positive in the `# type: ignore`'d
code.

All of the other ecosystem hits are of the type

```diff
trio (https://github.com/python-trio/trio)
+ src/trio/_core/_tests/test_guest_mode.py:532:24: error[not-iterable] Object of type `MemorySendChannel[int] | MemoryReceiveChannel[int]` may not be iterable
```

The message is correct, because only `MemoryReceiveChannel` has an
`__aiter__` method, but `MemorySendChannel` does not. What's not correct
is our inferred type here. It should be `MemoryReceiveChannel[int]`, not
the union of the two. This is due to missing unpacking support for tuple
subclasses, which @AlexWaygood is working on. I don't think this should
block merging this PR, because those wrong types are already there,
without this PR.

## Test Plan

New Markdown tests and snapshot tests for diagnostics.
2025-07-30 17:40:24 +02:00
David Peter 4ecf1d205a
[ty] Support `async`/`await`, `async with` and `yield from` (#19595)
## Summary

- Add support for the return types of `async` functions
- Add type inference for `await` expressions
- Add support for `async with` / async context managers
- Add support for `yield from` expressions

This PR is generally lacking proper error handling in some cases (e.g.
illegal `__await__` attributes). I'm planning to work on this in a
follow-up.

part of https://github.com/astral-sh/ty/issues/151

closes https://github.com/astral-sh/ty/issues/736

## Ecosystem

There are a lot of true positives on `prefect` which look similar to:
```diff
prefect (https://github.com/PrefectHQ/prefect)
+ src/integrations/prefect-aws/tests/workers/test_ecs_worker.py:406:12: error[unresolved-attribute] Type `str` has no attribute `status_code`
```

This is due to a wrong return type annotation
[here](e926b8c4c1/src/integrations/prefect-aws/tests/workers/test_ecs_worker.py (L355-L391)).

```diff
mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ test/mitmproxy/addons/test_clientplayback.py:18:1: error[invalid-argument-type] Argument to function `asynccontextmanager` is incorrect: Expected `(...) -> AsyncIterator[Unknown]`, found `def tcp_server(handle_conn, **server_args) -> Unknown | tuple[str, int]`
```


[This](a4d794c59a/test/mitmproxy/addons/test_clientplayback.py (L18-L19))
is a true positive. That function should return
`AsyncIterator[Address]`, not `Address`.

I looked through almost all of the other new diagnostics and they all
look like known problems or true positives.

## Typing conformance

The typing conformance diff looks good.

## Test Plan

New Markdown tests
2025-07-30 11:51:21 +02:00
Micha Reiser b033fb6bfd
[ty] Split `ScopedPlaceId` into `ScopedSymbolId` and `ScopedMemberId` (#19497) 2025-07-25 13:54:33 +02:00
Shunsuke Shibayama b124e182ca
[ty] improve lazy scope place lookup (#19321)
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-07-25 07:11:11 +00:00
David Peter 2a00eca66b
[ty] Exhaustiveness checking & reachability for `match` statements (#19508)
## Summary

Implements proper reachability analysis and — in effect — exhaustiveness
checking for `match` statements. This allows us to check the following
code without any errors (leads to *"can implicitly return `None`"* on
`main`):

```py
from enum import Enum, auto

class Color(Enum):
    RED = auto()
    GREEN = auto()
    BLUE = auto()

def hex(color: Color) -> str:
    match color:
        case Color.RED:
            return "#ff0000"
        case Color.GREEN:
            return "#00ff00"
        case Color.BLUE:
            return "#0000ff"
```

Note that code like this already worked fine if there was a
`assert_never(color)` statement in a catch-all case, because we would
then consider that `assert_never` call terminal. But now this also works
without the wildcard case. Adding a member to the enum would still lead
to an error here, if that case would not be handled in `hex`.

What needed to happen to support this is a new way of evaluating match
pattern constraints. Previously, we would simply compare the type of the
subject expression against the patterns. For the last case here, the
subject type would still be `Color` and the value type would be
`Literal[Color.BLUE]`, so we would infer an ambiguous truthiness.

Now, before we compare the subject type against the pattern, we first
generate a union type that corresponds to the set of all values that
would have *definitely been matched* by previous patterns. Then, we
build a "narrowed" subject type by computing `subject_type &
~already_matched_type`, and compare *that* against the pattern type. For
the example here, `already_matched_type = Literal[Color.RED] |
Literal[Color.GREEN]`, and so we have a narrowed subject type of `Color
& ~(Literal[Color.RED] | Literal[Color.GREEN]) = Literal[Color.BLUE]`,
which allows us to infer a reachability of `AlwaysTrue`.

<details>

<summary>A note on negated reachability constraints</summary>

It might seem that we now perform duplicate work, because we also record
*negated* reachability constraints. But that is still important for
cases like the following (and possibly also for more realistic
scenarios):

```py
from typing import Literal

def _(x: int | str):
    match x:
        case None:
            pass # never reachable
        case _:
            y = 1

    y
```

</details>

closes https://github.com/astral-sh/ty/issues/99

## Test Plan

* I verified that this solves all examples from the linked ticket (the
first example needs a PEP 695 type alias, because we don't support
legacy type aliases yet)
* Verified that the ecosystem changes are all because of removed false
positives
* Updated tests
2025-07-23 22:45:45 +02:00
David Peter 3d17897c02
[ty] Fix narrowing and reachability of class patterns with arguments (#19512)
## Summary

I noticed that our type narrowing and reachability analysis was
incorrect for class patterns that are not irrefutable. The test cases
below compare the old and the new behavior:

```py
from dataclasses import dataclass

@dataclass
class Point:
    x: int
    y: int

class Other: ...

def _(target: Point):
    y = 1

    match target:
        case Point(0, 0):
            y = 2
        case Point(x=0, y=1):
            y = 3
        case Point(x=1, y=0):
            y = 4
    
    reveal_type(y)  # revealed: Literal[1, 2, 3, 4]    (previously: Literal[2])


def _(target: Point | Other):
    match target:
        case Point(0, 0):
            reveal_type(target)  # revealed: Point
        case Point(x=0, y=1):
            reveal_type(target)  # revealed: Point    (previously: Never)
        case Point(x=1, y=0):
            reveal_type(target)  # revealed: Point    (previously: Never)
        case Other():
            reveal_type(target)  # revealed: Other    (previously: Other & ~Point)
```

## Test Plan

New Markdown test
2025-07-23 18:45:03 +02:00
Andrew Gallant 4573a0f6a0 [ty] Make `Module` a Salsa ingredient
We want to write queries that depend on `Module` for caching. While it
seems it can be done without making `Module` an ingredient, it seems it
is best practice to do so.

[best practice to do so]: https://github.com/astral-sh/ruff/pull/19408#discussion_r2215867301
2025-07-23 09:46:40 -04:00
Micha Reiser dc10ab81bd
[ty] Use `ThinVec` for sub segments in `PlaceExpr` (#19470) 2025-07-22 20:39:39 +02:00
Douglas Creager 88de5727df
[ty] Garbage-collect reachability constraints (#19414)
This is a follow-on to #19410 that further reduces the memory usage of
our reachability constraints. When finishing the building of a use-def
map, we walk through all of the "final" states and mark only those
reachability constraints as "used". We then throw away the interior TDD
nodes of any reachability constraints that weren't marked as used.

(This helps because we build up quite a few intermediate TDD nodes when
constructing complex reachability constraints. These nodes can never be
accessed if they were _only_ used as an intermediate TDD node. The
marking step ensures that we keep any nodes that ended up being referred
to in some accessible use-def map state.)
2025-07-21 14:16:27 -04:00
David Peter b8dec79182
[ty] Implicit instance attributes declared `Final` (#19462)
## Summary

Adds proper type inference for implicit instance attributes that are
declared with a "bare" `Final` and adds `invalid-assignment` diagnostics
for all implicit instance attributes that are declared `Final` or
`Final[…]`.

## Test Plan

New and updated MD tests.

## Ecosystem analysis

```diff
pytest (https://github.com/pytest-dev/pytest)
+ error[invalid-return-type] src/_pytest/fixtures.py:1662:24: Return type does not match returned value: expected `Scope`, found `Scope | (Unknown & ~None & ~((...) -> object) & ~str) | (((str, Config, /) -> Unknown) & ~((...) -> object) & ~str) | (Unknown & ~str)
```

The definition of the `scope` attribute is [here](

5f99385635/src/_pytest/fixtures.py (L1020-L1028)).
Looks like this is a new false positive due to missing `TypeAlias`
support that is surfaced here because we now infer a more precise type
for `FixtureDef._scope`.
2025-07-21 20:01:07 +02:00
Jack O'Connor e9a64e5825
[ty] make `del x` force local resolution of `x` in the current scope (#19389)
Fixes https://github.com/astral-sh/ty/issues/769.

**Updated:** The preferred approach here is to keep the SemanticIndex
simple (`del` of any name marks that name "bound" in the current scope)
and to move complexity to type inference (free variable resolution stops
when it finds a binding, unless that binding is declared `nonlocal`). As
part of this change, free variable resolution will now union the types
it finds as it walks in enclosing scopes. This approach is still
incomplete, because it doesn't consider inner scopes or sibling scopes,
but it improves the common case.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-07-18 14:58:32 -07:00
Micha Reiser 630c7a3152
[ty] Reduce number of inline stored definitions per place (#19409) 2025-07-18 18:28:46 +02:00