Commit Graph

81 Commits

Author SHA1 Message Date
Douglas Creager 1e33d25d1c fix test 2025-12-03 16:38:01 -05:00
Douglas Creager 3bcca62472 doc 2025-12-03 12:12:00 -05:00
Douglas Creager 85e6143e07 use self annotation in synthesized __init__ callable 2025-12-03 12:09:04 -05:00
Douglas Creager 77ce24a5bf allow multiple overloads/callables when inferring 2025-12-03 12:04:59 -05:00
Douglas Creager db5834dfd7 add failing tests 2025-12-03 12:04:00 -05:00
Douglas Creager 2b949b3e67 Merge remote-tracking branch 'origin/main' into dcreager/callable-return
* origin/main: (67 commits)
  Move `Token`, `TokenKind` and `Tokens` to `ruff-python-ast` (#21760)
  [ty] Don't confuse multiple occurrences of `typing.Self` when binding bound methods (#21754)
  Use our org-wide Renovate preset (#21759)
  Delete `my-script.py` (#21751)
  [ty] Move `all_members`, and related types/routines, out of `ide_support.rs` (#21695)
  [ty] Fix find-references for import aliases (#21736)
  [ty] add tests for workspaces (#21741)
  [ty] Stop testing the (brittle) constraint set display implementation (#21743)
  [ty] Use generator over list comprehension to avoid cast (#21748)
  [ty] Add a diagnostic for prohibited `NamedTuple` attribute overrides (#21717)
  [ty] Fix subtyping with `type[T]` and unions (#21740)
  Use `npm ci --ignore-scripts` everywhere (#21742)
  [`flake8-simplify`] Fix truthiness assumption for non-iterable arguments in tuple/list/set calls (`SIM222`, `SIM223`) (#21479)
  [`flake8-use-pathlib`] Mark fixes unsafe for return type changes (`PTH104`, `PTH105`, `PTH109`, `PTH115`) (#21440)
  [ty] Fix auto-import code action to handle pre-existing import
  Enable PEP 740 attestations when publishing to PyPI (#21735)
  [ty] Fix find references for type defined in stub (#21732)
  Use OIDC instead of codspeed token (#21719)
  [ty] Exclude `typing_extensions` from completions unless it's really available
  [ty] Fix false positives for `class F(Generic[*Ts]): ...` (#21723)
  ...
2025-12-02 14:23:15 -05:00
Alex Waygood 3a11e714c6
[ty] Show the user where the type variable was defined in `invalid-type-arguments` diagnostics (#21727) 2025-12-01 12:25:49 +00:00
Dhruv Manilawala 8795d9f0cb
[ty] Split `ParamSpec` mdtests to separate legacy and PEP 695 tests (#21687)
## Summary

This is another small refactor for
https://github.com/astral-sh/ruff/pull/21445 that splits the single
`paramspec.md` into `generics/legacy/paramspec.md` and
`generics/pep695/paramspec.md`.

## Test Plan

Make sure that all mdtests pass.
2025-11-29 06:49:39 +00:00
Ibraheem Ahmed 3ed537e9f1
[ty] Support `type[T]` with type variables (#21650)
## Summary

Adds support for `type[T]`, where `T` is a type variable.

- Resolves https://github.com/astral-sh/ty/issues/501
- Resolves https://github.com/astral-sh/ty/issues/783
- Resolves https://github.com/astral-sh/ty/issues/662
2025-11-28 09:20:24 +01:00
Carl Meyer 77f8fa6906
[ty] more precise inference for a failed specialization (#21651)
## Summary

Previously if an explicit specialization failed (e.g. wrong number of
type arguments or violates an upper bound) we just inferred `Unknown`
for the entire type. This actually caused us to panic on an a case of a
recursive upper bound with invalid specialization; the upper bound would
oscillate indefinitely in fixpoint iteration between `Unknown` and the
given specialization. This could be fixed with a cycle recovery
function, but in this case there's a simpler fix: if we infer
`C[Unknown]` instead of `Unknown` for an invalid attempt to specialize
`C`, that allows fixpoint iteration to quickly converge, as well as
giving a more precise type inference.

Other type checkers actually just go with the attempted specialization
even if it's invalid. So if `C` has a type parameter with upper bound
`int`, and you say `C[str]`, they'll emit a diagnostic but just go with
`C[str]`. Even weirder, if `C` has a single type parameter and you say
`C[str, bytes]`, they'll just go with `C[str]` as the type. I'm not
convinced by this approach; it seems odd to have specializations
floating around that explicitly violate the declared upper bound, or in
the latter case aren't even the specialization the annotation requested.
I prefer `C[Unknown]` for this case.

Fixing this revealed an issue with `collections.namedtuple`, which
returns `type[tuple[Any, ...]]`. Due to
https://github.com/astral-sh/ty/issues/1649 we consider that to be an
invalid specialization. So previously we returned `Unknown`; after this
PR it would be `type[tuple[Unknown]]`, leading to more false positives
from our lack of functional namedtuple support. To avoid that I added an
explicit Todo type for functional namedtuples for now.

## Test Plan

Added and updated mdtests.

The conformance suite changes have to do with `ParamSpec`, so no
meaningful signal there.

The ecosystem changes appear to be the expected effects of having more
precise type information (including occurrences of known issues such as
https://github.com/astral-sh/ty/issues/1495 ). Most effects are just
changes to types in diagnostics.
2025-11-27 13:44:28 +01:00
Dhruv Manilawala c7107a5a90
[ty] Use `zip` to perform explicit specialization (#21635)
## Summary

This PR updates the explicit specialization logic to avoid using the
call machinery.

Previously, the logic would use the call machinery by converting the
list of type variables into a `Binding` with a single `Signature` where
all the type variables are positional-only parameters with bounds and
constraints as the annotated type and the default type as the default
parameter value. This has the advantage that it doesn't need to
implement any specific logic but the disadvantages are subpar diagnostic
messages as it would use the ones specific to a function call. But, an
important disadvantage is that the kind of type variable is lost in this
translation which becomes important in #21445 where a `ParamSpec` can
specialize into a list of types which is provided using list literal.
For example,

```py
class Foo[T, **P]: ...

Foo[int, [int, str]]
```

This PR converts the logic to use a simple loop using `zip_longest` as
all type variables and their corresponding type argument maps on a 1-1
basis. They cannot be specified using keyword argument either e.g.,
`dict[_VT=str, _KT=int]` is invalid.

This PR also makes an initial attempt to improve the diagnostic message
to specifically target the specialization part by using words like "type
argument" instead of just "argument" and including information like the
type variable, bounds, and constraints. Further improvements can be made
by highlighting the type variable definition or the bounds / constraints
as a sub-diagnostic but I'm going to leave that as a follow-up.

## Test Plan

Update messages in existing test cases.
2025-11-27 03:52:22 +00:00
Douglas Creager fedc75463b this gets recursively expanded now 2025-11-26 18:35:15 -05:00
Douglas Creager 9950c126fe these need to be positional only to be assignable 2025-11-26 18:35:15 -05:00
Ibraheem Ahmed 294f863523
[ty] Avoid expression reinference for diagnostics (#21267)
## Summary

We now use the type context for a lot of things, so re-inferring without
type context actually makes diagnostics more confusing (in most cases).
2025-11-25 09:24:00 -08:00
Douglas Creager 6cc502781f
[ty] Remove brittle constraint set reveal tests (#21568)
These were added to try to make it clearer that assignability checks
will eventually return more detailed answers than true or false.
However, the constraint set display rendering is still more brittle than
I'd like it to be, and it's more trouble than it's worth to keep them
updated with semantically identically but textually different edits. The
`static_assert`s are sufficient to check correctness, and we can always
add `reveal_type` when needed for further debugging.
2025-11-21 13:57:55 -05:00
Douglas Creager 97935518e9
[ty] Create a specialization from a constraint set (#21414)
This patch lets us create specializations from a constraint set. The
constraint encodes the restrictions on which types each typevar can
specialize to. Given a generic context and a constraint set, we iterate
through all of the generic context's typevars. For each typevar, we
abstract the constraint set so that it only mentions the typevar in
question (propagating derived facts if needed). We then find the "best
representative type" for the typevar given the abstracted constraint
set.

When considering the BDD structure of the abstracted constraint set,
each path from the BDD root to the `true` terminal represents one way
that the constraint set can be satisfied. (This is also one of the
clauses in the DNF representation of the constraint set's boolean
formula.) Each of those paths is the conjunction of the individual
constraints of each internal node that we traverse as we walk that path,
giving a single lower/upper bound for the path. We use the upper bound
as the "best" (i.e. "closest to `object`") type for that path.

If there are multiple paths in the BDD, they technically represent
independent possible specializations. If there's a single specialization
that satisfies all of them, we will return that as the specialization.
If not, then the constraint set is ambiguous. (This happens most often
with constrained typevars.) We could in the future turn _each_ of the
paths into separate specializations, but it's not clear what we would do
with that, so instead we just report the ambiguity as a specialization
failure.
2025-11-19 14:20:33 -05:00
Ibraheem Ahmed c5d654bce8
[ty] Improve literal promotion heuristics (#21439)
## Summary

Extends literal promotion to apply to any generic method, as opposed to
only generic class constructors. This PR also improves our literal
promotion heuristics to only promote literals in non-covariant position
in the return type, and avoid promotion if the literal is present in
non-covariant position in any argument type.

Resolves https://github.com/astral-sh/ty/issues/1357.
2025-11-14 16:13:56 -05:00
Douglas Creager 33b942c7ad
[ty] Handle annotated `self` parameter in constructor of non-invariant generic classes (#21325)
This manifested as an error when inferring the type of a PEP-695 generic
class via its constructor parameters:

```py
class D[T, U]:
    @overload
    def __init__(self: "D[str, U]", u: U) -> None: ...
    @overload
    def __init__(self, t: T, u: U) -> None: ...
    def __init__(self, *args) -> None: ...

# revealed: D[Unknown, str]
# SHOULD BE: D[str, str]
reveal_type(D("string"))
```

This manifested because `D` is inferred to be bivariant in both `T` and
`U`. We weren't seeing this in the equivalent example for legacy
typevars, since those default to invariant. (This issue also showed up
for _covariant_ typevars, so this issue was not limited to bivariance.)

The underlying cause was because of a heuristic that we have in our
current constraint solver, which attempts to handle situations like
this:

```py
def f[T](t: T | None): ...
f(None)
```

Here, the `None` argument matches the non-typevar union element, so this
argument should not add any constraints on what `T` can specialize to.
Our previous heuristic would check for this by seeing if the argument
type is a subtype of the parameter annotation as a whole — even if it
isn't a union! That would cause us to erroneously ignore the `self`
parameter in our constructor call, since bivariant classes are
equivalent to each other, regardless of their specializations.

The quick fix is to move this heuristic "down a level", so that we only
apply it when the parameter annotation is a union. This heuristic should
go away completely 🤞 with the new constraint solver.
2025-11-10 19:46:49 -05:00
Ibraheem Ahmed 5c69e00d1c
[ty] Simplify unions containing multiple type variables during inference (#21275)
## Summary

Splitting this one out from https://github.com/astral-sh/ruff/pull/21210. This is also something that should be made obselete by the new constraint solver, but is easy enough to fix now.
2025-11-05 15:03:19 +00:00
Douglas Creager 17850eee4b
[ty] Reformat constraint set mdtests (#21111)
This PR updates the mdtests that test how our generics solver interacts
with our new constraint set implementation. Because the rendering of a
constraint set can get long, this standardizes on putting the `revealed`
assertion on a separate line. We also add a `static_assert` test for
each constraint set to verify that they are all coerced into simple
`bool`s correctly.

This is a pure reformatting (not even a refactoring!) that changes no
behavior. I've pulled it out of #20093 to reduce the amount of effort
that will be required to review that PR.
2025-10-28 14:59:49 -04:00
Alex Waygood db0e921db1
[ty] Fix bug where ty would think all types had an `__mro__` attribute (#20995) 2025-10-27 11:19:12 +00:00
Shunsuke Shibayama 48f1771877
[ty] fix infinite recursion with generic type aliases (#20969)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-10-23 14:14:30 +00:00
Alex Waygood 16efe53a72
[ty] Fix panic on recursive class definitions in a stub that use constrained type variables (#20955) 2025-10-18 13:02:55 +00:00
Shunsuke Shibayama e4384fc212
[ty] impl `VarianceInferable` for `KnownInstanceType` (#20924)
## Summary

Derived from #20900

Implement `VarianceInferable` for `KnownInstanceType` (especially for
`KnownInstanceType::TypeAliasType`).

The variance of a type alias matches its value type. In normal usage,
type aliases are expanded to value types, so the variance of a type
alias can be obtained without implementing this. However, for example,
if we want to display the variance when hovering over a type alias, we
need to be able to obtain the variance of the type alias itself (cf.
#20900).

## Test Plan

I couldn't come up with a way to test this in mdtest, so I'm testing it
in a test submodule at the end of `types.rs`.
I also added a test to `mdtest/generics/pep695/variance.md`, but it
passes without the changes in this PR.
2025-10-17 21:12:19 +02:00
Shunsuke Shibayama dc64c08633
[ty] bidirectional type inference using function return type annotations (#20528)
## Summary

Implements bidirectional type inference using function return type
annotations.

This PR was originally proposed to solve astral-sh/ty#1167, but this
does not fully resolve it on its own.
Additionally, I believe we need to allow dataclasses to generate their
own `__new__` methods, [use constructor return types ​​for
inference](5844c0103d/crates/ty_python_semantic/src/types.rs (L5326-L5328)),
and a mechanism to discard type narrowing like `& ~AlwaysFalsy` if
necessary (at a more general level than this PR).

## Test Plan

`mdtest/bidirectional.md` is added.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Ibraheem Ahmed <ibraheem@ibraheem.ca>
2025-10-11 00:38:35 +00:00
Alex Waygood ff386b4797
[ty] Improve diagnostics for bad `@overload` definitions (#20745) 2025-10-07 21:52:57 +00:00
Douglas Creager 416e956fe0
[ty] Infer better specializations of unions with `None` (etc) (#20749)
This PR adds a specialization inference special case that lets us handle
the following examples better:

```py
def f[T](t: T | None) -> T: ...
def g[T](t: T | int | None) -> T | int: ...

def _(x: str | None):
    reveal_type(f(x))  # revealed: str (previously str | None)

def _(y: str | int | None):
    reveal_type(g(x))  # revealed: str | int (previously str | int | None)
```

We already have a special case for when the formal is a union where one
element is a typevar, but it maps the entire actual type to the typevar
(as you can see in the "previously" results above).

The new special case kicks in when the actual is also a union. Now, we
filter out any actual union elements that are already subtypes of the
formal, and only bind whatever types remain to the typevar. (The `|
None` pattern appears quite often in the ecosystem results, but it's
more general and works with any number of non-typevar union elements.)

The new constraint solver should handle this case as well, but it's
worth adding this heuristic now with the old solver because it
eliminates some false positives from the ecosystem report, and makes the
ecosystem report less noisy on the other constraint solver PRs.
2025-10-07 13:33:42 -04:00
Alex Waygood 0639da2552
[ty] `~T` should never be assignable to `T` (#20606)
## Summary

Currently we do not emit an error on this code:

```py
from ty_extensions import Not

def f[T](x: T, y: Not[T]) -> T:
    x = y
    return x
```

But we should do! `~T` should never be assignable to `T`.

This fixes a small regression introduced in
14fe1228e7 (diff-8049ab5af787dba29daa389bbe2b691560c15461ef536f122b1beab112a4b48aR1443-R1446),
where a branch that previously returned `false` was replaced with a
branch that returns `C::always_satisfiable` -- the opposite of what it
used to be! The regression occurred because we didn't have any tests for
this -- so I added some tests in this PR that fail on `main`. I only
spotted the problem because I was going through the code of
`has_relation_to_impl` with a fine toothcomb for
https://github.com/astral-sh/ruff/pull/20602 😄
2025-10-02 07:52:47 +01:00
David Peter 130a794c2b
[ty] Add tests for nested generic functions (#20631)
## Summary

Add two simple tests that we recently discussed with @dcreager. They
demonstrate that the `TypeMapping::MarkTypeVarsInferable` operation
really does need to keep track of the binding context.

## Test Plan

Made sure that those tests fail if we create
`TypeMapping::MarkTypeVarsInferable(None)`s everywhere.
2025-09-30 08:44:18 +02:00
David Peter 0092794302
[ty] Use `typing.Self` for the first parameter of instance methods (#20517)
## Summary

Modify the (external) signature of instance methods such that the first
parameter uses `Self` unless it is explicitly annotated. This allows us
to correctly type-check more code, and allows us to infer correct return
types for many functions that return `Self`. For example:

```py
from pathlib import Path
from datetime import datetime, timedelta

reveal_type(Path(".config") / ".ty")  # now Path, previously Unknown

def _(dt: datetime, delta: timedelta):
    reveal_type(dt - delta)  # now datetime, previously Unknown
```

part of https://github.com/astral-sh/ty/issues/159

## Performance

I ran benchmarks locally on `attrs`, `freqtrade` and `colour`, the
projects with the largest regressions on CodSpeed. I see much smaller
effects locally, but can definitely reproduce the regression on `attrs`.
From looking at the profiling results (on Codspeed), it seems that we
simply do more type inference work, which seems plausible, given that we
now understand much more return types (of many stdlib functions). In
particular, whenever a function uses an implicit `self` and returns
`Self` (without mentioning `Self` anywhere else in its signature), we
will now infer the correct type, whereas we would previously return
`Unknown`. This also means that we need to invoke the generics solver in
more cases. Comparing half a million lines of log output on attrs, I can
see that we do 5% more "work" (number of lines in the log), and have a
lot more `apply_specialization` events (7108 vs 4304). On freqtrade, I
see similar numbers for `apply_specialization` (11360 vs 5138 calls).
Given these results, I'm not sure if it's generally worth doing more
performance work, especially since none of the code modifications
themselves seem to be likely candidates for regressions.

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/attrs` | 92.6 ± 3.6 | 85.9 |
102.6 | 1.00 |
| `./ty_self check /home/shark/ecosystem/attrs` | 101.7 ± 3.5 | 96.9 |
113.8 | 1.10 ± 0.06 |

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/freqtrade` | 599.0 ± 20.2 |
568.2 | 627.5 | 1.00 |
| `./ty_self check /home/shark/ecosystem/freqtrade` | 607.9 ± 11.5 |
594.9 | 626.4 | 1.01 ± 0.04 |

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/colour` | 423.9 ± 17.9 | 394.6
| 447.4 | 1.00 |
| `./ty_self check /home/shark/ecosystem/colour` | 426.9 ± 24.9 | 373.8
| 456.6 | 1.01 ± 0.07 |

## Test Plan

New Markdown tests

## Ecosystem report

* apprise: ~300 new diagnostics related to problematic stubs in apprise
😩
* attrs: a new true positive, since [this
function](4e2c89c823/tests/test_make.py (L2135))
is missing a `@staticmethod`?
* Some legitimate true positives
* sympy: lots of new `invalid-operator` false positives in [matrix
multiplication](cf9f4b6805/sympy/matrices/matrixbase.py (L3267-L3269))
due to our limited understanding of [generic `Callable[[Callable[[T1,
T2], T3]], Callable[[T1, T2], T3]]` "identity"
types](cf9f4b6805/sympy/core/decorators.py (L83-L84))
of decorators. This is not related to type-of-self.

## Typing conformance results

The changes are all correct, except for
```diff
+generics_self_usage.py:50:5: error[invalid-assignment] Object of type `def foo(self) -> int` is not assignable to `(typing.Self, /) -> int`
```
which is related to an assignability problem involving type variables on
both sides:
```py
class CallableAttribute:
    def foo(self) -> int:
        return 0

    bar: Callable[[Self], int] = foo  # <- we currently error on this assignment
```

---------

Co-authored-by: Shaygan Hooshyari <sh.hooshyari@gmail.com>
2025-09-29 21:08:08 +02:00
David Peter 3932f7c849
[ty] Fix subtyping for dynamic specializations (#20592)
## Summary

Fixes a bug observed by @AlexWaygood where `C[Any] <: C[object]` should
hold for a class that is covariant in its type parameter (and similar
subtyping relations involving dynamic types for other variance
configurations).

## Test Plan

New and updated Markdown tests
2025-09-26 15:05:03 +02:00
David Peter 742f8a4ee6
[ty] Use `C[T]` instead of `C[Unknown]` for the upper bound of `Self` (#20479)
### Summary

This PR includes two changes, both of which are necessary to resolve
https://github.com/astral-sh/ty/issues/1196:

* For a generic class `C[T]`, we previously used `C[Unknown]` as the
upper bound of the `Self` type variable. There were two problems with
this. For one, when `Self` appeared in contravariant position, we would
materialize its upper bound to `Bottom[C[Unknown]]` (which might
simplify to `C[Never]` if `C` is covariant in `T`) when accessing
methods on `Top[C[Unknown]]`. This would result in `invalid-argument`
errors on the `self` parameter. Also, using an upper bound of
`C[Unknown]` would mean that inside methods, references to `T` would be
treated as `Unknown`. This could lead to false negatives. To fix this,
we now use `C[T]` (with a "nested" typevar) as the upper bound for
`Self` on `C[T]`.
* In order to make this work, we needed to allow assignability/subtyping
of inferable typevars to other types, since we now check assignability
of e.g. `C[int]` to `C[T]` (when checking assignability to the upper
bound of `Self`) when calling an instance-method on `C[int]` whose
`self` parameter is annotated as `self: Self` (or implicitly `Self`,
following https://github.com/astral-sh/ruff/pull/18007).

closes https://github.com/astral-sh/ty/issues/1196
closes https://github.com/astral-sh/ty/issues/1208


### Test Plan

Regression tests for both issues.
2025-09-23 14:02:25 +02:00
Eric Mark Martin 2502ff7638
[ty] Make TypeIs invariant in its type argument (#20428)
## Summary

What it says on the tin. See the [typing
spec](https://docs.python.org/3/library/typing.html#typing.TypeIs) for
justification.

## Test Plan

Add more tests to PEP 695 `variance.md` suite.
2025-09-18 07:53:13 -07:00
Alex Waygood 0e3697a643
[ty] Minor fixes to `Protocol` tests (#20347) 2025-09-11 14:42:13 +00:00
Douglas Creager 2ac4147435
[ty] Add mdtests that exercise constraint sets (#20319)
This PR adds a new `ty_extensions.ConstraintSet` class, which is used to
expose constraint sets to our mdtest framework. This lets us write a
large collection of unit tests that exercise the invariants and rewrite
rules of our constraint set implementation.

As part of this, `is_assignable_to` and friends are updated to return a
`ConstraintSet` instead of a `bool`, and we implement
`ConstraintSet.__bool__` to return when a constraint set is always
satisfied. That lets us still use
`static_assert(is_assignable_to(...))`, since the assertion will coerce
the constraint set to a bool, and also lets us
`reveal_type(is_assignable_to(...))` to see more detail about
whether/when the two types are assignable. That lets us get rid of
`reveal_when_assignable_to` and friends, since they are now redundant
with the expanded capabilities of `is_assignable_to`.
2025-09-10 13:22:19 -04:00
Ibraheem Ahmed aa5d665d52
[ty] Add support for generic PEP695 type aliases (#20219)
## Summary

Adds support for generic PEP695 type aliases, e.g.,
```python
type A[T] = T
reveal_type(A[int]) # A[int]
```

Resolves https://github.com/astral-sh/ty/issues/677.
2025-09-08 13:26:21 -07:00
David Peter d55edb3d74
[ty] Support "legacy" `typing.Self` in combination with PEP 695 generic contexts (#20304)
## Summary

Support cases like the following, where we need the generic context to
include both `Self` and `T` (not just `T`):

```py
from typing import Self

class C:
    def method[T](self: Self, arg: T): ...

C().method(1)
```

closes https://github.com/astral-sh/ty/issues/1131

## Test Plan

Added regression test
2025-09-08 16:57:09 +02:00
Douglas Creager 77b2cee223
[ty] Add functions for revealing assignability/subtyping constraints (#20217)
This PR adds two new `ty_extensions` functions,
`reveal_when_assignable_to` and `reveal_when_subtype_of`. These are
closely related to the existing `is_assignable_to` and `is_subtype_of`,
but instead of returning when the property (always) holds, it produces a
diagnostic that describes _when_ the property holds. (This will let us
construct mdtests that print out constraints that are not always true or
always false — though we don't currently have any instances of those.)

I did not replace _every_ occurrence of the `is_property` variants in
the mdtest suite, instead focusing on the generics-related tests where
it will be important to see the full detail of the constraint sets.

As part of this, I also updated the mdtest harness to accept the shorter
`# revealed:` assertion format for more than just `reveal_type`, and
updated the existing uses of `reveal_protocol_interface` to take
advantage of this.
2025-09-03 16:44:35 -04:00
Douglas Creager 14fe1228e7
[ty] Perform assignability etc checks using new `Constraints` trait (#19838)
"Why would you do this? This looks like you just replaced `bool` with an
overly complex trait"

Yes that's correct!

This should be a no-op refactoring. It replaces all of the logic in our
assignability, subtyping, equivalence, and disjointness methods to work
over an arbitrary `Constraints` trait instead of only working on `bool`.

The methods that `Constraints` provides looks very much like what we get
from `bool`. But soon we will add a new impl of this trait, and some new
methods, that let us express "fuzzy" constraints that aren't always true
or false. (In particular, a constraint will express the upper and lower
bounds of the allowed specializations of a typevar.)

Even once we have that, most of the operations that we perform on
constraint sets will be the usual boolean operations, just on sets.
(`false` becomes empty/never; `true` becomes universe/always; `or`
becomes union; `and` becomes intersection; `not` becomes negation.) So
it's helpful to have this separate PR to refactor how we invoke those
operations without introducing the new functionality yet.

Note that we also have translations of `Option::is_some_and` and
`is_none_or`, and of `Iterator::any` and `all`, and that the `and`,
`or`, `when_any`, and `when_all` methods are meant to short-circuit,
just like the corresponding boolean operations. For constraint sets,
that depends on being able to implement the `is_always` and `is_never`
trait methods.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-08-21 09:30:09 -04:00
Eric Mark Martin 33030b34cd
[ty] linear variance inference for PEP-695 type parameters (#18713)
## Summary

Implement linear-time variance inference for type variables
(https://github.com/astral-sh/ty/issues/488).

Inspired by Martin Huschenbett's [PyCon 2025
Talk](https://www.youtube.com/watch?v=7uixlNTOY4s&t=9705s).

## Test Plan

update tests, add new tests, including for mutually recursive classes

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-08-19 17:54:09 -07:00
Alex Waygood 662d18bd05
[ty] Add precise inference for unpacking a TypeVar if the TypeVar has an upper bound with a precise tuple spec (#19985) 2025-08-19 22:11:30 +01:00
Alex Waygood 3314cf90ed
[ty] Add more regression tests for `tuple` (#19974) 2025-08-18 18:30:05 +01:00
Douglas Creager b892e4548e
[ty] Track when type variables are inferable or not (#19786)
`Type::TypeVar` now distinguishes whether the typevar in question is
inferable or not.

A typevar is _not inferable_ inside the body of the generic class or
function that binds it:

```py
def f[T](t: T) -> T:
    return t
```

The infered type of `t` in the function body is `TypeVar(T,
NotInferable)`. This represents how e.g. assignability checks need to be
valid for all possible specializations of the typevar. Most of the
existing assignability/etc logic only applies to non-inferable typevars.

Outside of the function body, the typevar is _inferable_:

```py
f(4)
```

Here, the parameter type of `f` is `TypeVar(T, Inferable)`. This
represents how e.g. assignability doesn't need to hold for _all_
specializations; instead, we need to find the constraints under which
this specific assignability check holds.

This is in support of starting to perform specialization inference _as
part of_ performing the assignability check at the call site.

In the [[POPL2015][]] paper, this concept is called _monomorphic_ /
_polymorphic_, but I thought _non-inferable_ / _inferable_ would be
clearer for us.

Depends on #19784 

[POPL2015]: https://doi.org/10.1145/2676726.2676991

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-08-16 18:25:03 -04:00
Carl Meyer 5a570c8e6d
[ty] fix deferred name loading in PEP695 generic classes/functions (#19888)
## Summary

For PEP 695 generic functions and classes, there is an extra "type
params scope" (a child of the outer scope, and wrapping the body scope)
in which the type parameters are defined; class bases and function
parameter/return annotations are resolved in that type-params scope.

This PR fixes some longstanding bugs in how we resolve name loads from
inside these PEP 695 type parameter scopes, and also defers type
inference of PEP 695 typevar bounds/constraints/default, so we can
handle cycles without panicking.

We were previously treating these type-param scopes as lazy nested
scopes, which is wrong. In fact they are eager nested scopes; the class
`C` here inherits `int`, not `str`, and previously we got that wrong:

```py
Base = int

class C[T](Base): ...

Base = str
```

But certain syntactic positions within type param scopes (typevar
bounds/constraints/defaults) are lazy at runtime, and we should use
deferred name resolution for them. This also means they can have cycles;
in order to handle that without panicking in type inference, we need to
actually defer their type inference until after we have constructed the
`TypeVarInstance`.

PEP 695 does specify that typevar bounds and constraints cannot be
generic, and that typevar defaults can only reference prior typevars,
not later ones. This reduces the scope of (valid from the type-system
perspective) cycles somewhat, although cycles are still possible (e.g.
`class C[T: list[C]]`). And this is a type-system-only restriction; from
the runtime perspective an "invalid" case like `class C[T: T]` actually
works fine.

I debated whether to implement the PEP 695 restrictions as a way to
avoid some cycles up-front, but I ended up deciding against that; I'd
rather model the runtime name-resolution semantics accurately, and
implement the PEP 695 restrictions as a separate diagnostic on top.
(This PR doesn't yet implement those diagnostics, thus some `# TODO:
error` in the added tests.)

Introducing the possibility of cyclic typevars made typevar display
potentially stack overflow. For now I've handled this by simply removing
typevar details (bounds/constraints/default) from typevar display. This
impacts display of two kinds of types. If you `reveal_type(T)` on an
unbound `T` you now get just `typing.TypeVar` instead of
`typing.TypeVar("T", ...)` where `...` is the bound/constraints/default.
This matches pyright and mypy; pyrefly uses `type[TypeVar[T]]` which
seems a bit confusing, but does include the name. (We could easily
include the name without cycle issues, if there's a syntax we like for
that.)

It also means that displaying a generic function type like `def f[T:
int](x: T) -> T: ...` now displays as `f[T](x: T) -> T` instead of `f[T:
int](x: T) -> T`. This matches pyright and pyrefly; mypy does include
bound/constraints/defaults of typevars in function/callable type
display. If we wanted to add this, we would either need to thread a
visitor through all the type display code, or add a `decycle` type
transformation that replaced recursive reoccurrence of a type with a
marker.

## Test Plan

Added mdtests and modified existing tests to improve their correctness.

After this PR, there's only a single remaining py-fuzzer seed in the
0-500 range that panics! (Before this PR, there were 10; the fuzzer
likes to generate cyclic PEP 695 syntax.)

## Ecosystem report

It's all just the changes to `TypeVar` display.
2025-08-13 15:51:59 -07:00
Alex Waygood d2fbf2af8f
[ty] Remove `Type::Tuple` (#19669) 2025-08-11 22:03:32 +01:00
Douglas Creager dc84645c36
[ty] Use separate Rust types for bound and unbound type variables (#19796)
This PR creates separate Rust types for bound and unbound type
variables, as proposed in https://github.com/astral-sh/ty/issues/926.

Closes https://github.com/astral-sh/ty/issues/926

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-08-11 15:29:58 -04:00
Alex Waygood 8489816edc
[ty] Improve ability to solve TypeVars when they appear in unions (#19829) 2025-08-08 17:50:37 +01:00
Alex Waygood c401a6d86e
[ty] Add failing tests for tuple subclasses (#19803) 2025-08-07 13:11:15 +00:00
Douglas Creager 585ce12ace
[ty] `typing.Self` is bound by the method, not the class (#19784)
This fixes our logic for binding a legacy typevar with its binding
context. (To recap, a legacy typevar starts out "unbound" when it is
first created, and each time it's used in a generic class or function,
we "bind" it with the corresponding `Definition`.)

We treat `typing.Self` the same as a legacy typevar, and so we apply
this binding logic to it too. Before, we were using the enclosing class
as its binding context. But that's not correct — it's the method where
`typing.Self` is used that binds the typevar. (Each invocation of the
method will find a new specialization of `Self` based on the specific
instance type containing the invoked method.)

This required plumbing through some additional state to the
`in_type_expression` method.

This also revealed that we weren't handling `Self`-typed instance
attributes correctly (but were coincidentally not getting the expected
false positive diagnostics).
2025-08-06 17:26:17 -04:00
Alex Waygood 4090297a11
[ty] Fix more false positives related to `Generic` or `Protocol` being subscripted with a `ParamSpec` or `TypeVarTuple` (#19764) 2025-08-05 15:45:56 +01:00