Commit Graph

1129 Commits

Author SHA1 Message Date
Dhruv Manilawala 3420cdda09 [ty] Add truncation policy for displaying overloads on single line 2025-12-12 20:24:59 +05:30
Dhruv Manilawala 76ffc56e85 [ty] Add basic support for overloads in `ParamSpec` 2025-12-12 16:23:48 +05:30
Carl Meyer 0138cd238a
[ty] avoid fixpoint unioning of types containing current-cycle Divergent (#21910)
Partially addresses https://github.com/astral-sh/ty/issues/1732

## Summary

Don't union the previous type in fixpoint iteration if the previous type
contains a `Divergent` from the current cycle and the latest type does
not. The theory here, as outlined by @mtshiba at
https://github.com/astral-sh/ty/issues/1732#issuecomment-3609937420, is
that oscillation can't occur by removing and then reintroducing a
`Divergent` type repeatedly, since `Divergent` types are only introduced
at the start of fixpoint iteration.

## Test Plan

Removes a `Divergent` type from the added mdtest, doesn't otherwise
regress any tests.
2025-12-11 19:52:34 -08:00
Shunsuke Shibayama 5e42926eee
[ty] improve bad specialization results & error messages (#21840)
## Summary

This PR includes the following changes:

* When attempting to specialize a non-generic type (or a type that is
already specialized), the result is `Unknown`. Also, the error message
is improved.
* When an implicit type alias is incorrectly specialized, the result is
`Unknown`. Also, the error message is improved.
* When only some of the type alias bounds and constraints are not
satisfied, not all substitutions are `Unknown`.
* Double specialization is prohibited. e.g. `G[int][int]`

Furthermore, after applying this PR, the fuzzing tests for seeds 1052
and 4419, which panic in main, now pass.
This is because the false recursions on type variables have been
removed.

```python
# name_2[0] => Unknown
class name_1[name_2: name_2[0]]:
    def name_4(name_3: name_2, /):
        if name_3:
            pass

#  (name_5 if unique_name_0 else name_1)[0] => Unknown
def name_4[name_5: (name_5 if unique_name_0 else name_1)[0], **name_1](): ...
```

## Test Plan

New corpus test
mdtest files updated
2025-12-11 19:21:34 -08:00
Jack O'Connor ddb7645e9d
[ty] support `NewType`s of `float` and `complex` (#21886)
Fixes https://github.com/astral-sh/ty/issues/1818.
2025-12-12 00:43:09 +00:00
Douglas Creager c8851ecf70
[ty] Defer all parameter and return type annotations (#21906)
As described in astral-sh/ty#1729, we previously had a salsa cycle when
inferring the signature of many function definitions.

The most obvious case happened when (a) the function was decorated, (b)
it had no PEP-695 type params, and (c) annotations were not always
deferred (e.g. in a stub file). We currently evaluate and apply function
decorators eagerly, as part of `infer_function_definition`. Applying a
decorator requires knowing the signature of the function being
decorated. There were two places where signature construction called
`infer_definition_types` cyclically.

The simpler case was that we were looking up the generic context and
decorator list of the function to determine whether it has an implicit
`self` parameter. Before, we used `infer_definition_types` to determine
that information. But since we're in the middle of signature
construction for the function, we can just thread the information
through directly.

The harder case is that signature construction requires knowing the
inferred parameter and return type annotations. When (b) and (c) hold,
those type annotations are inferred in `infer_function_definition`! (In
theory, we've already finished that by the time we start applying
decorators, but signature construction doesn't know that.)

If annotations are deferred, the params/return annotations are inferred
in `infer_deferred_types`; if there are PEP-695 type params, they're
inferred in `infer_function_type_params`. Both of those are different
salsa queries, and don't induce this cycle.

So the quick fix here is to always defer inference of the function
params/return, so that they are always inferred under a different salsa
query.

A more principled fix would be to apply decorators lazily, just like we
construct signatures lazily. But that is a more invasive fix.

Fixes astral-sh/ty#1729

---------

Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-12-11 15:00:18 -05:00
Micha Reiser 34f7a04ef7
[ty] Handle `Definition`s in `SemanticModel::scope` (#21919) 2025-12-11 18:04:57 +00:00
Micha Reiser fbeeb050af
[ty] Don't show hover for expressions with no inferred type (#21924) 2025-12-11 18:55:32 +01:00
Carl Meyer 4fdb4e8219
[ty] avoid unions of generic aliases of the same class in fixpoint (#21909)
Partially addresses https://github.com/astral-sh/ty/issues/1732
Fixes https://github.com/astral-sh/ty/issues/1800

## Summary

At each fixpoint iteration, we union the "previous" and "current"
iteration types, to ensure that the type can only widen at each
iteration. This prevents oscillation and ensures convergence.

But some unions triggered by this behavior (in particular, unions of
differently-specialized generic-aliases of the same class) never
simplify, and cause spurious errors. Since we haven't seen examples of
oscillating types involving class-literal or generic-alias types, just
don't union those.

There may be more thorough/principled ways to avoid undesirable unions
in fixpoint iteration, but this narrow change seems like it results in
strict improvement.

## Test Plan

Removes two false positive `unsupported-class-base` in mdtests, and
several in the ecosystem, without causing other regression.
2025-12-11 09:53:43 -08:00
Andrew Gallant c548ef2027 [ty] Squash false positive logs for failing to find `builtins` as a real module
I recently started noticing this showing up in the logs for every scope
based completion request:

```
2025-12-11 11:25:35.704329935 DEBUG request{id=29 method="textDocument/completion"}:map_stub_definition: Module `builtins` not found while looking in parent dirs
```

And in particular, it was repeated several times. This was confusing to
me because, well, of course `builtins` should resolve.

This particular code path comes from looking for the docstrings
of completion items. This involves a spelunking that ultimately
tries to resolve a "real" module if the stub doesn't have available
docstrings. But I guess there is no "real" `builtins` module, so
`resolve_real_module` fails. Which is fine, but the noisy logs were
annoying since this is an expected case.

So here, we carve out a short circuit for `builtins` and also improve
the log message.
2025-12-11 12:50:08 -05:00
Luca Chiodini 5a9d6a91ea
[ty] Uniformly use "not supported" in diagnostics (#21916) 2025-12-11 15:03:55 +00:00
Andrew Gallant 8647844572 [ty] Adjust scope completions to use all reachable symbols
Fixes astral-sh/ty#1294
2025-12-11 08:26:15 -05:00
Andrew Gallant 1dcb7f89f1 [ty] Rename `all_members_of_scope` to `all_end_of_scope_members`
This reflects more precisely its behavior based on how it uses the
use-def map.
2025-12-11 08:26:15 -05:00
Andrew Gallant c1c45a6a13 [ty] Remove `all_` prefix from some routines on UseDefMap
These routines don't return *all* symbols/members, but rather,
only *for* a particular scope. We do specifically want to add
some routines that return *all* symbols/members, and this naming
scheme made that confusing. It was also inconsistent with other
routines like `all_end_of_scope_symbol_declarations` which *do*
return *all* symbols.
2025-12-11 08:26:15 -05:00
David Peter 71540c03b6
[ty] Revert "Do not infer types for invalid binary expressions in annotations" (#21914)
See discussion here:
https://github.com/astral-sh/ruff/pull/21911#discussion_r2610155157
2025-12-11 11:57:45 +00:00
Charlie Marsh 5c320990f7
[ty] Avoid inferring types for invalid binary expressions in string annotations (#21911)
## Summary

Closes https://github.com/astral-sh/ty/issues/1847.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-12-11 09:40:19 +01:00
Dhruv Manilawala 24ed28e314
[ty] Improve overload call resolution tracing (#21913)
This PR improves the overload call resolution tracing messages as:
- Use `trace` level instead of `debug` level
- Add a `trace_span` which contains the call arguments and signature
- Remove the signature from individual tracing messages
2025-12-11 12:28:45 +05:30
Carl Meyer 2d0681da08
[ty] fix missing heap_size on Salsa query (#21912) 2025-12-10 18:34:00 -08:00
Ibraheem Ahmed 29bf2cd201
[ty] Support implicit type of `cls` in signatures (#21771)
## Summary

Extends https://github.com/astral-sh/ruff/pull/20517 to support the
implicit type of `cls` in `@classmethod` signatures. Part of
https://github.com/astral-sh/ty/issues/159.
2025-12-10 16:56:20 -05:00
Jack O'Connor 1b44d7e2a7
[ty] add `SyntheticTypedDictType` and implement `normalized` and `is_equivalent_to` (#21784) 2025-12-10 20:36:36 +00:00
Ibraheem Ahmed a2fb2ee06c
[ty] Fix disjointness checks with type-of `@final` classes (#21770)
## Summary

We currently perform a subtyping check, similar to what we were doing
for `@final` instances before
https://github.com/astral-sh/ruff/pull/21167, which is incorrect, e.g.
we currently consider `type[X[Any]]` and `type[X[T]]]` disjoint (where
`X` is `@final`).
2025-12-10 15:15:10 -05:00
Douglas Creager 3e00221a6c
[ty] Fix negation upper bounds in constraint sets (#21897)
This fixes the logic error that @sharkdp
[found](https://github.com/astral-sh/ruff/pull/21871#discussion_r2605755588)
in the constraint set upper bound normalization logic I introduced in
#21871.

I had originally claimed that `(T ≤ α & ~β)` should simplify into `(T ≤
α) ∧ ¬(T ≤ β)`. But that also suggests that `T ≤ ~β` should simplify to
`¬(T ≤ β)` on its own, and that's not correct.

The correct simplification is that `~α` is an "atomic" type, not an
"intersection" for the purposes of our upper bound simplifcation. So `(T
≤ α & ~β)` should simplify to `(T ≤ α) ∧ (T ≤ ~β)`. That is, break apart
the elements of a (proper) intersection, regardless of whether each
element is negated or not.

This PR fixes the logic, adds a test case, and updates the comments to
be hopefully more clear and accurate.
2025-12-10 15:07:50 -05:00
Ibraheem Ahmed 5dc0079e78
[ty] Fix disjointness checks on `@final` class instances (#21769)
## Summary

This was left unfinished in
https://github.com/astral-sh/ruff/pull/21167. This is required to fix
our disjointness checks with type-of a final class, which is currently
broken, and blocking https://github.com/astral-sh/ty/issues/159.
2025-12-10 14:17:22 -05:00
Carl Meyer 951766d1fb
[ty] default-specialize class-literal types in assignment to generic-alias types (#21883)
Fixes https://github.com/astral-sh/ty/issues/1832, fixes
https://github.com/astral-sh/ty/issues/1513

## Summary

A class object `C` (for which we infer an unspecialized `ClassLiteral`
type) should always be assignable to the type `type[C]` (which is
default-specialized, if `C` is generic). We already implemented this for
most cases, but we missed the case of a generic final type, where we
simplify `type[C]` to the `GenericAlias` type for the default
specialization of `C`. So we also need to implement this assignability
of generic `ClassLiteral` types as-if default-specialized.

## Test Plan

Added mdtests that failed before this PR.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-12-10 17:18:08 +01:00
David Peter 7bf50e70a7
[ty] Generics: Respect typevar bounds when matching against a union (#21893)
## Summary

Respect typevar bounds and constraints when matching against a union.
For example:

```py
def accepts_t_or_int[T_str: str](x: T_str | int) -> T_str:
    raise NotImplementedError

reveal_type(accepts_t_or_int("a"))  # ok, reveals `Literal["a"]`
reveal_type(accepts_t_or_int(1))  # ok, reveals `Unknown`

class Unrelated: ...

# error: [invalid-argument-type] "Argument type `Unrelated` does not
# satisfy upper bound `str` of type variable `T_str`"
accepts_t_or_int(Unrelated())
```

Previously, the last call succeed without any errors. Worse than that,
we also incorrectly solved `T_str = Unrelated`, which often lead to
downstream errors.

closes https://github.com/astral-sh/ty/issues/1837

## Ecosystem impact

Looks good!

* Lots of removed false positives, often because we previously selected
a wrong overload for a generic function (because we didn't respect the
typevar bound in an earlier overload).
* We now understand calls to functions accepting an argument of type
`GenericPath: TypeAlias = AnyStr | PathLike[AnyStr]`. Previously, we
would incorrectly match a `Path` argument against the `AnyStr` typevar
(violating its constraints), but now we match against `PathLike`.

## Performance

Another regression on `colour`. This package uses `numpy` heavily. And
`numpy` is the codebase that originally lead me to this bug. The fix
here allows us to infer more precise `np.array` types in some cases, so
it's reasonable that we just need to perform more work.

The fix here also requires us to look at more union elements when we
would previously short-circuit incorrectly, so some more work needs to
be done in the solver.

## Test Plan

New Markdown tests
2025-12-10 14:58:57 +01:00
Ibraheem Ahmed ff7086d9ad
[ty] Infer type of implicit `cls` parameter in method bodies (#21685)
## Summary

Extends https://github.com/astral-sh/ruff/pull/20922 to infer
unannotated `cls` parameters as `type[Self]` in method bodies.

Part of https://github.com/astral-sh/ty/issues/159.
2025-12-10 10:31:28 +01:00
Charlie Marsh d2aabeaaa2
[ty] Respect `kw_only` from parent class (#21820)
## Summary

Closes https://github.com/astral-sh/ty/issues/1769.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-12-10 10:12:18 +01:00
Jack O'Connor aaadf16b1b
[ty] bump dependencies to pull in Salsa support for `ordermap` (#21854) 2025-12-09 19:08:03 -08:00
Douglas Creager c343e94ac5
[ty] Simplify union lower bounds and intersection upper bounds in constraint sets (#21871)
In a constraint set, it's not useful for an upper bound to be an
intersection type, or for a lower bound to be a union type. Both of
those can be rewritten as simpler BDDs:

```
T ≤ α & β  ⇒ (T ≤ α) ∧ (T ≤ β)
T ≤ α & ¬β ⇒ (T ≤ α) ∧ ¬(T ≤ β)
α | β ≤ T  ⇒ (α ≤ T) ∧ (β ≤ T)
```

We were seeing performance issues on #21551 when _not_ performing this
simplification. For instance, `pandas` was producing some constraint
sets involving intersections of 8-9 different types. Our sequent map
calculation was timing out calculating all of the different permutations
of those types:

```
t1 & t2 & t3 → t1
t1 & t2 & t3 → t2
t1 & t2 & t3 → t3
t1 & t2 & t3 → t1 & t2
t1 & t2 & t3 → t1 & t3
t1 & t2 & t3 → t2 & t3
```

(and then imagine what that looks like for 9 types instead of 3...)

With this change, all of those permutations are now encoded in the BDD
structure itself, which is very good at simplifying that kind of thing.

Pulling this out of #21551 for separate review.
2025-12-09 19:49:17 -05:00
Douglas Creager 270b8d1d14
[ty] Collapse `never` paths in constraint set BDDs (#21880)
#21744 fixed some non-determinism in our constraint set implementation
by switching our BDD representation from being "fully reduced" to being
"quasi-reduced". We still deduplicate identical nodes (via salsa
interning), but we removed the logic to prune redundant nodes (one with
identical outgoing true and false edges). This ensures that the BDD
"remembers" all of the individual constraints that it was created with.

However, that comes at the cost of creating larger BDDs, and on #21551
that was causing performance issues. `scikit-learn` was producing a
function signature with dozens of overloads, and we were trying to
create a constraint set that would map a return type typevar to any of
those overload's return types. This created a combinatorial explosion in
the BDD, with by far most of the BDD paths leading to the `never`
terminal.

This change updates the quasi-reduction logic to prune nodes that are
redundant _because both edges lead to the `never` terminal_. In this
case, we don't need to "remember" that constraint, since no assignment
to it can lead to a valid specialization. So we keep the "memory" of our
quasi-reduced structure, while still pruning large unneeded portions of
the BDD structure.

Pulling this out of https://github.com/astral-sh/ruff/pull/21551 for
separate review.
2025-12-09 18:22:54 -05:00
David Peter a9be810c38
[ty] Type inference for `@asynccontextmanager` (#21876)
## Summary

This PR adds special handling for `asynccontextmanager` calls as a
temporary solution for https://github.com/astral-sh/ty/issues/1804. We
will be able to remove this soon once we have support for generic
protocols in the solver.

closes https://github.com/astral-sh/ty/issues/1804

## Ecosystem

```diff
+ tests/test_downloadermiddleware.py:305:56: error[invalid-argument-type] Argument to bound method `download` is incorrect: Expected `Spider`, found `Unknown | Spider | None`
+ tests/test_downloadermiddleware.py:305:56: warning[possibly-missing-attribute] Attribute `spider` may be missing on object of type `Crawler | None`
```
These look like true positives

```diff
+ pymongo/asynchronous/database.py:1021:35: error[invalid-assignment] Object of type `(AsyncClientSession & ~AlwaysTruthy & ~AlwaysFalsy) | (_ServerMode & ~AlwaysFalsy) | Unknown | Primary` is not assignable to `_ServerMode | None`
+ pymongo/asynchronous/database.py:1025:17: error[invalid-argument-type] Argument to bound method `_conn_for_reads` is incorrect: Expected `_ServerMode`, found `_ServerMode | None`
```

Known problems or true positives, just caused by the new type for
`session`

```diff
- src/integrations/prefect-sqlalchemy/prefect_sqlalchemy/database.py:269:16: error[invalid-return-type] Return type does not match returned value: expected `Connection | AsyncConnection`, found `_GeneratorContextManager[Unknown, None, None] | _AsyncGeneratorContextManager[Unknown, None] | Connection | AsyncConnection`
+ src/integrations/prefect-sqlalchemy/prefect_sqlalchemy/database.py:269:16: error[invalid-return-type] Return type does not match returned value: expected `Connection | AsyncConnection`, found `_GeneratorContextManager[Unknown, None, None] | _AsyncGeneratorContextManager[AsyncConnection, None] | Connection | AsyncConnection`
```

Just a more concrete type

```diff
- src/prefect/flow_engine.py:1277:24: error[missing-argument] No argument provided for required parameter `cls`
- src/prefect/server/api/server.py:696:49: error[missing-argument] No argument provided for required parameter `cls`
- src/prefect/task_engine.py:1426:24: error[missing-argument] No argument provided for required parameter `cls`
```

Good

## Test Plan

* Adapted and newly added Markdown tests
* Tested on internal codebase
2025-12-09 22:49:00 +01:00
Carl Meyer 8727a7b179
Fix stack overflow with recursive generic protocols (depth limit) (#21858)
## Summary

This fixes https://github.com/astral-sh/ty/issues/1736 where recursive
generic protocols with growing specializations caused a stack overflow.

The issue occurred with protocols like:
```python
class C[T](Protocol):
    a: 'C[set[T]]'
```

When checking `C[set[int]]` against e.g. `C[Unknown]`, member `a`
requires checking `C[set[set[int]]]`, which requires
`C[set[set[set[int]]]]`, etc. Each level has different type
specializations, so the existing cycle detection (using full types as
cache keys) didn't catch the infinite recursion.

This fix adds a simple recursion depth limit (64) to the CycleDetector.
When the depth exceeds the limit, we return the fallback value (assume
compatible) to safely terminate the recursion.

This is a bit of a blunt hammer, but it should be broadly effective to
prevent stack overflow in any nested-relation case, and it's hard to
imagine that non-recursive nested relation comparisons of depth > 64
exist much in the wild.

## Test Plan

Added mdtest.
2025-12-09 09:05:18 -08:00
David Peter aea2bc2308
[ty] Infer type variables within generic unions (#21862)
## Summary

This PR allows our generics solver to find a solution for `T` in cases
like the following:
```py
def extract_t[T](x: P[T] | Q[T]) -> T:
    raise NotImplementedError

reveal_type(extract_t(P[int]()))  # revealed: int
reveal_type(extract_t(Q[str]()))  # revealed: str
```

closes https://github.com/astral-sh/ty/issues/1772
closes https://github.com/astral-sh/ty/issues/1314

## Ecosystem

The impact here looks very good!

It took me a long time to figure this out, but the new diagnostics on
bokeh are actually true positives. I should have tested with another
type-checker immediately, I guess. All other type checkers also emit
errors on these `__init__` calls. MRE
[here](https://play.ty.dev/5c19d260-65e2-4f70-a75e-1a25780843a2) (no
error on main, diagnostic on this branch)

A lot of false positives on home-assistant go away for calls to
functions like
[`async_listen`](180053fe98/homeassistant/core.py (L1581-L1587))
which take a `event_type: EventType[_DataT] | str` parameter. We can now
solve for `_DataT` here, which was previously falling back to its
default value, and then caused problems because it was used as an
argument to an invariant generic class.

## Test Plan

New Markdown tests
2025-12-09 16:22:59 +01:00
Dhruv Manilawala c35bf8f441
[ty] Fix overload filtering to prefer more "precise" match (#21859)
## Summary

fixes: https://github.com/astral-sh/ty/issues/1809

I took this chance to add some debug level tracing logs for overload
call evaluation similar to Doug's implementation in `constraints.rs`.

## Test Plan

- Add new mdtests
- Tested it against `sqlalchemy.select` in pyx which results in the
correct overload being matched
2025-12-09 20:29:34 +05:30
Rasmus Nygren e548ce1ca9 [ty] Enrich function argument auto-complete suggestions with annotated types 2025-12-08 14:19:44 -05:00
Rasmus Nygren eac8a90cc4 [ty] Add autocomplete suggestions for function arguments
This adds autocomplete suggestions for function arguments. For example,
`okay` in:

```python
def foo(okay=None):

foo(o<CURSOR>
```

This also ensures that we don't suggest a keyword argument if it has
already been used.

Closes astral-sh/issues#1550
2025-12-08 14:19:44 -05:00
Alex Waygood 0ccd84136a
[ty] Make Python-version subdiagnostics less verbose (#21849) 2025-12-08 15:58:23 +00:00
Charlie Marsh 385dd2770b
[ty] Avoid double-inference on non-tuple argument to `Annotated` (#21837)
## Summary

If you pass a non-tuple to `Annotated`, we end up running inference on
it twice. I _think_ the only case here is `Annotated[]`, where we insert
a (fake) empty `Name` node in the slice.

Closes https://github.com/astral-sh/ty/issues/1801.
2025-12-08 10:24:05 -05:00
David Peter 4686111681
[ty] More SQLAlchemy test updates (#21846)
Minor updates to the SQLAlchemy test suite. I verified all expected
results using pyright.
2025-12-08 15:22:55 +01:00
David Peter c99e10eedc
[ty] Increase SQLAlchemy test coverage (#21843)
## Summary

Increase our SQLAlchemy test coverage to make sure we understand
`Session.scalar`, `Session.scalars`, `Session.execute` (and their async
equivalents), as well as `Result.tuples`, `Result.one_or_none`,
`Row._tuple`.
2025-12-08 14:36:13 +01:00
Dhruv Manilawala a364195335
[ty] Avoid diagnostic when `typing_extensions.ParamSpec` uses `default` parameter (#21839)
## Summary

fixes: https://github.com/astral-sh/ty/issues/1798

## Test Plan

Add mdtest.
2025-12-08 12:34:30 +00:00
David Peter dfd6ed0524
[ty] mdtests with external dependencies (#20904)
## Summary

This PR adds the possibility to write mdtests that specify external
dependencies in a `project` section of TOML blocks. For example, here is
a test that makes sure that we understand Pydantic's dataclass-transform
setup:

````markdown
```toml
[environment]
python-version = "3.12"
python-platform = "linux"

[project]
dependencies = ["pydantic==2.12.2"]
```

```py
from pydantic import BaseModel

class User(BaseModel):
    id: int
    name: str

user = User(id=1, name="Alice")
reveal_type(user.id)  # revealed: int
reveal_type(user.name)  # revealed: str

# error: [missing-argument] "No argument provided for required parameter
`name`"
invalid_user = User(id=2)
```
````

## How?

Using the `python-version` and the `dependencies` fields from the
Markdown section, we generate a `pyproject.toml` file, write it to a
temporary directory, and use `uv sync` to install the dependencies into
a virtual environment. We then copy the Python source files from that
venv's `site-packages` folder to a corresponding directory structure in
the in-memory filesystem. Finally, we configure the search paths
accordingly, and run the mdtest as usual.

I fully understand that there are valid concerns here:
* Doesn't this require network access? (yes, it does)
* Is this fast enough? (`uv` caching makes this almost unnoticeable,
actually)
* Is this deterministic? ~~(probably not, package resolution can depend
on the platform you're on)~~ (yes, hopefully)

For this reason, this first version is opt-in, locally. ~~We don't even
run these tests in CI (even though they worked fine in a previous
iteration of this PR).~~ You need to set `MDTEST_EXTERNAL=1`, or use the
new `-e/--enable-external` command line option of the `mdtest.py`
runner. For example:
```bash
# Skip mdtests with external dependencies (default):
uv run crates/ty_python_semantic/mdtest.py

# Run all mdtests, including those with external dependencies:
uv run crates/ty_python_semantic/mdtest.py -e

# Only run the `pydantic` tests. Use `-e` to make sure it is not skipped:
uv run crates/ty_python_semantic/mdtest.py -e pydantic
```

## Why?

I believe that this can be a useful addition to our testing strategy,
which lies somewhere between ecosystem tests and normal mdtests.
Ecosystem tests cover much more code, but they have the disadvantage
that we only see second- or third-order effects via diagnostic diffs. If
we unexpectedly gain or lose type coverage somewhere, we might not even
notice (assuming the gradual guarantee holds, and ecosystem code is
mostly correct). Another disadvantage of ecosystem checks is that they
only test checked-in code that is usually correct. However, we also want
to test what happens on wrong code, like the code that is momentarily
written in an editor, before fixing it. On the other end of the spectrum
we have normal mdtests, which have the disadvantage that they do not
reflect the reality of complex real-world code. We experience this
whenever we're surprised by an ecosystem report on a PR.

That said, these tests should not be seen as a replacement for either of
these things. For example, we should still strive to write detailed
self-contained mdtests for user-reported issues. But we might use this
new layer for regression tests, or simply as a debugging tool. It can
also serve as a tool to document our support for popular third-party
libraries.

## Test Plan

* I've been locally using this for a couple of weeks now.
* `uv run crates/ty_python_semantic/mdtest.py -e`
2025-12-08 11:44:20 +01:00
Dhruv Manilawala ac882f7e63
[ty] Handle various invalid explicit specializations for `ParamSpec` (#21821)
## Summary

fixes: https://github.com/astral-sh/ty/issues/1788

## Test Plan

Add new mdtests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-12-08 05:20:41 +00:00
Alex Waygood 857fd4f683
[ty] Add test case for fixed panic (#21832) 2025-12-07 15:58:11 +00:00
Charlie Marsh 285d6410d3
[ty] Avoid double-analyzing tuple in `Final` subscript (#21828)
## Summary

As-is, a single-element tuple gets destructured via:

```rust
let arguments = if let ast::Expr::Tuple(tuple) = slice {
    &*tuple.elts
} else {
    std::slice::from_ref(slice)
};
```

But then, because it's a single element, we call
`infer_annotation_expression_impl`, passing in the tuple, rather than
the first element.

Closes https://github.com/astral-sh/ty/issues/1793.
Closes https://github.com/astral-sh/ty/issues/1768.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-07 14:27:14 +00:00
Charlie Marsh ef45c97dab
[ty] Allow `tuple[Any, ...]` to assign to `tuple[int, *tuple[int, ...]]` (#21803)
## Summary

Closes https://github.com/astral-sh/ty/issues/1750.
2025-12-05 19:04:23 +00:00
Micha Reiser 9714c589e1
[ty] Support renaming import aliases (#21792) 2025-12-05 19:12:13 +01:00
Shunsuke Shibayama 2f05ffa2c8
[ty] more detailed description of "Size limit on unions of literals" in mdtest (#21804) 2025-12-05 17:34:39 +00:00
Dhruv Manilawala b623189560
[ty] Complete support for `ParamSpec` (#21445)
## Summary

Closes: https://github.com/astral-sh/ty/issues/157

This PR adds support for the following capabilities involving a
`ParamSpec` type variable:
- Representing `P.args` and `P.kwargs` in the type system
- Matching against a callable containing `P` to create a type mapping
- Specializing `P` against the stored parameters

The value of a `ParamSpec` type variable is being represented using
`CallableType` with a `CallableTypeKind::ParamSpecValue` variant. This
`CallableTypeKind` is expanded from the existing `is_function_like`
boolean flag. An `enum` is used as these variants are mutually
exclusive.

For context, an initial iteration made an attempt to expand the
`Specialization` to use `TypeOrParameters` enum that represents that a
type variable can specialize into either a `Type` or `Parameters` but
that increased the complexity of the code as all downstream usages would
need to handle both the variants appropriately. Additionally, we'd have
also need to establish an invariant that a regular type variable always
maps to a `Type` while a paramspec type variable always maps to a
`Parameters`.

I've intentionally left out checking and raising diagnostics when the
`ParamSpec` type variable and it's components are not being used
correctly to avoid scope increase and it can easily be done as a
follow-up. This would also include the scoping rules which I don't think
a regular type variable implements either.

## Test Plan

Add new mdtest cases and update existing test cases.

Ran this branch on pyx, no new diagnostics.

### Ecosystem analysis

There's a case where in an annotated assignment like:
```py
type CustomType[P] = Callable[...]

def value[**P](...): ...

def another[**P](...):
	target: CustomType[P] = value
```
The type of `value` is a callable and it has a paramspec that's bound to
`value`, `CustomType` is a type alias that's a callable and `P` that's
used in it's specialization is bound to `another`. Now, ty infers the
type of `target` same as `value` and does not use the declared type
`CustomType[P]`. [This is the
assignment](0980b9d9ab/src/async_utils/gen_transform.py (L108))
that I'm referring to which then leads to error in downstream usage.
Pyright and mypy does seem to use the declared type.

There are multiple diagnostics in `dd-trace-py` that requires support
for `cls`.

I'm seeing `Divergent` type for an example like which ~~I'm not sure
why, I'll look into it tomorrow~~ is because of a cycle as mentioned in
https://github.com/astral-sh/ty/issues/1729#issuecomment-3612279974:
```py
from typing import Callable

def decorator[**P](c: Callable[P, int]) -> Callable[P, str]: ...

@decorator
def func(a: int) -> int: ...

# ((a: int) -> str) | ((a: Divergent) -> str)
reveal_type(func)
```

I ~~need to look into why are the parameters not being specialized
through multiple decorators in the following code~~ think this is also
because of the cycle mentioned in
https://github.com/astral-sh/ty/issues/1729#issuecomment-3612279974 and
the fact that we don't support `staticmethod` properly:
```py
from contextlib import contextmanager

class Foo:
    @staticmethod
    @contextmanager
    def method(x: int):
        yield

foo = Foo()
# ty: Revealed type: `() -> _GeneratorContextManager[Unknown, None, None]` [revealed-type]
reveal_type(foo.method)
```

There's some issue related to `Protocol` that are generic over a
`ParamSpec` in `starlette` which might be related to
https://github.com/astral-sh/ty/issues/1635 but I'm not sure. Here's a
minimal example to reproduce:

<details><summary>Code snippet:</summary>
<p>

```py
from collections.abc import Awaitable, Callable, MutableMapping
from typing import Any, Callable, ParamSpec, Protocol

P = ParamSpec("P")

Scope = MutableMapping[str, Any]
Message = MutableMapping[str, Any]
Receive = Callable[[], Awaitable[Message]]
Send = Callable[[Message], Awaitable[None]]

ASGIApp = Callable[[Scope, Receive, Send], Awaitable[None]]

_Scope = Any
_Receive = Callable[[], Awaitable[Any]]
_Send = Callable[[Any], Awaitable[None]]

# Since `starlette.types.ASGIApp` type differs from `ASGIApplication` from `asgiref`
# we need to define a more permissive version of ASGIApp that doesn't cause type errors.
_ASGIApp = Callable[[_Scope, _Receive, _Send], Awaitable[None]]


class _MiddlewareFactory(Protocol[P]):
    def __call__(
        self, app: _ASGIApp, *args: P.args, **kwargs: P.kwargs
    ) -> _ASGIApp: ...


class Middleware:
    def __init__(
        self, factory: _MiddlewareFactory[P], *args: P.args, **kwargs: P.kwargs
    ) -> None:
        self.factory = factory
        self.args = args
        self.kwargs = kwargs


class ServerErrorMiddleware:
    def __init__(
        self,
        app: ASGIApp,
        value: int | None = None,
        flag: bool = False,
    ) -> None:
        self.app = app
        self.value = value
        self.flag = flag

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: ...


# ty: Argument to bound method `__init__` is incorrect: Expected `_MiddlewareFactory[(...)]`, found `<class 'ServerErrorMiddleware'>` [invalid-argument-type]
Middleware(ServerErrorMiddleware, value=500, flag=True)
```

</p>
</details> 

### Conformance analysis

> ```diff
> -constructors_callable.py:36:13: info[revealed-type] Revealed type:
`(...) -> Unknown`
> +constructors_callable.py:36:13: info[revealed-type] Revealed type:
`(x: int) -> Unknown`
> ```

Requires return type inference i.e.,
https://github.com/astral-sh/ruff/pull/21551

> ```diff
> +constructors_callable.py:194:16: error[invalid-argument-type]
Argument is incorrect: Expected `list[T@__init__]`, found `list[Unknown
| str]`
> +constructors_callable.py:194:22: error[invalid-argument-type]
Argument is incorrect: Expected `list[T@__init__]`, found `list[Unknown
| str]`
> +constructors_callable.py:195:4: error[invalid-argument-type] Argument
is incorrect: Expected `list[T@__init__]`, found `list[Unknown | int]`
> +constructors_callable.py:195:9: error[invalid-argument-type] Argument
is incorrect: Expected `list[T@__init__]`, found `list[Unknown | str]`
> ```

I might need to look into why this is happening...

> ```diff
> +generics_defaults.py:79:1: error[type-assertion-failure] Type
`type[Class_ParamSpec[(str, int, /)]]` does not match asserted type
`<class 'Class_ParamSpec'>`
> ```

which is on the following code
```py
DefaultP = ParamSpec("DefaultP", default=[str, int])

class Class_ParamSpec(Generic[DefaultP]): ...

assert_type(Class_ParamSpec, type[Class_ParamSpec[str, int]])
```

It's occurring because there's no equivalence relationship defined
between `ClassLiteral` and `KnownInstanceType::TypeGenericAlias` which
is what these types are.

Everything else looks good to me!
2025-12-05 22:00:06 +05:30
Douglas Creager e42cdf8495
[ty] Carry generic context through when converting class into `Callable` (#21798)
When converting a class (whether specialized or not) into a `Callable`
type, we should carry through any generic context that the constructor
has. This includes both the generic context of the class itself (if it's
generic) and of the constructor methods (if they are separately
generic).

To help test this, this also updates the `generic_context` extension
function to work on `Callable` types and unions; and adds a new
`into_callable` extension function that works just like
`CallableTypeOf`, but on value forms instead of type forms.

Pulled this out of #21551 for separate review.
2025-12-05 08:57:21 -05:00