Commit Graph

298 Commits

Author SHA1 Message Date
Alperen Keleş 932f941d15
[ty] fix binary expression inference between boolean literals and `bool` instances (#18663) 2025-06-17 18:02:40 +01:00
Alex Waygood 685eac10e5
Revert "[ty] Offer "Did you mean...?" suggestions for unresolved `from` imports and unresolved attributes (#18705)" (#18721) 2025-06-17 15:48:09 +01:00
Alex Waygood 913f136d33
[ty] Offer "Did you mean...?" suggestions for unresolved `from` imports and unresolved attributes (#18705)
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2025-06-17 11:10:34 +01:00
Dhruv Manilawala c7e020df6b
[ty] Filter overloads based on `Any` / `Unknown` (#18607)
## Summary

Closes: astral-sh/ty#552

This PR adds support for step 5 of the overload call evaluation
algorithm which specifies:

> For all arguments, determine whether all possible materializations of
the argument’s type are
> assignable to the corresponding parameter type for each of the
remaining overloads. If so,
> eliminate all of the subsequent remaining overloads.

The algorithm works in two parts:

1. Find out the participating parameter indexes. These are the
parameters that aren't gradual equivalent to one or more parameter types
at the same index in other overloads.
2. Loop over each overload and check whether that would be the _final_
overload for the argument types i.e., the remaining overloads will never
be matched against these argument types

For step 1, the participating parameter indexes are computed by just
comparing whether all the parameter types at the corresponding index for
all the overloads are **gradual equivalent**.

The step 2 of the algorithm used is described in [this
comment](https://github.com/astral-sh/ty/issues/552#issuecomment-2969165421).

## Test Plan

Update the overload call tests.
2025-06-17 15:35:09 +05:30
Alex Waygood 1d458d4314
[ty] Fix panics when pulling types for various special forms that have the wrong number of parameters (#18642) 2025-06-17 10:40:50 +01:00
Shunsuke Shibayama 342b2665db
[ty] basic narrowing on attribute and subscript expressions (#17643)
## Summary

This PR closes astral-sh/ty#164.

This PR introduces a basic type narrowing mechanism for
attribute/subscript expressions.
Member accesses, int literal subscripts, string literal subscripts are
supported (same as mypy and pyright).

## Test Plan

New test cases are added to `mdtest/narrow/complex_target.md`.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-06-17 11:07:46 +02:00
David Peter a1c69ca460
[ty] Enable ecosystem check for 'pywin32' (#18716)
## Summary

Follow-up to #18621
2025-06-17 09:52:26 +02:00
David Peter 3a77768f79
[ty] Reachability constraints (#18621)
## Summary



* Completely removes the concept of visibility constraints. Reachability
constraints are now used to model the static visibility of bindings and
declarations. Reachability constraints are *much* easier to reason about
/ work with, since they are applied at the beginning of a branch, and
not applied retroactively. Removing the duplication between visibility
and reachability constraints also leads to major code simplifications
[^1]. For an overview of how the new constraint system works, see the
updated doc comment in `reachability_constraints.rs`.
* Fixes a [control-flow modeling bug
(panic)](https://github.com/astral-sh/ty/issues/365) involving `break`
statements in loops
* Fixes a [bug where](https://github.com/astral-sh/ty/issues/624) where
`elif` branches would have wrong reachability constraints
* Fixes a [bug where](https://github.com/astral-sh/ty/issues/648) code
after infinite loops would not be considered unreachble
* Fixes a panic on the `pywin32` ecosystem project, which we should be
able to move to `good.txt` once this has been merged.
* Removes some false positives in unreachable code because we infer
`Never` more often, due to the fact that reachability constraints now
apply retroactively to *all* active bindings, not just to bindings
inside a branch.
* As one example, this removes the `division-by-zero` diagnostic from
https://github.com/astral-sh/ty/issues/443 because we now infer `Never`
for the divisor.
* Supersedes and includes similar test changes as
https://github.com/astral-sh/ruff/pull/18392


closes https://github.com/astral-sh/ty/issues/365
closes https://github.com/astral-sh/ty/issues/624
closes https://github.com/astral-sh/ty/issues/642
closes https://github.com/astral-sh/ty/issues/648

## Benchmarks

Benchmarks on black, pandas, and sympy showed that this is neither a
performance improvement, nor a regression.

## Test Plan

Regression tests for:
- [x] https://github.com/astral-sh/ty/issues/365
- [x] https://github.com/astral-sh/ty/issues/624
- [x] https://github.com/astral-sh/ty/issues/642
- [x] https://github.com/astral-sh/ty/issues/648

[^1]: I'm afraid this is something that @carljm advocated for since the
beginning, and I'm not sure anymore why we have never seriously tried
this before. So I suggest we do *not* attempt to do a historical deep
dive to find out exactly why this ever became so complicated, and just
enjoy the fact that we eventually arrived here.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-06-17 09:24:28 +02:00
Micha Reiser c22f809049
Hug closing `}` when f-string expression has a format specifier (#18704) 2025-06-17 07:39:42 +02:00
Alex Waygood 2b731d19b9
[ty] Fix panic when attempting to provide autocompletions for an instance of a class that assigns attributes to `self[0]` (#18707) 2025-06-16 21:58:05 +00:00
Felix Scherz 373a3bfcd6
[ty] allow `T: Never` as subtype of `Never` (#18687) 2025-06-16 17:46:17 +00:00
Alex Waygood 5e57e4680f
[ty] Use more parallelism when running corpus tests (#18711) 2025-06-16 17:38:55 +00:00
Abhijeet Prasad Bodas 2b15f1d240
[ty] Support `dataclasses.KW_ONLY` (#18677) 2025-06-16 17:27:55 +00:00
Micha Reiser 3a430fa6da
[ty] Allow overriding rules for specific files (#18648) 2025-06-15 14:27:39 +01:00
Ibraheem Ahmed 5e02d839d5
[ty] Avoid accessing class literal with incorrect AST (#18670) 2025-06-14 06:02:53 +01:00
InSync 6d56ee803e
[ty] Add partial support for `TypeIs` (#18589)
## Summary

Part of [#117](https://github.com/astral-sh/ty/issues/117).

`TypeIs[]` is a special form that allows users to define their own
narrowing functions. Despite the syntax, `TypeIs` is not a generic and,
on its own, it is meaningless as a type.
[Officially](https://typing.python.org/en/latest/spec/narrowing.html#typeis),
a function annotated as returning a `TypeIs[T]` is a <i>type narrowing
function</i>, where `T` is called the <i>`TypeIs` return type</i>.

A `TypeIs[T]` may or may not be bound to a symbol. Only bound types have
narrowing effect:

```python
def f(v: object = object()) -> TypeIs[int]: ...

a: str = returns_str()

if reveal_type(f()):   # Unbound: TypeIs[int]
	reveal_type(a)     # str

if reveal_type(f(a)):  # Bound:   TypeIs[a, int]
	reveal_type(a)     # str & int
```

Delayed usages of a bound type has no effect, however:

```python
b = f(a)

if b:
	reveal_type(a)     # str
```

A `TypeIs[T]` type:

* Is fully static when `T` is fully static.
* Is a singleton/single-valued when it is bound.
* Has exactly two runtime inhabitants when it is unbound: `True` and
`False`.
  In other words, an unbound type have ambiguous truthiness.
It is possible to infer more precise truthiness for bound types;
however, that is not part of this change.

`TypeIs[T]` is a subtype of or otherwise assignable to `bool`. `TypeIs`
is invariant with respect to the `TypeIs` return type: `TypeIs[int]` is
neither a subtype nor a supertype of `TypeIs[bool]`. When ty sees a
function marked as returning `TypeIs[T]`, its `return`s will be checked
against `bool` instead. ty will also report such functions if they don't
accept a positional argument. Addtionally, a type narrowing function
call with no positional arguments (e.g., `f()` in the example above)
will be considered invalid.

## Test Plan

Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-06-13 15:27:45 -07:00
David Peter 89d915a1e3
[ty] Delay computation of 'unbound' visibility for implicit instance attributes (#18669)
## Summary

Consider the following example, which leads to a excessively large
runtime on `main`. The reason for this is the following. When inferring
types for `self.a`, we look up the `a` attribute on `C`. While looking
for implicit instance attributes, we go through every method and check
for `self.a = …` assignments. There are no such assignments here, but we
always have an implicit `self.a = <unbound>` binding at the beginning
over every method. This binding accumulates a complex visibility
constraint in `C.f`, due to the `isinstance` checks. While evaluating
that constraint, we need to infer the type of `self.b`. There's no
binding for `self.b` either, but there's also an implicit `self.b =
<unbound>` binding with the same complex visibility constraint
(involving `self.b` recursively). This leads to a combinatorial
explosion:

```py
class C:
    def f(self: "C"):
        if isinstance(self.a, str):
            return

        if isinstance(self.b, str):
            return
        if isinstance(self.b, str):
            return
        if isinstance(self.b, str):
            return
        # repeat 20 times
```
(note that the `self` parameter here is annotated explicitly because we
currently still infer `Unknown` for `self` otherwise)

The fix proposed here is rather simple: when there are no `self.name =
…` attribute assignments in a given method, we skip evaluating the
visibility constraint of the implicit `self.name = <unbound>` binding.
This should also generally help with performance, because that's a very
common case.

This is *not* a fix for cases where there *are* actual bindings in the
method. When we add `self.a = 1; self.b = 1` to that example above, we
still see that combinatorial explosion of runtime. I still think it's
worth to make this optimization, as it fixes the problems with `pandas`
and `sqlalchemy` reported by users. I will open a ticket to track that
separately.

closes https://github.com/astral-sh/ty/issues/627
closes https://github.com/astral-sh/ty/issues/641

## Test Plan

* Made sure that `ty` finishes quickly on the MREs in
https://github.com/astral-sh/ty/issues/627
* Made sure that `ty` finishes quickly on `pandas`
* Made sure that `ty` finishes quickly on `sqlalchemy`
2025-06-13 12:50:57 -07:00
Ibraheem Ahmed c9dff5c7d5
[ty] AST garbage collection (#18482)
## Summary

Garbage collect ASTs once we are done checking a given file. Queries
with a cross-file dependency on the AST will reparse the file on demand.
This reduces ty's peak memory usage by ~20-30%.

The primary change of this PR is adding a `node_index` field to every
AST node, that is assigned by the parser. `ParsedModule` can use this to
create a flat index of AST nodes any time the file is parsed (or
reparsed). This allows `AstNodeRef` to simply index into the current
instance of the `ParsedModule`, instead of storing a pointer directly.

The indices are somewhat hackily (using an atomic integer) assigned by
the `parsed_module` query instead of by the parser directly. Assigning
the indices in source-order in the (recursive) parser turns out to be
difficult, and collecting the nodes during semantic indexing is
impossible as `SemanticIndex` does not hold onto a specific
`ParsedModuleRef`, which the pointers in the flat AST are tied to. This
means that we have to do an extra AST traversal to assign and collect
the nodes into a flat index, but the small performance impact (~3% on
cold runs) seems worth it for the memory savings.

Part of https://github.com/astral-sh/ty/issues/214.
2025-06-13 08:40:11 -04:00
Shunsuke Shibayama ef564094a9
[ty] support del statement and deletion of except handler names (#18593)
## Summary

This PR closes https://github.com/astral-sh/ty/issues/238.

Since `DefinitionState::Deleted` was introduced in #18041, support for
the `del` statement (and deletion of except handler names) is
straightforward.

However, it is difficult to determine whether references to attributes
or subscripts are unresolved after they are deleted. This PR only
invalidates narrowing by assignment if the attribute or subscript is
deleted.

## Test Plan

`mdtest/del.md` is added.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-06-12 07:44:42 -07:00
Alex Waygood 324e5cbc19
[ty] Pull types on synthesized Python files created by mdtest (#18539) 2025-06-12 10:32:17 +01:00
Dhruv Manilawala ef4108af2a
[ty] Generate the top and bottom materialization of a type (#18594)
## Summary

This is to support https://github.com/astral-sh/ruff/pull/18607.

This PR adds support for generating the top materialization (or upper
bound materialization) and the bottom materialization (or lower bound
materialization) of a type. This is the most general and the most
specific form of the type which is fully static, respectively.
    
More concretely, `T'`, the top materialization of `T`, is the type `T`
with all occurrences
of dynamic type (`Any`, `Unknown`, `@Todo`) replaced as follows:

- In covariant position, it's replaced with `object`
- In contravariant position, it's replaced with `Never`
- In invariant position, it's replaced with an unresolved type variable

(For an invariant position, it should actually be replaced with an
existential type, but this is not currently representable in our type
system, so we use an unresolved type variable for now instead.)

The bottom materialization is implemented in the same way, except we
start out in "contravariant" position.

## Test Plan

Add test cases for various types.
2025-06-12 12:06:16 +05:30
Alex Waygood e84406d8be
[ty] Infer the Python version from `--python=<system installation>` on Unix (#18550) 2025-06-11 14:32:33 +00:00
Micha Reiser 3aae1cd59b
Fix incorrect salsa `return_ref` attribute (#18605) 2025-06-11 09:19:57 +02:00
Micha Reiser 5dcfc9f074
Move corpus tests to `ty_python_semantic` (#18609) 2025-06-11 08:55:30 +02:00
Carl Meyer a2de81cb27
[ty] implement disjointness of Callable vs SpecialForm (#18503)
## Summary

Fixes https://github.com/astral-sh/ty/issues/557

## Test Plan

Stable property tests succeed with a million iterations. Added mdtests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-06-10 20:25:08 +00:00
Carl Meyer eb60bd64fd
[ty] more simplification of infer_parameterized_legacy_typing_alias (#18526)
Address post-land review on https://github.com/astral-sh/ruff/pull/18489
2025-06-10 13:22:25 -07:00
Suneet Tipirneni 161446a47a
[ty] Add support for global __debug__ constant (#18540)
## Summary

Closes https://github.com/astral-sh/ty/issues/577. Make global
`__debug__` a `bool` constant.

## Test Plan

Mdtest `global-constants.md` was created to check if resolved type was
`bool`.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-06-10 06:48:59 +00:00
Alex Waygood aa3c312f5f
[ty] Fix panic when trying to pull types for subscript expressions inside `Callable` type expressions (#18534) 2025-06-09 11:26:10 +01:00
renovate[bot] 475a02b725
Update pre-commit dependencies (#18581) 2025-06-09 08:08:17 +02:00
Ben Bar-Or 1dc8f8f903
[ty] Add hints to `invalid-type-form` for common mistakes (#18543)
Co-authored-by: Ben Bar-Or <ben.baror@ridewithvia.com>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-06-09 00:40:05 +01:00
Alex Waygood 72552f31e4
[ty] Fix panic when pulling types for `UnaryOp` expressions inside `Literal` slices (#18536) 2025-06-07 15:26:10 +00:00
Alex Waygood 95497ffaab
[ty] Fix panic when trying to pull types for attribute expressions inside `Literal` type expressions (#18535) 2025-06-07 15:59:12 +01:00
Alex Waygood 6e785867c3
[ty] Unify `Type::is_subtype_of()` and `Type::is_assignable_to()` (#18430) 2025-06-06 17:28:55 +00:00
Alex Waygood 1274521f9f
[ty] Track the origin of the `environment.python` setting for better error messages (#18483) 2025-06-06 13:36:41 +01:00
Carl Meyer db8db536f8
[ty] clarify requirements for scope_id argument to in_type_expression (#18488) 2025-06-05 22:46:26 -07:00
Carl Meyer cb8246bc5f
[ty] remove unnecessary Either (#18489)
Just a quick review-comment follow-up.
2025-06-05 18:39:22 -07:00
Ibraheem Ahmed 8531f4b3ca
[ty] Add infrastructure for AST garbage collection (#18445)
## Summary

https://github.com/astral-sh/ty/issues/214 will require a couple
invasive changes that I would like to get merged even before garbage
collection is fully implemented (to avoid rebasing):
- `ParsedModule` can no longer be dereferenced directly. Instead you
need to load a `ParsedModuleRef` to access the AST, which requires a
reference to the salsa database (as it may require re-parsing the AST if
it was collected).
- `AstNodeRef` can only be dereferenced with the `node` method, which
takes a reference to the `ParsedModuleRef`. This allows us to encode the
fact that ASTs do not live as long as the database and may be collected
as soon a given instance of a `ParsedModuleRef` is dropped. There are a
number of places where we currently merge the `'db` and `'ast`
lifetimes, so this requires giving some types/functions two separate
lifetime parameters.
2025-06-05 11:43:18 -04:00
Andrew Gallant 55100209c7
[ty] IDE: add support for `object.<CURSOR>` completions (#18468)
This PR adds logic for detecting `Name Dot [Name]` token patterns,
finding the corresponding `ExprAttribute`, getting the type of the
object and returning the members available on that object.

Here's a video demonstrating this working:

https://github.com/user-attachments/assets/42ce78e8-5930-4211-a18a-fa2a0434d0eb

Ref astral-sh/ty#86
2025-06-05 11:15:19 -04:00
Alex Waygood 8485dbb324
[ty] Fix `--python` argument for Windows, and improve error messages for bad `--python` arguments (#18457)
## Summary

Fixes https://github.com/astral-sh/ty/issues/556.

On Windows, system installations have different layouts to virtual
environments. In Windows virtual environments, the Python executable is
found at `<sys.prefix>/Scripts/python.exe`. But in Windows system
installations, the Python executable is found at
`<sys.prefix>/python.exe`. That means that Windows users were able to
point to Python executables inside virtual environments with the
`--python` flag, but they weren't able to point to Python executables
inside system installations.

This PR fixes that issue. It also makes a couple of other changes:
- Nearly all `sys.prefix` resolution is moved inside `site_packages.rs`.
That was the original design of the `site-packages` resolution logic,
but features implemented since the initial implementation have added
some resolution and validation to `resolver.rs` inside the module
resolver. That means that we've ended up with a somewhat confusing code
structure and a situation where several checks are unnecessarily
duplicated between the two modules.
- I noticed that we had quite bad error messages if you e.g. pointed to
a path that didn't exist on disk with `--python` (we just gave a
somewhat impenetrable message saying that we "failed to canonicalize"
the path). I improved the error messages here and added CLI tests for
`--python` and the `environment.python` configuration setting.

## Test Plan

- Existing tests pass
- Added new CLI tests
- I manually checked that virtual-environment discovery still works if
no configuration is given
- Micha did some manual testing to check that pointing `--python` to a
system-installation executable now works on Windows
2025-06-05 08:19:15 +01:00
Shunsuke Shibayama 0858896bc4
[ty] type narrowing by attribute/subscript assignments (#18041)
## Summary

This PR partially solves https://github.com/astral-sh/ty/issues/164
(derived from #17643).

Currently, the definitions we manage are limited to those for simple
name (symbol) targets, but we expand this to track definitions for
attribute and subscript targets as well.

This was originally planned as part of the work in #17643, but the
changes are significant, so I made it a separate PR.
After merging this PR, I will reflect this changes in #17643.

There is still some incomplete work remaining, but the basic features
have been implemented, so I am publishing it as a draft PR.
Here is the TODO list (there may be more to come):
* [x] Complete rewrite and refactoring of documentation (removing
`Symbol` and replacing it with `Place`)
* [x] More thorough testing
* [x] Consolidation of duplicated code (maybe we can consolidate the
handling related to name, attribute, and subscript)

This PR replaces the current `Symbol` API with the `Place` API, which is
a concept that includes attributes and subscripts (the term is borrowed
from Rust).

## Test Plan

`mdtest/narrow/assignment.md` is added.

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-06-04 17:24:27 -07:00
Alex Waygood ce8b744f17
[ty] Only calculate information for unresolved-reference subdiagnostic if we know we'll emit the diagnostic (#18465)
## Summary

This optimizes some of the logic added in
https://github.com/astral-sh/ruff/pull/18444. In general, we only
calculate information for subdiagnostics if we know we'll actually emit
the diagnostic. The check to see whether we'll emit the diagnostic is
work we'll definitely have to do whereas the the work to gather
information for a subdiagnostic isn't work we necessarily have to do if
the diagnostic isn't going to be emitted at all.

This PR makes us lazier about gathering the information we need for the
subdiagnostic, and moves all the subdiagnostic logic into one function
rather than having some `unresolved-reference` subdiagnostic logic in
`infer.rs` and some in `diagnostic.rs`.

## Test Plan

`cargo test -p ty_python_semantic`
2025-06-04 20:41:00 +01:00
Alex Waygood 5a8cdab771
[ty] Only consider a type `T` a subtype of a protocol `P` if all of `P`'s members are fully bound on `T` (#18466)
## Summary

Fixes https://github.com/astral-sh/ty/issues/578

## Test Plan

mdtests
2025-06-04 19:39:14 +00:00
Alex Waygood 3a8191529c
[ty] Exclude members starting with `_abc_` from a protocol interface (#18467)
## Summary

As well as excluding a hardcoded set of special attributes, CPython at
runtime also excludes any attributes or declarations starting with
`_abc_` from the set of members that make up a protocol interface. I
missed this in my initial implementation.

This is a bit of a CPython implementation detail, but I do think it's
important that we try to model the runtime as best we can here. The
closer we are to the runtime behaviour, the closer we come to sound
behaviour when narrowing types from `isinstance()` checks against
runtime-checkable protocols (for example)

## Test Plan

Extended an existing mdtest
2025-06-04 20:34:09 +01:00
lipefree e658778ced
[ty] Add subdiagnostic suggestion to `unresolved-reference` diagnostic when variable exists on `self` (#18444)
## Summary

Closes https://github.com/astral-sh/ty/issues/502.

In the following example:
```py
class Foo:
    x: int

    def method(self):
        y = x
```
The user may intended to use `y = self.x` in `method`. 

This is now added as a subdiagnostic in the following form : 

`info: An attribute with the same name as 'x' is defined, consider using
'self.x'`

## Test Plan

Added mdtest with snapshot diagnostics.
2025-06-04 08:13:50 -07:00
David Peter f1883d71a4
[ty] IDE: only provide declarations and bindings as completions (#18456)
## Summary

Previously, all symbols where provided as possible completions. In an
example like the following, both `foo` and `f` were suggested as
completions, because `f` itself is a symbol.
```py
foo = 1

f<CURSOR>
```
Similarly, in the following example, `hidden_symbol` was suggested, even
though it is not statically visible:
```py
if 1 + 2 != 3:
    hidden_symbol = 1

hidden_<CURSOR>
```

With the change suggested here, we only use statically visible
declarations and bindings as a source for completions.


## Test Plan

- Updated snapshot tests
- New test for statically hidden definitions
- Added test for star import
2025-06-04 16:11:05 +02:00
David Peter 11db567b0b
[ty] ty_ide: Hotfix for `expression_scope_id` panics (#18455)
## Summary

Implement a hotfix for the playground/LSP crashes related to missing
`expression_scope_id`s.

relates to: https://github.com/astral-sh/ty/issues/572

## Test Plan

* Regression tests from https://github.com/astral-sh/ruff/pull/18441
* Ran the playground locally to check if panics occur / completions
still work.

---------

Co-authored-by: Andrew Gallant <andrew@astral.sh>
2025-06-04 10:39:16 +02:00
David Peter 9f8c3de462
[ty] Improve docs for Class{Literal,Type}::instance_member (#18454)
## Summary

Mostly just refer to `Type::instance_member` which has much more
details.
2025-06-04 09:55:45 +02:00
David Peter 293d4ac388
[ty] Add meta-type tests for legavy TypeVars (#18453)
## Summary

Follow up to the comment by @dcreager
[here](https://github.com/astral-sh/ruff/pull/18439#discussion_r2123802784).
2025-06-04 07:44:44 +00:00
Dhruv Manilawala 453e5f5934
[ty] Add tests for empty list/tuple unpacking (#18451)
## Summary

This PR is to address this comment:
https://github.com/astral-sh/ruff/pull/18438#issuecomment-2935344415

## Test Plan

Run mdtest
2025-06-04 02:40:26 +00:00
Dhruv Manilawala 7ea773daf2
[ty] Argument type expansion for overload call evaluation (#18382)
## Summary

Part of astral-sh/ty#104, closes: astral-sh/ty#468

This PR implements the argument type expansion which is step 3 of the
overload call evaluation algorithm.

Specifically, this step needs to be taken if type checking resolves to
no matching overload and there are argument types that can be expanded.

## Test Plan

Add new test cases.

## Ecosystem analysis

This PR removes 174 `no-matching-overload` false positives -- I looked
at a lot of them and they all are false positives.

One thing that I'm not able to understand is that in
2b7e3adf27/sphinx/ext/autodoc/preserve_defaults.py (L179)
the inferred type of `value` is `str | None` by ty and Pyright, which is
correct, but it's only ty that raises `invalid-argument-type` error
while Pyright doesn't. The constructor method of `DefaultValue` has
declared type of `str` which is invalid.

There are few cases of false positives resulting due to the fact that ty
doesn't implement narrowing on attribute expressions.
2025-06-04 02:12:00 +00:00
Alex Waygood 0079cc6817
[ty] Minor cleanup for `site-packages` discovery logic (#18446) 2025-06-03 18:49:14 +00:00
Matthew Mckee e8ea40012a
[ty] Add generic inference for dataclasses (#18443)
## Summary

An issue seen here https://github.com/astral-sh/ty/issues/500

The `__init__` method of dataclasses had no inherited generic context,
so we could not infer the type of an instance from a constructor call
with generics

## Test Plan

Add tests to classes.md` in generics folder
2025-06-03 09:59:43 -07:00
Abhijeet Prasad Bodas 71d8a5da2a
[ty] dataclasses: Allow using dataclasses.dataclass as a function. (#18440)
## Summary

Part of https://github.com/astral-sh/ty/issues/111

Using `dataclass` as a function, instead of as a decorator did not work
as expected prior to this.
Fix that by modifying the dataclass overload's return type.

## Test Plan

New mdtests, fixing the existing TODO.
2025-06-03 09:50:29 -07:00
Douglas Creager 2c3b3d3230
[ty] Create separate `FunctionLiteral` and `FunctionType` types (#18360)
This updates our representation of functions to more closely match our
representation of classes.

The new `OverloadLiteral` and `FunctionLiteral` classes represent a
function definition in the AST. If a function is generic, this is
unspecialized. `FunctionType` has been updated to represent a function
type, which is specialized if the function is generic. (These names are
chosen to match `ClassLiteral` and `ClassType` on the class side.)

This PR does not add a separate `Type` variant for `FunctionLiteral`.
Maybe we should? Possibly as a follow-on PR?

Part of https://github.com/astral-sh/ty/issues/462

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-06-03 10:59:31 -04:00
Dhruv Manilawala 8d98c601d8
[ty] Infer `list[T]` when unpacking non-tuple type (#18438)
## Summary

Follow-up from #18401, I was looking at whether that would fix the issue
at https://github.com/astral-sh/ty/issues/247#issuecomment-2917656676
and it didn't, which made me realize that the PR only inferred `list[T]`
when the value type was tuple but it could be other types as well.

This PR fixes the actual issue by inferring `list[T]` for the non-tuple
type case.

## Test Plan

Add test cases for starred expression involved with non-tuple type. I
also added a few test cases for list type and list literal.

I also verified that the example in the linked issue comment works:
```py
def _(line: str):
    a, b, *c = line.split(maxsplit=2)
    c.pop()
```
2025-06-03 19:17:47 +05:30
David Peter 0986edf427
[ty] Meta-type of type variables should be type[..] (#18439)
## Summary

Came across this while debugging some ecosystem changes in
https://github.com/astral-sh/ruff/pull/18347. I think the meta-type of a
typevar-annotated variable should be equal to `type`, not `<class
'object'>`.

## Test Plan

New Markdown tests.
2025-06-03 15:22:00 +02:00
lipefree f23d2c9b9e
[ty] Support using legacy typing aliases for generic classes in type annotations (#18404)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-06-03 12:09:51 +01:00
Dhruv Manilawala 2289187b74
Infer `list[T]` for starred target in unpacking (#18401)
## Summary

Closes: astral-sh/ty#191

## Test Plan

Update existing tests.
2025-06-03 07:25:07 +05:30
lipefree f379eb6e62
[ty] Treat lambda functions as instances of types.FunctionType (#18431) 2025-06-02 16:46:26 +01:00
Alex Waygood 47698883ae
[ty] Fix false positives for legacy `ParamSpec`s inside `Callable` type expressions (#18426) 2025-06-02 14:10:00 +01:00
Alex Waygood e2d96df501
[ty] Improve diagnostics if the user attempts to import a stdlib module that does not exist on their configured Python version (#18403) 2025-06-02 10:52:26 +00:00
Matthew Mckee 97b824db3e
[ty] Ensure `Literal` types are considered assignable to anything their `Instance` supertypes are assignable to (#18351) 2025-06-01 16:39:56 +01:00
Micha Reiser 220ab88779
[ty] Promote projects to good that now no longer hang (#18370) 2025-06-01 17:25:46 +02:00
Micha Reiser 54f597658c
[ty] Fix multithreading related hangs and panics (#18238) 2025-06-01 11:07:55 +02:00
Zanie Blue 88866f0048
[ty] Infer the Python version from the environment if feasible (#18057)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-05-30 21:22:51 +00:00
Dylan 9bbf4987e8
Implement template strings (#17851)
This PR implements template strings (t-strings) in the parser and
formatter for Ruff.

Minimal changes necessary to compile were made in other parts of the code (e.g. ty, the linter, etc.). These will be covered properly in follow-up PRs.
2025-05-30 15:00:56 -05:00
Carl Meyer ad024f9a09
[ty] support callability of bound/constrained typevars (#18389)
## Summary

Allow a typevar to be callable if it is bound to a callable type, or
constrained to callable types.

I spent some time digging into why this support didn't fall out
naturally, and ultimately the reason is that we look up `__call__` on
the meta type (since its a dunder), and our implementation of
`Type::to_meta_type` for `Type::Callable` does not return a type with
`__call__`.

A more general solution here would be to have `Type::to_meta_type` for
`Type::Callable` synthesize a protocol with `__call__` and return an
intersection with that protocol (since for a type to be callable, we
know its meta-type must have `__call__`). That solution could in
principle also replace the special-case handling of `Type::Callable`
itself, here in `Type::bindings`. But that more general approach would
also be slower, and our protocol support isn't quite ready for that yet,
and handling this directly in `Type::bindings` is really not bad.

Fixes https://github.com/astral-sh/ty/issues/480

## Test Plan

Added mdtests.
2025-05-30 12:01:51 -07:00
Andrew Gallant fc549bda94
[ty] Minor tweaks to "list all members" docs and tests (#18388)
Ref https://github.com/astral-sh/ruff/pull/18251#pullrequestreview-2881810681
2025-05-30 13:36:57 -04:00
Alex Waygood 77c8ddf101
[ty] Fix broken property tests for disjointness (#18384) 2025-05-30 16:49:20 +01:00
David Peter e730f27f80
[ty] List available members for a given type (#18251)
This PR adds initial support for listing all attributes of
an object. It is exposed through a new `all_members`
routine in `ty_extensions`, which is in turn used to test
the functionality.

The purpose of listing all members is for code
completion. That is, given a `object.<CURSOR>`, we
would like to list all available attributes on
`object`.
2025-05-30 11:24:20 -04:00
Micha Reiser 8005ebb405
Update salsa past generational id change (#18362) 2025-05-30 15:31:33 +02:00
Alex Waygood ad2f667ee4
[ty] Improve tests for `site-packages` discovery (#18374)
## Summary

- Convert tests demonstrating our resilience to malformed/absent
`version` fields in `pyvenf.cfg` files to mdtests. Also make them more
expansive.
- Convert the regression test I added in
https://github.com/astral-sh/ruff/pull/18157 to an mdtest
- Add comments next to unit tests that cannot be converted to mdtests
(but where it's not obvious why they can't) so I don't have to do this
exercise again 😄
- In `site_packages.rs`, factor out the logic for figuring out where we
expect the system-installation `site-packages` to be. Currently we have
the same logic twice.

## Test Plan

`cargo test -p ty_python_semantic`
2025-05-30 07:32:21 +01:00
Carl Meyer 363f061f09
[ty] _typeshed.Self is not a special form (#18377)
## Summary

This change was based on a mis-reading of a comment in typeshed, and a
wrong assumption about what was causing a test failure in a prior PR.
Reverting it doesn't cause any tests to fail.

## Test Plan

Existing tests.
2025-05-29 17:11:13 -07:00
InSync 9b0dfc505f
[ty] Callable types are disjoint from non-callable `@final` nominal instance types (#18368)
## Summary

Resolves [#513](https://github.com/astral-sh/ty/issues/513).

Callable types are now considered to be disjoint from nominal instance
types where:

* The class is `@final`, and
* Its `__call__` either does not exist or is not assignable to `(...) ->
Unknown`.

## Test Plan

Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-29 23:27:27 +00:00
lipefree 695de4f27f
[ty] Add diagnosis for function with no return statement but with return type annotation (#18359)
## Summary

Partially implement https://github.com/astral-sh/ty/issues/538, 
```py
from pathlib import Path

def setup_test_project(registry_name: str, registry_url: str, project_dir: str) -> Path:
    pyproject_file = Path(project_dir) / "pyproject.toml"
    pyproject_file.write_text("...", encoding="utf-8")
```
As no return statement is defined in the function `setup_test_project`
with annotated return type `Path`, we provide the following diagnosis :

- error[invalid-return-type]: Function **always** implicitly returns
`None`, which is not assignable to return type `Path`

with a subdiagnostic : 
- note: Consider changing your return annotation to `-> None` or adding a `return` statement
 
## Test Plan

mdtests with snapshots to capture the subdiagnostic. I have to mention
that existing snapshots were modified since they now fall in this
category.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-29 23:17:18 +00:00
Andrew Gallant 33ed502edb ty_ide: improve completions by using scopes
Previously, completions were based on just returning every identifier
parsed in the current Python file. In this commit, we change it to
identify an expression under the cursor and then return all symbols
available to the scope containing that expression.

This is still returning too much, and also, in some cases, not enough.
Namely, it doesn't really take the specific context into account other
than scope. But this does improve on the status quo. For example:

    def foo(): ...
    def bar():
        def fast(): ...
    def foofoo(): ...

    f<CURSOR>

When asking for completions here, the LSP will no longer include `fast`
as a possible completion in this context.

Ref https://github.com/astral-sh/ty/issues/86
2025-05-29 10:31:30 -04:00
Alex Waygood 47a2ec002e
[ty] Split `Type::KnownInstance` into two type variants (#18350) 2025-05-29 14:47:55 +01:00
Matthew Mckee c60b4d7f30
[ty] Add subtyping between Callable types and class literals with `__init__` (#17638)
## Summary

Allow classes with `__init__` to be subtypes of `Callable`

Fixes https://github.com/astral-sh/ty/issues/358

## Test Plan

Update is_subtype_of.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-28 13:43:07 -07:00
Douglas Creager 452f992fbc
[ty] Simplify signature types, use them in `CallableType` (#18344)
There were many fields in `Signature` and friends that really had more
to do with how a signature was being _used_ — how it was looked up,
details about an individual call site, etc. Those fields more properly
belong in `Bindings` and friends.

This is a pure refactoring, and should not affect any tests or ecosystem
projects.

I started on this journey in support of
https://github.com/astral-sh/ty/issues/462. It seemed worth pulling out
as a separate PR.

One major concrete benefit of this refactoring is that we can now use
`CallableSignature` directly in `CallableType`. (We can't use
`CallableSignature` directly in that `Type` variant because signatures
are not currently interned.)
2025-05-28 13:11:45 -04:00
Alex Waygood a5ebb3f3a2
[ty] Support ephemeral uv virtual environments (#18335) 2025-05-28 14:54:59 +00:00
David Peter bbcd7e0196
[ty] Synthetic function-like callables (#18242)
## Summary

We create `Callable` types for synthesized functions like the `__init__`
method of a dataclass. These generated functions are real functions
though, with descriptor-like behavior. That is, they can bind `self`
when accessed on an instance. This was modeled incorrectly so far.

## Test Plan

Updated tests
2025-05-28 10:00:56 +02:00
Carl Meyer 3eada01153
put similar dunder-call tests next to each other (#18343)
Follow-up from post-land review on
https://github.com/astral-sh/ruff/pull/18260
2025-05-27 12:16:41 -07:00
Alex Waygood 3e811fc369
[ty] Derive `PartialOrd, Ord` for `KnownInstanceType` (#18340) 2025-05-27 19:37:01 +01:00
Alex Waygood 743764d384
[ty] Simplify `Type::try_bool()` (#18342)
## Summary

I don't think we're ever going to add any `KnownInstanceType` variants
that evaluate to `False` in a boolean context; the
`KnownInstanceType::bool()` method just seems like unnecessary
complexity.

## Test Plan

`cargo test -p ty_python_semantic`
2025-05-27 19:32:17 +01:00
Alex Waygood e03e05d2b3
[ty] Simplify `Type::normalized` slightly (#18339) 2025-05-27 18:08:59 +00:00
Alex Waygood 9ec4a178a4
[ty] Move arviz off the list of selected primer projects (#18336) 2025-05-27 17:51:19 +01:00
Alex Waygood 6453ac9ea1
[ty] Tell the user why we inferred a certain Python version when reporting version-specific syntax errors (#18295) 2025-05-26 20:44:43 +00:00
Alex Waygood 0a11baf29c
[ty] Implement implicit inheritance from `Generic[]` for PEP-695 generic classes (#18283) 2025-05-26 20:40:16 +01:00
lipefree 1d20cf9570
[ty] Add hint if async context manager is used in non-async with statement (#18299)
# Summary

Adds a subdiagnostic hint in the following scenario where a
synchronous `with` is used with an async context manager:
```py
class Manager:
    async def __aenter__(self): ...
    async def __aexit__(self, *args): ...

# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because it does not implement `__enter__` and `__exit__`"
# note: Objects of type `Manager` *can* be used as async context managers
# note: Consider using `async with` here
with Manager():
    ...
```

closes https://github.com/astral-sh/ty/issues/508

## Test Plan

New MD snapshot tests

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-05-26 21:34:47 +02:00
David Peter 4e68dd96a6
[ty] Infer types for ty_extensions.Intersection[A, B] tuple expressions (#18321)
## Summary

fixes astral-sh/ty#366

## Test Plan

* Added panic corpus regression tests
* I also wrote a hover regression test (see below), but decided not to
include it. The corpus tests are much more "effective" at finding these
types of errors, since they exhaustively check all expressions for
types.

<details>

```rs
#[test]
fn hover_regression_test_366() {
    let test = cursor_test(
        r#"
    from ty_extensions import Intersection

    class A: ...
    class B: ...

    def _(x: Intersection[A,<CURSOR> B]):
        pass
    "#,
    );

    assert_snapshot!(test.hover(), @r"
    A & B
    ---------------------------------------------
    ```text
    A & B
    ```
    ---------------------------------------------
    info[hover]: Hovered content is
     --> main.py:7:31
      |
    5 |         class B: ...
    6 |
    7 |         def _(x: Intersection[A, B]):
      |                               ^^-^
      |                               | |
      |                               | Cursor offset
      |                               source
    8 |             pass
      |
    ");
}
```

</details>
2025-05-26 17:08:52 +02:00
David Peter e1b662bf5d
[ty] Always pass `NO_INSTANCE_FALLBACK` in `try_call_dunder_with_policy` (#18315)
## Summary

The previous `try_call_dunder_with_policy` API was a bit of a footgun
since you needed to pass `NO_INSTANCE_FALLBACK` in *addition* to other
policies that you wanted for the member lookup. Implicit calls to dunder
methods never access instance members though, so we can do this
implicitly in `try_call_dunder_with_policy`.

No functional changes.
2025-05-26 13:20:27 +02:00
Felix Scherz f885cb8a2f
[ty] use `__getattribute__` to lookup unknown members on a type (#18280)
## Summary

`Type::member_lookup_with_policy` now falls back to calling
`__getattribute__` when a member cannot be found as a second fallback
after `__getattr__`.


closes https://github.com/astral-sh/ty/issues/441

## Test Plan

Added markdown tests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: David Peter <mail@david-peter.de>
2025-05-26 12:59:45 +02:00
David Peter 4ef2c223c9
[ty] Respect `MRO_NO_OBJECT_FALLBACK` policy when looking up symbols on `type` instances (#18312)
## Summary

This should address a problem that came up while working on
https://github.com/astral-sh/ruff/pull/18280. When looking up an
attribute (typically a dunder method) with the `MRO_NO_OBJECT_FALLBACK`
policy, the attribute is first looked up on the meta type. If the meta
type happens to be `type`, we go through the following branch in
`find_name_in_mro_with_policy`:


97ff015c88/crates/ty_python_semantic/src/types.rs (L2565-L2573)

The problem is that we now look up the attribute on `object` *directly*
(instead of just having `object` in the MRO). In this case,
`MRO_NO_OBJECT_FALLBACK` has no effect in `class_member_from_mro`:


c3feb8ce27/crates/ty_python_semantic/src/types/class.rs (L1081-L1082)

So instead, we need to explicitly respect the `MRO_NO_OBJECT_FALLBACK`
policy here by returning `Symbol::Unbound`.

## Test Plan

Added new Markdown tests that explain the ecosystem changes that we
observe.
2025-05-26 12:03:29 +02:00
David Peter 7eca6f96e3
[ty] Fix attribute writes to unions/intersections including modules (#18313)
## Summary

Fix a bug that involved writes to attributes on union/intersection types
that included modules as elements.

This is a prerequisite to avoid some ecosystem false positives in
https://github.com/astral-sh/ruff/pull/18312

## Test Plan

Added regression test
2025-05-26 11:41:03 +02:00
Alex Waygood e293411679
[ty] `get_protocol_members` returns a frozenset, not a tuple (#18284) 2025-05-23 23:20:34 +00:00
lipefree 53d19f8368
[ty] Resolving Python path using `CONDA_PREFIX` variable to support Conda and Pixi (#18267) 2025-05-23 20:00:42 +02:00
InSync a1399656c9
[ty] Fix binary intersection comparison inference logic (#18266)
## Summary

Resolves https://github.com/astral-sh/ty/issues/485.

`infer_binary_intersection_type_comparison()` now checks for all
positive members before concluding that an operation is unsupported for
a given intersection type.

## Test Plan

Markdown tests.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-05-23 12:55:17 +02:00
David Peter 93ac0934dd
[ty] Type compendium (#18263)
## Summary

This is something I wrote a few months ago, and continued to update from
time to time. It was mostly written for my own education. I found a few
bugs while writing it at the time (there are still one or two TODOs in
the test assertions that are probably bugs). Our other tests are fairly
comprehensive, but they are usually structured around a certain
functionality or operation (subtyping, assignability, narrowing). The
idea here was to focus on individual *types and their properties*.

closes #197 (added `JustFloat` and `JustComplex` to `ty_extensions`).
2025-05-23 11:41:31 +02:00
Alex Waygood d02c9ada5d
[ty] Do not carry the generic context of `Protocol` or `Generic` in the `ClassBase` enum (#17989)
## Summary

It doesn't seem to be necessary for our generics implementation to carry
the `GenericContext` in the `ClassBase` variants. Removing it simplifies
the code, fixes many TODOs about `Generic` or `Protocol` appearing
multiple times in MROs when each should only appear at most once, and
allows us to more accurately detect runtime errors that occur due to
`Generic` or `Protocol` appearing multiple times in a class's bases.

In order to remove the `GenericContext` from the `ClassBase` variant, it
turns out to be necessary to emulate
`typing._GenericAlias.__mro_entries__`, or we end up with a large number
of false-positive `inconsistent-mro` errors. This PR therefore also does
that.

Lastly, this PR fixes the inferred MROs of PEP-695 generic classes,
which implicitly inherit from `Generic` even if they have no explicit
bases.

## Test Plan

mdtests
2025-05-22 21:37:03 -04:00
Carl Meyer 0b181bc2ad
Fix instance vs callable subtyping/assignability (#18260)
## Summary

Fix some issues with subtying/assignability for instances vs callables.
We need to look up dunders on the class, not the instance, and we should
limit our logic here to delegating to the type of `__call__`, so it
doesn't get out of sync with the calls we allow.

Also, we were just entirely missing assignability handling for
`__call__` implemented as anything other than a normal bound method
(though we had it for subtyping.)

A first step towards considering what else we want to change in
https://github.com/astral-sh/ty/issues/491

## Test Plan

mdtests

---------

Co-authored-by: med <medioqrity@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-05-22 19:47:05 +00:00