## Summary
Fixes https://github.com/astral-sh/ty/issues/2363
Fixes https://github.com/astral-sh/ty/issues/2013
And several other bugs with the same root cause. And makes any similar
bugs impossible by construction.
Previously we distinguished "no annotation" (Rust `None`) from
"explicitly annotated with something of type `Unknown`" (which is not an
error, and results in the annotation being of Rust type
`Some(Type::DynamicType(Unknown))`), even though semantically these
should be treated the same.
This was a bit of a bug magnet, because it was easy to forget to make
this `None` -> `Unknown` translation everywhere we needed to. And in
fact we did fail to do it in the case of materializing a callable,
leading to a top-materialized callable still having (rust) `None` return
type, which should have instead materialized to `object`.
This also fixes several other bugs related to not handling un-annotated
return types correctly:
1. We previously considered the return type of an unannotated `async
def` to be `Unknown`, where it should be `CoroutineType[Any, Any,
Unknown]`.
2. We previously failed to infer a ParamSpec if the return type of the
callable we are inferring against was not annotated.
3. We previously wrongly returned `Unknown` from `some_dict.get("key",
None)` if the value type of `some_dict` included a callable type with
un-annotated return type.
We now make signature return types and annotated parameter types
required, and we eagerly insert `Unknown` if there's no annotation. Most
of the diff is just a bunch of mechanical code changes where we
construct these types, and simplifications where we use them.
One exception is type display: when a callable type has un-annotated
parameters, we want to display them as un-annotated, but if it has a
parameter explicitly annotated with something of `Unknown` type, we want
to display that parameter as `x: Unknown` (it would be confusing if it
looked like your annotation just disappeared entirely).
Fortunately, we already have a mechanism in place for handling this: the
`inferred_annotation` flag, which suppresses display of an annotation.
Previously we used it only for `self` and `cls` parameters with an
inferred annotated type -- but we now also set it for any un-annotated
parameter, for which we infer `Unknown` type.
We also need to normalize `inferred_annotation`, since it's display-only
and shouldn't impact type equivalence. (This is technically a
previously-existing bug, it just never came up when it only affected
self types -- now it comes up because we have tests asserting that `def
f(x)` and `def g(x: Unknown)` are equivalent.)
## Test Plan
Added mdtests.
## Summary
I wondered if this might improve performance a little. It doesn't seem
to, but it's a net reduction in LOC and I think the changes make sense.
I think it's worth it anyway just in terms of simplifying the code.
## Test Plan
Our existing tests all pass and the primer report is clean (aside from
our usual flakes).
## Summary
fixes: https://github.com/astral-sh/ty/issues/2027
This PR fixes a bug where the type mapping for a `ParamSpec` was not
being applied in an overloaded function.
This PR also fixes https://github.com/astral-sh/ty/issues/2081 and
reveals new diagnostics which doesn't look related to the bug:
```py
from prefect import flow, task
@task
def task_get() -> int:
"""Task get integer."""
return 42
@task
def task_add(x: int, y: int) -> int:
"""Task add two integers."""
print(f"Adding {x} and {y}")
return x + y
@flow
def my_flow():
"""My flow."""
x = 23
future_y = task_get.submit()
# error: [no-matching-overload]
task_add(future_y, future_y)
# error: [no-matching-overload]
task_add(x, future_y)
```
The reason is that the type of `future_y` is `PrefectFuture[int]` while
the type of `task_add` is `Task[(x: int, y: int), int]` which means that
the assignment between `int` and `PrefectFuture[int]` fails which
results in no overload matching. Pyright also raises the invalid
argument type error on all three usages of `future_y` in those two
calls.
## Test Plan
Add regression mdtest from the linked issue.
@dhruvmanila encountered this in #22416 — there are two different
`TypeMapping` variants for apply a specialization to a type. One
operates on a full `Specialization` instance, the other on a partially
constructed one. If we move this enum-ness "down a level" it reduces
some copy/paste in places where we are operating on a `TypeMapping`.
## Summary
Fixes https://github.com/astral-sh/ty/issues/2292
When solving a bounded typevar, we preferred the upper bound over the
actual type seen in the call. This change fixes that.
## Test Plan
Added mdtest, existing tests pass.
## Summary
This raises a `ValueError` at runtime:
```python
from functools import total_ordering
@total_ordering
class NoOrdering:
def __eq__(self, other: object) -> bool:
return True
```
Specifically:
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py", line 193, in total_ordering
raise ValueError('must define at least one ordering operation: < > <= >=')
ValueError: must define at least one ordering operation: < > <= >=
```
See: https://github.com/astral-sh/ty/issues/1202.
Snapshot tests recently started reporting this warning:
> Snapshot test passes but the existing value is in a legacy format.
> Please run cargo insta test --force-update-snapshots to update to a
> newer format.
This PR is the result of that forced update.
One file (crates/ruff_db/src/diagnostic/render/full.rs) seems to get
corrupted, because it contains strings with unprintable characters that
trigger some bug in cargo-insta. I've manually reverted that file, and
also manually reverted the `input_file:` lines, which we like.
## Summary
Decorators are now called with the class as an argument, and the return
type becomes the class's type. This mirrors how function decorators
already work.
Closes https://github.com/astral-sh/ty/issues/2313.
## Summary
As-is, the following rejects `return self.value` in `def other` in the
subclass
([link](https://play.ty.dev/f55b47b2-313e-45d1-ba45-fde410bed32e))
because `self.value` is resolving to `Unknown | int | float | property`:
```python
class Base:
_value: float = 0.0
@property
def value(self) -> float:
return self._value
@value.setter
def value(self, v: float) -> None:
self._value = v
@property
def other(self) -> float:
return self.value
@other.setter
def other(self, v: float) -> None:
self.value = v
class Derived(Base):
@property
def other(self) -> float:
return self.value
@other.setter
def other(self, v: float) -> None:
reveal_type(self.value) # revealed: int | float
self.value = v
```
I believe the root cause is that we're not excluding properties when
searching for class methods, so we're treating the `other` setter as a
classmethod. I don't fully understand how that ends up materializing as
`| property` on the union though.
## Summary
If we match on an `TestEnum | None`, then when adding a case like
`~Literal[TestEnum.FOO]` (i.e., after `if value == TestEnum.FOO:
return`), we'd distribute `Literal[TestEnum.BAR]` on the entire builder,
creating `None & Literal[TestEnum.BAR]` which simplified to `Never`.
Instead, we should only expand to the remaining members for pieces of
the intersection that contain the enum.
Now, `(TestEnum | None) & ~Literal[TestEnum.FOO] &
~Literal[TestEnum.BAR]` correctly simplifies to `None` instead of
`Never`.
Closes https://github.com/astral-sh/ty/issues/2260.
## Summary
`apply_type_mapping` always expands type aliases and operates on the
resulting types, which can lead to cluttered results due to excessive
type alias expansion in places where it is not actually needed.
Specifically, type aliases are expanded when displaying method
signatures, because we use `TypeMapping::BindSelf` to get the method
signature.
```python
type Scalar = int | float
type Array1d = list[Scalar] | tuple[Scalar]
def f(x: Scalar | Array1d) -> None: pass
reveal_type(f) # revealed: def f(x: Scalar | Array1d) -> None
class Foo:
def f(self, x: Scalar | Array1d) -> None: pass
# should be `bound method Foo.f(x: Scalar | Array1d) -> None`
reveal_type(Foo().f) # revealed: bound method Foo.f(x: int | float | list[int | float] | tuple[int | float]) -> None
```
In this PR, when type mapping is performed on a type alias, the
expansion result without type mapping is compared with the expansion
result after type mapping, and if the two are equivalent, the expansion
is deemed redundant and canceled.
## Test Plan
mdtest updated
## Summary
Resolve(s) astral-sh/ty#117, astral-sh/ty#1569
Implement `typing.TypeGuard`. Due to the fact that it [overrides
anything previously known about the checked
value](https://typing.python.org/en/latest/spec/narrowing.html#typeguard)---
> When a conditional statement includes a call to a user-defined type
guard function, and that function returns true, the expression passed as
the first positional argument to the type guard function should be
assumed by a static type checker to take on the type specified in the
TypeGuard return type, unless and until it is further narrowed within
the conditional code block.
---we have to substantially rework the constraints system. In
particular, we make constraints represented as a disjunctive normal form
(DNF) where each term includes a regular constraint, and one or more
disjuncts with a typeguard constraint. Some test cases (including some
with more complex boolean logic) are added to `type_guards.md`.
## Test Plan
- update existing tests
- add new tests for more complex boolean logic with `TypeGuard`
- add new tests for `TypeGuard` variance
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
I only noticed this in the ecosystem report of
https://github.com/astral-sh/ruff/pull/22213 after merging it. The
change to displaying `Top[]` wrapper around the entire signature instead
of just the parameters had the side effect of not showing it at all when
displaying a top ParamSpec specialization. This PR fixes that.
Marking internal since this is a fixup of a not-released PR.
## Test Plan
Added mdtest that fails without this PR.
## Summary
A couple things I noticed when taking another look at the callable type
materializations.
1) Previously we wrongly ignored the return type when
bottom-materializing a callable with gradual signature, and always
changed it to `Never`.
2) We weren't correctly handling overloads that included a gradual
signature. Rather than separately materializing each overload, we would
just mark the entire callable as "top" or replace the entire callable
with the bottom signature.
Really, "top parameters" is something that belongs on the `Parameters`,
not on the entire `CallableType`. Conveniently, we already have
`ParametersKind` where we can track this, right next to where we already
track `ParametersKind::Gradual`. This saves a bit of memory, fixes the
two bugs above, and simplifies the implementation considerably (net
removal of 100+ LOC, a bunch of places that shouldn't need to care about
topness of a callable no longer need to.)
One user-visible change from this is that I now display the "top
callable" as `(Top[...]) -> object` instead of `Top[(...) -> object]`. I
think this is a (minor) improvement, because it wraps exactly the part
in `Top` that needs to be, rather than misleadingly wrapping the entire
callable type, including the return type (which has already been
separately materialized). I think the prior display would be
particularly confusing if the return type also has its own `Top` in it:
previously we could have e.g. `Top[(...) -> Top[list[Unknown]]]`, which
I think is less clear than the new `(Top[...]) -> Top[list[Unknown]]`.
## Test Plan
Added mdtests that failed before this PR and pass after it.
### Ecosystem
The changed diagnostics are all either the change to `Top` display, or
else known non-deterministic output. The added diagnostics are all true
positives:
The added diagnostic at
aa35ca1965/torchvision/transforms/v2/_utils.py (L149)
is a true positive that wasn't caught by the previous version. `str` is
not assignable to `Callable[[Any], Any]` (strings are not callable), nor
is the top callable (top callable includes callables that do not take a
single required positional argument.)
The added diagnostic at
081535ad9b/starlette/routing.py (L67)
is also a (pedantic) true positive. It's the same case as #1567 -- the
code assumes that it is impossible for a subclass of `Response` to
implement `__await__` (yielding something other than a `Response`).
The pytest added diagnostics are also both similar true positives: they
make the assumption that an object cannot simultaneously be a `Sequence`
and callable, or an `Iterable` and callable.
