[ty] Use `typing.Self` for the first parameter of instance methods (#20517)

## Summary

Modify the (external) signature of instance methods such that the first
parameter uses `Self` unless it is explicitly annotated. This allows us
to correctly type-check more code, and allows us to infer correct return
types for many functions that return `Self`. For example:

```py
from pathlib import Path
from datetime import datetime, timedelta

reveal_type(Path(".config") / ".ty")  # now Path, previously Unknown

def _(dt: datetime, delta: timedelta):
    reveal_type(dt - delta)  # now datetime, previously Unknown
```

part of https://github.com/astral-sh/ty/issues/159

## Performance

I ran benchmarks locally on `attrs`, `freqtrade` and `colour`, the
projects with the largest regressions on CodSpeed. I see much smaller
effects locally, but can definitely reproduce the regression on `attrs`.
From looking at the profiling results (on Codspeed), it seems that we
simply do more type inference work, which seems plausible, given that we
now understand much more return types (of many stdlib functions). In
particular, whenever a function uses an implicit `self` and returns
`Self` (without mentioning `Self` anywhere else in its signature), we
will now infer the correct type, whereas we would previously return
`Unknown`. This also means that we need to invoke the generics solver in
more cases. Comparing half a million lines of log output on attrs, I can
see that we do 5% more "work" (number of lines in the log), and have a
lot more `apply_specialization` events (7108 vs 4304). On freqtrade, I
see similar numbers for `apply_specialization` (11360 vs 5138 calls).
Given these results, I'm not sure if it's generally worth doing more
performance work, especially since none of the code modifications
themselves seem to be likely candidates for regressions.

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/attrs` | 92.6 ± 3.6 | 85.9 |
102.6 | 1.00 |
| `./ty_self check /home/shark/ecosystem/attrs` | 101.7 ± 3.5 | 96.9 |
113.8 | 1.10 ± 0.06 |

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/freqtrade` | 599.0 ± 20.2 |
568.2 | 627.5 | 1.00 |
| `./ty_self check /home/shark/ecosystem/freqtrade` | 607.9 ± 11.5 |
594.9 | 626.4 | 1.01 ± 0.04 |

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/colour` | 423.9 ± 17.9 | 394.6
| 447.4 | 1.00 |
| `./ty_self check /home/shark/ecosystem/colour` | 426.9 ± 24.9 | 373.8
| 456.6 | 1.01 ± 0.07 |

## Test Plan

New Markdown tests

## Ecosystem report

* apprise: ~300 new diagnostics related to problematic stubs in apprise
😩
* attrs: a new true positive, since [this
function](4e2c89c823/tests/test_make.py (L2135))
is missing a `@staticmethod`?
* Some legitimate true positives
* sympy: lots of new `invalid-operator` false positives in [matrix
multiplication](cf9f4b6805/sympy/matrices/matrixbase.py (L3267-L3269))
due to our limited understanding of [generic `Callable[[Callable[[T1,
T2], T3]], Callable[[T1, T2], T3]]` "identity"
types](cf9f4b6805/sympy/core/decorators.py (L83-L84))
of decorators. This is not related to type-of-self.

## Typing conformance results

The changes are all correct, except for
```diff
+generics_self_usage.py:50:5: error[invalid-assignment] Object of type `def foo(self) -> int` is not assignable to `(typing.Self, /) -> int`
```
which is related to an assignability problem involving type variables on
both sides:
```py
class CallableAttribute:
    def foo(self) -> int:
        return 0

    bar: Callable[[Self], int] = foo  # <- we currently error on this assignment
```

---------

Co-authored-by: Shaygan Hooshyari <sh.hooshyari@gmail.com>
This commit is contained in:
David Peter 2025-09-29 21:08:08 +02:00 committed by GitHub
parent 1d3e4a9153
commit 0092794302
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
23 changed files with 555 additions and 131 deletions

View File

@ -117,7 +117,7 @@ static COLOUR_SCIENCE: std::sync::LazyLock<Benchmark<'static>> = std::sync::Lazy
max_dep_date: "2025-06-17",
python_version: PythonVersion::PY310,
},
477,
500,
)
});

View File

@ -3030,6 +3030,12 @@ impl Parameters {
.find(|arg| arg.parameter.name.as_str() == name)
}
/// Returns the index of the parameter with the given name
pub fn index(&self, name: &str) -> Option<usize> {
self.iter_non_variadic_params()
.position(|arg| arg.parameter.name.as_str() == name)
}
/// Returns an iterator over all parameters included in this [`Parameters`] node.
pub fn iter(&self) -> ParametersIterator<'_> {
ParametersIterator::new(self)

View File

@ -33,11 +33,6 @@ class Shape:
reveal_type(x) # revealed: Self@nested_func_without_enclosing_binding
inner(self)
def implicit_self(self) -> Self:
# TODO: first argument in a method should be considered as "typing.Self"
reveal_type(self) # revealed: Unknown
return self
reveal_type(Shape().nested_type()) # revealed: list[Shape]
reveal_type(Shape().nested_func()) # revealed: Shape
@ -53,6 +48,150 @@ class Outer:
return self
```
## Type of (unannotated) `self` parameters
In instance methods, the first parameter (regardless of its name) is assumed to have the type
`typing.Self`, unless it has an explicit annotation. This does not apply to `@classmethod` and
`@staticmethod`s.
```toml
[environment]
python-version = "3.11"
```
```py
from typing import Self
class A:
def implicit_self(self) -> Self:
# TODO: This should be Self@implicit_self
reveal_type(self) # revealed: Unknown
return self
def a_method(self) -> int:
def first_arg_is_not_self(a: int) -> int:
reveal_type(a) # revealed: int
return a
return first_arg_is_not_self(1)
@classmethod
def a_classmethod(cls) -> Self:
# TODO: This should be type[Self@bar]
reveal_type(cls) # revealed: Unknown
return cls()
@staticmethod
def a_staticmethod(x: int): ...
a = A()
reveal_type(a.implicit_self()) # revealed: A
reveal_type(a.implicit_self) # revealed: bound method A.implicit_self() -> A
```
Calling an instance method explicitly verifies the first argument:
```py
A.implicit_self(a)
# error: [invalid-argument-type] "Argument to function `implicit_self` is incorrect: Argument type `Literal[1]` does not satisfy upper bound `A` of type variable `Self`"
A.implicit_self(1)
```
Passing `self` implicitly also verifies the type:
```py
from typing import Never
class Strange:
def can_not_be_called(self: Never) -> None: ...
# error: [invalid-argument-type] "Argument to bound method `can_not_be_called` is incorrect: Expected `Never`, found `Strange`"
Strange().can_not_be_called()
```
If the method is a class or static method then first argument is not inferred as `Self`:
```py
A.a_classmethod()
A.a_classmethod(a) # error: [too-many-positional-arguments]
A.a_staticmethod(1)
a.a_staticmethod(1)
A.a_staticmethod(a) # error: [invalid-argument-type]
```
The first parameter of instance methods always has type `Self`, if it is not explicitly annotated.
The name `self` is not special in any way.
```py
class B:
def name_does_not_matter(this) -> Self:
# TODO: Should reveal Self@name_does_not_matter
reveal_type(this) # revealed: Unknown
return this
def positional_only(self, /, x: int) -> Self:
# TODO: Should reveal Self@positional_only
reveal_type(self) # revealed: Unknown
return self
def keyword_only(self, *, x: int) -> Self:
# TODO: Should reveal Self@keyword_only
reveal_type(self) # revealed: Unknown
return self
@property
def a_property(self) -> Self:
# TODO: Should reveal Self@a_property
reveal_type(self) # revealed: Unknown
return self
reveal_type(B().name_does_not_matter()) # revealed: B
reveal_type(B().positional_only(1)) # revealed: B
reveal_type(B().keyword_only(x=1)) # revealed: B
# TODO: this should be B
reveal_type(B().a_property) # revealed: Unknown
```
This also works for generic classes:
```py
from typing import Self, Generic, TypeVar
T = TypeVar("T")
class G(Generic[T]):
def id(self) -> Self:
# TODO: Should reveal Self@id
reveal_type(self) # revealed: Unknown
return self
reveal_type(G[int]().id()) # revealed: G[int]
reveal_type(G[str]().id()) # revealed: G[str]
```
Free functions and nested functions do not use implicit `Self`:
```py
def not_a_method(self):
reveal_type(self) # revealed: Unknown
# error: [invalid-type-form]
def does_not_return_self(self) -> Self:
return self
class C:
def outer(self) -> None:
def inner(self):
reveal_type(self) # revealed: Unknown
reveal_type(not_a_method) # revealed: def not_a_method(self) -> Unknown
```
## typing_extensions
```toml
@ -208,6 +347,47 @@ class MyMetaclass(type):
return super().__new__(cls)
```
## Explicit annotations override implicit `Self`
If the first parameter is explicitly annotated, that annotation takes precedence over the implicit
`Self` type.
```toml
[environment]
python-version = "3.12"
```
```py
from __future__ import annotations
from typing import final
@final
class Disjoint: ...
class Explicit:
# TODO: We could emit a warning if the annotated type of `self` is disjoint from `Explicit`
def bad(self: Disjoint) -> None:
reveal_type(self) # revealed: Disjoint
def forward(self: Explicit) -> None:
reveal_type(self) # revealed: Explicit
# error: [invalid-argument-type] "Argument to bound method `bad` is incorrect: Expected `Disjoint`, found `Explicit`"
Explicit().bad()
Explicit().forward()
class ExplicitGeneric[T]:
def special(self: ExplicitGeneric[int]) -> None:
reveal_type(self) # revealed: ExplicitGeneric[int]
ExplicitGeneric[int]().special()
# TODO: this should be an `invalid-argument-type` error
ExplicitGeneric[str]().special()
```
## Binding a method fixes `Self`
When a method is bound, any instances of `Self` in its signature are "fixed", since we now know the

View File

@ -69,7 +69,9 @@ reveal_type(bound_method(1)) # revealed: str
When we call the function object itself, we need to pass the `instance` explicitly:
```py
C.f(1) # error: [missing-argument]
# error: [invalid-argument-type] "Argument to function `f` is incorrect: Expected `C`, found `Literal[1]`"
# error: [missing-argument]
C.f(1)
reveal_type(C.f(C(), 1)) # revealed: str
```

View File

@ -431,6 +431,8 @@ def _(flag: bool):
reveal_type(C7.union_of_class_data_descriptor_and_attribute) # revealed: Literal["data", 2]
C7.union_of_metaclass_attributes = 2 if flag else 1
# TODO: https://github.com/astral-sh/ty/issues/1163
# error: [invalid-assignment]
C7.union_of_metaclass_data_descriptor_and_attribute = 2 if flag else 100
C7.union_of_class_attributes = 2 if flag else 1
C7.union_of_class_data_descriptor_and_attribute = 2 if flag else DataDescriptor()

View File

@ -562,17 +562,17 @@ class C(Generic[T]):
return u
reveal_type(generic_context(C)) # revealed: tuple[T@C]
reveal_type(generic_context(C.method)) # revealed: None
reveal_type(generic_context(C.generic_method)) # revealed: tuple[U@generic_method]
reveal_type(generic_context(C.method)) # revealed: tuple[Self@method]
reveal_type(generic_context(C.generic_method)) # revealed: tuple[Self@generic_method, U@generic_method]
reveal_type(generic_context(C[int])) # revealed: None
reveal_type(generic_context(C[int].method)) # revealed: None
reveal_type(generic_context(C[int].generic_method)) # revealed: tuple[U@generic_method]
reveal_type(generic_context(C[int].method)) # revealed: tuple[Self@method]
reveal_type(generic_context(C[int].generic_method)) # revealed: tuple[Self@generic_method, U@generic_method]
c: C[int] = C[int]()
reveal_type(c.generic_method(1, "string")) # revealed: Literal["string"]
reveal_type(generic_context(c)) # revealed: None
reveal_type(generic_context(c.method)) # revealed: None
reveal_type(generic_context(c.generic_method)) # revealed: tuple[U@generic_method]
reveal_type(generic_context(c.method)) # revealed: tuple[Self@method]
reveal_type(generic_context(c.generic_method)) # revealed: tuple[Self@generic_method, U@generic_method]
```
## Specializations propagate

View File

@ -504,17 +504,17 @@ class C[T]:
def cannot_shadow_class_typevar[T](self, t: T): ...
reveal_type(generic_context(C)) # revealed: tuple[T@C]
reveal_type(generic_context(C.method)) # revealed: None
reveal_type(generic_context(C.generic_method)) # revealed: tuple[U@generic_method]
reveal_type(generic_context(C.method)) # revealed: tuple[Self@method]
reveal_type(generic_context(C.generic_method)) # revealed: tuple[Self@generic_method, U@generic_method]
reveal_type(generic_context(C[int])) # revealed: None
reveal_type(generic_context(C[int].method)) # revealed: None
reveal_type(generic_context(C[int].generic_method)) # revealed: tuple[U@generic_method]
reveal_type(generic_context(C[int].method)) # revealed: tuple[Self@method]
reveal_type(generic_context(C[int].generic_method)) # revealed: tuple[Self@generic_method, U@generic_method]
c: C[int] = C[int]()
reveal_type(c.generic_method(1, "string")) # revealed: Literal["string"]
reveal_type(generic_context(c)) # revealed: None
reveal_type(generic_context(c.method)) # revealed: None
reveal_type(generic_context(c.generic_method)) # revealed: tuple[U@generic_method]
reveal_type(generic_context(c.method)) # revealed: tuple[Self@method]
reveal_type(generic_context(c.generic_method)) # revealed: tuple[Self@generic_method, U@generic_method]
```
## Specializations propagate

View File

@ -534,6 +534,5 @@ class C:
def _(x: int):
reveal_type(C().explicit_self(x)) # revealed: tuple[C, int]
# TODO: this should be `tuple[C, int]` as well, once we support implicit `self`
reveal_type(C().implicit_self(x)) # revealed: tuple[Unknown, int]
reveal_type(C().implicit_self(x)) # revealed: tuple[C, int]
```

View File

@ -117,6 +117,7 @@ reveal_type(bound_method.__func__) # revealed: def f(self, x: int) -> str
reveal_type(C[int]().f(1)) # revealed: str
reveal_type(bound_method(1)) # revealed: str
# error: [invalid-argument-type] "Argument to function `f` is incorrect: Argument type `Literal[1]` does not satisfy upper bound `C[T@C]` of type variable `Self`"
C[int].f(1) # error: [missing-argument]
reveal_type(C[int].f(C[int](), 1)) # revealed: str
@ -154,7 +155,7 @@ from ty_extensions import generic_context
legacy.m("string", None) # error: [invalid-argument-type]
reveal_type(legacy.m) # revealed: bound method Legacy[int].m[S](x: int, y: S@m) -> S@m
reveal_type(generic_context(Legacy)) # revealed: tuple[T@Legacy]
reveal_type(generic_context(legacy.m)) # revealed: tuple[S@m]
reveal_type(generic_context(legacy.m)) # revealed: tuple[Self@m, S@m]
```
With PEP 695 syntax, it is clearer that the method uses a separate typevar:

View File

@ -278,8 +278,7 @@ reveal_type(Person._make(("Alice", 42))) # revealed: Unknown
person = Person("Alice", 42)
reveal_type(person._asdict()) # revealed: dict[str, Any]
# TODO: should be `Person` once we support implicit type of `self`
reveal_type(person._replace(name="Bob")) # revealed: Unknown
reveal_type(person._replace(name="Bob")) # revealed: Person
```
When accessing them on child classes of generic `NamedTuple`s, the return type is specialized
@ -296,8 +295,7 @@ class Box(NamedTuple, Generic[T]):
class IntBox(Box[int]):
pass
# TODO: should be `IntBox` once we support the implicit type of `self`
reveal_type(IntBox(1)._replace(content=42)) # revealed: Unknown
reveal_type(IntBox(1)._replace(content=42)) # revealed: IntBox
```
## `collections.namedtuple`

View File

@ -324,8 +324,7 @@ a covariant generic, this is equivalent to using the upper bound of the type par
from typing import Self
class Covariant[T]:
# TODO: remove the explicit `Self` annotation, once we support the implicit type of `self`
def get(self: Self) -> T:
def get(self) -> T:
raise NotImplementedError
def _(x: object):
@ -338,8 +337,7 @@ Similarly, contravariant type parameters use their lower bound of `Never`:
```py
class Contravariant[T]:
# TODO: remove the explicit `Self` annotation, once we support the implicit type of `self`
def push(self: Self, x: T) -> None: ...
def push(self, x: T) -> None: ...
def _(x: object):
if isinstance(x, Contravariant):
@ -354,10 +352,8 @@ the type system, so we represent it with the internal `Top[]` special form.
```py
class Invariant[T]:
# TODO: remove the explicit `Self` annotation, once we support the implicit type of `self`
def push(self: Self, x: T) -> None: ...
# TODO: remove the explicit `Self` annotation, once we support the implicit type of `self`
def get(self: Self) -> T:
def push(self, x: T) -> None: ...
def get(self) -> T:
raise NotImplementedError
def _(x: object):

View File

@ -325,7 +325,7 @@ type A = list[Union["A", str]]
def f(x: A):
reveal_type(x) # revealed: list[A | str]
for item in x:
reveal_type(item) # revealed: list[A | str] | str
reveal_type(item) # revealed: list[Any | str] | str
```
#### With new-style union
@ -336,7 +336,7 @@ type A = list["A" | str]
def f(x: A):
reveal_type(x) # revealed: list[A | str]
for item in x:
reveal_type(item) # revealed: list[A | str] | str
reveal_type(item) # revealed: list[Any | str] | str
```
#### With Optional
@ -349,7 +349,7 @@ type A = list[Optional[Union["A", str]]]
def f(x: A):
reveal_type(x) # revealed: list[A | str | None]
for item in x:
reveal_type(item) # revealed: list[A | str | None] | str | None
reveal_type(item) # revealed: list[Any | str | None] | str | None
```
### Tuple comparison

View File

@ -1977,12 +1977,12 @@ from typing_extensions import TypeVar, Self, Protocol
from ty_extensions import is_equivalent_to, static_assert, is_assignable_to, is_subtype_of
class NewStyleClassScoped[T](Protocol):
def method(self: Self, input: T) -> None: ...
def method(self, input: T) -> None: ...
S = TypeVar("S")
class LegacyClassScoped(Protocol[S]):
def method(self: Self, input: S) -> None: ...
def method(self, input: S) -> None: ...
# TODO: these should pass
static_assert(is_equivalent_to(NewStyleClassScoped, LegacyClassScoped)) # error: [static-assert-error]

View File

@ -1948,8 +1948,6 @@ static_assert(is_subtype_of(TypeOf[A.g], Callable[[int], int]))
static_assert(not is_subtype_of(TypeOf[a.f], Callable[[float], int]))
static_assert(not is_subtype_of(TypeOf[A.g], Callable[[], int]))
# TODO: This assertion should be true
# error: [static-assert-error] "Static assertion error: argument of type `ty_extensions.ConstraintSet[never]` is statically known to be falsy"
static_assert(is_subtype_of(TypeOf[A.f], Callable[[A, int], int]))
```

View File

@ -657,16 +657,14 @@ alice: Employee = {"name": "Alice", "employee_id": 1}
eve: Employee = {"name": "Eve"}
def combine(p: Person, e: Employee):
# TODO: Should be `Person` once we support the implicit type of self
reveal_type(p.copy()) # revealed: Unknown
# TODO: Should be `Employee` once we support the implicit type of self
reveal_type(e.copy()) # revealed: Unknown
reveal_type(p.copy()) # revealed: Person
reveal_type(e.copy()) # revealed: Employee
reveal_type(p | p) # revealed: Person
reveal_type(e | e) # revealed: Employee
# TODO: Should be `Person` once we support the implicit type of self and subtyping for TypedDicts
reveal_type(p | e) # revealed: Employee
# TODO: Should be `Person` once we support subtyping for TypedDicts
reveal_type(p | e) # revealed: Person | Employee
```
When inheriting from a `TypedDict` with a different `total` setting, inherited fields maintain their

View File

@ -254,8 +254,7 @@ async def long_running_task():
async def main():
async with asyncio.TaskGroup() as tg:
# TODO: should be `TaskGroup`
reveal_type(tg) # revealed: Unknown
reveal_type(tg) # revealed: TaskGroup
tg.create_task(long_running_task())
```

View File

@ -52,7 +52,8 @@ use crate::types::function::{
DataclassTransformerParams, FunctionSpans, FunctionType, KnownFunction,
};
use crate::types::generics::{
GenericContext, PartialSpecialization, Specialization, bind_typevar, walk_generic_context,
GenericContext, PartialSpecialization, Specialization, bind_typevar, typing_self,
walk_generic_context,
};
pub use crate::types::ide_support::{
CallSignatureDetails, Member, MemberWithDefinition, all_members, call_signature_details,
@ -5355,12 +5356,10 @@ impl<'db> Type<'db> {
Some(generic_context) => (
Some(class),
Some(generic_context),
Type::from(class.apply_specialization(db, |_| {
// It is important that identity_specialization specializes the class with
// _inferable_ typevars, so that our specialization inference logic will
// try to find a specialization for them.
generic_context.identity_specialization(db)
})),
// It is important that identity_specialization specializes the class with
// _inferable_ typevars, so that our specialization inference logic will
// try to find a specialization for them.
Type::from(class.identity_specialization(db, &Type::TypeVar)),
),
_ => (None, None, self),
},
@ -5717,39 +5716,13 @@ impl<'db> Type<'db> {
});
};
let upper_bound = Type::instance(
Ok(typing_self(
db,
class.apply_specialization(db, |generic_context| {
let types = generic_context
.variables(db)
.iter()
.map(|typevar| Type::NonInferableTypeVar(*typevar));
generic_context.specialize(db, types.collect())
}),
);
let class_definition = class.definition(db);
let typevar = TypeVarInstance::new(
db,
ast::name::Name::new_static("Self"),
Some(class_definition),
Some(TypeVarBoundOrConstraints::UpperBound(upper_bound).into()),
// According to the [spec], we can consider `Self`
// equivalent to an invariant type variable
// [spec]: https://typing.python.org/en/latest/spec/generics.html#self
Some(TypeVarVariance::Invariant),
None,
TypeVarKind::TypingSelf,
);
Ok(bind_typevar(
db,
index,
scope_id.file_scope_id(db),
scope_id,
typevar_binding_context,
typevar,
class,
&Type::NonInferableTypeVar,
)
.map(Type::NonInferableTypeVar)
.unwrap_or(*self))
}
SpecialFormType::TypeAlias => Ok(Type::Dynamic(DynamicType::TodoTypeAlias)),

View File

@ -376,8 +376,8 @@ pub enum ClassType<'db> {
#[salsa::tracked]
impl<'db> ClassType<'db> {
pub(super) const fn is_not_generic(self) -> bool {
matches!(self, Self::NonGeneric(_))
pub(super) const fn is_generic(self) -> bool {
matches!(self, Self::Generic(_))
}
pub(super) const fn into_generic_alias(self) -> Option<GenericAlias<'db>> {
@ -1527,6 +1527,18 @@ impl<'db> ClassLiteral<'db> {
})
}
/// Returns a specialization of this class where each typevar is mapped to itself. The second
/// parameter can be `Type::TypeVar` or `Type::NonInferableTypeVar`, depending on the use case.
pub(crate) fn identity_specialization(
self,
db: &'db dyn Db,
typevar_to_type: &impl Fn(BoundTypeVarInstance<'db>) -> Type<'db>,
) -> ClassType<'db> {
self.apply_specialization(db, |generic_context| {
generic_context.identity_specialization(db, typevar_to_type)
})
}
/// Return an iterator over the inferred types of this class's *explicit* bases.
///
/// Note that any class (except for `object`) that has no explicit
@ -2625,7 +2637,14 @@ impl<'db> ClassLiteral<'db> {
.map_type(|ty|
ty.apply_type_mapping(
db,
&TypeMapping::ReplaceSelf {new_upper_bound: determine_upper_bound(db, self, specialization, ClassBase::is_typed_dict) }
&TypeMapping::ReplaceSelf {
new_upper_bound: determine_upper_bound(
db,
self,
specialization,
ClassBase::is_typed_dict
)
}
)
)
}
@ -4174,6 +4193,91 @@ impl KnownClass {
}
}
/// Return `true` if this class is a typeshed fallback class which is used to provide attributes and
/// methods for another type (e.g. `NamedTupleFallback` for actual `NamedTuple`s). These fallback
/// classes need special treatment in some places. For example, implicit usages of `Self` should not
/// be eagerly replaced with the fallback class itself. Instead, `Self` should eventually be treated
/// as referring to the destination type (e.g. the actual `NamedTuple`).
pub(crate) const fn is_fallback_class(self) -> bool {
match self {
KnownClass::Bool
| KnownClass::Object
| KnownClass::Bytes
| KnownClass::Bytearray
| KnownClass::Type
| KnownClass::Int
| KnownClass::Float
| KnownClass::Complex
| KnownClass::Str
| KnownClass::List
| KnownClass::Tuple
| KnownClass::Set
| KnownClass::FrozenSet
| KnownClass::Dict
| KnownClass::Slice
| KnownClass::Property
| KnownClass::BaseException
| KnownClass::Exception
| KnownClass::BaseExceptionGroup
| KnownClass::ExceptionGroup
| KnownClass::Staticmethod
| KnownClass::Classmethod
| KnownClass::Super
| KnownClass::Enum
| KnownClass::EnumType
| KnownClass::Auto
| KnownClass::Member
| KnownClass::Nonmember
| KnownClass::StrEnum
| KnownClass::ABCMeta
| KnownClass::GenericAlias
| KnownClass::ModuleType
| KnownClass::FunctionType
| KnownClass::MethodType
| KnownClass::MethodWrapperType
| KnownClass::WrapperDescriptorType
| KnownClass::UnionType
| KnownClass::GeneratorType
| KnownClass::AsyncGeneratorType
| KnownClass::CoroutineType
| KnownClass::NotImplementedType
| KnownClass::BuiltinFunctionType
| KnownClass::EllipsisType
| KnownClass::NoneType
| KnownClass::Awaitable
| KnownClass::Generator
| KnownClass::Deprecated
| KnownClass::StdlibAlias
| KnownClass::SpecialForm
| KnownClass::TypeVar
| KnownClass::ParamSpec
| KnownClass::ParamSpecArgs
| KnownClass::ParamSpecKwargs
| KnownClass::ProtocolMeta
| KnownClass::TypeVarTuple
| KnownClass::TypeAliasType
| KnownClass::NoDefaultType
| KnownClass::NewType
| KnownClass::SupportsIndex
| KnownClass::Iterable
| KnownClass::Iterator
| KnownClass::ChainMap
| KnownClass::Counter
| KnownClass::DefaultDict
| KnownClass::Deque
| KnownClass::OrderedDict
| KnownClass::VersionInfo
| KnownClass::Field
| KnownClass::KwOnly
| KnownClass::NamedTupleLike
| KnownClass::Template
| KnownClass::Path
| KnownClass::ConstraintSet
| KnownClass::InitVar => false,
KnownClass::NamedTupleFallback | KnownClass::TypedDictFallback => true,
}
}
pub(crate) fn name(self, db: &dyn Db) -> &'static str {
match self {
Self::Bool => "bool",

View File

@ -1209,11 +1209,13 @@ impl Display for DisplayParameter<'_> {
if let Some(name) = self.param.display_name() {
f.write_str(&name)?;
if let Some(annotated_type) = self.param.annotated_type() {
write!(
f,
": {}",
annotated_type.display_with(self.db, self.settings.clone())
)?;
if self.param.should_annotation_be_displayed() {
write!(
f,
": {}",
annotated_type.display_with(self.db, self.settings.clone())
)?;
}
}
// Default value can only be specified if `name` is given.
if let Some(default_ty) = self.param.default_type() {

View File

@ -4,9 +4,9 @@ use itertools::Itertools;
use ruff_python_ast as ast;
use rustc_hash::FxHashMap;
use crate::semantic_index::SemanticIndex;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::scope::{FileScopeId, NodeWithScopeKind};
use crate::semantic_index::scope::{FileScopeId, NodeWithScopeKind, ScopeId};
use crate::semantic_index::{SemanticIndex, semantic_index};
use crate::types::class::ClassType;
use crate::types::class_base::ClassBase;
use crate::types::infer::infer_definition_types;
@ -14,10 +14,10 @@ use crate::types::instance::{Protocol, ProtocolInstanceType};
use crate::types::signatures::{Parameter, Parameters, Signature};
use crate::types::tuple::{TupleSpec, TupleType, walk_tuple_type};
use crate::types::{
ApplyTypeMappingVisitor, BoundTypeVarInstance, FindLegacyTypeVarsVisitor, HasRelationToVisitor,
IsEquivalentVisitor, KnownClass, KnownInstanceType, MaterializationKind, NormalizedVisitor,
Type, TypeMapping, TypeRelation, TypeVarBoundOrConstraints, TypeVarInstance, TypeVarKind,
TypeVarVariance, UnionType, binding_type, declaration_type,
ApplyTypeMappingVisitor, BoundTypeVarInstance, ClassLiteral, FindLegacyTypeVarsVisitor,
HasRelationToVisitor, IsEquivalentVisitor, KnownClass, KnownInstanceType, MaterializationKind,
NormalizedVisitor, Type, TypeMapping, TypeRelation, TypeVarBoundOrConstraints, TypeVarInstance,
TypeVarKind, TypeVarVariance, UnionType, binding_type, declaration_type,
};
use crate::{Db, FxOrderSet};
@ -98,6 +98,45 @@ pub(crate) fn bind_typevar<'db>(
})
}
/// Create a `typing.Self` type variable for a given class.
pub(crate) fn typing_self<'db>(
db: &'db dyn Db,
scope_id: ScopeId,
typevar_binding_context: Option<Definition<'db>>,
class: ClassLiteral<'db>,
typevar_to_type: &impl Fn(BoundTypeVarInstance<'db>) -> Type<'db>,
) -> Option<Type<'db>> {
let index = semantic_index(db, scope_id.file(db));
let typevar = TypeVarInstance::new(
db,
ast::name::Name::new_static("Self"),
Some(class.definition(db)),
Some(
TypeVarBoundOrConstraints::UpperBound(Type::instance(
db,
class.identity_specialization(db, typevar_to_type),
))
.into(),
),
// According to the [spec], we can consider `Self`
// equivalent to an invariant type variable
// [spec]: https://typing.python.org/en/latest/spec/generics.html#self
Some(TypeVarVariance::Invariant),
None,
TypeVarKind::TypingSelf,
);
bind_typevar(
db,
index,
scope_id.file_scope_id(db),
typevar_binding_context,
typevar,
)
.map(typevar_to_type)
}
/// A list of formal type variables for a generic function, class, or type alias.
///
/// TODO: Handle nested generic contexts better, with actual parent links to the lexically
@ -286,14 +325,17 @@ impl<'db> GenericContext<'db> {
}
/// Returns a specialization of this generic context where each typevar is mapped to itself.
/// (And in particular, to an _inferable_ version of itself, since this will be used in our
/// class constructor invocation machinery to infer a specialization for the class from the
/// arguments passed to its constructor.)
pub(crate) fn identity_specialization(self, db: &'db dyn Db) -> Specialization<'db> {
/// The second parameter can be `Type::TypeVar` or `Type::NonInferableTypeVar`, depending on
/// the use case.
pub(crate) fn identity_specialization(
self,
db: &'db dyn Db,
typevar_to_type: &impl Fn(BoundTypeVarInstance<'db>) -> Type<'db>,
) -> Specialization<'db> {
let types = self
.variables(db)
.iter()
.map(|typevar| Type::TypeVar(*typevar))
.map(|typevar| typevar_to_type(*typevar))
.collect();
self.specialize(db, types)
}

View File

@ -5950,7 +5950,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
// but constructor calls to `tuple[int]`, `tuple[int, ...]`, `tuple[int, *tuple[str, ...]]` (etc.)
// are handled by the default constructor-call logic (we synthesize a `__new__` method for them
// in `ClassType::own_class_member()`).
class.is_known(self.db(), KnownClass::Tuple) && class.is_not_generic()
class.is_known(self.db(), KnownClass::Tuple) && !class.is_generic()
);
// temporary special-casing for all subclasses of `enum.Enum`

View File

@ -13,20 +13,58 @@
use std::{collections::HashMap, slice::Iter};
use itertools::{EitherOrBoth, Itertools};
use ruff_python_ast::ParameterWithDefault;
use smallvec::{SmallVec, smallvec_inline};
use super::{DynamicType, Type, TypeVarVariance, definition_expression_type};
use super::{
DynamicType, Type, TypeVarVariance, definition_expression_type, infer_definition_types,
semantic_index,
};
use crate::semantic_index::definition::Definition;
use crate::types::constraints::{ConstraintSet, IteratorConstraintsExtension};
use crate::types::generics::{GenericContext, walk_generic_context};
use crate::types::function::FunctionType;
use crate::types::generics::{GenericContext, typing_self, walk_generic_context};
use crate::types::infer::nearest_enclosing_class;
use crate::types::{
ApplyTypeMappingVisitor, BindingContext, BoundTypeVarInstance, FindLegacyTypeVarsVisitor,
HasRelationToVisitor, IsEquivalentVisitor, KnownClass, MaterializationKind, NormalizedVisitor,
TypeMapping, TypeRelation, VarianceInferable, todo_type,
ApplyTypeMappingVisitor, BindingContext, BoundTypeVarInstance, ClassType,
FindLegacyTypeVarsVisitor, HasRelationToVisitor, IsEquivalentVisitor, KnownClass,
MaterializationKind, NormalizedVisitor, TypeMapping, TypeRelation, VarianceInferable,
todo_type,
};
use crate::{Db, FxOrderSet};
use ruff_python_ast::{self as ast, name::Name};
#[derive(Clone, Copy, Debug)]
struct MethodInformation<'db> {
method: FunctionType<'db>,
class: ClassType<'db>,
}
fn infer_method_information<'db>(
db: &'db dyn Db,
definition: Definition<'db>,
) -> Option<MethodInformation<'db>> {
let class_scope_id = definition.scope(db);
let file = class_scope_id.file(db);
let index = semantic_index(db, file);
let class_scope = index.scope(class_scope_id.file_scope_id(db));
let class_node = class_scope.node().as_class()?;
let method = infer_definition_types(db, definition)
.declaration_type(definition)
.inner_type()
.into_function_literal()?;
let class_def = index.expect_single_definition(class_node);
let class_literal = infer_definition_types(db, class_def)
.declaration_type(class_def)
.inner_type();
let class = class_literal.to_class_type(db)?;
Some(MethodInformation { method, class })
}
/// The signature of a single callable. If the callable is overloaded, there is a separate
/// [`Signature`] for each overload.
#[derive(Clone, Debug, PartialEq, Eq, Hash, salsa::Update, get_size2::GetSize)]
@ -1185,16 +1223,72 @@ impl<'db> Parameters<'db> {
.map(|default| definition_expression_type(db, definition, default))
};
let method_info = infer_method_information(db, definition);
let is_static_or_classmethod = method_info
.is_some_and(|f| f.method.is_staticmethod(db) || f.method.is_classmethod(db));
let inferred_annotation = |arg: &ParameterWithDefault| {
if let Some(MethodInformation { method, class }) = method_info
&& !is_static_or_classmethod
&& arg.parameter.annotation().is_none()
&& parameters.index(arg.name().id()) == Some(0)
{
let method_has_self_in_generic_context =
method.signature(db).overloads.iter().any(|s| {
s.generic_context.is_some_and(|context| {
context
.variables(db)
.iter()
.any(|v| v.typevar(db).is_self(db))
})
});
if method_has_self_in_generic_context
|| class.is_generic()
|| class.known(db).is_some_and(KnownClass::is_fallback_class)
{
let scope_id = definition.scope(db);
let typevar_binding_context = Some(definition);
let index = semantic_index(db, scope_id.file(db));
let class = nearest_enclosing_class(db, index, scope_id).unwrap();
Some(
typing_self(db, scope_id, typevar_binding_context, class, &Type::TypeVar)
.expect("We should always find the surrounding class for an implicit self: Self annotation"),
)
} else {
// For methods of non-generic classes that are not otherwise generic (e.g. return `Self` or
// have additional type parameters), the implicit `Self` type of the `self` parameter would
// be the only type variable, so we can just use the class directly.
Some(Type::instance(db, class))
}
} else {
None
}
};
let pos_only_param = |param: &ast::ParameterWithDefault| {
Parameter::from_node_and_kind(
db,
definition,
&param.parameter,
ParameterKind::PositionalOnly {
name: Some(param.parameter.name.id.clone()),
default_type: default_type(param),
},
)
if let Some(inferred_annotation_type) = inferred_annotation(param) {
Parameter {
annotated_type: Some(inferred_annotation_type),
inferred_annotation: true,
kind: ParameterKind::PositionalOnly {
name: Some(param.parameter.name.id.clone()),
default_type: default_type(param),
},
form: ParameterForm::Value,
}
} else {
Parameter::from_node_and_kind(
db,
definition,
&param.parameter,
ParameterKind::PositionalOnly {
name: Some(param.parameter.name.id.clone()),
default_type: default_type(param),
},
)
}
};
let mut positional_only: Vec<Parameter> = posonlyargs.iter().map(pos_only_param).collect();
@ -1218,15 +1312,27 @@ impl<'db> Parameters<'db> {
}
let positional_or_keyword = pos_or_keyword_iter.map(|arg| {
Parameter::from_node_and_kind(
db,
definition,
&arg.parameter,
ParameterKind::PositionalOrKeyword {
name: arg.parameter.name.id.clone(),
default_type: default_type(arg),
},
)
if let Some(inferred_annotation_type) = inferred_annotation(arg) {
Parameter {
annotated_type: Some(inferred_annotation_type),
inferred_annotation: true,
kind: ParameterKind::PositionalOrKeyword {
name: arg.parameter.name.id.clone(),
default_type: default_type(arg),
},
form: ParameterForm::Value,
}
} else {
Parameter::from_node_and_kind(
db,
definition,
&arg.parameter,
ParameterKind::PositionalOrKeyword {
name: arg.parameter.name.id.clone(),
default_type: default_type(arg),
},
)
}
});
let variadic = vararg.as_ref().map(|arg| {
@ -1399,6 +1505,10 @@ pub(crate) struct Parameter<'db> {
/// Annotated type of the parameter.
annotated_type: Option<Type<'db>>,
/// Does the type of this parameter come from an explicit annotation, or was it inferred from
/// the context, like `Self` for the `self` parameter of instance methods.
inferred_annotation: bool,
kind: ParameterKind<'db>,
pub(crate) form: ParameterForm,
}
@ -1407,6 +1517,7 @@ impl<'db> Parameter<'db> {
pub(crate) fn positional_only(name: Option<Name>) -> Self {
Self {
annotated_type: None,
inferred_annotation: false,
kind: ParameterKind::PositionalOnly {
name,
default_type: None,
@ -1418,6 +1529,7 @@ impl<'db> Parameter<'db> {
pub(crate) fn positional_or_keyword(name: Name) -> Self {
Self {
annotated_type: None,
inferred_annotation: false,
kind: ParameterKind::PositionalOrKeyword {
name,
default_type: None,
@ -1429,6 +1541,7 @@ impl<'db> Parameter<'db> {
pub(crate) fn variadic(name: Name) -> Self {
Self {
annotated_type: None,
inferred_annotation: false,
kind: ParameterKind::Variadic { name },
form: ParameterForm::Value,
}
@ -1437,6 +1550,7 @@ impl<'db> Parameter<'db> {
pub(crate) fn keyword_only(name: Name) -> Self {
Self {
annotated_type: None,
inferred_annotation: false,
kind: ParameterKind::KeywordOnly {
name,
default_type: None,
@ -1448,6 +1562,7 @@ impl<'db> Parameter<'db> {
pub(crate) fn keyword_variadic(name: Name) -> Self {
Self {
annotated_type: None,
inferred_annotation: false,
kind: ParameterKind::KeywordVariadic { name },
form: ParameterForm::Value,
}
@ -1486,6 +1601,7 @@ impl<'db> Parameter<'db> {
.annotated_type
.map(|ty| ty.apply_type_mapping_impl(db, type_mapping, visitor)),
kind: self.kind.apply_type_mapping_impl(db, type_mapping, visitor),
inferred_annotation: self.inferred_annotation,
form: self.form,
}
}
@ -1501,6 +1617,7 @@ impl<'db> Parameter<'db> {
) -> Self {
let Parameter {
annotated_type,
inferred_annotation,
kind,
form,
} = self;
@ -1544,6 +1661,7 @@ impl<'db> Parameter<'db> {
Self {
annotated_type: Some(annotated_type),
inferred_annotation: *inferred_annotation,
kind,
form: *form,
}
@ -1566,6 +1684,7 @@ impl<'db> Parameter<'db> {
}),
kind,
form: ParameterForm::Value,
inferred_annotation: false,
}
}
@ -1620,6 +1739,11 @@ impl<'db> Parameter<'db> {
&self.kind
}
/// Whether or not the type of this parameter should be displayed.
pub(crate) fn should_annotation_be_displayed(&self) -> bool {
!self.inferred_annotation
}
/// Name of the parameter (if it has one).
pub(crate) fn name(&self) -> Option<&ast::name::Name> {
match &self.kind {

View File

@ -103,6 +103,6 @@ class NamedTupleLike(Protocol):
@classmethod
def _make(cls: type[Self], iterable: Iterable[Any]) -> Self: ...
def _asdict(self, /) -> dict[str, Any]: ...
def _replace(self: Self, /, **kwargs) -> Self: ...
def _replace(self, /, **kwargs) -> Self: ...
if sys.version_info >= (3, 13):
def __replace__(self: Self, **kwargs) -> Self: ...
def __replace__(self, **kwargs) -> Self: ...