[ty] Support typevar-specialized dynamic types in generic type aliases (#21730)

## Summary

For a type alias like the one below, where `UnknownClass` is something
with a dynamic type, we previously lost track of the fact that this
dynamic type was explicitly specialized *with a type variable*. If that
alias is then later explicitly specialized itself (`MyAlias[int]`), we
would miscount the number of legacy type variables and emit a
`invalid-type-arguments` diagnostic
([playground](https://play.ty.dev/886ae6cc-86c3-4304-a365-510d29211f85)).
```py
T = TypeVar("T")

MyAlias: TypeAlias = UnknownClass[T] | None
```
The solution implemented here is not pretty, but we can hopefully get
rid of it via https://github.com/astral-sh/ty/issues/1711. Also, once we
properly support `ParamSpec` and `Concatenate`, we should be able to
remove some of this code.

This addresses many of the `invalid-type-arguments` false-positives in
https://github.com/astral-sh/ty/issues/1685. With this change, there are
still some diagnostics of this type left. Instead of implementing even
more (rather sophisticated) workarounds for these cases as well, it
might be much easier to wait for full `ParamSpec`/`Concatenate` support
and then try again.

A disadvantage of this implementation is that we lose track of some
`@Todo` types and replace them with `Unknown`. We could spend more
effort and try to preserve them, but I'm unsure if this is the best use
of our time right now.

## Test Plan

New Markdown tests.
This commit is contained in:
David Peter 2025-12-03 10:00:02 +01:00 committed by GitHub
parent f4e4229683
commit 21e5a57296
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
9 changed files with 285 additions and 49 deletions

View File

@ -149,6 +149,101 @@ def _(x: MyAlias):
reveal_type(x) # revealed: int | list[str] | set[str]
```
## Typevar-specialized dynamic types
We still recognize type aliases as being generic if a symbol of a dynamic type is explicitly
specialized with a type variable:
```py
from typing import TypeVar, TypeAlias
from unknown_module import UnknownClass # type: ignore
T = TypeVar("T")
MyAlias1: TypeAlias = UnknownClass[T] | None
def _(a: MyAlias1[int]):
reveal_type(a) # revealed: Unknown | None
```
This also works with multiple type arguments:
```py
U = TypeVar("U")
V = TypeVar("V")
MyAlias2: TypeAlias = UnknownClass[T, U, V] | int
def _(a: MyAlias2[int, str, bytes]):
reveal_type(a) # revealed: Unknown | int
```
If we specialize with fewer or more type arguments than expected, we emit an error:
```py
def _(
# error: [invalid-type-arguments] "No type argument provided for required type variable `V`"
too_few: MyAlias2[int, str],
# error: [invalid-type-arguments] "Too many type arguments: expected 3, got 4"
too_many: MyAlias2[int, str, bytes, float],
): ...
```
We can also reference these type aliases from other type aliases:
```py
MyAlias3: TypeAlias = MyAlias1[str] | MyAlias2[int, str, bytes]
def _(c: MyAlias3):
reveal_type(c) # revealed: Unknown | None | int
```
Here, we test some other cases that might involve `@Todo` types, which also need special handling:
```py
from typing_extensions import Callable, Concatenate, TypeAliasType
MyAlias4: TypeAlias = Callable[Concatenate[dict[str, T], ...], list[U]]
def _(c: MyAlias4[int, str]):
# TODO: should be (int, / ...) -> str
reveal_type(c) # revealed: Unknown
T = TypeVar("T")
MyList = TypeAliasType("MyList", list[T], type_params=(T,))
MyAlias5 = Callable[[MyList[T]], int]
def _(c: MyAlias5[int]):
# TODO: should be (list[int], /) -> int
reveal_type(c) # revealed: (Unknown, /) -> int
K = TypeVar("K")
V = TypeVar("V")
MyDict = TypeAliasType("MyDict", dict[K, V], type_params=(K, V))
MyAlias6 = Callable[[MyDict[K, V]], int]
def _(c: MyAlias6[str, bytes]):
# TODO: should be (dict[str, bytes], /) -> int
reveal_type(c) # revealed: (Unknown, /) -> int
ListOrDict: TypeAlias = MyList[T] | dict[str, T]
def _(x: ListOrDict[int]):
# TODO: should be list[int] | dict[str, int]
reveal_type(x) # revealed: Unknown | dict[str, int]
MyAlias7: TypeAlias = Callable[Concatenate[T, ...], None]
def _(c: MyAlias7[int]):
# TODO: should be (int, / ...) -> None
reveal_type(c) # revealed: Unknown
```
## Imported
`alias.py`:

View File

@ -223,7 +223,8 @@ T = TypeVar("T")
IntAndT = TypeAliasType("IntAndT", tuple[int, T], type_params=(T,))
def f(x: IntAndT[str]) -> None:
reveal_type(x) # revealed: @Todo(Generic manual PEP-695 type alias)
# TODO: This should be `tuple[int, str]`
reveal_type(x) # revealed: Unknown
```
### Error cases

View File

@ -763,7 +763,7 @@ impl<'db> DataclassParams<'db> {
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, salsa::Update, get_size2::GetSize)]
pub enum Type<'db> {
/// The dynamic type: a statically unknown set of values
Dynamic(DynamicType),
Dynamic(DynamicType<'db>),
/// The empty set of values
Never,
/// A specific function object
@ -889,7 +889,10 @@ impl<'db> Type<'db> {
}
pub const fn is_unknown(&self) -> bool {
matches!(self, Type::Dynamic(DynamicType::Unknown))
matches!(
self,
Type::Dynamic(DynamicType::Unknown | DynamicType::UnknownGeneric(_))
)
}
pub(crate) const fn is_never(&self) -> bool {
@ -959,7 +962,10 @@ impl<'db> Type<'db> {
pub(crate) fn is_todo(&self) -> bool {
self.as_dynamic().is_some_and(|dynamic| match dynamic {
DynamicType::Any | DynamicType::Unknown | DynamicType::Divergent(_) => false,
DynamicType::Any
| DynamicType::Unknown
| DynamicType::UnknownGeneric(_)
| DynamicType::Divergent(_) => false,
DynamicType::Todo(_) | DynamicType::TodoStarredExpression | DynamicType::TodoUnpack => {
true
}
@ -1146,7 +1152,7 @@ impl<'db> Type<'db> {
}
}
pub(crate) const fn as_dynamic(self) -> Option<DynamicType> {
pub(crate) const fn as_dynamic(self) -> Option<DynamicType<'db>> {
match self {
Type::Dynamic(dynamic_type) => Some(dynamic_type),
_ => None,
@ -1160,7 +1166,7 @@ impl<'db> Type<'db> {
}
}
pub(crate) const fn expect_dynamic(self) -> DynamicType {
pub(crate) const fn expect_dynamic(self) -> DynamicType<'db> {
self.as_dynamic().expect("Expected a Type::Dynamic variant")
}
@ -7851,14 +7857,18 @@ impl<'db> Type<'db> {
typevars: &mut FxOrderSet<BoundTypeVarInstance<'db>>,
visitor: &FindLegacyTypeVarsVisitor<'db>,
) {
let is_matching_typevar = |bound_typevar: &BoundTypeVarInstance<'db>| {
matches!(
bound_typevar.typevar(db).kind(db),
TypeVarKind::Legacy | TypeVarKind::TypingSelf | TypeVarKind::ParamSpec
) && binding_context.is_none_or(|binding_context| {
bound_typevar.binding_context(db) == BindingContext::Definition(binding_context)
})
};
match self {
Type::TypeVar(bound_typevar) => {
if matches!(
bound_typevar.typevar(db).kind(db),
TypeVarKind::Legacy | TypeVarKind::TypingSelf | TypeVarKind::ParamSpec
) && binding_context.is_none_or(|binding_context| {
bound_typevar.binding_context(db) == BindingContext::Definition(binding_context)
}) {
if is_matching_typevar(&bound_typevar) {
typevars.insert(bound_typevar);
}
}
@ -7998,6 +8008,14 @@ impl<'db> Type<'db> {
}
},
Type::Dynamic(DynamicType::UnknownGeneric(generic_context)) => {
for variable in generic_context.variables(db) {
if is_matching_typevar(&variable) {
typevars.insert(variable);
}
}
}
Type::Dynamic(_)
| Type::Never
| Type::AlwaysTruthy
@ -8029,6 +8047,26 @@ impl<'db> Type<'db> {
}
}
/// Bind all unbound legacy type variables to the given context and then
/// add all legacy typevars to the provided set.
pub(crate) fn bind_and_find_all_legacy_typevars(
self,
db: &'db dyn Db,
binding_context: Option<Definition<'db>>,
variables: &mut FxOrderSet<BoundTypeVarInstance<'db>>,
) {
self.apply_type_mapping(
db,
&TypeMapping::BindLegacyTypevars(
binding_context
.map(BindingContext::Definition)
.unwrap_or(BindingContext::Synthetic),
),
TypeContext::default(),
)
.find_legacy_typevars(db, None, variables);
}
/// Replace default types in parameters of callables with `Unknown`.
pub(crate) fn replace_parameter_defaults(self, db: &'db dyn Db) -> Type<'db> {
self.apply_type_mapping(
@ -8177,7 +8215,7 @@ impl<'db> Type<'db> {
Self::SpecialForm(special_form) => special_form.definition(db),
Self::Never => Type::SpecialForm(SpecialFormType::Never).definition(db),
Self::Dynamic(DynamicType::Any) => Type::SpecialForm(SpecialFormType::Any).definition(db),
Self::Dynamic(DynamicType::Unknown) => Type::SpecialForm(SpecialFormType::Unknown).definition(db),
Self::Dynamic(DynamicType::Unknown | DynamicType::UnknownGeneric(_)) => Type::SpecialForm(SpecialFormType::Unknown).definition(db),
Self::AlwaysTruthy => Type::SpecialForm(SpecialFormType::AlwaysTruthy).definition(db),
Self::AlwaysFalsy => Type::SpecialForm(SpecialFormType::AlwaysFalsy).definition(db),
@ -8839,11 +8877,18 @@ pub struct DivergentType {
impl get_size2::GetSize for DivergentType {}
#[derive(Copy, Clone, Debug, Eq, Hash, PartialEq, salsa::Update, get_size2::GetSize)]
pub enum DynamicType {
pub enum DynamicType<'db> {
/// An explicitly annotated `typing.Any`
Any,
/// An unannotated value, or a dynamic type resulting from an error
Unknown,
/// Similar to `Unknown`, this represents a dynamic type that has been explicitly specialized
/// with legacy typevars, e.g. `UnknownClass[T]`, where `T` is a legacy typevar. We keep track
/// of the type variables in the generic context in case this type is later specialized again.
///
/// TODO: Once we implement <https://github.com/astral-sh/ty/issues/1711>, this variant might
/// not be needed anymore.
UnknownGeneric(GenericContext<'db>),
/// Temporary type for symbols that can't be inferred yet because of missing implementations.
///
/// This variant should eventually be removed once ty is spec-compliant.
@ -8862,7 +8907,7 @@ pub enum DynamicType {
Divergent(DivergentType),
}
impl DynamicType {
impl DynamicType<'_> {
fn normalized(self) -> Self {
if matches!(self, Self::Divergent(_)) {
self
@ -8880,11 +8925,11 @@ impl DynamicType {
}
}
impl std::fmt::Display for DynamicType {
impl std::fmt::Display for DynamicType<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
DynamicType::Any => f.write_str("Any"),
DynamicType::Unknown => f.write_str("Unknown"),
DynamicType::Unknown | DynamicType::UnknownGeneric(_) => f.write_str("Unknown"),
// `DynamicType::Todo`'s display should be explicit that is not a valid display of
// any other type
DynamicType::Todo(todo) => write!(f, "@Todo{todo}"),

View File

@ -172,7 +172,7 @@ impl<'db> BoundSuperError<'db> {
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, get_size2::GetSize)]
pub enum SuperOwnerKind<'db> {
Dynamic(DynamicType),
Dynamic(DynamicType<'db>),
Class(ClassType<'db>),
Instance(NominalInstanceType<'db>),
}

View File

@ -18,7 +18,7 @@ use crate::types::{
/// automatically construct the default specialization for that class.
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, salsa::Update, get_size2::GetSize)]
pub enum ClassBase<'db> {
Dynamic(DynamicType),
Dynamic(DynamicType<'db>),
Class(ClassType<'db>),
/// Although `Protocol` is not a class in typeshed's stubs, it is at runtime,
/// and can appear in the MRO of a class.
@ -62,7 +62,7 @@ impl<'db> ClassBase<'db> {
match self {
ClassBase::Class(class) => class.name(db),
ClassBase::Dynamic(DynamicType::Any) => "Any",
ClassBase::Dynamic(DynamicType::Unknown) => "Unknown",
ClassBase::Dynamic(DynamicType::Unknown | DynamicType::UnknownGeneric(_)) => "Unknown",
ClassBase::Dynamic(
DynamicType::Todo(_) | DynamicType::TodoUnpack | DynamicType::TodoStarredExpression,
) => "@Todo",

View File

@ -9571,6 +9571,9 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
(unknown @ Type::Dynamic(DynamicType::Unknown), _, _)
| (_, unknown @ Type::Dynamic(DynamicType::Unknown), _) => Some(unknown),
(unknown @ Type::Dynamic(DynamicType::UnknownGeneric(_)), _, _)
| (_, unknown @ Type::Dynamic(DynamicType::UnknownGeneric(_)), _) => Some(unknown),
(
todo @ Type::Dynamic(
DynamicType::Todo(_)
@ -10969,6 +10972,19 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
);
}
}
Type::KnownInstance(KnownInstanceType::TypeAliasType(TypeAliasType::ManualPEP695(
_,
))) => {
let slice_ty = self.infer_expression(slice, TypeContext::default());
let mut variables = FxOrderSet::default();
slice_ty.bind_and_find_all_legacy_typevars(
self.db(),
self.typevar_binding_context,
&mut variables,
);
let generic_context = GenericContext::from_typevar_instances(self.db(), variables);
return Type::Dynamic(DynamicType::UnknownGeneric(generic_context));
}
Type::KnownInstance(KnownInstanceType::TypeAliasType(type_alias)) => {
if let Some(generic_context) = type_alias.generic_context(self.db()) {
return self.infer_explicit_type_alias_type_specialization(
@ -11107,33 +11123,74 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
));
}
Type::SpecialForm(SpecialFormType::Callable) => {
// TODO: Remove this once we support ParamSpec properly. This is necessary to avoid
// a lot of false positives downstream, because we can't represent the specialized
// `Callable[P, _]` type yet.
if let Some(first_arg) = subscript
.slice
.as_ref()
.as_tuple_expr()
.and_then(|args| args.elts.first())
&& first_arg.is_name_expr()
{
let first_arg_ty = self.infer_expression(first_arg, TypeContext::default());
let arguments = if let ast::Expr::Tuple(tuple) = &*subscript.slice {
&*tuple.elts
} else {
std::slice::from_ref(&*subscript.slice)
};
if let Type::KnownInstance(KnownInstanceType::TypeVar(typevar)) = first_arg_ty
&& typevar.kind(self.db()).is_paramspec()
{
return todo_type!("Callable[..] specialized with ParamSpec");
}
// TODO: Remove this once we support ParamSpec and Concatenate properly. This is necessary
// to avoid a lot of false positives downstream, because we can't represent the typevar-
// specialized `Callable` types yet.
let num_arguments = arguments.len();
if num_arguments == 2 {
let first_arg = &arguments[0];
let second_arg = &arguments[1];
if let Some(builder) = self.context.report_lint(&INVALID_TYPE_FORM, subscript) {
builder.into_diagnostic(format_args!(
"The first argument to `Callable` must be either a list of types, \
ParamSpec, Concatenate, or `...`",
if first_arg.is_name_expr() {
let first_arg_ty = self.infer_expression(first_arg, TypeContext::default());
if let Type::KnownInstance(KnownInstanceType::TypeVar(typevar)) =
first_arg_ty
&& typevar.kind(self.db()).is_paramspec()
{
return todo_type!("Callable[..] specialized with ParamSpec");
}
if let Some(builder) =
self.context.report_lint(&INVALID_TYPE_FORM, subscript)
{
builder.into_diagnostic(format_args!(
"The first argument to `Callable` must be either a list of types, \
ParamSpec, Concatenate, or `...`",
));
}
return Type::KnownInstance(KnownInstanceType::Callable(
CallableType::unknown(self.db()),
));
} else if first_arg.is_subscript_expr() {
let first_arg_ty = self.infer_expression(first_arg, TypeContext::default());
if let Type::Dynamic(DynamicType::UnknownGeneric(generic_context)) =
first_arg_ty
{
let mut variables = generic_context
.variables(self.db())
.collect::<FxOrderSet<_>>();
let return_ty =
self.infer_expression(second_arg, TypeContext::default());
return_ty.bind_and_find_all_legacy_typevars(
self.db(),
self.typevar_binding_context,
&mut variables,
);
let generic_context =
GenericContext::from_typevar_instances(self.db(), variables);
return Type::Dynamic(DynamicType::UnknownGeneric(generic_context));
}
if let Some(builder) =
self.context.report_lint(&INVALID_TYPE_FORM, subscript)
{
builder.into_diagnostic(format_args!(
"The first argument to `Callable` must be either a list of types, \
ParamSpec, Concatenate, or `...`",
));
}
return Type::KnownInstance(KnownInstanceType::Callable(
CallableType::unknown(self.db()),
));
}
return Type::KnownInstance(KnownInstanceType::Callable(
CallableType::unknown(self.db()),
));
}
let callable = self
@ -11240,6 +11297,17 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
) => {
return self.infer_explicit_type_alias_specialization(subscript, value_ty, false);
}
Type::Dynamic(DynamicType::Unknown) => {
let slice_ty = self.infer_expression(slice, TypeContext::default());
let mut variables = FxOrderSet::default();
slice_ty.bind_and_find_all_legacy_typevars(
self.db(),
self.typevar_binding_context,
&mut variables,
);
let generic_context = GenericContext::from_typevar_instances(self.db(), variables);
return Type::Dynamic(DynamicType::UnknownGeneric(generic_context));
}
_ => {}
}
@ -11696,6 +11764,18 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
Some(Type::Dynamic(DynamicType::TodoUnpack))
}
(Type::SpecialForm(SpecialFormType::Concatenate), _) => {
// TODO: Add proper support for `Concatenate`
let mut variables = FxOrderSet::default();
slice_ty.bind_and_find_all_legacy_typevars(
db,
self.typevar_binding_context,
&mut variables,
);
let generic_context = GenericContext::from_typevar_instances(self.db(), variables);
Some(Type::Dynamic(DynamicType::UnknownGeneric(generic_context)))
}
(Type::SpecialForm(special_form), _) if special_form.class().is_special_form() => {
Some(todo_type!("Inference of subscript on special form"))
}

View File

@ -946,8 +946,17 @@ impl<'db> TypeInferenceBuilder<'db, '_> {
}
}
KnownInstanceType::TypeAliasType(TypeAliasType::ManualPEP695(_)) => {
self.infer_type_expression(slice);
todo_type!("Generic manual PEP-695 type alias")
// TODO: support generic "manual" PEP 695 type aliases
let slice_ty = self.infer_expression(slice, TypeContext::default());
let mut variables = FxOrderSet::default();
slice_ty.bind_and_find_all_legacy_typevars(
self.db(),
self.typevar_binding_context,
&mut variables,
);
let generic_context =
GenericContext::from_typevar_instances(self.db(), variables);
Type::Dynamic(DynamicType::UnknownGeneric(generic_context))
}
KnownInstanceType::LiteralStringAlias(_) => {
self.infer_type_expression(slice);
@ -984,6 +993,9 @@ impl<'db> TypeInferenceBuilder<'db, '_> {
Type::unknown()
}
},
Type::Dynamic(DynamicType::UnknownGeneric(_)) => {
self.infer_explicit_type_alias_specialization(subscript, value_ty, true)
}
Type::Dynamic(_) => {
// Infer slice as a value expression to avoid false-positive
// `invalid-type-form` diagnostics, when we have e.g.

View File

@ -322,7 +322,7 @@ impl<'db> VarianceInferable<'db> for SubclassOfType<'db> {
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, salsa::Update, get_size2::GetSize)]
pub(crate) enum SubclassOfInner<'db> {
Class(ClassType<'db>),
Dynamic(DynamicType),
Dynamic(DynamicType<'db>),
TypeVar(BoundTypeVarInstance<'db>),
}
@ -362,7 +362,7 @@ impl<'db> SubclassOfInner<'db> {
}
}
pub(crate) const fn into_dynamic(self) -> Option<DynamicType> {
pub(crate) const fn into_dynamic(self) -> Option<DynamicType<'db>> {
match self {
Self::Class(_) | Self::TypeVar(_) => None,
Self::Dynamic(dynamic) => Some(dynamic),
@ -465,8 +465,8 @@ impl<'db> From<ClassType<'db>> for SubclassOfInner<'db> {
}
}
impl From<DynamicType> for SubclassOfInner<'_> {
fn from(value: DynamicType) -> Self {
impl<'db> From<DynamicType<'db>> for SubclassOfInner<'db> {
fn from(value: DynamicType<'db>) -> Self {
SubclassOfInner::Dynamic(value)
}
}

View File

@ -265,6 +265,9 @@ fn dynamic_elements_ordering(left: DynamicType, right: DynamicType) -> Ordering
(DynamicType::Unknown, _) => Ordering::Less,
(_, DynamicType::Unknown) => Ordering::Greater,
(DynamicType::UnknownGeneric(_), _) => Ordering::Less,
(_, DynamicType::UnknownGeneric(_)) => Ordering::Greater,
#[cfg(debug_assertions)]
(DynamicType::Todo(TodoType(left)), DynamicType::Todo(TodoType(right))) => left.cmp(right),