Python
/
ruff
mirror of https://github.com/astral-sh/ruff
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[ty] New Type variant for TypedDict

This commit is contained in:
David Peter 2025-08-04 10:46:04 +02:00
parent b95d22c08e
commit efcb7468c6
10 changed files with 61 additions and 626 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
Cargo.lock generated
View File

@ -3441,7 +3441,7 @@ checksum = "28d3b2b1366ec20994f1fd18c3c594f05c5dd4bc44d8bb0c1c632c8d6829481f"
[[package]]
name = "salsa"
version = "0.23.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=86ca4a9d70e97dd5107e6111a09647dd10ff7535#86ca4a9d70e97dd5107e6111a09647dd10ff7535"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c2f4827b512b82842dbc84b1ccc4367500e301ed#c2f4827b512b82842dbc84b1ccc4367500e301ed"
dependencies = [
"boxcar",
"compact_str",
@ -3465,12 +3465,12 @@ dependencies = [
[[package]]
name = "salsa-macro-rules"
version = "0.23.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=86ca4a9d70e97dd5107e6111a09647dd10ff7535#86ca4a9d70e97dd5107e6111a09647dd10ff7535"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c2f4827b512b82842dbc84b1ccc4367500e301ed#c2f4827b512b82842dbc84b1ccc4367500e301ed"
[[package]]
name = "salsa-macros"
version = "0.23.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=86ca4a9d70e97dd5107e6111a09647dd10ff7535#86ca4a9d70e97dd5107e6111a09647dd10ff7535"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c2f4827b512b82842dbc84b1ccc4367500e301ed#c2f4827b512b82842dbc84b1ccc4367500e301ed"
dependencies = [
"proc-macro2",
"quote",

2
Cargo.toml
View File

@ -141,7 +141,7 @@ regex-automata = { version = "0.4.9" }
rustc-hash = { version = "2.0.0" }
rustc-stable-hash = { version = "0.1.2" }
# When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "86ca4a9d70e97dd5107e6111a09647dd10ff7535", default-features = false, features = [
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "c2f4827b512b82842dbc84b1ccc4367500e301ed", default-features = false, features = [
"compact_str",
"macros",
"salsa_unstable",

1
crates/ty_python_semantic/resources/mdtest/call/function.md
View File

@ -611,6 +611,7 @@ from ty_extensions import static_assert
static_assert()
# error: [too-many-positional-arguments] "Too many positional arguments to function `static_assert`: expected 2, got 3"
# error: 21 [invalid-argument-type] "Argument to function `static_assert` is incorrect: Expected `LiteralString | None`, found `Literal[2]`"
static_assert(True, 2, 3)
```

583
crates/ty_python_semantic/resources/mdtest/unreachable.md
View File

@ -1,583 +0,0 @@
# Unreachable code
This document describes our approach to handling unreachable code. There are two aspects to this.
One is to detect and mark blocks of code that are unreachable. This is useful for notifying the
user, as it can often be indicative of an error. The second aspect of this is to make sure that we
do not emit (incorrect) diagnostics in unreachable code.
## Detecting unreachable code
In this section, we look at various scenarios how sections of code can become unreachable. We should
eventually introduce a new diagnostic that would detect unreachable code. In an editor/LSP context,
there are ways to 'gray out' sections of code, which is helpful for blocks of code that are not
'dead' code, but inactive under certain conditions, like platform-specific code.
### Terminal statements
In the following examples, the `print` statements are definitely unreachable.
```py
def f1():
return
# TODO: we should mark this as unreachable
print("unreachable")
def f2():
raise Exception()
# TODO: we should mark this as unreachable
print("unreachable")
def f3():
while True:
break
# TODO: we should mark this as unreachable
print("unreachable")
def f4():
for _ in range(10):
continue
# TODO: we should mark this as unreachable
print("unreachable")
```
### Infinite loops
```py
def f1():
while True:
pass
# TODO: we should mark this as unreachable
print("unreachable")
```
### Statically known branches
In the following examples, the `print` statements are also unreachable, but it requires type
inference to determine that:
```py
def f1():
if 2 + 3 > 10:
# TODO: we should mark this as unreachable
print("unreachable")
def f2():
if True:
return
# TODO: we should mark this as unreachable
print("unreachable")
def f3():
if False:
return
elif True:
return
else:
pass
# TODO: we should mark this as unreachable
print("unreachable")
```
### `Never` / `NoReturn`
If a function is annotated with a return type of `Never` or `NoReturn`, we can consider all code
after the call to that function unreachable.
```py
from typing_extensions import NoReturn
def always_raises() -> NoReturn:
raise Exception()
def f():
always_raises()
# TODO: we should mark this as unreachable
print("unreachable")
```
### Python version and platform checks
It is common to have code that is specific to a certain Python version or platform. This case is
special because whether or not the code is reachable depends on externally configured constants. And
if we are checking for a set of parameters that makes one of these branches unreachable, that is
likely not something that the user wants to be warned about, because there are probably other sets
of parameters that make the branch reachable.
#### `sys.version_info` branches
Consider the following example. If we check with a Python version lower than 3.11, the import
statement is unreachable. If we check with a Python version equal to or greater than 3.11, the
import statement is definitely reachable. We should not emit any diagnostics in either case.
##### Checking with Python version 3.10
```toml
[environment]
python-version = "3.10"
```
```py
import sys
if sys.version_info >= (3, 11):
from typing import Self
```
##### Checking with Python version 3.12
```toml
[environment]
python-version = "3.12"
```
```py
import sys
if sys.version_info >= (3, 11):
from typing import Self
```
#### `sys.platform` branches
The problem is even more pronounced with `sys.platform` branches, since we don't necessarily have
the platform information available.
##### Checking with platform `win32`
```toml
[environment]
python-platform = "win32"
```
```py
import sys
if sys.platform == "win32":
sys.getwindowsversion()
```
##### Checking with platform `linux`
```toml
[environment]
python-platform = "linux"
```
```py
import sys
if sys.platform == "win32":
sys.getwindowsversion()
```
##### Checking with platform set to `all`
```toml
[environment]
python-platform = "all"
```
If `python-platform` is set to `all`, we treat the platform as unspecified. This means that we do
not infer a literal type like `Literal["win32"]` for `sys.platform`, but instead fall back to
`LiteralString` (the `typeshed` annotation for `sys.platform`). This means that we can not
statically determine the truthiness of a branch like `sys.platform == "win32"`.
See <https://github.com/astral-sh/ruff/issues/16983#issuecomment-2777146188> for a plan on how this
could be improved.
```py
import sys
if sys.platform == "win32":
# TODO: we should not emit an error here
# error: [possibly-unbound-attribute]
sys.getwindowsversion()
```
## No (incorrect) diagnostics in unreachable code
```toml
[environment]
python-version = "3.10"
```
In this section, we demonstrate that we do not emit (incorrect) diagnostics in unreachable sections
of code.
It could be argued that no diagnostics at all should be emitted in unreachable code. The reasoning
is that any issues inside the unreachable section would not cause problems at runtime. And type
checking the unreachable code under the assumption that it *is* reachable might lead to false
positives (see the "Global constants" example below).
On the other hand, it could be argued that code like `1 + "a"` is incorrect, no matter if it is
reachable or not. Some developers like to use things like early `return` statements while debugging,
and for this use case, it is helpful to still see some diagnostics in unreachable sections.
We currently follow the second approach, but we do not attempt to provide the full set of
diagnostics in unreachable sections. In fact, a large number of diagnostics are suppressed in
unreachable code, simply due to the fact that we infer `Never` for most of the symbols.
### Use of variables in unreachable code
We should not emit any diagnostics for uses of symbols in unreachable code:
```py
def f():
x = 1
return
print("unreachable")
print(x)
```
### Use of variable in nested function
This is a regression test for a behavior that previously caused problems when the public type still
referred to the end-of-scope, which would result in an unresolved-reference error here since the end
of the scope is unreachable.
```py
def outer():
x = 1
def inner():
reveal_type(x) # revealed: Literal[1]
while True:
pass
```
### Global constants
```py
from typing import Literal
FEATURE_X_ACTIVATED: Literal[False] = False
if FEATURE_X_ACTIVATED:
def feature_x():
print("Performing 'X'")
def f():
if FEATURE_X_ACTIVATED:
# Type checking this particular section as if it were reachable would
# lead to a false positive, so we should not emit diagnostics here.
feature_x()
```
### Exhaustive check of syntactic constructs
We include some more examples here to make sure that silencing of diagnostics works for
syntactically different cases. To test this, we use `ExceptionGroup`, which is only available in
Python 3.11 and later. We have set the Python version to 3.10 for this whole section, to have
`match` statements available, but not `ExceptionGroup`.
To start, we make sure that we do not emit a diagnostic in this simple case:
```py
import sys
if sys.version_info >= (3, 11):
ExceptionGroup # no error here
```
Similarly, if we negate the logic, we also emit no error:
```py
if sys.version_info < (3, 11):
pass
else:
ExceptionGroup # no error here
```
This also works for more complex `if`-`elif`-`else` chains:
```py
if sys.version_info >= (3, 13):
ExceptionGroup # no error here
elif sys.version_info >= (3, 12):
ExceptionGroup # no error here
elif sys.version_info >= (3, 11):
ExceptionGroup # no error here
elif sys.version_info >= (3, 10):
pass
else:
# This branch is also unreachable, because the previous `elif` branch is always true
ExceptionGroup # no error here
```
And for nested `if` statements:
```py
def _(flag: bool):
if flag:
if sys.version_info >= (3, 11):
ExceptionGroup # no error here
else:
pass
if sys.version_info < (3, 11):
pass
else:
ExceptionGroup # no error here
```
The same works for ternary expressions:
```py
class ExceptionGroupPolyfill: ...
MyExceptionGroup1 = ExceptionGroup if sys.version_info >= (3, 11) else ExceptionGroupPolyfill
MyExceptionGroup1 = ExceptionGroupPolyfill if sys.version_info < (3, 11) else ExceptionGroup
```
Due to short-circuiting, this also works for Boolean operators:
```py
sys.version_info >= (3, 11) and ExceptionGroup
sys.version_info < (3, 11) or ExceptionGroup
```
And in `match` statements:
```py
reveal_type(sys.version_info.minor) # revealed: Literal[10]
match sys.version_info.minor:
case 13:
ExceptionGroup
case 12:
ExceptionGroup
case 11:
ExceptionGroup
case _:
pass
```
Terminal statements can also lead to unreachable code:
```py
def f():
if sys.version_info < (3, 11):
raise RuntimeError("this code only works for Python 3.11+")
ExceptionGroup
```
Similarly, assertions with statically-known falsy conditions can lead to unreachable code:
```py
def f():
assert sys.version_info > (3, 11)
ExceptionGroup
```
Finally, not that anyone would ever use it, but it also works for `while` loops:
```py
while sys.version_info >= (3, 11):
ExceptionGroup
```
### Infinite loops
We also do not emit diagnostics in unreachable code after an infinite loop:
```py
def f():
while True:
pass
ExceptionGroup # no error here
```
### Silencing errors for actually unknown symbols
We currently also silence diagnostics for symbols that are not actually defined anywhere. It is
conceivable that this could be improved, but is not a priority for now.
```py
if False:
does_not_exist
def f():
return
does_not_exist
```
### Attributes
When attribute expressions appear in unreachable code, we should not emit `unresolved-attribute`
diagnostics:
```py
import sys
import builtins
if sys.version_info >= (3, 11):
builtins.ExceptionGroup
```
### Imports
When import statements appear in unreachable code, we should not emit `unresolved-import`
diagnostics:
```py
import sys
if sys.version_info >= (3, 11):
from builtins import ExceptionGroup
import builtins.ExceptionGroup
# See https://docs.python.org/3/whatsnew/3.11.html#new-modules
import tomllib
import wsgiref.types
```
### Nested scopes
When we have nested scopes inside the unreachable section, we should not emit diagnostics either:
```py
if False:
x = 1
def f():
print(x)
class C:
def __init__(self):
print(x)
```
### Type annotations
Silencing of diagnostics also works for type annotations, even if they are stringified:
```py
import sys
import typing
if sys.version_info >= (3, 11):
from typing import Self
class C:
def name_expr(self) -> Self:
return self
def name_expr_stringified(self) -> "Self":
return self
def attribute_expr(self) -> typing.Self:
return self
def attribute_expr_stringified(self) -> "typing.Self":
return self
```
### Use of unreachable symbols in type annotations, or as class bases
We should not show any diagnostics in type annotations inside unreachable sections.
```py
def _():
class C:
class Inner: ...
return
c1: C = C()
c2: C.Inner = C.Inner()
c3: tuple[C, C] = (C(), C())
c4: tuple[C.Inner, C.Inner] = (C.Inner(), C.Inner())
class Sub(C): ...
```
### Emit diagnostics for definitely wrong code
Even though the expressions in the snippet below are unreachable, we still emit diagnostics for
them:
```py
if False:
1 + "a" # error: [unsupported-operator]
def f():
return
1 / 0 # error: [division-by-zero]
```
### Conflicting type information
We also support cases where type information for symbols conflicts between mutually exclusive
branches:
```py
import sys
if sys.version_info >= (3, 11):
x: int = 1
else:
x: str = "a"
if sys.version_info >= (3, 11):
other: int = x
else:
other: str = x
```
This is also supported for function calls, attribute accesses, etc.:
```py
from typing import Literal
if False:
def f(x: int): ...
def g(*, a: int, b: int): ...
class C:
x: int = 1
class D:
def __call__(self):
pass
number: Literal[1] = 1
else:
def f(x: str): ...
def g(*, a: int): ...
class C:
x: str = "a"
class D: ...
number: Literal[0] = 0
if False:
f(2)
g(a=2, b=3)
C.x = 2
d: D = D()
d()
1 / number
```

33
crates/ty_python_semantic/src/types.rs
View File

@ -5199,11 +5199,30 @@ impl<'db> Type<'db> {
KnownClass::Float.to_instance(db),
],
),
_ => Type::instance(db, class.default_specialization(db)),
_ => {
if class
.iter_mro(db, None)
.any(|base| matches!(base, ClassBase::TypedDict))
{
todo_type!("Support for `TypedDict`")
} else {
Type::instance(db, class.default_specialization(db))
}
}
};
Ok(ty)
}
Type::GenericAlias(alias) => Ok(Type::instance(db, ClassType::from(*alias))),
Type::GenericAlias(alias) => {
let class_type = ClassType::from(*alias);
if class_type
.iter_mro(db)
.any(|base| matches!(base, ClassBase::TypedDict))
{
Ok(todo_type!("Support for `TypedDict`"))
} else {
Ok(Type::instance(db, class_type))
}
}
Type::SubclassOf(_)
| Type::BooleanLiteral(_)
@ -6250,9 +6269,6 @@ pub enum DynamicType {
/// A special Todo-variant for type aliases declared using `typing.TypeAlias`.
/// A temporary variant to detect and special-case the handling of these aliases in autocomplete suggestions.
TodoTypeAlias,
/// A special Todo-variant for classes inheriting from `TypedDict`.
/// A temporary variant to avoid false positives while we wait for full support.
TodoTypedDict,
}
impl DynamicType {
@ -6284,13 +6300,6 @@ impl std::fmt::Display for DynamicType {
f.write_str("@Todo")
}
}
DynamicType::TodoTypedDict => {
if cfg!(debug_assertions) {
f.write_str("@Todo(Support for `TypedDict`)")
} else {
f.write_str("@Todo")
}
}
}
}
}

22
crates/ty_python_semantic/src/types/class.rs
View File

@ -23,9 +23,9 @@ use crate::types::signatures::{CallableSignature, Parameter, Parameters, Signatu
use crate::types::tuple::{TupleSpec, TupleType};
use crate::types::{
BareTypeAliasType, Binding, BoundSuperError, BoundSuperType, CallableType, DataclassParams,
DeprecatedInstance, DynamicType, KnownInstanceType, TypeAliasType, TypeMapping, TypeRelation,
DeprecatedInstance, KnownInstanceType, TypeAliasType, TypeMapping, TypeRelation,
TypeTransformer, TypeVarBoundOrConstraints, TypeVarInstance, TypeVarKind, declaration_type,
infer_definition_types,
infer_definition_types, todo_type,
};
use crate::{
Db, FxIndexMap, FxOrderSet, Program,
@ -419,15 +419,6 @@ impl<'db> ClassType<'db> {
other: Self,
relation: TypeRelation,
) -> bool {
// TODO: remove this branch once we have proper support for TypedDicts.
if self.is_known(db, KnownClass::Dict)
&& other
.iter_mro(db)
.any(|b| matches!(b, ClassBase::Dynamic(DynamicType::TodoTypedDict)))
{
return true;
}
self.iter_mro(db).any(|base| {
match base {
ClassBase::Dynamic(_) => match relation {
@ -451,6 +442,8 @@ impl<'db> ClassType<'db> {
(ClassType::Generic(_), ClassType::NonGeneric(_))
| (ClassType::NonGeneric(_), ClassType::Generic(_)) => false,
},
ClassBase::TypedDict => true,
}
})
}
@ -1687,6 +1680,10 @@ impl<'db> ClassLiteral<'db> {
)
});
}
ClassBase::TypedDict => {
return Place::bound(todo_type!("Support for `TypedDict`"))
.with_qualifiers(TypeQualifiers::empty());
}
}
if lookup_result.is_ok() {
break;
@ -2142,6 +2139,9 @@ impl<'db> ClassLiteral<'db> {
union = union.add(ty);
}
}
ClassBase::TypedDict => {
return Place::bound(todo_type!("Support for `TypedDict`")).into();
}
}
}

23
crates/ty_python_semantic/src/types/class_base.rs
View File

@ -24,6 +24,7 @@ pub enum ClassBase<'db> {
/// but nonetheless appears in the MRO of classes that inherit from `Generic[T]`,
/// `Protocol[T]`, or bare `Protocol`.
Generic,
TypedDict,
}
impl<'db> ClassBase<'db> {
@ -39,7 +40,7 @@ impl<'db> ClassBase<'db> {
match self {
Self::Dynamic(dynamic) => Self::Dynamic(dynamic.normalized()),
Self::Class(class) => Self::Class(class.normalized_impl(db, visitor)),
Self::Protocol | Self::Generic => self,
Self::Protocol | Self::Generic | Self::TypedDict => self,
}
}
@ -51,11 +52,11 @@ impl<'db> ClassBase<'db> {
ClassBase::Dynamic(
DynamicType::Todo(_)
| DynamicType::TodoPEP695ParamSpec
| DynamicType::TodoTypeAlias
| DynamicType::TodoTypedDict,
| DynamicType::TodoTypeAlias,
) => "@Todo",
ClassBase::Protocol => "Protocol",
ClassBase::Generic => "Generic",
ClassBase::TypedDict => "TypedDict",
}
}
@ -234,7 +235,7 @@ impl<'db> ClassBase<'db> {
SpecialFormType::OrderedDict => {
Self::try_from_type(db, KnownClass::OrderedDict.to_class_literal(db))
}
SpecialFormType::TypedDict => Some(Self::Dynamic(DynamicType::TodoTypedDict)),
SpecialFormType::TypedDict => Some(Self::TypedDict),
SpecialFormType::Callable => {
Self::try_from_type(db, todo_type!("Support for Callable as a base class"))
}
@ -245,14 +246,14 @@ impl<'db> ClassBase<'db> {
pub(super) fn into_class(self) -> Option<ClassType<'db>> {
match self {
Self::Class(class) => Some(class),
Self::Dynamic(_) | Self::Generic | Self::Protocol => None,
Self::Dynamic(_) | Self::Generic | Self::Protocol | Self::TypedDict => None,
}
}
fn apply_type_mapping<'a>(self, db: &'db dyn Db, type_mapping: &TypeMapping<'a, 'db>) -> Self {
match self {
Self::Class(class) => Self::Class(class.apply_type_mapping(db, type_mapping)),
Self::Dynamic(_) | Self::Generic | Self::Protocol => self,
Self::Dynamic(_) | Self::Generic | Self::Protocol | Self::TypedDict => self,
}
}
@ -276,7 +277,10 @@ impl<'db> ClassBase<'db> {
.try_mro(db, specialization)
.is_err_and(MroError::is_cycle)
}
ClassBase::Dynamic(_) | ClassBase::Generic | ClassBase::Protocol => false,
ClassBase::Dynamic(_)
| ClassBase::Generic
| ClassBase::Protocol
| ClassBase::TypedDict => false,
}
}
@ -288,7 +292,9 @@ impl<'db> ClassBase<'db> {
) -> impl Iterator<Item = ClassBase<'db>> {
match self {
ClassBase::Protocol => ClassBaseMroIterator::length_3(db, self, ClassBase::Generic),
ClassBase::Dynamic(_) | ClassBase::Generic => ClassBaseMroIterator::length_2(db, self),
ClassBase::Dynamic(_) | ClassBase::Generic | ClassBase::TypedDict => {
ClassBaseMroIterator::length_2(db, self)
}
ClassBase::Class(class) => {
ClassBaseMroIterator::from_class(db, class, additional_specialization)
}
@ -309,6 +315,7 @@ impl<'db> From<ClassBase<'db>> for Type<'db> {
ClassBase::Class(class) => class.into(),
ClassBase::Protocol => Type::SpecialForm(SpecialFormType::Protocol),
ClassBase::Generic => Type::SpecialForm(SpecialFormType::Generic),
ClassBase::TypedDict => Type::SpecialForm(SpecialFormType::TypedDict),
}
}
}

6
crates/ty_python_semantic/src/types/infer.rs
View File

@ -7267,8 +7267,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
todo @ Type::Dynamic(
DynamicType::Todo(_)
| DynamicType::TodoPEP695ParamSpec
| DynamicType::TodoTypeAlias
| DynamicType::TodoTypedDict,
| DynamicType::TodoTypeAlias,
),
_,
_,
@ -7278,8 +7277,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
todo @ Type::Dynamic(
DynamicType::Todo(_)
| DynamicType::TodoPEP695ParamSpec
| DynamicType::TodoTypeAlias
| DynamicType::TodoTypedDict,
| DynamicType::TodoTypeAlias,
),
_,
) => Some(todo),

5
crates/ty_python_semantic/src/types/mro.rs
View File

@ -269,7 +269,10 @@ impl<'db> Mro<'db> {
continue;
}
match base {
ClassBase::Class(_) | ClassBase::Generic | ClassBase::Protocol => {
ClassBase::Class(_)
| ClassBase::Generic
| ClassBase::Protocol
| ClassBase::TypedDict => {
errors.push(DuplicateBaseError {
duplicate_base: base,
first_index: *first_index,

6
crates/ty_python_semantic/src/types/type_ordering.rs
View File

@ -166,6 +166,9 @@ pub(super) fn union_or_intersection_elements_ordering<'db>(
(ClassBase::Generic, _) => Ordering::Less,
(_, ClassBase::Generic) => Ordering::Greater,
(ClassBase::TypedDict, _) => Ordering::Less,
(_, ClassBase::TypedDict) => Ordering::Greater,
(ClassBase::Dynamic(left), ClassBase::Dynamic(right)) => {
dynamic_elements_ordering(left, right)
}
@ -257,9 +260,6 @@ fn dynamic_elements_ordering(left: DynamicType, right: DynamicType) -> Ordering
(DynamicType::TodoTypeAlias, _) => Ordering::Less,
(_, DynamicType::TodoTypeAlias) => Ordering::Greater,
(DynamicType::TodoTypedDict, _) => Ordering::Less,
(_, DynamicType::TodoTypedDict) => Ordering::Greater,
}
}