mirror of https://github.com/astral-sh/ruff synced 2026-01-23 06:20:55 -05:00

Files

Carl Meyer 192c37d540 [ty] tighten up handling of subscripts in type expressions (#21503 )

## Summary

Get rid of the catch-all todo type from subscripting a base type we
haven't implemented handling for yet in a type expression, and turn it
into a diagnostic instead.

Handle a few more cases explicitly, to avoid false positives from the
above change:
1. Subscripting any dynamic type (not just a todo type) in a type
expression should just result in that same dynamic type. This is
important for gradual guarantee, and matches other type checkers.
2. Subscripting a generic alias may be an error or not, depending
whether the specialization itself contains typevars. Don't try to handle
this yet (it should be handled in a later PR for specializing generic
non-PEP695 type aliases), just use a dedicated todo type for it.
3. Add a temporary todo branch to avoid false positives from string PEP
613 type aliases. This can be removed in the next PR, with PEP 613 type
alias support.

## Test Plan

Adjusted mdtests, ecosystem.

All new diagnostics in conformance suite are supposed to be diagnostics,
so this PR is a strict improvement there.

New diagnostics in the ecosystem are surfacing cases where we already
don't understand an annotation, but now we emit a diagnostic about it.
They are mostly intentional choices. Analysis of particular cases:

* `attrs`, `bokeh`, `django-stubs`, `dulwich`, `ibis`, `kornia`,
`mitmproxy`, `mongo-python-driver`, `mypy`, `pandas`, `poetry`,
`prefect`, `pydantic`, `pytest`, `scrapy`, `trio`, `werkzeug`, and
`xarray` are all cases where under `from __future__ import annotations`
or Python 3.14 deferred-annotations semantics, we follow normal
name-scoping rules, whereas some other type checkers prefer global names
over local names. This means we don't like it if e.g. you have a class
with a method or attribute named `type` or `tuple`, and you also try to
use `type` or `tuple` in method/attribute annotations of that class.
This PR isn't changing those semantics, just revealing them in more
cases where previously we just silently fell back to `Unknown`. I think
failing with a diagnostic (so authors can alias names as needed to avoid
relying on scoping rules that differ between type checkers) is better
than failing silently here.
* `beartype` assumes we support `TypeForm` (because it only supports
mypy and pyright, it uses `if MYPY:` to hide the `TypeForm` from mypy,
and pyright supports `TypeForm`), and we don't yet.
* `graphql-core` likes to use a `try: ... except ImportError: ...`
pattern for importing special forms from `typing` with fallback to
`typing_extensions`, instead of using `sys.version_info` checks. We
don't handle this well when type checking under an older Python version
(where the import from `typing` is not found); we see the imported name
as of type e.g. `Unknown | SpecialFormType(...)`, and because of the
union with `Unknown` we fail to handle it as the special form type. Mypy
and pyright also don't seem to support this pattern. They don't complain
about subscripting such special forms, but they do silently fail to
treat them as the desired special form. Again here, if we are going to
fail I'd rather fail with a diagnostic rather than silently.
* `ibis` is [trying to
use](https://github.com/ibis-project/ibis/blob/main/ibis/common/collections.py#L372)
`frozendict: type[FrozenDict]` as a way to create a "type alias" to
`FrozenDict`, but this is wrong: that means `frozendict:
type[FrozenDict[Any, Any]]`.
* `mypy` has some errors due to the fact that type-checking `typing.pyi`
itself (without knowing that it's the real `typing.pyi`) doesn't work
very well.
* `mypy-protobuf` imports some types from the protobufs library that end
up unioned with `Unknown` for some reason, and so we don't allow
explicit-specialization of them. Depending on the reason they end up
unioned with `Unknown`, we might want to better support this? But it's
orthogonal to this PR -- we aren't failing any worse here, just alerting
the author that we didn't understand their annotation.
* `pwndbg` has unresolved references due to star-importing from a
dependency that isn't installed, and uses un-imported names like `Dict`
in annotation expressions. Some of the unresolved references were hidden
by
https://github.com/astral-sh/ruff/blob/main/crates/ty_python_semantic/src/types/infer/builder.rs#L7223-L7228
when some annotations previously resolved to a Todo type that no longer
do.

2025-11-18 10:43:07 -08:00

9.8 KiB

Raw Blame History

Literal

https://typing.python.org/en/latest/spec/literal.html#literals

Parameterization

from typing import Literal
from enum import Enum

mode: Literal["w", "r"]
a1: Literal[26]
a2: Literal[0x1A]
a3: Literal[-4]
a4: Literal["hello world"]
a5: Literal[b"hello world"]
a6: Literal[True]
a7: Literal[None]
a8: Literal[Literal[1]]

class Color(Enum):
    RED = 0
    GREEN = 1
    BLUE = 2

b1: Literal[Color.RED]

MissingT = Enum("MissingT", {"MISSING": "MISSING"})
b2: Literal[MissingT.MISSING]

def f():
    reveal_type(mode)  # revealed: Literal["w", "r"]
    reveal_type(a1)  # revealed: Literal[26]
    reveal_type(a2)  # revealed: Literal[26]
    reveal_type(a3)  # revealed: Literal[-4]
    reveal_type(a4)  # revealed: Literal["hello world"]
    reveal_type(a5)  # revealed: Literal[b"hello world"]
    reveal_type(a6)  # revealed: Literal[True]
    reveal_type(a7)  # revealed: None
    reveal_type(a8)  # revealed: Literal[1]
    reveal_type(b1)  # revealed: Literal[Color.RED]
    # TODO should be `Literal[MissingT.MISSING]`
    reveal_type(b2)  # revealed: @Todo(functional `Enum` syntax)

# error: [invalid-type-form]
invalid1: Literal[3 + 4]
# error: [invalid-type-form]
invalid2: Literal[4 + 3j]
# error: [invalid-type-form]
invalid3: Literal[(3, 4)]

hello = "hello"
invalid4: Literal[
    1 + 2,  # error: [invalid-type-form]
    "foo",
    hello,  # error: [invalid-type-form]
    (1, 2, 3),  # error: [invalid-type-form]
]

class NotAnEnum:
    x: int = 1

# error: [invalid-type-form]
invalid5: Literal[NotAnEnum.x]

a_list: list[int] = [1, 2, 3]
# error: [invalid-type-form]
invalid6: Literal[a_list[0]]

Parameterizing with a type alias

typing.Literal can also be parameterized with a type alias for any literal type or union of literal types.

PEP 695 type alias

[environment]
python-version = "3.12"

from typing import Literal
from enum import Enum

import mod

class E(Enum):
    A = 1
    B = 2

type SingleInt = Literal[1]
type SingleStr = Literal["foo"]
type SingleBytes = Literal[b"bar"]
type SingleBool = Literal[True]
type SingleNone = Literal[None]
type SingleEnum = Literal[E.A]
type UnionLiterals = Literal[1, "foo", b"bar", True, None, E.A]
# We support this because it is an equivalent type to the following union of literals, but maybe
# we should not, because it doesn't use `Literal` form? Other type checkers do not.
type AnEnum1 = E
type AnEnum2 = Literal[E.A, E.B]
# Similarly, we support this because it is equivalent to `Literal[True, False]`.
type Bool1 = bool
type Bool2 = Literal[True, False]

def _(
    single_int: Literal[SingleInt],
    single_str: Literal[SingleStr],
    single_bytes: Literal[SingleBytes],
    single_bool: Literal[SingleBool],
    single_none: Literal[SingleNone],
    single_enum: Literal[SingleEnum],
    union_literals: Literal[UnionLiterals],
    an_enum1: Literal[AnEnum1],
    an_enum2: Literal[AnEnum2],
    bool1: Literal[Bool1],
    bool2: Literal[Bool2],
    multiple: Literal[SingleInt, SingleStr, SingleEnum],
    single_int_other_module: Literal[mod.SingleInt],
):
    reveal_type(single_int)  # revealed: Literal[1]
    reveal_type(single_str)  # revealed: Literal["foo"]
    reveal_type(single_bytes)  # revealed: Literal[b"bar"]
    reveal_type(single_bool)  # revealed: Literal[True]
    reveal_type(single_none)  # revealed: None
    reveal_type(single_enum)  # revealed: Literal[E.A]
    reveal_type(union_literals)  # revealed: Literal[1, "foo", b"bar", True, E.A] | None
    reveal_type(an_enum1)  # revealed: E
    reveal_type(an_enum2)  # revealed: E
    reveal_type(bool1)  # revealed: bool
    reveal_type(bool2)  # revealed: bool
    reveal_type(multiple)  # revealed: Literal[1, "foo", E.A]
    reveal_type(single_int_other_module)  # revealed: Literal[2]

mod.py:

from typing import Literal

type SingleInt = Literal[2]

PEP 613 type alias

from typing import Literal, TypeAlias
from enum import Enum

class E(Enum):
    A = 1
    B = 2

SingleInt: TypeAlias = Literal[1]
SingleStr: TypeAlias = Literal["foo"]
SingleBytes: TypeAlias = Literal[b"bar"]
SingleBool: TypeAlias = Literal[True]
SingleNone: TypeAlias = Literal[None]
SingleEnum: TypeAlias = Literal[E.A]
UnionLiterals: TypeAlias = Literal[1, "foo", b"bar", True, None, E.A]
AnEnum1: TypeAlias = E
AnEnum2: TypeAlias = Literal[E.A, E.B]
Bool1: TypeAlias = bool
Bool2: TypeAlias = Literal[True, False]

def _(
    single_int: Literal[SingleInt],
    single_str: Literal[SingleStr],
    single_bytes: Literal[SingleBytes],
    single_bool: Literal[SingleBool],
    single_none: Literal[SingleNone],
    single_enum: Literal[SingleEnum],
    union_literals: Literal[UnionLiterals],
    # Could also not error
    an_enum1: Literal[AnEnum1],  # error: [invalid-type-form]
    an_enum2: Literal[AnEnum2],
    # Could also not error
    bool1: Literal[Bool1],  # error: [invalid-type-form]
    bool2: Literal[Bool2],
    multiple: Literal[SingleInt, SingleStr, SingleEnum],
):
    reveal_type(single_int)  # revealed: Literal[1]
    reveal_type(single_str)  # revealed: Literal["foo"]
    reveal_type(single_bytes)  # revealed: Literal[b"bar"]
    reveal_type(single_bool)  # revealed: Literal[True]
    reveal_type(single_none)  # revealed: None
    reveal_type(single_enum)  # revealed: Literal[E.A]
    reveal_type(union_literals)  # revealed: Literal[1, "foo", b"bar", True, E.A] | None
    # Could also be `E`
    reveal_type(an_enum1)  # revealed: Unknown
    reveal_type(an_enum2)  # revealed: E
    # Could also be `bool`
    reveal_type(bool1)  # revealed: Unknown
    reveal_type(bool2)  # revealed: bool
    reveal_type(multiple)  # revealed: Literal[1, "foo", E.A]

Implicit type alias

from typing import Literal
from enum import Enum

class E(Enum):
    A = 1
    B = 2

SingleInt = Literal[1]
SingleStr = Literal["foo"]
SingleBytes = Literal[b"bar"]
SingleBool = Literal[True]
SingleNone = Literal[None]
SingleEnum = Literal[E.A]
UnionLiterals = Literal[1, "foo", b"bar", True, None, E.A]
# For implicit type aliases, we may not want to support this. It's simpler not to, and no other
# type checker does.
AnEnum1 = E
AnEnum2 = Literal[E.A, E.B]
# For implicit type aliases, we may not want to support this.
Bool1 = bool
Bool2 = Literal[True, False]

def _(
    single_int: Literal[SingleInt],
    single_str: Literal[SingleStr],
    single_bytes: Literal[SingleBytes],
    single_bool: Literal[SingleBool],
    single_none: Literal[SingleNone],
    single_enum: Literal[SingleEnum],
    union_literals: Literal[UnionLiterals],
    an_enum1: Literal[AnEnum1],  # error: [invalid-type-form]
    an_enum2: Literal[AnEnum2],
    bool1: Literal[Bool1],  # error: [invalid-type-form]
    bool2: Literal[Bool2],
    multiple: Literal[SingleInt, SingleStr, SingleEnum],
):
    reveal_type(single_int)  # revealed: Literal[1]
    reveal_type(single_str)  # revealed: Literal["foo"]
    reveal_type(single_bytes)  # revealed: Literal[b"bar"]
    reveal_type(single_bool)  # revealed: Literal[True]
    reveal_type(single_none)  # revealed: None
    reveal_type(single_enum)  # revealed: Literal[E.A]
    reveal_type(union_literals)  # revealed: Literal[1, "foo", b"bar", True, E.A] | None
    reveal_type(an_enum1)  # revealed: Unknown
    reveal_type(an_enum2)  # revealed: E
    reveal_type(bool1)  # revealed: Unknown
    reveal_type(bool2)  # revealed: bool
    reveal_type(multiple)  # revealed: Literal[1, "foo", E.A]

Shortening unions of literals

When a Literal is parameterized with more than one value, it’s treated as exactly to equivalent to the union of those types.

from typing import Literal

def x(
    a1: Literal[Literal[Literal[1, 2, 3], "foo"], 5, None],
    a2: Literal["w"] | Literal["r"],
    a3: Literal[Literal["w"], Literal["r"], Literal[Literal["w+"]]],
    a4: Literal[True] | Literal[1, 2] | Literal["foo"],
):
    reveal_type(a1)  # revealed: Literal[1, 2, 3, 5, "foo"] | None
    reveal_type(a2)  # revealed: Literal["w", "r"]
    reveal_type(a3)  # revealed: Literal["w", "r", "w+"]
    reveal_type(a4)  # revealed: Literal[True, 1, 2, "foo"]

Display of heterogeneous unions of literals

from typing import Literal, Union

def foo(x: int) -> int:
    return x + 1

def bar(s: str) -> str:
    return s

class A: ...
class B: ...

def union_example(
    x: Union[
        # unknown type
        # error: [unresolved-reference]
        y,
        Literal[-1],
        Literal["A"],
        Literal[b"A"],
        Literal[b"\x00"],
        Literal[b"\x07"],
        Literal[0],
        Literal[1],
        Literal["B"],
        Literal["foo"],
        Literal["bar"],
        Literal["B"],
        Literal[True],
        None,
    ],
):
    reveal_type(x)  # revealed: Unknown | Literal[-1, 0, 1, "A", "B", "foo", "bar", b"A", b"\x00", b"\x07", True] | None

Detecting Literal outside typing and typing_extensions

Only Literal that is defined in typing and typing_extension modules is detected as the special Literal.

other.pyi:

from typing import _SpecialForm

Literal: _SpecialForm

from other import Literal

# TODO: can we add a subdiagnostic here saying something like:
#
#     `other.Literal` and `typing.Literal` have similar names, but are different symbols and don't have the same semantics
#
# ?
#
# error: [invalid-type-form] "Int literals are not allowed in this context in a type expression"
# error: [invalid-type-form] "Invalid subscript of object of type `_SpecialForm` in type expression"
a1: Literal[26]

def f():
    reveal_type(a1)  # revealed: Unknown

Detecting typing_extensions.Literal

from typing_extensions import Literal

a1: Literal[26]

def f():
    reveal_type(a1)  # revealed: Literal[26]

Invalid

from typing import Literal

# error: [invalid-type-form] "`typing.Literal` requires at least one argument when used in a type expression"
def _(x: Literal):
    reveal_type(x)  # revealed: Unknown

9.8 KiB Raw Blame History Unescape Escape