mirror of https://github.com/astral-sh/ruff synced 2026-01-21 21:40:51 -05:00

Files

Carl Meyer 30902497db [ty] Make signature return and parameter types non-optional (#22425 )

## Summary

Fixes https://github.com/astral-sh/ty/issues/2363
Fixes https://github.com/astral-sh/ty/issues/2013

And several other bugs with the same root cause. And makes any similar
bugs impossible by construction.

Previously we distinguished "no annotation" (Rust `None`) from
"explicitly annotated with something of type `Unknown`" (which is not an
error, and results in the annotation being of Rust type
`Some(Type::DynamicType(Unknown))`), even though semantically these
should be treated the same.

This was a bit of a bug magnet, because it was easy to forget to make
this `None` -> `Unknown` translation everywhere we needed to. And in
fact we did fail to do it in the case of materializing a callable,
leading to a top-materialized callable still having (rust) `None` return
type, which should have instead materialized to `object`.

This also fixes several other bugs related to not handling un-annotated
return types correctly:
1. We previously considered the return type of an unannotated `async
def` to be `Unknown`, where it should be `CoroutineType[Any, Any,
Unknown]`.
2. We previously failed to infer a ParamSpec if the return type of the
callable we are inferring against was not annotated.
3. We previously wrongly returned `Unknown` from `some_dict.get("key",
None)` if the value type of `some_dict` included a callable type with
un-annotated return type.

We now make signature return types and annotated parameter types
required, and we eagerly insert `Unknown` if there's no annotation. Most
of the diff is just a bunch of mechanical code changes where we
construct these types, and simplifications where we use them.

One exception is type display: when a callable type has un-annotated
parameters, we want to display them as un-annotated, but if it has a
parameter explicitly annotated with something of `Unknown` type, we want
to display that parameter as `x: Unknown` (it would be confusing if it
looked like your annotation just disappeared entirely).

Fortunately, we already have a mechanism in place for handling this: the
`inferred_annotation` flag, which suppresses display of an annotation.
Previously we used it only for `self` and `cls` parameters with an
inferred annotated type -- but we now also set it for any un-annotated
parameter, for which we infer `Unknown` type.

We also need to normalize `inferred_annotation`, since it's display-only
and shouldn't impact type equivalence. (This is technically a
previously-existing bug, it just never came up when it only affected
self types -- now it comes up because we have tests asserting that `def
f(x)` and `def g(x: Unknown)` are equivalent.)

## Test Plan

Added mdtests.

2026-01-07 09:18:39 -08:00

2.5 KiB

Raw Blame History

`async` / `await`

Basic

async def retrieve() -> int:
    return 42

async def main():
    result = await retrieve()

    reveal_type(result)  # revealed: int

Generic `async` functions

from typing import TypeVar

T = TypeVar("T")

async def persist(x: T) -> T:
    return x

async def f(x: int):
    result = await persist(x)

    reveal_type(result)  # revealed: int

Use cases

`Future`

import asyncio
import concurrent.futures

def blocking_function() -> int:
    return 42

async def main():
    loop = asyncio.get_event_loop()
    with concurrent.futures.ThreadPoolExecutor() as pool:
        result = await loop.run_in_executor(pool, blocking_function)
        reveal_type(result)  # revealed: int

`asyncio.Task`

import asyncio

async def f() -> int:
    return 1

async def main():
    task = asyncio.create_task(f())

    result = await task

    reveal_type(result)  # revealed: int

`asyncio.gather`

import asyncio

async def task(name: str) -> int:
    return len(name)

async def main():
    (a, b) = await asyncio.gather(
        task("A"),
        task("B"),
    )

    reveal_type(a)  # revealed: int
    reveal_type(b)  # revealed: int

Under the hood

[environment]
python-version = "3.12"  # Use 3.12 to be able to use PEP 695 generics

Let's look at the example from the beginning again:

async def retrieve() -> int:
    return 42

When we look at the signature of this function, we see that it actually returns a CoroutineType:

reveal_type(retrieve)  # revealed: def retrieve() -> CoroutineType[Any, Any, int]

The expression await retrieve() desugars into a call to the __await__ dunder method on the CoroutineType object, followed by a yield from. Let's first see the return type of __await__:

reveal_type(retrieve().__await__())  # revealed: Generator[Any, None, int]

We can see that this returns a Generator that yields Any, and eventually returns int. For the final type of the await expression, we retrieve that third argument of the Generator type:

from typing import Generator

def _():
    result = yield from retrieve().__await__()
    reveal_type(result)  # revealed: int

Un-annotated async functions

An async def with no annotated return type is still known to return CoroutineType of Unknown, not just Unknown:

async def f():
    pass

reveal_type(f())  # revealed: CoroutineType[Any, Any, Unknown]

2.5 KiB Raw Blame History

async / await