## Summary
Marks fixes as unsafe when they change return types (`None` → `Path`,
`str`/`bytes` → `Path`, `str` → `Path`), except when the call is a
top-level expression.
Fixes#21431.
## Problem
Fixes for `os.rename`, `os.replace`, `os.getcwd`/`os.getcwdb`, and
`os.readlink` were marked safe despite changing return types, which can
break code that uses the return value.
## Approach
Added `is_top_level_expression_call` helper to detect when a call is a
top-level expression (return value unused). Updated
`check_os_pathlib_two_arg_calls` and `check_os_pathlib_single_arg_calls`
to mark fixes as unsafe unless the call is a top-level expression.
Updated PTH109 to use the helper for applicability determination.
## Test Plan
Updated snapshots for `preview_full_name.py`, `preview_import_as.py`,
`preview_import_from.py`, and `preview_import_from_as.py` to reflect
unsafe markers.
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
Previously, the code action to do auto-import on a pre-existing symbol
assumed that the auto-importer would always generate an import
statement. But sometimes an import statement already exists.
A good example of this is the following snippet:
```
import warnings
@deprecated
def myfunc(): pass
```
Specifically, `deprecated` exists in `warnings` but isn't currently
imported. A code action to fix this could feasibly do two
transformations here. One is:
```
import warnings
@warnings.deprecated
def myfunc(): pass
```
Another is:
```
from warnings import deprecated
import warnings
@deprecated
def myfunc(): pass
```
The existing auto-import infrastructure chooses the former, since it
reuses a pre-existing import statement. But this PR chooses the latter
for the case of a code action. I'm not 100% sure this is the correct
choice, but it seems to defer more strongly to what the user has typed.
That is, that they want to use it unqualified because it's what has been
typed. So we should add the necessary import statement to make that
work.
Fixesastral-sh/ty#1668
This works by adding a third module resolution mode that lets the caller
opt into _some_ shadowing of modules that is otherwise not allowed (for
`typing` and `typing_extensions`).
Fixesastral-sh/ty#1658
## Summary
If you manage to create an `typing.GenericAlias` instance without us
knowing how that was created, then we don't know what to do with this in
a type annotation. So it's better to be explicit and show an error
instead of failing silently with a `@Todo` type.
## Test Plan
* New Markdown tests
* Zero ecosystem impact
## Summary
We had tests for this already, but they used generic classes that were
bivariant in their type parameter, and so this case wasn't captured.
closes https://github.com/astral-sh/ty/issues/1702
## Test Plan
Updated Markdown tests
## Summary
These projects from `mypy_primer` were missing from both `good.txt` and
`bad.txt` for some reason. I thought about writing a script that would
verify that `good.txt` + `bad.txt` = `mypy_primer.projects`, but that's
not completely trivial since there are projects like `cpython` only
appear once in `good.txt`. Given that we can hopefully soon get rid of
both of these files (and always run on all projects), it's probably not
worth the effort. We are usually notified of all `mypy_primer` changes.
## Test Plan
CI on this PR
## Summary
The exact behavior around what's allowed vs. disallowed was partly
detected through trial and error in the runtime.
I was a little confused by [this
comment](https://github.com/python/cpython/pull/129352) that says
"`NamedTuple` subclasses cannot be inherited from" because in practice
that doesn't appear to error at runtime.
Closes [#1683](https://github.com/astral-sh/ty/issues/1683).
## Summary
This is another small refactor for
https://github.com/astral-sh/ruff/pull/21445 that splits the single
`paramspec.md` into `generics/legacy/paramspec.md` and
`generics/pep695/paramspec.md`.
## Test Plan
Make sure that all mdtests pass.
## Summary
Add support for generic PEP 613 type aliases and generic implicit type
aliases:
```py
from typing import TypeVar
T = TypeVar("T")
ListOrSet = list[T] | set[T]
def _(xs: ListOrSet[int]):
reveal_type(xs) # list[int] | set[int]
```
closes https://github.com/astral-sh/ty/issues/1643
closes https://github.com/astral-sh/ty/issues/1629
closes https://github.com/astral-sh/ty/issues/1596
closes https://github.com/astral-sh/ty/issues/573
closes https://github.com/astral-sh/ty/issues/221
## Typing conformance
```diff
-aliases_explicit.py:52:5: error[type-assertion-failure] Type `list[int]` does not match asserted type `@Todo(specialized generic alias in type expression)`
-aliases_explicit.py:53:5: error[type-assertion-failure] Type `tuple[str, ...] | list[str]` does not match asserted type `@Todo(Generic specialization of types.UnionType)`
-aliases_explicit.py:54:5: error[type-assertion-failure] Type `tuple[int, int, int, str]` does not match asserted type `@Todo(specialized generic alias in type expression)`
-aliases_explicit.py:56:5: error[type-assertion-failure] Type `(int, str, /) -> str` does not match asserted type `@Todo(Generic specialization of typing.Callable)`
-aliases_explicit.py:59:5: error[type-assertion-failure] Type `int | str | None | list[list[int]]` does not match asserted type `int | str | None | list[@Todo(specialized generic alias in type expression)]`
```
New true negatives ✔️
```diff
+aliases_explicit.py:41:36: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
-aliases_explicit.py:57:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `@Todo(Generic specialization of typing.Callable)`
+aliases_explicit.py:57:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `(...) -> Unknown`
```
These require `ParamSpec`
```diff
+aliases_explicit.py:67:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
+aliases_explicit.py:68:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
+aliases_explicit.py:69:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
+aliases_explicit.py:70:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
+aliases_explicit.py:71:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
+aliases_explicit.py:102:20: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
```
New true positives ✔️
```diff
-aliases_implicit.py:63:5: error[type-assertion-failure] Type `list[int]` does not match asserted type `@Todo(specialized generic alias in type expression)`
-aliases_implicit.py:64:5: error[type-assertion-failure] Type `tuple[str, ...] | list[str]` does not match asserted type `@Todo(Generic specialization of types.UnionType)`
-aliases_implicit.py:65:5: error[type-assertion-failure] Type `tuple[int, int, int, str]` does not match asserted type `@Todo(specialized generic alias in type expression)`
-aliases_implicit.py:67:5: error[type-assertion-failure] Type `(int, str, /) -> str` does not match asserted type `@Todo(Generic specialization of typing.Callable)`
-aliases_implicit.py:70:5: error[type-assertion-failure] Type `int | str | None | list[list[int]]` does not match asserted type `int | str | None | list[@Todo(specialized generic alias in type expression)]`
-aliases_implicit.py:71:5: error[type-assertion-failure] Type `list[bool]` does not match asserted type `@Todo(specialized generic alias in type expression)`
```
New true negatives ✔️
```diff
+aliases_implicit.py:54:36: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
-aliases_implicit.py:68:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `@Todo(Generic specialization of typing.Callable)`
+aliases_implicit.py:68:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `(...) -> Unknown`
```
These require `ParamSpec`
```diff
+aliases_implicit.py:76:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
+aliases_implicit.py:77:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
+aliases_implicit.py:78:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
+aliases_implicit.py:79:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
+aliases_implicit.py:80:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
+aliases_implicit.py:81:25: error[invalid-type-arguments] Type `str` is not assignable to upper bound `int | float` of type variable `TFloat@GoodTypeAlias12`
+aliases_implicit.py:135:20: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
```
New true positives ✔️
```diff
+callables_annotation.py:172:19: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
+callables_annotation.py:175:19: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
+callables_annotation.py:188:25: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
+callables_annotation.py:189:25: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
```
These require `ParamSpec` and `Concatenate`.
```diff
-generics_defaults_specialization.py:26:5: error[type-assertion-failure] Type `SomethingWithNoDefaults[int, str]` does not match asserted type `SomethingWithNoDefaults[int, typing.TypeVar]`
+generics_defaults_specialization.py:26:5: error[type-assertion-failure] Type `SomethingWithNoDefaults[int, str]` does not match asserted type `SomethingWithNoDefaults[int, DefaultStrT]`
```
Favorable diagnostic change ✔️
```diff
-generics_defaults_specialization.py:27:5: error[type-assertion-failure] Type `SomethingWithNoDefaults[int, bool]` does not match asserted type `@Todo(specialized generic alias in type expression)`
```
New true negative ✔️
```diff
-generics_defaults_specialization.py:30:1: error[non-subscriptable] Cannot subscript object of type `<class 'SomethingWithNoDefaults[int, typing.TypeVar]'>` with no `__class_getitem__` method
+generics_defaults_specialization.py:30:15: error[invalid-type-arguments] Too many type arguments: expected between 0 and 1, got 2
```
Correct new diagnostic ✔️
```diff
-generics_variance.py:175:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:175:35: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:179:29: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:179:39: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:183:21: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:183:27: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:187:25: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:187:31: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:191:33: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:191:43: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:191:49: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:196:5: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:196:15: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
-generics_variance.py:196:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
```
One of these should apparently be an error, but not of this kind, so
this is good ✔️
```diff
-specialtypes_type.py:152:16: error[invalid-type-form] `typing.TypeVar` is not a generic class
-specialtypes_type.py:156:16: error[invalid-type-form] `typing.TypeVar` is not a generic class
```
Good, those were false positives. ✔️
I skipped the analysis for everything involving `TypeVarTuple`.
## Ecosystem impact
**[Full report with detailed
diff](https://david-generic-implicit-alias.ecosystem-663.pages.dev/diff)**
Previous iterations of this PR showed all kinds of problems. In it's
current state, I do not see any large systematic problems, but it is
hard to tell with 5k diagnostic changes.
## Performance
* There is a huge 4x regression in `colour-science/colour`, related to
[this large
file](https://github.com/colour-science/colour/blob/develop/colour/io/luts/tests/test_lut.py)
with [many assignments of hard-coded arrays (lists of lists) to
`np.NDArray`
types](83e754c8b6/colour/io/luts/tests/test_lut.py (L701-L781))
that we now understand. We now take ~2 seconds to check this file, so
definitely not great, but maybe acceptable for now.
## Test Plan
Updated and new Markdown tests
## Summary
This is a bugfix for subtyping of `type[Any]` / `type[T]` and protocols.
## Test Plan
Regression test that will only be really meaningful once
https://github.com/astral-sh/ruff/pull/21553 lands.
## Summary
**This is the final goto-targets with missing
goto-definition/declaration implementations!
You can now theoretically click on all the user-defined names in all the
syntax. 🎉**
This adds:
* goto definition/declaration on patterns/typevars
* find-references/rename on patterns/typevars
* fixes syntax highlighting of `*rest` patterns
This notably *does not* add:
* goto-type for patterns/typevars
* hover for patterns/typevars (because that's just goto-type for names)
Also I realized we were at the precipice of one of the great GotoTarget
sins being resolved, and so I made import aliases also resolve to a
ResolvedDefinition. This removes a ton of cruft and prevents further
backsliding.
Note however that import aliases are, in general, completely jacked up
when it comes to find-references/renames (both before and after this
PR). Previously you could try to rename an import alias and it just
wouldn't do anything. With this change we instead refuse to even let you
try to rename it.
Sorting out why import aliases are jacked up is an ongoing thing I hope
to handle in a followup.
## Test Plan
You'll surely not regret checking in 86 snapshot tests
## Summary
* Fixes https://github.com/astral-sh/ty/issues/1650
* Part of https://github.com/astral-sh/ty/issues/1610
We now handle:
* `.. warning::` (and friends) by bolding the line and rendering the
block as normal (non-code) text
* `.. code::` (and friends) by treating it the same as `::` (fully
deleted if seen, introduce a code block)
* `.. code:: lang` (and friends) by letting it set the language on the
codefence
* `.. versionchanged:: 1.2.3` (and friends) by rendering it like
`warning` but with the version included and italicized
* `.. dsfsdf-unknown:: (lang)` by assuming it's the same as `.. code::
(lang)`
## Test Plan
Snapshots added/updated. I also deleted a bunch of useless checks on
plaintext rendering. It's important for some edge-case tests but not for
the vast majority of tests.
## Summary
This PR adds a new `db` parameter to `Parameters::new` for
https://github.com/astral-sh/ruff/pull/21445. This change creates a
large diff so thought to split it out as it's just a mechanical change.
The `Parameters::new` method not only creates the `Parameters` but also
analyses the parameters to check what kind it is. For `ParamSpec`
support, it's going to require the `db` to check whether the annotated
type is `ParamSpec` or not. For the current set of parameters that isn't
required because it's only checking whether it's dynamic or not which
doesn't require `db`.
## Summary
Pulls in an ecosystem-analyzer change with a few updates to the diff
report:
* Breakdown of added/removed/changed diagnostics by project
* Option to filter diagnostics by project
* Small button to copy a file path to the clipboard
* `(-R +A ~C)` indicators in the filter dropdowns (removed, added,
changed)
* More concise layout, less scrolling
## Test Plan
Tested on https://github.com/astral-sh/ruff/pull/21553 =>
https://david-generic-implicit-alias.ecosystem-663.pages.dev/diff
## Summary
Originally I planned to feed this in as a `fix` but I realized that we
probably don't want to be trying to resolve import suggestions while
we're doing type inference. Thus I implemented this as a fallback when
there's no fixes on a diagnostic, which can use the full lsp machinery.
Fixes https://github.com/astral-sh/ty/issues/1552
## Test Plan
Works in the IDE, added some e2e tests.
## Summary
Previously if an explicit specialization failed (e.g. wrong number of
type arguments or violates an upper bound) we just inferred `Unknown`
for the entire type. This actually caused us to panic on an a case of a
recursive upper bound with invalid specialization; the upper bound would
oscillate indefinitely in fixpoint iteration between `Unknown` and the
given specialization. This could be fixed with a cycle recovery
function, but in this case there's a simpler fix: if we infer
`C[Unknown]` instead of `Unknown` for an invalid attempt to specialize
`C`, that allows fixpoint iteration to quickly converge, as well as
giving a more precise type inference.
Other type checkers actually just go with the attempted specialization
even if it's invalid. So if `C` has a type parameter with upper bound
`int`, and you say `C[str]`, they'll emit a diagnostic but just go with
`C[str]`. Even weirder, if `C` has a single type parameter and you say
`C[str, bytes]`, they'll just go with `C[str]` as the type. I'm not
convinced by this approach; it seems odd to have specializations
floating around that explicitly violate the declared upper bound, or in
the latter case aren't even the specialization the annotation requested.
I prefer `C[Unknown]` for this case.
Fixing this revealed an issue with `collections.namedtuple`, which
returns `type[tuple[Any, ...]]`. Due to
https://github.com/astral-sh/ty/issues/1649 we consider that to be an
invalid specialization. So previously we returned `Unknown`; after this
PR it would be `type[tuple[Unknown]]`, leading to more false positives
from our lack of functional namedtuple support. To avoid that I added an
explicit Todo type for functional namedtuples for now.
## Test Plan
Added and updated mdtests.
The conformance suite changes have to do with `ParamSpec`, so no
meaningful signal there.
The ecosystem changes appear to be the expected effects of having more
precise type information (including occurrences of known issues such as
https://github.com/astral-sh/ty/issues/1495 ). Most effects are just
changes to types in diagnostics.
Dan Parizher
Andrew Gallant
William Woodruff
Micha Reiser
Alex Waygood
David Peter
Carl Meyer
Kieran Ryan
Shunsuke Shibayama
renovate[bot]
Charlie Marsh
RasmusNygren
Dhruv Manilawala
Dylan
Matthew Mckee
Aria Desires
Ibraheem Ahmed
Shahar Naveh
Tsvika Shapira