The implementation here is (to me) surprisingly complicated. The main
complications are:
1. Trying to limit the redundant detection to as few of the symbols
we extract as possible. In particular, while I haven't done benchmarking
on this, I perceive the redundancy detection to be somewhat expensive
and auto-import can return lots of symbols. So we're careful to only do
this extra checking on (typically) small groups of symbols that could
possibly be merged.
2. Even by restricting our work, this merging process could still be
called quite a bit. (Thousands of times in my "standard data scienc-y
test environment.") So I went out of my way to amortize allocs.
3. Re-exports can form a chain and we want to find all of them.
4. We (probably) don't want to remove redundant re-exports unless
they share the same top-level module. Otherwise, e.g., a library
that re-exports another library's symbols could have all of its
re-exports dropped.
5. We want to only keep the top-most re-exports, and there may be
multiple such re-exports. So we keep all of them.
6. We can't assume anything about the relationship of re-exports
and the original definition. A re-export could be deeper in the
module hierarchy than the original definition or above it.
Previously, we were constructing this at a higher level layer. But this
commit pushes it down a layer of abstraction. This shouldn't result in
constructing the fully qualified name any more frequently than we
previously did. Namely, we're still only doing it for symbols that match
the caller provided search query.
The motivation here is that we want to do some de-duplication based on
module name, and having the fully qualified name of a symbol seems like
the most straight-forward way to go about this. (We'll need one more
piece of information that we add in a subsequent commit.)
## Summary
Prefer completions of type `ClassLiteral` inside class-def context.
Fixes https://github.com/astral-sh/ty/issues/1776
## Test Plan
New completion-eval test that has incorrect ranking on main but correct
after this patch.
## Summary
This PR adds support for 'dangling' `type(...)` constructors, e.g.:
```python
class Foo(type("Bar", ...)):
...
```
As opposed to:
```python
Bar = type("Bar", ...)
```
The former doesn't have a `Definition` since it doesn't get bound to a
place, so we instead need to store the `NodeIndex`. Per @MichaReiser's
suggestion, we can use a Salsa tracked struct for this.
## Summary
This PR adds support for dynamic classes created via `type()`. The core
of the change is that `ClassLiteral` is now an enum:
```rust
pub enum ClassLiteral<'db> {
/// A class defined via a `class` statement.
Stmt(StmtClassLiteral<'db>),
/// A class created via the functional form `type(name, bases, dict)`.
Functional(FunctionalClassLiteral<'db>),
}
```
And, in turn, various methods on `ClassLiteral` like `body_scope` now
return `Option` or similar (and callers must adjust to that change in
signature).
Over time, we can expand the enum to include functional namedtuples,
etc. (I already have this working in a separate branch, and I believe it
slots in well.)
(I'd love help with the names -- I think `StmtClassLiteral` is kind of
lame. Maybe `DeclarativeClassLiteral`?)
Closes https://github.com/astral-sh/ty/issues/740.
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
## Summary
The type inference system already correctly special-cases `__file__` to
return `str` for the current module (since the code is executing from an
existing file). However, the completion system was bypassing this logic
and pulling `__file__: str | None` directly from `types.ModuleType` in
typeshed.
This PR adds implicit module globals (like `__file__`, `__name__`, etc.)
with their correctly-typed values to completions, reusing the existing
`module_type_implicit_global_symbol` function that already handles the
special-casing.
Closes https://github.com/astral-sh/ty/issues/2445.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This makes it so, e.g., `os<CURSOR>` will suggest the top-level stdlib
`os` module even if there is an `os` symbol elsewhere in your project.
The way this is done is somewhat overwrought, but it's done to avoid
suggesting top-level modules over other symbols already in scope.
Fixesastral-sh/issues#1852
Part of this was already done, but it was half-assed. We now look at the
search path that a symbol came from and centralize a symbol's origin
classification.
The preference ordering here is maybe not the right one, but we can
iterate as users give us feedback. Note also that the preference
ordering based on the origin is pretty low in the relevance sorting.
This means that other more specific criteria will and can override this.
This results in some nice improvements to our evaluation tasks.
Previously, we would only sort by name and file path (unless the symbol
refers to an entire module). This led to somewhat confusing ordering,
since the file path is absolute and can be anything.
Sorting by module name after symbol name gives a more predictable
ordering in common practice.
This is mostly just an improvement for debugging purposes, i.e., when
looking at the output of `all_symbols` directly. This mostly shouldn't
impact completion ordering since completions do their own ranking.
This moves the information we want to use to rank completions into
`Completion` itself. (This was the primary motivation for creating a
`CompletionBuilder` in the first place.)
The principal advantage here is that we now only need to compute the
relevance information for each completion exactly once. Previously, we
were computing it on every comparison, which might end up doing
redundant work for a not insignifcant number of completions.
The relevance information is also specifically constructed from the
builder so that, in the future, we might choose to short-circuit
construction if we know we'll never send it back to the client (e.g.,
its ranking is worse than the lowest ranked completion). But we don't
implement that optimization yet.
I want to be able to attach extra data to each `Completion`, but not
burden callers with the need to construct it. This commit helps get us
to that point by requiring callers to use a `CompletionBuilder` for
construction instead of a `Completion` itself.
I think this will also help in the future if it proves to be the case
that we can improve performance by delaying work until we actually build
a `Completion`, which might only happen if we know we won't throw it
out. But we aren't quite there yet.
This also lets us tighten things up a little bit and makes completion
construction less noisy. The downside is that callers no longer need to
consider "every" completion field.
There should not be any behavior changes here.
## Summary
Fixes https://github.com/astral-sh/ty/issues/2363
Fixes https://github.com/astral-sh/ty/issues/2013
And several other bugs with the same root cause. And makes any similar
bugs impossible by construction.
Previously we distinguished "no annotation" (Rust `None`) from
"explicitly annotated with something of type `Unknown`" (which is not an
error, and results in the annotation being of Rust type
`Some(Type::DynamicType(Unknown))`), even though semantically these
should be treated the same.
This was a bit of a bug magnet, because it was easy to forget to make
this `None` -> `Unknown` translation everywhere we needed to. And in
fact we did fail to do it in the case of materializing a callable,
leading to a top-materialized callable still having (rust) `None` return
type, which should have instead materialized to `object`.
This also fixes several other bugs related to not handling un-annotated
return types correctly:
1. We previously considered the return type of an unannotated `async
def` to be `Unknown`, where it should be `CoroutineType[Any, Any,
Unknown]`.
2. We previously failed to infer a ParamSpec if the return type of the
callable we are inferring against was not annotated.
3. We previously wrongly returned `Unknown` from `some_dict.get("key",
None)` if the value type of `some_dict` included a callable type with
un-annotated return type.
We now make signature return types and annotated parameter types
required, and we eagerly insert `Unknown` if there's no annotation. Most
of the diff is just a bunch of mechanical code changes where we
construct these types, and simplifications where we use them.
One exception is type display: when a callable type has un-annotated
parameters, we want to display them as un-annotated, but if it has a
parameter explicitly annotated with something of `Unknown` type, we want
to display that parameter as `x: Unknown` (it would be confusing if it
looked like your annotation just disappeared entirely).
Fortunately, we already have a mechanism in place for handling this: the
`inferred_annotation` flag, which suppresses display of an annotation.
Previously we used it only for `self` and `cls` parameters with an
inferred annotated type -- but we now also set it for any un-annotated
parameter, for which we infer `Unknown` type.
We also need to normalize `inferred_annotation`, since it's display-only
and shouldn't impact type equivalence. (This is technically a
previously-existing bug, it just never came up when it only affected
self types -- now it comes up because we have tests asserting that `def
f(x)` and `def g(x: Unknown)` are equivalent.)
## Test Plan
Added mdtests.
By stripping leading indents from codefence lines to ensure they're
properly understood by markdown (but otherwise preserving the indent in
the codeblock so all the code renders roughly at the right indent).
As described in [this
comment](https://github.com/astral-sh/ty/issues/2352#issuecomment-3711686053)
this solution is very "do what I mean" for when a user has an explicit
markdown codeblock in e.g. a `Returns:` section which "has" to be
indented but that indent makes the verbatim codefence invalid markdown.
* Fixes https://github.com/astral-sh/ty/issues/2352
Snapshot tests recently started reporting this warning:
> Snapshot test passes but the existing value is in a legacy format.
> Please run cargo insta test --force-update-snapshots to update to a
> newer format.
This PR is the result of that forced update.
One file (crates/ruff_db/src/diagnostic/render/full.rs) seems to get
corrupted, because it contains strings with unprintable characters that
trigger some bug in cargo-insta. I've manually reverted that file, and
also manually reverted the `input_file:` lines, which we like.
## Summary
Resolve(s) astral-sh/ty#117, astral-sh/ty#1569
Implement `typing.TypeGuard`. Due to the fact that it [overrides
anything previously known about the checked
value](https://typing.python.org/en/latest/spec/narrowing.html#typeguard)---
> When a conditional statement includes a call to a user-defined type
guard function, and that function returns true, the expression passed as
the first positional argument to the type guard function should be
assumed by a static type checker to take on the type specified in the
TypeGuard return type, unless and until it is further narrowed within
the conditional code block.
---we have to substantially rework the constraints system. In
particular, we make constraints represented as a disjunctive normal form
(DNF) where each term includes a regular constraint, and one or more
disjuncts with a typeguard constraint. Some test cases (including some
with more complex boolean logic) are added to `type_guards.md`.
## Test Plan
- update existing tests
- add new tests for more complex boolean logic with `TypeGuard`
- add new tests for `TypeGuard` variance
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Resolve https://github.com/astral-sh/ty/issues/2226
We need to add a special case in `apply_type_mapping` instead of
directly in `promote_literals_impl` because we do not reach this with
non generic non tuple nominal instances. We still ensure we apply the
normal mapping if we do not see `float` or `complex` instances.
## Test Plan
Update existing mdtest and add a new case to `literal_promotion.md`
## Summary
* Related to, but does not handle
https://github.com/astral-sh/ty/issues/2021
## Test Plan
I also added some snapshot tests for future work on non-standard
attribute docstrings (didn't want to highlight them if we don't
recognize them elsewhere).
