Commit Graph

295 Commits

Author SHA1 Message Date
David Peter 53fa32a389
[red-knot] Remove `Type::Unbound` (#13980)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

- Remove `Type::Unbound`
- Handle (potential) unboundness as a concept orthogonal to the type
system (see new `Symbol` type)
- Improve existing and add new diagnostics related to (potential)
unboundness

closes #13671 

## Test Plan

- Update existing markdown-based tests
- Add new tests for added/modified functionality
2024-10-31 20:05:53 +01:00
Alex Waygood d1189c20df
[red-knot] Add failing tests for iterating over maybe-iterable unions (#14016) 2024-10-31 18:20:21 +00:00
Micha Reiser 76e4277696
[red-knot] Handle context managers in (sync) with statements (#13998) 2024-10-31 08:18:18 +00:00
Dhruv Manilawala 2629527559
Fix panic when filling up types vector during unpacking (#14006)
## Summary

This PR fixes a panic which can occur in an unpack assignment when:
* (number of target expressions) - (number of tuple types) > 2
* There's a starred expression

The reason being that the `insert` panics because the index is greater
than the length.

This is an error case and so practically it should occur very rarely.
The solution is to resize the types vector to match the number of
expressions and then insert the starred expression type.

## Test Plan

Add a new test case.
2024-10-30 19:13:57 +00:00
Dhruv Manilawala bf20061268
Separate type check diagnostics builder (#13978)
## Summary

This PR creates a new `TypeCheckDiagnosticsBuilder` for the
`TypeCheckDiagnostics` struct. The main motivation behind this is to
separate the helpers required to build the diagnostics from the type
inference builder itself. This allows us to use such helpers outside of
the inference builder like for example in the unpacking logic in
https://github.com/astral-sh/ruff/pull/13979.

## Test Plan

`cargo insta test`
2024-10-30 18:50:31 +00:00
Charlie Marsh eddc8d7644
Add failing tests for augmented assignments with partial binding (#14002)
## Summary

These cases aren't handled correctly yet -- some of them are waiting on
refactors to `Unbound` before fixing. Part of #12699.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-30 14:22:34 -04:00
Charlie Marsh b1ce8a3949
Use `Never` instead of `None` for stores (#13984)
## Summary

See:
https://github.com/astral-sh/ruff/pull/13981#issuecomment-2445472433
2024-10-30 12:03:50 -04:00
Charlie Marsh 262c04f297
Use binary semantics when `__iadd__` et al are unbound (#13987)
## Summary

I noticed that augmented assignments on floats were yielding "not
supported" diagnostics. If the dunder isn't bound at all, we should use
binary operator semantics, rather than treating it as not-callable.
2024-10-30 13:09:22 +00:00
Charlie Marsh 71536a43db
Add remaining augmented assignment dunders (#13985)
## Summary

See: https://github.com/astral-sh/ruff/issues/12699
2024-10-30 13:02:29 +00:00
Alex Waygood e6dcdf3e49
Switch off the `single_match_else` Clippy lint (#13994) 2024-10-30 12:24:16 +00:00
Alex Waygood 42c70697d8
[red-knot] Fix bug where union of two iterable types was not recognised as iterable (#13992) 2024-10-30 11:54:16 +00:00
Charlie Marsh 1607d88c22
Use consistent diagnostic messages in augmented assignment inference (#13986) 2024-10-29 22:57:53 -04:00
Charlie Marsh c6b82151dd
Add augmented assignment inference for `-=` operator (#13981)
## Summary

See: https://github.com/astral-sh/ruff/issues/12699
2024-10-29 22:14:27 -04:00
Alex Waygood 39cf46ecd6
[red-knot] Improve ergonomics for the `PySlice` trait (#13983) 2024-10-29 20:40:59 +00:00
David Peter 96b3c400fe
[red-knot] Minor follow-up on slice expression inference (#13982)
## Summary

Minor follow-up to #13917 — thanks @AlexWaygood for the post-merge
review.

- Add
SliceLiteralType::as_tuple
- Use .expect() instead of SAFETY
comment
- Match on ::try_from
result
- Add TODO comment regarding raising a diagnostic for `"foo"["bar":"baz"]`
2024-10-29 19:40:57 +00:00
Alex Waygood 8d98aea6c4
[red-knot] Infer attribute expressions in type annotations (#13967) 2024-10-29 11:06:44 +00:00
Alex Waygood d2c9f5e43c
[red-knot] Fallback to attributes on types.ModuleType if a symbol can't be found in locals or globals (#13904) 2024-10-29 10:59:03 +00:00
Alex Waygood 7dd0c7f4bd
[red-knot] Infer `tuple` types from annotations (#13943)
## Summary

This PR adds support for heterogenous `tuple` annotations to red-knot.

It does the following:
- Extends `infer_type_expression` so that it understands tuple
annotations
- Changes `infer_type_expression` so that `ExprStarred` nodes in type
annotations are inferred as `Todo` rather than `Unknown` (they're valid
in PEP-646 tuple annotations)
- Extends `Type::is_subtype_of` to understand when one heterogenous
tuple type can be understood to be a subtype of another (without this
change, the PR would have introduced new false-positive errors to some
existing mdtests).
2024-10-29 10:30:03 +00:00
David Peter 56c796acee
[red-knot] Slice expression types & subscript expressions with slices (#13917)
## Summary

- Add a new `Type::SliceLiteral` variant
- Infer `SliceLiteral` types for slice expressions, such as
`<int-literal>:<int-literal>:<int-literal>`.
- Infer "sliced" literal types for subscript expressions using slices,
such as `<string-literal>[<slice-literal>]`.
- Infer types for expressions involving slices of tuples:
`<tuple>[<slice-literal>]`.

closes #13853

## Test Plan

- Unit tests for indexing/slicing utility functions
- Markdown-based tests for
  - Subscript expressions `tuple[slice]`
  - Subscript expressions `string_literal[slice]`
  - Subscript expressions `bytes_literal[slice]`
2024-10-29 10:17:31 +01:00
Raphael Gaschignard 2fe203292a
[red-knot] Distribute intersections on negation (#13962)
## Summary

This does two things:
- distribute negated intersections when building up intersections (i.e.
going from `A & ~(B & C)` to `(A & ~B) | (A & ~C)`) (fixing #13931)

## Test Plan

`cargo test`
2024-10-29 02:56:04 +00:00
Charlie Marsh b19862c64a
Rename `operator-unsupported` to `unsupported-operator` (#13973)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13959.
2024-10-28 21:34:12 -04:00
TomerBin 9a0dade925
[red-knot] Type narrowing inside boolean expressions (#13970)
## Summary

This PR adds type narrowing in `and` and `or` expressions, for example:

```py
class A: ...

x: A | None = A() if bool_instance() else None

isinstance(x, A) or reveal_type(x)  # revealed: None
``` 

## Test Plan
New mdtests 😍
2024-10-28 18:17:48 -07:00
TomerBin 74cf66e4c2
[red-knot] Narrowing - Not operator (#13942)
## Summary

After #13918 has landed, narrowing constraint negation became easy, so
adding support for `not` operator.

## Test Plan

Added a new mdtest file for `not` expression.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-28 20:27:26 +00:00
Charlie Marsh 6f52d573ef
Support inference for PEP 604 union annotations (#13964)
## Summary

Supports return type inference for, e.g., `def f() -> int | None:`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-28 10:13:01 -04:00
TomerBin 66c3aaa307
[red-knot] - Flow-control for boolean operations (#13940)
## Summary

As python uses short-circuiting boolean operations in runtime, we should
mimic that logic in redknot as well.
For example, we should detect that in the following code `x` might be
undefined inside the block:

```py
if flag or (x := 1):
    print(x) 
```

## Test Plan

Added mdtest suit for boolean expressions.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-27 03:33:01 +00:00
cake-monotone b6ffa51c16
[red-knot] Type inference for comparisons between arbitrary instances (#13903)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-10-26 18:19:56 +00:00
TomerBin 35f007f17f
[red-knot] Type narrow in else clause (#13918)
## Summary

Add support for type narrowing in elif and else scopes as part of
#13694.

## Test Plan

- mdtest
- builder unit test for union negation.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-26 16:22:57 +00:00
Micha Reiser 6aaf1d9446
[red-knot] Remove lint-phase (#13922)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-25 18:40:52 +00:00
Alex Waygood 5eb87aa56e
[red-knot] Infer `Todo`, not `Unknown`, for PEP-604 unions in annotations (#13908) 2024-10-25 18:21:31 +00:00
Micha Reiser 32b57b2ee4
Enable nursery rules: 'redundant_clone', 'debug_assert_with_mut_call', and 'unused_peekable' (#13920) 2024-10-25 09:46:30 +02:00
Alex Waygood 3eb454699a
[red-knot] Format mdtest Python snippets more concisely (#13905) 2024-10-24 11:09:31 +00:00
David Peter 77ae0ccf0f
[red-knot] Infer subscript expression types for bytes literals (#13901)
## Summary

Infer subscript expression types for bytes literals:
```py
b = b"\x00abc\xff"

reveal_type(b[0])  # revealed: Literal[b"\x00"]
reveal_type(b[1])  # revealed: Literal[b"a"]
reveal_type(b[-1])  # revealed: Literal[b"\xff"]
reveal_type(b[-2])  # revealed: Literal[b"c"]

reveal_type(b[False])  # revealed: Literal[b"\x00"]
reveal_type(b[True])  # revealed: Literal[b"a"]
```


part of #13689
(https://github.com/astral-sh/ruff/issues/13689#issuecomment-2404285064)

## Test Plan

- New Markdown-based tests (see `mdtest/subscript/bytes.md`)
- Added missing test for `string_literal[bool_literal]`
2024-10-24 12:07:41 +02:00
David Peter 2c57c2dc8a
[red-knot] Type narrowing for `isinstance` checks (#13894)
## Summary

Add type narrowing for `isinstance(object, classinfo)` [1] checks:
```py
x = 1 if flag else "a"

if isinstance(x, int):
    reveal_type(x)  # revealed: Literal[1]
```

closes #13893

[1] https://docs.python.org/3/library/functions.html#isinstance

## Test Plan

New Markdown-based tests in `narrow/isinstance.md`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-23 20:51:33 +02:00
David Peter 387076d212
[red-knot] Use track_caller for expect_ methods (#13884)
## Summary

A minor quality-of-life improvement: add
[`#[track_caller]`](https://doc.rust-lang.org/reference/attributes/codegen.html#the-track_caller-attribute)
attribute to `Type::expect_xyz()` methods and some `TypeInference` methods such that the panic-location
is reported one level higher up in the stack trace.

before: reports location inside the `Type::expect_class_literal()`
method. Not very useful.
```
thread 'types::infer::tests::deferred_annotation_builtin' panicked at crates/red_knot_python_semantic/src/types.rs:304:14:
Expected a Type::ClassLiteral variant
```

after: reports location at the `Type::expect_class_literal()` call site,
where the error was made.
```
thread 'types::infer::tests::deferred_annotation_builtin' panicked at crates/red_knot_python_semantic/src/types/infer.rs:4302:14:
Expected a Type::ClassLiteral variant
```

## Test Plan

Called `expect_class_literal()` on something that's not a
`Type::ClassLiteral` and saw that the error was reported at the call
site.
2024-10-23 12:48:19 +02:00
David Peter f335fe4d4a
[red-knot] rename {Class,Module,Function} => {Class,Module,Function}Literal (#13873)
## Summary

* Rename `Type::Class` => `Type::ClassLiteral`
* Rename `Type::Function` => `Type::FunctionLiteral`
* Do not rename `Type::Module`
* Remove `*Literal` suffixes in `display::LiteralTypeKind` variants, as
per clippy suggestion
* Get rid of `Type::is_class()` in favor of `is_subtype_of(…, 'type')`;
modifiy `is_subtype_of` to support this.
* Add new `Type::is_xyz()` methods and use them instead of matching on
`Type` variants.

closes #13863 

## Test Plan

New `is_subtype_of_class_literals` unit test.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-22 22:10:53 +02:00
David Peter c6ce52c29e
[red-knot] Treat empty intersection as 'object', fix intersection simplification (#13880)
## Summary

- Properly treat the empty intersection as being of type `object`.
- Consequently, change the simplification method to explicitly add
`Never` to the positive side of the intersection when collapsing a type
such as `int & str` to `Never`, as opposed to just clearing both the
positive and the negative side.
- Minor code improvement in `bindings_ty`: use `peekable()` to check
whether the iterator over constraints is empty, instead of handling
first and subsequent elements separately.

fixes #13870

## Test Plan

- New unit tests for `IntersectionBuilder` to make sure the empty
intersection represents `object`.
- Markdown-based regression test for the original issue in #13870
2024-10-22 21:02:46 +02:00
David Peter 46c0961b0b
[red-knot] is_subtype_of: treat literals as subtype of 'object' (#13876)
Add the following subtype relations:
- `BooleanLiteral <: object`
- `IntLiteral <: object`
- `StringLiteral <: object`
- `LiteralString <: object`
- `BytesLiteral <: object`

Added a test case for `bool <: int`.

## Test Plan

New unit tests.
2024-10-22 13:32:51 +02:00
Alex 9d102799f9
[red-knot] Support for not-equal narrowing (#13749)
Add type narrowing for `!=` expression as stated in
#13694.

###  Test Plan

Add tests in new md format.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2024-10-21 23:08:33 +02:00
TomerBin a77512df68
[red-knot] Improve chained comparisons handling (#13825)
## Summary

A small fix for comparisons of multiple comparators.
Instead of comparing each comparator to the leftmost item, we should
compare it to the closest item on the left.

While implementing this, I noticed that we don’t yet narrow Yoda
comparisons (e.g., `True is x`), so I didn’t change that behavior in
this PR.

## Test Plan

Added some mdtests 🎉
2024-10-21 12:38:08 -07:00
David Peter fa7626160b
[red-knot] handle unions on the LHS of is_subtype_of (#13857)
## Summary

Just a drive-by change that occurred to me while I was looking at
`Type::is_subtype_of`: the existing pattern for unions on the *right
hand side*:
```rs
            (ty, Type::Union(union)) => union
                .elements(db)
                .iter()
                .any(|&elem_ty| ty.is_subtype_of(db, elem_ty)),
```
is not (generally) correct if the *left hand side* is a union.

## Test Plan

Added new test cases for `is_subtype_of` and `!is_subtype_of`
2024-10-21 20:12:03 +02:00
David Peter d9ef83bfef
[red-knot] Consistently rename BoolLiteral => BooleanLiteral (#13856)
## Summary

- Consistent naming: `BoolLiteral` => `BooleanLiteral` (it's mainly the
`Ty::BoolLiteral` variant that was renamed)

  I tripped over this a few times now, so I thought I'll smooth it out.
- Add a new test case for `Literal[True] <: bool`, as suggested here:
https://github.com/astral-sh/ruff/pull/13781#discussion_r1804922827
2024-10-21 13:55:50 +02:00
Alex Waygood 02bca9a8d0
Modernize build scripts (#13837)
Use the modern `cargo::KEY=VALUE` syntax that was stabilised in MSRV 1.77, rather than the deprecated `cargo:KEY=VALUE` syntax.
2024-10-20 22:35:35 +01:00
Alex Waygood 72adb09bf3
Simplify iteration idioms (#13834)
Remove unnecessary uses of `.as_ref()`, `.iter()`, `&**` and similar, mostly in situations when iterating over variables. Many of these changes are only possible following #13826, when we bumped our MSRV to 1.80: several useful implementations on `&Box<[T]>` were only stabilised in Rust 1.80. Some of these changes we could have done earlier, however.
2024-10-20 22:25:27 +01:00
Aditya Pratap Singh 7fd8e30eed
[red-knot] Cleanup generated names of mdtest tests (#13831)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-20 15:11:53 +00:00
Shaygan Hooshyari 0f0fff4d5a
[red-knot] Implement more types in binary and unary expressions (#13803)
Implemented some points from
https://github.com/astral-sh/ruff/issues/12701

- Handle Unknown and Any in Unary operation
- Handle Boolean in binary operations
- Handle instances in unary operation
- Consider division by False to be division by zero

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-20 01:57:21 +00:00
cake-monotone fb66f715f3
[red-knot] Enhancing Diagnostics for Compare Expression Inference (#13819)
## Summary

- Refactored comparison type inference functions in `infer.rs`: Changed
the return type from `Option` to `Result` to lay the groundwork for
providing more detailed diagnostics.
- Updated diagnostic messages.

This is a small step toward improving diagnostics in the future.

Please refer to #13787

## Test Plan

mdtest included!

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-19 18:17:01 +00:00
Alex Waygood 55bccf6680
[red-knot] Fix edge case for binary-expression inference where the lhs and rhs are the exact same type (#13823)
## Summary

This fixes an edge case that @carljm and I missed when implementing
https://github.com/astral-sh/ruff/pull/13800. Namely, if the left-hand
operand is the _exact same type_ as the right-hand operand, the
reflected dunder on the right-hand operand is never tried:

```pycon
>>> class Foo:
...     def __radd__(self, other):
...         return 42
...         
>>> Foo() + Foo()
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    Foo() + Foo()
    ~~~~~~^~~~~~~
TypeError: unsupported operand type(s) for +: 'Foo' and 'Foo'
```

This edge case _is_ covered in Brett's blog at
https://snarky.ca/unravelling-binary-arithmetic-operations-in-python/,
but I missed it amongst all the other subtleties of this algorithm. The
motivations and history behind it were discussed in
https://mail.python.org/archives/list/python-dev@python.org/thread/7NZUCODEAPQFMRFXYRMGJXDSIS3WJYIV/

## Test Plan

I added an mdtest for this cornercase.
2024-10-19 11:09:54 -07:00
Carl Meyer f4b5e70fae
[red-knot] binary arithmetic on instances (#13800)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-19 15:22:54 +00:00
Alex Waygood 36cb1199cc
[red-knot] Autoformat `mdtest` Python snippets using `blacken-docs` (#13809) 2024-10-19 15:57:06 +01:00
David Peter 6964eef369
[red knot] add `Type::is_disjoint_from` and intersection simplifications (#13775)
## Summary

- Add `Type::is_disjoint_from` as a way to test whether two types
overlap
- Add a first set of simplification rules for intersection types
  - `S & T = S` for `S <: T`
  - `S & ~T = Never` for `S <: T`
  - `~S & ~T = ~T` for `S <: T`
  - `A & ~B = A` for `A` disjoint from `B`
  - `A & B = Never` for `A` disjoint from `B`
  - `bool & ~Literal[bool] = Literal[!bool]`

resolves one item in #12694

## Open questions:

- Can we somehow leverage the (anti) symmetry between `positive` and
`negative` contributions? I could imagine that there would be a way if
we had `Type::Not(type)`/`Type::Negative(type)`, but with the
`positive`/`negative` architecture, I'm not sure. Note that there is a
certain duplication in the `add_positive`/`add_negative` functions (e.g.
`S & ~T = Never` is implemented twice), but other rules are actually not
perfectly symmetric: `S & T = S` vs `~S & ~T = ~T`.
- I'm not particularly proud of the way `add_positive`/`add_negative`
turned out. They are long imperative-style functions with some
mutability mixed in (`to_remove`). I'm happy to look into ways to
improve this code *if we decide to go with this approach* of
implementing a set of ad-hoc rules for simplification.
- ~~Is it useful to perform simplifications eagerly in
`add_positive`/`add_negative`? (@carljm)~~ This is what I did for now.

## Test Plan

- Unit tests for `Type::is_disjoint_from`
- Observe changes in Markdown-based tests
- Unit tests for `IntersectionBuilder::build()`

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-18 21:34:43 +00:00
David Peter c2f7c39987
[red-knot] mdtest suite: formatting and cleanup (#13806)
Minor cleanup and consistent formatting of the Markdown-based tests.

- Removed lots of unnecessary `a`, `b`, `c`, … variables.
- Moved test assertions (`# revealed:` comments) closer to the tested
object.
- Always separate `# revealed` and `# error` comments from the code by
two spaces, according to the discussion
[here](https://github.com/astral-sh/ruff/pull/13746/files#r1799385758).
This trades readability for consistency in some cases.
- Fixed some headings
2024-10-18 11:07:53 +02:00
Raphael Gaschignard 3d0bdb426a
[red-knot] Use the right scope when considering class bases (#13766)
Summary
---------

PEP 695 Generics introduce a scope inside a class statement's arguments
and keywords.

```
class C[T](A[T]):  # the T in A[T] is not from the global scope but from a type-param-specfic scope
   ...
```

When doing inference on the class bases, we currently have been doing
base class expression lookups in the global scope. Not an issue without
generics (since a scope is only created when generics are present).

This change instead makes sure to stop the global scope inference from
going into expressions within this sub-scope. Since there is a separate
scope, `check_file` and friends will trigger inference on these
expressions still.

Another change as a part of this is making sure that `ClassType` looks
up its bases in the right scope.

Test Plan
----------
`cargo test --package red_knot_python_semantic generics` will run the
markdown test that previously would panic due to scope lookup issues

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-17 22:29:46 +00:00
Carl Meyer e2a30b71f4
[red-knot] revert change to emit fewer division by zero errors (#13801)
This reverts https://github.com/astral-sh/ruff/pull/13799, and restores
the previous behavior, which I think was the most pragmatic and useful
version of the divide-by-zero error, if we will emit it at all.

In general, a type checker _does_ emit diagnostics when it can detect
something that will definitely be a problem for some inhabitants of a
type, but not others. For example, `x.foo` if `x` is typed as `object`
is a type error, even though some inhabitants of the type `object` will
have a `foo` attribute! The correct fix is to make your type annotations
more precise, so that `x` is assigned a type which definitely has the
`foo` attribute.

If we will emit it divide-by-zero errors, it should follow the same
logic. Dividing an inhabitant of the type `int` by zero may not emit an
error, if the inhabitant is an instance of a subclass of `builtins.int`
that overrides division. But it may emit an error (more likely it will).
If you don't want the diagnostic, you can clarify your type annotations
to require an instance of your safe subclass.

Because the Python type system doesn't have the ability to explicitly
reflect the fact that divide-by-zero is an error in type annotations
(e.g. for `int.__truediv__`), or conversely to declare a type as safe
from divide-by-zero, or include a "nonzero integer" type which it is
always safe to divide by, the analogy doesn't fully apply. You can't
explicitly mark your subclass of `int` as safe from divide-by-zero, we
just semi-arbitrarily choose to silence the diagnostic for subclasses,
to avoid false positives.

Also, if we fully followed the above logic, we'd have to error on every
`int / int` because the RHS `int` might be zero! But this would likely
cause too many false positives, because of the lack of a "nonzero
integer" type.

So this is just a pragmatic choice to emit the diagnostic when it is
very likely to be an error. It's unclear how useful this diagnostic is
in practice, but this version of it is at least very unlikely to cause
harm.
2024-10-17 20:17:22 +00:00
Carl Meyer 5c537b6dbb
[red-knot] don't emit divide-by-zero error if we can't be sure (#13799)
If the LHS is just `int` or `float` type, that type includes custom
subclasses which can arbitrarily override division behavior, so we
shouldn't emit a divide-by-zero error in those cases.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-17 17:11:07 +00:00
David Peter 5c3c0c4705
[red-knot] Inference for comparison of union types (#13781)
## Summary

Add type inference for comparisons involving union types. For example:
```py
one_or_two = 1 if flag else 2

reveal_type(one_or_two <= 2)  # revealed: Literal[True]
reveal_type(one_or_two <= 1)  # revealed: bool
reveal_type(one_or_two <= 0)  # revealed: Literal[False]
```

closes #13779

## Test Plan

See `resources/mdtest/comparison/unions.md`
2024-10-17 11:03:37 +02:00
aditya pillai ed4a0b34ba
[red-knot] don't include Unknown in the type for a conditionally-defined import (#13563)
## Summary

Fixes the bug described in #13514 where an unbound public type defaulted
to the type or `Unknown`, whereas it should only be the type if unbound.

## Test Plan

Added a new test case

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-16 13:46:03 -07:00
Micha Reiser 2095ea8372
Add scope assertion to `TypeInference.extend` (#13764)
## Summary

This PR adds a debug assertion that asserts that `TypeInference::extend`
is only called on results that have the same scope.
This is critical because `expressions` uses `ScopedExpressionId` that
are local and merging expressions from different
scopes would lead to incorrect expression types.

We could consider storing `scope` only on `TypeInference` for debug
builds. Doing so has the advantage that the `TypeInference` type is
smaller of which we'll have many. However, a `ScopeId` is a `u32`... so
it shouldn't matter that much and it avoids storing the `scope` both on
`TypeInference` and `TypeInferenceBuilder`

## Test Plan

`cargo test`
2024-10-16 08:44:25 -07:00
Alex Waygood 6282402a8c
[red-knot] Add control flow for try/except blocks (#13729) 2024-10-16 13:03:59 +00:00
Raphael Gaschignard d25673f664
[red-knot] Do not panic if named expressions show up in assignment position (#13711)
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-16 12:42:39 +00:00
cake-monotone 2ffc3fad47
[red-knot] Implement `Type::Tuple` Comparisons (#13712)
## Summary

This PR implements comparisons for (tuple, tuple).

It will close #13688 and complete an item in #13618 once merged.

## Test Plan

Basic tests are included for (tuple, tuple) comparisons.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-16 11:39:55 +00:00
David Peter b85be6297e
[red knot] Minor follow-up tasks regarding singleton types (#13769)
## Summary

- Do not treat empty tuples as singletons after discussion [1]
- Improve comment regarding intersection types
- Resolve unnecessary TODO in Markdown test

[1]
https://discuss.python.org/t/should-we-specify-in-the-language-reference-that-the-empty-tuple-is-a-singleton/67957

## Test Plan

—
2024-10-16 11:30:03 +02:00
Alex Waygood fb1d1e3241
[red-knot] Simplify some branches in `infer_subscript_expression` (#13762)
## Summary

Just a small simplification to remove some unnecessary complexity here.
Rather than using separate branches for subscript expressions involving
boolean literals, we can simply convert them to integer literals and
reuse the logic in the `IntLiteral` branches.

## Test Plan

`cargo test -p red_knot_python_semantic`
2024-10-16 07:58:24 +01:00
Dhruv Manilawala b16f665a81
[red-knot] Infer target types for unpacked tuple assignment (#13316)
## Summary

This PR adds support for unpacking tuple expression in an assignment
statement where the target expression can be a tuple or a list (the
allowed sequence targets).

The implementation introduces a new `infer_assignment_target` which can
then be used for other targets like the ones in for loops as well. This
delegates it to the `infer_definition`. The final implementation uses a
recursive function that visits the target expression in source order and
compares the variable node that corresponds to the definition. At the
same time, it keeps track of where it is on the assignment value type.

The logic also accounts for the number of elements on both sides such
that it matches even if there's a gap in between. For example, if
there's a starred expression like `(a, *b, c) = (1, 2, 3)`, then the
type of `a` will be `Literal[1]` and the type of `b` will be
`Literal[2]`.

There are a couple of follow-ups that can be done:
* Use this logic for other target positions like `for` loop
* Add diagnostics for mis-match length between LHS and RHS

## Test Plan

Add various test cases using the new markdown test framework.
Validate that existing test cases pass.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-15 19:07:11 +00:00
Alex d77480768d
[red-knot] Port type inference tests to new test framework (#13719)
## Summary

Porting infer tests to new markdown tests framework.

Link to the corresponding issue: #13696

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-15 11:23:46 -07:00
github-actions[bot] 5fa82fb0cd
Sync vendored typeshed stubs (#13753) 2024-10-15 13:36:11 +00:00
David Peter 74bf4b0653
[red knot] Fix narrowing for '… is not …' type guards, add '… is …' type guards (#13758)
## Summary

- Fix a bug with `… is not …` type guards.
 
  Previously, in an example like
  ```py
  x = [1]
  y = [1]
  
  if x is not y:
      reveal_type(x)
  ```
  we would infer a type of `list[int] & ~list[int] == Never` for `x`
  inside the conditional (instead of `list[int]`), since we built a
  (negative) intersection with the type of the right hand side (`y`).
  However, as this example shows, this assumption can only be made for
  singleton types (types with a single inhabitant) such as `None`.
- Add support for `… is …` type guards.

closes #13715

## Test Plan

Moved existing `narrow_…` tests to Markdown-based tests and added new
ones (including a regression test for the bug described above). Note
that will create some conflicts with
https://github.com/astral-sh/ruff/pull/13719. I tried to establish the
correct organizational structure as proposed in
https://github.com/astral-sh/ruff/pull/13719#discussion_r1800188105
2024-10-15 14:49:32 +02:00
Micha Reiser 5f65e842e8
Upgrade salsa (#13757) 2024-10-15 11:06:32 +00:00
David Peter 04b636cba2
[red knot] Use memmem::find instead of custom version (#13750)
This is a follow-up on #13746:

- Use `memmem::find` instead of rolling our own inferior version.
- Avoid `x.as_ref()` calls using `&**x`
2024-10-14 15:17:19 +02:00
Alex Waygood 6048f331d9
[red-knot] Add a build.rs file to `red_knot_python_semantic`, and document pitfalls of using `rstest` in combination with `mdtest` (#13747) 2024-10-14 13:02:03 +01:00
David Peter 93097f1c53
[red-knot] feat: Inference for `BytesLiteral` comparisons (#13746)
Implements inference for `BytesLiteral` comparisons along the lines of
https://github.com/astral-sh/ruff/pull/13634.

closes #13687

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-14 14:01:23 +02:00
Carl Meyer a3dc5c0529
[red-knot] document test framework (#13695)
This adds documentation for the new test framework.

I also added documentation for the planned design of features we haven't
built yet (clearly marked as such), so that this doc can become the sole
source of truth for the test framework design (we don't need to refer
back to the original internal design document.)

Also fixes a few issues in the test framework implementation that were
discovered in writing up the docs.

---------

Co-authored-by: T-256 <132141463+T-256@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-10-10 12:02:01 -07:00
Carl Meyer 93eff7f174
[red-knot] type inference/checking test framework (#13636)
## Summary

Adds a markdown-based test framework for writing tests of type inference
and type checking. Fixes #11664.

Implements the basic required features. A markdown test file is a suite
of tests, each test can contain one or more Python files, with
optionally specified path/name. The test writes all files to an
in-memory file system, runs red-knot, and matches the resulting
diagnostics against `Type: ` and `Error: ` assertions embedded in the
Python source as comments.

We will want to add features like incremental tests, setting custom
configuration for tests, writing non-Python files, testing syntax
errors, capturing full diagnostic output, etc. There's also plenty of
room for improved UX (colored output?).

## Test Plan

Lots of tests!

Sample of the current output when a test fails:

```
     Running tests/inference.rs (target/debug/deps/inference-7c96590aa84de2a4)

running 1 test
test inference::path_1_resources_inference_numbers_md ... FAILED

failures:

---- inference::path_1_resources_inference_numbers_md stdout ----
inference/numbers.md - Numbers - Floats
  /src/test.py
    line 2: unexpected error: [invalid-assignment] "Object of type `Literal["str"]` is not assignable to `int`"

thread 'inference::path_1_resources_inference_numbers_md' panicked at crates/red_knot_test/src/lib.rs:60:5:
Some tests failed.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    inference::path_1_resources_inference_numbers_md

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s

error: test failed, to rerun pass `-p red_knot_test --test inference`
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-08 12:33:19 -07:00
Alex Waygood 71b52b83e4
[red-knot] Allow `type[]` to be subscripted (#13667)
Fixed a TODO by adding another TODO. It's the red-knot way!

## Summary

`builtins.type` can be subscripted at runtime on Python 3.9+, even
though it has no `__class_getitem__` method and its metaclass (which
is... itself) has no `__getitem__` method. The special case is
[hardcoded directly into `PyObject_GetItem` in
CPython](744caa8ef4/Objects/abstract.c (L181-L184)).
We just have to replicate the special case in our semantic model.

This will fail at runtime on Python <3.9. However, there's a bunch of
outstanding questions (detailed in the TODO comment I added) regarding
how we deal with subscriptions of other generic types on lower Python
versions. Since we want to avoid too many false positives for now, I
haven't tried to address this; I've just made `type` subscriptable on
all Python versions.

## Test Plan

`cargo test -p red_knot_python_semantic --lib`
2024-10-07 19:43:47 +01:00
Alex Waygood d7484e6942
[red-knot] Improve type inference for except handlers where a tuple of exception classes is caught (#13646) 2024-10-07 16:13:06 +01:00
renovate[bot] 38d872ea4c
Update Rust crate hashbrown to 0.15.0 (#13652)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-07 08:50:59 +02:00
Simon 8108f83810
[red-knot] feat: add `StringLiteral` and `LiteralString` comparison (#13634)
## Summary

Implements string literal comparisons and fallbacks to `str` instance
for `LiteralString`.
Completes an item in #13618

## Test Plan

- Adds a dedicated test with non exhaustive cases

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-05 12:22:30 -07:00
Simon f1205177fd
[red-knot] fix: when simplifying union, True & False -> instance(bool) (#13644) 2024-10-05 19:01:10 +01:00
Simon 1c2cafc101
[red-knot] more ergonomic and efficient handling of known builtin classes (#13615) 2024-10-05 18:03:46 +01:00
Alex Waygood 7c5a7d909c
[red-knot] Improve tests relating to type inference for exception handlers (#13643) 2024-10-05 16:59:36 +00:00
Simon 888930b7d3
[red-knot] feat: implement integer comparison (#13571)
## Summary

Implements the comparison operator for `[Type::IntLiteral]` and
`[Type::BooleanLiteral]` (as an artifact of special handling of `True` and
`False` in python).
Sets the framework to implement more comparison for types known at
static time (e.g. `BooleanLiteral`, `StringLiteral`), allowing us to only
implement cases of the triplet `<left> Type`, `<right> Type`, `CmpOp`.
Contributes to #12701 (without checking off an item yet).

## Test Plan

- Added a test for the comparison of literals that should include most
cases of note.
- Added a test for the comparison of int instances

Please note that the cases do not cover 100% of the branches as there
are many and the current testing strategy with variables make this
fairly confusing once we have too many in one test.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-04 10:40:59 -07:00
Charlie Marsh c3b40da0d2
Use backticks for code in red-knot messages (#13599)
## Summary

...and remove periods from messages that don't span more than a single
sentence.

This is more consistent with how we present user-facing messages in uv
(which has a defined style guide).
2024-10-02 03:14:28 +00:00
Charlie Marsh ef45185dbc
Allow users to provide custom diagnostic messages when unwrapping calls (#13597)
## Summary

You can now call `return_ty_result` to operate on a `Result` directly
thereby using your own diagnostics, as in:

```rust
return dunder_getitem_method
    .call(self.db, &[slice_ty])
    .return_ty_result(self.db, value.as_ref().into(), self)
    .unwrap_or_else(|err| {
        self.add_diagnostic(
            (&**value).into(),
            "call-non-callable",
            format_args!(
                "Method `__getitem__` is not callable on object of type '{}'.",
                value_ty.display(self.db),
            ),
        );
        err.return_ty()
    });
```
2024-10-01 21:22:13 +00:00
Charlie Marsh 961fc98344
Use `__class_getitem__` for more specific non-subscript errors (#13596) 2024-10-01 18:16:00 +00:00
Charlie Marsh 0a6dc8e1b8
Support `__getitem__` type inference for subscripts (#13579)
## Summary

Follow-up to https://github.com/astral-sh/ruff/pull/13562, to add
support for "arbitrary" subscript operations.
2024-10-01 18:04:16 +00:00
Charlie Marsh 8d54996ffb
Avoid indirection in `class.__call__` lookup (#13595) 2024-10-01 18:01:36 +00:00
Alex Waygood 73e884b232
[red-knot] [minor] Improve helper methods for builtin types (#13594) 2024-10-01 18:38:33 +01:00
Charlie Marsh edba60106b
Support classes that implement `__call__` (#13580)
## Summary

This looked straightforward and removes some TODOs.
2024-10-01 17:15:46 +00:00
Alex Waygood 043fba7a57
[red-knot] Fix a few details around `Type::call` (#13593) 2024-10-01 16:49:09 +00:00
Alex Waygood 82324678cf
Rename the `ruff_vendored` crate to `red_knot_vendored` (#13586) 2024-10-01 16:16:59 +01:00
Zanie Blue cfd5d63917
Use operator specific messaging in division by zero diagnostics (#13588)
Requested at
https://github.com/astral-sh/ruff/pull/13576#discussion_r1782530971
2024-10-01 08:58:38 -05:00
Alex Waygood 2a36b47f13
[red-knot] Remove `Type::RevealType` (#13567) 2024-10-01 10:01:03 +00:00
Zanie Blue 45f01e7872
Add diagnostic for integer division by zero (#13576)
Adds a diagnostic for division by the integer zero in `//`, `/`, and
`%`.

Doesn't handle `<int> / 0.0` because we don't track the values of float
literals.
2024-09-30 22:38:52 +00:00
Simon 6cdf996af6
[red-knot] feat: introduce a new `[Type::Todo]` variant (#13548)
This variant shows inference that is not yet implemented..

## Summary

PR #13500 reopened the idea of adding a new type variant to keep track
of not-implemented features in Red Knot.

It was based off of #12986 with a more generic approach of keeping track
of different kind of unknowns. Discussion in #13500 agreed that keeping
track of different `Unknown` is complicated for now, and this feature is
better achieved through a new variant of `Type`.

### Requirements

Requirements for this implementation can be summed up with some extracts
of comment from @carljm on the previous PR

> So at the moment we are leaning towards simplifying this PR to just
use a new top-level variant, which behaves like Any and Unknown but
represents inference that is not yet implemented in red-knot.

> I think the general rule should be that Todo should propagate only
when the presence of the input Todo caused the output to be unknown.
>
> To take a specific example, the inferred result of addition must be
Unknown if either operand is Unknown. That is, Unknown + X will always
be Unknown regardless of what X is. (Same for X + Unknown.) In this
case, I believe that Unknown + Todo (or Todo + Unknown) should result in
Unknown, not result in Todo. If we fix the upstream source of the Todo,
the result would still be Unknown, so it's not useful to propagate the
Todo in this case: it wrongly suggests that the output is unknown
because of a todo item.

## Test Plan

This PR does not introduce new tests, but it did required to edit some
tests with the display of `[Type::Todo]` (currently `@Todo`), which
suggests that those test are placeholders requirements for features we
don't support yet.
2024-09-30 14:28:06 -07:00
Zanie Blue 9d8a4c0057
Improve display of `assert_public_ty` assertion failures (#13577)
While working on https://github.com/astral-sh/ruff/pull/13576 I noticed
that it was really hard to tell which assertion failed in some of these
test cases. This could be expanded to elsewhere, but I've heard this
test suite format won't be around for long?
2024-09-30 16:12:26 -05:00
Charlie Marsh c9c748a79e
Add some basic subscript type inference (#13562)
## Summary

Just for tuples and strings -- the easiest cases. I think most of the
rest require generic support?
2024-09-30 16:50:46 -04:00
Zanie Blue 32c746bd82
Fix inference when integers are divided (#13575)
Fixes the `Operator::Div` case and adds `Operator::FloorDiv` support

Closes https://github.com/astral-sh/ruff/issues/13570
2024-09-30 15:50:37 -05:00
Charlie Marsh d86b73eb3d
Add unary inference for integer and boolean literals (#13559)
## Summary

Just trying to familiarize myself with the general patterns, testing,
etc.

Part of https://github.com/astral-sh/ruff/issues/12701.
2024-09-30 16:29:06 +00:00
Alex Waygood 5f4b282327
[red-knot] Allow calling `bool()` with no arguments (#13568) 2024-09-30 13:18:01 +00:00
aditya pillai d9267132d6
Fix leftover references to `red_knot_python_semantic/vendor/` (#13561)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-09-30 11:32:02 +00:00
TomerBin ec72e675d9
Red Knot - Infer the return value of bool() (#13538)
## Summary
Following #13449, this PR adds custom handling for the bool constructor,
so when the input type has statically known truthiness value, it will be
used as the return value of the bool function.
For example, in the following snippet x will now be resolved to
`Literal[True]` instead of `bool`.
```python
x = bool(1)
```

## Test Plan
Some cargo tests were added.
2024-09-27 12:11:55 -07:00