## Summary
This is a new rule to implement the renaming of PEP 695 type parameters
with leading underscores after they have (presumably) been converted
from standalone type variables by either UP046 or UP047. Part of #15642.
I'm not 100% sure the fix is always safe, but I haven't come up with any
counterexamples yet. `Renamer` seems pretty precise, so I don't think
the usual issues with comments apply.
I initially tried writing this as a rule that receives a `Stmt` rather
than a `Binding`, but in that case the
`checker.semantic().current_scope()` was the global scope, rather than
the scope of the type parameters as I needed. Most of the other rules
using `Renamer` also used `Binding`s, but it does have the downside of
offering separate diagnostics for each parameter to rename.
## Test Plan
New snapshot tests for UP049 alone and the combination of UP046, UP049,
and PYI018.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
I experimented with [not trimming trailing newlines in code
snippets](https://github.com/astral-sh/ruff/pull/15926#discussion_r1940992090),
but since came to the conclusion that the current behavior is better
because otherwise, there is no way to write snippets without a trailing
newline at all. And when you copy the code from a Markdown snippet in
GitHub, you also don't get a trailing newline.
I was surprised to see some test failures when I played with this
though, and decided to make this test independent from this
implementation detail.
## Summary
Fix line number reporting in MDTest error messages.
## Test Plan
Introduced an error in a Markdown test and made sure that the line in
the error message matches.
## Summary
This is a follow-up to #15726, #15778, and #15794 to preserve the triple
quote and prefix flags in plain strings, bytestrings, and f-strings.
I also added a `StringLiteralFlags::without_triple_quotes` method to
avoid passing along triple quotes in rules like SIM905 where it might
not make sense, as discussed
[here](https://github.com/astral-sh/ruff/pull/15726#discussion_r1930532426).
## Test Plan
Existing tests, plus many new cases in the `generator::tests::quote`
test that should cover all combinations of quotes and prefixes, at least
for simple string bodies.
Closes#7799 when combined with #15694, #15726, #15778, and #15794.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Replaces our existing Markdown test parser with a fully hand-written
parser. I tried to fix this bug using the old approach and kept running
into problems. Eventually this seemed like the easier way. It's more
code (+50 lines, excluding the new test), but I hope it's relatively
straightforward to understand, compared to the complex interplay between
the byte-stream-manipulation and regex-parsing that we had before.
I did not really focus on performance, as the parsing time does not
dominate the test execution time, but this seems to be slightly faster
than what we had before (executing all MD tests; debug):
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| this branch | 2.775 ± 0.072 | 2.690 | 2.877 | 1.00 |
| `main` | 2.921 ± 0.034 | 2.865 | 2.967 | 1.05 ± 0.03 |
closes#15923
## Test Plan
One new regression test.
## Summary
Extend AIR302 with
* `airflow.operators.bash.BashOperator →
airflow.providers.standard.operators.bash.BashOperator`
* change existing rules `airflow.operators.bash_operator.BashOperator →
airflow.operators.bash.BashOperator` to
`airflow.operators.bash_operator.BashOperator →
airflow.providers.standard.operators.bash.BashOperator`
## Test Plan
a test fixture has been updated
## Summary
Resolves#15695, rework of #15704.
This change modifies the Mdtests framework so that:
* Paths must now be specified in a separate preceding line:
`````markdown
`a.py`:
```py
x = 1
```
`````
If the path of a file conflicts with its `lang`, an error will be
thrown.
* Configs are no longer accepted. The pattern still take them into
account, however, to avoid "Unterminated code block" errors.
* Unnamed files are now assigned unique, `lang`-respecting paths
automatically.
Additionally, all legacy usages have been updated.
## Test Plan
Unit tests and Markdown tests.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
This extracts some pure refactoring noise from
https://github.com/astral-sh/ruff/pull/15861. This changes the API for
creating and evaluating visibility constraints, but does not change how
they are respresented internally. There should be no behavioral or
performance changes in this PR.
Changes:
- Hide the internal representation isn't changed, so that we can make
changes to it in #15861.
- Add a separate builder type for visibility constraints. (With TDDs, we
will have some additional builder state that we can throw away once
we're done constructing.)
- Remove a layer of helper methods from `UseDefMapBuilder`, making
`SemanticIndexBuilder` responsible for constructing whatever visibility
constraints it needs.
## Summary
Add support for implicitly-defined instance attributes, i.e. support
type inference for cases like this:
```py
class C:
def __init__(self) -> None:
self.x: int = 1
self.y = None
reveal_type(C().x) # int
reveal_type(C().y) # Unknown | None
```
## Benchmarks
Codspeed reports no change in a cold-cache benchmark, and a -1%
regression in the incremental benchmark. On `black`'s `src` folder, I
don't see a statistically significant difference between the branches:
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./red_knot_main check --project /home/shark/black/src` | 133.7 ± 9.5 | 126.7 | 164.7 | 1.01 ± 0.08 |
| `./red_knot_feature check --project /home/shark/black/src` | 132.2 ± 5.1 | 118.1 | 140.9 | 1.00 |
## Test Plan
Updated and new Markdown tests
## Summary
Given the following code:
```python
set(([x for x in range(5)]))
```
the current implementation of C403 results in
```python
{(x for x in range(5))}
```
which is a set containing a generator rather than the result of the
generator.
This change removes the extraneous parentheses so that the resulting
code is:
```python
{x for x in range(5)}
```
## Test Plan
`cargo nextest run` and `cargo insta test`
## Summary
Related to #15848, this PR adds the imports explicitly as we'll now flag
these symbols as undefined.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This change does a simple swap of the existing renderer for one that
uses our vendored copy of `annotate-snippets`. We don't change anything
about the diagnostic data model, but this alone already makes
diagnostics look a lot nicer!
This mimics a simplification we have on the OR side, where we simplify
`A ∨ !A` to true. This requires changes to how we add `while` statements
to the semantic index, since we now need distinct
`VisibilityConstraint`s if we need to model evaluating a `Constraint`
multiple times at different points in the execution of the program.
Something Alex and I threw together during our 1:1 this morning. Allows
us to collect statistics on the prevalence of various types in a file,
most usefully TODO types or other dynamic types.
## Summary
This is a follow-up to #15565, tracked in #15642, to reuse the string
replacement logic from the other PEP 695 rules instead of the
`Generator`, which has the benefit of preserving more comments. However,
comments in some places are still dropped, so I added a check for this
and update the fix safety accordingly. I also added a `## Fix safety`
section to the docs to reflect this and the existing `isinstance`
caveat.
## Test Plan
Existing UP040 tests, plus some new cases.
## Summary
Resolves#10063 and follow-up to #15521.
The fix is now marked as unsafe if there are any comments within its
range. Tests are adapted from that of #15521.
## Test Plan
`cargo nextest run` and `cargo insta test`.
Both `list` and `dict` expect only a single positional argument. Giving
more positional arguments, or a keyword argument, is a `TypeError` and
neither the lint rule nor its fix make sense in that context.
Closes#15810
## Summary
Fixes https://github.com/astral-sh/ruff/issues/15812 by visiting the
second argument as a type definition.
## Test Plan
New F401 tests based on the report.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Builtin bindings are given a range of `0..0`, which causes strange
behavior when range checks are made at the top of the file. In this
case, the logic of the rule demands that the value of the dict
comprehension is not self-referential (i.e. it does not contain
definitions for any of the variables used within it). This logic was
confused by builtins which looked like they were defined "in the
comprehension", if the comprehension appeared at the top of the file.
Closes#15830
## Summary
This PR removes a trailing semicolon after an interface definition in
the custom TypeScript section of `ruff_wasm`. Currently, this semicolon
triggers the error "TS1036: Statements are not allowed in ambient
contexts" when including the file and compiling with e.g `tsc`.
## Test Plan
I made the change, ran `wasm-pack` and copied the generated directory
manually to my `node_modules` folder. I then compiled a file importing
`@astral-sh/ruff-wasm-web` again and confirmed that the compilation
error was gone.
If there is any `ParenthesizedWhitespace` (in the sense of LibCST) after
the function name `sorted` and before the arguments, then we must wrap
`sorted` with parentheses after removing the surrounding function.
Closes#15789
This PR uses the tokens of the parsed annotation available in the
`Checker`, instead of re-lexing (using `SimpleTokenizer`) the
annotation. This avoids some limitations of the `SimpleTokenizer`, such
as not being able to handle number and string literals.
Closes#15816 .
## Summary
Permits suspicious imports (the `S4` namespaced diagnostics) from stub
files.
Closes#15207.
## Test Plan
Added tests and ran `cargo nextest run`. The test files are copied from
the `.py` variants.
`FlowSnapshot` now tracks a `reachable` bool, which indicates whether we
have encountered a terminal statement on that control flow path. When
merging flow states together, we skip any that have been marked
unreachable. This ensures that bindings that can only be reached through
unreachable paths are not considered visible.
## Test Plan
The new mdtests failed (with incorrect `reveal_type` results, and
spurious `possibly-unresolved-reference` errors) before adding the new
visibility constraints.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
This is another follow-up to #15726 and #15778, extending the
quote-preserving behavior to f-strings and deleting the now-unused
`Generator::quote` field.
## Details
I also made one unrelated change to `rules/flynt/helpers.rs` to remove a
`to_string` call for making a `Box<str>` and tweaked some arguments to
some of the `Generator::unparse_f_string` methods to make the code
easier to follow, in my opinion. Happy to revert especially the latter
of these if needed.
Unfortunately this still does not fix the issue in #9660, which appears
to be more of an escaping issue than a quote-preservation issue. After
#15726, the result is now `a = f'# {"".join([])}' if 1 else ""` instead
of `a = f"# {''.join([])}" if 1 else ""` (single quotes on the outside
now), but we still don't have the desired behavior of double quotes
everywhere on Python 3.12+. I added a test for this but split it off
into another branch since it ended up being unaddressed here, but my
`dbg!` statements showed the correct preferred quotes going into
[`UnicodeEscape::with_preferred_quote`](https://github.com/astral-sh/ruff/blob/main/crates/ruff_python_literal/src/escape.rs#L54).
## Test Plan
Existing rule and `Generator` tests.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Implements some of #14738, by adding support for 6 new patterns:
```py
re.search("abc", s) is None # ⇒ "abc" not in s
re.search("abc", s) is not None # ⇒ "abc" in s
re.match("abc", s) is None # ⇒ not s.startswith("abc")
re.match("abc", s) is not None # ⇒ s.startswith("abc")
re.fullmatch("abc", s) is None # ⇒ s != "abc"
re.fullmatch("abc", s) is not None # ⇒ s == "abc"
```
## Test Plan
```shell
cargo nextest run
cargo insta review
```
And ran the fix on my startup's repo.
## Note
One minor limitation here:
```py
if not re.match('abc', s) is None:
pass
```
will get fixed to this (technically correct, just not nice):
```py
if not not s.startswith('abc'):
pass
```
This seems fine given that Ruff has this covered: the initial code
should be caught by
[E714](https://docs.astral.sh/ruff/rules/not-is-test/) and the fixed
code should be caught by
[SIM208](https://docs.astral.sh/ruff/rules/double-negation/).
## Summary
When we discussed the plan on how to proceed with instance attributes,
we said that we should first extend our research into the behavior of
existing type checkers. The result of this research is summarized in the
newly added / modified tests in this PR. The TODO comments align with
existing behavior of other type checkers. If we deviate from the
behavior, it is described in a comment.
## Summary
Resolves#12717.
This change incorporates the logic added in #15588.
## Test Plan
`cargo nextest run` and `cargo insta test`.
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
## Summary
As promised in #15603 - the **highly** sophisticated change - adding
missing config docstrings that are used in command completions.
## Test Plan
I actually made a local change to emit all empty items and verified
there are none now, before opening the PR.
## Summary
This is a very closely related follow-up to #15726, adding the same
quote-preserving behavior to bytestrings. Only one rule (UP018) was
affected this time, and it was easy to mirror the plain string changes.
## Test Plan
Existing tests
## Summary
This is a first step toward fixing #7799 by using the quoting style
stored in the `flags` field on `ast::StringLiteral`s to select a quoting
style. This PR does not include support for f-strings or byte strings.
Several rules also needed small updates to pass along existing quoting
styles instead of using `StringLiteralFlags::default()`. The remaining
snapshot changes are intentional and should preserve the quotes from the
input strings.
## Test Plan
Existing tests with some accepted updates, plus a few new RUF055 tests
for raw strings.
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
* feat
* add is_execute_method_inherits_from_airflow_operator for checking the
removed context key in the execute method
* refactor: rename
* is_airflow_task as is_airflow_task_function_def
* in_airflow_task as in_airflow_task_function_def
* removed_in_3 as airflow_3_removal_expr
* removed_in_3_function_def as airflow_3_removal_function_def
* test:
* reorganize test cases
## Test Plan
a test fixture has been updated
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
## Summary
Not the most important feature, but hey... was marked as the good first
issue ;-) fixes#4551
Unfortunately, looks like clap only generates proper completions for
zsh, so this would not make any difference for bash/fish.
## Test Plan
- cargo nextest run
- manual test by sourcing completions and then triggering autocomplete:
```shell
misha@PandaBook ruff % source <(target/debug/ruff generate-shell-completion zsh)
misha@PandaBook ruff % target/debug/ruff config lin
line-length -- The line length to use when enforcing long-lines violations
lint -- Configures how Ruff checks your code.
lint.allowed-confusables -- A list of allowed 'confusable' Unicode characters to ignore
lint.dummy-variable-rgx -- A regular expression used to identify 'dummy' variables, or
lint.exclude -- A list of file patterns to exclude from linting in addition
lint.explicit-preview-rules -- Whether to require exact codes to select preview rules. Whe
lint.extend-fixable -- A list of rule codes or prefixes to consider fixable, in ad
lint.extend-ignore -- A list of rule codes or prefixes to ignore, in addition to
lint.extend-per-file-ignores -- A list of mappings from file pattern to rule codes or prefi
lint.extend-safe-fixes -- A list of rule codes or prefixes for which unsafe fixes sho
lint.extend-select -- A list of rule codes or prefixes to enable, in addition to
lint.extend-unsafe-fixes -- A list of rule codes or prefixes for which safe fixes shoul
lint.external -- A list of rule codes or prefixes that are unsupported by Ru
lint.fixable -- A list of rule codes or prefixes to consider fixable. By de
lint.flake8-annotations -- Print a list of available options
lint.flake8-annotations.allow-star-arg-any -- Whether to suppress `ANN401` for dynamically typed `*args`
...
```
- check command help
```shell
❯ target/debug/ruff config -h
List or describe the available configuration options
Usage: ruff config [OPTIONS] [OPTION]
Arguments:
[OPTION] Config key to show
Options:
--output-format <OUTPUT_FORMAT> Output format [default: text] [possible values: text, json]
-h, --help Print help
Log levels:
-v, --verbose Enable verbose logging
-q, --quiet Print diagnostics, but nothing else
-s, --silent Disable all logging (but still exit with status code "1" upon detecting diagnostics)
Global options:
--config <CONFIG_OPTION> Either a path to a TOML configuration file (`pyproject.toml` or `ruff.toml`), or a TOML `<KEY> =
<VALUE>` pair (such as you might find in a `ruff.toml` configuration file) overriding a specific
configuration option. Overrides of individual settings using this option always take precedence over
all configuration files, including configuration files that were also specified using `--config`
--isolated Ignore all configuration files
```
- running original command
```shell
❯ target/debug/ruff config
cache-dir
extend
output-format
fix
unsafe-fixes
fix-only
show-fixes
required-version
preview
exclude
extend-exclude
extend-include
force-exclude
include
respect-gitignore
builtins
namespace-packages
target-version
src
line-length
indent-width
lint
format
analyze
```
## Summary
Adds a slightly more comprehensive documentation of our behavior
regarding type inference for public uses of symbols. In particular:
- What public type do we infer for `x: int = any()`?
- What public type do we infer for `x: Unknown = 1`?
## Summary
Found a comment that looks to be intended as docstring but accidentally
is just a normal comment.
Didn't create an issue as the readme said it's not neccessary for
trivial changes.
## Test Plan
<!-- How was it tested? -->
Can be tested by regenerating the docs.
Co-authored-by: Marcus Näslund <vidaochmarcus@gmail.com>
## Summary
On `main`, red-knot:
- Considers `P | Q` equivalent to `Q | P`
- Considered `tuple[P | Q]` equivalent to `tuple[Q | P]`
- Considers `tuple[P | tuple[P | Q]]` equivalent to `tuple[tuple[Q | P]
| P]`
- ‼️ Does _not_ consider `tuple[tuple[P | Q]]` equivalent to
`tuple[tuple[Q | P]]`
The key difference for the last one of these is that the union appears
inside a tuple that is directly nested inside another tuple.
This PR fixes this so that differently ordered unions are considered
equivalent even when they appear inside arbitrarily nested tuple types.
## Test Plan
- Added mdtests that fails on `main`
- Checked that all property tests continue to pass with this PR
This is a follow-up to #15702 that hopefully claws back the 1%
performance regression. Assuming it works, the trick is to iterate over
the constraints vectors via mut reference (aka a single pointer), so
that we're not copying `BitSet`s into and out of the zip tuples as we
iterate. We use `std::mem::take` as a poor-man's move constructor only
at the very end, when we're ready to emplace it into the result. (C++
idioms intended! 😄)
With local testing via hyperfine, I'm seeing this be 1-3% faster than
`main` most of the time — though a small number of runs (1 in 10,
maybe?) are a wash or have `main` faster. Codspeed reports a 2%
gain.
## Summary
Use `Unknown | T_inferred` as the type for *undeclared* public symbols.
## Test Plan
- Updated existing tests
- New test for external `__slots__` modifications.
- New tests for external modifications of public symbols.
**Summary**
Airflow 3.0 removes a set of deprecated context variables that were
phased out in 2.x. This PR introduces lint rules to detect usage of
these removed variables in various patterns, helping identify
incompatibilities. The removed context variables include:
```
conf
execution_date
next_ds
next_ds_nodash
next_execution_date
prev_ds
prev_ds_nodash
prev_execution_date
prev_execution_date_success
tomorrow_ds
yesterday_ds
yesterday_ds_nodash
```
**Detected Patterns and Examples**
The linter now flags the use of removed context variables in the
following scenarios:
1. **Direct Subscript Access**
```python
execution_date = context["execution_date"] # Flagged
```
2. **`.get("key")` Method Calls**
```python
print(context.get("execution_date")) # Flagged
```
3. **Variables Assigned from `get_current_context()`**
If a variable is assigned from `get_current_context()` and then used to
access a removed key:
```python
c = get_current_context()
print(c.get("execution_date")) # Flagged
```
4. **Function Parameters in `@task`-Decorated Functions**
Parameters named after removed context variables in functions decorated
with `@task` are flagged:
```python
from airflow.decorators import task
@task
def my_task(execution_date, **kwargs): # Parameter 'execution_date'
flagged
pass
```
5. **Removed Keys in Task Decorator `kwargs` and Other Scenarios**
Other similar patterns where removed context variables appear (e.g., as
part of `kwargs` in a `@task` function) are also detected.
```
from airflow.decorators import task
@task
def process_with_execution_date(**context):
execution_date = lambda: context["execution_date"] # flagged
print(execution_date)
@task(kwargs={"execution_date": "2021-01-01"}) # flagged
def task_with_kwargs(**context):
pass
```
**Test Plan**
Test fixtures covering various patterns of deprecated context usage are
included in this PR. For example:
```python
from airflow.decorators import task, dag, get_current_context
from airflow.models import DAG
from airflow.operators.dummy import DummyOperator
import pendulum
from datetime import datetime
@task
def access_invalid_key_task(**context):
print(context.get("conf")) # 'conf' flagged
@task
def print_config(**context):
execution_date = context["execution_date"] # Flagged
prev_ds = context["prev_ds"] # Flagged
@task
def from_current_context():
context = get_current_context()
print(context["execution_date"]) # Flagged
# Usage outside of a task decorated function
c = get_current_context()
print(c.get("execution_date")) # Flagged
@task
def some_task(execution_date, **kwargs):
print("execution date", execution_date) # Parameter flagged
@dag(
start_date=pendulum.datetime(2021, 1, 1, tz="UTC")
)
def my_dag():
task1 = DummyOperator(
task_id="task1",
params={
"execution_date": "{{ execution_date }}", # Flagged in template context
},
)
access_invalid_key_task()
print_config()
from_current_context()
dag = my_dag()
class CustomOperator(BaseOperator):
def execute(self, context):
execution_date = context.get("execution_date") # Flagged
next_ds = context.get("next_ds") # Flagged
next_execution_date = context["next_execution_date"] # Flagged
```
Ruff will emit `AIR302` diagnostics for each deprecated usage, with
suggestions when applicable, aiding in code migration to Airflow 3.0.
related: https://github.com/apache/airflow/issues/44409,
https://github.com/apache/airflow/issues/41641
---------
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
## Summary
Fixes#9663 and also improves the fixes for
[RUF055](https://docs.astral.sh/ruff/rules/unnecessary-regular-expression/)
since regular expressions are often written as raw strings.
This doesn't include raw f-strings.
## Test Plan
Existing snapshots for RUF055 and PT009, plus a new `Generator` test and
a regression test for the reported `PIE810` issue.
## Summary
Another small PR to focus #15674 solely on the relevant changes. This
makes our Markdown tests less dependent on precise types of public
symbols, without actually changing anything semantically in these tests.
Best reviewed using ignore-whitespace-mode.
## Test Plan
Tested these changes on `main` and on the branch from #15674.
## Summary
Make the remaining `infer.rs` unit tests independent from public symbol
type inference decisions (see upcoming change in #15674).
## Test Plan
- Made sure that the unit tests actually fail if one of the
`assert_type` assertions is changed.
## Summary
Port comprehension tests from Rust to Markdown
I don' think the remaining tests in `infer.rs` should be ported to
Markdown, maybe except for the incremental-checking tests when (if ever)
we have support for that in the MD tests.
closes#13696
## Summary
- Port "deferred annotations" unit tests to Markdown
- Port `implicit_global_in_function` unit test to Markdown
- Removed `resolve_method` and `local_inference` unit tests. These seem
like relics from a time where type inference was in it's early stages.
There is no way that these tests would fail today without lots of other
things going wrong as well.
part of #13696
based on #15683
## Test Plan
New MD tests for existing Rust unit tests.
## Summary
- Add feature to specify a custom typeshed from within Markdown-based
tests
- Port "builtins" unit tests from `infer.rs` to Markdown tests, part of
#13696
## Test Plan
- Tests for the custom typeshed feature
- New Markdown tests for deleted Rust unit tests
## Summary
Addresses the second follow up to #15565 in #15642. This was easier than
expected by using this cool destructuring syntax I hadn't used before,
and by assuming
[PYI059](https://docs.astral.sh/ruff/rules/generic-not-last-base-class/)
(`generic-not-last-base-class`).
## Test Plan
Using an existing test, plus two new tests combining multiple base
classes and multiple generics. It looks like I deleted a relevant test,
which I did, but I meant to rename this in #15565. It looks like instead
I copied it and renamed the copy.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
This PR extends our [PEP 695](https://peps.python.org/pep-0695) handling
from the type aliases handled by `UP040` to generic function and class
parameters, as suggested in the latter two examples from #4617:
```python
# Input
T = TypeVar("T", bound=float)
class A(Generic[T]):
...
def f(t: T):
...
# Output
class A[T: float]:
...
def f[T: float](t: T):
...
```
I first implemented this as part of `UP040`, but based on a brief
discussion during a very helpful pairing session with @AlexWaygood, I
opted to split them into rules separate from `UP040` and then also
separate from each other. From a quick look, and based on [this
issue](https://github.com/asottile/pyupgrade/issues/836), I'm pretty
sure neither of these rules is currently in pyupgrade, so I just took
the next available codes, `UP046` and `UP047`.
The last main TODO, noted in the rule file and in the fixture, is to
handle generic method parameters not included in the class itself, `S`
in this case:
```python
T = TypeVar("T")
S = TypeVar("S")
class Foo(Generic[T]):
def bar(self, x: T, y: S) -> S: ...
```
but Alex mentioned that that might be okay to leave for a follow-up PR.
I also left a TODO about handling multiple subclasses instead of bailing
out when more than one is present. I'm not sure how common that would
be, but I can still handle it here, or follow up on that too.
I think this is unrelated to the PR, but when I ran `cargo dev
generate-all`, it removed the rule code `PLW0101` from
`ruff.schema.json`. It seemed unrelated, so I left that out, but I
wanted to mention it just in case.
## Test Plan
New test fixture, `cargo nextest run`
Closes#4617, closes#12542
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Somehow, I managed to crash the `mdtest` runner today. I struggled to
reproduce this again to see if it's actually fixed (even with an
artificial `sleep` between the two `cargo test` invocations), but the
original backtrace clearly showed that this is where the problem
originated from. And it seems like a clear TOCTOU problem.
## Summary
Raise "invalid-assignment" diagnostics for incorrect assignments to
attributes, for example:
```py
class C:
var: str = "a"
C.var = 1 # error: "Object of type `Literal[1]` is not assignable to `str`"
```
closes#15456
## Test Plan
- Updated test assertions
- New test for assignments to module-attributes
## Summary
This PR generalizes some of the logic we have in `Type::is_subtype_of`
and `Type::is_disjoint_from` so that we fallback to the instance type of
the metaclass more often in `Type::ClassLiteral` and `Type::SubclassOf`
branches. This simplifies the code (we end up with one less branch in
`is_subtype_of`, and we can remove a helper method that's no longer
used), makes the code more robust (any fixes made to subtyping or
disjointness of instance types will automatically improve our
understanding of subtyping/disjointness for class-literal types and
`type[]` types) and more elegantly expresses the type-system invariants
encoded in these branches.
## Test Plan
No new tests added (it's a pure refactor, adding no new functionality).
All existing tests pass, however, including the property tests.
The AST generator creates a reference enum for each syntax group — an
enum where each variant contains a reference to the relevant syntax
node. Previously you could customize the name of the reference enum for
a group — primarily because there was an existing `ExpressionRef` type
that wouldn't have lined up with the auto-derived name `ExprRef`. This
follow-up PR is a simple search/replace to switch over to the
auto-derived name, so that we can remove this customization point.
This is a minor cleanup to the AST generation script to make a clearer
separation between nodes that do appear in a group enum, and those that
don't. There are some types and methods that we create for every syntax
node, and others that refer to the group that the syntax node belongs
to, and which therefore don't make sense for ungrouped nodes. This new
separation makes it clearer which category each definition is in, since
you're either inside of a `for group in ast.groups` loop, or a `for node
in ast.all_nodes` loop.
## Summary
Test executables usually write failure messages (including panics) to
stdout, but I just managed to make a mdtest crash with
```
thread 'mdtest__unary_not' has overflowed its stack
fatal runtime error: stack overflow
```
which is printed to stderr. This test simply appends stderr to stdout
(`stderr=subprocess.STDOUT` can not be used with `capture_output`)
## Test Plan
Make sure that the error message is now visible in the output of `uv -q
run crates/red_knot_python_semantic/mdtest.py`
## Summary
The `Options` struct is intended to capture the user's configuration
options but
`EnvironmentOptions::venv_path` supports both a `SitePackages::Known`
and `SitePackages::Derived`.
Users should only be able to provide `SitePackages::Derived`—they
specify a path to a venv, and Red Knot derives the path to the
site-packages directory. We'll only use the `Known` variant once we
automatically discover the Python installation.
That's why this PR changes `EnvironmentOptions::venv_path` from
`Option<SitePackages>` to `Option<SystemPathBuf>`.
This requires making some changes to the file watcher test, and I
decided to use `extra_paths` over venv path
because our venv validation is annoyingly correct -- making mocking a
venv rather involved.
## Test Plan
`cargo test`
## Summary
We were mistakenly using `CommentRanges::has_comments` to determine
whether our edits
were safe, which sometimes expands the checked range to the end of a
line. But in order to
determine safety we need to check exactly the range we're replacing.
This bug affected the rules `runtime-cast-value` (`TC006`) and
`quoted-type-alias` (`TC008`)
although it was very unlikely to be hit for `TC006` and for `TC008` we
never hit it because we
were checking the wrong expression.
## Test Plan
`cargo nextest run`
This commit fixes RUF055 rule to format `re.fullmatch(pattern, var)` to
`var == pattern` instead of the current `pattern == var` behaviour. This
is more idiomatic and easy to understand.
## Summary
This changes the current formatting behaviour of `re.fullmatch(pattern,
var)` to format it to `var == pattern` instead of `pattern == var`.
## Test Plan
I used a code file locally to see the updated formatting behaviour.
Fixes https://github.com/astral-sh/ruff/issues/14733
## Summary
As more and more tests move to Markdown, running the mdtest suite
becomes one of the most common tasks for developers working on Red Knot.
There are a few pain points when doing so, however:
- The `build.rs` script enforces recompilation (~five seconds) whenever
something changes in the `resource/mdtest` folder. This is strictly
necessary, because whenever files are added or removed, the test harness
needs to be updated. But this is very rarely the case! The most common
scenario is that a Markdown file has *changed*, and in this case, no
recompilation is necessary. It is currently not possible to distinguish
these two cases using `cargo::rerun-if-changed`. One can work around
this by running the test executable manually, but it requires finding
the path to the correct `mdtest-<random-hash>` executable.
- All Markdown tests are run by default. This is needed whenever Rust
code changes, but while working on the tests themselves, it is often
much more convenient to only run the tests for a single file. This can
be done by using a `mdtest__path_to_file` filter, but this needs to be
manually spelled out or copied from the test output.
- `cargo`s test output for a failing Markdown test is often
unnecessarily verbose. Unless there is an *actual* panic somewhere in
the code, mdtests usually fail with the explicit *"Some tests failed"*
panic in the mdtest suite. But in those cases, we are not interested in
the pointer to the source of this panic, but only in the mdtest suite
output.
This PR adds a Markdown test runner tool that attempts to make the
developer experience better.
Once it is started using
```bash
uv run -q crates/red_knot_python_semantic/mdtest.py
```
it will first recompile the tests once (if cargo requires it), find the
path to the `mdtest` executable, and then enter into a mode where it
watches for changes in the `red_knot_python_semantic` crate. Whenever …
* … a Markdown file changes, it will rerun the mdtest for this specific
file automatically (no recompilation!).
* … a Markdown file is added, it will recompile the tests and then run
the mdtest for the new file
* … Rust code is changed, it will recompile the tests and run all of
them
The tool also trims down `cargo test` output and only shows the actual
mdtest errors.
The tool will certainly require a few more iterations before it becomes
mature, but I'm curious to hear if there is any interest for something
like this.
## Test Plan
- Tested the new runner under various scenarios.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Rename two functions with outdated names (they used to return `Type`s):
* `bindings_ty` => `symbol_from_bindings` (returns `Symbol`)
* `declarations_ty` => `symbol_from_declarations` (returns a
`SymbolAndQualifiers` result)
I chose `symbol_from_*` instead of `*_symbol` as I found the previous
name quite confusing. Especially since `binding_ty` and `declaration_ty`
also exist (singular).
## Test Plan
—
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
## Summary
In preperation for https://github.com/astral-sh/ruff/pull/15558
Isolate the `show_settings` test instead of reading Ruff's
`pyproject.toml` for better test isolation.
## Test Plan
`cargo test`
## Summary
I noticed this while reviewing
https://github.com/astral-sh/ruff/pull/15541 that the code inside the
large closure cannot be formatted by the Rust formatter. This PR
extracts the qualified name and inlines the match expression.
## Test Plan
`cargo clippy` and `cargo insta`
## Summary
Right now, these are being applied in random order, since if we have two
`RedefinitionWhileUnused`, it just takes the first-generated (whereas
the next comparator in the sort here orders by location)... Which means
we frequently have to re-run!
## Summary
The fix range for sorting imports accounts for trailing whitespace, but
we should only show the trimmed range to the user when displaying the
diagnostic. So this PR changes the diagnostic range.
Closes#15504
## Test Plan
Reviewed snapshot changes
## Summary
Added some extra notes on why you should have focused try...except
blocks to
[TRY300](https://docs.astral.sh/ruff/rules/try-consider-else/).
When fixing a violation of this rule, a co-worker of mine (very
understandably) asked why this was better. The current docs just say
putting the return in the else is "more explicit", but if you look at
the [linked reference in the python
documentation](https://docs.python.org/3/tutorial/errors.html) they are
more clear on why violations like this is bad:
> The use of the else clause is better than adding additional code to
the [try](https://docs.python.org/3/reference/compound_stmts.html#try)
clause because it avoids accidentally catching an exception that wasn’t
raised by the code being protected by the try … except statement.
This is my attempt at adding more context to the docs on this. Open to
suggestions for wording!
---------
Co-authored-by: dylwil3 <dylwil3@gmail.com>
In the following situation:
```python
class Grandparent:
__slots__ = "a"
class Parent(Grandparent): ...
class Child(Parent):
__slots__ = "a"
```
the message for `W0244` now specifies that `a` is overwriting a slot
from `Grandparent`.
To implement this, we introduce a helper function `iter_super_classes`
which does a breadth-first traversal of the superclasses of a given
class (as long as they are defined in the same file, due to the usual
limitations of the semantic model).
Note: Python does not allow conflicting slots definitions under multiple
inheritance. Unless I'm misunderstanding something, I believe It follows
that the subposet of superclasses of a given class that redefine a given
slot is in fact totally ordered. There is therefore a unique _nearest_
superclass whose slot is being overwritten. So, you know, in case anyone
was super worried about that... you can just chill.
This is a followup to #9640 .
## Summary
Add support for `typing.ClassVar`, i.e. emit a diagnostic in this
scenario:
```py
from typing import ClassVar
class C:
x: ClassVar[int] = 1
c = C()
c.x = 3 # error: "Cannot assign to pure class variable `x` from an instance of type `C`"
```
## Test Plan
- New tests for the `typing.ClassVar` qualifier
- Fixed one TODO in `attributes.md`
While looking into potential AST optimizations, I noticed the `AstNode`
trait and `AnyNode` type aren't used anywhere in Ruff or Red Knot. It
looks like they might be historical artifacts of previous ways of
consuming AST nodes?
- `AstNode::cast`, `AstNode::cast_ref`, and `AstNode::can_cast` are not
used anywhere.
- Since `cast_ref` isn't needed anymore, the `Ref` associated type isn't
either.
This is a pure refactoring, with no intended behavior changes.
This PR replaces most of the hard-coded AST definitions with a
generation script, similar to what happens in `rust_python_formatter`.
I've replaced every "rote" definition that I could find, where the
content is entirely boilerplate and only depends on what syntax nodes
there are and which groups they belong to.
This is a pretty massive diff, but it's entirely a refactoring. It
should make absolutely no changes to the API or implementation. In
particular, this required adding some configuration knobs that let us
override default auto-generated names where they don't line up with
types that we created previously by hand.
## Test plan
There should be no changes outside of the `rust_python_ast` crate, which
verifies that there were no API changes as a result of the
auto-generation. Aggressive `cargo clippy` and `uvx pre-commit` runs
after each commit in the branch.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes parentheses not being stripped in C401. Pretty much the same as
#11607 which fixed it for C400.
## Test Plan
`cargo nextest run`
## Summary
This is a small, tentative step towards the bigger goal of understanding
instance attributes.
- Adds partial support for pure instance variables declared in the class
body, i.e. this case:
```py
class C:
variable1: str = "a"
variable2 = "b"
reveal_type(C().variable1) # str
reveal_type(C().variable2) # Unknown | Literal["b"]
```
- Adds `property` as a known class to query for `@property` decorators
- Splits up various `@Todo(instance attributes)` cases into
sub-categories.
## Test Plan
Modified existing MD tests.
## Summary
This PR adds support for configuring Red Knot in the `tool.knot` section
of the project's
`pyproject.toml` section. Options specified on the CLI precede the
options in the configuration file.
This PR only supports the `environment` and the `src.root` options for
now.
Other options will be added as separate PRs.
There are also a few concerns that I intentionally ignored as part of
this PR:
* Handling of relative paths: We need to anchor paths relative to the
current working directory (CLI), or the project (`pyproject.toml` or
`knot.toml`)
* Tracking the source of a value. Diagnostics would benefit from knowing
from which configuration a value comes so that we can point the user to
the right configuration file (or CLI) if the configuration is invalid.
* Schema generation and there's a lot more; see
https://github.com/astral-sh/ruff/issues/15491
This PR changes the default for first party codes: Our existing default
was to only add the project root. Now, Red Knot adds the project root
and `src` (if such a directory exists).
Theoretically, we'd have to add a file watcher event that changes the
first-party search paths if a user later creates a `src` directory. I
think this is pretty uncommon, which is why I ignored the complexity for
now but I can be persuaded to handle it if it's considered important.
Part of https://github.com/astral-sh/ruff/issues/15491
## Test Plan
Existing tests, new file watching test demonstrating that changing the
python version and platform is correctly reflected.
## Summary
Closes https://github.com/astral-sh/ruff/issues/15508
For any two instance types `T` and `S`, we know they are disjoint if
either `T` is final and `T` is not a subclass of `S` or `S` is final and
`S` is not a subclass of `T`.
Correspondingly, for any two types `type[T]` and `S` where `S` is an
instance type, `type[T]` can be said to be disjoint from `S` if `S` is
disjoint from `U`, where `U` is the type that represents all instances
of `T`'s metaclass.
And a heterogeneous tuple type can be said to be disjoint from an
instance type if the instance type is disjoint from `tuple` (a type
representing all instances of the `tuple` class at runtime).
## Test Plan
- A new mdtest added. Most of our `is_disjoint_from()` tests are not
written as mdtests just yet, but it's pretty hard to test some of these
edge cases from a Rust unit test!
- Ran `QUICKCHECK_TESTS=1000000 cargo test --release -p
red_knot_python_semantic -- --ignored types::property_tests::stable`
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Add a setting to allow ignoring one line docstrings for the pydoclint
rules.
Resolves#13086
Part of #12434
## Test Plan
Run tests with setting enabled.
---------
Co-authored-by: dylwil3 <dylwil3@gmail.com>
## Summary
This fixes the infinite loop reported in #14389 by raising an error to
the user about conflicting ICN001 (`unconventional-import-alias`) and
I002 (`missing-required-import`) configuration options.
## Test Plan
Added a CLI integration test reproducing the old behavior and then
confirming the fix.
Closes#14389
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
This fixes the infinite loop reported in #12897, where an
`unused-import` that is undefined at the scope of `__all__` is "fixed"
by adding it to `__all__` repeatedly. These changes make it so that only
imports in the global scope will be suggested to add to `__all__` and
the unused local import is simply removed.
## Test Plan
Added a CLI integration test that sets up the same module structure as
the original report
Closes#12897
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Ref: https://github.com/astral-sh/ruff/pull/15387#discussion_r1917796907
This PR updates `F722` to show syntax error message instead of the
string content.
I think it's more useful to show the syntax error message than the
string content. In the future, when the diagnostics renderer is more
capable, we could even highlight the exact location of the syntax error
along with the annotation string.
This is also in line with how we show the diagnostic in red knot.
## Test Plan
Update existing test snapshots.
## Summary
Resolves#9467
Parse quoted annotations as if the string content is inside parenthesis.
With this logic `x` and `y` in this example are equal:
```python
y: """
int |
str
"""
z: """(
int |
str
)
"""
```
Also this rule only applies to triple
quotes([link](https://github.com/python/typing-council/issues/9#issuecomment-1890808610)).
This PR is based on the
[comments](https://github.com/astral-sh/ruff/issues/9467#issuecomment-2579180991)
on the issue.
I did one extra change, since we don't want any indentation tokens I am
setting the `State::Other` as the initial state of the Lexer.
Remaining work:
- [x] Add a test case for red-knot.
- [x] Add more tests.
## Test Plan
Added a test which previously failed because quoted annotation contained
indentation.
Added an mdtest for red-knot.
Updated previous test.
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Allow links to issues that appear on the same line as the TODO
directive, if they conform to the format that VSCode's GitHub PR
extension produces.
Revival of #9627 (the branch was stale enough that rebasing was a lot
harder than just making the changes anew). Credit should go to the
author of that PR though.
Closes#8061
Co-authored-by: Martin Bernstorff <martinbernstorff@gmail.com>
Instead of doing this on a lint-by-lint basis, we now just do it right
before rendering. This is more broadly applicable.
Note that this doesn't fix the diagnostic rendering for the Python
parser. But that's using a different path anyway (`annotate-snippets` is
only used in tests).
Previously, these were pointing to the right place, but were missing the
`^`. With the `annotate-snippets` upgrade, the `^` was added, but they
started pointing to the end of the previous line instead of the
beginning of the following line. In this case, we really want it to
point to the beginning of the following line since we're calling out
indentation issues.
As in a prior commit, we fix this by tweaking the offsets emitted by the
lint itself. Instead of an empty range at the beginning of the line, we
point to the first character in the line. This "forces" the renderer to
point to the beginning of the line instead of the end of the preceding
line.
The end effect here is that the rendering is fixed by adding `^` in the
proper location.
This update includes some missing `^` in the diagnostic annotations.
This update also includes some shifting of "syntax error" annotations to
the end of the preceding line. I believe this is technically a
regression, but fixing them has proven quite difficult. I *think* the
best way to do that might be to tweak the spans generated by the Python
parser errors, but I didn't want to dig into that. (Another approach
would be to change the `annotate-snippets` rendering, but when I tried
that and managed to fix these regressions, I ended up causing a bunch of
other regressions.)
Ref 77d454525e (r1915458616)
This change also requires some shuffling to the offsets we generate for
the diagnostic. Previously, we were generating an empty range
immediately *after* the line terminator and immediate before the first
byte of the subsequent line. How this is rendered is somewhat open to
interpretation, but the new version of `annotate-snippets` chooses to
render this at the end of the preceding line instead of the beginning of
the following line.
In this case, we want the diagnostic to point to the beginning of the
following line. So we either need to change `annotate-snippets` to
render such spans at the beginning of the following line, or we need to
change our span to point to the first full character in the following
line. The latter will force `annotate-snippets` to move the caret to the
proper location.
I ended up deciding to change our spans instead of changing how
`annotate-snippets` renders empty spans after a line terminator. While I
didn't investigate it, my guess is that they probably had good reason
for doing so, and it doesn't necessarily strike me as _wrong_.
Furthermore, fixing up our spans seems like a good idea regardless, and
was pretty easy to do.
This looks like a bug fix since the caret is now pointing right at the
position of the unprintable character. I'm not sure if this is a result
of an improvement via the `annotate-snippets` upgrade, or because of
more accurate tracking of annotation ranges even after unprintable
characters are replaced. I'm tempted to say the former since in theory
the offsets were never wrong before because they were codepoint offsets.
Regardless, this looks like an improvement.
This updates snapshots where long lines now get trimmed with
`annotate-snippets`. And an ellipsis is inserted to indicate trimming.
This is a little hokey to test since in tests we don't do any styling.
And I believe this just uses the default "max term width" for rendering.
But in real life, it seems like a big improvement to have long lines
trimmed if they would otherwise wrap in the terminal. So this seems like
an improvement to me.
There are some other fixes here that overlap with previous categories.
We do this because `...` is valid Python, which makes it pretty likely
that some line trimming will lead to ambiguous output. So we add support
for overriding the cut indicator. This also requires changing some of
the alignment math, which was previously tightly coupled to `...`.
For Ruff, we go with `…` (`U+2026 HORIZONTAL ELLIPSIS`) for our cut
indicator.
For more details, see the patch sent to upstream:
https://github.com/rust-lang/annotate-snippets-rs/pull/172
This fix was sent upstream and the PR description includes more details:
https://github.com/rust-lang/annotate-snippets-rs/pull/170
Without this fix, there was an errant snapshot diff that looked like
this:
|
1 | version = "0.1.0"
2 | # Ensure that the spans from toml handle utf-8 correctly
3 | authors = [
| ___________^
4 | | { name = "Z͑ͫ̓ͪ̂ͫ̽͏̴̙...A̴̵̜̰͔ͫ͗͢L̠ͨͧͩ͘G̴̻͈͍̑͗̎̅͛́Ǫ̵̹̻̝̳͂̌̌͘", email = 1 }
5 | | ]
| |_^ RUF200
|
That ellipsis should _not_ be inserted since the line is not actually
truncated. The handling of line length (in bytes versus actual rendered
length) wasn't quite being handled correctly in all cases.
With this fix, there's (correctly) no snapshot diff.
The change to the rendering code is elaborated on in more detail here,
where I attempted to upstream it:
https://github.com/rust-lang/annotate-snippets-rs/pull/169
Otherwise, the snapshot diff also shows a bug fix: a `^` is now rendered
where as it previously was not.
This one almost looks like it fits into the other failure categories,
but without identifying root causes, it's hard to say for sure. The span
here does end after a line terminator, so it feels like it's like the
rest.
I also isolated this change since I found the snapshot diff pretty hard
to read and wanted to look at it more closely. In this case, the before
is:
E204.py:31:2: E204 [*] Whitespace after decorator
|
30 | # E204
31 | @ \
| __^
32 | | foo
| |_^ E204
33 | def baz():
34 | print('baz')
|
= help: Remove whitespace
And the after is:
E204.py:31:2: E204 [*] Whitespace after decorator
|
30 | # E204
31 | @ \
| ^^ E204
32 | foo
33 | def baz():
34 | print('baz')
|
= help: Remove whitespace
The updated rendering is clearly an improvement, since `foo` itself is
not really the subject of the diagnostic. The whitespace is.
Also, the new rendering matches the span fed to `annotate-snippets`,
where as the old rendering does not.
I separated out this snapshot update since the string of `^` including
whitespace looked a little odd. I investigated this one specifically,
and indeed, our span in this case is telling `annotate-snippets` to
point at the whitespace. So this is `annotate-snippets` doing what it's
told with a mildly sub-optimal span.
For clarity, the before rendering is:
skip.py:34:1: I001 [*] Import block is un-sorted or un-formatted
|
32 | import sys; import os # isort:skip
33 | import sys; import os # isort:skip # isort:skip
34 | / import sys; import os
|
= help: Organize imports
And now after is:
skip.py:34:1: I001 [*] Import block is un-sorted or un-formatted
|
32 | import sys; import os # isort:skip
33 | import sys; import os # isort:skip # isort:skip
34 | import sys; import os
| ^^^^^^^^^^^^^^^^^^^^^^^^^ I001
|
= help: Organize imports
This is a clear bug fix since it adds in the `I001` annotation, even
though the carets look a little funny by including the whitespace
preceding `import sys; import os`.
This group of updates is similar to the last one, but they call out the
fact that while the change is an improvement, it does still seem to be a
little buggy.
As one example, previously we would have this:
|
1 | / from __future__ import annotations
2 | |
3 | | from typing import Any
4 | |
5 | | from requests import Session
6 | |
7 | | from my_first_party import my_first_party_object
8 | |
9 | | from . import my_local_folder_object
10 | |
11 | |
12 | |
13 | | class Thing(object):
| |_^ I001
14 | name: str
15 | def __init__(self, name: str):
|
= help: Organize imports
And now here's what it looks like after:
|
1 | / from __future__ import annotations
2 | |
3 | | from typing import Any
4 | |
5 | | from requests import Session
6 | |
7 | | from my_first_party import my_first_party_object
8 | |
9 | | from . import my_local_folder_object
10 | |
11 | |
12 | |
| |__^ Organize imports
13 | class Thing(object):
14 | name: str
15 | def __init__(self, name: str):
|
= help: Organize imports
So at least now, the diagnostic is not pointing to a completely
unrelated thing (`class Thing`), but it's still not quite pointing to
the imports directly. And the `^` is a bit offset. After looking at
some examples more closely, I think this is probably more of a bug
with how we're generating offsets, since we are actually pointing to
a location that is a few empty lines _below_ the last import. And
`annotate-snippets` is rendering that part correctly. However, the
offset from the left (the `^` is pointing at `r` instead of `f` or even
at the end of `from . import my_local_folder_object`) appears to be a
problem with `annotate-snippets` itself.
We accept this under the reasoning that it's an improvement, albeit not
perfect.
I believe this case is different from the last in that it happens when
the end of a *multi-line* annotation occurs after a line terminator.
Previously, the diagnostic would render on the next line, which is
definitely a bit weird. This new update renders it at the end of the
line the annotation ends on.
In some cases, the annotation was previously rendered to point at source
lines below where the error occurred, which is probably pretty
confusing.
This looks like a bug fix that occurs when the annotation is a
zero-width span immediately following a line terminator. Previously, the
caret seems to be rendered on the next line, but it should be rendered
at the end of the line the span corresponds to.
I admit that this one is kinda weird. I would somewhat expect that our
spans here are actually incorrect, and that to obtain this sort of
rendering, we should identify a span just immediately _before_ the line
terminator and not after it. But I don't want to dive into that rabbit
hole for now (and given how `annotate-snippets` now renders these
spans, perhaps there is more to it than I see), and this does seem like
a clear improvement given the spans we feed to `annotate-snippets`.
The previous rendering just seems wrong in that a `^` is omitted. The
new version of `annotate-snippets` seems to get this right. I checked a
pseudo random sample of these, and it seems to only happen when the
position pointed at a line terminator.
It's hard to grok the change from the snapshot diffs alone, so here's
one example. Before:
PYI021.pyi:15:5: PYI021 [*] Docstrings should not be included in stubs
|
14 | class Baz:
15 | """Multiline docstring
| _____^
16 | |
17 | | Lorem ipsum dolor sit amet
18 | | """
| |_______^ PYI021
19 |
20 | def __init__(self) -> None: ...
|
= help: Remove docstring
And now after:
PYI021.pyi:15:5: PYI021 [*] Docstrings should not be included in stubs
|
14 | class Baz:
15 | / """Multiline docstring
16 | |
17 | | Lorem ipsum dolor sit amet
18 | | """
| |_______^ PYI021
19 |
20 | def __init__(self) -> None: ...
|
= help: Remove docstring
I personally think both of these are fine. If we felt strongly, I could
investigate reverting to the old style, but the new style seems okay to
me.
In other words, these updates I believe are just cosmetic and not a bug
fix.
These updates center around the addition of annotations in the
diagnostic rendering. Previously, the annotation was just not rendered
at all. With the `annotate-snippets` upgrade, it is now rendered. I
examined a pseudo random sample of these, and they all look correct.
As will be true in future batches, some of these snapshots also have
changes to whitespace in them as well.
These snapshot changes should *all* only be a result of changes to
trailing whitespace in the output. I checked a psuedo random sample of
these, and the whitespace found in the previous snapshots seems to be an
artifact of the rendering and _not_ of the source data. So this seems
like a strict bug fix to me.
There are other snapshots with whitespace changes, but they also have
other changes that we split out into separate commits. Basically, we're
going to do approximately one commit per category of change.
This represents, by far, the biggest chunk of changes to snapshots as a
result of the `annotate-snippets` upgrade.
Previously, we were replacing unprintable ASCII characters with a
printable representation of them via fancier Unicode characters. Since
`annotate-snippets` used to use codepoint offsets, this didn't make our
ranges incorrect: we swapped one codepoint for another.
But now, with the `annotate-snippets` upgrade, we use byte offsets
(which is IMO the correct choice). However, this means our ranges can be
thrown off since an ASCII codepoint is always one byte and a non-ASCII
codepoint is always more than one byte.
Instead of tweaking the `ShowNonprinting` trait and making it more
complicated (which is used in places other than this diagnostic
rendering it seems), we instead change `replace_whitespace` to handle
non-printable characters. This works out because `replace_whitespace`
was already updating the annotation range to account for the tab
replacement. We copy that approach for unprintable characters.
This is pretty much just moving to the new API and taking care to use
byte offsets. This is *almost* enough. The next commit will fix a bug
involving the handling of unprintable characters as a result of
switching to byte offsets.
This is a tiny change that, perhaps slightly shady, permits us to use
the `annotate-snippets` renderer without its mandatory header (which
wasn't there in `annotate-snippets 0.9`). Specifically, we can now do
this:
Level::None.title("")
The combination of a "none" level and an empty label results in the
`annotate-snippets` header being skipped entirely. (Not even an empty
line is written.)
This is maybe not the right API for upstream `annotate-snippets`, but
it's very easy for us to do and unblocks the upgrade (albeit relying on
a vendored copy).
Ref https://github.com/rust-lang/annotate-snippets-rs/issues/167
This merely adds the crate to our repository. Some cosmetic changes are
made to make it work in our repo and follow our conventions, such as
changing the name to `ruff_annotate_snippets`. We retain the original
license information. We do drop some things, such as benchmarks, but
keep tests and examples.
## Summary
The initial purpose was to fix#15043, where code like this:
```python
from fastapi import FastAPI, Query
app = FastAPI()
@app.get("/test")
def handler(echo: str = Query("")):
return echo
```
was being fixed to the invalid code below:
```python
from typing import Annotated
from fastapi import FastAPI, Query
app = FastAPI()
@app.get("/test")
def handler(echo: Annotated[str, Query("")]): # changed
return echo
```
As @MichaReiser pointed out, the correct fix is:
```python
from typing import Annotated
from fastapi import FastAPI, Query
app = FastAPI()
@app.get("/test")
def handler(echo: Annotated[str, Query()] = ""): # changed
return echo
```
After fixing the issue for `Query`, I realized that other classes like
`Path`, `Body`, `Cookie`, `Header`, `File`, and `Form` also looked
susceptible to this issue. The last few commits should handle these too,
which I think means this will also close#12913.
I had to reorder the arguments to the `do_stuff` test case because the
new fix removes some default argument values (eg for `Path`:
`some_path_param: str = Path()` becomes `some_path_param: Annotated[str,
Path()]`).
There's also #14484 related to this rule. I'm happy to take a stab at
that here or in a follow up PR too.
## Test Plan
`cargo test`
I also checked the fixed output with `uv run --with fastapi
FAST002_0.py`, but it required making a bunch of additional changes to
the test file that I wasn't sure we wanted in this PR.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
If `S <: T`, then `~T <: ~S`. This test currently fails with example
like:
```
S = tuple[()]
T = ~Literal[True] & ~Literal[False]
```
`T` is equivalent to `~(Literal[True] | Literal[False])` and therefore
equivalent to `~bool`, but the minimal example for a failure is what is
stated above. We correctly recognize that `S <: T`, but fail to see that
`~T <: ~S`, i.e. `bool <: ~tuple[()]`.
This is why the tests goes into the "flaky" section as well.
## Test Plan
```
export QUICKCHECK_TESTS=100000
while cargo test --release -p red_knot_python_semantic -- --ignored types::property_tests::flaky::negation_reverses_subtype_order; do :; done
```
## Summary
Adds some initial tests for class and instance attributes, mostly to
document (and discuss) what we want to support eventually. These
tests are not exhaustive yet. The idea is to specify the coarse-grained
behavior first.
Things that we'll eventually want to test:
- Interplay with inheritance
- Support `Final` in addition to `ClassVar`
- Specific tests for `ClassVar`, like making sure that we support things
like `x: Annotated[ClassVar[int], "metadata"]`
- … or making sure that we raise an error here:
```py
class Foo:
def __init__(self):
self.x: ClassVar[str] = "x"
```
- Add tests for `__new__` in addition to the tests for `__init__`
- Add tests that show that we use the union of types if multiple methods
define the symbol with different types
- Make sure that diagnostics are raised if, e.g., the inferred type of
an assignment within a method does not match the declared type in the
class body.
- https://github.com/astral-sh/ruff/pull/15474#discussion_r1916556284
- Method calls are completely left out for now.
- Same for `@property`
- … and the descriptor protocol
## Test Plan
New Markdown tests
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
The next sync of typeshed would have failed without manual changes
anyway, so I'm doing one manual sync + the required changes in our
`sys.platform` tests (which are necessary because of my tiny typeshed PR
here: https://github.com/python/typeshed/pull/13378).
closes#15485 (the next run of the pipeline in two weeks should be fine
as the bug has been fixed upstream)
## Summary
Adds two additional tests for `is_equivalent_to` so that we cover all
properties of an [equivalence relation].
## Test Plan
```
while cargo test --release -p red_knot_python_semantic -- --ignored types::property_tests::stable; do :; done
```
[equivalence relation]:
https://en.wikipedia.org/wiki/Equivalence_relation
## Summary
This PR fixes the `show_*_msg` macros to pass all the tokens instead of
just a single token. This allows for using various expressions right in
the macro similar to how it would be in `format_args!`.
## Test Plan
`cargo clippy`
## Summary
This PR creates separate functions to check whether the document path is
excluded for linting or formatting. The main motivation is to avoid the
double `Option` for the call sites and makes passing the correct
settings simpler.
## Summary
This changeset adds new tests for public uses of symbols,
considering all possible declaredness and boundness states.
Note that this is a mere documentation of the current behavior. There is
still an [open ticket] questioning some of these choices (or unintential
behaviors).
## Test plan
Made sure that the respective test fails if I add the questionable case
again in `symbol_by_id`:
```rs
Symbol::Type(inferred_ty, Boundness::Bound) => {
Symbol::Type(inferred_ty, Boundness::Bound)
}
```
[open ticket]: https://github.com/astral-sh/ruff/issues/14297
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Replace typo "security_managr" in AIR303 as "security_manager"
## Test Plan
<!-- How was it tested? -->
a test fixture has been updated
## Summary
Simplification follow-up to #15413.
There's no need to have a dedicated `CallOutcome` variant for every
known function, it's only necessary if the special-cased behavior of the
known function includes emitting extra diagnostics. For `typing.cast`,
there's no such need; we can use the regular `Callable` outcome variant,
and update the return type according to the cast. (This is the same way
we already handle `len`.)
One reason to avoid proliferating unnecessary `CallOutcome` variants is
that currently we have to explicitly add emitting call-binding
diagnostics, for each outcome variant. So we were previously wrongly
silencing any binding diagnostics on calls to `typing.cast`. Fixing this
revealed a separate bug, that we were emitting a bogus error anytime
more than one keyword argument mapped to a `**kwargs` parameter. So this
PR also adds test and fix for that bug.
## Test Plan
Existing `cast` tests pass unchanged, added new test for `**kwargs` bug.
## Summary
I noticed this while trying out
https://github.com/astral-sh/ruff-vscode/issues/665 that we use the
`Display` implementation to show the error which hides the context. This
PR changes it to use the `Debug` implementation and adds the message as
a context.
## Test Plan
**Before:**
```
0.001228084s ERROR main ruff_server::session::index::ruff_settings: Unable to find editor-specified configuration file: Failed to parse /private/tmp/hatch-test/ruff.toml
```
**After:**
```
0.002348750s ERROR main ruff_server::session::index::ruff_settings: Unable to load editor-specified configuration file
Caused by:
0: Failed to parse /private/tmp/hatch-test/ruff.toml
1: TOML parse error at line 2, column 18
|
2 | extend-select = ["ASYNC101"]
| ^^^^^^^^^^
Unknown rule selector: `ASYNC101`
```
## Summary
In `SymbolState` merging, use `BitSet::union` instead of inserting
declarations one by one. This used to be the case but was changed in
https://github.com/astral-sh/ruff/pull/15019 because we had to iterate
over declarations anyway.
This is an alternative to https://github.com/astral-sh/ruff/pull/15419
by @MichaReiser. It's similar in performance, but a bit more
declarative and less imperative.
## Summary
Follow-up PR from https://github.com/astral-sh/ruff/pull/15415🥲
The exact same property test already exists:
`intersection_assignable_to_both` and
`all_type_pairs_can_be_assigned_from_their_intersection`
## Test Plan
`cargo test -p red_knot_python_semantic -- --ignored
types::property_tests::flaky`
## Summary
Implements upstream diagnostics `PT029`, `PT030`, `PT031` that function
as pytest.warns corollaries of `PT010`, `PT011`, `PT012` respectively.
Most of the implementation and documentation is designed to mirror those
existing diagnostics.
Closes#14239
## Test Plan
Tests for `PT029`, `PT030`, `PT031` largely copied from `PT010`,
`PT011`, `PT012` respectively.
`cargo nextest run`
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
A small PR to reduce some of the code duplication between the various
branches, make it a little more readable and move the API closer to what
we already have for `KnownClass`
Brent Westbrook
Alex Waygood
David Peter
Mike Perlov
Alexander Nordin
Wei Lee
InSync
Douglas Creager
Justin Bramley
Vasco Schiavo
renovate[bot]
Dhruv Manilawala
Tom Kuson
Micha Reiser
github-actions[bot]
Andrew Gallant
Carl Meyer
Dylan
Leo Gaskin
Garrett Reynolds
Marek Hanuš
Marcus Näslund
Charlie Marsh
Jelle Zijlstra
Zanie Blue
Ankit Chaurasia
David Salvisberg
Chandra Kiran G
Calum Young
wooly18
Will Lachance
Akira Noda
guillaumeLepape
Auguste Lalande
Aleksei Latyshev
Shaygan Hooshyari
cake-monotone