Commit Graph

5596 Commits

Author SHA1 Message Date
Alex Waygood a06099dffe
[red-knot] Move attribute access on `ModuleLiteral` types into a dedicated method (#15067) 2024-12-19 16:02:16 +00:00
Alex Waygood bb43085939
[red-knot] Reduce TODOs in `Type::member()` (#15066) 2024-12-19 15:54:01 +00:00
Dylan c1eaf6ff72
Modify parsing of raise with cause when exception is absent (#15049)
When confronted with `raise from exc` the parser will now create a
`StmtRaise` that has `None` for the exception and `exc` for the cause.

Before, the parser created a `StmtRaise` with `from` for the exception,
no cause, and a spurious expression `exc` afterwards.
2024-12-19 13:36:32 +00:00
Dhruv Manilawala 3bb0dac235
Bump version to 0.8.4 (#15064) 2024-12-19 13:15:42 +00:00
Alex Waygood 40cba5dc8a
[red-knot] Cleanup various `todo_type!()` messages (#15063)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-19 13:03:41 +00:00
Dylan 596d80cc8e
[`perflint`] Parenthesize walrus expressions in autofix for `manual-list-comprehension` (`PERF401`) (#15050) 2024-12-19 06:56:45 -06:00
Taras Matsyk 85e71ba91a
[`flake8-bandit`] Check `S105` for annotated assignment (#15059)
## Summary

A follow up PR on https://github.com/astral-sh/ruff/issues/14991
Ruff ignores hardcoded passwords for typed variables. Add a rule to
catch passwords in typed code bases

## Test Plan

Includes 2 more test typed variables
2024-12-19 12:26:40 +00:00
Douglas Creager 2802cbde29
Don't special-case class instances in unary expression inference (#15045)
We have a handy `to_meta_type` that does the right thing for class
instances, and also works for all of the other types that are “instances
of” something. Unless I'm missing something, this should let us get rid
of the catch-all clause in one fell swoop.

cf #14548
2024-12-18 14:37:17 -05:00
InSync ed2bce6ebb
[red-knot] Report invalid exceptions (#15042)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-18 18:31:24 +00:00
InSync f0012df686
Fix typos in `RUF043.py` (#15044)
(Accidentally introduced in #14966.)
2024-12-18 15:39:55 +00:00
Micha Reiser 0fc4e8f795
Introduce `InferContext` (#14956)
## Summary

I'm currently on the fence about landing the #14760 PR because it's
unclear how we'd support tracking used and unused suppression comments
in a performant way:
* Salsa adds an "untracked" dependency to every query reading
accumulated values. This has the effect that the query re-runs on every
revision. For example, a possible future query
`unused_suppression_comments(db, file)` would re-run on every
incremental change and for every file. I don't expect the operation
itself to be expensive, but it all adds up in a project with 100k+ files
* Salsa collects the accumulated values by traversing the entire query
dependency graph. It can skip over sub-graphs if it is known that they
contain no accumulated values. This makes accumulators a great tool for
when they are rare; diagnostics are a good example. Unfortunately,
suppressions are more common, and they often appear in many different
files, making the "skip over subgraphs" optimization less effective.

Because of that, I want to wait to adopt salsa accumulators for type
check diagnostics (we could start using them for other diagnostics)
until we have very specific reasons that justify regressing incremental
check performance.

This PR does a "small" refactor that brings us closer to what I have in
#14760 but without using accumulators. To emit a diagnostic, a method
needs:

* Access to the db
* Access to the currently checked file

This PR introduces a new `InferContext` that holds on to the db, the
current file, and the reported diagnostics. It replaces the
`TypeCheckDiagnosticsBuilder`. We pass the `InferContext` instead of the
`db` to methods that *might* emit diagnostics. This simplifies some of
the `Outcome` methods, which can now be called with a context instead of
a `db` and the diagnostics builder. Having the `db` and the file on a
single type like this would also be useful when using accumulators.

This PR doesn't solve the issue that the `Outcome` types feel somewhat
complicated nor that it can be annoying when you need to report a
`Diagnostic,` but you don't have access to an `InferContext` (or the
file). However, I also believe that accumulators won't solve these
problems because:

* Even with accumulators, it's necessary to have a reference to the file
that's being checked. The struggle would be to get a reference to that
file rather than getting a reference to `InferContext`.
* Users of the `HasTy` trait (e.g., a linter) don't want to bother
getting the `File` when calling `Type::return_ty` because they aren't
interested in the created diagnostics. They just want to know what
calling the current expression would return (and if it even is a
callable). This is what the different methods of `Outcome` enable today.
I can ask for the return type without needing extra data that's only
relevant for emitting a diagnostic.

A shortcoming of this approach is that it is now a bit confusing when to
pass `db` and when an `InferContext`. An option is that we'd make the
`file` on `InferContext` optional (it won't collect any diagnostics if
`None`) and change all methods on `Type` to take `InferContext` as the
first argument instead of a `db`. I'm interested in your opinion on
this.

Accumulators are definitely harder to use incorrectly because they
remove the need to merge the diagnostics explicitly and there's no risk
that we accidentally merge the diagnostics twice, resulting in
duplicated diagnostics. I still value performance more over making our
life slightly easier.
2024-12-18 12:22:33 +00:00
InSync ac81c72bf3
[`ruff`] Ambiguous pattern passed to `pytest.raises()` (`RUF043`) (#14966) 2024-12-18 11:53:48 +00:00
David Salvisberg c0b7c36d43
[`ruff`] Avoid false positives for RUF027 for typing context bindings. (#15037)
Closes #14000 

## Summary

For typing context bindings we know that they won't be available at
runtime. We shouldn't recommend a fix, that will result in name errors
at runtime.

## Test Plan

`cargo nextest run`
2024-12-18 08:50:49 +01:00
Douglas Creager e8e461da6a
Prioritize attribute in from/import statement (#15041)
This tweaks the new semantics from #15026 a bit when a symbol could be
interpreted both as an attribute and a submodule of a package. For
`from...import`, we should actually prioritize the attribute, because of
how the statement itself is implemented [1].

> 1. check if the imported module has an attribute by that name
> 2. if not, attempt to import a submodule with that name and then check
the imported module again for that attribute

[1] https://docs.python.org/3/reference/simple_stmts.html#the-import-statement
2024-12-17 16:58:23 -05:00
Douglas Creager 91c9168dd7
Handle nested imports correctly in `from ... import` (#15026)
#14946 fixed our handling of nested imports with the `import` statement,
but didn't touch `from...import` statements.

cf
https://github.com/astral-sh/ruff/issues/14826#issuecomment-2525344515
2024-12-17 14:23:34 -05:00
cake-monotone f463fa7b7c
[red-knot] Narrowing For Truthiness Checks (`if x` or `if not x`) (#14687)
## Summary

Fixes #14550.

Add `AlwaysTruthy` and `AlwaysFalsy` types, representing the set of objects whose `__bool__` method can only ever return `True` or `False`, respectively, and narrow `if x` and `if not x` accordingly.


## Test Plan

- New Markdown test for truthiness narrowing `narrow/truthiness.md`
- unit tests in `types.rs` and `builders.rs` (`cargo test --package
red_knot_python_semantic --lib -- types`)
2024-12-17 08:37:07 -08:00
Micha Reiser c3b6139f39
Upgrade salsa (#15039)
The only code change is that Salsa now requires the `Db` to implement
`Clone` to create "lightweight" snapshots.
2024-12-17 15:50:33 +00:00
InSync c9fdb1f5e3
[`pylint`] Preserve original value format (`PLR6104`) (#14978)
## Summary

Resolves #11672.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-17 16:07:07 +01:00
Alex Waygood 463046ae07
[red-knot] Explicitly test diagnostics are emitted for unresolvable submodule imports (#15035) 2024-12-17 12:55:50 +00:00
Micha Reiser dcb99cc817
Fix stale File status in tests (#15030)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/15027

The `MemoryFileSystem::write_file` API automatically creates
non-existing ancestor directoryes
but we failed to update the status of the now created ancestor
directories in the `Files` data structure.


## Test Plan

Tested that the case in https://github.com/astral-sh/ruff/issues/15027
now passes regardless of whether the *Simple* case is commented out or
not
2024-12-17 12:45:36 +01:00
InSync 7c2e7cf25e
[red-knot] Basic support for other legacy `typing` aliases (#14998)
## Summary

Resolves #14997.

## Test Plan

Markdown tests.
2024-12-17 09:33:15 +00:00
Wei Lee 867a8f9497
feat(AIR302): extend the following rules (#15015)
## Summary


Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names.

* `airflow.api_connexion.security.requires_access_dataset` →
`airflow.api_connexion.security.requires_access_asset`
* `airflow.auth.managers.base_auth_manager.is_authorized_dataset` →
`airflow.auth.managers.base_auth_manager.is_authorized_asset`
* `airflow.auth.managers.models.resource_details.DatasetDetails` →
`airflow.auth.managers.models.resource_details.AssetDetails`
* `airflow.lineage.hook.DatasetLineageInfo` →
`airflow.lineage.hook.AssetLineageInfo`
* `airflow.security.permissions.RESOURCE_DATASET` →
`airflow.security.permissions.RESOURCE_ASSET`
* `airflow.www.auth.has_access_dataset` →
`airflow.www.auth.has_access_dataset.has_access_asset`
* remove `airflow.datasets.DatasetAliasEvent`
* `airflow.datasets.Dataset` → `airflow.sdk.definitions.asset.Asset`
* `airflow.Dataset` → `airflow.sdk.definitions.asset.Asset`
* `airflow.datasets.DatasetAlias` →
`airflow.sdk.definitions.asset.AssetAlias`
* `airflow.datasets.DatasetAll` →
`airflow.sdk.definitions.asset.AssetAll`
* `airflow.datasets.DatasetAny` →
`airflow.sdk.definitions.asset.AssetAny`
* `airflow.datasets.metadata` → `airflow.sdk.definitions.asset.metadata`
* `airflow.datasets.expand_alias_to_datasets` →
`airflow.sdk.definitions.asset.expand_alias_to_assets`
* `airflow.datasets.manager.dataset_manager` → `airflow.assets.manager`
* `airflow.datasets.manager.resolve_dataset_manager` →
`airflow.assets.resolve_asset_manager`
* `airflow.datasets.manager.DatasetManager` →
`airflow.assets.AssetManager`
* `airflow.listeners.spec.dataset.on_dataset_created` →
`airflow.listeners.spec.asset.on_asset_created`
* `airflow.listeners.spec.dataset.on_dataset_changed` →
`airflow.listeners.spec.asset.on_asset_changed`
* `airflow.timetables.simple.DatasetTriggeredTimetable` →
`airflow.timetables.simple.AssetTriggeredTimetable`
* `airflow.timetables.datasets.DatasetOrTimeSchedule` →
`airflow.timetables.assets.AssetOrTimeSchedule`
*
`airflow.providers.amazon.auth_manager.avp.entities.AvpEntities.DATASET`
→ `airflow.providers.amazon.auth_manager.avp.entities.AvpEntities.ASSET`
* `airflow.providers.amazon.aws.datasets.s3.create_dataset` →
`airflow.providers.amazon.aws.assets.s3.create_asset`
*
`airflow.providers.amazon.aws.datasets.s3.convert_dataset_to_openlineage`
→
`airflow.providers.amazon.aws.datasets.s3.convert_dataset_to_openlineage`
* `airflow.providers.amazon.aws.datasets.s3.sanitize_uri` →
`airflow.providers.amazon.aws.assets.s3.sanitize_uri`
*
`airflow.providers.common.io.datasets.file.convert_dataset_to_openlineage`
→ `airflow.providers.common.io.assets.file.convert_asset_to_openlineage`
* `airflow.providers.common.io.datasets.file.sanitize_uri` →
`airflow.providers.common.io.assets.file.sanitize_uri`
* `airflow.providers.common.io.datasets.file.create_dataset` →
`airflow.providers.common.io.assets.file.create_asset`
* `airflow.providers.google.datasets.bigquery.sanitize_uri` →
`airflow.providers.google.assets.bigquery.sanitize_uri`
* `airflow.providers.google.datasets.gcs.create_dataset` →
`airflow.providers.google.assets.gcs.create_asset`
* `airflow.providers.google.datasets.gcs.sanitize_uri` →
`airflow.providers.google.assets.gcs.sanitize_uri`
* `airflow.providers.google.datasets.gcs.convert_dataset_to_openlineage`
→ `airflow.providers.google.assets.gcs.convert_asset_to_openlineage`
*
`airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_dataset`
→
`airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset`
* `airflow.providers.openlineage.utils.utils.DatasetInfo` →
`airflow.providers.openlineage.utils.utils.AssetInfo`
* `airflow.providers.openlineage.utils.utils.translate_airflow_dataset`
→ `airflow.providers.openlineage.utils.utils.translate_airflow_asset`
* `airflow.providers.postgres.datasets.postgres.sanitize_uri` →
`airflow.providers.postgres.assets.postgres.sanitize_uri`
* `airflow.providers.mysql.datasets.mysql.sanitize_uri` →
`airflow.providers.mysql.assets.mysql.sanitize_uri`
* `airflow.providers.trino.datasets.trino.sanitize_uri` →
`airflow.providers.trino.assets.trino.sanitize_uri`

In additional to the newly added rules above, the message for
`airflow.contrib.*` and `airflow.subdag.*` has been extended,
`airflow.sensors.external_task.ExternalTaskSensorLink` error has been
fixed and the test fixture has been reorganized

## Test Plan

A test fixture is included in the PR.
2024-12-17 08:32:48 +01:00
w0nder1ng e22718f25f
[`perflint`] Simplify finding the loop target in `PERF401` (#15025)
Fixes #15012.

```python
def f():
    # panics when the code can't find the loop variable
    values = [1, 2, 3]
    result = []
    for i in values:
        result.append(i + 1)
    del i
```

I'm not sure exactly why this test case panics, but I suspect the `del
i` removes the binding from the semantic model's symbols.

I changed the code to search for the correct binding by directly
iterating through the bindings. Since we know exactly which binding we
want, this should find the loop variable without any complications.
2024-12-17 08:30:32 +01:00
Dhruv Manilawala dcdc6e7c64
[red-knot] Avoid undeclared path when raising conflicting declarations (#14958)
## Summary

This PR updates the logic when raising conflicting declarations
diagnostic to avoid the undeclared path if present.

The conflicting declaration diagnostics is added when there are two or
more declarations in the control flow path of a definition whose type
isn't equivalent to each other. This can be seen in the following
example:

```py
if flag:
	x: int
x = 1  # conflicting-declarations: Unknown, int
```

After this PR, we'd avoid considering "Unknown" as part of the
conflicting declarations. This means we'd still flag it for the
following case:

```py
if flag:
	x: int
else:
	x: str
x = 1  # conflicting-declarations: int, str
```

A solution that's local to the exception control flow was also explored
which required updating the logic for merging the flow snapshot to avoid
considering declarations using a flag. This is preserved here:
https://github.com/astral-sh/ruff/compare/dhruv/control-flow-no-declarations?expand=1.

The main motivation to avoid that is we don't really understand what the
user experience is w.r.t. the Unknown type and the
conflicting-declaration diagnostics. This makes us unsure on what the
right semantics are as to whether that diagnostics should be raised or
not and when to raise them. For now, we've decided to move forward with
this PR and could decide to adopt another solution or remove the
conflicting-declaration diagnostics in the future.

Closes: #13966 

## Test Plan

Update the existing mdtest case. Add an additional case specific to
exception control flow to verify that the diagnostic is not being raised
now.
2024-12-17 09:49:39 +05:30
Douglas Creager 4ddf9228f6
Bind top-most parent when importing nested module (#14946)
When importing a nested module, we were correctly creating a binding for
the top-most parent, but we were binding that to the nested module, not
to that parent module. Moreover, we weren't treating those submodules as
members of their containing parents. This PR addresses both issues, so
that nested imports work as expected.

As discussed in ~Slack~ whatever chat app I find myself in these days
😄, this requires keeping track of which modules have been imported
within the current file, so that when we resolve member access on a
module reference, we can see if that member has been imported as a
submodule. If so, we return the submodule reference immediately, instead
of checking whether the parent module's definition defines the symbol.

This is currently done in a flow insensitive manner. The `SemanticIndex`
now tracks all of the modules that are imported (via `import`, not via
`from...import`). The member access logic mentioned above currently only
considers module imports in the file containing the attribute
expression.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-16 16:15:40 -05:00
Dylan 6a5eff6017
[`pydocstyle`] Skip leading whitespace for `D403` (#14963)
This PR introduces three changes to `D403`, which has to do with
capitalizing the first word in a docstring.

1. The diagnostic and fix now skip leading whitespace when determining
what counts as "the first word".
2. The name has been changed to `first-word-uncapitalized` from
`first-line-capitalized`, for both clarity and compliance with our rule
naming policy.
3. The diagnostic message and documentation has been modified slightly
to reflect this.

Closes #14890
2024-12-16 09:09:27 -06:00
Dhruv Manilawala aa429b413f
Check diagnostic refresh support from client capability (#15014)
## Summary

Per the LSP spec, the property name is `workspace.diagnostics` with an
`s` at the end but the `lsp-types` dependency uses
`workspace.diagnostic` (without an `s`). Our fork contains this fix
(0f58d62879)
so we should avoid the hardcoded value.

The implication of this is that the client which doesn't support
workspace refresh capability didn't support the [dynamic
configuration](https://docs.astral.sh/ruff/editors/features/#dynamic-configuration)
feature because the server would _always_ send the workspace refresh
request but the client would ignore it. We have a fallback logic to
publish the diagnostics instead:


5f6fc3988b/crates/ruff_server/src/server/api/notifications/did_change_watched_files.rs (L28-L40)

fixes: #15013 

## Test Plan

### VS Code


https://github.com/user-attachments/assets/61ac8e6f-aa20-41cc-b398-998e1866b5bc

### Neovim



https://github.com/user-attachments/assets/4131e13c-3fba-411c-9bb7-478d26eb8d56
2024-12-16 16:26:40 +05:30
w0nder1ng 4a7536dc94
[`perflint`] Fix panic in `perf401` (#14971)
Fixes #14969.

The issue was that this line:

```rust
let from_assign_to_loop = TextRange::new(binding_stmt.end(), for_stmt.start());
```

was not safe if the binding was after the target. The only way (at least
that I can think of) this can happen is if they are in different scopes,
so it now checks for that before checking if there are usages between
the two.
2024-12-15 16:22:04 +01:00
Dimitri Papadopoulos Orfanos 2d15d7d1af
Improve the documentation of E201/E202 (#14983)
## Summary

The summary is misleading, as well as the
`whitespace-after-open-bracket` and `whitespace-before-close-bracket`
names - it's not only brackets, but also parentheses and braces. Align
the documentation with the actual behaviour.

Don't change the names, but align the documentation with the behaviour.

## Test Plan

No test (documentation).
2024-12-15 16:20:04 +01:00
Rebecca Chen 112e9d2d82
[ruff_python_ast] Add name and default functions to TypeParam. (#14964)
## Summary

This change adds `name` and `default` functions to `TypeParam` to access
the corresponding attributes more conveniently. I currently have these
as helper functions in code built on top of ruff_python_ast, and they
seemed like they might be generally useful.

## Test Plan

Ran the checks listed in CONTRIBUTING.md#development.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-15 12:04:51 +00:00
Alex Waygood 1389cb8e59
[red-knot] Emit an error if a bare `Annotated` or `Literal` is used in a type expression (#14973) 2024-12-15 02:00:52 +00:00
Alex Waygood fa46ba2306
[red-knot] Fix bugs relating to assignability of dynamic `type[]` types (#14972) 2024-12-15 01:15:10 +00:00
github-actions[bot] 53c7ef8bfe
Sync vendored typeshed stubs (#14977)
Co-authored-by: typeshedbot <>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-15 01:02:41 +00:00
Alex Waygood 4d64cdb83c
[red-knot] `ClassLiteral(<T>)` is not a disjoint type from `Instance(<metaclass of T>)` (#14970)
## Summary

A class is an instance of its metaclass, so `ClassLiteral("ABC")` is not
disjoint from `Instance("ABCMeta")`. However, we erroneously consider
the two types disjoint on the `main` branch. This PR fixes that.

This bug was uncovered by adding some more core types to the property
tests that provide coverage for classes that have custom metaclasses.
The additions to the property tests are included in this PR.

## Test Plan

New unit tests and property tests added. Tested with:
- `cargo test -p red_knot_python_semantic`
- `QUICKCHECK_TESTS=100000 cargo test -p red_knot_python_semantic --
--ignored types::property_tests::stable`

The assignability property test fails on this branch, but that's a known
issue that exists on `main`, due to
https://github.com/astral-sh/ruff/issues/14899.
2024-12-14 11:28:09 -08:00
Carl Meyer ac31b26a0e
[red-knot] type[] is disjoint from None, LiteralString (#14967)
## Summary

Teach red-knot that `type[...]` is always disjoint from `None` and from
`LiteralString`. Fixes #14925.

This should properly be generalized to "all instances of final types
which are not subclasses of `type`", but until we support finality,
hardcoding `None` (which is known to be final) allows us to fix the
subtype transitivity property test.

## Test Plan

Existing tests pass, added new unit tests for `is_disjoint_from` and
`is_subtype_of`.

`QUICKCHECK_TESTS=100000 cargo test -p red_knot_python_semantic --
--ignored types::property_tests::stable` fails only the "assignability
is reflexive" test, which is known to fail on `main` (#14899).

The same command, with `property_tests.rs` edited to prevent generating
intersection tests (the cause of #14899), passes all quickcheck tests.
2024-12-14 11:02:49 +01:00
InSync a80e934838
[red-knot] Error out when an mdtest code block is unterminated (#14965)
## Summary

Resolves #14934.

## Test Plan

Added a unit test.
2024-12-13 21:51:21 -08:00
Alex Waygood 224c8438bd
[red-knot] Minor simplifications to `types.rs` (#14962) 2024-12-13 20:31:51 +00:00
Alex Waygood 90a5439791
[red-knot] Use `type[Unknown]` rather than `Unknown` as the fallback metaclass for invalid classes (#14961) 2024-12-13 19:48:51 +00:00
Alex Waygood 4b2b126b9f
[red-knot] Make `is_subtype_of` exhaustive (#14924) 2024-12-13 19:31:22 +00:00
InSync 9798556eb5
[red-knot] Alphabetize rules (#14960)
## Summary

Follow-up from #14950.

## Test Plan

Purely stylistic change. Shouldn't affect any functionalities.
2024-12-13 10:39:18 -08:00
InSync aa1938f6ba
[red-knot] Understand `Annotated` (#14950)
## Summary

Resolves #14922.

## Test Plan

Markdown tests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-13 09:41:37 -08:00
Dhruv Manilawala 3533d7f5b4
[red-knot] Display definition range in trace logs (#14955)
I've mainly opened this PR to get some opinions. I've found having some
additional information in the tracing logs to be useful to determine
what we are currently inferring. For the `Definition` ingredient, the
range seems to be much useful. I thought of using the identifier name
but we would have to deconstruct the `Expr` to find out the identifier
which seems a lot for just trace logs. Additionally, multiple
identifiers _could_ have the same name where range would be useful.

The ranges are isolated to the names that have been defined by the
definition except for the `except` block where the entire range is being
used because the name is optional.

***Before:***

```
3      ├─   0.074671s  54ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(1402)) } }
3      └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1402), file=/Users/dhruv/playground/ruff/type_inference/isolated3/play.py}
3      ┌─┘
3      ├─   0.074768s  54ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: inner_fn_name_(Id(2800)) } }
3      ├─   0.074807s  54ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_deferred_types(Id(1735)) } }
3      └─┐red_knot_python_semantic::types::infer::infer_deferred_types{definition=Id(1735), file=vendored://stdlib/typing.pyi}
3        ├─   0.074842s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(14f3)) } }
3        └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(14f3), file=vendored://stdlib/typing.pyi}
3          ├─   0.074871s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_expression_types(Id(1820)) } }
3          └─┐red_knot_python_semantic::types::infer::infer_expression_types{expression=Id(1820), file=vendored://stdlib/typing.pyi}
3            ├─   0.074924s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(1429)) } }
3            └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1429), file=vendored://stdlib/typing.pyi}
3              ├─   0.074958s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(3), kind: WillExecute { database_key: infer_definition_types(Id(1428)) } }
3              └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1428), file=vendored://stdlib/typing.pyi}
3              ┌─┘
```

***After:***

```
12      ├─   0.074609s  55ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(1402)) } }
12      └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1402), range=36..37, file=/Users/dhruv/playground/ruff/type_inference/isolated3/play.py}
12      ┌─┘
12      ├─   0.074705s  55ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: inner_fn_name_(Id(2800)) } }
12      ├─   0.074742s  55ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_deferred_types(Id(1735)) } }
12      └─┐red_knot_python_semantic::types::infer::infer_deferred_types{definition=Id(1735), range=30225..30236, file=vendored://stdlib/typing.pyi}
12        ├─   0.074775s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(14f3)) } }
12        └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(14f3), range=9472..9474, file=vendored://stdlib/typing.pyi}
12          ├─   0.074803s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_expression_types(Id(1820)) } }
12          └─┐red_knot_python_semantic::types::infer::infer_expression_types{expression=Id(1820), range=9477..9490, file=vendored://stdlib/typing.pyi}
12            ├─   0.074855s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(1429)) } }
12            └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1429), range=3139..3146, file=vendored://stdlib/typing.pyi}
12              ├─   0.074892s   0ms TRACE red_knot_workspace::db Salsa event: Event { thread_id: ThreadId(12), kind: WillExecute { database_key: infer_definition_types(Id(1428)) } }
12              └─┐red_knot_python_semantic::types::infer::infer_definition_types{definition=Id(1428), range=3102..3107, file=vendored://stdlib/typing.pyi}
12              ┌─┘
```
2024-12-13 14:29:53 +00:00
Alex Waygood 0bbe166720
[red-knot] Move the `ClassBase` enum to its own submodule (#14957) 2024-12-13 13:12:39 +00:00
David Peter c3a64b44b7
[red-knot] mdtest: python version requirements (#14954)
## Summary

This is not strictly required yet, but makes these tests future-proof.
They need a `python-version` requirement as they rely on language
features that are not available in 3.9.
2024-12-13 10:40:38 +01:00
Wei Lee dfd7f38009
[airflow]: Import modules that has been moved to airflow providers (AIR303) (#14764)
## Summary

Many core Airflow features have been deprecated and moved to Airflow
Providers since users might need to install an additional package (e.g.,
`apache-airflow-provider-fab==1.0.0`); a separate rule (AIR303) is
created for this.

As some of the changes only relate to the module/package moved, instead
of listing out all the functions, variables, and classes in a module or
a package, it warns the user to import from the new path instead of the
specific name.

The following is the ones that has been moved to
`apache-airflow-provider-fab==1.0.0`

* module moved
* `airflow.api.auth.backend.basic_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.basic_auth`
* `airflow.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.security_manager.override` →
`airflow.providers.fab.auth_manager.security_manager.override`
* classes (e.g., functions, classes) moved
* `airflow.www.security.FabAirflowSecurityManagerOverride` →
`airflow.providers.fab.auth_manager.security_manager.override.FabAirflowSecurityManagerOverride`
* `airflow.auth.managers.fab.fab_auth_manager.FabAuthManager` →
`airflow.providers.fab.auth_manager.security_manager.FabAuthManager`

## Test Plan


A test fixture has been included for the rule.
2024-12-13 10:38:07 +01:00
David Peter e96b13c027
[red-knot] Support `typing.TYPE_CHECKING` (#14952)
## Summary

Add support for `typing.TYPE_CHECKING` and
`typing_extensions.TYPE_CHECKING`.

relates to: https://github.com/astral-sh/ruff/issues/14170

## Test Plan

New Markdown-based tests
2024-12-13 09:24:48 +00:00
Micha Reiser f52b1f4a4d
Add tracing support to mdtest (#14935)
## Summary

This PR extends the mdtest configuration with a `log` setting that can
be any of:

* `true`: Enables tracing
* `false`: Disables tracing (default)
* String: An ENV_FILTER similar to `RED_KNOT_LOG`

```toml
log = true
```

Closes https://github.com/astral-sh/ruff/issues/13865

## Test Plan

I changed a test and tried `log=true`, `log=false`, and `log=INFO`
2024-12-13 09:10:01 +00:00
David Peter 2ccc9b19a7
[red-knot] Improve `match` mdtests (#14951)
## Summary

Minor improvement for the `match` tests to make sure we can't infer
statically whether or not a certain `case` applies.
2024-12-13 09:50:17 +01:00
Micha Reiser c1837e4189
Rename `custom-typeshed-dir`, `target-version` and `current-directory` CLI options (#14930)
## Summary

This PR renames the `--custom-typeshed-dir`, `target-version`, and
`--current-directory` cli options to `--typeshed`,
`--python-version`, and `--project` as discussed in the CLI proposal
document.
I added aliases for `--target-version` (for Ruff compat) and
`--custom-typeshed-dir` (for Alex)

## Test Plan

Long help

```
An extremely fast Python type checker.

Usage: red_knot [OPTIONS] [COMMAND]

Commands:
  server  Start the language server
  help    Print this message or the help of the given subcommand(s)

Options:
      --project <PROJECT>
          Run the command within the given project directory.
          
          All `pyproject.toml` files will be discovered by walking up the directory tree from the project root, as will the project's virtual environment (`.venv`).
          
          Other command-line arguments (such as relative paths) will be resolved relative to the current working directory."#,

      --venv-path <PATH>
          Path to the virtual environment the project uses.
          
          If provided, red-knot will use the `site-packages` directory of this virtual environment to resolve type information for the project's third-party dependencies.

      --typeshed-path <PATH>
          Custom directory to use for stdlib typeshed stubs

      --extra-search-path <PATH>
          Additional path to use as a module-resolution source (can be passed multiple times)

      --python-version <VERSION>
          Python version to assume when resolving types
          
          [possible values: 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13]

  -v, --verbose...
          Use verbose output (or `-vv` and `-vvv` for more verbose output)

  -W, --watch
          Run in watch mode by re-running whenever files change

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version
```

Short help 

```
An extremely fast Python type checker.

Usage: red_knot [OPTIONS] [COMMAND]

Commands:
  server  Start the language server
  help    Print this message or the help of the given subcommand(s)

Options:
      --project <PROJECT>         Run the command within the given project directory
      --venv-path <PATH>          Path to the virtual environment the project uses
      --typeshed-path <PATH>      Custom directory to use for stdlib typeshed stubs
      --extra-search-path <PATH>  Additional path to use as a module-resolution source (can be passed multiple times)
      --python-version <VERSION>  Python version to assume when resolving types [possible values: 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13]
  -v, --verbose...                Use verbose output (or `-vv` and `-vvv` for more verbose output)
  -W, --watch                     Run in watch mode by re-running whenever files change
  -h, --help                      Print help (see more with '--help')
  -V, --version                   Print version

```

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-13 08:21:52 +00:00
David Peter d7ce548893
[red-knot] Add narrowing for 'while' loops (#14947)
## Summary

Add type narrowing for `while` loops and corresponding `else` branches.

closes #14861 

## Test Plan

New Markdown tests.
2024-12-13 07:40:14 +01:00
Krishnan Chandra be4ce16735
[`ruff`] Skip SQLModel base classes for `mutable-class-default` (`RUF012`) (#14949)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14892, by adding
`sqlmodel.SQLModel` to the list of classes with default copy semantics.

## Test Plan

Added a test into `RUF012.py` containing the example from the original
issue.
2024-12-12 22:19:21 -06:00
David Peter 657d26ff20
[red-knot] Tests for 'while' loop boundness (#14944)
## Summary

Regression test(s) for something that broken while implementing #14759.
We have similar tests for other control flow elements, but feel free to
let me know if this seems superfluous.

## Test Plan

New mdtests
2024-12-12 21:06:56 +01:00
Alex Waygood dbc191d2d6
[red-knot] Fixes to `Type::to_meta_type` (#14942) 2024-12-12 19:55:11 +00:00
David Peter d2712c7669
ruff_python_ast: Make `Singleton` `Copy` (#14943)
## Summary

Minor changed pulled out from #14759, as it seems to make sense in
isolation.

## Test Plan

—
2024-12-12 20:49:54 +01:00
Chandra Kiran G e5cb4d6388
[`flake8-pyi`]: More autofixes for redundant-none-literal (`PYI061`) (#14872)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-12 19:44:32 +00:00
Sergey Mezentsev 68e8496260
[`flake8-use-pathlib`] Extend check for invalid path suffix to include the case `"."` (`PTH210`) (#14902)
## Summary

`PTH210` renamed to `invalid-pathlib-with-suffix` and extended to check for `.with_suffix(".")`. This caused the fix availability to be downgraded to "Sometimes", since there is no fix offered in this case.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Dylan <53534755+dylwil3@users.noreply.github.com>
2024-12-12 13:30:17 -06:00
Alex Waygood 71239f248e
[red-knot] Add explicit TODO branches for many typing special forms and qualifiers (#14936) 2024-12-12 17:57:26 +00:00
Alex Waygood 58930905eb
[red-knot] Fixup a few edge cases regarding `type[]` (#14918) 2024-12-12 16:53:03 +00:00
Dylan 37f433814c
Bump version to 0.8.3 (#14937)
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-12-12 14:13:06 +00:00
Alex Waygood 45b565cbb5
[red-knot] `Any` cannot be parameterized (#14933) 2024-12-12 11:50:34 +00:00
Micha Reiser 82faa9bb62
Add tests demonstrating f-strings with debug expressions in replacements that contain escaped characters (#14929) 2024-12-12 09:33:20 +00:00
w0nder1ng 2eac00c60f
[`perflint`] fix invalid hoist in `perf401` (#14369)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-12 09:11:09 +01:00
InSync e4885a2fb2
[red-knot] Understand `typing.Tuple` (#14927)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-12 00:58:06 +00:00
David Peter a7e5e42b88
[red-knot] Make `attributes.md` test future-proof (#14923)
## Summary

Using `typing.LiteralString` breaks as soon as we understand
`sys.version_info` branches, as it's only available in 3.11 and later.

## Test Plan

Made sure it didn't fail on my #14759 branch anymore.
2024-12-11 20:46:24 +01:00
Alex Waygood c361cf66ad
[red-knot] Precise inference for `__class__` attributes on objects of all types (#14921) 2024-12-11 17:30:34 +00:00
Alex Waygood a54353392f
[red-knot] Add failing test for use of `type[]` as a base class (#14913)
We support using `typing.Type[]` as a base class (and we have tests for
it), but not yet `builtins.type[]`. At some point we should fix that,
but I don't think it';s worth spending much time on now (and it might be
easier once we've implemented generics?). This PR just adds a failing
test with a TODO.
2024-12-11 17:08:00 +00:00
Alex Waygood ef153a0cce
[red-knot] Remove an unnecessary branch and a confusing TODO comment (#14915) 2024-12-11 16:57:40 +00:00
InSync 0c85023cd9
Fix a typo in `if_key_in_dict_del.rs` (#14920)
(Accidentally introduced in #14553).
2024-12-11 16:28:27 +00:00
Alex Waygood 7135a49aea
[red-knot] Record the TODO message in `ClassBase::Todo`, same as in `Type::Todo` (#14919) 2024-12-11 15:17:56 +00:00
Micha Reiser 6e11086c98
Support `lint:<rule>` in mdtests (#14914)
## Summary

Fixes a small scoping issue in `DiagnosticId::matches`

Note: I don't think we should use `lint:id` in mdtests just yet. I worry
that it could lead to many unnecessary churns if we decide **not** to
use `lint:<id>` as the format (e.g., `lint/id`).

The reason why users even see `lint:<rule>` is because the mdtest
framework uses the diagnostic infrastructure

Closes #14910

## Test Plan

Added tests
2024-12-11 14:37:12 +01:00
Micha Reiser 28653c7c47
Strip `lint:` prefix from mdtest diagnostics 2024-12-11 14:33:31 +01:00
Alex Waygood 1d91dae11f
[red-knot] Minor simplifications to `mro.rs` (#14912) 2024-12-11 13:14:12 +00:00
Micha Reiser 881375a8d9
[red-knot] Lint registry and rule selection (#14874)
## Summary

This is the third and last PR in this stack that adds support for
toggling lints at a per-rule level.

This PR introduces a new `LintRegistry`, a central index of known lints.
The registry is required because we want to support lint rules from many
different crates but need a way to look them up by name, e.g., when
resolving a lint from a name in the configuration or analyzing a
suppression comment.

Adding a lint now requires two steps:

1. Declare the lint with `declare_lint`
2. Register the lint in the registry inside the `register_lints`
function.

I considered some more involved macros to avoid changes in two places.
Still, I ultimately decided against it because a) it's just two places
and b) I'd expect that registering a type checker lint will differ from
registering a lint that runs as a rule in the linter. I worry that any
more opinionated design could limit our options when working on the
linter, so I kept it simple.

The second part of this PR is the `RuleSelection`. It stores which lints
are enabled and what severity they should use for created diagnostics.
For now, the `RuleSelection` always gets initialized with all known
lints and it uses their default level.

## Linter crates

Each crate that defines lints should export a `register_lints` function
that accepts a `&mut LintRegistryBuilder` to register all its known
lints in the registry. This should make registering all known lints in a
top-level crate easy: Just call `register_lints` of every crate that
defines lint rules.

I considered defining a `LintCollection` trait and even some fancy
macros to accomplish the same but decided to go for this very simplistic
approach for now. We can add more abstraction once needed.

## Lint rules

This is a bit hand-wavy. I don't have a good sense for how our linter
infrastructure will look like, but I expect we'll need a way to register
the rules that should run as part of the red knot linter. One way is to
keep doing what Ruff does by having one massive `checker` and each lint
rule adds a call to itself in the relevant AST visitor methods. An
alternative is that we have a `LintRule` trait that provides common
hooks and implementations will be called at the "right time". Such a
design would need a way to register all known lint implementations,
possibly with the lint. This is where we'd probably want a dedicated
`register_rule` method. A third option is that lint rules are handled
separately from the `LintRegistry` and are specific to the linter crate.

The current design should be flexible enough to support the three
options.


## Documentation generation

The documentation for all known lints can be generated by creating a
factory, registering all lints by calling the `register_lints` methods,
and then querying the registry for the metadata.

## Deserialization and Schema generation

I haven't fully decided what the best approach is when it comes to
deserializing lint rule names:

* Reject invalid names in the deserializer. This gives us error messages
with line and column numbers (by serde)
* Don't validate lint rule names during deserialization; defer the
validation until the configuration is resolved. This gives us more
control over handling the error, e.g. emit a warning diagnostic instead
of aborting when a rule isn't known.

One technical challenge for both deserialization and schema generation
is that the `Deserialize` and `JSONSchema` traits do not allow passing
the `LintRegistry`, which is required to look up the lints by name. I
suggest that we either rely on the salsa db being set for the current
thread (`salsa::Attach`) or build our own thread-local storage for the
`LintRegistry`. It's the caller's responsibility to make the lint
registry available before calling `Deserialize` or `JSONSchema`.


## CLI support

I prefer deferring adding support for enabling and disabling lints from
the CLI for now because I think it will be easier
to add once I've figured out how to handle configurations. 

## Bitset optimization

Ruff tracks the enabled rules using a cheap copyable `Bitset` instead of
a hash map. This helped improve performance by a few percent (see
https://github.com/astral-sh/ruff/pull/3606). However, this approach is
no longer possible because lints have no "cheap" way to compute their
index inside the registry (other than using a hash map).

We could consider doing something similar to Salsa where each
`LintMetadata` stores a `LazyLintIndex`.

```
pub struct LazyLintIndex {
	cached: OnceLock<(Nonce, LintIndex)>
}

impl LazyLintIndex {
	pub fn get(registry: &LintRegistry, lint: &'static LintMetadata) {
	
	let (nonce, index) = self.cached.get_or_init(|| registry.lint_index(lint));

	if registry.nonce() == nonce {
		index
	} else {
		registry.lint_index(lint)
	}
}
```

Each registry keeps a map from `LintId` to `LintIndex` where `LintIndex`
is in the range of `0...registry.len()`. The `LazyLintIndex` is based on
the assumption that every program has exactly **one** registry. This
assumption allows to cache the `LintIndex` directly on the
`LintMetadata`. The implementation falls back to the "slow" path if
there is more than one registry at runtime.

I was very close to implementing this optimization because it's kind of
fun to implement. I ultimately decided against it because it adds
complexity and I don't think it's worth doing in Red Knot today:

* Red Knot only queries the rule selection when deciding whether or not
to emit a diagnostic. It is rarely used to detect if a certain code
block should run. This is different from Ruff where the rule selection
is queried many times for every single AST node to determine which rules
*should* run.
* I'm not sure if a 2-3% performance improvement is worth the complexity

I suggest revisiting this decision when working on the linter where a
fast path for deciding if a rule is enabled might be more important (but
that depends on how lint rules are implemented)


## Test Plan

I removed a lint from the default rule registry, and the MD tests
started failing because the diagnostics were no longer emitted.
2024-12-11 13:25:19 +01:00
InSync 6f8d8fa36b
[`ruff`] `if k in d: del d[k]` (`RUF051`) (#14553)
## Summary

Resolves #7537.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-11 11:12:23 +00:00
InSync f30227c436
[red-knot] Understand `typing.Type` (#14904)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-11 11:01:38 +00:00
InSync c8d505c8ea
[`pyupgrade`] Do not report when a UTF-8 comment is followed by a non-UTF-8 one (`UP009`) (#14728)
## Summary

Resolves #14704.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-11 10:30:41 +00:00
Dimitri Papadopoulos Orfanos a55722e740
Revert disjointness->disjointedness (#14906)
## Summary

Partially revert #14880. While `disjointness` is missing from the
[OED](https://www.oed.com/search/dictionary/?q=disjointness) and [SCOWL
(And
Friends)](http://app.aspell.net/lookup?dict=en_US-large;words=disjointness),
it is commonly used in mathematics to describe disjoint sets.

## Test Plan

CI tests.
2024-12-11 08:26:45 +00:00
Dylan a3bb0cd5ec
Raise syntax error for mixing `except` and `except*` (#14895)
This PR adds a syntax error if the parser encounters a `TryStmt` that
has except clauses both with and without a star.

The displayed error points to each except clause that contradicts the
original except clause kind. So, for example,

```python
try:
    ....
except:     #<-- we assume this is the desired except kind
    ....
except*:    #<---  error will point here
    ....
except*:    #<--- and here
    ....
```

Closes #14860
2024-12-10 17:50:55 -06:00
Douglas Creager d4126f6049
Handle type[Any] correctly (#14876)
This adds support for `type[Any]`, which represents an unknown type (not
an instance of an unknown type), and `type`, which we are choosing to
interpret as `type[object]`.

Closes #14546
2024-12-10 16:12:37 -05:00
Carl Meyer 03fb2e5ac1
[red-knot] split call-outcome enums to their own submodule (#14898)
## Summary

This is already several hundred lines of code, and it will get more
complex with call-signature checking.

## Test Plan

This is a pure code move; the moved code wasn't changed, just imports.
Existing tests pass.
2024-12-10 12:03:29 -08:00
David Peter 1a3c311ac5
[red-knot] Property tests: account non-fully-static types (#14897)
## Summary

Add a `is_fully_static` premise to the equivalence on subtyping property tests.

## Test Plan

```
cargo test -p red_knot_python_semantic -- --ignored types::property_tests::stable
```
2024-12-10 19:55:45 +00:00
Andrew Gallant 2ecd164adb
ruff: add worktree support to build.rs (#14896)
Without this, `cargo insta test` re-compiles every time it is run, even
if there are no changes. With this, I can re-run `cargo insta test` (or
other `cargo build` commands) without it resulting in re-compiles.

I made an identical change to uv a while back:
https://github.com/astral-sh/uv/pull/6825
2024-12-10 14:06:59 -05:00
Micha Reiser 5fc8e5d80e
[red-knot] Add infrastructure to declare lints (#14873)
## Summary

This is the second PR out of three that adds support for
enabling/disabling lint rules in Red Knot. You may want to take a look
at the [first PR](https://github.com/astral-sh/ruff/pull/14869) in this
stack to familiarize yourself with the used terminology.

This PR adds a new syntax to define a lint: 

```rust
declare_lint! {
    /// ## What it does
    /// Checks for references to names that are not defined.
    ///
    /// ## Why is this bad?
    /// Using an undefined variable will raise a `NameError` at runtime.
    ///
    /// ## Example
    ///
    /// ```python
    /// print(x)  # NameError: name 'x' is not defined
    /// ```
    pub(crate) static UNRESOLVED_REFERENCE = {
        summary: "detects references to names that are not defined",
        status: LintStatus::preview("1.0.0"),
        default_level: Level::Warn,
    }
}
```

A lint has a name and metadata about its status (preview, stable,
removed, deprecated), the default diagnostic level (unless the
configuration changes), and documentation. I use a macro here to derive
the kebab-case name and extract the documentation automatically.

This PR doesn't yet add any mechanism to discover all known lints. This
will be added in the next and last PR in this stack.


## Documentation
I documented some rules but then decided that it's probably not my best
use of time if I document all of them now (it also means that I play
catch-up with all of you forever). That's why I left some rules
undocumented (marked with TODO)

## Where is the best place to define all lints?

I'm not sure. I think what I have in this PR is fine but I also don't
love it because most lints are in a single place but not all of them. If
you have ideas, let me know.


## Why is the message not part of the lint, unlike Ruff's `Violation`

I understand that the main motivation for defining `message` on
`Violation` in Ruff is to remove the need to repeat the same message
over and over again. I'm not sure if this is an actual problem. Most
rules only emit a diagnostic in a single place and they commonly use
different messages if they emit diagnostics in different code paths,
requiring extra fields on the `Violation` struct.

That's why I'm not convinced that there's an actual need for it and
there are alternatives that can reduce the repetition when creating a
diagnostic:

* Create a helper function. We already do this in red knot with the
`add_xy` methods
* Create a custom `Diagnostic` implementation that tailors the entire
diagnostic and pre-codes e.g. the message

Avoiding an extra field on the `Violation` also removes the need to
allocate intermediate strings as it is commonly the place in Ruff.
Instead, Red Knot can use a borrowed string with `format_args`

## Test Plan

`cargo test`
2024-12-10 16:14:44 +00:00
Micha Reiser 5f548072d9
[red-knot] Typed diagnostic id (#14869)
## Summary

This PR introduces a structured `DiagnosticId` instead of using a plain
`&'static str`. It is the first of three in a stack that implements a
basic rules infrastructure for Red Knot.

`DiagnosticId` is an enum over all known diagnostic codes. A closed enum
reduces the risk of accidentally introducing two identical diagnostic
codes. It also opens the possibility of generating reference
documentation from the enum in the future (not part of this PR).

The enum isn't *fully closed* because it uses a `&'static str` for lint
names. This is because we want the flexibility to define lints in
different crates, and all names are only known in `red_knot_linter` or
above. Still, lower-level crates must already reference the lint names
to emit diagnostics. We could define all lint-names in `DiagnosticId`
but I decided against it because:

* We probably want to share the `DiagnosticId` type between Ruff and Red
Knot to avoid extra complexity in the diagnostic crate, and both tools
use different lint names.
* Lints require a lot of extra metadata beyond just the name. That's why
I think defining them close to their implementation is important.

In the long term, we may also want to support plugins, which would make
it impossible to know all lint names at compile time. The next PR in the
stack introduces extra syntax for defining lints.

A closed enum does have a few disadvantages:

* rustc can't help us detect unused diagnostic codes because the enum is
public
* Adding a new diagnostic in the workspace crate now requires changes to
at least two crates: It requires changing the workspace crate to add the
diagnostic and the `ruff_db` crate to define the diagnostic ID. I
consider this an acceptable trade. We may want to move `DiagnosticId` to
its own crate or into a shared `red_knot_diagnostic` crate.


## Preventing duplicate diagnostic identifiers

One goal of this PR is to make it harder to introduce ambiguous
diagnostic IDs, which is achieved by defining a closed enum. However,
the enum isn't fully "closed" because it doesn't explicitly list the IDs
for all lint rules. That leaves the possibility that a lint rule and a
diagnostic ID share the same name.

I made the names unambiguous in this PR by separating them into
different namespaces by using `lint/<rule>` for lint rule codes. I don't
mind the `lint` prefix in a *Ruff next* context, but it is a bit weird
for a standalone type checker. I'd like to not overfocus on this for now
because I see a few different options:

* We remove the `lint` prefix and add a unit test in a top-level crate
that iterates over all known lint rules and diagnostic IDs to ensure the
names are non-overlapping.
* We only render `[lint]` as the error code and add a note to the
diagnostic mentioning the lint rule. This is similar to clippy and has
the advantage that the header line remains short
(`lint/some-long-rule-name` is very long ;))
* Any other form of adjusting the diagnostic rendering to make the
distinction clear

I think we can defer this decision for now because the `DiagnosticId`
contains all the relevant information to change the rendering
accordingly.


## Why `Lint` and not `LintRule`

I see three kinds of diagnostics in Red Knot:

* Non-suppressable: Reveal type, IO errors, configuration errors, etc.
(any `DiagnosticId`)
* Lints: code-related diagnostics that are suppressable. 
* Lint rules: The same as lints, but they can be enabled or disabled in
the configuration. The majority of lints in Red Knot and the Ruff
linter.

Our current implementation doesn't distinguish between lints and Lint
rules because we aren't aware of a suppressible code-related lint that
can't be configured in the configuration. The only lint that comes to my
mind is maybe `division-by-zero` if we're 99.99% sure that it is always
right. However, I want to keep the door open to making this distinction
in the future if it proves useful.

Another reason why I chose lint over lint rule (or just rule) is that I
want to leave room for a future lint rule and lint phase concept:

* lint is the *what*: a specific code smell, pattern, or violation 
* the lint rule is the *how*: I could see a future `LintRule` trait in
`red_knot_python_linter` that provides the necessary hooks to run as
part of the linter. A lint rule produces diagnostics for exactly one
lint. A lint rule differs from all lints in `red_knot_python_semantic`
because they don't run as "rules" in the Ruff sense. Instead, they're a
side-product of type inference.
* the lint phase is a different form of *how*: A lint phase can produce
many different lints in a single pass. This is a somewhat common pattern
in Ruff where running one analysis collects the necessary information
for finding many different lints
* diagnostic is the *presentation*: Unlike a lint, the diagnostic isn't
the what, but how a specific lint gets presented. I expect that many
lints can use one generic `LintDiagnostic`, but a few lints might need
more flexibility and implement their custom diagnostic rendering (at
least custom `Diagnostic` implementation).


## Test Plan

`cargo test`
2024-12-10 15:58:07 +00:00
Karthikeyan Singaravelan dc0d944608
[`airflow`] Add fix to remove deprecated keyword arguments (`AIR302`) (#14887)
## Summary

Add replacement fixes to deprecated arguments of a DAG.

Ref #14582 #14626

## Test Plan

Diff was verified and snapshots were updated.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-12-10 18:49:28 +05:30
InSync 15fe540251
Improve mdtests style (#14884)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-10 13:05:51 +00:00
Micha Reiser 7a0e9b34d0
Reference `suppress-dummy-regex-options` in documentation of rules supporting it (#14888)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/14663
2024-12-10 09:53:53 +00:00
InSync 4b8c815b27
[`flake8-bugbear`] `itertools.batched()` without explicit `strict` (`B911`) (#14408)
## Summary

Resolves #14387.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-10 08:39:46 +00:00
Alex Waygood e3f34b8f5b
[`ruff`] Mark autofix for `RUF052` as always unsafe (#14824) 2024-12-09 23:11:44 +00:00
Alex Waygood ab26d9cf9a
[red-knot] Improve type inference for except handlers (#14838) 2024-12-09 22:49:58 +00:00
Dimitri Papadopoulos Orfanos 64944f2cf5
More typos found by codespell (#14880) 2024-12-09 22:47:34 +00:00
Carl Meyer 533e8a6ee6
[red-knot] move standalone expression_ty to TypeInferenceBuilder::file_expression_ty (#14879)
## Summary

Per suggestion in
https://github.com/astral-sh/ruff/pull/14802#discussion_r1875455417

This is a bit less error-prone and allows us to handle both expressions
in the current scope or a different scope. Also, there's currently no
need for this method outside of `TypeInferenceBuilder`, so no reason to
expose it in `types.rs`.

## Test Plan

Pure refactor, no functional change; existing tests pass.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-09 17:02:14 +00:00
InSync c62ba48ad4
[`ruff`] Do not simplify `round()` calls (`RUF046`) (#14832)
## Summary

Part 1 of the big change introduced in #14828. This temporarily causes
all fixes for `round(...)` to be considered unsafe, but they will
eventually be enhanced.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-12-09 16:51:27 +01:00
Dimitri Papadopoulos Orfanos 68eb0a2511
Stop referring to early ruff versions (#14862)
## Summary

Referring to old versions has become more distracting than useful.

## Test Plan

—
2024-12-09 16:47:26 +01:00
InSync 0f4350e10e
Fix a typo in `class.rs` (#14877)
(Accidentally introduced in #14801.)
2024-12-09 15:36:42 +00:00
InSync aa6b812a73
[`flake8-pyi`] Also remove `self` and `cls`'s annotation (`PYI034`) (#14801)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-09 14:59:12 +00:00
Harutaka Kawamura 0e9427255f
[`pyupgrade`] Remove unreachable code in `UP015` implementation (#14871) 2024-12-09 14:54:57 +00:00
Harutaka Kawamura 9c3c59aca9
[`flake8-bugbear`] Skip `B028` if `warnings.warn` is called with `*args` or `**kwargs` (#14870) 2024-12-09 14:32:37 +00:00
Harutaka Kawamura 172143ae77
[`flake8-bugbear`] Fix `B028` to allow `stacklevel` to be explicitly assigned as a positional argument (#14868) 2024-12-09 13:15:43 +00:00
InSync 3865fb6641
[red-knot] Understanding `type[Union[A, B]]` (#14858)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-09 12:47:14 +00:00
Dimitri Papadopoulos Orfanos 59145098d6
Fix typos found by codespell (#14863)
## Summary

Just fix typos.

## Test Plan

CI tests.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-09 09:32:12 +00:00
Harutaka Kawamura 3d9ac535e9
Fix `pytest-parametrize-names-wrong-type (PT006)` to edit both `argnames` and `argvalues` if both of them are single-element tuples/lists (#14699)
## Summary

Close #11243. Fix `pytest-parametrize-names-wrong-type (PT006)` to edit
both `argnames` and `argvalues` if both of them are single-element
tuples/lists.

```python
# Before fix
@pytest.mark.parametrize(("x",), [(1,), (2,)])
def test_foo(x):
    ...

# After fix:
@pytest.mark.parametrize("x", [1, 2])
def test_foo(x):
    ...
```

## Test Plan

New test cases
2024-12-09 09:58:52 +01:00
Dylan 9d641fa714
[`pylint`] Include parentheses and multiple comparators in check for `boolean-chained-comparison (PLR1716)` (#14781)
This PR introduces three changes to the diagnostic and fix behavior
(still under preview) for [boolean-chained-comparison
(PLR1716)](https://docs.astral.sh/ruff/rules/boolean-chained-comparison/#boolean-chained-comparison-plr1716).

1. We now offer a _fix_ in the case of parenthesized expressions like
`(a < b) and b < c`. The fix will merge the chains of comparisons and
then balance parentheses by _adding_ parentheses to one side of the
expression.
2. We now trigger a diagnostic (and fix) in the case where some
comparisons have multiple comparators like `a < b < c and c < d`.
3. When adjacent comparators are parenthesized, we prefer the left
parenthesization and apply the replacement to the whole parenthesized
range. So, for example, `a < (b) and ((b)) < c` becomes `a < (b) < c`.

While these seem like somewhat disconnected changes, they are actually
related. If we only offered (1), then we would see the following fix
behavior:

```diff
- (a < b) and b < c and ((c < d))
+ (a < b < c) and ((c < d))
```

This is because the fix which add parentheses to the first pair of
comparisons overlaps with the fix that removes the `and` between the
second two comparisons. So the latter fix is deferred. However, the
latter fix does not get a second chance because, upon the next lint
iteration, there is no violation of `PLR1716`.

Upon adopting (2), however, both fixes occur by the time ruff completes
several iterations and we get:

```diff
- (a < b) and b < c and ((c < d))
+ ((a < b < c < d))
```

Finally, (3) fixes a previously unobserved bug wherein the autofix for
`a < (b) and b < c` used to result in `a<(b<c` which gives a syntax
error. It could in theory have been fixed in a separate PR, but seems to
be on theme here.


----------

- Closes #13524
- (1), (2), and (3) are implemented in separate commits for ease of
review and modification.
- Technically a user can trigger an error in ruff (by reaching max
iterations) if they have a humongous boolean chained comparison with
differing parentheses levels.
2024-12-08 22:58:45 -06:00
Dmitry Shachnev d53e5cd25a
[flake8-commas]: Fix example replacement in docs (#14843)
## Summary

Minor change for the documentation of COM818 rule. This was a block
called “In the event that a tuple is intended”, but the suggested change
did not produce a tuple.

## Test Plan

```python
>>> import json
>>> (json.dumps({"bar": 1}),)  # this is a tuple
('{"bar": 1}',)
>>> (json.dumps({"bar": 1}))  # not a tuple
'{"bar": 1}'
```
2024-12-09 00:51:59 +00:00
Alex Waygood 9b8ceb9a2e
[`pyupgrade`] Mark fixes for `convert-typed-dict-functional-to-class` and `convert-named-tuple-functional-to-class` as unsafe if they will remove comments (`UP013`, `UP014`) (#14842) 2024-12-08 18:51:37 +00:00
Thibaut Decombe 8d9e408dbb
Fix `PLW1508` false positive for default string created via a mult operation (#14841) 2024-12-08 18:25:47 +00:00
ABDULRAHMAN ALRAHMA 85402097fc
Improve error messages for except* (B025, B029, B030, B904) #14791 (#14815)
Improves error message for [except*](https://peps.python.org/pep-0654/)
(Rules: B025, B029, B030, B904)

Example python snippet:
```python
try:
    a = 1
except* ValueError:
    a = 2
except* ValueError:
    a = 2

try:
    pass
except* ():
    pass

try:
    pass
except* 1:  # error
    pass

try:
    raise ValueError
except* ValueError:
    raise UserWarning
```
Error messages
Before:
```
$ ruff check --select=B foo.py
foo.py:6:9: B025 try-except block with duplicate exception `ValueError`
foo.py:11:1: B029 Using `except ():` with an empty tuple does not catch anything; add exceptions to handle
foo.py:16:9: B030 `except` handlers should only be exception classes or tuples of exception classes
foo.py:22:5: B904 Within an `except` clause, raise exceptions with `raise ... from err` or `raise ... from None` to distinguish them from errors in exception handling
Found 4 errors.
```
After:
```
$ ruff check --select=B foo.py
foo.py:6:9: B025 try-except* block with duplicate exception `ValueError`
foo.py:11:1: B029 Using `except* ():` with an empty tuple does not catch anything; add exceptions to handle
foo.py:16:9: B030 `except*` handlers should only be exception classes or tuples of exception classes
foo.py:22:5: B904 Within an `except*` clause, raise exceptions with `raise ... from err` or `raise ... from None` to distinguish them from errors in exception handling
Found 4 errors.
```

Closes https://github.com/astral-sh/ruff/issues/14791

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-08 17:37:34 +00:00
Shaygan Hooshyari 269e47be96
Understand `type[A | B]` special form in annotations (#14830)
resolves https://github.com/astral-sh/ruff/issues/14703

I decided to use recursion to get the type, so if anything is added to
the single element inference it will be applied for the union.
Also added this
[change](https://github.com/astral-sh/ruff/issues/14703#issuecomment-2510286217)
in this PR since it was easy.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-07 17:34:50 +00:00
Dylan d34013425f
[`flake8-bugbear`] Offer unsafe autofix for `no-explicit-stacklevel` (`B028`) (#14829)
This PR introduces an unsafe autofix for [no-explicit-stacklevel
(B028)](https://docs.astral.sh/ruff/rules/no-explicit-stacklevel/#no-explicit-stacklevel-b028):
we add the `stacklevel` argument, set to `2`.

Closes #14805
2024-12-07 08:24:37 -05:00
Dylan 2c13e6513d
[`flake8-comprehensions`] Skip iterables with named expressions in `unnecessary-map` (`C417`) (#14827)
This PR modifies [unnecessary-map
(C417)](https://docs.astral.sh/ruff/rules/unnecessary-map/#unnecessary-map-c417)
to skip `map` expressions if the iterable contains a named expression,
since those cannot appear in comprehensions.

Closes #14808
2024-12-06 22:00:33 -05:00
Douglas Creager 8fdd88013d
Support `type[a.X]` with qualified class names (#14825)
This adds support for `type[a.X]`, where the `type` special form is
applied to a qualified name that resolves to a class literal. This works
for both nested classes and classes imported from another module.

Closes #14545
2024-12-06 17:14:51 -05:00
Carl Meyer 3017b3b687
[red-knot] function parameter types (#14802)
## Summary

Inferred and declared types for function parameters, in the function
body scope.

Fixes #13693.

## Test Plan

Added mdtests.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-06 12:55:56 -08:00
Alex Waygood 2119dcab6f
[`ruff`] Teach autofix for `used-dummy-variable` about TypeVars etc. (`RUF052`) (#14819) 2024-12-06 17:05:50 +00:00
Wei Lee 3ea14d7a74
[airflow]: extend removed args (AIR302) (#14765)
## Summary

Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names.

* in `DAG`
    * `sla_miss_callback` was removed
* in `airflow.operators.trigger_dagrun.TriggerDagRunOperator`
    * `execution_date` was removed
* in `airflow.operators.weekday.DayOfWeekSensor`,
`airflow.operators.datetime.BranchDateTimeOperator` and
`airflow.operators.weekday.BranchDayOfWeekOperator`
* `use_task_execution_day` was removed in favor of
`use_task_logical_date`

The full list of rules we will extend
https://github.com/apache/airflow/issues/44556

## Test Plan

<!-- How was it tested? -->
A test fixture is included in the PR.
2024-12-06 17:00:23 +01:00
Alex Waygood 4fdd4ddfaa
[`ruff`] Don't emit `used-dummy-variable` on function parameters (`RUF052`) (#14818) 2024-12-06 14:54:42 +00:00
David Peter 6b9f3d7d7c
[red-knot] Import `LiteralString`/`Never` from `typing_extensions` (#14817)
## Summary

`typing.Never` and `typing.LiteralString` are only conditionally
exported from `typing` for Python versions 3.11 and later. We run the
Markdown tests with the default Python version of 3.9, so here we change
the import to `typing_extensions` instead, and add a new test to make
sure we'll continue to understand the `typing`-version of these symbols
for newer versions.

This didn't cause problems so far, as we don't understand
`sys.version_info` branches yet.

## Test Plan

New Markdown tests to make sure this will continue to work in the
future.
2024-12-06 13:57:51 +01:00
Alex Waygood 4cb8392523
Further simplifications to PTH210 (#14816) 2024-12-06 12:52:26 +00:00
Alex Waygood 9ee438b02f
Minor nitpicks for PTH210 (#14814) 2024-12-06 12:23:21 +00:00
InSync 89368a62a8
[`flake8-use-pathlib`] Dotless suffix passed to `Path.with_suffix()` (`PTH901`) (#14779)
## Summary

Resolves #14441.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-12-06 13:08:20 +01:00
Micha Reiser 1559c73fcd
Fix fstring formatting removing overlong implicit concatenated string in expression part (#14811)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/14778


The formatter incorrectly removed the inner implicitly concatenated
string for following single-line f-string:

```py
f"{'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' 'a' if True else ""}"

# formatted
f"{ if True else ''}"
```

This happened because I changed the `RemoveSoftlinesBuffer` in
https://github.com/astral-sh/ruff/pull/14489 to remove any content
wrapped in `if_group_breaks`. After all, it emulates an *all flat*
layout. This works fine when `if_group_breaks` is only used to **add**
content if the gorup breaks. It doesn't work if the same content is
rendered differently depending on if the group fits using
`if_group_breaks` and `if_groups_fits` because the enclosing `group`
might still *break* if the entire content exceeds the line-length limit.

This PR fixes this by unwrapping any `if_group_fits` content by removing
the `if_group_fits` start and end tags.


## Test Plan

added test
2024-12-06 13:01:04 +01:00
Wei Lee 39623f8d40
[airflow]: extend removed names (AIR302) (#14804)
## Summary

Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names.
The full list of rules we will extend
https://github.com/apache/airflow/issues/44556


#### package
* `airflow.contrib.*` 

#### module
* `airflow.operators.subdag.*` 

#### class
* `airflow.sensors.external_task.ExternalTaskSensorLink` →
`airflow.sensors.external_task.ExternalDagLin`
* `airflow.operators.bash_operator.BashOperator` →
`airflow.operators.bash.BashOperator`
* `airflow.operators.branch_operator.BaseBranchOperator` →
`airflow.operators.branch.BaseBranchOperator`
* `airflow.operators.dummy.EmptyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy.DummyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy_operator.EmptyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy_operator.DummyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.email_operator.EmailOperator` →
`airflow.operators.email.EmailOperator`
* `airflow.sensors.base_sensor_operator.BaseSensorOperator` →
`airflow.sensors.base.BaseSensorOperator`
* `airflow.sensors.date_time_sensor.DateTimeSensor` →
`airflow.sensors.date_time.DateTimeSensor`
* `airflow.sensors.external_task_sensor.ExternalTaskMarker` →
`airflow.sensors.external_task.ExternalTaskMarker`
* `airflow.sensors.external_task_sensor.ExternalTaskSensor` →
`airflow.sensors.external_task.ExternalTaskSensor`
* `airflow.sensors.external_task_sensor.ExternalTaskSensorLink` →
`airflow.sensors.external_task.ExternalTaskSensorLink`
* `airflow.sensors.time_delta_sensor.TimeDeltaSensor` →
`airflow.sensors.time_delta.TimeDeltaSensor`

#### function
* `airflow.utils.decorators.apply_defaults`
* `airflow.www.utils.get_sensitive_variables_fields` →
`airflow.utils.log.secrets_masker.get_sensitive_variables_fields`
* `airflow.www.utils.should_hide_value_for_key` →
`airflow.utils.log.secrets_masker.should_hide_value_for_key`
* `airflow.configuration.get` → `airflow.configuration.conf.get` 
* `airflow.configuration.getboolean` →
`airflow.configuration.conf.getboolean`
* `airflow.configuration.getfloat` →
`airflow.configuration.conf.getfloat`
* `airflow.configuration.getint` → `airflow.configuration.conf.getint` 
* `airflow.configuration.has_option` →
`airflow.configuration.conf.has_option`
* `airflow.configuration.remove_option` →
`airflow.configuration.conf.remove_option`
* `airflow.configuration.as_dict` → `airflow.configuration.conf.as_dict`
* `airflow.configuration.set` → `airflow.configuration.conf.set` 
* `airflow.secrets.local_filesystem.load_connections` →
`airflow.secrets.local_filesystem.load_connections_dict`
* `airflow.secrets.local_filesystem.get_connection` →
`airflow.secrets.local_filesystem.load_connections_dict`
* `airflow.utils.helpers.chain` → `airflow.models.baseoperator.chain` 
* `airflow.utils.helpers.cross_downstream` →
`airflow.models.baseoperator.cross_downstream`

#### attribute
* in `airflow.utils.trigger_rule.TriggerRule`
    * `DUMMY` 
    * `NONE_FAILED_OR_SKIPPED` 

#### constant / variable
* `airflow.PY\d\d`
2024-12-06 11:34:48 +01:00
Douglas Creager 918358aaa6
Migrate some inference tests to mdtests (#14795)
As part of #13696, this PR ports a smallish number of inference tests
over to the mdtest framework.
2024-12-06 11:19:22 +01:00
David Peter b01a651e69
[red-knot] Support for TOML configs in Markdown tests (#14785)
## Summary

This adds support for specifying the target Python version from a
Markdown test. It is a somewhat limited ad-hoc solution, but designed to
be future-compatible. TOML blocks can be added to arbitrary sections in
the Markdown block. They have the following format:

````markdown
```toml
[tool.knot.environment]
target-version = "3.13"
```
````

So far, there is nothing else that can be configured, but it should be
straightforward to extend this to things like a custom typeshed path.

This is in preparation for the statically-known branches feature where
we are going to have to specify the target version for lots of tests.

## Test Plan

- New Markdown test that fails without the explicitly specified
`target-version`.
- Manually tested various error paths when specifying a wrong
`target-version` field.
- Made sure that running tests is as fast as before.
2024-12-06 10:22:08 +01:00
Micha Reiser 56afb12ae7
Fix infinite watch loop by ignoring 'uninteresting' watch events (#14809)
## Summary
Fixes https://github.com/astral-sh/ruff/issues/14807

I suspect that this broke when we updated notify, although I'm not quiet
sure how this *ever* worked...

The problem was that the file watcher didn't skip over `Access` events,
but Ruff itself accesses the `pyproject.toml` when checking the project.
That means, Ruff triggers `Access` events but it also schedules a
re-check on every `Access` event... and this goes one forever.

This PR skips over `Access` and `Other` event. `Access` events are
uninteresting because they're only reads, they don't change any file
metadata or content.
The `Other` events should be rare and are mainly to inform about file
watcher changes... we don't need those.

I also added an explicit handling for the `Rescan` event. File watchers
emit a `Rescan` event if they failed to capture some file watching
changes
and it signals that the program should assume that all files might have
changed (the program should do a rescan to *get up to date*).

## Test Plan

I tested that Ruff no longer loops when running `check --watch`. I
verified that Ruff rechecks file after making content changes.
2024-12-06 08:50:29 +00:00
Dhruv Manilawala 40b0b67dd9
[red-knot] Separate invalid syntax code snippets (#14803)
Ref: https://github.com/astral-sh/ruff/pull/14788#discussion_r1872242283

This PR:
* Separates code snippets as individual tests for the invalid syntax
cases
* Adds a general comment explaining why the parser could emit more
syntax errors than expected
2024-12-06 02:41:33 +00:00
Dylan 1bd8fbb6e8
[`flake8-pyi`] Skip all type definitions in `string-or-bytes-too-long (PYI053)` (#14797) 2024-12-05 18:48:54 -06:00
Dhruv Manilawala b0e26e6fc8
Bump version to 0.8.2 (#14789) 2024-12-05 18:06:35 +05:30
Dhruv Manilawala e9941cd714
[red-knot] Move standalone expr inference to `for` non-name target (#14788)
## Summary

Ref: https://github.com/astral-sh/ruff/pull/14754#discussion_r1871040646

## Test Plan

Remove the TODO comment and update the mdtest.
2024-12-05 18:06:20 +05:30
Dhruv Manilawala 43bf1a8907
Add tests for "keyword as identifier" syntax errors (#14754)
## Summary

This is related to #13778, more specifically
https://github.com/astral-sh/ruff/issues/13778#issuecomment-2513556004.

This PR adds various test cases where a keyword is being where an
identifier is expected. The tests are to make sure that red knot doesn't
panic, raises the syntax error and the identifier is added to the symbol
table. The final part allows editor related features like renaming the
symbol.
2024-12-05 17:32:48 +05:30
InSync fda8b1f884
[`ruff`] Unnecessary cast to `int` (`RUF046`) (#14697)
## Summary

Resolves #11412.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-05 10:30:06 +01:00
David Peter 2d3f557875
[red-knot] Fallback for `typing._NoDefaultType` (#14783)
## Summary

`typing_extensions` has a `>=3.13` re-export for the `typing.NoDefault`
singleton, but not for `typing._NoDefaultType`. This causes problems as
soon as we understand `sys.version_info` branches, so we explicity
switch to `typing._NoDefaultType` for Python 3.13 and later.

This is a part of #14759 that I thought might make sense to break out
and merge in isolation.

## Test Plan

New test that will become more meaningful with #12700

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-05 09:17:55 +01:00
David Peter bd27bfab5d
[red-knot] Unify `setup_db()` functions, add `TestDb` builder (#14777)
## Summary

- Instead of seven (more or less similar) `setup_db` functions, use just
one in a single central place.
- For every test that needs customization beyond that, offer a
`TestDbBuilder` that can control the Python target version, custom
typeshed, and pre-existing files.

The main motivation for this is that we're soon going to need
customization of the Python version, and I didn't feel like adding this
to each of the existing `setup_db` functions.
2024-12-04 21:36:54 +01:00
InSync 155d34bbb9
[red-knot] Infer precise types for `len()` calls (#14599)
## Summary

Resolves #14598.

## Test Plan

Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-04 11:16:53 -08:00
Well404 04c887c8fc
Fix references for `async-busy-wait` (#14775) 2024-12-04 18:05:49 +01:00
David Peter af43bd4b0f
[red-knot] Gradual forms do not participate in equivalence/subtyping (#14758)
## Summary

This changeset contains various improvements concerning non-fully-static
types and their relationships:

- Make sure that non-fully-static types do not participate in
equivalence or subtyping.
- Clarify what `Type::is_equivalent_to` actually implements.
- Introduce `Type::is_fully_static`
- New tests making sure that multiple `Any`/`Unknown`s inside unions and
intersections are collapsed.

closes #14524

## Test Plan

- Added new unit tests for union and intersection builder
- Added new unit tests for `Type::is_equivalent_to`
- Added new unit tests for `Type::is_subtype_of`
- Added new property test making sure that non-fully-static types do not
participate in subtyping
2024-12-04 17:11:25 +01:00
Harutaka Kawamura 614917769e
Remove `@` in `pytest.mark.parametrize` rule messages (#14770) 2024-12-04 16:01:10 +01:00
Douglas Creager 8b23086eac
[red-knot] Add `typing.Any` as a spelling for the Any type (#14742)
We already had a representation for the Any type, which we would use
e.g. for expressions without type annotations. We now recognize
`typing.Any` as a way to refer to this type explicitly. Like other
special forms, this is tracked correctly through aliasing, and isn't
confused with local definitions that happen to have the same name.

Closes #14544
2024-12-04 09:56:36 -05:00
David Peter 948549fcdc
[red-knot] Test: Hashable/Sized => A/B (#14769)
## Summary

Minor change that uses two plain classes `A` and `B` instead of
`typing.Sized` and `typing.Hashable`.

The motivation is twofold: I remember that I was confused when I first
saw this test. Was there anything specific to `Sized` and `Hashable`
that was relevant here? (there is, these classes are not overlapping;
and you can build a proper intersection from them; but that's true for
almost all non-builtin classes).

I now ran into another problem while working on #14758: `Sized` and
`Hashable` are protocols that we don't fully understand yet. This
causing some trouble when trying to infer whether these are fully-static
types or not.
2024-12-04 15:00:27 +01:00
David Salvisberg e67f7f243d
[`flake8-type-checking`] Expands TC006 docs to better explain itself (#14749)
Closes: #14676

I think the consensus generally was to keep the rule as-is, but expand
the docs.

## Summary

Expands the docs for TC006 with an explanation for why the type
expression is always quoted, including mention of another potential
benefit to this style.
2024-12-04 13:16:31 +00:00
Dylan c617b2a48a
[`pycodestyle`] Handle f-strings properly for `invalid-escape-sequence (W605)` (#14748)
When fixing an invalid escape sequence in an f-string, each f-string
element is analyzed for valid escape characters prior to creating the
diagnostic and fix. This allows us to safely prefix with `r` to create a
raw string if no valid escape characters were found anywhere in the
f-string, and otherwise insert backslashes.

This fixes a bug in the original implementation: each "f-string part"
was treated separately, so it was not possible to tell whether a valid
escape character was or would be used elsewhere in the f-string.

Progress towards #11491 but format specifiers are not handled in this
PR.
2024-12-04 06:59:14 -06:00
Dhruv Manilawala 575deb5d4d
Check `AIR001` from builtin or providers `operators` module (#14631)
## Summary

This PR makes changes to the `AIR001` rule as per
https://github.com/astral-sh/ruff/pull/14627#discussion_r1860212307.

Additionally,
* Avoid returning the `Diagnostic` and update the checker in the rule
logic for consistency
* Remove test case for different keyword position (I don't think it's
required here)

## Test Plan

Add test cases for multiple operators from various modules.
2024-12-04 13:30:47 +05:30
Wei Lee edce559431
[airflow]: extend removed names (AIR302) (#14734) 2024-12-03 21:39:43 +00:00
Brent Westbrook 62e358e929
[`ruff`] Extend unnecessary-regular-expression to non-literal strings (`RUF055`) (#14679)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-03 15:17:20 +00:00
Alex Waygood 81bfcc9899
Minor followups to RUF052 (#14755)
## Summary

Just some minor followups to the recently merged RUF052 rule, that was
added in bf0fd04:
- Some small tweaks to the docs
- A minor code-style nit
- Some more tests for my peace of mind, just to check that the new
methods on the semantic model are working correctly

I'm adding the "internal" label as this doesn't deserve a changelog
entry. RUF052 is a new rule that hasn't been released yet.

## Test Plan

`cargo test -p ruff_linter`
2024-12-03 13:33:29 +00:00
David Peter 74309008fd
[red-knot] Property tests (#14178)
## Summary

This PR adds a new `property_tests` module with quickcheck-based tests
that verify certain properties of types. The following properties are
currently checked:

* `is_equivalent_to`:
  * is reflexive: `T` is equivalent to itself
* `is_subtype_of`:
  * is reflexive: `T` is a subtype of `T`
* is antisymmetric: if `S <: T` and `T <: S`, then `S` is equivalent to
`T`
  * is transitive: `S <: T` & `T <: U` => `S <: U`
* `is_disjoint_from`:
  * is irreflexive: `T` is not disjoint from `T`
  * is symmetric: `S` disjoint from `T` => `T` disjoint from `S`
* `is_assignable_to`:
  * is reflexive
* `negate`:
  * is an involution: `T.negate().negate()` is equivalent to `T`

There are also some tests that validate higher-level properties like:

* `S <: T` implies that `S` is not disjoint from `T`
* `S <: T` implies that `S` is assignable to `T`
* A singleton type must also be single-valued

These tests found a few bugs so far:

- #14177 
- #14195 
- #14196 
- #14210
- #14731

Some additional notes:

- Quickcheck-based property tests are non-deterministic and finding
counter-examples might take an arbitrary long time. This makes them bad
candidates for running in CI (for every PR). We can think of running
them in a cron-job way from time to time, similar to fuzzing. But for
now, it's only possible to run them locally (see instructions in source
code).
- Some tests currently find false positive "counterexamples" because our
understanding of equivalence of types is not yet complete. We do not
understand that `int | str` is the same as `str | int`, for example.
These tests are in a separate `property_tests::flaky` module.
- Properties can not be formulated in every way possible, due to the
fact that `is_disjoint_from` and `is_subtype_of` can produce false
negative answers.
- The current shrinking implementation is very naive, which leads to
counterexamples that are very long (`str & Any & ~tuple[Any] &
~tuple[Unknown] & ~Literal[""] & ~Literal["a"] | str & int & ~tuple[Any]
& ~tuple[Unknown]`), requiring the developer to simplify manually. It
has not been a major issue so far, but there is a comment in the code
how this can be improved.
- The tests are currently implemented using a macro. This is a single
commit on top which can easily be reverted, if we prefer the plain code
instead. With the macro:
  ```rs
  // `S <: T` implies that `S` can be assigned to `T`.
  type_property_test!(
      subtype_of_implies_assignable_to, db,
forall types s, t. s.is_subtype_of(db, t) => s.is_assignable_to(db, t)
  );
  ```
  without the macro:
  ```rs
  /// `S <: T` implies that `S` can be assigned to `T`.
  #[quickcheck]
  fn subtype_of_implies_assignable_to(s: Ty, t: Ty) -> bool {
      let db = get_cached_db();
  
      let s = s.into_type(&db);
      let t = t.into_type(&db);
  
      !s.is_subtype_of(&*db, t) || s.is_assignable_to(&*db, t)
  }
  ```

## Test Plan

```bash
while cargo test --release -p red_knot_python_semantic --features property_tests types::property_tests; do :; done
```
2024-12-03 13:54:54 +01:00
David Peter a255d79087
[red-knot] `is_subtype_of` fix for `KnownInstance` types (#14750)
## Summary

`KnownInstance::instance_fallback` may return instances of supertypes.
For example, it returns an instance of `_SpecialForm` for `Literal`.
This means it can't be used on the right-hand side of `is_subtype_of`
relationships, because it might lead to false positives.

I can lead to false negatives on the left hand side of `is_subtype_of`,
but this is at least a known limitation. False negatives are fine for
most applications, but false positives can lead to wrong results in
intersection-simplification, for example.

closes #14731

## Test Plan

Added regression test
2024-12-03 12:03:26 +01:00
Alex Waygood 70bd10614f
Improve docs for flake8-use-pathlib rules (#14741)
Flag the perf impact more clearly, add more links, clarify the rule
about the glob module
2024-12-03 07:47:31 +00:00
Lokejoke bf0fd04e4e
[`ruff`] Implemented `used-dummy-variable` (`RUF052`) (#14611)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-03 08:36:16 +01:00
David Peter a69dfd4a74
[red-knot] Simplify tuples containing `Never` (#14744)
## Summary

Simplify tuples containing `Never` to `Never`:

```py
from typing import Never

def never() -> Never: ...

reveal_type((1, never(), "foo"))  # revealed: Never
```

I should note that mypy and pyright do *not* perform this
simplification. I don't know why.


There is [only one
place](5137fcc9c8/crates/red_knot_python_semantic/src/types/infer.rs (L1477-L1484))
where we use `TupleType::new` directly (instead of `Type::tuple`, which
changes behavior here). This appears when creating `TypeVar`
constraints, and it looks to me like it should stay this way, because
we're using `TupleType` to store a list of constraints there, instead of
an actual type. We also store `tuple[constraint1, constraint2, …]` as
the type for the `constraint1, constraint2, …` tuple expression. This
would mean that we infer a type of `tuple[str, Never]` for the following
type variable constraints, without simplifying it to `Never`. This seems
like a weird edge case that's maybe not worth looking further into?!
```py
from typing import Never

#         vvvvvvvvvv
def f[T: (str, Never)](x: T):
    pass
```

## Test Plan

- Added a new unit test. Did not add additional Markdown tests as that
seems superfluous.
- Tested the example above using red knot, mypy, pyright.
- Verified that this allows us to remove `contains_never` from the
property tests
(https://github.com/astral-sh/ruff/pull/14178#discussion_r1866473192)
2024-12-03 08:28:36 +01:00
Micha Reiser c2e17d0399
Possible fix for flaky file watching test (#14543) 2024-12-03 08:22:42 +01:00
Dylan 10fef8bd5d
[`flake8-import-conventions`] Improve syntax check for aliases supplied in configuration for `unconventional-import-alias (ICN001)` (#14745)
This PR improves on #14477 by:

- Ensuring user's do not require the module alias "__debug__", which is unassignable
- Validating the linter settings for
`lint.flake8-import-conventions.extend-aliases` (whereas previously we
only did this for `lint.flake8-import-conventions.aliases`).

Closes #14662
2024-12-02 22:41:47 -06:00
InSync 246a6df87d
[red-knot] Deeper understanding of `LiteralString` (#14649)
## Summary

Resolves #14648.

## Test Plan

Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-03 03:31:58 +00:00
Connor Skees 3e702e12f7
red-knot: support narrowing for bool(E) (#14668)
Resolves https://github.com/astral-sh/ruff/issues/14547 by delegating
narrowing to `E` for `bool(E)` where `E` is some expression.

This change does not include other builtin class constructors which
should also work in this position, like `int(..)` or `float(..)`, as the
original issue does not mention these. It should be easy enough to add
checks for these as well if we want to.

I don't see a lot of markdown tests for malformed input, maybe there's a
better place for the no args and too many args cases to go?

I did see after the fact that it looks like this task was intended for a
new hire.. my apologies. I got here from
https://github.com/astral-sh/ruff/issues/13694, which is marked
help-wanted.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2024-12-03 03:04:59 +00:00
Dylan 91e2d9a139
[`refurb`] Handle non-finite decimals in `verbose-decimal-constructor (FURB157)` (#14596)
This PR extends the Decimal parsing used in [verbose-decimal-constructor
(FURB157)](https://docs.astral.sh/ruff/rules/verbose-decimal-constructor/)
to better handle non-finite `Decimal` objects, avoiding some false
negatives.

Closes #14587

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-02 18:13:20 -06:00
David Peter 5137fcc9c8
[red-knot] Re-enable linter corpus tests (#14736)
## Summary

Seeing the fuzzing results from @dhruvmanila in #13778, I think we can
re-enable these tests. We also had one regression that would have been
caught by these tests, so there is some value in having them enabled.
2024-12-02 20:11:30 +01:00
Matt Ord 83651deac7
[`pylint`] Ignore overload in `PLR0904` (#14730)
Fixes #14727

## Summary

Fixes #14727

## Test Plan

cargo test

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-12-02 14:36:51 +00:00
Alex Waygood 6dfe125f44
Improve error messages and docs for `flake8-comprehensions` rules (#14729) 2024-12-02 13:36:15 +00:00
Micha Reiser f96dfc179f
Revert: [pyflakes] Avoid false positives in `@no_type_check` contexts (F821, F722) (#14615) (#14726) 2024-12-02 14:28:27 +01:00
Tzu-ping Chung 76d2e56501
[airflow] Avoid deprecated values (AIR302) (#14582) 2024-12-02 07:39:26 +00:00
Micha Reiser 30d80d9746
Sort discovered workspace packages for consistent cross-platform package discovery (#14725) 2024-12-02 07:36:08 +00:00
renovate[bot] 5a67d3269b
Update pre-commit dependencies (#14719) 2024-12-02 06:02:56 +00:00
Simon Brugman 48ec3a8add
[`refurb`] Guard `hashlib` imports and mark `hashlib-digest-hex` fix as safe (`FURB181`) (#14694)
## Summary

- Check if `hashlib` and `crypt` imports have been seen for `FURB181`
and `S324`
- Mark the fix for `FURB181` as safe: I think it was accidentally marked
as unsafe in the first place. The rule does not support user-defined
classes as the "fix safety" section suggests.
- Removed `hashlib._Hash`, as it's not part of the `hashlib` module.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

Updated the test snapshots
2024-12-01 20:24:49 -05:00
github-actions[bot] 84748be163
Sync vendored typeshed stubs (#14696)
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
2024-12-01 01:38:31 +00:00
Brent Westbrook 9e017634cb
[`pep8-naming`] Avoid false positive for `class Bar(type(foo))` (`N804`) (#14683) 2024-11-30 22:37:28 +00:00
Simon Brugman 56ae73a925
[`pylint`] Fix false negatives for `ascii` and `sorted` in `len-as-condition` (PLC1802) (#14692) 2024-11-30 14:10:30 -06:00
InSync be07424e80
Increase rule set size (#14689) 2024-11-30 15:12:10 +01:00
Connor Skees 579ef01294
mdtest: include test name in printed rerun command (#14684)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-30 11:01:06 +00:00
Micha Reiser 90487b8cbd
Skip panda rules if panda module hasn't been seen (#14671) 2024-11-29 21:32:51 +00:00
Alex Waygood f3d8c023d3
[`ruff`] Avoid emitting `assignment-in-assert` when all references to the assigned variable are themselves inside `assert`s (`RUF018`) (#14661) 2024-11-29 13:36:59 +00:00
Micha Reiser b63c2e126b
Upgrade Rust toolchain to 1.83 (#14677) 2024-11-29 12:05:05 +00:00
Connor Skees a6402fb51e
mdtest: allow specifying a specific test inside a file (#14670) 2024-11-29 12:59:07 +01:00
Simon Brugman abb3c6ea95
[`flake8-pyi`] Avoid rewriting invalid type expressions in `unnecessary-type-union` (PYI055) (#14660) 2024-11-28 18:30:50 +00:00
Brent Westbrook 224fe75a76
[`ruff`] Implement `unnecessary-regular-expression` (`RUF055`) (#14659)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Simon Brugman <sbrugman@users.noreply.github.com>
2024-11-28 18:29:23 +00:00
Simon Brugman dc29f52750
[`flake8-pyi`, `ruff`] Fix traversal of nested literals and unions (`PYI016`, `PYI051`, `PYI055`, `PYI062`, `RUF041`) (#14641) 2024-11-28 18:07:12 +00:00
David Salvisberg d9cbf2fe44
Avoids unnecessary overhead for `TC004`, when `TC001-003` are disabled (#14657) 2024-11-28 16:28:24 +01:00
Samodya Abeysiriwardane 3f6c65e78c
[red-knot] Fix merged type after if-else without explicit else branch (#14621)
## Summary

Closes: https://github.com/astral-sh/ruff/issues/14593

The final type of a variable after if-statement without explicit else
branch should be similar to having an explicit else branch.

## Test Plan

Originally failed test cases from the bug are added.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-28 06:23:55 -08:00
Dhruv Manilawala 976c37a849
Bump version to 0.8.1 (#14655) 2024-11-28 19:12:50 +05:30
David Peter a378ff38dc
[red-knot] Fix Boolean flags in mdtests (#14654)
## Summary

Similar to #14652, but now with conditions that are `Literal[True]`
(instead of `Literal[False]`), where we want them to be `bool`.
2024-11-28 14:29:35 +01:00
Alex Waygood d8bca0d3a2
Fix bug where methods defined using lambdas were flagged by FURB118 (#14639) 2024-11-28 12:58:23 +00:00
David Peter 6f1cf5b686
[red-knot] Minor fix in MRO tests (#14652)
## Summary

`bool()` is equal to `False`, and we infer `Literal[False]` for it. Which
means that the test here will fail as soon as we treat the body of
this `if` as unreachable.
2024-11-28 10:17:15 +01:00
Alex Waygood f1b2e85339
py-fuzzer: recommend using `uvx` rather than `uv run` to run the fuzzer (#14645) 2024-11-27 22:19:52 +00:00
David Salvisberg 6d61c8aa16
Fixes minor bug in `SemanticModel::lookup_symbol` (#14643)
## Summary

This came up as part of #12927 when implementing
`SemanticModel::simulate_runtime_load`.

Should be fairly self-explanatory, if the scope returns a binding with
`BindingKind::Annotation` the bottom part of the loop gets skipped, so
there's no chance for `seen_function` to have been updated. So unless
there's something subtle going on here, like function scopes never
containing bindings with `BindingKind::Annotation`, this seems like a
bug.

## Test Plan

`cargo nextest run`
2024-11-27 16:50:19 -05:00
David Salvisberg 8a7ba5d2df
[`flake8-type-checking`] Fixes `quote_type_expression` (#14634) 2024-11-27 18:58:48 +01:00
Brent Westbrook 6fcbe8efb4
[`ruff`] Detect redirected-noqa in file-level comments (`RUF101`) (#14635) 2024-11-27 18:25:47 +01:00
Alexandra Valentine-Ketchum c40b37aa36
N811 & N814: eliminate false positives for single-letter names (#14584)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-27 14:38:36 +00:00
Alex Waygood ef0e2a6e1b
Refactor `crates/ruff_python_stdlib/src/builtins.rs` to make it easier to add support for new Python versions (#14632) 2024-11-27 12:20:21 +00:00
Alex Waygood 4fb1416bf4
Minor stylistic improvements for functions detecting PEP-604 unions (#14633) 2024-11-27 11:29:37 +00:00
Dhruv Manilawala f96fa6b0e2
Do not consider f-strings with escaped newlines as multiline (#14624)
## Summary

This PR fixes a bug in the f-string formatting to not consider the
escaped newlines for `is_multiline`. This is done by checking if the
f-string is triple-quoted or not similar to normal string literals.

This is not required to be gated behind preview because the logic change
for `is_multiline` was added in
https://github.com/astral-sh/ruff/pull/14454.

## Test Plan

Add a test case which formats differently on `main`:
https://play.ruff.rs/ea3c55c2-f0fe-474e-b6b8-e3365e0ede5e
2024-11-27 10:25:38 +00:00
Dhruv Manilawala 4cd2b9926e
Gate `is_multiline` change behind preview (#14630)
## Summary

Ref:
https://github.com/astral-sh/ruff/pull/14624#pullrequestreview-2464127254

## Test Plan

The test case in the follow-up PR showcases the difference between
preview and non-preview formatting:
https://github.com/astral-sh/ruff/pull/14624/files#diff-dc25bd4df280d9a9180598075b5bc2d0bac30af956767b373561029309c8f024
2024-11-27 15:50:28 +05:30
Simon Brugman 11a2929ed7
[`ruff`] Implement `unnecessary-nested-literal` (`RUF041`) (#14323)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-27 10:01:50 +00:00
InSync 187974eff4
[`flake8-use-pathlib`] Recommend `Path.iterdir()` over `os.listdir()` (`PTH208`) (#14509)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-27 09:53:13 +00:00
Micha Reiser 14ba469fc0
Use a derive macro for Violations (#14557)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-27 09:41:40 +00:00
David Salvisberg 6fd10e2fe7
[`flake8-type-checking`] Adds implementation for TC007 and TC008 (#12927)
Co-authored-by: Simon Brugman <sbrugman@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-11-27 09:51:20 +01:00
Alex Waygood e0f3eaf1dd
Turn the `fuzz-parser` script into a properly packaged Python project (#14606)
## Summary

This PR gets rid of the `requirements.in` and `requirements.txt` files
in the `scripts/fuzz-parser` directory, and replaces them with
`pyproject.toml` and `uv.lock` files. The script is renamed from
`fuzz-parser` to `py-fuzzer` (since it can now also be used to fuzz
red-knot as well as the parser, following
https://github.com/astral-sh/ruff/pull/14566), and moved from the
`scripts/` directory to the `python/` directory, since it's now a
(uv)-pip-installable project in its own right.

I've been resisting this for a while, because conceptually this script
just doesn't feel "complicated" enough to me for it to be a full-blown
package. However, I think it's time to do this. Making it a proper
package has several advantages:
- It means we can run it from the project root using `uv run` without
having to activate a virtual environment and ensure that all required
dependencies are installed into that environment
- Using a `pyproject.toml` file means that we can express that the
project requires Python 3.12+ to run properly; this wasn't possible
before
- I've been running mypy on the project locally when I've been working
on it or reviewing other people's PRs; now I can put the mypy config for
the project in the `pyproject.toml` file

## Test Plan

I manually tested that all the commands detailed in
`python/py-fuzzer/README.md` work for me locally.

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2024-11-27 08:09:04 +00:00
Dhruv Manilawala c84c690f1e
Avoid invalid syntax for format-spec with quotes for all Python versions (#14625)
## Summary

fixes: #14608

The logic that was only applied for 3.12+ target version needs to be
applied for other versions as well.

## Test Plan

I've moved the existing test cases for 3.12 only to `f_string.py` so
that it's tested against the default target version.

I think we should probably enabled testing for two target version (pre
3.12 and 3.12) but it won't highlight any issue because the parser
doesn't consider this. Maybe we should enable this once we have target
version specific syntax errors in place
(https://github.com/astral-sh/ruff/issues/6591).
2024-11-27 13:19:33 +05:30
Dhruv Manilawala 0d649f9afd
Check that `airflow` module is seen for `AIR001` (#14627) 2024-11-27 07:25:08 +00:00
Lokejoke 82c01aa662
`[pylint]` Implement `len-test` (`PLC1802`) (#14309)
## Summary

This PR implements [`use-implicit-booleaness-not-len` /
`C1802`](https://pylint.pycqa.org/en/latest/user_guide/messages/convention/use-implicit-booleaness-not-len.html)
> For sequences, (strings, lists, tuples), use the fact that empty
sequences are false.

---------

Co-authored-by: xbrtnik1 <524841@mail.muni.cz>
Co-authored-by: xbrtnik1 <xbrtnik1@mail.muni.cz>
2024-11-26 13:30:17 -06:00
Brent Westbrook 9f446faa6c
[pyflakes] Avoid false positives in `@no_type_check` contexts (F821, F722) (#14615) 2024-11-26 19:13:43 +00:00
David Peter b94d6cf567
[red-knot] Fix panic related to f-strings in annotations (#14613)
## Summary

Fix panics related to expressions without inferred types in invalid
syntax examples like:
```py
x: f"Literal[{1 + 2}]" = 3
```
where the `1 + 2` expression (and its sub-expressions) inside the
annotation did not have an inferred type.

## Test Plan

Added new corpus test.
2024-11-26 16:35:44 +01:00
David Peter cd0c97211c
[red-knot] Update KNOWN_FAILURES (#14612)
## Summary

Remove entry that was prevously fixed in
5a30ec0df6.

## Test Plan

```sh
cargo test -p red_knot_workspace -- --ignored linter_af linter_gz
```
2024-11-26 15:56:42 +01:00
David Peter 0e71c9e3bb
[red-knot] Fix unit tests in release mode (#14604)
## Summary

This is about the easiest patch that I can think of. It has a drawback
in that there is no real guarantee this won't happen again. I think this
might be acceptable, given that all of this is a temporary thing.

And we also add a new CI job to prevent regressions like this in the
future.

For the record though, I'm listing alternative approaches I thought of:

- We could get rid of the debug/release distinction and just add `@Todo`
type metadata everywhere. This has possible affects on runtime. The main
reason I didn't follow through with this is that the size of `Type`
increases. We would either have to adapt the `assert_eq_size!` test or
get rid of it. Even if we add messages everywhere and get rid of the
file-and-line-variant in the enum, it's not enough to get back to the
current release-mode size of `Type`.
- We could generally discard `@Todo` meta information when using it in
tests. I think this would be a huge drawback. I like that we can have
the actual messages in the mdtest. And make sure we get the expected
`@Todo` type, not just any `@Todo`. It's also helpful when debugging
tests.

closes #14594

## Test Plan

```rs
cargo nextest run --release
```
2024-11-26 15:40:02 +01:00
Dylan 24c90d6953
[`pylint`] Do not wrap function calls in parentheses in the fix for unnecessary-dunder-call (PLC2801) (#14601) 2024-11-26 06:47:01 -06:00
Tzu-ping Chung fbff4dec3a
[airflow] Avoid implicit DAG schedule (AIR301) (#14581) 2024-11-26 13:38:18 +01:00
Dhruv Manilawala f3dac27e9a
Fix f-string formatting in assignment statement (#14454)
## Summary

fixes: #13813

This PR fixes a bug in the formatting assignment statement when the
value is an f-string.

This is resolved by using custom best fit layouts if the f-string is (a)
not already a flat f-string (thus, cannot be multiline) and (b) is not a
multiline string (thus, cannot be flattened). So, it is used in cases
like the following:
```py
aaaaaaaaaaaaaaaaaa = f"testeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee{
    expression}moreeeeeeeeeeeeeeeee"
```
Which is (a) `FStringLayout::Multiline` and (b) not a multiline.

There are various other examples in the PR diff along with additional
explanation and context as code comments.

## Test Plan

Add multiple test cases for various scenarios.
2024-11-26 15:07:18 +05:30
Simon Brugman e4cefd9bf9
Extend test cases for `flake8-pyi` (#14280) 2024-11-26 09:10:38 +01:00
Lokejoke 9e4ee98109
[`ruff`] Implement `invalid-assert-message-literal-argument` (`RUF040`) (#14488)
## Summary

This PR implements new rule discussed
[here](https://github.com/astral-sh/ruff/discussions/14449).
In short, it searches for assert messages which were unintentionally
used as a expression to be matched against.

## Test Plan

`cargo test` and review of `ruff-ecosystem`
2024-11-25 17:41:07 -06:00
Shaygan Hooshyari 557d583e32
Support `typing.NoReturn` and `typing.Never` (#14559)
Fix #14558 
## Summary

- Add `typing.NoReturn` and `typing.Never` to known instances and infer
them as `Type::Never`
- Add `is_assignable_to` cases for `Type::Never`

I skipped emitting diagnostic for when a function is annotated as
`NoReturn` but it actually returns.

## Test Plan

Added tests from

https://github.com/python/typing/blob/main/conformance/tests/specialtypes_never.py
except from generics and checking if the return value of the function
and the annotations match.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-11-25 21:37:55 +00:00
cake-monotone f98eebdbab
[red-knot] Fix Leaking Narrowing Constraint in `ast::ExprIf` (#14590)
## Summary

Closes #14588


```py
x: Literal[42, "hello"] = 42 if bool_instance() else "hello"
reveal_type(x)  # revealed: Literal[42] | Literal["hello"]

_ = ... if isinstance(x, str) else ...

# The `isinstance` test incorrectly narrows the type of `x`.
# As a result, `x` is revealed as Literal["hello"], but it should remain Literal[42, "hello"].
reveal_type(x)  # revealed: Literal["hello"]
```

## Test Plan
mdtest included!

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-25 10:36:37 -08:00
Simon Brugman c606bf014e
[`flake8-pyi`] Improve autofix safety for `redundant-none-literal` (PYI061) (#14583)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-25 17:40:57 +00:00
Simon Brugman e8fce20736
[`ruff`] Improve autofix safety for `never-union` (RUF020) (#14589) 2024-11-25 18:35:07 +01:00
Dhruv Manilawala 5a30ec0df6
Avoid inferring invalid expr types for string annotation (#14447)
## Summary

fixes: #14440

## Test Plan

Add a test case with all the invalid expressions in a string annotation
context.
2024-11-25 21:27:03 +05:30
Harutaka Kawamura fa22bd604a
Fix `pytest.mark.parametrize` rules to check calls instead of decorators (#14515)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-25 13:55:18 +01:00
Dhruv Manilawala 0c9165fc3a
Use `Result` for failed text document retrieval in LSP requests (#14579)
## Summary

Ref:
https://github.com/astral-sh/ruff-vscode/issues/644#issuecomment-2496588452

## Test Plan

Not sure how to test this as this is mainly to get more context on the
panic that the server is raising.
2024-11-25 15:14:30 +05:30
Dhruv Manilawala efe54081d6
Remove `FormatFStringPart` (#14448)
## Summary

This is just a small refactor to remove the `FormatFStringPart` as it's
only used in the case when the f-string is not implicitly concatenated
in which case the only part is going to be `FString`. In implicitly
concatenated f-strings, we use `StringLike` instead.
2024-11-25 10:29:22 +05:30
Alex Waygood ac23c99744
[`ruff`] Mark fixes for `unsorted-dunder-all` and `unsorted-dunder-slots` as unsafe when there are complex comments in the sequence (`RUF022`, `RUF023`) (#14560) 2024-11-24 12:49:29 +00:00
Charlie Marsh de62e39eba
Use truthiness check in `auto_attribs` detection (#14562) 2024-11-23 22:06:10 -05:00
InSync d285717da8
[`ruff`] Handle `attrs`'s `auto_attribs` correctly (`RUF009`) (#14520)
## Summary

Resolves #14519.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-23 21:46:38 -05:00
InSync 545e9deba3
[`flake8-builtins`] Exempt private built-in modules (`A005`) (#14505)
## Summary

Resolves #12949.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-23 21:39:04 -05:00
Harutaka Kawamura e3d792605f
[`flake8-bugbear`] Fix `mutable-contextvar-default (B039)` to resolve annotated function calls properly (#14532)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix #14525

## Test Plan

<!-- How was it tested? -->

New test cases

---------

Signed-off-by: harupy <hkawamura0130@gmail.com>
2024-11-23 21:29:25 -05:00
Harutaka Kawamura 1f303a5eb6
Simplify `flake8_pytest_style::rules::fail_call` implementation (#14556) 2024-11-23 15:14:28 +01:00
Nikolas Hearp 07d13c6b4a
[B028-doc-update] Update documentation for B028 (#14338)
## Summary
Resolves #14289
The documentation for B028 no_explicit_stacklevel is updated to be more
clear.

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
2024-11-23 07:45:28 +00:00
Dylan e1838aac29
Ignore more rules for stub files (#14541)
This PR causes the following rules to ignore stub files, on the grounds
that it is not under the author's control to appease these lints:

- `PLR0904` https://docs.astral.sh/ruff/rules/too-many-public-methods/
- `PLR0913` https://docs.astral.sh/ruff/rules/too-many-arguments/
- `PLR0917`
https://docs.astral.sh/ruff/rules/too-many-positional-arguments/
- `PLW3201` https://docs.astral.sh/ruff/rules/bad-dunder-method-name/
- `SLOT` https://docs.astral.sh/ruff/rules/#flake8-slots-slot
- `FBT` https://docs.astral.sh/ruff/rules/#flake8-boolean-trap-fbt
(except for FBT003 since that involves a function call.)

Progress towards #14535
2024-11-23 07:41:10 +00:00
Carl Meyer 4ba847f250
[red-knot] remove wrong typevar attribute implementations (#14540) 2024-11-22 13:17:16 -08:00
Dylan 3fda2d17c7
[`ruff`] Auto-add `r` prefix when string has no backslashes for `unraw-re-pattern (RUF039)` (#14536)
This PR adds a sometimes-available, safe autofix for [unraw-re-pattern
(RUF039)](https://docs.astral.sh/ruff/rules/unraw-re-pattern/#unraw-re-pattern-ruf039),
which prepends an `r` prefix. It is used only when the string in
question has no backslahses (and also does not have a `u` prefix, since
that causes a syntax error.)

Closes #14527

Notes: 
- Test fixture unchanged, but snapshot changed to include fix messages.
- This fix is automatically only available in preview since the rule
itself is in preview
2024-11-22 15:09:53 -06:00
Harutaka Kawamura 931fa06d85
Extend `invalid-envvar-default (PLW1508)` to detect `os.environ.get` (#14512) 2024-11-22 19:13:58 +00:00
Micha Reiser e53ac7985d
Enable logging for directory-renamed test (#14533) 2024-11-22 16:41:46 +00:00
David Salvisberg e25e7044ba
[`flake8-type-checking`] Adds implementation for TC006 (#14511)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-22 15:22:59 +01:00
Micha Reiser b80de52592
Consider quotes inside format-specs when choosing the quotes for an f-string (#14493) 2024-11-22 12:43:53 +00:00
David Peter f6b2cd5588
[red-knot] Semantic index: handle invalid `break`s (#14522)
## Summary

This fix addresses panics related to invalid syntax like the following
where a `break` statement is used in a nested definition inside a
loop:

```py
while True:

    def b():
        x: int

        break
```

closes #14342

## Test Plan

* New corpus regression tests.
* New unit test to make sure we handle nested while loops correctly.
This test is passing on `main`, but can easily fail if the
`is_inside_loop` state isn't properly saved/restored.
2024-11-22 13:13:55 +01:00
Micha Reiser 302fe76c2b
Fix unnecessary space around power op in overlong f-string expressions (#14489) 2024-11-22 13:01:22 +01:00
David Peter a90e404c3f
[red-knot] PEP 695 type aliases (#14357)
## Summary

Add support for (non-generic) type aliases. The main motivation behind
this was to get rid of panics involving expressions in (generic) type
aliases. But it turned out the best way to fix it was to implement
(partial) support for type aliases.

```py
type IntOrStr = int | str

reveal_type(IntOrStr)  # revealed: typing.TypeAliasType
reveal_type(IntOrStr.__name__)  # revealed: Literal["IntOrStr"]

x: IntOrStr = 1

reveal_type(x)  # revealed: Literal[1]

def f() -> None:
    reveal_type(x)  # revealed: int | str
```

## Test Plan

- Updated corpus test allow list to reflect that we don't panic anymore.
- Added Markdown-based test for type aliases (`type_alias.md`)
2024-11-22 08:47:14 +01:00
Micha Reiser 8358ad8d25
Ruff 0.8 release (#14486)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: David Salvisberg <dave@daverball.com>
2024-11-22 08:45:19 +01:00
Alex Waygood 2b8b1ef178
Improve docs for some pycodestyle rules (#14517) 2024-11-21 17:26:06 +00:00
Dylan 2efa3fbb62
[`flake8-import-conventions`] Syntax check aliases supplied in configuration for `unconventional-import-alias (ICN001)` (#14477)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-21 15:54:49 +00:00
cmp0xff b9da4305e6
doc(B024): #14455 add annotated but unassgined class variables (#14502)
# Summary

Closes #14455, migrated from https://github.com/astral-sh/docs/pull/106.
2024-11-21 09:08:02 -06:00
Micha Reiser 87043a2415
Limit type size assertion to 64bit (#14514) 2024-11-21 12:49:55 +00:00
David Peter f684b6fff4
[red-knot] Fix: Infer type for typing.Union[..] tuple expression (#14510)
## Summary

Fixes a panic related to sub-expressions of `typing.Union` where we fail
to store a type for the `int, str` tuple-expression in code like this:
```
x: Union[int, str] = 1
```

relates to [my
comment](https://github.com/astral-sh/ruff/pull/14499#discussion_r1851794467)
on #14499.

## Test Plan

New corpus test
2024-11-21 11:49:20 +01:00
David Peter 47f39ed1a0
[red-knot] Meta data for `Type::Todo` (#14500)
## Summary

Adds meta information to `Type::Todo`, allowing developers to easily
trace back the origin of a particular `@Todo` type they encounter.

Instead of `Type::Todo`, we now write either `type_todo!()` which
creates a `@Todo[path/to/source.rs:123]` type with file and line
information, or using `type_todo!("PEP 604 unions not supported")`,
which creates a variant with a custom message.

`Type::Todo` now contains a `TodoType` field. In release mode, this is
just a zero-sized struct, in order not to create any overhead. In debug
mode, this is an `enum` that contains the meta information.

`Type` implements `Copy`, which means that `TodoType` also needs to be
copyable. This limits the design space. We could intern `TodoType`, but
I discarded this option, as it would require us to have access to the
salsa DB everywhere we want to use `Type::Todo`. And it would have made
the macro invocations less ergonomic (requiring us to pass `db`).

So for now, the meta information is simply a `&'static str` / `u32` for
the file/line variant, or a `&'static str` for the custom message.
Anything involving a chain/backtrace of several `@Todo`s or similar is
therefore currently not implemented. Also because we currently don't see
any direct use cases for this, and because all of this will eventually
go away.

Note that the size of `Type` increases from 16 to 24 bytes, but only in
debug mode.

## Test Plan

- Observed the changes in Markdown tests.
- Added custom messages for all `Type::Todo`s that were revealed in the
tests
- Ran red knot in release and debug mode on the following Python file:
  ```py
  def f(x: int) -> int:
      reveal_type(x)
  ```
Prints `@Todo` in release mode and `@Todo(function parameter type)` in
debug mode.
2024-11-21 09:59:47 +01:00
Shaygan Hooshyari aecdb8c144
[red-knot] support `typing.Union` in type annotations (#14499)
Fix #14498

## Summary

This PR adds `typing.Union` support

## Test Plan

I created new tests in mdtest.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-11-20 21:55:33 +00:00
Micha Reiser 942d6eeb9f Stabilize `A004` (#14480) 2024-11-20 13:11:51 +01:00
Alex Waygood 4ccacc80f9 [ruff-0.8] [`FAST`] Further improve docs for `fast-api-non-annotated-depencency` (`FAST002`) (#14467) 2024-11-20 13:11:51 +01:00
Micha Reiser b2bb119c6a Fix failing tests for Ruff 0.8 branch (#14482) 2024-11-20 13:11:51 +01:00
Alex Waygood cef12f4925 [ruff-0.8] Spruce up docs for newly stabilised rules (#14466)
## Summary

- Expand some docs where they're unclear about the motivation, or assume
some knowledge that hasn't been introduced yet
- Add more links to external docs
- Rename PYI063 from `PrePep570PositionalArgument` to
`Pep484StylePositionalOnlyParameter`
- Rename the file `parenthesize_logical_operators.rs` to
`parenthesize_chained_operators.rs`, since the rule is called
`ParenthesizeChainedOperators`, not `ParenthesizeLogicalOperators`

## Test Plan

`cargo test`
2024-11-20 13:11:51 +01:00
Alex Waygood aa7ac2ce0f [ruff-0.8] [`ruff`] Stabilise `unsorted-dunder-all` and `unsorted-dunder-slots` (#14468)
## Summary

These rules were implemented in January, have been very stable, and have
no open issues about them. They were highly requested by the community
prior to being implemented. Let's stabilise them!

## Test Plan

Ecosystem check on this PR.
2024-11-20 13:11:51 +01:00
Micha Reiser adfa723464 Stabilize multiple rules (#14462) 2024-11-20 13:11:51 +01:00
Alex Waygood 11d20a1a51 [ruff-0.8] [`ruff`] Stabilise `parenthesize-chained-operators` (`RUF021`) (#14450) 2024-11-20 13:11:51 +01:00
Micha Reiser e9079e7d95 Remove the deprecated `E999` rule code (#14428) 2024-11-20 13:11:51 +01:00
Alex Waygood c400725713 [ruff 0.8] [`flake8-pytest-style`] Remove deprecated rules PT004 and PT005 (#14385)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-20 13:11:51 +01:00
Alex Waygood 1081694140 [ruff 0.8] [`flake8-annotations`] Remove deprecated rules ANN101 and ANN102 (#14384)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-20 13:11:51 +01:00
Micha Reiser 52f526eb38 Warn instead of error when removed rules are used in ignore (#14435)
Closes https://github.com/astral-sh/ruff/issues/13505
2024-11-20 13:11:51 +01:00
David Salvisberg dc05b38165 [ruff 0.8][`flake8-type-checking`] Rename `TCH` to `TC` (#14438)
Closes #9573
2024-11-20 13:11:51 +01:00
konsti b46cc6ac0b Update pyproject-toml to support PEP 639 (#13902)
Fixes #13869
2024-11-20 13:11:51 +01:00
Dylan 8b925ea626 [`pycodestyle`] Stabilize behavior to ignore stub files in `ambiguous-variable-name (E741)` (#14405) 2024-11-20 13:11:51 +01:00
yataka 1b180c8342 Change default for Python version from 3.8 to 3.9 (#13896)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-20 13:11:51 +01:00
Dylan afeb217452 [`pyupgrade`] Stabilize behavior to show diagnostic even when unfixable in `printf-string-formatting (UP031)` (#14406) 2024-11-20 13:11:51 +01:00
Dylan c0b3dd3745 [`ruff`] Stabilize unsafe fix for `zip-instead-of-pairwise (RUF007)` (#14401)
This PR stabilizes the unsafe fix for [zip-instead-of-pairwise
(RUF007)](https://docs.astral.sh/ruff/rules/zip-instead-of-pairwise/#zip-instead-of-pairwise-ruf007),
which replaces the use of zip with that of itertools.pairwise and has
been available under preview since version 0.5.7.

There are no open issues regarding RUF007 at the time of this writing.
2024-11-20 13:11:51 +01:00
Alex Waygood 5f6607bf54 [ruff 0.8] Remove deprecated rule UP027 (#14382) 2024-11-20 13:11:51 +01:00
InSync b9c53a74f9
[`pycodestyle`] Exempt `pytest.importorskip()` calls (`E402`) (#14474)
## Summary

Resolves #13537.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-19 22:08:15 -05:00
cake-monotone 6a4d207db7
[red-knot] Refactoring the inference logic of lexicographic comparisons (#14422)
## Summary

closes #14279

### Limitations of the Current Implementation
#### Incorrect Error Propagation

In the current implementation of lexicographic comparisons, if the
result of an Eq operation is Ambiguous, the comparison stops
immediately, returning a bool instance. While this may yield correct
inferences, it fails to capture unsupported-operation errors that might
occur in subsequent comparisons.
```py
class A: ...

(int_instance(), A()) < (int_instance(), A())  # should error
```

#### Weak Inference in Specific Cases

> Example: `(int_instance(), "foo") == (int_instance(), "bar")`
> Current result: `bool`
> Expected result: `Literal[False]`

`Eq` and `NotEq` have unique behavior in lexicographic comparisons
compared to other operators. Specifically:
- For `Eq`, if any non-equal pair exists within the tuples being
compared, we can immediately conclude that the tuples are not equal.
- For `NotEq`, if any equal pair exists, we can conclude that the tuples
are unequal.

```py
a = (str_instance(), int_instance(), "foo")

reveal_type(a == a)  # revealed: bool
reveal_type(a != a)  # revealed: bool

b = (str_instance(), int_instance(), "bar")

reveal_type(a == b)  # revealed: bool  # should be Literal[False]
reveal_type(a != b)  # revealed: bool  # should be Literal[True]
```
#### Incorrect Support for Non-Boolean Rich Comparisons

In CPython, aside from `==` and `!=`, tuple comparisons return a
non-boolean result as-is. Tuples do not convert the value into `bool`.

Note: If all pairwise `==` comparisons between elements in the tuples
return Truthy, the comparison then considers the tuples' lengths.
Regardless of the return type of the dunder methods, the final result
can still be a boolean.

```py
from __future__ import annotations

class A:
    def __eq__(self, o: object) -> str:
        return "hello"

    def __ne__(self, o: object) -> bytes:
        return b"world"

    def __lt__(self, o: A) -> float:
        return 3.14

a = (A(), A())

reveal_type(a == a)  # revealed: bool
reveal_type(a != a)  # revealed: bool
reveal_type(a < a)  # revealed: bool # should be: `float | Literal[False]`

```

### Key Changes
One of the major changes is that comparisons no longer end with a `bool`
result when a pairwise `Eq` result is `Ambiguous`. Instead, the function
attempts to infer all possible cases and unions the results. This
improvement allows for more robust type inference and better error
detection.

Additionally, as the function is now optimized for tuple comparisons,
the name has been changed from the more general
`infer_lexicographic_comparison` to `infer_tuple_rich_comparison`.

## Test Plan

mdtest included
2024-11-19 17:32:43 -08:00
Micha Reiser dbbe7a773c
Mark UP043 fix unsafe when the type annotation contains any comments (#14458) 2024-11-19 15:24:02 +01:00
InSync 5f09d4a90a
[`ruff`] `re` and `regex` calls with unraw string as first argument (`RUF039`) (#14446) 2024-11-19 13:44:55 +01:00
David Peter f8c20258ae
[red-knot] Do not panic on f-string format spec expressions (#14436)
## Summary

Previously, we panicked on expressions like `f"{v:{f'0.2f'}}"` because
we did not infer types for expressions nested inside format spec
elements.

## Test Plan

```
cargo nextest run -p red_knot_workspace -- --ignored linter_af linter_gz
```
2024-11-19 10:04:51 +01:00
David Peter d8538d8c98
[red-knot] Narrowing for `type(x) is C` checks (#14432)
## Summary

Add type narrowing for `type(x) is C` conditions (and `else` clauses of
`type(x) is not C` conditionals):

```py
if type(x) is A:
    reveal_type(x)  # revealed: A
else:
    reveal_type(x)  # revealed: A | B
```

closes: #14431, part of: #13694

## Test Plan

New Markdown-based tests.
2024-11-18 16:21:46 +01:00
InSync 3642381489
[`ruff`] Add rule forbidding `map(int, package.__version__.split('.'))` (`RUF048`) (#14373)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-18 13:43:24 +00:00
Micha Reiser 1f07880d5c
Add tests for python version compatibility (#14430) 2024-11-18 12:26:55 +00:00
David Peter d81b6cd334
[red-knot] Types for subexpressions of annotations (#14426)
## Summary

This patches up various missing paths where sub-expressions of type
annotations previously had no type attached. Examples include:
```py
tuple[int, str]
#     ~~~~~~~~

type[MyClass]
#    ~~~~~~~

Literal["foo"]
#       ~~~~~

Literal["foo", Literal[1, 2]]
#              ~~~~~~~~~~~~~

Literal[1, "a", random.illegal(sub[expr + ession])]
#               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

## Test Plan

```
cargo nextest run -p red_knot_workspace -- --ignored linter_af linter_gz
```
2024-11-18 13:03:27 +01:00
Micha Reiser d99210c049
[red-knot] Default to python 3.9 (#14429) 2024-11-18 11:27:40 +00:00
Steve C 577653551c
[`pylint`] - use sets when possible for `PLR1714` autofix (`repeated-equality-comparison`) (#14372) 2024-11-18 08:57:43 +01:00
Dhruv Manilawala 38a385fb6f
Simplify quote annotator logic for list expression (#14425)
## Summary

Follow-up to #14371, this PR simplifies the visitor logic for list
expressions to remove the state management. We just need to make sure
that we visit the nested expressions using the `QuoteAnnotator` and not
the `Generator`. This is similar to what's being done for binary
expressions.

As per the
[grammar](https://typing.readthedocs.io/en/latest/spec/annotations.html#grammar-token-expression-grammar-annotation_expression),
list expressions can be present which can contain other type expressions
(`Callable`):
```
       | <Callable> '[' <Concatenate> '[' (type_expression ',')+
                    (name | '...') ']' ',' type_expression ']'
             (where name must be a valid in-scope ParamSpec)
       | <Callable> '[' '[' maybe_unpacked (',' maybe_unpacked)*
                    ']' ',' type_expression ']'
```

## Test Plan

`cargo insta test`
2024-11-18 12:33:19 +05:30
Charlie Marsh fccbe56d23
Reverse order of `__contains__` arguments (#14424)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14423.
2024-11-18 03:58:12 +00:00
Shantanu c46555da41
Drive by typo fix (#14420)
Introduced in
https://github.com/astral-sh/ruff/pull/14397/files#diff-42314c006689490bbdfbeeb973de64046b3e069e3d88f67520aeba375f20e655
2024-11-18 03:03:36 +00:00
InSync 0a27c9dabd
[`flake8-pie`] Mark fix as unsafe if the following statement is a string literal (`PIE790`) (#14393)
## Summary

Resolves #12616.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-18 02:30:06 +00:00
InSync 3c9e76eb66
[`flake8-datetimez`] Also exempt `.time()` (`DTZ901`) (#14394)
## Summary

Resolves #14378.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-18 02:24:35 +00:00
Charlie Marsh e1eb188049
Avoid panic in unfixable `redundant-numeric-union` (#14402)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14396.
2024-11-17 12:15:44 -05:00
Shaygan Hooshyari ff19629b11
Understand `typing.Optional` in annotations (#14397) 2024-11-17 17:04:58 +00:00
Micha Reiser cd80c9d907
Fix Red Knot benchmarks on Windows (#14400) 2024-11-17 16:21:09 +00:00
Matt Norton abb34828bd
Improve rule & options documentation (#14329)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-17 10:16:47 +01:00
InSync cab7caf80b
[`flake8-logging`] Suggest `.getLogger(__name__)` instead of `.getLogger(__file__)` (`LOG015`) (#14392) 2024-11-17 09:22:52 +01:00
David Peter d470f29093
[red-knot] Disable linter-corpus tests (#14391)
## Summary

Disable the no-panic tests for the linter corpus, as there are too many
problems right now, requiring linter-contributors to add their test
files to the allow-list.

We can still run the tests using `cargo test -p red_knot_workspace --
--ignored linter_af linter_gz`. This is also why I left the
`crates/ruff_linter/` entries in the allow list for now, even if they
will get out of sync. But let me know if I should rather remove them.
2024-11-16 23:33:19 +01:00
Simon Brugman 1fbed6c325
[`ruff`] Implement `redundant-bool-literal` (`RUF038`) (#14319)
## Summary

Implements `redundant-bool-literal`

## Test Plan

<!-- How was it tested? -->

`cargo test`

The ecosystem results are all correct, but for `Airflow` the rule is not
relevant due to the use of overloading (and is marked as unsafe
correctly).

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-16 21:52:51 +00:00
David Peter 4dcb7ddafe
[red-knot] Remove duplicates from KNOWN_FAILURES (#14386)
## Summary

- Sort the list of `KNOWN_FAILURES`
- Remove accidental duplicates
2024-11-16 20:54:21 +01:00
Micha Reiser 5be90c3a67
Split the corpus tests into smaller tests (#14367)
## Summary

This PR splits the corpus tests into smaller chunks because running all
of them takes 8s on my windows machine and it's by far the longest test
in `red_knot_workspace`.

Splitting the tests has the advantage that they run in parallel. This PR
brings down the wall time from 8s to 4s.

This PR also limits the glob for the linter tests because it's common to
clone cpython into the `ruff_linter/resources/test` folder for
benchmarks (because that's what's written in the contributing guides)

## Test Plan

`cargo test`
2024-11-16 20:29:21 +01:00
Tom Kuson d0dca7bfcf
[`pydoclint`] Update diagnostics to target the docstring (#14381)
## Summary

Updates the `pydoclint` diagnostics to target the docstring instead of a
related statement.

Closes #13184

## Test Plan

`cargo nextest run`
2024-11-16 13:32:20 -05:00
Simon Brugman 78210b198b
[`flake8-pyi`] Implement `redundant-none-literal` (`PYI061`) (#14316)
## Summary

`Literal[None]` can be simplified into `None` in type annotations.

Surprising to see that this is not that rare:
-
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chat_models/base.py#L54
-
https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/sql/annotation.py#L69
- https://github.com/jax-ml/jax/blob/main/jax/numpy/__init__.pyi#L961
-
https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/inference/_common.py#L179

## Test Plan

`cargo test`

Reviewed all ecosystem results, and they are true positives.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-16 18:22:51 +00:00
Simon Brugman 4a2310b595
[`flake8-pyi`] Implement autofix for `redundant-numeric-union` (`PYI041`) (#14273)
## Summary

This PR adds autofix for `redundant-numeric-union` (`PYI041`)

There are some comments below to explain the reasoning behind some
choices that might help review.

<!-- What's the purpose of the change? What does it do, and why? -->

Resolves part of https://github.com/astral-sh/ruff/issues/14185.

## Test Plan

<!-- How was it tested? -->

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-16 18:13:23 +00:00
Dylan fc392c663a
[`flake8-type-checking`] Fix helper function which surrounds annotations in quotes (#14371)
This PR adds corrected handling of list expressions to the `Visitor`
implementation of `QuotedAnnotator` in `flake8_type_checking::helpers`.

Closes #14368
2024-11-16 12:58:02 -05:00
Alex Waygood 81d3c419e9
[red-knot] Simplify some traits in `ast_ids.rs` (#14379) 2024-11-16 17:22:23 +00:00
Micha Reiser a6a3d3f656
Fix file watcher panic when event has no paths (#14364) 2024-11-16 08:36:57 +01:00
Micha Reiser c847cad389
Update insta snapshots (#14366) 2024-11-15 19:31:15 +01:00
Micha Reiser 81e5830585
Workspace discovery (#14308) 2024-11-15 19:20:15 +01:00
Micha Reiser 2b58705cc1
Remove the optional salsa dependency from the AST crate (#14363) 2024-11-15 16:46:04 +00:00
David Peter 9f3235a37f
[red-knot] Expand test corpus (#14360)
## Summary

- Add 383 files from `crates/ruff_python_parser/resources` to the test
corpus
- Add 1296 files from `crates/ruff_linter/resources` to the test corpus
- Use in-memory file system for tests
- Improve test isolation by cleaning the test environment between checks
- Add a mechanism for "known failures". Mark ~80 files as known
failures.
- The corpus test is now a lot slower (6 seconds).

Note:
While `red_knot` as a command line tool can run over all of these
files without panicking, we still have a lot of test failures caused by
explicitly "pulling" all types.

## Test Plan

Run `cargo test -p red_knot_workspace` while making sure that
- Introducing code that is known to lead to a panic fails the test
- Removing code that is known to lead to a panic from
`KNOWN_FAILURES`-files also fails the test
2024-11-15 17:09:15 +01:00
Alex Waygood 62d650226b
[red-knot] Derive more `Default` methods (#14361) 2024-11-15 13:15:41 +00:00
David Peter 5d8a391a3e
[red-knot] Mark LoggingGuard as `must_use` (#14356) 2024-11-15 12:47:25 +01:00
Dhruv Manilawala ed7b98cf9b
Bump version to 0.7.4 (#14358) 2024-11-15 11:17:32 +00:00
Shaygan Hooshyari 6591775cd9
[`flake8-type-checking`] Skip quoting annotation if it becomes invalid syntax (`TCH001`) (#14285)
Fix: #13934 

## Summary

Current implementation has a bug when the current annotation contains a
string with single and double quotes.

TL;DR: I think these cases happen less than other use cases of Literal.
So instead of fixing them we skip the fix in those cases.

One of the problematic cases:

```
from typing import Literal
from third_party import Type

def error(self, type1: Type[Literal["'"]]):
    pass
```

The outcome is:

```
- def error(self, type1: Type[Literal["'"]]):
+ def error(self, type1: "Type[Literal[''']]"):
```

While it should be:

```
"Type[Literal['\'']"
```

The solution in this case is that we check if there’s any quotes same as
the quote style we want to use for this Literal parameter then escape
that same quote used in the string.

Also this case is not uncommon to have:
<https://grep.app/search?current=2&q=Literal["'>

But this can get more complicated for example in case of:

```
- def error(self, type1: Type[Literal["\'"]]):
+ def error(self, type1: "Type[Literal[''']]"):
```

Here we escaped the inner quote but in the generated annotation it gets
removed. Then we flip the quote style of the Literal paramter and the
formatting is wrong.

In this case the solution is more complicated.
1. When generating the string of the source code preserve the backslash.
2. After we have the annotation check if there isn’t any escaped quote
of the same type we want to use for the Literal parameter. In this case
check if we have any `’` without `\` before them. This can get more
complicated since there can be multiple backslashes so checking for only
`\’` won’t be enough.

Another problem is when the string contains `\n`. In case of
`Type[Literal["\n"]]` we generate `'Type[Literal["\n"]]'` and both
pyright and mypy reject this annotation.

https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAySMApiAIYA2AUAMaXkDOjUAKoiQNqsC6AXFAB0w6tQAmJYLBKMYAfQCOAVzCk5tMChjlUjOQCNytANaMGjABYAKRiUrAANLA4BGAQHJ2CLkVIVKnABEADoogTw87gCUfNRQ8VAITIyiElKksooqahpaOih6hiZmTNa29k7w3m5sHJy%2BZFRBoeE8MXEJScxAA

## Test Plan

I added test cases for the original code in the reported issue and two
more cases for backslash and new line.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-11-15 11:11:46 +00:00
Dhruv Manilawala 1f82731856
Use CWD to resolve settings from `ruff.configuration` (#14352)
## Summary

This PR fixes a bug in the Ruff language server where the
editor-specified configuration was resolved relative to the
configuration directory and not the current working directory.

The existing behavior is confusing given that this config file is
specified by the user and is not _discovered_ by Ruff itself. The
behavior of resolving this configuration file should be similar to that
of the `--config` flag on the command-line which uses the current
working directory:
3210f1a23b/crates/ruff/src/resolve.rs (L34-L48)

This creates problems where certain configuration options doesn't work
because the paths resolved in that case are relative to the
configuration directory and not the current working directory in which
the editor is expected to be in. For example, the
`lint.per-file-ignores` doesn't work as mentioned in the linked issue
along with `exclude`, `extend-exclude`, etc.

fixes: #14282 

## Test Plan

Using the following directory tree structure:
```
.
├── .config
│   └── ruff.toml
└── src
    └── migrations
        └── versions
            └── a.py
```

where, the `ruff.toml` is:
```toml
# 1. Comment this out to test `per-file-ignores`
extend-exclude = ["**/versions/*.py"]

[lint]
select = ["D"]

# 2. Comment this out to test `extend-exclude`
[lint.per-file-ignores]
"**/versions/*.py" = ["D"]

# 3. Comment both `per-file-ignores` and `extend-exclude` to test selection works
```

And, the content of `a.py`:
```py
"""Test"""
```

And, the VS Code settings:
```jsonc
{
  "ruff.nativeServer": "on",
  "ruff.path": ["/Users/dhruv/work/astral/ruff/target/debug/ruff"],
  // For single-file mode where current working directory is `/`
  // "ruff.configuration": "/tmp/ruff-repro/.config/ruff.toml",
  // When a workspace is opened containing this path
  "ruff.configuration": "./.config/ruff.toml",
  "ruff.trace.server": "messages",
  "ruff.logLevel": "trace"
}
```

I also tested out just opening the file in single-file mode where the
current working directory is `/` in VS Code. Here, the
`ruff.configuration` needs to be updated to use absolute path as shown
in the above VS Code settings.
2024-11-15 13:45:00 +05:30
Dhruv Manilawala 874da9c400
[red-knot] Display raw characters for string literal (#14351)
## Summary

Closes: #14330 

| `main` | PR |
|--------|--------|
| <img width="693" alt="Screenshot 2024-11-15 at 9 41 09 AM"
src="https://github.com/user-attachments/assets/0d10f2be-2155-4387-8d39-eb1b5027cfd4">
| <img width="800" alt="Screenshot 2024-11-15 at 9 40 27 AM"
src="https://github.com/user-attachments/assets/ba68911c-f4bf-405a-a597-44207b4bde7a">
|


## Test Plan

Add test cases for escape and quote characters.
2024-11-15 13:44:04 +05:30
github-actions[bot] 375cead202
Sync vendored typeshed stubs (#14350) 2024-11-14 22:29:29 -08:00
Dhruv Manilawala 9ec690b8f8
[red-knot] Add support for string annotations (#14151)
## Summary

This PR adds support for parsing and inferring types within string
annotations.

### Implementation (attempt 1)

This is preserved in
6217f48924.

The implementation here would separate the inference of string
annotations in the deferred query. This requires the following:
* Two ways of evaluating the deferred definitions - lazily and eagerly. 
* An eager evaluation occurs right outside the definition query which in
this case would be in `binding_ty` and `declaration_ty`.
* A lazy evaluation occurs on demand like using the
`definition_expression_ty` to determine the function return type and
class bases.
* The above point means that when trying to get the binding type for a
variable in an annotated assignment, the definition query won't include
the type. So, it'll require going through the deferred query to get the
type.

This has the following limitations:
* Nested string annotations, although not necessarily a useful feature,
is difficult to implement unless we convert the implementation in an
infinite loop
* Partial string annotations require complex layout because inferring
the types for stringified and non-stringified parts of the annotation
are done in separate queries. This means we need to maintain additional
information

### Implementation (attempt 2)

This is the final diff in this PR.

The implementation here does the complete inference of string annotation
in the same definition query by maintaining certain state while trying
to infer different parts of an expression and take decisions
accordingly. These are:
* Allow names that are part of a string annotation to not exists in the
symbol table. For example, in `x: "Foo"`, if the "Foo" symbol is not
defined then it won't exists in the symbol table even though it's being
used. This is an invariant which is being allowed only for symbols in a
string annotation.
* Similarly, lookup name is updated to do the same and if the symbol
doesn't exists, then it's not bounded.
* Store the final type of a string annotation on the string expression
itself and not for any of the sub-expressions that are created after
parsing. This is because those sub-expressions won't exists in the
semantic index.

Design document:
https://www.notion.so/astral-sh/String-Annotations-12148797e1ca801197a9f146641e5b71?pvs=4

Closes: #13796 

## Test Plan

* Add various test cases in our markdown framework
* Run `red_knot` on LibCST (contains a lot of string annotations,
specifically
https://github.com/Instagram/LibCST/blob/main/libcst/matchers/_matcher_base.py),
FastAPI (good amount of annotated code including `typing.Literal`) and
compare against the `main` branch output
2024-11-15 04:10:18 +00:00