## Summary
This change resolves#15814 to ensure that `SIM401` is only triggered on
known dictionary types. Before, the rule was getting triggered even on
types that _resemble_ a dictionary but are not actually a dictionary.
I did this using the `is_known_to_be_of_type_dict(...)` functionality.
The logic for this function was duplicated in a few spots, so I moved
the code to a central location, removed redundant definitions, and
updated existing calls to use the single definition of the function!
## Test Plan
Since this PR only modifies an existing rule, I made changes to the
existing test instead of adding new ones. I made sure that `SIM401` is
triggered on types that are clearly dictionaries and that it's not
triggered on a simple custom dictionary-like type (using a modified
version of [the code in the issue](#15814))
The additional changes to de-duplicate `is_known_to_be_of_type_dict`
don't break any existing tests -- I think this should be fine since the
logic remains the same (please let me know if you think otherwise, I'm
excited to get feedback and work towards a good fix 🙂).
---------
Co-authored-by: Junhson Jean-Baptiste <junhsonjb@naan.mynetworksettings.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Resolves#15997.
Ruff used to introduce syntax errors while fixing these cases, but no
longer will:
```python
{"a": [], **{},}
# ^^^^ Removed, leaving two contiguous commas
{"a": [], **({})}
# ^^^^^ Removed, leaving a stray closing parentheses
```
Previously, the function would take a shortcut if the unpacked
dictionary is empty; now, both cases are handled using the same logic
introduced in #15394. This change slightly modifies that logic to also
remove the first comma following the dictionary, if and only if it is
empty.
## Test Plan
`cargo nextest run` and `cargo insta test`.
## Summary
Resolves#15936.
The fixes will now attempt to preserve the original iterable's format
and quote it if necessary. For `FURB142`, comments within the fix range
will make it unsafe as well.
## Test Plan
`cargo nextest run` and `cargo insta test`.
## Summary
Resolves#15925.
`N803` now checks for functions instead of parameters. In preview mode,
if a method is decorated with `@override` and the current scope is that
of a class, it will be ignored.
## Test Plan
`cargo nextest run` and `cargo insta test`.
## Summary
Follow-up to #15779.
Prior to this change, non-name expressions are not reported at all:
```python
type(a.b) is type(None) # no error
```
This change enhances the rule so that such cases are also reported in
preview. Additionally:
* The fix will now be marked as unsafe if there are any comments within
its range.
* Error messages are slightly modified.
## Test Plan
`cargo nextest run` and `cargo insta test`.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
This is a new rule to implement the renaming of PEP 695 type parameters
with leading underscores after they have (presumably) been converted
from standalone type variables by either UP046 or UP047. Part of #15642.
I'm not 100% sure the fix is always safe, but I haven't come up with any
counterexamples yet. `Renamer` seems pretty precise, so I don't think
the usual issues with comments apply.
I initially tried writing this as a rule that receives a `Stmt` rather
than a `Binding`, but in that case the
`checker.semantic().current_scope()` was the global scope, rather than
the scope of the type parameters as I needed. Most of the other rules
using `Renamer` also used `Binding`s, but it does have the downside of
offering separate diagnostics for each parameter to rename.
## Test Plan
New snapshot tests for UP049 alone and the combination of UP046, UP049,
and PYI018.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
This is a follow-up to #15726, #15778, and #15794 to preserve the triple
quote and prefix flags in plain strings, bytestrings, and f-strings.
I also added a `StringLiteralFlags::without_triple_quotes` method to
avoid passing along triple quotes in rules like SIM905 where it might
not make sense, as discussed
[here](https://github.com/astral-sh/ruff/pull/15726#discussion_r1930532426).
## Test Plan
Existing tests, plus many new cases in the `generator::tests::quote`
test that should cover all combinations of quotes and prefixes, at least
for simple string bodies.
Closes#7799 when combined with #15694, #15726, #15778, and #15794.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Extend AIR302 with
* `airflow.operators.bash.BashOperator →
airflow.providers.standard.operators.bash.BashOperator`
* change existing rules `airflow.operators.bash_operator.BashOperator →
airflow.operators.bash.BashOperator` to
`airflow.operators.bash_operator.BashOperator →
airflow.providers.standard.operators.bash.BashOperator`
## Test Plan
a test fixture has been updated
## Summary
Given the following code:
```python
set(([x for x in range(5)]))
```
the current implementation of C403 results in
```python
{(x for x in range(5))}
```
which is a set containing a generator rather than the result of the
generator.
This change removes the extraneous parentheses so that the resulting
code is:
```python
{x for x in range(5)}
```
## Test Plan
`cargo nextest run` and `cargo insta test`
## Summary
This is a follow-up to #15565, tracked in #15642, to reuse the string
replacement logic from the other PEP 695 rules instead of the
`Generator`, which has the benefit of preserving more comments. However,
comments in some places are still dropped, so I added a check for this
and update the fix safety accordingly. I also added a `## Fix safety`
section to the docs to reflect this and the existing `isinstance`
caveat.
## Test Plan
Existing UP040 tests, plus some new cases.
## Summary
Resolves#10063 and follow-up to #15521.
The fix is now marked as unsafe if there are any comments within its
range. Tests are adapted from that of #15521.
## Test Plan
`cargo nextest run` and `cargo insta test`.
Both `list` and `dict` expect only a single positional argument. Giving
more positional arguments, or a keyword argument, is a `TypeError` and
neither the lint rule nor its fix make sense in that context.
Closes#15810
## Summary
Fixes https://github.com/astral-sh/ruff/issues/15812 by visiting the
second argument as a type definition.
## Test Plan
New F401 tests based on the report.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Builtin bindings are given a range of `0..0`, which causes strange
behavior when range checks are made at the top of the file. In this
case, the logic of the rule demands that the value of the dict
comprehension is not self-referential (i.e. it does not contain
definitions for any of the variables used within it). This logic was
confused by builtins which looked like they were defined "in the
comprehension", if the comprehension appeared at the top of the file.
Closes#15830
If there is any `ParenthesizedWhitespace` (in the sense of LibCST) after
the function name `sorted` and before the arguments, then we must wrap
`sorted` with parentheses after removing the surrounding function.
Closes#15789
This PR uses the tokens of the parsed annotation available in the
`Checker`, instead of re-lexing (using `SimpleTokenizer`) the
annotation. This avoids some limitations of the `SimpleTokenizer`, such
as not being able to handle number and string literals.
Closes#15816 .
## Summary
Permits suspicious imports (the `S4` namespaced diagnostics) from stub
files.
Closes#15207.
## Test Plan
Added tests and ran `cargo nextest run`. The test files are copied from
the `.py` variants.
## Summary
This is another follow-up to #15726 and #15778, extending the
quote-preserving behavior to f-strings and deleting the now-unused
`Generator::quote` field.
## Details
I also made one unrelated change to `rules/flynt/helpers.rs` to remove a
`to_string` call for making a `Box<str>` and tweaked some arguments to
some of the `Generator::unparse_f_string` methods to make the code
easier to follow, in my opinion. Happy to revert especially the latter
of these if needed.
Unfortunately this still does not fix the issue in #9660, which appears
to be more of an escaping issue than a quote-preservation issue. After
#15726, the result is now `a = f'# {"".join([])}' if 1 else ""` instead
of `a = f"# {''.join([])}" if 1 else ""` (single quotes on the outside
now), but we still don't have the desired behavior of double quotes
everywhere on Python 3.12+. I added a test for this but split it off
into another branch since it ended up being unaddressed here, but my
`dbg!` statements showed the correct preferred quotes going into
[`UnicodeEscape::with_preferred_quote`](https://github.com/astral-sh/ruff/blob/main/crates/ruff_python_literal/src/escape.rs#L54).
## Test Plan
Existing rule and `Generator` tests.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Implements some of #14738, by adding support for 6 new patterns:
```py
re.search("abc", s) is None # ⇒ "abc" not in s
re.search("abc", s) is not None # ⇒ "abc" in s
re.match("abc", s) is None # ⇒ not s.startswith("abc")
re.match("abc", s) is not None # ⇒ s.startswith("abc")
re.fullmatch("abc", s) is None # ⇒ s != "abc"
re.fullmatch("abc", s) is not None # ⇒ s == "abc"
```
## Test Plan
```shell
cargo nextest run
cargo insta review
```
And ran the fix on my startup's repo.
## Note
One minor limitation here:
```py
if not re.match('abc', s) is None:
pass
```
will get fixed to this (technically correct, just not nice):
```py
if not not s.startswith('abc'):
pass
```
This seems fine given that Ruff has this covered: the initial code
should be caught by
[E714](https://docs.astral.sh/ruff/rules/not-is-test/) and the fixed
code should be caught by
[SIM208](https://docs.astral.sh/ruff/rules/double-negation/).
## Summary
Resolves#12717.
This change incorporates the logic added in #15588.
## Test Plan
`cargo nextest run` and `cargo insta test`.
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
## Summary
This is a first step toward fixing #7799 by using the quoting style
stored in the `flags` field on `ast::StringLiteral`s to select a quoting
style. This PR does not include support for f-strings or byte strings.
Several rules also needed small updates to pass along existing quoting
styles instead of using `StringLiteralFlags::default()`. The remaining
snapshot changes are intentional and should preserve the quotes from the
input strings.
## Test Plan
Existing tests with some accepted updates, plus a few new RUF055 tests
for raw strings.
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
* feat
* add is_execute_method_inherits_from_airflow_operator for checking the
removed context key in the execute method
* refactor: rename
* is_airflow_task as is_airflow_task_function_def
* in_airflow_task as in_airflow_task_function_def
* removed_in_3 as airflow_3_removal_expr
* removed_in_3_function_def as airflow_3_removal_function_def
* test:
* reorganize test cases
## Test Plan
a test fixture has been updated
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
**Summary**
Airflow 3.0 removes a set of deprecated context variables that were
phased out in 2.x. This PR introduces lint rules to detect usage of
these removed variables in various patterns, helping identify
incompatibilities. The removed context variables include:
```
conf
execution_date
next_ds
next_ds_nodash
next_execution_date
prev_ds
prev_ds_nodash
prev_execution_date
prev_execution_date_success
tomorrow_ds
yesterday_ds
yesterday_ds_nodash
```
**Detected Patterns and Examples**
The linter now flags the use of removed context variables in the
following scenarios:
1. **Direct Subscript Access**
```python
execution_date = context["execution_date"] # Flagged
```
2. **`.get("key")` Method Calls**
```python
print(context.get("execution_date")) # Flagged
```
3. **Variables Assigned from `get_current_context()`**
If a variable is assigned from `get_current_context()` and then used to
access a removed key:
```python
c = get_current_context()
print(c.get("execution_date")) # Flagged
```
4. **Function Parameters in `@task`-Decorated Functions**
Parameters named after removed context variables in functions decorated
with `@task` are flagged:
```python
from airflow.decorators import task
@task
def my_task(execution_date, **kwargs): # Parameter 'execution_date'
flagged
pass
```
5. **Removed Keys in Task Decorator `kwargs` and Other Scenarios**
Other similar patterns where removed context variables appear (e.g., as
part of `kwargs` in a `@task` function) are also detected.
```
from airflow.decorators import task
@task
def process_with_execution_date(**context):
execution_date = lambda: context["execution_date"] # flagged
print(execution_date)
@task(kwargs={"execution_date": "2021-01-01"}) # flagged
def task_with_kwargs(**context):
pass
```
**Test Plan**
Test fixtures covering various patterns of deprecated context usage are
included in this PR. For example:
```python
from airflow.decorators import task, dag, get_current_context
from airflow.models import DAG
from airflow.operators.dummy import DummyOperator
import pendulum
from datetime import datetime
@task
def access_invalid_key_task(**context):
print(context.get("conf")) # 'conf' flagged
@task
def print_config(**context):
execution_date = context["execution_date"] # Flagged
prev_ds = context["prev_ds"] # Flagged
@task
def from_current_context():
context = get_current_context()
print(context["execution_date"]) # Flagged
# Usage outside of a task decorated function
c = get_current_context()
print(c.get("execution_date")) # Flagged
@task
def some_task(execution_date, **kwargs):
print("execution date", execution_date) # Parameter flagged
@dag(
start_date=pendulum.datetime(2021, 1, 1, tz="UTC")
)
def my_dag():
task1 = DummyOperator(
task_id="task1",
params={
"execution_date": "{{ execution_date }}", # Flagged in template context
},
)
access_invalid_key_task()
print_config()
from_current_context()
dag = my_dag()
class CustomOperator(BaseOperator):
def execute(self, context):
execution_date = context.get("execution_date") # Flagged
next_ds = context.get("next_ds") # Flagged
next_execution_date = context["next_execution_date"] # Flagged
```
Ruff will emit `AIR302` diagnostics for each deprecated usage, with
suggestions when applicable, aiding in code migration to Airflow 3.0.
related: https://github.com/apache/airflow/issues/44409,
https://github.com/apache/airflow/issues/41641
---------
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
## Summary
Fixes#9663 and also improves the fixes for
[RUF055](https://docs.astral.sh/ruff/rules/unnecessary-regular-expression/)
since regular expressions are often written as raw strings.
This doesn't include raw f-strings.
## Test Plan
Existing snapshots for RUF055 and PT009, plus a new `Generator` test and
a regression test for the reported `PIE810` issue.
## Summary
Addresses the second follow up to #15565 in #15642. This was easier than
expected by using this cool destructuring syntax I hadn't used before,
and by assuming
[PYI059](https://docs.astral.sh/ruff/rules/generic-not-last-base-class/)
(`generic-not-last-base-class`).
## Test Plan
Using an existing test, plus two new tests combining multiple base
classes and multiple generics. It looks like I deleted a relevant test,
which I did, but I meant to rename this in #15565. It looks like instead
I copied it and renamed the copy.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
This PR extends our [PEP 695](https://peps.python.org/pep-0695) handling
from the type aliases handled by `UP040` to generic function and class
parameters, as suggested in the latter two examples from #4617:
```python
# Input
T = TypeVar("T", bound=float)
class A(Generic[T]):
...
def f(t: T):
...
# Output
class A[T: float]:
...
def f[T: float](t: T):
...
```
I first implemented this as part of `UP040`, but based on a brief
discussion during a very helpful pairing session with @AlexWaygood, I
opted to split them into rules separate from `UP040` and then also
separate from each other. From a quick look, and based on [this
issue](https://github.com/asottile/pyupgrade/issues/836), I'm pretty
sure neither of these rules is currently in pyupgrade, so I just took
the next available codes, `UP046` and `UP047`.
The last main TODO, noted in the rule file and in the fixture, is to
handle generic method parameters not included in the class itself, `S`
in this case:
```python
T = TypeVar("T")
S = TypeVar("S")
class Foo(Generic[T]):
def bar(self, x: T, y: S) -> S: ...
```
but Alex mentioned that that might be okay to leave for a follow-up PR.
I also left a TODO about handling multiple subclasses instead of bailing
out when more than one is present. I'm not sure how common that would
be, but I can still handle it here, or follow up on that too.
I think this is unrelated to the PR, but when I ran `cargo dev
generate-all`, it removed the rule code `PLW0101` from
`ruff.schema.json`. It seemed unrelated, so I left that out, but I
wanted to mention it just in case.
## Test Plan
New test fixture, `cargo nextest run`
Closes#4617, closes#12542
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
We were mistakenly using `CommentRanges::has_comments` to determine
whether our edits
were safe, which sometimes expands the checked range to the end of a
line. But in order to
determine safety we need to check exactly the range we're replacing.
This bug affected the rules `runtime-cast-value` (`TC006`) and
`quoted-type-alias` (`TC008`)
although it was very unlikely to be hit for `TC006` and for `TC008` we
never hit it because we
were checking the wrong expression.
## Test Plan
`cargo nextest run`
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes parentheses not being stripped in C401. Pretty much the same as
#11607 which fixed it for C400.
## Test Plan
`cargo nextest run`
## Summary
This fixes the infinite loop reported in #12897, where an
`unused-import` that is undefined at the scope of `__all__` is "fixed"
by adding it to `__all__` repeatedly. These changes make it so that only
imports in the global scope will be suggested to add to `__all__` and
the unused local import is simply removed.
## Test Plan
Added a CLI integration test that sets up the same module structure as
the original report
Closes#12897
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Resolves#9467
Parse quoted annotations as if the string content is inside parenthesis.
With this logic `x` and `y` in this example are equal:
```python
y: """
int |
str
"""
z: """(
int |
str
)
"""
```
Also this rule only applies to triple
quotes([link](https://github.com/python/typing-council/issues/9#issuecomment-1890808610)).
This PR is based on the
[comments](https://github.com/astral-sh/ruff/issues/9467#issuecomment-2579180991)
on the issue.
I did one extra change, since we don't want any indentation tokens I am
setting the `State::Other` as the initial state of the Lexer.
Remaining work:
- [x] Add a test case for red-knot.
- [x] Add more tests.
## Test Plan
Added a test which previously failed because quoted annotation contained
indentation.
Added an mdtest for red-knot.
Updated previous test.
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Allow links to issues that appear on the same line as the TODO
directive, if they conform to the format that VSCode's GitHub PR
extension produces.
Revival of #9627 (the branch was stale enough that rebasing was a lot
harder than just making the changes anew). Credit should go to the
author of that PR though.
Closes#8061
Co-authored-by: Martin Bernstorff <martinbernstorff@gmail.com>
## Summary
The initial purpose was to fix#15043, where code like this:
```python
from fastapi import FastAPI, Query
app = FastAPI()
@app.get("/test")
def handler(echo: str = Query("")):
return echo
```
was being fixed to the invalid code below:
```python
from typing import Annotated
from fastapi import FastAPI, Query
app = FastAPI()
@app.get("/test")
def handler(echo: Annotated[str, Query("")]): # changed
return echo
```
As @MichaReiser pointed out, the correct fix is:
```python
from typing import Annotated
from fastapi import FastAPI, Query
app = FastAPI()
@app.get("/test")
def handler(echo: Annotated[str, Query()] = ""): # changed
return echo
```
After fixing the issue for `Query`, I realized that other classes like
`Path`, `Body`, `Cookie`, `Header`, `File`, and `Form` also looked
susceptible to this issue. The last few commits should handle these too,
which I think means this will also close#12913.
I had to reorder the arguments to the `do_stuff` test case because the
new fix removes some default argument values (eg for `Path`:
`some_path_param: str = Path()` becomes `some_path_param: Annotated[str,
Path()]`).
There's also #14484 related to this rule. I'm happy to take a stab at
that here or in a follow up PR too.
## Test Plan
`cargo test`
I also checked the fixed output with `uv run --with fastapi
FAST002_0.py`, but it required making a bunch of additional changes to
the test file that I wasn't sure we wanted in this PR.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Implements upstream diagnostics `PT029`, `PT030`, `PT031` that function
as pytest.warns corollaries of `PT010`, `PT011`, `PT012` respectively.
Most of the implementation and documentation is designed to mirror those
existing diagnostics.
Closes#14239
## Test Plan
Tests for `PT029`, `PT030`, `PT031` largely copied from `PT010`,
`PT011`, `PT012` respectively.
`cargo nextest run`
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fix infinite loop issue reported here #15248.
The issue was caused by the break inside the if block, which caused the
flow to exit in an unforeseen way. This caused other issues, eventually
leading to an infinite loop.
Resolves#15248. Resolves#15336.
## Test Plan
Added failing code to fixture.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: dylwil3 <dylwil3@gmail.com>
When removing `int` in calls like `int(expr)` we may need to keep
parentheses around `expr` even when it is a function call or subscript,
since there may be newlines in between the function/value name and the
opening parentheses/bracket of the argument.
This PR implements that logic.
Closes#15263
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Note: `PLW0101` remains in testing rather than preview, so this PR does
not modify any public behavior (hence the title beginning with
`internal` rather than `pylint`, for the sake of the changelog.)
Fixes an error in the processing of `try` statements in the control flow
graph builder.
When processing a try statement, the block following a `return` was
forced to point to the `finally` block. However, if the return was _in_
the `finally` block, this caused the block to point to itself. In the
case where the whole `try-finally` statement was also included inside of
a loop, this caused an infinite loop in the builder for the control flow
graph as it attempted to resolve edges.
Closes#15248
## Test function
### Source
```python
def l():
while T:
try:
while ():
if 3:
break
finally:
return
```
### Control Flow Graph
```mermaid
flowchart TD
start(("Start"))
return(("End"))
block0[["`*(empty)*`"]]
block1[["Loop continue"]]
block2["return\n"]
block3[["Loop continue"]]
block4["break\n"]
block5["if 3:
break\n"]
block6["while ():
if 3:
break\n"]
block7[["Exception raised"]]
block8["try:
while ():
if 3:
break
finally:
return\n"]
block9["while T:
try:
while ():
if 3:
break
finally:
return\n"]
start --> block9
block9 -- "T" --> block8
block9 -- "else" --> block0
block8 -- "Exception raised" --> block7
block8 -- "else" --> block6
block7 --> block2
block6 -- "()" --> block5
block6 -- "else" --> block2
block5 -- "3" --> block4
block5 -- "else" --> block3
block4 --> block2
block3 --> block6
block2 --> return
block1 --> block9
block0 --> return
```
## Summary
Adds `class-as-data-structure` rule (`B903`). Also compare pylint's `too-few-public-methods` (`PLR0903`).
Took some creative liberty with this by allowing the class to have any
decorators or base classes. There are years-old issues on pylint that
don't approve of the strictness when it comes to these things.
Especially considering that dataclass is a decorator and namedtuple _can
be_ a base class. I feel ignoring those explicitly is redundant all
things considered, but it's not a hill I'm willing to die on!
See: #970
## Test Plan
`cargo test`
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: dylwil3 <dylwil3@gmail.com>
## Summary
This PR re-introduces the control-flow graph implementation which was
first introduced in #5384, and then removed in #9463 due to not being
feature complete. Mainly, it lacked the ability to process
`try`-`except` blocks, along with some more minor bugs.
Closes#8958 and #8959 and #14881.
## Overview of Changes
I will now highlight the major changes implemented in this PR, in order
of implementation.
1. Introduced a post-processing step in loop handling to find any
`continue` or `break` statements within the loop body and redirect them
appropriately.
2. Introduced a loop-continue block which is always placed at the end of
loop blocks, and ensures proper looping regardless of the internal logic
of the block. This resolves#8958.
3. Implemented `try` processing with the following logic (resolves
#8959):
1. In the example below the cfg first encounters a conditional
`ExceptionRaised` forking if an exception was (or will be) raised in the
try block. This is not possible to know (except for trivial cases) so we
assume both paths can be taken unconditionally.
2. Going down the `try` path the cfg goes `try`->`else`->`finally`
unconditionally.
3. Going down the `except` path the cfg will meet several conditional
`ExceptionCaught` which fork depending on the nature of the exception
caught. Again there's no way to know which exceptions may be raised so
both paths are assumed to be taken unconditionally.
4. If none of the exception blocks catch the exception then the cfg
terminates by raising a new exception.
5. A post-processing step is also implemented to redirect any `raises`
or `returns` within the blocks appropriately.
```python
def func():
try:
print("try")
except Exception:
print("Exception")
except OtherException as e:
print("OtherException")
else:
print("else")
finally:
print("finally")
```
```mermaid
flowchart TD
start(("Start"))
return(("End"))
block0[["`*(empty)*`"]]
block1["print(#quot;finally#quot;)\n"]
block2["print(#quot;else#quot;)\n"]
block3["print(#quot;try#quot;)\n"]
block4[["Exception raised"]]
block5["print(#quot;OtherException#quot;)\n"]
block6["try:
print(#quot;try#quot;)
except Exception:
print(#quot;Exception#quot;)
except OtherException as e:
print(#quot;OtherException#quot;)
else:
print(#quot;else#quot;)
finally:
print(#quot;finally#quot;)\n"]
block7["print(#quot;Exception#quot;)\n"]
block8["try:
print(#quot;try#quot;)
except Exception:
print(#quot;Exception#quot;)
except OtherException as e:
print(#quot;OtherException#quot;)
else:
print(#quot;else#quot;)
finally:
print(#quot;finally#quot;)\n"]
block9["try:
print(#quot;try#quot;)
except Exception:
print(#quot;Exception#quot;)
except OtherException as e:
print(#quot;OtherException#quot;)
else:
print(#quot;else#quot;)
finally:
print(#quot;finally#quot;)\n"]
start --> block9
block9 -- "Exception raised" --> block8
block9 -- "else" --> block3
block8 -- "Exception" --> block7
block8 -- "else" --> block6
block7 --> block1
block6 -- "OtherException" --> block5
block6 -- "else" --> block4
block5 --> block1
block4 --> return
block3 --> block2
block2 --> block1
block1 --> block0
block0 --> return
```
6. Implemented `with` processing with the following logic:
1. `with` statements have no conditional execution (apart from the
hidden logic handling the enter and exit), so the block is assumed to
execute unconditionally.
2. The one exception is that exceptions raised within the block may
result in control flow resuming at the end of the block. Since it is not
possible know if an exception will be raised, or if it will be handled
by the context manager, we assume that execution always continues after
`with` blocks even if the blocks contain `raise` or `return` statements.
This is handled in a post-processing step.
## Test Plan
Additional test fixtures and control-flow fixtures were added.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: dylwil3 <dylwil3@gmail.com>
## Summary
Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR add rules for the
following.
* Removed class attribute
* `airflow.providers_manager.ProvidersManager.dataset_factories` →
`airflow.providers_manager.ProvidersManager.asset_factories`
* `airflow.providers_manager.ProvidersManager.dataset_uri_handlers` →
`airflow.providers_manager.ProvidersManager.asset_uri_handlers`
*
`airflow.providers_manager.ProvidersManager.dataset_to_openlineage_converters`
→
`airflow.providers_manager.ProvidersManager.asset_to_openlineage_converters`
* `airflow.lineage.hook.DatasetLineageInfo.dataset` →
`airflow.lineage.hook.AssetLineageInfo.asset`
* Removed class method (subclasses in airflow should also checked)
* `airflow.secrets.base_secrets.BaseSecretsBackend.get_conn_uri` →
`airflow.secrets.base_secrets.BaseSecretsBackend.get_conn_value`
* `airflow.secrets.base_secrets.BaseSecretsBackend.get_connections` →
`airflow.secrets.base_secrets.BaseSecretsBackend.get_connection`
* `airflow.hooks.base.BaseHook.get_connections` → use `get_connection`
* `airflow.datasets.BaseDataset.iter_datasets` →
`airflow.sdk.definitions.asset.BaseAsset.iter_assets`
* `airflow.datasets.BaseDataset.iter_dataset_aliases` →
`airflow.sdk.definitions.asset.BaseAsset.iter_asset_aliases`
* Removed constructor args (subclasses in airflow should also checked)
* argument `filename_template`
in`airflow.utils.log.file_task_handler.FileTaskHandler`
* in `BaseOperator`
* `sla`
* `task_concurrency` → `max_active_tis_per_dag`
* in `BaseAuthManager`
* `appbuilder`
* Removed class variable (subclasses anywhere should be checked)
* in `airflow.plugins_manager.AirflowPlugin`
* `executors` (from #43289)
* `hooks`
* `operators`
* `sensors`
* Replaced names
* `airflow.hooks.base_hook.BaseHook` → `airflow.hooks.base.BaseHook`
* `airflow.operators.dagrun_operator.TriggerDagRunLink` →
`airflow.operators.trigger_dagrun.TriggerDagRunLink`
* `airflow.operators.dagrun_operator.TriggerDagRunOperator` →
`airflow.operators.trigger_dagrun.TriggerDagRunOperator`
* `airflow.operators.python_operator.BranchPythonOperator` →
`airflow.operators.python.BranchPythonOperator`
* `airflow.operators.python_operator.PythonOperator` →
`airflow.operators.python.PythonOperator`
* `airflow.operators.python_operator.PythonVirtualenvOperator` →
`airflow.operators.python.PythonVirtualenvOperator`
* `airflow.operators.python_operator.ShortCircuitOperator` →
`airflow.operators.python.ShortCircuitOperator`
* `airflow.operators.latest_only_operator.LatestOnlyOperator` →
`airflow.operators.latest_only.LatestOnlyOperator`
In additional to the changes above, this PR also add utility functions
and improve docstring.
## Test Plan
A test fixture is included in the PR.
## Summary
This PR fixes an issue where Ruff's `D403` rule
(`first-word-uncapitalized`) was not detecting some single-word edge
cases that are picked up by `pydocstyle`.
The change involves extracting the first word of the docstring by
identifying the first whitespace character. This is consistent with
`pydocstyle` which uses `.split()` - see
8d0cdfc93e/src/pydocstyle/checker.py (L581C13-L581C64)
## Example
Here is a playground example -
https://play.ruff.rs/eab9ea59-92cf-4e44-b1a9-b54b7f69b178
```py
def example1():
"""foo"""
def example2():
"""foo
Hello world!
"""
def example3():
"""foo bar
Hello world!
"""
def example4():
"""
foo
"""
def example5():
"""
foo bar
"""
```
`pydocstyle` detects all five cases:
```bash
$ pydocstyle test.py --select D403
dev/test.py:2 in public function `example1`:
D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:5 in public function `example2`:
D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:11 in public function `example3`:
D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:17 in public function `example4`:
D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:22 in public function `example5`:
D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
```
Ruff (`0.8.4`) fails to catch example2 and example4.
## Test Plan
* Added two new test cases to cover the previously missed single-word
docstring cases.
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Fix#11482. Applies
https://github.com/adamchainz/flake8-comprehensions/pull/205 to ruff.
`C416` should be skipped if comprehension contains unpacking. Here's an
example:
```python
list_of_lists = [[1, 2], [3, 4]]
# ruff suggests `list(list_of_lists)` here, but that would change the result.
# `list(list_of_lists)` is not `[(1, 2), (3, 4)]`
a = [(x, y) for x, y in list_of_lists]
# This is equivalent to `list(list_of_lists)`
b = [x for x in list_of_lists]
```
## Test Plan
<!-- How was it tested? -->
Existing checks
---------
Signed-off-by: harupy <hkawamura0130@gmail.com>
## Summary
Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names and add a function for removed methods
* `airflow.datasets.manager.DatasetManager.register_dataset_change` →
`airflow.assets.manager.AssetManager.register_asset_change`
* `airflow.datasets.manager.DatasetManager.create_datasets` →
`airflow.assets.manager.AssetManager.create_assets`
* `airflow.datasets.manager.DatasetManager.notify_dataset_created` →
`airflow.assets.manager.AssetManager.notify_asset_created`
* `airflow.datasets.manager.DatasetManager.notify_dataset_changed` →
`airflow.assets.manager.AssetManager.notify_asset_changed`
* `airflow.datasets.manager.DatasetManager.notify_dataset_alias_created`
→ `airflow.assets.manager.AssetManager.notify_asset_alias_created`
*
`airflow.providers.amazon.auth_manager.aws_auth_manager.AwsAuthManager.is_authorized_dataset`
→
`airflow.providers.amazon.auth_manager.aws_auth_manager.AwsAuthManager.is_authorized_asset`
* `airflow.lineage.hook.HookLineageCollector.create_dataset` →
`airflow.lineage.hook.HookLineageCollector.create_asset`
* `airflow.lineage.hook.HookLineageCollector.add_input_dataset` →
`airflow.lineage.hook.HookLineageCollector.add_input_asset`
* `airflow.lineage.hook.HookLineageCollector.add_output_dataset` →
`airflow.lineage.hook.HookLineageCollector.dd_output_asset`
* `airflow.lineage.hook.HookLineageCollector.collected_datasets` →
`airflow.lineage.hook.HookLineageCollector.collected_assets`
*
`airflow.providers_manager.ProvidersManager.initialize_providers_dataset_uri_resources`
→
`airflow.providers_manager.ProvidersManager.initialize_providers_asset_uri_resources`
## Test Plan
A test fixture is included in the PR.
## Summary
A follow up PR on https://github.com/astral-sh/ruff/issues/14991
Ruff ignores hardcoded passwords for typed variables. Add a rule to
catch passwords in typed code bases
## Test Plan
Includes 2 more test typed variables
Closes#14000
## Summary
For typing context bindings we know that they won't be available at
runtime. We shouldn't recommend a fix, that will result in name errors
at runtime.
## Test Plan
`cargo nextest run`
Fixes#15012.
```python
def f():
# panics when the code can't find the loop variable
values = [1, 2, 3]
result = []
for i in values:
result.append(i + 1)
del i
```
I'm not sure exactly why this test case panics, but I suspect the `del
i` removes the binding from the semantic model's symbols.
I changed the code to search for the correct binding by directly
iterating through the bindings. Since we know exactly which binding we
want, this should find the loop variable without any complications.
This PR introduces three changes to `D403`, which has to do with
capitalizing the first word in a docstring.
1. The diagnostic and fix now skip leading whitespace when determining
what counts as "the first word".
2. The name has been changed to `first-word-uncapitalized` from
`first-line-capitalized`, for both clarity and compliance with our rule
naming policy.
3. The diagnostic message and documentation has been modified slightly
to reflect this.
Closes#14890
Fixes#14969.
The issue was that this line:
```rust
let from_assign_to_loop = TextRange::new(binding_stmt.end(), for_stmt.start());
```
was not safe if the binding was after the target. The only way (at least
that I can think of) this can happen is if they are in different scopes,
so it now checks for that before checking if there are usages between
the two.
## Summary
Many core Airflow features have been deprecated and moved to Airflow
Providers since users might need to install an additional package (e.g.,
`apache-airflow-provider-fab==1.0.0`); a separate rule (AIR303) is
created for this.
As some of the changes only relate to the module/package moved, instead
of listing out all the functions, variables, and classes in a module or
a package, it warns the user to import from the new path instead of the
specific name.
The following is the ones that has been moved to
`apache-airflow-provider-fab==1.0.0`
* module moved
* `airflow.api.auth.backend.basic_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.basic_auth`
* `airflow.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.security_manager.override` →
`airflow.providers.fab.auth_manager.security_manager.override`
* classes (e.g., functions, classes) moved
* `airflow.www.security.FabAirflowSecurityManagerOverride` →
`airflow.providers.fab.auth_manager.security_manager.override.FabAirflowSecurityManagerOverride`
* `airflow.auth.managers.fab.fab_auth_manager.FabAuthManager` →
`airflow.providers.fab.auth_manager.security_manager.FabAuthManager`
## Test Plan
A test fixture has been included for the rule.
## Summary
Closes https://github.com/astral-sh/ruff/issues/14892, by adding
`sqlmodel.SQLModel` to the list of classes with default copy semantics.
## Test Plan
Added a test into `RUF012.py` containing the example from the original
issue.
## Summary
`PTH210` renamed to `invalid-pathlib-with-suffix` and extended to check for `.with_suffix(".")`. This caused the fix availability to be downgraded to "Sometimes", since there is no fix offered in this case.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Dylan <53534755+dylwil3@users.noreply.github.com>
Junhson Jean-Baptiste
InSync
Vasco Schiavo
Dylan
Brent Westbrook
Alex Waygood
Mike Perlov
Wei Lee
Justin Bramley
Tom Kuson
Garrett Reynolds
Jelle Zijlstra
Ankit Chaurasia
David Salvisberg
Charlie Marsh
wooly18
Akira Noda
guillaumeLepape
Aleksei Latyshev
Shaygan Hooshyari
Micha Reiser
Auguste Lalande
David Salvisberg
Victor Westerhuis
Steve C
w0nder1ng
Mike Bernard
Arnav Gupta
wookie184
Enoch Kan
Harutaka Kawamura
TomerBin
my1e5
Taras Matsyk
Krishnan Chandra
Sergey Mezentsev