Commit Graph

919 Commits

Author SHA1 Message Date
InSync ac81c72bf3
[`ruff`] Ambiguous pattern passed to `pytest.raises()` (`RUF043`) (#14966) 2024-12-18 11:53:48 +00:00
David Salvisberg c0b7c36d43
[`ruff`] Avoid false positives for RUF027 for typing context bindings. (#15037)
Closes #14000 

## Summary

For typing context bindings we know that they won't be available at
runtime. We shouldn't recommend a fix, that will result in name errors
at runtime.

## Test Plan

`cargo nextest run`
2024-12-18 08:50:49 +01:00
InSync c9fdb1f5e3
[`pylint`] Preserve original value format (`PLR6104`) (#14978)
## Summary

Resolves #11672.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-17 16:07:07 +01:00
Wei Lee 867a8f9497
feat(AIR302): extend the following rules (#15015)
## Summary


Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names.

* `airflow.api_connexion.security.requires_access_dataset` →
`airflow.api_connexion.security.requires_access_asset`
* `airflow.auth.managers.base_auth_manager.is_authorized_dataset` →
`airflow.auth.managers.base_auth_manager.is_authorized_asset`
* `airflow.auth.managers.models.resource_details.DatasetDetails` →
`airflow.auth.managers.models.resource_details.AssetDetails`
* `airflow.lineage.hook.DatasetLineageInfo` →
`airflow.lineage.hook.AssetLineageInfo`
* `airflow.security.permissions.RESOURCE_DATASET` →
`airflow.security.permissions.RESOURCE_ASSET`
* `airflow.www.auth.has_access_dataset` →
`airflow.www.auth.has_access_dataset.has_access_asset`
* remove `airflow.datasets.DatasetAliasEvent`
* `airflow.datasets.Dataset` → `airflow.sdk.definitions.asset.Asset`
* `airflow.Dataset` → `airflow.sdk.definitions.asset.Asset`
* `airflow.datasets.DatasetAlias` →
`airflow.sdk.definitions.asset.AssetAlias`
* `airflow.datasets.DatasetAll` →
`airflow.sdk.definitions.asset.AssetAll`
* `airflow.datasets.DatasetAny` →
`airflow.sdk.definitions.asset.AssetAny`
* `airflow.datasets.metadata` → `airflow.sdk.definitions.asset.metadata`
* `airflow.datasets.expand_alias_to_datasets` →
`airflow.sdk.definitions.asset.expand_alias_to_assets`
* `airflow.datasets.manager.dataset_manager` → `airflow.assets.manager`
* `airflow.datasets.manager.resolve_dataset_manager` →
`airflow.assets.resolve_asset_manager`
* `airflow.datasets.manager.DatasetManager` →
`airflow.assets.AssetManager`
* `airflow.listeners.spec.dataset.on_dataset_created` →
`airflow.listeners.spec.asset.on_asset_created`
* `airflow.listeners.spec.dataset.on_dataset_changed` →
`airflow.listeners.spec.asset.on_asset_changed`
* `airflow.timetables.simple.DatasetTriggeredTimetable` →
`airflow.timetables.simple.AssetTriggeredTimetable`
* `airflow.timetables.datasets.DatasetOrTimeSchedule` →
`airflow.timetables.assets.AssetOrTimeSchedule`
*
`airflow.providers.amazon.auth_manager.avp.entities.AvpEntities.DATASET`
→ `airflow.providers.amazon.auth_manager.avp.entities.AvpEntities.ASSET`
* `airflow.providers.amazon.aws.datasets.s3.create_dataset` →
`airflow.providers.amazon.aws.assets.s3.create_asset`
*
`airflow.providers.amazon.aws.datasets.s3.convert_dataset_to_openlineage`
→
`airflow.providers.amazon.aws.datasets.s3.convert_dataset_to_openlineage`
* `airflow.providers.amazon.aws.datasets.s3.sanitize_uri` →
`airflow.providers.amazon.aws.assets.s3.sanitize_uri`
*
`airflow.providers.common.io.datasets.file.convert_dataset_to_openlineage`
→ `airflow.providers.common.io.assets.file.convert_asset_to_openlineage`
* `airflow.providers.common.io.datasets.file.sanitize_uri` →
`airflow.providers.common.io.assets.file.sanitize_uri`
* `airflow.providers.common.io.datasets.file.create_dataset` →
`airflow.providers.common.io.assets.file.create_asset`
* `airflow.providers.google.datasets.bigquery.sanitize_uri` →
`airflow.providers.google.assets.bigquery.sanitize_uri`
* `airflow.providers.google.datasets.gcs.create_dataset` →
`airflow.providers.google.assets.gcs.create_asset`
* `airflow.providers.google.datasets.gcs.sanitize_uri` →
`airflow.providers.google.assets.gcs.sanitize_uri`
* `airflow.providers.google.datasets.gcs.convert_dataset_to_openlineage`
→ `airflow.providers.google.assets.gcs.convert_asset_to_openlineage`
*
`airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_dataset`
→
`airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset`
* `airflow.providers.openlineage.utils.utils.DatasetInfo` →
`airflow.providers.openlineage.utils.utils.AssetInfo`
* `airflow.providers.openlineage.utils.utils.translate_airflow_dataset`
→ `airflow.providers.openlineage.utils.utils.translate_airflow_asset`
* `airflow.providers.postgres.datasets.postgres.sanitize_uri` →
`airflow.providers.postgres.assets.postgres.sanitize_uri`
* `airflow.providers.mysql.datasets.mysql.sanitize_uri` →
`airflow.providers.mysql.assets.mysql.sanitize_uri`
* `airflow.providers.trino.datasets.trino.sanitize_uri` →
`airflow.providers.trino.assets.trino.sanitize_uri`

In additional to the newly added rules above, the message for
`airflow.contrib.*` and `airflow.subdag.*` has been extended,
`airflow.sensors.external_task.ExternalTaskSensorLink` error has been
fixed and the test fixture has been reorganized

## Test Plan

A test fixture is included in the PR.
2024-12-17 08:32:48 +01:00
w0nder1ng e22718f25f
[`perflint`] Simplify finding the loop target in `PERF401` (#15025)
Fixes #15012.

```python
def f():
    # panics when the code can't find the loop variable
    values = [1, 2, 3]
    result = []
    for i in values:
        result.append(i + 1)
    del i
```

I'm not sure exactly why this test case panics, but I suspect the `del
i` removes the binding from the semantic model's symbols.

I changed the code to search for the correct binding by directly
iterating through the bindings. Since we know exactly which binding we
want, this should find the loop variable without any complications.
2024-12-17 08:30:32 +01:00
Dylan 6a5eff6017
[`pydocstyle`] Skip leading whitespace for `D403` (#14963)
This PR introduces three changes to `D403`, which has to do with
capitalizing the first word in a docstring.

1. The diagnostic and fix now skip leading whitespace when determining
what counts as "the first word".
2. The name has been changed to `first-word-uncapitalized` from
`first-line-capitalized`, for both clarity and compliance with our rule
naming policy.
3. The diagnostic message and documentation has been modified slightly
to reflect this.

Closes #14890
2024-12-16 09:09:27 -06:00
w0nder1ng 4a7536dc94
[`perflint`] Fix panic in `perf401` (#14971)
Fixes #14969.

The issue was that this line:

```rust
let from_assign_to_loop = TextRange::new(binding_stmt.end(), for_stmt.start());
```

was not safe if the binding was after the target. The only way (at least
that I can think of) this can happen is if they are in different scopes,
so it now checks for that before checking if there are usages between
the two.
2024-12-15 16:22:04 +01:00
Wei Lee dfd7f38009
[airflow]: Import modules that has been moved to airflow providers (AIR303) (#14764)
## Summary

Many core Airflow features have been deprecated and moved to Airflow
Providers since users might need to install an additional package (e.g.,
`apache-airflow-provider-fab==1.0.0`); a separate rule (AIR303) is
created for this.

As some of the changes only relate to the module/package moved, instead
of listing out all the functions, variables, and classes in a module or
a package, it warns the user to import from the new path instead of the
specific name.

The following is the ones that has been moved to
`apache-airflow-provider-fab==1.0.0`

* module moved
* `airflow.api.auth.backend.basic_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.basic_auth`
* `airflow.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.api.auth.backend.kerberos_auth` →
`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth`
* `airflow.auth.managers.fab.security_manager.override` →
`airflow.providers.fab.auth_manager.security_manager.override`
* classes (e.g., functions, classes) moved
* `airflow.www.security.FabAirflowSecurityManagerOverride` →
`airflow.providers.fab.auth_manager.security_manager.override.FabAirflowSecurityManagerOverride`
* `airflow.auth.managers.fab.fab_auth_manager.FabAuthManager` →
`airflow.providers.fab.auth_manager.security_manager.FabAuthManager`

## Test Plan


A test fixture has been included for the rule.
2024-12-13 10:38:07 +01:00
Krishnan Chandra be4ce16735
[`ruff`] Skip SQLModel base classes for `mutable-class-default` (`RUF012`) (#14949)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14892, by adding
`sqlmodel.SQLModel` to the list of classes with default copy semantics.

## Test Plan

Added a test into `RUF012.py` containing the example from the original
issue.
2024-12-12 22:19:21 -06:00
Sergey Mezentsev 68e8496260
[`flake8-use-pathlib`] Extend check for invalid path suffix to include the case `"."` (`PTH210`) (#14902)
## Summary

`PTH210` renamed to `invalid-pathlib-with-suffix` and extended to check for `.with_suffix(".")`. This caused the fix availability to be downgraded to "Sometimes", since there is no fix offered in this case.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Dylan <53534755+dylwil3@users.noreply.github.com>
2024-12-12 13:30:17 -06:00
w0nder1ng 2eac00c60f
[`perflint`] fix invalid hoist in `perf401` (#14369)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-12 09:11:09 +01:00
InSync 6f8d8fa36b
[`ruff`] `if k in d: del d[k]` (`RUF051`) (#14553)
## Summary

Resolves #7537.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-11 11:12:23 +00:00
InSync c8d505c8ea
[`pyupgrade`] Do not report when a UTF-8 comment is followed by a non-UTF-8 one (`UP009`) (#14728)
## Summary

Resolves #14704.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-11 10:30:41 +00:00
InSync 4b8c815b27
[`flake8-bugbear`] `itertools.batched()` without explicit `strict` (`B911`) (#14408)
## Summary

Resolves #14387.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-10 08:39:46 +00:00
InSync c62ba48ad4
[`ruff`] Do not simplify `round()` calls (`RUF046`) (#14832)
## Summary

Part 1 of the big change introduced in #14828. This temporarily causes
all fixes for `round(...)` to be considered unsafe, but they will
eventually be enhanced.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-12-09 16:51:27 +01:00
InSync aa6b812a73
[`flake8-pyi`] Also remove `self` and `cls`'s annotation (`PYI034`) (#14801)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-09 14:59:12 +00:00
Harutaka Kawamura 9c3c59aca9
[`flake8-bugbear`] Skip `B028` if `warnings.warn` is called with `*args` or `**kwargs` (#14870) 2024-12-09 14:32:37 +00:00
Harutaka Kawamura 172143ae77
[`flake8-bugbear`] Fix `B028` to allow `stacklevel` to be explicitly assigned as a positional argument (#14868) 2024-12-09 13:15:43 +00:00
Dimitri Papadopoulos Orfanos 59145098d6
Fix typos found by codespell (#14863)
## Summary

Just fix typos.

## Test Plan

CI tests.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-09 09:32:12 +00:00
Harutaka Kawamura 3d9ac535e9
Fix `pytest-parametrize-names-wrong-type (PT006)` to edit both `argnames` and `argvalues` if both of them are single-element tuples/lists (#14699)
## Summary

Close #11243. Fix `pytest-parametrize-names-wrong-type (PT006)` to edit
both `argnames` and `argvalues` if both of them are single-element
tuples/lists.

```python
# Before fix
@pytest.mark.parametrize(("x",), [(1,), (2,)])
def test_foo(x):
    ...

# After fix:
@pytest.mark.parametrize("x", [1, 2])
def test_foo(x):
    ...
```

## Test Plan

New test cases
2024-12-09 09:58:52 +01:00
Dylan 9d641fa714
[`pylint`] Include parentheses and multiple comparators in check for `boolean-chained-comparison (PLR1716)` (#14781)
This PR introduces three changes to the diagnostic and fix behavior
(still under preview) for [boolean-chained-comparison
(PLR1716)](https://docs.astral.sh/ruff/rules/boolean-chained-comparison/#boolean-chained-comparison-plr1716).

1. We now offer a _fix_ in the case of parenthesized expressions like
`(a < b) and b < c`. The fix will merge the chains of comparisons and
then balance parentheses by _adding_ parentheses to one side of the
expression.
2. We now trigger a diagnostic (and fix) in the case where some
comparisons have multiple comparators like `a < b < c and c < d`.
3. When adjacent comparators are parenthesized, we prefer the left
parenthesization and apply the replacement to the whole parenthesized
range. So, for example, `a < (b) and ((b)) < c` becomes `a < (b) < c`.

While these seem like somewhat disconnected changes, they are actually
related. If we only offered (1), then we would see the following fix
behavior:

```diff
- (a < b) and b < c and ((c < d))
+ (a < b < c) and ((c < d))
```

This is because the fix which add parentheses to the first pair of
comparisons overlaps with the fix that removes the `and` between the
second two comparisons. So the latter fix is deferred. However, the
latter fix does not get a second chance because, upon the next lint
iteration, there is no violation of `PLR1716`.

Upon adopting (2), however, both fixes occur by the time ruff completes
several iterations and we get:

```diff
- (a < b) and b < c and ((c < d))
+ ((a < b < c < d))
```

Finally, (3) fixes a previously unobserved bug wherein the autofix for
`a < (b) and b < c` used to result in `a<(b<c` which gives a syntax
error. It could in theory have been fixed in a separate PR, but seems to
be on theme here.


----------

- Closes #13524
- (1), (2), and (3) are implemented in separate commits for ease of
review and modification.
- Technically a user can trigger an error in ruff (by reaching max
iterations) if they have a humongous boolean chained comparison with
differing parentheses levels.
2024-12-08 22:58:45 -06:00
Alex Waygood 9b8ceb9a2e
[`pyupgrade`] Mark fixes for `convert-typed-dict-functional-to-class` and `convert-named-tuple-functional-to-class` as unsafe if they will remove comments (`UP013`, `UP014`) (#14842) 2024-12-08 18:51:37 +00:00
Thibaut Decombe 8d9e408dbb
Fix `PLW1508` false positive for default string created via a mult operation (#14841) 2024-12-08 18:25:47 +00:00
ABDULRAHMAN ALRAHMA 85402097fc
Improve error messages for except* (B025, B029, B030, B904) #14791 (#14815)
Improves error message for [except*](https://peps.python.org/pep-0654/)
(Rules: B025, B029, B030, B904)

Example python snippet:
```python
try:
    a = 1
except* ValueError:
    a = 2
except* ValueError:
    a = 2

try:
    pass
except* ():
    pass

try:
    pass
except* 1:  # error
    pass

try:
    raise ValueError
except* ValueError:
    raise UserWarning
```
Error messages
Before:
```
$ ruff check --select=B foo.py
foo.py:6:9: B025 try-except block with duplicate exception `ValueError`
foo.py:11:1: B029 Using `except ():` with an empty tuple does not catch anything; add exceptions to handle
foo.py:16:9: B030 `except` handlers should only be exception classes or tuples of exception classes
foo.py:22:5: B904 Within an `except` clause, raise exceptions with `raise ... from err` or `raise ... from None` to distinguish them from errors in exception handling
Found 4 errors.
```
After:
```
$ ruff check --select=B foo.py
foo.py:6:9: B025 try-except* block with duplicate exception `ValueError`
foo.py:11:1: B029 Using `except* ():` with an empty tuple does not catch anything; add exceptions to handle
foo.py:16:9: B030 `except*` handlers should only be exception classes or tuples of exception classes
foo.py:22:5: B904 Within an `except*` clause, raise exceptions with `raise ... from err` or `raise ... from None` to distinguish them from errors in exception handling
Found 4 errors.
```

Closes https://github.com/astral-sh/ruff/issues/14791

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-08 17:37:34 +00:00
Dylan d34013425f
[`flake8-bugbear`] Offer unsafe autofix for `no-explicit-stacklevel` (`B028`) (#14829)
This PR introduces an unsafe autofix for [no-explicit-stacklevel
(B028)](https://docs.astral.sh/ruff/rules/no-explicit-stacklevel/#no-explicit-stacklevel-b028):
we add the `stacklevel` argument, set to `2`.

Closes #14805
2024-12-07 08:24:37 -05:00
Dylan 2c13e6513d
[`flake8-comprehensions`] Skip iterables with named expressions in `unnecessary-map` (`C417`) (#14827)
This PR modifies [unnecessary-map
(C417)](https://docs.astral.sh/ruff/rules/unnecessary-map/#unnecessary-map-c417)
to skip `map` expressions if the iterable contains a named expression,
since those cannot appear in comprehensions.

Closes #14808
2024-12-06 22:00:33 -05:00
Alex Waygood 2119dcab6f
[`ruff`] Teach autofix for `used-dummy-variable` about TypeVars etc. (`RUF052`) (#14819) 2024-12-06 17:05:50 +00:00
Wei Lee 3ea14d7a74
[airflow]: extend removed args (AIR302) (#14765)
## Summary

Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names.

* in `DAG`
    * `sla_miss_callback` was removed
* in `airflow.operators.trigger_dagrun.TriggerDagRunOperator`
    * `execution_date` was removed
* in `airflow.operators.weekday.DayOfWeekSensor`,
`airflow.operators.datetime.BranchDateTimeOperator` and
`airflow.operators.weekday.BranchDayOfWeekOperator`
* `use_task_execution_day` was removed in favor of
`use_task_logical_date`

The full list of rules we will extend
https://github.com/apache/airflow/issues/44556

## Test Plan

<!-- How was it tested? -->
A test fixture is included in the PR.
2024-12-06 17:00:23 +01:00
Alex Waygood 4fdd4ddfaa
[`ruff`] Don't emit `used-dummy-variable` on function parameters (`RUF052`) (#14818) 2024-12-06 14:54:42 +00:00
InSync 89368a62a8
[`flake8-use-pathlib`] Dotless suffix passed to `Path.with_suffix()` (`PTH901`) (#14779)
## Summary

Resolves #14441.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-12-06 13:08:20 +01:00
Wei Lee 39623f8d40
[airflow]: extend removed names (AIR302) (#14804)
## Summary

Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names.
The full list of rules we will extend
https://github.com/apache/airflow/issues/44556


#### package
* `airflow.contrib.*` 

#### module
* `airflow.operators.subdag.*` 

#### class
* `airflow.sensors.external_task.ExternalTaskSensorLink` →
`airflow.sensors.external_task.ExternalDagLin`
* `airflow.operators.bash_operator.BashOperator` →
`airflow.operators.bash.BashOperator`
* `airflow.operators.branch_operator.BaseBranchOperator` →
`airflow.operators.branch.BaseBranchOperator`
* `airflow.operators.dummy.EmptyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy.DummyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy_operator.EmptyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy_operator.DummyOperator` →
`airflow.operators.empty.EmptyOperator`
* `airflow.operators.email_operator.EmailOperator` →
`airflow.operators.email.EmailOperator`
* `airflow.sensors.base_sensor_operator.BaseSensorOperator` →
`airflow.sensors.base.BaseSensorOperator`
* `airflow.sensors.date_time_sensor.DateTimeSensor` →
`airflow.sensors.date_time.DateTimeSensor`
* `airflow.sensors.external_task_sensor.ExternalTaskMarker` →
`airflow.sensors.external_task.ExternalTaskMarker`
* `airflow.sensors.external_task_sensor.ExternalTaskSensor` →
`airflow.sensors.external_task.ExternalTaskSensor`
* `airflow.sensors.external_task_sensor.ExternalTaskSensorLink` →
`airflow.sensors.external_task.ExternalTaskSensorLink`
* `airflow.sensors.time_delta_sensor.TimeDeltaSensor` →
`airflow.sensors.time_delta.TimeDeltaSensor`

#### function
* `airflow.utils.decorators.apply_defaults`
* `airflow.www.utils.get_sensitive_variables_fields` →
`airflow.utils.log.secrets_masker.get_sensitive_variables_fields`
* `airflow.www.utils.should_hide_value_for_key` →
`airflow.utils.log.secrets_masker.should_hide_value_for_key`
* `airflow.configuration.get` → `airflow.configuration.conf.get` 
* `airflow.configuration.getboolean` →
`airflow.configuration.conf.getboolean`
* `airflow.configuration.getfloat` →
`airflow.configuration.conf.getfloat`
* `airflow.configuration.getint` → `airflow.configuration.conf.getint` 
* `airflow.configuration.has_option` →
`airflow.configuration.conf.has_option`
* `airflow.configuration.remove_option` →
`airflow.configuration.conf.remove_option`
* `airflow.configuration.as_dict` → `airflow.configuration.conf.as_dict`
* `airflow.configuration.set` → `airflow.configuration.conf.set` 
* `airflow.secrets.local_filesystem.load_connections` →
`airflow.secrets.local_filesystem.load_connections_dict`
* `airflow.secrets.local_filesystem.get_connection` →
`airflow.secrets.local_filesystem.load_connections_dict`
* `airflow.utils.helpers.chain` → `airflow.models.baseoperator.chain` 
* `airflow.utils.helpers.cross_downstream` →
`airflow.models.baseoperator.cross_downstream`

#### attribute
* in `airflow.utils.trigger_rule.TriggerRule`
    * `DUMMY` 
    * `NONE_FAILED_OR_SKIPPED` 

#### constant / variable
* `airflow.PY\d\d`
2024-12-06 11:34:48 +01:00
Dylan 1bd8fbb6e8
[`flake8-pyi`] Skip all type definitions in `string-or-bytes-too-long (PYI053)` (#14797) 2024-12-05 18:48:54 -06:00
InSync fda8b1f884
[`ruff`] Unnecessary cast to `int` (`RUF046`) (#14697)
## Summary

Resolves #11412.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-05 10:30:06 +01:00
Dylan c617b2a48a
[`pycodestyle`] Handle f-strings properly for `invalid-escape-sequence (W605)` (#14748)
When fixing an invalid escape sequence in an f-string, each f-string
element is analyzed for valid escape characters prior to creating the
diagnostic and fix. This allows us to safely prefix with `r` to create a
raw string if no valid escape characters were found anywhere in the
f-string, and otherwise insert backslashes.

This fixes a bug in the original implementation: each "f-string part"
was treated separately, so it was not possible to tell whether a valid
escape character was or would be used elsewhere in the f-string.

Progress towards #11491 but format specifiers are not handled in this
PR.
2024-12-04 06:59:14 -06:00
Dhruv Manilawala 575deb5d4d
Check `AIR001` from builtin or providers `operators` module (#14631)
## Summary

This PR makes changes to the `AIR001` rule as per
https://github.com/astral-sh/ruff/pull/14627#discussion_r1860212307.

Additionally,
* Avoid returning the `Diagnostic` and update the checker in the rule
logic for consistency
* Remove test case for different keyword position (I don't think it's
required here)

## Test Plan

Add test cases for multiple operators from various modules.
2024-12-04 13:30:47 +05:30
Wei Lee edce559431
[airflow]: extend removed names (AIR302) (#14734) 2024-12-03 21:39:43 +00:00
Brent Westbrook 62e358e929
[`ruff`] Extend unnecessary-regular-expression to non-literal strings (`RUF055`) (#14679)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-12-03 15:17:20 +00:00
Alex Waygood 81bfcc9899
Minor followups to RUF052 (#14755)
## Summary

Just some minor followups to the recently merged RUF052 rule, that was
added in bf0fd04:
- Some small tweaks to the docs
- A minor code-style nit
- Some more tests for my peace of mind, just to check that the new
methods on the semantic model are working correctly

I'm adding the "internal" label as this doesn't deserve a changelog
entry. RUF052 is a new rule that hasn't been released yet.

## Test Plan

`cargo test -p ruff_linter`
2024-12-03 13:33:29 +00:00
Lokejoke bf0fd04e4e
[`ruff`] Implemented `used-dummy-variable` (`RUF052`) (#14611)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-03 08:36:16 +01:00
Dylan 91e2d9a139
[`refurb`] Handle non-finite decimals in `verbose-decimal-constructor (FURB157)` (#14596)
This PR extends the Decimal parsing used in [verbose-decimal-constructor
(FURB157)](https://docs.astral.sh/ruff/rules/verbose-decimal-constructor/)
to better handle non-finite `Decimal` objects, avoiding some false
negatives.

Closes #14587

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-02 18:13:20 -06:00
Matt Ord 83651deac7
[`pylint`] Ignore overload in `PLR0904` (#14730)
Fixes #14727

## Summary

Fixes #14727

## Test Plan

cargo test

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-12-02 14:36:51 +00:00
Micha Reiser f96dfc179f
Revert: [pyflakes] Avoid false positives in `@no_type_check` contexts (F821, F722) (#14615) (#14726) 2024-12-02 14:28:27 +01:00
Tzu-ping Chung 76d2e56501
[airflow] Avoid deprecated values (AIR302) (#14582) 2024-12-02 07:39:26 +00:00
Brent Westbrook 9e017634cb
[`pep8-naming`] Avoid false positive for `class Bar(type(foo))` (`N804`) (#14683) 2024-11-30 22:37:28 +00:00
Simon Brugman 56ae73a925
[`pylint`] Fix false negatives for `ascii` and `sorted` in `len-as-condition` (PLC1802) (#14692) 2024-11-30 14:10:30 -06:00
Alex Waygood f3d8c023d3
[`ruff`] Avoid emitting `assignment-in-assert` when all references to the assigned variable are themselves inside `assert`s (`RUF018`) (#14661) 2024-11-29 13:36:59 +00:00
Simon Brugman abb3c6ea95
[`flake8-pyi`] Avoid rewriting invalid type expressions in `unnecessary-type-union` (PYI055) (#14660) 2024-11-28 18:30:50 +00:00
Brent Westbrook 224fe75a76
[`ruff`] Implement `unnecessary-regular-expression` (`RUF055`) (#14659)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Simon Brugman <sbrugman@users.noreply.github.com>
2024-11-28 18:29:23 +00:00
Simon Brugman dc29f52750
[`flake8-pyi`, `ruff`] Fix traversal of nested literals and unions (`PYI016`, `PYI051`, `PYI055`, `PYI062`, `RUF041`) (#14641) 2024-11-28 18:07:12 +00:00
Alex Waygood d8bca0d3a2
Fix bug where methods defined using lambdas were flagged by FURB118 (#14639) 2024-11-28 12:58:23 +00:00
David Salvisberg 8a7ba5d2df
[`flake8-type-checking`] Fixes `quote_type_expression` (#14634) 2024-11-27 18:58:48 +01:00
Brent Westbrook 6fcbe8efb4
[`ruff`] Detect redirected-noqa in file-level comments (`RUF101`) (#14635) 2024-11-27 18:25:47 +01:00
Alexandra Valentine-Ketchum c40b37aa36
N811 & N814: eliminate false positives for single-letter names (#14584)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-27 14:38:36 +00:00
Simon Brugman 11a2929ed7
[`ruff`] Implement `unnecessary-nested-literal` (`RUF041`) (#14323)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-27 10:01:50 +00:00
InSync 187974eff4
[`flake8-use-pathlib`] Recommend `Path.iterdir()` over `os.listdir()` (`PTH208`) (#14509)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-27 09:53:13 +00:00
David Salvisberg 6fd10e2fe7
[`flake8-type-checking`] Adds implementation for TC007 and TC008 (#12927)
Co-authored-by: Simon Brugman <sbrugman@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-11-27 09:51:20 +01:00
Lokejoke 82c01aa662
`[pylint]` Implement `len-test` (`PLC1802`) (#14309)
## Summary

This PR implements [`use-implicit-booleaness-not-len` /
`C1802`](https://pylint.pycqa.org/en/latest/user_guide/messages/convention/use-implicit-booleaness-not-len.html)
> For sequences, (strings, lists, tuples), use the fact that empty
sequences are false.

---------

Co-authored-by: xbrtnik1 <524841@mail.muni.cz>
Co-authored-by: xbrtnik1 <xbrtnik1@mail.muni.cz>
2024-11-26 13:30:17 -06:00
Brent Westbrook 9f446faa6c
[pyflakes] Avoid false positives in `@no_type_check` contexts (F821, F722) (#14615) 2024-11-26 19:13:43 +00:00
Dylan 24c90d6953
[`pylint`] Do not wrap function calls in parentheses in the fix for unnecessary-dunder-call (PLC2801) (#14601) 2024-11-26 06:47:01 -06:00
Tzu-ping Chung fbff4dec3a
[airflow] Avoid implicit DAG schedule (AIR301) (#14581) 2024-11-26 13:38:18 +01:00
Simon Brugman e4cefd9bf9
Extend test cases for `flake8-pyi` (#14280) 2024-11-26 09:10:38 +01:00
Lokejoke 9e4ee98109
[`ruff`] Implement `invalid-assert-message-literal-argument` (`RUF040`) (#14488)
## Summary

This PR implements new rule discussed
[here](https://github.com/astral-sh/ruff/discussions/14449).
In short, it searches for assert messages which were unintentionally
used as a expression to be matched against.

## Test Plan

`cargo test` and review of `ruff-ecosystem`
2024-11-25 17:41:07 -06:00
Simon Brugman c606bf014e
[`flake8-pyi`] Improve autofix safety for `redundant-none-literal` (PYI061) (#14583)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-25 17:40:57 +00:00
Simon Brugman e8fce20736
[`ruff`] Improve autofix safety for `never-union` (RUF020) (#14589) 2024-11-25 18:35:07 +01:00
Harutaka Kawamura fa22bd604a
Fix `pytest.mark.parametrize` rules to check calls instead of decorators (#14515)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-25 13:55:18 +01:00
Charlie Marsh de62e39eba
Use truthiness check in `auto_attribs` detection (#14562) 2024-11-23 22:06:10 -05:00
InSync d285717da8
[`ruff`] Handle `attrs`'s `auto_attribs` correctly (`RUF009`) (#14520)
## Summary

Resolves #14519.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-23 21:46:38 -05:00
InSync 545e9deba3
[`flake8-builtins`] Exempt private built-in modules (`A005`) (#14505)
## Summary

Resolves #12949.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-23 21:39:04 -05:00
Harutaka Kawamura e3d792605f
[`flake8-bugbear`] Fix `mutable-contextvar-default (B039)` to resolve annotated function calls properly (#14532)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix #14525

## Test Plan

<!-- How was it tested? -->

New test cases

---------

Signed-off-by: harupy <hkawamura0130@gmail.com>
2024-11-23 21:29:25 -05:00
Harutaka Kawamura 931fa06d85
Extend `invalid-envvar-default (PLW1508)` to detect `os.environ.get` (#14512) 2024-11-22 19:13:58 +00:00
David Salvisberg e25e7044ba
[`flake8-type-checking`] Adds implementation for TC006 (#14511)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-22 15:22:59 +01:00
Alex Waygood c400725713 [ruff 0.8] [`flake8-pytest-style`] Remove deprecated rules PT004 and PT005 (#14385)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-20 13:11:51 +01:00
Alex Waygood 1081694140 [ruff 0.8] [`flake8-annotations`] Remove deprecated rules ANN101 and ANN102 (#14384)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-20 13:11:51 +01:00
David Salvisberg dc05b38165 [ruff 0.8][`flake8-type-checking`] Rename `TCH` to `TC` (#14438)
Closes #9573
2024-11-20 13:11:51 +01:00
konsti b46cc6ac0b Update pyproject-toml to support PEP 639 (#13902)
Fixes #13869
2024-11-20 13:11:51 +01:00
Alex Waygood 5f6607bf54 [ruff 0.8] Remove deprecated rule UP027 (#14382) 2024-11-20 13:11:51 +01:00
InSync b9c53a74f9
[`pycodestyle`] Exempt `pytest.importorskip()` calls (`E402`) (#14474)
## Summary

Resolves #13537.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-19 22:08:15 -05:00
Micha Reiser dbbe7a773c
Mark UP043 fix unsafe when the type annotation contains any comments (#14458) 2024-11-19 15:24:02 +01:00
InSync 5f09d4a90a
[`ruff`] `re` and `regex` calls with unraw string as first argument (`RUF039`) (#14446) 2024-11-19 13:44:55 +01:00
InSync 3642381489
[`ruff`] Add rule forbidding `map(int, package.__version__.split('.'))` (`RUF048`) (#14373)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-18 13:43:24 +00:00
Steve C 577653551c
[`pylint`] - use sets when possible for `PLR1714` autofix (`repeated-equality-comparison`) (#14372) 2024-11-18 08:57:43 +01:00
Charlie Marsh fccbe56d23
Reverse order of `__contains__` arguments (#14424)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14423.
2024-11-18 03:58:12 +00:00
InSync 0a27c9dabd
[`flake8-pie`] Mark fix as unsafe if the following statement is a string literal (`PIE790`) (#14393)
## Summary

Resolves #12616.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-18 02:30:06 +00:00
InSync 3c9e76eb66
[`flake8-datetimez`] Also exempt `.time()` (`DTZ901`) (#14394)
## Summary

Resolves #14378.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-18 02:24:35 +00:00
Simon Brugman 1fbed6c325
[`ruff`] Implement `redundant-bool-literal` (`RUF038`) (#14319)
## Summary

Implements `redundant-bool-literal`

## Test Plan

<!-- How was it tested? -->

`cargo test`

The ecosystem results are all correct, but for `Airflow` the rule is not
relevant due to the use of overloading (and is marked as unsafe
correctly).

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-16 21:52:51 +00:00
Simon Brugman 78210b198b
[`flake8-pyi`] Implement `redundant-none-literal` (`PYI061`) (#14316)
## Summary

`Literal[None]` can be simplified into `None` in type annotations.

Surprising to see that this is not that rare:
-
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chat_models/base.py#L54
-
https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/sql/annotation.py#L69
- https://github.com/jax-ml/jax/blob/main/jax/numpy/__init__.pyi#L961
-
https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/inference/_common.py#L179

## Test Plan

`cargo test`

Reviewed all ecosystem results, and they are true positives.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-16 18:22:51 +00:00
Simon Brugman 4a2310b595
[`flake8-pyi`] Implement autofix for `redundant-numeric-union` (`PYI041`) (#14273)
## Summary

This PR adds autofix for `redundant-numeric-union` (`PYI041`)

There are some comments below to explain the reasoning behind some
choices that might help review.

<!-- What's the purpose of the change? What does it do, and why? -->

Resolves part of https://github.com/astral-sh/ruff/issues/14185.

## Test Plan

<!-- How was it tested? -->

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-16 18:13:23 +00:00
Shaygan Hooshyari 6591775cd9
[`flake8-type-checking`] Skip quoting annotation if it becomes invalid syntax (`TCH001`) (#14285)
Fix: #13934 

## Summary

Current implementation has a bug when the current annotation contains a
string with single and double quotes.

TL;DR: I think these cases happen less than other use cases of Literal.
So instead of fixing them we skip the fix in those cases.

One of the problematic cases:

```
from typing import Literal
from third_party import Type

def error(self, type1: Type[Literal["'"]]):
    pass
```

The outcome is:

```
- def error(self, type1: Type[Literal["'"]]):
+ def error(self, type1: "Type[Literal[''']]"):
```

While it should be:

```
"Type[Literal['\'']"
```

The solution in this case is that we check if there’s any quotes same as
the quote style we want to use for this Literal parameter then escape
that same quote used in the string.

Also this case is not uncommon to have:
<https://grep.app/search?current=2&q=Literal["'>

But this can get more complicated for example in case of:

```
- def error(self, type1: Type[Literal["\'"]]):
+ def error(self, type1: "Type[Literal[''']]"):
```

Here we escaped the inner quote but in the generated annotation it gets
removed. Then we flip the quote style of the Literal paramter and the
formatting is wrong.

In this case the solution is more complicated.
1. When generating the string of the source code preserve the backslash.
2. After we have the annotation check if there isn’t any escaped quote
of the same type we want to use for the Literal parameter. In this case
check if we have any `’` without `\` before them. This can get more
complicated since there can be multiple backslashes so checking for only
`\’` won’t be enough.

Another problem is when the string contains `\n`. In case of
`Type[Literal["\n"]]` we generate `'Type[Literal["\n"]]'` and both
pyright and mypy reject this annotation.

https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAySMApiAIYA2AUAMaXkDOjUAKoiQNqsC6AXFAB0w6tQAmJYLBKMYAfQCOAVzCk5tMChjlUjOQCNytANaMGjABYAKRiUrAANLA4BGAQHJ2CLkVIVKnABEADoogTw87gCUfNRQ8VAITIyiElKksooqahpaOih6hiZmTNa29k7w3m5sHJy%2BZFRBoeE8MXEJScxAA

## Test Plan

I added test cases for the original code in the reported issue and two
more cases for backslash and new line.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-11-15 11:11:46 +00:00
Dylan 8095ff0e55
enforce required imports even with useless alias (#14287)
This PR handles a panic that occurs when applying unsafe fixes if a user
inserts a required import (I002) that has a "useless alias" in it, like
`import numpy as numpy`, and also selects PLC0414 (useless-import-alias)

In this case, the fixes alternate between adding the required import
statement, then removing the alias, until the recursion limit is
reached. See linked issue for an example.

Closes #14283

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-14 15:39:38 -06:00
Simon Brugman a40bc6a460
[`ruff`] Implement `none-not-at-end-of-union` (`RUF036`) (#14314) 2024-11-14 19:37:13 +01:00
InSync d8b1afbc6e
[`ruff`] Also report problems for `attrs` dataclasses in preview mode (`RUF008`, `RUF009`) (#14327)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-14 15:13:49 +00:00
David Salvisberg 89aa804b2d
[flake8-type-checking] Fix false positives for `typing.Annotated` (#14311) 2024-11-13 12:17:52 +00:00
InSync f789b12705
[`flake8-logging`] Root logger calls (`LOG015`) (#14302) 2024-11-13 09:11:55 +00:00
InSync 5c548dcc04
[`flake8-datetimez`] Usages of `datetime.max`/`datetime.min` (`DTZ901`) (#14288)
## Summary

Resolves #13217.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-12 20:36:07 +00:00
Simon Brugman bd30701980
[`flake8-pyi`] Improve autofix for nested and mixed type unions `unnecessary-type-union` (`PYI055`) (#14272)
## Summary

This PR improves the fix for `PYI055` to be able to handle nested and
mixed type unions.

It also marks the fix as unsafe when comments are present. 
 
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->
2024-11-12 15:33:51 -05:00
Harutaka Kawamura 2b6d66b793
Fix `pytest-raises-too-broad (PT011)` to flag `pytest.raises` call with keyword `expected_exception` (#14298)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

`pytest-raises-too-broad (PT011)` should be raised when
`expected_exception` is provided as a keyword argument.

```python
def test_foo():
    with pytest.raises(ValueError):  # raises PT011
        raise ValueError("Can't divide 1 by 0")

    # This is minor but a valid pytest.raises call
    with pytest.raises(expected_exception=ValueError):  # doesn't raise PT011 but should
        raise ValueError("Can't divide 1 by 0")
```

`pytest.raises` doc:
https://docs.pytest.org/en/8.3.x/reference/reference.html#pytest.raises

## Test Plan

<!-- How was it tested? -->

Unit tests

Signed-off-by: harupy <hkawamura0130@gmail.com>
2024-11-12 14:28:42 -05:00
InSync 13a1483f1e
[`flake8-pyi`] Add "replace with `Self`" fix (`PYI019`) (#14238)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-12 11:13:15 +00:00
InSync be69f61b3e
[`flake8-simplify`] Infer "unknown" truthiness for literal iterables whose items are all unpacks (`SIM222`) (#14263)
## Summary

Resolves #14237.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-11-11 15:23:34 -05:00
David Salvisberg f82ee8ea59
[flake8-markupsafe] Adds Implementation for MS001 via RUF035 (#14224)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-11 18:30:03 +00:00
Simon Brugman f8aae9b1d6
[`flake8-pyi`] Mark fix as unsafe when type annotation contains comments for `duplicate-literal-member` (`PYI062`) (#14268) 2024-11-11 12:48:14 +00:00