Commit Graph

129 Commits

Author SHA1 Message Date
Aarni Koskela
5f1bbf0b6b Move pycodestyle violations to rule modules (#2483) 2023-02-02 14:29:23 -05:00
Martin Fischer
56ad160c05 refactor: Move test_path helper to new test module 2023-02-01 09:17:53 -05:00
Simon Brugman
413acdf83c feat: introduce macro for testing snap files across platforms 2023-01-25 18:00:39 -05:00
Martin Fischer
f6fd702d41 Add #![warn(clippy::pedantic)] to lib.rs and main.rs files
We already enforced pedantic clippy lints via the
following command in .github/workflows/ci.yaml:

    cargo clippy --workspace --all-targets --all-features -- -D warnings -W clippy::pedantic

Additionally adding #![warn(clippy::pedantic)] to all main.rs and lib.rs
has the benefit that violations of pedantic clippy lints are also
reported when just running `cargo clippy` without any arguments and
are thereby also picked up by LSP[1] servers such as rust-analyzer[2].
However for rust-analyzer to run clippy you'll have to configure:

    "rust-analyzer.check.command": "clippy",

in your editor.[3]

[1]: https://microsoft.github.io/language-server-protocol/
[2]: https://rust-analyzer.github.io/
[3]: https://rust-analyzer.github.io/manual.html#configuration
2023-01-25 00:40:29 -05:00
Martin Fischer
2125d0bb54 refactor: Move #![forbid(unsafe_code)] attributes up
What's forbidden is more important than which clippy lints are
ignored and more important directives should come first.
2023-01-25 00:40:29 -05:00
Martin Fischer
269926cec4 refactor: Move redirects out of RuleCodePrefix
This commit removes rule redirects such as ("U" -> "UP") from the
RuleCodePrefix enum because they complicated the generation of that enum
(which we want to change to be prefix-agnostic in the future).

To preserve backwards compatibility redirects are now resolved
before the strum-generated RuleCodePrefix::from_str is invoked.

This change also brings two other advantages:

* Redirects are now only defined once
  (previously they had to be defined twice:
  once in ruff_macros/src/rule_code_prefix.rs
  and a second time in src/registry.rs).

* The deprecated redirects will no longer be suggested in IDE
  autocompletion within pyproject.toml since they are now no
  longer part of the ruff.schema.json.
2023-01-24 09:26:19 -05:00
Martin Fischer
abc9810e2b refactor: Turn RuleSelector into a newtype around RuleCodePrefix
Yet another refactor to let us implement the many-to-many mapping
between codes and rules in a prefix-agnostic way.

We want to break up the RuleCodePrefix[1] enum into smaller enums.
To facilitate that this commit  introduces a new wrapping type around
RuleCodePrefix so that we can start breaking it apart.

[1]: Actually `RuleCodePrefix` is the previous name of the autogenerated
enum ... I renamed it in b19258a243 to
RuleSelector since `ALL` isn't a prefix. This commit now renames it back
but only because the new `RuleSelector` wrapper type, introduced in this
commit, will let us move the `ALL` variant from `RuleCodePrefix` to
`RuleSelector` in the next commit.
2023-01-24 09:26:19 -05:00
Martin Fischer
8f6d8e215c derive-msg-formats 3/5: Introduce Violation::AUTOFIX associated constant
ruff_dev::generate_rules_table previously documented which rules are
autofixable via DiagnosticKind::fixable ... since the DiagnosticKind was
obtained via Rule::kind (and Violation::placeholder) which we both want
to get rid of we have to obtain the autofixability via another way.

This commit implements such another way by adding an AUTOFIX
associated constant to the Violation trait. The constant is of the type
Option<AutoFixkind>, AutofixKind is a new struct containing an
Availability enum { Sometimes, Always}, letting us additionally document
that some autofixes are only available sometimes (which previously
wasn't documented). We intentionally introduce this information in a
struct so that we can easily introduce further autofix metadata in the
future such as autofix applicability[1].

[1]: https://doc.rust-lang.org/stable/nightly-rustc/rustc_errors/enum.Applicability.html
2023-01-19 11:03:32 -05:00
Charlie Marsh
97f55b8e97 Convert remaining call path sites to use SmallVec (#1972) 2023-01-18 14:50:33 -05:00
Martin Fischer
d77675f30d Make ruff::{ast, autofix, directives, rustpython_helpers} private 2023-01-14 18:23:59 -05:00
Martin Fischer
fe7658199d Make ruff::violations private 2023-01-14 18:23:59 -05:00
Martin Fischer
cfa25ea4b0 Make ruff::rules private 2023-01-14 18:23:59 -05:00
Martin Fischer
3b36030461 Introduce ruff::rules module
Resolves #1547.
2023-01-14 11:48:02 -05:00
Martin Fischer
69b356e9b9 Add top-level doc comments for crates
Test by running:

    cargo doc --no-deps --all --open
2023-01-14 10:11:30 -05:00
Martin Fischer
82aff5f9ec Split off ruff_cli crate from ruff library
This lets you test the ruff linters or use the ruff library
without having to compile the ~100 additional dependencies
that are needed by the CLI.

Because we set the following in the [workspace] section of Cargo.toml:

   default-members = [".", "ruff_cli"]

`cargo run` still runs the CLI and `cargo test` still tests
the code in src/ as well as the code in the new ruff_cli crate.
(But you can now also run `cargo test -p ruff` to only test the linters.)
2023-01-13 21:37:54 -05:00
Charlie Marsh
403a004e03 Refactor import-tracking to leverage existing AST bindings (#1856)
This PR refactors our import-tracking logic to leverage our existing
logic for tracking bindings. It's both a significant simplification, a
significant improvement (as we can now track reassignments), and closes
out a bunch of subtle bugs.

Though the AST tracks all bindings (e.g., when parsing `import os as
foo`, we bind the name `foo` to a `BindingKind::Importation` that points
to the `os` module), when I went to implement import tracking (e.g., to
ensure that if the user references `List`, it's actually `typing.List`),
I added a parallel system specifically for this use-case.

That was a mistake, for a few reasons:

1. It didn't track reassignments, so if you had `from typing import
List`, but `List` was later overridden, we'd still consider any
reference to `List` to be `typing.List`.
2. It required a bunch of extra logic, include complex logic to try and
optimize the lookups, since it's such a hot codepath.
3. There were a few bugs in the implementation that were just hard to
correct under the existing abstractions (e.g., if you did `from typing
import Optional as Foo`, then we'd treat any reference to `Foo` _or_
`Optional` as `typing.Optional` (even though, in that case, `Optional`
was really unbound).

The new implementation goes through our existing binding tracking: when
we get a reference, we find the appropriate binding given the current
scope stack, and normalize it back to its original target.

Closes #1690.
Closes #1790.
2023-01-13 20:39:54 -05:00
Martin Fischer
38f5e8f423 Decouple linter module from cache module 2023-01-12 13:09:59 -05:00
Charlie Marsh
b36d4a15b0 Modify visibility and shuffle around some modules (#1807) 2023-01-11 23:57:05 -05:00
Charlie Marsh
f450e2e79d Implement doc line length enforcement (#1804)
This PR implements `W505` (`DocLineTooLong`), which is similar to `E501`
(`LineTooLong`) but confined to doc lines.

I based the "doc line" definition on pycodestyle, which defines a doc
line as a standalone comment or string statement. Our definition is a
bit more liberal, since we consider any string statement a doc line
(even if it's part of a multi-line statement) -- but that seems fine to
me.

Note that, unusually, this rule requires custom extraction from both the
token stream (to find standalone comments) and the AST (to find string
statements).

Closes #1784.
2023-01-11 22:32:14 -05:00
Charlie Marsh
09dc3c7225 Rename Check to Diagnostic (#1725)
Along with:

- `CheckKind` -> `DiagnosticKind`
- `CheckCode` -> `RuleCode`
- `CheckCodePrefix` -> `RuleCodePrefix`
2023-01-08 17:46:20 -05:00
Martin Fischer
90198f7563 structs 2/9: Generate violations.rs from registry.rs
# The changes of this commit were generated using the following code:
# (followed by cargo +nightly fmt)

import re
import subprocess
import io

indent = ' ' * 4

def split_words(s):
    return re.split(r'\b', s)

def checkkind_match_arms(f, strip_some=False):
    bodies = {}

    while (line := next(f).rstrip()) != indent * 2 + '}':
        if line.lstrip().startswith('//'):
            continue

        if line.strip() == '':
            continue
        parts = line.split('=>', maxsplit=1)
        if parts[0].strip() == '_':
            break
        left, body = parts
        left = left.strip()
        body = body.strip()
        if body == '{':
            body = ''
            while (line := next(f).rstrip()) != indent * 3 + '}':
                body += line + '\n'
        else:
            body = body.rstrip(',')
        kind = split_words(left)[3]
        if strip_some:
            body = re.sub('\)$', '', re.sub(r'Some\(', '', body, 1).rstrip(), 1)
        if ('(' in left and not '(..)' in left) or '{' in left:
            body = (
                'let '
                + left.replace('CheckKind::', '').replace('CheckCode::', '')
                + ' = self;\n'
                + body
            )
        bodies[kind] = body

    return bodies

with open('src/registry.rs') as f:
    orig_registry_code = f.read()

    # simplify the parsing of multiline match arms
    registry_code = subprocess.check_output(
        ['rustfmt', '+nightly', '--config', 'force_multiline_blocks=true'],
        encoding='utf-8',
        input=orig_registry_code,
    )

    f = io.StringIO(registry_code)

    # 1. Parse the CheckCode enum
    while next(f).strip() != 'pub enum CheckCode {':
        pass

    checkcode_lines = []
    while (line := next(f).strip().rstrip(',')) != '}':
        checkcode_lines.append(line)

    # 2. Parse the CheckKind enum
    while next(f).strip() != 'pub enum CheckKind {':
        pass

    struct_defs = {}

    while (line := next(f).strip()) != '}':
        if line.startswith('//'):
            continue
        line = line.rstrip(',')
        line = re.sub(r'{', '{ pub ', line)
        line = re.sub(r'\(', '(pub ', line)
        line = re.sub(',', ', pub', line)
        kind = split_words(line)[1]
        struct_defs[kind] = 'pub struct ' + line + (';' * (line[-1] != '}'))

    # 3. parse the kind() impl for CheckKind
    while next(f).rstrip() != "    pub fn kind(&self) -> CheckKind {":
        pass
    assert next(f).strip() == 'match self {'

    placeholders = checkkind_match_arms(f)

    # 4. parse the CheckKind -> CheckCode mapping
    while next(f).strip() != "pub fn code(&self) -> &'static CheckCode {":
        pass
    assert next(f).strip() == 'match self {'

    kind2code = {}
    while (line := next(f).strip().rstrip(',')) != '}':
        if line.startswith('//'):
            continue
        parts = re.split(r'\b', line)
        kind2code[parts[3]] = parts[-2]

    code2kind = {code: kind for kind, code in kind2code.items()}

    # 5. parse the body() impl for CheckKind

    while next(f).rstrip() != "    pub fn body(&self) -> String {":
        pass
    assert next(f).strip() == 'match self {'

    bodies = checkkind_match_arms(f)

    # 6. find fixable
    always_fixable = []
    sometimes_fixable = []

    while next(f).strip() != "// Always-fixable checks.":
        pass

    while (line := next(f).strip()) != '// Conditionally-fixable checks.':
        always_fixable.append(split_words(line)[3])

    while (line := next(f).strip()) != '// Non-fixable checks.':
        sometimes_fixable.append(split_words(line)[3])

    # 7. find autofix message
    while next(f).rstrip() != indent + "pub fn commit(&self) -> Option<String> {":
        pass
    assert next(f).strip() == 'match self {'

    autofix_msg = checkkind_match_arms(f, strip_some=True)

reg = '''\
macro_rules! define_rule_mapping {
    ($($code:ident => $mod:ident::$name:ident,)+) => {
        // TODO: implement
    };
}

define_rule_mapping!(
'''
for line in checkcode_lines:
    if line.startswith('//'):
        reg += indent + line + '\n'
        continue
    code = line
    reg += indent + code + ' => violations::' + code2kind[code] + ',\n'
reg += ');\n\n'

with open('src/registry.rs', 'w') as f:
    marker = '#[derive'
    f.write(orig_registry_code.replace(marker, reg + marker, 1))

out = '''\
use itertools::Itertools;

use crate::define_violation;
use crate::flake8_debugger::types::DebuggerUsingType;
use crate::flake8_pytest_style::types::{ParametrizeNameType, ParametrizeValuesType, ParametrizeValuesRowType};
use crate::flake8_quotes::settings::Quote;
use crate::flake8_tidy_imports::settings::Strictness;
use crate::pyupgrade::types::Primitive;
use crate::registry::{
    Branch, DeferralKeyword, EqCmpop, IsCmpop, LiteralType, MockReference, UnusedCodes,
};
use crate::violation::{AlwaysAutofixableViolation, Violation};
'''

for line in checkcode_lines:
    if line.startswith('//'):
        out += '\n' + line + '\n\n'
        continue

    code = line
    kind = code2kind[code]
    out += 'define_violation!(' + struct_defs[kind] + ');\n'
    if kind in always_fixable:
        out += f'impl AlwaysAutofixableViolation for {kind} {{\n'
    else:
        out += f'impl Violation for {kind} {{\n'
    out += 'fn message(&self) -> String {'
    out += bodies[kind]
    out += '}'
    if kind in always_fixable:
        out += 'fn autofix_title(&self) -> String {'
        out += autofix_msg[kind]
        out += '}'
    elif kind in sometimes_fixable:
        out += 'fn autofix_title_formatter(&self) -> Option<fn(&Self) -> String> {'
        out += 'todo!()'
        out += '}'
    out += 'fn placeholder() -> Self {'
    out += placeholders[code].replace('CheckKind::', '')
    out += '}'
    out += '}\n\n'

with open('src/violations.rs', 'w') as f:
    f.write(out)

with open('src/lib.rs', 'r') as f:
    mod = f.read()
with open('src/lib.rs', 'w') as f:
    marker = 'mod violation;'
    f.write(mod.replace(marker, marker + '\nmod violations;'))
2023-01-07 15:14:58 -05:00
Martin Fischer
eea1379a74 structs 0/9: Introduce Violation trait
For every available rule registry.rs currently defines:

1. A CheckCode variant to identify the rule.
2. A CheckKind variant to represent violations of the rule.
3. A mapping from the CheckCode variant to a placeholder CheckKind instance.
4. A mapping from the CheckKind variant to CheckCode.
5. A mapping from the CheckKind to a string description.
6. A mapping from the CheckKind to a boolean indicating if autofix is available.
7. A mapping from the CheckKind to a string describing the autofix if available.

Since registry.rs defines all of this for every single rule and
ruff has hundreds of rules, this results in the lines specific to
a particular rule to be hundreds of lines apart, making the code
cumbersome to read and edit.

This commit introduces a new Violation trait so that the rule-specific
details of the above steps 5.-7. can be defined next to the rule
implementation. The idea is that once all CheckCode/CheckKind variants
have been converted to this new approach then the steps 1.-4. in
registry.rs could simply be generated via a declarative macro, e.g:

    define_rule_mapping!(
        E501 => pycodestyle::LineTooLong,
        ...
    );

(where `pycodestyle::LineTooLong` would be a struct that implements the
new Violation trait).

The define_rule_mapping! macro would then take care of generating the
CheckCode and CheckKind enums, as well as all of the implementations for
the previously mentioned mappings, by simply calling the methods of the
new Violation trait.

There is another nice benefit from this approach: We want to introduce
more thorough documentation for rules and we want the documentation of a
rule to be defined close by the implementation (so that it's easier to
check if they match and that they're less likely to become out of sync).
Since these new Violation structs can be defined close by the
implementation of a rule we can also use them as an anchor point for the
documentation of a rule by simply adding the documentation in the form
of a Rust doc comment to the struct.
2023-01-07 15:14:58 -05:00
Anders Kaseorg
dd35e724dd Forbid unsafe code (#1704)
We can reverse this later if it really becomes necessary, but I expect
safe Rust to be sufficient for all our needs.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-01-06 20:25:59 -05:00
messense
1991d618a3 Add proc-macro to derive CheckCodePrefix (#1656)
IMO a derive macro is a natural way to generate new code, and it reduces
the chance of merge conflicts.
2023-01-05 11:39:16 -05:00
Charlie Marsh
5dd9e99a4b Add flake8-pie plugin with prefer_list_builtin (#1578) 2023-01-02 21:47:38 -05:00
Charlie Marsh
8647bec3cb Rename checks.rs to registry.rs (#1566) 2023-01-02 17:26:51 -05:00
Edgar R. M
c4014ef2d3 Implement flake8-pytest-style (#1506) 2023-01-02 16:34:17 -05:00
Charlie Marsh
b078050732 Implicit flake8-implicit-str-concat (#1463) 2022-12-29 23:00:55 -05:00
Charlie Marsh
d22e96916c Automatically detect and respect indentation and quotation code style (#1413) 2022-12-27 19:45:50 -05:00
Charlie Marsh
1392e4cced Default to double quotes in code_gen.rs (#1412) 2022-12-27 16:17:49 -05:00
Charlie Marsh
6a7c3728ee Set convention in flake8-to-ruff (#1410) 2022-12-27 13:51:24 -05:00
Reiner Gerecke
bdb1505262 Web playground with WASM (#1279) 2022-12-26 12:09:17 -05:00
Charlie Marsh
8c018e8261 Add settings validation to lib.rs 2022-12-26 10:12:07 -05:00
Charlie Marsh
b0f30bef8f Add support for ruff.toml (#1378) 2022-12-25 21:55:07 -05:00
Yasu_umi
8a4831dd5b Implement flake8-datetimez (#1270) 2022-12-18 22:06:06 -05:00
Charlie Marsh
5ad77fbc8d Move checkers into their own module (#1268) 2022-12-16 22:55:47 -05:00
Charlie Marsh
9cb18a481b Separate line-based checker from noqa enforcement (#1267) 2022-12-16 22:49:27 -05:00
Charlie Marsh
ecf0dd05d6 Auto-detect same-package imports (#1266) 2022-12-16 21:19:11 -05:00
Charlie Marsh
e4d36bae57 Replace ignore_noqa and autofix booleans with enums (#1264) 2022-12-16 14:01:25 -05:00
Edgar R. M
8281d414ca Implement flake8-errmsg (#1258) 2022-12-15 23:10:59 -05:00
Edgar R. M
27de342e75 Implement pandas-vet (#1235) 2022-12-15 14:01:01 -05:00
Charlie Marsh
549ea2f85f Ignore any pyproject.toml without a [tool.ruff] section (#1243) 2022-12-14 19:35:52 -05:00
Charlie Marsh
9853b0728b Enable configuration files to "extend" other configuration files (#1219) 2022-12-12 20:28:22 -05:00
Charlie Marsh
cd69610741 Use --config everywhere if provided (#1210) 2022-12-12 10:28:00 -05:00
Charlie Marsh
ac6fa1dc88 Simplify some logic around configuration detection (#1197) 2022-12-12 10:15:05 -05:00
Charlie Marsh
0adc9ed259 Support hierarchical settings for nested directories (#1190) 2022-12-12 10:12:23 -05:00
Reiner Gerecke
7f25d1ec70 Implement SIM118 (key in dict) of flake8-simplify (#1195) 2022-12-11 10:05:11 -05:00
Charlie Marsh
bb67fbb73a Implement unused argument detection (ARG) (#1126)
Detect unused arguments
2022-12-07 12:15:41 -05:00
Edgar R. M
ea550abd3c Implement flake8-import-conventions (#1098) 2022-12-06 16:01:17 -05:00
Charlie Marsh
71543eeabc Rename rules mod to ruff (#1104) 2022-12-05 23:35:36 -05:00