730e6b2b4c
## Summary
Previously, `StmtIf` was defined recursively as
```rust
pub struct StmtIf {
pub range: TextRange,
pub test: Box<Expr>,
pub body: Vec<Stmt>,
pub orelse: Vec<Stmt>,
}
```
Every `elif` was represented as an `orelse` with a single `StmtIf`. This
means that this representation couldn't differentiate between
```python
if cond1:
x = 1
else:
if cond2:
x = 2
```
and
```python
if cond1:
x = 1
elif cond2:
x = 2
```
It also makes many checks harder than they need to be because we have to
recurse just to iterate over an entire if-elif-else and because we're
lacking nodes and ranges on the `elif` and `else` branches.
We change the representation to a flat
```rust
pub struct StmtIf {
pub range: TextRange,
pub test: Box<Expr>,
pub body: Vec<Stmt>,
pub elif_else_clauses: Vec<ElifElseClause>,
}
pub struct ElifElseClause {
pub range: TextRange,
pub test: Option<Expr>,
pub body: Vec<Stmt>,
}
```
where `test: Some(_)` represents an `elif` and `test: None` an else.
This representation is different tradeoff, e.g. we need to allocate the
`Vec<ElifElseClause>`, the `elif`s are now different than the `if`s
(which matters in rules where want to check both `if`s and `elif`s) and
the type system doesn't guarantee that the `test: None` else is actually
last. We're also now a bit more inconsistent since all other `else`,
those from `for`, `while` and `try`, still don't have nodes. With the
new representation some things became easier, e.g. finding the `elif`
token (we can use the start of the `ElifElseClause`) and formatting
comments for if-elif-else (no more dangling comments splitting, we only
have to insert the dangling comment after the colon manually and set
`leading_alternate_branch_comments`, everything else is taken of by
having nodes for each branch and the usual placement.rs fixups).
## Merge Plan
This PR requires coordination between the parser repo and the main ruff
repo. I've split the ruff part, into two stacked PRs which have to be
merged together (only the second one fixes all tests), the first for the
formatter to be reviewed by @michareiser and the second for the linter
to be reviewed by @charliermarsh.
* MH: Review and merge
https://github.com/astral-sh/RustPython-Parser/pull/20
* MH: Review and merge or move later in stack
https://github.com/astral-sh/RustPython-Parser/pull/21
* MH: Review and approve
https://github.com/astral-sh/RustPython-Parser/pull/22
* MH: Review and approve formatter PR
https://github.com/astral-sh/ruff/pull/5459
* CM: Review and approve linter PR
https://github.com/astral-sh/ruff/pull/5460
* Merge linter PR in formatter PR, fix ecosystem checks (ecosystem
checks can't run on the formatter PR and won't run on the linter PR, so
we need to merge them first)
* Merge https://github.com/astral-sh/RustPython-Parser/pull/22
* Create tag in the parser, update linter+formatter PR
* Merge linter+formatter PR https://github.com/astral-sh/ruff/pull/5459
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
|
||
|---|---|---|
| .. | ||
| corpus | ||
| fuzz_targets | ||
| .gitignore | ||
| Cargo.toml | ||
| README.md | ||
| init-fuzzer.sh | ||
| reinit-fuzzer.sh | ||
README.md
ruff-fuzz
Fuzzers and associated utilities for automatic testing of Ruff.
Usage
To use the fuzzers provided in this directory, start by invoking:
./fuzz/init-fuzzers.sh
This will install cargo-fuzz and optionally download a
dataset which improves the efficacy of the testing.
This step is necessary for initialising the corpus directory, as all fuzzers share a common
corpus.
The dataset may take several hours to download and clean, so if you're just looking to try out the
fuzzers, skip the dataset download, though be warned that some features simply cannot be tested
without it (very unlikely for the fuzzer to generate valid python code from "thin air").
Once you have initialised the fuzzers, you can then execute any fuzzer with:
cargo fuzz run -s none name_of_fuzzer -- -timeout=1
Users using Apple M1 devices must use a nightly compiler and omit the -s none portion of this
command, as this architecture does not support fuzzing without a sanitizer.
You can view the names of the available fuzzers with cargo fuzz list.
For specific details about how each fuzzer works, please read this document in its entirety.
IMPORTANT: You should run ./reinit-fuzzer.sh after adding more file-based testcases. This will
allow the testing of new features that you've added unit tests for.
Debugging a crash
Once you've found a crash, you'll need to debug it.
The easiest first step in this process is to minimise the input such that the crash is still
triggered with a smaller input.
cargo-fuzz supports this out of the box with:
cargo fuzz tmin -s none name_of_fuzzer artifacts/name_of_fuzzer/crash-...
From here, you will need to analyse the input and potentially the behaviour of the program. The debugging process from here is unfortunately less well-defined, so you will need to apply some expertise here. Happy hunting!
A brief introduction to fuzzers
Fuzzing, or fuzz testing, is the process of providing generated data to a program under test. The most common variety of fuzzers are mutational fuzzers; given a set of existing inputs (a "corpus"), it will attempt to slightly change (or "mutate") these inputs into new inputs that cover parts of the code that haven't yet been observed. Using this strategy, we can quite efficiently generate testcases which cover significant portions of the program, both with expected and unexpected data. This is really quite effective for finding bugs.
The fuzzers here use cargo-fuzz, a utility which allows
Rust to integrate with libFuzzer, the fuzzer library built
into LLVM.
Each source file present in fuzz_targets is a harness, which is, in effect, a unit
test which can handle different inputs.
When an input is provided to a harness, the harness processes this data and libFuzzer observes the
code coverage and any special values used in comparisons over the course of the run.
Special values are preserved for future mutations and inputs which cover new regions of code are
added to the corpus.
Each fuzzer harness in detail
Each fuzzer harness in fuzz_targets targets a different aspect of Ruff and tests
them in different ways. While there is implementation-specific documentation in the source code
itself, each harness is briefly described below.
ruff_parse_simple
This fuzz harness does not perform any "smart" testing of Ruff; it merely checks that the parsing and unparsing of a particular input (what would normally be a source code file) does not crash. It also attempts to verify that the locations of tokens and errors identified do not fall in the middle of a UTF-8 code point, which may cause downstream panics. While this is unlikely to find any issues on its own, it executes very quickly and covers a large and diverse code region that may speed up the generation of inputs and therefore make a more valuable corpus quickly. It is particularly useful if you skip the dataset generation.
ruff_parse_idempotency
This fuzz harness checks that Ruff's parser is idempotent in order to check that it is not
incorrectly parsing or unparsing an input.
It can be built in two modes: default (where it is only checked that the parser does not enter an
unstable state) or full idempotency (the parser is checked to ensure that it will always produce
the same output after the first unparsing).
Full idempotency mode can be used by enabling the full-idempotency feature when running the
fuzzer, but this may be too strict of a restriction for initial testing.
ruff_fix_validity
This fuzz harness checks that fixes applied by Ruff do not introduce new errors using the existing
ruff::test::test_snippet testing utility.
It currently is only configured to use default settings, but may be extended in future versions to
test non-default linter settings.