mirror of https://github.com/astral-sh/ruff
68f605e80a
## Summary This PR attempts to address a problem in the parser related to the range's of `WithItem` nodes in certain contexts -- specifically, `WithItem` nodes in parentheses that do not have an `as` token after them. For example, [here](https://play.ruff.rs/71be2d0b-2a04-4c7e-9082-e72bff152679): ```python with (a, b): pass ``` The range of the `WithItem` `a` is set to the range of `(a, b)`, as is the range of the `WithItem` `b`. In other words, when we have this kind of sequence, we use the range of the entire parenthesized context, rather than the ranges of the items themselves. Note that this also applies to cases [like](https://play.ruff.rs/c551e8e9-c3db-4b74-8cc6-7c4e3bf3713a): ```python with (a, b, c as d): pass ``` You can see the issue in the parser here: ```rust #[inline] WithItemsNoAs: Vec<ast::WithItem> = { <location:@L> <all:OneOrMore<Test<"all">>> <end_location:@R> => { all.into_iter().map(|context_expr| ast::WithItem { context_expr, optional_vars: None, range: (location..end_location).into() }).collect() }, } ``` Fixing this issue is... very tricky. The naive approach is to use the range of the `context_expr` as the range for the `WithItem`, but that range will be incorrect when the `context_expr` is itself parenthesized. For example, _that_ solution would fail here, since the range of the first `WithItem` would be that of `a`, rather than `(a)`: ```python with ((a), b): pass ``` The `with` parsing in general is highly precarious due to ambiguities in the grammar. Changing it in _any_ way seems to lead to an ambiguous grammar that LALRPOP fails to translate. Consensus seems to be that we don't really understand _why_ the current grammar works (i.e., _how_ it avoids these ambiguities as-is). The solution implemented here is to avoid changing the grammar itself, and instead change the shape of the nodes returned by various rules in the grammar. Specifically, everywhere that we return `Expr`, we instead return `ParenthesizedExpr`, which includes a parenthesized range and the underlying `Expr` itself. (If an `Expr` isn't parenthesized, the ranges will be equivalent.) In `WithItemsNoAs`, we can then use the parenthesized range as the range for the `WithItem`. |
||
|---|---|---|
| .. | ||
| flake8_to_ruff | ||
| ruff | ||
| ruff_benchmark | ||
| ruff_cache | ||
| ruff_cli | ||
| ruff_dev | ||
| ruff_diagnostics | ||
| ruff_formatter | ||
| ruff_index | ||
| ruff_macros | ||
| ruff_python_ast | ||
| ruff_python_codegen | ||
| ruff_python_formatter | ||
| ruff_python_index | ||
| ruff_python_literal | ||
| ruff_python_parser | ||
| ruff_python_resolver | ||
| ruff_python_semantic | ||
| ruff_python_stdlib | ||
| ruff_python_trivia | ||
| ruff_shrinking | ||
| ruff_source_file | ||
| ruff_text_size | ||
| ruff_wasm | ||
| ruff_workspace | ||