mirror of https://github.com/astral-sh/ruff
3711f8ad59
## Summary For #6485, I need to be able to use the `SimpleTokenizer` to lex the space between any two adjacent expressions (i.e., the space between a preceding and following node). This requires that we support a wider range of keywords (like `and`, to connect the pieces of `x and y`), and some additional single-character tokens (like `-` and `>`, to support `->`). Note that the `SimpleTokenizer` does not support multi-character tokens, so the `->` in a function signature is lexed as a `-` followed by a `>` -- but this is fine for our purposes. |
||
|---|---|---|
| .. | ||
| snapshots | ||
| cursor.rs | ||
| lib.rs | ||
| textwrap.rs | ||
| tokenizer.rs | ||
| whitespace.rs | ||