Commit Graph

5 Commits

Author SHA1 Message Date
Micha Reiser 40f54375cb
Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
Micha Reiser 2cf00fee96
Remove parser dependency from ruff-python-ast (#6096) 2023-07-26 17:47:22 +02:00
Micha Reiser d351761f5d
`SimpleTokenizer`: Fix infinite loop when lexing empty quotes (#5917) 2023-07-20 15:18:35 +02:00
Micha Reiser 76e9ce6dc0
Fix `SimpleTokenizer`'s backward lexing of `# ` (#5878) 2023-07-20 11:54:18 +02:00
Charlie Marsh 5f3da9955a
Rename `ruff_python_whitespace` to `ruff_python_trivia` (#5886)
## Summary

This crate now contains utilities for dealing with trivia more broadly:
whitespace, newlines, "simple" trivia lexing, etc. So renaming it to
reflect its increased responsibilities.

To avoid conflicts, I've also renamed `Token` and `TokenKind` to
`SimpleToken` and `SimpleTokenKind`.
2023-07-19 11:48:27 -04:00