Commit Graph

248 Commits

Author SHA1 Message Date
Zanie
126652b684 Fix decorator ranges
Incorrectly merged LALRPOP file
2023-07-17 14:49:14 -05:00
Zanie Blue
a843a00f6b Add parsing of type alias statements i.e. the type keyword (#97)
Extends #95
Closes #82

Adds parsing of new `type` soft keyword for defining type aliases.

Supports type alias statements as defined in PEP 695 e.g.

```python
type IntOrStr = int | str

type ListOrSet[T] = list[T] | set[T]

type AnimalOrVegetable = Animal | "Vegetable"

type RecursiveList[T] = T | list[RecursiveList[T]]
```

All type parameter kinds are supported as in #95.

Builds on soft keyword abstractions introduced in https://github.com/RustPython/RustPython/pull/4519
2023-07-17 14:49:14 -05:00
Zanie Blue
f846f1ea42 Parse type parameters in function definitions (#96)
* Parse type parameters in function definitions
* Add test for combined items
2023-07-17 14:49:14 -05:00
Zanie
1d4b7a395f Consolidate tests and add coverage for trailing comma 2023-07-17 14:49:14 -05:00
Zanie
ce3ce0734b Add bound to test case test_parse_class_with_all_possible_generic_types 2023-07-17 14:49:14 -05:00
Zanie
b0e119f049 Add test for tuple bounds 2023-07-17 14:49:14 -05:00
Zanie
1f5e707829 Remove test for empty generic class Foo[]: ...
Not valid syntax
2023-07-17 14:49:14 -05:00
Zanie
ed7acfe477 Parse type parameters in class definitions 2023-07-17 14:49:14 -05:00
Zanie
c31b58eb39 Move type_param stubs into LALRPOP definition 2023-07-17 14:49:14 -05:00
Zanie
05ae26b935 Regenerate code with latest ASDL 2023-07-17 14:49:14 -05:00
konsti
2d1f69cbb9 Remove asdl (#21)
This removes the ASDL code generation in favor of handwriting the AST. 

The motivations for moving away from the ASDL are:

* CPython compatibility is no longer a goal
* The ASDL grammar isn't as expressive as we would like
* The codegen scripts have a high complexity which makes extensions time
consuming
* We don't make heavy use of code generation (compared to e.g.
RustPython that generates Pyo3 bindings, a fold implementation etc).

We may want to revisit a grammar based code generation in the future,
e.g. by using [ungrammar](https://github.com/rust-analyzer/ungrammar)
2023-07-05 14:25:26 +02:00
Micha Reiser
8078663b6c Remove Range type parameter from AST nodes (#15) 2023-06-23 21:18:55 +01:00
Micha Reiser
f60e204b73 Fix range of keyword identifier (#14) 2023-06-22 10:00:25 +02:00
Micha Reiser
08ebbe40d7 Fix ArgWithDefault TextRange (#13) 2023-06-20 18:21:09 +02:00
Charlie Marsh
ed3b4eb72b Add TextRange to Identifier (#8)
## Summary

This PR adds `TextRange` to `Identifier`. Right now, the AST only
includes ranges for identifiers in certain cases (`Expr::Name`,
`Keyword`, etc.), namely when the identifier comprises an entire AST
node. In Ruff, we do additional ad-hoc lexing to extract identifiers
from source code.

One frequent example: given a function `def f(): ...`, we lex to find
the range of `f`, for use in diagnostics.

Another: `except ValueError as e`, for which the AST doesn't include a
range for `e`.

Note that, as an optimization, we avoid storing the `TextRange` for
`Expr::Name`, since it's already included.
2023-06-20 11:19:27 -04:00
Charlie Marsh
f0d200c8a1 Remove fold, unparse, and location features (#9) 2023-06-19 17:26:17 -04:00
Charlie Marsh
21aa0b8d84 Optimize validate_arguments (#10) 2023-06-19 15:32:58 -04:00
Charlie Marsh
9cb00518e5 Update snapshot tests 2023-06-19 13:16:07 -04:00
Micha Reiser
6f65c5cba7 Add Decorator node (#7) 2023-06-19 12:53:05 -04:00
Micha Reiser
8a415fa61e Include argument parentheses in range (#5) 2023-06-19 12:51:18 -04:00
Micha Reiser
5054cbe84f Merge branch 'RustPython:main' into main 2023-06-19 12:38:24 -04:00
Jeong, YunWon
69d27d924c Rename unconventional nodes (#74) 2023-06-17 01:54:00 +09:00
Steve Shi
a2e3209c42 Replace num-bigint with malachite-bigint (#18)
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-06-02 17:06:18 +09:00
Jeong, YunWon
fdec727f80 New Arguments and Arg/ArgWithDefault AST representation (#59) 2023-06-01 01:15:23 +09:00
Jeong, YunWon
3fbf4f6804 Parse for expr and stmt variants + identifier, constant (#78) 2023-05-31 20:03:46 +09:00
Micha Reiser
342cd19f50 Add safety comment 2023-05-31 09:37:48 +02:00
Micha Reiser
4a2c4aad0b Align MatchCase end location 2023-05-30 17:31:51 +02:00
Jeong, YunWon
4de0cb1827 Parse Trait (#71) 2023-05-28 21:03:27 +09:00
Micha Reiser
5493c9f4e3 Avoid removing elements from the beginning of a vec (#69) 2023-05-25 19:44:49 +09:00
Micha Reiser
41a0ef8740 Add test 2023-05-19 10:14:46 +02:00
Micha Reiser
25cc1da319 Include decorators in Class and FunctionDef range 2023-05-19 08:18:43 +02:00
Micha Reiser
fac0c25343 Update python.rs 2023-05-19 08:08:35 +02:00
Micha Reiser
4ff779c298 Update parser/src/python.lalrpop
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
2023-05-18 22:32:05 +02:00
Micha Reiser
726884b287 Avoid allocating vecs for each statement 2023-05-18 21:56:17 +02:00
Micha Reiser
851f23668f Use named parameters to reducue copying 2023-05-18 21:41:06 +02:00
Jeong, YunWon
6c5c311bab Fix range field order (#56)
* Skip validate_arguments when empty

* Fix `range` field order

* Fix unused variable
2023-05-18 21:44:01 +09:00
Jeong, YunWon
ff17f6e178 Add utilities to enum (#44)
* Add utilities to enum

* Fix unexpected pyo3 dependency propagation
2023-05-16 23:29:49 +09:00
Jeong, YunWon
9d47d3d212 specialize ConversionFlag (#42)
* specialize ConversionFlag

* Change value of ConversionFlag to i8 and None to -1

* is_* methods to ConversionFlag
2023-05-16 22:52:50 +09:00
Jeong, YunWon
0c7d16b61a Add is_* methods to Tok (#39) 2023-05-16 16:08:25 +09:00
Jeong, YunWon
735c06d5f4 Fix full-lexer feature (#38) 2023-05-16 15:45:03 +09:00
Charlie Marsh
10dda125ff Always emit non-logical newlines for 'empty' lines (#27) 2023-05-15 14:13:05 -04:00
Jeong, YunWon
27e3873dc2 Add full-lexer feature (#36) 2023-05-16 02:21:34 +09:00
Micha Reiser
dd4cc25227 Reduce copying elements when parsing (#35) 2023-05-15 23:20:44 +09:00
Micha Reiser
192379cede Move range from Attributed to Nodes (#22)
* Move `range` from `Attributed` to `Node`s

* No Attributed + custom for Range PoC

* Generate all located variants, generate enum implementations

* Implement `Copy` on simple enums

* Move `Suite` to `ranged` and `located`

* Update tests
---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-05-15 15:08:12 +09:00
Micha Reiser
a983f4383f Add format and cformat modules from RustPython (#24)
* Add `format` and `cformat` modules from `RustPython`

* Introduce `rustpython-format` crate

* Remove unused dependencies
2023-05-12 18:27:05 +09:00
Jeong YunWon
cbe4e8c5f3 Make parser location optional 2023-05-11 04:40:10 +09:00
Jeong YunWon
aabc96dde9 ruff integration support 2023-05-11 04:08:57 +09:00
Jeong YunWon
822cac5aa0 parse_expression{=>_starts}_at 2023-05-10 21:25:58 +09:00
Jeong YunWon
aa101e4f26 fix ImportDots 2023-05-10 20:31:04 +09:00
Jeong YunWon
6fa3d0f90a Fine-tune int types 2023-05-10 19:33:39 +09:00