Commit Graph

78 Commits

Author SHA1 Message Date
Micha Reiser 08ebbe40d7
Fix ArgWithDefault TextRange (#13) 2023-06-20 18:21:09 +02:00
Charlie Marsh ed3b4eb72b
Add `TextRange` to `Identifier` (#8)
## Summary

This PR adds `TextRange` to `Identifier`. Right now, the AST only
includes ranges for identifiers in certain cases (`Expr::Name`,
`Keyword`, etc.), namely when the identifier comprises an entire AST
node. In Ruff, we do additional ad-hoc lexing to extract identifiers
from source code.

One frequent example: given a function `def f(): ...`, we lex to find
the range of `f`, for use in diagnostics.

Another: `except ValueError as e`, for which the AST doesn't include a
range for `e`.

Note that, as an optimization, we avoid storing the `TextRange` for
`Expr::Name`, since it's already included.
2023-06-20 11:19:27 -04:00
Charlie Marsh f0d200c8a1
Remove `fold`, `unparse`, and `location` features (#9) 2023-06-19 17:26:17 -04:00
Micha Reiser 6f65c5cba7 Add `Decorator` node (#7) 2023-06-19 12:53:05 -04:00
Jeong, YunWon 69d27d924c
Rename unconventional nodes (#74) 2023-06-17 01:54:00 +09:00
Steve Shi a2e3209c42
Replace num-bigint with malachite-bigint (#18)
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-06-02 17:06:18 +09:00
Jeong, YunWon 5e9e8a7589
Linear Locator (#46) 2023-06-01 13:53:31 +09:00
Jeong, YunWon fdec727f80
New Arguments and Arg/ArgWithDefault AST representation (#59) 2023-06-01 01:15:23 +09:00
Jeong, YunWon 3fbf4f6804
Parse for expr and stmt variants + identifier, constant (#78) 2023-05-31 20:03:46 +09:00
Jeong, YunWon 531aeb3511
Cmpop::as_str (#72)
* clean up pyo3 generation

* Cmpop::as_str
2023-05-29 01:53:54 +09:00
Jeong YunWon 7d384d88d0 Separate rustpython_ast_pyo3 2023-05-23 02:50:12 +09:00
Jeong YunWon b81273e9bc to_pyo3_ast to return &'py 2023-05-23 00:12:22 +09:00
Jeong, YunWon e1f02fced7
Fix Vec::to_pyo3_ast (#63) 2023-05-22 15:15:05 +09:00
Jeong, YunWon d4084fb17a
impl From<T> for ast::Ast (#62) 2023-05-21 22:44:07 +09:00
Jeong, YunWon 3654cf0bdf
Add `Ast` as top level enum (#58) 2023-05-19 01:53:39 +09:00
Jeong, YunWon 6c5c311bab
Fix range field order (#56)
* Skip validate_arguments when empty

* Fix `range` field order

* Fix unused variable
2023-05-18 21:44:01 +09:00
Jeong, YunWon 531e41ae2c
Upgrade bitflags to remove clippy wanrings (#54)
* Upgrade bitflags to remove clippy wanrings

* Fix python lint warnings
2023-05-18 01:50:00 +09:00
Jeong, YunWon b48834fe2d
More flexible map_user and fold for new constructor nodes (#53)
* make fold.rs file

* Split user_map steps

* Fold for new constructor nodes
2023-05-18 00:16:04 +09:00
Jeong, YunWon fc301ab1b0
Fix build (#45)
* remove dedent

* Revert parser-pyo3 crate
2023-05-16 23:54:49 +09:00
Jeong, YunWon e820928f11
Add experimental pyo3-wrapper feature (#41)
* Fix pyo3 unit type value error

* Add experimental pyo3-wrapper feature

* location support
2023-05-16 23:45:31 +09:00
Jeong, YunWon ff17f6e178
Add utilities to enum (#44)
* Add utilities to enum

* Fix unexpected pyo3 dependency propagation
2023-05-16 23:29:49 +09:00
Jeong, YunWon 9d47d3d212
specialize ConversionFlag (#42)
* specialize ConversionFlag

* Change value of ConversionFlag to i8 and None to -1

* is_* methods to ConversionFlag
2023-05-16 22:52:50 +09:00
Jeong, YunWon 611dcc2e9b
rustpython_ast + pyo3 (#25) 2023-05-16 18:06:54 +09:00
Jeong, YunWon 53de75efc3
Add all node classes to ast module (#40) 2023-05-16 16:55:26 +09:00
Jeong, YunWon 02f13abf50
Add Node trait for node type information (#31) 2023-05-16 15:41:30 +09:00
Jeong, YunWon b78001c953
Generic types to generic (#30) 2023-05-15 20:22:42 +09:00
Jeong, YunWon 718354673e
Move range to node (#23)
* black + clippy

* Fix module generation
2023-05-15 16:20:22 +09:00
Micha Reiser 192379cede
Move `range` from `Attributed` to `Node`s (#22)
* Move `range` from `Attributed` to `Node`s

* No Attributed + custom for Range PoC

* Generate all located variants, generate enum implementations

* Implement `Copy` on simple enums

* Move `Suite` to `ranged` and `located`

* Update tests
---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-05-15 15:08:12 +09:00
Micha Reiser a983f4383f
Add `format` and `cformat` modules from `RustPython` (#24)
* Add `format` and `cformat` modules from `RustPython`

* Introduce `rustpython-format` crate

* Remove unused dependencies
2023-05-12 18:27:05 +09:00
Jeong, YunWon 947fb53d0b
Field accessor and utilities (#20)
* Apply is-macro to Constant and ast nodes
2023-05-11 19:44:20 +09:00
Jeong YunWon 2af9805662 A few more Identifier utilities 2023-05-11 16:05:11 +09:00
Jeong YunWon cbe4e8c5f3 Make parser location optional 2023-05-11 04:40:10 +09:00
Jeong YunWon aabc96dde9 ruff integration support 2023-05-11 04:08:57 +09:00
Jeong YunWon d6b6df5d1c ast::Int::to_usize 2023-05-10 21:31:02 +09:00
Jeong YunWon 6fa3d0f90a Fine-tune int types 2023-05-10 19:33:39 +09:00
Jeong YunWon 455bcc01a0 Give identifier and int ast types 2023-05-10 19:33:39 +09:00
Jeong YunWon d8822d1091 spell check ast/asdl_rs.py 2023-05-10 18:05:39 +09:00
Jeong YunWon 75f6ce1ae5 Setup spell checker 2023-05-10 17:57:15 +09:00
Jeong YunWon 17c8abcec1 asdl_rs.py to multiple file output 2023-05-10 17:41:01 +09:00
Jeong YunWon e000b1c304 Remove redundant types 2023-05-10 17:15:01 +09:00
Jeong YunWon 243ca16b34 Fix visitor to fit in new structure 2023-05-10 17:14:44 +09:00
Joshua 4de9580b92 Generate a visitor trait to ast_gen.rs 2023-05-10 17:14:34 +09:00
Jeong YunWon 1d366d52ab Let `located` only for python located stuff 2023-05-10 14:35:38 +09:00
Jeong YunWon a3d9d8cb14 numerous refactoring
- Split parser core and compiler core. Fix #14
- AST int type to `u32`
- Updated asdl_rs.py and update_asdl.sh fix #6
- Use `ruff_python_ast::SourceLocation` for Python source location. Deleted our own Location.
- Renamed ast::Located to ast::Attributed to distinguish terms for TextSize and SourceLocation
- `ast::<Node>`s for TextSize located ast. `ast::located::<Node>` for Python source located ast.
- And also strictly renaming `located` to refer only python location related interfaces.
- `SourceLocator` to convert locations.
- New `source-code` features of to disable python locations when unnecessary.
- Also including fully merging https://github.com/astral-sh/RustPython/pull/4 closes #9
2023-05-10 14:35:38 +09:00
Jeong YunWon 09a6afdd04 Adapt SourceLocation 2023-05-09 20:34:48 +09:00
Jeong YunWon a14e43e03a Separate byteoffset ast and located ast 2023-05-09 00:21:52 +09:00
Jeong YunWon f47dfca4e3 Rename compiler Location to TextSize 2023-05-08 03:38:10 +09:00
Micha Reiser 58c35ab458 Replace row/column based Location with byte-offsets. 2023-05-08 03:38:10 +09:00
Jeong YunWon 6d7358090b Refactor ast to hold data as seperated type 2023-05-07 19:20:47 +09:00
Jeong YunWon a73bac9ed1 Add python lint 2023-05-06 14:34:31 +09:00