Commit Graph

197 Commits

Author SHA1 Message Date
Jeong YunWon 1d366d52ab Let `located` only for python located stuff 2023-05-10 14:35:38 +09:00
Jeong YunWon a3d9d8cb14 numerous refactoring
- Split parser core and compiler core. Fix #14
- AST int type to `u32`
- Updated asdl_rs.py and update_asdl.sh fix #6
- Use `ruff_python_ast::SourceLocation` for Python source location. Deleted our own Location.
- Renamed ast::Located to ast::Attributed to distinguish terms for TextSize and SourceLocation
- `ast::<Node>`s for TextSize located ast. `ast::located::<Node>` for Python source located ast.
- And also strictly renaming `located` to refer only python location related interfaces.
- `SourceLocator` to convert locations.
- New `source-code` features of to disable python locations when unnecessary.
- Also including fully merging https://github.com/astral-sh/RustPython/pull/4 closes #9
2023-05-10 14:35:38 +09:00
Jeong YunWon 09a6afdd04 Adapt SourceLocation 2023-05-09 20:34:48 +09:00
Jeong YunWon a14e43e03a Separate byteoffset ast and located ast 2023-05-09 00:21:52 +09:00
Jeong YunWon f47dfca4e3 Rename compiler Location to TextSize 2023-05-08 03:38:10 +09:00
Micha Reiser 58c35ab458 Replace row/column based Location with byte-offsets. 2023-05-08 03:38:10 +09:00
Jeong YunWon 6d7358090b Refactor ast to hold data as seperated type 2023-05-07 19:20:47 +09:00
Jeong YunWon 39b2dbe04d Update lalrpop to 0.20.0 2023-05-06 13:48:20 +09:00
Jeong YunWon e1f70100ac Update parser/build.rs to embed python.rs 2023-05-06 05:28:30 +09:00
Micha Reiser ae9d3c3193 Add `Located::start`, `Located::end` and impl `Deref` 2023-04-26 10:24:34 -06:00
Micha Reiser 3873414b30 Use `Located::new` over struct initializer 2023-04-25 18:10:13 -06:00
Jeong YunWon 6d6155413b Update lalrpop 2023-03-25 22:18:02 +09:00
Jeong YunWon f9b5469642 Update cspell for compiler 2023-03-16 22:39:09 +09:00
Charlie Marsh ef38eb6b1a Include Derive feature with optional Serde dependency 2023-03-12 14:46:10 -04:00
Charlie Marsh 55fc0e83f3 Treat match and case as soft keywords in lambda assignments (#4623) 2023-03-04 12:42:05 -05:00
Charlie Marsh 713dd2b91e Add optional serde dependency 2023-03-02 15:38:40 -05:00
Noa b80bbec8e6 Custom marshal enc/decoding impl 2023-03-01 20:47:21 -06:00
DimitrisJim 8f425e9ce2 Use insta to verify values. 2023-02-28 20:00:47 +09:00
Jeong YunWon d7e2e7361e Use git version of unicode_names2 to avoid alias search failure 2023-02-28 20:00:47 +09:00
Charlie Marsh c37e0c7f03 Use proper locations for sub-expressions in constant matches 2023-02-26 23:10:38 -05:00
Charlie Marsh 0d7b94817d Allow type variable tuple for *args 2023-02-23 08:39:59 -05:00
Jeong YunWon f43e5b72e2 Merge pull request #4552 from charliermarsh/charlie/loc
Limit match range to end of last statement
2023-02-23 02:11:42 +09:00
Jeong YunWon a19f294b0c Merge pull request #4543 from youknowone/flatten-parser
Flatten parser interface
2023-02-23 02:09:46 +09:00
Charlie Marsh 7ebef61c47 Limit match range to end of last statement 2023-02-22 11:25:50 -05:00
Charlie Marsh b61f4d7b69 Allow trailing commas in MappingPattern 2023-02-22 10:02:41 -05:00
Jeong YunWon e26369a34e use super::* from tests submodules 2023-02-22 21:01:39 +09:00
Jeong YunWon 97a08ee77b remove #[macro_use] 2023-02-22 20:41:27 +09:00
Jeong YunWon cb8c6fb78d Flatten rustpython_parser interface 2023-02-22 20:32:31 +09:00
Jeong YunWon 8580e4ebb5 make_tokenizer -> lex to integrate terms
we don't distinguish scanner or tokenizer from lexer
2023-02-22 20:28:15 +09:00
Jeong YunWon 66e3080173 Fix ModeParseError message 2023-02-22 20:28:03 +09:00
Jeong YunWon 1511b6631b Break down rustpython_parser::error module
because it doesn't share any common errors but specific error for each sub module
2023-02-22 20:28:01 +09:00
Charlie Marsh 2a8aa6f308 Always wrap in SoftKeywordTransformer 2023-02-21 19:18:42 -05:00
Charlie Marsh dc628cab8f Expose SoftKeywordTransformer on public API 2023-02-21 19:00:32 -05:00
Jim Fasarakis-Hilliard c137bc9d77 Merge pull request #4519 from charliermarsh/charlie/match
Add support for match statements to parser
2023-02-21 19:43:28 +02:00
Jeong YunWon 60180fd54c Merge pull request #4531 from charliermarsh/charlie/exception-groups
Implement except* syntax
2023-02-21 13:20:18 +09:00
Charlie Marsh c7ed645cc6 Implement except* syntax 2023-02-21 12:19:54 +09:00
Charlie Marsh 8aa3bc93f3 Allow starred expressions in subscripts 2023-02-20 17:59:35 -05:00
Charlie Marsh f39ffef370 Update compiler/parser/src/soft_keywords.rs
Co-authored-by: Jim Fasarakis-Hilliard <d.f.hilliard@gmail.com>
2023-02-20 15:03:39 -05:00
Jeong YunWon e093d2bee6 clean up soft-keyword transform 2023-02-20 15:03:39 -05:00
Charlie Marsh ca5b474d45 Use muiltipeek 2023-02-20 15:03:39 -05:00
Charlie Marsh 2b43d45bd5 Add support for match statements to parser 2023-02-20 15:03:39 -05:00
Jeong YunWon 09b82e41d5 Merge pull request #4490 from DimitrisJim/function_parser
Add tests, some comments, to function.rs.
2023-02-13 20:00:35 +09:00
Jeong YunWon cc6d8a1c58 Merge pull request #4492 from DimitrisJim/doc_parser_uno
Document parser crate.
2023-02-13 17:25:14 +09:00
Dimitris Fasarakis Hilliard 07918f0a9a Document parser crate. 2023-02-12 17:58:19 +02:00
Dimitris Fasarakis Hilliard a0786ea872 Add tests, some comments, to function.rs. 2023-02-11 23:07:57 +02:00
Dimitris Fasarakis Hilliard 4713b2b3ab Refactor: Join string and string_parser. 2023-02-11 18:05:06 +02:00
Jeong YunWon e7f14ab9b8 Add test_generator_expression_argument 2023-02-11 05:20:39 +09:00
Charlie Marsh 56c73cc63d Use entire range for generators-as-arguments 2023-02-10 10:39:40 -05:00
Dimitris Fasarakis Hilliard 659f4dd8bf Document lexer. 2023-02-07 21:43:57 +02:00
Dimitris Fasarakis Hilliard bd158089e0 Move NewLineHandler inline, don't check each character twice. 2023-02-07 20:58:53 +02:00