Commit Graph

19 Commits

Author SHA1 Message Date
harupy 581f6e176c Fix dict spreading in dict literal 2023-01-15 13:01:59 +09:00
Ran Benita e5fe037e38 Fixup parse_tokens after "Add NonLogicalNewline token"
I only updated `parse()` to ignore `NonLogicalNewline`, didn't notice
it's also needed in `parse_tokens()`.
2023-01-14 11:52:33 +02:00
Ran Benita 674eeec29c Add NonLogicalNewline token
This token is completely ignored by the parser, but it's useful for
other users of the lexer, such as the Ruff linter. For example, the
token is helpful for a "trailing comma" lint.

The same idea exists in Python's `tokenize` module - there is a NEWLINE
token (logical newline), and a NL token (non-logical newline).

Fixes #4385.
2023-01-12 16:47:12 +02:00
harupy a4a5366504 Include comment text in token 2023-01-06 23:29:20 +09:00
Dimitris Fasarakis Hilliard 92b2574d52 Move tests for with into parser. 2023-01-01 21:36:07 +02:00
harupy 439298e735 Fix FormattedValue location 2022-12-30 21:39:29 +09:00
harupy b2ac4f60f1 Fix slice location 2022-12-25 09:37:07 +09:00
harupy 1b7a272b77 Fix end location of nodes containing body 2022-12-11 12:35:28 +09:00
harupy 6f6b7b2312 add tests 2022-12-04 05:59:51 +09:00
Charlie Marsh b6c230f3ca Implement Tok::Comment 2022-11-07 10:33:55 -05:00
dvermd a5b59f3c9d improve col_offset in new line and lalr 2022-10-26 21:30:51 +02:00
dvermd fa41a1e2f6 Fix ast types' _fields and use 0-based column 2022-10-27 03:09:38 +09:00
Charlie Marsh 02953b9fe6 Remove parse_program_tokens 2022-10-17 12:04:30 -04:00
Charlie Marsh 8adc74fe26 Expose a method to parse AST from tokens directly 2022-10-17 09:39:48 -04:00
Charlie Marsh 1cc342e4ed Add end locations to all nodes (#4192) 2022-10-17 13:18:30 +09:00
Jeong YunWon 1fd898c14c new_syntax_error working without full compiler
Fix #4100
2022-08-23 05:08:24 +09:00
Jeong YunWon 904fc477f1 integrate ast::Location into compilre-core::Location 2022-08-22 08:42:20 +09:00
Jeong YunWon 3742f9117b Add source_path to ParseError 2022-08-22 08:42:20 +09:00
Jeong YunWon 53c48bf6b9 reorganize compiler crates 2022-08-22 04:42:42 +09:00