Commit Graph

144 Commits

Author SHA1 Message Date
Dimitris Fasarakis Hilliard 38cf933bcb Add initial capacities, use u32s for indents/spaces. 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard 838990ae15 Don't call is_emoji_presentation for each invocation of consume_normal 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard aa0290bbfc Match on ascii start/continuation characters before calling functions. 2023-01-31 12:26:05 +02:00
Aarni Koskela f74e44d1e8 Bump phf to 0.11 series
string_cache is still using phf_shared 0.10.0 though
2023-01-25 19:58:43 +02:00
Jeong YunWon d9df131720 Merge pull request #4449 from harupy/fix-dict-spread-in-dict
Fix AST generated from a dict literal containing dict unpacking
2023-01-22 20:44:26 +09:00
Anders Kaseorg 6dba8430be Fix end location for elif blocks
Since we parse an `elif:` block as an `If` node, its location should
include its `orelse` node like it would for an `if:` block.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-01-17 22:39:44 -05:00
harupy b26365b215 Remove useless String::from
Signed-off-by: harupy <hkawamura0130@gmail.com>
2023-01-16 21:27:57 +09:00
harupy 2d019930e9 Rename test 2023-01-15 23:36:07 +09:00
harupy d5fc7c4c87 Improve test 2023-01-15 16:53:13 +09:00
harupy 4edd2bf78a Remove commented-out code 2023-01-15 16:10:51 +09:00
harupy 581f6e176c Fix dict spreading in dict literal 2023-01-15 13:01:59 +09:00
Ran Benita e5fe037e38 Fixup parse_tokens after "Add NonLogicalNewline token"
I only updated `parse()` to ignore `NonLogicalNewline`, didn't notice
it's also needed in `parse_tokens()`.
2023-01-14 11:52:33 +02:00
Ran Benita 674eeec29c Add NonLogicalNewline token
This token is completely ignored by the parser, but it's useful for
other users of the lexer, such as the Ruff linter. For example, the
token is helpful for a "trailing comma" lint.

The same idea exists in Python's `tokenize` module - there is a NEWLINE
token (logical newline), and a NL token (non-logical newline).

Fixes #4385.
2023-01-12 16:47:12 +02:00
Martin Fischer 4f1e7c6291 Fix docs.rs build for rustpython-parser
docs.rs failed to build the documentation of the recently released
rustpython-parser 0.2.0 because the build.rs script couldn't write the
parser.rs file because docs.rs builds the documentation in a sandbox
with a read-only filesystem.

This commit fixes this by writing the parser.rs file to the cargo output
directory instead, as recommended by the docs.rs documentation.[1]

Fixes #4436.

[1]: https://docs.rs/about/builds#read-only-directories
2023-01-11 09:58:10 +01:00
Noa 884a7bdb15 Bump all crate versions to 0.2.0 2023-01-11 00:14:28 -06:00
Jeong YunWon 7885344bcf first cspell dict 2023-01-09 19:57:23 +09:00
harupy a4a5366504 Include comment text in token 2023-01-06 23:29:20 +09:00
harupy 2dfd053bed Implement Default for Location 2023-01-05 22:48:47 +09:00
harupy 7f552e4594 Address comments 2023-01-05 18:24:54 +09:00
harupy 9efa872023 Use try_from 2023-01-05 01:18:30 +09:00
harupy fd8468c5eb Simplify string check 2023-01-04 23:57:50 +09:00
Jeong YunWon 958c7e33ad Merge pull request #4417 from harupy/add-with-offset-methods
Add `with_col_offset` and `with_row_offset` to `Location` for conveniece
2023-01-04 17:07:31 +09:00
Jeong YunWon 0f311cd5e5 Merge pull request #4413 from harupy/more-generic-window-impl
Update `CharWindow` in `compiler/parser/src/lexer.rs` to allow slicing
2023-01-04 17:06:06 +09:00
harupy 6d140426c1 Add with_col_offset and with_row_offset to Location 2023-01-04 13:16:03 +09:00
harupy 33a62789f7 Address comment 2023-01-04 00:03:18 +09:00
harupy 84dff79ddc Remove incorrect EmptyExpression in parse_formatted_value 2023-01-03 23:55:50 +09:00
harupy 300710f7db Improve CharWindow 2023-01-03 17:27:35 +09:00
harupy 515dceb07b Remove repetitive to_string in parse_escaped_char 2023-01-03 14:48:00 +09:00
Jeong YunWon 2858c315bf Merge pull request #4409 from harupy/improve-error-conversion-in-string-parser
Improve error conversion in `string_parsers.rs`
2023-01-03 14:39:01 +09:00
harupy 3b0fd61b3b Fix clippy error 2023-01-03 12:51:31 +09:00
harupy 9030679193 Improve error conversion in string_parsers.rs 2023-01-03 12:46:10 +09:00
Jeong YunWon 71ba4226c1 Merge pull request #4405 from harupy/use-drain
Use `drain` to simplify `compiler/parser/src/string_parser.rs`
2023-01-03 12:41:23 +09:00
Jeong YunWon 3d03439618 Merge pull request #4399 from branai/shell-continuing-fix
Fix IndentationError works differently with cpython in interective shell
2023-01-03 04:32:03 +09:00
harupy 39f410c909 Use drain 2023-01-03 01:23:44 +09:00
Jim Fasarakis-Hilliard 4222b13e6c Merge pull request #4404 from harupy/merge-match-arms
Merge match arms in `StringParser.parse_formatted_value`
2023-01-02 18:18:40 +02:00
Jim Fasarakis-Hilliard d691527ead Merge pull request #4402 from harupy/remove-unreachable-if
Remove unreachable code in `compiler/parser/src/string_parser.rs`
2023-01-02 17:05:45 +02:00
harupy 40e2150f7e Merge match arms in parse_formatted_value 2023-01-02 23:16:51 +09:00
harupy 92bf96608c Fix match 2023-01-02 22:54:48 +09:00
harupy fac6a857f6 Simplify code using match 2023-01-02 22:26:09 +09:00
harupy 68586f8e3c Remove unreachable code in compiler/parser/src/string_parser.rs 2023-01-02 20:48:40 +09:00
Jeong YunWon f8787a9377 Move (c)format basic implementations to rustpython-common 2023-01-02 20:21:36 +09:00
Bijan Naimi 530a20cc96 forgot to add formatted errors.rs 2023-01-01 17:28:49 -08:00
Bijan Naimi 6cb57b2075 changed the shell logic for handling indents 2023-01-01 15:41:51 -08:00
Dimitris Fasarakis Hilliard 92b2574d52 Move tests for with into parser. 2023-01-01 21:36:07 +02:00
Jim Fasarakis-Hilliard 95fb938bd6 Merge pull request #4389 from harupy/4384-follow-up
Follow-up for #4384
2023-01-01 14:53:54 +02:00
harupy 9683314264 Remove unreachable code 2023-01-01 17:43:25 +09:00
harupy d9bbeeb9b3 Fix NamedExpr location 2022-12-31 23:32:08 +09:00
harupy 68116d5c11 Move tests 2022-12-31 12:15:33 +09:00
Jeong YunWon e164a41723 Merge pull request #4373 from andersk/named
Allow named expression in subscript and set comprehension
2022-12-31 10:58:09 +09:00
Jeong YunWon aa32a73c5b Merge pull request #4379 from harupy/refactor-FStringParser
Refactor `FStringParser`
2022-12-31 10:56:52 +09:00