Commit Graph

190 Commits

Author SHA1 Message Date
Jeong YunWon d9df131720 Merge pull request #4449 from harupy/fix-dict-spread-in-dict
Fix AST generated from a dict literal containing dict unpacking
2023-01-22 20:44:26 +09:00
Anders Kaseorg 6dba8430be Fix end location for elif blocks
Since we parse an `elif:` block as an `If` node, its location should
include its `orelse` node like it would for an `if:` block.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-01-17 22:39:44 -05:00
harupy b26365b215 Remove useless String::from
Signed-off-by: harupy <hkawamura0130@gmail.com>
2023-01-16 21:27:57 +09:00
harupy 2d019930e9 Rename test 2023-01-15 23:36:07 +09:00
harupy d5fc7c4c87 Improve test 2023-01-15 16:53:13 +09:00
harupy 4edd2bf78a Remove commented-out code 2023-01-15 16:10:51 +09:00
harupy 581f6e176c Fix dict spreading in dict literal 2023-01-15 13:01:59 +09:00
Ran Benita e5fe037e38 Fixup parse_tokens after "Add NonLogicalNewline token"
I only updated `parse()` to ignore `NonLogicalNewline`, didn't notice
it's also needed in `parse_tokens()`.
2023-01-14 11:52:33 +02:00
Ran Benita 674eeec29c Add NonLogicalNewline token
This token is completely ignored by the parser, but it's useful for
other users of the lexer, such as the Ruff linter. For example, the
token is helpful for a "trailing comma" lint.

The same idea exists in Python's `tokenize` module - there is a NEWLINE
token (logical newline), and a NL token (non-logical newline).

Fixes #4385.
2023-01-12 16:47:12 +02:00
Martin Fischer 4f1e7c6291 Fix docs.rs build for rustpython-parser
docs.rs failed to build the documentation of the recently released
rustpython-parser 0.2.0 because the build.rs script couldn't write the
parser.rs file because docs.rs builds the documentation in a sandbox
with a read-only filesystem.

This commit fixes this by writing the parser.rs file to the cargo output
directory instead, as recommended by the docs.rs documentation.[1]

Fixes #4436.

[1]: https://docs.rs/about/builds#read-only-directories
2023-01-11 09:58:10 +01:00
Noa 884a7bdb15 Bump all crate versions to 0.2.0 2023-01-11 00:14:28 -06:00
Jeong YunWon 7885344bcf first cspell dict 2023-01-09 19:57:23 +09:00
harupy a4a5366504 Include comment text in token 2023-01-06 23:29:20 +09:00
harupy 2dfd053bed Implement Default for Location 2023-01-05 22:48:47 +09:00
harupy 7f552e4594 Address comments 2023-01-05 18:24:54 +09:00
harupy 9efa872023 Use try_from 2023-01-05 01:18:30 +09:00
harupy fd8468c5eb Simplify string check 2023-01-04 23:57:50 +09:00
Jeong YunWon 958c7e33ad Merge pull request #4417 from harupy/add-with-offset-methods
Add `with_col_offset` and `with_row_offset` to `Location` for conveniece
2023-01-04 17:07:31 +09:00
Jeong YunWon 0f311cd5e5 Merge pull request #4413 from harupy/more-generic-window-impl
Update `CharWindow` in `compiler/parser/src/lexer.rs` to allow slicing
2023-01-04 17:06:06 +09:00
harupy 6d140426c1 Add with_col_offset and with_row_offset to Location 2023-01-04 13:16:03 +09:00
harupy 33a62789f7 Address comment 2023-01-04 00:03:18 +09:00
harupy 84dff79ddc Remove incorrect EmptyExpression in parse_formatted_value 2023-01-03 23:55:50 +09:00
harupy 300710f7db Improve CharWindow 2023-01-03 17:27:35 +09:00
harupy 515dceb07b Remove repetitive to_string in parse_escaped_char 2023-01-03 14:48:00 +09:00
Jeong YunWon 2858c315bf Merge pull request #4409 from harupy/improve-error-conversion-in-string-parser
Improve error conversion in `string_parsers.rs`
2023-01-03 14:39:01 +09:00
harupy 3b0fd61b3b Fix clippy error 2023-01-03 12:51:31 +09:00
harupy 9030679193 Improve error conversion in string_parsers.rs 2023-01-03 12:46:10 +09:00
Jeong YunWon 71ba4226c1 Merge pull request #4405 from harupy/use-drain
Use `drain` to simplify `compiler/parser/src/string_parser.rs`
2023-01-03 12:41:23 +09:00
Jeong YunWon 3d03439618 Merge pull request #4399 from branai/shell-continuing-fix
Fix IndentationError works differently with cpython in interective shell
2023-01-03 04:32:03 +09:00
harupy 39f410c909 Use drain 2023-01-03 01:23:44 +09:00
Jim Fasarakis-Hilliard 4222b13e6c Merge pull request #4404 from harupy/merge-match-arms
Merge match arms in `StringParser.parse_formatted_value`
2023-01-02 18:18:40 +02:00
Jim Fasarakis-Hilliard d691527ead Merge pull request #4402 from harupy/remove-unreachable-if
Remove unreachable code in `compiler/parser/src/string_parser.rs`
2023-01-02 17:05:45 +02:00
harupy 40e2150f7e Merge match arms in parse_formatted_value 2023-01-02 23:16:51 +09:00
harupy 92bf96608c Fix match 2023-01-02 22:54:48 +09:00
harupy fac6a857f6 Simplify code using match 2023-01-02 22:26:09 +09:00
harupy 68586f8e3c Remove unreachable code in compiler/parser/src/string_parser.rs 2023-01-02 20:48:40 +09:00
Jeong YunWon f8787a9377 Move (c)format basic implementations to rustpython-common 2023-01-02 20:21:36 +09:00
Bijan Naimi 530a20cc96 forgot to add formatted errors.rs 2023-01-01 17:28:49 -08:00
Bijan Naimi 6cb57b2075 changed the shell logic for handling indents 2023-01-01 15:41:51 -08:00
Dimitris Fasarakis Hilliard 92b2574d52 Move tests for with into parser. 2023-01-01 21:36:07 +02:00
Jim Fasarakis-Hilliard 95fb938bd6 Merge pull request #4389 from harupy/4384-follow-up
Follow-up for #4384
2023-01-01 14:53:54 +02:00
harupy 9683314264 Remove unreachable code 2023-01-01 17:43:25 +09:00
harupy d9bbeeb9b3 Fix NamedExpr location 2022-12-31 23:32:08 +09:00
harupy 68116d5c11 Move tests 2022-12-31 12:15:33 +09:00
Jeong YunWon e164a41723 Merge pull request #4373 from andersk/named
Allow named expression in subscript and set comprehension
2022-12-31 10:58:09 +09:00
Jeong YunWon aa32a73c5b Merge pull request #4379 from harupy/refactor-FStringParser
Refactor `FStringParser`
2022-12-31 10:56:52 +09:00
Jeong YunWon 9d7d629cef Merge pull request #4384 from harupy/parse-formatted-value
Fix the location of `FormattedValue`
2022-12-31 10:52:44 +09:00
harupy 439298e735 Fix FormattedValue location 2022-12-30 21:39:29 +09:00
anilbey 16173bf581 Update compiler/parser/src/error.rs
Co-authored-by: fanninpm <fanninpm@miamioh.edu>
2022-12-29 22:56:34 +01:00
Anil Tuncel 7e7b2eadee format using cargo fmt 2022-12-29 22:15:44 +01:00
Anil Tuncel 80116c768d arg name to be written upon duplicate kwargs error #4381 2022-12-29 22:06:41 +01:00
Nick Liu 200390c1ab format code 2022-12-29 22:49:26 +08:00
Nick Liu 24d2ab8b0a use is_none 2022-12-29 22:49:26 +08:00
Nick Liu 41f21a7b5d add arg_name in duplicate argument error msg 2022-12-29 22:49:26 +08:00
Nick Liu 63e4a36e27 added check: named arguments must follow bare star 2022-12-29 22:49:26 +08:00
Nick Liu 15ab44384c added lex error: DuplicateArguments 2022-12-29 22:49:26 +08:00
Jim Fasarakis-Hilliard 5380b85579 Merge pull request #4367 from andersk/star-order
Prohibit starred arguments after double-starred arguments
2022-12-29 16:27:16 +02:00
Harutaka Kawamura b707f53f23 Update compiler/parser/src/fstring.rs
Co-authored-by: Zgarbul Andrey <zgarbul.andrey@gmail.com>
2022-12-29 08:10:33 +09:00
Anders Kaseorg 6439867f78 Allow named expression in set comprehension: `{a := b for c in d}`
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-28 09:57:32 -08:00
Anders Kaseorg 1904d095f9 Allow named expression in subscript: `a[b := c]`
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-28 09:57:32 -08:00
harupy 7e8f683808 Rename 2022-12-29 01:01:41 +09:00
harupy 37b1894834 Clean up FStringParser 2022-12-29 00:08:59 +09:00
Jim Fasarakis-Hilliard 201d08583a Merge pull request #4377 from andersk/duplicate-from
Remove duplicate declaration of "from" token
2022-12-28 11:19:54 +02:00
Anders Kaseorg 107b2e11ae Remove duplicate declaration of "from" token
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-27 18:56:24 -08:00
Anders Kaseorg 661b210391 Prohibit starred arguments after double-starred arguments
CPython prohibits ‘f(**kwargs, *args)’; we should too.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-27 12:49:50 -08:00
harupy 9aed9143fe Refactor parse_formatted_value 2022-12-28 00:23:25 +09:00
harupy c21d0d9283 Fix make_tokenizer_located 2022-12-27 21:54:18 +09:00
Jeong YunWon 313fd7d28c Merge pull request #4359 from yt2b/check_bom
Add BOM check
2022-12-26 16:03:51 +09:00
Jeong YunWon 3aa0096212 Merge pull request #4358 from harupy/fix-slice-location
Fix `Slice` location
2022-12-26 16:03:09 +09:00
Jeong YunWon 53dec88029 Merge pull request #4356 from andersk/with-tuple-named
Fix parsing of tuple with named expression as context manager
2022-12-26 16:02:17 +09:00
yt2b bd0c15d34e Fix comment 2022-12-26 09:30:12 +09:00
yt2b ce0be73841 Add BOM check 2022-12-25 11:15:29 +09:00
harupy b2ac4f60f1 Fix slice location 2022-12-25 09:37:07 +09:00
Anders Kaseorg 1fdfa5fe1b Fix parsing of tuple with named expression as context manager
Because the ‘with’ item grammar disallows named expressions, CPython
parses ‘with (a := 0, b := 1):’ as a tuple rather than two ‘with’
items.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 13:15:53 -08:00
harupy c0f390ebc6 Fix IfExp location 2022-12-25 00:33:02 +09:00
Anders Kaseorg c387b5d523 Simplify parenthesized context manager parsing with LALRPOP conditions
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 23:13:10 +09:00
harupy ea95e1a715 Fix the location of BinOp 2022-12-21 22:05:05 +09:00
Jim Fasarakis-Hilliard c16e08d59b Merge pull request #4340 from harupy/fix-locations-of-parethesized-expressions
Fix the start and end locations of `Tuple`
2022-12-18 15:17:40 +02:00
harupy ff00460ff4 Fix locations of parethesized expressions 2022-12-18 20:53:30 +09:00
Anders Kaseorg f4672e4256 Remove unnecessary boxing of ASDL product children
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-14 02:30:02 -08:00
Dimitris Fasarakis Hilliard 6e89b3ab1a Fix end location in with statements. 2022-12-13 13:28:17 +02:00
Jim Fasarakis-Hilliard 6a174dae45 Merge pull request #4327 from harupy/fix-end-location-body
Fix end location of compound statements
2022-12-13 12:30:21 +02:00
Charlie Marsh 8936ab2f8d Set ExprContext::Store on parenthesized with expressions 2022-12-12 09:09:15 -05:00
harupy 165b979733 Refactor 2022-12-12 22:36:34 +09:00
harupy 06c92bb899 Update snapshot 2022-12-12 22:26:03 +09:00
harupy 1a657731dd Format 2022-12-12 22:18:26 +09:00
harupy d8cfc7e84f Resolve conflict 2022-12-12 22:16:46 +09:00
harupy de2e88656e Address comments
Signed-off-by: harupy <hkawamura0130@gmail.com>
2022-12-12 22:14:05 +09:00
Anders Kaseorg 052dee72b8 Parse Python 3.9+ parenthesized context managers
Since the upstream grammar for this is not LR(1), we abuse LALRPOP
macros and the Into/TryInto traits to build a cover grammar that
converts to either tuples or `with` items after additional validation.
It’s annoying and ugly, but something like this is basically our only
option short of switching to a more powerful parser algorithm.

Fixes #4145.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-12 00:47:33 -08:00
Anders Kaseorg 751f9d304f Split and simplify some LALRPOP rules
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-11 22:02:08 -08:00
harupy d6f9dd0763 Use method chaining 2022-12-12 10:24:00 +09:00
harupy 5fc8c4d0b1 Fix other compound statements 2022-12-12 01:10:42 +09:00
harupy 1b7a272b77 Fix end location of nodes containing body 2022-12-11 12:35:28 +09:00
harupy 3abdc87076 Refactor 2022-12-10 22:01:42 +09:00
harupy 99b02be35a Fix 2022-12-10 18:45:36 +09:00
harupy f99167d4ed Fix plain string 2022-12-10 18:09:26 +09:00
harupy 816e1e711c Fix the end location of an implicitly-concatenated string 2022-12-10 17:49:57 +09:00
Jeong YunWon e2b28d07c8 Merge pull request #4310 from youknowone/fix-clippy
Fix nightly clippy warnings
2022-12-05 12:57:01 +09:00
Jeong YunWon 28785784b2 Fix nightly clippy warnings 2022-12-05 12:18:16 +09:00
harupy 6f6b7b2312 add tests 2022-12-04 05:59:51 +09:00