mirror of
https://github.com/morluto/rea
synced 2026-10-30 23:57:30 -04:00
fix(python): preserve source boundaries in pyc previews (#6492)
Preserve source code-point prefix endpoints in the shared marshal UTF-8 decoder, including separately encoded surrogate values. Keep native TextDecoder spans, full validation, byte identity and the existing 80-unit budget. Update the PyInstaller consumer without changing its text contract.
Validation: 207 relevant source tests;24 affected compiled CLI/MCP/filesystem CI tests; independent CPython3.11.2 compiler/marshal oracle and real argparse;18 equivalent performance tasks; build/check passed. Codex reviewed7c0f7abc6b without major findings.
CI bypass authorized for demonstrated baseline-only failures:101 source and179 built cases reproduced on actual tested main ba9c5bc68. Cancelled shard remainder is unverified. See PR body and discussion for scope and evidence.
Fixes #6476
This commit is contained in:
@@ -1625,3 +1625,21 @@ it("leaves 3.11 alpha magics with other code layouts unsupported", () => {
|
||||
parse(pythonPycFixture("py311", { magic: 3495 })).header,
|
||||
).toMatchObject({ python_version: "3.11", code_layout: "py311" });
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
"preserves a source-encoded lone surrogate at the preview boundary (truncated=%s)",
|
||||
(truncated) => {
|
||||
const text = Buffer.concat([
|
||||
Buffer.from("a".repeat(79)),
|
||||
Buffer.from([0xed, 0xa0, 0x80]),
|
||||
...(truncated ? [Buffer.from("z")] : []),
|
||||
]);
|
||||
const report = parse(withConsts([marshalUnicode(text)]));
|
||||
expect(root(report).consts[0]).toEqual({
|
||||
kind: "str",
|
||||
value: "a".repeat(79) + "\ud800",
|
||||
bytes: truncated ? 83 : 82,
|
||||
truncated,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user