mirror/rea
1
0
Fork 0
mirror of https://github.com/morluto/rea synced 2026-10-25 22:08:11 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(python): preserve marshal names and packaged bytecode

Browse Source 
Use one marshal surrogatepass decoder for PYZ directories and Python bytecode.
Validate ordinary UTF-8 spans with the native fatal decoder, preserving BOMs,
lone surrogates, raw bytes and marshal reference slots.

Distinguish legacy full-pyc CArchive modules from modern marshal-only modules
before reconstructing headers. Share typed source/codec failures between stream
output and completion while retaining native-read drain and cleanup ownership.

Validated focused producer, reader, lifecycle, CLI and MCP regressions, plus
public inventory/extraction of CPython 3.14-generated marshal tables and code.
No packaged application was executed. Document original versus derived bytes.

Fixes #6800
Fixes #6789
This commit is contained in:
morluto
2026-10-24 01:24:33 +08:00
parent 2443b949a4
commit f8f52f205a
18 changed files with 986 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/python-packaged-applications.md
+12 -7
View File
@@ -115,8 +115,9 @@ returns a resource failure.
Supported final Python bytecode magic values cover Python 3.7 through 3.14.
Supported marshal directory values are lists, tuples, strings, integers and
bytes, including interned Unicode strings used for module names. Directory
values follow the identified CPython marshal semantics; for
bytes, including interned Unicode strings used for module names. Unicode strings
use CPython's UTF-8 `surrogatepass` encoding, preserving lone surrogate code
points in decoded module names. Directory values follow the identified CPython marshal semantics; for
example, a long integer with a zero most-significant digit is malformed and is
not normalized into a valid table value. An unsupported magic or malformed or
unsupported directory stays an opaque member with a limitation and
@@ -133,11 +134,15 @@ The directory declares compressed module lengths, so uncompressed declared
sizes and source spans inside decoded data remain unknown.
PYZ modules receive a derived 16-byte `.pyc` header from their own supported
magic. CArchive bytecode receives that header only when every sibling PYZ magic
and the cookie's Python version agree. Otherwise it retains its original bytes
and `bytecode_header: "unknown"`. `bytecode_magic` and graph derivation edges
identify synthesized headers; member digests cover the resulting derived files.
These headers do not establish successful execution or decompilation.
magic. CArchive scripts receive that header only when every sibling PYZ magic
and the cookie's Python version agree. Under the same agreement, CArchive
modules (`m`/`M`) receive a header only when their decompressed prefix identifies
headerless marshal code. Legacy modules that retain a `.pyc` header, and modules
whose prefix is unrecognized, preserve their original bytes and report
`bytecode_header: "unknown"`. A header prefix does not establish that the entire
file is valid. `bytecode_magic` and graph derivation edges identify synthesized
headers; member digests cover the resulting derived files. These headers do not
establish successful execution or decompilation.
The nonempty CArchive Python module `pyimod00_crypto_key` (`m`/`M`) is recorded
as an encryption marker; a same-named data file or script is not. Affected