Integrate PR645 prepared image discovery and bounded BPE decoding

2026-10-26 02:11:36 -04:00 · 2026-10-19 12:29:25 -07:00
parent e8180f1dd3
commit d1b3022c0d
8 changed files with 792 additions and 22 deletions
@@ -57,6 +57,11 @@ Profiles declare these reusable methods:
 | Field / method | Contract |
 | --- | --- |
 | `disc_hashes` | Hash the cue's original data track before extracting. |
+| `disc_set` | Instead of `disc_hashes`: `[{name, disc_hashes}]`, every disc of a release whose executable images are identical. Any one matches; the images are pinned by their own content hashes, so which disc was read does not change the output. |
+| `file_aliases` | `{logical: [per-disc names]}` for one file that discs name differently (`SCUS_944.91`, `SCUS_945.84`, ...). Exactly one candidate must exist; the logical name reads as that file everywhere in the profile. |
+| `prepared_files[]` | Virtual files derived from disc bytes by a declared transform (see below). Each is verified, then usable as a `file` by every method and check. |
+| `cross_image_call_roots` | Opt in (`true`): a `jal` target that other images call becomes a static discovery root of the image holding it, when the evidence at the target is the same as for local calls and exactly one image occupies that address. Finds routines only an overlay (or boot code) calls. |
+| `generic_exclude_files[]` | `{files, reason}`: disc files the generic detector must not consider because the profile owns them (aliased program files are excluded automatically). |
 | `images[].method = fixed_address_files` | Whole original files placed at a verified `load_addr`. Optional duplicate-leaf checks require identical bytes. |
 | `images[].method = fixed_address_extents` | Declared `{file, file_offset, base, address, size, sha256}` byte extents placed at a verified `load_addr`. Offset rules match `words` checks; size and SHA-256 are required. |
 | `mod_packages[]` / `mod_package` | A named, verified selection of a trusted mod package. Images and checks carrying `mod_package` read the original disc with that selection applied; see below. |
@@ -71,7 +76,7 @@ Profiles declare these reusable methods:
 | `images[].method = indexed_lzss_members` | Read a u32 count and `{offset, size, stored}` descriptors, decode every member (verbatim when `size == stored`, otherwise bit-stream LZSS with declared parameters), require exact sizes, zero padding and a fully accounted file, and classify every member as an executable image at a verified `load_addr` or an explained exclusion. |
 | `checks[].method = words` | Verify loader instructions or descriptors at explicit file offsets / virtual addresses. |
 | `checks[].method = tagged_relocations` | Validate a complete relocation stream, retained `image_size` and `relocation_count`, including modules whose heap placement remains unresolved and which are excluded from native production. |
-| `checks[].method = pointer_strings` | Verify a pointer-indexed filename table against exact expected strings. |
+| `checks[].method = pointer_strings` | Verify a pointer-indexed filename table against exact expected strings. Optional `stride` (default 4, a multiple of 4) reads tables of `{id, name*}` records. |
 | `checks[].method = bcd_extent_table` | Verify an indexed BCD-MSF/size table against an ISO file's actual extent. |
 | `checks[].method = adjacent_files` | Require the named ISO extents to be sector aligned and physically adjacent, in order. |
 | `compositions` | Combine declared simultaneous producers in address order. `max_gap` bounds their separation; gap and alignment bytes remain unowned. |
@@ -127,12 +132,50 @@ Current consumers:
 - Ape Escape USA: `ApeEscapeRecomp/aot/overlays.json`, 47 established packed
  sector-table overlays plus two minigame PS-X EXEs. Two additional archive
  members have unresolved load paths and are explicitly excluded.
+- The Legend of Dragoon USA: `aot/overlays.json`, `disc_set` over all four discs
+  (identical executable images). The BPE launcher's embedded container is the engine
+  (`prepared_files`, two code extents at 0x80010000) and the ten BPE overlays fill
+  slot A (0x800C6688) or slot B (0x800FB778); all linked with `static`.
 - WipEout 3 SE PAL: `aot/overlays.json`, the front-end menu overlay (member 5
  of `WIPEOUT3/NFE.PB`, `indexed_lzss_members`) from the stock disc and from the
  NTSC / PAL Mode package's patched disc, plus that package's engine image; all
  linked with `static` (see below). The other 88 packs on the disc (`TRACK*.PBP`)
  hold data only.

+### Prepared images: packed executables and overlays
+
+Some engines run code that no disc file holds verbatim: a launcher EXE that
+unpacks the real executable, overlays stored compressed. The loader's transform
+is deterministic, so its output is an original input like any other, provided the
+transform and every byte it reads and writes are pinned. `prepared_files` declares
+such outputs; the pipeline rebuilds them from the user's own disc every time and
+nothing derived is stored or committed. The profile carries hashes, offsets, sizes
+and addresses only.
+
+```json
+"prepared_files": [{
+  "name": "PREPARED/ENGINE", "method": "bpe_container",
+  "source": {"file": "BOOT.EXE", "file_offset": "0xA00"}, "magic": "0x1A455042",
+  "consumed": "0x36228", "source_sha256": "...", "size": "0xD7800", "sha256": "..."
+}]
+```
+
+The decoder must consume exactly `consumed` bytes, whose SHA-256 is
+`source_sha256`, and produce exactly `size` bytes with SHA-256 `sha256`; any
+difference stops the run. A prepared name may not be a real disc file or alias, and
+a later entry may read an earlier one. Prepared names then work in
+`fixed_address_files`, `fixed_address_extents`, `words` and `pointer_strings`, so the
+loader code that fixes the destination is verified against the very bytes that run.
+
+Methods: `bpe_container` (`tools/bpe_container.py`): block-wise byte-pair
+expansion, `u32 size, u32 magic, {u32 block size, pair table, symbols}*, 0`. The
+magic is a required parameter.
+
+A packed image is rarely all code (its BSS, tables and replaced overlay area are
+data). Use `fixed_address_extents` on the prepared file to name only the code runs,
+each with its `sha256`, so data is never a producer: a data run decoded as code
+fails the generated-C audit and costs the whole image.
+
 ### Executable images from a mod package

 A trusted package can add code that no disc file contains: patched boot-EXE
@@ -1171,6 +1171,7 @@ if(BUILD_TESTING)
        # template, scaffold, check_generated.sh, Project Studio audit/ops).
        foreach(_t IN ITEMS
                aligned_lzss_banks
+                bpe_container
                audit_playtest_diagnostics_package
                bundled_release_layout
                collect_game_misses
@@ -27,6 +27,7 @@ from indexed_lzss_pack import members as extract_pack_members
 from mips_tagged_relocations import parse as parse_tagged_relocations, relocate as relocate_tagged_image
 from counted_mips_relocations import parse as parse_counted_relocations, relocate as relocate_counted_image
 from mod_package_images import ModPackageView
+import prepared_files

 FRAMEWORK = Path(__file__).resolve().parents[1]
 # KSEG0 main-RAM decode window. Retail 2 MiB DRAM mirrors across all of it and
@@ -68,9 +69,29 @@ def load_profile(profile_path, config):
    profile = json.loads(Path(profile_path).read_text(encoding='utf-8-sig'))
    require(profile.get('schema') == 'psxrecomp AOT methods v1', 'Unsupported AOT profile')
    require(config['game']['id'] == profile['game_id'], 'AOT profile/game mismatch')
+    require(('disc_hashes' in profile) != ('disc_set' in profile),
+            'An AOT profile declares exactly one of disc_hashes (one disc) and disc_set (several)')
    return profile


+def disc_identities(profile):
+    """[(name, {algorithm: digest})] for every disc this profile accepts.
+
+    `disc_hashes` is one disc. `disc_set` is a list of {name, disc_hashes}: a
+    release spread over several discs whose executable images are identical.
+    The profile's image sources are pinned by content hashes, so they - not the
+    disc - decide what is built; the disc identity only gates applicability."""
+    if 'disc_hashes' in profile:
+        return [('disc', profile['disc_hashes'])]
+    entries = profile['disc_set']
+    require(isinstance(entries, list) and entries, 'disc_set needs at least one disc')
+    for entry in entries:
+        require(entry.get('name') and entry.get('disc_hashes'),
+                'Every disc_set entry needs a name and disc_hashes')
+    require(len({entry['name'] for entry in entries}) == len(entries), 'Duplicate disc_set name')
+    return [(entry['name'], entry['disc_hashes']) for entry in entries]
+
+
 def static_output(profile, project_root):
    """The overlays_static.c this title's build links, or None.

@@ -214,8 +235,10 @@ def verify_evidence(disc, checks, views=None):
                actual = struct.unpack_from('<I', data, offset)[0]
                require(actual == number(expected), f"Loader evidence changed: {check['file']} {address}")
        elif method == 'pointer_strings':
+            stride = number(check.get('stride', 4))
+            require(stride >= 4 and stride % 4 == 0, 'Invalid pointer table stride')
            for index, expected in enumerate(check['strings']):
-                offset = origin + number(check['table']) - base + index * 4
+                offset = origin + number(check['table']) - base + index * stride
                pointer = struct.unpack_from('<I', data, offset)[0]
                start = origin + pointer - base
                require(0 <= start < len(data), 'String pointer outside source')
@@ -556,10 +579,31 @@ def declared_entries(source, disc):
    return entries


-def make_fixed_record(source, disc):
+def foreign_call_roots(source, sources):
+    """Entries of `source` that other images call directly (`jal`).
+
+    A call target is a function entry by construction, but an image's own scan
+    cannot see calls made from elsewhere: an overlay calling a frameless engine
+    routine, boot code calling its constructors. Only targets inside exactly one
+    image count: an address that several images occupy alternatively (overlays
+    sharing a load slot) names no particular one. The same boundary evidence as
+    for local calls applies to the target bytes."""
+    lo, hi = source['base'], source['base'] + len(source['body'])
+    others = [other for other in sources if other is not source]
+    targets = set()
+    for other in others:
+        targets |= {t for t in extractor.jal_targets(other['body']) if lo <= t < hi}
+    targets = {t for t in targets
+               if not any(o['base'] <= t < o['base'] + len(o['body']) for o in others)}
+    return extractor.direct_jal_roots(source['body'], source['base'], targets)
+
+
+def make_fixed_record(source, disc, sources=(), cross_image=False):
    body, base, spec = source['body'], source['base'], source['spec']
    require(spec.get('allow_missing'), f"Generic extractor missed required image: {source['name']}")
    direct = set(extractor.direct_jal_roots(body, base))
+    if cross_image:
+        direct |= set(foreign_call_roots(source, sources))
    seeds = set(extractor.prologues(body, base)) | extractor.frameless_leaf_entries(body, base)
    if spec.get('supplemental_entries', True):
        seeds |= extractor.supplemental_callable_seeds(body, base)
@@ -642,7 +686,7 @@ def prepare(profile, disc, records, output, views=None):
                singles[source['name']] = record
    for source in sources:
        if source['spec']['method'] in ('fixed_address_files', 'fixed_address_extents', 'packed_sector_members', 'sector_extent_members', 'aligned_lzss_banks', 'indexed_lzss_members', 'tagged_relocated_files', 'counted_relocated_members') and source['name'] not in singles:
-            record = make_fixed_record(source, disc)
+            record = make_fixed_record(source, disc, sources, bool(profile.get('cross_image_call_roots')))
            singles[source['name']] = record
            records.append(record)
    # Loader-established exports remain required even when heuristic discovery
@@ -713,6 +757,7 @@ def prepare(profile, disc, records, output, views=None):
                     original_disc_sha256=digest(disc.binary), required_images=sorted(by_name),
                     recipe_count=len(jobs), full_static_coverage_proven=False, jobs=jobs)
    inventory['mod_packages'] = [view.receipt() for view in (views or {}).values()]
+    inventory['prepared_files'] = list(getattr(disc, 'receipts', []))
    inventory['source_images'] = [dict(name=s['name'], aliases=s['aliases'],
        method=s['spec']['method'], load_addr=hex(s['base']), size=len(s['body']),
        sha256=hashlib.sha256(s['body']).hexdigest(),
@@ -724,19 +769,28 @@ def prepare(profile, disc, records, output, views=None):
 def check_disc(profile, disc, cue):
    """Digests of the disc's data track, or DiscNotApplicable naming both sides.

-    A profile is facts about one exact disc. Any other dump (another region,
-    revision, or a modified image) shares none of its verified images."""
-    digests = {}
-    for algorithm, expected in sorted(profile['disc_hashes'].items()):
-        actual = digest(disc.binary, algorithm)
-        digests[algorithm] = actual
-        if actual != expected:
-            raise DiscNotApplicable(
-                f"the AOT profile for {profile['game_id']} describes the disc whose data "
-                f"track has {algorithm} {expected}, but {Path(cue).name} has {actual}. None "
-                'of its images apply to this disc (another region or revision?), so no '
-                'native overlay code can be built for it')
-    return digests
+    A profile is facts about its exact disc(s). Any other dump (another region,
+    revision, or a modified image) shares none of its verified images. A
+    `disc_set` accepts the disc if it is any one of the declared discs."""
+    identities = disc_identities(profile)
+    digests, failures = {}, []
+    for name, expected in identities:
+        matched = True
+        for algorithm, wanted in sorted(expected.items()):
+            if algorithm not in digests:
+                digests[algorithm] = digest(disc.binary, algorithm)
+            if digests[algorithm] != wanted:
+                failures.append(f'{name} {algorithm} {wanted}')
+                matched = False
+                break
+        if matched:
+            return digests
+    actual = ', '.join(f'{algorithm} {value}' for algorithm, value in sorted(digests.items()))
+    raise DiscNotApplicable(
+        f"the AOT profile for {profile['game_id']} describes the disc whose data "
+        f"track has {'; or '.join(failures)}, but {Path(cue).name} has {actual}. None "
+        'of its images apply to this disc (another region or revision?), so no '
+        'native overlay code can be built for it')


 def game_disc(game_toml, config, cue=None):
@@ -753,6 +807,10 @@ def extract(profile_path, game_toml, recompiler, output, cue=None, disc=None, pr
        disc = Disc(cue)
        check_disc(profile, disc, cue)
    output.mkdir(parents=True, exist_ok=True)
+    try:
+        disc = prepared_files.prepare(disc, profile)
+    except prepared_files.PreparedError as exc:
+        raise ProfileError(str(exc)) from exc
    # The config supplies relative paths and code-generation settings; the disc
    # is passed explicitly so the verified dump is the one that is read.
    command = [sys.executable, str(FRAMEWORK / 'tools/aot_overlay_spike/extract_generic.py'),
@@ -760,6 +818,8 @@ def extract(profile_path, game_toml, recompiler, output, cue=None, disc=None, pr
               '--disc', str(cue),
               '--out', str(output / 'generic.json'), '--tmp', str(output / 'extract-tmp'),
               '--project-root', str(FRAMEWORK)]
+    for name in generic_exclusions(profile):
+        command += ['--exclude-file', name]
    command += ['--require-bios-resident'] if profile.get('bios_resident') else ['--no-bios-resident']
    with (output / 'extract.log').open('w') as log:
        result = subprocess.run(command, stdout=log, stderr=subprocess.STDOUT)
@@ -771,6 +831,22 @@ def extract(profile_path, game_toml, recompiler, output, cue=None, disc=None, pr
                   output, views)


+def generic_exclusions(profile):
+    """Disc files the generic detector must not consider, each with a reason.
+
+    Files the profile owns through other declarations: aliased program files
+    (their static analysis is the title's main program or a declared image) and
+    packed files whose bytes only a prepared_files transform makes visible."""
+    names = set()
+    for logical, candidates in profile.get('file_aliases', {}).items():
+        names.update(candidate.upper() for candidate in candidates)
+    for item in profile.get('generic_exclude_files', []):
+        require(item.get('files') and item.get('reason', '').strip(),
+                'generic_exclude_files entries need files and a reason')
+        names.update(name.upper() for name in item['files'])
+    return sorted(names)
+
+
 def audit(game_toml, recompiler, cache, inventory, output, static_dispatch=None, flavor=0):
    target = ['--static-dispatch', str(static_dispatch)] if static_dispatch else ['--cache-root', str(cache)]
    subprocess.run([sys.executable, str(FRAMEWORK / 'tools/audit_aot_cache.py'),
@@ -947,7 +1023,9 @@ def generation_inputs(profile_path, profile, game_toml, recompiler, disc_digests
        schema=GENERATION_INPUTS_SCHEMA,
        profile_sha256=digest(profile_path),
        game_toml_sha256=digest(game_toml),
-        disc=disc_digests,
+        # A disc_set profile builds from content-pinned images that are identical on
+        # every disc it lists, so which of them was read does not change the output.
+        disc=disc_digests if 'disc_hashes' in profile else dict(disc_set=True),
        recompiler_sha256=digest(recompiler),
        cache_tag=compiler.cache_tag(include, str(recompiler), str(game_toml), 0),
        cps=bool(cps),
@@ -1118,7 +1196,8 @@ def run():
                                 args.workers, config.parent, args.cps)
        receipt = audit(config, recompiler, None, work / 'runtime-input-inventory.json',
                        work / 'audit.json', build_dir / STATIC_OUTPUT_NAME)
-        for key in ('profile_sha256', 'original_disc_sha256', 'required_images', 'mod_packages'):
+        for key in ('profile_sha256', 'original_disc_sha256', 'required_images', 'mod_packages',
+                    'prepared_files'):
            receipt[key] = inventory[key]
        receipt['generation_inputs'] = inputs
        receipt['generation_inputs_sha256'] = inputs_sha256
@@ -216,10 +216,13 @@ def jal_targets(data):
            ts.add(0x80000000 | ((w & 0x03FFFFFF) << 2))
    return ts

-def direct_jal_roots(data, base):
-    """Locally bounded call targets with independent boundary evidence."""
+def direct_jal_roots(data, base, targets=None):
+    """Call targets inside [base, base+len) with independent boundary evidence.
+
+    `targets` are the jal targets to consider: this image's own by default; an
+    AOT profile can pass the targets other images call into this one."""
    lo,hi=base,base+len(data)
-    return sorted(t for t in jal_targets(data)
+    return sorted(t for t in (jal_targets(data) if targets is None else targets)
                  if lo <= t < hi and (t&3)==0 and
                  not _dense_local_pointer_table(data,base,t-base) and
                  (_callable_boundary(data,t-base) or
@@ -975,6 +978,9 @@ def main():
    ap.add_argument('--project-root', default=None,
                    help='root holding bios/SCPH1001.toml for normal-mode discovery '
                         '(default: this framework)')
+    ap.add_argument('--exclude-file', action='append', default=[],
+                    help='disc file the detector must not consider (an AOT profile '
+                         'declares it another way); repeatable')
    ap.add_argument('--bios', default=None,
                    help='BIOS ROM for exact-hash resident-code recipes (default: '
                         'PSXRECOMP_BIOS_ROM or framework bios/SCPH1001.BIN)')
@@ -1036,9 +1042,12 @@ def main():
    records=[]; np=nh=nr=na=ne=nc=0; seen_crc=set(); header_files=[]; raw_files=[]
    relocatable=[]
    positioned=[]
+    excluded={name.replace('\\','/').lstrip('/').upper() for name in a.exclude_file}
    for p,l,s in sorted(files):
        if p.upper() in hed_consumed:
            continue
+        if p.replace('\\','/').lstrip('/').upper() in excluded:
+            continue
        if s<8 or s>2_000_000: continue
        if exe_base and os.path.basename(p).upper()==exe_base:
            continue   # skip the statically-compiled base executable
@@ -0,0 +1,115 @@
+"""Block-wise byte-pair-expansion ("BPE") containers.
+
+Some PlayStation engines pack their executable image and overlays with a
+byte-pair coder: a table of 256 optional (left, right) pairs per block, then a
+stream of symbols each expanded through that table. The container is
+
+    u32  decoded size            (all blocks together)
+    u32  magic                   (a title-declared constant, "BPE\\x1a" in the
+                                  titles seen so far)
+    block*:
+        u32  block size          bytes this block decodes to (0 ends the container)
+        table                    256 entries, run-length coded (below)
+        symbols                  as many input bytes as it takes to produce
+                                 `block size` output bytes, each expanded
+                                 through the table
+
+The decoder reads its input as little-endian 32-bit words, handing the words
+out as bytes. Anything left unread in the last word of a table or block is
+discarded, so every block header starts on a word boundary; `consumed` counts
+whole words including the terminating zero count.
+
+Table coding, entry index `c` starting at 0 until it reaches 256:
+  * a byte `n` >= 0x80 first skips `n - 0x7F` entries (they stay identity
+    pairs) and is then treated as `n = 0`;
+  * the next `n + 1` entries are given in turn, each by a byte `b` that sets
+    `left[c] = b` and, if `b != c`, a second byte that sets `right[c]`. An
+    entry with `left[c] == c` is a literal.
+Expanding a symbol `b`: while `left[b] != b`, the byte `right[b]` is
+queued to follow, and `b` becomes `left[b]`; the final `b` is emitted, then the
+queued bytes are expanded the same way, nearest first. A queued byte is
+processed before any unread input. Each emitted byte counts against the block
+size, whether it came from a symbol or from the queue.
+
+This module is a pure function of its input. It rejects truncated input, a
+declared size that the blocks do not produce, a different magic, and tables
+that never finish expanding, instead of returning partial output.
+"""
+import struct
+
+
+class BpeError(ValueError):
+    """A container is truncated, inconsistent, or not the declared format."""
+
+
+def decode(data, *, magic, offset=0, max_output=1 << 28):
+    """Decode the container at `offset`. Returns (output bytes, consumed bytes).
+
+    `magic` is the second header word the title's loader checks; it is a
+    required argument because it is evidence about the title, not a default.
+    """
+    if offset < 0 or offset % 4 or len(data) - offset < 12:
+        raise BpeError('BPE container is truncated or misaligned')
+    declared, found = struct.unpack_from('<II', data, offset)
+    if found != magic:
+        raise BpeError(f'BPE magic {found:#010x} is not the declared {magic:#010x}')
+    if declared > max_output:
+        raise BpeError('BPE declared size exceeds the output limit')
+    end = len(data)
+    position = offset + 8
+    output = bytearray()
+
+    def word():
+        nonlocal position
+        if position + 4 > end:
+            raise BpeError('BPE container is truncated')
+        value = struct.unpack_from('<I', data, position)[0]
+        position += 4
+        return value
+
+    while True:
+        count = word()
+        if count == 0:
+            break
+        if len(output) + count > declared:
+            raise BpeError('BPE blocks decode past the declared size')
+        left, right = list(range(256)), [0] * 256
+        window = []          # next byte to hand out is window[-1]
+
+        def take():
+            if not window:
+                window.extend(reversed(data[position:position + 4]))
+                advance = word          # consume the word through the bounds check
+                advance()
+            return window.pop()
+
+        index = 0
+        while index < 256:
+            run = take()
+            if run >= 0x80:
+                index += run - 0x7F
+                run = 0
+            if index >= 256:
+                break
+            for _ in range(run + 1):
+                if index >= 256:
+                    raise BpeError('BPE table run passes its last entry')
+                symbol = take()
+                left[index] = symbol
+                if symbol != index:
+                    right[index] = take()
+                index += 1
+        budget = (count + 1) * 256     # a table that never reaches a literal
+        for _ in range(count):
+            symbol = take()
+            while left[symbol] != symbol:
+                window.append(right[symbol])
+                symbol = left[symbol]
+                budget -= 1
+                if budget < 0:
+                    raise BpeError('BPE table does not terminate')
+            output.append(symbol)
+        window.clear()
+    if len(output) != declared:
+        raise BpeError(f'BPE decoded {len(output):#x} bytes, header declares {declared:#x}')
+    return bytes(output), position - offset
@@ -0,0 +1,138 @@
+"""Virtual files an AOT profile derives from the original disc.
+
+An engine can run code that no disc file contains verbatim: a boot EXE that
+unpacks an inner image, overlays stored compressed. The loader's transform is
+deterministic, so its output is as much an original input as a raw file, as
+long as the transform and every byte it consumes and produces are pinned.
+
+A profile declares such outputs in `prepared_files`; each is then an ordinary
+named file for every image method and evidence check (`fixed_address_files`,
+`words`, `pointer_strings`, ...). Nothing derived is stored: it is rebuilt from
+the user's own disc whenever the profile is used, and the profile carries only
+hashes, offsets and sizes.
+
+    "prepared_files": [{
+      "name": "ENGINE.IMG",                     # unique; never a real disc file
+      "method": "bpe_container",
+      "source": {"file": "BOOT.EXE", "file_offset": "0xA00"},
+      "magic": "0x1A455042",                    # declared by the method
+      "consumed": "0x36228",                    # input bytes the decoder reads
+      "source_sha256": "...",                   # of exactly those bytes
+      "size": "0xD7800",                        # decoded size
+      "sha256": "..."                           # of the decoded bytes
+    }]
+
+`file_aliases` maps a logical name to the per-disc names of one file, for sets
+whose discs name an identical file differently (SCUS_944.91, SCUS_945.84, ...).
+Exactly one candidate must exist on the disc being read.
+
+Methods are decoders over bytes only; they never see a path, the disc, or the
+profile's other sections.
+"""
+import hashlib
+
+import bpe_container
+
+
+class PreparedError(ValueError):
+    """A declared derived file did not reproduce from the disc."""
+
+
+def number(value):
+    return int(value, 0) if isinstance(value, str) else int(value)
+
+
+def _bpe_container(data, spec):
+    return bpe_container.decode(data, magic=number(spec['magic']), offset=0)
+
+
+# method -> (decoder(data_from_source_offset, spec) -> (output, consumed),
+#            keys the spec must declare beyond the common ones)
+METHODS = {'bpe_container': (_bpe_container, ('magic',))}
+COMMON_KEYS = ('name', 'method', 'source', 'consumed', 'source_sha256', 'size', 'sha256')
+
+
+class PreparedDisc:
+    """An original disc plus the profile's aliases and derived files.
+
+    Reads of a declared alias or prepared name are answered here; everything
+    else (`files`, `binary`, `reader`, ...) is the underlying disc's."""
+
+    def __init__(self, disc, aliases=None, prepared=None, receipts=None):
+        self.disc = disc
+        self.aliases = dict(aliases or {})
+        self.prepared = dict(prepared or {})
+        self.receipts = list(receipts or [])
+
+    def __getattr__(self, name):
+        # Only reached for attributes this wrapper does not define.
+        return getattr(self.disc, name)
+
+    def read(self, name):
+        key = name.upper()
+        if key in self.prepared:
+            return self.prepared[key]
+        return self.disc.read(self.aliases.get(key, key))
+
+
+def resolve_aliases(disc, profile):
+    """logical name -> the one candidate that exists on this disc."""
+    resolved = {}
+    for logical, candidates in profile.get('file_aliases', {}).items():
+        key = logical.upper()
+        if key in disc.files:
+            raise PreparedError(f'File alias {logical} is the name of a real disc file')
+        if not candidates or not all(isinstance(c, str) and c for c in candidates):
+            raise PreparedError(f'File alias {logical} needs candidate file names')
+        present = [c.upper() for c in candidates if c.upper() in disc.files]
+        if len(present) != 1:
+            raise PreparedError(
+                f'File alias {logical} needs exactly one of {sorted(c.upper() for c in candidates)} '
+                f'on the disc, found {present}')
+        resolved[key] = present[0]
+    return resolved
+
+
+def prepare(disc, profile):
+    """The disc as the profile sees it: aliases resolved, derived files built.
+
+    Every derived file is verified (input extent, decoded size and hash) before
+    any image method can read it; a mismatch is a PreparedError."""
+    aliases = resolve_aliases(disc, profile)
+    view = PreparedDisc(disc, aliases)
+    for spec in profile.get('prepared_files', []):
+        method = METHODS.get(spec.get('method'))
+        if method is None:
+            raise PreparedError(f"Unknown prepared file method: {spec.get('method')}")
+        decoder, extra = method
+        missing = [key for key in COMMON_KEYS + extra if key not in spec]
+        if missing:
+            raise PreparedError(f"Prepared file {spec.get('name')} lacks {missing}")
+        name = spec['name'].upper()
+        if name in view.prepared or name in aliases or name in disc.files:
+            raise PreparedError(f'Prepared file name is not unique: {name}')
+        source = spec['source']
+        data = view.read(source['file'])
+        start = number(source.get('file_offset', 0))
+        if not 0 <= start < len(data):
+            raise PreparedError(f'Prepared file {name}: source offset outside {source["file"]}')
+        try:
+            output, consumed = decoder(data[start:], spec)
+        except ValueError as exc:
+            raise PreparedError(f'Prepared file {name} does not decode: {exc}') from exc
+        if consumed != number(spec['consumed']):
+            raise PreparedError(f'Prepared file {name}: decoder consumed {consumed:#x} bytes, '
+                                f"profile pins {number(spec['consumed']):#x}")
+        if hashlib.sha256(data[start:start + consumed]).hexdigest() != spec['source_sha256']:
+            raise PreparedError(f'Prepared file {name}: source bytes changed')
+        if len(output) != number(spec['size']):
+            raise PreparedError(f'Prepared file {name}: decoded {len(output):#x} bytes, '
+                                f"profile pins {number(spec['size']):#x}")
+        if hashlib.sha256(output).hexdigest() != spec['sha256']:
+            raise PreparedError(f'Prepared file {name}: decoded bytes changed')
+        view.prepared[name] = output
+        view.receipts.append(dict(name=name, method=spec['method'], source=source['file'].upper(),
+                                  source_offset=start, consumed=consumed,
+                                  source_sha256=spec['source_sha256'], size=len(output),
+                                  sha256=spec['sha256']))
+    return view
@@ -15,6 +15,8 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
 import aot_overlay_pipeline as pipeline
 import audit_aot_cache as auditor
 import indexed_lzss_pack as lzss_pack
+import prepared_files
+from test_bpe_container import MAGIC as BPE_MAGIC, container as bpe_container_bytes, encode_block


 class PgxpReleaseNamespaceTests(unittest.TestCase):
@@ -904,6 +906,248 @@ class IndexedLzssPackTest(unittest.TestCase):
                pipeline.positioned_sources(disc, [self.spec(**change)])


+class PreparedImageTests(unittest.TestCase):
+    """Images a loader unpacks: derived files, per-disc aliases, disc sets."""
+    CODE = bytes.fromhex('27bdffe8afbf001000000000') * 4          # invented words
+    OVERLAY = bytes.fromhex('3c028001244200108c430000') * 3
+
+    def boot(self):
+        """A launcher: 8 bytes of header, then a container holding CODE."""
+        packed = bpe_container_bytes([encode_block(list(self.CODE), len(self.CODE))], len(self.CODE))
+        return b'LAUNCH!!' + packed + b'tail', 8, len(packed), packed
+
+    def profile(self, **overrides):
+        boot, offset, consumed, packed = self.boot()
+        profile = dict(
+            game_id='TEST',
+            file_aliases={'BOOT.EXE': ['SLUS_001.01', 'SLUS_002.02']},
+            prepared_files=[dict(name='PREPARED/ENGINE', method='bpe_container',
+                                 source=dict(file='BOOT.EXE', file_offset=offset), magic=hex(BPE_MAGIC),
+                                 consumed=consumed, source_sha256=sha(packed), size=len(self.CODE),
+                                 sha256=sha(self.CODE))])
+        return {**profile, **overrides}
+
+    def disc(self, boot_name='SLUS_001.01', extra=None):
+        return FakeDisc({boot_name: self.boot()[0], **(extra or {})})
+
+    def test_prepared_file_reads_like_a_disc_file_for_images_and_evidence(self):
+        view = prepared_files.prepare(self.disc(), self.profile())
+        self.assertEqual(view.read('prepared/engine'), self.CODE)
+        self.assertEqual(view.read('BOOT.EXE'), self.boot()[0])
+        self.assertEqual(view.receipts[0]['sha256'], sha(self.CODE))
+        source, = pipeline.positioned_sources(view, [dict(
+            method='fixed_address_files', files=['PREPARED/ENGINE'], load_addr='0x80010000')])
+        self.assertEqual((source['name'], source['base'], source['body']),
+                         ('PREPARED/ENGINE', 0x80010000, self.CODE))
+        first, second = struct.unpack_from('<II', self.CODE)
+        pipeline.verify_evidence(view, [dict(method='words', file='PREPARED/ENGINE', base='0x80010000',
+                                             values={'0x80010004': hex(second), '0x80010000': hex(first)})])
+        with self.assertRaisesRegex(ValueError, 'Loader evidence changed'):
+            pipeline.verify_evidence(view, [dict(method='words', file='PREPARED/ENGINE', base='0x80010000',
+                                                 values={'0x80010000': hex(first + 1)})])
+
+    def test_alias_names_one_file_per_disc_and_is_never_a_real_name(self):
+        for name in ('SLUS_001.01', 'SLUS_002.02'):
+            with self.subTest(name=name):
+                view = prepared_files.prepare(self.disc(name), self.profile())
+                self.assertEqual(view.read('BOOT.EXE'), self.boot()[0])
+                self.assertEqual(view.read('PREPARED/ENGINE'), self.CODE)
+        cases = [(self.disc('OTHER.EXE'), self.profile(), 'exactly one'),
+                 (FakeDisc({'SLUS_001.01': b'a', 'SLUS_002.02': b'b'}), self.profile(), 'exactly one'),
+                 (self.disc(extra={'BOOT.EXE': b'x'}), self.profile(), 'real disc file'),
+                 (self.disc(), self.profile(file_aliases={'BOOT.EXE': []}), 'candidate')]
+        for disc, profile, error in cases:
+            with self.subTest(error=error), self.assertRaisesRegex(prepared_files.PreparedError, error):
+                prepared_files.prepare(disc, profile)
+
+    def test_prepared_files_fail_closed_on_any_drift(self):
+        good = self.profile()['prepared_files'][0]
+
+        def with_change(**change):
+            return self.profile(prepared_files=[{**good, **change}])
+        cases = [
+            (with_change(consumed=good['consumed'] - 4), 'consumed'),
+            (with_change(source_sha256='0' * 64), 'source bytes changed'),
+            (with_change(size=good['size'] + 4), 'profile pins'),
+            (with_change(sha256='0' * 64), 'decoded bytes changed'),
+            (with_change(magic='0x12345678'), 'does not decode'),
+            (with_change(method='lzss_container'), 'Unknown prepared file method'),
+            (with_change(source=dict(file='BOOT.EXE', file_offset=1 << 20)), 'outside'),
+            (with_change(source=dict(file='BOOT.EXE', file_offset=12)), 'does not decode'),
+            (self.profile(prepared_files=[good, good]), 'not unique'),
+            (with_change(name='SLUS_001.01'), 'not unique'),
+            (with_change(name='BOOT.EXE'), 'not unique'),
+            (self.profile(prepared_files=[{k: v for k, v in good.items() if k != 'magic'}]), 'lacks'),
+        ]
+        for profile, error in cases:
+            with self.subTest(error=error), self.assertRaisesRegex(prepared_files.PreparedError, error):
+                prepared_files.prepare(self.disc(), profile)
+
+    def test_prepared_files_can_be_built_from_earlier_prepared_files(self):
+        inner = self.OVERLAY
+        packed = bpe_container_bytes([encode_block(list(inner), len(inner))], len(inner))
+        spec = dict(name='PREPARED/OVERLAY', method='bpe_container', source=dict(file='PACK.BIN'),
+                    magic=hex(BPE_MAGIC), consumed=len(packed), source_sha256=sha(packed),
+                    size=len(inner), sha256=sha(inner))
+        view = prepared_files.prepare(FakeDisc({'PACK.BIN': packed + self.CODE}), dict(prepared_files=[spec]))
+        self.assertEqual(view.read('PREPARED/OVERLAY'), inner)
+        chained = dict(spec, name='PREPARED/AGAIN', source=dict(file='PREPARED/OVERLAY'),
+                       consumed=0, source_sha256=sha(b''), size=0)
+        with self.assertRaises(prepared_files.PreparedError):
+            prepared_files.prepare(FakeDisc({'PACK.BIN': packed}), dict(prepared_files=[spec, chained]))
+
+    def test_disc_set_accepts_any_listed_disc_and_rejects_others_naming_all(self):
+        with tempfile.TemporaryDirectory() as directory:
+            track = Path(directory) / 'track.bin'
+            track.write_bytes(b'second disc')
+            disc = mock.Mock(binary=track)
+            discs = [dict(name='Disc 1', disc_hashes=dict(sha256='ab' * 32)),
+                     dict(name='Disc 2', disc_hashes=dict(sha256=sha(track.read_bytes())))]
+            profile = dict(game_id='TEST', disc_set=discs)
+            self.assertEqual(pipeline.check_disc(profile, disc, 'd2.cue'),
+                             dict(sha256=sha(track.read_bytes())))
+            profile['disc_set'] = discs[:1] + [dict(name='Disc 3', disc_hashes=dict(sha256='cd' * 32))]
+            with self.assertRaises(pipeline.DiscNotApplicable) as caught:
+                pipeline.check_disc(profile, disc, 'd2.cue')
+            text = str(caught.exception)
+            self.assertTrue('ab' * 32 in text and 'cd' * 32 in text and sha(track.read_bytes()) in text, text)
+            self.assertIn('d2.cue', text)
+
+    def test_profile_declares_one_disc_or_a_disc_set(self):
+        with tempfile.TemporaryDirectory() as directory:
+            path = Path(directory) / 'overlays.json'
+            config = dict(game=dict(id='TEST'))
+            base = dict(schema='psxrecomp AOT methods v1', game_id='TEST')
+            one = dict(name='d', disc_hashes=dict(sha256='a'))
+            for extra, ok in [(dict(disc_hashes=dict(sha256='a')), True), (dict(disc_set=[one]), True),
+                              ({}, False), (dict(disc_hashes=dict(sha256='a'), disc_set=[one]), False)]:
+                path.write_text(json.dumps({**base, **extra}))
+                with self.subTest(keys=sorted(extra)):
+                    if ok:
+                        pipeline.load_profile(path, config)
+                    else:
+                        with self.assertRaisesRegex(pipeline.ProfileError, 'exactly one'):
+                            pipeline.load_profile(path, config)
+            bad = [dict(disc_set=[]), dict(disc_set=[dict(name='d')]),
+                   dict(disc_set=[one, dict(name='d', disc_hashes=dict(sha256='b'))])]
+            for extra in bad:
+                with self.subTest(extra=extra), self.assertRaises(pipeline.ProfileError):
+                    pipeline.disc_identities({**base, **extra})
+
+    def test_generation_inputs_do_not_depend_on_which_disc_of_a_set_was_read(self):
+        tmp = tempfile.TemporaryDirectory()
+        self.addCleanup(tmp.cleanup)
+        root = Path(tmp.name)
+        (root / 'game.toml').write_text('[game]\nid = "TEST"\n')
+        (root / 'overlays.json').write_text('{}')
+        recompiler = root / 'emitter'
+        recompiler.write_bytes(b'emitter')
+        sets = dict(disc_set=[dict(name='a', disc_hashes=dict(sha256='1')),
+                              dict(name='b', disc_hashes=dict(sha256='2'))])
+
+        def inputs(profile, digests):
+            with mock.patch.object(pipeline.compiler, 'cache_tag', return_value='tag'):
+                return pipeline.generation_inputs(root / 'overlays.json', profile, root / 'game.toml',
+                                                  recompiler, digests, 'gcc', True)[1]
+        self.assertEqual(inputs(sets, dict(sha256='1')), inputs(sets, dict(sha256='2')))
+        single = dict(disc_hashes=dict(sha256='1'))
+        self.assertNotEqual(inputs(single, dict(sha256='1')), inputs(single, dict(sha256='2')))
+
+    def test_pointer_strings_stride_reads_interleaved_tables(self):
+        # {id, name*} records: the pointers sit every eight bytes.
+        base = 0x80010000
+        data = bytearray(64)
+        struct.pack_into('<IIII', data, 0, 0xFFFF, base + 32, 0xFFFF, base + 40)
+        data[32:36], data[40:44] = b'ONE\0', b'TWO\0'
+        disc = FakeDisc({'IMG.BIN': bytes(data)})
+        check = dict(method='pointer_strings', file='IMG.BIN', base=hex(base), table=hex(base + 4),
+                     stride=8, strings=['ONE', 'TWO'])
+        pipeline.verify_evidence(disc, [check])
+        for change, error in [(dict(strings=['TWO', 'ONE']), 'Filename table changed'),
+                              (dict(stride=3), 'stride'), (dict(stride=4), 'Filename table changed|outside')]:
+            with self.subTest(change=change), self.assertRaisesRegex(Exception, error):
+                pipeline.verify_evidence(disc, [{**check, **change}])
+
+    def test_generic_detector_is_told_which_files_the_profile_owns(self):
+        profile = self.profile(generic_exclude_files=[dict(files=['OVL/A.OV_', 'ovl/b.ov_'],
+                                                           reason='compressed on disc')])
+        self.assertEqual(pipeline.generic_exclusions(profile),
+                         ['OVL/A.OV_', 'OVL/B.OV_', 'SLUS_001.01', 'SLUS_002.02'])
+        for bad in (dict(files=['X'], reason=' '), dict(files=[], reason='r'), dict(reason='r')):
+            with self.subTest(bad=bad), self.assertRaisesRegex(pipeline.ProfileError, 'reason'):
+                pipeline.generic_exclusions(dict(generic_exclude_files=[bad]))
+        with tempfile.TemporaryDirectory() as directory:
+            commands = []
+
+            def fail(command, **kwargs):
+                commands.append(command)
+                return mock.Mock(returncode=1)
+            work = Path(directory)
+            (work / 'game.toml').write_text('[game]\nid = "TEST"\n')
+            with mock.patch.object(pipeline.subprocess, 'run', side_effect=fail), \
+                 contextlib.redirect_stderr(io.StringIO()), \
+                 self.assertRaises(pipeline.subprocess.CalledProcessError):
+                pipeline.extract(None, work / 'game.toml', work / 'emitter', work / 'out',
+                                 cue=work / 'game.cue', disc=self.disc(), profile=profile)
+            command = commands[0]
+            excluded = [command[i + 1] for i, word in enumerate(command) if word == '--exclude-file']
+            self.assertEqual(excluded, pipeline.generic_exclusions(profile))
+
+    def test_inventory_carries_the_prepared_file_receipts(self):
+        profile = dict(game_id='TEST', expected_records=1, strict_bounds=True,
+                       file_aliases=self.profile()['file_aliases'],
+                       prepared_files=self.profile()['prepared_files'],
+                       images=[dict(method='fixed_address_files', files=['PREPARED/ENGINE'],
+                                    load_addr='0x80010000', allow_missing=True, entries=['0x80010000'])])
+        with tempfile.TemporaryDirectory() as directory:
+            disc = self.disc()
+            disc.binary = Path(directory) / 'track.bin'
+            disc.binary.write_bytes(b'synthetic disc')
+            view = prepared_files.prepare(disc, profile)
+            inventory = pipeline.prepare(profile, view, [], Path(directory))
+        self.assertEqual(inventory['required_images'], ['PREPARED/ENGINE'])
+        self.assertEqual([item['sha256'] for item in inventory['prepared_files']], [sha(self.CODE)])
+        self.assertEqual(inventory['source_images'][0]['sha256'], sha(self.CODE))
+
+
+class CrossImageRootsTests(unittest.TestCase):
+    """Calls made from one image into another are entries of the callee image."""
+
+    @staticmethod
+    def image(name, base, words):
+        body = b''.join(struct.pack('<I', w) for w in words)
+        return dict(name=name, base=base, body=body, spec=dict(method='fixed_address_files',
+                                                               allow_missing=True), aliases=[])
+
+    def setUp(self):
+        nop, ret = 0, 0x03E00008
+        prologue = 0x27BDFFE8                          # addiu sp, sp, -0x18
+        # Engine: data-free code with a routine at +0x10 nothing inside it calls.
+        self.engine = self.image('ENGINE', 0x80010000,
+                                 [ret, nop, nop, nop, prologue, 0, ret, nop])
+        self.target = 0x80010010
+        jal = lambda t: 0x0C000000 | ((t & 0x0FFFFFFF) >> 2)
+        self.caller = self.image('CALLER', 0x80100000, [jal(self.target), nop, ret, nop])
+
+    def test_a_target_only_another_image_calls_becomes_a_root(self):
+        engine, caller = self.engine, self.caller
+        self.assertEqual(pipeline.foreign_call_roots(engine, [engine, caller]), [self.target])
+        self.assertEqual(pipeline.foreign_call_roots(caller, [engine, caller]), [])
+        local = pipeline.make_fixed_record(engine, None, [engine, caller], False)
+        cross = pipeline.make_fixed_record(engine, None, [engine, caller], True)
+        self.assertNotIn(hex(self.target), local.get('static_discovery_entry_pcs', []))
+        self.assertIn(hex(self.target), cross['static_discovery_entry_pcs'])
+
+    def test_an_address_several_images_occupy_names_no_one_of_them(self):
+        engine = self.engine
+        twin = self.image('TWIN', 0x80010000, [0x03E00008, 0, 0, 0, 0x27BDFFE8, 0, 0x03E00008, 0])
+        self.assertEqual(pipeline.foreign_call_roots(engine, [engine, twin, self.caller]), [])
+
+    def test_a_target_without_boundary_evidence_is_not_a_root(self):
+        data = self.image('DATA', 0x80010000, [0x11111111] * 8)
+        self.assertEqual(pipeline.foreign_call_roots(data, [data, self.caller]), [])
+
+
 class StaticGenerationTest(unittest.TestCase):
    """The `static` action as `psxrecomp_cli.py generate` drives it."""

@@ -0,0 +1,141 @@
+"""BPE container contract on invented bytes; no game data."""
+import struct
+import sys
+import unittest
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+import bpe_container
+from bpe_container import BpeError, decode
+
+MAGIC = 0x1A455042
+
+
+def encode_table(left, right):
+    """Run-length table coding with identity stretches skipped."""
+    out, c = bytearray(), 0
+    while c < 256:
+        stretch = 0
+        while c + stretch < 256 and stretch < 128 and left[c + stretch] == c + stretch:
+            stretch += 1
+        if stretch >= 3:
+            out.append(0x7E + stretch)          # skips stretch - 1 entries, then one entry
+            out.append(c + stretch - 1)         # the last identity entry of the stretch
+            c += stretch
+            continue
+        run = 0
+        while c + run < 256 and run < 128 and not (
+                left[c + run] == c + run and
+                all(left[c + run + k] == c + run + k for k in range(1, 3) if c + run + k < 256) and
+                c + run + 3 <= 256):
+            run += 1
+        run = max(run, 1)
+        out.append(run - 1)
+        for k in range(c, c + run):
+            out.append(left[k])
+            if left[k] != k:
+                out.append(right[k])
+        c += run
+    return bytes(out)
+
+
+def encode_block(symbols, decoded_size, left=None, right=None):
+    """A block of input `symbols` that decodes to `decoded_size` bytes."""
+    left = list(range(256)) if left is None else left
+    right = [0] * 256 if right is None else right
+    body = encode_table(left, right) + bytes(symbols)
+    body += bytes(-len(body) % 4)
+    return struct.pack('<I', decoded_size) + body
+
+
+def container(blocks, declared, magic=MAGIC):
+    return struct.pack('<II', declared, magic) + b''.join(blocks) + struct.pack('<I', 0)
+
+
+class BpeContainerTests(unittest.TestCase):
+    def pairs(self):
+        left, right = list(range(256)), [0] * 256
+        left[0xF0], right[0xF0] = ord('A'), ord('B')       # 0xF0 -> "AB"
+        left[0xF1], right[0xF1] = 0xF0, ord('C')           # 0xF1 -> "ABC" (chained)
+        left[0x10], right[0x10] = 0x20, 0x30                # 0x10 -> 0x20 0x30
+        return left, right
+
+    def one(self, symbols, size, left=None, right=None):
+        return container([encode_block(symbols, size, left, right)], size)
+
+    def test_literals_pairs_chains_and_multiple_blocks(self):
+        left, right = self.pairs()
+        first = encode_block([0xF1, ord('x'), 0xF0, 0x10], 8, left, right)
+        second = encode_block(list(b'plain'), 5)
+        data = container([first, second], 13)
+        body, consumed = decode(data, magic=MAGIC)
+        self.assertEqual(body, b'ABCxAB' + bytes([0x20, 0x30]) + b'plain')
+        self.assertEqual(consumed, len(data))
+
+    def test_consumed_counts_words_and_offset_selects_the_container(self):
+        data = self.one(list(b'abc'), 3)
+        for lead in (b'', bytes(8)):
+            with self.subTest(lead=len(lead)):
+                body, consumed = decode(lead + data + b'trailing', magic=MAGIC, offset=len(lead))
+                self.assertEqual((body, consumed), (b'abc', len(data)))
+
+    def test_skip_coded_identity_stretches_keep_tables_short_and_decode_correctly(self):
+        left, right = self.pairs()
+        table = encode_table(left, right)
+        self.assertLess(len(table), 40, 'identity stretches must be skip-coded')
+        body, _ = decode(self.one([0xF1, 0x10, 0x41, 0xF0], 8, left, right), magic=MAGIC)
+        self.assertEqual(body, b'ABC' + bytes([0x20, 0x30]) + b'A' + b'AB')
+
+    def test_a_skip_that_reaches_the_last_entry_ends_the_table(self):
+        # 128 explicit entries (entry 0 is a pair), then a 128-entry skip: index 256.
+        table = bytes([127, ord('Z'), ord('Y')]) + bytes(range(1, 128)) + bytes([0xFF])
+        block = struct.pack('<I', 2) + table + bytes([0])
+        block += bytes(-len(block) % 4)
+        body, _ = decode(container([block], 2), magic=MAGIC)
+        self.assertEqual(body, b'ZY')
+        # After a skip that stays inside the table, one entry still follows.
+        table = bytes([127]) + bytes(range(128)) + bytes([0xFE, 0x41, 0x42])  # skip 127 -> 255, entry 255
+        block = struct.pack('<I', 1) + table + bytes([5])
+        block += bytes(-len(block) % 4)
+        self.assertEqual(decode(container([block], 1), magic=MAGIC)[0], b'')
+
+    def test_rejects_wrong_magic_size_truncation_and_misalignment(self):
+        good = self.one(list(b'abc'), 3)
+        cases = [
+            (self.one(list(b'abc'), 3).replace(struct.pack('<I', MAGIC), struct.pack('<I', 0x12345678), 1),
+             dict(), 'magic'),
+            (container([encode_block(list(b'abc'), 3)], 4), dict(), 'declares'),
+            (container([encode_block(list(b'abc'), 3)], 2), dict(), 'declared size'),
+            (good[:-4], dict(), 'truncated'),
+            (good[:12], dict(), 'truncated'),
+            (good[:9], dict(), 'truncated'),
+            (good, dict(offset=2), 'misaligned'),
+            (good, dict(max_output=2), 'output limit'),
+        ]
+        for data, options, error in cases:
+            with self.subTest(error=error, options=options), self.assertRaisesRegex(BpeError, error):
+                decode(data, magic=MAGIC, **options)
+
+    def test_rejects_cyclic_tables_and_table_runs_past_the_last_entry(self):
+        left, right = list(range(256)), [0] * 256
+        left[0xF0], right[0xF0] = 0xF1, ord('x')
+        left[0xF1], right[0xF1] = 0xF0, ord('y')
+        with self.assertRaisesRegex(BpeError, 'does not terminate'):
+            decode(self.one([0xF0], 1, left, right), magic=MAGIC)
+        # 128 + 101 + 41 entries = 270 > 256.
+        table = (bytes([127]) + bytes(range(128)) + bytes([100]) + bytes(range(128, 229)) +
+                 bytes([40]) + bytes(range(229, 256)) + bytes(16))
+        block = struct.pack('<I', 1) + table + bytes([5])
+        block += bytes(-len(block) % 4)
+        with self.assertRaisesRegex(BpeError, 'passes its last entry'):
+            decode(container([block], 1), magic=MAGIC)
+
+    def test_magic_is_required_evidence_not_a_default(self):
+        with self.assertRaises(TypeError):
+            decode(self.one(list(b'a'), 1))     # no magic given
+        self.assertTrue(issubclass(BpeError, ValueError))
+        self.assertIs(bpe_container.BpeError, BpeError)
+
+
+if __name__ == '__main__':
+    unittest.main()