#!/usr/bin/env python3
"""Build the classified-negatives worker pools, minus everything already excluded.

Usage:
    sf3_negpool --negatives config/near_match_negatives.tsv \\
                --regions config/regions.tsv --extents config/function_extents.tsv \\
                --exe 'extracted/SCUS_946.40;1' \\
                --exclude 0x8005DEF8 --exclude 0x800F3160 ... \\
                --out-dir .run/p12 --prefix negatives [--workers 4]

*** THE ROOT CAUSE OF EVERY POOL DEFECT: I NEVER READ THE INDEX'S SCHEMA ***

The tracked index documents itself on line 2:

    # Columns: address<TAB>size<TAB>status<TAB>class.
    # status: near-match | blocked | deferred. class: '-' where not confidently stated.

Phase 12 re-dispatched every worker onto the ~193-row negatives index, because that is where the
measured bodies were (the per-row reading cost is pre-paid: an earlier session already named the
mechanism). The first pools were generated by a one-off script that read `address`, `size` and
`class` and **silently skipped `status`**, and that knew nothing about two collaborator lists.

Four defects came from that, **all four shipped to the workers, and all four were found by them**:

  1. `status=blocked` (5 rows) and `blocked,deferred` (1) are rows **charter section 7 forbids
     attempting** -- trapping arithmetic, the `0x80012xxx` primitive-init family, the maspsx
     rare-epilogue mutual exclusion. **Six forbidden rows were dispatched at prio 1**, the top of the
     list. Worker C found three of them in its own pool and refused them. Now dropped outright.
  2. The Makefile's `NAMED_EXCLUSIONS` (12 rows) was invisible here: **8 of the 12 leaked back** as
     fresh work -- `$gp`-switch thunk halves, a fragment, false extent starts. Worker A found two and
     asked whether they should be filtered. Eight were leaking, not two.
  3. The workers' own `negatives.tsv` staging was invisible: **94 of the 115 rows they had already
     classified were served as fresh work**, sitting at the TOP of the prio-1 order. Worker C:
     *"they are the first four rows in the file's own prio-1 order, so a worker starting at the top
     spends its first hours re-deriving my floor."*  4. `named = class not in ('-', '')` is a test for
     a NON-EMPTY STRING, not for a mechanism. Worker D's staging labels 56 of its 92 rows
     **`no-mechanism-yet`** and 6 more `no-extent`, so **62 rows were ranked prio 1, "cheapest,
     mechanism already named", when their label says the exact opposite.** That is why worker A's
     pool led with rows it could not close.

A coordinator aside that belongs in the record: I had already "fixed" one instance of defect 2 by
hand, adding `--exclude 0x8001DC20` to the `worklist` target -- and **the counter caught it**:
`excluded_named_exclusion` did not move, because that row was already excluded from the worklist
under a different rule, and it had reached the worker through the *negatives pool* instead. Fixing
one row by hand while the generator leaked was treating a symptom, and the tool said so.

This is a **schema error, not a logic error**, and it is worth naming as such: five separate bugs all
trace to generating a dispatch file from a table whose header I had not read.

*** THE FIVE FILTERS, AND WHERE EACH COMES FROM ***

1. `status=blocked` -> dropped. Charter section 7.  `status=deferred` -> **prio 4**, last but
   visible.
2. `--exclude` -> the Makefile's `NAMED_EXCLUSIONS`, which is the single source of truth and is also
   handed to the `worklist` target, so the two can no longer disagree.
3. `--classified` -> the workers' staging. See the policy section below.
4. `names_a_mechanism()` -> absence tokens are not mechanisms.
5. Already-registered rows -> dropped, by exact start address (a registered region's EXCLUSIVE END
   must not count; a naive `grep` reports a free address as taken, which worker B had to check by hand
   on `0x800AC9D8`).

*** MEMBERSHIP IS COMPUTED BEFORE THE FILTER, DELIBERATELY ***

Buckets are assigned over the **unregistered negatives rows**, and *then* excluded rows are dropped
from whichever bucket they landed in. This is not an accident:

- If the filter ran first, every exclusion would shift the split and **move unrelated rows between
  workers**. Workers hold a pool file, remember their position in it, and have claimed `wip` rows
  from it; moving their rows underneath them invalidates that. I have already made
  partition-membership mistakes twice this phase by re-cutting a split.
- Filtering after the split removes only the excluded rows. **Every other row stays exactly where it
  was**, so a re-cut is safe to hand to a worker mid-flight.

*** PRIORITY **

`prio 1` a named mechanism is present -> cheapest, and cookbook 182 says such a negative is
overturnable. `prio 2` the body has >= 2 `jr $31`, which is the gate's precondition for naming an
alternative cc1 -- written with `--exclude`-style care because the gate, not this tool, is what
enforces it. `prio 3` nothing named yet. `prio 4` the index marks it `deferred`. Within a priority:
more exit jumps first, then smaller.

*** "no-mechanism-yet" IS NOT A MECHANISM (defect 4, by rule) ***

The rule is `names_a_mechanism()` below: a class is a mechanism only if it is non-empty, is not `-`,
and does not start with `no-` (nor equal `fragment-suspect`, `none`, `blocked`). It is derived from the
labels' own vocabulary rather than from a list of known-good mechanism names, so it will not rot as
workers invent new class tokens.

*** ROWS CLASSIFIED IN THIS PHASE: KEPT ONLY IF THEY NAME A MECHANISM ***

Pass each worker's `negatives.tsv` staging with `--classified`. The default policy is driven by a
measured result rather than by tidiness:

- A row classified this phase that **names a mechanism** is **KEPT at prio 1**. This is worker D's
  measured 4-bodies-from-4-attempts-at-one-spelling case: Goal B classified the rows, and the bodies
  then came from *retrying* them. The classification is what makes the retry cheap, so throwing these
  away would discard the phase's best material.
- A row classified this phase with **no mechanism** (`no-mechanism-yet`, `no-extent`) is **DROPPED**:
  a worker already read it and found nothing, so a second worker re-deriving it is duplication.
  Worker C asked for exactly this, in its own words: *"treat them as done, not as fresh rows"*.

`--keep-classified-no-mechanism` restores the old behaviour if a deliberate re-sweep is ever wanted.
"""

import argparse
import re
import sys
import zlib
from collections import Counter
from pathlib import Path

EXIT_INSN = 0x03E00008  # `jr $31`, the only real return encoding
EXE_BASE = 0x800
EXE_VADDR = 0x80010000
LETTERS = "abcdefgh"

# A class is a MECHANISM only if it is not one of these. See the module docstring: treating
# `no-mechanism-yet` as a named mechanism ranked 62 rows backwards.
ABSENCE_PREFIXES = ("no-", "none", "blocked", "fragment", "deferred", "unmatched")
ABSENCE_TOKENS = {"", "-", "?", "none", "null", "n/a", "na", "unknown", "tbd"}
# Staging rows carry columns that are NOT class labels -- counts, flags, statuses. A cell is only
# evidence of a mechanism if it could plausibly BE a class label, so these are rejected outright.
# (Without this, worker D's row `0x80101C5C 32 1 0 None gte-cop2` had its `1` accepted as a
# mechanism, the test became vacuous, and 0 rows were dropped instead of ~62.)
NON_CLASS_TOKENS = {
    "match", "matched", "near-match", "nearmatch", "negative", "neg", "ok", "true", "false",
    "none", "null", "yes", "no", "y", "n", "ghi", "-1",
}

# CHARTER SECTION 7's blocked families, by the wording the CHARTER uses.
#
# `status=blocked` in the index marks SOME of these but not all: `0x80012A48` records status
# `near-match` with class `primitive-init scheduler-bound family`, and three `rare-epilogue` rows
# record `near-match` too. So the status column alone lets a forbidden row through, which is how
# `0x80012A48` reached prio 1 in worker C's pool.
#
# Deliberately NOT included: `rare-epilogue-ORDER`. Worker A drew this line precisely -- "the class
# is the *epilogue order*, not the maspsx rare-epilogue *mutual exclusion*" -- and cookbook
# 140/147/165 shipped `maspsx=epilogue` for exactly that shape, so those rows are legitimate work.
# Only the mutual-EXCLUSION variant is blocked, and no index row is classed with it by that name.
BLOCKED_CLASS_SUBSTRINGS = ("trapping-arithmetic", "primitive-init")

# *** A CLASS STRING CAN NAME ITS FAMILY BY ADDRESS INSTEAD OF BY KEYWORD ***
#
# Worker D found the gap: `0x80012A98` is charter-blocked (the `0x80012xxx` primitive-init family)
# but its class reads
#
#     cc1-scheduler-bound (constant stores before stack-arg loads; 2 coordinator spellings incl.
#     named locals, 72B both; same family as 0x80012A10/0x80012AE0)
#
# -- it names the blocked family **by address**, so the keyword substring test passes it straight
# through to prio 1. This is the same "half-right class string" pattern the workers kept hitting.
#
# First fix: read the ADDRESSES out of the class text and block the row if any of them is itself
# excluded. That catches `0x80012A98`, which names its family as "same family as
# 0x80012A10/0x80012AE0".
#
# **THAT WAS NOT ENOUGH, AND WORKER D FOUND THE SECOND LEAK: `0x80012D54`, whose class is `-`.**
# There is no class text to parse, so no text rule can ever catch it -- and D read its BYTES and
# confirmed it IS the family: `li v0,4 / sh v0,6(a0) / lui v0,0x400 / sw v0,8(a0) / lui v0,0x5000 /
# sw zero,0(a0)`.
#
# So the band itself is blocked, which is what the CHARTER actually says: it names the family as
# **"the `0x80012xxx` primitive-init family"**, i.e. BY ADDRESS BAND. I first declined this because
# band-blocking would discard unclassified rows "on an inference the charter does not make" -- and
# that was backwards: the charter makes exactly that inference, and it had already been acted on,
# since `0x80012A10`, `0x80012AE0`, `0x80012B20` and `0x80012CFC` (all class `-`) are excluded BY NAME
# in the Makefile. **Two workers found the leak twice and the second instance had no text to match on,
# which is the evidence that settled it.** Blocked rows are still REPORTED with their reason.
FAMILY_ADDRESS_RE = re.compile(r"0[xX][0-9A-Fa-f]{6,8}")
BLOCKED_ADDRESS_BANDS = (
    # The charter's own wording. See the comment above for why the band, not a string.
    (0x80012000, 0x80013000, "charter s7: the `0x80012xxx` primitive-init family"),
)

# *** THE TRIAGE KEY WAS IMPLEMENTED, TESTED AGAINST THE PHASE'S OWN RECORD, AND IT FAILED ***
#
# Three workers converged on the same split from three directions in one hour:
#
#   D, from its scoreboard: "what matters is whether the row has been attempted WITH THE CURRENT
#     LEVER SET, not which pool it is in."
#   C, from 11 classifications: "prio-1 does not distinguish 'a name that implies a SOURCE SHAPE'
#     from 'a name that implies a COMPILER BEHAVIOUR'."
#   A, from 14 rows touched: "what predicts closability is the wording of the row's own note, not
#     its prio", and a grep would separate them in one pass.
#
# The INSIGHT is right and is kept: some residuals are compiler behaviours that no spelling reaches.
# The DISCRIMINATOR they proposed -- the note's wording -- is NOT predictive, and that is measured
# rather than argued. I classified every row A and C named by outcome against the pool's own record:
#
#   MATCHED (8 rows): 0x8010036C, 0x800690E4, 0x80018284, 0x80094370, 0x80065494, 0x800C3514,
#                     0x80022E44, 0x8002DF1C
#     -> source-shape 2 | unknown 2 | **residual-class 4**
#   RESISTED (7 rows): 0x8010400C, 0x800A6C34, 0x80107CCC, 0x80050674, 0x80023D40, 0x80024630,
#                     0x80101E50
#     -> source-shape 1 | **unknown 3** | residual-class 3
#
# Precision of "residual-class => will resist": **3/7, worse than a coin flip**, and the rule would
# have DEPRIORITISED FOUR OF THE EIGHT ACTUAL SUCCESSES. The decisive examples are C's own and B's
# own: `0x800C3514` is classed `alloc+layout (...)` and MATCHED (the lever was arity), and
# `0x8002DF1C` is classed `candidate_bytes=104 = CORRECT length, 58 differing bytes` and MATCHED (the
# lever was a `goto` into the body). Both would have been sent to the back of the queue.
#
# So the key is DISARMED as an ordering rule and retained only as a LABEL: `kind` is written into the
# pool file as a stated hypothesis a worker can read and overrule, and it does NOT participate in the
# sort. This is the same shape of error as defect 5 of this tool (`no-mechanism-yet` read as a named
# mechanism, 62 rows inverted) -- a text heuristic over free prose -- and the difference this time is
# only that it was TESTED BEFORE BEING BELIEVED. The real discriminator is whether a residual has
# been LOCALISED to an allocation or scheduling decision, and that requires reading the bytes, not
# the note. Free prose is not evidence about closability.
SOURCE_SHAPE_WORDS = (
    "goto", "epilogue", "token", "twin", "struct", "arity", "volatile", "binding",
    "named local", "literal", "cast", "comma", "early return", "fall through", "fallthrough",
    "sf3_family", "maspsx=", "gp=-", "statement order", "loop shape", "do/while", "indexed",
    "argument", "pass-through", "passthrough", "5th param", "sixth param", "prototype",
)
RESIDUAL_CLASS_WORDS = (
    "hoist", "order (`", "operand order", "position", "scheduling", "scheduler", "layout",
    "allocator", "alloc", "tie-break", "tiebreak", "tie", "correct length", "differing",
    "rematerialis", "rematerializ", "reorg", "cse", "delay slot", "prologue", "epilogue order",
    "permutation", "rotation", "shuffle", "strength-reduce", "fold", "register field",
    "register fields", "allocation", "block order", "no untried", "exhausted", "toolchain",
)


def triage_kind(klass):
    """'source-shape' | 'residual-class' | 'unknown' -- a LABEL ONLY, never an ordering rule.

    Tests against the phase's own record gave this 3/7 precision and showed it would demote four of
    the eight real successes (see the block comment above). It is written into the pool file as a
    stated hypothesis a worker may read and overrule, and it is deliberately absent from the sort
    key. Do not re-enable it without a measurement that beats this one.
    """
    k = (klass or "").lower()
    if not k.strip().strip("-"):
        return "unknown"
    # A residual-class word is a stronger signal than a source word, because these notes routinely
    # CONTAIN both: "hoist ... the untried lever is a source order that stops the hoist" names a
    # lever that was tried and failed, so the presence of a residual class means a previous session
    # already reached that floor. That ordering is worker A's, verbatim.
    if any(w in k for w in RESIDUAL_CLASS_WORDS):
        return "residual-class"
    if any(w in k for w in SOURCE_SHAPE_WORDS):
        return "source-shape"
    return "unknown"


KIND_RANK = {"source-shape": 0, "unknown": 1, "residual-class": 2}


def names_a_mechanism(cell):
    """True iff this CELL could be a class label naming a mechanism.

    Deliberately conservative, and derived from the labels' own shape rather than from a list of
    known-good mechanism names, so it will not rot as workers invent new class tokens:

    * a NEGATION or an empty/placeholder cell means "nothing was found here" -- the opposite of a
      lead. `no-mechanism-yet`, `no-extent`, `fragment-suspect`, `-`, `None`.
    * a NUMBER or a flag (`32`, `1`, `0`) is a dashboard column, not a class. Worker D's staging is
      `address size exits jal frame class`, so four of its six columns are numeric.
    * a bare STATUS word (`negative`, `MATCH`, `near-match`) says how the attempt ended, not what
      mechanism was found.
    * a row of PROSE that names a lever DOES count -- that is how workers A and C write their class.
      Prose is accepted on the same footing as a token: both must contain letters.

    `gp-relative-access`, `gte-cop2`, `trapping-addi`, `multi-exit-2`,
    `twin-of-registered(sibling <==, 1/18 words differ)` and
    `cc1 `combine` folds the copy+negate` all pass. `no-mechanism-yet` and `32` do not.
    """
    k = (cell or "").strip()
    if k.lower() in ABSENCE_TOKENS or k.lower() in NON_CLASS_TOKENS:
        return False
    if k.lower().startswith(ABSENCE_PREFIXES):
        return False
    # Numbers, signed or not, in any base a worker might write.
    try:
        int(k, 0)
        return False
    except ValueError:
        pass
    try:
        float(k)
        return False
    except ValueError:
        pass
    # A class label contains letters. A bare flag or punctuation does not.
    return any(ch.isalpha() for ch in k)


def die(msg):
    print(f"sf3_negpool: {msg}", file=sys.stderr)
    raise SystemExit(2)


def read_tsv(path, what):
    p = Path(path)
    if not p.is_file():
        die(f"{what} not found: {path}")
    rows = []
    for line in p.read_text().splitlines():
        if line.startswith("#") or not line.strip():
            continue
        rows.append(line.split("\t"))
    return rows


def load_exe(path):
    p = Path(path)
    if not p.is_file():
        die(f"executable not found: {path}")
    return p.read_bytes()


def load_extents(path):
    ext = {}
    for f in read_tsv(path, "function extents"):
        try:
            ext[int(f[0], 16)] = int(f[1], 16)
        except (ValueError, IndexError):
            continue
    return ext


def load_registered(path):
    """Registry addresses as a set. A row is 'already registered' iff its START address matches;
    containment is not the test -- a registered region's exclusive end must not count."""
    return {int(f[0], 16) for f in read_tsv(path, "regions registry") if f}


def load_classified(paths):
    """Rows a worker classified THIS PHASE, mapped to {'workers': {...}, 'class': str}.

    Tolerant of the workers' differing staging formats on purpose: the address is always column 0,
    but the class lives at column 5 (worker D) or is free text (workers A and C). Any non-empty
    column that names a mechanism is accepted; absence detection happens in names_a_mechanism().
    """
    out = {}
    for path in paths:
        p = Path(path)
        if not p.is_file():
            die(f"classified staging not found: {path}")
        worker = p.parent.name
        for f in read_tsv(path, f"classified staging {path}"):
            try:
                addr = int(f[0], 16)
            except (ValueError, IndexError):
                continue
            rec = out.setdefault(addr, {"workers": set(), "classes": []})
            rec["workers"].add(worker)
            for cell in f[2:]:
                rec["classes"].append(cell)
    for rec in out.values():
        # The row names a mechanism if ANY of its free-text cells does. Worker C writes its class
        # into a prose sentence and worker D into a token column, so this is the only robust test --
        # but it must reject dashboard columns, or every row with a numeric cell 'names' one.
        rec["mechanism"] = any(names_a_mechanism(c) for c in rec["classes"])
        rec["classes"] = [c.strip() for c in rec["classes"] if c.strip()]
    return out


def exit_jumps(exe, ext, addr):
    """Count `jr $31` inside the extent. Returns 0 when the extent is unknown."""
    end = ext.get(addr)
    if end is None or end <= addr:
        return 0
    off = EXE_BASE + (addr - EXE_VADDR)
    return sum(
        1
        for i in range((end - addr) // 4)
        if int.from_bytes(exe[off + 4 * i: off + 4 * i + 4], "little") == EXIT_INSN
    )


def main():
    ap = argparse.ArgumentParser(description="Build classified-negatives worker pools.")
    ap.add_argument("--negatives", required=True)
    ap.add_argument("--regions", required=True)
    ap.add_argument("--extents", required=True)
    ap.add_argument("--exe", required=True)
    ap.add_argument("--exclude", action="append", default=[],
                    help="addresses to keep OUT of the pools (repeatable); "
                         "pass every one the worklist excludes")
    ap.add_argument("--classified", action="append", default=[],
                    help="a worker's negatives.tsv staging (repeatable). Rows classified this "
                         "phase are kept at prio 1 when they name a mechanism and DROPPED when "
                         "they do not -- see the module docstring")
    ap.add_argument("--keep-classified-no-mechanism", action="store_true",
                    help="do not drop rows a worker already classified with no mechanism")
    ap.add_argument("--keep-own-classified", action="store_true",
                    help="do not drop a row from the very worker that classified it")
    ap.add_argument("--out-dir", required=True)
    ap.add_argument("--prefix", default="negatives")
    ap.add_argument("--workers", type=int, default=4)
    ap.add_argument("--quiet", action="store_true")
    args = ap.parse_args()

    if not 1 <= args.workers <= len(LETTERS):
        die(f"--workers must be 1..{len(LETTERS)}")

    try:
        excluded = {int(x, 16) for x in args.exclude}
    except ValueError as exc:
        die(f"--exclude wants addresses like 0x800C3490: {exc}")

    exe = load_exe(args.exe)
    ext = load_extents(args.extents)
    registered = load_registered(args.regions)
    classified = load_classified(args.classified) if args.classified else {}

    # ---- read the negatives index -------------------------------------------------
    # Schema, from the file's own header: address<TAB>size<TAB>status<TAB>class.
    # `status` is READ HERE and was the omission behind six forbidden rows reaching the top of the
    # dispatch order. `blocked` is dropped; `deferred` is ranked last.
    rows, dupes = [], set()
    for f in read_tsv(args.negatives, "negatives index"):
        try:
            addr = int(f[0], 16)
        except (ValueError, IndexError):
            continue
        size = int(f[1]) if len(f) > 1 and f[1].isdigit() else None
        status = f[2].strip().lower() if len(f) > 2 else ""
        klass = f[3] if len(f) > 3 else "-"
        if addr in dupes:
            continue
        dupes.add(addr)
        rows.append((addr, size, status, klass))

    # ---- drop what is already matched --------------------------------------------
    unregistered = [r for r in rows if r[0] not in registered]
    already = len(rows) - len(unregistered)

    # Addresses the charter blocks or the Makefile excludes. A row whose class text NAMES one of
    # these is referencing a blocked family, so it is blocked too (see FAMILY_ADDRESS_RE).
    blocked_addrs = {r[0] for r in unregistered if "blocked" in r[2]}

    # ---- MEMBERSHIP FIRST (see the module docstring), then the filter -------------
    names = LETTERS[: args.workers]
    buckets = {n: [] for n in names}
    leaked = []
    dropped_done = []
    dropped_blocked = []
    dropped_own = []
    for addr, size, status, klass in unregistered:
        bucket = names[zlib.crc32(f"{addr:#010x}".encode()) % args.workers]
        # CHARTER SECTION 7. A blocked class must not be attempted at all, so it must not be
        # dispatched either -- six of these went out at prio 1 before `status` was read, and a
        # seventh (`0x80012A48`) arrived by CLASS with status `near-match`.
        blob = f"{status} {klass}".lower()
        # A family reference BY ADDRESS counts exactly like a keyword. D's `0x80012A98` case.
        family_refs = {int(x, 16) for x in FAMILY_ADDRESS_RE.findall(klass or "")}
        band = next((reason for lo, hi, reason in BLOCKED_ADDRESS_BANDS if lo <= addr < hi), None)
        if ("blocked" in status
                or band is not None
                or any(s in blob for s in BLOCKED_CLASS_SUBSTRINGS)
                or (family_refs & (excluded | blocked_addrs))):
            dropped_blocked.append((addr, bucket, status, klass))
            continue
        if addr in excluded:
            leaked.append((addr, bucket))
            continue
        c = classified.get(addr)
        if c is not None and not c["mechanism"] and not args.keep_classified_no_mechanism:
            dropped_done.append((addr, bucket, ",".join(sorted(c["workers"]))))
            continue
        # A worker's OWN classification must not come back to that worker: it already read the row,
        # derived the structure and reached a floor, so re-serving it is re-deriving a known answer.
        # Worker C reported exactly this -- "they are the first four rows in the file's own prio-1
        # order, so a worker starting at the top spends its first hours re-deriving my floor".
        # The row is still worth dispatching to a DIFFERENT worker, who can use the recorded
        # mechanism as a lead; only the owning worker is spared it.
        if c is not None and f"w-{bucket}" in c["workers"] and not args.keep_own_classified:
            dropped_own.append((addr, bucket))
            continue
        n = exit_jumps(exe, ext, addr)
        # A class that NAMES a mechanism ranks first; a negation does not. A row a worker already
        # classified this phase is kept here precisely because the classification is the lead.
        prio = 1 if names_a_mechanism(klass) else 3
        if c is not None and c["mechanism"]:
            prio = 1
        if n >= 2:
            prio = min(prio, 2)
        if "deferred" in status:
            prio = 4
        # A NOTE column, because free-text staging cannot be parsed reliably enough to decide
        # "done" vs "cheap retry" in every case. Where the tool CAN decide it drops the row (the
        # absence-token case, above); where it cannot, it says who classified it and lets the worker
        # read one label instead of re-deriving a floor. A `INFLIGHT:wip:<worker>` note means the
        # row is held in the ledger and must not be claimed again.
        note = "index-only"
        if c is not None:
            who = ",".join(sorted(w.replace("w-", "") for w in c["workers"]))
            note = f"classified-by:{who}"
        # The three-worker triage key, advisory. See triage_kind().
        kind = triage_kind(klass)
        buckets[bucket].append((prio, n, addr, size, klass, note, kind))

    for b in buckets.values():
        # prio, then more exit jumps, then smaller, then address. **`kind` is deliberately NOT in
        # this key**: it was tested against the phase record and got 3/7, demoting half the real
        # successes. It is a label in the output, not a sort.
        b.sort(key=lambda r: (r[0], -r[1], r[3] if r[3] else 10 ** 9, r[2]))

    # ---- write --------------------------------------------------------------------
    out = Path(args.out_dir)
    out.mkdir(parents=True, exist_ok=True)
    written = {}
    for n in names:
        path = out / f"{args.prefix}-{n}-pool.tsv"
        with path.open("w") as fh:
            fh.write("# Phase 12 classified-negatives pool. Generated by tools/sf3_negpool.\n")
            fh.write("# Ordered: prio1 = named mechanism (cheapest), prio2 = multi-exit (the\n")
            fh.write("# `cc1bin` class), prio3 = no mechanism yet. Within a prio: more exit\n")
            fh.write("# jumps first, then smaller. Generated MINUS the worklist's named\n")
            fh.write("# exclusions, which are the single source of truth in the Makefile.\n")
            fh.write("# columns: prio\texit_jumps\taddress\tsize\tclass\tnote\tkind\n")
            fh.write("# note: `index-only` = never worked this phase; `classified-by:XY` = a worker\n")
            fh.write("# stopped here already, so READ THEIR NOTE BEFORE SPENDING SPELLINGS.\n")
            fh.write("# kind: a STATED HYPOTHESIS about the note's wording, from worker A's triage key\n")
            fh.write("# and corroborated in spirit by D and C. **IT IS NOT USED FOR ORDERING** -- it was\n")
            fh.write("# tested against this phase's own record, got 3/7 precision, and would have\n")
            fh.write("# demoted four of the eight actual successes (0x800C3514 is classed `alloc+layout`\n")
            fh.write("# and MATCHED; 0x8002DF1C is classed `correct length, 58 differing` and MATCHED).\n")
            fh.write("# It is here so you can read it and overrule it. The real discriminator is whether a\n")
            fh.write("# residual is LOCALISED to an allocation or scheduling decision, and that needs the\n")
            fh.write("# bytes, not the note.\n")
            for prio, n_exits, addr, size, klass, note, kind in buckets[n]:
                fh.write(f"{prio}\t{n_exits}\t{addr:#010x}\t{size if size else '-'}\t{klass}\t{note}\t{kind}\n")
        written[n] = len(buckets[n])

    if not args.quiet:
        print(f"negatives index rows         {len(rows)}")
        print(f"  already registered         {already}   (dropped)")
        print(f"  unregistered               {len(unregistered)}")
        print(f"  filtered as excluded       {len(leaked)}")
        for addr, bucket in sorted(leaked):
            print(f"      dropped {addr:#010x} (would have been in pool {bucket})")
        print(f"  dropped, CHARTER-BLOCKED class (section 7)                  {len(dropped_blocked)}")
        for addr, bucket, status, klass in sorted(dropped_blocked):
            print(f"      dropped {addr:#010x} (was pool {bucket}; status={status} class={klass[:60]})")
        print(f"  dropped, already classified this phase with NO mechanism   {len(dropped_done)}")
        for addr, bucket, who in sorted(dropped_done)[:5]:
            print(f"      dropped {addr:#010x} (was pool {bucket}; classified by {who})")
        if len(dropped_done) > 5:
            print(f"      ... and {len(dropped_done) - 5} more")
        print(f"  dropped, this bucket's OWN prior classification          {len(dropped_own)}")
        print(f"pool rows written            {sum(written.values())}"
              f"  (check: {len(unregistered) - len(leaked) - len(dropped_done) - len(dropped_blocked) - len(dropped_own)})")
        for n in names:
            c = Counter(r[0] for r in buckets[n])
            k = Counter(r[6] for r in buckets[n])
            print(f"  {args.prefix}-{n}-pool.tsv  n={written[n]:3d}"
                  f"  prio1={c[1]:3d} prio2={c[2]:3d} prio3={c[3]:3d} prio4={c[4]:3d}"
                  f"  | kind: source-shape={k['source-shape']:3d} unknown={k['unknown']:3d}"
                  f" residual-class={k['residual-class']:3d}")

    # A leak here means the generator and the caller disagree; fail loudly rather than
    # hand a worker a row that was excluded on purpose.
    if leaked:
        print(f"sf3_negpool: WARNING {len(leaked)} excluded row(s) found in the negatives index "
              f"and filtered out -- the index and the exclusion list overlap; that is expected "
              f"but should be reviewed.", file=sys.stderr)
    return 0


if __name__ == "__main__":
    sys.exit(main())
