mirror/rea
1
0
Fork 0
mirror of https://github.com/morluto/rea synced 2026-10-25 22:08:11 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(analysis): add opt-in retained inventory and WASM views (#2122)

Browse Source 
Add opt-in retained inventory and WASM Evidence views with exact nested output contracts and a small static WASM descriptor. Preserve old complete CLI/MCP workflows, producer choices, original authenticated Evidence and cleanup ownership.

Validated with real public applications, genuine old Evidence migration, 123 focused assertions, 11 synchronized description checks and 42 installed-package WABT cases. Detailed generated experiment JSON stays outside the final tree; concise findings and reproduction inputs remain. Main-existing completion/package CI failures are separately attributed and left unchanged.

Refs #2016

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
Ysopwnable
2026-10-12 21:17:52 +08:00
committed by GitHub
parent 3159732597
commit 58fb372071
55 changed files with 4322 additions and 235 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/reverse-engineer-anything/references/evidence-workflows.md
+13
View File
@@ -11,6 +11,19 @@ returned inline. When complete `inspect_binary_layout` or
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
page instead of repeating the producer.
Completed `inspect_artifact` inventory also supports a summary and
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
Omit `logical_paths` to page all available occurrences; `[]` selects none.
Other inventory collections are `nodes`, `edges` and
`integrity_contradictions`. Keep `source_complete` separate from page exhaustion:
a missing match in partial source Evidence does not establish absence. Use the
returned parent Evidence ID for subsequent views and export before closing the
session when the source must survive it.
When `tools/list` advertises `inspect_artifact.detail`, request `"summary"` to
retain its complete source and receive that view immediately. Without that
advertised choice, keep the existing complete call. CLI inspection returns
portable complete Evidence; save it before projecting a selected inline view.
Use `record_unknown` to track unresolved questions in the user's investigation
and name the authority or environment still required. Supply supporting and
contradicting evidence IDs. It updates the session registry directly.
.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md
+10
View File
@@ -36,6 +36,16 @@ retained, call `inspect_analysis_view` for a summary, mitigations or linkage
facet, one section or symbol, or a stable page. Do not re-run the decoder to
read a single object.
## Offline WASM
Use `inspect_wasm_artifact` with caller-supplied WABT for selected local bytes.
For a completed result, use `inspect_analysis_view` with its reported retained
reference for a summary or `view: {kind: "wasm", facet: "exports", offset: 0, limit: 32}`.
Facets also include sections, imports, candidates, glue, WAT, headers, details
and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
The complete source remains retained after an oversized MCP reply. No target
execution or network acquisition is implied, and WAT is decoded representation.
## Native targets
After `open_binary`, use focused search, procedure, or function tools directly.
.gitignore
+1
View File
@@ -47,6 +47,7 @@ Thumbs.db
# Local reference sources and analysis artifacts
/_reference/
/experiments/analysis-efficiency/results/
# Generated website download
/website/public/examples/notes-example.zip
docs/architecture.mermaid
+5 -1
View File
@@ -6,7 +6,7 @@ graph TD
end
subgraph "Production Composition"
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading"]
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM descriptor: named contract, factory, retryable lifetime"]
end
subgraph "Adapters"
@@ -27,6 +27,7 @@ graph TD
subgraph "Provider Layers"
PWNTOOLS["src/native/pwntools/ + src/native/pwndbg/<br/>BYO offline ELF/core adapters<br/>owned snapshots/Python and optional core-only GDB<br/>unchanged upstream parsers, ABI structures and mapping API"]
WASM["src/wasm/<br/>caller-supplied WABT; immutable artifact inspection<br/>owned process supervision and retryable workspace cleanup"]
EVM["src/evm/<br/>unchanged EVMole/WASM in bounded owned worker<br/>explicit carrier parsing and inferred interface adapter"]
HOPPER["src/hopper/<br/>Hopper launch & Unix-socket protocol<br/>BridgeLauncher, HopperClient, protocol"]
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
@@ -94,6 +95,9 @@ graph TD
APP -->|record callbacks| RECORDS
SERVER -->|narrow record ports| RECORDS
RECORDS --> DOMAIN
COMPOSITION --> WASM
WASM --> PROCESS
WASM --> DOMAIN
COMPOSITION --> PWNTOOLS
PWNTOOLS --> PROCESS
PWNTOOLS --> DOMAIN
docs/mcp-contracts.md
+36 -1
View File
@@ -412,7 +412,7 @@ Ordinary responses keep their existing complete result contract.
## Retained application Evidence inputs
`analyze_javascript_application` and `inspect_binary_layout` accept `detail`:
`analyze_javascript_application`, `inspect_binary_layout`, and `inspect_artifact` accept `detail`:
`complete` (default) returns the complete analysis Evidence; `summary` records
that complete Evidence in the session and returns the derived
`inspect_analysis_view` summary Evidence instead. Its
@@ -420,6 +420,9 @@ that complete Evidence in the session and returns the derived
analysis, so later views and application workflows read it without repeating
analysis or moving the complete record through the response. A server without
session retention refuses summary detail with `capability_unavailable`.
For `inspect_artifact`, omission also keeps the original request parameters and
complete output shape. Delivery selection never reaches the inventory engine;
the complete inspection and its atomic inventory Evidence remain retained.
`inspect_analysis_view` also accepts `api-surface` for retained HAR capture Evidence,
grouping request lines by method, origin and format-rule path template. Templated
@@ -446,6 +449,34 @@ application and semantic coverage. The result carries a digest of the selected
facts. Actual serialized size determines MCP transport admission: reduce the
page size or export retained Evidence if the view is too large.
Completed `inspect_artifact` and atomic `inventory_artifact` Evidence also
support summary and inventory views. A summary reports the manifest's expected
counts separately from available record counts, observed entry kinds, inventory
Evidence IDs and inspection coverage. An inventory view selects `nodes`,
`occurrences`, `edges`, or `integrity_contradictions`, with `offset` and `limit`
defaulting to 0 and 64. For occurrences, optional `logical_paths` matches exact
recorded strings: omission selects all available records; `[]` selects none.
These selectors do not reopen the artifact or resolve filesystem paths.
`source_complete` records whether the available inventory passes the complete
graph checks. Exhausting a page from a partial source does not establish that
unreturned paths are absent. Views carry their own Evidence IDs and preserve
links to the original parent; complete producer output and historical inline
Evidence input parsing retain their existing shapes.
Completed `inspect_wasm_artifact` Evidence supports a summary and
`view: {kind: "wasm", facet: "exports", offset: 0, limit: 64}`. Other facets
are `sections`, `imports`, `candidates`, `glue`, `wat`, `headers`, `details`, and
the singleton `tool_profile` (offset 0). Arrays retain original WABT forms and
records. Text facets page by UTF-16 code units and preserve complete Unicode
characters. Summaries report the selected artifact's identity, validation,
available counts, decoded WAT digest/length and exact tool profile. Runtime
execution and network acquisition remain `not-performed`. A valid WAT digest
is checked at the owning model boundary for current output and portable input.
The original producer still returns complete Evidence by default. When its
complete MCP reply exceeds the frame, use the exact reported retained
reference with these views, or export the complete source. Views do not run
WABT again and do not turn decoded WAT into original source.
`trace_application_feature`, `trace_javascript_semantics`,
`compare_application_versions`, `compare_source_to_bundle`, and
`compare_javascript_export_shapes` accept complete inline application Evidence
@@ -484,6 +515,10 @@ CLI application workflows continue to read portable inline Evidence from files
and run the same analysis workflows; a standalone CLI invocation cannot resolve
another MCP connection's retained records. No additional lookup call, provider
selection, or approval step is required for a same-session follow-up.
CLI `inspect-artifact` continues to return complete portable Evidence. Save that
JSON to a file and use `inspect-analysis-view` with inline source Evidence for
later inventory summaries or pages; a one-shot CLI session cannot retain a
parent for another invocation.
## Aggregate native context
experiments/analysis-efficiency/README.md
+540
View File
@@ -0,0 +1,540 @@
# Analysis efficiency measurements
Tracks the experiments for [#2016](https://github.com/morluto/rea/issues/2016).
The production CLI/MCP contracts and defaults remain the compatibility baseline.
Downloaded artifacts, runtime copies, exported Evidence, transcripts and detailed
JSON measurements belong outside the checkout. This document keeps the measured
findings, verification scope and revision references; the scripts and `corpus.json`
keep the reproduction inputs. Generated `results/` is ignored.
The 12 detailed checkpoint receipts were archived byte-for-byte before removal
from the final PR tree. The archive `review-receipts-00d51e0fc-20261012` is in the
private experiment lab outside the checkout; its `archive-index.json` records
source revision, filenames, sizes and SHA-256.
Raw producer replies, model transcripts and exported Evidence remain in the same
private lab. These local files are not required to build or use REA. Historical
checkpoint commits retain their original receipts; no history was rewritten.
## Optimization priorities and gradual migration
Agent token consumption and end-to-end task time are the primary objectives.
Memory and CPU consumption are also measured and controlled. Correctness,
Evidence provenance, caller choices and existing-user compatibility are
acceptance conditions for every optimization.
- Tokens: minimize unnecessary model-visible data while returning the requested
facts and useful provenance inline. Count discovery, repeated calls, errors,
retries and follow-up turns; report total/cached/uncached input and output
separately. Compact JSON bytes are a delivery measurement, not a token count.
- Time: measure completion of the same analyst question, including startup,
provider work, transport, client processing and model corrections. Separate
measured stages and cold/warm state; use repeated comparable trials before
generalizing an observed latency improvement.
- Memory: record aggregate process-tree peaks and limit/cleanup events, account
for retained Evidence and representation expansion, and keep host safeguards.
- CPU: record aggregate CPU time for the completed task. A one-CPU quota is an
experiment control; it does not itself establish reduced CPU consumption.
Migrate domain by domain with focused real-artifact checks and relevant shared
contract checks. The domain matrix is a growing roadmap; native PE and DOS, for
example, remain unverified in this study. A passed lane does not imply all
advertised platforms/providers have passed. Newly discovered REA bugs remain
in the separate local handoff document for later team repair.
Old clients using existing CLI/MCP requests against the candidate retain the
complete default path. New summary/view options require explicit selection.
Actual old Evidence import/view/export checks preserve original authenticated
records and identities. These checks establish the recorded baseline/workflow
scope, not every historical release or third-party client. A new client targeting
an older server must discover the actual schema and use a supported equivalent
where available; this study does not implement a universal automatic fallback.
The catalog intentionally gains more precise/additive schemas, so its bytes and
digest are not frozen: measured runtime `259e99dd9` has about 2.6% larger complete
catalog JSON than the frozen baseline. The separate post-integration receipt
records the newer runtime, including upstream changes. Optional views can reduce
model-visible result data; old complete requests do not silently switch delivery
or acquire that reduction automatically.
## Reproduce a measurement
Use the repository's pinned Node/npm and locked dependencies. Build each selected
revision, then copy its `dist/`, `bridge/`, `scripts/`, `skills/`, `package.json`
and `package-lock.json` to a separate runtime directory before editing production
sources. That copy needs the matching locked dependencies. The initial frozen
baseline is `42b0ed126df7d5b908ac369715378d5e519051b7`; this study does not
silently substitute a newer `main` or the previous Code Mode experiment.
The public artifacts and acquisition digests are in `corpus.json`. Supply their
actual local paths and an optional JSON object of explicit provider environment
settings. Do not add credentials to that file. Provider settings are inherited
by the child, but the report persists only the selected settings' digest. Runs
reject undeclared inherited REA/Hopper and Ghidra/Java configuration keys before
starting a measured server or creating a model profile; include those settings
explicitly in `--providers`. Both drivers use `provider-environment.json` and
record its digest. The system/runtime environment still needs comparable
controlled conditions; its contents are not serialized. Both drivers also
fingerprint the effective provider `PATH` without recording its value, and
the SDK comparator rejects different or unrecorded search paths. This detects different
search configuration, not replacement of executable bytes at the same path.
Historical receipts
predate this guard, so their selected-settings digest alone does not certify
that no undeclared provider setting was inherited.
```sh
export NODE_OPTIONS=--max-old-space-size=512
export UV_THREADPOOL_SIZE=1
export GOMAXPROCS=1
export GOMEMLIMIT=2GiB
python3 experiments/analysis-efficiency/resource-run.py -- \
node experiments/analysis-efficiency/measure.mjs \
--runtime-root /absolute/frozen-runtime \
--revision 42b0ed126df7d5b908ac369715378d5e519051b7 \
--artifact obsidian.asar --path /absolute/obsidian-1.14.4.asar \
--providers /absolute/selected-providers.json \
--output /absolute/new-measurement-directory
```
The wrapper requires Linux cgroup v2 and a working user systemd manager. It owns
a unique transient scope, restricts the whole process tree to one CPU, caps
memory at 4 GiB with no swap and 192 tasks, and samples memory/tasks/CPU every
100 ms. It stops its own scope above 3 GiB or below 4 GiB host available memory;
startup requires 8 GiB available. Cleanup failures retain the owned scope's
identity. Run heavy builds, providers and model trials serially. Sampled peaks
can miss short spikes; cgroup limit events are separately reported.
An actual Codex + Chromium trial exceeded the initial 192-task allowance and
was stopped automatically below 1 GiB sampled memory, with no OOM. The isolated
browser model retry uses `resource-run.py --task-ceiling 256 -- COMMAND...` to
account for both process trees; memory, swap and CPU controls are unchanged.
Keep the original aborted run and cleanup receipt. This is experiment capacity,
not a REA defect or a successful browser analysis.
## What the numbers mean
`summary.json` records the selected artifact's before/after SHA-256, runtime
file-table digest, dispatcher and lockfile digests, runner source digest,
Node/SDK versions, catalog sizes, caller arguments and producer Evidence IDs.
`runtime-files.json` identifies the exact compiled files measured. Actual
provider settings and runtime files must stay fixed during a comparison.
Every tool call measures three intervals at the public SDK Transport boundary:
- Client work before `send`, including first-use schema preparation.
- Transport/server roundtrip, including framing, I/O, server work and client
JSON parsing. This interval is **not** a Ghidra-only or provider-only timer.
- Client work after the parsed response reaches `onmessage`.
The SDK call's total excludes the runner's subsequent encoding and disk writes.
Result byte counts measure compact JSON of the parsed client result, not raw
wire bytes or tokens. Progress notifications retain their actual arrival times;
they do not imply more precise internal engine timings than the producer emits.
Scope CPU time and aggregate process-tree memory come from the wrapper's final
JSON line. Fresh processes have cold application caches; host page caches are
not globally flushed.
`--delivery summary` is an explicit candidate workflow, initially for artifact
inventory. It must be supported by the selected server's advertised schema.
The default `complete` runs the existing calls unchanged. Compare the same
analyst facts and complete source Evidence before crediting a smaller response.
Projection can reduce delivery bytes without reducing upstream analysis time.
For SQLite WASM, summary delivery reads the completed retained result through
the summary and exports views after preserving the original transport error.
`--javascript-question selected` analyzes only the exact Obsidian sources
`package.json` and `main.js`, keeps inventory identity, and reads each observed
`main.js` module by its exact node ID. Package metadata is not a module selector.
Compare paired runs with the same runner source and provider settings:
```sh
node experiments/analysis-efficiency/compare.mjs \
--left /absolute/complete-run --right /absolute/summary-run \
--output /absolute/new-comparison.json
```
This checks all selected analyst facts, including inventory source IDs, exact
path records, manifest, counts, completeness, contradictions, coverage,
unexplored branches and limitations. It excludes only delivery observation IDs
and the named recovery-path description. Different settings, changed facts or
failed workflows make the comparator fail. Binwalk's per-invocation region UUIDs
are separately preserved with both original values and JSON paths; only those
opaque lookup values are excluded from physical-region fact equality. Original
producer replies are never rewritten. Selected JavaScript comparisons likewise
preserve both values of the two named parent-heap-dependent byte budgets before
excluding their values from fact equality; fixed limits and coverage rows remain
checked. Counts include extra view calls and
observed tool errors. Use repeated runs with alternating order before claiming
a distribution or a speedup; one pair is only one observation.
For a completed WASM result exceeding the MCP response budget, the runner keeps
the original error, exports retained Evidence through the public tool, and
authenticates the exact reported record using the selected runtime's parser.
The report names this recovery; it does not relabel the oversized inline reply
as successful delivery.
## Initial observations and exclusions
The frozen baseline has 2,326,609 bytes of complete catalog JSON, of which
494,207 bytes describe names, descriptions and input schemas. One Obsidian
trial spent about 21.9 s before the first request was sent, then about 0.77 s
in that request's transport/server roundtrip. Other cold runs differed
substantially. These single observations identify a measurement boundary;
they do not establish a speedup or a distribution.
The initial matrix exercised shipped Obsidian, NewPipe, ShareX, OpenWrt, SQLite
WASM and jq artifacts. SQLite produced a complete result larger than the
transport frame and requires an explicit recovery run. A first jq attempt hit
the SDK's default 60 s timeout because the experimental runner used a superseded
three-argument call signature. SDK 2.3.1 accepts request options as argument two;
the corrected jq run completed real Ghidra analysis in about 265 s. That earlier
driver timeout is excluded from REA regression claims. Provisional runs with
different runner source digests are not one paired statistical cohort.
No cross-session engine database cache, default analyzer changes, external
plugin loading, or broad plugin SDK is enabled by this experiment.
## Obsidian delivery checkpoint
The two recorded pairs compare frozen baseline
`42b0ed12` and candidate `f1106181` in complete → summary and summary → complete
order using the same runner source, bytes and provider settings. Both preserve
every selected fact and the atomic inventory Evidence ID.
| Measurement | Complete | Summary + exact path view |
| --------------------------------------------------- | --------------: | ------------------------: |
| Tool-result compact JSON, including lifecycle calls | 937,404 bytes | 16,322 bytes |
| Tool calls per question | 3 | 4 |
| Complete catalog compact JSON | 2,326,609 bytes | 2,370,505 bytes |
| Workflow elapsed, first pair | 11.04 s | 10.23 s |
| Workflow elapsed, second pair | 10.57 s | 10.18 s |
Reply JSON falls by about 98.3%; catalog JSON grows by about 1.9%. The two elapsed
observations do not establish a statistical speedup. Inventory coverage and
upstream byte acquisition are unchanged. The candidate remains opt-in; existing
requests continue to deliver the complete inspection. Each run verifies the
selected artifact's before/after digest and reports no observed OOM or task
limit events. Source typing and projection improve selected delivery without
introducing an engine database cache or an external plugin protocol.
## Task-scoped analyzer experiment
The recorded jq pair explicitly disables
`Decompiler Switch Analysis` through the existing `REA_GHIDRA_DISABLED_ANALYZERS`
setting and compares it with the unchanged Ghidra default. Both use the same
frozen runtime and real jq bytes, one CPU, a fresh ephemeral database and the
same entry/main question. Completed main dossiers exceed the MCP frame in both
runs; existing retained native views recover the selected facts, preserving the
observed transport errors in the reports.
One disabled/default pair takes 174.77/276.23 seconds. Entry and main pseudocode,
main strings and its block count agree. However, callee `0x40daa0` has 604
recovered body bytes in eight ranges by default, and 164 bytes in four ranges
with the same 646-byte span after disabling switch analysis. Equal top-level
counts do not establish call-graph equivalence. **Keep the production default.**
Use the existing control only for a caller-selected task whose required facts
have been checked; no general fast preset is introduced. A separate real
coreutils/Ghidra probe rejects a nonexistent analyzer before bridge readiness,
preserves the named failure and closes the session. Source bytes remain unchanged.
`--schemas selected` is a separate SDK 2.3.1 client experiment. It passes the exact
live named `tools/list` definition to the SDK's public `callTool` option, retaining
output validation. It does not change REA or the default runner mode. Selected
definitions compile per call rather than preparing the entire cached catalog;
report this client effect separately from engine analysis or model behavior.
## Real Codex CLI trials
`agent-trial.py` accepts an explicit frozen runtime and revision, the same
question templates in `questions/`, and optional explicit provider settings.
It fingerprints the actual runtime, runs pinned local Codex CLI with a private
temporary profile, copies only the authentication file with mode 0600, disables
shell/web/multi-agent tools, and removes copied profile state afterward. Raw
events and answers remain outside Git. A zero exit code is not a factual grade.
Review the original tool calls, failed requests, result limitations and answer.
The optional `--enabled-tools FILE` supplies a list of exact names to the
temporary Codex MCP configuration's `enabled_tools`, as described in the
[official MCP configuration](https://learn.chatgpt.com/docs/extend/mcp?surface=cli).
It leaves the REA catalog and other clients intact. Initial Obsidian trials
showed parameter-correction attempts and different delivery choices even with
the same question; filtering alone has not established a cost or quality win.
Requested model aliases are recorded without inventing a resolved model identity.
The browser question uses `--authority browser` to permit only its explicitly
selected public scenario in a fresh, owned headless profile. Browser observations
and cleanup are graded separately from static artifact analysis.
## Real artifact compatibility checkpoint
Six frozen baseline/pilot pairs exercise
Obsidian, NewPipe, ShareX, OpenWrt, SQLite WASM and published historical WETH code
through public MCP tools. All selected facts agree under the stated comparison
rules. Obsidian complete delivery and atomic inventory Evidence identity remain
unchanged. ShareX native-boundary claims remain scoped to the selected DLL;
NewPipe retains a null DEX descriptor rather than inventing one.
SQLite's complete 15,382,128-code-unit WAT exceeds the MCP response budget in both
revisions. The control recovers complete Evidence through file export; the pilot
reads summary and four exact export forms inline, without running WABT again.
Inline byte totals exclude file exports, so they do not establish a size win.
OpenWrt comparison preserves both sets of upstream-generated region UUIDs before
comparing offsets, extents, signatures, coverage and limitations. WETH is a
published historical example; a live RPC request returned HTTP 403, so this is
not live-chain verification. Every pair records unchanged target bytes and zero
observed OOM/task-limit events. The Linux x64 results do not establish behavior
for IDA, Hopper, macOS, Windows or Harmony.
## Same-question model and selected-source follow-up
Two actual SQLite model trials preserve all
12 requested facts. The question, developer instruction, provider settings and
tool allowlist hashes match. A description-only follow-up supplies the exact
summary kind and its fields. Calls change from 42 to 3, workflow time from
292.64 to 63.79 seconds, and reported input tokens from 304,698 to 74,387. The
follow-up reads no unnecessary WAT pages. Both keep the original completed
analysis's transport error and authenticate the retained source/view identities.
This is one stochastic model pair, not a statistical or general causal claim.
The selected JavaScript pair preserves
all 24 observed `main.js` module identities and the exact two-source selection.
The strict comparison initially differs on parent budget values derived from
observed heap availability; both values and paths remain recorded separately.
Other coverage facts, per-family rows, fixed worker limits, omitted/unknown
counts, statistics and Electron observations agree. Coverage remains partial,
with 36 unselected relevant files and 384 semantic unknowns. These static facts
do not establish runtime behavior. The comparator does not rewrite Evidence.
Ten real CLI trial reviews cover native,
managed, Android, firmware, Electron/JavaScript, WASM, EVM and browser tasks.
The selected JavaScript model preserves core facts and partial coverage but
omits full budget/family provenance. The browser producer captures complete
DOM/accessibility/URL evidence and terminates its owned process; direct model
answers lose two accessibility metadata fields to client context clipping and
report them unknown. A requested under-development Codex code-mode configuration
still produces a direct MCP call in the actual trace, so no executor/composition
win is credited. Profile cleanup, original errors, actual calls, usage, facts and
scope limits remain independently reviewed. Enabling a configuration flag does
not prove that a capability was used.
Discovered REA defects are recorded separately for team repair at the user's
request. This follow-up performs no additional REA bug fixes.
## Nonempty managed boundaries and final verification
A second shipped managed library checks
11 ModuleRefs, 147 PInvoke declarations and 225 native implementations from
ShareX HelpersLib 19.0.2. All complete normalized artifact and native-boundary
fields match between baseline and pilot, including source Evidence identities.
This covers nonempty declarations beyond the main ShareX DLL's empty import
tables. It does not verify native exports, execute the assembly or resolve CLR
dependencies. The curated receipt keeps identities, coverage, all ModuleRefs,
the first three PInvoke records and digests of the complete boundary payloads.
Full producer tables and comparison inputs stay in the private lab outputs.
The earlier verification includes 42 real
WABT CLI/MCP cases and the complete real SQLite CLI JSON: the unchanged default
returns the same original source Evidence ID, normalized result and raw result
as the baseline exported MCP record. Source bytes are unchanged. The seven-file
contract/composition run passes 51/52 assertions and hits the existing 20-second
presentation-test deadline while sharing one CPU. That same unmodified assertion
passes alone in 10.835 seconds. The original combined failure and isolated pass
remain recorded; no test deadline or REA bug was patched. Build, type/lint/module
checks and generated-document checks pass. No full `check:pr` or unverified
provider/platform coverage is implied.
## Actual old Evidence migration
The migration experiment imports the original
baseline SQLite Evidence bundle into the pilot runtime, reads a typed summary
and four export forms, and exports the retained bundle again. The complete old
WASM record is deeply equal after import/export, with its original authenticated
Evidence ID. The original baseline Obsidian complete inspection also works as
inline input to the new inventory summary, retaining the old parent and atomic
inventory identities and manifest. Original input files remain byte-identical.
The runtime explicitly selects an absent WABT directory and requests no producer
operation. These views consume existing Evidence without another provider run;
this is portable-result reuse through existing import/export, not an engine
database cache. The five-call workflow completes in 5.458 seconds with a sampled
aggregate peak of 700,542,976 bytes and no observed limit or OOM events.
To repeat it with genuine old public outputs:
```sh
python3 experiments/analysis-efficiency/resource-run.py -- \
node experiments/analysis-efficiency/verify-migration.mjs \
--runtime-root /absolute/frozen-pilot-runtime \
--revision 259e99dd906290189217cf0fb1a19769bcf87101 \
--bundle /absolute/baseline-exported-evidence.json \
--inspection-reply /absolute/baseline-complete-inspection-reply.json \
--output /absolute/new-migration-directory
```
## PR integration verification
The separate post-integration verification pins runtime
`0eb42a1eb8ab17aed04240174cb77612e733a1d8`, integrated with main
`c22111eca2c4bc84a084f12711f405b3e7422a40`. Conflict resolution preserves main's
HAR API-surface views, portable capture admission, exact layout/JavaScript
summary carriers and existing CLI tests alongside inventory/WASM views.
Earlier real-model observations keep their original runtime fingerprints.
Build, the complete `check` static gate and generated documentation checks pass.
Twelve focused files pass 123 assertions, including the presentation test run
separately with its original deadline. Actual package creation, private-prefix
installation and 42 real WABT CLI/MCP cases pass. No package is published.
The complete generic package verifier and full local `check:pr` were not run.
Genuine old SQLite/Obsidian Evidence migration passes again with WABT unavailable.
A fresh public SDK cohort preserves the original Obsidian default result size
(937,404 compact JSON bytes), atomic inventory identity and every selected fact;
explicit summary plus exact paths still returns 16,322 bytes. Complete catalog
JSON is 2,359,082 bytes versus the original baseline's 2,326,609 (about 1.4%
larger). That difference includes intervening main changes and does not isolate
this PR's catalog contribution. One cohort is not a latency distribution.
Recorded integration scopes have no OOM/OOM-kill or hard-limit abort. The full
source-check scope reports three `memory.high` events near its 3 GiB soft
threshold. The subsequent cold pre-push type/lint scope reports 37 such events,
with 3,221,155,840 sampled peak bytes and no hard-limit/OOM event. Heavy lanes stay
serial with the existing CPU, memory and no-swap controls. No further REA bug repairs
are included; separate local findings remain for team follow-up.
## CI attribution
Review of the actual main and PR job logs separates two PR-related
description/test mismatches from pre-existing main failures. The complete-graph
wording check now accepts capitalization changes; the existing effects audit
records why returning WASM execution/network metadata is not foreign execution.
Those two files pass 11 focused assertions without production changes. Main's
own CI already fails the same live prompt-completion assertion and Darwin
packaged-completion checks; the affected implementation and fixture files are
byte-identical here. Those failures remain unchanged for team review at the
maintainer's request. CI failures are not all infrastructure faults, and the
failed CI gate is not presented as passed.
| Failure | Main `c22111eca` | Reviewed PR `8d6f7eb1c` |
| ---------------------------------------- | ---------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| Empty prompt completion after open | [main shard](https://github.com/morluto/rea/actions/runs/38224038323/job/114727940506) | [PR shard](https://github.com/morluto/rea/actions/runs/38228724531/job/114742955092) |
| Packaged prompt completion, Darwin arm64 | [main package](https://github.com/morluto/rea/actions/runs/38224038323/job/114728030045) | [PR package](https://github.com/morluto/rea/actions/runs/38228724531/job/114743092314) |
| Packaged prompt completion, Darwin x64 | [main package](https://github.com/morluto/rea/actions/runs/38224038323/job/114728029923) | [PR package](https://github.com/morluto/rea/actions/runs/38228724531/job/114743092079) |
`src/server/promptCompletion.ts`, `src/application/binary/BinarySession.ts`,
`tests/boundary/mcp/promptCompletion.test.ts` and
`scripts/verify-package-prompts.mjs` are byte-identical to that main revision.
Main commit `07b36c148946f46b99a4ce04e55726e97d0ee03a` introduces ready-provider-only
completion; the failing fixtures expect populated completion immediately after
opening. Whether those expectations or the readiness behavior should change is
left for team review. Commit `00d51e0fcd1d1bba60bf56281725e82d525a27ae`
synchronizes only the two PR-related description checks.
## Frozen runtime references
### Review follow-up
Two substantive bot findings were reproduced and addressed before merge. Atomic
inventory projections now reuse the owning subject/digest/format validation.
Actual public SDK calls with original Obsidian data demonstrated the old
acceptance of null subjects and inconsistent formats; the corrected runtime
rejects both with `invalid_request`. Six boundary assertions also cover missing
subject, wrong format and wrong digest across summary and node views.
The related three-file run passes 37 assertions, and full static/docs gates
pass. Genuine baseline SQLite/Obsidian import/view/export still preserves whole
original records and source/view identities with WABT unavailable. A new real
Obsidian summary workflow preserves every selected fact, atomic source identity
and 16,322 delivered JSON bytes; catalog bytes remain 2,359,082.
Both experimental drivers reject undeclared inherited provider controls before
server/profile creation and log no values. Explicit settings still override
the corresponding inherited values. This prevents claiming equal declared
settings while silently inheriting a different analysis option; it does not
certify every aspect of the inherited system/runtime environment. These are
research-driver controls, with no production configuration changes.
The source/static scope records 98 `memory.high` events, no hard limit/OOM event,
and a sampled peak of 3,221,209,088 bytes. The real migration/Obsidian scope peaks
at 677,367,808 bytes with no limit/OOM events. All raw receipts remain outside Git.
A later WASM finding was also reproduced: portable sources with absent subject
or a different subject path could report another artifact path. The producer and
view now share an exact path/digest identity rule. Actual WABT inspection of the
published VS Code Tree-sitter INI grammar (0.3.1, 4,716 bytes, SHA-256
`1e8cdf3027209100fa54a43d0faed69721ae1116428164c8ad6dcc496a9aa6af`)
supplied the unchanged report for public SDK before/after checks. The old runtime
accepts both inconsistencies; the corrected runtime rejects both with
`invalid_request` and WABT unavailable. Its 26 focused assertions and type/lint
checks pass, and genuine old records still round-trip unchanged. The verification
scope peaks at 3,204,509,696 bytes with no observed limit/OOM event. The initial
CSS producer exceeded the MCP reply budget as expected; that failed probe is
preserved separately and is not credited as a completed identity reproduction.
These revisions identify different experiment stages; model measurements were
not rerun or relabeled as post-integration measurements.
| Stage | Source revision |
| ---------------------------------------------- | ------------------------------------------ |
| Original baseline | `42b0ed126df7d5b908ac369715378d5e519051b7` |
| Inventory delivery pairs | `f110618184e99f89fa776bc5e59703a61baa7c57` |
| Initial WASM pilot | `0002caa07ffd7319672a21ce901dbfbc55e21aac` |
| Exact WASM summary guidance | `259e99dd906290189217cf0fb1a19769bcf87101` |
| Main-integrated package/migration verification | `0eb42a1eb8ab17aed04240174cb77612e733a1d8` |
The baseline runtime file-table SHA-256 is
`3a1765a6fec0031db0b288112d4db9bc0dd503c9c3aa0946f07ce57f7673ab4b`;
the integrated runtime file-table SHA-256 is
`e99d3bdc851822b77e7d5bfaa9b60c57e7b99bd66e27a739bdea2e9300560d5f`.
The integrated lockfile SHA-256 is
`0a55042141899a442a39dce3d49755c65a861f86d1930bd4f9c6d89cd53f5eba`.
Actual old migration inputs have SHA-256
`e1ff3d28801436bcc11693f79cc847a4f31234f49d88449f9e0d77215c3765fc`
(SQLite bundle) and
`7cecd07012bb79f3afad2ea9e70357e056e9eab48c1aa13bbbd310cdcb43eb9f`
(Obsidian complete inspection reply). The source WASM identity
`ev_16b9f6e0fcefcab9603f14fd5e63ea00c4749accd94fef4a4e9fbb260d2e0140`
and atomic inventory identity
`ev_22c17e52489f3ec0e7298a62da37bd985dbbec202d7208a0bb1c20d863790aed`
remain unchanged after the recorded import/view/export workflows.
### Complete carrier identity and provider search configuration
The outer `inspect_artifact` carrier now uses the same owning subject/digest/format
validator as its atomic inventory. Invalid imported/inline carriers are rejected
before selected-view construction; the generic historical Evidence parser and
valid complete records are unchanged. Six additional boundary assertions cover
missing subject, wrong format and wrong digest across summary/node projections.
The negative checks use the small keyed-archive workflow so the original large
ZIP retention/capacity workflow keeps its existing deadline and distinct coverage.
An initial large-workflow placement timed out and is retained as a failed attempt.
The three public MCP workflows, build, fast static gate and generated docs pass.
The final static/test scope observed one soft `memory.high` event, with no hard
memory, task-limit, OOM or OOM-kill event; it is not a performance trial.
Both research drivers fingerprint the effective provider `PATH`, including an
explicit caller override, without serializing its value or the ambient environment.
The SDK comparator rejects absent or unequal search-path fingerprints. This
records search configuration, not executable replacement at an unchanged path;
provider/runtime identities and controlled host conditions still matter.
Actual public SDK calls using the original shipped Obsidian inspection reproduce
accepted null/wrong-format outer subjects on frozen `2ef0467bb` and `invalid_request`
on the corrected compiled runtime. Two fresh Obsidian replays with different
inherited search paths preserve selected facts and source identities, but the
comparator rejects matching settings with only `provider_search_path_sha256`
different. The original records and failed/accepted replies stay in private lab
receipts; no environment value is copied into them.
### Resource-counter closeout qualification
The experimental wrapper now attempts a final cgroup event read before waiting
or cleanup. Disappeared/incomplete counters or still-owned descendants make
`resource_counters_verified` false; last observations remain available separately.
The model grader treats missing final counters as unverified, never as zero,
and any recorded soft/hard/task-limit event prevents an unrestricted resource
pass. Lightweight parser probes cover missing/incomplete and nonzero counters;
an actual short command confirms disappearance is reported as unknown. Regrading
one already-correct SQLite model answer preserves its facts but rejects the old
receipt's absent final resource counters. Earlier receipts record last-sampled
observations, not verified final counters. Their bytes, facts and token reports
remain original observations; resource-qualified performance conclusions need
this additional closeout evidence. No analyzer/build/schema/package behavior is
changed, so unrelated verification is not repeated for this research-driver fix.
experiments/analysis-efficiency/agent-trial.py
+278
View File
@@ -0,0 +1,278 @@
"""Evaluate one real Codex CLI task with an isolated REA MCP registration."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import shutil
import signal
import subprocess
import tempfile
import time
def file_digest(path):
digest = hashlib.sha256()
with path.open('rb') as data:
for chunk in iter(lambda: data.read(1024 * 1024), b''):
digest.update(chunk)
return digest.hexdigest()
def runtime_fingerprint(root):
files = []
for name in ['dist', 'bridge']:
for path in sorted((root / name).rglob('*')):
if path.is_symlink():
raise ValueError('Runtime fingerprint does not follow symlinks')
if path.is_file():
files.append({'path': path.relative_to(root).as_posix(), 'sha256': file_digest(path)})
files.sort(key=lambda row: row['path'])
text = json.dumps(files, separators=(',', ':'), ensure_ascii=False)
return files, hashlib.sha256(text.encode()).hexdigest()
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--mode', choices=['direct', 'client-code'], default='direct')
parser.add_argument('--runtime-root', type=Path, required=True)
parser.add_argument('--revision', required=True)
parser.add_argument('--prompt', type=Path, required=True)
parser.add_argument('--artifact-path', help='Replace {{artifact_path}} in a question template')
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--model', default='gpt-5.6-terra')
parser.add_argument('--reasoning', default='high')
parser.add_argument('--timeout', type=int, default=900)
parser.add_argument('--codex', default='/usr/bin/codex')
parser.add_argument('--providers', type=Path)
parser.add_argument('--enabled-tools', type=Path)
parser.add_argument('--authority', choices=['static', 'browser'], default='static')
args = parser.parse_args()
def terminate(signum, _frame):
# The outer resource scope terminates descendants together. Ensure the
# runner's copied credentials are also removed on a resource abort.
raise SystemExit(128 + signum)
signal.signal(signal.SIGTERM, terminate)
if not args.runtime_root.is_absolute() or not args.output.is_absolute():
raise ValueError('Runtime and output paths must be absolute')
repository = args.runtime_root
driver_repository = Path(__file__).resolve().parents[2]
experiment = Path(__file__).resolve().parent
node = shutil.which('node')
if subprocess.check_output([node, '--version'], text=True).strip() != 'v24.18.0':
raise SystemExit('Use pinned Node 24.18.0 on PATH')
provider_env = {'REA_LOG_LEVEL': 'silent', 'NODE_OPTIONS': '--max-old-space-size=512'}
if args.providers:
declared = json.loads(args.providers.read_text())
if not isinstance(declared, dict) or not all(
isinstance(value, str) for value in declared.values()
):
raise ValueError('--providers must contain an object of explicit string settings')
provider_env.update(declared)
provider_policy_text = (experiment / 'provider-environment.json').read_text()
provider_policy = json.loads(provider_policy_text)
undeclared = sorted(key for key in os.environ if (
any(key.startswith(prefix) for prefix in provider_policy['namespace_prefixes'])
or key in provider_policy['additional_keys']
) and key not in provider_env)
if undeclared:
raise ValueError('Declare inherited provider settings explicitly in --providers: '
+ ', '.join(undeclared) + '. Values are not logged.')
args.output.mkdir(parents=True, exist_ok=False, mode=0o700)
runtime_files, runtime_digest = runtime_fingerprint(repository)
(args.output / 'runtime-files.json').write_text(json.dumps(runtime_files))
profile = Path(tempfile.mkdtemp(prefix='profile-', dir=args.output))
trial_codex_home = profile / 'codex'
account = profile / 'account'
workspace = profile / 'workspace'
for directory in [trial_codex_home, account, workspace]:
directory.mkdir(mode=0o700)
source_auth = Path(os.environ.get('CODEX_HOME', str(Path.home() / '.codex'))) / 'auth.json'
auth = trial_codex_home / 'auth.json'
started = time.monotonic()
timeout = False
exit_code = None
environment = dict(os.environ)
environment.update({
'CODEX_HOME': str(trial_codex_home), 'HOME': str(account),
'XDG_CONFIG_HOME': str(account / '.config'),
'XDG_CACHE_HOME': str(account / '.cache'),
'XDG_DATA_HOME': str(account / '.local/share'),
'NODE_OPTIONS': '--max-old-space-size=512',
})
target_digest = file_digest(Path(args.artifact_path)) if args.artifact_path else None
command_args = [str(repository / 'scripts/rea.mjs'), 'mcp']
instruction = (
'Use the registered REA MCP server for this investigation. Static analysis '
'only: do not execute target code, fetch URLs, launch a GUI, or modify targets. '
'Preserve observations, derivations, inferences, limitations, and real Evidence IDs. '
'An Evidence value has evidence_id, normalized_result and limitations; '
'non-Evidence session values have result. '
'The analyzed target digest is Evidence.subject.digest.sha256. '
'normalized_result.engine.artifact_sha256 identifies the analysis engine '
'artifact and must never be substituted for the target digest. '
'Prefer summary/focused views where appropriate. Do not invent successful facts '
'after a tool error. No other agents. Stop once the requested facts are supported.'
)
if args.authority == 'browser':
instruction = (
'Use the registered REA tools for the explicitly selected public browser scenario. '
'Launching the selected headless browser with a fresh empty profile and observing '
'the supplied public URL is authorized. Keep all actions within that scenario. '
'No injected code, credentials, unrelated targets, shell/file tools or other agents. '
'Preserve real Evidence IDs, observation completeness, failed captures and limitations. '
'Do not infer original source or global behavior from one observed session.'
)
if args.mode == 'client-code':
instruction += (
' Use the client JavaScript executor to compose registered MCP calls when '
'possible and select the requested normalized_result fields before returning '
'intermediate output to model context. Keep real Evidence IDs and limitations.'
)
enabled_tools = None
if args.enabled_tools:
enabled_tools = json.loads(args.enabled_tools.read_text())
if not isinstance(enabled_tools, list) or not all(isinstance(name, str) and name for name in enabled_tools) or len(enabled_tools) != len(set(enabled_tools)):
raise ValueError('enabled-tools must be a unique list of exact tool names')
quote = json.dumps
config = '\n'.join([
'approval_policy = "never"', 'sandbox_mode = "read-only"',
'web_search = "disabled"', 'model = ' + quote(args.model),
'model_reasoning_effort = ' + quote(args.reasoning),
'developer_instructions = ' + quote(instruction),
'[features]', 'multi_agent = false', 'shell_snapshot = false',
'shell_tool = false', 'unified_exec = false',
'code_mode = ' + ('true' if args.mode == 'client-code' else 'false'),
'code_mode_only = ' + ('true' if args.mode == 'client-code' else 'false'),
'[history]', 'persistence = "none"',
'[mcp_servers.rea]', 'command = ' + quote(node),
'args = ' + quote(command_args), 'startup_timeout_sec = 60',
'tool_timeout_sec = 600',
*([] if enabled_tools is None else ['enabled_tools = ' + quote(enabled_tools)]),
'[mcp_servers.rea.env]',
*[key + ' = ' + quote(value) for key, value in provider_env.items()],
]) + '\n'
# MCP read-only hints cover normal static tools. Explicitly authorize only
# the session-opening primitive needed by the requested native tasks.
config += '[mcp_servers.rea.tools.open_binary]\napproval_mode = "approve"\n'
for tool in ['analyze_javascript_application', 'inspect_analysis_view',
'inspect_android_package', 'search_android_classes',
'inspect_android_class', 'inspect_android_method',
'inspect_firmware_regions', 'inspect_managed_artifact',
'inspect_managed_members', 'inspect_managed_native_boundaries',
'inspect_artifact', 'inspect_wasm_artifact',
'inspect_evm_interface', 'get_evidence_bundle', 'export_evidence_bundle',
'capture_browser_scenario',
'resolve_containing_procedure', 'analyze_function', 'close_binary']:
config += '[mcp_servers.rea.tools.' + tool + ']\napproval_mode = "approve"\n'
try:
if source_auth.exists():
shutil.copyfile(source_auth, auth)
auth.chmod(0o600)
(trial_codex_home / 'config.toml').write_text(config)
(trial_codex_home / 'config.toml').chmod(0o600)
prompt = args.prompt.read_text()
template_digest = hashlib.sha256(prompt.encode()).hexdigest()
if '{{artifact_path}}' in prompt:
if not args.artifact_path or not os.path.isabs(args.artifact_path):
raise ValueError('A question template requires an absolute --artifact-path')
prompt = prompt.replace('{{artifact_path}}', args.artifact_path)
elif args.artifact_path:
raise ValueError('--artifact-path requires the literal template marker')
(args.output / 'prompt.txt').write_text(prompt)
(args.output / 'trial-settings.json').write_text(json.dumps({
'mode': args.mode, 'model': args.model, 'reasoning': args.reasoning,
'driver_commit': subprocess.check_output(
['git', 'rev-parse', 'HEAD'], cwd=driver_repository, text=True).strip(),
'runtime_revision': args.revision,
'runtime_root': str(repository),
'selected_artifact_path': args.artifact_path,
'selected_artifact_sha256_before': target_digest,
'enabled_tools': enabled_tools,
'authority': args.authority,
'runtime_files_sha256': runtime_digest,
'runtime_dispatcher_sha256': file_digest(repository / 'scripts/rea.mjs'),
'package_lock_sha256': file_digest(repository / 'package-lock.json'),
'experiment_sources_sha256': {
name: hashlib.sha256((experiment / name).read_bytes()).hexdigest()
for name in ['agent-trial.py', 'provider-environment.json']
},
'developer_instructions': instruction,
'developer_instructions_sha256': hashlib.sha256(instruction.encode()).hexdigest(),
'prompt_sha256': hashlib.sha256(prompt.encode()).hexdigest(),
'question_template_sha256': template_digest,
'provider_settings_sha256': hashlib.sha256(
json.dumps(provider_env, sort_keys=True).encode()).hexdigest(),
'provider_search_path_sha256': hashlib.sha256(
provider_env.get('PATH', os.environ.get('PATH', '')).encode()).hexdigest(),
'provider_environment_policy_sha256': hashlib.sha256(
provider_policy_text.encode()).hexdigest(),
'features': {'shell_tool': False, 'unified_exec': False,
'multi_agent': False, 'code_mode': args.mode == 'client-code',
'code_mode_only': args.mode == 'client-code'},
}, indent=2) + '\n')
invocation = [args.codex, '--no-daemon', 'exec', '--strict-config', '--ephemeral',
'--skip-git-repo-check', '--ignore-rules', '--json',
'--color', 'never', '-C', str(workspace),
'--output-last-message', str(args.output.resolve() / 'answer.txt'), '-']
with (args.output / 'events.jsonl').open('w') as events, (args.output / 'stderr.log').open('w') as errors:
child = subprocess.Popen(invocation, env=environment, cwd=workspace,
stdin=subprocess.PIPE, stdout=events, stderr=errors, text=True)
child.stdin.write(prompt)
child.stdin.close()
try:
exit_code = child.wait(timeout=args.timeout)
except subprocess.TimeoutExpired:
timeout = True
child.terminate()
try:
exit_code = child.wait(timeout=10)
except subprocess.TimeoutExpired:
child.kill()
exit_code = child.wait()
usage = []
tools = []
event_types = {}
for line in (args.output / 'events.jsonl').open():
if not line.strip():
continue
event = json.loads(line)
kind = event.get('type', 'unknown')
event_types[kind] = event_types.get(kind, 0) + 1
if 'usage' in event:
usage.append(event['usage'])
item = event.get('item', {})
if item.get('type') == 'mcp_tool_call' and kind == 'item.completed':
tools.append({
**{k: item.get(k) for k in ['server', 'tool', 'arguments', 'status', 'error']},
'codex_event_result_json_utf8_bytes': len(json.dumps(
item.get('result'), separators=(',', ':'), ensure_ascii=False
).encode()),
})
final_target_digest = file_digest(Path(args.artifact_path)) if args.artifact_path else None
summary = {
'mode': args.mode, 'requested_model': args.model, 'reasoning': args.reasoning,
'runtime_revision': args.revision,
'selected_artifact_sha256_before': target_digest,
'selected_artifact_sha256_after': final_target_digest,
'selected_artifact_bytes_unchanged': target_digest == final_target_digest if target_digest else None,
'model_identity_scope': 'requested alias; resolved server-side model identity is not inferred',
'codex_version': subprocess.check_output([args.codex, '--version'], text=True).strip(),
'exit_code': exit_code, 'timeout': timeout,
'wall_seconds': round(time.monotonic() - started, 3),
'usage': usage, 'completed_mcp_calls': tools, 'event_types': event_types,
'factual_correctness': 'not_assessed',
'scope': 'real_cli_trial_requires_transcript_and_fact_review',
}
(args.output / 'summary.json').write_text(json.dumps(summary, indent=2) + '\n')
print(json.dumps(summary))
finally:
# Always remove copied credentials, config, caches and profile state.
shutil.rmtree(profile)
(args.output / 'cleanup.json').write_text(json.dumps({'profile_removed': not profile.exists()}) + '\n')
raise SystemExit(124 if timeout else exit_code)
if __name__ == '__main__':
main()
experiments/analysis-efficiency/compare.mjs
+157
View File
@@ -0,0 +1,157 @@
import { isDeepStrictEqual } from "node:util";
import { readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { parseArgs } from "node:util";
const { values } = parseArgs({
options: {
left: { type: "string" },
right: { type: "string" },
output: { type: "string" },
},
});
if (!values.left || !values.right)
throw new Error(
"Use --left MEASUREMENT_DIR --right MEASUREMENT_DIR [--output JSON]",
);
const read = async (directory, name) =>
JSON.parse(await readFile(resolve(directory, name), "utf8"));
const left = await read(values.left, "summary.json");
const right = await read(values.right, "summary.json");
const leftAnswer = await read(values.left, "answer.json");
const rightAnswer = await read(values.right, "answer.json");
const sameSettings = [
"artifact",
"artifact_sha256",
"artifact_sha256_after",
"native_question",
"javascript_question",
"provider_settings_sha256",
"provider_search_path_sha256",
"provider_environment_policy_sha256",
"measure_source_sha256",
"package_lock_sha256",
"node_version",
"sdk_client_version",
"sdk_server_version",
"platform",
"architecture",
];
const settingDifferences = sameSettings.filter(
(key) =>
!isDeepStrictEqual(left[key], right[key]) ||
(key === "provider_search_path_sha256" &&
!/^[a-f0-9]{64}$/.test(left[key] ?? "")),
);
// Producer/view IDs identify different delivery observations; a named recovery
// describes the delivery path. All selected analyst facts still must agree.
const facts = (answer) =>
Object.fromEntries(
Object.entries(answer).filter(
([key]) => key !== "evidence_ids" && key !== "recovery",
),
);
const a = facts(leftAnswer);
const b = facts(rightAnswer);
const runSpecific = [];
// Binwalk assigns a new opaque region UUID in each invocation. Preserve both
// original lookup values in this report; compare the observed region facts
// separately. The product and original replies retain those IDs unchanged.
if (left.artifact === "openwrt.bin") {
for (const [side, answer] of [
["left", a],
["right", b],
]) {
for (const [index, region] of answer.result.regions.entries()) {
runSpecific.push({
side,
path: ["result", "regions", index, "provider_id"],
value: region.provider_id,
});
delete region.provider_id;
}
}
}
// JavaScript result budgets depend on observed parent heap availability.
// Preserve their exact values on both sides, then compare the coverage facts
// and fixed worker limits. Never exclude status, omitted counts or families.
if (
left.artifact === "obsidian.asar" &&
left.javascript_question === "selected"
) {
const dynamicBudgets = new Set([
"javascript_analysis_parent_available_result_budget_bytes",
"javascript_analysis_parent_result_budget_bytes",
]);
for (const [side, answer] of [
["left", a],
["right", b],
]) {
for (const layer of ["application", "semantic"]) {
for (const [index, limit] of answer.coverage[layer].limits.entries()) {
if (!dynamicBudgets.has(limit.name) || limit.unit !== "bytes") continue;
runSpecific.push({
side,
path: ["coverage", layer, "limits", index, "value"],
name: limit.name,
unit: limit.unit,
value: limit.value,
});
delete limit.value;
}
}
}
}
const factDifferences = [
...new Set([...Object.keys(a), ...Object.keys(b)]),
].filter((key) => !isDeepStrictEqual(a[key], b[key]));
const cost = (report) => ({
revision: report.revision,
runtime_files_sha256: report.runtime_files_sha256,
delivery: report.delivery,
sdk_schema_mode: report.sdk_schema_mode,
elapsed_ms: report.elapsed_ms,
catalog_json_utf8_bytes: report.catalog_json_utf8_bytes,
calls: report.calls.length,
observed_tool_errors: report.calls.filter((call) => call.is_error).length,
client_result_json_utf8_bytes: report.calls.reduce(
(sum, call) => sum + (call.client_result_json_utf8_bytes ?? 0),
0,
),
client_before_send_ms: report.calls.reduce(
(sum, call) => sum + (call.client_before_send_ms ?? 0),
0,
),
transport_roundtrip_ms: report.calls.reduce(
(sum, call) => sum + (call.transport_roundtrip_ms ?? 0),
0,
),
client_after_receive_ms: report.calls.reduce(
(sum, call) => sum + (call.client_after_receive_ms ?? 0),
0,
),
});
const comparison = {
artifact: left.artifact,
artifact_sha256: left.artifact_sha256,
settings_match: settingDifferences.length === 0,
setting_differences: settingDifferences,
facts_match: factDifferences.length === 0,
fact_differences: factDifferences,
preserved_run_specific_values: runSpecific,
workflow_failures: [left.failure, right.failure],
left: cost(left),
right: cost(right),
scope:
"Same selected analyst facts; includes extra view calls and preserves observed tool errors. Result JSON bytes exclude file exports, catalog responses and raw wire framing. This is one paired observation, not a statistical speedup or independent ground-truth proof.",
};
const text = JSON.stringify(comparison, null, 2) + "\n";
if (values.output) await writeFile(values.output, text, { flag: "wx" });
process.stdout.write(text);
if (
settingDifferences.length ||
factDifferences.length ||
left.failure ||
right.failure
)
process.exitCode = 1;
experiments/analysis-efficiency/corpus.json
+120
View File
@@ -0,0 +1,120 @@
{
"schema_version": 1,
"artifacts": [
{
"id": "sharex.dll",
"file": "ShareX.dll",
"bytes": 2162688,
"sha256": "191a191be964e434daf0f9d76d1e0d8ddce90d7e248cba2a6b5ac9577f288d53",
"provenance": {
"project": "ShareX",
"version": "19.0.2",
"upstreamFile": "ShareX-19.0.2-portable.zip",
"url": "https://github.com/ShareX/ShareX/releases/download/v19.0.2/ShareX-19.0.2-portable.zip",
"transport_sha256": "967b5d838df5ce25fb5fd7cbd16f1bd797427e2df16907b022b74a4bc0fb9d54",
"transport_bytes": 168796487,
"derivation": "selected regular ZIP member ShareX.dll"
}
},
{
"id": "sqlite.wasm",
"file": "sqlite3.wasm",
"bytes": 868907,
"sha256": "2ee8f3dab694532afc8840e07703127287662d08b74e6ff50491ce63f00d5752",
"provenance": {
"project": "SQLite Wasm",
"version": "3.53.4-build1",
"upstreamFile": "sqlite-wasm-3.53.4-build1.tgz",
"url": "https://registry.npmjs.org/@sqlite.org/sqlite-wasm/-/sqlite-wasm-3.53.4-build1.tgz",
"transport_sha256": "24f46db4ce5248fa8f8f24b67e81909ab1a9c152f42a4796a32c6d385a6fb62b",
"transport_bytes": 977616,
"transport_integrity": "sha512-U8pvL0patYHMgieD8JxxYJTlh9m1LUX8+BDYb7MiKQzBI89f2qbdmdcG/PvnPaLm9wsBbv4VlVG453ixnxBOnA==",
"derivation": "selected regular tar member package/dist/sqlite3.wasm"
}
},
{
"id": "obsidian.asar",
"file": "obsidian-1.14.4.asar",
"bytes": 27477966,
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f",
"provenance": {
"project": "Obsidian",
"version": "1.14.4",
"upstreamFile": "obsidian-1.14.4.asar",
"transport": {
"file": "obsidian-1.14.4.asar.gz",
"url": "https://github.com/obsidianmd/obsidian-releases/releases/download/v1.14.4/obsidian-1.14.4.asar.gz",
"sha256": "d1ed428c363968774f0f3906e67d8a53a058ece0b6cdd924b07865977b2ada21",
"bytes": 9200670
},
"derivation": "gzip decompression"
}
},
{
"id": "newpipe.apk",
"file": "NewPipe_v0.29.1.apk",
"bytes": 11507122,
"sha256": "18447bfb1e06d113edc88df93f471827280de06f6f3d4dc42f56f28b9c1bab79",
"provenance": {
"project": "NewPipe",
"version": "0.29.1",
"upstreamFile": "NewPipe_v0.29.1.apk",
"url": "https://github.com/TeamNewPipe/NewPipe/releases/download/v0.29.1/NewPipe_v0.29.1.apk"
}
},
{
"id": "jq",
"file": "jq-linux-amd64",
"bytes": 2267912,
"sha256": "b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f",
"provenance": {
"project": "jq",
"version": "1.8.2",
"upstreamFile": "jq-linux-amd64",
"url": "https://github.com/jqlang/jq/releases/download/jq-1.8.2/jq-linux-amd64"
}
},
{
"id": "openwrt.bin",
"file": "openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin",
"bytes": 7405862,
"sha256": "f15b16b36df7fce8a5cf1302320fdb428c3bb6fbc2949a4858afa257f4f364a2",
"provenance": {
"project": "OpenWrt Archer C7 v5",
"version": "25.12.5",
"upstreamFile": "openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin",
"url": "https://downloads.openwrt.org/releases/25.12.5/targets/ath79/generic/openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin"
}
},
{
"id": "weth.hex",
"file": "weth-published.hex",
"bytes": 6251,
"sha256": "eb67ea4da28b88358df0385358bad3a58b131317835b45b3ede85c61851335e4",
"provenance": {
"project": "Published historical WETH runtime code example",
"url": "https://ethereum.org/developers/docs/apis/json-rpc/",
"contract_address": "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2",
"published_block": "0x5daf3b",
"transport_sha256": "98c131a1bd99a0c2051dd6d6716fd306c36cdd45995d5ffd559bd9c690569c0f",
"derivation": "Parse the named eth_getCode JSON result from the official published example and preserve the hex carrier with one trailing LF",
"authority": "published historical RPC example; live retrieval returned HTTP 403; deployment authenticity remains unverified"
}
},
{
"id": "sharex.helpers.dll",
"file": "ShareX.HelpersLib.dll",
"bytes": 1076224,
"sha256": "e91bd1ec904e2bb8a1add4bb6187443b878bf33d24fe9e27f0b799bfee9de33d",
"provenance": {
"project": "ShareX HelpersLib",
"version": "19.0.2",
"upstreamFile": "ShareX-19.0.2-portable.zip",
"url": "https://github.com/ShareX/ShareX/releases/download/v19.0.2/ShareX-19.0.2-portable.zip",
"transport_sha256": "967b5d838df5ce25fb5fd7cbd16f1bd797427e2df16907b022b74a4bc0fb9d54",
"transport_bytes": 168796487,
"derivation": "selected regular ZIP member ShareX.HelpersLib.dll; no target execution"
}
}
]
}
experiments/analysis-efficiency/measure.mjs
+754
View File
@@ -0,0 +1,754 @@
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import {
mkdir,
open,
readFile,
readdir,
stat,
writeFile,
} from "node:fs/promises";
import { dirname, isAbsolute, resolve } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
const root = resolve(dirname(fileURLToPath(import.meta.url)), "../..");
const { values } = parseArgs({
options: {
"runtime-root": { type: "string", default: root },
revision: { type: "string" },
artifact: { type: "string" },
path: { type: "string" },
providers: { type: "string" },
output: { type: "string" },
delivery: { type: "string", default: "complete" },
"native-question": { type: "string", default: "entry" },
"javascript-question": { type: "string", default: "inventory" },
schemas: { type: "string", default: "default" },
},
});
if (
!values.revision ||
!values.artifact ||
!values.path ||
!values.output ||
!isAbsolute(values.path) ||
!isAbsolute(values.output) ||
!["complete", "summary"].includes(values.delivery) ||
!["entry", "entry-and-main"].includes(values["native-question"]) ||
!["inventory", "selected"].includes(values["javascript-question"]) ||
!["default", "selected"].includes(values.schemas) ||
(values.artifact !== "jq" && values["native-question"] !== "entry") ||
(values.artifact !== "obsidian.asar" &&
values["javascript-question"] !== "inventory")
)
throw new Error(
"Use --revision SHA --artifact ID --path ABS --output NEW_ABS_DIR [--runtime-root ABS] [--providers JSON] [--delivery complete|summary]",
);
const sha = (text) => createHash("sha256").update(text).digest("hex");
const fileSha = async (path) => {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return hash.digest("hex");
};
const corpus = JSON.parse(
await readFile(new URL("./corpus.json", import.meta.url), "utf8"),
);
const artifact = corpus.artifacts.find(({ id }) => id === values.artifact);
if (!artifact) throw new Error("Select an artifact from the pinned corpus");
const inputSha = await fileSha(values.path);
if (
artifact.sha256 !== inputSha ||
artifact.bytes !== (await stat(values.path)).size
)
throw new Error("Selected bytes differ from the pinned public corpus");
const selectedProviders = values.providers
? JSON.parse(await readFile(values.providers, "utf8"))
: {};
if (
selectedProviders === null ||
Array.isArray(selectedProviders) ||
typeof selectedProviders !== "object" ||
Object.values(selectedProviders).some((value) => typeof value !== "string")
)
throw new Error(
"--providers must contain an object of explicit string settings",
);
const providerPolicyText = await readFile(
new URL("./provider-environment.json", import.meta.url),
"utf8",
);
const providerPolicy = JSON.parse(providerPolicyText);
const undeclaredProviderKeys = Object.keys(process.env).filter(
(key) =>
(providerPolicy.namespace_prefixes.some((prefix) =>
key.startsWith(prefix),
) ||
providerPolicy.additional_keys.includes(key)) &&
!Object.hasOwn(selectedProviders, key),
);
if (undeclaredProviderKeys.length > 0)
throw new Error(
`Declare inherited provider settings explicitly in --providers: ${undeclaredProviderKeys.sort().join(", ")}. Values are not logged.`,
);
await mkdir(values.output, { recursive: false, mode: 0o700 });
const runtimeFiles = [];
const fingerprintTree = async (directory) => {
for (const entry of await readdir(directory, { withFileTypes: true })) {
const path = resolve(directory, entry.name);
if (entry.isDirectory()) await fingerprintTree(path);
else if (entry.isFile())
runtimeFiles.push({
path: path.slice(resolve(values["runtime-root"]).length + 1),
sha256: await fileSha(path),
});
else throw new Error("Runtime fingerprint does not follow symlinks");
}
};
await fingerprintTree(resolve(values["runtime-root"], "dist"));
await fingerprintTree(resolve(values["runtime-root"], "bridge"));
runtimeFiles.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
await writeFile(
resolve(values.output, "runtime-files.json"),
JSON.stringify(runtimeFiles),
);
const fingerprints = {
measure_source_sha256: await fileSha(fileURLToPath(import.meta.url)),
runtime_files_sha256: sha(JSON.stringify(runtimeFiles)),
runtime_dispatcher_sha256: await fileSha(
resolve(values["runtime-root"], "scripts/rea.mjs"),
),
package_lock_sha256: await fileSha(
resolve(values["runtime-root"], "package-lock.json"),
),
};
const clientVersion = JSON.parse(
await readFile(
resolve(root, "node_modules/@modelcontextprotocol/client/package.json"),
"utf8",
),
).version;
const serverVersion = JSON.parse(
await readFile(
resolve(
values["runtime-root"],
"node_modules/@modelcontextprotocol/server/package.json",
),
"utf8",
),
).version;
class ToolFailure extends Error {
constructor(value) {
super("REA returned a tool error");
this.value = value;
}
}
const client = new Client({ name: "rea-analysis-efficiency", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [resolve(values["runtime-root"], "scripts/rea.mjs"), "mcp"],
env: { ...process.env, ...selectedProviders },
stderr: "pipe",
});
const diagnostics = [];
const calls = [];
const catalogDefinitions = new Map();
const pendingRpc = new Map();
let activeCall;
const started = performance.now();
let connectedMs, catalogMs, catalogBytes, catalogInputBytes, result, failure;
const decode = (reply) =>
reply.structuredContent ??
JSON.parse(
reply.content.find((item) => item.type === "text")?.text ?? "null",
);
const call = async (name, arguments_) => {
const toolDefinition =
values.schemas === "selected" ? catalogDefinitions.get(name) : undefined;
if (values.schemas === "selected" && toolDefinition === undefined)
throw new Error(`No live advertised definition for ${name}`);
const began = performance.now();
const progress = [];
const rpc = {};
activeCall = { began, rpc };
try {
const reply = await client.callTool(
{ name, arguments: arguments_ },
{
...(toolDefinition === undefined ? {} : { toolDefinition }),
timeout: 600_000,
onprogress: (update) =>
progress.push({ elapsed_ms: performance.now() - began, ...update }),
},
);
const elapsedMs = performance.now() - began;
const encodingBegan = performance.now();
const text = JSON.stringify(reply);
const value = decode(reply);
calls.push({
name,
arguments: arguments_,
elapsed_ms: elapsedMs,
client_before_send_ms: rpc.sent === undefined ? null : rpc.sent - began,
transport_roundtrip_ms:
rpc.received === undefined || rpc.sent === undefined
? null
: rpc.received - rpc.sent,
client_after_receive_ms:
rpc.received === undefined ? null : began + elapsedMs - rpc.received,
client_result_json_utf8_bytes: Buffer.byteLength(text),
runner_encoding_ms: performance.now() - encodingBegan,
producer_evidence_id: value?.evidence_id ?? null,
is_error: reply.isError === true,
progress,
});
await writeFile(resolve(values.output, `call-${calls.length}.json`), text);
if (reply.isError) throw new ToolFailure(value);
return value;
} catch (error) {
if (calls.at(-1)?.name !== name || calls.at(-1)?.arguments !== arguments_)
calls.push({
name,
arguments: arguments_,
elapsed_ms: performance.now() - began,
failure: String(error),
progress,
});
throw error;
} finally {
activeCall = undefined;
for (const [id, pending] of pendingRpc)
if (pending.rpc === rpc) pendingRpc.delete(id);
}
};
const normalized = (value) => value.normalized_result;
const targetSha = (value) => value.subject?.digest.sha256;
const inventoryPaths = ["package.json", "main.js", "preload.js"];
const unique = (values) => [...new Set(values)].sort();
const inspectManagedTarget = async () => {
const info = await call("inspect_managed_artifact", { path: values.path });
const boundaries = await call("inspect_managed_native_boundaries", {
path: values.path,
});
return {
artifact_sha256: targetSha(info),
artifact: normalized(info),
boundaries: normalized(boundaries),
evidence_ids: [info.evidence_id, boundaries.evidence_id],
};
};
const workflows = {
"obsidian.asar": async () => {
if (values["javascript-question"] === "selected") {
const inspected = await call("analyze_javascript_application", {
input_path: values.path,
source_paths: ["package.json", "main.js"],
detail: "summary",
});
const receipt = normalized(inspected);
const parent = receipt.parent_evidence_id;
const evidenceIds = [parent, inspected.evidence_id];
const modules = [];
let offset = 0;
do {
const page = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: parent },
view: { kind: "page", collection: "modules", offset, limit: 64 },
});
evidenceIds.push(page.evidence_id);
const view = normalized(page);
for (const module of view.items.filter(
(row) => row.path === "main.js",
)) {
const selected = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: parent },
view: {
kind: "item",
collection: "modules",
selector: { node_id: module.node_id },
},
});
evidenceIds.push(selected.evidence_id);
modules.push(normalized(selected).item);
}
if (view.coverage.exhausted) break;
if (
view.coverage.next_offset === null ||
view.coverage.next_offset <= offset
)
throw new Error("Module page did not advance");
offset = view.coverage.next_offset;
} while (true);
return {
artifact_sha256: targetSha(inspected),
...receipt.summary,
modules,
limitations: receipt.limitations,
evidence_ids: evidenceIds,
};
}
await call("open_binary", { path: values.path });
try {
const inspected = await call(
"inspect_artifact",
values.delivery === "summary" ? { detail: "summary" } : {},
);
if (values.delivery === "summary") {
const view = normalized(inspected);
const summary = view.summary;
if (!summary.source_complete || !summary.inspection)
throw new Error(
"Expected a complete inventory with inspection coverage",
);
const parent = view.parent_evidence_id;
const selected = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: parent },
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: inventoryPaths,
offset: 0,
limit: 64,
},
});
const contradictions = [];
let offset = 0;
while (offset < summary.available_counts.integrity_contradictions) {
const page = normalized(
await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: parent },
view: {
kind: "inventory",
collection: "integrity_contradictions",
offset,
limit: 64,
},
}),
);
contradictions.push(...page.items);
if (page.coverage.exhausted) break;
if (
page.coverage.next_offset === null ||
page.coverage.next_offset <= offset
)
throw new Error("Inventory page did not advance");
offset = page.coverage.next_offset;
}
return {
artifact_sha256: targetSha(inspected),
manifest: summary.manifest,
source_complete: summary.source_complete,
entry_kind_counts: summary.observed_entry_kind_counts,
requested_paths: normalized(selected).items,
inventory_evidence_ids: summary.inventory_evidence_ids,
integrity_contradictions: contradictions,
coverage: summary.inspection.coverage,
unexplored_branches: summary.inspection.unexplored_branches,
limitations: view.limitations,
evidence_ids: [inspected.evidence_id, selected.evidence_id],
};
}
const inspection = normalized(inspected);
const inventory = inspection.substeps[0].evidence;
const graph = normalized(inventory);
const counts = { file: 0, directory: 0, symlink: 0, slice: 0 };
for (const row of graph.occurrences) counts[row.entry_kind]++;
const { parseArtifactInventoryEvidence } = await import(
pathToFileURL(
resolve(
values["runtime-root"],
"dist/domain/artifactInventoryEvidence.js",
),
).href
);
const checked = parseArtifactInventoryEvidence(inventory);
return {
artifact_sha256: targetSha(inspected),
manifest: graph.manifest,
source_complete: checked.inventory.complete,
entry_kind_counts: counts,
requested_paths: graph.occurrences.filter((row) =>
inventoryPaths.includes(row.logical_path),
),
inventory_evidence_ids: [inventory.evidence_id],
integrity_contradictions: graph.integrity_contradictions,
coverage: inspection.coverage,
unexplored_branches: inspection.unexplored_branches,
evidence_ids: [inspected.evidence_id, inventory.evidence_id],
limitations: unique([
...inspected.limitations,
...inspection.limitations,
...inventory.limitations,
]),
};
} finally {
await call("close_binary", {});
}
},
"newpipe.apk": async () => {
const pkg = await call("inspect_android_package", { path: values.path });
const found = await call("search_android_classes", {
path: values.path,
query: "MainActivity",
});
const cls = await call("inspect_android_class", {
path: values.path,
class_name: "org.schabi.newpipe.MainActivity",
});
return {
artifact_sha256: targetSha(pkg),
package_name: normalized(pkg).package_name,
version_name: normalized(pkg).version_name,
version_code: normalized(pkg).version_code,
matches: normalized(found).matches,
coverage: normalized(found).coverage,
onCreate: normalized(cls).methods.filter(
(row) => row.name === "onCreate",
),
evidence_ids: [pkg.evidence_id, found.evidence_id, cls.evidence_id],
};
},
"sharex.dll": inspectManagedTarget,
"sharex.helpers.dll": inspectManagedTarget,
"openwrt.bin": async () => {
const info = await call("inspect_firmware_regions", { path: values.path });
return {
artifact_sha256: targetSha(info),
result: normalized(info),
evidence_ids: [info.evidence_id],
};
},
"sqlite.wasm": async () => {
let info;
let recovery = null;
try {
info = await call("inspect_wasm_artifact", {
path: values.path,
glue_paths: [],
candidate_paths: [],
});
} catch (error) {
const reference =
error instanceof ToolFailure &&
error.value?.error?.code === "resource_constraint"
? error.value.error.details?.reported_limits?.evidence_reference
: undefined;
if (reference?.kind !== "retained-evidence") throw error;
if (values.delivery === "summary") {
const summaryEvidence = await call("inspect_analysis_view", {
source: reference,
view: { kind: "summary" },
});
const summary = normalized(summaryEvidence).summary;
const exportsEvidence = await call("inspect_analysis_view", {
source: reference,
view: { kind: "wasm", facet: "exports", offset: 0, limit: 4 },
});
return {
artifact_sha256: targetSha(summaryEvidence),
validation: summary.validation,
section_count: summary.available_counts.sections,
import_count: summary.available_counts.imports,
export_count: summary.available_counts.exports,
exports_first4: normalized(exportsEvidence).item,
wat_sha256: summary.wat.sha256,
wat_code_units: summary.wat.code_units,
tool_profile: summary.tool_profile,
runtime_execution: summary.runtime_execution,
network_fetch: summary.network_fetch,
evidence_ids: [
reference.evidence_id,
summaryEvidence.evidence_id,
exportsEvidence.evidence_id,
],
recovery:
"existing completed producer reference read through typed WASM views",
};
}
const path = resolve(values.output, "retained-evidence.json");
await call("export_evidence_bundle", { path });
const bundle = JSON.parse(await readFile(path, "utf8"));
const record = bundle.records.find(
(row) => row.evidence_id === reference.evidence_id,
);
const { parseEvidence } = await import(
pathToFileURL(
resolve(values["runtime-root"], "dist/domain/evidence.js"),
).href
);
info = parseEvidence(record);
recovery =
"complete producer Evidence exported after an observed transport constraint";
}
const n = normalized(info);
return {
artifact_sha256: targetSha(info),
validation: n.validation,
section_count: n.sections.length,
import_count: n.imports.length,
export_count: n.exports.length,
exports_first4: n.exports.slice(0, 4),
wat_sha256: n.wat.sha256,
wat_code_units: n.wat.text.length,
tool_profile: n.tool_profile,
runtime_execution: n.runtime_execution,
network_fetch: n.network_fetch,
evidence_ids: [info.evidence_id],
recovery,
};
},
"weth.hex": async () => {
const info = await call("inspect_evm_interface", {
path: values.path,
encoding: "hex",
});
const n = normalized(info);
return {
artifact_sha256: targetSha(info),
artifact: n.artifact,
bytecode: n.bytecode,
functions: n.functions,
discovery_completeness: n.discovery_completeness,
runtime_execution: n.runtime_execution,
limitations: n.limitations,
evidence_ids: [info.evidence_id],
};
},
jq: async () => {
const file = await open(values.path, "r");
let header;
try {
header = (await file.read(Buffer.alloc(32), 0, 32, 0)).buffer;
} finally {
await file.close();
}
if (header.subarray(0, 6).toString("hex") !== "7f454c460201")
throw new Error("Expected ELF64 little endian entry seed");
const address = `0x${header.readBigUInt64LE(24).toString(16)}`;
await call("open_binary", { path: values.path, provider_id: "ghidra" });
try {
const containing = await call("resolve_containing_procedure", {
address,
});
const info = await call("analyze_function", { procedure: address });
const n = normalized(info);
// This exact application entry is materialized in the pinned jq entry's
// observed MOV RDI, 0x401f00 instruction, not an original-source claim.
let main;
const mainEvidenceIds = [];
if (values["native-question"] === "entry-and-main") {
let sourceId;
try {
const completed = await call("analyze_function", {
procedure: "0x401f00",
});
sourceId = completed.evidence_id;
} catch (error) {
const reference =
error instanceof ToolFailure &&
error.value?.error?.code === "resource_constraint"
? error.value.error.details?.reported_limits?.evidence_reference
: undefined;
if (reference?.kind !== "retained-evidence") throw error;
sourceId = reference.evidence_id;
}
mainEvidenceIds.push(sourceId);
main = {};
for (const facet of [
"procedure",
"pseudocode",
"referenced_strings",
"callees",
"callers",
"basic_blocks",
]) {
let offset = 0;
const items = [];
const pages = [];
do {
const selected = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: sourceId },
view: {
kind: "native",
facet,
offset,
limit:
facet === "pseudocode"
? 65536
: facet === "basic_blocks"
? 1
: 64,
},
});
mainEvidenceIds.push(selected.evidence_id);
const v = normalized(selected);
items.push(v.item);
pages.push({
coverage: v.coverage,
limitations: v.limitations,
unknowns: v.unknowns,
});
if (v.coverage.exhausted || facet === "basic_blocks") break;
if (
v.coverage.next_offset === null ||
v.coverage.next_offset <= offset
)
throw new Error("Native view did not advance");
offset = v.coverage.next_offset;
} while (true);
main[facet] = { items, pages };
}
}
return {
artifact_sha256: targetSha(info),
entry_address: address,
containing_procedure: normalized(containing),
procedure: n.procedure,
pseudocode_prefix: n.pseudocode.slice(0, 512),
caller_count: n.callers.length,
callee_count: n.callees.length,
...(main === undefined ? {} : { main }),
analysis_profile: info.analysis_profile,
evidence_ids: [
containing.evidence_id,
info.evidence_id,
...mainEvidenceIds,
],
};
} finally {
await call("close_binary", {});
}
},
};
try {
await client.connect(transport);
connectedMs = performance.now() - started;
// Observe the public Transport seam without replacing its codec or lifecycle.
// These intervals separate SDK work from the transport/server roundtrip;
// the roundtrip still includes server work, framing, IO and client parsing.
const send = transport.send.bind(transport);
transport.send = (message, options) => {
if (message.method === "tools/call" && activeCall !== undefined) {
activeCall.rpc.sent = performance.now();
pendingRpc.set(message.id, activeCall);
}
return send(message, options);
};
const receive = transport.onmessage;
transport.onmessage = (message, extra) => {
if ("result" in message || "error" in message) {
const pending = pendingRpc.get(message.id);
if (pending !== undefined) {
pending.rpc.received = performance.now();
pendingRpc.delete(message.id);
}
}
receive?.(message, extra);
};
transport.stderr?.on("data", (chunk) => diagnostics.push(chunk.toString()));
const catalogStarted = performance.now();
const tools = [];
let cursor;
do {
const page = await client.listTools(cursor === undefined ? {} : { cursor });
tools.push(...page.tools);
cursor = page.nextCursor;
} while (cursor !== undefined);
catalogMs = performance.now() - catalogStarted;
for (const tool of tools) {
if (catalogDefinitions.has(tool.name))
throw new Error("Duplicate advertised tool name");
catalogDefinitions.set(tool.name, tool);
}
catalogBytes = Buffer.byteLength(JSON.stringify(tools));
catalogInputBytes = Buffer.byteLength(
JSON.stringify(
tools.map(({ name, description, inputSchema }) => ({
name,
description,
inputSchema,
})),
),
);
await writeFile(
resolve(values.output, "catalog.json"),
JSON.stringify(tools),
);
const workflow = workflows[values.artifact];
if (!workflow) throw new Error("No pinned question for selected artifact");
result = await workflow();
if (result.artifact_sha256 !== inputSha)
throw new Error("Producer target digest differs from selected bytes");
} catch (error) {
failure = String(error);
} finally {
try {
await client.close();
} catch (error) {
failure = `${failure ?? ""} cleanup: ${String(error)}`;
}
}
await writeFile(
resolve(values.output, "diagnostics.log"),
diagnostics.join(""),
);
const finalInputSha = await fileSha(values.path);
if (finalInputSha !== inputSha)
failure = `${failure ?? ""} Selected artifact bytes changed during the workflow`;
if (result)
await writeFile(
resolve(values.output, "answer.json"),
JSON.stringify(result),
);
const report = {
revision: values.revision,
runtime_root: values["runtime-root"],
artifact: values.artifact,
artifact_sha256: inputSha,
artifact_sha256_after: finalInputSha,
delivery: values.delivery,
sdk_schema_mode: values.schemas,
native_question: values.artifact === "jq" ? values["native-question"] : null,
javascript_question:
values.artifact === "obsidian.asar" ? values["javascript-question"] : null,
provider_settings_sha256: sha(JSON.stringify(selectedProviders)),
provider_search_path_sha256: sha(
selectedProviders.PATH ?? process.env.PATH ?? "",
),
provider_environment_policy_sha256: sha(providerPolicyText),
...fingerprints,
node_version: process.version,
sdk_client_version: clientVersion,
sdk_server_version: serverVersion,
platform: process.platform,
architecture: process.arch,
connected_ms: connectedMs,
catalog_ms: catalogMs,
catalog_json_utf8_bytes: catalogBytes,
catalog_input_definition_utf8_bytes: catalogInputBytes,
elapsed_ms: performance.now() - started,
calls,
failure: failure ?? null,
measurement_scope:
"Public SDK Transport timestamps split client work from the transport/server roundtrip; parsed result JSON bytes are not raw wire bytes; runner encoding and writes are outside each call; engine-stage durations and independent producer truth are not inferred",
};
await writeFile(
resolve(values.output, "summary.json"),
JSON.stringify(report, null, 2),
);
process.stdout.write(
JSON.stringify({
artifact: values.artifact,
delivery: values.delivery,
elapsed_ms: report.elapsed_ms,
failure: failure?.slice(0, 800) ?? null,
}) + "\n",
);
if (failure) process.exitCode = 1;
experiments/analysis-efficiency/probe-analyzers.mjs
+116
View File
@@ -0,0 +1,116 @@
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { mkdir, open, readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
const { values } = parseArgs({
options: {
"runtime-root": { type: "string" },
path: { type: "string" },
providers: { type: "string" },
output: { type: "string" },
},
});
if (
!values["runtime-root"] ||
!values.path ||
!values.providers ||
!values.output
)
throw new Error(
"Use --runtime-root ABS --path ABS --providers JSON --output NEW_ABS_DIR",
);
const selected = JSON.parse(await readFile(values.providers, "utf8"));
const expectedName = selected.REA_GHIDRA_DISABLED_ANALYZERS;
if (!expectedName)
throw new Error(
"The probe requires an explicitly selected invalid analyzer name",
);
const digest = async () => {
const hash = createHash("sha256");
for await (const chunk of createReadStream(values.path)) hash.update(chunk);
return hash.digest("hex");
};
const before = await digest();
await mkdir(values.output, { mode: 0o700 });
const file = await open(values.path, "r");
let address;
try {
const header = Buffer.alloc(32);
const { bytesRead } = await file.read(header, 0, header.length, 0);
if (
bytesRead !== 32 ||
header.subarray(0, 6).toString("hex") !== "7f454c460201"
)
throw new Error(
"This counterexample uses a real ELF64 little-endian application",
);
address = `0x${header.readBigUInt64LE(24).toString(16)}`;
} finally {
await file.close();
}
const client = new Client({ name: "rea-analyzer-verification", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [resolve(values["runtime-root"], "scripts/rea.mjs"), "mcp"],
env: { ...process.env, ...selected },
stderr: "pipe",
});
const calls = [];
const call = async (name, arguments_) => {
const reply = await client.callTool(
{ name, arguments: arguments_ },
{ timeout: 600_000 },
);
await writeFile(
resolve(values.output, `call-${calls.length + 1}.json`),
JSON.stringify(reply),
);
calls.push({ name, is_error: reply.isError === true });
return reply;
};
let rejected = false;
let named = false;
let failure = null;
try {
await client.connect(transport);
const opened = await call("open_binary", {
path: values.path,
provider_id: "ghidra",
});
const reply = opened.isError
? opened
: await call("resolve_containing_procedure", { address });
rejected = reply.isError === true;
named = rejected && JSON.stringify(reply).includes(expectedName);
} catch (error) {
failure = String(error);
} finally {
try {
await call("close_binary", {});
} catch (error) {
failure = `${failure ?? ""} close: ${String(error)}`;
}
await client.close();
}
const after = await digest();
const report = {
target_sha256: before,
target_sha256_after: after,
expected_analyzer: expectedName,
rejected,
rejection_names_requested_analyzer: named,
failure,
calls,
oracle:
"A syntactically valid but nonexistent analyzer must produce a meaningful public refusal, not successful Evidence claiming that preset. A not-found procedure still means Ghidra reached readiness.",
};
await writeFile(
resolve(values.output, "summary.json"),
JSON.stringify(report, null, 2),
);
process.stdout.write(JSON.stringify(report) + "\n");
if (!rejected || !named || failure || before !== after) process.exitCode = 1;
experiments/analysis-efficiency/probe-browser.mjs
+85
View File
@@ -0,0 +1,85 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
const { values } = parseArgs({
options: {
"runtime-root": { type: "string" },
scenario: { type: "string" },
output: { type: "string" },
},
});
if (!values["runtime-root"] || !values.scenario || !values.output)
throw new Error(
"Use --runtime-root ABS --scenario JSON --output NEW_ABS_DIR",
);
await mkdir(values.output, { mode: 0o700 });
const scenario = JSON.parse(await readFile(values.scenario, "utf8"));
const client = new Client({ name: "rea-real-browser-app", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [resolve(values["runtime-root"], "scripts/rea.mjs"), "mcp"],
env: {
...process.env,
REA_LOG_LEVEL: "silent",
NODE_OPTIONS: "--max-old-space-size=512",
},
stderr: "pipe",
});
let failure;
try {
await client.connect(transport);
transport.stderr?.resume();
const reply = await client.callTool(
{ name: "capture_browser_scenario", arguments: scenario },
{ timeout: 120_000 },
);
await writeFile(resolve(values.output, "reply.json"), JSON.stringify(reply));
const value =
reply.structuredContent ??
JSON.parse(
reply.content.find((item) => item.type === "text")?.text ?? "null",
);
if (reply.isError) {
failure = value.error;
} else {
const capture = value.normalized_result;
await writeFile(
resolve(values.output, "summary.json"),
JSON.stringify(
{
evidence_id: value.evidence_id,
browser: capture.browser,
scenario: capture.scenario,
completeness: capture.completeness,
step_count: capture.steps.length,
event_count: capture.events.retained,
limitations: value.limitations,
},
null,
2,
),
);
process.stdout.write(
JSON.stringify({
evidence_id: value.evidence_id,
browser: capture.browser,
step_count: capture.steps.length,
}) + "\n",
);
}
} finally {
await client.close();
}
if (failure) {
await writeFile(
resolve(values.output, "failure.json"),
JSON.stringify(failure, null, 2),
);
process.stdout.write(
JSON.stringify({ code: failure.code, message: failure.message }) + "\n",
);
process.exitCode = 1;
}
experiments/analysis-efficiency/provider-environment.json
+10
View File
@@ -0,0 +1,10 @@
{
"namespace_prefixes": ["REA_", "HOPPER_"],
"additional_keys": [
"GHIDRA_INSTALL_DIR",
"JAVA_HOME",
"JAVA_TOOL_OPTIONS",
"JDK_JAVA_OPTIONS",
"_JAVA_OPTIONS"
]
}
experiments/analysis-efficiency/questions/android.txt
+3
View File
@@ -0,0 +1,3 @@
Statically investigate the actual shipped APK at {{artifact_path}} using only the registered REA tools. Report the exact artifact SHA-256, package name, version name/code, every class-name match for the case-sensitive substring MainActivity, and all onCreate method descriptors reported for org.schabi.newpipe.MainActivity. Preserve reported signatures and explicit unknowns. For each onCreate entry include reported_signature, overload_index and dex_descriptor exactly as reported, preserving null. Final answer must be JSON with artifact_sha256, package_name, version_name, version_code, matches, onCreate, evidence_ids, limitations. Include real Evidence IDs supporting each result. Do not execute the APK or use shell, host file tools, network or other agents. Stop once these facts are supported.
When a client JavaScript executor is available, compose the required REA calls in one execution and return only selected facts from normalized_result, real Evidence IDs, target identity from subject.digest.sha256 and limitations. Parse original MCP text content if necessary. Do not return raw_result, whole Evidence or unrelated inventories. All matching classes and onCreate overloads must still be included.
experiments/analysis-efficiency/questions/browser.txt
+1
View File
@@ -0,0 +1 @@
Observe the real public Excalidraw application through REA capture_browser_scenario using exactly this authorized fresh-profile headless scenario: {"browser": {"mode": "launch", "executable_path": "/home/yann/.cache/ms-playwright/chromium-1234/chrome-linux64/chrome", "headless": true}, "start_url": {"url": "https://excalidraw.com/", "query": []}, "actions": [{"step_id": "settled", "action": "wait_for_timeout", "duration_ms": 2000}], "capture": {"at_end": ["url", "dom", "accessibility"], "events": ["network", "page-errors"]}}. Report actual browser product/version, start origin, final sanitized URL, DOM/accessibility artifact SHA-256 and byte counts when captured, step count, network/page-error observations, completeness, provider-owned process cleanup, real Evidence ID and limitations. Preserve unavailable and partial capture states. This is one runtime observation, not a claim about original source or all behavior. Do not supply credentials, interact with other targets or inject JavaScript. Final JSON fields: browser, start_origin, final_url, dom, accessibility, step_count, network, page_errors, completeness, evidence_ids, limitations.
experiments/analysis-efficiency/questions/electron-inventory.txt
+1
View File
@@ -0,0 +1 @@
Statically inventory {{artifact_path}} using only the registered REA tools. Open this ASAR as an archive, inspect its artifact inventory, and close the session. Report exact target SHA-256, the complete inventory manifest, counts of every entry_kind across all occurrences, the original occurrence records for root paths package.json, main.js and preload.js if present, all integrity contradictions, inspection coverage, unexplored branches, real Evidence IDs and limitations. This question asks about package bytes and inventory, not AST interpretation or runtime behavior. Do not execute target code, analyze all JavaScript, extract files, launch a GUI, use shell/file tools/network or other agents. When a client JavaScript executor is available, compose original calls in dependency order and return only requested fields from normalized_result; the full inventory is the Evidence nested in the inspect_artifact inventory_artifact substep. Use Evidence.subject.digest.sha256 for the target identity and retain both inspection and inventory Evidence IDs. Final JSON fields: artifact_sha256, manifest, entry_kind_counts, requested_paths, integrity_contradictions, coverage, unexplored_branches, evidence_ids, limitations. Stop when these facts and cleanup are supported.
experiments/analysis-efficiency/questions/evm.txt
+1
View File
@@ -0,0 +1 @@
Statically inspect the published historical WETH hex carrier at {{artifact_path}} using only REA inspect_evm_interface with encoding hex. Report original carrier SHA-256/byte count, decoded bytecode SHA-256/byte count, every inferred function selector with bytecode_offset, dispatch, inferred_arguments and inferred_state_mutability exactly including null. Preserve evidence_kind, discovery_completeness, bytecode kind/hardfork/deployment_authenticity, runtime_execution, real Evidence IDs and limitations. This file comes from the official Ethereum published historical eth_getCode example; do not infer live deployment authenticity, original source signatures or runtime behavior. No RPC, transactions, target execution, shell/file tools or other agents. Final JSON fields: artifact_sha256, artifact_bytes, bytecode_sha256, bytecode_bytes, functions, evidence_kind, discovery_completeness, bytecode_kind, hardfork, deployment_authenticity, runtime_execution, evidence_ids, limitations.
experiments/analysis-efficiency/questions/firmware.txt
+1
View File
@@ -0,0 +1 @@
Statically inspect {{artifact_path}} using only the registered REA tools. Report exact target SHA-256 and every reported firmware region with offsets, signatures, reported extents, provider provenance and examined coverage. Preserve unknowns and limitations; a signature scan does not prove partition boundaries or boot/runtime validity. Do not execute target code, extract files, launch a GUI, use shell/file tools/network or other agents. When a client JavaScript executor is available, select the requested normalized_result fields, Evidence.subject.digest.sha256 and real Evidence IDs before returning output. Final JSON fields: artifact_sha256, result, evidence_ids, limitations. Stop when the requested facts are supported.
experiments/analysis-efficiency/questions/javascript-selected.txt
+1
View File
@@ -0,0 +1 @@
Investigate the actual shipped Obsidian ASAR at {{artifact_path}} using REA analyze_javascript_application with source_paths ["package.json","main.js"] and detail summary. Keep complete inventory identity while analyzing only those exact sources. Read retained module views for each main.js module identity; module pages may contain several identities for a path, so preserve their exact node_id values. package.json is package metadata and need not be a module. Report selected input path/root artifact SHA-256, requested_paths and unselected_files, graph/statistics, Electron observations, exact selected main.js module identities/types/digests, analysis and semantic coverage, retained source Evidence ID, view Evidence IDs and limitations. Do not execute target JavaScript, analyze other sources, extract files, use network/shell/file tools or other agents. Static AST results do not prove actual runtime behavior or original source. Final JSON fields: input_path, artifact_sha256, source_selection, statistics, electron, modules, coverage, evidence_ids, limitations.
experiments/analysis-efficiency/questions/managed-nonempty.txt
+1
View File
@@ -0,0 +1 @@
Statically inspect the shipped ShareX HelpersLib assembly at {{artifact_path}} using only registered REA tools. Report exact target SHA-256 and module MVID, native ModuleRef/PInvoke/native-implementation counts, every original ModuleRef record, the first three complete PInvoke declaration records in producer order, coverage and limitations. Preserve tokens, import scope/name, flags, nulls and verification states exactly; declarations do not prove native-library/export availability or runtime behavior. No assembly load, native symbol lookup, target execution, GUI, shell/file tools, network or other agents. Final JSON fields: artifact_sha256, module_mvid, summary, module_refs, pinvoke_first3, coverage, evidence_ids, limitations. Stop once those selected facts are supported. Use compact selected evidence when the client supports it, but do not invent fields hidden by client output clipping.
experiments/analysis-efficiency/questions/managed.txt
+1
View File
@@ -0,0 +1 @@
Statically inspect {{artifact_path}} using only the registered REA tools. Report exact target SHA-256, assembly name/version, module name/MVID, managed classification and architecture with its evidence, every reported target framework, and all declared native ModuleRef/PInvoke boundaries with coverage and limitations. Do not load or execute the assembly, launch a GUI, use shell/file tools/network or other agents. When a client JavaScript executor is available, compose the required original MCP calls and return only requested normalized_result fields, target identity from subject.digest.sha256 and real Evidence IDs. Do not return whole Evidence, raw_result or unrelated resources/custom-attribute inventories. Preserve dependency order for operations sharing the selected managed target. Final JSON fields: artifact_sha256, assembly, module, classification, target_frameworks, native_boundaries, coverage, evidence_ids, limitations. Stop when these facts are supported.
experiments/analysis-efficiency/questions/native.txt
+1
View File
@@ -0,0 +1 @@
Statically inspect {{artifact_path}} using only the registered REA tools and Ghidra. The pinned ELF header was independently observed to report entry address 0x403a50; treat that address as the analyst's seed, not as a function-name assumption. Keep default auto-analysis enabled and allow at least 600 seconds for native analysis (including the script deadline if using a bridge). Open the target with provider_id ghidra, resolve the containing procedure for that address, analyze it, and close the ephemeral session in a finally block or a subsequent cleanup call. Report exact target SHA-256, the entry_address, containing_procedure, procedure metadata, the first 512 UTF-16 units of pseudocode, caller_count, callee_count, real Evidence IDs and limitations. Do not execute target code, launch a GUI, use shell/file tools/network or other agents. When a client JavaScript executor is available, compose the original calls in dependency order and select requested fields before returning output; use Evidence.subject.digest.sha256 for the target identity. Final JSON fields: artifact_sha256, entry_address, containing_procedure, procedure, pseudocode_prefix, caller_count, callee_count, evidence_ids, limitations. The address seed applies only to the pinned jq artifact in corpus.json. Stop when these facts and cleanup are supported.
experiments/analysis-efficiency/questions/wasm.txt
+1
View File
@@ -0,0 +1 @@
Statically inspect the real shipped SQLite WASM at {{artifact_path}} using only REA tools. Report exact selected artifact SHA-256, valid/invalid status, section count, import count, export count, first four complete export WAT forms in producer order, decoded WAT SHA-256 and code-unit length, WABT release/source commit, and exact runtime_execution/network_fetch states. If complete delivery exceeds the MCP frame, use the exact retained reference for summary and wasm exports views. Preserve original strings, uncertainties and real Evidence IDs. Do not execute WASM, fetch URLs or use shell/file tools. Final JSON fields: artifact_sha256, validation, section_count, import_count, export_count, exports_first4, wat_sha256, wat_code_units, wabt_release, wabt_source_commit, runtime_execution, network_fetch, evidence_ids, limitations.
experiments/analysis-efficiency/resource-run.py
+158
View File
@@ -0,0 +1,158 @@
"""Run one owned experiment in a bounded, transient Linux systemd scope."""
import argparse
import json
import os
from pathlib import Path
import subprocess
import sys
import time
import uuid
def available_bytes():
for line in Path('/proc/meminfo').read_text().splitlines():
if line.startswith('MemAvailable:'):
return int(line.split()[1]) * 1024
raise RuntimeError('MemAvailable unavailable')
def scope_has_processes(group):
if group is None:
return False
try:
return bool((group / 'cgroup.procs').read_text().strip())
except FileNotFoundError:
return False
def final_resource_events(group):
"""An exited transient cgroup may disappear; missing counters are unknown."""
if group is None:
return None
try:
result = {name: dict((key, int(value)) for key, value in
(line.split() for line in (group / name).read_text().splitlines()))
for name in ['memory.events', 'pids.events']}
except (OSError, ValueError):
return None
required = {'memory.events': ['high', 'max', 'oom', 'oom_kill'],
'pids.events': ['max']}
if not all(all(key in result[name] for key in keys)
for name, keys in required.items()):
return None
return result
def stop_owned_scope(unit, process, group):
subprocess.run(['systemctl', '--user', 'stop', '--no-block', unit],
check=False, timeout=5)
for attempt in range(2):
deadline = time.monotonic() + 10
while process.poll() is None or scope_has_processes(group):
if time.monotonic() >= deadline:
break
time.sleep(0.1)
else:
return
if attempt == 1:
raise RuntimeError('Cleanup unverified; retain owned scope ' + unit)
subprocess.run(['systemctl', '--user', 'kill', '--kill-who=all',
'--signal=SIGKILL', unit], check=False, timeout=5)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--task-ceiling', type=int, default=192)
parser.add_argument('command', nargs=argparse.REMAINDER)
options = parser.parse_args()
if options.task_ceiling <= 0:
parser.error('--task-ceiling must be positive')
args = options.command
if args and args[0] == '--':
args = args[1:]
if not args:
raise SystemExit('Usage: resource-run.py -- COMMAND ARGS...')
gib = 1024 ** 3
if available_bytes() < 8 * gib:
raise SystemExit('Refusing experiment: less than 8 GiB available')
unit = 'rea-efficiency-' + uuid.uuid4().hex + '.scope'
cpu = max(os.sched_getaffinity(0))
command = [
'systemd-run', '--user', '--scope', '--unit=' + unit,
'-p', 'MemoryMax=4G', '-p', 'MemoryHigh=3G', '-p', 'MemorySwapMax=0',
'-p', 'CPUQuota=100%', '-p', 'TasksMax=' + str(options.task_ceiling),
'taskset', '-c', str(cpu), *args,
]
process = subprocess.Popen(command)
peak = 0
peak_tasks = 0
events = {}
cpu_stats = {}
abort = None
final_events = None
started = time.monotonic()
try:
group = None
while process.poll() is None:
if group is None:
reply = subprocess.run(
['systemctl', '--user', 'show', unit, '--value',
'--property=ControlGroup'],
capture_output=True, text=True, check=False,
)
path = reply.stdout.strip()
if path:
group = Path('/sys/fs/cgroup') / path.lstrip('/')
current = group / 'memory.current' if group else None
if current and current.exists():
try:
peak = max(peak, int(current.read_text()))
peak_tasks = max(peak_tasks, int((group / 'pids.current').read_text()))
cpu_stats = dict((key, int(value)) for key, value in
(line.split() for line in (group / 'cpu.stat').read_text().splitlines()))
for event_file in ['memory.events', 'pids.events']:
events[event_file] = dict(
(key, int(value)) for key, value in
(line.split() for line in (group / event_file).read_text().splitlines())
)
except FileNotFoundError:
# A completed transient scope can disappear between reads.
pass
if peak > 3 * gib:
abort = 'scope_memory_early_stop'
elif available_bytes() < 4 * gib:
abort = 'host_available_memory_early_stop'
elif events.get('pids.events', {}).get('max', 0):
abort = 'scope_task_limit_observed'
if abort:
stop_owned_scope(unit, process, group)
break
time.sleep(0.1)
# Read while the owned group still exists, before wait/cleanup can remove it.
final_events = final_resource_events(group)
code = process.wait()
except BaseException:
stop_owned_scope(unit, process, group)
raise
finally:
if scope_has_processes(group):
final_events = None
stop_owned_scope(unit, process, group)
print(json.dumps({
'resource_scope': unit, 'peak_cgroup_bytes_sampled': peak,
'elapsed_seconds': round(time.monotonic() - started, 3),
'abort': abort, 'hard_memory_bytes': 4 * gib,
'early_stop_bytes': 3 * gib, 'cpu_quota_percent': 100,
'cpu_affinity': [cpu],
'task_ceiling': options.task_ceiling, 'peak_tasks_sampled': peak_tasks,
'resource_events_last_observed': events,
'resource_events_final': final_events,
'resource_counters_verified': final_events is not None,
'cpu_stat_last_observed': cpu_stats,
}), file=sys.stderr)
raise SystemExit(125 if abort else code)
if __name__ == '__main__':
main()
experiments/analysis-efficiency/review-model.py
+265
View File
@@ -0,0 +1,265 @@
"""Check selected model facts against public producer observations, not exit status."""
import argparse
import hashlib
import json
from pathlib import Path
import re
def read(path):
return json.loads(path.read_text())
def strings(value):
if isinstance(value, str):
yield value
elif isinstance(value, list):
for entry in value:
yield from strings(entry)
elif isinstance(value, dict):
for entry in value.values():
yield from strings(entry)
def evidence_ids(value):
return {s for s in strings(value) if re.fullmatch(r'ev_[a-f0-9]{64}', s)}
def decode(result):
value = result.get('structured_content', result.get('structuredContent'))
if value is not None:
return value
for entry in result.get('content', []):
if entry.get('type') == 'text':
try:
return json.loads(entry['text'])
except json.JSONDecodeError:
pass
return None
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
parser.add_argument('--trial', type=Path, required=True)
parser.add_argument('--reference', type=Path)
parser.add_argument('--resource-log', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
summary = read(args.trial / 'summary.json')
settings = read(args.trial / 'trial-settings.json')
answer_text = (args.trial / 'answer.txt').read_text().strip()
fenced = answer_text.startswith('```json\n') and answer_text.endswith('\n```')
answer = json.loads(answer_text[8:-4] if fenced else answer_text)
reference = read(args.reference / 'answer.json') if args.reference else None
observed_ids = set()
producer_results = {}
tool_errors = []
for line in (args.trial / 'events.jsonl').open():
event = json.loads(line)
item = event.get('item', {})
if event.get('type') != 'item.completed' or item.get('type') != 'mcp_tool_call':
continue
result = item.get('result') or {}
parsed = decode(result)
if item['status'] == 'completed':
# Only response text contributes observed IDs, never model arguments.
for text in strings(result):
observed_ids.update(re.findall(r'ev_[a-f0-9]{64}', text))
if parsed is not None:
producer_results.setdefault(item['tool'], []).append(parsed)
else:
tool_errors.append({'tool': item['tool'], 'arguments': item['arguments'], 'response': parsed if parsed is not None else result})
if parsed is not None and parsed.get('error', {}).get('code') == 'resource_constraint':
reference_value = parsed['error'].get('details', {}).get('reported_limits', {}).get('evidence_reference')
observed_ids.update(evidence_ids(reference_value))
reported_ids = evidence_ids(answer.get('evidence_ids'))
checks = {
'reported_evidence_ids_observed_in_responses': bool(reported_ids) and reported_ids <= observed_ids,
'temporary_profile_removed': read(args.trial / 'cleanup.json')['profile_removed'],
}
if summary.get('selected_artifact_sha256_before') is not None:
checks['selected_artifact_bytes_unchanged'] = summary['selected_artifact_bytes_unchanged']
if reference is not None:
checks['target_digest'] = answer.get('artifact_sha256') == reference['artifact_sha256']
differences = {}
def check(name, actual, expected):
checks[name] = actual == expected
if not checks[name]:
differences[name] = {'answer': actual, 'producer': expected}
domain = args.domain
if domain == 'wasm':
for key in ['validation', 'section_count', 'import_count', 'export_count', 'exports_first4', 'wat_sha256', 'wat_code_units', 'runtime_execution', 'network_fetch']:
check(key, answer.get(key), reference[key])
for key in ['release', 'source_commit']:
check('wabt_' + key, answer.get('wabt_' + key), reference['tool_profile'][key])
elif domain == 'evm':
n = producer_results['inspect_evm_interface'][-1]['normalized_result']
for key, expected in {'artifact_bytes': n['artifact']['bytes'], 'bytecode_sha256': n['bytecode']['sha256'], 'bytecode_bytes': n['bytecode']['bytes'], 'functions': n['functions'], 'evidence_kind': n['evidence_kind'], 'discovery_completeness': n['discovery_completeness'], 'runtime_execution': n['runtime_execution'], 'bytecode_kind': n['bytecode']['kind'], 'hardfork': n['bytecode']['hardfork'], 'deployment_authenticity': n['bytecode']['deployment_authenticity']}.items():
check(key, answer.get(key), expected)
elif domain == 'android':
for key in ['package_name', 'version_name', 'version_code', 'matches']:
check(key, answer.get(key), reference[key])
fields = ['reported_signature', 'overload_index', 'dex_descriptor']
checks['onCreate_fields_present_including_explicit_null'] = all(all(k in row for k in fields) for row in answer.get('onCreate', []))
check('all_onCreate_descriptors', [{k: row.get(k) for k in fields} for row in answer.get('onCreate', [])], [{k: row[k] for k in fields} for row in reference['onCreate']])
elif domain == 'managed':
n = reference['artifact']
for group, fields in [('assembly', ['name', 'version']), ('module', ['name', 'mvid'])]:
for field in fields:
check(group + '.' + field, answer.get(group, {}).get(field), n[group][field])
for key in ['classification', 'target_frameworks']:
check(key, answer.get(key), n[key])
for key in ['module_refs', 'pinvoke_imports']:
check(key, answer.get('native_boundaries', {}).get(key), reference['boundaries'][key])
check('artifact_coverage', answer.get('coverage', {}).get('artifact'), n['coverage'])
check('native_boundaries_coverage', answer.get('coverage', {}).get('native_boundaries'), reference['boundaries']['coverage'])
elif domain == 'managed-nonempty':
n = reference['boundaries']
check('module_mvid', answer.get('module_mvid'), n['module']['mvid'])
check('summary', {key: answer.get('summary', {}).get(key) for key in n['summary']}, n['summary'])
check('all_original_module_refs', answer.get('module_refs'), n['module_refs'])
check('first3_complete_pinvoke_records', answer.get('pinvoke_first3'), n['pinvoke_imports'][:3])
check('coverage', answer.get('coverage'), n['coverage'])
elif domain == 'firmware':
source = producer_results['inspect_firmware_regions'][-1]
n = source['normalized_result']
result = answer['result']
check('input_size', result.get('input_size'), n['input_size'])
check('examined_coverage', result.get('coverage', result.get('examined_coverage')), n['coverage'])
fields = ['provider_id', 'offset', 'reported_size', 'signature', 'description', 'reported_confidence', 'size_basis']
check('all_original_region_records', [{k: row.get(k) for k in fields} for row in result.get('regions', [])], [{k: row[k] for k in fields} for row in n['regions']])
engine = result.get('engine', result.get('provider_provenance', {}).get('engine', {}))
for key in ['name', 'version']:
check('engine.' + key, engine.get(key), n['engine'][key])
check('engine_executable_digest', engine.get('executable_sha256', engine.get('artifact_sha256')), n['engine']['executable_sha256'])
elif domain == 'inventory':
for key in ['manifest', 'entry_kind_counts', 'integrity_contradictions']:
check(key, answer.get(key), reference[key])
original = producer_results['inspect_artifact'][-1]['normalized_result']
if original.get('kind') == 'summary':
branches = original['summary']['inspection']['unexplored_branches']
else:
branches = original['unexplored_branches']
# Caller-selected integrity policy participates in atomic Evidence and
# branch identity. Match those IDs to this original response, not a
# reference invocation that may use a different policy.
check('unexplored_branches_with_original_ids', answer.get('unexplored_branches'), branches)
coverage = answer.get('coverage', {})
check('inspection_coverage', coverage.get('inspection', coverage), reference['coverage'])
paths = answer.get('requested_paths')
if isinstance(paths, dict):
paths = [row for value in paths.values() for row in (value if isinstance(value, list) else [value]) if row is not None]
canonical = lambda row: json.dumps(row, sort_keys=True, separators=(',', ':'))
check('original_selected_path_records', sorted(map(canonical, paths or [])), sorted(map(canonical, reference['requested_paths'])))
elif domain == 'javascript':
for key in ['input_path', 'source_selection', 'statistics', 'electron']:
check(key, answer.get(key), reference[key])
coverage = answer['coverage']
expected = reference['coverage']
for key in ['application_unknowns', 'semantic_unknowns']:
check('coverage.' + key, coverage.get(key), expected[key])
for layer in ['application', 'semantic']:
for key, value in expected[layer].items():
if key in ['limits', 'families']: continue
check('coverage.' + layer + '.' + key, coverage[layer].get(key), value)
# Budgets vary with parent heap and belong to this actual invocation.
# Exact optional provenance remains a separate strict check; omissions
# must not be hidden by accepting the coarse coverage summary.
original = producer_results['analyze_javascript_application'][-1]['normalized_result']['summary']['coverage']
check('full_coverage_provenance', coverage, original)
# The question names types/digests, not literal answer property names.
# Exact node IDs select the producer's main.js records. No source or
# Evidence values are rewritten to accept these answer labels.
select = lambda rows: sorted([{
'node_id': row.get('node_id'),
'kind': row.get('kind', row.get('type')),
'identity': row.get('identity'),
'hashes': row.get('hashes', row.get('digests')),
} for row in rows], key=lambda row: row['node_id'])
check('all_selected_module_identities', select(answer.get('modules', [])), select(reference['modules']))
elif domain == 'native':
for key in ['entry_address', 'pseudocode_prefix', 'caller_count', 'callee_count']:
check(key, answer.get(key), reference[key])
containing = answer['containing_procedure']
expected_containing = reference['containing_procedure']['procedure']
for key in ['address', 'name']:
check('containing_procedure.' + key, containing.get(key, containing.get('procedure', {}).get(key)), expected_containing[key])
procedure = answer['procedure']
expected = reference['procedure']
for key in ['address', 'name', 'classification', 'signature', 'locals']:
check('procedure.' + key, procedure.get(key), expected[key])
check('procedure.body_ranges', procedure.get('body_ranges', procedure.get('body', {}).get('ranges')), expected['body']['ranges'])
check('procedure.total_bytes', procedure.get('total_bytes', procedure.get('body', {}).get('total_bytes')), expected['body']['total_bytes'])
calls = summary['completed_mcp_calls']
checks['default_analysis_not_disabled'] = all(not call['arguments'].get('skip_auto_analysis', False) for call in calls if call['tool'] == 'open_binary')
checks['session_close_completed'] = any(call['tool'] == 'close_binary' and call['status'] == 'completed' for call in calls)
elif domain == 'browser':
source = producer_results['capture_browser_scenario'][-1]
n = source['normalized_result']
for key in ['browser', 'completeness']:
check(key, answer.get(key), n[key])
check('start_origin', answer.get('start_origin'), n['scenario']['start_origin'])
step_count = answer.get('step_count')
check('step_count', step_count.get('captured_steps') if isinstance(step_count, dict) else step_count, len(n['steps']))
final = n['steps'][-1]
expected_url = final['after_url'] if isinstance(answer.get('final_url'), dict) else final['after_url']['url']
check('final_url', answer.get('final_url'), expected_url)
for key in ['dom', 'accessibility']:
expected = final['artifacts'][key]
actual = answer.get(key, {})
if expected['state'] == 'captured':
actual = actual.get('value', actual)
for field in ['sha256', 'bytes']:
check(key + '.' + field, actual.get(field), expected['value'][field])
else:
check(key + '.state', actual.get('state'), expected['state'])
checks['owned_browser_terminated'] = n['browser']['cleanup'] == 'terminated-owned-process'
events = n['events']['items']
page_errors = [event for event in events if event['kind'] == 'page-error']
if isinstance(answer.get('page_errors'), dict):
check('page_error_count', answer['page_errors'].get('count'), len(page_errors))
network = answer.get('network', {})
if 'observed_transactions' in network:
check('network_transaction_count', network['observed_transactions'], len({event['transaction_id'] for event in events if event.get('transaction_id') is not None}))
if 'observed_response_statuses' in network:
statuses = {}
for event in events:
if event['kind'] == 'response':
status = str(event['status'])
statuses[status] = statuses.get(status, 0) + 1
check('network_response_statuses', network['observed_response_statuses'], statuses)
scopes = [json.loads(line) for line in args.resource_log.read_text().splitlines() if line.startswith('{')]
scope = next(value for value in reversed(scopes) if 'resource_scope' in value)
checks['no_resource_abort'] = scope['abort'] is None
events = scope.get('resource_events_final')
checks['resource_counters_verified'] = scope.get('resource_counters_verified') is True
checks['no_observed_oom_or_task_limit'] = (
isinstance(events, dict)
and all(events.get('memory.events', {}).get(k) == 0 for k in ['max', 'oom', 'oom_kill'])
and events.get('pids.events', {}).get('max') == 0
)
checks['no_resource_limit_event'] = (
checks['resource_counters_verified']
and checks['no_observed_oom_or_task_limit']
and events['memory.events'].get('high') == 0
)
report = {
'domain': domain, 'checks': checks, 'all_selected_checks_pass': all(checks.values()),
'differences': differences, 'tool_errors': tool_errors, 'summary': summary,
'fingerprints': {key: value for key, value in settings.items() if 'sha256' in key or key in ['runtime_revision', 'enabled_tools', 'authority']},
'answer_file_sha256': hashlib.sha256((args.trial / 'answer.txt').read_bytes()).hexdigest(),
'answer_had_json_fence': fenced, 'reported_evidence_ids': sorted(reported_ids), 'resource': scope,
'scope': 'Fixed selected-fact review against actual public producer observations. Equivalent explicitly named answer representations are accepted; unknowns/nulls remain checked. Limitations, unsupported claims and broader behavior still require transcript review. Token usage and failed calls are retained; zero exit status is not a factual grade.',
}
with args.output.open('x') as output:
output.write(json.dumps(report, indent=2, ensure_ascii=False) + '\n')
print(json.dumps({'domain': domain, 'passed': report['all_selected_checks_pass'], 'failed_checks': [key for key, value in checks.items() if not value]}))
raise SystemExit(0 if report['all_selected_checks_pass'] else 1)
if __name__ == '__main__':
main()
experiments/analysis-efficiency/verify-migration.mjs
+163
View File
@@ -0,0 +1,163 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { readFile, writeFile, mkdir } from "node:fs/promises";
import { isAbsolute, resolve } from "node:path";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
const { values } = parseArgs({
options: {
"runtime-root": { type: "string" },
revision: { type: "string" },
bundle: { type: "string" },
"inspection-reply": { type: "string" },
output: { type: "string" },
},
});
for (const key of ["runtime-root", "bundle", "inspection-reply", "output"])
assert.ok(values[key] && isAbsolute(values[key]), `Supply --${key} ABS`);
assert.ok(values.revision);
await mkdir(values.output, { mode: 0o700 });
const sha = (value) => createHash("sha256").update(value).digest("hex");
const bundleBytes = await readFile(values.bundle);
const oldBundle = JSON.parse(bundleBytes.toString("utf8"));
const oldWasm = oldBundle.records.find(
(record) => record.operation === "inspect_wasm_artifact",
);
assert.ok(oldWasm);
const inspectionBytes = await readFile(values["inspection-reply"]);
const oldInspection = JSON.parse(
inspectionBytes.toString("utf8"),
).structuredContent;
assert.equal(oldInspection.operation, "inspect_artifact");
const definitions = new Map();
const client = new Client({
name: "rea-real-evidence-migration",
version: "1",
});
// An absent explicitly selected WABT directory makes accidental re-analysis
// fail. Imported/portable views must operate on existing Evidence alone.
const transport = new StdioClientTransport({
command: process.execPath,
args: [resolve(values["runtime-root"], "scripts/rea.mjs"), "mcp"],
env: {
...process.env,
REA_LOG_LEVEL: "silent",
NODE_OPTIONS: "--max-old-space-size=512",
REA_WABT_BIN_DIRECTORY: resolve(values.output, "absent-wabt"),
},
stderr: "pipe",
});
const calls = [];
const call = async (name, arguments_) => {
const reply = await client.callTool(
{ name, arguments: arguments_ },
{ toolDefinition: definitions.get(name), timeout: 120_000 },
);
await writeFile(
resolve(values.output, `call-${calls.length + 1}.json`),
JSON.stringify(reply),
);
assert.notEqual(reply.isError, true, JSON.stringify(reply));
const result = reply.structuredContent ?? JSON.parse(reply.content[0].text);
calls.push({
name,
arguments:
name === "inspect_analysis_view" && arguments_.source.kind === "inline"
? {
source: {
kind: "inline",
original_evidence_id: arguments_.source.evidence.evidence_id,
},
view: arguments_.view,
}
: arguments_,
evidence_id: result.evidence_id ?? null,
});
return result;
};
let report;
try {
await client.connect(transport);
transport.stderr?.resume();
for (const definition of (await client.listTools()).tools)
definitions.set(definition.name, definition);
await call("import_evidence_bundle", { path: values.bundle });
const source = {
kind: "retained-evidence",
evidence_id: oldWasm.evidence_id,
};
const summary = await call("inspect_analysis_view", {
source,
view: { kind: "summary" },
});
const exports = await call("inspect_analysis_view", {
source,
view: { kind: "wasm", facet: "exports", offset: 0, limit: 4 },
});
assert.equal(
summary.normalized_result.parent_evidence_id,
oldWasm.evidence_id,
);
assert.equal(summary.subject.digest.sha256, oldWasm.subject.digest.sha256);
assert.equal(
summary.normalized_result.summary.wat.sha256,
oldWasm.normalized_result.wat.sha256,
);
assert.deepEqual(
exports.normalized_result.item,
oldWasm.normalized_result.exports.slice(0, 4),
);
const inventory = await call("inspect_analysis_view", {
source: { kind: "inline", evidence: oldInspection },
view: { kind: "summary" },
});
assert.equal(
inventory.normalized_result.parent_evidence_id,
oldInspection.evidence_id,
);
const oldAtomic = oldInspection.normalized_result.substeps[0].evidence;
assert.deepEqual(
inventory.normalized_result.summary.manifest,
oldAtomic.normalized_result.manifest,
);
assert.deepEqual(inventory.normalized_result.summary.inventory_evidence_ids, [
oldAtomic.evidence_id,
]);
const exportedPath = resolve(values.output, "exported-after-import.json");
await call("export_evidence_bundle", { path: exportedPath });
const exported = JSON.parse(await readFile(exportedPath, "utf8"));
assert.deepEqual(
exported.records.find(
(record) => record.evidence_id === oldWasm.evidence_id,
),
oldWasm,
);
assert.equal(sha(await readFile(values.bundle)), sha(bundleBytes));
assert.equal(
sha(await readFile(values["inspection-reply"])),
sha(inspectionBytes),
);
report = {
runtime_revision: values.revision,
original_bundle_sha256: sha(bundleBytes),
original_inspection_reply_sha256: sha(inspectionBytes),
source_files_unchanged: true,
imported_wasm_source_evidence_id: oldWasm.evidence_id,
original_wasm_record_preserved_exactly: true,
typed_summary_and_first4_exports_match: true,
portable_old_inspection_and_atomic_inventory_identity_preserved: true,
original_inventory_evidence_id: oldAtomic.evidence_id,
calls,
scope:
"Actual old complete Evidence imported or passed inline to the new runtime; original authentication, IDs, records, source bytes and selected facts preserved. WABT is unavailable and no producer analysis is requested. Existing file import/export is used, not an engine database cache.",
};
} finally {
await client.close();
}
await writeFile(
resolve(values.output, "summary.json"),
JSON.stringify(report, null, 2),
);
process.stdout.write(JSON.stringify(report) + "\n");
src/application/analysisView/AnalysisViewService.ts
+1
View File
@@ -69,6 +69,7 @@ export const inspectAnalysisViewValidated = (
evidenceId: parent.evidence_id,
operation: parent.operation,
normalizedResult: parent.normalized_result,
subject: parent.subject,
limitations: parent.limitations,
parameters: parent.parameters,
artifact:
src/application/wasm/WasmArtifactService.ts
+2 -5
View File
@@ -1,4 +1,3 @@
import { createHash } from "node:crypto";
import { isAbsolute } from "node:path";
import type { ExecutionOptions } from "../AnalysisProvider.js";
import type { WasmArtifactPort } from "./WasmArtifactPort.js";
@@ -12,6 +11,7 @@ import { createEvidence, type Evidence } from "../../domain/evidence.js";
import {
inspectWasmArtifactInputSchema,
wasmArtifactSchema,
wasmArtifactMatchesSubject,
} from "../../domain/wasm/wasmArtifact.js";
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
import { err, ok, type Result } from "../../domain/result.js";
@@ -66,10 +66,7 @@ export class WasmArtifactService {
if (
!report.success ||
report.data.artifact.path !== input.data.path ||
execution.subject?.path !== input.data.path ||
execution.subject.sha256 !== report.data.artifact.sha256 ||
createHash("sha256").update(report.data.wat.text).digest("hex") !==
report.data.wat.sha256 ||
!wasmArtifactMatchesSubject(report.data.artifact, execution.subject) ||
JSON.stringify(report.data.glue.map((item) => item.artifact.path)) !==
JSON.stringify(input.data.glue_paths) ||
JSON.stringify(report.data.candidates.map((item) => item.path)) !==
src/cli/wasmCommands.ts
+4 -5
View File
@@ -1,7 +1,6 @@
import { z } from "incur";
import { resolve } from "node:path";
import { createWasmArtifactService } from "../composition/wasm.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { wasmArtifactModule } from "../composition/wasm.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
@@ -14,8 +13,8 @@ export const registerWasmCommands = (
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
): void => {
const service = createWasmArtifactService(environment);
cli.command(CLI_COMMANDS.inspectWasmArtifact, {
const service = wasmArtifactModule.create(environment);
cli.command(wasmArtifactModule.cliCommand, {
description: "Inspect an explicit local WASM artifact with WABT",
args: z.object({
path: z.string().describe("Local WASM artifact"),
@@ -35,7 +34,7 @@ export const registerWasmCommands = (
try {
return await logCliCommand(
logger,
CLI_COMMANDS.inspectWasmArtifact,
wasmArtifactModule.cliCommand,
async () => {
const result = await service.inspect(
{
src/composition/wasm.ts
+22
View File
@@ -1,7 +1,29 @@
import { WasmArtifactService } from "../application/wasm/WasmArtifactService.js";
import { WabtArtifactProvider } from "../wasm/WabtArtifactProvider.js";
import { INSPECT_WASM_ARTIFACT_CONTRACT } from "../contracts/wasm/wasmToolContracts.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
/** Compose the caller-supplied offline toolchain without acquisition or import-time I/O. */
export const createWasmArtifactService = (
environment: Readonly<NodeJS.ProcessEnv> = process.env,
): WasmArtifactService =>
new WasmArtifactService(new WabtArtifactProvider(environment));
/**
* Internal static descriptor for one named domain operation and its lifetime.
* CLI/MCP share this binding; the service keeps failed resource ownership so
* close can be retried. Construction performs no acquisition or target I/O.
*/
export const wasmArtifactModule = {
contract: INSPECT_WASM_ARTIFACT_CONTRACT,
cliCommand: CLI_COMMANDS.inspectWasmArtifact,
create(
environment: Readonly<NodeJS.ProcessEnv>,
injected?: WasmArtifactService,
): Pick<WasmArtifactService, "inspect" | "close"> {
const service = injected ?? createWasmArtifactService(environment);
return {
inspect: service.inspect.bind(service),
close: service.close.bind(service),
};
},
} as const;
src/contracts/analysisViewToolContracts.ts
+19 -3
View File
@@ -9,23 +9,39 @@ import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
const RETAINED_EXAMPLE_ID = `ev_${"a".repeat(64)}`;
/** Selected views of already completed layout, JavaScript and native function Evidence. */
/** Selected views of completed layout, JavaScript, native, inventory, WASM and web network Evidence. */
export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
{
name: "inspect_analysis_view",
...toolContractMetadata("inspect_analysis_view"),
kind: "application",
description:
"Inspect selected views of completed binary-layout, JavaScript application or native analyze_function Evidence without repeating analysis. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.",
'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact or inspect_web_network_capture Evidence without repeating analysis. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: "<parent_evidence_id>"}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: <complete Evidence>}. For layout, JavaScript, native, inventory and WASM sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.',
inputSchema: inspectAnalysisViewInputSchema,
outputSchema: evidenceResultOf(analysisViewResultSchema),
examples: [
{
title: "Read exact inventory paths from retained inspection Evidence",
input: {
source: {
kind: "retained-evidence",
evidence_id: RETAINED_EXAMPLE_ID,
},
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: ["package.json", "main.js"],
offset: 0,
limit: 32,
},
},
},
{
title: "Group requests from retained HAR Evidence",
input: API_SURFACE_EXAMPLE.input,
},
{
title: "Summarize retained JavaScript application Evidence",
title: "Summarize retained analysis Evidence, including WASM",
input: {
source: {
kind: "retained-evidence",
src/contracts/artifactToolContracts.ts
+14 -2
View File
@@ -9,11 +9,23 @@ import { requireOutputSchema } from "./toolOutputSchemaPrimitives.js";
import { appleAssetCatalogInputSchema } from "../domain/apple/appleAssetCatalog.js";
import { dylibResolutionInputSchema } from "../domain/apple/dylibResolution.js";
import { keyedArchiveInputSchema } from "../domain/apple/keyedArchive.js";
import { analysisDetailSchema } from "./analysisDetail.js";
/** Exact caller boundary for deterministic artifact inventory. */
export const artifactInventoryInputSchema = z.strictObject({
integrity_policy: artifactIntegrityPolicySchema,
});
/** Delivery selection belongs to the adapter, not the inventory producer. */
export const artifactInspectionInputSchema =
artifactInventoryInputSchema.extend({
detail: analysisDetailSchema
.unwrap()
.optional()
.describe(
"Omitted or complete returns the complete inspection Evidence. summary retains that complete Evidence in this session and returns its inspect_analysis_view summary; use normalized_result.parent_evidence_id for inventory views.",
),
});
/** Extraction needs no selector: it materializes every regular child file. */
export const artifactExtractionInputSchema = z.strictObject({
integrity_policy: artifactIntegrityPolicySchema,
@@ -67,8 +79,8 @@ const artifact = <
export const ARTIFACT_TOOL_CONTRACTS = [
artifact(
"inspect_artifact",
"Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph once, as the nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
artifactInventoryInputSchema,
"Inspect an active archive or application package in one call. By default returns the complete content-addressed artifact graph once, as the typed nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Optional detail summary retains the complete inspection and returns its derived summary view; read exact paths or inventory pages with inspect_analysis_view using parent_evidence_id. This changes delivery, not inventory coverage. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
artifactInspectionInputSchema,
),
artifact(
"extract_artifact",
src/contracts/toolEffects.test.ts
+2
View File
@@ -25,6 +25,8 @@ const EXECUTION_WORDING_EXCEPTIONS: Readonly<Record<string, string>> = {
"decodes bytecode in REA's bundled worker without executing it",
inspect_wasm_artifact:
"WABT validates and decompiles the module without executing it",
inspect_analysis_view:
"reports retained WASM execution/network states without performing execution",
inspect_managed_artifact: "reads metadata without executing target code",
inspect_managed_members: "reads metadata without executing target code",
inspect_managed_native_boundaries:
src/contracts/toolOutputSchemaGroups.ts
+8 -2
View File
@@ -1,4 +1,5 @@
import { z } from "zod";
import { artifactInventorySummaryViewResultSchema } from "../domain/analysisView/analysisView.js";
import { evidenceBundleSummarySchema } from "../domain/evidenceRecordSummary.js";
import { BINARY_ARCHITECTURES } from "../domain/binaryTargetTypes.js";
import {
@@ -37,7 +38,7 @@ import {
listArchitecturesSchema,
} from "../domain/native/nativeInspection.js";
import { artifactExtractionResultSchema } from "../domain/artifactGraph.js";
import { artifactInspectionResultSchema } from "../domain/artifactInspection.js";
import { artifactInspectionOutputSchema } from "../domain/artifactInspection.js";
import { interfaceBuilderAnalysisSchema } from "../domain/apple/interfaceBuilderGraph.js";
import { dylibResolutionResultSchema } from "../domain/apple/dylibResolution.js";
import { keyedArchiveResultSchema } from "../domain/apple/keyedArchive.js";
@@ -385,7 +386,12 @@ export const nativeOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
/** Exact Evidence schemas for provider-neutral artifact graph operations. */
export const artifactOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
inspect_artifact: evidenceResultOf(artifactInspectionResultSchema),
inspect_artifact: evidenceResultOf(
z.union([
artifactInspectionOutputSchema,
artifactInventorySummaryViewResultSchema,
]),
),
extract_artifact: evidenceResultOf(artifactExtractionResultSchema),
decode_interface_builder: evidenceResultOf(interfaceBuilderAnalysisSchema),
inspect_keyed_archive: evidenceResultOf(keyedArchiveResultSchema),
src/contracts/wasm/wasmToolContracts.ts
+21 -20
View File
@@ -6,24 +6,25 @@ import type { ToolContract } from "../toolContractTypes.js";
import { toolContractMetadata } from "../toolEffects.js";
import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js";
/** Optional caller-supplied WABT capability with target-free CLI/MCP parity. */
export const WASM_TOOL_CONTRACTS = [
{
name: "inspect_wasm_artifact",
...toolContractMetadata("inspect_wasm_artifact"),
kind: "artifact-provider",
description:
"Validate an explicit local WASM artifact with caller-supplied WABT 1.0.42, retain exact byte SHA-256, sections, upstream import/export WAT forms and decoded WAT inline as Evidence. Optionally associate selected JavaScript glue literals with explicit local candidates while preserving ambiguous URL/basename matches. Configure absolute REA_WABT_BIN_DIRECTORY. No installation, implicit fetch, WASM execution or original-source claim.",
inputSchema: inspectWasmArtifactInputSchema,
outputSchema: evidenceResultOf(wasmArtifactSchema),
examples: [
{
title: "Inspect a selected local WASM module",
input: {
path: "/artifacts/module.wasm",
glue_paths: [],
candidate_paths: [],
},
export const INSPECT_WASM_ARTIFACT_CONTRACT = {
name: "inspect_wasm_artifact",
...toolContractMetadata("inspect_wasm_artifact"),
kind: "artifact-provider",
description:
"Validate an explicit local WASM artifact with caller-supplied WABT 1.0.42, retain exact byte SHA-256, sections, upstream import/export WAT forms and decoded WAT inline as Evidence. Optionally associate selected JavaScript glue literals with explicit local candidates while preserving ambiguous URL/basename matches. Configure absolute REA_WABT_BIN_DIRECTORY. No installation, implicit fetch, WASM execution or original-source claim.",
inputSchema: inspectWasmArtifactInputSchema,
outputSchema: evidenceResultOf(wasmArtifactSchema),
examples: [
{
title: "Inspect a selected local WASM module",
input: {
path: "/artifacts/module.wasm",
glue_paths: [],
candidate_paths: [],
},
],
},
] as const satisfies readonly ToolContract[];
},
],
} as const satisfies ToolContract;
/** Keep the public catalog in its existing order, using its named contract. */
export const WASM_TOOL_CONTRACTS = [INSPECT_WASM_ARTIFACT_CONTRACT] as const;
src/domain/analysisView/analysisView.ts
+143 -31
View File
@@ -8,12 +8,20 @@ import {
} from "../analysisErrorCore.js";
import { digestCanonicalValue } from "../canonicalDigest.js";
import { digestSchema, prefixedDigestSchema } from "../digests.js";
import { evidenceSchema } from "../evidence.js";
import { evidenceSchema, type Evidence } from "../evidence.js";
import { jsonObjectSchema, jsonValueSchema } from "../jsonValue.js";
import { err, ok, type Result } from "../result.js";
import type { AnalysisError } from "../analysisErrorBase.js";
import { binaryLayoutSchema } from "../native/binaryLayout.js";
import { functionDossierSchema } from "../hopperValues.js";
import {
artifactGraphManifestSchema,
artifactNodeSchema,
artifactOccurrenceSchema,
artifactEdgeSchema,
integrityContradictionSchema,
} from "../artifactGraph.js";
import { artifactInspectionResultSchema } from "../artifactInspection.js";
import {
parseOwnedJavaScriptApplicationAnalysisSteps,
type JavaScriptApplicationAnalysisResult,
@@ -27,6 +35,13 @@ import {
apiSurfaceViewSchema,
} from "./webNetworkApiSurfaceView.js";
import { projectNativeFunctionView } from "./nativeFunctionView.js";
import { projectArtifactInventoryView } from "./artifactInventoryView.js";
import {
projectWasmArtifactView,
wasmViewRequestSchema,
wasmViewItemSchema,
wasmSummarySchema,
} from "./wasmArtifactView.js";
import type { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js";
/** Public name of the selected-view workflow. */
@@ -108,6 +123,28 @@ const nativeViewSchema = z.strictObject({
limit: z.number().int().positive().default(64),
});
/** Original inventory records in stable order, optionally matching exact paths. */
const inventoryViewSchema = z.discriminatedUnion("collection", [
z.strictObject({
kind: z.literal("inventory"),
collection: z.literal("occurrences"),
logical_paths: z
.array(z.string())
.optional()
.describe(
"Exact recorded logical paths. Omitted selects all available occurrences; an empty array selects none. This does not resolve filesystem paths.",
),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
}),
z.strictObject({
kind: z.literal("inventory"),
collection: z.enum(["nodes", "edges", "integrity_contradictions"]),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
}),
]);
/** Caller-selected projection of already completed analysis Evidence. */
export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
summaryViewSchema,
@@ -115,6 +152,8 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
itemViewSchema,
pageViewSchema,
nativeViewSchema,
inventoryViewSchema,
wasmViewRequestSchema,
apiSurfaceViewSchema,
]);
@@ -192,30 +231,30 @@ const javascriptSummarySchema = z.strictObject({
limitation_count: z.number().int().nonnegative(),
});
const summaryViewResultFields = {
kind: z.literal("summary"),
view: summaryViewSchema,
parent_evidence_id: prefixedDigestSchema("ev"),
parent_digest: digestSchema,
view_digest: digestSchema,
artifact: artifactSchema,
coverage: coverageSchema,
limitations: z.array(z.string()),
unknowns: z.array(z.string()),
} as const;
/** Summary `inspect_binary_layout` detail returns; no other view kind. */
export const binaryLayoutSummaryViewResultSchema = z.strictObject({
...summaryViewResultFields,
summary: layoutSummarySchema,
parent_operation: z.literal("inspect_binary_layout"),
});
/** Summary `analyze_javascript_application` detail returns; no other view kind. */
export const javascriptApplicationSummaryViewResultSchema = z.strictObject({
...summaryViewResultFields,
summary: javascriptSummarySchema,
parent_operation: z.literal("analyze_javascript_application"),
const inventorySummarySchema = z.strictObject({
manifest: artifactGraphManifestSchema,
available_counts: z.strictObject({
nodes: z.number().int().nonnegative(),
occurrences: z.number().int().nonnegative(),
edges: z.number().int().nonnegative(),
integrity_contradictions: z.number().int().nonnegative(),
}),
observed_entry_kind_counts: z.strictObject({
file: z.number().int().nonnegative(),
directory: z.number().int().nonnegative(),
symlink: z.number().int().nonnegative(),
slice: z.number().int().nonnegative(),
}),
source_complete: z.boolean(),
inventory_evidence_ids: z.array(prefixedDigestSchema("ev")).min(1),
inspection: z
.strictObject({
coverage: artifactInspectionResultSchema.shape.coverage,
unexplored_branches:
artifactInspectionResultSchema.shape.unexplored_branches,
})
.nullable(),
limitation_count: z.number().int().nonnegative(),
});
const viewResultBase = {
@@ -224,6 +263,9 @@ const viewResultBase = {
"inspect_binary_layout",
"analyze_javascript_application",
"analyze_function",
"inspect_artifact",
"inventory_artifact",
"inspect_wasm_artifact",
]),
parent_digest: digestSchema,
view_digest: digestSchema,
@@ -233,6 +275,33 @@ const viewResultBase = {
unknowns: z.array(z.string()),
} as const;
const summaryViewResultBase = {
kind: z.literal("summary"),
view: summaryViewSchema,
...viewResultBase,
} as const;
/** Summary `inspect_binary_layout` detail returns; no other view kind. */
export const binaryLayoutSummaryViewResultSchema = z.strictObject({
...summaryViewResultBase,
summary: layoutSummarySchema,
parent_operation: z.literal("inspect_binary_layout"),
});
/** Summary `analyze_javascript_application` detail returns; no other view kind. */
export const javascriptApplicationSummaryViewResultSchema = z.strictObject({
...summaryViewResultBase,
summary: javascriptSummarySchema,
parent_operation: z.literal("analyze_javascript_application"),
});
/** Exact summary carrier for artifact inspection's optional delivery contract. */
export const artifactInventorySummaryViewResultSchema = z.strictObject({
...summaryViewResultBase,
parent_operation: z.enum(["inspect_artifact", "inventory_artifact"]),
summary: inventorySummarySchema,
});
/** Projected facts for one selected view of retained analysis Evidence. */
export const analysisViewResultSchema = z.discriminatedUnion("kind", [
z.strictObject({
@@ -247,10 +316,13 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
}),
}),
z.strictObject({
kind: z.literal("summary"),
view: summaryViewSchema,
summary: z.union([layoutSummarySchema, javascriptSummarySchema]),
...viewResultBase,
...summaryViewResultBase,
summary: z.union([
layoutSummarySchema,
javascriptSummarySchema,
inventorySummarySchema,
wasmSummarySchema,
]),
}),
z.strictObject({
kind: z.literal("facet"),
@@ -277,6 +349,28 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
item: jsonValueSchema,
...viewResultBase,
}),
z.strictObject({
kind: z.literal("wasm"),
view: wasmViewRequestSchema,
item: wasmViewItemSchema,
...viewResultBase,
}),
z.strictObject({
kind: z.literal("inventory"),
view: inventoryViewSchema,
manifest: artifactGraphManifestSchema,
inventory_evidence_ids: z.array(prefixedDigestSchema("ev")).min(1),
source_complete: z.boolean(),
items: z.array(
z.union([
artifactNodeSchema,
artifactOccurrenceSchema,
artifactEdgeSchema,
integrityContradictionSchema,
]),
),
...viewResultBase,
}),
]);
export type AnalysisViewRequest = z.output<typeof analysisViewRequestSchema>;
@@ -291,6 +385,8 @@ export interface AnalysisViewParent {
readonly evidenceId: string;
readonly operation: string;
readonly normalizedResult: unknown;
/** Preserve the authenticated source identity, including explicit absence. */
readonly subject?: Evidence["subject"];
readonly limitations: readonly string[];
readonly artifact?: { readonly path: string; readonly sha256: string } | null;
/** Parent Evidence parameters, when the parent operation records them. */
@@ -319,7 +415,12 @@ export type UnsignedAnalysisView =
| Omit<
Extract<AnalysisViewResult, { readonly kind: "native" }>,
"view_digest"
>;
>
| Omit<
Extract<AnalysisViewResult, { readonly kind: "inventory" }>,
"view_digest"
>
| Omit<Extract<AnalysisViewResult, { readonly kind: "wasm" }>, "view_digest">;
/** Attach the content digest that must change when projected bytes change. */
export const sealAnalysisView = (
@@ -364,7 +465,7 @@ const unsupportedParent = (
new AnalysisUnsupportedTargetError(
INSPECT_ANALYSIS_VIEW_OPERATION,
parentArtifactPath(parent),
`Parent Evidence operation ${JSON.stringify(parent.operation)} is not inspect_binary_layout, analyze_javascript_application or analyze_function.`,
`Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, or web network capture Evidence.`,
);
const parentArtifactPath = (parent: AnalysisViewParent): string => {
@@ -459,5 +560,16 @@ export const projectAnalysisView = (
const projected = projectNativeFunctionView(parent, native.data, view);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
if (parent.operation === "inspect_wasm_artifact") {
const projected = projectWasmArtifactView(parent, view);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
if (
parent.operation === "inspect_artifact" ||
parent.operation === "inventory_artifact"
) {
const projected = projectArtifactInventoryView(parent, view);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
return err(unsupportedParent(parent));
};
src/domain/analysisView/analysisViewText.ts
+55
View File
@@ -0,0 +1,55 @@
import type { AnalysisError } from "../analysisErrorBase.js";
import { err, ok, type Result } from "../result.js";
import {
analysisViewInputError,
type AnalysisViewCoverage,
} from "./analysisView.js";
import { pageViewCoverage } from "./analysisViewCoverage.js";
const viewError = (message: string): AnalysisError =>
analysisViewInputError([
{ path: ["view"], reason: "invalid_value", message },
]);
/** Slice text by UTF-16 code units without losing a partial Unicode character. */
export const analysisTextPage = (
body: string,
offset: number,
limit: number,
label = "Pseudocode",
): Result<
{
readonly item: { readonly text: string; readonly unit: "utf16-code-units" };
readonly coverage: AnalysisViewCoverage;
},
AnalysisError
> => {
const start = Math.min(offset, body.length);
const high = (n: number) => n >= 0xd800 && n <= 0xdbff;
const low = (n: number) => n >= 0xdc00 && n <= 0xdfff;
if (
start > 0 &&
start < body.length &&
high(body.charCodeAt(start - 1)) &&
low(body.charCodeAt(start))
)
return err(viewError(`${label} offset splits a UTF-16 surrogate pair.`));
let end = Math.min(start + limit, body.length);
if (
end > start &&
end < body.length &&
high(body.charCodeAt(end - 1)) &&
low(body.charCodeAt(end))
)
end--;
if (end === start && start < body.length)
return err(
viewError(
`Increase ${label.toLowerCase()} limit to include the next complete Unicode character.`,
),
);
return ok({
item: { text: body.slice(start, end), unit: "utf16-code-units" },
coverage: pageViewCoverage(start, end - start, body.length),
});
};
src/domain/analysisView/artifactInventoryView.ts
+203
View File
@@ -0,0 +1,203 @@
import { z } from "zod";
import { AnalysisInputError } from "../analysisErrorCore.js";
import type { AnalysisError } from "../analysisErrorBase.js";
import { uniqueSorted } from "../canonicalOrdering.js";
import { err, ok, type Result } from "../result.js";
import {
parseArtifactInventoryEvidence,
parseArtifactInventoryResult,
validateArtifactInventorySubject,
type InventorySet,
} from "../artifactInventoryEvidence.js";
import {
artifactInspectionResultSchema,
type ArtifactInspectionResult,
} from "../artifactInspection.js";
import type {
AnalysisViewParent,
AnalysisViewRequest,
UnsignedAnalysisView,
} from "./analysisView.js";
import {
completeWithinViewCoverage,
pageViewCoverage,
} from "./analysisViewCoverage.js";
const inputError = (message: string): AnalysisInputError =>
new AnalysisInputError("inspect_analysis_view", undefined, [
{
path: ["source"],
reason: "invalid_value",
message,
},
]);
interface InventorySource {
readonly inventory: InventorySet;
readonly evidenceIds: readonly string[];
readonly inspection: ArtifactInspectionResult | null;
}
/** Reuse the owning inventory integrity/completeness parser for both carriers. */
const parseSource = (
parent: AnalysisViewParent,
): Result<InventorySource, AnalysisError> => {
try {
if (parent.operation === "inventory_artifact")
return ok({
inventory: parseArtifactInventoryResult(
parent.normalizedResult,
parent.subject ?? null,
),
evidenceIds: [parent.evidenceId],
inspection: null,
});
const inspection = artifactInspectionResultSchema.parse(
parent.normalizedResult,
);
for (const step of inspection.substeps) {
if (
step.evidence_id !== step.evidence.evidence_id ||
step.evidence.operation !== step.operation
)
throw new TypeError(
"Inspection substep identity differs from its source Evidence",
);
}
const parsed = parseArtifactInventoryEvidence(
inspection.substeps.map(({ evidence }) => evidence),
);
const manifest = parsed.inventory.manifest;
validateArtifactInventorySubject(parent.subject ?? null, manifest);
if (
inspection.subject.manifest_id !== manifest.manifest_id ||
inspection.subject.root_artifact_id !== manifest.root_artifact_id ||
inspection.subject.root_sha256 !== manifest.root_sha256 ||
inspection.subject.root_format !== manifest.root_format
)
throw new TypeError(
"Inspection subject differs from its inventory manifest",
);
return ok({
inventory: parsed.inventory,
evidenceIds: parsed.evidence.map(({ evidence_id }) => evidence_id),
inspection,
});
} catch (cause: unknown) {
if (!(cause instanceof TypeError) && !(cause instanceof z.ZodError))
throw cause;
return err(inputError(cause.message));
}
};
/** Project exact source records; exhausted pages cover available records, not runtime behavior. */
export const projectArtifactInventoryView = (
parent: AnalysisViewParent,
view: AnalysisViewRequest,
): Result<UnsignedAnalysisView, AnalysisError> => {
const operation = parent.operation;
if (operation !== "inventory_artifact" && operation !== "inspect_artifact")
return err(
inputError(
"Inventory views require inventory_artifact or inspect_artifact Evidence.",
),
);
if (view.kind !== "summary" && view.kind !== "inventory")
return err(
inputError("Artifact Evidence supports summary and inventory views."),
);
const parsed = parseSource(parent);
if (!parsed.ok) return parsed;
const { inventory, evidenceIds, inspection } = parsed.value;
const limitations = uniqueSorted([
...parent.limitations,
...inventory.limitations,
...(inspection?.limitations ?? []),
]);
const shared = {
parent_evidence_id: parent.evidenceId,
parent_operation: operation,
parent_digest: parent.evidenceId.slice(3),
artifact: {
path: parent.artifact?.path ?? "",
sha256: inventory.manifest.root_sha256,
},
limitations,
unknowns: [
"Inventory does not establish application runtime behavior or complete format-specific analysis.",
...(inventory.complete
? []
: [
"Parent inventory is partial; counts and exhausted pages cover only available records. Missing matches do not establish absence.",
]),
],
} satisfies Pick<
UnsignedAnalysisView,
| "parent_evidence_id"
| "parent_operation"
| "parent_digest"
| "artifact"
| "limitations"
| "unknowns"
>;
if (view.kind === "summary") {
const counts = { file: 0, directory: 0, symlink: 0, slice: 0 };
for (const occurrence of inventory.occurrences)
counts[occurrence.entry_kind]++;
const examined =
inventory.nodes.length +
inventory.occurrences.length +
inventory.edges.length;
return ok({
kind: "summary",
view,
summary: {
manifest: inventory.manifest,
available_counts: {
nodes: inventory.nodes.length,
occurrences: inventory.occurrences.length,
edges: inventory.edges.length,
integrity_contradictions: inventory.integrityContradictions.length,
},
observed_entry_kind_counts: counts,
source_complete: inventory.complete,
inventory_evidence_ids: [...evidenceIds],
inspection:
inspection === null
? null
: {
coverage: inspection.coverage,
unexplored_branches: inspection.unexplored_branches,
},
limitation_count: limitations.length,
},
coverage: completeWithinViewCoverage(examined, examined),
...shared,
});
}
const paths =
view.collection === "occurrences" && view.logical_paths !== undefined
? new Set(view.logical_paths)
: undefined;
const rows =
view.collection === "occurrences" && paths !== undefined
? inventory.occurrences.filter((row) => paths.has(row.logical_path))
: view.collection === "occurrences"
? inventory.occurrences
: view.collection === "nodes"
? inventory.nodes
: view.collection === "edges"
? inventory.edges
: inventory.integrityContradictions;
const items = rows.slice(view.offset, view.offset + view.limit);
return ok({
kind: "inventory",
view,
manifest: inventory.manifest,
inventory_evidence_ids: [...evidenceIds],
source_complete: inventory.complete,
items,
coverage: pageViewCoverage(view.offset, items.length, rows.length),
...shared,
});
};
src/domain/analysisView/binaryLayoutView.ts
+8
View File
@@ -156,6 +156,14 @@ export const projectBinaryLayoutView = (
return err(
layoutIncompatible("Native views require analyze_function Evidence."),
);
if (view.kind === "inventory")
return err(
layoutIncompatible("Inventory views require artifact Evidence."),
);
if (view.kind === "wasm")
return err(
layoutIncompatible("WASM views require inspect_wasm_artifact Evidence."),
);
if (view.kind === "summary")
return ok({
kind: "summary",
src/domain/analysisView/javascriptApplicationView.ts
+6
View File
@@ -247,6 +247,12 @@ export const projectJavaScriptApplicationView = (
return err(
jsIncompatible("Native views require analyze_function Evidence."),
);
if (view.kind === "inventory")
return err(jsIncompatible("Inventory views require artifact Evidence."));
if (view.kind === "wasm")
return err(
jsIncompatible("WASM views require inspect_wasm_artifact Evidence."),
);
if (view.kind === "summary")
return ok({
kind: "summary",
src/domain/analysisView/nativeFunctionView.ts
+2 -43
View File
@@ -1,3 +1,4 @@
import { analysisTextPage } from "./analysisViewText.js";
import type { AnalysisError } from "../analysisErrorBase.js";
import { uniqueSorted } from "../canonicalOrdering.js";
import type { FunctionDossier } from "../hopperValues.js";
@@ -5,7 +6,6 @@ import { jsonValueSchema } from "../jsonValue.js";
import { err, ok, type Result } from "../result.js";
import {
analysisViewInputError,
type AnalysisViewCoverage,
type AnalysisViewParent,
type AnalysisViewRequest,
type UnsignedAnalysisView,
@@ -59,47 +59,6 @@ const parentFields = (
};
};
const pseudocodePage = (
body: string,
offset: number,
limit: number,
): Result<
{
readonly item: { readonly text: string; readonly unit: "utf16-code-units" };
readonly coverage: AnalysisViewCoverage;
},
AnalysisError
> => {
const start = Math.min(offset, body.length);
const high = (n: number) => n >= 0xd800 && n <= 0xdbff;
const low = (n: number) => n >= 0xdc00 && n <= 0xdfff;
if (
start > 0 &&
start < body.length &&
high(body.charCodeAt(start - 1)) &&
low(body.charCodeAt(start))
)
return err(viewError("Pseudocode offset splits a UTF-16 surrogate pair."));
let end = Math.min(start + limit, body.length);
if (
end > start &&
end < body.length &&
high(body.charCodeAt(end - 1)) &&
low(body.charCodeAt(end))
)
end--;
if (end === start && start < body.length)
return err(
viewError(
"Increase pseudocode limit to include the next complete Unicode character.",
),
);
return ok({
item: { text: body.slice(start, end), unit: "utf16-code-units" },
coverage: pageViewCoverage(start, end - start, body.length),
});
};
const nativeRows = (
dossier: FunctionDossier,
facet: Exclude<
@@ -193,7 +152,7 @@ export const projectNativeFunctionView = (
return ok(singleton(status));
}
if (view.facet === "pseudocode") {
const page = pseudocodePage(dossier.pseudocode, view.offset, view.limit);
const page = analysisTextPage(dossier.pseudocode, view.offset, view.limit);
return page.ok
? ok({ kind: "native", view, ...page.value, ...common })
: page;
src/domain/analysisView/wasmArtifactView.ts
+207
View File
@@ -0,0 +1,207 @@
import { z } from "zod";
import type { AnalysisError } from "../analysisErrorBase.js";
import { uniqueSorted } from "../canonicalOrdering.js";
import { err, ok, type Result } from "../result.js";
import {
wasmArtifactSchema,
wasmArtifactMatchesSubject,
} from "../wasm/wasmArtifact.js";
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
import {
analysisViewInputError,
type AnalysisViewParent,
type AnalysisViewRequest,
type UnsignedAnalysisView,
} from "./analysisView.js";
import {
completeWithinViewCoverage,
pageViewCoverage,
} from "./analysisViewCoverage.js";
import { analysisTextPage } from "./analysisViewText.js";
/** Source forms and text from completed WASM Evidence; this never executes a VM. */
export const wasmViewRequestSchema = z.strictObject({
kind: z.literal("wasm"),
facet: z.enum([
"sections",
"imports",
"exports",
"candidates",
"glue",
"wat",
"headers",
"details",
"tool_profile",
]),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
});
export const wasmViewItemSchema = z.union([
wasmArtifactSchema.shape.sections,
wasmArtifactSchema.shape.imports,
wasmArtifactSchema.shape.candidates,
wasmArtifactSchema.shape.glue,
wasmArtifactSchema.shape.tool_profile,
z.strictObject({ text: z.string(), unit: z.literal("utf16-code-units") }),
]);
/** Counts are observations of the completed producer, independent of runtime semantics. */
export const wasmSummarySchema = z.strictObject({
bytes: wasmArtifactSchema.shape.artifact.shape.bytes,
validation: wasmArtifactSchema.shape.validation,
available_counts: z.strictObject({
sections: z.number().int().nonnegative(),
imports: z.number().int().nonnegative(),
exports: z.number().int().nonnegative(),
candidates: z.number().int().nonnegative(),
glue: z.number().int().nonnegative(),
}),
wat: wasmArtifactSchema.shape.wat
.omit({ text: true })
.extend({ code_units: z.number().int().nonnegative() }),
tool_profile: wasmArtifactSchema.shape.tool_profile,
runtime_execution: wasmArtifactSchema.shape.runtime_execution,
network_fetch: wasmArtifactSchema.shape.network_fetch,
limitation_count: z.number().int().nonnegative(),
});
/** Read the exact retained WABT forms with source identity and explicit uncertainty. */
export const projectWasmArtifactView = (
parent: AnalysisViewParent,
view: AnalysisViewRequest,
): Result<UnsignedAnalysisView, AnalysisError> => {
if (view.kind !== "summary" && view.kind !== "wasm")
return err(
analysisViewInputError([
{
path: ["view"],
reason: "invalid_value",
message: "WASM Evidence supports summary and wasm views.",
},
]),
);
const parsed = wasmArtifactSchema.safeParse(parent.normalizedResult);
if (!parsed.success)
return err(
analysisInputErrorFromIssues(
"inspect_analysis_view",
parsed.error.issues.map((issue) => ({
...issue,
path: ["source", "normalized_result", ...issue.path],
})),
{ source: { normalized_result: parent.normalizedResult } },
{ cause: parsed.error },
),
);
const report = parsed.data;
if (
!wasmArtifactMatchesSubject(
report.artifact,
parent.subject == null
? null
: {
path: parent.subject.local_path,
sha256: parent.subject.digest.sha256,
},
)
)
return err(
analysisViewInputError([
{
path: ["source", "subject"],
reason: "invalid_value",
message:
"WASM parent Evidence subject must identify the report's exact path and SHA-256.",
},
]),
);
const limitations = uniqueSorted([
...parent.limitations,
...report.limitations,
]);
const shared = {
parent_evidence_id: parent.evidenceId,
parent_operation: "inspect_wasm_artifact" as const,
parent_digest: parent.evidenceId.slice(3),
artifact: { path: report.artifact.path, sha256: report.artifact.sha256 },
limitations,
unknowns: [
"Decoded WAT is not original source. Runtime behavior and network acquisition were not performed.",
...(report.glue.some((glue) => glue.parse_status !== "complete")
? [
"Selected glue parsing is incomplete; unmatched references do not establish absence.",
]
: []),
],
};
if (view.kind === "summary")
return ok({
kind: "summary",
view,
...shared,
summary: {
bytes: report.artifact.bytes,
validation: report.validation,
available_counts: {
sections: report.sections.length,
imports: report.imports.length,
exports: report.exports.length,
candidates: report.candidates.length,
glue: report.glue.length,
},
wat: {
sha256: report.wat.sha256,
representation: report.wat.representation,
code_units: report.wat.text.length,
},
tool_profile: report.tool_profile,
runtime_execution: report.runtime_execution,
network_fetch: report.network_fetch,
limitation_count: limitations.length,
},
coverage: completeWithinViewCoverage(
report.sections.length + report.imports.length + report.exports.length,
report.sections.length + report.imports.length + report.exports.length,
),
});
if (view.facet === "tool_profile") {
if (view.offset !== 0)
return err(
analysisViewInputError([
{
path: ["view", "offset"],
reason: "invalid_value",
message: "Singleton tool_profile requires offset 0.",
},
]),
);
return ok({
kind: "wasm",
view,
item: report.tool_profile,
coverage: completeWithinViewCoverage(1, 1),
...shared,
});
}
if (
view.facet === "wat" ||
view.facet === "headers" ||
view.facet === "details"
) {
const text = view.facet === "wat" ? report.wat.text : report[view.facet];
const page = analysisTextPage(text, view.offset, view.limit, view.facet);
return page.ok
? ok({ kind: "wasm", view, ...page.value, ...shared })
: page;
}
const rows = report[view.facet];
const item = rows.slice(view.offset, view.offset + view.limit);
return ok({
kind: "wasm",
view,
item,
coverage: pageViewCoverage(view.offset, item.length, rows.length),
...shared,
});
};
src/domain/artifactGraph.ts
+4 -4
View File
@@ -101,7 +101,7 @@ const artifactCommandSchema = z.object({
export type ArtifactCommand = z.infer<typeof artifactCommandSchema>;
/** One content-addressed application artifact in deterministic manifest order. */
const artifactNodeSchema = z.object({
export const artifactNodeSchema = z.object({
artifact_id: artifactIdSchema,
format: artifactContentFormatSchema,
sha256: sha256Schema,
@@ -113,7 +113,7 @@ const artifactNodeSchema = z.object({
});
/** One path or virtual location where an artifact occurs in its parent. */
const artifactOccurrenceSchema = z.object({
export const artifactOccurrenceSchema = z.object({
occurrence_id: occurrenceIdSchema,
artifact_id: artifactIdSchema.nullable(),
parent_occurrence_id: occurrenceIdSchema.nullable(),
@@ -159,7 +159,7 @@ export const integrityContradictionSchema = z.object({
* Parent-child derivation relation between two content-addressed artifacts.
* The child artifact and its logical path are those of the edge's occurrence.
*/
const artifactEdgeSchema = z.object({
export const artifactEdgeSchema = z.object({
edge_id: edgeIdSchema,
ordinal: z.number().int().min(0),
parent_artifact_id: artifactIdSchema,
@@ -177,7 +177,7 @@ const artifactEdgeSchema = z.object({
});
/** Stable cryptographic commitment and counts for a complete artifact graph. */
const artifactGraphManifestSchema = z.object({
export const artifactGraphManifestSchema = z.object({
manifest_id: manifestIdSchema,
root_artifact_id: artifactIdSchema,
root_sha256: sha256Schema,
src/domain/artifactInspection.ts
+18
View File
@@ -103,6 +103,24 @@ export type ArtifactInspectionResult = z.infer<
typeof artifactInspectionResultSchema
>;
/**
* The current producer guarantees a concrete inventory payload. Specialize
* its output without narrowing the historical inspection/Evidence input
* parser or applying inventory defaults inside an authenticated imported record.
*/
export const artifactInspectionOutputSchema =
artifactInspectionResultSchema.safeExtend({
substeps: z
.array(
artifactInspectionResultSchema.shape.substeps.element.safeExtend({
evidence: evidenceSchema.safeExtend({
normalized_result: artifactInventoryResultSchema,
}),
}),
)
.min(1),
});
/** Project one successful inventory Evidence record into an inspection report. */
export const createArtifactInspection = (
inventoryInput: unknown,
src/domain/artifactInventoryEvidence.ts
+70 -59
View File
@@ -37,37 +37,63 @@ export interface ParsedInventorySet {
/** Parse, merge, and verify inventory Evidence pages. */
export const parseArtifactInventoryEvidence = (
input: unknown,
): ParsedInventorySet =>
assembleInventorySet(
(Array.isArray(input) ? input : [input]).map((value) => {
const evidence = parseEvidence(value);
const inventoryEvidence =
evidence.operation === "inventory_artifact"
? evidence
: evidence.operation === "inspect_artifact"
? artifactInspectionResultSchema.parse(evidence.normalized_result)
.substeps[0]?.evidence
: undefined;
if (inventoryEvidence === undefined)
throw new TypeError(
"Artifact comparison requires inspect_artifact Evidence",
);
const inventory = artifactInventoryResultSchema.parse(
inventoryEvidence.normalized_result,
): ParsedInventorySet => {
const pages = (Array.isArray(input) ? input : [input]).map((value) => {
const evidence = parseEvidence(value);
const inventoryEvidence =
evidence.operation === "inventory_artifact"
? evidence
: evidence.operation === "inspect_artifact"
? artifactInspectionResultSchema.parse(evidence.normalized_result)
.substeps[0]?.evidence
: undefined;
if (inventoryEvidence === undefined)
throw new TypeError(
"Artifact comparison requires inspect_artifact Evidence",
);
if (
inventoryEvidence.subject === null ||
inventoryEvidence.subject.digest.sha256 !==
inventory.manifest.root_sha256 ||
inventoryEvidence.subject.format !== inventory.manifest.root_format
)
throw new TypeError(
"Artifact inventory Evidence subject does not match its root digest",
);
validateInventoryPage(inventory);
return { evidence, inventory };
}),
);
const inventory = artifactInventoryResultSchema.parse(
inventoryEvidence.normalized_result,
);
validateArtifactInventorySubject(
inventoryEvidence.subject,
inventory.manifest,
);
validateInventoryPage(inventory);
return { evidence, inventory };
});
return {
evidence: pages.map(({ evidence }) => evidence),
inventory: assembleInventoryResults(
pages.map(({ inventory }) => inventory),
),
};
};
/** Validate one already authenticated parent payload with the same graph proofs. */
export const parseArtifactInventoryResult = (
input: unknown,
subject: Evidence["subject"],
): InventorySet => {
const inventory = artifactInventoryResultSchema.parse(input);
validateArtifactInventorySubject(subject, inventory.manifest);
validateInventoryPage(inventory);
return assembleInventoryResults([inventory]);
};
/** Both inventory carriers require the same authenticated root identity. */
export const validateArtifactInventorySubject = (
subject: Evidence["subject"],
manifest: ArtifactInventoryResult["manifest"],
): void => {
if (
subject === null ||
subject.digest.sha256 !== manifest.root_sha256 ||
subject.format !== manifest.root_format
)
throw new TypeError(
"Artifact inventory Evidence subject does not match its root digest",
);
};
const validateInventoryPage = (inventory: ArtifactInventoryResult): void => {
assertUnique(
@@ -91,45 +117,30 @@ const validateInventoryPage = (inventory: ArtifactInventoryResult): void => {
throw new TypeError("Artifact node ID is not content-addressed");
};
const assembleInventorySet = (
pages: readonly {
readonly evidence: Evidence;
readonly inventory: ArtifactInventoryResult;
}[],
): ParsedInventorySet => {
const assembleInventoryResults = (
pages: readonly ArtifactInventoryResult[],
): InventorySet => {
const first = pages[0];
if (first === undefined)
throw new TypeError("Artifact inventory requires Evidence pages");
const manifestJson = canonicalJson(
first.inventory.manifest,
"Artifact inventory",
);
const manifestJson = canonicalJson(first.manifest, "Artifact inventory");
const nodes = new Map<string, ArtifactNode>();
const occurrences = new Map<string, ArtifactOccurrence>();
const paths = new Map<string, ArtifactOccurrence>();
const edges = new Map<string, ArtifactInventoryResult["edges"][number]>();
for (const page of pages) {
if (
canonicalJson(page.inventory.manifest, "Artifact inventory") !==
manifestJson
)
if (canonicalJson(page.manifest, "Artifact inventory") !== manifestJson)
throw new TypeError("Artifact inventory pages do not share one manifest");
if (
canonicalJson(
page.inventory.integrity_contradictions,
"Artifact inventory",
) !==
canonicalJson(
first.inventory.integrity_contradictions,
"Artifact inventory",
)
canonicalJson(page.integrity_contradictions, "Artifact inventory") !==
canonicalJson(first.integrity_contradictions, "Artifact inventory")
)
throw new TypeError(
"Artifact inventory pages do not share integrity contradictions",
);
for (const node of page.inventory.nodes)
for (const node of page.nodes)
mergeExact(nodes, node.artifact_id, node, "node");
for (const occurrence of page.inventory.occurrences) {
for (const occurrence of page.occurrences) {
mergeExact(
occurrences,
occurrence.occurrence_id,
@@ -138,21 +149,21 @@ const assembleInventorySet = (
);
mergeExact(paths, occurrence.logical_path, occurrence, "logical path");
}
for (const edge of page.inventory.edges)
for (const edge of page.edges)
mergeExact(edges, edge.edge_id, edge, "edge");
}
const inventory = orderedInventory(first.inventory, pages, {
const inventory = orderedInventory(first, pages, {
nodes,
occurrences,
edges,
});
if (inventory.complete) validateCompleteInventory(inventory);
return { evidence: pages.map(({ evidence }) => evidence), inventory };
return inventory;
};
const orderedInventory = (
first: ArtifactInventoryResult,
pages: readonly { readonly inventory: ArtifactInventoryResult }[],
pages: readonly ArtifactInventoryResult[],
values: {
readonly nodes: ReadonlyMap<string, ArtifactNode>;
readonly occurrences: ReadonlyMap<string, ArtifactOccurrence>;
@@ -178,7 +189,7 @@ const orderedInventory = (
edges,
integrityContradictions: first.integrity_contradictions,
limitations: [
...new Set(pages.flatMap(({ inventory }) => inventory.limitations)),
...new Set(pages.flatMap((inventory) => inventory.limitations)),
].sort(compareUnicodeCodePoints),
complete:
nodes.length === first.manifest.node_count &&
src/domain/wasm/wasmArtifact.ts
+20
View File
@@ -1,4 +1,5 @@
import { z } from "zod";
import { createHash } from "node:crypto";
import { localPathStringSchema } from "../localPath.js";
@@ -83,6 +84,15 @@ export const wasmArtifactSchema = z
limitations: z.array(z.string()),
})
.superRefine((value, context) => {
if (
createHash("sha256").update(value.wat.text, "utf8").digest("hex") !==
value.wat.sha256
)
context.addIssue({
code: "custom",
path: ["wat", "sha256"],
message: "Decoded WAT digest does not match its UTF-8 text.",
});
const selected = value.candidates[0];
if (
selected?.path !== value.artifact.path ||
@@ -125,3 +135,13 @@ export const wasmArtifactSchema = z
export type InspectWasmArtifactInput = z.infer<
typeof inspectWasmArtifactInputSchema
>;
export type WasmArtifact = z.infer<typeof wasmArtifactSchema>;
/** A WASM observation must identify the exact selected path and byte digest. */
export const wasmArtifactMatchesSubject = (
artifact: WasmArtifact["artifact"],
subject: Pick<WasmArtifact["artifact"], "path" | "sha256"> | null | undefined,
): subject is Pick<WasmArtifact["artifact"], "path" | "sha256"> =>
subject != null &&
subject.path === artifact.path &&
subject.sha256 === artifact.sha256;
src/server/createServer.ts
+8 -5
View File
@@ -1,5 +1,5 @@
import type { WasmArtifactService } from "../application/wasm/WasmArtifactService.js";
import { createWasmArtifactService } from "../composition/wasm.js";
import { wasmArtifactModule } from "../composition/wasm.js";
import { registerWasmTools } from "./registerWasmTools.js";
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { createPeResourcesService } from "../composition/binaryDiagnostics.js";
@@ -246,8 +246,10 @@ export const createServer = (
);
const android =
options.androidAnalysis ?? createAndroidAnalysisProvider(environment);
const wasmArtifact =
options.wasmArtifact ?? createWasmArtifactService(environment);
const wasmArtifact = wasmArtifactModule.create(
environment,
options.wasmArtifact,
);
const apktool =
options.apktoolAnalysis ??
createApktoolResourceAnalysisProvider(environment);
@@ -399,7 +401,7 @@ const registerConfiguredAnalysisTools = (
withAdmittedAnalysis,
}: ServerToolContext,
android: AndroidAnalysisPort,
wasmArtifact: WasmArtifactService,
wasmArtifact: Pick<WasmArtifactService, "inspect" | "close">,
jeb: JebAnalysisPort,
firmwareAnalysis: FirmwareAnalysisService,
): void => {
@@ -557,6 +559,7 @@ const registerBinaryAnalysisTools = ({
activeTarget,
recordEvidence,
recordEvidenceWithUnknown,
evidenceById,
withAdmittedAnalysis,
}: ServerToolContext): void => {
const recordUnknown =
@@ -592,7 +595,7 @@ const registerBinaryAnalysisTools = ({
withAdmittedAnalysis,
};
registerEvidenceTools(server, NATIVE_TOOL_CONTRACTS, evidenceOptions);
registerArtifactTools(server, evidenceOptions);
registerArtifactTools(server, { ...evidenceOptions, evidenceById });
registerManagedTools(server, evidenceOptions);
if (session !== undefined)
registerManagedWorkflowTools(server, {
src/server/registerArtifactTools.ts
+26 -1
View File
@@ -1,7 +1,14 @@
import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
import { ARTIFACT_TOOL_CONTRACTS } from "../contracts/artifactToolContracts.js";
import {
ARTIFACT_TOOL_CONTRACTS,
artifactInspectionInputSchema,
} from "../contracts/artifactToolContracts.js";
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
import { summarizeRetainedAnalysis } from "../application/analysisView/AnalysisViewService.js";
import { createArtifactExtractionDestination } from "../application/artifacts/ArtifactExtractionDestination.js";
import { ok } from "../domain/result.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import type { Logger } from "pino";
import { registerEvidenceTools } from "./registerEvidenceTools.js";
@@ -14,11 +21,29 @@ export const registerArtifactTools = (
readonly logger: Logger;
readonly activeTarget: (() => BinaryTarget | undefined) | undefined;
readonly recordEvidence: EvidenceWriter["recordEvidence"] | undefined;
readonly evidenceById?: EvidenceLookup | undefined;
readonly withAdmittedAnalysis: import("./analysisAdmission.js").WithAdmittedAnalysis;
},
): void => {
registerEvidenceTools(server, ARTIFACT_TOOL_CONTRACTS, {
...options,
parametersForExecution: (operation, parameters) => {
if (operation === "inspect_artifact") {
const { detail: _detail, ...execution } =
artifactInspectionInputSchema.parse(parameters);
return execution;
}
return operation === "extract_artifact"
? { ...parameters, output_root: createArtifactExtractionDestination() }
: parameters;
},
projectEvidence: (operation, parameters, evidence) =>
operation === "inspect_artifact" && parameters.detail === "summary"
? summarizeRetainedAnalysis(evidence, {
recordEvidence: options.recordEvidence,
evidenceById: options.evidenceById,
})
: ok(evidence),
sourceEvidence: (operation, result) =>
operation === "inspect_artifact"
? artifactInspectionResultSchema
src/server/registerEvidenceTools.ts
+21 -10
View File
@@ -8,12 +8,13 @@ import type {
import type { ToolContract } from "../contracts/toolContractTypes.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import { jsonObjectSchema } from "../domain/jsonValue.js";
import { jsonObjectSchema, type JsonValue } from "../domain/jsonValue.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { ok, type Result } from "../domain/result.js";
import type { Logger } from "pino";
import { mcpProgressReporter } from "./mcpProgress.js";
import { logToolExecution } from "./toolLogging.js";
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
import { createArtifactExtractionDestination } from "../application/artifacts/ArtifactExtractionDestination.js";
import type { WithAdmittedAnalysis } from "./analysisAdmission.js";
interface EvidenceToolRegistration {
@@ -28,6 +29,15 @@ interface EvidenceToolRegistration {
operation: Exclude<AnalysisOperation, "health">,
result: import("../domain/jsonValue.js").JsonValue,
) => readonly Evidence[];
readonly parametersForExecution?: (
operation: Exclude<AnalysisOperation, "health">,
parameters: Readonly<Record<string, JsonValue>>,
) => Readonly<Record<string, JsonValue>>;
readonly projectEvidence?: (
operation: Exclude<AnalysisOperation, "health">,
parameters: Readonly<Record<string, JsonValue>>,
evidence: Evidence,
) => Result<Evidence, AnalysisError>;
}
/** Register provider-backed contracts that return atomic Evidence observations. */
@@ -57,12 +67,8 @@ export const registerEvidenceTools = (
});
const parameters = jsonObjectSchema.parse(input);
const executionParameters =
contract.name === "extract_artifact"
? {
...parameters,
output_root: createArtifactExtractionDestination(),
}
: parameters;
options.parametersForExecution?.(contract.name, parameters) ??
parameters;
const execution = await logToolExecution(
options.logger,
contract.name,
@@ -108,9 +114,14 @@ export const registerEvidenceTools = (
contract,
);
}
const recorded = options.recordEvidence?.(evidence);
const projected =
options.projectEvidence?.(contract.name, parameters, evidence) ??
ok(evidence);
if (!projected.ok)
return server.delivery.toCallToolResult(projected, contract);
const recorded = options.recordEvidence?.(projected.value);
return server.delivery.toEvidenceToolResult(
evidence,
projected.value,
contract,
recorded,
);
src/server/registerWasmTools.ts
+3 -3
View File
@@ -2,7 +2,7 @@ import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
import type { WasmArtifactService } from "../application/wasm/WasmArtifactService.js";
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
import { toolContract } from "../contracts/toolContracts.js";
import { wasmArtifactModule } from "../composition/wasm.js";
import type { Logger } from "pino";
import { createEvidenceToolHandler } from "./contractToolHandler.js";
import type { WithAdmittedAnalysis } from "./analysisAdmission.js";
@@ -11,12 +11,12 @@ import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
/** Bind the named WASM contract to shared validation and session Evidence recording. */
export const registerWasmTools = (
server: EvidenceMcpServer,
service: WasmArtifactService,
service: Pick<WasmArtifactService, "inspect">,
logger: Logger,
recordEvidence?: EvidenceWriter["recordEvidence"],
withAdmittedAnalysis?: WithAdmittedAnalysis,
): void => {
const contract = toolContract("inspect_wasm_artifact");
const contract = wasmArtifactModule.contract;
const handler = createEvidenceToolHandler(
server,
(input, options) => service.inspect(input, options),
tests/boundary/cli/analysisViewCli.test.ts
+48 -3
View File
@@ -1,10 +1,10 @@
import { apiSurfaceHar } from "../../fixtures/historicalHar.js";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { createHash } from "node:crypto";
import { mkdir, writeFile } from "node:fs/promises";
import { mkdir, unlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect } from "vitest";
import { TextReader, Uint8ArrayWriter, ZipWriter } from "@zip.js/zip.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
@@ -15,11 +15,56 @@ import {
analysisViewCaptureEvidence,
analysisViewObjectDenseCapture,
} from "../../fixtures/analysisView.js";
import { createEvidence } from "../../../src/domain/evidence.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
import { inspectEvidenceBundle } from "../../../src/application/investigation/InspectEvidenceBundle.js";
cliTest(
"projects portable artifact inventory after the original ZIP is removed",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-inventory-view-cli-");
const archive = join(root, "fixture.zip");
const writer = new ZipWriter(new Uint8ArrayWriter());
await writer.add("main.js", new TextReader("console.log('portable');\n"));
await writeFile(archive, await writer.close());
const inspected = await cli.run({
arguments: ["inspect-artifact", archive, "--json"],
});
expect(inspected.exitCode).toBe(0);
const parent = parseEvidence(inspected.json);
await unlink(archive);
const request = join(root, "view.json");
await writeFile(
request,
JSON.stringify({
source: { kind: "inline", evidence: parent },
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: ["main.js"],
},
}),
);
const projected = await cli.run({
arguments: ["inspect-analysis-view", request, "--json"],
});
expect(projected.exitCode).toBe(0);
expect(projected.json).toMatchObject({
operation: "inspect_analysis_view",
evidence_links: [parent.evidence_id],
normalized_result: {
parent_evidence_id: parent.evidence_id,
kind: "inventory",
source_complete: true,
artifact: { path: archive, sha256: parent.subject?.digest.sha256 },
items: [expect.objectContaining({ logical_path: "main.js" })],
coverage: { total: 1, exhausted: true, next_offset: null },
},
});
},
);
cliTest(
"admits portable captures before parsing and preserves capture caveats",
async ({ cli }) => {
tests/boundary/mcp/artifactMcp.inlineInventory.test.ts
+431 -23
View File
@@ -1,8 +1,9 @@
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { readFile, writeFile } from "node:fs/promises";
import { readFile, unlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import { TextReader, Uint8ArrayWriter, ZipWriter } from "@zip.js/zip.js";
import { expect, it } from "vitest";
import { z } from "zod";
@@ -12,8 +13,81 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { ArtifactProvider } from "../../../src/artifacts/ArtifactProvider.js";
import { createServer } from "../../../src/server/createServer.js";
import {
createEvidence,
parseEvidence,
type Evidence,
} from "../../../src/domain/evidence.js";
import {
artifactInspectionResultSchema,
createArtifactInspection,
} from "../../../src/domain/artifactInspection.js";
import {
artifactInventoryResultSchema,
type ArtifactInventoryResult,
} from "../../../src/domain/artifactGraph.js";
import {
analysisViewResultSchema,
type AnalysisViewResult,
} from "../../../src/domain/analysisView/analysisView.js";
it("inspects a standalone keyed archive through MCP with original object identities", async () => {
const assertInventorySubjectValidation = async (
client: Client,
source: Evidence,
): Promise<void> => {
const result =
source.operation === "inspect_artifact"
? artifactInspectionResultSchema.parse(source.normalized_result)
.substeps[0]?.evidence.normalized_result
: source.normalized_result;
const inventory = artifactInventoryResultSchema.parse(result);
if (source.subject === null)
throw new Error("Fixture requires a known subject");
for (const target of [
undefined,
{
path: source.subject.local_path,
sha256: inventory.manifest.root_sha256,
format: "file" as const,
},
{
path: source.subject.local_path,
sha256: "f".repeat(64),
format: inventory.manifest.root_format,
},
]) {
const inconsistent = createEvidence(target, source.provider, {
operation: source.operation,
parameters: source.parameters,
result: source.normalized_result,
});
for (const view of [
{ kind: "summary" },
{ kind: "inventory", collection: "nodes", offset: 0, limit: 1 },
]) {
const reply = await client.callTool({
name: "inspect_analysis_view",
arguments: { source: { kind: "inline", evidence: inconsistent }, view },
});
expect(parseMcpToolError(reply)).toMatchObject({
error: {
code: "invalid_request",
details: {
issues: [
{
path: ["source"],
reason: "invalid_value",
message: expect.stringContaining("subject does not match"),
},
],
},
},
});
}
}
};
it("preserves keyed archive identities and rejects inconsistent inventory carrier subjects", async () => {
const directory = await createTestTempDirectory("rea-keyed-mcp-");
const path = join(directory, "archive.plist");
await writeFile(
@@ -99,6 +173,19 @@ it("inspects a standalone keyed archive through MCP with original object identit
},
},
});
const inspected = await client.callTool({
name: "inspect_artifact",
arguments: {},
});
expect(inspected.isError).not.toBe(true);
const parent = parseEvidence(inspected.structuredContent);
const inspection = artifactInspectionResultSchema.parse(
parent.normalized_result,
);
const inventory = inspection.substeps[0]?.evidence;
if (inventory === undefined) throw new Error("Missing inventory Evidence");
await assertInventorySubjectValidation(client, inventory);
await assertInventorySubjectValidation(client, parent);
} finally {
await client.close();
await server.close();
@@ -106,7 +193,201 @@ it("inspects a standalone keyed archive through MCP with original object identit
}
});
it("returns all 520 ZIP file occurrences in one inspect_artifact MCP result", async () => {
const createInventoryViewReader = async (client: Client, parent: Evidence) => {
const advertised = (await client.listTools()).tools.find(
(tool) => tool.name === "inspect_analysis_view",
);
if (advertised?.outputSchema === undefined)
throw new Error("Missing advertised view schemas");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const inputSchema: Record<string, unknown> = advertised.inputSchema;
const outputSchema: Record<string, unknown> = advertised.outputSchema;
const validateInput = ajv.compile(inputSchema);
const validateOutput = ajv.compile(outputSchema);
expect(
validateInput({
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
view: { kind: "inventory", collection: "nodes", logical_paths: [] },
}),
).toBe(false);
const readView = async (
view: Record<string, unknown>,
source = {
kind: "retained-evidence",
evidence_id: parent.evidence_id,
},
) => {
const arguments_ = { source, view };
expect(
validateInput(arguments_),
JSON.stringify(validateInput.errors),
).toBe(true);
const reply = await client.callTool({
name: "inspect_analysis_view",
arguments: arguments_,
});
if (reply.isError)
throw new Error(JSON.stringify(parseMcpToolError(reply)));
expect(
validateOutput(reply.structuredContent),
JSON.stringify(validateOutput.errors),
).toBe(true);
const evidence = parseEvidence(reply.structuredContent);
return {
evidence,
result: analysisViewResultSchema.parse(evidence.normalized_result),
};
};
return readView;
};
const assertHistoricalInventory = async (
client: Client,
input: {
archive: string;
parent: Evidence;
inventoryEvidence: Evidence;
graph: ArtifactInventoryResult;
},
) => {
const { archive, parent, inventoryEvidence, graph } = input;
// Historical records may omit a later inventory field. Input parsing must
// not insert defaults inside that authenticated child Evidence.
const { integrity_contradictions: _contradictions, ...legacyInventory } =
graph;
const legacyTarget = {
path: archive,
sha256: graph.manifest.root_sha256,
format: "zip" as const,
};
const legacySource = createEvidence(
legacyTarget,
inventoryEvidence.provider,
{
operation: "inventory_artifact",
parameters: {},
result: legacyInventory,
},
);
const legacyInspection = createArtifactInspection(legacySource);
expect(legacyInspection.substeps[0]?.evidence).toEqual(legacySource);
expect(
legacyInspection.substeps[0]?.evidence.normalized_result,
).not.toHaveProperty("integrity_contradictions");
const legacyParent = createEvidence(legacyTarget, parent.provider, {
operation: "inspect_artifact",
parameters: {},
result: legacyInspection,
});
const legacyReply = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "inline", evidence: legacyParent },
view: { kind: "summary" },
},
});
expect(
legacyReply.isError,
JSON.stringify(legacyReply.structuredContent),
).not.toBe(true);
expect(
parseEvidence(legacyReply.structuredContent).normalized_result,
).toMatchObject({
parent_evidence_id: legacyParent.evidence_id,
summary: {
source_complete: true,
inventory_evidence_ids: [legacySource.evidence_id],
},
});
};
const assertPortableRoundtrip = async (
client: Client,
directory: string,
parent: Evidence,
originalView: AnalysisViewResult,
) => {
const saved = join(directory, "evidence.json");
const exported = await client.callTool({
name: "export_evidence_bundle",
arguments: { path: saved },
});
if (exported.isError)
throw new Error(JSON.stringify(parseMcpToolError(exported)));
const closed = await client.callTool({
name: "close_binary",
arguments: {},
});
if (closed.isError)
throw new Error(JSON.stringify(parseMcpToolError(closed)));
const missing = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
view: { kind: "summary" },
},
});
expect(parseMcpToolError(missing)).toMatchObject({
error: { details: { reason: "missing" } },
});
const imported = await client.callTool({
name: "import_evidence_bundle",
arguments: { path: saved },
});
if (imported.isError)
throw new Error(JSON.stringify(parseMcpToolError(imported)));
const restored = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
view: { kind: "summary" },
},
});
if (restored.isError)
throw new Error(JSON.stringify(parseMcpToolError(restored)));
expect(parseEvidence(restored.structuredContent).normalized_result).toEqual(
originalView,
);
};
const assertSummaryDelivery = async (
client: Client,
session: ReturnType<typeof createTestBinarySession>,
complete: Evidence,
) => {
expect(complete.parameters).not.toHaveProperty("detail");
const reply = await client.callTool({
name: "inspect_artifact",
arguments: { detail: "summary" },
});
expect(reply.isError, JSON.stringify(reply.structuredContent)).not.toBe(true);
const projected = parseEvidence(reply.structuredContent);
const view = analysisViewResultSchema.parse(projected.normalized_result);
expect(projected.operation).toBe("inspect_analysis_view");
expect(view.kind).toBe("summary");
const retained = session.evidenceById(view.parent_evidence_id);
if (retained === undefined) throw new Error("Missing complete inspection");
expect(retained.operation).toBe("inspect_artifact");
expect(retained.parameters).toMatchObject({ detail: "summary" });
expect(retained.normalized_result).toEqual(complete.normalized_result);
expect(retained.evidence_id).not.toBe(projected.evidence_id);
const inspection = artifactInspectionResultSchema.parse(
retained.normalized_result,
);
expect(
session.evidenceById(inspection.substeps[0]?.evidence_id ?? ""),
).toEqual(inspection.substeps[0]?.evidence);
const explicit = await client.callTool({
name: "inspect_artifact",
arguments: { detail: "complete" },
});
expect(explicit.isError).not.toBe(true);
expect(parseEvidence(explicit.structuredContent).normalized_result).toEqual(
complete.normalized_result,
);
};
it("keeps complete ZIP output and reads retained inventory views after the source is removed", async () => {
const directory = await createTestTempDirectory("rea-artifact-inline-");
const archive = join(directory, "many.zip");
const writer = new ZipWriter(new Uint8ArrayWriter());
@@ -138,27 +419,154 @@ it("returns all 520 ZIP file occurrences in one inspect_artifact MCP result", as
expect(result.isError, JSON.stringify(result.structuredContent)).not.toBe(
true,
);
const inspection = z
.object({
substeps: z.array(
z.object({
evidence: z.object({
normalized_result: z.object({
occurrences: z.array(z.object({ logical_path: z.string() })),
}),
}),
}),
),
})
.parse(
z
.object({ normalized_result: z.unknown() })
.parse(result.structuredContent).normalized_result,
);
const parent = parseEvidence(result.structuredContent);
await assertSummaryDelivery(client, session, parent);
const parsedInspection = artifactInspectionResultSchema.parse(
parent.normalized_result,
);
const inventoryEvidence = parsedInspection.substeps[0]?.evidence;
if (inventoryEvidence === undefined)
throw new Error("Missing inventory Evidence");
const graph = artifactInventoryResultSchema.parse(
inventoryEvidence.normalized_result,
);
expect(graph.occurrences).toHaveLength(fileCount + 1);
// Follow-ups must use retained source facts, not reopen this archive.
await unlink(archive);
const readView = await createInventoryViewReader(client, parent);
const summary = await readView({ kind: "summary" });
expect(summary.result).toMatchObject({
kind: "summary",
parent_evidence_id: parent.evidence_id,
artifact: { path: archive, sha256: graph.manifest.root_sha256 },
summary: {
manifest: graph.manifest,
source_complete: true,
available_counts: { occurrences: fileCount + 1 },
observed_entry_kind_counts: { file: fileCount + 1 },
inventory_evidence_ids: [inventoryEvidence.evidence_id],
},
});
expect(summary.evidence.evidence_links).toEqual([parent.evidence_id]);
const paths = ["files/0.txt", "files/519.txt", "missing.txt"];
const selected = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: paths,
offset: 0,
limit: 1,
});
expect(selected.result).toMatchObject({
kind: "inventory",
source_complete: true,
items: graph.occurrences
.filter((row) => paths.includes(row.logical_path))
.slice(0, 1),
coverage: { examined: 1, total: 2, next_offset: 1, exhausted: false },
});
const next = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: paths,
offset: 1,
limit: 1,
});
expect(next.result).toMatchObject({
coverage: { examined: 1, next_offset: null, exhausted: true },
});
const none = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: [],
});
expect(none.result).toMatchObject({
items: [],
coverage: { total: 0, exhausted: true },
});
const partial = createEvidence(
{ path: archive, sha256: graph.manifest.root_sha256, format: "zip" },
inventoryEvidence.provider,
{
operation: "inventory_artifact",
parameters: {},
result: { ...graph, occurrences: graph.occurrences.slice(0, 1) },
},
);
const partialReply = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "inline", evidence: partial },
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: ["missing.txt"],
},
},
});
expect(
inspection.substeps[0]?.evidence.normalized_result.occurrences,
).toHaveLength(fileCount + 1);
parseEvidence(partialReply.structuredContent).normalized_result,
).toMatchObject({
source_complete: false,
items: [],
unknowns: expect.arrayContaining([
expect.stringContaining("Missing matches do not establish absence"),
]),
});
await assertHistoricalInventory(client, {
archive,
parent,
inventoryEvidence,
graph,
});
await assertPortableRoundtrip(client, directory, parent, summary.result);
} finally {
await Promise.allSettled([client.close(), server.close(), session.close()]);
await client.close();
await server.close();
await session.close();
}
});
it("refuses summary delivery without a retained Evidence owner and preserves complete delivery", async () => {
const directory = await createTestTempDirectory("rea-inventory-fixed-");
const archive = join(directory, "fixture.zip");
const writer = new ZipWriter(new Uint8ArrayWriter());
await writer.add("main.js", new TextReader("console.log('fixed');\n"));
await writeFile(archive, await writer.close());
const session = createTestBinarySession(new ArtifactProvider(process.env));
const opened = await session.open(archive);
if (!opened.ok) throw opened.error;
const server = createServer({ kind: "fixed", analysis: session });
const client = new Client({ name: "artifact-fixed-test", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const summary = await client.callTool({
name: "inspect_artifact",
arguments: { detail: "summary" },
});
expect(parseMcpToolError(summary)).toMatchObject({
error: {
code: "capability_unavailable",
message: expect.stringContaining("does not retain Evidence"),
},
});
const complete = await client.callTool({
name: "inspect_artifact",
arguments: {},
});
expect(complete.isError).not.toBe(true);
expect(
parseEvidence(complete.structuredContent).normalized_result,
).toMatchObject({
substeps: [expect.objectContaining({ operation: "inventory_artifact" })],
});
} finally {
await client.close();
await server.close();
await session.close();
}
});
tests/boundary/mcp/sessionMcp.test.ts
+1 -1
View File
@@ -281,7 +281,7 @@ describe("target-free MCP workflow", () => {
);
expect(
beforeTools.find(({ name }) => name === "inspect_artifact")?.description,
).toContain("Returns the complete content-addressed artifact graph");
).toMatch(/\breturns the complete content-addressed artifact graph\b/iu);
expect(beforeNames).toContain("open_binary");
expect(beforeNames).toContain("binary_session");
expect(beforeNames).toContain("current_document");
tests/boundary/mcp/wasmArtifactMcp.test.ts
+216 -2
View File
@@ -1,11 +1,14 @@
import type { Client } from "@modelcontextprotocol/client";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it, onTestFinished, vi } from "vitest";
import { readFile, realpath, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { WasmArtifactService } from "../../../src/application/wasm/WasmArtifactService.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
import { createHash } from "node:crypto";
import { z } from "zod";
import { createServer } from "../../../src/server/createServer.js";
import { ok } from "../../../src/domain/result.js";
import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
@@ -141,3 +144,214 @@ it("advertises exact valid schemas and records inline WASM interface evidence wi
await harness.close();
expect(close).toHaveBeenCalled();
});
it("reads an oversized completed WASM result through its reported retained reference", async () => {
const text = "(module)\n" + ";;😀\n" + " ".repeat(6 * 1024 * 1024);
const value = {
...wasmArtifactFixture(),
imports: ['(import "env" "memory.with punctuation" (memory 1))'],
exports: ['(export "first" (func 0))', '(export "second" (func 1))'],
wat: {
text,
sha256: createHash("sha256").update(text, "utf8").digest("hex"),
representation: "decoded-wat" as const,
},
};
const service = new WasmArtifactService({
close: () => Promise.resolve(),
inspect: () => Promise.resolve(ok(wasmArtifactExecution(value))),
});
const harness = await createApplicationMcpHarness({
serverOptions: { wasmArtifact: service },
});
onTestFinished(harness.close);
const reply = await harness.client.callTool({
name: "inspect_wasm_artifact",
arguments: { path: value.artifact.path },
});
const reference = z
.object({
details: z.object({
reported_limits: z.object({
evidence_reference: z.object({
kind: z.literal("retained-evidence"),
evidence_id: z.string(),
}),
}),
}),
})
.parse(parseMcpToolError(reply).error).details
.reported_limits.evidence_reference;
const catalog = await harness.client.listTools();
const advertised = catalog.tools.find(
(tool) => tool.name === "inspect_analysis_view",
);
if (advertised?.outputSchema === undefined)
throw new Error("Missing WASM view schemas");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const input: Record<string, unknown> = advertised.inputSchema;
const output: Record<string, unknown> = advertised.outputSchema;
for (const [view, expected] of [
[
{ kind: "summary" },
{
summary: {
validation: "valid",
available_counts: { imports: 1, exports: 2 },
runtime_execution: "not-performed",
network_fetch: "not-performed",
wat: { sha256: value.wat.sha256, code_units: text.length },
},
},
],
[
{ kind: "wasm", facet: "exports", offset: 1, limit: 1 },
{
item: [value.exports[1]],
coverage: { total: 2, next_offset: null, exhausted: true },
},
],
[{ kind: "wasm", facet: "imports" }, { item: value.imports }],
[
{ kind: "wasm", facet: "wat", offset: 0, limit: 16 },
{
item: { text: text.slice(0, 16), unit: "utf16-code-units" },
coverage: { total: text.length, next_offset: 16, exhausted: false },
},
],
] as const) {
const args = { source: reference, view };
expect(ajv.validate(input, args), JSON.stringify(ajv.errors)).toBe(true);
const selected = await harness.client.callTool({
name: "inspect_analysis_view",
arguments: args,
});
expect(
selected.isError,
JSON.stringify(selected.structuredContent),
).not.toBe(true);
expect(
ajv.validate(output, selected.structuredContent),
JSON.stringify(ajv.errors),
).toBe(true);
const evidence = parseEvidence(selected.structuredContent);
expect(evidence.confidence).toBe("derived");
expect(evidence.evidence_links).toEqual([reference.evidence_id]);
expect(evidence.normalized_result).toMatchObject({
parent_evidence_id: reference.evidence_id,
...expected,
});
}
const original = harness.session.evidenceById(reference.evidence_id);
expect(original?.normalized_result).toEqual(value);
const inconsistent = createEvidence(
{
path: value.artifact.path,
sha256: value.artifact.sha256,
format: "file",
},
{ id: "historical-wabt", name: "Historical WABT", version: "1" },
{
operation: "inspect_wasm_artifact",
parameters: {},
result: { ...value, wat: { ...value.wat, sha256: "f".repeat(64) } },
},
);
const invalid = await harness.client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "inline", evidence: inconsistent },
view: { kind: "summary" },
},
});
expect(parseMcpToolError(invalid)).toMatchObject({
error: {
code: "invalid_request",
details: {
issues: [
{
path: ["source", "normalized_result", "wat", "sha256"],
reason: "invalid_value",
},
],
},
},
});
await harness.client.ping();
});
it("rejects absent or inconsistent WASM source identity before deriving portable views", async () => {
const value = wasmArtifactFixture();
const harness = await createApplicationMcpHarness({
serverOptions: { environment: {} },
});
onTestFinished(harness.close);
for (const target of [
undefined,
{
path: `${value.artifact.path}.other`,
sha256: value.artifact.sha256,
format: "file" as const,
},
{
path: value.artifact.path,
sha256: "f".repeat(64),
format: "file" as const,
},
]) {
const source = createEvidence(
target,
{ id: "historical-wabt", name: "Historical WABT", version: "1" },
{ operation: "inspect_wasm_artifact", parameters: {}, result: value },
);
for (const view of [
{ kind: "summary" },
{ kind: "wasm", facet: "exports", offset: 0, limit: 1 },
]) {
const reply = await harness.client.callTool({
name: "inspect_analysis_view",
arguments: { source: { kind: "inline", evidence: source }, view },
});
expect(parseMcpToolError(reply)).toMatchObject({
error: {
code: "invalid_request",
details: {
issues: [{ path: ["source", "subject"], reason: "invalid_value" }],
},
},
});
}
}
await harness.client.ping();
});
it("retries cleanup of the injected WASM module after server shutdown fails", async () => {
const close = vi
.fn()
.mockRejectedValueOnce(new Error("retained WASM workspace"))
.mockResolvedValue(undefined);
const service = new WasmArtifactService({
close,
inspect: () => Promise.resolve(ok(wasmArtifactExecution())),
});
const session = createTestBinarySession(() => {
throw new Error("No deep provider");
});
const server = createServer(
{ kind: "session", session },
{ wasmArtifact: service },
);
const client = new Client({ name: "wasm-cleanup-retry", version: "1" });
const [a, b] = InMemoryTransport.createLinkedPair();
try {
await server.connect(b);
await client.connect(a);
await expect(server.close()).rejects.toThrow("REA session cleanup failed");
await server.close();
expect(close).toHaveBeenCalledTimes(2);
} finally {
await client.close();
await server.close();
await session.close();
}
});
tests/fixtures/wasm/artifact.ts
Vendored
+16 -11
View File
@@ -1,7 +1,10 @@
import { createHash } from "node:crypto";
import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js";
import type { WasmArtifact } from "../../../src/domain/wasm/wasmArtifact.js";
/** Source-owned transport seam; upstream binaries are verified in a separate lane. */
export const wasmArtifactFixture = (path = "/artifacts/module.wasm") => ({
export const wasmArtifactFixture = (
path = "/artifacts/module.wasm",
): WasmArtifact => ({
artifact: { path, sha256: "a".repeat(64), bytes: 8 },
validation: "valid" as const,
sections: [],
@@ -18,16 +21,18 @@ export const wasmArtifactFixture = (path = "/artifacts/module.wasm") => ({
source: "https://github.com/WebAssembly/wabt" as const,
release: "1.0.42" as const,
source_commit: "ff0ef7e0009402740c805a9744c09b05be063e48" as const,
commands: ["wasm-validate", "wasm-objdump", "wasm2wat"].map((tool) => ({
tool,
path: `/tools/${tool}`,
sha256: "b".repeat(64),
version_banner: "1.0.42\n",
arguments:
tool === "wasm-objdump"
? ["-h", "-x", "module.wasm"]
: ["--enable-all", "module.wasm"],
})),
commands: (["wasm-validate", "wasm-objdump", "wasm2wat"] as const).map(
(tool) => ({
tool,
path: `/tools/${tool}`,
sha256: "b".repeat(64),
version_banner: "1.0.42\n",
arguments:
tool === "wasm-objdump"
? ["-h", "-x", "module.wasm"]
: ["--enable-all", "module.wasm"],
}),
),
timeout_ms: 30_000,
output_budget_bytes: 32 * 1024 * 1024,
},