Files
rea/docs/verification/native-function-summary-2026-10-23.md
Aaron K. Clark 1a48a82020 feat(analysis)!: default function MCP output to retained summaries (#5660)
* feat(ghidra): bound one procedure's decompile without closing the session

Fixes #5340

* fix(ghidra): report decompile limit as a request timeout

Keeps BinarySession from marking every Ghidra operation unavailable after one procedure's decompile reaches the limit, and names the MCP tool-call deadline in the remediation.

* fix(ghidra): budget the decompile limit and report cancellation-closed sessions

Adds REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS to the derived Codex/Kimi tool deadline and forwards it through setup. Marks a remote failure that settles after caller cancellation so the timeout projection no longer promises session reuse.

* feat(ghidra)!: default analyze_function to a retained summary

analyze_function was the one analysis tool never wired into REA's
detail: summary|complete convention, so it always returned the complete
native dossier inline (0.2-10 MB, dominated by value-flow and
references). Agents on tool-result-capped harnesses never reached the
pseudocode.

Add `detail` to analyze_function, defaulting to summary: return a
compact dossier (procedure identity, pseudocode size, facet counts,
value-flow availability) and retain the complete dossier Evidence in
the session. Page pseudocode, references, assembly and value-flow
through normalized_result.parent_evidence_id with inspect_analysis_view,
which already projects every native facet. detail: complete returns the
full dossier inline as before. Delivery detail never reaches the
provider or Evidence identity.

BREAKING CHANGE: analyze_function now defaults to the compact summary
view instead of the complete dossier inline. Pass detail: "complete"
for the previous behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* feat(cli): expose analyze_function detail on the function command

Wire `rea function` into the shared CLI detail-delivery convention
(--detail, --evidence_output) so the CLI and MCP adapters offer the same
summary | complete choice for analyze_function. Summary delivery writes
the complete dossier Evidence to a bundle and returns the compact
summary view; it fails before analysis when --evidence_output is
missing, so no provider work is wasted. The CLI default stays complete
because a one-shot command has no in-process Evidence retention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* refactor(ghidra): harden analyze_function summary after independent review

FlatlineRoundtable cross-model review (6 lanes) converged on the summary
projection's availability semantics being underspecified. Document that
facet_availability is authoritative over raw pseudocode length, that
code_units are UTF-16 (the unit inspect_analysis_view pages by), and that
value_flow.truncated is meaningful only when value-flow is available.
Parenthesize the nullish-coalescing/comparison for clarity. No behavior
change; declined claims (value-flow "truncated collapse", unresolved_calls
dead code) were schema-impossible or already correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* test(ghidra): cover analyze_function detail wiring on both adapters

Close the roundtable's top convergent finding: the summary path had only
domain-projection coverage, not the server/CLI wiring where the breaking
behavior lives.

MCP server (registerEnhancedTools): the default delivers a retained
summary whose complete parent is recorded for paging; delivery detail is
stripped before the provider request; detail: complete returns the full
dossier inline; a registration without retention rejects a summary
request before any provider work.

CLI (rea function): the default returns the complete dossier; --detail
summary without --evidence_output fails before analysis; with the bundle
sink it returns the compact summary and writes the complete Evidence
bundle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): keep analyze_function delivery detail out of provider inputs

Address the five automated review findings on #5660.

P1: `detail` lived in the shared enhancedInputSchemas.analyze_function, so
inspect_native_api composed `{procedure, detail: "summary"}` and the strict
Ghidra analyze_function schema rejected it. Delivery detail is now owned by
the analyze_function MCP tool contract; the handler separates it before any
provider call, and executeFunctionAnalysisEvidence parses the provider input
only. The MCP regression test validates provider input exactly as the
production Ghidra client does and exercises inspect_native_api.

P2: the analyze_function summary Evidence now carries the parent's
relationship_coverage like native facet views, so partial or unknown
caller/callee discovery is not reported as complete.

P2: the summary reports the dossier's declared facet_availability, nulls the
counts of facets the provider declared unavailable, and lists each reason in
unknowns, so Kuna's empty references, comments and basic blocks are not
mistaken for observed absence.

P2: inspect-analysis-view evidence-file sources accept the Evidence bundle
that --evidence_output writes, selecting a record with source.evidence_id
when the bundle holds several; the CLI test reads the saved bundle back
through that route.

Tests that assert the complete dossier over public analyze_function now pass
detail complete, and the oversized-dossier test asserts the summary view it
previously expected to be unsupported.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): keep omitted function facets unknown in the summary

The analyze_function summary coerced an omitted unresolved_calls record to
a count of 0 while the native facet view reports the same dossier as "not
recorded", and it carried no counts at all for the pageable
swift_string_literals facets. Count optional records as null when the
dossier omits them, name them in unknowns, and add nullable counts for both
Swift literal facets. The summary Evidence envelope also dropped the
function address: createAnalysisViewEvidence now derives one function-view
predicate for both the address location and relationship coverage.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(scripts): request complete analyze_function dossiers in real-provider checks

The real-provider check scripts call analyze_function over MCP and then
consume the complete dossier: requireFunctionDossier, functionDossierSchema,
.procedure/.pseudocode reads, operation assertions, compare_functions inputs
and the oversized-dossier rejection path. The summary default broke them.
Every such call site now passes detail: "complete". Provider-direct callers
are unchanged because detail is an MCP contract field; the admission-error
sites stay unchanged because provider failures return before the summary
branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): keep unavailable value-flow counts unknown in the summary

summarizeDossier coerced the five value_flow_* counts to zero when
native_value_flow was null (IDA, Kuna) or declared unavailable (Ghidra),
so the summary reported empty collections while value_flow.available said
otherwise. The counts are now null unless value flow is available, with the
reason already named in unknowns.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(evidence): budget authentication headroom by file shape and serialized size

The evidence-file headroom reserved two parsed copies, calibrated for one
record, and charged strings at two bytes per character. Bundle
authentication additionally holds the schema-parsed bundle, the re-parsed
records and two JSON.stringify results, which expand control characters and
lone surrogates sixfold: a bundle whose raw_result is a million NUL
characters estimated at 2 MB parsed but needed 12 MB of serialized text.

The budget now has a parsed term and a serialized term per file shape, owned
by EvidenceAuthenticationHeadroom and shared by the analysis-view reader and
the bundle importer, which had no guard. Two transient peaks the budget did
not name are removed at their producers: the canonical digest hashes a large
leaf directly instead of concatenating it into its buffer, and the JSON
validator walks containers by cursor so a wide array no longer queues one
entry per member.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(mcp): route analyze_function summary replies to dossier consumers in docs and prompts

The MCP guide, the version-comparison and crash-triage prompt contracts,
the authored skill and the Kuna, IDA and function-matching docs still told
agents to pass analyze_function replies straight to compare_functions or
build_call_path. Under the summary default those consumers reject the reply.
They now name the retained-evidence reference for compare_functions, the
observations fields of match_native_functions, and detail: "complete" for
build_call_path, which accepts only inline dossiers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* test(evidence): pin validator member order and keep interrupted validation yielding

Independent review of the cursor validator found the interrupted-validation
fixture relied on the old last-first walk: its over-deep head was reached
before the first cooperative yield. The padding now precedes the deep
element so the walk yields first, which is the invariant the test guards.
A new case pins first-defect-in-member-order reporting, skipped
non-enumerable keys no longer count toward the yield cadence, and the
import-headroom test loads the reader through a fresh module graph so it
holds under the services project's shared workers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): align deadline bounds and cancellation outcomes

* refactor(ghidra): shorten deadline recovery guidance

* fix(mcp): clarify the scope of inferred decompile deadlines

* test(ghidra): verify deadline workflows and document measured costs

Separate failure and registration cases by their distinct contracts.
Record actual Linux Ghidra, CLI and MCP evidence with measured context,
elapsed time, CPU and memory, and explicit remaining coverage limits.

* docs(ghidra): pin integration and distinguish baseline format failures

* fix(ghidra): own failure lifecycle wording in the provider

* test(ghidra): budget timeout fixture generation before allocation

* fix(analysis): preserve summary provenance and bound bundle guidance

* fix(ghidra): align capability and traversal deadline guidance

* style(config): align deadline guidance formatting

* docs(ghidra): record capability and traversal verification

* docs(analysis): use absolute paths for Evidence files

* docs(analysis): record completed function summary measurements

* docs(analysis): update main integration validation

---------

Co-authored-by: Syed Rishad <rishadzzzz@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-24 08:18:53 +08:00

8.6 KiB

Native function summary verification — 2026-10-23

Behavior and source

MCP analyze_function defaults to a compact derived summary. The complete function Evidence stays in the session and can be selected by the returned parent_evidence_id through inspect_analysis_view. detail: "complete" keeps the historical full reply. Delivery detail stays outside provider inputs and parent Evidence identity. This MCP default is a breaking change.

CLI rea function still defaults to complete output. Opting into summary requires an exclusive --evidence_output bundle destination. The authenticated bundle can be paged by an absolute evidence-file source and explicit record ID after the provider closes. Plain records also honor an explicit ID.

The implementation reuses the existing JSON authentication/headroom owner and one bundle-authentication function for import and file views. It reads one file once per request. Ambiguous-bundle guidance retains complete IDs and bounds JSON-encoded reference previews to 2 KiB, omitting oversized operation labels. Native summaries inherit function locations and relationship coverage; omitted or unavailable facts stay unknown rather than becoming zero.

Measured source:

  • Before: fb6c88d518ab9b84ed396cf82dcb1c28a608c501.
  • After: 4e3d930ada93014efa43610265b58307de49a687.
  • Harness SHA-256: d67b23db21024361b34f796add1ff91bf3fbcda1991ea9e369ef8bcf64c71ec4.
  • Linux x86-64; pinned Node 24.22.1 / npm 11.16.0; bring-your-own Ghidra 12.1.4 and JDK 21.0.12.1+1. No target execution.
  • Target SHA-256: 9c49d2eef9c2083f6f66f38de4ed0ee1bac17ca5b73dc8f3a90d9ba62a8d1d5d. Its 3,000-branch C source is the timeout verifier fixture, SHA-256 f99074876faae8e18e45265dab007b6fcc396fe909d57efab72adc8925a960c7. Known entries 0x1010f9 and 0x128ed6 apply only to this exact artifact.

Completed public MCP comparison

One fresh production stdio MCP client discovers all 221 tools, opens the target, then obtains two answers: the small function's identity, availability/counts and complete pseudocode; and the large function's identity, availability, assembly/ block counts and first 20 assembly rows. It repeats both reads in the same session, performs one matched complete-small compatibility read and closes.

Before, each default function reply carries the full dossier. After, each answer uses the default summary and one retained-Evidence page. Both lanes use a one-second native decompile limit, an explicit 600-second SDK tool deadline and matching 128 MiB server/client receive budgets. Selected facts match between implementations and rounds. The large function has 45,007 assembly rows and 9,001 blocks with pseudocode unavailable; the small pseudocode is 64 bytes, SHA-256 c947ff22cab1ffef9b962b12d82e12d0e5f5e2a6c5634a2ed5ea3c353e1d5807. Each run retained the same observed JVM start identity through its tasks and completed cleanup with an unchanged target digest.

Whole benchmark Before After
Estimated exchange tokens 44,217,047 1,033,322
Elapsed seconds 237.715 128.600
Waited process-family CPU seconds 211.656 123.083
Sampled family RSS MiB 3951.7 2827.8
Catalog/tool exchanges 8 12

Estimated context falls 97.66% for this completed task, including the larger catalog and four extra page calls. The catalog alone grows from 982,457 to 985,662 estimated tokens. The complete-small compatibility read remains included in both totals.

MCP phase elapsed seconds Before After
First small function, including deferred JVM startup 80.3747 69.0745
First large function, with the JVM already warm 57.2394 19.8205
Repeated small function 0.0534 0.0601
Repeated large function 56.4148 19.7744

The repeated small task adds a page call and takes about 6.7 ms longer in this pair. Large partial dossiers remain live and retry native analysis; healthy small-function snapshot reuse remains available. This change reduces agent output and its serialization/transport work while still producing and retaining the complete parent dossier.

Tokens are tokenx 1.3.0 estimates over the actual catalog, arguments and whole SDK results, including both text and structured projections. Protocol framing, handshake, prompts, client rendering/replay and actual model usage are unmeasured. The whole benchmark includes token estimation, hashing, resource sampling and exchange-file writes. Summed measured MCP phases exclude the subsequent counting/writing overhead: 217.327 → 127.538 seconds. The whole-benchmark timings must not be presented as pure provider analysis time.

Node heaps were 2 GiB, Ghidra's heap 1 GiB, affinity CPUs 0/1 and Java active processor count two. The observed-family early-stop guard allowed 6 GiB RSS, 600 seconds CPU and 900 seconds elapsed; neither completed run hit a guard. RSS was sampled every 50 ms through recursive /proc observations. Shared pages can be double-counted and brief/unobserved child transitions can be missed; this is not a kernel cgroup peak or hard family memory limit. Waited CPU includes reaped descendants. Baseline ran first, OS caches were not flushed and the host was shared with other work. This is one paired investigation, with cold/result-cache distinctions above, rather than a universal speed claim.

An earlier baseline attempt used a 240-second SDK deadline and began with the large function. It timed out, then closed successfully; the whole attempt took 280.628 seconds and 219.562 seconds waited CPU, with no resource-guard stop. Its failed receipt/exchanges remain separate. It is excluded from completed-task comparisons. The final pair uses the same longer deadline and task order on both implementations.

Public CLI and retained files

The real default CLI command returned a complete small-function dossier with the expected pseudocode (70.627 s elapsed, 70.034 s CPU). The first verification harness assumed bare Evidence and failed to unwrap the successful public ok/data/meta envelope. After correcting the harness, its completed receipt and output were verified and reused without another provider analysis.

The summary CLI command analyzed the large function and wrote a 29,141,012-byte bundle. It returned an 8,499-byte summary envelope in 78.163 seconds, with 94.196 seconds CPU and 2,381.2 MiB sampled family RSS. Two fresh CLI processes, with the Ghidra installation path unavailable, authenticated that bundle and returned its first and final 20 assembly rows. Coverage reports 45,007 total rows, correct continuation and exhaustion. The target digest remained unchanged.

Those independent file-page commands took 17.218 / 18.175 seconds, 15.150 / 17.015 seconds CPU and about 666 / 669 MiB RSS. Each fresh command reads and authenticates the complete bundle; this is not sublinear disk access or a cross-process cache. Complete evidence remains available at the caller's file.

Validation and limits

  • Full npm run check passed after integration with main a062529bb2aa064be37e82df39c59c4fa798fe50 at source b7c3f0cd79c4a02b135da56fff06de08dd456ff7. docs:check also passed.
  • The latest focused contract/progress/snapshot/local-path run passed 139 tests; the measured integration's selected run passed 136 and an earlier broader CLI-selector/contract run passed 169. Overlaps are not summed. The earlier domain/Evidence run had 87 passes and one legacy progress-fixture failure. The explicit-complete fixture and new summary's increasing notifications subsequently passed in a separate eight-case MCP run.
  • Actual SDK/schema checks passed all 15 cases after regenerating ignored test catalog metadata. The initial run had 14 passes and one stale-catalog equality failure; its receipt is retained.
  • The production CLI/MCP observations above use real Linux Ghidra. Hopper, IDA, Kuna, macOS and Windows behavior is covered only to the extent of the selected source/boundary tests, not new real-provider verification.
  • Earlier source integration fixes and later main changes were not included in every historical test/performance pin. The measured pair remains tied to the explicit source commits above.

Raw exchanges, protocol observations, control failures, scripts and resource receipts are retained under /tmp/rea-maintenance-20261023/5660/. Provider projects, generated binaries, build outputs and local planning files are not committed.