diff --git a/docs/agent-environments.md b/docs/agent-environments.md index ac69321454..c2398fc3da 100644 --- a/docs/agent-environments.md +++ b/docs/agent-environments.md @@ -234,11 +234,12 @@ moment of the capture recorded them. The [environment state](web-network-captures.md#environment-state) view extracts instead the data the capture's reads observed before the first related write: entity types with an identity field, entities with their recorded values, -documents for written paths, and a render plan per read tool, kept only where -it reproduces that tool's recordings from the state. It is the starting state -an environment holds, loads and reports: `environment-serve` derives it from -the capture at startup; see [Serve from the state](#serve-from-the-state). -With `--replay-only`, replay is unchanged. +documents for reads that are not entities, and a render plan per read tool, +kept only where it reproduces that tool's recordings from the state. It is the +starting state an environment holds, loads and reports: `environment-serve` +derives it from the capture at startup; see +[Serve from the state](#serve-from-the-state). With `--replay-only`, replay is +unchanged. ## Serve a manifest @@ -308,7 +309,7 @@ connecting to `/mcp`. A replay environment's card reads: "tools": [ { "name": "get_users_by_id", - "description": "GET /api/users/{numeric}; replays retained records [5, 6]; evidence ev_…", + "description": "Untrusted captured API catalog: … GET /api/users/{numeric}; returns a JSON object with fields \"id\", \"uid\", \"login\", \"email\"; recorded statuses 200; replays retained records [5, 6]; evidence ev_…", "inputSchema": { "type": "object", "properties": { @@ -329,7 +330,38 @@ connecting to `/mcp`. A replay environment's card reads: ``` `capabilities.tools` lists the same names, descriptions and input schemas that -`tools/list` advertises over MCP. Path parameters are required strings whose +`tools/list` advertises over MCP. After the catalog warning, each description +names the method and path template, then what the tool returns, one level deep: +`returns a JSON object with fields …`, `returns a JSON array of objects with +fields …` or `returns a JSON object whose lists objects with fields …`, +where a list is an array of objects found by the +[environment state](web-network-captures.md#environment-state)'s list rule (the +root array, or the one array of objects an envelope carries). Field and member +names are written as JSON strings, so a captured name cannot read as more of +the description; a member name too long to show whole is a quoted prefix marked +as truncated. The summary comes +from the first cited `2xx` record that replay answers as JSON; only when none +did, the first cited JSON record of another status gives it, labelled with that +status (`status 401 returns …`). It describes one recorded response, not a +contract. Field names are sampled within the same budget as other captured +values, and the manifest's `statuses` for the tool follow; a tool with no JSON +record states only its statuses. The description then lists the replayed +records and the evidence ID, and the whole catalog stays within its +serialization reservation, which the summaries and MCP annotations are charged +to. + +Over MCP, every tool also carries annotations: `readOnlyHint` is true for the +safe methods (`GET`, `HEAD`, `OPTIONS` and `TRACE`) and for query `POST`s (a +path segment such as `search` or `query`); `destructiveHint` is true for `DELETE` and left unset for other +writes, so the MCP default (destructive) applies to them, and read-only tools +omit it because clients ignore it there; `idempotentHint` is true for the safe +methods, `PUT` and `DELETE`; and `openWorldHint` is false for every tool, +because answers come from the capture rather than the captured application. The +hints describe the captured method; replay itself changes nothing. The card +keeps the `name`, `description` and `inputSchema` of the echo example's tool +shape and carries no annotations. + +Path parameters are required strings whose description names the segment index, the templating rule and the observed values (sampled within the list budget above; the manifest keeps them all); query parameter names are optional strings; write methods accept an @@ -399,7 +431,9 @@ naming the entity, collection or document path, or `{"source": "recording", "reason": …}` with the view's reason, `no-view` or `unrecorded-request`. A state answer carries the base record's status, content type and `replay` provenance with `truncated: false`; one whose JSON would exceed 256 KiB is -refused as a `resource_constraint` tool error rather than cut. Serving is +refused as a `resource_constraint` tool error rather than cut. Extraction does +not state-back a document whose rendered JSON exceeds 256 KiB; that read stays +recorded (`rendered-oversize`) and replays the captured body. Serving is read-only: `POST`, `PUT`, `PATCH` and `DELETE` tools, other than query `POST`s, are not served, so nothing a call does changes the state. Query `POST`s stay served and answer from their recordings with the reason `query-body`. diff --git a/docs/mcp-contracts.md b/docs/mcp-contracts.md index 905c320e9a..a1540590b4 100644 --- a/docs/mcp-contracts.md +++ b/docs/mcp-contracts.md @@ -332,7 +332,7 @@ See [environment manifests](web-network-captures.md#environment-manifest). `inspect_analysis_view` also accepts `environment-state` for the same parents, extracting the initial state the recorded reads observed: entity types with an -identity field, entities, documents for written non-entity paths, and a render +identity field, entities, documents for non-entity reads, and a render plan per read tool that reproduces its recordings from the state or names why it cannot. Types and identity are inferences, so the Evidence is always inferred. See [environment state](web-network-captures.md#environment-state). diff --git a/docs/specs/agent-environments/state-coverage/plan.md b/docs/specs/agent-environments/state-coverage/plan.md new file mode 100644 index 0000000000..dfe6d688da --- /dev/null +++ b/docs/specs/agent-environments/state-coverage/plan.md @@ -0,0 +1,32 @@ +# Implementation plan + +1. In `environmentStateRules.ts`, parse pointers into tokens and let `at` and + `replaceAt` read and replace element 0 of a one-element array at the root + or in one member; extend `locateItem` with those forms after the object + forms, requiring an identity name there; add `identityCandidates`, the + identity names then suffix-named fields, and let `locateList` prefer arrays + identified by name, then arrays of two or more items identified by suffix. +2. In `EnvironmentStateExtraction.ts`, plan lists with the shared candidates, + treat an empty recording as unconstraining, accept a suffix-named field only + once a recording shows it distinct across two or more items, leave reads + without a list or identity to documents, drop the written-path requirement + and the `not-entity` fallback, record `path-varies` apart from + `query-varies`, and keep each entity's earliest sighting; + `environmentStateMerge.ts` cites records in timeline order. Remove + `not-entity` from the reason type and result schema, add `path-varies`, and + describe the new pointer forms and `field-name` basis. +3. Move the classification of retained bodies (none, base64, cut, JSON, text) + into `replayedBody` in `environmentSource.ts`, which owns recorded + exchanges; replay renders from it and tool descriptions read its JSON case. +4. Make `environmentToolDefinitions` take the `EnvironmentSource`; describe the + first `2xx` JSON record, else the first JSON record labelled with its status, + one level deep with the state's list rule and the existing sample budget; + list the manifest tool's statuses; and charge the whole description and the + tool's MCP annotations to the catalog reservation. Update the serve and + bundle callers. +5. Let the tool definitions own the MCP annotations derived from the method and + the query-`POST` rule; `serveEnvironment.ts` registers them and the card + leaves them out. +6. Cover domain, services, adapters and the compiled MCP boundary; document the + rules in the capture guide, descriptions and hints in the agent environments + guide, and the contract text. diff --git a/docs/specs/agent-environments/state-coverage/spec.md b/docs/specs/agent-environments/state-coverage/spec.md new file mode 100644 index 0000000000..0477bb9f50 --- /dev/null +++ b/docs/specs/agent-environments/state-coverage/spec.md @@ -0,0 +1,99 @@ +# Wider environment state coverage and tool descriptions + +Issue: https://github.com/morluto/rea/issues/4933 + +## Outcome + +The `environment-state` view backs more read tools with state, and a served +environment tells agents more about each tool. Read-only paths whose responses +are not entities become documents, listed items may be identified by an +`…Id` or `…_id` field, an item wrapped in a one-element array is located +inside it, and reads that vary by path get their own recorded reason. Each tool description says what the tool returns and which statuses +were recorded, and every MCP tool carries method hints. This increment adds no +tool, view kind, provider, network activity or write effect. + +## Requirements + +- A `GET` list tool (its template does not end in a parameter) that yields no + item or list plan holds its latest initial value as a `document` when its + initial reads have one request line, whether or not the capture writes its + path, unless its value changed between two of those reads across any + recorded write (even under another root), which leaves it `fields-differ`. A + value whose rendered JSON exceeds 256 KiB stays `rendered-oversize` and is + replayed from the recorded body, which itself fit. More request lines give + `path-varies` when they differ by path (a template with a parameter before + its end) and `query-varies` when they share one path. With every such read + now a document, `path-varies`, `query-varies`, `body-unusable` or + `rendered-oversize`, the unreachable `not-entity` reason leaves the result + schema, and `no-identity` names item tools only. A document holds the value + of its one `path`. +- When no path parameter proves an identity and no identity name identifies + the listed items, the `field-name` identity is the first key of the first + listed item, in its order, whose name ends in `Id` or `_id`, that every item + carries as a distinct string or number, and that at least one initial + recording shows distinct across two or more items. The last condition goes + beyond "every item carries it distinctly": one item satisfies that + vacuously, and a lone item's suffix-named fields are often foreign keys (a + reaction keyed by its user, a comment by its post), so a list that only ever + held one item stays a document. Empty recordings constrain no field. + Identity names keep precedence, also when choosing among an envelope's + arrays, where a suffix-named field identifies only an array of two or more + items. +- An item response that is a root array of exactly one object, or an object + with one member holding such an array, locates its entity at `/0` or + `//0` when one of the identity names in that object repeats the path + value. Objects at the root or in one member keep precedence. `at` and + `replaceAt` read and replace exactly these forms besides the root and one + member, so the gate and renderers run unchanged. +- An entity's state order is its earliest sighting on the timeline and it cites + its records in timeline order, whichever plan observed it first. +- Each tool description reads `METHOD template; ; recorded statuses + …; replays retained records […]; evidence ev_…` after the untrusted-catalog + warning. `` describes one level deep the first cited `2xx` record + replay answers as JSON: an object's fields, an array of objects' fields, or + an envelope whose member lists objects, where a list is an array of objects + found by the state's list rule; an empty field set reads `with no fields`. + Only when no `2xx` record replays as JSON does the first cited JSON record of + another status give it, labelled `status returns …`; with no JSON + record it is omitted. The statuses are the manifest tool's `statuses`, the + one owner of that set (`no recorded status` when it is empty). Field names + and statuses are sampled within the description sample budget and charged + with the whole description to the catalog reservation before it is retained. + One owner in the environment source classifies each retained body as none, + base64, cut, JSON or text; replay answers from that classification and + descriptions read only its JSON case. +- MCP tools carry `readOnlyHint` (true for the safe methods `GET`, `HEAD`, + `OPTIONS` and `TRACE`, and for query `POST`s), + `destructiveHint: true` for `DELETE` (unset for other writes, so the MCP + default applies, and for reads, where clients ignore it), `idempotentHint` + (true for the safe methods, `PUT` and `DELETE`) and `openWorldHint: false`. The + tool definitions own the hints and charge them to the catalog reservation; + the environment card keeps its `name`, `description`, `inputSchema` tool + shape. + +## Acceptance and limits + +Synthetic captures cover read-only documents, `path-varies` and +`query-varies`, suffix-named identities with their key order, their +two-item proof, empty first recordings and their precedence (including the +choice between an envelope's arrays), a Ghost-like `{"posts":[...]}` item under +a trailing-slash path merging with its list view, a root one-element array, a +one-element filtered list that stays unidentified, and the pointer forms. The +Grafana-shaped fixture covers documents for its read-only paths (its one-member +team list among them) and object, array and enveloped-list descriptions; +synthetic sources cover the `2xx` preference over an earlier error body, the +first of several JSON records, the labelled fallback, the manifest's statuses, +empty, scalar and fieldless bodies, a truncated member name, a name whose +JSON quoting exceeds the sample, a recorded body under 256 KiB whose rendered +JSON exceeds it, non-JSON bodies and the catalog budget; the in-process and +compiled MCP servers advertise the +hints. A filtered list may still answer with one element whose identity name +repeats the requested value; list filters, pagination, lossless big integers +and cross-endpoint entity linking remain out of scope. Documents inline whole +recorded values and enlarge the state result; agent tokens, CPU time and peak +memory of the state and description build were not measured. + +## Deferred issue criteria + +Serving from the state, the AgentEnv data plane and write effects remain later +increments of #4933. This PR must reference rather than close the issue. diff --git a/docs/specs/agent-environments/state-coverage/tasks.md b/docs/specs/agent-environments/state-coverage/tasks.md new file mode 100644 index 0000000000..d2abdab579 --- /dev/null +++ b/docs/specs/agent-environments/state-coverage/tasks.md @@ -0,0 +1,16 @@ +# Tasks + +- [x] Record the coverage and description contract and its limits. +- [x] Extend pointers, item location and identity candidates. +- [x] Require two distinct items before a suffix-named field identifies a list. +- [x] Hold read-only non-entity reads as documents, retire `not-entity` and + separate `path-varies` from `query-varies`. +- [x] Order entities by earliest sighting and cite records in timeline order. +- [x] Classify retained bodies in one owner and describe tool responses, + preferring `2xx` records, with the manifest's statuses. +- [x] Advertise MCP tool annotations, charged to the catalog, without changing + the card. +- [x] Add domain, services, adapters and compiled-boundary coverage. +- [x] Document the rules, descriptions and hints; update the contract text. +- [x] Run static, formatting, duplication, documentation and test gates and the + real-capture measurement. diff --git a/docs/specs/agent-environments/state-coverage/verification.md b/docs/specs/agent-environments/state-coverage/verification.md new file mode 100644 index 0000000000..2ecb0b5047 --- /dev/null +++ b/docs/specs/agent-environments/state-coverage/verification.md @@ -0,0 +1,123 @@ +# Verification + +Verified base: 7a668b73f4783b42eb0e8ce1eeff079f86d53e22 (the environment-state +view branch). + +Executed on the implementation worktree with its existing locked installation +under Node.js v22.23.1 (the repository pins 24.18.0, which was not available); +every check below ran as written in this pass, after the last source change. + +- `npm run format` then `npm run format:check`: passed. +- `npm run check:fast`: typecheck and lint passed (3959 files, no errors). The + touched files carry two lint warnings, both present on the base: `max-params` + in `environmentReplay.ts` and an unused `_` in `environmentCapacity.test.ts`. +- `npx knip`: passed (exit 0). +- `npm run jscpd`: 3.40% duplicated lines over 3348 files, under the 3.5% + gate. The three clones that touch changed test files pair the same blocks as + on the base; none involves code this change adds. +- `npm run scan:todos`: no markers. +- `npm run docs:check`: the built documentation checks and the README + consistency conformance test (5 tests) passed. +- `npx vitest run --project domain --project services`: 341 files, 5859 tests + passed, 5 pre-existing skips. +- `npm run build:cached`: passed. +- `npm run test:focused --` the environment serve adapters and capacity tests, + the environment serve MCP boundary, the environment serve and export CLI + boundaries, the environment serve shutdown process boundary, the + analysis-view MCP and CLI boundaries, the compact input-schema contract and + MCP boundary, the tool-schema validity boundary, the analysis-detail and + advertised-input-example contracts and the qlog acceptance test: 14 files, + 113 tests passed, 1 pre-existing skip. + +Real captures (outside the test suite): the built view ran through the +application service on the same 44 recorded captures (39 HAR, 5 browser +scenario) with all three template rules, in 7 to 50 ms each. 42 produced a +state, and the same two browser-scenario captures are refused with +`resource_constraint` as on the base. The base column is the brief's baseline, +which the same script reproduced on the base build in an earlier pass; this +pass did not rebuild the base. + +| Measure | Base | This change | +| ---------------------------------- | ----- | ----------- | +| Types | 568 | 576 | +| Entities | 2,213 | 2,246 | +| Documents | 2 | 299 | +| Item views | 211 | 213 | +| List views | 497 | 504 | +| Document views | 2 | 299 | +| State-backed read tools (of 1,258) | 710 | 1,016 | +| Recorded: `not-entity` | 306 | (retired) | +| Recorded: `no-identity` | 129 | 32 | +| Recorded: `path-varies` | (new) | 32 | +| Recorded: `query-varies` | 5 | 69 | +| Recorded: `body-unusable` | 32 | 32 | +| Recorded: `list-differs` | 31 | 31 | +| Recorded: `no-init-recording` | 26 | 26 | +| Recorded: `query-body` | 12 | 12 | +| Recorded: `fields-differ` | 3 | 4 | +| Recorded: `envelope-varies` | 4 | 4 | + +Seven list types gained a suffix-named identity (`permissionId`, `product_id`, +`ItemId`, `user_id`, `channel_id`, `spl_id` and `eventId`). Seven more had one +before the two-item proof and lost it, because every initial response listed a +single item: an Airbyte organization, a Mattermost reaction and team +membership, and a Redpanda broker and schema reference became documents; a +NOAA station read is `query-varies` and a Redpanda consumer read +`path-varies`. Two Ghost item tools render from `/members/0` and `/tags/0`. A +third Ghost item tool is located at `/posts/0` but stays `fields-differ`: its +recordings differ by an `include` query. + +State result size, as the JSON length of each Evidence summed over the 42 +states: 2,677 KiB on the base and 3,196 KiB now (+19%), measured by the same +script. The largest growth per capture is Metabase (133 to 236 KiB), Directus +(21 to 93), Redpanda Console (26 to 68) and Vault (9 to 43). Documents take +492,516 of the 3,274,243 UTF-8 bytes of the results now; the base held two. +Agent tokens, CPU time and peak memory of the state view were not measured. + +Tool descriptions on the 44 captures' manifests: all 44 catalogs were admitted +within their reservation. 1,336 of 1,400 tools state what they return: 1,333 +from a `2xx` record and three labelled with another status (Miniflux and +NocoDB 404, Shlink 501), because none of their records succeeded as JSON. The +summaries add 261,502 bytes to a 1,185,893-byte catalog of names, descriptions +and input schemas (median 136, 95th percentile 554, maximum 1,133 bytes per +tool); five reached the field sample bound (one Keycloak, one Lychee and three +Sentry tools with 46 to 105 fields). The MCP annotations add 113,182 bytes to +`tools/list` and none to the card. Building a catalog took a median 1.6 ms and +at most 7.6 ms (Metabase), after the source was loaded. Agent tokens, CPU time +and peak memory of the description build were not measured. + +After merging the environment state view's review fixes (its branch at +6ad63bcc, which also brings main's evidence-file sources), every gate ran +again on the merged tree: `npm run check:fast`, `npm run format:check`, +`npx knip`, `npm run scan:todos` and `npm run docs:check` passed; `npm run +jscpd`: 3.39% duplicated lines over 3362 files; `npx vitest run --project +domain --project services`: 344 files, 5914 tests passed, 5 pre-existing +skips; `npm run build:cached`, then `npm run test:focused --` the serve and +tool-definition adapters, the serve MCP and CLI boundaries, the export CLI +boundary, the analysis-view MCP and CLI boundaries, the compact input-schema +budget and the analysis-detail and advertised-input-example contracts: 10 +files, 98 tests passed. + +The real-capture comparison was repeated on the merged code, with the base +measured on the merged state view without this change: all 44 captures now +produce a state, and 1,011 of 1,277 read tools are state-backed against 625 +on the base (216 item, 416 list and 379 document views against 214, 409 and +2). Recorded reasons: `no-identity` 216 to 32 and `not-entity` 310 to none; +`query-varies` 5 to 76 and `path-varies` 36 are document reads under more +than one query or path; `list-differs` 35, `body-unusable` 33, +`no-init-recording` 28, `query-body` 12, `envelope-varies` 4 are unchanged, +and `fields-differ` is 9 to 10. + +Not covered here: serving from the state, the AgentEnv data plane and write +effects, which are later increments. + +## After the squash onto main + +The branch was squashed onto main once the state-serve branch merged (166fb082). Against a merge of the previous head +into that main, the squash keeps both sides of every conflict and drops nothing of this branch. Two tests then failed +on it and were refitted with the oversized-document fix in 2a926a0a: the bundle test of a state catalog over the +discovery budget uses 300 read tools, inside the window where the state's catalog is refused and replay's, with its +response summaries, still fits (280–320 by a probe), and the rea-state validation test names the pushed document by +the count of documents the state holds. The replay-only card test pinned main's 11,131-byte card; it now pins this +branch's 12,818-byte card, whose descriptions name each tool's recorded responses. The branch then merged main at +d7ca93cc without conflicts. diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index d41b50a1e1..107e2b748b 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -522,9 +522,9 @@ Pass the same parent to `inspect_analysis_view` with `view: {"kind":"environment-state"}` to extract the initial state the capture's reads observed, and a render plan per read tool that reproduces those reads from it. Where the environment manifest names an application's tools, the -state holds its data: entities by type and identity, plus documents for written -paths whose reads are not entities. Use the `template_rules` of the manifest the -state accompanies so tool names match; the view groups records the same way. +state holds its data: entities by type and identity, plus documents for reads +that are not entities. Use the `template_rules` of the manifest the state +accompanies so tool names match; the view groups records the same way. ```json { @@ -560,13 +560,27 @@ refused for them, as the manifest is not. Each `GET` tool whose template ends in a parameter is an item tool, and its type's `collection_template` is the template without that parameter; any other -`GET` tool is a list tool of its own template. An item tool proves its type's +`GET` tool is a list tool of its own template. Empty segments are ignored, so a +trailing slash names the same collection. An item tool proves its type's `id_field` when every initial response carries the requested path value in the -same field, at the root or in one wrapper member (`id_basis: "path-parameter"`). -Otherwise the field is the first of `id`, `uid`, `uuid`, `_id`, `ID`, `UID`, -`Id`, `guid`, `key`, `slug` and `name` that every listed item carries as a -distinct string or number (`id_basis: "field-name"`). A list is the response -array, or the one array of objects in an envelope that is not itself an entity. +same field of one entity object (`id_basis: "path-parameter"`). The object is +the response itself or one wrapper member; failing both, it is the only element +of a one-element array at the root (pointer `/0`) or in one member +(`//0`), as in `{"posts":[{...}]}` or `{"data":[{...}]}`, provided one +of the identity names below repeats the value, since a filtered list may also +answer with one element that repeats its parent's value. Otherwise the field is +the first of `id`, `uid`, `uuid`, `_id`, `ID`, `UID`, `Id`, `guid`, `key`, +`slug` and `name` that every listed item carries as a distinct string or +number; when none does, it is the first key of the first listed item, in its +order, whose name ends in `Id` or `_id` (such as `orgId` or `user_id`), that +every item carries as a distinct string or number, and that at least one +initial response shows distinct across two or more items +(`id_basis: "field-name"`). A single listed item is distinct from nothing, and +its suffix-named fields are often foreign keys, so a list that only ever held +one item is left to a document. Empty initial lists constrain no field. A list +is the response array, or the one array of objects in an envelope that is not +itself an entity, preferring an array whose items an identity name identifies, +then an array of two or more items a suffix-named field identifies. `entities` merge every initial observation of one identity within one collection path: an item view's latest values win over a list view's, a view @@ -583,9 +597,9 @@ Every entity cites its `record_ordinals`. | Shape | Plan | | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `item` | The entity object at `pointer` (the root or one member), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares | +| `item` | The entity object at `pointer` (the root, one member, or `/0` or `//0`), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares | | `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query, no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed | -| `document` | The path's latest initial value, held in `documents` | +| `document` | The latest initial value of a list tool whose responses hold no identified list, when its initial reads share one request line; held in `documents` with that `path`, whether or not the capture writes it, unless its value changed across any recorded write between those reads (`fields-differ`) | | `recorded` | A `reason` the tool replays its recordings instead | An item or list plan is kept only when it reproduces the latest initial @@ -594,11 +608,13 @@ regard to member order; a volatile field compares its recorded value. Each plan lists the `checked_record_ordinals`. Otherwise the tool is a recorded view with reason `envelope-varies`, `fields-differ` or `list-differs`; the other reasons are `query-body` (a query `POST`), `no-init-recording`, -`body-unusable`, `not-entity`, `no-identity` and `query-varies` (a document -read under several queries). Documents are held only for list-shaped paths -that a `POST`, `PUT` or `PATCH` tool writes: a non-entity read of an item path -such as `/api/files/{id}`, or of a path that is only deleted, stays recorded -with `no-identity` or `not-entity`. A body is usable when its content type is +`body-unusable`, `no-identity` (an item read without a proved identity), +`path-varies` (another read under more than one path, as a template with a +parameter before its end), `query-varies` (another read of one path under +more than one query) and `rendered-oversize` (a document value whose JSON +renders to more than 256 KiB). A document holds the value of its one `path`, so for such +a template it says nothing about the template's other paths. A body is usable +when its content type is JSON-like, it is UTF-8 text of at most 256 KiB, every number survives native conversion exactly, and no member is named `__proto__`. @@ -610,7 +626,9 @@ that the state establishes no authorization model, no way the application computes responses or derived values, and no effects of writes. The state accumulator charges the manifest's admission and grouping, every record line, each body as temporary bytes before it is decoded and parsed, and every value -it keeps, within the same 96 MiB budget. The view performs no network activity +it keeps, within the same 96 MiB budget. Documents hold whole recorded response +values whose recorded text and rendered JSON are each at most 256 KiB, so a +capture with many read-only paths yields a correspondingly larger state result. The view performs no network activity and serves nothing; `rea environment-serve` derives it from the capture at startup, serves read tools from it and loads replacement states through the AgentEnv data plane, as diff --git a/src/application/environment/environmentBundle.test.ts b/src/application/environment/environmentBundle.test.ts index 5f2727b6cd..276dcfcf02 100644 --- a/src/application/environment/environmentBundle.test.ts +++ b/src/application/environment/environmentBundle.test.ts @@ -387,10 +387,11 @@ describe("planEnvironmentBundle without a state", () => { it("serves and predicts replay when the state's catalog exceeds the discovery budget that replay's fits", () => { // Read tools only, so leaving out write tools saves nothing while each state description is longer. + // 300 sits between the two catalogs on this fixture: replay still fits and the state catalog does not. const reads = servedSource.manifest.tools.filter( ({ method }) => method === "GET", ); - const tools = Array.from({ length: 350 }, (_, index) => { + const tools = Array.from({ length: 300 }, (_, index) => { const tool = reads[index % reads.length]!; return index < reads.length ? tool @@ -416,7 +417,7 @@ describe("planEnvironmentBundle without a state", () => { served: false, code: "resource_constraint", }); - expect(planned.value.toolCount).toBe(350); + expect(planned.value.toolCount).toBe(300); expect(planned.value.check).toEqual({ tool: "get_annotations", recordOrdinal: 25, diff --git a/src/application/environment/environmentReplay.ts b/src/application/environment/environmentReplay.ts index 7da4391072..4b66ea08ef 100644 --- a/src/application/environment/environmentReplay.ts +++ b/src/application/environment/environmentReplay.ts @@ -1,8 +1,5 @@ -import { isJsonLike } from "../../domain/analysisView/EnvironmentManifestAccumulator.js"; -import { mediaTypeEssence } from "../../domain/analysisView/webNetworkRouteTemplates.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import { AnalysisInputError } from "../../domain/analysisErrorCore.js"; -import { parseExactJson } from "../../domain/exactJsonNumber.js"; import type { JsonValue } from "../../domain/jsonValue.js"; import { iterateUrlQueryParameters, @@ -13,6 +10,7 @@ import { environmentToolArguments } from "./environmentTools.js"; import { REPLAY_BODY_LIMIT_BYTES, ENVIRONMENT_SERVE_OPERATION, + replayedBody, type EnvironmentManifestTool, type EnvironmentSource, type RecordedExchange, @@ -183,11 +181,6 @@ const utf8Prefix = (text: string, maxBytes: number): string => { return text.slice(0, read); }; -/** Bytes a padded base64 string decodes to. */ -const decodedBase64Bytes = (encoded: string): number => - Math.floor((encoded.length * 3) / 4) - - (encoded.endsWith("==") ? 2 : encoded.endsWith("=") ? 1 : 0); - /** Base64 characters, in whole groups, whose bytes fit the replay limit. */ const BASE64_LIMIT_CHARS = Math.floor(REPLAY_BODY_LIMIT_BYTES / 3) * 4; @@ -196,29 +189,26 @@ interface ReplayBody { readonly truncated: boolean; } -/** The limit applies to the recorded bytes, not to their base64 spelling. */ -const replayBase64Body = (encoded: string, cut: boolean): ReplayBody => - !cut && decodedBase64Bytes(encoded) <= REPLAY_BODY_LIMIT_BYTES - ? { body: encoded, truncated: false } - : { body: encoded.slice(0, BASE64_LIMIT_CHARS), truncated: true }; - -/** Parse a retained body as JSON when its content type is JSON-like; otherwise return the text. */ +/** The reply body and whether it was cut, as the source classifies the retained body. */ const replayBody = (exchange: RecordedExchange): ReplayBody => { - const text = exchange.bodyText; - if (text === null) return { body: null, truncated: false }; - if (exchange.bodyEncoding === "base64") - return replayBase64Body(text, exchange.bodyCut === true); - const prefix = utf8Prefix(text, REPLAY_BODY_LIMIT_BYTES); - if (prefix.length < text.length || exchange.bodyCut === true) - return { body: prefix, truncated: true }; - if ( - exchange.contentType !== null && - isJsonLike(mediaTypeEssence(exchange.contentType)) - ) { - const value = parseExactJson(text); - if (value !== undefined) return { body: value, truncated: false }; + const body = replayedBody(exchange); + switch (body.kind) { + case "none": + return { body: null, truncated: false }; + case "base64": + return body.whole + ? { body: body.encoded, truncated: false } + : { body: body.encoded.slice(0, BASE64_LIMIT_CHARS), truncated: true }; + case "cut": + return { + body: utf8Prefix(body.text, REPLAY_BODY_LIMIT_BYTES), + truncated: true, + }; + case "json": + return { body: body.value, truncated: false }; + case "text": + return { body: body.text, truncated: false }; } - return { body: text, truncated: false }; }; /** Recorded paths and queries are cut to this many UTF-8 bytes in replies. */ diff --git a/src/application/environment/environmentSource.ts b/src/application/environment/environmentSource.ts index c412dbc4a3..f7ec97321e 100644 --- a/src/application/environment/environmentSource.ts +++ b/src/application/environment/environmentSource.ts @@ -9,7 +9,11 @@ import { readRecordedResponseBody, type RecordedResponseBody, } from "../../domain/analysisView/recordedResponseBody.js"; -import { readRecordedRequestLine } from "../../domain/analysisView/webNetworkRouteTemplates.js"; +import { isJsonLike } from "../../domain/analysisView/EnvironmentManifestAccumulator.js"; +import { + mediaTypeEssence, + readRecordedRequestLine, +} from "../../domain/analysisView/webNetworkRouteTemplates.js"; import { browserScenarioCaptureSchema } from "../../domain/browserScenarioCapture.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import { @@ -17,8 +21,10 @@ import { type AnalysisInputIssue, } from "../../domain/analysisErrorCore.js"; import { immutableEvidence, type Evidence } from "../../domain/evidence.js"; +import { parseExactJson } from "../../domain/exactJsonNumber.js"; import { projectInputIssues } from "../../domain/inputIssueProjection.js"; import { immutableJsonSnapshotMember } from "../../domain/immutableJson.js"; +import type { JsonValue } from "../../domain/jsonValue.js"; import { err, ok, type Result } from "../../domain/result.js"; import { safeParseOwnedWebNetworkCapture, @@ -47,6 +53,52 @@ export interface RecordedExchange extends RecordedResponseBody { /** Replayed bodies larger than this are cut at a UTF-8 boundary and flagged. */ export const REPLAY_BODY_LIMIT_BYTES = 256 * 1024; +/** How replay answers with an exchange's body; replay and tool descriptions both read it from here. */ +export type ReplayedBody = + | { readonly kind: "none" } + /** Non-UTF-8 bytes as base64; `whole` when every recorded byte fits the replay limit. */ + | { + readonly kind: "base64"; + readonly encoded: string; + readonly whole: boolean; + } + /** UTF-8 text over the replay limit or cut when retained; replay answers with a prefix. */ + | { readonly kind: "cut"; readonly text: string } + | { readonly kind: "json"; readonly value: JsonValue } + | { readonly kind: "text"; readonly text: string }; + +/** Bytes a padded base64 string decodes to. */ +const decodedBase64Bytes = (encoded: string): number => + Math.floor((encoded.length * 3) / 4) - + (encoded.endsWith("==") ? 2 : encoded.endsWith("=") ? 1 : 0); + +/** Classify a retained body once: whole JSON-like UTF-8 text within the limit that parses exactly is JSON. */ +export const replayedBody = (exchange: RecordedExchange): ReplayedBody => { + const text = exchange.bodyText; + if (text === null) return { kind: "none" }; + if (exchange.bodyEncoding === "base64") + return { + kind: "base64", + encoded: text, + whole: + exchange.bodyCut !== true && + decodedBase64Bytes(text) <= REPLAY_BODY_LIMIT_BYTES, + }; + if ( + exchange.bodyCut === true || + // UTF-8 never takes fewer bytes than UTF-16 units, so the length check avoids encoding huge text. + text.length > REPLAY_BODY_LIMIT_BYTES || + Buffer.byteLength(text, "utf8") > REPLAY_BODY_LIMIT_BYTES + ) + return { kind: "cut", text }; + const value = + exchange.contentType !== null && + isJsonLike(mediaTypeEssence(exchange.contentType)) + ? parseExactJson(text) + : undefined; + return value === undefined ? { kind: "text", text } : { kind: "json", value }; +}; + /** A HAR inspected by inspect_web_network_capture, or a session recorded by capture_browser_scenario. */ export type EnvironmentCaptureKind = "web-network-capture" | "browser-scenario"; diff --git a/src/application/environment/environmentState.test.ts b/src/application/environment/environmentState.test.ts index 655414555b..ad36076289 100644 --- a/src/application/environment/environmentState.test.ts +++ b/src/application/environment/environmentState.test.ts @@ -485,12 +485,12 @@ describe("rea-state/v1 validation", () => { "a document for a tool that is not a document view", (document: Editable) => { document.rea_state.documents.push({ - tool: "get_org", - path: "/api/org", + tool: "get_folders", + path: "/api/folders", value: {}, }); }, - "rea_state/documents/0/tool", + `rea_state/documents/${String(state.documents.length)}/tool`, ], [ "a member the format does not define", diff --git a/src/application/environment/environmentState.ts b/src/application/environment/environmentState.ts index 6d6b97e381..c1fd5fc53d 100644 --- a/src/application/environment/environmentState.ts +++ b/src/application/environment/environmentState.ts @@ -61,17 +61,19 @@ export const servedCatalog = ( operation: string = ENVIRONMENT_SERVE_OPERATION, ): Result => { const dataPlane: EnvironmentDataPlane = fromState ? "state" : "none"; - const tools = environmentToolDefinitions( - fromState - ? { + const catalogSource: EnvironmentSource = fromState + ? { + ...source, + manifest: { ...source.manifest, tools: source.manifest.tools.filter(servedFromState), - } - : source.manifest, - source.evidenceId, - name, - { operation, dataPlane }, - ); + }, + } + : source; + const tools = environmentToolDefinitions(catalogSource, name, { + operation, + dataPlane, + }); return tools.ok ? ok({ tools: tools.value, dataPlane }) : tools; }; diff --git a/src/application/environment/environmentStateAnswer.test.ts b/src/application/environment/environmentStateAnswer.test.ts index cc4d9e588f..52ac7667ef 100644 --- a/src/application/environment/environmentStateAnswer.test.ts +++ b/src/application/environment/environmentStateAnswer.test.ts @@ -311,6 +311,17 @@ describe("recorded and document answers", () => { }); }); + it("replays a read whose rendered document exceeds the body limit", () => { + const values = Array.from({ length: 60_000 }, () => "1e3").join(","); + const body = `{"values":[${values}]}`; + const serving = servingFrom([["GET", "/api/config", body]]); + expect(serving.store.state.documents).toEqual([]); + expect(answer(serving, "get_config", {})).toMatchObject({ + replay: { record_ordinal: 0 }, + state: { source: "recording", reason: "rendered-oversize" }, + }); + }); + it("answers a document's recorded line with the state's value for it", () => { const capture = timedCaptureEvidence([ ["GET", "/api/settings", { theme: "dark", density: "compact" }], diff --git a/src/application/environment/environmentTools.test.ts b/src/application/environment/environmentTools.test.ts index 5efc052337..67be8b2cff 100644 --- a/src/application/environment/environmentTools.test.ts +++ b/src/application/environment/environmentTools.test.ts @@ -1,8 +1,12 @@ import { describe, expect, it } from "vitest"; import { environmentServeSource } from "../../../tests/fixtures/environmentServe.js"; -import type { EnvironmentManifestTool } from "./environmentSource.js"; +import type { + EnvironmentManifestTool, + RecordedExchange, +} from "./environmentSource.js"; import { + ENVIRONMENT_CATALOG_LIMIT_BYTES, environmentCard, environmentToolArguments, environmentToolDefinitions as prepareDefinitions, @@ -17,18 +21,16 @@ const environmentToolDefinitions = ( if (!result.ok) throw result.error; return result.value; }; +const withTools = (tools: readonly EnvironmentManifestTool[]) => ({ + ...source, + manifest: { ...source.manifest, tools: [...tools] }, +}); const environmentToolInputSchema = (tool: EnvironmentManifestTool) => { - const [definition] = environmentToolDefinitions( - { ...source.manifest, tools: [tool] }, - source.evidenceId, - ); + const [definition] = environmentToolDefinitions(withTools([tool])); if (definition === undefined) throw new Error("Missing tool definition"); return definition.inputSchema; }; -const definitions = environmentToolDefinitions( - source.manifest, - source.evidenceId, -); +const definitions = environmentToolDefinitions(source); const tool = (name: string): EnvironmentManifestTool => { const found = source.manifest.tools.find((entry) => entry.name === name); @@ -44,7 +46,7 @@ describe("environmentToolDefinitions", () => { const users = tool("get_users_by_id"); const definition = definitions.find(({ name }) => name === users.name); expect(definition?.description).toBe( - `${REPLAY_CATALOG_INSTRUCTIONS} GET ${users.path_template}; replays retained records [${users.record_ordinals.join(", ")}]; evidence ${source.evidenceId}`, + `${REPLAY_CATALOG_INSTRUCTIONS} GET ${users.path_template}; returns a JSON object with fields "id", "uid", "login", "email"; recorded statuses 200; replays retained records [${users.record_ordinals.join(", ")}]; evidence ${source.evidenceId}`, ); expect(definition?.tool).toBe(users); }); @@ -108,6 +110,243 @@ describe("environmentToolDefinitions", () => { }); }); +const described = (name: string): string | undefined => + definitions.find((definition) => definition.name === name)?.description; +/** The users tool citing exactly the given exchanges, with the statuses its manifest recorded. */ +const usersAnswering = ( + answers: readonly Partial[], + statuses: readonly number[] = [200], +): string | undefined => { + const exchanges = new Map( + answers.map((answer, index): [number, RecordedExchange] => [ + index, + { + ordinal: index, + path: "/api/users/2", + query: null, + status: 200, + contentType: "application/json", + bodyText: null, + ...answer, + }, + ]), + ); + const users = { + ...tool("get_users_by_id"), + record_ordinals: [...exchanges.keys()], + statuses: [...statuses], + }; + const [definition] = environmentToolDefinitions({ + ...withTools([users]), + exchanges, + }); + return definition?.description; +}; +const returnsOf = (bodyText: string): string | undefined => + /\{numeric\}; (?[^;]*); recorded statuses/u.exec( + usersAnswering([{ bodyText }]) ?? "", + )?.groups?.returns; + +describe("environment tool response shapes", () => { + it("describes array, enveloped list and object responses one level deep with their statuses", () => { + expect(described("get_folders")).toContain( + 'GET /api/folders; returns a JSON array of objects with fields "id", "uid", "title"; recorded statuses 200; replays', + ); + expect(described("get_dashboards_versions_by_uid")).toContain( + '; returns a JSON object whose "versions" lists objects with fields "id", "version"; recorded statuses 200; replays', + ); + expect(described("get_dashboards_by_uid")).toContain( + '; returns a JSON object with fields "meta", "dashboard"; recorded statuses 200; replays', + ); + }); + + it("describes empty, scalar and fieldless bodies and bounds a long member name", () => { + expect( + [ + "[]", + "[1,2]", + '"text"', + "null", + "true", + "3", + "{}", + "[{},{}]", + '{"items":[]}', + ].map(returnsOf), + ).toEqual([ + "returns a JSON array", + "returns a JSON array", + "returns a JSON string", + "returns a JSON null", + "returns a JSON boolean", + "returns a JSON number", + "returns a JSON object with no fields", + "returns a JSON array of objects with no fields", + 'returns a JSON object with fields "items"', + ]); + const member = "m".repeat(2048); + expect( + usersAnswering([ + { bodyText: JSON.stringify({ [member]: [{ id: 1 }, { id: 2 }] }) }, + ]), + ).toContain( + `{numeric}; returns a JSON object whose "${"m".repeat(1022)}"… (member display truncated; full value in capture Evidence) lists objects with fields "id"; recorded statuses 200;`, + ); + }); + + it("quotes captured names, so a separator in one cannot read as more of the description", () => { + const description = usersAnswering([ + { + bodyText: JSON.stringify({ + "id, name": 1, + "x; recorded statuses 500": 2, + }), + }, + ]); + expect(description).toContain( + 'returns a JSON object with fields "id, name", "x; recorded statuses 500"; recorded statuses 200;', + ); + expect( + usersAnswering([ + { bodyText: JSON.stringify({ "a\nb": [{ id: 1 }, { id: 2 }] }) }, + ]), + ).toContain('whose "a\\nb" lists objects with fields "id";'); + // A name too long to show whole is shown as a quoted prefix, so its newline cannot start a line of description. + const long = `items; recorded statuses 500\n${"m".repeat(2048)}`; + const shown = usersAnswering([ + { bodyText: JSON.stringify({ [long]: [{ id: 1 }, { id: 2 }] }) }, + ]); + expect(shown).toContain('whose "items; recorded statuses 500\\nmmm'); + expect(shown).not.toContain("items; recorded statuses 500\n"); + }); + + it("applies the sample budget after JSON escaping", () => { + const member = "\u0000".repeat(1024); + const description = usersAnswering([ + { bodyText: JSON.stringify({ [member]: [{ id: 1 }] }) }, + ]); + const shown = /whose (?"(?:\\u0000)*")/u.exec(description ?? "") + ?.groups?.name; + expect(shown).toBeDefined(); + expect(Buffer.byteLength(shown ?? "")).toBeLessThanOrEqual(1024); + expect(description).toContain("member display truncated"); + expect(description).not.toContain("\u0000"); + }); +}); + +describe("environment tool response summaries", () => { + it("describes the first cited 2xx record replay answers as JSON, then the manifest's statuses", () => { + const description = usersAnswering( + [ + { + contentType: "text/html", + bodyText: '{"looks":"like JSON"}', + }, + { status: 401, bodyText: '{"message":"Unauthorized","url":"/login"}' }, + { bodyText: '{"id":2,"login":"probe"}' }, + { bodyText: '[{"id":3,"email":"viewer@example.test"}]' }, + ], + [200, 401], + ); + expect(description).toContain( + '{numeric}; returns a JSON object with fields "id", "login"; recorded statuses 200, 401; replays retained records [0, 1, 2, 3];', + ); + }); + + it("labels a JSON record of another status only when no 2xx record replays as JSON", () => { + expect( + usersAnswering( + [ + { contentType: "text/html", bodyText: "

Signed out

" }, + { status: 401, bodyText: '{"message":"Unauthorized"}' }, + { status: 403, bodyText: '{"errors":[{"message":"Forbidden"}]}' }, + ], + [200, 401, 403], + ), + ).toContain( + '{numeric}; status 401 returns a JSON object with fields "message"; recorded statuses 200, 401, 403; replays', + ); + expect( + usersAnswering([{ status: null, bodyText: '{"id":2}' }], []), + ).toContain( + '{numeric}; a record without a status returns a JSON object with fields "id"; no recorded status; replays', + ); + }); + + it("states only the statuses when no cited exchange replays as JSON", () => { + for (const answer of [ + { contentType: "text/html", bodyText: "

Granite Ledger

" }, + { bodyText: '{"id":9007199254740993}' }, + { bodyText: '{"id":2}', bodyEncoding: "base64" as const }, + { bodyText: '{"id":2}', bodyCut: true as const }, + { bodyText: null }, + ]) + expect(usersAnswering([answer])).toMatch( + /\{numeric\}; recorded statuses 200; replays retained records/u, + ); + }); + + it("charges response summaries to the catalog reservation before retaining them", () => { + const fields = Object.fromEntries( + Array.from({ length: 300 }, (_, index) => [ + `field_${String(index).padStart(3, "0")}`, + index, + ]), + ); + const users = tool("get_users_by_id"); + const ordinal = users.record_ordinals[0] ?? 0; + const catalog = (contentType: string) => + prepareDefinitions({ + ...withTools( + Array.from({ length: 200 }, (_, index) => ({ + ...users, + name: `get_users_${String(index)}`, + record_ordinals: [ordinal], + })), + ), + exchanges: new Map([ + [ + ordinal, + { + ordinal, + path: "/api/users/2", + query: null, + status: 200, + contentType, + bodyText: JSON.stringify(fields), + }, + ], + ]), + }); + expect(catalog("text/html").ok).toBe(true); + const bounded = catalog("application/json"); + expect(bounded).toMatchObject({ + ok: false, + error: { + _tag: "AnalysisResourceConstraintError", + resource: "capacity", + reportedLimits: { max_catalog_bytes: ENVIRONMENT_CATALOG_LIMIT_BYTES }, + }, + }); + const [single] = environmentToolDefinitions({ + ...withTools([users]), + exchanges: new Map([ + [ + ordinal, + { + ...source.exchanges.get(ordinal)!, + bodyText: JSON.stringify(fields), + }, + ], + ]), + }); + expect(Buffer.byteLength(single?.description ?? "")).toBeLessThan(1700); + expect(single?.description).toContain( + "/300 shown; full list in capture Evidence)", + ); + }); +}); + describe("environment catalog description budgets", () => { it("bounds captured scalar prose while preserving complete lookup identities", () => { const query = "query-".repeat(1000); @@ -117,10 +356,7 @@ describe("environment catalog description budgets", () => { path_template: `/${"😀".repeat(1000)}`, query_parameter_names: [query], }; - const [definition] = environmentToolDefinitions( - { ...source.manifest, tools: [expanded] }, - source.evidenceId, - ); + const [definition] = environmentToolDefinitions(withTools([expanded])); expect(definition?.tool).toBe(expanded); expect(Buffer.byteLength(definition?.description ?? "")).toBeLessThan(3500); expect(definition?.description).toContain( @@ -152,10 +388,7 @@ describe("environment catalog description budgets", () => { })), record_ordinals: ordinals, }; - const [definition] = environmentToolDefinitions( - { ...source.manifest, tools: [expanded] }, - source.evidenceId, - ); + const [definition] = environmentToolDefinitions(withTools([expanded])); expect(definition?.tool).toBe(expanded); expect(expanded.params[0]?.observed_values).toBe(observations); expect(expanded.record_ordinals).toBe(ordinals); @@ -180,10 +413,7 @@ describe("environment catalog description budgets", () => { ...users, path_template: `/api/${"segment/".repeat(200_000)}{id}`, }; - const [definition] = environmentToolDefinitions( - { ...source.manifest, tools: [long] }, - source.evidenceId, - ); + const [definition] = environmentToolDefinitions(withTools([long])); expect(Buffer.byteLength(definition?.description ?? "")).toBeLessThan(2048); expect(definition?.description).toContain( "path display truncated; full value in manifest Evidence", @@ -330,3 +560,22 @@ describe("environmentCard", () => { expect(environmentCard("renamed", definitions).name).toBe("renamed"); }); }); + +describe("environment tool annotations", () => { + it("marks OPTIONS and TRACE safe and idempotent, as GET is", () => { + const get = tool("get_users_by_id"); + const hints = environmentToolDefinitions( + withTools( + (["OPTIONS", "TRACE"] as const).map((method) => ({ + ...get, + name: `${method.toLowerCase()}_users_by_id`, + method, + })), + ), + ).map((definition) => definition.annotations); + expect(hints).toEqual([ + { readOnlyHint: true, idempotentHint: true, openWorldHint: false }, + { readOnlyHint: true, idempotentHint: true, openWorldHint: false }, + ]); + }); +}); diff --git a/src/application/environment/environmentTools.ts b/src/application/environment/environmentTools.ts index 26fc41bf03..847be3d76d 100644 --- a/src/application/environment/environmentTools.ts +++ b/src/application/environment/environmentTools.ts @@ -1,9 +1,20 @@ +import { + isObject, + isQueryPost, + listItems, + locateList, + pointerTokens, + type JsonObject, +} from "../../domain/analysisView/environmentStateRules.js"; import { AnalysisResourceConstraintError } from "../../domain/analysisErrorCore.js"; +import type { JsonValue } from "../../domain/jsonValue.js"; import { err, ok, type Result } from "../../domain/result.js"; -import { ENVIRONMENT_SERVE_OPERATION } from "./environmentSource.js"; -import type { - EnvironmentManifest, - EnvironmentManifestTool, +import { + ENVIRONMENT_SERVE_OPERATION, + replayedBody, + type EnvironmentManifestTool, + type EnvironmentSource, + type RecordedExchange, } from "./environmentSource.js"; /** Methods whose tools accept an optional request body object. */ @@ -27,36 +38,98 @@ export interface EnvironmentToolInputSchema { readonly additionalProperties: false; } -/** One manifest tool as an MCP tool: name, description and input schema. */ +/** MCP hints for the captured method; answers come from the capture, never the open world. */ +export interface EnvironmentToolAnnotations { + readonly readOnlyHint: boolean; + /** Set for `DELETE` only; other writes leave the MCP default (destructive). */ + readonly destructiveHint?: true; + readonly idempotentHint: boolean; + readonly openWorldHint: false; +} + +/** One manifest tool as an MCP tool: name, description, input schema and the hints MCP advertises; the card omits the hints. */ export interface EnvironmentToolDefinition { readonly name: string; readonly description: string; readonly inputSchema: EnvironmentToolInputSchema; + readonly annotations: EnvironmentToolAnnotations; readonly tool: EnvironmentManifestTool; } +/** RFC 9110 safe methods: a preflight `OPTIONS` reads, as `GET` does. */ +const SAFE_METHODS: ReadonlySet = new Set([ + "GET", + "HEAD", + "OPTIONS", + "TRACE", +]); + +const IDEMPOTENT_METHODS: ReadonlySet = new Set([ + ...SAFE_METHODS, + "PUT", + "DELETE", +]); + +const environmentToolAnnotations = ({ + method, + path_template: path, +}: EnvironmentManifestTool): EnvironmentToolAnnotations => ({ + readOnlyHint: SAFE_METHODS.has(method) || isQueryPost(method, path), + ...(method === "DELETE" ? { destructiveHint: true as const } : {}), + idempotentHint: IDEMPOTENT_METHODS.has(method), + openWorldHint: false, +}); + /** Agent-context budget for each captured display sample, excluding its provenance notice. */ const DESCRIPTION_SAMPLE_BYTES = 1024; -/** A display prefix is explicitly separate from the complete identity held by the manifest. */ -const describeText = (text: string, label: string): string => { +/** Whether a rendered sample fits the display budget. Callers' `show` never shortens its input. */ +const sampleFits = (shown: string): boolean => + shown.length <= DESCRIPTION_SAMPLE_BYTES && + Buffer.byteLength(shown, "utf8") <= DESCRIPTION_SAMPLE_BYTES; + +/** + * A display prefix is explicitly separate from the complete identity held by the Evidence named. + * `show` renders the text or its prefix. The budget applies to that rendering, so JSON quoting of a + * name full of NULs or newlines cannot outgrow the sample. + */ +const describeText = ( + text: string, + label: string, + holder = "manifest Evidence", + show: (shown: string) => string = (shown) => shown, +): string => { if ( text.length <= DESCRIPTION_SAMPLE_BYTES && Buffer.byteLength(text, "utf8") <= DESCRIPTION_SAMPLE_BYTES - ) - return text; - const { read } = new TextEncoder().encodeInto( - text, - new Uint8Array(DESCRIPTION_SAMPLE_BYTES), - ); - return `${text.slice(0, read)}… (${label} display truncated; full value in manifest Evidence)`; + ) { + const shown = show(text); + if (sampleFits(shown)) return shown; + } + let low = 0; + let high = Math.min(text.length, DESCRIPTION_SAMPLE_BYTES); + let best = 0; + while (low <= high) { + const mid = (low + high) >> 1; + if (sampleFits(show(text.slice(0, mid)))) { + best = mid; + low = mid + 1; + } else high = mid - 1; + } + // A lone surrogate is not a character. Step off a high surrogate so the prefix stays whole. + if (best > 0 && best < text.length) { + const unit = text.charCodeAt(best - 1); + if (unit >= 0xd8_00 && unit <= 0xdb_ff) best -= 1; + } + return `${show(text.slice(0, best))}… (${label} display truncated; full value in ${holder})`; }; -/** Sample whole values before joining; keep full lists in the manifest that owns them. */ +/** Sample whole values before joining; keep full lists in the Evidence that owns them. */ const describeList = ( values: Iterable, count: number, display: (value: Value) => string = String, + holder = "manifest Evidence", ): string => { if (count === 0) return "none"; const sample: string[] = []; @@ -76,7 +149,80 @@ const describeList = ( const listed = sample.join(", "); return sample.length === count ? listed - : `${listed}${sample.length === 0 ? "" : ", "}… (${String(sample.length)}/${String(count)} shown; full list in manifest Evidence)`; + : `${listed}${sample.length === 0 ? "" : ", "}… (${String(sample.length)}/${String(count)} shown; full list in ${holder})`; +}; + +/** + * A captured name as a JSON string, so a separator, quote or newline in it cannot read as more of the + * description. A name too long for a sample is left as it is: the sample refuses it either way. + */ +const quotedName = (name: string): string => + name.length > DESCRIPTION_SAMPLE_BYTES ? name : JSON.stringify(name); + +/** Field names of the objects in first-seen order; the recorded bodies stay in the capture Evidence. */ +const withFields = (objects: readonly JsonObject[]): string => { + const names = new Set(); + for (const object of objects) + for (const name of Object.keys(object)) names.add(name); + return names.size === 0 + ? "with no fields" + : `with fields ${describeList(names, names.size, quotedName, "capture Evidence")}`; +}; + +/** One level of a JSON body; a list is an array of objects found by the state's list rule. */ +const describeJson = (value: JsonValue): string => { + const pointer = locateList(value, undefined); + const items = pointer === undefined ? [] : (listItems(value, pointer) ?? []); + if (pointer !== undefined && items.length > 0) { + const [member] = pointerTokens(pointer); + return member === undefined + ? `a JSON array of objects ${withFields(items)}` + : `a JSON object whose ${describeText(member, "member", "capture Evidence", JSON.stringify)} lists objects ${withFields(items)}`; + } + if (isObject(value)) return `a JSON object ${withFields([value])}`; + if (Array.isArray(value)) return "a JSON array"; + return `a JSON ${value === null ? "null" : typeof value}`; +}; + +const isSuccess = (status: number | null): boolean => + status !== null && status >= 200 && status < 300; + +/** The first cited record replay answers as JSON, preferring a 2xx one; each body is parsed at most once. */ +const firstJson = ( + tool: EnvironmentManifestTool, + exchanges: ReadonlyMap, +): + | { readonly status: number | null; readonly value: JsonValue } + | undefined => { + for (const success of [true, false]) + for (const ordinal of tool.record_ordinals) { + const exchange = exchanges.get(ordinal); + if (exchange === undefined || isSuccess(exchange.status) !== success) + continue; + const body = replayedBody(exchange); + if (body.kind === "json") + return { status: exchange.status, value: body.value }; + } + return undefined; +}; + +/** What the tool returns as JSON, labelled with the status unless it succeeded, then the manifest's recorded statuses. */ +const describeResponses = ( + tool: EnvironmentManifestTool, + exchanges: ReadonlyMap, +): string => { + const recorded = + tool.statuses.length === 0 + ? "no recorded status" + : `recorded statuses ${describeList(tool.statuses, tool.statuses.length)}`; + const found = firstJson(tool, exchanges); + if (found === undefined) return recorded; + const returns = isSuccess(found.status) + ? "returns" + : found.status === null + ? "a record without a status returns" + : `status ${String(found.status)} returns`; + return `${returns} ${describeJson(found.value)}; ${recorded}`; }; /** Captured argument names longer than this are advertised under a short alias; the manifest keeps the name. */ @@ -208,9 +354,8 @@ export interface EnvironmentCatalogOptions { /** Admit complete discovery before binding; six bytes per UTF-16 unit covers JSON escaping and UTF-8 expansion. */ export const environmentToolDefinitions = ( - manifest: EnvironmentManifest, - evidenceId: string, - name: string = manifest.name, + source: EnvironmentSource, + name: string = source.manifest.name, { operation = ENVIRONMENT_SERVE_OPERATION, dataPlane = "none", @@ -254,15 +399,22 @@ export const environmentToolDefinitions = ( ? "answers from the served environment state where the state backs this tool, otherwise replays retained records" : "replays retained records"; const definitions: EnvironmentToolDefinition[] = []; - for (const tool of manifest.tools) { - if (!reserve(EMPTY_TOOL_BYTES) || !text(tool.name)) return refused(); + for (const tool of source.manifest.tools) { + const annotations = environmentToolAnnotations(tool); + // Only MCP discovery carries the hints; charging them per tool over-charges the card harmlessly. + if ( + !reserve(EMPTY_TOOL_BYTES + JSON.stringify({ annotations }).length) || + !text(tool.name) + ) + return refused(); const count = tool.params.length + tool.query_parameter_names.length + (WRITE_METHODS.has(tool.method) ? 1 : 0); // A lower bound rejects enormous argument lists before constructing their alias map. if (!reserve(count * EMPTY_PROPERTY_BYTES)) return refused(); - const description = `${REPLAY_CATALOG_INSTRUCTIONS} ${describeText(tool.method, "method")} ${describeText(tool.path_template, "path")}; ${answers} [${describeList(tool.record_ordinals, tool.record_ordinals.length)}]; evidence ${evidenceId}`; + // The response summary is bounded by its samples and charged with the whole description. + const description = `${REPLAY_CATALOG_INSTRUCTIONS} ${describeText(tool.method, "method")} ${describeText(tool.path_template, "path")}; ${describeResponses(tool, source.exchanges)}; ${answers} [${describeList(tool.record_ordinals, tool.record_ordinals.length)}]; evidence ${source.evidenceId}`; if (!text(description)) return refused(); const args = environmentToolArguments(tool); const properties = new Map(); @@ -289,6 +441,7 @@ export const environmentToolDefinitions = ( required, additionalProperties: false, }, + annotations, tool, }); } diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index fc073abde9..4dc7042b17 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed Frida session status, GDB/objdump command observations, Go build metadata, Rust embedded dependencies, binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot, inspect_il2cpp_metadata, inspect_model_artifact, inspect_gpu_code_objects or inspect_web_network_capture Evidence without repeating analysis. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized non-entity paths written by POST/PUT/PATCH (item-shaped and delete-only paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state, and every other read tool is a recorded view with a reason. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, and a scenario parent must have selected response bodies; its values keep the capture redaction markers, and a write whose URL it redacted ends the state of every type. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. A completed Evidence JSON file on this host uses {kind: "evidence-file", path: ""}: REA reads and authenticates it, then projects it like retained Evidence, so a large capture is bounded by the file reader rather than by inline-input accounting. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions. Occurrences accept exact logical_paths and occurrence_ids; nodes accept artifact_ids. Batch filters intersect (omitted means all available, [] means none); fields selects columns while retaining identity links. source_complete describes retained graph membership; source_coverage describes producer coverage, including unknown encrypted PCK directories. Page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_accesses/user_strings and carry the source selection and coverage. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Protobuf views use kind protobuf, collection bodies/frames/fields, with record_ordinals, body_pointers and fields; field pages also select frame_indices, field_ids, field_numbers or schema_names. Byte identity, offset basis and body status remain present. Text and schema_value are opt-in page columns. Model views use kind model-artifact and collection metadata (keys), tensors (names/types) or types (derived totals), with indices, fields and paging. Header SHA-256 scopes declarations; full-file identity remains unknown unless full hashing was selected. Arrays retain typed eight-element previews with explicit omissions. IL2CPP metadata views use kind il2cpp-metadata and collection sections/images/types/methods/fields/properties/events/parameters/string_literals/code_gen_modules/method_addresses/return_types/field_types/parameter_types. Select indices, fields and pages; named rows accept names, types also accept image_indices/namespaces/declaring_types, and methods/members accept declaring_types. Parameters and parameter_types accept method_indices; code_gen_modules accept image_indices; method_addresses accept addresses/states, preserving every folded alternative. Row index is scoped to the metadata digest; native pages add binary_source. Uncollected native tables have null counts and remain unavailable. Desktop directory views use kind desktop-application and collection files/omissions/candidates/components/next_probes, with fields and pagination; files and omissions accept exact paths or their stable IDs, and candidates accept families. Source coverage and stable row IDs remain available. Native inventory Evidence from list_names, list/search_procedures or list/search_strings uses view: {kind: "native-inventory", offset: 0, limit: 64, fields: ["value"]}. Rows keep source_index and address even with selected fields. Summary counts describe retained rows; provider discovery coverage remains unknown. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed Frida session status, GDB/objdump command observations, Go build metadata, Rust embedded dependencies, binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot, inspect_il2cpp_metadata, inspect_model_artifact, inspect_gpu_code_objects or inspect_web_network_capture Evidence without repeating analysis. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state, and every other read tool is a recorded view with a reason. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, and a scenario parent must have selected response bodies; its values keep the capture redaction markers, and a write whose URL it redacted ends the state of every type. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. A completed Evidence JSON file on this host uses {kind: "evidence-file", path: ""}: REA reads and authenticates it, then projects it like retained Evidence, so a large capture is bounded by the file reader rather than by inline-input accounting. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions. Occurrences accept exact logical_paths and occurrence_ids; nodes accept artifact_ids. Batch filters intersect (omitted means all available, [] means none); fields selects columns while retaining identity links. source_complete describes retained graph membership; source_coverage describes producer coverage, including unknown encrypted PCK directories. Page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_accesses/user_strings and carry the source selection and coverage. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Protobuf views use kind protobuf, collection bodies/frames/fields, with record_ordinals, body_pointers and fields; field pages also select frame_indices, field_ids, field_numbers or schema_names. Byte identity, offset basis and body status remain present. Text and schema_value are opt-in page columns. Model views use kind model-artifact and collection metadata (keys), tensors (names/types) or types (derived totals), with indices, fields and paging. Header SHA-256 scopes declarations; full-file identity remains unknown unless full hashing was selected. Arrays retain typed eight-element previews with explicit omissions. IL2CPP metadata views use kind il2cpp-metadata and collection sections/images/types/methods/fields/properties/events/parameters/string_literals/code_gen_modules/method_addresses/return_types/field_types/parameter_types. Select indices, fields and pages; named rows accept names, types also accept image_indices/namespaces/declaring_types, and methods/members accept declaring_types. Parameters and parameter_types accept method_indices; code_gen_modules accept image_indices; method_addresses accept addresses/states, preserving every folded alternative. Row index is scoped to the metadata digest; native pages add binary_source. Uncollected native tables have null counts and remain unavailable. Desktop directory views use kind desktop-application and collection files/omissions/candidates/components/next_probes, with fields and pagination; files and omissions accept exact paths or their stable IDs, and candidates accept families. Source coverage and stable row IDs remain available. Native inventory Evidence from list_names, list/search_procedures or list/search_strings uses view: {kind: "native-inventory", offset: 0, limit: 64, fields: ["value"]}. Rows keep source_index and address even with selected fields. Summary counts describe retained rows; provider discovery coverage remains unknown. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/domain/analysisView/EnvironmentStateAccumulator.ts b/src/domain/analysisView/EnvironmentStateAccumulator.ts index d1c3711fd0..6413a213c7 100644 --- a/src/domain/analysisView/EnvironmentStateAccumulator.ts +++ b/src/domain/analysisView/EnvironmentStateAccumulator.ts @@ -18,6 +18,7 @@ import { deeperThan, hasPrototypeKey, isStateWrite, + STATE_BODY_LIMIT_BYTES, type ParsedBody, type StateRecordFacts, type StateTimeline, @@ -50,8 +51,7 @@ type StateFields = Pick< | "limitations" >; -/** Bodies beyond this many UTF-8 bytes are not parsed into state; their views stay recorded. */ -export const STATE_BODY_LIMIT_BYTES = 256 * 1024; +export { STATE_BODY_LIMIT_BYTES }; /** * Levels the state's own nesting adds above a recorded value: Evidence, a `data/get` reply and a diff --git a/src/domain/analysisView/EnvironmentStateExtraction.ts b/src/domain/analysisView/EnvironmentStateExtraction.ts index 10ec4424f8..c3c2168011 100644 --- a/src/domain/analysisView/EnvironmentStateExtraction.ts +++ b/src/domain/analysisView/EnvironmentStateExtraction.ts @@ -4,24 +4,28 @@ import { compositeKey, } from "../unicodeCodePointOrder.js"; import { - IDENTITY_NAMES, + apiBaseLengths, at, collectionPathOf, + hasPrototypeKey, identifies, + IDENTITY_NAMES, + identityCandidates, isObject, isParam, isQueryPost, isStateWrite, jsonBytes, lastSegmentValue, + STATE_BODY_LIMIT_BYTES, listItems, - normalizedPath, locateItem, locateList, + normalizedPath, rootOf, + same, scalarText, segmentsOf, - apiBaseLengths, writeSpanCheck, type ExtractedState, type JsonObject, @@ -142,10 +146,7 @@ export class EnvironmentStateExtraction { plan.tool.name, plan.item ? checkItemPlan(gate, plan) : checkListPlan(gate, plan), ); - const documents = this.#documents(candidates); - for (const { tool } of candidates) - if (!this.#views.has(tool.name)) this.#recorded(tool, "not-entity"); - return this.#result(documents); + return this.#result(this.#documents(candidates)); } catch (cause: unknown) { if (cause instanceof StateBudgetExhausted) return null; throw cause; @@ -302,15 +303,26 @@ export class EnvironmentStateExtraction { this.#addPlan(candidate, pointer, recordings); } - /** List tools use the proved field, else the first identity name every non-empty recording's items carry. */ + /** + * List tools use the proved field, else the first identity candidate every non-empty recording's + * items share; a suffix-named field must also show distinct values across two items of one + * recording. Reads without a list or identity are left to documents. + */ #planList(candidate: Candidate): void { const { tool, type } = candidate; const recordings = this.#initRecordingsOf(candidate); if (recordings === undefined) return; const proved = this.#identity.get(type)?.field; + const fieldsOf = (items: readonly JsonObject[]): string[] => + proved === undefined + ? identityCandidates(items) + : identifies(items, proved) + ? [proved] + : []; let pointer: string | undefined; let names: readonly string[] | undefined; - let sawItems = false; + // A single listed item is distinct from nothing, so it shows no field distinct. + const shown = new Set(); for (const record of recordings) { const found = this.#withBody(record.ordinal, (body) => { if (body === undefined) return null; @@ -321,29 +333,32 @@ export class EnvironmentStateExtraction { located === undefined ? undefined : listItems(body, located); if (items === undefined) return undefined; pointer = located; - if (items.length > 0) sawItems = true; + // An empty list constrains no field. return items.length === 0 - ? (names ?? (proved === undefined ? IDENTITY_NAMES : [proved])) - : (proved === undefined ? IDENTITY_NAMES : [proved]).filter((name) => - identifies(items, name), - ); + ? true + : { fields: fieldsOf(items), several: items.length > 1 }; }); - if (found === null || found === undefined) { - this.#recorded(tool, found === null ? "body-unusable" : "not-entity"); + if (found === undefined) return; + if (found === null) { + this.#recorded(tool, "body-unusable"); return; } - names = (names ?? found).filter((name) => found.includes(name)); + if (found === true) continue; + // The first recording's candidates stand; later ones intersect through a set, so a wide + // item costs one pass per recording. + if (names === undefined) names = found.fields; + else { + const matching = new Set(found.fields); + names = names.filter((name) => matching.has(name)); + } + if (found.several) for (const name of found.fields) shown.add(name); } - const field = names?.[0]; // An identity name is only inferred from items that carry it; empty lists prove none. - if ( - field === undefined || - pointer === undefined || - (proved === undefined && !sawItems) - ) { - this.#recorded(tool, "no-identity"); - return; - } + const field = (names ?? (proved === undefined ? [] : [proved])).find( + (name) => + name === proved || IDENTITY_NAMES.includes(name) || shown.has(name), + ); + if (field === undefined || pointer === undefined) return; if (!this.#identity.has(type)) this.#identity.set(type, { field, basis: "field-name" }); this.#addPlan(candidate, pointer, recordings); @@ -500,39 +515,55 @@ export class EnvironmentStateExtraction { return this.#collections.get(JSON.stringify([type, collection])) ?? []; } - /** A written path whose read is not an entity holds one document value: its latest init recording. */ + /** A list tool without a plan or recorded reason holds one document value: its latest init recording, under one request line. */ #documents(candidates: readonly Candidate[]): StateDocument[] { - const written = new Set( - this.#tools - .filter( - (tool) => - ["POST", "PUT", "PATCH"].includes(tool.method) && - !isQueryPost(tool.method, tool.path_template), - ) - .map((tool) => tool.path_template), - ); const documents: StateDocument[] = []; for (const { tool, item } of candidates) { - const current = this.#views.get(tool.name); - const entityless = - current === undefined || - (current.shape === "recorded" && - (current.reason === "not-entity" || - current.reason === "no-identity")); - if (item || !entityless || !written.has(tool.path_template)) continue; - const latest = this.#latestPerLine( - this.#initRecordings(tool, tool.path_template), - ); + if (item || this.#views.has(tool.name)) continue; + const recordings = this.#initRecordings(tool, tool.path_template); + const latest = this.#latestPerLine(recordings); if (latest.length !== 1) { - if (latest.length > 1) this.#recorded(tool, "query-varies"); + const paths = new Set(latest.map((record) => record.path)); + this.#recorded( + tool, + latest.length === 0 + ? "no-init-recording" + : paths.size > 1 + ? "path-varies" + : "query-varies", + ); continue; } const record = latest[0]!; + // A value that changed across a write, even under another root, is not one initial state. + if ( + recordings.some( + (after, index) => + index > 0 && + this.#writeWithin(recordings[index - 1]!, after) && + this.#withBody(recordings[index - 1]!.ordinal, (before) => + this.#withBody(after.ordinal, (body) => !same(before, body)), + ), + ) + ) { + this.#recorded(tool, "fields-differ"); + continue; + } const value = this.#withBody(record.ordinal, (body) => body); if (value === undefined) { this.#recorded(tool, "body-unusable"); continue; } + // Parsing can lengthen a body that was admitted under the recorded-text limit: + // `1e3` is four bytes once rendered as `1000`. Serving refuses that rendering, + // so an oversized document stays the recorded replay instead of a failing read. + if ( + Buffer.byteLength(JSON.stringify(value), "utf8") > + STATE_BODY_LIMIT_BYTES + ) { + this.#recorded(tool, "rendered-oversize"); + continue; + } this.#retain( ENTRY_BYTES + jsonBytes(value) + textBytes(record.path.length), ); diff --git a/src/domain/analysisView/environmentStateMerge.ts b/src/domain/analysisView/environmentStateMerge.ts index f0bb102dfb..cca91fb5b7 100644 --- a/src/domain/analysisView/environmentStateMerge.ts +++ b/src/domain/analysisView/environmentStateMerge.ts @@ -64,8 +64,13 @@ export const mergeObservedEntity = (entity: ObservedEntity): StateEntity => { })) .filter((view) => Object.keys(view.fields).length > 0) .sort((left, right) => compare(left.tool, right.tool)), + // Cited in timeline order, whichever plan observed them first. record_ordinals: [ - ...new Set(entity.observations.map((observation) => observation.ordinal)), + ...new Set( + entity.observations + .toSorted((left, right) => left.position - right.position) + .map((observation) => observation.ordinal), + ), ], }; }; diff --git a/src/domain/analysisView/environmentStateRules.ts b/src/domain/analysisView/environmentStateRules.ts index e287f3f0c2..e62cbda51f 100644 --- a/src/domain/analysisView/environmentStateRules.ts +++ b/src/domain/analysisView/environmentStateRules.ts @@ -42,12 +42,20 @@ export type RecordedViewReason = | "query-body" | "no-init-recording" | "body-unusable" - | "not-entity" | "no-identity" + | "path-varies" | "query-varies" | "fields-differ" | "envelope-varies" - | "list-differs"; + | "list-differs" + | "rendered-oversize"; + +/** + * Recorded response text, and a document value rendered back to JSON, stay within this many UTF-8 + * bytes. A larger recorded body is not parsed; a parsed value whose JSON rendering exceeds it stays + * a recorded replay (`rendered-oversize`) rather than a document the server would have to refuse. + */ +export const STATE_BODY_LIMIT_BYTES = 256 * 1024; export interface StateType { readonly collection_template: string; @@ -314,29 +322,55 @@ export const hasPrototypeKey = (value: JsonValue): boolean => export const pointerTo = (name: string): string => `/${name.replaceAll("~", "~0").replaceAll("/", "~1")}`; -const pointerName = (pointer: string): string => - pointer.slice(1).replaceAll("~1", "/").replaceAll("~0", "~"); +/** The unescaped reference tokens of a pointer. */ +export const pointerTokens = (pointer: string): string[] => + pointer === "" + ? [] + : pointer + .slice(1) + .split("/") + .map((token) => token.replaceAll("~1", "/").replaceAll("~0", "~")); -/** The value at a pointer of depth at most one. */ +const memberOf = ( + body: JsonValue | undefined, + name: string, +): JsonValue | undefined => + isObject(body) && Object.hasOwn(body, name) ? body[name] : undefined; + +/** The element of a one-element array. */ +const soleElement = (value: JsonValue | undefined): JsonValue | undefined => + Array.isArray(value) && value.length === 1 ? value[0] : undefined; + +/** The value at a pointer: the root, one member, or element 0 of a one-element array at the root or in one member. */ export const at = (body: JsonValue, pointer: string): JsonValue | undefined => { - if (pointer === "") return body; - if (!isObject(body)) return undefined; - const name = pointerName(pointer); - return Object.hasOwn(body, name) ? body[name] : undefined; + const [first, second, ...rest] = pointerTokens(pointer); + if (first === undefined) return body; + if (second !== undefined) + return second === "0" && rest.length === 0 + ? soleElement(memberOf(body, first)) + : undefined; + return Array.isArray(body) + ? first === "0" + ? soleElement(body) + : undefined + : memberOf(body, first); }; -/** A copy of `body` with the value at a pointer of depth at most one replaced. */ +/** A copy of `body` with the value `at` reads at the pointer replaced; unchanged when it reads none. */ export const replaceAt = ( body: JsonValue, pointer: string, value: JsonValue, ): JsonValue => { - if (pointer === "" || !isObject(body)) return value; - const name = pointerName(pointer); + const [first, second] = pointerTokens(pointer); + if (first === undefined) return value; + if (at(body, pointer) === undefined) return body; + if (!isObject(body)) return [value]; + const replaced = second === undefined ? value : [value]; return Object.fromEntries( Object.entries(body).map(([key, item]) => [ key, - key === name ? value : item, + key === first ? replaced : item, ]), ); }; @@ -344,20 +378,36 @@ export const replaceAt = ( const matchingKeys = (object: JsonObject, value: string): string[] => Object.keys(object).filter((key) => scalarText(object[key]) === value); -/** The entity object an item response carries, at the root or in one wrapper property, with the keys repeating its path value. */ +/** The entity object an item response carries with the keys repeating its path value: at the root or in one wrapper member, else as the element of a one-element array there. */ export const locateItem = ( body: JsonValue, value: string, ): { readonly pointer: string; readonly keys: string[] } | undefined => { - if (!isObject(body)) return undefined; - const root = matchingKeys(body, value); - if (root.length > 0) return { pointer: "", keys: root }; - const wrapped = Object.entries(body).flatMap(([name, item]) => { - if (!isObject(item)) return []; - const keys = matchingKeys(item, value); - return keys.length > 0 ? [{ pointer: pointerTo(name), keys }] : []; - }); - return wrapped.length === 1 ? wrapped[0] : undefined; + const located = ( + object: JsonValue | undefined, + pointer: string, + named = false, + ) => { + const keys = isObject(object) + ? matchingKeys(object, value).filter( + (key) => !named || IDENTITY_NAMES.includes(key), + ) + : []; + return keys.length > 0 ? [{ pointer, keys }] : []; + }; + const members = isObject(body) ? Object.entries(body) : []; + // Objects take precedence; an array element needs an identity name, since a filtered list may + // answer with one element that repeats its parent's value. + const tiers = [ + located(body, ""), + members.flatMap(([name, item]) => located(item, pointerTo(name))), + located(soleElement(body), "/0", true), + members.flatMap(([name, item]) => + located(soleElement(item), `${pointerTo(name)}/0`, true), + ), + ]; + const chosen = tiers.find((tier) => tier.length > 0); + return chosen?.length === 1 ? chosen[0] : undefined; }; /** Whether every item carries `name` as a distinct string or number. */ @@ -365,13 +415,34 @@ export const identifies = ( items: readonly JsonObject[], name: string, ): boolean => { - const values = items.map((item) => scalarText(item[name])); - return ( - values.every((value) => value !== undefined) && - new Set(values).size === values.length - ); + // Stop at the first missing or repeated value, so a sparse list costs one read per field. + const seen = new Set(); + for (const item of items) { + const value = scalarText(item[name]); + if (value === undefined || seen.has(value)) return false; + seen.add(value); + } + return true; }; +/** Fields ending in one of these may identify listed items that no identity name identifies. */ +const IDENTITY_SUFFIX = /(?:Id|_id)$/u; + +/** Keys of the first item, in its order, that end like an identity field and identify every item. */ +const suffixIdentities = (items: readonly JsonObject[]): string[] => + Object.keys(items[0] ?? {}).filter( + (name) => + IDENTITY_SUFFIX.test(name) && + !IDENTITY_NAMES.includes(name) && + identifies(items, name), + ); + +/** Fields that identify every listed item, in precedence order: identity names, then suffix-named fields. */ +export const identityCandidates = (items: readonly JsonObject[]): string[] => [ + ...IDENTITY_NAMES.filter((name) => identifies(items, name)), + ...suffixIdentities(items), +]; + /** The array of entity objects a list response carries, at the root or in one property of an envelope that is not itself an entity. */ export const locateList = ( body: JsonValue, @@ -387,20 +458,24 @@ export const locateList = ( (entry): entry is [string, JsonObject[]] => Array.isArray(entry[1]) && entry[1].every(isObject), ); - const carrying = arrays.filter( - ([, items]) => - items.length > 0 && - (idField === undefined - ? IDENTITY_NAMES.some((name) => identifies(items, name)) - : identifies(items, idField)), - ); - const chosen = - carrying.length === 1 - ? carrying[0] - : carrying.length === 0 && arrays.length === 1 - ? arrays[0] - : undefined; - return chosen === undefined ? undefined : pointerTo(chosen[0]); + const carrying = (test: (items: JsonObject[]) => boolean) => + arrays.filter(([, items]) => items.length > 0 && test(items)); + // Identity names take precedence over suffix-named fields, which need two items to show + // distinct values; with neither, the one array of objects. + const tiers = + idField === undefined + ? [ + carrying((items) => + IDENTITY_NAMES.some((name) => identifies(items, name)), + ), + carrying( + (items) => items.length > 1 && suffixIdentities(items).length > 0, + ), + arrays, + ] + : [carrying((items) => identifies(items, idField)), arrays]; + const chosen = tiers.find((entries) => entries.length > 0); + return chosen?.length === 1 ? pointerTo(chosen[0]![0]) : undefined; }; export const listItems = ( diff --git a/src/domain/analysisView/environmentStateView.test.ts b/src/domain/analysisView/environmentStateView.test.ts index 736a6a3349..863e5fefbd 100644 --- a/src/domain/analysisView/environmentStateView.test.ts +++ b/src/domain/analysisView/environmentStateView.test.ts @@ -19,6 +19,12 @@ import { type AnalysisViewResult, } from "./analysisView.js"; import { EnvironmentStateAccumulator } from "./EnvironmentStateAccumulator.js"; +import { + at, + identityCandidates, + replaceAt, + type JsonObject, +} from "./environmentStateRules.js"; type State = Extract; const ALL_RULES = [ @@ -112,7 +118,23 @@ it("extracts the state the Grafana-shaped capture's reads observed", () => { shape: "recorded", reason: "no-init-recording", }); - expect(state.documents).toEqual([]); + // The one team member listed proves no suffix-named identity, so the list is a document of its path. + expect( + state.documents.map((document) => [document.tool, document.path]), + ).toEqual([ + [ + "get_datasources_health_by_uid", + "/api/datasources/uid/dg1jo96tern5sb/health", + ], + ["get_org", "/api/org"], + ["get_teams_members_by_id", "/api/teams/1/members"], + ["get_user", "/api/user"], + ]); + expect(viewOf(state, "get_plugins_settings_by_plugin_id")).toEqual({ + tool: "get_plugins_settings_by_plugin_id", + shape: "recorded", + reason: "path-varies", + }); expect(state.unknowns.map((unknown) => unknown.kind)).toEqual([ "authorization-model", "response-semantics", @@ -367,22 +389,317 @@ it("holds a written path's non-entity read as a document", () => { }); }); -it.each([ - ["/api/files/123", "PUT", "get_files_by_id"], - ["/api/settings", "DELETE", "get_settings"], -] as const)( - "leaves a deferred document path recorded (%s, %s)", - (path, method, tool) => { - const state = stateOf( - captureOf([ - ["GET", path, { text: "initial document" }], - [method, path, null], - ]), - ); - expect(state.documents).toEqual([]); - expect(viewOf(state, tool)).toMatchObject({ shape: "recorded" }); - }, -); +it("holds a read-only non-entity read as a document unless it varies by request line", () => { + const state = stateOf( + captureOf([ + ["GET", "/api/config", { theme: "dark", locale: "en-GB" }], + [ + "GET", + "/api/rates", + [ + { currency: "EUR", rate: 1.1 }, + { currency: "EUR", rate: 1.2 }, + ], + ], + ["GET", "/api/stats?range=day", { visits: 3 }], + ["GET", "/api/stats?range=week", { visits: 9 }], + ]), + ); + expect(state.documents).toEqual([ + { + tool: "get_config", + path: "/api/config", + value: { theme: "dark", locale: "en-GB" }, + record_ordinal: 0, + }, + { + tool: "get_rates", + path: "/api/rates", + value: [ + { currency: "EUR", rate: 1.1 }, + { currency: "EUR", rate: 1.2 }, + ], + record_ordinal: 1, + }, + ]); + expect(viewOf(state, "get_stats")).toEqual({ + tool: "get_stats", + shape: "recorded", + reason: "query-varies", + }); + expect(state.types).toEqual([]); +}); + +it("leaves a document recorded when rendering it exceeds the body limit", () => { + const small = stateOf( + captureOf([["GET", "/api/config", '{"values":[1e3]}']]), + ); + expect(viewOf(small, "get_config")).toMatchObject({ shape: "document" }); + expect(small.documents[0]?.value).toEqual({ values: [1000] }); + // 60,000 `1e3` tokens are 240,012 recorded bytes and 300,012 once rendered as `1000`. + const values = Array.from({ length: 60_000 }, () => "1e3").join(","); + const body = `{"values":[${values}]}`; + expect(Buffer.byteLength(body, "utf8")).toBe(240_012); + const state = stateOf(captureOf([["GET", "/api/config", body]])); + expect(state.documents).toEqual([]); + expect(viewOf(state, "get_config")).toEqual({ + tool: "get_config", + shape: "recorded", + reason: "rendered-oversize", + }); +}); + +it("infers a suffix-named identity only when no identity name identifies the listed items", () => { + const state = stateOf( + captureOf([ + [ + "GET", + "/api/memberships", + [ + { orgId: 7, userId: 1, role: "Admin" }, + { orgId: 8, userId: 1, role: "Viewer" }, + ], + ], + [ + "GET", + "/api/seats", + { + total: 2, + seats: [ + { parentId: 1, seat_id: "s-1" }, + { parentId: 1, seat_id: "s-2" }, + ], + }, + ], + [ + "GET", + "/api/projects", + [ + { projectId: "a", id: 2 }, + { projectId: "b", id: 3 }, + ], + ], + [ + "GET", + "/api/grants", + [ + { userId: 1, orgId: 7 }, + { userId: 2, orgId: 8 }, + ], + ], + ["GET", "/api/members", []], + ["GET", "/api/members", [{ memberId: "m-1" }, { memberId: "m-2" }]], + ["GET", "/api/comments", [{ postId: 9, authorId: 3, text: "hi" }]], + ]), + ); + expect( + state.types.map(({ collection_template, id_field, id_basis }) => [ + collection_template, + id_field, + id_basis, + ]), + ).toEqual([ + // Both identify the grants; the earlier key in item order wins. + ["/api/grants", "userId", "field-name"], + // The empty first recording constrains no field. + ["/api/members", "memberId", "field-name"], + ["/api/memberships", "orgId", "field-name"], + ["/api/projects", "id", "field-name"], + ["/api/seats", "seat_id", "field-name"], + ]); + expect(viewOf(state, "get_seats")).toMatchObject({ + shape: "list", + pointer: "/seats", + membership: "all", + counters: ["total"], + }); + expect(viewOf(state, "get_members")).toMatchObject({ shape: "list" }); + expect( + state.entities + .filter((entity) => entity.type === "/api/memberships") + .map((entity) => entity.id), + ).toEqual(["7", "8"]); + // One listed item shows no field distinct, so its foreign keys identify nothing. + expect(viewOf(state, "get_comments")).toMatchObject({ shape: "document" }); +}); + +it("chooses an envelope's array by identity name, then by a suffix-named field", () => { + const state = stateOf( + captureOf([ + [ + "GET", + "/api/board", + { + labels: [{ text: "urgent" }, { text: "later" }], + cards: [ + { cardId: 1, text: "Granite Ledger" }, + { cardId: 2, text: "Blue Heron" }, + ], + }, + ], + [ + "GET", + "/api/directory", + { + owners: [{ orgId: 1 }, { orgId: 2 }], + teams: [ + { id: 1, orgId: 1 }, + { id: 2, orgId: 1 }, + ], + }, + ], + ]), + ); + expect( + state.types.map(({ collection_template, id_field }) => [ + collection_template, + id_field, + ]), + ).toEqual([ + ["/api/board", "cardId"], + ["/api/directory", "id"], + ]); + expect(viewOf(state, "get_board")).toMatchObject({ + shape: "list", + pointer: "/cards", + }); + expect(viewOf(state, "get_directory")).toMatchObject({ + shape: "list", + pointer: "/teams", + }); +}); + +it("locates an item entity wrapped in a one-element array and merges it with the list view", () => { + const first = "6ad47b5c1023b500010873e3"; + const second = "6ad47b5c1023b500010873e4"; + const state = stateOf( + captureOf([ + [ + "GET", + "/ghost/api/admin/posts/", + { + posts: [ + { id: first, title: "Granite Ledger", status: "draft" }, + { id: second, title: "Blue Heron", status: "published" }, + ], + meta: { pagination: { page: 1, pages: 1 } }, + }, + ], + [ + "GET", + `/ghost/api/admin/posts/${first}/`, + { + posts: [ + { + id: first, + title: "Granite Ledger", + status: "draft", + html: "

Quarry count

", + }, + ], + }, + ], + ["GET", "/api/depots/5", [{ id: 5, name: "Saltmarsh Depot" }]], + ["GET", "/api/rosters/9", [{ id: 31, roster: 9, name: "Ada Finch" }]], + ]), + ); + expect(viewOf(state, "get_rosters_by_id")).toEqual({ + tool: "get_rosters_by_id", + shape: "recorded", + reason: "no-identity", + }); + const posts = "/ghost/api/admin/posts"; + expect(state.types).toContainEqual({ + collection_template: posts, + id_field: "id", + id_basis: "path-parameter", + init_before_ordinal: null, + volatile_fields: [], + }); + expect(state.views).toContainEqual( + expect.objectContaining({ + shape: "item", + type: posts, + pointer: "/posts/0", + fields: ["id", "title", "status", "html"], + checked_record_ordinals: [1], + }), + ); + expect(state.views).toContainEqual( + expect.objectContaining({ + shape: "list", + type: posts, + pointer: "/posts", + membership: "all", + }), + ); + expect(state.views).toContainEqual( + expect.objectContaining({ + shape: "item", + type: "/api/depots", + pointer: "/0", + }), + ); + expect(state.entities.filter((entity) => entity.type === posts)).toEqual([ + { + type: posts, + collection: posts, + id: first, + fields: { + id: first, + title: "Granite Ledger", + status: "draft", + html: "

Quarry count

", + }, + view_fields: [], + record_ordinals: [0, 1], + }, + expect.objectContaining({ id: second, record_ordinals: [0] }), + ]); +}); + +it("reads and replaces only the root, one member or the element of a one-element array", () => { + const body = { posts: [{ id: 1 }], meta: { posts: 1 } }; + expect(at(body, "/posts/0")).toEqual({ id: 1 }); + expect(replaceAt(body, "/posts/0", null)).toEqual({ + posts: [null], + meta: { posts: 1 }, + }); + expect(at([{ id: 1 }], "/0")).toEqual({ id: 1 }); + expect(replaceAt([{ id: 1 }], "/0", { id: 2 })).toEqual([{ id: 2 }]); + for (const pointer of ["/posts/1", "/meta/posts", "/posts/0/id"]) + expect(at(body, pointer)).toBeUndefined(); + expect(at({ posts: [{ id: 1 }, { id: 2 }] }, "/posts/0")).toBeUndefined(); + expect(replaceAt(body, "/meta/posts", null)).toBe(body); +}); + +it("leaves an item path's non-entity read recorded", () => { + const state = stateOf( + captureOf([ + ["GET", "/api/files/123", { text: "initial document" }], + ["PUT", "/api/files/123", null], + ]), + ); + expect(state.documents).toEqual([]); + expect(viewOf(state, "get_files_by_id")).toMatchObject({ shape: "recorded" }); +}); + +it("holds a deleted path's initial non-entity read as a document", () => { + const state = stateOf( + captureOf([ + ["GET", "/api/settings", { text: "initial document" }], + ["DELETE", "/api/settings", null], + ]), + ); + expect(state.documents).toEqual([ + { + tool: "get_settings", + path: "/api/settings", + value: { text: "initial document" }, + record_ordinal: 0, + }, + ]); +}); it("orders a scenario capture by request order and needs its response bodies", () => { const state = stateOf(parentOf(scenarioCaptureEvidence())); @@ -459,11 +776,11 @@ it("ends every type's state at a write whose URL was redacted, and says so", () ); }); -it("infers no identity for a list whose recordings held no items", () => { +it("infers no identity for a list whose recordings held no items and holds it as a document", () => { const state = stateOf(captureOf([["GET", "/api/widgets", []]])); expect(state.types).toEqual([]); expect(state.views).toEqual([ - { tool: "get_widgets", shape: "recorded", reason: "no-identity" }, + { tool: "get_widgets", shape: "document", checked_record_ordinals: [0] }, ]); }); @@ -514,8 +831,9 @@ it("does not take a later recording that is itself an entity as the list", () => ], ]), ); + // Without one list across its recordings, the read is held as a document, not entities. expect(state.views).toContainEqual( - expect.objectContaining({ tool: "get_items", shape: "recorded" }), + expect.objectContaining({ tool: "get_items", shape: "document" }), ); expect(state.entities).toEqual([]); }); @@ -763,3 +1081,57 @@ it("reads only item totals and next-page markers as signs of a partial list", () ["recorded", []], ]); }); + +it("reads a sparse list once per candidate field when inferring identity", () => { + const fields = 2_000; + const wide: JsonObject = Object.fromEntries( + Array.from({ length: fields }, (_, index) => [ + `f${String(index)}Id`, + index, + ]), + ); + let reads = 0; + const sparse = Array.from( + { length: 5_000 }, + () => + new Proxy( + {}, + { + get: (target, key, receiver) => { + reads += 1; + return Reflect.get(target, key, receiver) as unknown; + }, + }, + ), + ); + expect(identityCandidates([wide, ...sparse])).toEqual([]); + // Each identity name and suffix field stops at the first sparse item. + expect(reads).toBeLessThanOrEqual(fields + 20); +}); + +it("keeps a document out of the state when its value changed across a write under another root", () => { + const changed = stateOf( + captureOf([ + ["GET", "/api/me", { credits: 10, plan: "free" }], + ["POST", "/api/orders", { id: "o1" }], + ["GET", "/api/me", { credits: 9, plan: "free" }], + ]), + ); + expect(changed.documents).toEqual([]); + expect(changed.views).toContainEqual({ + tool: "get_me", + shape: "recorded", + reason: "fields-differ", + }); + // A value the write left unchanged is still one initial state. + const unchanged = stateOf( + captureOf([ + ["GET", "/api/me", { credits: 10, plan: "free" }], + ["POST", "/api/orders", { id: "o1" }], + ["GET", "/api/me", { credits: 10, plan: "free" }], + ]), + ); + expect(unchanged.documents).toMatchObject([ + { tool: "get_me", value: { credits: 10, plan: "free" }, record_ordinal: 2 }, + ]); +}); diff --git a/src/domain/analysisView/environmentStateView.ts b/src/domain/analysisView/environmentStateView.ts index a43a7e05c5..fd8273c7dc 100644 --- a/src/domain/analysisView/environmentStateView.ts +++ b/src/domain/analysisView/environmentStateView.ts @@ -32,7 +32,7 @@ const entityViewFields = { pointer: z .string() .describe( - "JSON pointer, of depth at most one, to the entity object or array within the response.", + "JSON pointer to the entity object or array within the response: the root or one member, or for an item the element of a one-element array there (/0 or //0).", ), fields: z .array(z.string()) @@ -76,12 +76,13 @@ const viewSchema = z.discriminatedUnion("shape", [ "query-body", "no-init-recording", "body-unusable", - "not-entity", "no-identity", + "path-varies", "query-varies", "fields-differ", "envelope-varies", "list-differs", + "rendered-oversize", ]), }), ]); @@ -103,7 +104,7 @@ export const environmentStateFields = { id_basis: z .enum(["path-parameter", "field-name"]) .describe( - "path-parameter: item responses repeat the requested path value in this field; field-name: an inferred identity name every listed item carries distinctly.", + "path-parameter: item responses repeat the requested path value in this field; field-name: inferred from listed items as the first identity name, else the first field ending in Id or _id shown distinct across at least two items, that every listed item carries distinctly.", ), init_before_ordinal: z .number() diff --git a/src/server/environment/environmentCapacity.test.ts b/src/server/environment/environmentCapacity.test.ts index 70e887e2cf..406c572887 100644 --- a/src/server/environment/environmentCapacity.test.ts +++ b/src/server/environment/environmentCapacity.test.ts @@ -48,17 +48,18 @@ it("charges environment identity, aggregate tools and escaped text before produc query_parameter_names: [], })); expect( - environmentToolDefinitions({ ...source.manifest, tools }, source.evidenceId) - .ok, + environmentToolDefinitions({ + ...source, + manifest: { ...source.manifest, tools }, + }).ok, ).toBe(false); expect( environmentToolDefinitions( - source.manifest, - source.evidenceId, + source, "x".repeat(ENVIRONMENT_CATALOG_LIMIT_BYTES), ).ok, ).toBe(false); - const normal = environmentToolDefinitions(source.manifest, source.evidenceId); + const normal = environmentToolDefinitions(source); expect(normal.ok).toBe(true); if (!normal.ok) throw normal.error; expect( @@ -74,12 +75,9 @@ it("reserves the state card only when the state data plane is served", () => { const source = environmentServeSource(); const manifest = { ...source.manifest, tools: [] }; const admits = (length: number, dataPlane: "none" | "state") => - environmentToolDefinitions( - manifest, - source.evidenceId, - "x".repeat(length), - { dataPlane }, - ).ok; + environmentToolDefinitions({ ...source, manifest }, "x".repeat(length), { + dataPlane, + }).ok; // With no tools, the name and the card are the whole reservation: six bytes per name unit plus quotes. const longest = (dataPlane: "none" | "state") => Math.floor( diff --git a/src/server/environment/serveEnvironment.test.ts b/src/server/environment/serveEnvironment.test.ts index 359b82e5e1..8a88b40bc3 100644 --- a/src/server/environment/serveEnvironment.test.ts +++ b/src/server/environment/serveEnvironment.test.ts @@ -38,10 +38,7 @@ const serveEnvironment = async ( if (!result.ok) throw result.error; return result.value; }; -const definitions = environmentToolDefinitions( - source.manifest, - source.evidenceId, -); +const definitions = environmentToolDefinitions(source); /** * The agentenv echo example's card, key for key. Only `name` and the tool @@ -400,6 +397,70 @@ describe("environment catalog limits", () => { }); }); +describe("environment tool annotations", () => { + it("advertises method hints with closed-world answers over MCP and keeps them off the card", async () => { + const folders = source.manifest.tools.find( + (tool) => tool.name === "post_folders", + ); + if (folders === undefined) throw new Error("Missing fixture folders tool"); + const search = { + ...folders, + name: "post_search", + path_template: "/api/search", + }; + const entry = await serveEnvironment({ + source: { + ...source, + manifest: { + ...source.manifest, + tools: [...source.manifest.tools, search], + }, + }, + host: "127.0.0.1", + port: 0, + }); + served.push(entry); + const client = new Client({ name: "tool-hints", version: "1" }); + const transport = new StreamableHTTPClientTransport(new URL(entry.url)); + await client.connect(transport); + try { + const { tools } = await client.listTools(); + const hints = new Map( + tools.map(({ name, annotations }) => [name, annotations]), + ); + const read = { readOnlyHint: true, openWorldHint: false }; + expect(hints.get("get_users_by_id")).toEqual({ + ...read, + idempotentHint: true, + }); + expect(hints.get("post_search")).toEqual({ + ...read, + idempotentHint: false, + }); + expect(hints.get("post_folders")).toEqual({ + readOnlyHint: false, + idempotentHint: false, + openWorldHint: false, + }); + expect(hints.get("put_folders_by_uid")).toEqual({ + readOnlyHint: false, + idempotentHint: true, + openWorldHint: false, + }); + expect(hints.get("delete_teams_by_id")).toEqual({ + readOnlyHint: false, + destructiveHint: true, + idempotentHint: true, + openWorldHint: false, + }); + expect(JSON.stringify(entry.card)).not.toContain("Hint"); + } finally { + await transport.terminateSession(); + await client.close(); + } + }); +}); + describe("serveEnvironment", () => { it("serves the card, lists one MCP tool per manifest tool and replays target content with provenance", async () => { const entry = await start(); diff --git a/src/server/environment/serveEnvironment.ts b/src/server/environment/serveEnvironment.ts index aa9a0220a0..ff171ab02f 100644 --- a/src/server/environment/serveEnvironment.ts +++ b/src/server/environment/serveEnvironment.ts @@ -115,6 +115,7 @@ export const createEnvironmentMcpServer = ( { description: definition.description, inputSchema: fromJsonSchema(definition.inputSchema), + annotations: definition.annotations, }, (args): CallToolResult => { const replayed = answer(definition.tool, args); diff --git a/tests/boundary/mcp/environmentServeMcp.test.ts b/tests/boundary/mcp/environmentServeMcp.test.ts index 8fa7b44b4e..cf4f06e508 100644 --- a/tests/boundary/mcp/environmentServeMcp.test.ts +++ b/tests/boundary/mcp/environmentServeMcp.test.ts @@ -20,9 +20,9 @@ import { const HTTP_BOUNDARY_TIMEOUT_MS = 90_000; const CARD_PATH = "/.well-known/agent-env.json"; -/** SHA-256 of the 11,131-byte card main serves for the fixture named "renamed"; pure replay must serve the same bytes. */ -const MAIN_REPLAY_CARD_SHA256 = - "f12f16011feb5f675b2bbf6a04a8b28a2ed8041074e4fe188cf74e7a66b15207"; +/** SHA-256 of the 12,818-byte card served for the fixture named "renamed", its descriptions naming each tool's recorded responses; pure replay must serve these bytes. */ +const REPLAY_CARD_SHA256 = + "7c65ab63f0c00ce79b570024c4611c225729677b44fc49bbde32b2d65f868961"; /** Serve the fixture pair as pure replay through the built CLI. */ const serveReplayOnly = async ( @@ -69,6 +69,15 @@ describe("rea environment-serve --replay-only", () => { expect(tools.map(({ name }) => name).sort()).toEqual( source.manifest.tools.map(({ name }) => name).sort(), ); + const users = tools.find(({ name }) => name === "get_users_by_id"); + expect(users?.description).toContain( + '; returns a JSON object with fields "id", "uid", "login", "email"; recorded statuses 200;', + ); + expect(users?.annotations).toEqual({ + readOnlyHint: true, + idempotentHint: true, + openWorldHint: false, + }); const exact = await client.callTool({ name: "get_users_by_id", arguments: { id: "2" }, @@ -122,7 +131,7 @@ describe("rea environment-serve --replay-only", () => { expect(card.status).toBe(200); const text = await card.text(); expect(createHash("sha256").update(text).digest("hex")).toBe( - MAIN_REPLAY_CARD_SHA256, + REPLAY_CARD_SHA256, ); const body: unknown = JSON.parse(text); expect(body).toMatchObject({