mirror of
https://github.com/morluto/rea
synced 2026-10-26 06:16:17 -04:00
feat(environment): render more read tools from the state and describe responses (#5352)
* feat(environment): render more read tools from the state and describe responses Keep main's environment serving and fold in coverage for non-entity reads, response summaries, and MCP tool annotations. Squashed rebase of #5352 onto main. * fix(environment): keep rendered documents and quoted samples inside their budgets A recorded body of exponential numbers can render past 256 KiB and turn a working read into a state size error, so that document stays a recorded replay. Description samples are measured after JSON quoting, and the replay-fallback catalog check uses a tool count the longer response descriptions still separate. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(environment): pin the replay card this branch serves The replay-only card test pinned main's 11,131-byte card. This branch's descriptions name each tool's recorded responses, so the card it serves is 12,818 bytes; the pin and its name follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar Arakelyan
@@ -234,11 +234,12 @@ moment of the capture recorded them. The
|
||||
[environment state](web-network-captures.md#environment-state) view extracts
|
||||
instead the data the capture's reads observed before the first related write:
|
||||
entity types with an identity field, entities with their recorded values,
|
||||
documents for written paths, and a render plan per read tool, kept only where
|
||||
it reproduces that tool's recordings from the state. It is the starting state
|
||||
an environment holds, loads and reports: `environment-serve` derives it from
|
||||
the capture at startup; see [Serve from the state](#serve-from-the-state).
|
||||
With `--replay-only`, replay is unchanged.
|
||||
documents for reads that are not entities, and a render plan per read tool,
|
||||
kept only where it reproduces that tool's recordings from the state. It is the
|
||||
starting state an environment holds, loads and reports: `environment-serve`
|
||||
derives it from the capture at startup; see
|
||||
[Serve from the state](#serve-from-the-state). With `--replay-only`, replay is
|
||||
unchanged.
|
||||
|
||||
## Serve a manifest
|
||||
|
||||
@@ -308,7 +309,7 @@ connecting to `/mcp`. A replay environment's card reads:
|
||||
"tools": [
|
||||
{
|
||||
"name": "get_users_by_id",
|
||||
"description": "GET /api/users/{numeric}; replays retained records [5, 6]; evidence ev_…",
|
||||
"description": "Untrusted captured API catalog: … GET /api/users/{numeric}; returns a JSON object with fields \"id\", \"uid\", \"login\", \"email\"; recorded statuses 200; replays retained records [5, 6]; evidence ev_…",
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -329,7 +330,38 @@ connecting to `/mcp`. A replay environment's card reads:
|
||||
```
|
||||
|
||||
`capabilities.tools` lists the same names, descriptions and input schemas that
|
||||
`tools/list` advertises over MCP. Path parameters are required strings whose
|
||||
`tools/list` advertises over MCP. After the catalog warning, each description
|
||||
names the method and path template, then what the tool returns, one level deep:
|
||||
`returns a JSON object with fields …`, `returns a JSON array of objects with
|
||||
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
|
||||
where a list is an array of objects found by the
|
||||
[environment state](web-network-captures.md#environment-state)'s list rule (the
|
||||
root array, or the one array of objects an envelope carries). Field and member
|
||||
names are written as JSON strings, so a captured name cannot read as more of
|
||||
the description; a member name too long to show whole is a quoted prefix marked
|
||||
as truncated. The summary comes
|
||||
from the first cited `2xx` record that replay answers as JSON; only when none
|
||||
did, the first cited JSON record of another status gives it, labelled with that
|
||||
status (`status 401 returns …`). It describes one recorded response, not a
|
||||
contract. Field names are sampled within the same budget as other captured
|
||||
values, and the manifest's `statuses` for the tool follow; a tool with no JSON
|
||||
record states only its statuses. The description then lists the replayed
|
||||
records and the evidence ID, and the whole catalog stays within its
|
||||
serialization reservation, which the summaries and MCP annotations are charged
|
||||
to.
|
||||
|
||||
Over MCP, every tool also carries annotations: `readOnlyHint` is true for the
|
||||
safe methods (`GET`, `HEAD`, `OPTIONS` and `TRACE`) and for query `POST`s (a
|
||||
path segment such as `search` or `query`); `destructiveHint` is true for `DELETE` and left unset for other
|
||||
writes, so the MCP default (destructive) applies to them, and read-only tools
|
||||
omit it because clients ignore it there; `idempotentHint` is true for the safe
|
||||
methods, `PUT` and `DELETE`; and `openWorldHint` is false for every tool,
|
||||
because answers come from the capture rather than the captured application. The
|
||||
hints describe the captured method; replay itself changes nothing. The card
|
||||
keeps the `name`, `description` and `inputSchema` of the echo example's tool
|
||||
shape and carries no annotations.
|
||||
|
||||
Path parameters are required strings whose
|
||||
description names the segment index, the templating rule and the observed
|
||||
values (sampled within the list budget above; the manifest keeps them all);
|
||||
query parameter names are optional strings; write methods accept an
|
||||
@@ -399,7 +431,9 @@ naming the entity, collection or document path, or `{"source": "recording",
|
||||
"reason": …}` with the view's reason, `no-view` or `unrecorded-request`. A
|
||||
state answer carries the base record's status, content type and `replay`
|
||||
provenance with `truncated: false`; one whose JSON would exceed 256 KiB is
|
||||
refused as a `resource_constraint` tool error rather than cut. Serving is
|
||||
refused as a `resource_constraint` tool error rather than cut. Extraction does
|
||||
not state-back a document whose rendered JSON exceeds 256 KiB; that read stays
|
||||
recorded (`rendered-oversize`) and replays the captured body. Serving is
|
||||
read-only: `POST`, `PUT`, `PATCH` and `DELETE` tools, other than query `POST`s,
|
||||
are not served, so nothing a call does changes the state. Query `POST`s stay
|
||||
served and answer from their recordings with the reason `query-body`.
|
||||
|
||||
@@ -332,7 +332,7 @@ See [environment manifests](web-network-captures.md#environment-manifest).
|
||||
|
||||
`inspect_analysis_view` also accepts `environment-state` for the same parents,
|
||||
extracting the initial state the recorded reads observed: entity types with an
|
||||
identity field, entities, documents for written non-entity paths, and a render
|
||||
identity field, entities, documents for non-entity reads, and a render
|
||||
plan per read tool that reproduces its recordings from the state or names why it
|
||||
cannot. Types and identity are inferences, so the Evidence is always inferred.
|
||||
See [environment state](web-network-captures.md#environment-state).
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# Implementation plan
|
||||
|
||||
1. In `environmentStateRules.ts`, parse pointers into tokens and let `at` and
|
||||
`replaceAt` read and replace element 0 of a one-element array at the root
|
||||
or in one member; extend `locateItem` with those forms after the object
|
||||
forms, requiring an identity name there; add `identityCandidates`, the
|
||||
identity names then suffix-named fields, and let `locateList` prefer arrays
|
||||
identified by name, then arrays of two or more items identified by suffix.
|
||||
2. In `EnvironmentStateExtraction.ts`, plan lists with the shared candidates,
|
||||
treat an empty recording as unconstraining, accept a suffix-named field only
|
||||
once a recording shows it distinct across two or more items, leave reads
|
||||
without a list or identity to documents, drop the written-path requirement
|
||||
and the `not-entity` fallback, record `path-varies` apart from
|
||||
`query-varies`, and keep each entity's earliest sighting;
|
||||
`environmentStateMerge.ts` cites records in timeline order. Remove
|
||||
`not-entity` from the reason type and result schema, add `path-varies`, and
|
||||
describe the new pointer forms and `field-name` basis.
|
||||
3. Move the classification of retained bodies (none, base64, cut, JSON, text)
|
||||
into `replayedBody` in `environmentSource.ts`, which owns recorded
|
||||
exchanges; replay renders from it and tool descriptions read its JSON case.
|
||||
4. Make `environmentToolDefinitions` take the `EnvironmentSource`; describe the
|
||||
first `2xx` JSON record, else the first JSON record labelled with its status,
|
||||
one level deep with the state's list rule and the existing sample budget;
|
||||
list the manifest tool's statuses; and charge the whole description and the
|
||||
tool's MCP annotations to the catalog reservation. Update the serve and
|
||||
bundle callers.
|
||||
5. Let the tool definitions own the MCP annotations derived from the method and
|
||||
the query-`POST` rule; `serveEnvironment.ts` registers them and the card
|
||||
leaves them out.
|
||||
6. Cover domain, services, adapters and the compiled MCP boundary; document the
|
||||
rules in the capture guide, descriptions and hints in the agent environments
|
||||
guide, and the contract text.
|
||||
@@ -0,0 +1,99 @@
|
||||
# Wider environment state coverage and tool descriptions
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
The `environment-state` view backs more read tools with state, and a served
|
||||
environment tells agents more about each tool. Read-only paths whose responses
|
||||
are not entities become documents, listed items may be identified by an
|
||||
`…Id` or `…_id` field, an item wrapped in a one-element array is located
|
||||
inside it, and reads that vary by path get their own recorded reason. Each tool description says what the tool returns and which statuses
|
||||
were recorded, and every MCP tool carries method hints. This increment adds no
|
||||
tool, view kind, provider, network activity or write effect.
|
||||
|
||||
## Requirements
|
||||
|
||||
- A `GET` list tool (its template does not end in a parameter) that yields no
|
||||
item or list plan holds its latest initial value as a `document` when its
|
||||
initial reads have one request line, whether or not the capture writes its
|
||||
path, unless its value changed between two of those reads across any
|
||||
recorded write (even under another root), which leaves it `fields-differ`. A
|
||||
value whose rendered JSON exceeds 256 KiB stays `rendered-oversize` and is
|
||||
replayed from the recorded body, which itself fit. More request lines give
|
||||
`path-varies` when they differ by path (a template with a parameter before
|
||||
its end) and `query-varies` when they share one path. With every such read
|
||||
now a document, `path-varies`, `query-varies`, `body-unusable` or
|
||||
`rendered-oversize`, the unreachable `not-entity` reason leaves the result
|
||||
schema, and `no-identity` names item tools only. A document holds the value
|
||||
of its one `path`.
|
||||
- When no path parameter proves an identity and no identity name identifies
|
||||
the listed items, the `field-name` identity is the first key of the first
|
||||
listed item, in its order, whose name ends in `Id` or `_id`, that every item
|
||||
carries as a distinct string or number, and that at least one initial
|
||||
recording shows distinct across two or more items. The last condition goes
|
||||
beyond "every item carries it distinctly": one item satisfies that
|
||||
vacuously, and a lone item's suffix-named fields are often foreign keys (a
|
||||
reaction keyed by its user, a comment by its post), so a list that only ever
|
||||
held one item stays a document. Empty recordings constrain no field.
|
||||
Identity names keep precedence, also when choosing among an envelope's
|
||||
arrays, where a suffix-named field identifies only an array of two or more
|
||||
items.
|
||||
- An item response that is a root array of exactly one object, or an object
|
||||
with one member holding such an array, locates its entity at `/0` or
|
||||
`/<member>/0` when one of the identity names in that object repeats the path
|
||||
value. Objects at the root or in one member keep precedence. `at` and
|
||||
`replaceAt` read and replace exactly these forms besides the root and one
|
||||
member, so the gate and renderers run unchanged.
|
||||
- An entity's state order is its earliest sighting on the timeline and it cites
|
||||
its records in timeline order, whichever plan observed it first.
|
||||
- Each tool description reads `METHOD template; <returns>; recorded statuses
|
||||
…; replays retained records […]; evidence ev_…` after the untrusted-catalog
|
||||
warning. `<returns>` describes one level deep the first cited `2xx` record
|
||||
replay answers as JSON: an object's fields, an array of objects' fields, or
|
||||
an envelope whose member lists objects, where a list is an array of objects
|
||||
found by the state's list rule; an empty field set reads `with no fields`.
|
||||
Only when no `2xx` record replays as JSON does the first cited JSON record of
|
||||
another status give it, labelled `status <code> returns …`; with no JSON
|
||||
record it is omitted. The statuses are the manifest tool's `statuses`, the
|
||||
one owner of that set (`no recorded status` when it is empty). Field names
|
||||
and statuses are sampled within the description sample budget and charged
|
||||
with the whole description to the catalog reservation before it is retained.
|
||||
One owner in the environment source classifies each retained body as none,
|
||||
base64, cut, JSON or text; replay answers from that classification and
|
||||
descriptions read only its JSON case.
|
||||
- MCP tools carry `readOnlyHint` (true for the safe methods `GET`, `HEAD`,
|
||||
`OPTIONS` and `TRACE`, and for query `POST`s),
|
||||
`destructiveHint: true` for `DELETE` (unset for other writes, so the MCP
|
||||
default applies, and for reads, where clients ignore it), `idempotentHint`
|
||||
(true for the safe methods, `PUT` and `DELETE`) and `openWorldHint: false`. The
|
||||
tool definitions own the hints and charge them to the catalog reservation;
|
||||
the environment card keeps its `name`, `description`, `inputSchema` tool
|
||||
shape.
|
||||
|
||||
## Acceptance and limits
|
||||
|
||||
Synthetic captures cover read-only documents, `path-varies` and
|
||||
`query-varies`, suffix-named identities with their key order, their
|
||||
two-item proof, empty first recordings and their precedence (including the
|
||||
choice between an envelope's arrays), a Ghost-like `{"posts":[...]}` item under
|
||||
a trailing-slash path merging with its list view, a root one-element array, a
|
||||
one-element filtered list that stays unidentified, and the pointer forms. The
|
||||
Grafana-shaped fixture covers documents for its read-only paths (its one-member
|
||||
team list among them) and object, array and enveloped-list descriptions;
|
||||
synthetic sources cover the `2xx` preference over an earlier error body, the
|
||||
first of several JSON records, the labelled fallback, the manifest's statuses,
|
||||
empty, scalar and fieldless bodies, a truncated member name, a name whose
|
||||
JSON quoting exceeds the sample, a recorded body under 256 KiB whose rendered
|
||||
JSON exceeds it, non-JSON bodies and the catalog budget; the in-process and
|
||||
compiled MCP servers advertise the
|
||||
hints. A filtered list may still answer with one element whose identity name
|
||||
repeats the requested value; list filters, pagination, lossless big integers
|
||||
and cross-endpoint entity linking remain out of scope. Documents inline whole
|
||||
recorded values and enlarge the state result; agent tokens, CPU time and peak
|
||||
memory of the state and description build were not measured.
|
||||
|
||||
## Deferred issue criteria
|
||||
|
||||
Serving from the state, the AgentEnv data plane and write effects remain later
|
||||
increments of #4933. This PR must reference rather than close the issue.
|
||||
@@ -0,0 +1,16 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the coverage and description contract and its limits.
|
||||
- [x] Extend pointers, item location and identity candidates.
|
||||
- [x] Require two distinct items before a suffix-named field identifies a list.
|
||||
- [x] Hold read-only non-entity reads as documents, retire `not-entity` and
|
||||
separate `path-varies` from `query-varies`.
|
||||
- [x] Order entities by earliest sighting and cite records in timeline order.
|
||||
- [x] Classify retained bodies in one owner and describe tool responses,
|
||||
preferring `2xx` records, with the manifest's statuses.
|
||||
- [x] Advertise MCP tool annotations, charged to the catalog, without changing
|
||||
the card.
|
||||
- [x] Add domain, services, adapters and compiled-boundary coverage.
|
||||
- [x] Document the rules, descriptions and hints; update the contract text.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates and the
|
||||
real-capture measurement.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Verification
|
||||
|
||||
Verified base: 7a668b73f4783b42eb0e8ce1eeff079f86d53e22 (the environment-state
|
||||
view branch).
|
||||
|
||||
Executed on the implementation worktree with its existing locked installation
|
||||
under Node.js v22.23.1 (the repository pins 24.18.0, which was not available);
|
||||
every check below ran as written in this pass, after the last source change.
|
||||
|
||||
- `npm run format` then `npm run format:check`: passed.
|
||||
- `npm run check:fast`: typecheck and lint passed (3959 files, no errors). The
|
||||
touched files carry two lint warnings, both present on the base: `max-params`
|
||||
in `environmentReplay.ts` and an unused `_` in `environmentCapacity.test.ts`.
|
||||
- `npx knip`: passed (exit 0).
|
||||
- `npm run jscpd`: 3.40% duplicated lines over 3348 files, under the 3.5%
|
||||
gate. The three clones that touch changed test files pair the same blocks as
|
||||
on the base; none involves code this change adds.
|
||||
- `npm run scan:todos`: no markers.
|
||||
- `npm run docs:check`: the built documentation checks and the README
|
||||
consistency conformance test (5 tests) passed.
|
||||
- `npx vitest run --project domain --project services`: 341 files, 5859 tests
|
||||
passed, 5 pre-existing skips.
|
||||
- `npm run build:cached`: passed.
|
||||
- `npm run test:focused --` the environment serve adapters and capacity tests,
|
||||
the environment serve MCP boundary, the environment serve and export CLI
|
||||
boundaries, the environment serve shutdown process boundary, the
|
||||
analysis-view MCP and CLI boundaries, the compact input-schema contract and
|
||||
MCP boundary, the tool-schema validity boundary, the analysis-detail and
|
||||
advertised-input-example contracts and the qlog acceptance test: 14 files,
|
||||
113 tests passed, 1 pre-existing skip.
|
||||
|
||||
Real captures (outside the test suite): the built view ran through the
|
||||
application service on the same 44 recorded captures (39 HAR, 5 browser
|
||||
scenario) with all three template rules, in 7 to 50 ms each. 42 produced a
|
||||
state, and the same two browser-scenario captures are refused with
|
||||
`resource_constraint` as on the base. The base column is the brief's baseline,
|
||||
which the same script reproduced on the base build in an earlier pass; this
|
||||
pass did not rebuild the base.
|
||||
|
||||
| Measure | Base | This change |
|
||||
| ---------------------------------- | ----- | ----------- |
|
||||
| Types | 568 | 576 |
|
||||
| Entities | 2,213 | 2,246 |
|
||||
| Documents | 2 | 299 |
|
||||
| Item views | 211 | 213 |
|
||||
| List views | 497 | 504 |
|
||||
| Document views | 2 | 299 |
|
||||
| State-backed read tools (of 1,258) | 710 | 1,016 |
|
||||
| Recorded: `not-entity` | 306 | (retired) |
|
||||
| Recorded: `no-identity` | 129 | 32 |
|
||||
| Recorded: `path-varies` | (new) | 32 |
|
||||
| Recorded: `query-varies` | 5 | 69 |
|
||||
| Recorded: `body-unusable` | 32 | 32 |
|
||||
| Recorded: `list-differs` | 31 | 31 |
|
||||
| Recorded: `no-init-recording` | 26 | 26 |
|
||||
| Recorded: `query-body` | 12 | 12 |
|
||||
| Recorded: `fields-differ` | 3 | 4 |
|
||||
| Recorded: `envelope-varies` | 4 | 4 |
|
||||
|
||||
Seven list types gained a suffix-named identity (`permissionId`, `product_id`,
|
||||
`ItemId`, `user_id`, `channel_id`, `spl_id` and `eventId`). Seven more had one
|
||||
before the two-item proof and lost it, because every initial response listed a
|
||||
single item: an Airbyte organization, a Mattermost reaction and team
|
||||
membership, and a Redpanda broker and schema reference became documents; a
|
||||
NOAA station read is `query-varies` and a Redpanda consumer read
|
||||
`path-varies`. Two Ghost item tools render from `/members/0` and `/tags/0`. A
|
||||
third Ghost item tool is located at `/posts/0` but stays `fields-differ`: its
|
||||
recordings differ by an `include` query.
|
||||
|
||||
State result size, as the JSON length of each Evidence summed over the 42
|
||||
states: 2,677 KiB on the base and 3,196 KiB now (+19%), measured by the same
|
||||
script. The largest growth per capture is Metabase (133 to 236 KiB), Directus
|
||||
(21 to 93), Redpanda Console (26 to 68) and Vault (9 to 43). Documents take
|
||||
492,516 of the 3,274,243 UTF-8 bytes of the results now; the base held two.
|
||||
Agent tokens, CPU time and peak memory of the state view were not measured.
|
||||
|
||||
Tool descriptions on the 44 captures' manifests: all 44 catalogs were admitted
|
||||
within their reservation. 1,336 of 1,400 tools state what they return: 1,333
|
||||
from a `2xx` record and three labelled with another status (Miniflux and
|
||||
NocoDB 404, Shlink 501), because none of their records succeeded as JSON. The
|
||||
summaries add 261,502 bytes to a 1,185,893-byte catalog of names, descriptions
|
||||
and input schemas (median 136, 95th percentile 554, maximum 1,133 bytes per
|
||||
tool); five reached the field sample bound (one Keycloak, one Lychee and three
|
||||
Sentry tools with 46 to 105 fields). The MCP annotations add 113,182 bytes to
|
||||
`tools/list` and none to the card. Building a catalog took a median 1.6 ms and
|
||||
at most 7.6 ms (Metabase), after the source was loaded. Agent tokens, CPU time
|
||||
and peak memory of the description build were not measured.
|
||||
|
||||
After merging the environment state view's review fixes (its branch at
|
||||
6ad63bcc, which also brings main's evidence-file sources), every gate ran
|
||||
again on the merged tree: `npm run check:fast`, `npm run format:check`,
|
||||
`npx knip`, `npm run scan:todos` and `npm run docs:check` passed; `npm run
|
||||
jscpd`: 3.39% duplicated lines over 3362 files; `npx vitest run --project
|
||||
domain --project services`: 344 files, 5914 tests passed, 5 pre-existing
|
||||
skips; `npm run build:cached`, then `npm run test:focused --` the serve and
|
||||
tool-definition adapters, the serve MCP and CLI boundaries, the export CLI
|
||||
boundary, the analysis-view MCP and CLI boundaries, the compact input-schema
|
||||
budget and the analysis-detail and advertised-input-example contracts: 10
|
||||
files, 98 tests passed.
|
||||
|
||||
The real-capture comparison was repeated on the merged code, with the base
|
||||
measured on the merged state view without this change: all 44 captures now
|
||||
produce a state, and 1,011 of 1,277 read tools are state-backed against 625
|
||||
on the base (216 item, 416 list and 379 document views against 214, 409 and
|
||||
2). Recorded reasons: `no-identity` 216 to 32 and `not-entity` 310 to none;
|
||||
`query-varies` 5 to 76 and `path-varies` 36 are document reads under more
|
||||
than one query or path; `list-differs` 35, `body-unusable` 33,
|
||||
`no-init-recording` 28, `query-body` 12, `envelope-varies` 4 are unchanged,
|
||||
and `fields-differ` is 9 to 10.
|
||||
|
||||
Not covered here: serving from the state, the AgentEnv data plane and write
|
||||
effects, which are later increments.
|
||||
|
||||
## After the squash onto main
|
||||
|
||||
The branch was squashed onto main once the state-serve branch merged (166fb082). Against a merge of the previous head
|
||||
into that main, the squash keeps both sides of every conflict and drops nothing of this branch. Two tests then failed
|
||||
on it and were refitted with the oversized-document fix in 2a926a0a: the bundle test of a state catalog over the
|
||||
discovery budget uses 300 read tools, inside the window where the state's catalog is refused and replay's, with its
|
||||
response summaries, still fits (280–320 by a probe), and the rea-state validation test names the pushed document by
|
||||
the count of documents the state holds. The replay-only card test pinned main's 11,131-byte card; it now pins this
|
||||
branch's 12,818-byte card, whose descriptions name each tool's recorded responses. The branch then merged main at
|
||||
d7ca93cc without conflicts.
|
||||
@@ -522,9 +522,9 @@ Pass the same parent to `inspect_analysis_view` with
|
||||
`view: {"kind":"environment-state"}` to extract the initial state the
|
||||
capture's reads observed, and a render plan per read tool that reproduces those
|
||||
reads from it. Where the environment manifest names an application's tools, the
|
||||
state holds its data: entities by type and identity, plus documents for written
|
||||
paths whose reads are not entities. Use the `template_rules` of the manifest the
|
||||
state accompanies so tool names match; the view groups records the same way.
|
||||
state holds its data: entities by type and identity, plus documents for reads
|
||||
that are not entities. Use the `template_rules` of the manifest the state
|
||||
accompanies so tool names match; the view groups records the same way.
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -560,13 +560,27 @@ refused for them, as the manifest is not.
|
||||
|
||||
Each `GET` tool whose template ends in a parameter is an item tool, and its
|
||||
type's `collection_template` is the template without that parameter; any other
|
||||
`GET` tool is a list tool of its own template. An item tool proves its type's
|
||||
`GET` tool is a list tool of its own template. Empty segments are ignored, so a
|
||||
trailing slash names the same collection. An item tool proves its type's
|
||||
`id_field` when every initial response carries the requested path value in the
|
||||
same field, at the root or in one wrapper member (`id_basis: "path-parameter"`).
|
||||
Otherwise the field is the first of `id`, `uid`, `uuid`, `_id`, `ID`, `UID`,
|
||||
`Id`, `guid`, `key`, `slug` and `name` that every listed item carries as a
|
||||
distinct string or number (`id_basis: "field-name"`). A list is the response
|
||||
array, or the one array of objects in an envelope that is not itself an entity.
|
||||
same field of one entity object (`id_basis: "path-parameter"`). The object is
|
||||
the response itself or one wrapper member; failing both, it is the only element
|
||||
of a one-element array at the root (pointer `/0`) or in one member
|
||||
(`/<member>/0`), as in `{"posts":[{...}]}` or `{"data":[{...}]}`, provided one
|
||||
of the identity names below repeats the value, since a filtered list may also
|
||||
answer with one element that repeats its parent's value. Otherwise the field is
|
||||
the first of `id`, `uid`, `uuid`, `_id`, `ID`, `UID`, `Id`, `guid`, `key`,
|
||||
`slug` and `name` that every listed item carries as a distinct string or
|
||||
number; when none does, it is the first key of the first listed item, in its
|
||||
order, whose name ends in `Id` or `_id` (such as `orgId` or `user_id`), that
|
||||
every item carries as a distinct string or number, and that at least one
|
||||
initial response shows distinct across two or more items
|
||||
(`id_basis: "field-name"`). A single listed item is distinct from nothing, and
|
||||
its suffix-named fields are often foreign keys, so a list that only ever held
|
||||
one item is left to a document. Empty initial lists constrain no field. A list
|
||||
is the response array, or the one array of objects in an envelope that is not
|
||||
itself an entity, preferring an array whose items an identity name identifies,
|
||||
then an array of two or more items a suffix-named field identifies.
|
||||
|
||||
`entities` merge every initial observation of one identity within one
|
||||
collection path: an item view's latest values win over a list view's, a view
|
||||
@@ -583,9 +597,9 @@ Every entity cites its `record_ordinals`.
|
||||
|
||||
| Shape | Plan |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity object at `pointer` (the root or one member), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
|
||||
| `item` | The entity object at `pointer` (the root, one member, or `/0` or `/<member>/0`), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
|
||||
| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query, no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed |
|
||||
| `document` | The path's latest initial value, held in `documents` |
|
||||
| `document` | The latest initial value of a list tool whose responses hold no identified list, when its initial reads share one request line; held in `documents` with that `path`, whether or not the capture writes it, unless its value changed across any recorded write between those reads (`fields-differ`) |
|
||||
| `recorded` | A `reason` the tool replays its recordings instead |
|
||||
|
||||
An item or list plan is kept only when it reproduces the latest initial
|
||||
@@ -594,11 +608,13 @@ regard to member order; a volatile field compares its recorded value. Each
|
||||
plan lists the `checked_record_ordinals`. Otherwise the tool is a recorded view
|
||||
with reason `envelope-varies`, `fields-differ` or `list-differs`; the other
|
||||
reasons are `query-body` (a query `POST`), `no-init-recording`,
|
||||
`body-unusable`, `not-entity`, `no-identity` and `query-varies` (a document
|
||||
read under several queries). Documents are held only for list-shaped paths
|
||||
that a `POST`, `PUT` or `PATCH` tool writes: a non-entity read of an item path
|
||||
such as `/api/files/{id}`, or of a path that is only deleted, stays recorded
|
||||
with `no-identity` or `not-entity`. A body is usable when its content type is
|
||||
`body-unusable`, `no-identity` (an item read without a proved identity),
|
||||
`path-varies` (another read under more than one path, as a template with a
|
||||
parameter before its end), `query-varies` (another read of one path under
|
||||
more than one query) and `rendered-oversize` (a document value whose JSON
|
||||
renders to more than 256 KiB). A document holds the value of its one `path`, so for such
|
||||
a template it says nothing about the template's other paths. A body is usable
|
||||
when its content type is
|
||||
JSON-like, it is UTF-8 text of at most 256 KiB, every number survives native
|
||||
conversion exactly, and no member is named `__proto__`.
|
||||
|
||||
@@ -610,7 +626,9 @@ that the state establishes no authorization model, no way the application
|
||||
computes responses or derived values, and no effects of writes. The state
|
||||
accumulator charges the manifest's admission and grouping, every record line,
|
||||
each body as temporary bytes before it is decoded and parsed, and every value
|
||||
it keeps, within the same 96 MiB budget. The view performs no network activity
|
||||
it keeps, within the same 96 MiB budget. Documents hold whole recorded response
|
||||
values whose recorded text and rendered JSON are each at most 256 KiB, so a
|
||||
capture with many read-only paths yields a correspondingly larger state result. The view performs no network activity
|
||||
and serves nothing; `rea environment-serve` derives it from the capture at
|
||||
startup, serves read tools from it and loads replacement states through the
|
||||
AgentEnv data plane, as
|
||||
|
||||
@@ -387,10 +387,11 @@ describe("planEnvironmentBundle without a state", () => {
|
||||
|
||||
it("serves and predicts replay when the state's catalog exceeds the discovery budget that replay's fits", () => {
|
||||
// Read tools only, so leaving out write tools saves nothing while each state description is longer.
|
||||
// 300 sits between the two catalogs on this fixture: replay still fits and the state catalog does not.
|
||||
const reads = servedSource.manifest.tools.filter(
|
||||
({ method }) => method === "GET",
|
||||
);
|
||||
const tools = Array.from({ length: 350 }, (_, index) => {
|
||||
const tools = Array.from({ length: 300 }, (_, index) => {
|
||||
const tool = reads[index % reads.length]!;
|
||||
return index < reads.length
|
||||
? tool
|
||||
@@ -416,7 +417,7 @@ describe("planEnvironmentBundle without a state", () => {
|
||||
served: false,
|
||||
code: "resource_constraint",
|
||||
});
|
||||
expect(planned.value.toolCount).toBe(350);
|
||||
expect(planned.value.toolCount).toBe(300);
|
||||
expect(planned.value.check).toEqual({
|
||||
tool: "get_annotations",
|
||||
recordOrdinal: 25,
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
import { isJsonLike } from "../../domain/analysisView/EnvironmentManifestAccumulator.js";
|
||||
import { mediaTypeEssence } from "../../domain/analysisView/webNetworkRouteTemplates.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { parseExactJson } from "../../domain/exactJsonNumber.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import {
|
||||
iterateUrlQueryParameters,
|
||||
@@ -13,6 +10,7 @@ import { environmentToolArguments } from "./environmentTools.js";
|
||||
import {
|
||||
REPLAY_BODY_LIMIT_BYTES,
|
||||
ENVIRONMENT_SERVE_OPERATION,
|
||||
replayedBody,
|
||||
type EnvironmentManifestTool,
|
||||
type EnvironmentSource,
|
||||
type RecordedExchange,
|
||||
@@ -183,11 +181,6 @@ const utf8Prefix = (text: string, maxBytes: number): string => {
|
||||
return text.slice(0, read);
|
||||
};
|
||||
|
||||
/** Bytes a padded base64 string decodes to. */
|
||||
const decodedBase64Bytes = (encoded: string): number =>
|
||||
Math.floor((encoded.length * 3) / 4) -
|
||||
(encoded.endsWith("==") ? 2 : encoded.endsWith("=") ? 1 : 0);
|
||||
|
||||
/** Base64 characters, in whole groups, whose bytes fit the replay limit. */
|
||||
const BASE64_LIMIT_CHARS = Math.floor(REPLAY_BODY_LIMIT_BYTES / 3) * 4;
|
||||
|
||||
@@ -196,29 +189,26 @@ interface ReplayBody {
|
||||
readonly truncated: boolean;
|
||||
}
|
||||
|
||||
/** The limit applies to the recorded bytes, not to their base64 spelling. */
|
||||
const replayBase64Body = (encoded: string, cut: boolean): ReplayBody =>
|
||||
!cut && decodedBase64Bytes(encoded) <= REPLAY_BODY_LIMIT_BYTES
|
||||
? { body: encoded, truncated: false }
|
||||
: { body: encoded.slice(0, BASE64_LIMIT_CHARS), truncated: true };
|
||||
|
||||
/** Parse a retained body as JSON when its content type is JSON-like; otherwise return the text. */
|
||||
/** The reply body and whether it was cut, as the source classifies the retained body. */
|
||||
const replayBody = (exchange: RecordedExchange): ReplayBody => {
|
||||
const text = exchange.bodyText;
|
||||
if (text === null) return { body: null, truncated: false };
|
||||
if (exchange.bodyEncoding === "base64")
|
||||
return replayBase64Body(text, exchange.bodyCut === true);
|
||||
const prefix = utf8Prefix(text, REPLAY_BODY_LIMIT_BYTES);
|
||||
if (prefix.length < text.length || exchange.bodyCut === true)
|
||||
return { body: prefix, truncated: true };
|
||||
if (
|
||||
exchange.contentType !== null &&
|
||||
isJsonLike(mediaTypeEssence(exchange.contentType))
|
||||
) {
|
||||
const value = parseExactJson(text);
|
||||
if (value !== undefined) return { body: value, truncated: false };
|
||||
const body = replayedBody(exchange);
|
||||
switch (body.kind) {
|
||||
case "none":
|
||||
return { body: null, truncated: false };
|
||||
case "base64":
|
||||
return body.whole
|
||||
? { body: body.encoded, truncated: false }
|
||||
: { body: body.encoded.slice(0, BASE64_LIMIT_CHARS), truncated: true };
|
||||
case "cut":
|
||||
return {
|
||||
body: utf8Prefix(body.text, REPLAY_BODY_LIMIT_BYTES),
|
||||
truncated: true,
|
||||
};
|
||||
case "json":
|
||||
return { body: body.value, truncated: false };
|
||||
case "text":
|
||||
return { body: body.text, truncated: false };
|
||||
}
|
||||
return { body: text, truncated: false };
|
||||
};
|
||||
|
||||
/** Recorded paths and queries are cut to this many UTF-8 bytes in replies. */
|
||||
|
||||
@@ -9,7 +9,11 @@ import {
|
||||
readRecordedResponseBody,
|
||||
type RecordedResponseBody,
|
||||
} from "../../domain/analysisView/recordedResponseBody.js";
|
||||
import { readRecordedRequestLine } from "../../domain/analysisView/webNetworkRouteTemplates.js";
|
||||
import { isJsonLike } from "../../domain/analysisView/EnvironmentManifestAccumulator.js";
|
||||
import {
|
||||
mediaTypeEssence,
|
||||
readRecordedRequestLine,
|
||||
} from "../../domain/analysisView/webNetworkRouteTemplates.js";
|
||||
import { browserScenarioCaptureSchema } from "../../domain/browserScenarioCapture.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import {
|
||||
@@ -17,8 +21,10 @@ import {
|
||||
type AnalysisInputIssue,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import { immutableEvidence, type Evidence } from "../../domain/evidence.js";
|
||||
import { parseExactJson } from "../../domain/exactJsonNumber.js";
|
||||
import { projectInputIssues } from "../../domain/inputIssueProjection.js";
|
||||
import { immutableJsonSnapshotMember } from "../../domain/immutableJson.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
safeParseOwnedWebNetworkCapture,
|
||||
@@ -47,6 +53,52 @@ export interface RecordedExchange extends RecordedResponseBody {
|
||||
/** Replayed bodies larger than this are cut at a UTF-8 boundary and flagged. */
|
||||
export const REPLAY_BODY_LIMIT_BYTES = 256 * 1024;
|
||||
|
||||
/** How replay answers with an exchange's body; replay and tool descriptions both read it from here. */
|
||||
export type ReplayedBody =
|
||||
| { readonly kind: "none" }
|
||||
/** Non-UTF-8 bytes as base64; `whole` when every recorded byte fits the replay limit. */
|
||||
| {
|
||||
readonly kind: "base64";
|
||||
readonly encoded: string;
|
||||
readonly whole: boolean;
|
||||
}
|
||||
/** UTF-8 text over the replay limit or cut when retained; replay answers with a prefix. */
|
||||
| { readonly kind: "cut"; readonly text: string }
|
||||
| { readonly kind: "json"; readonly value: JsonValue }
|
||||
| { readonly kind: "text"; readonly text: string };
|
||||
|
||||
/** Bytes a padded base64 string decodes to. */
|
||||
const decodedBase64Bytes = (encoded: string): number =>
|
||||
Math.floor((encoded.length * 3) / 4) -
|
||||
(encoded.endsWith("==") ? 2 : encoded.endsWith("=") ? 1 : 0);
|
||||
|
||||
/** Classify a retained body once: whole JSON-like UTF-8 text within the limit that parses exactly is JSON. */
|
||||
export const replayedBody = (exchange: RecordedExchange): ReplayedBody => {
|
||||
const text = exchange.bodyText;
|
||||
if (text === null) return { kind: "none" };
|
||||
if (exchange.bodyEncoding === "base64")
|
||||
return {
|
||||
kind: "base64",
|
||||
encoded: text,
|
||||
whole:
|
||||
exchange.bodyCut !== true &&
|
||||
decodedBase64Bytes(text) <= REPLAY_BODY_LIMIT_BYTES,
|
||||
};
|
||||
if (
|
||||
exchange.bodyCut === true ||
|
||||
// UTF-8 never takes fewer bytes than UTF-16 units, so the length check avoids encoding huge text.
|
||||
text.length > REPLAY_BODY_LIMIT_BYTES ||
|
||||
Buffer.byteLength(text, "utf8") > REPLAY_BODY_LIMIT_BYTES
|
||||
)
|
||||
return { kind: "cut", text };
|
||||
const value =
|
||||
exchange.contentType !== null &&
|
||||
isJsonLike(mediaTypeEssence(exchange.contentType))
|
||||
? parseExactJson(text)
|
||||
: undefined;
|
||||
return value === undefined ? { kind: "text", text } : { kind: "json", value };
|
||||
};
|
||||
|
||||
/** A HAR inspected by inspect_web_network_capture, or a session recorded by capture_browser_scenario. */
|
||||
export type EnvironmentCaptureKind = "web-network-capture" | "browser-scenario";
|
||||
|
||||
|
||||
@@ -485,12 +485,12 @@ describe("rea-state/v1 validation", () => {
|
||||
"a document for a tool that is not a document view",
|
||||
(document: Editable) => {
|
||||
document.rea_state.documents.push({
|
||||
tool: "get_org",
|
||||
path: "/api/org",
|
||||
tool: "get_folders",
|
||||
path: "/api/folders",
|
||||
value: {},
|
||||
});
|
||||
},
|
||||
"rea_state/documents/0/tool",
|
||||
`rea_state/documents/${String(state.documents.length)}/tool`,
|
||||
],
|
||||
[
|
||||
"a member the format does not define",
|
||||
|
||||
@@ -61,17 +61,19 @@ export const servedCatalog = (
|
||||
operation: string = ENVIRONMENT_SERVE_OPERATION,
|
||||
): Result<ServedCatalog, AnalysisResourceConstraintError> => {
|
||||
const dataPlane: EnvironmentDataPlane = fromState ? "state" : "none";
|
||||
const tools = environmentToolDefinitions(
|
||||
fromState
|
||||
? {
|
||||
const catalogSource: EnvironmentSource = fromState
|
||||
? {
|
||||
...source,
|
||||
manifest: {
|
||||
...source.manifest,
|
||||
tools: source.manifest.tools.filter(servedFromState),
|
||||
}
|
||||
: source.manifest,
|
||||
source.evidenceId,
|
||||
name,
|
||||
{ operation, dataPlane },
|
||||
);
|
||||
},
|
||||
}
|
||||
: source;
|
||||
const tools = environmentToolDefinitions(catalogSource, name, {
|
||||
operation,
|
||||
dataPlane,
|
||||
});
|
||||
return tools.ok ? ok({ tools: tools.value, dataPlane }) : tools;
|
||||
};
|
||||
|
||||
|
||||
@@ -311,6 +311,17 @@ describe("recorded and document answers", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("replays a read whose rendered document exceeds the body limit", () => {
|
||||
const values = Array.from({ length: 60_000 }, () => "1e3").join(",");
|
||||
const body = `{"values":[${values}]}`;
|
||||
const serving = servingFrom([["GET", "/api/config", body]]);
|
||||
expect(serving.store.state.documents).toEqual([]);
|
||||
expect(answer(serving, "get_config", {})).toMatchObject({
|
||||
replay: { record_ordinal: 0 },
|
||||
state: { source: "recording", reason: "rendered-oversize" },
|
||||
});
|
||||
});
|
||||
|
||||
it("answers a document's recorded line with the state's value for it", () => {
|
||||
const capture = timedCaptureEvidence([
|
||||
["GET", "/api/settings", { theme: "dark", density: "compact" }],
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { environmentServeSource } from "../../../tests/fixtures/environmentServe.js";
|
||||
import type { EnvironmentManifestTool } from "./environmentSource.js";
|
||||
import type {
|
||||
EnvironmentManifestTool,
|
||||
RecordedExchange,
|
||||
} from "./environmentSource.js";
|
||||
import {
|
||||
ENVIRONMENT_CATALOG_LIMIT_BYTES,
|
||||
environmentCard,
|
||||
environmentToolArguments,
|
||||
environmentToolDefinitions as prepareDefinitions,
|
||||
@@ -17,18 +21,16 @@ const environmentToolDefinitions = (
|
||||
if (!result.ok) throw result.error;
|
||||
return result.value;
|
||||
};
|
||||
const withTools = (tools: readonly EnvironmentManifestTool[]) => ({
|
||||
...source,
|
||||
manifest: { ...source.manifest, tools: [...tools] },
|
||||
});
|
||||
const environmentToolInputSchema = (tool: EnvironmentManifestTool) => {
|
||||
const [definition] = environmentToolDefinitions(
|
||||
{ ...source.manifest, tools: [tool] },
|
||||
source.evidenceId,
|
||||
);
|
||||
const [definition] = environmentToolDefinitions(withTools([tool]));
|
||||
if (definition === undefined) throw new Error("Missing tool definition");
|
||||
return definition.inputSchema;
|
||||
};
|
||||
const definitions = environmentToolDefinitions(
|
||||
source.manifest,
|
||||
source.evidenceId,
|
||||
);
|
||||
const definitions = environmentToolDefinitions(source);
|
||||
|
||||
const tool = (name: string): EnvironmentManifestTool => {
|
||||
const found = source.manifest.tools.find((entry) => entry.name === name);
|
||||
@@ -44,7 +46,7 @@ describe("environmentToolDefinitions", () => {
|
||||
const users = tool("get_users_by_id");
|
||||
const definition = definitions.find(({ name }) => name === users.name);
|
||||
expect(definition?.description).toBe(
|
||||
`${REPLAY_CATALOG_INSTRUCTIONS} GET ${users.path_template}; replays retained records [${users.record_ordinals.join(", ")}]; evidence ${source.evidenceId}`,
|
||||
`${REPLAY_CATALOG_INSTRUCTIONS} GET ${users.path_template}; returns a JSON object with fields "id", "uid", "login", "email"; recorded statuses 200; replays retained records [${users.record_ordinals.join(", ")}]; evidence ${source.evidenceId}`,
|
||||
);
|
||||
expect(definition?.tool).toBe(users);
|
||||
});
|
||||
@@ -108,6 +110,243 @@ describe("environmentToolDefinitions", () => {
|
||||
});
|
||||
});
|
||||
|
||||
const described = (name: string): string | undefined =>
|
||||
definitions.find((definition) => definition.name === name)?.description;
|
||||
/** The users tool citing exactly the given exchanges, with the statuses its manifest recorded. */
|
||||
const usersAnswering = (
|
||||
answers: readonly Partial<RecordedExchange>[],
|
||||
statuses: readonly number[] = [200],
|
||||
): string | undefined => {
|
||||
const exchanges = new Map(
|
||||
answers.map((answer, index): [number, RecordedExchange] => [
|
||||
index,
|
||||
{
|
||||
ordinal: index,
|
||||
path: "/api/users/2",
|
||||
query: null,
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
bodyText: null,
|
||||
...answer,
|
||||
},
|
||||
]),
|
||||
);
|
||||
const users = {
|
||||
...tool("get_users_by_id"),
|
||||
record_ordinals: [...exchanges.keys()],
|
||||
statuses: [...statuses],
|
||||
};
|
||||
const [definition] = environmentToolDefinitions({
|
||||
...withTools([users]),
|
||||
exchanges,
|
||||
});
|
||||
return definition?.description;
|
||||
};
|
||||
const returnsOf = (bodyText: string): string | undefined =>
|
||||
/\{numeric\}; (?<returns>[^;]*); recorded statuses/u.exec(
|
||||
usersAnswering([{ bodyText }]) ?? "",
|
||||
)?.groups?.returns;
|
||||
|
||||
describe("environment tool response shapes", () => {
|
||||
it("describes array, enveloped list and object responses one level deep with their statuses", () => {
|
||||
expect(described("get_folders")).toContain(
|
||||
'GET /api/folders; returns a JSON array of objects with fields "id", "uid", "title"; recorded statuses 200; replays',
|
||||
);
|
||||
expect(described("get_dashboards_versions_by_uid")).toContain(
|
||||
'; returns a JSON object whose "versions" lists objects with fields "id", "version"; recorded statuses 200; replays',
|
||||
);
|
||||
expect(described("get_dashboards_by_uid")).toContain(
|
||||
'; returns a JSON object with fields "meta", "dashboard"; recorded statuses 200; replays',
|
||||
);
|
||||
});
|
||||
|
||||
it("describes empty, scalar and fieldless bodies and bounds a long member name", () => {
|
||||
expect(
|
||||
[
|
||||
"[]",
|
||||
"[1,2]",
|
||||
'"text"',
|
||||
"null",
|
||||
"true",
|
||||
"3",
|
||||
"{}",
|
||||
"[{},{}]",
|
||||
'{"items":[]}',
|
||||
].map(returnsOf),
|
||||
).toEqual([
|
||||
"returns a JSON array",
|
||||
"returns a JSON array",
|
||||
"returns a JSON string",
|
||||
"returns a JSON null",
|
||||
"returns a JSON boolean",
|
||||
"returns a JSON number",
|
||||
"returns a JSON object with no fields",
|
||||
"returns a JSON array of objects with no fields",
|
||||
'returns a JSON object with fields "items"',
|
||||
]);
|
||||
const member = "m".repeat(2048);
|
||||
expect(
|
||||
usersAnswering([
|
||||
{ bodyText: JSON.stringify({ [member]: [{ id: 1 }, { id: 2 }] }) },
|
||||
]),
|
||||
).toContain(
|
||||
`{numeric}; returns a JSON object whose "${"m".repeat(1022)}"… (member display truncated; full value in capture Evidence) lists objects with fields "id"; recorded statuses 200;`,
|
||||
);
|
||||
});
|
||||
|
||||
it("quotes captured names, so a separator in one cannot read as more of the description", () => {
|
||||
const description = usersAnswering([
|
||||
{
|
||||
bodyText: JSON.stringify({
|
||||
"id, name": 1,
|
||||
"x; recorded statuses 500": 2,
|
||||
}),
|
||||
},
|
||||
]);
|
||||
expect(description).toContain(
|
||||
'returns a JSON object with fields "id, name", "x; recorded statuses 500"; recorded statuses 200;',
|
||||
);
|
||||
expect(
|
||||
usersAnswering([
|
||||
{ bodyText: JSON.stringify({ "a\nb": [{ id: 1 }, { id: 2 }] }) },
|
||||
]),
|
||||
).toContain('whose "a\\nb" lists objects with fields "id";');
|
||||
// A name too long to show whole is shown as a quoted prefix, so its newline cannot start a line of description.
|
||||
const long = `items; recorded statuses 500\n${"m".repeat(2048)}`;
|
||||
const shown = usersAnswering([
|
||||
{ bodyText: JSON.stringify({ [long]: [{ id: 1 }, { id: 2 }] }) },
|
||||
]);
|
||||
expect(shown).toContain('whose "items; recorded statuses 500\\nmmm');
|
||||
expect(shown).not.toContain("items; recorded statuses 500\n");
|
||||
});
|
||||
|
||||
it("applies the sample budget after JSON escaping", () => {
|
||||
const member = "\u0000".repeat(1024);
|
||||
const description = usersAnswering([
|
||||
{ bodyText: JSON.stringify({ [member]: [{ id: 1 }] }) },
|
||||
]);
|
||||
const shown = /whose (?<name>"(?:\\u0000)*")/u.exec(description ?? "")
|
||||
?.groups?.name;
|
||||
expect(shown).toBeDefined();
|
||||
expect(Buffer.byteLength(shown ?? "")).toBeLessThanOrEqual(1024);
|
||||
expect(description).toContain("member display truncated");
|
||||
expect(description).not.toContain("\u0000");
|
||||
});
|
||||
});
|
||||
|
||||
describe("environment tool response summaries", () => {
|
||||
it("describes the first cited 2xx record replay answers as JSON, then the manifest's statuses", () => {
|
||||
const description = usersAnswering(
|
||||
[
|
||||
{
|
||||
contentType: "text/html",
|
||||
bodyText: '{"looks":"like JSON"}',
|
||||
},
|
||||
{ status: 401, bodyText: '{"message":"Unauthorized","url":"/login"}' },
|
||||
{ bodyText: '{"id":2,"login":"probe"}' },
|
||||
{ bodyText: '[{"id":3,"email":"viewer@example.test"}]' },
|
||||
],
|
||||
[200, 401],
|
||||
);
|
||||
expect(description).toContain(
|
||||
'{numeric}; returns a JSON object with fields "id", "login"; recorded statuses 200, 401; replays retained records [0, 1, 2, 3];',
|
||||
);
|
||||
});
|
||||
|
||||
it("labels a JSON record of another status only when no 2xx record replays as JSON", () => {
|
||||
expect(
|
||||
usersAnswering(
|
||||
[
|
||||
{ contentType: "text/html", bodyText: "<p>Signed out</p>" },
|
||||
{ status: 401, bodyText: '{"message":"Unauthorized"}' },
|
||||
{ status: 403, bodyText: '{"errors":[{"message":"Forbidden"}]}' },
|
||||
],
|
||||
[200, 401, 403],
|
||||
),
|
||||
).toContain(
|
||||
'{numeric}; status 401 returns a JSON object with fields "message"; recorded statuses 200, 401, 403; replays',
|
||||
);
|
||||
expect(
|
||||
usersAnswering([{ status: null, bodyText: '{"id":2}' }], []),
|
||||
).toContain(
|
||||
'{numeric}; a record without a status returns a JSON object with fields "id"; no recorded status; replays',
|
||||
);
|
||||
});
|
||||
|
||||
it("states only the statuses when no cited exchange replays as JSON", () => {
|
||||
for (const answer of [
|
||||
{ contentType: "text/html", bodyText: "<p>Granite Ledger</p>" },
|
||||
{ bodyText: '{"id":9007199254740993}' },
|
||||
{ bodyText: '{"id":2}', bodyEncoding: "base64" as const },
|
||||
{ bodyText: '{"id":2}', bodyCut: true as const },
|
||||
{ bodyText: null },
|
||||
])
|
||||
expect(usersAnswering([answer])).toMatch(
|
||||
/\{numeric\}; recorded statuses 200; replays retained records/u,
|
||||
);
|
||||
});
|
||||
|
||||
it("charges response summaries to the catalog reservation before retaining them", () => {
|
||||
const fields = Object.fromEntries(
|
||||
Array.from({ length: 300 }, (_, index) => [
|
||||
`field_${String(index).padStart(3, "0")}`,
|
||||
index,
|
||||
]),
|
||||
);
|
||||
const users = tool("get_users_by_id");
|
||||
const ordinal = users.record_ordinals[0] ?? 0;
|
||||
const catalog = (contentType: string) =>
|
||||
prepareDefinitions({
|
||||
...withTools(
|
||||
Array.from({ length: 200 }, (_, index) => ({
|
||||
...users,
|
||||
name: `get_users_${String(index)}`,
|
||||
record_ordinals: [ordinal],
|
||||
})),
|
||||
),
|
||||
exchanges: new Map([
|
||||
[
|
||||
ordinal,
|
||||
{
|
||||
ordinal,
|
||||
path: "/api/users/2",
|
||||
query: null,
|
||||
status: 200,
|
||||
contentType,
|
||||
bodyText: JSON.stringify(fields),
|
||||
},
|
||||
],
|
||||
]),
|
||||
});
|
||||
expect(catalog("text/html").ok).toBe(true);
|
||||
const bounded = catalog("application/json");
|
||||
expect(bounded).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "AnalysisResourceConstraintError",
|
||||
resource: "capacity",
|
||||
reportedLimits: { max_catalog_bytes: ENVIRONMENT_CATALOG_LIMIT_BYTES },
|
||||
},
|
||||
});
|
||||
const [single] = environmentToolDefinitions({
|
||||
...withTools([users]),
|
||||
exchanges: new Map([
|
||||
[
|
||||
ordinal,
|
||||
{
|
||||
...source.exchanges.get(ordinal)!,
|
||||
bodyText: JSON.stringify(fields),
|
||||
},
|
||||
],
|
||||
]),
|
||||
});
|
||||
expect(Buffer.byteLength(single?.description ?? "")).toBeLessThan(1700);
|
||||
expect(single?.description).toContain(
|
||||
"/300 shown; full list in capture Evidence)",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("environment catalog description budgets", () => {
|
||||
it("bounds captured scalar prose while preserving complete lookup identities", () => {
|
||||
const query = "query-".repeat(1000);
|
||||
@@ -117,10 +356,7 @@ describe("environment catalog description budgets", () => {
|
||||
path_template: `/${"😀".repeat(1000)}`,
|
||||
query_parameter_names: [query],
|
||||
};
|
||||
const [definition] = environmentToolDefinitions(
|
||||
{ ...source.manifest, tools: [expanded] },
|
||||
source.evidenceId,
|
||||
);
|
||||
const [definition] = environmentToolDefinitions(withTools([expanded]));
|
||||
expect(definition?.tool).toBe(expanded);
|
||||
expect(Buffer.byteLength(definition?.description ?? "")).toBeLessThan(3500);
|
||||
expect(definition?.description).toContain(
|
||||
@@ -152,10 +388,7 @@ describe("environment catalog description budgets", () => {
|
||||
})),
|
||||
record_ordinals: ordinals,
|
||||
};
|
||||
const [definition] = environmentToolDefinitions(
|
||||
{ ...source.manifest, tools: [expanded] },
|
||||
source.evidenceId,
|
||||
);
|
||||
const [definition] = environmentToolDefinitions(withTools([expanded]));
|
||||
expect(definition?.tool).toBe(expanded);
|
||||
expect(expanded.params[0]?.observed_values).toBe(observations);
|
||||
expect(expanded.record_ordinals).toBe(ordinals);
|
||||
@@ -180,10 +413,7 @@ describe("environment catalog description budgets", () => {
|
||||
...users,
|
||||
path_template: `/api/${"segment/".repeat(200_000)}{id}`,
|
||||
};
|
||||
const [definition] = environmentToolDefinitions(
|
||||
{ ...source.manifest, tools: [long] },
|
||||
source.evidenceId,
|
||||
);
|
||||
const [definition] = environmentToolDefinitions(withTools([long]));
|
||||
expect(Buffer.byteLength(definition?.description ?? "")).toBeLessThan(2048);
|
||||
expect(definition?.description).toContain(
|
||||
"path display truncated; full value in manifest Evidence",
|
||||
@@ -330,3 +560,22 @@ describe("environmentCard", () => {
|
||||
expect(environmentCard("renamed", definitions).name).toBe("renamed");
|
||||
});
|
||||
});
|
||||
|
||||
describe("environment tool annotations", () => {
|
||||
it("marks OPTIONS and TRACE safe and idempotent, as GET is", () => {
|
||||
const get = tool("get_users_by_id");
|
||||
const hints = environmentToolDefinitions(
|
||||
withTools(
|
||||
(["OPTIONS", "TRACE"] as const).map((method) => ({
|
||||
...get,
|
||||
name: `${method.toLowerCase()}_users_by_id`,
|
||||
method,
|
||||
})),
|
||||
),
|
||||
).map((definition) => definition.annotations);
|
||||
expect(hints).toEqual([
|
||||
{ readOnlyHint: true, idempotentHint: true, openWorldHint: false },
|
||||
{ readOnlyHint: true, idempotentHint: true, openWorldHint: false },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,9 +1,20 @@
|
||||
import {
|
||||
isObject,
|
||||
isQueryPost,
|
||||
listItems,
|
||||
locateList,
|
||||
pointerTokens,
|
||||
type JsonObject,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import { AnalysisResourceConstraintError } from "../../domain/analysisErrorCore.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import { ENVIRONMENT_SERVE_OPERATION } from "./environmentSource.js";
|
||||
import type {
|
||||
EnvironmentManifest,
|
||||
EnvironmentManifestTool,
|
||||
import {
|
||||
ENVIRONMENT_SERVE_OPERATION,
|
||||
replayedBody,
|
||||
type EnvironmentManifestTool,
|
||||
type EnvironmentSource,
|
||||
type RecordedExchange,
|
||||
} from "./environmentSource.js";
|
||||
|
||||
/** Methods whose tools accept an optional request body object. */
|
||||
@@ -27,36 +38,98 @@ export interface EnvironmentToolInputSchema {
|
||||
readonly additionalProperties: false;
|
||||
}
|
||||
|
||||
/** One manifest tool as an MCP tool: name, description and input schema. */
|
||||
/** MCP hints for the captured method; answers come from the capture, never the open world. */
|
||||
export interface EnvironmentToolAnnotations {
|
||||
readonly readOnlyHint: boolean;
|
||||
/** Set for `DELETE` only; other writes leave the MCP default (destructive). */
|
||||
readonly destructiveHint?: true;
|
||||
readonly idempotentHint: boolean;
|
||||
readonly openWorldHint: false;
|
||||
}
|
||||
|
||||
/** One manifest tool as an MCP tool: name, description, input schema and the hints MCP advertises; the card omits the hints. */
|
||||
export interface EnvironmentToolDefinition {
|
||||
readonly name: string;
|
||||
readonly description: string;
|
||||
readonly inputSchema: EnvironmentToolInputSchema;
|
||||
readonly annotations: EnvironmentToolAnnotations;
|
||||
readonly tool: EnvironmentManifestTool;
|
||||
}
|
||||
|
||||
/** RFC 9110 safe methods: a preflight `OPTIONS` reads, as `GET` does. */
|
||||
const SAFE_METHODS: ReadonlySet<string> = new Set([
|
||||
"GET",
|
||||
"HEAD",
|
||||
"OPTIONS",
|
||||
"TRACE",
|
||||
]);
|
||||
|
||||
const IDEMPOTENT_METHODS: ReadonlySet<string> = new Set([
|
||||
...SAFE_METHODS,
|
||||
"PUT",
|
||||
"DELETE",
|
||||
]);
|
||||
|
||||
const environmentToolAnnotations = ({
|
||||
method,
|
||||
path_template: path,
|
||||
}: EnvironmentManifestTool): EnvironmentToolAnnotations => ({
|
||||
readOnlyHint: SAFE_METHODS.has(method) || isQueryPost(method, path),
|
||||
...(method === "DELETE" ? { destructiveHint: true as const } : {}),
|
||||
idempotentHint: IDEMPOTENT_METHODS.has(method),
|
||||
openWorldHint: false,
|
||||
});
|
||||
|
||||
/** Agent-context budget for each captured display sample, excluding its provenance notice. */
|
||||
const DESCRIPTION_SAMPLE_BYTES = 1024;
|
||||
|
||||
/** A display prefix is explicitly separate from the complete identity held by the manifest. */
|
||||
const describeText = (text: string, label: string): string => {
|
||||
/** Whether a rendered sample fits the display budget. Callers' `show` never shortens its input. */
|
||||
const sampleFits = (shown: string): boolean =>
|
||||
shown.length <= DESCRIPTION_SAMPLE_BYTES &&
|
||||
Buffer.byteLength(shown, "utf8") <= DESCRIPTION_SAMPLE_BYTES;
|
||||
|
||||
/**
|
||||
* A display prefix is explicitly separate from the complete identity held by the Evidence named.
|
||||
* `show` renders the text or its prefix. The budget applies to that rendering, so JSON quoting of a
|
||||
* name full of NULs or newlines cannot outgrow the sample.
|
||||
*/
|
||||
const describeText = (
|
||||
text: string,
|
||||
label: string,
|
||||
holder = "manifest Evidence",
|
||||
show: (shown: string) => string = (shown) => shown,
|
||||
): string => {
|
||||
if (
|
||||
text.length <= DESCRIPTION_SAMPLE_BYTES &&
|
||||
Buffer.byteLength(text, "utf8") <= DESCRIPTION_SAMPLE_BYTES
|
||||
)
|
||||
return text;
|
||||
const { read } = new TextEncoder().encodeInto(
|
||||
text,
|
||||
new Uint8Array(DESCRIPTION_SAMPLE_BYTES),
|
||||
);
|
||||
return `${text.slice(0, read)}… (${label} display truncated; full value in manifest Evidence)`;
|
||||
) {
|
||||
const shown = show(text);
|
||||
if (sampleFits(shown)) return shown;
|
||||
}
|
||||
let low = 0;
|
||||
let high = Math.min(text.length, DESCRIPTION_SAMPLE_BYTES);
|
||||
let best = 0;
|
||||
while (low <= high) {
|
||||
const mid = (low + high) >> 1;
|
||||
if (sampleFits(show(text.slice(0, mid)))) {
|
||||
best = mid;
|
||||
low = mid + 1;
|
||||
} else high = mid - 1;
|
||||
}
|
||||
// A lone surrogate is not a character. Step off a high surrogate so the prefix stays whole.
|
||||
if (best > 0 && best < text.length) {
|
||||
const unit = text.charCodeAt(best - 1);
|
||||
if (unit >= 0xd8_00 && unit <= 0xdb_ff) best -= 1;
|
||||
}
|
||||
return `${show(text.slice(0, best))}… (${label} display truncated; full value in ${holder})`;
|
||||
};
|
||||
|
||||
/** Sample whole values before joining; keep full lists in the manifest that owns them. */
|
||||
/** Sample whole values before joining; keep full lists in the Evidence that owns them. */
|
||||
const describeList = <Value>(
|
||||
values: Iterable<Value>,
|
||||
count: number,
|
||||
display: (value: Value) => string = String,
|
||||
holder = "manifest Evidence",
|
||||
): string => {
|
||||
if (count === 0) return "none";
|
||||
const sample: string[] = [];
|
||||
@@ -76,7 +149,80 @@ const describeList = <Value>(
|
||||
const listed = sample.join(", ");
|
||||
return sample.length === count
|
||||
? listed
|
||||
: `${listed}${sample.length === 0 ? "" : ", "}… (${String(sample.length)}/${String(count)} shown; full list in manifest Evidence)`;
|
||||
: `${listed}${sample.length === 0 ? "" : ", "}… (${String(sample.length)}/${String(count)} shown; full list in ${holder})`;
|
||||
};
|
||||
|
||||
/**
|
||||
* A captured name as a JSON string, so a separator, quote or newline in it cannot read as more of the
|
||||
* description. A name too long for a sample is left as it is: the sample refuses it either way.
|
||||
*/
|
||||
const quotedName = (name: string): string =>
|
||||
name.length > DESCRIPTION_SAMPLE_BYTES ? name : JSON.stringify(name);
|
||||
|
||||
/** Field names of the objects in first-seen order; the recorded bodies stay in the capture Evidence. */
|
||||
const withFields = (objects: readonly JsonObject[]): string => {
|
||||
const names = new Set<string>();
|
||||
for (const object of objects)
|
||||
for (const name of Object.keys(object)) names.add(name);
|
||||
return names.size === 0
|
||||
? "with no fields"
|
||||
: `with fields ${describeList(names, names.size, quotedName, "capture Evidence")}`;
|
||||
};
|
||||
|
||||
/** One level of a JSON body; a list is an array of objects found by the state's list rule. */
|
||||
const describeJson = (value: JsonValue): string => {
|
||||
const pointer = locateList(value, undefined);
|
||||
const items = pointer === undefined ? [] : (listItems(value, pointer) ?? []);
|
||||
if (pointer !== undefined && items.length > 0) {
|
||||
const [member] = pointerTokens(pointer);
|
||||
return member === undefined
|
||||
? `a JSON array of objects ${withFields(items)}`
|
||||
: `a JSON object whose ${describeText(member, "member", "capture Evidence", JSON.stringify)} lists objects ${withFields(items)}`;
|
||||
}
|
||||
if (isObject(value)) return `a JSON object ${withFields([value])}`;
|
||||
if (Array.isArray(value)) return "a JSON array";
|
||||
return `a JSON ${value === null ? "null" : typeof value}`;
|
||||
};
|
||||
|
||||
const isSuccess = (status: number | null): boolean =>
|
||||
status !== null && status >= 200 && status < 300;
|
||||
|
||||
/** The first cited record replay answers as JSON, preferring a 2xx one; each body is parsed at most once. */
|
||||
const firstJson = (
|
||||
tool: EnvironmentManifestTool,
|
||||
exchanges: ReadonlyMap<number, RecordedExchange>,
|
||||
):
|
||||
| { readonly status: number | null; readonly value: JsonValue }
|
||||
| undefined => {
|
||||
for (const success of [true, false])
|
||||
for (const ordinal of tool.record_ordinals) {
|
||||
const exchange = exchanges.get(ordinal);
|
||||
if (exchange === undefined || isSuccess(exchange.status) !== success)
|
||||
continue;
|
||||
const body = replayedBody(exchange);
|
||||
if (body.kind === "json")
|
||||
return { status: exchange.status, value: body.value };
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** What the tool returns as JSON, labelled with the status unless it succeeded, then the manifest's recorded statuses. */
|
||||
const describeResponses = (
|
||||
tool: EnvironmentManifestTool,
|
||||
exchanges: ReadonlyMap<number, RecordedExchange>,
|
||||
): string => {
|
||||
const recorded =
|
||||
tool.statuses.length === 0
|
||||
? "no recorded status"
|
||||
: `recorded statuses ${describeList(tool.statuses, tool.statuses.length)}`;
|
||||
const found = firstJson(tool, exchanges);
|
||||
if (found === undefined) return recorded;
|
||||
const returns = isSuccess(found.status)
|
||||
? "returns"
|
||||
: found.status === null
|
||||
? "a record without a status returns"
|
||||
: `status ${String(found.status)} returns`;
|
||||
return `${returns} ${describeJson(found.value)}; ${recorded}`;
|
||||
};
|
||||
|
||||
/** Captured argument names longer than this are advertised under a short alias; the manifest keeps the name. */
|
||||
@@ -208,9 +354,8 @@ export interface EnvironmentCatalogOptions {
|
||||
|
||||
/** Admit complete discovery before binding; six bytes per UTF-16 unit covers JSON escaping and UTF-8 expansion. */
|
||||
export const environmentToolDefinitions = (
|
||||
manifest: EnvironmentManifest,
|
||||
evidenceId: string,
|
||||
name: string = manifest.name,
|
||||
source: EnvironmentSource,
|
||||
name: string = source.manifest.name,
|
||||
{
|
||||
operation = ENVIRONMENT_SERVE_OPERATION,
|
||||
dataPlane = "none",
|
||||
@@ -254,15 +399,22 @@ export const environmentToolDefinitions = (
|
||||
? "answers from the served environment state where the state backs this tool, otherwise replays retained records"
|
||||
: "replays retained records";
|
||||
const definitions: EnvironmentToolDefinition[] = [];
|
||||
for (const tool of manifest.tools) {
|
||||
if (!reserve(EMPTY_TOOL_BYTES) || !text(tool.name)) return refused();
|
||||
for (const tool of source.manifest.tools) {
|
||||
const annotations = environmentToolAnnotations(tool);
|
||||
// Only MCP discovery carries the hints; charging them per tool over-charges the card harmlessly.
|
||||
if (
|
||||
!reserve(EMPTY_TOOL_BYTES + JSON.stringify({ annotations }).length) ||
|
||||
!text(tool.name)
|
||||
)
|
||||
return refused();
|
||||
const count =
|
||||
tool.params.length +
|
||||
tool.query_parameter_names.length +
|
||||
(WRITE_METHODS.has(tool.method) ? 1 : 0);
|
||||
// A lower bound rejects enormous argument lists before constructing their alias map.
|
||||
if (!reserve(count * EMPTY_PROPERTY_BYTES)) return refused();
|
||||
const description = `${REPLAY_CATALOG_INSTRUCTIONS} ${describeText(tool.method, "method")} ${describeText(tool.path_template, "path")}; ${answers} [${describeList(tool.record_ordinals, tool.record_ordinals.length)}]; evidence ${evidenceId}`;
|
||||
// The response summary is bounded by its samples and charged with the whole description.
|
||||
const description = `${REPLAY_CATALOG_INSTRUCTIONS} ${describeText(tool.method, "method")} ${describeText(tool.path_template, "path")}; ${describeResponses(tool, source.exchanges)}; ${answers} [${describeList(tool.record_ordinals, tool.record_ordinals.length)}]; evidence ${source.evidenceId}`;
|
||||
if (!text(description)) return refused();
|
||||
const args = environmentToolArguments(tool);
|
||||
const properties = new Map<string, EnvironmentToolProperty>();
|
||||
@@ -289,6 +441,7 @@ export const environmentToolDefinitions = (
|
||||
required,
|
||||
additionalProperties: false,
|
||||
},
|
||||
annotations,
|
||||
tool,
|
||||
});
|
||||
}
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -18,6 +18,7 @@ import {
|
||||
deeperThan,
|
||||
hasPrototypeKey,
|
||||
isStateWrite,
|
||||
STATE_BODY_LIMIT_BYTES,
|
||||
type ParsedBody,
|
||||
type StateRecordFacts,
|
||||
type StateTimeline,
|
||||
@@ -50,8 +51,7 @@ type StateFields = Pick<
|
||||
| "limitations"
|
||||
>;
|
||||
|
||||
/** Bodies beyond this many UTF-8 bytes are not parsed into state; their views stay recorded. */
|
||||
export const STATE_BODY_LIMIT_BYTES = 256 * 1024;
|
||||
export { STATE_BODY_LIMIT_BYTES };
|
||||
|
||||
/**
|
||||
* Levels the state's own nesting adds above a recorded value: Evidence, a `data/get` reply and a
|
||||
|
||||
@@ -4,24 +4,28 @@ import {
|
||||
compositeKey,
|
||||
} from "../unicodeCodePointOrder.js";
|
||||
import {
|
||||
IDENTITY_NAMES,
|
||||
apiBaseLengths,
|
||||
at,
|
||||
collectionPathOf,
|
||||
hasPrototypeKey,
|
||||
identifies,
|
||||
IDENTITY_NAMES,
|
||||
identityCandidates,
|
||||
isObject,
|
||||
isParam,
|
||||
isQueryPost,
|
||||
isStateWrite,
|
||||
jsonBytes,
|
||||
lastSegmentValue,
|
||||
STATE_BODY_LIMIT_BYTES,
|
||||
listItems,
|
||||
normalizedPath,
|
||||
locateItem,
|
||||
locateList,
|
||||
normalizedPath,
|
||||
rootOf,
|
||||
same,
|
||||
scalarText,
|
||||
segmentsOf,
|
||||
apiBaseLengths,
|
||||
writeSpanCheck,
|
||||
type ExtractedState,
|
||||
type JsonObject,
|
||||
@@ -142,10 +146,7 @@ export class EnvironmentStateExtraction {
|
||||
plan.tool.name,
|
||||
plan.item ? checkItemPlan(gate, plan) : checkListPlan(gate, plan),
|
||||
);
|
||||
const documents = this.#documents(candidates);
|
||||
for (const { tool } of candidates)
|
||||
if (!this.#views.has(tool.name)) this.#recorded(tool, "not-entity");
|
||||
return this.#result(documents);
|
||||
return this.#result(this.#documents(candidates));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof StateBudgetExhausted) return null;
|
||||
throw cause;
|
||||
@@ -302,15 +303,26 @@ export class EnvironmentStateExtraction {
|
||||
this.#addPlan(candidate, pointer, recordings);
|
||||
}
|
||||
|
||||
/** List tools use the proved field, else the first identity name every non-empty recording's items carry. */
|
||||
/**
|
||||
* List tools use the proved field, else the first identity candidate every non-empty recording's
|
||||
* items share; a suffix-named field must also show distinct values across two items of one
|
||||
* recording. Reads without a list or identity are left to documents.
|
||||
*/
|
||||
#planList(candidate: Candidate): void {
|
||||
const { tool, type } = candidate;
|
||||
const recordings = this.#initRecordingsOf(candidate);
|
||||
if (recordings === undefined) return;
|
||||
const proved = this.#identity.get(type)?.field;
|
||||
const fieldsOf = (items: readonly JsonObject[]): string[] =>
|
||||
proved === undefined
|
||||
? identityCandidates(items)
|
||||
: identifies(items, proved)
|
||||
? [proved]
|
||||
: [];
|
||||
let pointer: string | undefined;
|
||||
let names: readonly string[] | undefined;
|
||||
let sawItems = false;
|
||||
// A single listed item is distinct from nothing, so it shows no field distinct.
|
||||
const shown = new Set<string>();
|
||||
for (const record of recordings) {
|
||||
const found = this.#withBody(record.ordinal, (body) => {
|
||||
if (body === undefined) return null;
|
||||
@@ -321,29 +333,32 @@ export class EnvironmentStateExtraction {
|
||||
located === undefined ? undefined : listItems(body, located);
|
||||
if (items === undefined) return undefined;
|
||||
pointer = located;
|
||||
if (items.length > 0) sawItems = true;
|
||||
// An empty list constrains no field.
|
||||
return items.length === 0
|
||||
? (names ?? (proved === undefined ? IDENTITY_NAMES : [proved]))
|
||||
: (proved === undefined ? IDENTITY_NAMES : [proved]).filter((name) =>
|
||||
identifies(items, name),
|
||||
);
|
||||
? true
|
||||
: { fields: fieldsOf(items), several: items.length > 1 };
|
||||
});
|
||||
if (found === null || found === undefined) {
|
||||
this.#recorded(tool, found === null ? "body-unusable" : "not-entity");
|
||||
if (found === undefined) return;
|
||||
if (found === null) {
|
||||
this.#recorded(tool, "body-unusable");
|
||||
return;
|
||||
}
|
||||
names = (names ?? found).filter((name) => found.includes(name));
|
||||
if (found === true) continue;
|
||||
// The first recording's candidates stand; later ones intersect through a set, so a wide
|
||||
// item costs one pass per recording.
|
||||
if (names === undefined) names = found.fields;
|
||||
else {
|
||||
const matching = new Set(found.fields);
|
||||
names = names.filter((name) => matching.has(name));
|
||||
}
|
||||
if (found.several) for (const name of found.fields) shown.add(name);
|
||||
}
|
||||
const field = names?.[0];
|
||||
// An identity name is only inferred from items that carry it; empty lists prove none.
|
||||
if (
|
||||
field === undefined ||
|
||||
pointer === undefined ||
|
||||
(proved === undefined && !sawItems)
|
||||
) {
|
||||
this.#recorded(tool, "no-identity");
|
||||
return;
|
||||
}
|
||||
const field = (names ?? (proved === undefined ? [] : [proved])).find(
|
||||
(name) =>
|
||||
name === proved || IDENTITY_NAMES.includes(name) || shown.has(name),
|
||||
);
|
||||
if (field === undefined || pointer === undefined) return;
|
||||
if (!this.#identity.has(type))
|
||||
this.#identity.set(type, { field, basis: "field-name" });
|
||||
this.#addPlan(candidate, pointer, recordings);
|
||||
@@ -500,39 +515,55 @@ export class EnvironmentStateExtraction {
|
||||
return this.#collections.get(JSON.stringify([type, collection])) ?? [];
|
||||
}
|
||||
|
||||
/** A written path whose read is not an entity holds one document value: its latest init recording. */
|
||||
/** A list tool without a plan or recorded reason holds one document value: its latest init recording, under one request line. */
|
||||
#documents(candidates: readonly Candidate[]): StateDocument[] {
|
||||
const written = new Set(
|
||||
this.#tools
|
||||
.filter(
|
||||
(tool) =>
|
||||
["POST", "PUT", "PATCH"].includes(tool.method) &&
|
||||
!isQueryPost(tool.method, tool.path_template),
|
||||
)
|
||||
.map((tool) => tool.path_template),
|
||||
);
|
||||
const documents: StateDocument[] = [];
|
||||
for (const { tool, item } of candidates) {
|
||||
const current = this.#views.get(tool.name);
|
||||
const entityless =
|
||||
current === undefined ||
|
||||
(current.shape === "recorded" &&
|
||||
(current.reason === "not-entity" ||
|
||||
current.reason === "no-identity"));
|
||||
if (item || !entityless || !written.has(tool.path_template)) continue;
|
||||
const latest = this.#latestPerLine(
|
||||
this.#initRecordings(tool, tool.path_template),
|
||||
);
|
||||
if (item || this.#views.has(tool.name)) continue;
|
||||
const recordings = this.#initRecordings(tool, tool.path_template);
|
||||
const latest = this.#latestPerLine(recordings);
|
||||
if (latest.length !== 1) {
|
||||
if (latest.length > 1) this.#recorded(tool, "query-varies");
|
||||
const paths = new Set(latest.map((record) => record.path));
|
||||
this.#recorded(
|
||||
tool,
|
||||
latest.length === 0
|
||||
? "no-init-recording"
|
||||
: paths.size > 1
|
||||
? "path-varies"
|
||||
: "query-varies",
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const record = latest[0]!;
|
||||
// A value that changed across a write, even under another root, is not one initial state.
|
||||
if (
|
||||
recordings.some(
|
||||
(after, index) =>
|
||||
index > 0 &&
|
||||
this.#writeWithin(recordings[index - 1]!, after) &&
|
||||
this.#withBody(recordings[index - 1]!.ordinal, (before) =>
|
||||
this.#withBody(after.ordinal, (body) => !same(before, body)),
|
||||
),
|
||||
)
|
||||
) {
|
||||
this.#recorded(tool, "fields-differ");
|
||||
continue;
|
||||
}
|
||||
const value = this.#withBody(record.ordinal, (body) => body);
|
||||
if (value === undefined) {
|
||||
this.#recorded(tool, "body-unusable");
|
||||
continue;
|
||||
}
|
||||
// Parsing can lengthen a body that was admitted under the recorded-text limit:
|
||||
// `1e3` is four bytes once rendered as `1000`. Serving refuses that rendering,
|
||||
// so an oversized document stays the recorded replay instead of a failing read.
|
||||
if (
|
||||
Buffer.byteLength(JSON.stringify(value), "utf8") >
|
||||
STATE_BODY_LIMIT_BYTES
|
||||
) {
|
||||
this.#recorded(tool, "rendered-oversize");
|
||||
continue;
|
||||
}
|
||||
this.#retain(
|
||||
ENTRY_BYTES + jsonBytes(value) + textBytes(record.path.length),
|
||||
);
|
||||
|
||||
@@ -64,8 +64,13 @@ export const mergeObservedEntity = (entity: ObservedEntity): StateEntity => {
|
||||
}))
|
||||
.filter((view) => Object.keys(view.fields).length > 0)
|
||||
.sort((left, right) => compare(left.tool, right.tool)),
|
||||
// Cited in timeline order, whichever plan observed them first.
|
||||
record_ordinals: [
|
||||
...new Set(entity.observations.map((observation) => observation.ordinal)),
|
||||
...new Set(
|
||||
entity.observations
|
||||
.toSorted((left, right) => left.position - right.position)
|
||||
.map((observation) => observation.ordinal),
|
||||
),
|
||||
],
|
||||
};
|
||||
};
|
||||
|
||||
@@ -42,12 +42,20 @@ export type RecordedViewReason =
|
||||
| "query-body"
|
||||
| "no-init-recording"
|
||||
| "body-unusable"
|
||||
| "not-entity"
|
||||
| "no-identity"
|
||||
| "path-varies"
|
||||
| "query-varies"
|
||||
| "fields-differ"
|
||||
| "envelope-varies"
|
||||
| "list-differs";
|
||||
| "list-differs"
|
||||
| "rendered-oversize";
|
||||
|
||||
/**
|
||||
* Recorded response text, and a document value rendered back to JSON, stay within this many UTF-8
|
||||
* bytes. A larger recorded body is not parsed; a parsed value whose JSON rendering exceeds it stays
|
||||
* a recorded replay (`rendered-oversize`) rather than a document the server would have to refuse.
|
||||
*/
|
||||
export const STATE_BODY_LIMIT_BYTES = 256 * 1024;
|
||||
|
||||
export interface StateType {
|
||||
readonly collection_template: string;
|
||||
@@ -314,29 +322,55 @@ export const hasPrototypeKey = (value: JsonValue): boolean =>
|
||||
|
||||
export const pointerTo = (name: string): string =>
|
||||
`/${name.replaceAll("~", "~0").replaceAll("/", "~1")}`;
|
||||
const pointerName = (pointer: string): string =>
|
||||
pointer.slice(1).replaceAll("~1", "/").replaceAll("~0", "~");
|
||||
/** The unescaped reference tokens of a pointer. */
|
||||
export const pointerTokens = (pointer: string): string[] =>
|
||||
pointer === ""
|
||||
? []
|
||||
: pointer
|
||||
.slice(1)
|
||||
.split("/")
|
||||
.map((token) => token.replaceAll("~1", "/").replaceAll("~0", "~"));
|
||||
|
||||
/** The value at a pointer of depth at most one. */
|
||||
const memberOf = (
|
||||
body: JsonValue | undefined,
|
||||
name: string,
|
||||
): JsonValue | undefined =>
|
||||
isObject(body) && Object.hasOwn(body, name) ? body[name] : undefined;
|
||||
|
||||
/** The element of a one-element array. */
|
||||
const soleElement = (value: JsonValue | undefined): JsonValue | undefined =>
|
||||
Array.isArray(value) && value.length === 1 ? value[0] : undefined;
|
||||
|
||||
/** The value at a pointer: the root, one member, or element 0 of a one-element array at the root or in one member. */
|
||||
export const at = (body: JsonValue, pointer: string): JsonValue | undefined => {
|
||||
if (pointer === "") return body;
|
||||
if (!isObject(body)) return undefined;
|
||||
const name = pointerName(pointer);
|
||||
return Object.hasOwn(body, name) ? body[name] : undefined;
|
||||
const [first, second, ...rest] = pointerTokens(pointer);
|
||||
if (first === undefined) return body;
|
||||
if (second !== undefined)
|
||||
return second === "0" && rest.length === 0
|
||||
? soleElement(memberOf(body, first))
|
||||
: undefined;
|
||||
return Array.isArray(body)
|
||||
? first === "0"
|
||||
? soleElement(body)
|
||||
: undefined
|
||||
: memberOf(body, first);
|
||||
};
|
||||
|
||||
/** A copy of `body` with the value at a pointer of depth at most one replaced. */
|
||||
/** A copy of `body` with the value `at` reads at the pointer replaced; unchanged when it reads none. */
|
||||
export const replaceAt = (
|
||||
body: JsonValue,
|
||||
pointer: string,
|
||||
value: JsonValue,
|
||||
): JsonValue => {
|
||||
if (pointer === "" || !isObject(body)) return value;
|
||||
const name = pointerName(pointer);
|
||||
const [first, second] = pointerTokens(pointer);
|
||||
if (first === undefined) return value;
|
||||
if (at(body, pointer) === undefined) return body;
|
||||
if (!isObject(body)) return [value];
|
||||
const replaced = second === undefined ? value : [value];
|
||||
return Object.fromEntries(
|
||||
Object.entries(body).map(([key, item]) => [
|
||||
key,
|
||||
key === name ? value : item,
|
||||
key === first ? replaced : item,
|
||||
]),
|
||||
);
|
||||
};
|
||||
@@ -344,20 +378,36 @@ export const replaceAt = (
|
||||
const matchingKeys = (object: JsonObject, value: string): string[] =>
|
||||
Object.keys(object).filter((key) => scalarText(object[key]) === value);
|
||||
|
||||
/** The entity object an item response carries, at the root or in one wrapper property, with the keys repeating its path value. */
|
||||
/** The entity object an item response carries with the keys repeating its path value: at the root or in one wrapper member, else as the element of a one-element array there. */
|
||||
export const locateItem = (
|
||||
body: JsonValue,
|
||||
value: string,
|
||||
): { readonly pointer: string; readonly keys: string[] } | undefined => {
|
||||
if (!isObject(body)) return undefined;
|
||||
const root = matchingKeys(body, value);
|
||||
if (root.length > 0) return { pointer: "", keys: root };
|
||||
const wrapped = Object.entries(body).flatMap(([name, item]) => {
|
||||
if (!isObject(item)) return [];
|
||||
const keys = matchingKeys(item, value);
|
||||
return keys.length > 0 ? [{ pointer: pointerTo(name), keys }] : [];
|
||||
});
|
||||
return wrapped.length === 1 ? wrapped[0] : undefined;
|
||||
const located = (
|
||||
object: JsonValue | undefined,
|
||||
pointer: string,
|
||||
named = false,
|
||||
) => {
|
||||
const keys = isObject(object)
|
||||
? matchingKeys(object, value).filter(
|
||||
(key) => !named || IDENTITY_NAMES.includes(key),
|
||||
)
|
||||
: [];
|
||||
return keys.length > 0 ? [{ pointer, keys }] : [];
|
||||
};
|
||||
const members = isObject(body) ? Object.entries(body) : [];
|
||||
// Objects take precedence; an array element needs an identity name, since a filtered list may
|
||||
// answer with one element that repeats its parent's value.
|
||||
const tiers = [
|
||||
located(body, ""),
|
||||
members.flatMap(([name, item]) => located(item, pointerTo(name))),
|
||||
located(soleElement(body), "/0", true),
|
||||
members.flatMap(([name, item]) =>
|
||||
located(soleElement(item), `${pointerTo(name)}/0`, true),
|
||||
),
|
||||
];
|
||||
const chosen = tiers.find((tier) => tier.length > 0);
|
||||
return chosen?.length === 1 ? chosen[0] : undefined;
|
||||
};
|
||||
|
||||
/** Whether every item carries `name` as a distinct string or number. */
|
||||
@@ -365,13 +415,34 @@ export const identifies = (
|
||||
items: readonly JsonObject[],
|
||||
name: string,
|
||||
): boolean => {
|
||||
const values = items.map((item) => scalarText(item[name]));
|
||||
return (
|
||||
values.every((value) => value !== undefined) &&
|
||||
new Set(values).size === values.length
|
||||
);
|
||||
// Stop at the first missing or repeated value, so a sparse list costs one read per field.
|
||||
const seen = new Set<string>();
|
||||
for (const item of items) {
|
||||
const value = scalarText(item[name]);
|
||||
if (value === undefined || seen.has(value)) return false;
|
||||
seen.add(value);
|
||||
}
|
||||
return true;
|
||||
};
|
||||
|
||||
/** Fields ending in one of these may identify listed items that no identity name identifies. */
|
||||
const IDENTITY_SUFFIX = /(?:Id|_id)$/u;
|
||||
|
||||
/** Keys of the first item, in its order, that end like an identity field and identify every item. */
|
||||
const suffixIdentities = (items: readonly JsonObject[]): string[] =>
|
||||
Object.keys(items[0] ?? {}).filter(
|
||||
(name) =>
|
||||
IDENTITY_SUFFIX.test(name) &&
|
||||
!IDENTITY_NAMES.includes(name) &&
|
||||
identifies(items, name),
|
||||
);
|
||||
|
||||
/** Fields that identify every listed item, in precedence order: identity names, then suffix-named fields. */
|
||||
export const identityCandidates = (items: readonly JsonObject[]): string[] => [
|
||||
...IDENTITY_NAMES.filter((name) => identifies(items, name)),
|
||||
...suffixIdentities(items),
|
||||
];
|
||||
|
||||
/** The array of entity objects a list response carries, at the root or in one property of an envelope that is not itself an entity. */
|
||||
export const locateList = (
|
||||
body: JsonValue,
|
||||
@@ -387,20 +458,24 @@ export const locateList = (
|
||||
(entry): entry is [string, JsonObject[]] =>
|
||||
Array.isArray(entry[1]) && entry[1].every(isObject),
|
||||
);
|
||||
const carrying = arrays.filter(
|
||||
([, items]) =>
|
||||
items.length > 0 &&
|
||||
(idField === undefined
|
||||
? IDENTITY_NAMES.some((name) => identifies(items, name))
|
||||
: identifies(items, idField)),
|
||||
);
|
||||
const chosen =
|
||||
carrying.length === 1
|
||||
? carrying[0]
|
||||
: carrying.length === 0 && arrays.length === 1
|
||||
? arrays[0]
|
||||
: undefined;
|
||||
return chosen === undefined ? undefined : pointerTo(chosen[0]);
|
||||
const carrying = (test: (items: JsonObject[]) => boolean) =>
|
||||
arrays.filter(([, items]) => items.length > 0 && test(items));
|
||||
// Identity names take precedence over suffix-named fields, which need two items to show
|
||||
// distinct values; with neither, the one array of objects.
|
||||
const tiers =
|
||||
idField === undefined
|
||||
? [
|
||||
carrying((items) =>
|
||||
IDENTITY_NAMES.some((name) => identifies(items, name)),
|
||||
),
|
||||
carrying(
|
||||
(items) => items.length > 1 && suffixIdentities(items).length > 0,
|
||||
),
|
||||
arrays,
|
||||
]
|
||||
: [carrying((items) => identifies(items, idField)), arrays];
|
||||
const chosen = tiers.find((entries) => entries.length > 0);
|
||||
return chosen?.length === 1 ? pointerTo(chosen[0]![0]) : undefined;
|
||||
};
|
||||
|
||||
export const listItems = (
|
||||
|
||||
@@ -19,6 +19,12 @@ import {
|
||||
type AnalysisViewResult,
|
||||
} from "./analysisView.js";
|
||||
import { EnvironmentStateAccumulator } from "./EnvironmentStateAccumulator.js";
|
||||
import {
|
||||
at,
|
||||
identityCandidates,
|
||||
replaceAt,
|
||||
type JsonObject,
|
||||
} from "./environmentStateRules.js";
|
||||
|
||||
type State = Extract<AnalysisViewResult, { kind: "environment-state" }>;
|
||||
const ALL_RULES = [
|
||||
@@ -112,7 +118,23 @@ it("extracts the state the Grafana-shaped capture's reads observed", () => {
|
||||
shape: "recorded",
|
||||
reason: "no-init-recording",
|
||||
});
|
||||
expect(state.documents).toEqual([]);
|
||||
// The one team member listed proves no suffix-named identity, so the list is a document of its path.
|
||||
expect(
|
||||
state.documents.map((document) => [document.tool, document.path]),
|
||||
).toEqual([
|
||||
[
|
||||
"get_datasources_health_by_uid",
|
||||
"/api/datasources/uid/dg1jo96tern5sb/health",
|
||||
],
|
||||
["get_org", "/api/org"],
|
||||
["get_teams_members_by_id", "/api/teams/1/members"],
|
||||
["get_user", "/api/user"],
|
||||
]);
|
||||
expect(viewOf(state, "get_plugins_settings_by_plugin_id")).toEqual({
|
||||
tool: "get_plugins_settings_by_plugin_id",
|
||||
shape: "recorded",
|
||||
reason: "path-varies",
|
||||
});
|
||||
expect(state.unknowns.map((unknown) => unknown.kind)).toEqual([
|
||||
"authorization-model",
|
||||
"response-semantics",
|
||||
@@ -367,22 +389,317 @@ it("holds a written path's non-entity read as a document", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
["/api/files/123", "PUT", "get_files_by_id"],
|
||||
["/api/settings", "DELETE", "get_settings"],
|
||||
] as const)(
|
||||
"leaves a deferred document path recorded (%s, %s)",
|
||||
(path, method, tool) => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", path, { text: "initial document" }],
|
||||
[method, path, null],
|
||||
]),
|
||||
);
|
||||
expect(state.documents).toEqual([]);
|
||||
expect(viewOf(state, tool)).toMatchObject({ shape: "recorded" });
|
||||
},
|
||||
);
|
||||
it("holds a read-only non-entity read as a document unless it varies by request line", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/config", { theme: "dark", locale: "en-GB" }],
|
||||
[
|
||||
"GET",
|
||||
"/api/rates",
|
||||
[
|
||||
{ currency: "EUR", rate: 1.1 },
|
||||
{ currency: "EUR", rate: 1.2 },
|
||||
],
|
||||
],
|
||||
["GET", "/api/stats?range=day", { visits: 3 }],
|
||||
["GET", "/api/stats?range=week", { visits: 9 }],
|
||||
]),
|
||||
);
|
||||
expect(state.documents).toEqual([
|
||||
{
|
||||
tool: "get_config",
|
||||
path: "/api/config",
|
||||
value: { theme: "dark", locale: "en-GB" },
|
||||
record_ordinal: 0,
|
||||
},
|
||||
{
|
||||
tool: "get_rates",
|
||||
path: "/api/rates",
|
||||
value: [
|
||||
{ currency: "EUR", rate: 1.1 },
|
||||
{ currency: "EUR", rate: 1.2 },
|
||||
],
|
||||
record_ordinal: 1,
|
||||
},
|
||||
]);
|
||||
expect(viewOf(state, "get_stats")).toEqual({
|
||||
tool: "get_stats",
|
||||
shape: "recorded",
|
||||
reason: "query-varies",
|
||||
});
|
||||
expect(state.types).toEqual([]);
|
||||
});
|
||||
|
||||
it("leaves a document recorded when rendering it exceeds the body limit", () => {
|
||||
const small = stateOf(
|
||||
captureOf([["GET", "/api/config", '{"values":[1e3]}']]),
|
||||
);
|
||||
expect(viewOf(small, "get_config")).toMatchObject({ shape: "document" });
|
||||
expect(small.documents[0]?.value).toEqual({ values: [1000] });
|
||||
// 60,000 `1e3` tokens are 240,012 recorded bytes and 300,012 once rendered as `1000`.
|
||||
const values = Array.from({ length: 60_000 }, () => "1e3").join(",");
|
||||
const body = `{"values":[${values}]}`;
|
||||
expect(Buffer.byteLength(body, "utf8")).toBe(240_012);
|
||||
const state = stateOf(captureOf([["GET", "/api/config", body]]));
|
||||
expect(state.documents).toEqual([]);
|
||||
expect(viewOf(state, "get_config")).toEqual({
|
||||
tool: "get_config",
|
||||
shape: "recorded",
|
||||
reason: "rendered-oversize",
|
||||
});
|
||||
});
|
||||
|
||||
it("infers a suffix-named identity only when no identity name identifies the listed items", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
[
|
||||
"GET",
|
||||
"/api/memberships",
|
||||
[
|
||||
{ orgId: 7, userId: 1, role: "Admin" },
|
||||
{ orgId: 8, userId: 1, role: "Viewer" },
|
||||
],
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"/api/seats",
|
||||
{
|
||||
total: 2,
|
||||
seats: [
|
||||
{ parentId: 1, seat_id: "s-1" },
|
||||
{ parentId: 1, seat_id: "s-2" },
|
||||
],
|
||||
},
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"/api/projects",
|
||||
[
|
||||
{ projectId: "a", id: 2 },
|
||||
{ projectId: "b", id: 3 },
|
||||
],
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"/api/grants",
|
||||
[
|
||||
{ userId: 1, orgId: 7 },
|
||||
{ userId: 2, orgId: 8 },
|
||||
],
|
||||
],
|
||||
["GET", "/api/members", []],
|
||||
["GET", "/api/members", [{ memberId: "m-1" }, { memberId: "m-2" }]],
|
||||
["GET", "/api/comments", [{ postId: 9, authorId: 3, text: "hi" }]],
|
||||
]),
|
||||
);
|
||||
expect(
|
||||
state.types.map(({ collection_template, id_field, id_basis }) => [
|
||||
collection_template,
|
||||
id_field,
|
||||
id_basis,
|
||||
]),
|
||||
).toEqual([
|
||||
// Both identify the grants; the earlier key in item order wins.
|
||||
["/api/grants", "userId", "field-name"],
|
||||
// The empty first recording constrains no field.
|
||||
["/api/members", "memberId", "field-name"],
|
||||
["/api/memberships", "orgId", "field-name"],
|
||||
["/api/projects", "id", "field-name"],
|
||||
["/api/seats", "seat_id", "field-name"],
|
||||
]);
|
||||
expect(viewOf(state, "get_seats")).toMatchObject({
|
||||
shape: "list",
|
||||
pointer: "/seats",
|
||||
membership: "all",
|
||||
counters: ["total"],
|
||||
});
|
||||
expect(viewOf(state, "get_members")).toMatchObject({ shape: "list" });
|
||||
expect(
|
||||
state.entities
|
||||
.filter((entity) => entity.type === "/api/memberships")
|
||||
.map((entity) => entity.id),
|
||||
).toEqual(["7", "8"]);
|
||||
// One listed item shows no field distinct, so its foreign keys identify nothing.
|
||||
expect(viewOf(state, "get_comments")).toMatchObject({ shape: "document" });
|
||||
});
|
||||
|
||||
it("chooses an envelope's array by identity name, then by a suffix-named field", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
[
|
||||
"GET",
|
||||
"/api/board",
|
||||
{
|
||||
labels: [{ text: "urgent" }, { text: "later" }],
|
||||
cards: [
|
||||
{ cardId: 1, text: "Granite Ledger" },
|
||||
{ cardId: 2, text: "Blue Heron" },
|
||||
],
|
||||
},
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"/api/directory",
|
||||
{
|
||||
owners: [{ orgId: 1 }, { orgId: 2 }],
|
||||
teams: [
|
||||
{ id: 1, orgId: 1 },
|
||||
{ id: 2, orgId: 1 },
|
||||
],
|
||||
},
|
||||
],
|
||||
]),
|
||||
);
|
||||
expect(
|
||||
state.types.map(({ collection_template, id_field }) => [
|
||||
collection_template,
|
||||
id_field,
|
||||
]),
|
||||
).toEqual([
|
||||
["/api/board", "cardId"],
|
||||
["/api/directory", "id"],
|
||||
]);
|
||||
expect(viewOf(state, "get_board")).toMatchObject({
|
||||
shape: "list",
|
||||
pointer: "/cards",
|
||||
});
|
||||
expect(viewOf(state, "get_directory")).toMatchObject({
|
||||
shape: "list",
|
||||
pointer: "/teams",
|
||||
});
|
||||
});
|
||||
|
||||
it("locates an item entity wrapped in a one-element array and merges it with the list view", () => {
|
||||
const first = "6ad47b5c1023b500010873e3";
|
||||
const second = "6ad47b5c1023b500010873e4";
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
[
|
||||
"GET",
|
||||
"/ghost/api/admin/posts/",
|
||||
{
|
||||
posts: [
|
||||
{ id: first, title: "Granite Ledger", status: "draft" },
|
||||
{ id: second, title: "Blue Heron", status: "published" },
|
||||
],
|
||||
meta: { pagination: { page: 1, pages: 1 } },
|
||||
},
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
`/ghost/api/admin/posts/${first}/`,
|
||||
{
|
||||
posts: [
|
||||
{
|
||||
id: first,
|
||||
title: "Granite Ledger",
|
||||
status: "draft",
|
||||
html: "<p>Quarry count</p>",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
["GET", "/api/depots/5", [{ id: 5, name: "Saltmarsh Depot" }]],
|
||||
["GET", "/api/rosters/9", [{ id: 31, roster: 9, name: "Ada Finch" }]],
|
||||
]),
|
||||
);
|
||||
expect(viewOf(state, "get_rosters_by_id")).toEqual({
|
||||
tool: "get_rosters_by_id",
|
||||
shape: "recorded",
|
||||
reason: "no-identity",
|
||||
});
|
||||
const posts = "/ghost/api/admin/posts";
|
||||
expect(state.types).toContainEqual({
|
||||
collection_template: posts,
|
||||
id_field: "id",
|
||||
id_basis: "path-parameter",
|
||||
init_before_ordinal: null,
|
||||
volatile_fields: [],
|
||||
});
|
||||
expect(state.views).toContainEqual(
|
||||
expect.objectContaining({
|
||||
shape: "item",
|
||||
type: posts,
|
||||
pointer: "/posts/0",
|
||||
fields: ["id", "title", "status", "html"],
|
||||
checked_record_ordinals: [1],
|
||||
}),
|
||||
);
|
||||
expect(state.views).toContainEqual(
|
||||
expect.objectContaining({
|
||||
shape: "list",
|
||||
type: posts,
|
||||
pointer: "/posts",
|
||||
membership: "all",
|
||||
}),
|
||||
);
|
||||
expect(state.views).toContainEqual(
|
||||
expect.objectContaining({
|
||||
shape: "item",
|
||||
type: "/api/depots",
|
||||
pointer: "/0",
|
||||
}),
|
||||
);
|
||||
expect(state.entities.filter((entity) => entity.type === posts)).toEqual([
|
||||
{
|
||||
type: posts,
|
||||
collection: posts,
|
||||
id: first,
|
||||
fields: {
|
||||
id: first,
|
||||
title: "Granite Ledger",
|
||||
status: "draft",
|
||||
html: "<p>Quarry count</p>",
|
||||
},
|
||||
view_fields: [],
|
||||
record_ordinals: [0, 1],
|
||||
},
|
||||
expect.objectContaining({ id: second, record_ordinals: [0] }),
|
||||
]);
|
||||
});
|
||||
|
||||
it("reads and replaces only the root, one member or the element of a one-element array", () => {
|
||||
const body = { posts: [{ id: 1 }], meta: { posts: 1 } };
|
||||
expect(at(body, "/posts/0")).toEqual({ id: 1 });
|
||||
expect(replaceAt(body, "/posts/0", null)).toEqual({
|
||||
posts: [null],
|
||||
meta: { posts: 1 },
|
||||
});
|
||||
expect(at([{ id: 1 }], "/0")).toEqual({ id: 1 });
|
||||
expect(replaceAt([{ id: 1 }], "/0", { id: 2 })).toEqual([{ id: 2 }]);
|
||||
for (const pointer of ["/posts/1", "/meta/posts", "/posts/0/id"])
|
||||
expect(at(body, pointer)).toBeUndefined();
|
||||
expect(at({ posts: [{ id: 1 }, { id: 2 }] }, "/posts/0")).toBeUndefined();
|
||||
expect(replaceAt(body, "/meta/posts", null)).toBe(body);
|
||||
});
|
||||
|
||||
it("leaves an item path's non-entity read recorded", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/files/123", { text: "initial document" }],
|
||||
["PUT", "/api/files/123", null],
|
||||
]),
|
||||
);
|
||||
expect(state.documents).toEqual([]);
|
||||
expect(viewOf(state, "get_files_by_id")).toMatchObject({ shape: "recorded" });
|
||||
});
|
||||
|
||||
it("holds a deleted path's initial non-entity read as a document", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/settings", { text: "initial document" }],
|
||||
["DELETE", "/api/settings", null],
|
||||
]),
|
||||
);
|
||||
expect(state.documents).toEqual([
|
||||
{
|
||||
tool: "get_settings",
|
||||
path: "/api/settings",
|
||||
value: { text: "initial document" },
|
||||
record_ordinal: 0,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("orders a scenario capture by request order and needs its response bodies", () => {
|
||||
const state = stateOf(parentOf(scenarioCaptureEvidence()));
|
||||
@@ -459,11 +776,11 @@ it("ends every type's state at a write whose URL was redacted, and says so", ()
|
||||
);
|
||||
});
|
||||
|
||||
it("infers no identity for a list whose recordings held no items", () => {
|
||||
it("infers no identity for a list whose recordings held no items and holds it as a document", () => {
|
||||
const state = stateOf(captureOf([["GET", "/api/widgets", []]]));
|
||||
expect(state.types).toEqual([]);
|
||||
expect(state.views).toEqual([
|
||||
{ tool: "get_widgets", shape: "recorded", reason: "no-identity" },
|
||||
{ tool: "get_widgets", shape: "document", checked_record_ordinals: [0] },
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -514,8 +831,9 @@ it("does not take a later recording that is itself an entity as the list", () =>
|
||||
],
|
||||
]),
|
||||
);
|
||||
// Without one list across its recordings, the read is held as a document, not entities.
|
||||
expect(state.views).toContainEqual(
|
||||
expect.objectContaining({ tool: "get_items", shape: "recorded" }),
|
||||
expect.objectContaining({ tool: "get_items", shape: "document" }),
|
||||
);
|
||||
expect(state.entities).toEqual([]);
|
||||
});
|
||||
@@ -763,3 +1081,57 @@ it("reads only item totals and next-page markers as signs of a partial list", ()
|
||||
["recorded", []],
|
||||
]);
|
||||
});
|
||||
|
||||
it("reads a sparse list once per candidate field when inferring identity", () => {
|
||||
const fields = 2_000;
|
||||
const wide: JsonObject = Object.fromEntries(
|
||||
Array.from({ length: fields }, (_, index) => [
|
||||
`f${String(index)}Id`,
|
||||
index,
|
||||
]),
|
||||
);
|
||||
let reads = 0;
|
||||
const sparse = Array.from(
|
||||
{ length: 5_000 },
|
||||
() =>
|
||||
new Proxy<JsonObject>(
|
||||
{},
|
||||
{
|
||||
get: (target, key, receiver) => {
|
||||
reads += 1;
|
||||
return Reflect.get(target, key, receiver) as unknown;
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
expect(identityCandidates([wide, ...sparse])).toEqual([]);
|
||||
// Each identity name and suffix field stops at the first sparse item.
|
||||
expect(reads).toBeLessThanOrEqual(fields + 20);
|
||||
});
|
||||
|
||||
it("keeps a document out of the state when its value changed across a write under another root", () => {
|
||||
const changed = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/me", { credits: 10, plan: "free" }],
|
||||
["POST", "/api/orders", { id: "o1" }],
|
||||
["GET", "/api/me", { credits: 9, plan: "free" }],
|
||||
]),
|
||||
);
|
||||
expect(changed.documents).toEqual([]);
|
||||
expect(changed.views).toContainEqual({
|
||||
tool: "get_me",
|
||||
shape: "recorded",
|
||||
reason: "fields-differ",
|
||||
});
|
||||
// A value the write left unchanged is still one initial state.
|
||||
const unchanged = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/me", { credits: 10, plan: "free" }],
|
||||
["POST", "/api/orders", { id: "o1" }],
|
||||
["GET", "/api/me", { credits: 10, plan: "free" }],
|
||||
]),
|
||||
);
|
||||
expect(unchanged.documents).toMatchObject([
|
||||
{ tool: "get_me", value: { credits: 10, plan: "free" }, record_ordinal: 2 },
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -32,7 +32,7 @@ const entityViewFields = {
|
||||
pointer: z
|
||||
.string()
|
||||
.describe(
|
||||
"JSON pointer, of depth at most one, to the entity object or array within the response.",
|
||||
"JSON pointer to the entity object or array within the response: the root or one member, or for an item the element of a one-element array there (/0 or /<member>/0).",
|
||||
),
|
||||
fields: z
|
||||
.array(z.string())
|
||||
@@ -76,12 +76,13 @@ const viewSchema = z.discriminatedUnion("shape", [
|
||||
"query-body",
|
||||
"no-init-recording",
|
||||
"body-unusable",
|
||||
"not-entity",
|
||||
"no-identity",
|
||||
"path-varies",
|
||||
"query-varies",
|
||||
"fields-differ",
|
||||
"envelope-varies",
|
||||
"list-differs",
|
||||
"rendered-oversize",
|
||||
]),
|
||||
}),
|
||||
]);
|
||||
@@ -103,7 +104,7 @@ export const environmentStateFields = {
|
||||
id_basis: z
|
||||
.enum(["path-parameter", "field-name"])
|
||||
.describe(
|
||||
"path-parameter: item responses repeat the requested path value in this field; field-name: an inferred identity name every listed item carries distinctly.",
|
||||
"path-parameter: item responses repeat the requested path value in this field; field-name: inferred from listed items as the first identity name, else the first field ending in Id or _id shown distinct across at least two items, that every listed item carries distinctly.",
|
||||
),
|
||||
init_before_ordinal: z
|
||||
.number()
|
||||
|
||||
@@ -48,17 +48,18 @@ it("charges environment identity, aggregate tools and escaped text before produc
|
||||
query_parameter_names: [],
|
||||
}));
|
||||
expect(
|
||||
environmentToolDefinitions({ ...source.manifest, tools }, source.evidenceId)
|
||||
.ok,
|
||||
environmentToolDefinitions({
|
||||
...source,
|
||||
manifest: { ...source.manifest, tools },
|
||||
}).ok,
|
||||
).toBe(false);
|
||||
expect(
|
||||
environmentToolDefinitions(
|
||||
source.manifest,
|
||||
source.evidenceId,
|
||||
source,
|
||||
"x".repeat(ENVIRONMENT_CATALOG_LIMIT_BYTES),
|
||||
).ok,
|
||||
).toBe(false);
|
||||
const normal = environmentToolDefinitions(source.manifest, source.evidenceId);
|
||||
const normal = environmentToolDefinitions(source);
|
||||
expect(normal.ok).toBe(true);
|
||||
if (!normal.ok) throw normal.error;
|
||||
expect(
|
||||
@@ -74,12 +75,9 @@ it("reserves the state card only when the state data plane is served", () => {
|
||||
const source = environmentServeSource();
|
||||
const manifest = { ...source.manifest, tools: [] };
|
||||
const admits = (length: number, dataPlane: "none" | "state") =>
|
||||
environmentToolDefinitions(
|
||||
manifest,
|
||||
source.evidenceId,
|
||||
"x".repeat(length),
|
||||
{ dataPlane },
|
||||
).ok;
|
||||
environmentToolDefinitions({ ...source, manifest }, "x".repeat(length), {
|
||||
dataPlane,
|
||||
}).ok;
|
||||
// With no tools, the name and the card are the whole reservation: six bytes per name unit plus quotes.
|
||||
const longest = (dataPlane: "none" | "state") =>
|
||||
Math.floor(
|
||||
|
||||
@@ -38,10 +38,7 @@ const serveEnvironment = async (
|
||||
if (!result.ok) throw result.error;
|
||||
return result.value;
|
||||
};
|
||||
const definitions = environmentToolDefinitions(
|
||||
source.manifest,
|
||||
source.evidenceId,
|
||||
);
|
||||
const definitions = environmentToolDefinitions(source);
|
||||
|
||||
/**
|
||||
* The agentenv echo example's card, key for key. Only `name` and the tool
|
||||
@@ -400,6 +397,70 @@ describe("environment catalog limits", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("environment tool annotations", () => {
|
||||
it("advertises method hints with closed-world answers over MCP and keeps them off the card", async () => {
|
||||
const folders = source.manifest.tools.find(
|
||||
(tool) => tool.name === "post_folders",
|
||||
);
|
||||
if (folders === undefined) throw new Error("Missing fixture folders tool");
|
||||
const search = {
|
||||
...folders,
|
||||
name: "post_search",
|
||||
path_template: "/api/search",
|
||||
};
|
||||
const entry = await serveEnvironment({
|
||||
source: {
|
||||
...source,
|
||||
manifest: {
|
||||
...source.manifest,
|
||||
tools: [...source.manifest.tools, search],
|
||||
},
|
||||
},
|
||||
host: "127.0.0.1",
|
||||
port: 0,
|
||||
});
|
||||
served.push(entry);
|
||||
const client = new Client({ name: "tool-hints", version: "1" });
|
||||
const transport = new StreamableHTTPClientTransport(new URL(entry.url));
|
||||
await client.connect(transport);
|
||||
try {
|
||||
const { tools } = await client.listTools();
|
||||
const hints = new Map(
|
||||
tools.map(({ name, annotations }) => [name, annotations]),
|
||||
);
|
||||
const read = { readOnlyHint: true, openWorldHint: false };
|
||||
expect(hints.get("get_users_by_id")).toEqual({
|
||||
...read,
|
||||
idempotentHint: true,
|
||||
});
|
||||
expect(hints.get("post_search")).toEqual({
|
||||
...read,
|
||||
idempotentHint: false,
|
||||
});
|
||||
expect(hints.get("post_folders")).toEqual({
|
||||
readOnlyHint: false,
|
||||
idempotentHint: false,
|
||||
openWorldHint: false,
|
||||
});
|
||||
expect(hints.get("put_folders_by_uid")).toEqual({
|
||||
readOnlyHint: false,
|
||||
idempotentHint: true,
|
||||
openWorldHint: false,
|
||||
});
|
||||
expect(hints.get("delete_teams_by_id")).toEqual({
|
||||
readOnlyHint: false,
|
||||
destructiveHint: true,
|
||||
idempotentHint: true,
|
||||
openWorldHint: false,
|
||||
});
|
||||
expect(JSON.stringify(entry.card)).not.toContain("Hint");
|
||||
} finally {
|
||||
await transport.terminateSession();
|
||||
await client.close();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("serveEnvironment", () => {
|
||||
it("serves the card, lists one MCP tool per manifest tool and replays target content with provenance", async () => {
|
||||
const entry = await start();
|
||||
|
||||
@@ -115,6 +115,7 @@ export const createEnvironmentMcpServer = (
|
||||
{
|
||||
description: definition.description,
|
||||
inputSchema: fromJsonSchema<ReplayArguments>(definition.inputSchema),
|
||||
annotations: definition.annotations,
|
||||
},
|
||||
(args): CallToolResult => {
|
||||
const replayed = answer(definition.tool, args);
|
||||
|
||||
@@ -20,9 +20,9 @@ import {
|
||||
|
||||
const HTTP_BOUNDARY_TIMEOUT_MS = 90_000;
|
||||
const CARD_PATH = "/.well-known/agent-env.json";
|
||||
/** SHA-256 of the 11,131-byte card main serves for the fixture named "renamed"; pure replay must serve the same bytes. */
|
||||
const MAIN_REPLAY_CARD_SHA256 =
|
||||
"f12f16011feb5f675b2bbf6a04a8b28a2ed8041074e4fe188cf74e7a66b15207";
|
||||
/** SHA-256 of the 12,818-byte card served for the fixture named "renamed", its descriptions naming each tool's recorded responses; pure replay must serve these bytes. */
|
||||
const REPLAY_CARD_SHA256 =
|
||||
"7c65ab63f0c00ce79b570024c4611c225729677b44fc49bbde32b2d65f868961";
|
||||
|
||||
/** Serve the fixture pair as pure replay through the built CLI. */
|
||||
const serveReplayOnly = async (
|
||||
@@ -69,6 +69,15 @@ describe("rea environment-serve --replay-only", () => {
|
||||
expect(tools.map(({ name }) => name).sort()).toEqual(
|
||||
source.manifest.tools.map(({ name }) => name).sort(),
|
||||
);
|
||||
const users = tools.find(({ name }) => name === "get_users_by_id");
|
||||
expect(users?.description).toContain(
|
||||
'; returns a JSON object with fields "id", "uid", "login", "email"; recorded statuses 200;',
|
||||
);
|
||||
expect(users?.annotations).toEqual({
|
||||
readOnlyHint: true,
|
||||
idempotentHint: true,
|
||||
openWorldHint: false,
|
||||
});
|
||||
const exact = await client.callTool({
|
||||
name: "get_users_by_id",
|
||||
arguments: { id: "2" },
|
||||
@@ -122,7 +131,7 @@ describe("rea environment-serve --replay-only", () => {
|
||||
expect(card.status).toBe(200);
|
||||
const text = await card.text();
|
||||
expect(createHash("sha256").update(text).digest("hex")).toBe(
|
||||
MAIN_REPLAY_CARD_SHA256,
|
||||
REPLAY_CARD_SHA256,
|
||||
);
|
||||
const body: unknown = JSON.parse(text);
|
||||
expect(body).toMatchObject({
|
||||
|
||||
Reference in New Issue
Block a user