mirror of
https://github.com/morluto/rea
synced 2026-10-26 06:16:17 -04:00
feat(browser): retain compact scenario summaries and selected views (#7120)
Retain complete browser scenario Evidence while delivering compact summaries and selected step, event, transaction and text views through shared CLI/MCP workflows. Preserve initial CLI failures across cleanup retries and validate with real Chromium/TodoMVC. Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
gensokyo-dev
@@ -86,6 +86,8 @@ Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
For multi-step flows with trees or events, select `detail: "summary"` and page
|
||||
exact steps, transactions, text slices or events with `inspect_analysis_view` kind `browser-scenario`.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
|
||||
@@ -621,3 +621,58 @@ rules and preserve the same missing fact.
|
||||
Network normalization also commits the removal of capture-local transaction IDs,
|
||||
attribution to the requesting step, and unordered transaction comparison. These
|
||||
built-in rule identities participate in the normalization digest.
|
||||
|
||||
## Summary delivery and retained views
|
||||
|
||||
`capture_browser_scenario` accepts `detail: "summary"` to retain the complete
|
||||
capture in the MCP session and return its summary. The default remains
|
||||
`complete`. Delivery detail does not change scenario parameters or identity.
|
||||
|
||||
The summary reports each step's status, error, URL and captured/missing/truncated
|
||||
section names. It omits `not_requested` sections and repeated `before_url` values.
|
||||
The accessibility tree comes from the first unsuccessful step, or the final
|
||||
step on success; when that step captured no tree, its nearest earlier captured
|
||||
tree is identified explicitly. Event and network counts cover retained records,
|
||||
with capture omissions reported separately.
|
||||
|
||||
For the one-shot CLI, save the complete Evidence before selecting a summary:
|
||||
|
||||
```bash
|
||||
rea capture-browser-scenario scenario.json --detail summary --evidence_output /absolute/path/scenario-evidence.json --json
|
||||
```
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view` source `{kind: "retained-evidence", evidence_id: "..."}`.
|
||||
For a CLI bundle, use an `evidence-file` source with its absolute path and the
|
||||
parent Evidence ID. Views read the recorded capture; they never replay actions.
|
||||
|
||||
- `{kind: "summary"}` returns the same overview.
|
||||
- `{kind: "browser-scenario", collection: "steps", step_indices: [2], artifacts: ["accessibility"], offset: 0, limit: 1}` returns selected recorded steps and artifacts.
|
||||
- `{kind: "browser-scenario", collection: "events", event_kinds: ["response"], transaction_ids: ["request-2"], offset: 0, limit: 20}` pages selected events with original item indices.
|
||||
|
||||
Omitted filters select all; empty arrays select none. Pagination counts matching
|
||||
retained rows. Exhausting a page does not establish capture completeness or
|
||||
current application state.
|
||||
|
||||
Transaction pages use `{kind: "browser-scenario", collection: "transactions", limit: 20}`.
|
||||
Their stable identity is the parent Evidence ID plus `transaction_id`; each row
|
||||
keeps original event indices. Metadata events are returned by default. Select
|
||||
`event_kinds: ["network-content"]` for recorded bodies and header values, or
|
||||
`event_kinds: []` for identity and indices alone. A `step_indices` filter selects
|
||||
a whole transaction when any retained event belongs to a selected step; this
|
||||
is receipt attribution, not proof that the action caused the request.
|
||||
|
||||
For a large retained tree, request `{kind: "browser-scenario", collection: "text",
|
||||
step_index: 10, artifact: "accessibility", offset: 0, limit: 2048}`. `dom` is also
|
||||
supported. Offsets and limits count UTF-16 code units. The full artifact digest
|
||||
and byte size remain attached to each page. Missing, truncated and unrequested
|
||||
text stay explicit; source truncation counts are preserved without treating an
|
||||
exhausted page as a complete capture.
|
||||
|
||||
Network summary counts include every retained transaction identity, including
|
||||
those whose request event was dropped. Missing request events, conflicting
|
||||
metadata and observed request failures have separate counts. `transactions` is
|
||||
`null` when network observation was unrequested or its legacy selection is
|
||||
unknown and there are no retained transactions; an observed empty inventory is
|
||||
`0`. Full capture omissions remain available and each page carries source
|
||||
completeness, dropped-event counts and omitted-transaction accounting.
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
# Browser scenario summary verification
|
||||
|
||||
The verifier runs the actual compiled CLI/MCP and caller-selected Chromium against
|
||||
TodoMVC's JavaScript ES5 distribution served on loopback. Ten actions add five
|
||||
todos. Every step captures URL, DOM, accessibility and a screenshot; the oracle
|
||||
checks all completed steps, the five authored labels, retained events and
|
||||
transaction identities. Summary runs also read step, text and transaction pages
|
||||
and authenticate an exported complete parent.
|
||||
|
||||
Download the public distribution at the pinned revision into a disposable folder:
|
||||
|
||||
```bash
|
||||
export REA_TODOMVC_APP="$(mktemp -d)"
|
||||
python3 - "$REA_TODOMVC_APP" <<'PY'
|
||||
import hashlib, json, sys, urllib.request
|
||||
from pathlib import Path
|
||||
root = Path(sys.argv[1])
|
||||
revision = "ff43b02e59dfa604386bb382034b2cd07c2bcd8a"
|
||||
files = ["index.html", "app.js", "base.css", "base.js", "controller.js",
|
||||
"helpers.js", "index.css", "model.js", "store.js", "template.js", "view.js"]
|
||||
manifest = []
|
||||
for name in files:
|
||||
url = f"https://raw.githubusercontent.com/tastejs/todomvc/{revision}/examples/javascript-es5/dist/{name}"
|
||||
with urllib.request.urlopen(url, timeout=30) as reply:
|
||||
data = reply.read()
|
||||
(root / name).write_bytes(data)
|
||||
manifest.append({"path": name, "url": url, "bytes": len(data),
|
||||
"sha256": hashlib.sha256(data).hexdigest()})
|
||||
(root / "provenance.json").write_text(json.dumps({"revision": revision, "files": manifest}, indent=2))
|
||||
PY
|
||||
```
|
||||
|
||||
Build each selected checkout with the repository's pinned Node/npm toolchain.
|
||||
Use new output directories; exports never overwrite an earlier run:
|
||||
|
||||
```bash
|
||||
node experiments/analysis-efficiency/browser-scenario-summary.mjs \
|
||||
--repository /absolute/path/to/baseline --mode complete \
|
||||
--application "$REA_TODOMVC_APP" --browser /absolute/path/to/chromium \
|
||||
--output /absolute/path/to/new-baseline-report
|
||||
node experiments/analysis-efficiency/browser-scenario-summary.mjs \
|
||||
--mode summary --application "$REA_TODOMVC_APP" \
|
||||
--browser /absolute/path/to/chromium \
|
||||
--output /absolute/path/to/new-summary-report
|
||||
```
|
||||
|
||||
`cold` is the first capture in a fresh MCP process; `warm` repeats the capture in
|
||||
that process with a new owned browser/profile. Neither label establishes cold OS
|
||||
file caches. Summary mode intentionally performs additional retained queries.
|
||||
Compare equivalent answers and include those calls when reporting task cost;
|
||||
smaller replies alone do not establish a faster task.
|
||||
|
||||
Add `--verify-boundaries` to a separate summary run for a real failed-step focus,
|
||||
CLI summary publication, and MCP text readback from the CLI's complete bundle.
|
||||
Those extra checks make its whole-run resource cost unsuitable for comparison
|
||||
against a complete-only run.
|
||||
|
||||
Reports retain the actual catalog, arguments, MCP text and complete Evidence.
|
||||
When estimating tokens, name the tokenizer and count discovery, arguments and
|
||||
all replies needed for the chosen task. Raw serialized catalog estimates are
|
||||
not actual model usage or a measurement of a client's rendered tool definitions.
|
||||
Measure CPU and peak process-family memory externally, state the accounting
|
||||
method, and include cleanup; the verifier's elapsed phase excludes final teardown.
|
||||
@@ -0,0 +1,412 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:http";
|
||||
import { readFile, writeFile, mkdir, readdir } from "node:fs/promises";
|
||||
import { resolve, join, extname } from "node:path";
|
||||
import { pathToFileURL, fileURLToPath } from "node:url";
|
||||
import { createRequire } from "node:module";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify, parseArgs } from "node:util";
|
||||
// A developer verifier: the caller selects the pinned TodoMVC distribution,
|
||||
// browser executable, output directory and baseline/final repository checkout.
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
repository: {
|
||||
type: "string",
|
||||
default: fileURLToPath(new URL("../../", import.meta.url)),
|
||||
},
|
||||
mode: { type: "string", default: "summary" },
|
||||
application: { type: "string" },
|
||||
browser: { type: "string" },
|
||||
output: { type: "string" },
|
||||
port: { type: "string", default: "0" },
|
||||
"verify-boundaries": { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
assert.ok(
|
||||
values.application && values.browser && values.output,
|
||||
"Pass --application <TodoMVC javascript-es5 dist> --browser <executable> --output <new directory>",
|
||||
);
|
||||
const repo = resolve(values.repository);
|
||||
const mode = values.mode;
|
||||
const output = resolve(values.output);
|
||||
const appRoot = resolve(values.application);
|
||||
const executablePath = resolve(values.browser);
|
||||
assert.ok(["complete", "summary"].includes(mode));
|
||||
assert.ok(
|
||||
!values["verify-boundaries"] || mode === "summary",
|
||||
"--verify-boundaries requires summary mode",
|
||||
);
|
||||
process.chdir(repo);
|
||||
const require = createRequire(join(repo, "package.json"));
|
||||
const { Client } = await import(
|
||||
pathToFileURL(require.resolve("@modelcontextprotocol/client")).href
|
||||
);
|
||||
const { StdioClientTransport } = await import(
|
||||
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio")).href
|
||||
);
|
||||
await mkdir(output);
|
||||
const files = await readdir(appRoot);
|
||||
const server = createServer(async (request, response) => {
|
||||
try {
|
||||
const name =
|
||||
decodeURIComponent(
|
||||
new URL(request.url, "http://127.0.0.1").pathname,
|
||||
).slice(1) || "index.html";
|
||||
assert.ok(files.includes(name));
|
||||
const data = await readFile(join(appRoot, name));
|
||||
response.setHeader(
|
||||
"Content-Type",
|
||||
{ ".html": "text/html", ".js": "text/javascript", ".css": "text/css" }[
|
||||
extname(name)
|
||||
] ?? "application/octet-stream",
|
||||
);
|
||||
response.end(data);
|
||||
} catch {
|
||||
response.writeHead(404);
|
||||
response.end();
|
||||
}
|
||||
});
|
||||
await new Promise((done, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(Number(values.port), "127.0.0.1", done);
|
||||
});
|
||||
const address = server.address();
|
||||
assert.ok(address && typeof address === "object");
|
||||
const origin = `http://127.0.0.1:${address.port}`;
|
||||
const environment = {
|
||||
PATH: process.env.PATH,
|
||||
NODE_OPTIONS: "--max-old-space-size=3072",
|
||||
UV_THREADPOOL_SIZE: "2",
|
||||
REA_LOG_LEVEL: "silent",
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
};
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [join(repo, "scripts/rea.mjs"), "mcp"],
|
||||
cwd: repo,
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
let stderr = "";
|
||||
transport.stderr?.on("data", (chunk) => {
|
||||
if (stderr.length < 65536) stderr += chunk;
|
||||
});
|
||||
const client = new Client({
|
||||
name: "rea-browser-scenario-summary-verifier",
|
||||
version: "1",
|
||||
});
|
||||
const calls = [];
|
||||
const invoke = async (name, args, tools) => {
|
||||
const toolDefinition = tools.find((tool) => tool.name === name);
|
||||
assert.ok(
|
||||
toolDefinition?.outputSchema,
|
||||
`Missing named output schema: ${name}`,
|
||||
);
|
||||
const started = performance.now();
|
||||
const result = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ toolDefinition, timeout: 90000 },
|
||||
);
|
||||
calls.push({
|
||||
name,
|
||||
arguments: args,
|
||||
content: result.content,
|
||||
elapsed_ms: performance.now() - started,
|
||||
});
|
||||
assert.notEqual(result.isError, true, JSON.stringify(result));
|
||||
return result.structuredContent;
|
||||
};
|
||||
const allStarted = performance.now();
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const discoveryStarted = performance.now();
|
||||
const catalog = await client.listTools();
|
||||
const discoveryMs = performance.now() - discoveryStarted;
|
||||
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
|
||||
const actions = Array.from({ length: 5 }, (_, index) => [
|
||||
{
|
||||
step_id: `fill_${index}`,
|
||||
action: "fill",
|
||||
locator: { kind: "css", selector: ".new-todo" },
|
||||
value: { source: "literal", value: `Review evidence ${index + 1}` },
|
||||
timeout_ms: 10000,
|
||||
},
|
||||
{
|
||||
step_id: `add_${index}`,
|
||||
action: "press",
|
||||
locator: { kind: "css", selector: ".new-todo" },
|
||||
key: "Enter",
|
||||
timeout_ms: 10000,
|
||||
},
|
||||
]).flat();
|
||||
const scenario = {
|
||||
browser: {
|
||||
mode: "launch",
|
||||
executable_path: executablePath,
|
||||
headless: true,
|
||||
},
|
||||
start_url: { url: origin, query: [] },
|
||||
actions,
|
||||
capture: {
|
||||
after_each_step: ["url", "dom", "accessibility", "screenshot"],
|
||||
at_end: ["url", "dom", "accessibility"],
|
||||
events: ["network"],
|
||||
},
|
||||
};
|
||||
const runs = [];
|
||||
for (const temperature of ["cold", "warm"]) {
|
||||
const callStart = calls.length;
|
||||
const result = await invoke(
|
||||
"capture_browser_scenario",
|
||||
{ ...scenario, ...(mode === "summary" ? { detail: "summary" } : {}) },
|
||||
catalog.tools,
|
||||
);
|
||||
const parentId =
|
||||
mode === "summary"
|
||||
? result.normalized_result.parent_evidence_id
|
||||
: result.evidence_id;
|
||||
const bundlePath = join(output, `${temperature}-evidence.json`);
|
||||
await invoke("export_evidence_bundle", { path: bundlePath }, catalog.tools);
|
||||
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
|
||||
const parent = bundle.records.find((row) => row.evidence_id === parentId);
|
||||
assert.ok(parent);
|
||||
const capture = parent.normalized_result;
|
||||
assert.equal(capture.steps.length, 11);
|
||||
assert.ok(capture.steps.every((step) => step.status === "completed"));
|
||||
assert.equal(capture.browser.cleanup, "terminated-owned-process");
|
||||
const final = capture.steps.at(-1).artifacts.accessibility;
|
||||
assert.equal(final.state, "captured");
|
||||
for (let index = 1; index <= 5; index++)
|
||||
assert.ok(final.value.text.includes(`Review evidence ${index}`));
|
||||
assert.ok(final.value.text.includes('strong: "5"'));
|
||||
assert.ok(final.value.text.includes("text: items left"));
|
||||
const transactionIds = new Set(
|
||||
capture.events.items
|
||||
.filter((event) => "transaction_id" in event)
|
||||
.map((event) => event.transaction_id),
|
||||
);
|
||||
const details = {
|
||||
temperature,
|
||||
parentId,
|
||||
steps: capture.steps.length,
|
||||
retained_events: capture.events.retained,
|
||||
transactions: transactionIds.size,
|
||||
};
|
||||
if (mode === "summary") {
|
||||
const summary = result.normalized_result.summary;
|
||||
assert.equal(summary.steps.length, 11);
|
||||
assert.equal(
|
||||
summary.final_accessibility.accessibility.value.text,
|
||||
final.value.text,
|
||||
);
|
||||
assert.equal(summary.events.retained, capture.events.retained);
|
||||
assert.equal(summary.limitation_count, parent.limitations.length);
|
||||
assert.equal(summary.network.transactions, transactionIds.size);
|
||||
const source = { kind: "retained-evidence", evidence_id: parentId };
|
||||
const pagingStarted = performance.now();
|
||||
for (const stepIndex of [1, 5, 10]) {
|
||||
const page = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "steps",
|
||||
step_indices: [stepIndex],
|
||||
artifacts: [],
|
||||
limit: 1,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(
|
||||
page.normalized_result.items.rows[0].step_id,
|
||||
capture.steps[stepIndex].step_id,
|
||||
);
|
||||
}
|
||||
details.warm_page_total_ms = performance.now() - pagingStarted;
|
||||
const text = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 10,
|
||||
artifact: "accessibility",
|
||||
offset: 0,
|
||||
limit: 32,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(
|
||||
text.normalized_result.items.text,
|
||||
final.value.text.slice(0, 32),
|
||||
);
|
||||
const tx = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "transactions",
|
||||
limit: 2,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.deepEqual(
|
||||
tx.normalized_result.items.rows.map((row) => row.transaction_id),
|
||||
[...transactionIds].slice(0, 2),
|
||||
);
|
||||
details.returned_summary = summary;
|
||||
}
|
||||
details.calls = calls.slice(callStart);
|
||||
runs.push(details);
|
||||
}
|
||||
const boundaryChecks = [];
|
||||
if (values["verify-boundaries"]) {
|
||||
const failed = await invoke(
|
||||
"capture_browser_scenario",
|
||||
{
|
||||
...scenario,
|
||||
actions: [
|
||||
{
|
||||
step_id: "missing",
|
||||
action: "click",
|
||||
locator: {
|
||||
kind: "css",
|
||||
selector: "#rea-control-that-does-not-exist",
|
||||
},
|
||||
timeout_ms: 100,
|
||||
},
|
||||
],
|
||||
capture: {
|
||||
after_each_step: ["url", "accessibility"],
|
||||
at_end: ["url", "accessibility"],
|
||||
events: ["network"],
|
||||
},
|
||||
detail: "summary",
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(failed.normalized_result.summary.steps[1].status, "failed");
|
||||
assert.equal(
|
||||
failed.normalized_result.summary.final_accessibility.focus_step_index,
|
||||
1,
|
||||
);
|
||||
assert.equal(
|
||||
failed.normalized_result.summary.completeness.equality_eligible,
|
||||
false,
|
||||
);
|
||||
boundaryChecks.push("real failed-step focus and incomplete coverage");
|
||||
const execute = promisify(execFile);
|
||||
const bundlePath = join(output, "cli-evidence.json");
|
||||
const { stdout } = await execute(
|
||||
process.execPath,
|
||||
[
|
||||
join(repo, "scripts/rea.mjs"),
|
||||
"capture-browser-scenario",
|
||||
JSON.stringify(scenario),
|
||||
"--detail",
|
||||
"summary",
|
||||
"--evidence_output",
|
||||
bundlePath,
|
||||
"--json",
|
||||
],
|
||||
{
|
||||
cwd: repo,
|
||||
env: environment,
|
||||
maxBuffer: 64 * 1024 * 1024,
|
||||
timeout: 90000,
|
||||
},
|
||||
);
|
||||
const cli = JSON.parse(stdout);
|
||||
const parentId = cli.summary.normalized_result.parent_evidence_id;
|
||||
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
|
||||
const parent = bundle.records.find((row) => row.evidence_id === parentId);
|
||||
assert.ok(parent);
|
||||
assert.equal(parent.normalized_result.steps.length, 11);
|
||||
assert.ok(
|
||||
parent.normalized_result.steps.every(
|
||||
(step) => step.status === "completed",
|
||||
),
|
||||
);
|
||||
assert.ok(!("detail" in parent.parameters));
|
||||
assert.equal(
|
||||
cli.summary.normalized_result.summary.final_accessibility.accessibility
|
||||
.value.text,
|
||||
parent.normalized_result.steps.at(-1).artifacts.accessibility.value.text,
|
||||
);
|
||||
const text = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source: {
|
||||
kind: "evidence-file",
|
||||
path: bundlePath,
|
||||
evidence_id: parentId,
|
||||
},
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 10,
|
||||
artifact: "accessibility",
|
||||
offset: 0,
|
||||
limit: 32,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(
|
||||
text.normalized_result.items.text,
|
||||
parent.normalized_result.steps
|
||||
.at(-1)
|
||||
.artifacts.accessibility.value.text.slice(0, 32),
|
||||
);
|
||||
boundaryChecks.push(
|
||||
"real CLI summary, complete bundle and MCP evidence-file readback",
|
||||
);
|
||||
}
|
||||
await writeFile(
|
||||
join(output, "result.json"),
|
||||
JSON.stringify(
|
||||
{
|
||||
repo,
|
||||
mode,
|
||||
application: appRoot,
|
||||
browser: executablePath,
|
||||
input_paths: {
|
||||
repository: values.repository,
|
||||
application: values.application,
|
||||
browser: values.browser,
|
||||
output: values.output,
|
||||
},
|
||||
origin,
|
||||
discovery_ms: discoveryMs,
|
||||
elapsed_ms: performance.now() - allStarted,
|
||||
runs,
|
||||
boundaryChecks,
|
||||
stderr,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
mode,
|
||||
discovery_ms: discoveryMs,
|
||||
runs: runs.map(
|
||||
({ calls: _calls, returned_summary: _summary, ...rest }) => rest,
|
||||
),
|
||||
boundaryChecks,
|
||||
elapsed_ms: performance.now() - allStarted,
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
await client.close();
|
||||
server.closeAllConnections();
|
||||
await new Promise((done) => server.close(done));
|
||||
}
|
||||
@@ -0,0 +1,521 @@
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
SCENARIO_ORIGIN,
|
||||
scenarioCaptureEvidence,
|
||||
scenarioCaptureFixture,
|
||||
} from "../../../tests/fixtures/browserScenarioCapture.js";
|
||||
import { inspectAnalysisView } from "./AnalysisViewService.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { sanitizeBrowserUrl } from "../../domain/browserObservation.js";
|
||||
import type { BrowserScenarioCapture } from "../../domain/browserScenarioCapture.js";
|
||||
import type { AnalysisViewRequest } from "../../domain/analysisView/analysisView.js";
|
||||
import { projectBrowserScenarioView } from "../../domain/analysisView/browserScenarioView.js";
|
||||
|
||||
const tree = (text: string) => ({
|
||||
state: "captured" as const,
|
||||
value: {
|
||||
sha256: createHash("sha256").update(text).digest("hex"),
|
||||
bytes: Buffer.byteLength(text),
|
||||
text,
|
||||
},
|
||||
});
|
||||
|
||||
/** Three steps: navigate to /quiz, capture its tree, then fail on a missing control. */
|
||||
const quizCapture = (
|
||||
thirdStep: "failed" | "completed" = "failed",
|
||||
): BrowserScenarioCapture => {
|
||||
const base = scenarioCaptureFixture();
|
||||
const [initial, browse] = base.steps;
|
||||
if (initial === undefined || browse === undefined)
|
||||
throw new Error("Fixture lacks steps");
|
||||
const quiz = sanitizeBrowserUrl(`${SCENARIO_ORIGIN}/quiz`);
|
||||
return {
|
||||
...base,
|
||||
scenario: { ...base.scenario, action_count: 2 },
|
||||
steps: [
|
||||
{
|
||||
...initial,
|
||||
artifacts: {
|
||||
...initial.artifacts,
|
||||
url: { state: "captured", value: initial.after_url },
|
||||
},
|
||||
},
|
||||
{
|
||||
...browse,
|
||||
after_url: quiz,
|
||||
artifacts: {
|
||||
...browse.artifacts,
|
||||
accessibility: tree('- heading "Question 1"'),
|
||||
screenshot: { state: "missing", reason: "page closed" },
|
||||
},
|
||||
},
|
||||
{
|
||||
...browse,
|
||||
step_index: 2,
|
||||
step_id: "answer",
|
||||
action: "click",
|
||||
before_url: quiz,
|
||||
after_url: quiz,
|
||||
...(thirdStep === "failed"
|
||||
? { status: "failed" as const, error: "locator not found" }
|
||||
: { status: "completed" as const, error: null }),
|
||||
},
|
||||
],
|
||||
};
|
||||
};
|
||||
|
||||
const view = async (
|
||||
capture: BrowserScenarioCapture,
|
||||
request: AnalysisViewRequest,
|
||||
) => {
|
||||
const projected = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence(capture) },
|
||||
view: request,
|
||||
});
|
||||
if (!projected.ok) throw projected.error;
|
||||
return projected.value.normalized_result as Record<string, unknown>;
|
||||
};
|
||||
|
||||
describe("browser scenario analysis views", () => {
|
||||
it("counts inherited Evidence limitations alongside capture-local caveats", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
expect(capture.limitations).toEqual([]);
|
||||
const summary = await view(capture, { kind: "summary" });
|
||||
expect(summary).toMatchObject({
|
||||
limitations: [
|
||||
"Artifact identity is unavailable for this observation.",
|
||||
"Browser host platform and architecture were not observed.",
|
||||
],
|
||||
summary: { limitation_count: 2 },
|
||||
});
|
||||
});
|
||||
|
||||
it("summarizes steps compactly and focuses the first failing step's nearest captured tree", async () => {
|
||||
const summary = await view(quizCapture(), { kind: "summary" });
|
||||
expect(summary).toMatchObject({
|
||||
kind: "summary",
|
||||
parent_operation: "capture_browser_scenario",
|
||||
artifact: null,
|
||||
coverage: { status: "complete-within-view", exhausted: true },
|
||||
});
|
||||
const { steps, final_accessibility, network, events } =
|
||||
summary.summary as Record<string, unknown> & {
|
||||
steps: Record<string, unknown>[];
|
||||
};
|
||||
// before_url repeats only across a discontinuity; not_requested sections are omitted.
|
||||
expect(steps[0]).toHaveProperty("before_url");
|
||||
expect(steps[1]).not.toHaveProperty("before_url");
|
||||
expect(steps[2]).not.toHaveProperty("before_url");
|
||||
expect(steps[0]?.sections).toEqual({
|
||||
captured: ["url"],
|
||||
missing: [],
|
||||
truncated: [],
|
||||
});
|
||||
expect(steps[1]?.sections).toEqual({
|
||||
captured: ["accessibility"],
|
||||
missing: ["screenshot"],
|
||||
truncated: [],
|
||||
});
|
||||
expect(steps[2]).toMatchObject({
|
||||
status: "failed",
|
||||
error: "locator not found",
|
||||
after_url: { url: `${SCENARIO_ORIGIN}/quiz` },
|
||||
});
|
||||
expect(final_accessibility).toEqual({
|
||||
focus_step_index: 2,
|
||||
focus: "first-unsuccessful-step",
|
||||
step_index: 1,
|
||||
accessibility: tree('- heading "Question 1"'),
|
||||
});
|
||||
expect(network).toEqual({
|
||||
selected: true,
|
||||
transactions: 8,
|
||||
without_request: 0,
|
||||
ambiguous_transactions: 0,
|
||||
failed_transactions: 1,
|
||||
by_origin: [{ origin: SCENARIO_ORIGIN, transactions: 8 }],
|
||||
by_resource_type: [
|
||||
{ resource_type: "document", transactions: 1 },
|
||||
{ resource_type: "fetch", transactions: 7 },
|
||||
],
|
||||
by_status_class: [
|
||||
{ status_class: "2xx", transactions: 7 },
|
||||
{ status_class: "failed", transactions: 1 },
|
||||
],
|
||||
});
|
||||
expect(events).toMatchObject({ retained: 24, dropped: 0 });
|
||||
});
|
||||
|
||||
it("reports the last step's own state when no step captured a tree", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
expect(await view(capture, { kind: "summary" })).toMatchObject({
|
||||
summary: {
|
||||
final_accessibility: {
|
||||
focus_step_index: 1,
|
||||
focus: "last-step",
|
||||
step_index: null,
|
||||
accessibility: { state: "not_requested" },
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser scenario analysis view pages", () => {
|
||||
it("pages exact steps with only the selected captured sections", async () => {
|
||||
const page = await view(quizCapture("completed"), {
|
||||
kind: "browser-scenario",
|
||||
collection: "steps",
|
||||
step_indices: [1, 2],
|
||||
artifacts: ["accessibility"],
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
});
|
||||
expect(page).toMatchObject({
|
||||
kind: "browser-scenario",
|
||||
source_completeness: { status: "complete" },
|
||||
source_events: { retained: 24, dropped: 0 },
|
||||
coverage: { status: "page", examined: 1, total: 2, next_offset: 1 },
|
||||
items: {
|
||||
collection: "steps",
|
||||
rows: [
|
||||
{
|
||||
step_index: 1,
|
||||
artifacts: { accessibility: tree('- heading "Question 1"') },
|
||||
omitted_artifacts: [{ section: "screenshot", state: "missing" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("filters events by step, kind and transaction, keeping original item indices", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
const responses = (await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "events",
|
||||
event_kinds: ["response", "request-failed"],
|
||||
transaction_ids: ["request-2", "request-8"],
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
})) as { items: { rows: { item_index: number; event: unknown }[] } };
|
||||
expect(
|
||||
responses.items.rows.map(({ item_index, event }) => ({
|
||||
item_index,
|
||||
event,
|
||||
})),
|
||||
).toEqual(
|
||||
capture.events.items.flatMap((event, item_index) =>
|
||||
(event.kind === "response" || event.kind === "request-failed") &&
|
||||
["request-2", "request-8"].includes(event.transaction_id)
|
||||
? [{ item_index, event }]
|
||||
: [],
|
||||
),
|
||||
);
|
||||
expect(responses.items.rows).toHaveLength(2);
|
||||
for (const empty of [
|
||||
{ step_indices: [] },
|
||||
{ event_kinds: [] },
|
||||
{ transaction_ids: [] },
|
||||
{ step_indices: [0] },
|
||||
])
|
||||
expect(
|
||||
await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "events",
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
...empty,
|
||||
}),
|
||||
).toMatchObject({
|
||||
items: { rows: [] },
|
||||
coverage: { status: "empty", total: 0, exhausted: true },
|
||||
});
|
||||
});
|
||||
|
||||
it("states dropped events and refuses views of other parents", async () => {
|
||||
const base = scenarioCaptureFixture();
|
||||
const dropped = {
|
||||
...base,
|
||||
events: {
|
||||
...base.events,
|
||||
dropped: 3,
|
||||
omissions: {
|
||||
...base.events.omissions,
|
||||
by_kind: [
|
||||
{
|
||||
kind: "console" as const,
|
||||
count: 3,
|
||||
reasons: ["max-events" as const],
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
completeness: {
|
||||
status: "truncated" as const,
|
||||
equality_eligible: false as const,
|
||||
missing_sections: [],
|
||||
truncated_sections: ["events" as const],
|
||||
},
|
||||
};
|
||||
const summary = await view(dropped, { kind: "summary" });
|
||||
expect(summary.unknowns).toContain(
|
||||
"3 events were dropped by capture budgets; counts and pages cover retained events only.",
|
||||
);
|
||||
expect(summary).toMatchObject({
|
||||
summary: {
|
||||
events: {
|
||||
dropped: 3,
|
||||
omissions: {
|
||||
by_kind: [{ kind: "console", count: 3 }],
|
||||
network_transactions: 0,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
const refused = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence() },
|
||||
view: { kind: "go", collection: "dependencies" },
|
||||
});
|
||||
if (refused.ok) throw new Error("Expected an unsupported view refusal");
|
||||
expect(projectAnalysisError(refused.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser scenario retained boundaries", () => {
|
||||
it("counts transactions whose request event was omitted and keeps source omissions on an exhausted page", async () => {
|
||||
const base = scenarioCaptureFixture();
|
||||
const omitted = (
|
||||
event: BrowserScenarioCapture["events"]["items"][number],
|
||||
) =>
|
||||
event.kind === "request" ||
|
||||
(event.kind === "network-content" && event.phase === "request");
|
||||
const removed = base.events.items.filter(omitted);
|
||||
const items = base.events.items.filter((event) => !omitted(event));
|
||||
const capture: BrowserScenarioCapture = {
|
||||
...base,
|
||||
events: {
|
||||
...base.events,
|
||||
items,
|
||||
retained: items.length,
|
||||
dropped: removed.length,
|
||||
omissions: {
|
||||
...base.events.omissions,
|
||||
by_kind: [
|
||||
{ kind: "request", count: 8, reasons: ["max-events"] },
|
||||
{ kind: "network-content", count: 1, reasons: ["max-events"] },
|
||||
],
|
||||
},
|
||||
},
|
||||
completeness: {
|
||||
status: "truncated",
|
||||
equality_eligible: false,
|
||||
missing_sections: [],
|
||||
truncated_sections: ["events"],
|
||||
},
|
||||
};
|
||||
expect(await view(capture, { kind: "summary" })).toMatchObject({
|
||||
summary: {
|
||||
network: {
|
||||
transactions: 8,
|
||||
without_request: 8,
|
||||
failed_transactions: 1,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(
|
||||
await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "transactions",
|
||||
transaction_ids: ["request-8"],
|
||||
event_kinds: [],
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
}),
|
||||
).toMatchObject({
|
||||
source_completeness: { status: "truncated" },
|
||||
source_events: { dropped: 9 },
|
||||
coverage: { total: 1, exhausted: true },
|
||||
items: {
|
||||
rows: [
|
||||
{
|
||||
transaction_id: "request-8",
|
||||
events: [],
|
||||
omitted_events: [{ kind: "request-failed" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps network bodies opt-in while retaining each transaction's event identities", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
const request = {
|
||||
kind: "browser-scenario" as const,
|
||||
collection: "transactions" as const,
|
||||
transaction_ids: ["request-2"],
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
};
|
||||
// Parse defaults at the public boundary rather than supplying a manually normalized request.
|
||||
const page = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence(capture) },
|
||||
view: request,
|
||||
});
|
||||
if (!page.ok) throw page.error;
|
||||
const result = page.value.normalized_result;
|
||||
expect(result).toMatchObject({
|
||||
items: {
|
||||
collection: "transactions",
|
||||
rows: [{ transaction_id: "request-2" }],
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain('"content":');
|
||||
const selected = await view(capture, {
|
||||
...request,
|
||||
event_kinds: ["network-content"],
|
||||
});
|
||||
expect(JSON.stringify(selected)).toContain('"content":');
|
||||
expect(selected).toMatchObject({
|
||||
items: {
|
||||
rows: [
|
||||
{
|
||||
omitted_events: expect.arrayContaining([
|
||||
{ item_index: expect.any(Number), kind: "request" },
|
||||
]),
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("distinguishes unrequested and legacy-unknown network observation from an observed empty inventory", () => {
|
||||
const base = scenarioCaptureFixture([], {
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
});
|
||||
const project = (capture: unknown) => {
|
||||
const result = projectBrowserScenarioView(
|
||||
{
|
||||
evidenceId: `ev:${"a".repeat(64)}`,
|
||||
operation: "capture_browser_scenario",
|
||||
normalizedResult: capture,
|
||||
limitations: [],
|
||||
},
|
||||
{ kind: "summary" },
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
return result.value;
|
||||
};
|
||||
expect(project(base)).toMatchObject({
|
||||
summary: { network: { selected: true, transactions: 0 } },
|
||||
});
|
||||
expect(
|
||||
project({ ...base, scenario: { ...base.scenario, event_families: [] } }),
|
||||
).toMatchObject({
|
||||
summary: { network: { selected: false, transactions: null } },
|
||||
});
|
||||
const { event_families: _families, ...legacyScenario } = base.scenario;
|
||||
expect(project({ ...base, scenario: legacyScenario })).toMatchObject({
|
||||
summary: { network: { selected: null, transactions: null } },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser scenario text and selection boundaries", () => {
|
||||
it("pages exact Unicode text with the full artifact digest and reports unavailable text explicitly", async () => {
|
||||
const capture = quizCapture("completed");
|
||||
const step = capture.steps[1]!;
|
||||
step.artifacts.accessibility = tree("A💡B\nC");
|
||||
const request = {
|
||||
kind: "browser-scenario" as const,
|
||||
collection: "text" as const,
|
||||
step_index: 1,
|
||||
artifact: "accessibility" as const,
|
||||
offset: 1,
|
||||
limit: 2,
|
||||
};
|
||||
expect(await view(capture, request)).toMatchObject({
|
||||
items: {
|
||||
state: "captured",
|
||||
text: "💡",
|
||||
sha256: tree("A💡B\nC").value.sha256,
|
||||
bytes: Buffer.byteLength("A💡B\nC"),
|
||||
},
|
||||
coverage: { examined: 2, total: 6, next_offset: 3 },
|
||||
});
|
||||
expect(await view(capture, { ...request, offset: 99 })).toMatchObject({
|
||||
items: { text: "" },
|
||||
coverage: { status: "empty", total: 6, exhausted: true },
|
||||
});
|
||||
expect(await view(capture, { ...request, step_index: 0 })).toMatchObject({
|
||||
items: { state: "not_requested", text: null, sha256: null, bytes: null },
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects irrelevant selection fields and nonexistent text steps with typed diagnostics", async () => {
|
||||
for (const request of [
|
||||
{ kind: "browser-scenario", collection: "events", artifacts: [] },
|
||||
{
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 99,
|
||||
artifact: "dom",
|
||||
},
|
||||
]) {
|
||||
const result = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence() },
|
||||
view: request,
|
||||
});
|
||||
if (result.ok)
|
||||
throw new Error("Expected the invalid view to be rejected");
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps source truncation facts when a text page has no retained payload", async () => {
|
||||
const capture = quizCapture("completed");
|
||||
const completeness = {
|
||||
status: "truncated" as const,
|
||||
equality_eligible: false as const,
|
||||
missing_sections: [],
|
||||
truncated_sections: ["accessibility" as const],
|
||||
};
|
||||
capture.steps[1]!.artifacts.accessibility = {
|
||||
state: "truncated",
|
||||
observed: 1000,
|
||||
retained: 0,
|
||||
reason: "capture budget",
|
||||
};
|
||||
capture.steps[1]!.completeness = completeness;
|
||||
capture.completeness = completeness;
|
||||
expect(
|
||||
await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 1,
|
||||
artifact: "accessibility",
|
||||
offset: 0,
|
||||
limit: 20,
|
||||
}),
|
||||
).toMatchObject({
|
||||
source_completeness: { status: "truncated" },
|
||||
items: {
|
||||
state: "truncated",
|
||||
observed: 1000,
|
||||
retained: 0,
|
||||
reason: "capture budget",
|
||||
text: null,
|
||||
},
|
||||
coverage: { status: "empty", exhausted: true },
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -76,15 +76,15 @@ describe("environment manifests from capture_browser_scenario parents", () => {
|
||||
kind: "inline",
|
||||
evidence: scenarioCaptureEvidence(scenarioCaptureFixture()),
|
||||
},
|
||||
view: { kind: "summary" },
|
||||
view: { kind: "dart-aot", collection: "package_uris" },
|
||||
});
|
||||
if (refused.ok) throw new Error("Expected the unsupported view refusal");
|
||||
expect(projectAnalysisError(refused.error)).toMatchObject({
|
||||
code: "unsupported_target",
|
||||
message: expect.stringContaining(
|
||||
"supports only the environment-manifest, environment-state and environment-coverage views",
|
||||
),
|
||||
code: "invalid_request",
|
||||
});
|
||||
expect(JSON.stringify(projectAnalysisError(refused.error))).toContain(
|
||||
"supports summary and browser-scenario views",
|
||||
);
|
||||
});
|
||||
|
||||
it("serves and exports a scenario-built manifest from the same capture", () => {
|
||||
|
||||
@@ -21,6 +21,7 @@ export const finishAnalysisCommand = async <
|
||||
canonicalOperation: string,
|
||||
signal: AbortSignal,
|
||||
fallbackResource?: string,
|
||||
options: { readonly failureProjection?: "initial" | "final" } = {},
|
||||
): Promise<Value | ReturnType<typeof projectAnalysisError>> => {
|
||||
let result: Result<Value, AnalysisError>;
|
||||
let cleanup: AnalysisError | undefined;
|
||||
@@ -35,6 +36,12 @@ export const finishAnalysisCommand = async <
|
||||
: new ProviderAdapterError(providerId, canonicalOperation, { cause }),
|
||||
};
|
||||
}
|
||||
// Some one-shot commands promise the original failure snapshot even when
|
||||
// close retries and mutates the producer's live cleanup status.
|
||||
const initialFailure =
|
||||
!result.ok && options.failureProjection === "initial"
|
||||
? projectAnalysisError(result.error)
|
||||
: undefined;
|
||||
try {
|
||||
// Resource owners may resolve their retained errors during close. This
|
||||
// adapter cannot infer resolution of a producer's earlier cleanup failure.
|
||||
@@ -61,6 +68,7 @@ export const finishAnalysisCommand = async <
|
||||
}
|
||||
}
|
||||
await dispatchQueuedInterrupts();
|
||||
if (initialFailure !== undefined) return initialFailure;
|
||||
if (result.ok && signal.aborted)
|
||||
result = {
|
||||
ok: false,
|
||||
|
||||
@@ -2,15 +2,17 @@ import { Cli } from "incur";
|
||||
import { z } from "zod";
|
||||
|
||||
import { captureBrowserScenario } from "./application/BrowserScenarioCaptureService.js";
|
||||
import { finishAnalysisCommand } from "./cli/finishAnalysisCommand.js";
|
||||
import { withCommandCancellation } from "./cli/commandCancellation.js";
|
||||
import {
|
||||
analysisEvidenceDeliveryOptions,
|
||||
cliEvidenceDeliveryInputError,
|
||||
publishCliAnalysisEvidence,
|
||||
} from "./cli/analysisEvidenceDelivery.js";
|
||||
import { CLI_COMMANDS } from "./cliCommandNames.js";
|
||||
import { parseCliJsonInput, resolveCliJsonPaths } from "./cliJsonInput.js";
|
||||
import { logCliCommand } from "./cliLogging.js";
|
||||
import { projectAnalysisError } from "./domain/analysisErrorProjection.js";
|
||||
import {
|
||||
BrowserObservationError,
|
||||
browserCleanupObservation,
|
||||
} from "./domain/browserObservationError.js";
|
||||
import { analysisInputErrorFromIssues } from "./domain/inputIssueProjection.js";
|
||||
import { browserScenarioSchema } from "./domain/browserScenario.js";
|
||||
import type { BrowserScenarioCapturePort } from "./application/BrowserScenarioCapturePort.js";
|
||||
@@ -38,9 +40,16 @@ export const registerBrowserScenarioCommands = (
|
||||
.string()
|
||||
.describe("Inline browser scenario JSON or JSON file path"),
|
||||
}),
|
||||
run: ({ args }) =>
|
||||
options: z.object(analysisEvidenceDeliveryOptions),
|
||||
run: ({ args, options: delivery }) =>
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.captureBrowserScenario, async () => {
|
||||
const deliveryError = cliEvidenceDeliveryInputError({
|
||||
detail: delivery.detail,
|
||||
evidenceOutput: delivery.evidence_output,
|
||||
operation: OPERATION,
|
||||
});
|
||||
if (deliveryError !== undefined) return cliError(deliveryError);
|
||||
const input = await parseCliJsonInput(
|
||||
args.inputJson,
|
||||
OPERATION,
|
||||
@@ -67,28 +76,28 @@ export const registerBrowserScenarioCommands = (
|
||||
createBrowserScenarioProvider(environment),
|
||||
)
|
||||
: options.createProvider(environment);
|
||||
let result:
|
||||
| Awaited<ReturnType<typeof captureBrowserScenario>>
|
||||
| undefined;
|
||||
try {
|
||||
result = await captureBrowserScenario(provider, scenario.data, {
|
||||
signal,
|
||||
});
|
||||
return result.ok ? result.value : cliError(result.error);
|
||||
} finally {
|
||||
try {
|
||||
await provider?.close?.();
|
||||
} catch (cause: unknown) {
|
||||
// Failed captures already retain their primary error and partial
|
||||
// observations. A cleanup retry must not replace that outcome.
|
||||
if (result?.ok)
|
||||
throw new BrowserObservationError(OPERATION, "cleanup_failed", {
|
||||
cause,
|
||||
cleanup: browserCleanupObservation(cause),
|
||||
partialObservation: result.value,
|
||||
});
|
||||
}
|
||||
}
|
||||
const result = await finishAnalysisCommand(
|
||||
() => captureBrowserScenario(provider, scenario.data, { signal }),
|
||||
{
|
||||
close: async () => {
|
||||
await provider?.close?.();
|
||||
},
|
||||
},
|
||||
"rea-playwright-browser-scenario",
|
||||
OPERATION,
|
||||
signal,
|
||||
undefined,
|
||||
{ failureProjection: "initial" },
|
||||
);
|
||||
if (!("evidence_id" in result))
|
||||
return { error: "Browser scenario capture failed", ...result };
|
||||
return publishCliAnalysisEvidence(result, {
|
||||
providerId: "rea-playwright-browser-scenario",
|
||||
operation: OPERATION,
|
||||
signal,
|
||||
detail: delivery.detail,
|
||||
evidenceOutput: delivery.evidence_output,
|
||||
});
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,6 +1,10 @@
|
||||
import type { ToolContract } from "./toolContractTypes.js";
|
||||
import { toolContractMetadata } from "./toolEffects.js";
|
||||
import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
|
||||
import {
|
||||
analysisDetailOutputSchemaOf,
|
||||
analysisDetailSchema,
|
||||
} from "./analysisDetail.js";
|
||||
import { browserScenarioSummaryViewResultSchema } from "../domain/analysisView/analysisView.js";
|
||||
import { browserScenarioInputSchema } from "../domain/browserScenario.js";
|
||||
import { browserScenarioCaptureSchema } from "../domain/browserScenarioCapture.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
@@ -26,10 +30,15 @@ export const BROWSER_SCENARIO_TOOL_CONTRACTS = [
|
||||
name: "capture_browser_scenario",
|
||||
...toolContractMetadata("capture_browser_scenario"),
|
||||
description:
|
||||
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. A request action sends one HTTP request to the start origin through the page's own fetch, with header values from secrets, page cookies or local storage; write methods change application state. Environment, empty storage, a URL-only final capture, and measured event-retention budgets are defaults. capture.network selects request_body, response_body, and header_values independently; capture.limits lets callers raise or lower the event count, retained-allocation estimates, network transaction count, and network content budgets. Selected content is returned inline with transaction and event references, byte digests, and explicit unavailable states. Budget omissions identify affected event kinds and network transactions. Credentials and declared secrets are redacted. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Results are returned inline.",
|
||||
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. A request action sends one HTTP request to the start origin through the page's own fetch, with header values from secrets, page cookies or local storage; write methods change application state. Environment, empty storage, a URL-only final capture, and measured event-retention budgets are defaults. capture.network selects request_body, response_body, and header_values independently; capture.limits lets callers raise or lower the event count, retained-allocation estimates, network transaction count, and network content budgets. Selected content is returned inline with transaction and event references, byte digests, and explicit unavailable states. Budget omissions identify affected event kinds and network transactions. Credentials and declared secrets are redacted. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Default detail complete returns the capture inline. Detail summary retains complete Evidence and returns step outcomes, the final or first unsuccessful step’s accessibility tree, and event/network counts. Read exact steps with selected artifacts, UTF-16 text pages, transactions with opt-in content, or events filtered by step, kind and transaction, through inspect_analysis_view kind browser-scenario.",
|
||||
kind: "browser-provider",
|
||||
inputSchema: browserScenarioInputSchema,
|
||||
outputSchema: evidenceResultOf(browserScenarioCaptureSchema),
|
||||
inputSchema: browserScenarioInputSchema.extend({
|
||||
detail: analysisDetailSchema,
|
||||
}),
|
||||
outputSchema: analysisDetailOutputSchemaOf(
|
||||
browserScenarioCaptureSchema,
|
||||
browserScenarioSummaryViewResultSchema,
|
||||
),
|
||||
examples: [
|
||||
{ title: "Capture a deterministic browser scenario", input: example },
|
||||
],
|
||||
|
||||
@@ -122,6 +122,13 @@ import {
|
||||
|
||||
import { absoluteLocalPathStringSchema } from "../localPath.js";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
browserScenarioSummarySchema,
|
||||
browserScenarioViewContextSchema,
|
||||
browserScenarioViewItemsSchema,
|
||||
browserScenarioViewRequestSchema,
|
||||
} from "./browserScenarioViewSchemas.js";
|
||||
import { projectBrowserScenarioView } from "./browserScenarioView.js";
|
||||
import {
|
||||
desktopViewRequestSchema,
|
||||
desktopViewItemsSchema,
|
||||
@@ -393,6 +400,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
||||
environmentStateViewRequestSchema,
|
||||
environmentCoverageViewRequestSchema,
|
||||
captureViewRequestSchema,
|
||||
browserScenarioViewRequestSchema,
|
||||
protobufViewRequestSchema,
|
||||
qlogViewRequestSchema,
|
||||
pcapViewRequestSchema,
|
||||
@@ -729,6 +737,14 @@ export const webNetworkCaptureSummaryViewResultSchema = z.strictObject({
|
||||
summary: captureSummarySchema,
|
||||
});
|
||||
|
||||
/** Compact outcomes and the focus screen from a retained browser scenario. */
|
||||
export const browserScenarioSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
artifact: z.null(),
|
||||
parent_operation: z.literal("capture_browser_scenario"),
|
||||
summary: browserScenarioSummarySchema,
|
||||
});
|
||||
|
||||
/** Exact package declarations with retained decoded manifest text. */
|
||||
export const androidPackageSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
@@ -1022,6 +1038,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
managedArtifactSummaryViewResultSchema,
|
||||
managedMemberSummaryViewResultSchema,
|
||||
webNetworkCaptureSummaryViewResultSchema,
|
||||
browserScenarioSummaryViewResultSchema,
|
||||
androidPackageSummaryViewResultSchema,
|
||||
pcapCaptureSummaryViewResultSchema,
|
||||
dartAotSummaryViewResultSchema,
|
||||
@@ -1076,6 +1093,15 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
...viewResultBase,
|
||||
parent_operation: z.literal("inspect_web_network_capture"),
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("browser-scenario"),
|
||||
view: browserScenarioViewRequestSchema,
|
||||
items: browserScenarioViewItemsSchema,
|
||||
...viewResultBase,
|
||||
...browserScenarioViewContextSchema.shape,
|
||||
artifact: z.null(),
|
||||
parent_operation: z.literal("capture_browser_scenario"),
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("qlog"),
|
||||
view: qlogViewRequestSchema,
|
||||
@@ -1283,9 +1309,7 @@ const unsupportedParent = (
|
||||
new AnalysisUnsupportedTargetError(
|
||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||
parentArtifactPath(parent),
|
||||
parent.operation === "capture_browser_scenario"
|
||||
? `Parent Evidence operation "capture_browser_scenario" supports only the environment-manifest, environment-state and environment-coverage views.`
|
||||
: `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed Evidence from a supported parent operation with a compatible view. For a summary receipt, use its normalized_result.parent_evidence_id rather than the receipt's own evidence_id.`,
|
||||
`Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed Evidence from a supported parent operation with a compatible view. For a summary receipt, use its normalized_result.parent_evidence_id rather than the receipt's own evidence_id.`,
|
||||
);
|
||||
|
||||
const parentArtifactPath = (parent: AnalysisViewParent): string => {
|
||||
@@ -1331,6 +1355,7 @@ const retainedViewProjectors = new Map<string, typeof projectWasmArtifactView>([
|
||||
["inspect_managed_artifact", projectManagedArtifactView],
|
||||
["inspect_managed_members", projectManagedMemberView],
|
||||
["inspect_android_package", projectAndroidPackageView],
|
||||
["capture_browser_scenario", projectBrowserScenarioView],
|
||||
["inspect_dart_aot", projectDartAotView],
|
||||
["inspect_il2cpp_metadata", projectIl2cppMetadataView],
|
||||
["inspect_apk_signature", projectApkSignatureView],
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
import { z } from "zod";
|
||||
import type { RetainedBrowserScenarioCapture } from "../browserScenarioCapture.js";
|
||||
import { isImmutableJsonSnapshot } from "../immutableJson.js";
|
||||
import { compareAsciiStrings } from "../canonicalOrdering.js";
|
||||
|
||||
const count = z.int().nonnegative();
|
||||
const statusClass = z.enum([
|
||||
"1xx",
|
||||
"2xx",
|
||||
"3xx",
|
||||
"4xx",
|
||||
"5xx",
|
||||
"failed",
|
||||
"unknown",
|
||||
]);
|
||||
|
||||
/** Counts concern retained transaction identities, including those without a request event. */
|
||||
export const browserScenarioNetworkSummarySchema = z.strictObject({
|
||||
selected: z.boolean().nullable(),
|
||||
transactions: count.nullable(),
|
||||
without_request: count,
|
||||
ambiguous_transactions: count,
|
||||
failed_transactions: count,
|
||||
by_origin: z.array(
|
||||
z.strictObject({ origin: z.string().nullable(), transactions: count }),
|
||||
),
|
||||
by_resource_type: z.array(
|
||||
z.strictObject({
|
||||
resource_type: z.string().nullable(),
|
||||
transactions: count,
|
||||
}),
|
||||
),
|
||||
by_status_class: z.array(
|
||||
z.strictObject({ status_class: statusClass, transactions: count }),
|
||||
),
|
||||
});
|
||||
|
||||
interface TransactionGroup {
|
||||
readonly transaction_id: string;
|
||||
readonly source_item_indices: readonly number[];
|
||||
readonly origin: string | null;
|
||||
readonly resource_type: string | null;
|
||||
readonly status_class: z.infer<typeof statusClass>;
|
||||
readonly has_request: boolean;
|
||||
readonly failed: boolean;
|
||||
readonly ambiguous: boolean;
|
||||
}
|
||||
|
||||
interface TransactionFacts {
|
||||
indices: number[];
|
||||
origins: Set<string | null>;
|
||||
resourceTypes: Set<string>;
|
||||
statuses: Set<number>;
|
||||
requests: number;
|
||||
failed: boolean;
|
||||
}
|
||||
|
||||
/** Sealed historical captures own this metadata-only index until their retained lifetime ends. */
|
||||
const transactionsByCapture = new WeakMap<
|
||||
object,
|
||||
readonly TransactionGroup[]
|
||||
>();
|
||||
|
||||
/** Keep all observed alternatives; one missing event never erases its transaction. */
|
||||
export const browserScenarioTransactionGroups = (
|
||||
capture: RetainedBrowserScenarioCapture,
|
||||
): readonly TransactionGroup[] => {
|
||||
const reusable = isImmutableJsonSnapshot(capture);
|
||||
const known = reusable ? transactionsByCapture.get(capture) : undefined;
|
||||
if (known !== undefined) return known;
|
||||
const groups = new Map<string, TransactionFacts>();
|
||||
for (const [index, event] of capture.events.items.entries()) {
|
||||
if (!("transaction_id" in event)) continue;
|
||||
let group = groups.get(event.transaction_id);
|
||||
if (group === undefined) {
|
||||
group = {
|
||||
indices: [],
|
||||
origins: new Set(),
|
||||
resourceTypes: new Set(),
|
||||
statuses: new Set(),
|
||||
requests: 0,
|
||||
failed: false,
|
||||
};
|
||||
groups.set(event.transaction_id, group);
|
||||
}
|
||||
group.indices.push(index);
|
||||
if (event.kind === "network-content") continue;
|
||||
group.origins.add(event.url.origin);
|
||||
group.resourceTypes.add(event.resource_type);
|
||||
if (event.kind === "request") group.requests += 1;
|
||||
if (event.kind === "response") group.statuses.add(event.status);
|
||||
if (event.kind === "request-failed") group.failed = true;
|
||||
}
|
||||
const result = [...groups].map(
|
||||
([transaction_id, group]): TransactionGroup => {
|
||||
const status = group.statuses.values().next().value;
|
||||
return Object.freeze({
|
||||
transaction_id,
|
||||
source_item_indices: Object.freeze(group.indices),
|
||||
origin:
|
||||
group.origins.size === 1
|
||||
? (group.origins.values().next().value ?? null)
|
||||
: null,
|
||||
resource_type:
|
||||
group.resourceTypes.size === 1
|
||||
? (group.resourceTypes.values().next().value ?? null)
|
||||
: null,
|
||||
status_class:
|
||||
group.statuses.size === 1 && status !== undefined
|
||||
? (`${String(Math.floor(status / 100))}xx` as z.infer<
|
||||
typeof statusClass
|
||||
>)
|
||||
: group.statuses.size === 0 && group.failed
|
||||
? "failed"
|
||||
: "unknown",
|
||||
has_request: group.requests > 0,
|
||||
failed: group.failed,
|
||||
ambiguous:
|
||||
group.requests > 1 ||
|
||||
group.statuses.size > 1 ||
|
||||
group.origins.size > 1 ||
|
||||
group.resourceTypes.size > 1,
|
||||
});
|
||||
},
|
||||
);
|
||||
if (reusable) transactionsByCapture.set(capture, result);
|
||||
return result;
|
||||
};
|
||||
|
||||
const increment = <Key>(values: Map<Key, number>, key: Key): void => {
|
||||
values.set(key, (values.get(key) ?? 0) + 1);
|
||||
};
|
||||
const sorted = <Key extends string | null>(
|
||||
values: ReadonlyMap<Key, number>,
|
||||
): [Key, number][] =>
|
||||
[...values].sort(([left], [right]) =>
|
||||
left === right
|
||||
? 0
|
||||
: left === null
|
||||
? -1
|
||||
: right === null
|
||||
? 1
|
||||
: compareAsciiStrings(left, right),
|
||||
);
|
||||
|
||||
/** Null selection/counts preserve legacy unknowns and unrequested network observation. */
|
||||
export const summarizeBrowserScenarioNetwork = (
|
||||
capture: RetainedBrowserScenarioCapture,
|
||||
): z.infer<typeof browserScenarioNetworkSummarySchema> => {
|
||||
const groups = browserScenarioTransactionGroups(capture);
|
||||
const contentSelected = Object.values(capture.scenario.network_content).some(
|
||||
Boolean,
|
||||
);
|
||||
const selected =
|
||||
contentSelected ||
|
||||
capture.scenario.event_families?.includes("network") === true
|
||||
? true
|
||||
: capture.scenario.event_families === undefined
|
||||
? null
|
||||
: false;
|
||||
const origins = new Map<string | null, number>();
|
||||
const resources = new Map<string | null, number>();
|
||||
const statuses = new Map<z.infer<typeof statusClass>, number>();
|
||||
let withoutRequest = 0;
|
||||
let ambiguous = 0;
|
||||
let failed = 0;
|
||||
for (const group of groups) {
|
||||
if (!group.has_request) withoutRequest += 1;
|
||||
if (group.ambiguous) ambiguous += 1;
|
||||
if (group.failed) failed += 1;
|
||||
increment(origins, group.origin);
|
||||
increment(resources, group.resource_type);
|
||||
increment(statuses, group.status_class);
|
||||
}
|
||||
return {
|
||||
selected,
|
||||
transactions: selected === true || groups.length > 0 ? groups.length : null,
|
||||
without_request: withoutRequest,
|
||||
ambiguous_transactions: ambiguous,
|
||||
failed_transactions: failed,
|
||||
by_origin: sorted(origins).map(([origin, transactions]) => ({
|
||||
origin,
|
||||
transactions,
|
||||
})),
|
||||
by_resource_type: sorted(resources).map(
|
||||
([resource_type, transactions]) => ({ resource_type, transactions }),
|
||||
),
|
||||
by_status_class: sorted(statuses).map(([status_class, transactions]) => ({
|
||||
status_class,
|
||||
transactions,
|
||||
})),
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,388 @@
|
||||
import type { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import {
|
||||
retainedBrowserScenarioCaptureSchema,
|
||||
type RetainedBrowserScenarioCapture as BrowserScenarioCapture,
|
||||
} from "../browserScenarioCapture.js";
|
||||
import type { BrowserScenarioStep as Step } from "../browserScenarioCaptureValues.js";
|
||||
import { compareAsciiStrings } from "../canonicalOrdering.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { analysisViewInputError } from "./analysisViewErrors.js";
|
||||
import {
|
||||
completeWithinViewCoverage,
|
||||
pageViewCoverage,
|
||||
} from "./analysisViewCoverage.js";
|
||||
import { createAnalysisViewResultReader } from "./analysisViewParent.js";
|
||||
import type {
|
||||
AnalysisViewCoverage,
|
||||
AnalysisViewParent,
|
||||
AnalysisViewRequest,
|
||||
UnsignedAnalysisView,
|
||||
} from "./analysisView.js";
|
||||
import {
|
||||
ARTIFACT_SECTIONS,
|
||||
stepRowSchema,
|
||||
browserScenarioSummarySchema,
|
||||
stepSectionsSchema,
|
||||
summaryStepSchema,
|
||||
focusAccessibilitySchema,
|
||||
browserScenarioViewItemsSchema,
|
||||
type BrowserScenarioViewRequest,
|
||||
type ArtifactSection,
|
||||
} from "./browserScenarioViewSchemas.js";
|
||||
|
||||
import {
|
||||
browserScenarioTransactionGroups,
|
||||
summarizeBrowserScenarioNetwork,
|
||||
} from "./browserScenarioNetworkView.js";
|
||||
|
||||
const OPERATION = "capture_browser_scenario" as const;
|
||||
const readCapture = createAnalysisViewResultReader(
|
||||
retainedBrowserScenarioCaptureSchema,
|
||||
);
|
||||
|
||||
const unsupportedView = (): AnalysisError =>
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["view"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
'Parent Evidence operation "capture_browser_scenario" supports summary and browser-scenario views, plus compatible environment views.',
|
||||
},
|
||||
]);
|
||||
|
||||
const sameUrl = (
|
||||
left: Step["after_url"] | undefined,
|
||||
right: Step["before_url"],
|
||||
): boolean =>
|
||||
left !== undefined &&
|
||||
left.url === right.url &&
|
||||
left.redacted === right.redacted;
|
||||
|
||||
const stepSections = (step: Step): z.infer<typeof stepSectionsSchema> => {
|
||||
const sections: z.infer<typeof stepSectionsSchema> = {
|
||||
captured: [],
|
||||
missing: [],
|
||||
truncated: [],
|
||||
};
|
||||
for (const section of ARTIFACT_SECTIONS) {
|
||||
const { state } = step.artifacts[section];
|
||||
if (state !== "not_requested") sections[state].push(section);
|
||||
}
|
||||
return sections;
|
||||
};
|
||||
|
||||
const summaryStep = (
|
||||
step: Step,
|
||||
previous: Step | undefined,
|
||||
): z.infer<typeof summaryStepSchema> => ({
|
||||
step_index: step.step_index,
|
||||
step_id: step.step_id,
|
||||
action: step.action,
|
||||
status: step.status,
|
||||
error: step.error,
|
||||
elapsed_ms: step.elapsed_ms,
|
||||
...(sameUrl(previous?.after_url, step.before_url)
|
||||
? {}
|
||||
: { before_url: step.before_url }),
|
||||
after_url: step.after_url,
|
||||
events: {
|
||||
sequence_start: step.event_sequence_start,
|
||||
sequence_end: step.event_sequence_end,
|
||||
},
|
||||
sections: stepSections(step),
|
||||
completeness: step.completeness.status,
|
||||
});
|
||||
|
||||
/** The first unsuccessful step explains a failure; otherwise the final screen does. */
|
||||
const focusAccessibility = (
|
||||
steps: readonly Step[],
|
||||
): z.infer<typeof focusAccessibilitySchema> => {
|
||||
const unsuccessful = steps.findIndex(({ status }) => status !== "completed");
|
||||
const focusIndex = unsuccessful === -1 ? steps.length - 1 : unsuccessful;
|
||||
const focus = steps[focusIndex];
|
||||
if (focus === undefined) throw new Error("Capture has no steps");
|
||||
const kind =
|
||||
unsuccessful === -1 ? ("last-step" as const) : "first-unsuccessful-step";
|
||||
for (let index = focusIndex; index >= 0; index -= 1) {
|
||||
const accessibility = steps[index]?.artifacts.accessibility;
|
||||
if (accessibility?.state === "captured")
|
||||
return {
|
||||
focus_step_index: focusIndex,
|
||||
focus: kind,
|
||||
step_index: index,
|
||||
accessibility,
|
||||
};
|
||||
}
|
||||
return {
|
||||
focus_step_index: focusIndex,
|
||||
focus: kind,
|
||||
step_index: null,
|
||||
accessibility: focus.artifacts.accessibility,
|
||||
};
|
||||
};
|
||||
|
||||
/** Counts in canonical key order; a null origin sorts first. */
|
||||
const sortedCounts = <Key extends string | null>(
|
||||
counts: ReadonlyMap<Key, number>,
|
||||
): [Key, number][] =>
|
||||
[...counts].sort(([left], [right]) =>
|
||||
left === right
|
||||
? 0
|
||||
: left === null
|
||||
? -1
|
||||
: right === null
|
||||
? 1
|
||||
: compareAsciiStrings(left, right),
|
||||
);
|
||||
|
||||
const increment = <Key>(counts: Map<Key, number>, key: Key): void => {
|
||||
counts.set(key, (counts.get(key) ?? 0) + 1);
|
||||
};
|
||||
|
||||
const eventCounts = (
|
||||
capture: BrowserScenarioCapture,
|
||||
): z.infer<typeof browserScenarioSummarySchema>["events"] => {
|
||||
const kinds = new Map<
|
||||
BrowserScenarioCapture["events"]["items"][number]["kind"],
|
||||
number
|
||||
>();
|
||||
for (const event of capture.events.items) increment(kinds, event.kind);
|
||||
const { omissions } = capture.events;
|
||||
return {
|
||||
retained: capture.events.retained,
|
||||
dropped: capture.events.dropped,
|
||||
by_kind: sortedCounts(kinds).map(([kind, count]) => ({ kind, count })),
|
||||
omissions: {
|
||||
by_kind: omissions.by_kind,
|
||||
network_transactions: omissions.network_transactions.length,
|
||||
network_transaction_range: omissions.network_transaction_range,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const selected = <Value>(
|
||||
filter: readonly Value[] | undefined,
|
||||
): ((value: Value) => boolean) => {
|
||||
if (filter === undefined) return () => true;
|
||||
const values = new Set(filter);
|
||||
return (value) => values.has(value);
|
||||
};
|
||||
|
||||
const stepRow = (
|
||||
{ artifacts: recorded, ...step }: Step,
|
||||
selection: ReadonlySet<ArtifactSection>,
|
||||
): z.infer<typeof stepRowSchema> => {
|
||||
const artifacts: z.infer<typeof stepRowSchema>["artifacts"] = {};
|
||||
const omitted: z.infer<typeof stepRowSchema>["omitted_artifacts"] = [];
|
||||
for (const section of ARTIFACT_SECTIONS) {
|
||||
const artifact = recorded[section];
|
||||
if (selection.has(section))
|
||||
Object.assign(artifacts, { [section]: artifact });
|
||||
else if (artifact.state !== "not_requested")
|
||||
omitted.push({ section, state: artifact.state });
|
||||
}
|
||||
return { ...step, artifacts, omitted_artifacts: omitted };
|
||||
};
|
||||
|
||||
/** Scan matching rows once and materialize only the requested page. */
|
||||
const selectedPage = <Input, Row>(
|
||||
source: readonly Input[],
|
||||
view: { readonly offset: number; readonly limit: number },
|
||||
matches: (item: Input, index: number) => boolean,
|
||||
project: (item: Input, index: number) => Row,
|
||||
): { rows: Row[]; coverage: AnalysisViewCoverage } => {
|
||||
const rows: Row[] = [];
|
||||
let total = 0;
|
||||
for (const [index, item] of source.entries()) {
|
||||
if (!matches(item, index)) continue;
|
||||
if (total >= view.offset && rows.length < view.limit)
|
||||
rows.push(project(item, index));
|
||||
total += 1;
|
||||
}
|
||||
return { rows, coverage: pageViewCoverage(view.offset, rows.length, total) };
|
||||
};
|
||||
|
||||
/** Text offsets are UTF-16 positions in the exact digest-bound retained artifact. */
|
||||
const textPage = (
|
||||
capture: BrowserScenarioCapture,
|
||||
view: Extract<BrowserScenarioViewRequest, { collection: "text" }>,
|
||||
) => {
|
||||
const artifact = capture.steps[view.step_index]!.artifacts[view.artifact];
|
||||
const value = artifact.state === "captured" ? artifact.value : null;
|
||||
const text = value?.text.slice(view.offset, view.offset + view.limit) ?? null;
|
||||
return {
|
||||
items: {
|
||||
collection: "text" as const,
|
||||
step_index: view.step_index,
|
||||
artifact: view.artifact,
|
||||
state: artifact.state,
|
||||
reason: "reason" in artifact ? artifact.reason : null,
|
||||
sha256: value?.sha256 ?? null,
|
||||
bytes: value?.bytes ?? null,
|
||||
observed: artifact.state === "truncated" ? artifact.observed : null,
|
||||
retained: artifact.state === "truncated" ? artifact.retained : null,
|
||||
text,
|
||||
},
|
||||
coverage: pageViewCoverage(
|
||||
view.offset,
|
||||
text?.length ?? 0,
|
||||
value?.text.length ?? 0,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
/** One stable page of exact recorded facts; no whole-result filtering arrays. */
|
||||
const pageRecords = (
|
||||
capture: BrowserScenarioCapture,
|
||||
view: BrowserScenarioViewRequest,
|
||||
): {
|
||||
readonly items: z.infer<typeof browserScenarioViewItemsSchema>;
|
||||
readonly coverage: AnalysisViewCoverage;
|
||||
} => {
|
||||
if (view.collection === "text") return textPage(capture, view);
|
||||
const stepSelected = selected(view.step_indices);
|
||||
if (view.collection === "steps") {
|
||||
const artifacts = new Set(view.artifacts);
|
||||
const page = selectedPage(
|
||||
capture.steps,
|
||||
view,
|
||||
({ step_index }) => stepSelected(step_index),
|
||||
(step) => stepRow(step, artifacts),
|
||||
);
|
||||
return {
|
||||
items: { collection: "steps", rows: page.rows },
|
||||
coverage: page.coverage,
|
||||
};
|
||||
}
|
||||
const kindSelected = selected(view.event_kinds);
|
||||
const transactionSelected = selected(view.transaction_ids);
|
||||
if (view.collection === "transactions") {
|
||||
const page = selectedPage(
|
||||
browserScenarioTransactionGroups(capture),
|
||||
view,
|
||||
(group) =>
|
||||
transactionSelected(group.transaction_id) &&
|
||||
(view.step_indices === undefined ||
|
||||
group.source_item_indices.some((index) =>
|
||||
stepSelected(capture.events.items[index]!.step_index),
|
||||
)),
|
||||
(group) => {
|
||||
const events: {
|
||||
item_index: number;
|
||||
event: BrowserScenarioCapture["events"]["items"][number];
|
||||
}[] = [];
|
||||
const omitted: {
|
||||
item_index: number;
|
||||
kind: BrowserScenarioCapture["events"]["items"][number]["kind"];
|
||||
}[] = [];
|
||||
for (const item_index of group.source_item_indices) {
|
||||
const event = capture.events.items[item_index]!;
|
||||
if (kindSelected(event.kind)) events.push({ item_index, event });
|
||||
else omitted.push({ item_index, kind: event.kind });
|
||||
}
|
||||
return {
|
||||
transaction_id: group.transaction_id,
|
||||
source_item_indices: [...group.source_item_indices],
|
||||
events,
|
||||
omitted_events: omitted,
|
||||
};
|
||||
},
|
||||
);
|
||||
return {
|
||||
items: { collection: "transactions", rows: page.rows },
|
||||
coverage: page.coverage,
|
||||
};
|
||||
}
|
||||
const page = selectedPage(
|
||||
capture.events.items,
|
||||
view,
|
||||
(event) =>
|
||||
stepSelected(event.step_index) &&
|
||||
kindSelected(event.kind) &&
|
||||
(view.transaction_ids === undefined ||
|
||||
("transaction_id" in event &&
|
||||
transactionSelected(event.transaction_id))),
|
||||
(event, item_index) => ({ item_index, event }),
|
||||
);
|
||||
return {
|
||||
items: { collection: "events", rows: page.rows },
|
||||
coverage: page.coverage,
|
||||
};
|
||||
};
|
||||
|
||||
/** Project retained scenario Evidence without relaunching the browser or replaying actions. */
|
||||
export const projectBrowserScenarioView = (
|
||||
parent: AnalysisViewParent,
|
||||
view: AnalysisViewRequest,
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
if (view.kind !== "summary" && view.kind !== "browser-scenario")
|
||||
return err(unsupportedView());
|
||||
const parsed = readCapture(parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const capture = parsed.value;
|
||||
const shared = {
|
||||
parent_evidence_id: parent.evidenceId,
|
||||
parent_operation: OPERATION,
|
||||
parent_digest: parent.evidenceId.slice(3),
|
||||
artifact: null,
|
||||
limitations: [...parent.limitations],
|
||||
unknowns: [
|
||||
"Step states record one controlled replay; they do not establish the application's current state.",
|
||||
"Event step references record receipt attribution; they do not prove an action caused a transaction.",
|
||||
...(capture.events.dropped > 0
|
||||
? [
|
||||
`${String(capture.events.dropped)} events were dropped by capture budgets; counts and pages cover retained events only.`,
|
||||
]
|
||||
: []),
|
||||
],
|
||||
};
|
||||
if (view.kind === "summary")
|
||||
return ok({
|
||||
kind: "summary",
|
||||
view,
|
||||
...shared,
|
||||
summary: {
|
||||
browser: capture.browser,
|
||||
scenario: capture.scenario,
|
||||
duration_ms: capture.duration_ms,
|
||||
completeness: capture.completeness,
|
||||
steps: capture.steps.map((step, index) =>
|
||||
summaryStep(step, capture.steps[index - 1]),
|
||||
),
|
||||
final_accessibility: focusAccessibility(capture.steps),
|
||||
events: eventCounts(capture),
|
||||
network: summarizeBrowserScenarioNetwork(capture),
|
||||
limitation_count: shared.limitations.length,
|
||||
},
|
||||
coverage: completeWithinViewCoverage(1, 1),
|
||||
});
|
||||
if (
|
||||
view.collection === "text" &&
|
||||
capture.steps[view.step_index] === undefined
|
||||
)
|
||||
return err(
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["view", "step_index"],
|
||||
reason: "invalid_value",
|
||||
message: "The selected step is not present in the retained capture.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
return ok({
|
||||
kind: "browser-scenario",
|
||||
view,
|
||||
...shared,
|
||||
source_completeness: capture.completeness,
|
||||
source_events: {
|
||||
retained: capture.events.retained,
|
||||
dropped: capture.events.dropped,
|
||||
},
|
||||
source_network_omissions: {
|
||||
transactions: capture.events.omissions.network_transactions.length,
|
||||
transaction_range: capture.events.omissions.network_transaction_range,
|
||||
},
|
||||
...pageRecords(capture, view),
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,250 @@
|
||||
import { z } from "zod";
|
||||
import { digestSchema } from "../digests.js";
|
||||
import { browserScenarioNetworkSummarySchema } from "./browserScenarioNetworkView.js";
|
||||
import { sanitizedBrowserUrlSchema } from "../browserObservation.js";
|
||||
import { retainedBrowserScenarioCaptureSchema } from "../browserScenarioCapture.js";
|
||||
import {
|
||||
browserScenarioCompletenessSchema,
|
||||
browserScenarioEventKindSchema,
|
||||
browserScenarioEventSchema,
|
||||
browserStepArtifactsSchema,
|
||||
} from "../browserScenarioCaptureValues.js";
|
||||
|
||||
export const ARTIFACT_SECTIONS = browserStepArtifactsSchema.keyof().options;
|
||||
const artifactSectionSchema = browserStepArtifactsSchema.keyof();
|
||||
export type ArtifactSection = z.infer<typeof artifactSectionSchema>;
|
||||
|
||||
const countSchema = z.number().int().nonnegative();
|
||||
/** Section names by recorded state; not_requested sections are omitted. */
|
||||
export const stepSectionsSchema = z.strictObject({
|
||||
captured: z.array(artifactSectionSchema),
|
||||
missing: z.array(artifactSectionSchema),
|
||||
truncated: z.array(artifactSectionSchema),
|
||||
});
|
||||
|
||||
export const summaryStepSchema = z.strictObject({
|
||||
step_index: countSchema,
|
||||
step_id: z.string().min(1),
|
||||
action: z.string().min(1),
|
||||
status: z.enum(["completed", "failed", "cancelled"]),
|
||||
error: z.string().nullable(),
|
||||
elapsed_ms: countSchema,
|
||||
before_url: sanitizedBrowserUrlSchema
|
||||
.exactOptional()
|
||||
.describe(
|
||||
"Present only when it differs from the previous step's after_url (always for step 0).",
|
||||
),
|
||||
after_url: sanitizedBrowserUrlSchema,
|
||||
events: z.strictObject({
|
||||
sequence_start: z.number().int().min(1),
|
||||
sequence_end: countSchema,
|
||||
}),
|
||||
sections: stepSectionsSchema,
|
||||
completeness: z.enum(["complete", "incomplete", "truncated"]),
|
||||
});
|
||||
|
||||
export const focusAccessibilitySchema = z.strictObject({
|
||||
focus_step_index: countSchema,
|
||||
focus: z.enum(["first-unsuccessful-step", "last-step"]),
|
||||
/** The step whose tree is returned: the focus step, or the nearest earlier step that captured one. */
|
||||
step_index: countSchema.nullable(),
|
||||
accessibility: browserStepArtifactsSchema.shape.accessibility,
|
||||
});
|
||||
|
||||
/** Compact capture overview; complete records stay in the retained parent. */
|
||||
export const browserScenarioSummarySchema = z.strictObject({
|
||||
browser: retainedBrowserScenarioCaptureSchema.shape.browser,
|
||||
scenario: retainedBrowserScenarioCaptureSchema.shape.scenario,
|
||||
duration_ms: countSchema,
|
||||
completeness: browserScenarioCompletenessSchema,
|
||||
steps: z.array(summaryStepSchema),
|
||||
final_accessibility: focusAccessibilitySchema,
|
||||
events: z.strictObject({
|
||||
retained: countSchema,
|
||||
dropped: countSchema,
|
||||
by_kind: z.array(
|
||||
z.strictObject({
|
||||
kind: browserScenarioEventKindSchema,
|
||||
count: countSchema,
|
||||
}),
|
||||
),
|
||||
omissions: z.strictObject({
|
||||
by_kind:
|
||||
retainedBrowserScenarioCaptureSchema.shape.events.shape.omissions.shape
|
||||
.by_kind,
|
||||
network_transactions: countSchema,
|
||||
network_transaction_range:
|
||||
retainedBrowserScenarioCaptureSchema.shape.events.shape.omissions.shape
|
||||
.network_transaction_range,
|
||||
}),
|
||||
}),
|
||||
network: browserScenarioNetworkSummarySchema,
|
||||
limitation_count: countSchema.describe(
|
||||
"Number of parent Evidence limitations returned with this summary, including inherited provenance caveats.",
|
||||
),
|
||||
});
|
||||
|
||||
const pageShape = {
|
||||
kind: z.literal("browser-scenario"),
|
||||
offset: countSchema.default(0),
|
||||
limit: z.int().positive().default(64),
|
||||
};
|
||||
const stepSelection = {
|
||||
step_indices: z
|
||||
.array(countSchema)
|
||||
.optional()
|
||||
.describe(
|
||||
"Exact recorded step indices; omitted selects all, [] selects none.",
|
||||
),
|
||||
};
|
||||
const transactionSelection = {
|
||||
transaction_ids: z
|
||||
.array(z.string().min(1))
|
||||
.optional()
|
||||
.describe(
|
||||
"Exact recorded transaction IDs; omitted selects all, [] selects none.",
|
||||
),
|
||||
};
|
||||
|
||||
/** Collection-specific fields are validated rather than silently ignored. */
|
||||
export const browserScenarioViewRequestSchema = z.discriminatedUnion(
|
||||
"collection",
|
||||
[
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
...stepSelection,
|
||||
collection: z.literal("steps"),
|
||||
artifacts: z
|
||||
.array(artifactSectionSchema)
|
||||
.default([])
|
||||
.describe(
|
||||
"Return only selected artifact payloads; omitted_artifacts names the other recorded sections.",
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
...stepSelection,
|
||||
...transactionSelection,
|
||||
collection: z.literal("events"),
|
||||
event_kinds: z
|
||||
.array(browserScenarioEventKindSchema)
|
||||
.optional()
|
||||
.describe("Omitted selects all event kinds; [] selects none."),
|
||||
}),
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
...stepSelection,
|
||||
...transactionSelection,
|
||||
collection: z.literal("transactions"),
|
||||
event_kinds: z
|
||||
.array(browserScenarioEventKindSchema)
|
||||
.default([
|
||||
"request",
|
||||
"response",
|
||||
"request-finished",
|
||||
"request-failed",
|
||||
"request-unfinished",
|
||||
])
|
||||
.describe(
|
||||
"Metadata events by default; select network-content explicitly for bodies and header values. [] returns identity and source indices only.",
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
collection: z.literal("text"),
|
||||
step_index: countSchema,
|
||||
artifact: z.enum(["accessibility", "dom"]),
|
||||
limit: z
|
||||
.int()
|
||||
.positive()
|
||||
.default(2048)
|
||||
.describe("Caller-selected UTF-16 code units, matching offset."),
|
||||
}),
|
||||
],
|
||||
);
|
||||
export type BrowserScenarioViewRequest = z.output<
|
||||
typeof browserScenarioViewRequestSchema
|
||||
>;
|
||||
|
||||
/** One exact step whose unselected captured sections are named rather than returned. */
|
||||
export const stepRowSchema = z.strictObject({
|
||||
step_index: countSchema,
|
||||
step_id: z.string().min(1),
|
||||
action: z.string().min(1),
|
||||
status: z.enum(["completed", "failed", "cancelled"]),
|
||||
error: z.string().nullable(),
|
||||
elapsed_ms: countSchema,
|
||||
before_url: sanitizedBrowserUrlSchema,
|
||||
after_url: sanitizedBrowserUrlSchema,
|
||||
event_sequence_start: z.number().int().min(1),
|
||||
event_sequence_end: countSchema,
|
||||
completeness: browserScenarioCompletenessSchema,
|
||||
artifacts: browserStepArtifactsSchema.partial(),
|
||||
omitted_artifacts: z.array(
|
||||
z.strictObject({
|
||||
section: artifactSectionSchema,
|
||||
state: z.enum(["captured", "missing", "truncated"]),
|
||||
}),
|
||||
),
|
||||
});
|
||||
const eventRowSchema = z.strictObject({
|
||||
item_index: countSchema,
|
||||
event: browserScenarioEventSchema,
|
||||
});
|
||||
|
||||
export const browserScenarioViewItemsSchema = z.discriminatedUnion(
|
||||
"collection",
|
||||
[
|
||||
z.strictObject({
|
||||
collection: z.literal("steps"),
|
||||
rows: z.array(stepRowSchema),
|
||||
}),
|
||||
z.strictObject({
|
||||
collection: z.literal("events"),
|
||||
rows: z.array(eventRowSchema),
|
||||
}),
|
||||
z.strictObject({
|
||||
collection: z.literal("transactions"),
|
||||
rows: z.array(
|
||||
z.strictObject({
|
||||
transaction_id: z.string().min(1),
|
||||
source_item_indices: z.array(countSchema),
|
||||
events: z.array(eventRowSchema),
|
||||
omitted_events: z.array(
|
||||
z.strictObject({
|
||||
item_index: countSchema,
|
||||
kind: browserScenarioEventKindSchema,
|
||||
}),
|
||||
),
|
||||
}),
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
collection: z.literal("text"),
|
||||
step_index: countSchema,
|
||||
artifact: z.enum(["accessibility", "dom"]),
|
||||
state: z.enum(["captured", "missing", "truncated", "not_requested"]),
|
||||
reason: z.string().nullable(),
|
||||
sha256: digestSchema.nullable(),
|
||||
bytes: countSchema.nullable(),
|
||||
observed: countSchema.nullable(),
|
||||
retained: countSchema.nullable(),
|
||||
text: z.string().nullable(),
|
||||
}),
|
||||
],
|
||||
);
|
||||
|
||||
/** Capture-wide facts every page repeats so exhaustion is not read as completeness. */
|
||||
export const browserScenarioViewContextSchema = z.strictObject({
|
||||
source_completeness: browserScenarioCompletenessSchema,
|
||||
source_events: z.strictObject({
|
||||
retained: countSchema,
|
||||
dropped: countSchema,
|
||||
}),
|
||||
source_network_omissions: z.strictObject({
|
||||
transactions: countSchema,
|
||||
transaction_range:
|
||||
retainedBrowserScenarioCaptureSchema.shape.events.shape.omissions.shape
|
||||
.network_transaction_range,
|
||||
}),
|
||||
});
|
||||
@@ -1290,6 +1290,7 @@ const registerObservationTools = ({
|
||||
});
|
||||
registerBrowserScenarioTool(server, {
|
||||
...common,
|
||||
evidenceById,
|
||||
provider: options.browserScenarioCapture,
|
||||
loadFailure: options.optionalProviderLoadFailures?.browserScenarioCapture,
|
||||
});
|
||||
|
||||
@@ -1,82 +1,67 @@
|
||||
import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
|
||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
|
||||
import {
|
||||
optionalProviderUnavailable,
|
||||
type OptionalProviderLoadFailure,
|
||||
} from "../application/OptionalObservationProviders.js";
|
||||
import { err } from "../domain/result.js";
|
||||
|
||||
import { browserScenarioSchema } from "../domain/browserScenario.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import type { BrowserScenarioCapturePort } from "../application/BrowserScenarioCapturePort.js";
|
||||
import { captureBrowserScenario } from "../application/BrowserScenarioCaptureService.js";
|
||||
import { toolContract } from "../contracts/toolContracts.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import { browserScenarioSchema } from "../domain/browserScenario.js";
|
||||
import type { Logger } from "pino";
|
||||
import { logToolExecution } from "./toolLogging.js";
|
||||
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
import type { WithAdmittedAnalysis } from "./analysisAdmission.js";
|
||||
import { runAdmittedToolOperation } from "./admittedToolOperation.js";
|
||||
import { deliverEvidenceResult } from "./contractToolHandler.js";
|
||||
import { createRetainedSummaryToolHandler } from "./contractToolHandler.js";
|
||||
|
||||
interface BrowserScenarioToolRegistration {
|
||||
readonly logger: Logger;
|
||||
readonly loadFailure?: OptionalProviderLoadFailure | undefined;
|
||||
readonly provider: BrowserScenarioCapturePort | undefined;
|
||||
readonly recordEvidence: EvidenceWriter["recordEvidence"] | undefined;
|
||||
readonly evidenceById?: EvidenceLookup | undefined;
|
||||
readonly withAdmittedAnalysis?: WithAdmittedAnalysis;
|
||||
}
|
||||
|
||||
/** Register the browser scenario tool with execution-time provider diagnostics. */
|
||||
/** Validate one scenario, then share complete/retained delivery with other analyses. */
|
||||
export const registerBrowserScenarioTool = (
|
||||
server: EvidenceMcpServer,
|
||||
options: BrowserScenarioToolRegistration,
|
||||
): void => {
|
||||
const contract = toolContract("capture_browser_scenario");
|
||||
const handler = createRetainedSummaryToolHandler(
|
||||
server,
|
||||
contract,
|
||||
async (input, execution) => {
|
||||
// The SDK admits the advertised structure. The canonical semantic parser
|
||||
// owns field-path diagnostics, shared with the CLI rather than flattened
|
||||
// into the SDK's pre-handler validation message.
|
||||
const scenario = browserScenarioSchema.safeParse(input);
|
||||
if (!scenario.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
contract.name,
|
||||
scenario.error.issues,
|
||||
input,
|
||||
{ cause: scenario.error },
|
||||
),
|
||||
);
|
||||
return options.loadFailure === undefined
|
||||
? captureBrowserScenario(options.provider, scenario.data, execution)
|
||||
: err(optionalProviderUnavailable(options.loadFailure, contract.name));
|
||||
},
|
||||
options.logger,
|
||||
{
|
||||
recordEvidence: options.recordEvidence,
|
||||
evidenceById: options.evidenceById,
|
||||
},
|
||||
options.withAdmittedAnalysis,
|
||||
);
|
||||
server.registerTool(
|
||||
contract.name,
|
||||
toolRegistrationOptions(contract),
|
||||
async (input, context) => {
|
||||
const parsedScenario = browserScenarioSchema.safeParse(input);
|
||||
if (!parsedScenario.success)
|
||||
return server.delivery.toCallToolResult(
|
||||
err(
|
||||
analysisInputErrorFromIssues(
|
||||
contract.name,
|
||||
parsedScenario.error.issues,
|
||||
input,
|
||||
{ cause: parsedScenario.error },
|
||||
),
|
||||
),
|
||||
contract,
|
||||
);
|
||||
const scenario = parsedScenario.data;
|
||||
if (options.loadFailure !== undefined)
|
||||
return server.delivery.toCallToolResult(
|
||||
err(optionalProviderUnavailable(options.loadFailure, contract.name)),
|
||||
contract,
|
||||
);
|
||||
return runAdmittedToolOperation(
|
||||
server,
|
||||
options.withAdmittedAnalysis,
|
||||
contract.name,
|
||||
context.mcpReq.signal,
|
||||
async () => {
|
||||
const result = await logToolExecution(
|
||||
options.logger,
|
||||
contract.name,
|
||||
() =>
|
||||
captureBrowserScenario(options.provider, scenario, {
|
||||
signal: context.mcpReq.signal,
|
||||
}),
|
||||
);
|
||||
return deliverEvidenceResult(
|
||||
server,
|
||||
result,
|
||||
contract,
|
||||
options.recordEvidence,
|
||||
);
|
||||
},
|
||||
);
|
||||
},
|
||||
handler,
|
||||
);
|
||||
};
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
import { access } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { Cli } from "incur";
|
||||
import pino from "pino";
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { registerBrowserScenarioCommands } from "../../../src/cliBrowserScenarioCommands.js";
|
||||
import { ok } from "../../../src/domain/result.js";
|
||||
import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js";
|
||||
import { scenarioCaptureFixture } from "../../fixtures/browserScenarioCapture.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
it.each(["complete", "summary"] as const)(
|
||||
"retains complete Evidence on cancellation during CLI cleanup with %s delivery",
|
||||
async (detail) => {
|
||||
const root = await createTestTempDirectory("rea-browser-summary-cli-");
|
||||
const destination = join(root, "evidence.json");
|
||||
const originalExit = process.exitCode;
|
||||
const listeners = new Set(process.listeners("SIGINT"));
|
||||
const cli = Cli.create("rea-browser-summary-test", { sync: false });
|
||||
let stdout = "";
|
||||
let closed = false;
|
||||
registerBrowserScenarioCommands(
|
||||
cli,
|
||||
pino({ level: "silent" }),
|
||||
{},
|
||||
{
|
||||
createProvider: () => ({
|
||||
identity: () => ({ id: "fixture", name: "Fixture", version: "1" }),
|
||||
captureScenario: async () => ok(scenarioCaptureFixture()),
|
||||
close: async () => {
|
||||
closed = true;
|
||||
// Invoke only this command's newly installed listener, without emitting
|
||||
// a process-wide signal into the shared test worker.
|
||||
const installed = process
|
||||
.listeners("SIGINT")
|
||||
.filter((listener) => !listeners.has(listener));
|
||||
expect(installed).toHaveLength(1);
|
||||
installed[0]!("SIGINT");
|
||||
},
|
||||
}),
|
||||
},
|
||||
);
|
||||
try {
|
||||
await cli.serve(
|
||||
[
|
||||
"capture-browser-scenario",
|
||||
JSON.stringify({
|
||||
browser: { mode: "launch", executable_path: process.execPath },
|
||||
start_url: { url: "http://127.0.0.1:3101/" },
|
||||
actions: [
|
||||
{ step_id: "browse", action: "wait_for_timeout", duration_ms: 1 },
|
||||
],
|
||||
}),
|
||||
"--detail",
|
||||
detail,
|
||||
"--evidence_output",
|
||||
destination,
|
||||
"--json",
|
||||
],
|
||||
{
|
||||
env: {},
|
||||
stdout: (text) => {
|
||||
stdout += text;
|
||||
},
|
||||
exit: () => undefined,
|
||||
},
|
||||
);
|
||||
expect(JSON.parse(stdout)).toMatchObject({
|
||||
code: "cancelled",
|
||||
details: {
|
||||
partial_observation: {
|
||||
operation: "capture_browser_scenario",
|
||||
normalized_result: { steps: expect.any(Array) },
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(closed).toBe(true);
|
||||
await expect(access(destination)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
expect(new Set(process.listeners("SIGINT"))).toEqual(listeners);
|
||||
} finally {
|
||||
process.exitCode = originalExit;
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps the provider's cleanup failure and completed Evidence without publishing a summary bundle", async () => {
|
||||
const root = await createTestTempDirectory("rea-browser-summary-cleanup-");
|
||||
const destination = join(root, "evidence.json");
|
||||
const cli = Cli.create("rea-browser-cleanup-test", { sync: false });
|
||||
let stdout = "";
|
||||
const failure = new ProviderCleanupError(
|
||||
"owned-browser",
|
||||
["owned-profile"],
|
||||
{ reason: "cleanup denied" },
|
||||
{ operation: "capture_browser_scenario" },
|
||||
);
|
||||
registerBrowserScenarioCommands(
|
||||
cli,
|
||||
pino({ level: "silent" }),
|
||||
{},
|
||||
{
|
||||
createProvider: () => ({
|
||||
identity: () => ({ id: "fixture", name: "Fixture", version: "1" }),
|
||||
captureScenario: async () => ok(scenarioCaptureFixture()),
|
||||
close: async () => {
|
||||
throw failure;
|
||||
},
|
||||
}),
|
||||
},
|
||||
);
|
||||
await cli.serve(
|
||||
[
|
||||
"capture-browser-scenario",
|
||||
JSON.stringify({
|
||||
browser: { mode: "launch", executable_path: process.execPath },
|
||||
start_url: { url: "http://127.0.0.1:3101/" },
|
||||
actions: [
|
||||
{ step_id: "browse", action: "wait_for_timeout", duration_ms: 1 },
|
||||
],
|
||||
}),
|
||||
"--detail",
|
||||
"summary",
|
||||
"--evidence_output",
|
||||
destination,
|
||||
"--json",
|
||||
],
|
||||
{
|
||||
env: {},
|
||||
stdout: (text) => {
|
||||
stdout += text;
|
||||
},
|
||||
exit: () => undefined,
|
||||
},
|
||||
);
|
||||
expect(JSON.parse(stdout)).toMatchObject({
|
||||
code: "cleanup_incomplete",
|
||||
details: {
|
||||
resources: ["owned-profile"],
|
||||
partial_observation: { operation: "capture_browser_scenario" },
|
||||
},
|
||||
});
|
||||
await expect(access(destination)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
@@ -38,6 +38,38 @@ describe("scenario CLI schema diagnostics", () => {
|
||||
});
|
||||
});
|
||||
|
||||
cliTest(
|
||||
"refuses summary delivery without a complete Evidence bundle before launching",
|
||||
async ({ cli }) => {
|
||||
const scenario = {
|
||||
browser: { mode: "launch", executable_path: "/nonexistent/chromium" },
|
||||
start_url: { url: "https://app.example.test/", query: [] },
|
||||
actions: [
|
||||
{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 },
|
||||
],
|
||||
};
|
||||
const result = await cli.run({
|
||||
arguments: [
|
||||
"capture-browser-scenario",
|
||||
JSON.stringify(scenario),
|
||||
"--detail",
|
||||
"summary",
|
||||
"--json",
|
||||
],
|
||||
environment: { REA_LOG_LEVEL: "silent" },
|
||||
});
|
||||
|
||||
expect(result.exitCode).not.toBe(0);
|
||||
expect(result.json).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "capture_browser_scenario",
|
||||
issues: [{ path: ["evidence_output"], reason: "invalid_value" }],
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
cliTest("reports Electron scenario schema paths", async ({ cli }) => {
|
||||
const result = await cli.run({
|
||||
arguments: ["capture-electron-scenario", "{}", "--json"],
|
||||
|
||||
@@ -367,3 +367,111 @@ test("records a completed late-close cancellation as server Evidence", async ()
|
||||
await Promise.all([client.close(), server.close(), session.close()]);
|
||||
}
|
||||
});
|
||||
|
||||
test("delivers a retained summary and pages its exact steps and events", async () => {
|
||||
const provider = new FakeBrowserScenarioProvider();
|
||||
const session = createTestBinarySession(() => ({
|
||||
execute: () => Promise.resolve(observed(null)),
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
}));
|
||||
const server = createServer(
|
||||
{ kind: "session", session },
|
||||
{
|
||||
browserScenarioCapture: provider,
|
||||
availabilityPolicy: () => ({
|
||||
processCaptureEnabled: false,
|
||||
investigationInputRoots: 0,
|
||||
}),
|
||||
},
|
||||
);
|
||||
const client = new Client({ name: "scenario-summary", version: "1" });
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
try {
|
||||
const tools = (await client.listTools()).tools;
|
||||
const advertised = tools.find(
|
||||
({ name }) => name === "capture_browser_scenario",
|
||||
);
|
||||
if (advertised?.outputSchema === undefined)
|
||||
throw new Error("Missing advertised scenario output schema");
|
||||
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
||||
const validateInput = ajv.compile(advertised.inputSchema);
|
||||
const validateOutput = ajv.compile(
|
||||
z.record(z.string(), z.unknown()).parse(advertised.outputSchema),
|
||||
);
|
||||
const request = { ...minimalScenario(), detail: "summary" };
|
||||
expect(validateInput(request), JSON.stringify(validateInput.errors)).toBe(
|
||||
true,
|
||||
);
|
||||
|
||||
const response = await client.callTool({
|
||||
name: "capture_browser_scenario",
|
||||
arguments: request,
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
expect(
|
||||
validateOutput(response.structuredContent),
|
||||
JSON.stringify(validateOutput.errors),
|
||||
).toBe(true);
|
||||
const summary = z
|
||||
.object({
|
||||
normalized_result: z.object({
|
||||
kind: z.literal("summary"),
|
||||
parent_evidence_id: z.string(),
|
||||
summary: z.object({
|
||||
steps: z.array(z.object({ step_id: z.string() })),
|
||||
}),
|
||||
}),
|
||||
})
|
||||
.parse(response.structuredContent).normalized_result;
|
||||
expect(summary.summary.steps.map(({ step_id }) => step_id)).toEqual([
|
||||
"scenario_start",
|
||||
"settle",
|
||||
]);
|
||||
// Delivery detail is neither sent to the provider nor recorded as a scenario parameter.
|
||||
expect(provider.scenarios[0]).not.toHaveProperty("detail");
|
||||
const parent = session.evidenceById(summary.parent_evidence_id);
|
||||
expect(parent?.operation).toBe("capture_browser_scenario");
|
||||
expect(parent?.parameters).not.toHaveProperty("detail");
|
||||
|
||||
const page = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: summary.parent_evidence_id,
|
||||
},
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "steps",
|
||||
step_indices: [1],
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(page.isError, JSON.stringify(page)).not.toBe(true);
|
||||
expect(page.structuredContent).toMatchObject({
|
||||
normalized_result: {
|
||||
kind: "browser-scenario",
|
||||
parent_evidence_id: summary.parent_evidence_id,
|
||||
items: {
|
||||
collection: "steps",
|
||||
rows: [{ step_index: 1, step_id: "settle", artifacts: {} }],
|
||||
},
|
||||
coverage: { total: 1, exhausted: true },
|
||||
},
|
||||
});
|
||||
|
||||
const invalid = await client.callTool({
|
||||
name: "capture_browser_scenario",
|
||||
arguments: { ...minimalScenario(), detail: "brief" },
|
||||
});
|
||||
expect(parseMcpToolError(invalid)).toMatchObject({
|
||||
error: { code: "invalid_request" },
|
||||
});
|
||||
expect(provider.scenarios).toHaveLength(1);
|
||||
} finally {
|
||||
await Promise.all([client.close(), server.close(), session.close()]);
|
||||
}
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user