docs(browser): add an interaction example that captures the visible result (#750)

* docs(browser): add an interaction example that captures the visible result

Fixes #728

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(browser): wait for asynchronous search results before capture

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: N0zoM1z0 <1705004398@qq.com>
This commit is contained in:
Tufail Akram
2026-10-06 22:54:22 +05:30
committed by GitHub
parent 9a4e6da5a9
commit 91daaa68ae
3 changed files with 96 additions and 9 deletions
+6 -2
View File
@@ -400,8 +400,12 @@ the launched browser. Connect mode accepts one exact loopback CDP target and
disconnects without closing the external browser. The request supplies the
selected executable or endpoint, actions, and any origin or environment
selections needed by the scenario. Scenario JSON contains secret references and
environment-variable names, never secret values. See the
[browser scenario contract](docs/browser-scenario-contract.md).
environment-variable names, never secret values. The default capture retains
only the final URL; request `dom`, `accessibility`, or `screenshot` when an
interaction changes the page without navigating. For asynchronous updates,
wait for a page-specific result-ready condition before capture. See the
[browser scenario contract](docs/browser-scenario-contract.md), including its
interaction example.
### Node and Electron V8 Inspector observation
+81
View File
@@ -49,6 +49,87 @@ profile for this request; it does not start your fixture server or install a
browser. A schema-valid scenario does not prove that the executable or target
is available on the host.
## Interaction example: capture the visible result
The default final URL answers "where did the page end up?", not "what did the
page show?". Many interactions, such as a search button that renders results in
place, change the page without changing the URL. A capture with only the
default `url` artifact then reports that the click succeeded and the URL is
unchanged, which cannot tell you whether results appeared.
Request the artifacts that answer your question and wait for the state you want
to inspect. This scenario clicks a Search button, waits for the current search
results, and captures the final DOM, accessibility tree, and URL. It assumes the
local page clears `#results`'s `data-state` while loading and sets it to `ready`
after rendering the current response. Adapt the selectors and readiness marker
to your page; an already-visible container or previous results are insufficient.
```json
{
"browser": {
"mode": "launch",
"executable_path": "/absolute/path/to/chromium"
},
"start_url": { "url": "http://127.0.0.1:3000" },
"actions": [
{
"step_id": "search",
"action": "click",
"locator": { "kind": "css", "selector": "#search" },
"timeout_ms": 10000
},
{
"step_id": "results-ready",
"action": "wait_for",
"locator": { "kind": "css", "selector": "#results[data-state='ready']" },
"state": "visible",
"timeout_ms": 10000
}
],
"capture": { "at_end": ["dom", "accessibility", "url"] }
}
```
Save it as `browser-search.json` and run it the same way as the minimal
example:
```bash
rea capture-browser-scenario ./browser-search.json --json > browser-search-capture.json
```
Through MCP, pass the same object as the `capture_browser_scenario` arguments.
The final step's DOM and accessibility artifacts retain the page state at that
capture time inline. REA still launches and cleans up the browser profile.
The click's `timeout_ms` covers the click action. A non-navigating `fetch` or
delayed render can finish later, so use `wait_for` to observe a page-specific
ready condition before final capture. A page that updates synchronously can
omit that wait. REA does not infer application readiness from capture selection.
Choose captures by the question being asked:
- `dom` and `accessibility`: text or structure the page rendered, such as
search results, validation messages, or updated lists. The accessibility tree
is usually the more compact way to read visible text.
- `screenshot`: visual layout or styling, or content not exposed as text, such
as a canvas.
- `history` or `storage`: navigation history, or cookies and web storage the
interaction wrote. Request them only when that state is the question; they
are never captured implicitly.
- `after_each_step` instead of `at_end`: intermediate state between actions in
a multi-step scenario.
- `events` (`console`, `page-errors`, `network`, and so on): how the page
reached its result, such as the request a search issued or an error it
logged. Select only the event families you need.
Capture states are reported per artifact. An artifact you did not request is
`not_requested`, which means REA did not look, not that the page had nothing
to show. A step's `completeness` of `complete` covers only the sections that
were requested. A URL-only capture can be `complete` and still be unable to
answer a question about rendered content, so add the relevant artifact and run
the scenario again rather than inferring from its absence. Capture completeness
also does not assert that the page's background work has finished.
The browser boundary is part of the contract. Launch mode requires a
caller-selected executable and always uses a provider-owned temporary profile
that is closed and deleted during cleanup. Connect mode accepts only an
+9 -7
View File
@@ -64,22 +64,24 @@ afterEach(async () => {
});
describe("canonical product catalog", () => {
it("admits the documented minimal browser scenario through the named contract", async () => {
it("admits every documented browser scenario through the named contract", async () => {
const guide = await readFile(
join(root, "docs/browser-scenario-contract.md"),
"utf8",
);
const example = /```json\n([\s\S]*?)\n```/u.exec(guide)?.[1];
if (example === undefined)
throw new Error("Missing browser scenario example");
const examples = [
...guide.matchAll(/```json\r?\n([\s\S]*?)\r?\n```/gu),
].map((match) => match[1] ?? "");
expect(examples.length).toBeGreaterThanOrEqual(2);
const contract = TOOL_CONTRACTS.find(
({ name }) => name === "capture_browser_scenario",
);
if (contract === undefined)
throw new Error("Missing browser scenario contract");
expect(contract.inputSchema.safeParse(JSON.parse(example)).success).toBe(
true,
);
for (const example of examples)
expect(contract.inputSchema.safeParse(JSON.parse(example)).success).toBe(
true,
);
});
it("matches every source-derived checked-in product fact", async () => {