docs(environment): record the derived-values verification

Gates, tests, write plans over 69 recorded captures and served write tasks on the baseline and on this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar Arakelyan
2026-10-24 09:55:16 -07:00
parent a1edeafdd5
commit 8ce1127b5d
@@ -0,0 +1,74 @@
# Verification
Executed on macOS with Node v24.22.1 at `a1edeafd` (main `79845063` plus this branch). The prompt-built recordings and
evaluations used a local, unpushed build of this branch merged with the scenario-timing branch (#6967), whose request
times the clock rules need from browser scenarios, on Ubuntu 24.04 VMs with snap Chromium 155. The baseline is the same
merge without this branch.
## Gates
- `npm run -s typecheck`, `lint`, `format:check`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`,
`scan:todos` and `npx knip`: passed.
- `npx vitest run --project domain --project services --project adapters`: 17,246 passed, 25 skipped, 1 failed ("stops
Inspector target authorization after the current stat settles"), which fails the same way on main in this environment.
- `npm run test:prepare`, then `npm run test:focused --` the environment build, export and serve CLI boundaries, the
analysis-view CLI and MCP boundaries, the environment serve, served-writes and bundle MCP boundaries, MCP schema
validation and output schema constraints: 10 files, 82 passed.
## Tests
- View: Go-spelled tag creates and renames are served with a trimmed clock; bookmark creates are served with a composed
snapshot address and accept any URL; folder creates and renames are served with a composed address and a generated
integer id.
- Serving: a served tag create spells its clocks without trailing zeros; a served bookmark create composes its snapshot
address from the served clock and the sent URL; a served folder create gets a fresh sixteen-digit id and an address
that follows its uid and title, and a rename moves the address.
- Unit: fitting clock formats (trimmed, fixed width, separator and zone), calendar digits, seeded integers, generated
integer detection, composition and its refusals (one call alone, a held entity spelling otherwise, two sources with
one text, a member stored otherwise than sent, a value the parts do not spell before the call), and a wide body
composed within the existing time limit.
## Write plans over recorded captures
Every write plan of 69 captures, built by the baseline and by this branch: the 50 published captures, 18 captures
Claude Code recorded from a prompt for the prompt branch's checks, and one new timed Grafana recording.
| Captures | Write plans | Served on the baseline | Served on this branch |
| ------------------------- | ----------- | ---------------------- | --------------------- |
| 50 published | 187 | 68 | 68 |
| 19 recorded from a prompt | 120 | 56 | 61 |
- No write goes from served to withheld. The five gains are Linkding's bookmark create, Homebox's tag create, and
Grafana's folder create in two recordings and folder rename in the timed one.
- Published plans change only where a value is now composed: Linkding's bookmark create and BookStack's book create and
update (`description_html` as `<p>` around the description) now accept the member they read.
- Unchanged by design: Grafana's published folder, whose uid and title slug are the same text, so its address names two
sources; Ghost's post update, whose `lexical` keeps the `derived` rule that explained it before; BookStack's page HTML
and Jellyfin's playlist path, which other pages and playlists of the capture spell otherwise.
- What still withholds a create the composed value now explains: a later Linkding create that sends a member the first
did not (`shared`), Homebox item creates that differ from the first location create, and a Linkding create whose
read-back lists `tag_names` sorted.
## Served write tasks
The same timed captures, rebuilt by the baseline and by this branch, with the models `fireworks_ai/gpt-oss-120b` and
`fireworks_ai/glm-5p3-flash`. Create tasks are new; recorded write tasks are the applications' existing ones with entity
counts taken relative to each bundle's state.
| Application | Tasks | Baseline (gpt-oss, GLM) | This branch (gpt-oss, GLM) |
| ----------- | ---------------------------------- | ----------------------- | -------------------------- |
| Linkding | 2 bookmark creates | 0/2, 0/2 | 2/2, 2/2 |
| Linkding | 3 recorded (update, notes, delete) | 3/3, 3/3 | 3/3, 3/3 |
| Homebox | 2 tag creates | 0/2, 0/2 | 2/2, 2/2 |
| Homebox | 2 recorded deletes | 0/2, 0/2 | 0/2, 0/2 |
| Grafana | folder create and rename | not served | 1/2, 2/2 |
| Grafana | 2 recorded deletes | | 2/2, 2/2 |
- Smoke and retrieval match on both builds (Linkding 9/9 and 8/9, Homebox 3/3 and 3/3); Grafana's new recording passes
its smoke task and 9/9 retrieval with both models.
- gpt-oss missed the Grafana rename by sending the update without the title it was asked to change, ten times; the
served environment refused each call with the members it accepts.
- Homebox's recorded deletes fail on both builds: this recording's served entity list holds its locations only, so the
models never find the item to delete.
- The created bookmarks' snapshot addresses and the created and renamed folders' addresses are checked from the
environment's state.