From 38719a6fa9a1301eeee647e6749701a9b83f10bb Mon Sep 17 00:00:00 2001 From: Edgar Arakelyan Date: Sat, 24 Oct 2026 14:06:21 -0700 Subject: [PATCH] feat(environment): explain values an application derives from other fields (#7025) * feat(environment): explain values an application derives from other fields Prompt-built environments withheld creates whose responses carry values the application derives deterministically, which the write model kept as constants or could not match across calls: - A composed string is literal text plus parts read from the fields other rules write: a field's value, its slug, or a clock's calendar digits. A create composes a string only when another entity of the type spells it the same way and none spells it otherwise, and never from two sources serving could tell apart; an update composes an unsent change from members stored as sent and unchanged fields that also spell its value before the call. A composed field no longer pins the members it reads. - A trimmed clock format drops trailing fraction zeros, as Go spells nanoseconds. It is learned from the type's values, a later create or a later update, and spreads to the plan's other clocks unless a value shows a fixed width. - A large integer no entity of the type holds, among distinct ones of the same length and kind, is a generated unique value, served as a seeded integer. Co-Authored-By: Claude Opus 5.5 * docs(environment): record the derived-values verification Gates, tests, write plans over 69 recorded captures and served write tasks on the baseline and on this branch. Co-Authored-By: Claude Opus 5.5 * fix(environment): keep clock spellings, linked ids and word boundaries honest - A later update's clocks take the plan's spelling, sibling-trimmed clocks included, wherever the call's values fit it, so an update whose later call trims only one of two clocks is served. - A held value that ends its fraction in a zero keeps a create's clock at a fixed width, whatever order the held spellings come in. - A large integer that another field of the created entity also holds is linked, as a repeated token is: it is never unique, and the create is withheld as unexplained-change. - Composed parts are delimited by whole characters through the shared joinsBefore/joinsAfter, so an astral letter joins a match to its word. Co-Authored-By: Claude Opus 5.5 * test(environment): keep a same-width fixed clock out of trimming A held .123456780 beside a held .12345678 keeps the create's clock at nine digits in either order; the cases fail on 8ce1127b5. Co-Authored-By: Claude Opus 5.5 * fix(environment): check composed parts and clock fits before they shape a plan - A composed string whose part has nothing to spell refuses the create or update through the apply Result instead of keeping a stale value. - A composition learned on a later call is rebuilt from each earlier call's fields, so an earlier call that left the value otherwise contradicts it. - An update adopts its fitted clock spellings only after the unknown-entity and contradiction checks, so a call on an entity an unmodelled write left unknown no longer trims a served plan's clock. - Sibling trimming on a later create also checks that call's fields, so a sibling clock it shows with a trailing zero stays fixed-width. Co-Authored-By: Claude Opus 5.5 * test(environment): split the derived-values view tests by rule Co-Authored-By: Claude Opus 5.5 * fix(environment): keep fixed-width, zone and generated-integer evidence through a plan - A plan keeps the first value that showed each clock at a fixed width, from held entities and every accepted create or update. No later fit or sibling trim widens that field, and a later call that shows it shorter is withheld. - Two stamped clocks are one composed source only when they share a zone offset. A held entity that holds the target field as anything the parts do not spell, a non-string included, vetoes the composition. The token index is built once, and only when a string is left to compose. - A generated integer that another field repeats as a number at any depth is linked. Compare treats two integers of one JSON kind and length under a unique rule as generated values. Co-Authored-By: Claude Opus 5.5 * test(setup): share registration prewarm host preparation * fix(environment): retain write evidence and match trimmed response clocks * fix(environment): withhold creates whose generated integer another string repeats A unique integer whose digits show in another top-level string that no template or composed rule rebuilds from it (an echoed serialText, or a token like folder-) withholds the create as unexplained-change, since serving would seed the integer and keep the string. A templated copy is still served. Co-Authored-By: Claude Opus 5.5 * fix(environment): hold later creates and compare to the composed and integer forms - A later create whose response spells a composed field otherwise than the plan rebuilds it is withheld as unexplained-change. - A unique field learned from a generated integer accepts only an integer of the same length and kind from a later create, as serving seeds one. - Compare counts a composed field as equal when each side's response spells it from its own fields, which compare under their own rules. Co-Authored-By: Claude Opus 5.5 * fix(environment): preserve composed parts and unique generator families --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- docs/agent-environments.md | 19 +- .../write-derived-values/plan.md | 12 + .../write-derived-values/spec.md | 59 ++ .../write-derived-values/tasks.md | 10 + .../write-derived-values/verification.md | 91 +++ docs/web-network-captures.md | 25 +- .../environmentCompareDiff.test.ts | 352 ++++++++++++ .../environment/environmentCompareDiff.ts | 104 +++- .../environmentStateDerivedAnswer.test.ts | 84 +++ .../environmentStateWriteAnswer.ts | 12 +- .../EnvironmentStateWriteSimulation.ts | 1 + .../environmentStateDerivedWrites.test.ts | 517 ++++++++++++++++++ .../analysisView/environmentStateView.ts | 22 + .../environmentStateWriteApply.test.ts | 78 +++ .../environmentStateWriteApply.ts | 26 +- .../environmentStateWriteCalls.ts | 17 + .../environmentStateWriteChanges.ts | 5 + .../environmentStateWriteClockFormats.test.ts | 111 ++++ .../environmentStateWriteClockFormats.ts | 164 ++++++ .../environmentStateWriteComposed.test.ts | 302 ++++++++++ .../environmentStateWriteComposed.ts | 274 ++++++++++ .../environmentStateWriteCreate.ts | 119 +++- .../environmentStateWriteEffects.ts | 107 +++- .../environmentStateWriteExplain.test.ts | 86 +++ .../environmentStateWriteExplain.ts | 114 +++- .../environmentStateWriteRules.test.ts | 52 ++ .../environmentStateWriteRules.ts | 55 ++ .../environmentStateWriteValues.test.ts | 65 ++- .../environmentStateWriteValues.ts | 95 +++- .../cli/environmentCompareCli.test.ts | 90 +++ .../fixtures/environmentStateDerivedValues.ts | 484 ++++++++++++++++ 31 files changed, 3467 insertions(+), 85 deletions(-) create mode 100644 docs/specs/agent-environments/write-derived-values/plan.md create mode 100644 docs/specs/agent-environments/write-derived-values/spec.md create mode 100644 docs/specs/agent-environments/write-derived-values/tasks.md create mode 100644 docs/specs/agent-environments/write-derived-values/verification.md create mode 100644 src/application/environment/environmentStateDerivedAnswer.test.ts create mode 100644 src/domain/analysisView/environmentStateDerivedWrites.test.ts create mode 100644 src/domain/analysisView/environmentStateWriteClockFormats.test.ts create mode 100644 src/domain/analysisView/environmentStateWriteClockFormats.ts create mode 100644 src/domain/analysisView/environmentStateWriteComposed.test.ts create mode 100644 src/domain/analysisView/environmentStateWriteComposed.ts create mode 100644 src/domain/analysisView/environmentStateWriteRules.test.ts create mode 100644 tests/fixtures/environmentStateDerivedValues.ts diff --git a/docs/agent-environments.md b/docs/agent-environments.md index 62db30bb65..4b812f9659 100644 --- a/docs/agent-environments.md +++ b/docs/agent-environments.md @@ -721,7 +721,7 @@ every MCP session and `data/get` then see: | Effect | Call | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. | -| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, unique and renewed changes the plan models. | +| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, unique and renewed changes the plan models. | | `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), with any sent members; it is served only when a recorded call found those fields already set, so a toggle never is. | | `delete` | Removes the entity the path names. | | bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). | @@ -923,6 +923,9 @@ Known limits of served writes: updates require `updated_at` to match the entity (a `held-clock`). - A slug derived from a title can repeat across served creates with the same title, where the application would make it unique. +- A `composed` string copies sent text as sent: an application that escapes or + renders it (HTML entities, Markdown) differs for text that needs it, as + BookStack's `description_html` would for a description holding `&`. - A served create's sequence id is the state's next value, not the gaps an application's own sequence would leave. - Nested clocks are served in the spelling of the write's own response, which @@ -1057,9 +1060,11 @@ Numeric arrays retain that containing field's clock context through nested arrays; an object's own fields establish their own context. Numeric values at the body root, in root arrays, or under measurement fields remain literal. -Generated strings normalize only where this confirmed, served write's entity -response rules identify generation. A `unique` rule or UUID identity rule must -still compare two UUIDs; a `renewed` rule applies only at its recorded pointers, +Generated values normalize only where this confirmed, served write's entity +response rules identify generation. A UUID identity rule must still compare two +UUIDs, and a `unique` rule two UUIDs or two generated integers (10 to 20 +digits) of one JSON kind and length; a `renewed` rule applies only at its +recorded pointers, with `*` matching actual array items. Token/sequence strings retain the existing format check: equal length of at least eight, the same character classes, and a digit. Numeric sequences continue to compare literally unless an ID mapping @@ -1071,6 +1076,12 @@ Admitted generated values must also preserve aliases and distinct identities within that response. UUID identity uses the generator's case-insensitive matching; token and sequence strings retain their literal identity. This correspondence ends with the answer and is not inferred across calls. +A `composed` response field must spell the value its parts build from that +response's own fields. Its parts then compare by their rendered text or their +source fields' clock and ID rules. A coherent archive URL can therefore differ +only in normalized clock digits without reporting a value difference; an +incoherent URL still differs even when both sides repeat the same stale text. +Normalized compositions count under `generated`; source fields are counted once. The report counts each kind of normalised value; the generation rules remain model-derived facts, rather than new observations of the application. diff --git a/docs/specs/agent-environments/write-derived-values/plan.md b/docs/specs/agent-environments/write-derived-values/plan.md new file mode 100644 index 0000000000..a5ae3f29f8 --- /dev/null +++ b/docs/specs/agent-environments/write-derived-values/plan.md @@ -0,0 +1,12 @@ +# Implementation plan + +1. **Clock formats.** A `trimmed` ISO format, `fixedWidth` on a clock reading, fitting a format to another value, and + calendar digits, in a clock formats module. +2. **Composed strings.** A module that indexes a create's candidate parts once, segments a string, checks the type's + other entities and an update's before value, and spells the string from written fields. +3. **Explaining.** Create explanation composes unexplained strings after templates, learns trimmed clocks from the + type and generated integers; update explanation composes unsent changes after the existing rules. +4. **Later calls.** A later create or update widens the plan's clocks to a trimmed spelling it shows; a later create's + unique value may be a generated integer. +5. **Serving.** Derived fields are rebuilt after other rules; unique integers are seeded by length and kind. +6. **Schema, tests and docs.** The plan schema, view, serving and unit tests, and the write-plan docs. diff --git a/docs/specs/agent-environments/write-derived-values/spec.md b/docs/specs/agent-environments/write-derived-values/spec.md new file mode 100644 index 0000000000..efc90d707a --- /dev/null +++ b/docs/specs/agent-environments/write-derived-values/spec.md @@ -0,0 +1,59 @@ +# Explain values an application derives from other fields + +Issue: https://github.com/morluto/rea/issues/4933 + +## Outcome + +Environments an agent built from a prompt withheld creates in three of five applications, each for a value the write +model could not explain, though the application derives it deterministically: + +- Linkding's `web_archive_snapshot_url` is `https://web.archive.org/web//`. + The plan kept it as a constant, which pinned `url` to its recorded value, so a later create with another URL was + refused. +- Grafana's folder `url` is `/dashboards/f//`, which pinned `uid` the same way, and its numeric + `id` is a large time-based integer the plan copied from the first create, which the second create's read-back + contradicted. +- Homebox, written in Go, spells nanosecond clocks with trailing fraction zeros dropped (`…34.38812218Z`), so a later + create's clock did not follow the format learned from the first. + +## Behaviour + +- **Composed strings.** A created field no other rule explains may be a `composed` string: literal text and parts read + from the fields other rules write, each a field's value, its slug, or a clock's calendar digits (`YYYYMMDDhhmmss`, + `YYYYMMDD`, `YYYY-MM-DD`, `YYYY/MM/DD`, in the clock's zone). Parts are found left to right, longest first, delimited + by non-alphanumerics; generated values may be short, other values need eight characters, as a copy does. A create + composes a string only when at least one other entity of the type spells it with the same parts and none spells it + otherwise, and never when one text comes from two sources serving could tell apart (two clocks the write stamps are + one source when they spell the same zone), and never when another entity holds the field as something other than the + parts spell, a non-string value included. An update's unsent change may be composed from members it stored as sent and + fields it left as they were, when the same parts also spell the value before the call and no other entity spells it + otherwise. Serving rebuilds the string from the written fields after every other rule and refuses a call when a part's + field holds nothing to spell, and a composed field no longer pins the members it reads. A composition learned on a + later call is checked against every earlier call, and a later create whose response spells a composed field otherwise + is withheld. +- **Trimmed clocks.** A clock format may be `trimmed`: its values differ only in how many fraction digits they show and + none ends a fraction in a zero. It is learned from the type's held values of the field, from a later create, and from + a later update; other clocks the plan stamps take the spelling too, and no clock takes it once a held entity, the + exemplar or any call the plan accepted showed the field at a fixed width; a later call that then shows it shorter is + withheld. `fraction` is the most digits shown, and serving drops trailing zeros and the point with them. +- **Generated integers.** A created integer of 10 to 20 digits that no entity of the type holds, where every held entity + holds a distinct one of the same length and JSON kind and none is a clock, is `unique`. Serving draws a seeded integer + of that length and kind that no held entity has. One that another field of the created entity also holds, as a value, + a number at any depth or a delimited part of a nested string, is not `unique`, and the create is withheld as + `unexplained-change`, since seeding cannot keep the two equal. A create is also withheld when another string holds its + digits and no `template` or `composed` rule rebuilds that string from it, or when a later create answers the field + with anything but an integer of the same length and kind. Compare treats two such integers of one JSON kind and length + under a `unique` rule as generated values. + +Plans that need none of this are unchanged. + +Later creates must retain the exemplar's unique-value family (UUID, integer with +the same digit count and JSON kind, or opaque token). Their recorded composed +fields must equal the values the plan rebuilds; simulation cannot replace a +contradictory captured value and then count that replacement as confirmation. + +## Acceptance + +- The prompt-built Linkding, Homebox and Grafana captures serve the creates and renames they withheld, and write + tasks that use them pass. +- No write tool of the 50 published captures or the earlier prompt-built captures goes from served to withheld. diff --git a/docs/specs/agent-environments/write-derived-values/tasks.md b/docs/specs/agent-environments/write-derived-values/tasks.md new file mode 100644 index 0000000000..ae9e074b47 --- /dev/null +++ b/docs/specs/agent-environments/write-derived-values/tasks.md @@ -0,0 +1,10 @@ +# Tasks + +- [x] Read and fit trimmed clock formats, and spell calendar digits. +- [x] Compose created strings and unsent update changes from other fields. +- [x] Learn generated integers as unique values and serve them by length and kind. +- [x] Widen plan clocks on later creates and updates, and spread a trimmed spelling to the plan's other clocks. +- [x] Add view, serving and unit tests. +- [x] Document the rules. +- [x] Measure the prompt-built captures, the 50 published captures and the served write tasks. +- [x] Run static, formatting, documentation and test gates. diff --git a/docs/specs/agent-environments/write-derived-values/verification.md b/docs/specs/agent-environments/write-derived-values/verification.md new file mode 100644 index 0000000000..5f74a2d775 --- /dev/null +++ b/docs/specs/agent-environments/write-derived-values/verification.md @@ -0,0 +1,91 @@ +# Verification + +Executed on macOS with Node v24.22.1 at `a1edeafd` (main `79845063` plus this branch). The prompt-built recordings and +evaluations used a local, unpushed build of this branch merged with the scenario-timing branch (#6967), whose request +times the clock rules need from browser scenarios, on Ubuntu 24.04 VMs with snap Chromium 155. The baseline is the same +merge without this branch. + +## Gates + +Follow-up on 2026-10-25, after merging main `a5bfd0440`, with Node 24.22.1 and npm 11.16.0 on macOS: + +- Environment write/compare source tests: 26 files, 839 passed. +- Analysis-view CLI/MCP, environment served-write MCP, environment-compare CLI, managed bundle read guards, and setup + registration prewarm boundaries: 6 distinct files, 77 passed. The managed low-headroom failure is fixed by main's + Evidence ownership change (#7036); the setup tests share host preparation to satisfy lint without removing cases. +- `check:fast`, `format:check`, and `docs:check`: passed. +- New capture-to-plan regressions reproduce missing composition parts on earlier updates (null and structured sources) + and fixed-width clock evidence from either the selected entity or another held entity. Rule tests exercise several + separator/zone groups in both orders, choosing each group's widest trimmed peer while preserving fixed-width fields. +- Compare uses the domain clock matcher on parsed readings. Tests cover shorter fractions, absent fractions, maximum + width, trailing zeros, separators, zones, and response-entity scope; the compiled CLI accepts a Go-style create whose + live clocks have eight digits and served clocks have none, preserving all input Evidence files. +- The original application's recordings and model evaluations below were not rerun in this follow-up. + +Original author validation: + +- `npm run -s typecheck`, `lint`, `format:check`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`, + `scan:todos` and `npx knip`: passed. +- `npx vitest run --project domain --project services --project adapters`: 17,246 passed, 25 skipped, 1 failed ("stops + Inspector target authorization after the current stat settles"), which fails the same way on main in this environment. +- `npm run test:prepare`, then `npm run test:focused --` the environment build, export and serve CLI boundaries, the + analysis-view CLI and MCP boundaries, the environment serve, served-writes and bundle MCP boundaries, MCP schema + validation and output schema constraints: 10 files, 82 passed. + +## Tests + +- View: Go-spelled tag creates and renames are served with a trimmed clock; bookmark creates are served with a composed + snapshot address and accept any URL; folder creates and renames are served with a composed address and a generated + integer id. +- Serving: a served tag create spells its clocks without trailing zeros; a served bookmark create composes its snapshot + address from the served clock and the sent URL; a served folder create gets a fresh sixteen-digit id and an address + that follows its uid and title, and a rename moves the address. +- Unit: fitting clock formats (trimmed, fixed width, separator and zone), calendar digits, seeded integers, generated + integer detection, composition and its refusals (one call alone, a held entity spelling otherwise, two sources with + one text, a member stored otherwise than sent, a value the parts do not spell before the call), and a wide body + composed within the existing time limit. + +## Write plans over recorded captures + +Every write plan of 69 captures, built by the baseline and by this branch: the 50 published captures, 18 captures +Claude Code recorded from a prompt for the prompt branch's checks, and one new timed Grafana recording. + +| Captures | Write plans | Served on the baseline | Served on this branch | +| ------------------------- | ----------- | ---------------------- | --------------------- | +| 50 published | 187 | 68 | 68 | +| 19 recorded from a prompt | 120 | 56 | 61 | + +- No write goes from served to withheld. The five gains are Linkding's bookmark create, Homebox's tag create, and + Grafana's folder create in two recordings and folder rename in the timed one. +- Published plans change only where a value is now composed: Linkding's bookmark create and BookStack's book create and + update (`description_html` as `

` around the description) now accept the member they read. +- Unchanged by design: Grafana's published folder, whose uid and title slug are the same text, so its address names two + sources; Ghost's post update, whose `lexical` keeps the `derived` rule that explained it before; BookStack's page HTML + and Jellyfin's playlist path, which other pages and playlists of the capture spell otherwise. +- What still withholds a create the composed value now explains: a later Linkding create that sends a member the first + did not (`shared`), Homebox item creates that differ from the first location create, and a Linkding create whose + read-back lists `tag_names` sorted. + +## Served write tasks + +The same timed captures, rebuilt by the baseline and by this branch, with the models `fireworks_ai/gpt-oss-120b` and +`fireworks_ai/glm-5p3-flash`. Create tasks are new; recorded write tasks are the applications' existing ones with entity +counts taken relative to each bundle's state. + +| Application | Tasks | Baseline (gpt-oss, GLM) | This branch (gpt-oss, GLM) | +| ----------- | ---------------------------------- | ----------------------- | -------------------------- | +| Linkding | 2 bookmark creates | 0/2, 0/2 | 2/2, 2/2 | +| Linkding | 3 recorded (update, notes, delete) | 3/3, 3/3 | 3/3, 3/3 | +| Homebox | 2 tag creates | 0/2, 0/2 | 2/2, 2/2 | +| Homebox | 2 recorded deletes | 0/2, 0/2 | 0/2, 0/2 | +| Grafana | folder create and rename | not served | 1/2, 2/2 | +| Grafana | 2 recorded deletes | | 2/2, 2/2 | + +- Smoke and retrieval match on both builds (Linkding 9/9 and 8/9, Homebox 3/3 and 3/3); Grafana's new recording passes + its smoke task and 9/9 retrieval with both models. +- gpt-oss missed the Grafana rename by sending the update without the title it was asked to change, ten times; the + served environment refused each call with the members it accepts. +- Homebox's recorded deletes fail on both builds: this recording's served entity list holds its locations only, so the + models never find the item to delete. +- The created bookmarks' snapshot addresses and the created and renamed folders' addresses are checked from the + environment's state. diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index 866494acb7..7459535193 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -740,9 +740,22 @@ explains it: generated `identity` (a sequence, UUID or token, by the recorded value's shape), `echo` of a body member, `copy` of a body string of 8 or more characters, a `path` value, a `slug` of a body member, a `clock` value within 10 minutes of the request in its recorded format, a `sequence` one past the -largest value of its collection or type, a `unique` UUID or opaque token, a -`nested-clock` object whose clocks the request stamped, or a `template` around -a generated value; other fields keep the observed value. A body that wraps one +largest value of its collection or type, a `unique` UUID, opaque token or +integer of 10 or more digits that no entity of the type holds (where every one +holds a distinct integer of the same length, as time-based ids do), a +`nested-clock` object whose clocks the request stamped, a `template` around +a generated value, or a `composed` string; other fields keep the observed value. +A `composed` string is literal text and parts read from the fields other rules +write: a field's value, its slug, or a clock's calendar digits +(`YYYYMMDDhhmmss`, `YYYYMMDD`, `YYYY-MM-DD` or `YYYY/MM/DD`, in the clock's +zone). A create composes a string only when at least one other entity of the +type spells it with the same parts and none spells it otherwise, and never when +one text could come from two sources serving could tell apart. A clock format +is `trimmed` when the field's values differ only in how many fraction digits +they show and none ends a fraction in a zero, as Go spells nanoseconds: its +`fraction` is the most it shows, and serving drops trailing zeros and the point +with them. Other clocks the plan stamps take that spelling too unless their own +value shows a fixed width. A body that wraps one object in a one-element array is that object, wrapped so on every call (`body.envelope`). An update's members echo when stored as sent; a member stored differently is accepted only with its @@ -750,8 +763,10 @@ recorded value (`body.fixed`) and stores the value the call that showed it recorded (the `value` of its `constant` rule), and a field it changed without sending must be a clock, a nested clock, an increment, a slug, a fresh `unique` value, a `stamp` of the acting user, a value `derived` from the one member it -stores nowhere, or for an item action a `set` of a scalar to the value it -showed, or the plan is `unexplained-change`; an increment, slug, stamp, derived +stores nowhere, a `composed` string whose parts read members it stored as sent +or fields it left as they were and spell the value before the call too, or for +an item action a `set` of a scalar to the value it showed, or the plan is +`unexplained-change`; an increment, slug, stamp, derived value or set must hold for every call of the tool, and an action is served only once a call found what it sets already set. A write is served only with the query its first call sent, which serving applies to a call that diff --git a/src/application/environment/environmentCompareDiff.test.ts b/src/application/environment/environmentCompareDiff.test.ts index 53c52e2208..174cf341b4 100644 --- a/src/application/environment/environmentCompareDiff.test.ts +++ b/src/application/environment/environmentCompareDiff.test.ts @@ -4,6 +4,7 @@ import type { JsonValue } from "../../domain/jsonValue.js"; import type { FieldRule } from "../../domain/analysisView/environmentStateWriteRules.js"; import { compareAnswers } from "./environmentCompareDiff.js"; import { IdMap, learnIds } from "./environmentCompareIds.js"; +import { bookmark } from "../../../tests/fixtures/environmentStateDerivedValues.js"; const compare = ( live: unknown, @@ -34,6 +35,233 @@ describe("early-year clock formats", () => { ); }); +describe("composed response values", () => { + const rules: readonly FieldRule[] = [ + { + field: "date_added", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 0, zone: "Z" }, + }, + { + field: "date_modified", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 0, zone: "Z" }, + }, + { + field: "web_archive_snapshot_url", + rule: "composed", + parts: [ + { text: "https://web.archive.org/web/" }, + { from: "date_added", as: "clock", pattern: "YYYYMMDDhhmmss" }, + { text: "/" }, + { from: "url", as: "value" }, + ], + }, + ]; + const live = bookmark( + 1, + "https://example.test/page", + "A bookmark", + "2026-10-20T09:00:00Z", + ); + const env = bookmark( + 1, + "https://example.test/page", + "A bookmark", + "2026-10-17T18:00:00Z", + ); + + it("compares coherent archive URLs through their clock parts without counting source fields twice", () => { + const result = compare(live, env, new IdMap(), [201, 201], rules); + expect(result.differences).toEqual([]); + expect(result.normalised).toMatchObject({ clocks: 2, generated: 1 }); + }); + + it("compares a composed ID through the source field's mapping even when the address comes first", () => { + const map = new IdMap(); + map.learn(101, 1); + const result = compare( + { url: "/folders/101", id: 101 }, + { url: "/folders/1", id: 1 }, + map, + [201, 201], + [ + { + field: "url", + rule: "composed", + parts: [{ text: "/folders/" }, { from: "id", as: "value" }], + }, + ], + ); + expect(result.differences).toEqual([]); + expect(result.normalised).toMatchObject({ ids: 1, generated: 1 }); + }); + + it.each(["live", "env", "both"] as const)( + "reports an incoherent %s composition, including equal but stale URLs", + (side) => { + const invalid = "https://web.archive.org/web/stale"; + const result = compare( + side === "env" ? live : { ...live, web_archive_snapshot_url: invalid }, + side === "live" ? env : { ...env, web_archive_snapshot_url: invalid }, + new IdMap(), + [201, 201], + rules, + ); + expect(result.differences).toMatchObject([ + { category: "value", pointer: "/web_archive_snapshot_url" }, + ]); + expect(result.normalised.generated).toBe(0); + }, + ); + + it("preserves actual input differences although each address correctly spells its own response", () => { + const different = bookmark( + 1, + "https://example.test/other", + "A bookmark", + "2026-10-17T18:00:00Z", + ); + const result = compare(live, different, new IdMap(), [201, 201], rules); + expect(result.differences.map(({ pointer }) => pointer).sort()).toEqual([ + "/url", + "/web_archive_snapshot_url", + ]); + expect(result.normalised.generated).toBe(0); + }); + + it("does not normalize a source's unproven generated-looking business value", () => { + const result = compare( + { url: "/folders/token00001", token: "token00001" }, + { url: "/folders/token00002", token: "token00002" }, + new IdMap(), + [201, 201], + [ + { + field: "url", + rule: "composed", + parts: [{ text: "/folders/" }, { from: "token", as: "value" }], + }, + ], + ); + expect(result.differences.map(({ pointer }) => pointer).sort()).toEqual([ + "/token", + "/url", + ]); + }); + + it("compares the projected slug while retaining a difference in its source title", () => { + const result = compare( + { + title: "Hello!", + date_added: "2026-10-20T09:00:00Z", + url: "/snapshot/20261020/hello", + }, + { + title: "Hello", + date_added: "2026-10-17T18:00:00Z", + url: "/snapshot/20261017/hello", + }, + new IdMap(), + [201, 201], + [ + rules[0]!, + { + field: "url", + rule: "composed", + parts: [ + { text: "/snapshot/" }, + { from: "date_added", as: "clock", pattern: "YYYYMMDD" }, + { text: "/" }, + { from: "title", as: "slug" }, + ], + }, + ], + ); + expect(result.differences).toMatchObject([ + { category: "value", pointer: "/title" }, + ]); + expect(result.normalised).toMatchObject({ clocks: 1, generated: 1 }); + }); +}); + +describe("trimmed response clocks", () => { + const rules: readonly FieldRule[] = [ + { + field: "updatedAt", + rule: "clock", + format: { + kind: "iso", + separator: "T", + zone: "Z", + fraction: 8, + trimmed: true, + }, + }, + ]; + + it.each([".12345678", ".123", ""])( + "normalizes a learned trimmed width %s against the served millisecond width", + (fraction) => { + const result = compare( + { updatedAt: `2026-03-04T05:06:07${fraction}Z` }, + { updatedAt: "2026-03-04T05:06:08.321Z" }, + new IdMap(), + [200, 200], + rules, + ); + expect(result.differences).toEqual([]); + expect(result.normalised.clocks).toBe(1); + }, + ); + + it.each([ + ["2026-03-04T05:06:07.123456789Z", "2026-03-04T05:06:08.321Z"], + ["2026-03-04T05:06:07.123456780Z", "2026-03-04T05:06:08.321Z"], + ["2026-03-04T05:06:07.120Z", "2026-03-04T05:06:08.320Z"], + ["2026-03-04T05:06:07.123+01:00", "2026-03-04T05:06:08.321Z"], + ["2026-03-04 05:06:07.123Z", "2026-03-04T05:06:08.321Z"], + ])( + "reports a pair outside the learned trimmed format (%s, %s)", + (live, env) => { + const result = compare( + { updatedAt: live }, + { updatedAt: env }, + new IdMap(), + [200, 200], + rules, + ); + expect(result.differences).toMatchObject([ + { category: "clock-format", pointer: "/updatedAt" }, + ]); + expect(result.normalised.clocks).toBe(0); + }, + ); + + it("keeps trimmed normalization within the confirmed response entity", () => { + const answer = (fraction: string) => ({ + status: 200, + body: { + kind: "json" as const, + value: { + data: { updatedAt: `2026-03-04T05:06:07.${fraction}Z` }, + updatedAt: `2026-03-04T05:06:07.${fraction}Z`, + }, + }, + }); + const result = compareAnswers( + answer("12345678"), + answer("123"), + new IdMap(), + { at: "/data", rules }, + ); + expect(result.differences).toMatchObject([ + { category: "clock-format", pointer: "/updatedAt" }, + ]); + expect(result.normalised.clocks).toBe(1); + }); +}); + const registerGeneratedValueComparisonCases = () => { it.each([ { @@ -106,6 +334,57 @@ const registerGeneratedValueComparisonCases = () => { }); }; +describe("composed response fields", () => { + const rules: readonly FieldRule[] = [ + { + field: "date_added", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 0, zone: "Z" }, + }, + { field: "url", rule: "echo" }, + { + field: "web_archive_snapshot_url", + rule: "composed", + parts: [ + { text: "https://web.archive.org/web/" }, + { from: "date_added", as: "clock", pattern: "YYYYMMDDhhmmss" }, + { text: "/" }, + { from: "url", as: "value" }, + ], + }, + ]; + const bookmark = (stamp: string, snapshot: string) => ({ + date_added: `2026-10-01T09:00:${stamp}Z`, + url: "https://quarry.example.test/ledger", + web_archive_snapshot_url: `https://web.archive.org/web/${snapshot}/https://quarry.example.test/ledger`, + }); + + it("normalizes a composed field each side spells from its own fields", () => { + const result = compare( + bookmark("00", "20261001090000"), + bookmark("05", "20261001090005"), + new IdMap(), + [200, 200], + rules, + ); + expect(result.differences).toEqual([]); + expect(result.normalised).toMatchObject({ clocks: 1, generated: 1 }); + }); + + it("reports a composed field one side does not spell from its own fields", () => { + const result = compare( + bookmark("00", "20261001090000"), + bookmark("05", "20261001090009"), + new IdMap(), + [200, 200], + rules, + ); + expect( + result.differences.map(({ category, pointer }) => [category, pointer]), + ).toEqual([["value", "/web_archive_snapshot_url"]]); + }); +}); + describe("compareAnswers", () => { registerGeneratedValueComparisonCases(); @@ -671,3 +950,76 @@ describe("raw ids a mapping gives to other entities", () => { ).toEqual([]); }); }); + +describe("generated integer identities", () => { + const unique: readonly FieldRule[] = [{ field: "id", rule: "unique" }]; + + it.each([ + { + name: "numbers of one length", + live: 1_729_791_234_567_890, + env: 1_000_000_000_123_456, + rules: unique, + generated: 1, + }, + { + name: "digit strings of one length", + live: "1729791234567890", + env: "1000000000123456", + rules: unique, + generated: 1, + }, + { + name: "numbers of different lengths", + live: 1_729_791_234_567_890, + env: 100_000_000_012_345, + rules: unique, + generated: 0, + }, + { + name: "a number and a digit string", + live: 1_729_791_234_567_890, + env: "1000000000123456", + rules: unique, + generated: 0, + }, + { + name: "short integers", + live: 17, + env: 18, + rules: unique, + generated: 0, + }, + { + name: "integers no rule generates", + live: 1_729_791_234_567_890, + env: 1_000_000_000_123_456, + rules: [], + generated: 0, + }, + ])( + "matches $name only when a unique rule generates both", + ({ live, env, rules, generated }) => { + const result = compare( + { id: live }, + { id: env }, + new IdMap(), + [200, 200], + rules, + ); + expect(result.normalised.generated).toBe(generated); + expect(result.differences).toEqual( + generated === 1 + ? [] + : [ + { + category: "value", + pointer: "/id", + live: JSON.stringify(live), + env: JSON.stringify(env), + }, + ], + ); + }, + ); +}); diff --git a/src/application/environment/environmentCompareDiff.ts b/src/application/environment/environmentCompareDiff.ts index b028e99a12..25d2db8c13 100644 --- a/src/application/environment/environmentCompareDiff.ts +++ b/src/application/environment/environmentCompareDiff.ts @@ -1,9 +1,16 @@ import type { JsonObject } from "../../domain/analysisView/environmentStateRules.js"; import type { FieldRule } from "../../domain/analysisView/environmentStateWriteRules.js"; +import { matchesClockFormat } from "../../domain/analysisView/environmentStateWriteClockFormats.js"; import { + composedPartText, + composedValue, +} from "../../domain/analysisView/environmentStateWriteComposed.js"; +import { + likeGeneratedInteger, isUuid, readClock, type ClockFormat, + type ClockReading, } from "../../domain/analysisView/environmentStateWriteValues.js"; import type { JsonValue } from "../../domain/jsonValue.js"; import type { ReplayedBody } from "./environmentSource.js"; @@ -76,18 +83,17 @@ const formatKey = (format: ClockFormat): string => : `iso${format.separator}${String(format.fraction)}${format.zone === "" ? "" : format.zone === "Z" ? "Z" : "offset"}`; /** ISO text has its own clock syntax; numeric epochs require a time-named field. */ -const clockFormat = ( +const clockReading = ( value: JsonValue, key: string | undefined, -): string | undefined => { +): ClockReading | undefined => { if ( typeof value === "number" && (key === undefined || (!TIME_FIELD.test(key) && !TIME_FIELD_CAMEL.test(key))) ) return undefined; - const clock = readClock(value); - return clock === undefined ? undefined : formatKey(clock.format); + return readClock(value); }; const characterClasses = (text: string): string => @@ -108,14 +114,21 @@ interface Generation { readonly pointer: string; } -/** Whether a confirmed generation rule applies here and both strings retain its generated format. */ +/** Whether a confirmed generation rule applies here and both values retain its generated format. */ const generatedPair = ( - live: string, - env: string, + live: string | number, + env: string | number, generation: Generation | undefined, ): boolean => { if (generation === undefined) return false; const { rule, pointer } = generation; + if ( + rule.rule === "unique" && + pointer === "" && + likeGeneratedInteger(live, env) + ) + return true; + if (typeof live !== "string" || typeof env !== "string") return false; if (rule.rule === "renewed") return rule.pointers.includes(pointer) && isUuid(live) && isUuid(env); if (pointer !== "") return false; @@ -136,6 +149,7 @@ const generatedPair = ( class Differ { readonly #rules: ReadonlyMap; + readonly #composedRules: readonly Extract[]; readonly #generatedToEnv = new Map(); readonly #generatedToLive = new Map(); readonly differences: Difference[] = []; @@ -159,6 +173,9 @@ class Differ { }, ) { this.#rules = new Map(entity.rules.map((rule) => [rule.field, rule])); + this.#composedRules = entity.rules.filter( + (rule) => rule.rule === "composed", + ); } add(difference: Difference): void { @@ -166,13 +183,21 @@ class Differ { } /** One answer's generated identities must rename consistently in both directions. */ - #pairGenerated(live: string, env: string, rule: FieldRule): boolean { + #pairGenerated( + live: string | number, + env: string | number, + rule: FieldRule, + ): boolean { const uuid = rule.rule === "unique" || rule.rule === "renewed" || (rule.rule === "identity" && rule.generator === "uuid"); - const identity = (value: string) => - uuid ? `uuid:${value.toLowerCase()}` : `string:${value}`; + const identity = (value: string | number) => + typeof value === "number" + ? `number:${String(value)}` + : uuid + ? `uuid:${value.toLowerCase()}` + : `string:${value}`; const liveKey = identity(live); const envKey = identity(env); const knownEnv = this.#generatedToEnv.get(liveKey); @@ -222,12 +247,12 @@ class Differ { generation?: Generation, ): void { const key = at.clockKey ?? at.key; - const liveClock = clockFormat(live, key); - const envClock = clockFormat(env, key); + const liveClock = clockReading(live, key); + const envClock = clockReading(env, key); const generatedMatch = generation !== undefined && - typeof live === "string" && - typeof env === "string" && + (typeof live === "string" || typeof live === "number") && + (typeof env === "string" || typeof env === "number") && liveClock === undefined && envClock === undefined && !this.map.hasLive(live) && @@ -267,7 +292,17 @@ class Differ { return; } if (liveClock !== undefined && envClock !== undefined) { - if (liveClock === envClock) this.normalised.clocks += 1; + const rule = generation?.pointer === "" ? generation.rule : undefined; + const format = + rule?.rule === "clock" || rule?.rule === "held-clock" + ? rule.format + : undefined; + const matches = + format?.kind === "iso" && format.trimmed === true + ? matchesClockFormat(format, liveClock) && + matchesClockFormat(format, envClock) + : formatKey(liveClock.format) === formatKey(envClock.format); + if (matches) this.normalised.clocks += 1; else this.add({ category: "clock-format", @@ -387,6 +422,10 @@ class Differ { : key; }; const matched = new Set(); + const composed = + self && this.#composedRules.length > 0 + ? { agreed: new Set(), fields: new Map() } + : undefined; for (const [key, value] of Object.entries(live)) { const child = `${pointer}/${escapePointer(key)}`; const other = envKey(key); @@ -399,6 +438,11 @@ class Differ { else { matched.add(other); const rule = self ? this.#rules.get(key) : undefined; + if (rule?.rule === "composed" && composed !== undefined) { + composed.fields.set(key, other); + continue; + } + const before = this.differences.length; this.value( value, env[other]!, @@ -417,6 +461,36 @@ class Differ { pointer: `${generation.pointer}/${escapePointer(key)}`, }, ); + if (this.differences.length === before) composed?.agreed.add(key); + } + } + // Compare source fields once, then reuse their clock/identity decisions. + // Validate each response's spelling before normalizing its composition. + if (composed !== undefined) { + for (const rule of this.#composedRules) { + const other = composed.fields.get(rule.field); + if (other === undefined) continue; + const coherent = + composedValue(rule.parts, live) === live[rule.field] && + composedValue(rule.parts, env) === env[other]; + if ( + coherent && + rule.parts.every( + (part) => + !("from" in part) || + composed.agreed.has(part.from) || + composedPartText(part, live) === composedPartText(part, env), + ) + ) { + composed.agreed.add(rule.field); + if (live[rule.field] !== env[other]) this.normalised.generated += 1; + } else + this.add({ + category: "value", + pointer: `${pointer}/${escapePointer(rule.field)}`, + live: shown(live[rule.field]), + env: shown(env[other]), + }); } } for (const [key, value] of Object.entries(env)) diff --git a/src/application/environment/environmentStateDerivedAnswer.test.ts b/src/application/environment/environmentStateDerivedAnswer.test.ts new file mode 100644 index 0000000000..9ec2dca6c8 --- /dev/null +++ b/src/application/environment/environmentStateDerivedAnswer.test.ts @@ -0,0 +1,84 @@ +import { describe, expect, it } from "vitest"; + +import { + BOOKMARK_ROWS, + FOLDER_ROWS, + TAG_ROWS, +} from "../../../tests/fixtures/environmentStateDerivedValues.js"; +import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js"; +import { + bookClock, + numberedUuid, +} from "../../../tests/fixtures/environmentStateWrites.js"; +import type { JsonValue } from "../../domain/jsonValue.js"; + +/** The clock the first served write of a capture of `rows` stamps: one second past the served clock's start. */ +const firstServedClock = (rows: readonly unknown[]): string => + bookClock(rows.length + 1); + +const objectOf = (body: JsonValue | undefined): Record => { + if (typeof body !== "object" || body === null || Array.isArray(body)) + throw new Error("Expected an object body"); + return body; +}; + +describe("served values an application derives from others", () => { + it("stamps a created tag's clocks with trailing fraction zeros dropped, as the capture spelled them", () => { + const tags = servedFrom(TAG_ROWS); + const created = objectOf( + tags.answer("post_tags", { body: { name: "Spare" } }).body, + ); + expect(created).toMatchObject({ + name: "Spare", + createdAt: firstServedClock(TAG_ROWS), + updatedAt: firstServedClock(TAG_ROWS), + }); + }); + + it("composes a created bookmark's snapshot address from the served clock and the sent URL", () => { + const bookmarks = servedFrom(BOOKMARK_ROWS); + const url = "https://pier.example.test/inspection"; + const created = objectOf( + bookmarks.answer("post_bookmarks", { body: { url, title: "Pier" } }).body, + ); + const digits = firstServedClock(BOOKMARK_ROWS) + .replace(/\D/gu, "") + .slice(0, 14); + expect(created).toMatchObject({ + url, + web_archive_snapshot_url: `https://web.archive.org/web/${digits}/${url}`, + }); + }); + + it("gives a created folder a fresh numeric id and an address that follows its uid and title, and renames it", () => { + const folders = servedFrom(FOLDER_ROWS); + const uid = numberedUuid(200); + const created = objectOf( + folders.answer("post_folders", { + body: { uid, title: "New Folder Name" }, + }).body, + ); + expect(created).toMatchObject({ + uid, + url: `/dashboards/f/${uid}/new-folder-name`, + }); + expect(typeof created["id"]).toBe("number"); + expect(String(created["id"])).toHaveLength(16); + expect([ + 1655476044521472, 1656166527623168, 1656166745726976, + ]).not.toContain(created["id"]); + const renamed = objectOf( + folders.answer("put_folders_by_uuid", { + uuid: uid, + body: { title: "Moved Folder" }, + }).body, + ); + expect(renamed).toMatchObject({ + id: created["id"], + url: `/dashboards/f/${uid}/moved-folder`, + }); + expect( + folders.answer("get_folders_by_uuid", { uuid: uid }).body, + ).toMatchObject({ title: "Moved Folder", url: renamed["url"] }); + }); +}); diff --git a/src/application/environment/environmentStateWriteAnswer.ts b/src/application/environment/environmentStateWriteAnswer.ts index eebf4b35c0..7d803995c2 100644 --- a/src/application/environment/environmentStateWriteAnswer.ts +++ b/src/application/environment/environmentStateWriteAnswer.ts @@ -30,6 +30,8 @@ import { } from "../../domain/analysisView/environmentStateWriteRules.js"; import { formatClock, + generatedIntegerDigits, + seededInteger, seededLike, spelledLike, } from "../../domain/analysisView/environmentStateWriteValues.js"; @@ -212,14 +214,14 @@ export const generator = ( ); const unused = ( field: string, - make: (attempt: number) => string, - ): Result => { + make: (attempt: number) => JsonValue, + ): Result => { const held = new Set( ofType.map((entity) => scalarText(entity.fields[field])), ); for (let attempt = 0; attempt < GENERATION_ATTEMPTS; attempt += 1) { const value = make(attempt); - if (!held.has(value)) return ok(value); + if (!held.has(scalarText(value))) return ok(value); } return err( `Could not generate a unique value for ${field} after ${String(GENERATION_ATTEMPTS)} attempts without colliding with the held state.`, @@ -267,7 +269,9 @@ export const generator = ( ); case "unique": return unused(rule.field, (attempt) => - seededLike(seed(attempt), exemplar), + generatedIntegerDigits(exemplar) === undefined + ? seededLike(seed(attempt), exemplar) + : seededInteger(seed(attempt), exemplar), ); case "renewed": return ok( diff --git a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts index c24249fb96..2a9f21b129 100644 --- a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts +++ b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts @@ -219,6 +219,7 @@ export class EnvironmentStateWriteSimulation { sequences: new Map(), steady: new Set(), unstamped: new Set(), + fixedWidth: new Map(), linked: link.collection, }); } diff --git a/src/domain/analysisView/environmentStateDerivedWrites.test.ts b/src/domain/analysisView/environmentStateDerivedWrites.test.ts new file mode 100644 index 0000000000..8fec6c4f91 --- /dev/null +++ b/src/domain/analysisView/environmentStateDerivedWrites.test.ts @@ -0,0 +1,517 @@ +import { describe, expect, it } from "vitest"; + +import { + timedCaptureEvidence, + type TimedCaptureRow, +} from "../../../tests/fixtures/analysisView.js"; +import { + BOOKMARK_ROWS, + FIXED_HELD_TAG_ROWS, + FIXED_SIBLING_TAG_ROWS, + FOLDER_ROWS, + goClock, + LATER_FIXED_SIBLING_TAG_ROWS, + MIDDLE_FIXED_TAG_UPDATE_ROWS, + MIXED_WIDTH_TAG_UPDATE_ROWS, + restoredAddressRows, + TAG_ROWS, + TAG_UPDATE_ROWS, + TWO_CLOCK_TAG_UPDATE_ROWS, + UNKNOWN_TAG_UPDATE_ROWS, +} from "../../../tests/fixtures/environmentStateDerivedValues.js"; +import { + environmentStateViewOf, + numberedUuid, +} from "../../../tests/fixtures/environmentStateWrites.js"; +import type { JsonValue } from "../jsonValue.js"; +import { isObject, type JsonObject } from "./environmentStateRules.js"; + +const planOf = (rows: readonly TimedCaptureRow[], tool: string) => { + const plan = environmentStateViewOf(timedCaptureEvidence(rows)).writes.find( + (candidate) => candidate.tool === tool, + ); + if (plan === undefined) throw new Error(`No plan for ${tool}`); + return plan; +}; + +const TRIMMED = { + kind: "iso", + separator: "T", + fraction: 9, + zone: "Z", + trimmed: true, +} as const; + +describe("later creates must retain the learned value rules", () => { + it("withholds a later create whose captured composed address is null", () => { + const snapshot = null; + const rows = BOOKMARK_ROWS.slice(0, 6).map( + (row, index): TimedCaptureRow => { + if (index !== 5) return row; + const [method, path, body, ...rest] = row; + if (!isObject(body)) throw new Error("Expected a bookmark response"); + return [ + method, + path, + { ...body, web_archive_snapshot_url: snapshot }, + ...rest, + ]; + }, + ); + expect(planOf(rows, "post_bookmarks")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); + + it.each([ + { + name: "a string integer becomes a UUID", + exemplar: String, + later: numberedUuid(888), + }, + { + name: "a string integer becomes an equal-length token", + exemplar: String, + later: "a656166745726976", + }, + { + name: "a UUID becomes an equal-length opaque token", + exemplar: (id: number) => numberedUuid(400 + (id % 1000)), + later: "a".repeat(35) + "1", + }, + ])("withholds a create when $name", ({ exemplar, later }) => { + const changed = (value: JsonValue): JsonValue => + Array.isArray(value) + ? value.map(changed) + : isObject(value) + ? Object.fromEntries( + Object.entries(value).map(([field, child]) => [ + field, + field === "id" && typeof child === "number" + ? child === 1656166745726976 + ? later + : exemplar(child) + : changed(child), + ]), + ) + : value; + const rows = FOLDER_ROWS.map((row): TimedCaptureRow => { + const [method, path, body, ...rest] = row; + return body === undefined ? row : [method, path, changed(body), ...rest]; + }); + expect(planOf(rows, "post_folders")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); +}); + +describe("clock spellings an application derives", () => { + it("serves creates whose clocks drop trailing fraction zeros, as Go spells nanoseconds", () => { + expect(planOf(TAG_ROWS, "post_tags")).toMatchObject({ + served: true, + rules: expect.arrayContaining([ + { field: "updatedAt", rule: "clock", format: TRIMMED }, + ]), + }); + }); + + it.each([ + ["2026-10-01T09:00:00.12345678Z", "2026-10-01T10:00:00.120000000Z"], + ["2026-10-01T09:00:00.120000000Z", "2026-10-01T10:00:00.12345678Z"], + ["2026-10-01T09:00:00.12345678Z", "2026-10-01T10:00:00.123456780Z"], + ["2026-10-01T09:00:00.123456780Z", "2026-10-01T10:00:00.12345678Z"], + ])( + "keeps a create's clock at a fixed width when a held value ends its fraction in a zero (%s, %s)", + (first, second) => { + const held = [first, second].map((updatedAt, index) => ({ + id: numberedUuid(index + 1), + name: `Held ${index + 1}`, + updatedAt, + })); + const created = { + id: numberedUuid(3), + name: "Heavy", + updatedAt: goClock(3, "374763655"), + }; + const rows: TimedCaptureRow[] = [ + ["GET", "/api/tags", held], + ...held.map((tag): TimedCaptureRow => [ + "GET", + `/api/tags/${tag.id}`, + tag, + ]), + ["POST", "/api/tags", created, 201, { name: "Heavy" }], + ["GET", `/api/tags/${created.id}`, created], + ]; + expect(planOf(rows, "post_tags").rules).toContainEqual({ + field: "updatedAt", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 9, zone: "Z" }, + }); + }, + ); + + it("keeps a create's sibling clock at a fixed width when a later call trims one clock and shows the sibling's trailing zero", () => { + const rules = planOf(FIXED_SIBLING_TAG_ROWS, "post_tags").rules; + expect(rules).toContainEqual({ + field: "updatedAt", + rule: "clock", + format: TRIMMED, + }); + expect(rules).toContainEqual({ + field: "createdAt", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 9, zone: "Z" }, + }); + }); + + it("serves updates whose clock a later call shows trimmed", () => { + expect(planOf(TAG_UPDATE_ROWS, "patch_tags_by_uuid")).toMatchObject({ + served: true, + rules: expect.arrayContaining([ + { field: "updatedAt", rule: "clock", format: TRIMMED }, + ]), + }); + }); + + it("learns no clock spelling from an update to an entity an unmodelled write left unknown", () => { + expect(planOf(UNKNOWN_TAG_UPDATE_ROWS, "patch_tags_by_uuid")).toMatchObject( + { + served: true, + rules: expect.arrayContaining([ + { + field: "updatedAt", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 9, zone: "Z" }, + }, + ]), + }, + ); + }); + + it("serves updates whose two clocks a later call shows trimmed for one only", () => { + expect( + planOf(TWO_CLOCK_TAG_UPDATE_ROWS, "patch_tags_by_uuid"), + ).toMatchObject({ + served: true, + rules: expect.arrayContaining([ + { field: "updatedAt", rule: "clock", format: TRIMMED }, + { field: "syncedAt", rule: "clock", format: TRIMMED }, + ]), + }); + }); +}); + +describe("fixed-width clock evidence", () => { + const FIXED = { + kind: "iso", + separator: "T", + fraction: 9, + zone: "Z", + } as const; + + it.each([1, 2])( + "withholds a shorter update clock when held entity %s proves a fixed width", + (heldId) => { + const fixedHeld = (value: JsonValue): JsonValue => + Array.isArray(value) + ? value.map(fixedHeld) + : isObject(value) && value.id === numberedUuid(heldId) + ? { ...value, updatedAt: "2026-10-01T09:00:00.120000000Z" } + : value; + const rows = TAG_UPDATE_ROWS.map((row, index): TimedCaptureRow => { + const [method, path, body, ...rest] = row; + return index < 3 && body !== undefined + ? [method, path, fixedHeld(body), ...rest] + : row; + }); + expect(planOf(rows, "patch_tags_by_uuid")).toMatchObject({ + served: false, + reason: "unexplained-change", + rules: expect.arrayContaining([ + { field: "updatedAt", rule: "clock", format: FIXED }, + ]), + }); + }, + ); + + it("serves updates whose later call trims one clock and shows the other's trailing zero", () => { + expect( + planOf(MIXED_WIDTH_TAG_UPDATE_ROWS, "patch_tags_by_uuid"), + ).toMatchObject({ + served: true, + rules: expect.arrayContaining([ + { field: "updatedAt", rule: "clock", format: TRIMMED }, + { field: "syncedAt", rule: "clock", format: FIXED }, + ]), + }); + }); + + it("keeps a create's clock at a fixed width an earlier later call showed, whatever a third shows", () => { + expect( + planOf(LATER_FIXED_SIBLING_TAG_ROWS, "post_tags").rules, + ).toContainEqual({ field: "createdAt", rule: "clock", format: FIXED }); + }); + + it("keeps a create's clock at a fixed width a held entity showed while its sibling is learned trimmed", () => { + const rules = planOf(FIXED_HELD_TAG_ROWS, "post_tags").rules; + expect(rules).toContainEqual({ + field: "updatedAt", + rule: "clock", + format: TRIMMED, + }); + expect(rules).toContainEqual({ + field: "createdAt", + rule: "clock", + format: FIXED, + }); + }); + + it("withholds updates whose clock a later call trims after an earlier one showed a trailing zero", () => { + expect( + planOf(MIDDLE_FIXED_TAG_UPDATE_ROWS, "patch_tags_by_uuid"), + ).toMatchObject({ + served: false, + reason: "unexplained-change", + rules: expect.arrayContaining([ + { field: "updatedAt", rule: "clock", format: FIXED }, + ]), + }); + }); +}); + +describe("values an application composes or generates", () => { + it("withholds bookmark creates a later call answers with a snapshot address the composition does not spell", () => { + const special = "https://web.archive.org/web/special"; + const rows = BOOKMARK_ROWS.flatMap((row): TimedCaptureRow[] => { + const [method, path, body, ...rest] = row; + if (method === "GET" && path === "/api/bookmarks/4") return []; + return method === "POST" && isObject(body) && body.id === 4 + ? [ + [ + method, + path, + { ...body, web_archive_snapshot_url: special }, + ...rest, + ], + ] + : [row]; + }).slice(0, -1); + expect(planOf(rows, "post_bookmarks")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); + + it("serves creates whose snapshot address joins the creation time and the sent URL, accepting any URL", () => { + expect(planOf(BOOKMARK_ROWS, "post_bookmarks")).toMatchObject({ + served: true, + body: { accepted: ["url", "title"], fixed: [] }, + rules: expect.arrayContaining([ + { + field: "web_archive_snapshot_url", + rule: "composed", + parts: [ + { text: "https://web.archive.org/web/" }, + { from: "date_added", as: "clock", pattern: "YYYYMMDDhhmmss" }, + { text: "/" }, + { from: "url", as: "value" }, + ], + }, + ]), + }); + }); + + it("serves folder creates and renames whose address follows the uid and the title's slug, and whose numeric id is generated", () => { + const url = { + field: "url", + rule: "composed", + parts: [ + { text: "/dashboards/f/" }, + { from: "uid", as: "value" }, + { text: "/" }, + { from: "title", as: "slug" }, + ], + }; + expect(planOf(FOLDER_ROWS, "post_folders")).toMatchObject({ + served: true, + body: { accepted: ["uid", "title"], fixed: [] }, + rules: expect.arrayContaining([{ field: "id", rule: "unique" }, url]), + }); + expect(planOf(FOLDER_ROWS, "put_folders_by_uuid")).toMatchObject({ + served: true, + rules: expect.arrayContaining([url]), + }); + }); + + it("withholds renames whose address a later call composes when an earlier call left it otherwise", () => { + expect( + planOf(restoredAddressRows(true), "put_folders_by_uuid"), + ).toMatchObject({ + served: true, + rules: expect.arrayContaining([ + expect.objectContaining({ field: "url", rule: "composed" }), + ]), + }); + // The first rename left its address as it was, which the composition would rebuild. + expect( + planOf(restoredAddressRows(false), "put_folders_by_uuid"), + ).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); + + it.each([null, { path: "/legacy" }])( + "withholds a later composition whose parts an earlier rename could not spell (%j)", + (base) => { + const first = { id: 1, title: "First Folder", base, url: "/legacy" }; + const renamedFirst = { ...first, title: "Renamed First Folder" }; + const second = { + id: 2, + title: "Second Folder", + base: "https://example.test/folders", + url: "https://example.test/folders/second-folder", + }; + const renamedSecond = { + ...second, + title: "Renamed Second Folder", + url: "https://example.test/folders/renamed-second-folder", + }; + const rows: TimedCaptureRow[] = [ + ["GET", "/api/folders", [first, second]], + ["GET", "/api/folders/1", first], + ["GET", "/api/folders/2", second], + [ + "PUT", + "/api/folders/1", + renamedFirst, + 200, + { title: renamedFirst.title }, + ], + ["GET", "/api/folders/1", renamedFirst], + [ + "PUT", + "/api/folders/2", + renamedSecond, + 200, + { title: renamedSecond.title }, + ], + ["GET", "/api/folders/2", renamedSecond], + ]; + expect(planOf(rows, "put_folders_by_id")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }, + ); +}); + +/** The folder rows with each folder's numeric id repeated where `mirror` puts it, and in each create's body when `sent`. */ +const mirrored = ( + mirror: (id: number) => JsonObject, + sent = false, +): TimedCaptureRow[] => { + const repeated = (value: JsonValue): JsonValue => + Array.isArray(value) + ? value.map(repeated) + : isObject(value) && typeof value.id === "number" + ? { ...value, ...mirror(value.id) } + : value; + return FOLDER_ROWS.map((row): TimedCaptureRow => { + const [method, path, body, ...rest] = row; + if (body === undefined) return row; + const answered = repeated(body); + const [status, request] = rest; + return sent && + method === "POST" && + isObject(answered) && + status !== undefined && + isObject(request) + ? [ + method, + path, + answered, + status, + { ...request, ...mirror(Number(answered.id)) }, + ] + : [method, path, answered, ...rest]; + }); +}; + +describe("generated integers another field repeats", () => { + it("withholds folder creates a later call answers with a UUID where the exemplar generated an integer", () => { + const later = 1656166745726976; + const answered = numberedUuid(203); + const swap = (value: JsonValue): JsonValue => + Array.isArray(value) + ? value.map(swap) + : isObject(value) && value.id === later + ? { ...value, id: answered } + : value; + const rows = FOLDER_ROWS.map((row): TimedCaptureRow => { + const [method, path, body, ...rest] = row; + return body === undefined ? row : [method, path, swap(body), ...rest]; + }); + expect(planOf(rows, "post_folders")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); + + it("withholds folder creates whose generated numeric id another field repeats, which seeding cannot keep", () => { + expect( + planOf( + mirrored((id) => ({ folderId: id })), + "post_folders", + ), + ).toMatchObject({ served: false, reason: "unexplained-change" }); + }); + + it("withholds folder creates whose generated numeric id a number nested in another field repeats", () => { + expect( + planOf( + mirrored((id) => ({ metadata: { serial: id } })), + "post_folders", + ), + ).toMatchObject({ served: false, reason: "unexplained-change" }); + }); + + it.each([ + [ + "an echoed string of its digits", + (id: number) => ({ serialText: String(id) }), + true, + ], + [ + "a string another rule generates around its digits", + (id: number) => ({ ref: `folder-${id}` }), + false, + ], + ])( + "withholds folder creates whose generated numeric id %s repeats", + (_, mirror, sent) => { + expect(planOf(mirrored(mirror, sent), "post_folders")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }, + ); + + it("serves folder creates whose string of the id's digits a template rebuilds from it", () => { + expect( + planOf( + mirrored((id) => ({ idText: String(id) })), + "post_folders", + ), + ).toMatchObject({ + served: true, + rules: expect.arrayContaining([ + { field: "id", rule: "unique" }, + { field: "idText", rule: "template", from: "id" }, + ]), + }); + }); +}); diff --git a/src/domain/analysisView/environmentStateView.ts b/src/domain/analysisView/environmentStateView.ts index 87e92de8a7..c01c556b06 100644 --- a/src/domain/analysisView/environmentStateView.ts +++ b/src/domain/analysisView/environmentStateView.ts @@ -9,6 +9,7 @@ import type { } from "./analysisView.js"; import { EnvironmentStateAccumulator } from "./EnvironmentStateAccumulator.js"; import { isClockZone } from "./environmentStateWriteValues.js"; +import { CLOCK_PATTERNS } from "./environmentStateWriteClockFormats.js"; import { groupCaptureRecords, templateRulesSchema, @@ -97,6 +98,7 @@ const clockFormatSchema = z.discriminatedUnion("kind", [ separator: z.enum(["T", " "]), fraction: z.number().int().nonnegative().max(9), zone: z.string().refine(isClockZone), + trimmed: z.literal(true).exactOptional(), }), z.strictObject({ kind: z.literal("epoch"), unit: z.enum(["s", "ms"]) }), ]); @@ -256,6 +258,26 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [ z.strictObject({ ...ruled, rule: z.literal("path"), from: z.string() }), z.strictObject({ ...ruled, rule: z.literal("slug"), from: z.string() }), z.strictObject({ ...ruled, rule: z.literal("template"), from: z.string() }), + z.strictObject({ + ...ruled, + rule: z.literal("composed"), + parts: z + .array( + z.union([ + z.strictObject({ text: z.string().min(1) }), + z.strictObject({ + from: z.string(), + as: z.enum(["value", "slug"]), + }), + z.strictObject({ + from: z.string(), + as: z.literal("clock"), + pattern: z.enum(CLOCK_PATTERNS), + }), + ]), + ) + .min(1), + }), z.strictObject({ ...ruled, rule: z.literal("increment") }), z.strictObject({ ...ruled, diff --git a/src/domain/analysisView/environmentStateWriteApply.test.ts b/src/domain/analysisView/environmentStateWriteApply.test.ts index 46ed37f429..bce091a9c2 100644 --- a/src/domain/analysisView/environmentStateWriteApply.test.ts +++ b/src/domain/analysisView/environmentStateWriteApply.test.ts @@ -676,3 +676,81 @@ it("checks a member stored only under another name against the type of the field expect(created({ not: "a string" }).ok).toBe(false); expect(created(null).ok).toBe(false); }); + +const ADDRESS: FieldRule = { + field: "url", + rule: "composed", + parts: [ + { text: "/dashboards/f/" }, + { from: "uid", as: "value" }, + { text: "/" }, + { from: "title", as: "slug" }, + ], +}; +const UNSPELLED = + 'The write composes "url" from "uid", "title", and the entity holds no value one of them can spell.'; + +it("rebuilds an updated composed address, and refuses one whose entity holds no field a part reads", () => { + const plan: Plan = { + rules: [{ field: "title", rule: "echo" }, ADDRESS], + body: { + accepted: ["title"], + fixed: [], + required: [], + identity: "uid", + kinds: [], + }, + }; + const rename = (before: JsonObject) => + updatedFields( + plan, + { before, observed: before }, + { ...callWith({ title: "Moved Folder" }, { item: "7" }), id: "7" }, + generator().generate, + ); + expect( + rename({ id: 7, uid: "f-7", title: "Old", url: "/dashboards/f/f-7/old" }), + ).toEqual( + ok({ + id: 7, + uid: "f-7", + title: "Moved Folder", + url: "/dashboards/f/f-7/moved-folder", + }), + ); + expect(rename({ id: 7, title: "Old", url: "/dashboards/f/f-7/old" })).toEqual( + err(UNSPELLED), + ); +}); + +it("refuses a create whose composed address reads a member the call does not send", () => { + const plan: Plan = { + rules: [ + { field: "uid", rule: "echo" }, + { field: "title", rule: "echo" }, + ADDRESS, + ], + body: { + accepted: ["uid", "title"], + fixed: [], + required: ["title"], + identity: null, + kinds: [], + }, + }; + const observed: JsonObject = { + uid: "f-1", + title: "Recorded", + url: "/dashboards/f/f-1/recorded", + }; + const create = (body: JsonObject) => + createdFields(plan, observed, callWith(body), generator().generate); + expect(create({ uid: "f-2", title: "New Folder" })).toEqual( + ok({ + uid: "f-2", + title: "New Folder", + url: "/dashboards/f/f-2/new-folder", + }), + ); + expect(create({ title: "New Folder" })).toEqual(err(UNSPELLED)); +}); diff --git a/src/domain/analysisView/environmentStateWriteApply.ts b/src/domain/analysisView/environmentStateWriteApply.ts index 7a57b41d30..f980e1138b 100644 --- a/src/domain/analysisView/environmentStateWriteApply.ts +++ b/src/domain/analysisView/environmentStateWriteApply.ts @@ -24,6 +24,7 @@ import { stampProblem, nestedClockValue, } from "./environmentStateWriteStamps.js"; +import { composedValue } from "./environmentStateWriteComposed.js"; import { integerOf, readClock, @@ -260,13 +261,13 @@ const templated = ( : undefined; }; -/** Apply the rules that derive from fields already written or the call: slugs, templates and stamps. */ +/** Apply the rules that derive from fields already written or the call: slugs, templates, stamps and composed strings; why a composed string cannot be rebuilt, if one cannot. */ const derive = ( rules: readonly FieldRule[], observed: JsonObject, written: JsonObject, sent: JsonObject, -): void => { +): string | undefined => { for (const rule of rules) { if (rule.rule === "slug") { const from = sent[rule.from]; @@ -278,6 +279,19 @@ const derive = ( else if (rule.rule === "derived" && Object.hasOwn(sent, rule.member)) written[rule.field] = rule.value; } + // Composed strings read the fields the rules above wrote. + for (const rule of rules) + if (rule.rule === "composed") { + const value = composedValue(rule.parts, written); + if (value === undefined) { + const sources = new Set( + rule.parts.flatMap((part) => ("from" in part ? [part.from] : [])), + ); + return `The write composes ${JSON.stringify(rule.field)} from ${listed([...sources])}, and the entity holds no value one of them can spell.`; + } + written[rule.field] = value; + } + return undefined; }; /** Members stored only under another name, with the value of the field each is copied whole into, for type checks. */ @@ -313,8 +327,8 @@ export const createdFields = ( if (value.value !== undefined) written[rule.field] = value.value; else if (rule.rule === "echo") delete written[rule.field]; } - derive(plan.rules, observed, written, call.body ?? {}); - return ok(written); + const unspelled = derive(plan.rules, observed, written, call.body ?? {}); + return unspelled === undefined ? ok(written) : err(unspelled); }; /** Rules an update gives a new value on every call. */ @@ -384,7 +398,7 @@ export const updatedFields = ( if (value.value !== undefined) written[rule.field] = value.value; } } - derive( + const unspelled = derive( plan.rules.filter( (rule) => rule.rule !== "slug" || Object.hasOwn(sent, rule.from), ), @@ -392,7 +406,7 @@ export const updatedFields = ( written, sent, ); - return ok(written); + return unspelled === undefined ? ok(written) : err(unspelled); }; /** Why a document write's body does not fit what the capture showed: members its plan accepts, as the JSON types the document holds, a member a replacing write dropped keeping the type the initial document showed. */ diff --git a/src/domain/analysisView/environmentStateWriteCalls.ts b/src/domain/analysisView/environmentStateWriteCalls.ts index f8ad8ad29a..6b0402f909 100644 --- a/src/domain/analysisView/environmentStateWriteCalls.ts +++ b/src/domain/analysisView/environmentStateWriteCalls.ts @@ -3,6 +3,7 @@ import { digestCanonicalValue } from "../canonicalDigest.js"; import type { JsonValue } from "../jsonValue.js"; import { same, type JsonObject } from "./environmentStateRules.js"; import { storedKinds } from "./environmentStateWriteApply.js"; +import { composedValue } from "./environmentStateWriteComposed.js"; import type { explainUpdate } from "./environmentStateWriteExplain.js"; import type { FieldRule, @@ -15,6 +16,7 @@ import { integerOf, slugOf } from "./environmentStateWriteValues.js"; export const unsentChange = (rule: FieldRule): boolean => rule.rule === "increment" || rule.rule === "slug" || + rule.rule === "composed" || rule.rule === "stamp" || rule.rule === "derived" || rule.rule === "set"; @@ -81,6 +83,14 @@ const observedValue = (value: JsonValue): ObservedValue => ? { kind: "structured", digest: digestCanonicalValue(value) } : { kind: "scalar", value }; +/** The scalar fields a call showed, as a composed rule reads them. */ +const shownScalars = (shown: ReadonlyMap): JsonObject => + Object.fromEntries( + [...shown].flatMap(([field, observed]) => + observed.kind === "scalar" ? [[field, observed.value]] : [], + ), + ); + const sameObserved = (left: ObservedValue, right: ObservedValue): boolean => left.kind === "structured" ? right.kind === "structured" && left.digest === right.digest @@ -129,6 +139,13 @@ export const contradictsEarlierCalls = ( if (rule.rule === "stamp") return !sameObserved(observation, expected!); if (rule.rule === "derived") return sent.has(rule.member) && !sameObserved(observation, expected!); + if (rule.rule === "composed") { + const value = composedValue(rule.parts, shownScalars(shown)); + return ( + value === undefined || + !sameObserved(observation, observedValue(value)) + ); + } const from = rule.rule === "slug" ? sent.get(rule.from) : undefined; return ( typeof from === "string" && diff --git a/src/domain/analysisView/environmentStateWriteChanges.ts b/src/domain/analysisView/environmentStateWriteChanges.ts index a13b2f9f52..1786655e6f 100644 --- a/src/domain/analysisView/environmentStateWriteChanges.ts +++ b/src/domain/analysisView/environmentStateWriteChanges.ts @@ -3,6 +3,7 @@ import { isObject, same, type JsonObject } from "./environmentStateRules.js"; import type { FieldRule, WriteCall } from "./environmentStateWriteRules.js"; import { integerOf, readClock } from "./environmentStateWriteValues.js"; import { freshUuid, renewedRule } from "./environmentStateWriteGenerated.js"; +import { composedChange } from "./environmentStateWriteComposed.js"; import { derivedRule, freshToken, @@ -59,6 +60,10 @@ export const changeRule = ( return ( stampRule(evidence, field) ?? derivedRule(evidence, field) ?? + composedChange(field, { + ...evidence, + sent: evidence.call.body, + }) ?? actionRule(evidence, field) ); }; diff --git a/src/domain/analysisView/environmentStateWriteClockFormats.test.ts b/src/domain/analysisView/environmentStateWriteClockFormats.test.ts new file mode 100644 index 0000000000..deff640d20 --- /dev/null +++ b/src/domain/analysisView/environmentStateWriteClockFormats.test.ts @@ -0,0 +1,111 @@ +import { expect, it } from "vitest"; +import type { JsonValue } from "../jsonValue.js"; +import { + clockDigits, + fittedClockFormat, + trimmedLike, +} from "./environmentStateWriteClockFormats.js"; +import type { ClockFormat } from "./environmentStateWriteValues.js"; + +const iso = ( + separator: "T" | " ", + fraction: number, + zone: string, +): ClockFormat => ({ kind: "iso", separator, fraction, zone }); + +const WITH_MS = Date.UTC(2026, 2, 4, 5, 6, 7) + 123; + +const TRIMMED_9: ClockFormat = { + kind: "iso", + separator: "T", + fraction: 9, + zone: "Z", + trimmed: true, +}; + +it.each< + [string, ClockFormat, string, string | undefined, ClockFormat | undefined] +>([ + [ + "the format's own spelling", + iso("T", 9, "Z"), + "2026-03-04T05:06:07.123456789Z", + "2026-03-04T05:06:07.123456781Z", + iso("T", 9, "Z"), + ], + [ + "a shorter trimmed fraction", + iso("T", 9, "Z"), + "2026-03-04T05:06:07.12345678Z", + "2026-03-04T05:06:07.123456781Z", + TRIMMED_9, + ], + [ + "a longer one than a trimmed exemplar", + iso("T", 8, "Z"), + "2026-03-04T05:06:07.123456781Z", + "2026-03-04T05:06:07.12345678Z", + TRIMMED_9, + ], + [ + "no fraction beside a trimmed one", + iso("T", 9, "Z"), + "2026-03-04T05:06:07Z", + "2026-03-04T05:06:07.123456781Z", + TRIMMED_9, + ], + [ + "a fraction ending in a zero", + iso("T", 8, "Z"), + "2026-03-04T05:06:07.123456780Z", + "2026-03-04T05:06:07.12345678Z", + undefined, + ], + [ + "an exemplar ending in a zero", + iso("T", 3, "Z"), + "2026-03-04T05:06:07.123456Z", + "2026-03-04T05:06:07.120Z", + undefined, + ], + [ + "another zone", + iso("T", 9, "Z"), + "2026-03-04T05:06:07.1234567+00:00", + "2026-03-04T05:06:07.123456781Z", + undefined, + ], + [ + "a shorter value under a trimmed format", + TRIMMED_9, + "2026-03-04T05:06:07.1Z", + undefined, + TRIMMED_9, + ], +])("fits %s to a clock format", (_name, format, value, exemplar, expected) => { + expect(fittedClockFormat(format, value, exemplar)).toEqual(expected); +}); + +it("trims a sibling clock spelled alike unless its own value shows a fixed width", () => { + expect( + trimmedLike(iso("T", 9, "Z"), TRIMMED_9, "2026-03-04T05:06:07.123456789Z"), + ).toEqual(TRIMMED_9); + expect( + trimmedLike(iso("T", 9, "Z"), TRIMMED_9, "2026-03-04T05:06:07.123456780Z"), + ).toEqual(iso("T", 9, "Z")); + expect( + trimmedLike(iso(" ", 9, "Z"), TRIMMED_9, "2026-03-04 05:06:07.123456789Z"), + ).toEqual(iso(" ", 9, "Z")); +}); + +it.each<[JsonValue, string, string | undefined]>([ + ["2026-03-04T05:06:07.123Z", "YYYYMMDDhhmmss", "20260304050607"], + ["2026-03-04T05:06:07Z", "YYYYMMDD", "20260304"], + ["2026-03-04T23:30:00-05:00", "YYYY-MM-DD", "2026-03-04"], + [WITH_MS, "YYYY/MM/DD", "2026/03/04"], + ["not a clock", "YYYYMMDD", undefined], +])("spells the clock %j's wall clock in %s", (value, pattern, expected) => { + expect(clockDigits(value, pattern as Parameters[1])).toBe( + expected, + ); +}); diff --git a/src/domain/analysisView/environmentStateWriteClockFormats.ts b/src/domain/analysisView/environmentStateWriteClockFormats.ts new file mode 100644 index 0000000000..883b752df5 --- /dev/null +++ b/src/domain/analysisView/environmentStateWriteClockFormats.ts @@ -0,0 +1,164 @@ +import type { JsonValue } from "../jsonValue.js"; +import { + readClock, + zoneMinutesForFormat, + type ClockFormat, + type ClockReading, +} from "./environmentStateWriteValues.js"; + +const MINUTE_MS = 60_000; + +const pad = (value: number, width: number): string => + String(value).padStart(width, "0"); + +/** Whether a parsed clock fits a learned format without widening it or learning a new spelling. */ +export const matchesClockFormat = ( + format: ClockFormat, + reading: ClockReading | undefined, +): boolean => { + if (reading === undefined) return false; + const spelled = reading.format; + if (format.kind === "epoch") + return spelled.kind === "epoch" && spelled.unit === format.unit; + return ( + spelled.kind === "iso" && + spelled.separator === format.separator && + spelled.zone === format.zone && + (format.trimmed === true + ? !reading.fixedWidth && spelled.fraction <= format.fraction + : spelled.fraction === format.fraction) + ); +}; + +/** + * The format that spells both a format's values and another clock value: the format itself, or a trimmed one when the + * two differ only in how many fraction digits they show and neither ends a fraction in a zero. `exemplar` is the value + * the format was learned from, which shows whether its own width is fixed. + */ +export const fittedClockFormat = ( + format: ClockFormat, + value: JsonValue | undefined, + exemplar: JsonValue | undefined, +): ClockFormat | undefined => { + const reading = readClock(value); + if (reading === undefined) return undefined; + if (matchesClockFormat(format, reading)) return format; + const spelled = reading.format; + if ( + format.kind !== "iso" || + spelled.kind !== "iso" || + spelled.separator !== format.separator || + spelled.zone !== format.zone || + reading.fixedWidth + ) + return undefined; + if (format.trimmed !== true) { + const learned = readClock(exemplar); + return learned === undefined || learned.fixedWidth + ? undefined + : { + ...format, + fraction: Math.max(format.fraction, spelled.fraction), + trimmed: true, + }; + } + return { ...format, fraction: spelled.fraction }; +}; + +/** Whether a value spells a clock as an ISO format does with a fraction ending in a zero, which no trimmed spelling shows. */ +export const showsFixedWidth = ( + format: ClockFormat, + value: JsonValue | undefined, +): boolean => { + const reading = readClock(value); + return ( + reading !== undefined && + reading.fixedWidth && + format.kind === "iso" && + reading.format.kind === "iso" && + reading.format.separator === format.separator && + reading.format.zone === format.zone + ); +}; + +/** Keep the first value showing each field as an ISO clock at a fixed width, which no trimmed spelling of the field may contradict. */ +export const noteFixedWidths = ( + seen: Map, + fields: Readonly>, +): void => { + for (const [field, value] of Object.entries(fields)) { + const reading = readClock(value); + if ( + reading?.fixedWidth === true && + reading.format.kind === "iso" && + !seen.has(field) + ) + seen.set(field, value); + } +}; + +/** A clock format trimmed as a sibling's is, when it is spelled alike and its own value does not show a fixed width. */ +export const trimmedLike = ( + format: ClockFormat, + trimmed: ClockFormat, + value: JsonValue | undefined, +): ClockFormat => { + const reading = readClock(value); + return format.kind === "iso" && + trimmed.kind === "iso" && + trimmed.trimmed === true && + format.trimmed !== true && + format.separator === trimmed.separator && + format.zone === trimmed.zone && + reading !== undefined && + !reading.fixedWidth + ? { + ...format, + fraction: Math.max(format.fraction, trimmed.fraction), + trimmed: true, + } + : format; +}; + +/** Calendar digits a composed string spells a clock in: date and time run together, or a date. */ +export type ClockPattern = + | "YYYYMMDDhhmmss" + | "YYYYMMDD" + | "YYYY-MM-DD" + | "YYYY/MM/DD"; + +export const CLOCK_PATTERNS: readonly ClockPattern[] = [ + "YYYYMMDDhhmmss", + "YYYYMMDD", + "YYYY-MM-DD", + "YYYY/MM/DD", +]; + +/** A clock value's wall clock in its own zone, in a calendar pattern; undefined when it is not a clock. */ +export const clockDigits = ( + value: JsonValue | undefined, + pattern: ClockPattern, +): string | undefined => { + const clock = readClock(value); + if (clock === undefined) return undefined; + const zone = + clock.format.kind === "iso" ? zoneMinutesForFormat(clock.format.zone) : 0; + const at = new Date(clock.ms + zone * MINUTE_MS); + const year = at.getUTCFullYear(); + if (!Number.isInteger(year) || year < 0 || year > 9999) return undefined; + const [y, m, d] = [ + pad(year, 4), + pad(at.getUTCMonth() + 1, 2), + pad(at.getUTCDate(), 2), + ]; + switch (pattern) { + case "YYYYMMDDhhmmss": + return `${y}${m}${d}${pad(at.getUTCHours(), 2)}${pad(at.getUTCMinutes(), 2)}${pad(at.getUTCSeconds(), 2)}`; + case "YYYYMMDD": + return `${y}${m}${d}`; + case "YYYY-MM-DD": + return `${y}-${m}-${d}`; + case "YYYY/MM/DD": + return `${y}/${m}/${d}`; + } +}; diff --git a/src/domain/analysisView/environmentStateWriteComposed.test.ts b/src/domain/analysisView/environmentStateWriteComposed.test.ts new file mode 100644 index 0000000000..c350b4f62a --- /dev/null +++ b/src/domain/analysisView/environmentStateWriteComposed.test.ts @@ -0,0 +1,302 @@ +import { describe, expect, it } from "vitest"; + +import type { JsonValue } from "../jsonValue.js"; +import type { JsonObject } from "./environmentStateRules.js"; +import { + composedChange, + composedValue, + createdComposer, +} from "./environmentStateWriteComposed.js"; +import type { FieldRule } from "./environmentStateWriteRules.js"; + +const CLOCK: FieldRule = { + field: "added", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 0, zone: "Z" }, +}; + +const bookmark = (url: string, added: string): JsonObject => ({ + url, + added, + modified: added, + snapshot: `https://web.archive.org/web/${added.replace(/\D/gu, "").slice(0, 14)}/${url}`, +}); + +const SEED = bookmark( + "https://ferries.example.test/timetable", + "2026-10-01T09:00:00Z", +); +const CREATED = bookmark( + "https://quarry.example.test/ledger", + "2026-10-17T18:00:03Z", +); +const ECHO_URL: FieldRule = { field: "url", rule: "echo" }; + +describe("strings a create composes from other fields", () => { + it("spells a snapshot address from the write clock's digits and the sent URL, as held entities do", () => { + const rule = createdComposer( + CREATED, + [ECHO_URL, CLOCK], + [SEED], + )("snapshot"); + expect(rule).toEqual({ + field: "snapshot", + rule: "composed", + parts: [ + { text: "https://web.archive.org/web/" }, + { from: "added", as: "clock", pattern: "YYYYMMDDhhmmss" }, + { text: "/" }, + { from: "url", as: "value" }, + ], + }); + expect( + composedValue(rule!.rule === "composed" ? rule!.parts : [], SEED), + ).toBe(SEED["snapshot"]); + }); + + it("composes nothing one call alone shows, nor what a held entity spells otherwise", () => { + expect( + createdComposer(CREATED, [ECHO_URL, CLOCK], [])("snapshot"), + ).toBeUndefined(); + const stale = { + ...SEED, + snapshot: "https://web.archive.org/web/20250101000000/x", + }; + expect( + createdComposer(CREATED, [ECHO_URL, CLOCK], [SEED, stale])("snapshot"), + ).toBeUndefined(); + }); + + it.each([ + ["before", (url: string) => `𐐀${url}`], + ["after", (url: string) => `${url}𐐀`], + ])( + "composes nothing from a value an astral letter joins %s it", + (_, label) => { + const labelled = (entity: JsonObject): JsonObject => ({ + ...entity, + label: label(String(entity["url"])), + }); + expect( + createdComposer( + labelled(CREATED), + [ECHO_URL, CLOCK], + [labelled(SEED)], + )("label"), + ).toBeUndefined(); + }, + ); + + it("takes two clocks a create stamps alike as one source, but not two members that send the same text", () => { + const modified: FieldRule = { ...CLOCK, field: "modified" }; + expect( + createdComposer(CREATED, [ECHO_URL, CLOCK, modified], [SEED])("snapshot"), + ).toMatchObject({ rule: "composed" }); + const folder = { + uid: "scratch-pier", + title: "Scratch Pier", + url: "/dashboards/f/scratch-pier/scratch-pier", + }; + const seed = { + uid: "lighthouse-rota", + title: "Lighthouse Rota", + url: "/dashboards/f/lighthouse-rota/lighthouse-rota", + }; + expect( + createdComposer( + folder, + [ + { field: "uid", rule: "echo" }, + { field: "title", rule: "echo" }, + ], + [seed], + )("url"), + ).toBeUndefined(); + }); +}); + +describe("strings an update composes from other fields", () => { + const before = { + uid: "scratch-alpha", + title: "Scratch Alpha Harbor", + url: "/dashboards/f/scratch-alpha/scratch-alpha-harbor", + }; + const after = { + ...before, + title: "Renamed Alpha Lantern", + url: "/dashboards/f/scratch-alpha/renamed-alpha-lantern", + }; + const change = (overrides: { + readonly after?: JsonObject; + readonly sent?: JsonObject; + readonly others?: readonly JsonObject[]; + }) => + composedChange("url", { + before, + after: overrides.after ?? after, + sent: overrides.sent ?? { title: "Renamed Alpha Lantern" }, + idField: "uid", + others: overrides.others ?? [], + }); + + it("spells an address from an unchanged uid and the sent title's slug, which spell it before the call too", () => { + expect(change({})).toEqual({ + field: "url", + rule: "composed", + parts: [ + { text: "/dashboards/f/" }, + { from: "uid", as: "value" }, + { text: "/" }, + { from: "title", as: "slug" }, + ], + }); + }); + + it("composes nothing from a member the application stored otherwise than sent, nor what another entity spells otherwise", () => { + expect( + change({ sent: { title: "RENAMED ALPHA LANTERN" } }), + ).toBeUndefined(); + expect( + change({ + others: [ + { uid: "other-folder", title: "Other Folder", url: "/f/other" }, + ], + }), + ).toBeUndefined(); + }); + + it("composes nothing the same parts do not spell before the call", () => { + expect( + change({ + after: { + ...after, + url: "/dashboards/f/scratch-alpha/renamed-alpha-lantern-1", + }, + }), + ).toBeUndefined(); + }); +}); + +describe("compositions serving could not keep", () => { + const stamped = (zone: string): FieldRule => ({ + field: "stamped", + rule: "clock", + format: { kind: "iso", separator: "T", fraction: 0, zone }, + }); + const dated = (url: string, added: string, at: string): JsonObject => ({ + url, + added, + stamped: at, + day: `https://archive.example.test/${added.slice(0, 10).replace(/-/gu, "")}/${url}`, + }); + const composeDay = (zone: string, created: string, seed: string) => + createdComposer( + dated( + "https://quarry.example.test/ledger", + "2026-10-17T12:00:03Z", + created, + ), + [ECHO_URL, CLOCK, stamped(zone)], + [ + dated( + "https://ferries.example.test/timetable", + "2026-10-01T09:00:00Z", + seed, + ), + ], + )("day"); + + it.each([ + ["+02:00", "2026-10-17T14:00:03+02:00", "2026-10-01T11:00:00+02:00"], + ["+05:00", "2026-10-17T17:00:03+05:00", "2026-10-01T14:00:00+05:00"], + ])( + "takes no date from two clocks in different zones, %s beside UTC, which a served instant near midnight tells apart", + (zone, created, seed) => { + expect(composeDay(zone, created, seed)).toBeUndefined(); + }, + ); + + it("still takes a date from two clocks the create stamps in one zone", () => { + expect( + composeDay("Z", "2026-10-17T12:00:03Z", "2026-10-01T09:00:00Z"), + ).toMatchObject({ rule: "composed" }); + }); + + it.each([null, 42])( + "composes nothing a held entity holds as %s while every part it reads is there", + (held) => { + const other = { + ...bookmark( + "https://ferries.example.test/rota", + "2026-10-02T09:00:00Z", + ), + snapshot: held, + }; + expect( + createdComposer(CREATED, [ECHO_URL, CLOCK], [SEED, other])("snapshot"), + ).toBeUndefined(); + const { snapshot: _, ...without } = other; + expect( + createdComposer( + CREATED, + [ECHO_URL, CLOCK], + [SEED, without], + )("snapshot"), + ).toMatchObject({ rule: "composed" }); + const folder = (url: JsonValue) => ({ + uid: "other-folder", + title: "Other Folder", + url, + }); + const before = { + uid: "scratch-alpha", + title: "Scratch Alpha Harbor", + url: "/dashboards/f/scratch-alpha/scratch-alpha-harbor", + }; + const rename = (others: readonly JsonObject[]) => + composedChange("url", { + before, + after: { + ...before, + title: "Renamed Alpha Lantern", + url: "/dashboards/f/scratch-alpha/renamed-alpha-lantern", + }, + sent: { title: "Renamed Alpha Lantern" }, + idField: "uid", + others, + }); + expect(rename([folder(held)])).toBeUndefined(); + const { url: __, ...unaddressed } = folder(held); + expect(rename([unaddressed])).toMatchObject({ rule: "composed" }); + }, + ); +}); + +it( + "indexes a wide body's repeated values once, and only when a string is left to compose", + { timeout: 5_000 }, + () => { + const n = 8_000; + const shared = `https://shared.example.test/${"x".repeat(64)}`; + const body: JsonObject = Object.fromEntries( + Array.from({ length: n }, (_, index) => [`e${String(index)}`, shared]), + ); + const rules: FieldRule[] = Array.from({ length: n }, (_, index) => ({ + field: `e${String(index)}`, + rule: "echo", + })); + let reads = 0; + const counted = new Proxy(body, { + get: (target, key) => { + reads += 1; + return Reflect.get(target, key); + }, + }); + const composer = createdComposer(counted, rules, []); + expect(reads).toBe(0); + expect(composer("e0")).toBeUndefined(); + expect(composer("e1")).toBeUndefined(); + // One read per source to index them, once, and one per composed field. + expect(reads).toBe(n + 2); + }, +); diff --git a/src/domain/analysisView/environmentStateWriteComposed.ts b/src/domain/analysisView/environmentStateWriteComposed.ts new file mode 100644 index 0000000000..0017aa905f --- /dev/null +++ b/src/domain/analysisView/environmentStateWriteComposed.ts @@ -0,0 +1,274 @@ +import type { JsonValue } from "../jsonValue.js"; +import { same, scalarText, type JsonObject } from "./environmentStateRules.js"; +import { COPY_MIN_LENGTH } from "./environmentStateWriteCopies.js"; +import { GENERATED } from "./environmentStateWriteGenerated.js"; +import type { ComposedPart, FieldRule } from "./environmentStateWriteRules.js"; +import { + joinsAfter, + joinsBefore, + readClock, + slugOf, + zoneMinutesForFormat, +} from "./environmentStateWriteValues.js"; +import { + CLOCK_PATTERNS, + clockDigits, +} from "./environmentStateWriteClockFormats.js"; + +type SourcePart = Exclude; + +/** Text a field's value offers a composed string, and the part that spells it. */ +interface Candidate { + readonly token: string; + readonly part: SourcePart; +} + +/** Rules whose field holds a value the call or serving decides, which a created composed string may show. */ +const SOURCES: ReadonlySet = new Set([ + ...GENERATED, + "echo", + "copy", + "path", + "slug", + "clock", +]); + +/** The text a part spells from written fields; undefined when its field holds nothing it can spell. */ +export const composedPartText = ( + part: ComposedPart, + fields: JsonObject, +): string | undefined => { + if ("text" in part) return part.text; + const value = fields[part.from]; + if (part.as === "clock") return clockDigits(value, part.pattern); + if (part.as === "value") return scalarText(value); + return typeof value === "string" ? slugOf(value) : undefined; +}; + +/** A composed string spelled from written fields, or undefined when a part has nothing to spell. */ +export const composedValue = ( + parts: readonly ComposedPart[], + fields: JsonObject, +): string | undefined => { + let text = ""; + for (const part of parts) { + const spelled = composedPartText(part, fields); + if (spelled === undefined) return undefined; + text += spelled; + } + return text; +}; + +/** The texts a field's value offers: itself and its slug when long enough to show it, and a clock's calendar digits. */ +const candidatesOf = ( + field: string, + value: JsonValue | undefined, + shortest: number, +): Candidate[] => { + const candidates: Candidate[] = []; + const text = scalarText(value); + if (text !== undefined && text.length >= shortest) + candidates.push({ token: text, part: { from: field, as: "value" } }); + if (typeof value === "string") { + const slug = slugOf(value); + if (slug !== value && slug.length >= COPY_MIN_LENGTH) + candidates.push({ token: slug, part: { from: field, as: "slug" } }); + } + if (readClock(value) !== undefined) + for (const pattern of CLOCK_PATTERNS) { + const digits = clockDigits(value, pattern); + if (digits !== undefined) + candidates.push({ + token: digits, + part: { from: field, as: "clock", pattern }, + }); + } + return candidates; +}; + +/** Candidates by their text, and the lengths those texts take, longest first: looked up per position rather than searched. */ +interface CandidateIndex { + readonly byToken: ReadonlyMap; + readonly lengths: readonly number[]; +} + +const indexed = (candidates: readonly Candidate[]): CandidateIndex => { + const byToken = new Map(); + for (const candidate of candidates) { + const same = byToken.get(candidate.token); + if (same === undefined) byToken.set(candidate.token, [candidate]); + else same.push(candidate); + } + const lengths = [...new Set(candidates.map(({ token }) => token.length))]; + return { byToken, lengths: lengths.sort((left, right) => right - left) }; +}; + +/** The longest candidates whose text starts at a position and ends at a delimiter. */ +const matchesAt = ( + text: string, + at: number, + index: CandidateIndex, +): readonly Candidate[] => { + if (joinsBefore(text, at)) return []; + for (const length of index.lengths) { + if (at + length > text.length || joinsAfter(text, at + length)) continue; + const matches = index.byToken.get(text.slice(at, at + length)); + if (matches !== undefined) return matches; + } + return []; +}; + +/** + * Literal text around the longest delimited candidate at each position, left to right; undefined when none shows, or + * when one text comes from two sources serving could tell apart. `alike` names parts serving fills with one value. + */ +const segmented = ( + text: string, + index: CandidateIndex, + alike: (left: SourcePart, right: SourcePart) => boolean, +): ComposedPart[] | undefined => { + const parts: ComposedPart[] = []; + let literal = ""; + let shown = false; + for (let at = 0; at < text.length;) { + const matches = matchesAt(text, at, index); + const match = matches[0]; + if (match === undefined) { + literal += text[at]; + at += 1; + continue; + } + if (matches.some((other) => !alike(match.part, other.part))) + return undefined; + if (literal !== "") parts.push({ text: literal }); + literal = ""; + parts.push(match.part); + shown = true; + at += match.token.length; + } + if (literal !== "") parts.push({ text: literal }); + return shown ? parts : undefined; +}; + +const samePart = (left: SourcePart, right: SourcePart): boolean => + left.from === right.from && + left.as === right.as && + (left.as !== "clock" || + right.as !== "clock" || + left.pattern === right.pattern); + +/** How many entities holding the field and what its parts read spell it as the parts do, or undefined when one holds it otherwise, a non-string value included. */ +const heldSpellings = ( + parts: readonly ComposedPart[], + field: string, + held: readonly JsonObject[], +): number | undefined => { + let agreeing = 0; + for (const entity of held) { + if (!Object.hasOwn(entity, field)) continue; + const spelled = composedValue(parts, entity); + if (spelled === undefined) continue; + if (spelled !== entity[field]) return undefined; + agreeing += 1; + } + return agreeing; +}; + +/** + * For a created entity, each string field rebuilt from literal text and the fields other rules write, when it shows + * at least one and the type's other entities spell it the same way, at least one of them: one call alone cannot show + * which text is literal. + */ +export const createdComposer = ( + entity: JsonObject, + rules: readonly FieldRule[], + held: readonly JsonObject[], +): ((field: string) => FieldRule | undefined) => { + // Serving stamps every clock a create writes with the one write clock, so their digits in one zone are one source. + const zones = new Map( + rules.flatMap((rule) => + rule.rule === "clock" + ? [ + [ + rule.field, + rule.format.kind === "iso" + ? zoneMinutesForFormat(rule.format.zone) + : 0, + ] as const, + ] + : [], + ), + ); + const alike = (left: SourcePart, right: SourcePart): boolean => + samePart(left, right) || + (left.as === "clock" && + right.as === "clock" && + left.pattern === right.pattern && + zones.has(left.from) && + zones.get(left.from) === zones.get(right.from)); + let index: CandidateIndex | undefined; + return (field) => { + const text = entity[field]; + if (typeof text !== "string") return undefined; + index ??= indexed( + rules.flatMap((rule) => + SOURCES.has(rule.rule) + ? candidatesOf( + rule.field, + entity[rule.field], + // A generated value is short and still the call's own, as a template's is. + GENERATED.has(rule.rule) ? 1 : COPY_MIN_LENGTH, + ) + : [], + ), + ); + const parts = segmented(text, index, alike); + return parts === undefined || + parts.some((part) => "from" in part && part.from === field) || + !((heldSpellings(parts, field, held) ?? 0) > 0) + ? undefined + : { field, rule: "composed", parts }; + }; +}; + +/** + * An unsent string field an update rebuilt from literal text and fields serving knows, members stored as sent and those + * the call left as they were: the same parts spell it before the call and on the type's other entities. + */ +export const composedChange = ( + field: string, + entity: { + readonly before: JsonObject; + readonly after: JsonObject; + readonly sent: JsonObject; + readonly idField: string; + readonly others: readonly JsonObject[]; + }, +): FieldRule | undefined => { + const { before, after, sent, idField } = entity; + const text = after[field]; + if (typeof text !== "string" || typeof before[field] !== "string") + return undefined; + const known = Object.keys(after).filter( + (other) => + other !== field && + (Object.hasOwn(sent, other) + ? same(sent[other], after[other]) + : Object.hasOwn(before, other) && same(before[other], after[other])), + ); + const index = indexed( + known.flatMap((other) => + candidatesOf( + other, + after[other], + other === idField ? 1 : COPY_MIN_LENGTH, + ), + ), + ); + const parts = segmented(text, index, samePart); + return parts === undefined || + composedValue(parts, before) !== before[field] || + heldSpellings(parts, field, entity.others) === undefined + ? undefined + : { field, rule: "composed", parts }; +}; diff --git a/src/domain/analysisView/environmentStateWriteCreate.ts b/src/domain/analysisView/environmentStateWriteCreate.ts index f8f39e7202..78c1f27c4e 100644 --- a/src/domain/analysisView/environmentStateWriteCreate.ts +++ b/src/domain/analysisView/environmentStateWriteCreate.ts @@ -33,6 +33,7 @@ import { import { built, callOf, + clockEvidence, fail, itemReads, refuse, @@ -47,6 +48,7 @@ import { type WriteStep, } from "./environmentStateWriteEffects.js"; import { + createdInteger, explainCreate, insertCandidates, insertIndex, @@ -61,13 +63,20 @@ import { } from "./environmentStateWriteObserve.js"; import { pathValuesOf, + siblingTrimmedClocks, type FieldRule, type WriteCall, } from "./environmentStateWriteRules.js"; import { + fittedClockFormat, + noteFixedWidths, +} from "./environmentStateWriteClockFormats.js"; +import { + containsDelimited, + generatedIntegerDigits, isUuid, + likeGeneratedInteger, nextSequence, - readClock, sequenceNumber, spelledLike, } from "./environmentStateWriteValues.js"; @@ -133,6 +142,47 @@ const createdObservation = ( const fieldsOf = (entities: readonly StateEntity[]): JsonObject[] => entities.map((entity) => entity.fields); +/** Whether a created field holds a generated token or integer that another field holds too, which no rule rebuilds. */ +const linkedGenerated = ( + field: string, + entity: JsonObject, + type: readonly JsonObject[], +): boolean => + createdToken(field, entity, type) === "linked" || + createdInteger(field, entity, type) === "linked"; + +/** Whether a field's rule rebuilds it from `source`, as a template or a composition does. */ +const rebuiltFrom = (rule: FieldRule | undefined, source: string): boolean => + rule?.rule === "template" + ? rule.from === source + : rule?.rule === "composed" && + rule.parts.some((part) => "from" in part && part.from === source); + +/** Whether a unique integer's digits show in another string whose rule does not rebuild them from it, which seeding would leave behind. */ +const strandedInteger = ( + rules: readonly FieldRule[], + entity: JsonObject, +): boolean => + rules.some((rule) => { + const digits = + rule.rule === "unique" + ? generatedIntegerDigits(entity[rule.field]) + : undefined; + return ( + digits !== undefined && + Object.entries(entity).some( + ([other, kept]) => + other !== rule.field && + typeof kept === "string" && + containsDelimited(kept, digits) && + !rebuiltFrom( + rules.find((candidate) => candidate.field === other), + rule.field, + ), + ) + ); + }); + /** Build a create plan from its exemplar's observation. */ const buildCreate = ( context: EffectContext, @@ -163,8 +213,9 @@ const buildCreate = ( Object.keys(evidence.observed.object).some( (field) => !explainedFields.has(field) && - createdToken(field, evidence.observed.object, held) === "linked", - ) + linkedGenerated(field, evidence.observed.object, held), + ) || + strandedInteger(explained.rules, evidence.observed.object) ) { refuse(draft, step, "unexplained-change"); return false; @@ -221,6 +272,7 @@ const buildCreate = ( insert: draft.candidates[0]!, }); draft.observed = evidence.observed.object; + for (const entity of held) noteFixedWidths(draft.fixedWidth, entity); return true; }; @@ -269,21 +321,37 @@ const followsSequences = ( ); }); -/** Whether a later call's generated values take the forms serving generates them in: clocks in the learned format, unique values as the exemplar's kind of token, UUID identities as UUIDs, and sequences in order. */ -const followsGenerated = ( +/** Whether a later create preserves generation shapes and its derived fields reproduce the captured observation. */ +const followsCreateRules = ( context: EffectContext, draft: WriteDraft, - created: Pick, + created: Pick & { + readonly observed: JsonObject; + }, ): boolean => draft.plan.rules.every((rule) => { const value = created.fields[rule.field]; + const exemplar = draft.observed?.[rule.field]; + if (rule.rule === "composed") + return ( + !Object.hasOwn(created.observed, rule.field) || + value === created.observed[rule.field] + ); if (rule.rule === "clock") return ( - JSON.stringify(readClock(value)?.format) === JSON.stringify(rule.format) + fittedClockFormat( + rule.format, + value, + clockEvidence(draft, rule.field), + ) !== undefined ); if (rule.rule === "unique") return ( - (isUuid(value) || isOpaqueToken(value, draft.observed?.[rule.field])) && + (isUuid(exemplar) + ? isUuid(value) + : generatedIntegerDigits(exemplar) !== undefined + ? likeGeneratedInteger(value, exemplar) + : isOpaqueToken(value, exemplar)) && !context.state .ofType(created.type) .some((held) => held.fields[rule.field] === value) @@ -293,6 +361,28 @@ const followsGenerated = ( ); }) && followsSequences(context, draft, created); +/** Widen the plan's clock formats to the trimmed spelling a later call showed, and its other clocks with them unless a value the plan saw shows them at a fixed width. */ +const fitClocks = (draft: WriteDraft, fields: JsonObject): void => { + const fitted = draft.plan.rules.map((rule) => { + if (rule.rule !== "clock") return rule; + const format = fittedClockFormat( + rule.format, + fields[rule.field], + clockEvidence(draft, rule.field), + ); + return format === undefined || format === rule.format + ? rule + : { ...rule, format }; + }); + draft.plan.rules = siblingTrimmedClocks( + fitted, + draft.observed, + fields, + Object.fromEntries(draft.fixedWidth), + ); + noteFixedWidths(draft.fixedWidth, fields); +}; + /** Record the values a create's sequences took, which the next call follows. */ const takeSequences = ( draft: WriteDraft, @@ -458,15 +548,16 @@ export const applyCreate = ( const later = built(draft); if ( later && - draft.plan.rules.some( + (draft.plan.rules.some( (rule) => rule.rule === "unique" && - createdToken( + linkedGenerated( rule.field, observed.object, fieldsOf(context.state.ofType(type)), - ) === "linked", - ) + ), + ) || + strandedInteger(draft.plan.rules, observed.object)) ) { refuse(draft, step, "unexplained-change"); return recorded(); @@ -500,16 +591,18 @@ export const applyCreate = ( id === undefined || context.state.entity(type, collection, id) !== undefined || (later && - !followsGenerated(context, draft, { + !followsCreateRules(context, draft, { type, collection, fields: fields.value, + observed: observed.object, })) ) { refuse(draft, step, "unexplained-change"); return recorded(); } takeSequences(draft, { collection, fields: fields.value }); + fitClocks(draft, fields.value); // The call's own values follow the forms serving generates; the state holds a nested clock as the read-back restamped it, as an update's. const stored = restampedFields( searchFrom(context, step), diff --git a/src/domain/analysisView/environmentStateWriteEffects.ts b/src/domain/analysisView/environmentStateWriteEffects.ts index 47acc73df6..088f9d4dfb 100644 --- a/src/domain/analysisView/environmentStateWriteEffects.ts +++ b/src/domain/analysisView/environmentStateWriteEffects.ts @@ -48,12 +48,18 @@ import { } from "./environmentStateWriteObserve.js"; import { plannedBody, restampedFields } from "./environmentStateWriteStamps.js"; import { readClock } from "./environmentStateWriteValues.js"; +import { + fittedClockFormat, + matchesClockFormat, + noteFixedWidths, +} from "./environmentStateWriteClockFormats.js"; import { actionItemPath, observationNeeds, } from "./environmentStateWriteAddress.js"; import { pathValuesOf, + siblingTrimmedClocks, type FieldRule, type InsertRule, type StateWritePlan, @@ -84,8 +90,17 @@ export interface WriteDraft { readonly steady: Set; /** Unsent clocks a call left as they were in its own second, which it cannot show it stamps. */ readonly unstamped: Set; + /** A value a held entity or an accepted call showed each clock field at a fixed width in, which keeps it untrimmed. */ + readonly fixedWidth: Map; } +/** The value a draft fits a clock field against: one showing it at a fixed width, else the exemplar's. */ +export const clockEvidence = ( + draft: WriteDraft, + field: string, +): JsonValue | undefined => + draft.fixedWidth.get(field) ?? draft.observed?.[field]; + /** A write observation plus the fields its selected read view actually covers, if any. */ type UpdateObservation = Observed & { readonly readFields?: readonly string[]; @@ -435,6 +450,8 @@ const buildUpdate = ( rules: evidence.explained.rules, }); draft.observed = evidence.after.object; + for (const entity of context.state.ofType(draft.plan.target!)) + noteFixedWidths(draft.fixedWidth, entity.fields); }; /** @@ -509,19 +526,25 @@ export const updateEntity = ( } const unknown = context.unknown(type, collection, id); const call = { ...callOf(step, body), body }; - const explained = explainUpdate({ - before: entity.fields, - after: after.object, - call, - method: step.tool.method, - idField, - id, - startMs: startOf(context, step), - ...(after.readFields === undefined ? {} : { readFields: after.readFields }), - others: otherFields(context.state.ofType(type), { collection, id }), - shown: shownBefore(context.viewOf, entity, after.ordinal), - action: needs.action, - }); + const { explained, rules } = fitUpdateClocks( + draft, + after.object, + explainUpdate({ + before: entity.fields, + after: after.object, + call, + method: step.tool.method, + idField, + id, + startMs: startOf(context, step), + ...(after.readFields === undefined + ? {} + : { readFields: after.readFields }), + others: otherFields(context.state.ofType(type), { collection, id }), + shown: shownBefore(context.viewOf, entity, after.ordinal), + action: needs.action, + }), + ); if (explained.unexplained.length > 0 && !unknown) { refuse(draft, step, "unexplained-change"); return recorded(); @@ -529,11 +552,19 @@ export const updateEntity = ( // Each call must agree with the plan; one on an unknown entity cannot be checked, so it adds nothing to the plan. if ( built(draft) && - (unknown || contradictsPlan(draft, entity.fields, after.object, explained)) + (unknown || + contradictsPlan( + { ...draft, plan: { ...draft.plan, rules } }, + entity.fields, + after.object, + explained, + )) ) { if (!unknown) refuse(draft, step, "unexplained-change"); return recorded(); } + draft.plan.rules = rules; + noteFixedWidths(draft.fixedWidth, after.object); buildUpdate(context, draft, step, { after, explained, idField }); // Serving reads the plan's observation, so the gate does too; generated values are this call's. const written = updatedFields( @@ -586,6 +617,54 @@ export const updateEntity = ( }; }; +/** A later update's clocks in the plan's format, and the plan's rules widened to the trimmed spelling the call showed when one covers both, its siblings with it wherever the call's values fit; the plan adopts them only once the call is checked. */ +const fitUpdateClocks = ( + draft: WriteDraft, + after: JsonObject, + explained: ReturnType, +): { + readonly explained: ReturnType; + readonly rules: FieldRule[]; +} => { + if (!built(draft)) return { explained, rules: draft.plan.rules }; + const rules = draft.plan.rules.map((rule) => { + const later = explained.rules.find((other) => other.field === rule.field); + if ( + rule.rule !== "clock" || + later?.rule !== "clock" || + JSON.stringify(later.format) === JSON.stringify(rule.format) + ) + return rule; + const format = fittedClockFormat( + rule.format, + after[rule.field], + clockEvidence(draft, rule.field), + ); + return format === undefined ? rule : { ...rule, format }; + }); + const fitted = siblingTrimmedClocks( + rules, + draft.observed, + after, + Object.fromEntries(draft.fixedWidth), + ); + const planned = new Map(fitted.map((rule) => [rule.field, rule])); + return { + explained: { + ...explained, + rules: explained.rules.map((rule) => { + const held = planned.get(rule.field); + return rule.rule === "clock" && + held?.rule === "clock" && + matchesClockFormat(held.format, readClock(after[rule.field])) + ? { ...rule, format: held.format } + : rule; + }), + }, + rules: fitted, + }; +}; + /** Serving regenerates a learned clock on every update, so recorded calls must advance it. */ const contradictsGeneratedClock = ( before: JsonObject, diff --git a/src/domain/analysisView/environmentStateWriteExplain.test.ts b/src/domain/analysisView/environmentStateWriteExplain.test.ts index a7cb1bd845..101a18087d 100644 --- a/src/domain/analysisView/environmentStateWriteExplain.test.ts +++ b/src/domain/analysisView/environmentStateWriteExplain.test.ts @@ -1109,3 +1109,89 @@ it("accepts a copied member only with its recorded value when a template keeps i sent: "group-alpha-2", }); }); + +it.each<[string, JsonValue, readonly JsonValue[], boolean]>([ + [ + "a time-based id no held entity takes", + 1656166527623168, + [1655476044521472, 1655476044521999], + true, + ], + [ + "one a held entity already holds", + 1655476044521472, + [1655476044521472], + false, + ], + [ + "one among held ids of another length", + 1656166527623168, + [16554760445214], + false, + ], + [ + "one among held ids spelled as text", + 1656166527623168, + ["1655476044521472"], + false, + ], + [ + "one among held entities sharing an id", + 1656166527623168, + [1655476044521472, 1655476044521472], + false, + ], + ["one with no held entity to compare", 1656166527623168, [], false], + ["a short number", 123456789, [987654321], false], +])("learns %s as generated", (_name, value, held, unique) => { + const result = explainCreate( + created( + { id: 7, ref: value }, + {}, + { type: held.map((ref, index) => ({ id: index + 1, ref })) }, + ), + ); + expect(ruleFor(result.rules, "ref")).toEqual( + unique ? { field: "ref", rule: "unique" } : undefined, + ); +}); + +it("trims a created clock's fraction, and its sibling clock's, when held entities spell the field trimmed", () => { + const at = new Date(START).toISOString().replace(".000Z", ""); + const result = explainCreate( + created( + { + id: 7, + createdAt: `${at}.123456789Z`, + updatedAt: `${at}.123456791Z`, + }, + {}, + { + type: [ + { + id: 1, + createdAt: "2026-01-01T00:00:00.12345678Z", + updatedAt: "2026-01-01T00:00:00.123456789Z", + }, + ], + }, + ), + ); + const trimmed = { + kind: "iso", + separator: "T", + fraction: 9, + zone: "Z", + trimmed: true, + }; + expect(ruleFor(result.rules, "createdAt")).toEqual({ + field: "createdAt", + rule: "clock", + format: trimmed, + }); + expect(ruleFor(result.rules, "updatedAt")).toEqual({ + field: "updatedAt", + rule: "clock", + format: trimmed, + }); +}); diff --git a/src/domain/analysisView/environmentStateWriteExplain.ts b/src/domain/analysisView/environmentStateWriteExplain.ts index 26ad07dcea..9bcbdbb3b9 100644 --- a/src/domain/analysisView/environmentStateWriteExplain.ts +++ b/src/domain/analysisView/environmentStateWriteExplain.ts @@ -9,21 +9,30 @@ import { type JsonObject, type StateEntity, } from "./environmentStateRules.js"; -import type { - FieldRule, - InsertRule, - WriteBody, - WriteCall, - WriteResponse, +import { + siblingTrimmedClocks, + type FieldRule, + type InsertRule, + type WriteBody, + type WriteCall, + type WriteResponse, } from "./environmentStateWriteRules.js"; import { containsDelimited, + containsNumber, + generatedIntegerDigits, integerOf, isUuid, readClock, sequenceNumber, slugOf, + type ClockFormat, } from "./environmentStateWriteValues.js"; +import { createdComposer } from "./environmentStateWriteComposed.js"; +import { + fittedClockFormat, + showsFixedWidth, +} from "./environmentStateWriteClockFormats.js"; import { COPY_MIN_LENGTH, instantMembers, @@ -157,6 +166,64 @@ const requestRule = ( : { field, rule: "slug", from: member }; }; +/** A clock's format, trimmed when the type's values of the field show its fraction trimmed and none shows a fixed width. */ +const learnedClockFormat = ( + format: ClockFormat, + field: string, + value: JsonValue, + type: readonly JsonObject[], +): ClockFormat => { + let learned = format; + for (const member of type) { + const fitted = fittedClockFormat(learned, member[field], value); + if (fitted?.kind === "iso" && fitted.trimmed === true) learned = fitted; + } + return type.some((member) => showsFixedWidth(format, member[field])) + ? format + : learned; +}; + +/** A created large integer's proof, as a time-based id's: no entity of its type holds it and each holds a distinct one of the same length and kind; linked when another field holds it too, as a number at any depth or delimited in a nested string, which seeding cannot keep. */ +export const createdInteger = ( + field: string, + entity: JsonObject, + type: readonly JsonObject[], +): "independent" | "linked" | undefined => { + const value = entity[field]; + const digits = generatedIntegerDigits(value); + if ( + digits === undefined || + readClock(value) !== undefined || + type.length === 0 + ) + return undefined; + const held = new Set([digits]); + const distinct = type.every((member) => { + const other = generatedIntegerDigits(member[field]); + if ( + other === undefined || + typeof member[field] !== typeof value || + other.length !== digits.length || + held.has(other) + ) + return false; + held.add(other); + return true; + }); + if (!distinct) return undefined; + return Object.entries(entity).some( + ([other, kept]) => + other !== field && + (kept === value || + containsNumber(kept, digits) || + (typeof kept === "object" && + kept !== null && + containsDelimited(kept, digits))), + ) + ? "linked" + : "independent"; +}; + /** The rule a created field's generated value follows, if it follows one. */ const generatedRule = ( evidence: CreateEvidence, @@ -166,7 +233,11 @@ const generatedRule = ( if (field === evidence.idField) return identityRule(field, value); const clock = readClock(value); if (clock !== undefined && isClockNear(value, evidence.startMs)) - return { field, rule: "clock", format: clock.format }; + return { + field, + rule: "clock", + format: learnedClockFormat(clock.format, field, value, evidence.type), + }; const integer = sequenceNumber(value); if (integer !== undefined) { if (nextInteger(evidence.collection, field, value) === integer) @@ -175,9 +246,10 @@ const generatedRule = ( return { field, rule: "sequence", scope: "type" }; } if ( - isUuid(value) && - evidence.type.length > 0 && - evidence.type.every((member) => member[field] !== value) + (isUuid(value) && + evidence.type.length > 0 && + evidence.type.every((member) => member[field] !== value)) || + createdInteger(field, evidence.entity, evidence.type) === "independent" ) return { field, rule: "unique" }; if (createdToken(field, evidence.entity, evidence.type) === "independent") @@ -208,7 +280,7 @@ export const explainCreate = ( readonly body: WriteBody; readonly tied: boolean; } => { - const rules: FieldRule[] = []; + const ruled: FieldRule[] = []; const unexplained: string[] = []; const index = indexBody(evidence.call.body); for (const [field, value] of Object.entries(evidence.entity)) { @@ -216,8 +288,9 @@ export const explainCreate = ( requestRule(evidence, index, field, value) ?? generatedRule(evidence, field, value); if (rule === undefined) unexplained.push(field); - else rules.push(rule); + else ruled.push(rule); } + const rules = siblingTrimmedClocks(ruled, evidence.entity, ...evidence.type); // Generated values first, then copies: an echoed value may recur in unrelated text, a copied one is the call's own unless also echoed. const echoed = new Set( rules.flatMap((rule) => @@ -233,8 +306,10 @@ export const explainCreate = ( ), ]; const constants: string[] = []; + const composed = createdComposer(evidence.entity, rules, evidence.type); for (const field of unexplained) { - const template = templateRule(evidence.entity, field, sources); + const template = + templateRule(evidence.entity, field, sources) ?? composed(field); if (template === undefined) constants.push(field); else rules.push(template); } @@ -244,6 +319,12 @@ export const explainCreate = ( const templates = rules.flatMap((rule) => rule.rule === "template" ? [rule.field] : [], ); + // A composed string rebuilds the values it names, so only its literal text can tie it. + const literals = rules.flatMap((rule) => + rule.rule === "composed" + ? rule.parts.flatMap((part) => ("text" in part ? [part.text] : [])) + : [], + ); // A member whose value a template cannot rebuild, or that a constant derives from, is accepted only with its recorded value. const texts = [...constants, ...templates]; const stranded = strandedCopies(rules, evidence.entity, texts); @@ -277,9 +358,10 @@ export const explainCreate = ( Object.keys(evidence.call.body), ), // A template rebuilds only its generated value, so other text it keeps ties it like a constant. - tied: [...constants, ...templates].some((field) => - tokens.some((token) => containsDelimited(evidence.entity[field]!, token)), - ), + tied: [ + ...[...constants, ...templates].map((field) => evidence.entity[field]!), + ...literals, + ].some((text) => tokens.some((token) => containsDelimited(text, token))), }; }; diff --git a/src/domain/analysisView/environmentStateWriteRules.test.ts b/src/domain/analysisView/environmentStateWriteRules.test.ts new file mode 100644 index 0000000000..e0e3d7c2ff --- /dev/null +++ b/src/domain/analysisView/environmentStateWriteRules.test.ts @@ -0,0 +1,52 @@ +import { expect, it } from "vitest"; +import { + siblingTrimmedClocks, + type FieldRule, +} from "./environmentStateWriteRules.js"; + +it.each([false, true])( + "propagates the widest trimmed peer in each separator/zone group (reversed: %s)", + (reverse) => { + const clock = ( + field: string, + separator: "T" | " ", + zone: string, + fraction: number, + trimmed = false, + ): FieldRule => ({ + field, + rule: "clock", + format: { + kind: "iso", + separator, + zone, + fraction, + ...(trimmed ? { trimmed: true } : {}), + }, + }); + const rules = [ + clock("utc", "T", "Z", 9, true), + clock("shortOffset", "T", "+01:00", 6, true), + clock("wideOffset", "T", "+01:00", 9, true), + clock("offsetSibling", "T", "+01:00", 8), + clock("spaced", " ", "+01:00", 9, true), + clock("spacedSibling", " ", "+01:00", 8), + clock("fixedSibling", "T", "+01:00", 9), + clock("otherZone", "T", "-01:00", 8), + ]; + const fitted = siblingTrimmedClocks(reverse ? rules.toReversed() : rules, { + offsetSibling: "2026-03-04T05:06:07.12345678+01:00", + spacedSibling: "2026-03-04 05:06:07.12345678+01:00", + fixedSibling: "2026-03-04T05:06:07.123456780+01:00", + otherZone: "2026-03-04T05:06:07.12345678-01:00", + }); + expect(fitted).toContainEqual( + clock("offsetSibling", "T", "+01:00", 9, true), + ); + expect(fitted).toContainEqual( + clock("spacedSibling", " ", "+01:00", 9, true), + ); + expect(fitted).toContainEqual(clock("fixedSibling", "T", "+01:00", 9)); + expect(fitted).toContainEqual(clock("otherZone", "T", "-01:00", 8)); + }, +); diff --git a/src/domain/analysisView/environmentStateWriteRules.ts b/src/domain/analysisView/environmentStateWriteRules.ts index 4b70317bec..badc28748f 100644 --- a/src/domain/analysisView/environmentStateWriteRules.ts +++ b/src/domain/analysisView/environmentStateWriteRules.ts @@ -8,6 +8,11 @@ import { type JsonObject, } from "./environmentStateRules.js"; import type { ClockFormat } from "./environmentStateWriteValues.js"; +import { + showsFixedWidth, + trimmedLike, + type ClockPattern, +} from "./environmentStateWriteClockFormats.js"; /** What a write tool does to the state, by method and the target its path names. */ export type WriteEffect = "create" | "update" | "delete" | "document" | "none"; @@ -70,6 +75,12 @@ export type FieldRule = | { readonly field: string; readonly rule: "path"; readonly from: string } | { readonly field: string; readonly rule: "slug"; readonly from: string } | { readonly field: string; readonly rule: "template"; readonly from: string } + /** A string rebuilt from literal text and other written fields, in order. */ + | { + readonly field: string; + readonly rule: "composed"; + readonly parts: ComposedPart[]; + } | { readonly field: string; readonly rule: "increment" } /** An update stores `value` whenever the call sends the member, as the call that showed it did. */ | { @@ -98,6 +109,50 @@ export type FieldRule = readonly value: JsonValue; }; +/** Clock rules spelled as a trimmed sibling is, since an application spells its clocks alike: fields whose `values`, and every `later` call's, show no fixed width. */ +export const siblingTrimmedClocks = ( + rules: readonly FieldRule[], + values: Readonly> | undefined, + ...later: readonly Readonly>[] +): FieldRule[] => { + const trimmed = new Map>(); + for (const rule of rules) { + if ( + rule.rule !== "clock" || + rule.format.kind !== "iso" || + rule.format.trimmed !== true + ) + continue; + const format = rule.format; + const group = `${format.separator}${format.zone}`; + const widest = trimmed.get(group); + if (widest === undefined || format.fraction > widest.fraction) + trimmed.set(group, format); + } + if (trimmed.size === 0) return [...rules]; + return rules.map((rule) => { + if (rule.rule !== "clock" || rule.format.kind !== "iso") return rule; + const peer = trimmed.get(`${rule.format.separator}${rule.format.zone}`); + if (peer === undefined) return rule; + const format = later.some((fields) => + showsFixedWidth(rule.format, fields[rule.field]), + ) + ? rule.format + : trimmedLike(rule.format, peer, values?.[rule.field]); + return format === rule.format ? rule : { ...rule, format }; + }); +}; + +/** One part of a composed string: literal text, or another written field as written, slugged, or as a clock's calendar digits. */ +export type ComposedPart = + | { readonly text: string } + | { readonly from: string; readonly as: "value" | "slug" } + | { + readonly from: string; + readonly as: "clock"; + readonly pattern: ClockPattern; + }; + /** Where a create places its entity in its collection's order. */ export type InsertRule = | { readonly rule: "first" | "last" } diff --git a/src/domain/analysisView/environmentStateWriteValues.test.ts b/src/domain/analysisView/environmentStateWriteValues.test.ts index fc8d3103e4..a719b46a30 100644 --- a/src/domain/analysisView/environmentStateWriteValues.test.ts +++ b/src/domain/analysisView/environmentStateWriteValues.test.ts @@ -4,11 +4,13 @@ import { containsDelimited, delimitedOffsets, formatClock, + generatedIntegerDigits, integerOf, isUuid, readClock, replaceDelimited, respellClock, + seededInteger, seededToken, seededUuid, sequenceNumber, @@ -69,10 +71,24 @@ it.each<[string, number, ClockFormat, number?]>([ ])( "reads the ISO clock %s with its spelling", (text, ms, format, subMillisecond = 0) => { - expect(readClock(text)).toEqual({ ms, subMillisecond, format }); + expect(readClock(text)).toEqual({ + ms, + subMillisecond, + format, + fixedWidth: false, + }); }, ); +it.each<[string, boolean]>([ + ["2026-03-04T05:06:07.120Z", true], + ["2026-03-04T05:06:07.000Z", true], + ["2026-03-04T05:06:07.12Z", false], + ["2026-03-04T05:06:07Z", false], +])("reads whether the fraction of %s shows a fixed width", (text, fixed) => { + expect(readClock(text)?.fixedWidth).toBe(fixed); +}); + it.each([ "0001-02-29T00:00:00Z", "0099-02-29T00:00:00Z", @@ -153,11 +169,17 @@ it.each<[JsonValue | undefined, ReturnType]>([ ms: WHOLE_SECOND, subMillisecond: 0, format: { kind: "epoch", unit: "s" }, + fixedWidth: true, }, ], [ WITH_MS, - { ms: WITH_MS, subMillisecond: 0, format: { kind: "epoch", unit: "ms" } }, + { + ms: WITH_MS, + subMillisecond: 0, + format: { kind: "epoch", unit: "ms" }, + fixedWidth: true, + }, ], [999_999_999, undefined], [99_999_999_999, undefined], @@ -413,3 +435,42 @@ it("respells a clock at another offset with its own fraction digits, across a da "2026-10-21T17:06:13.250+00:00", ); }); + +const TRIMMED_9: ClockFormat = { + kind: "iso", + separator: "T", + fraction: 9, + zone: "Z", + trimmed: true, +}; + +it("spells a trimmed format without trailing fraction zeros, and without the point when none remain", () => { + expect(formatClock(WITH_MS, TRIMMED_9)).toBe("2026-03-04T05:06:07.123Z"); + expect(formatClock(WHOLE_SECOND, TRIMMED_9)).toBe("2026-03-04T05:06:07Z"); + expect(respellClock("2026-03-04T05:06:07.120000Z", TRIMMED_9)).toBe( + "2026-03-04T05:06:07.12Z", + ); +}); + +it("seeds integers as long as the recorded one, keeping its JSON kind and a safe number", () => { + const number = seededInteger("seed", 1656166527623168); + expect(typeof number).toBe("number"); + expect(Number.isSafeInteger(number)).toBe(true); + expect(String(number)).toMatch(/^[1-8]\d{15}$/u); + expect(seededInteger("seed", 1656166527623168)).toBe(number); + expect(seededInteger("other", 1656166527623168)).not.toBe(number); + expect(seededInteger("seed", "1234567890123456789")).toMatch( + /^[1-9]\d{18}$/u, + ); +}); + +it.each<[JsonValue, string | undefined]>([ + [1656166527623168, "1656166527623168"], + ["1234567890123456789", "1234567890123456789"], + [123456789, undefined], + ["0123456789012", undefined], + [-1656166527623168, undefined], + [1.5e15 + 0.5, undefined], +])("reads %j as a generated integer's digits", (value, expected) => { + expect(generatedIntegerDigits(value)).toBe(expected); +}); diff --git a/src/domain/analysisView/environmentStateWriteValues.ts b/src/domain/analysisView/environmentStateWriteValues.ts index 46b6afe0e0..8ab0d19bcb 100644 --- a/src/domain/analysisView/environmentStateWriteValues.ts +++ b/src/domain/analysisView/environmentStateWriteValues.ts @@ -9,14 +9,18 @@ export type ClockFormat = readonly separator: "T" | " "; readonly fraction: number; readonly zone: string; + /** Fraction digits end at the last non-zero one, and the point goes with them, as Go spells nanoseconds; `fraction` is the most shown. */ + readonly trimmed?: true; } | { readonly kind: "epoch"; readonly unit: "s" | "ms" }; -interface ClockReading { +export interface ClockReading { readonly ms: number; /** Nanoseconds after the floored millisecond, so distinct recorded instants stay distinct. */ readonly subMillisecond: number; readonly format: ClockFormat; + /** The fraction ends in a zero, which a trimmed spelling never shows, so its width is fixed. */ + readonly fixedWidth: boolean; } const MINUTE_MS = 60_000; @@ -37,7 +41,8 @@ const zoneMinutes = (zone: string): number | undefined => { export const isClockZone = (zone: string): boolean => zoneMinutes(zone) !== undefined; -const zoneMinutesForFormat = (zone: string): number => { +/** Minutes east of UTC a stored format's zone names. */ +export const zoneMinutesForFormat = (zone: string): number => { const minutes = zoneMinutes(zone); if (minutes === undefined) throw new TypeError(`Invalid clock zone in a derived format: ${zone}`); @@ -64,6 +69,8 @@ const readIsoClock = (value: string): ClockReading | undefined => { fraction: spelling.fractionDigits, zone: spelling.zone, }, + // The instant's fraction drops trailing zeros, so a longer spelling ended in one. + fixedWidth: spelling.fractionDigits > instant.fraction.length, }; }; @@ -79,12 +86,14 @@ export const readClock = ( ms: value * 1000, subMillisecond: 0, format: { kind: "epoch", unit: "s" }, + fixedWidth: true, }; if (value >= 1e12 && value < 1e13) return { ms: value, subMillisecond: 0, format: { kind: "epoch", unit: "ms" }, + fixedWidth: true, }; return undefined; }; @@ -116,7 +125,12 @@ export const formatClock = ( 0, format.fraction, ); - return isoText(at, format.separator, digits, format.zone); + return isoText( + at, + format.separator, + format.trimmed === true ? digits.replace(/0+$/u, "") : digits, + format.zone, + ); }; /** @@ -147,7 +161,9 @@ export const respellClock = ( isoText( at, format.separator, - digits.padEnd(format.fraction, "0"), + format.trimmed === true + ? digits.replace(/0+$/u, "") + : digits.padEnd(format.fraction, "0"), format.zone, ) ?? value ); @@ -158,6 +174,39 @@ const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/iu; export const isUuid = (value: JsonValue | undefined): value is string => typeof value === "string" && UUID.test(value); +/** Digits a generated integer shows at least, as a time-based id does; counts and codes are shorter. */ +const GENERATED_INTEGER_DIGITS = 10; + +/** A large integer's digits, from a safe JSON number or a string of digits without a leading zero. */ +export const generatedIntegerDigits = ( + value: JsonValue | undefined, +): string | undefined => { + const digits = + typeof value === "number" && Number.isSafeInteger(value) && value > 0 + ? String(value) + : typeof value === "string" && /^[1-9]\d*$/u.test(value) + ? value + : undefined; + return digits !== undefined && + digits.length >= GENERATED_INTEGER_DIGITS && + digits.length <= 20 + ? digits + : undefined; +}; + +/** Whether a generated integer retains the exemplar's digit count and JSON kind. */ +export const likeGeneratedInteger = ( + value: JsonValue | undefined, + exemplar: JsonValue | undefined, +): boolean => { + const digits = generatedIntegerDigits(value); + return ( + digits !== undefined && + typeof value === typeof exemplar && + digits.length === generatedIntegerDigits(exemplar)?.length + ); +}; + /** An integer value as a number, from a JSON number or a string of digits. */ export const integerOf = (value: JsonValue | undefined): number | undefined => typeof value === "number" && Number.isSafeInteger(value) @@ -260,6 +309,14 @@ const nextCharacter = (text: string, at: number): string | undefined => { return point === undefined ? undefined : String.fromCodePoint(point); }; +/** Whether the character before UTF-16 offset `at` is a letter or digit, joining text that starts there to its left. */ +export const joinsBefore = (text: string, at: number): boolean => + isAlphanumeric(previousCharacter(text, at)); + +/** Whether the character at UTF-16 offset `at` is a letter or digit, joining text that ends there to its right. */ +export const joinsAfter = (text: string, at: number): boolean => + isAlphanumeric(nextCharacter(text, at)); + /** Start offsets of `token` in `text` where neither neighbour is a letter or digit. */ export const delimitedOffsets = (text: string, token: string): number[] => { const offsets: number[] = []; @@ -267,8 +324,7 @@ export const delimitedOffsets = (text: string, token: string): number[] => { let at = text.indexOf(token); while (at >= 0) { const delimited = - !isAlphanumeric(previousCharacter(text, at)) && - !isAlphanumeric(nextCharacter(text, at + token.length)); + !joinsBefore(text, at) && !joinsAfter(text, at + token.length); if (delimited) offsets.push(at); // A rejected match may overlap a delimited one, so the search moves on by one character. at = text.indexOf(token, at + (delimited ? token.length : 1)); @@ -305,6 +361,19 @@ export const containsDelimited = (value: JsonValue, token: string): boolean => { return false; }; +/** Whether any number inside a value spells `digits`. */ +export const containsNumber = (value: JsonValue, digits: string): boolean => { + const pending: JsonValue[] = [value]; + for (let next = pending.pop(); next !== undefined; next = pending.pop()) { + if (typeof next === "number") { + if (String(next) === digits) return true; + } else if (typeof next === "object" && next !== null) + for (const child of Array.isArray(next) ? next : Object.values(next)) + pending.push(child); + } + return false; +}; + /** SHA-256 bytes of a seed, extended by counter blocks to the requested length. */ const seededBytes = (seed: string, length: number): Uint8Array => { const bytes = new Uint8Array(length); @@ -359,6 +428,20 @@ export const seededLike = ( ? seededUuid(seed, String(exemplar)) : seededToken(seed, String(exemplar)); +/** A seeded integer with as many digits as the recorded one and none leading zero, as a number when it is one and stays safe. */ +export const seededInteger = (seed: string, exemplar: JsonValue): JsonValue => { + const length = String(exemplar).length; + const bytes = seededBytes(seed, length); + // A number keeps under the largest safe integer, which a sixteen-digit one starting with 9 can pass. + const leading = typeof exemplar === "number" && length >= 16 ? 8 : 9; + const digits = [...bytes] + .map((byte, index) => + String(index === 0 ? 1 + (byte % leading) : byte % 10), + ) + .join(""); + return typeof exemplar === "number" ? Number(digits) : digits; +}; + /** A token as long as the recorded one, from its alphabet, keeping its other characters in place. */ export const seededToken = (seed: string, exemplar: string): string => { const alphabet = tokenAlphabet(exemplar); diff --git a/tests/boundary/cli/environmentCompareCli.test.ts b/tests/boundary/cli/environmentCompareCli.test.ts index a965ca12d3..b7776f3ac9 100644 --- a/tests/boundary/cli/environmentCompareCli.test.ts +++ b/tests/boundary/cli/environmentCompareCli.test.ts @@ -10,6 +10,11 @@ import { type TimedCaptureRow, } from "../../fixtures/analysisView.js"; import { environmentStateServeFixture } from "../../fixtures/environmentServe.js"; +import { + BOOKMARK_ROWS, + bookmark, + TAG_ROWS, +} from "../../fixtures/environmentStateDerivedValues.js"; import { BOOK_NOT_FOUND, BOOK_ROWS, @@ -349,6 +354,91 @@ describe("rea environment-compare value context", () => { }); describe("rea environment-compare recorded clock spelling", () => { + for (const scenario of [ + { + name: "variable fractional widths under a confirmed trimmed write plan", + rows: TAG_ROWS, + probe: [ + [ + "POST", + "/api/tags", + { + id: numberedUuid(20), + name: "Spare", + createdAt: "2026-10-20T10:00:00.12345678Z", + updatedAt: "2026-10-20T10:00:00.12345678Z", + }, + 201, + { name: "Spare" }, + ], + ] as const, + generated: 0, + }, + { + name: "a composed bookmark URL through its creation clock", + rows: BOOKMARK_ROWS, + probe: [ + [ + "POST", + "/api/bookmarks", + bookmark( + 20, + "https://pier.example.test/inspection", + "Pier", + "2026-10-20T10:00:00Z", + ), + 201, + { url: "https://pier.example.test/inspection", title: "Pier" }, + ], + ] as const, + generated: 1, + }, + ]) { + cliTest(`accepts ${scenario.name}`, async ({ cli }) => { + const root = await createTestTempDirectory( + "rea-environment-derived-compare-", + ); + const capture = timedCaptureEvidence(scenario.rows); + const fixture = environmentStateServeFixture(capture); + const probe = timedCaptureEvidence(scenario.probe); + const originals = new Map(); + for (const [name, value] of [ + ["manifest.json", fixture.manifest], + ["capture.json", capture], + ["probe.json", probe], + ] as const) { + const path = join(root, name); + const text = JSON.stringify(value); + originals.set(path, text); + await writeFile(path, text); + } + const result = await cli.run({ + arguments: [ + "environment-compare", + join(root, "manifest.json"), + "--evidence", + join(root, "capture.json"), + "--probe", + join(root, "probe.json"), + "--json", + ], + }); + expect(result.exitCode, result.stderr).toBe(0); + expect(result.json).toMatchObject({ + status: "complete", + summary: { probes: 1, steps: 1, equal: 1, agreement: 1 }, + probes: [ + { + normalised: { clocks: 2, generated: scenario.generated }, + results: [{ method: "POST", equal: true, differences: [] }], + }, + ], + }); + for (const [path, text] of originals) + expect(await readFile(path, "utf8")).toBe(text); + }); + } + cliTest( "replays compact-offset submillisecond probe timestamps in exact order", async ({ cli }) => { diff --git a/tests/fixtures/environmentStateDerivedValues.ts b/tests/fixtures/environmentStateDerivedValues.ts new file mode 100644 index 0000000000..dd18c8fab8 --- /dev/null +++ b/tests/fixtures/environmentStateDerivedValues.ts @@ -0,0 +1,484 @@ +import type { JsonValue } from "../../src/domain/jsonValue.js"; +import type { TimedCaptureRow } from "./analysisView.js"; +import { bookClock, numberedUuid } from "./environmentStateWrites.js"; + +type Entity = { [field: string]: JsonValue }; + +/** The instant `seconds` after the first exchange with nanosecond digits, trailing zeros trimmed as Go spells them. */ +export const goClock = (seconds: number, nanos: string): string => + bookClock(seconds).replace("Z", `.${nanos}`.replace(/\.?0+$/u, "") + "Z"); + +const tag = (n: number, name: string, created: string, updated: string) => ({ + id: numberedUuid(n), + name, + createdAt: created, + updatedAt: updated, +}); + +const FIRST_TAG = tag( + 1, + "Fragile", + "2026-10-01T09:00:00.123456789Z", + "2026-10-01T09:00:00.123456791Z", +); +const SECOND_TAG = tag( + 2, + "Outdoor", + "2026-10-01T10:00:00.223456789Z", + "2026-10-01T10:00:00.223456791Z", +); +/** The first created tag's clocks show nine digits; the second's update clock ends in a zero Go drops. */ +const HEAVY_TAG = tag( + 3, + "Heavy", + goClock(3, "374763655"), + goClock(3, "374763767"), +); +const BORROWED_TAG = tag( + 4, + "Borrowed", + goClock(5, "388122047"), + goClock(5, "388122180"), +); + +/** Two tag creates whose clocks Go spells with trailing fraction zeros trimmed, confirmed by later reads. */ +export const TAG_ROWS: readonly TimedCaptureRow[] = [ + ["GET", "/api/tags", [FIRST_TAG, SECOND_TAG]], + ["GET", `/api/tags/${numberedUuid(1)}`, FIRST_TAG], + ["GET", `/api/tags/${numberedUuid(2)}`, SECOND_TAG], + ["POST", "/api/tags", HEAVY_TAG, 201, { name: "Heavy" }], + ["GET", `/api/tags/${numberedUuid(3)}`, HEAVY_TAG], + ["POST", "/api/tags", BORROWED_TAG, 201, { name: "Borrowed" }], + ["GET", `/api/tags/${numberedUuid(4)}`, BORROWED_TAG], + ["GET", "/api/tags", [FIRST_TAG, SECOND_TAG, HEAVY_TAG, BORROWED_TAG]], +]; + +/** A later tag create whose update clock Go trims while its creation clock keeps a trailing zero. */ +const FIXED_CREATED_TAG = tag( + 4, + "Borrowed", + bookClock(5).replace("Z", ".388122040Z"), + goClock(5, "388122180"), +); + +/** Tag creates whose later call trims one clock and shows its sibling at a fixed width. */ +export const FIXED_SIBLING_TAG_ROWS: readonly TimedCaptureRow[] = [ + ["GET", "/api/tags", [FIRST_TAG, SECOND_TAG]], + ["GET", `/api/tags/${numberedUuid(1)}`, FIRST_TAG], + ["GET", `/api/tags/${numberedUuid(2)}`, SECOND_TAG], + ["POST", "/api/tags", HEAVY_TAG, 201, { name: "Heavy" }], + ["GET", `/api/tags/${numberedUuid(3)}`, HEAVY_TAG], + ["POST", "/api/tags", FIXED_CREATED_TAG, 201, { name: "Borrowed" }], + ["GET", `/api/tags/${numberedUuid(4)}`, FIXED_CREATED_TAG], + ["GET", "/api/tags", [FIRST_TAG, SECOND_TAG, HEAVY_TAG, FIXED_CREATED_TAG]], +]; + +/** A clock's UTC date and time digits, as the Wayback Machine names a snapshot. */ +const archiveDigits = (clock: string): string => + clock.replace(/\D/gu, "").slice(0, 14); + +/** A bookmark whose snapshot address names its creation time and its URL. */ +export const bookmark = ( + id: number, + url: string, + title: string, + added: string, +): Entity => ({ + id, + url, + title, + web_archive_snapshot_url: `https://web.archive.org/web/${archiveDigits(added)}/${url}`, + date_added: added, + date_modified: added, +}); + +const FIRST_BOOKMARK = bookmark( + 1, + "https://ferries.example.test/timetable", + "Ferry timetable", + "2026-10-01T09:00:00Z", +); +const SECOND_BOOKMARK = bookmark( + 2, + "https://harbour.example.test/tides", + "Harbour tides", + "2026-10-01T10:00:00Z", +); +const THIRD_BOOKMARK = bookmark( + 3, + "https://quarry.example.test/ledger", + "Quarry ledger", + bookClock(3), +); +const FOURTH_BOOKMARK = bookmark( + 4, + "https://kitchen.example.test/menus", + "Kitchen menus", + bookClock(5), +); + +/** Two bookmark creates whose snapshot addresses the application composes, confirmed by later reads. */ +export const BOOKMARK_ROWS: readonly TimedCaptureRow[] = [ + ["GET", "/api/bookmarks", [FIRST_BOOKMARK, SECOND_BOOKMARK]], + ["GET", "/api/bookmarks/1", FIRST_BOOKMARK], + ["GET", "/api/bookmarks/2", SECOND_BOOKMARK], + [ + "POST", + "/api/bookmarks", + THIRD_BOOKMARK, + 201, + { url: "https://quarry.example.test/ledger", title: "Quarry ledger" }, + ], + ["GET", "/api/bookmarks/3", THIRD_BOOKMARK], + [ + "POST", + "/api/bookmarks", + FOURTH_BOOKMARK, + 201, + { url: "https://kitchen.example.test/menus", title: "Kitchen menus" }, + ], + ["GET", "/api/bookmarks/4", FOURTH_BOOKMARK], + [ + "GET", + "/api/bookmarks", + [FIRST_BOOKMARK, SECOND_BOOKMARK, THIRD_BOOKMARK, FOURTH_BOOKMARK], + ], +]; + +const slugOf = (title: string): string => + title + .toLowerCase() + .replace(/[^a-z0-9]+/gu, "-") + .replace(/^-|-$/gu, ""); + +/** A folder named by the uid its creator chose, with an address from that uid and its title, and a time-based numeric id. */ +export const folder = ( + uid: string, + title: string, + id: number, + created: string, + updated = created, +): Entity => ({ + id, + uid, + title, + url: `/dashboards/f/${uid}/${slugOf(title)}`, + created, + updated, +}); + +const SEED_FOLDER = folder( + numberedUuid(101), + "Lighthouse Rota", + 1655476044521472, + "2026-10-01T09:00:00Z", +); +const ALPHA = folder( + numberedUuid(102), + "Scratch Alpha Harbor", + 1656166527623168, + bookClock(2), +); +const BRAVO = folder( + numberedUuid(103), + "Scratch Bravo Meadow", + 1656166745726976, + bookClock(4), +); +/** The first created folder after the recorded rename at exchange 6. */ +const RENAMED_ALPHA = folder( + numberedUuid(102), + "Renamed Alpha Lantern", + 1656166527623168, + bookClock(2), + bookClock(6), +); + +/** Two folder creates and a rename, whose addresses follow uid and title, confirmed by later reads. */ +export const FOLDER_ROWS: readonly TimedCaptureRow[] = [ + ["GET", "/api/folders", [SEED_FOLDER]], + ["GET", `/api/folders/${numberedUuid(101)}`, SEED_FOLDER], + [ + "POST", + "/api/folders", + ALPHA, + 200, + { uid: numberedUuid(102), title: "Scratch Alpha Harbor" }, + ], + ["GET", `/api/folders/${numberedUuid(102)}`, ALPHA], + [ + "POST", + "/api/folders", + BRAVO, + 200, + { uid: numberedUuid(103), title: "Scratch Bravo Meadow" }, + ], + ["GET", `/api/folders/${numberedUuid(103)}`, BRAVO], + [ + "PUT", + `/api/folders/${numberedUuid(102)}`, + RENAMED_ALPHA, + 200, + { title: "Renamed Alpha Lantern" }, + ], + ["GET", `/api/folders/${numberedUuid(102)}`, RENAMED_ALPHA], + ["GET", "/api/folders", [SEED_FOLDER, RENAMED_ALPHA, BRAVO]], +]; + +/** Two folder renames, the first leaving its address as it was unless `rebuilt`, which another tool then rebuilds. */ +export const restoredAddressRows = (rebuilt: boolean): TimedCaptureRow[] => { + const held = "2026-10-01T09:00:00Z"; + const alpha = folder( + numberedUuid(102), + "Scratch Alpha Harbor", + 1656166527623168, + held, + ); + const bravo = folder( + numberedUuid(103), + "Scratch Bravo Meadow", + 1656166745726976, + held, + ); + const renamed = folder( + numberedUuid(102), + "Renamed Alpha Lantern", + 1656166527623168, + held, + bookClock(3), + ); + const first = rebuilt ? renamed : { ...renamed, url: alpha.url ?? null }; + const second = folder( + numberedUuid(103), + "Second Bravo Lantern", + 1656166745726976, + held, + bookClock(7), + ); + return [ + ["GET", "/api/folders", [alpha, bravo]], + ["GET", `/api/folders/${numberedUuid(102)}`, alpha], + ["GET", `/api/folders/${numberedUuid(103)}`, bravo], + [ + "PUT", + `/api/folders/${numberedUuid(102)}`, + first, + 200, + { title: "Renamed Alpha Lantern" }, + ], + ["GET", `/api/folders/${numberedUuid(102)}`, first], + [ + "POST", + `/api/folders/${numberedUuid(102)}/address`, + renamed, + 200, + { url: renamed.url ?? null }, + ], + ["GET", `/api/folders/${numberedUuid(102)}`, renamed], + [ + "PUT", + `/api/folders/${numberedUuid(103)}`, + second, + 200, + { title: "Second Bravo Lantern" }, + ], + ["GET", `/api/folders/${numberedUuid(103)}`, second], + ]; +}; + +const RENAMED_FIRST_TAG = { + ...FIRST_TAG, + name: "Fragile glass", + updatedAt: goClock(3, "512345679"), +}; +const RENAMED_SECOND_TAG = { + ...SECOND_TAG, + name: "Outdoor gear", + updatedAt: goClock(5, "612345680"), +}; + +/** Two tag renames whose update clocks Go spells with nine and then eight fraction digits. */ +export const TAG_UPDATE_ROWS: readonly TimedCaptureRow[] = [ + ["GET", "/api/tags", [FIRST_TAG, SECOND_TAG]], + ["GET", `/api/tags/${numberedUuid(1)}`, FIRST_TAG], + ["GET", `/api/tags/${numberedUuid(2)}`, SECOND_TAG], + [ + "PATCH", + `/api/tags/${numberedUuid(1)}`, + RENAMED_FIRST_TAG, + 200, + { name: "Fragile glass" }, + ], + ["GET", `/api/tags/${numberedUuid(1)}`, RENAMED_FIRST_TAG], + [ + "PATCH", + `/api/tags/${numberedUuid(2)}`, + RENAMED_SECOND_TAG, + 200, + { name: "Outdoor gear" }, + ], + ["GET", `/api/tags/${numberedUuid(2)}`, RENAMED_SECOND_TAG], +]; + +/** The same renames with an archive the gate does not model before the second, which leaves that tag unknown. */ +export const UNKNOWN_TAG_UPDATE_ROWS: readonly TimedCaptureRow[] = [ + ...TAG_UPDATE_ROWS.slice(0, 5), + ["POST", `/api/tags/${numberedUuid(2)}/archive`, { archived: true }, 200, {}], + ...TAG_UPDATE_ROWS.slice(5), +]; + +const synced = (entity: Entity, syncedAt: string): Entity => ({ + ...entity, + syncedAt, +}); + +/** The same renames stamping a second clock, which the later call still shows with nine digits. */ +export const TWO_CLOCK_TAG_UPDATE_ROWS: readonly TimedCaptureRow[] = [ + [ + "GET", + "/api/tags", + [ + synced(FIRST_TAG, "2026-10-01T09:00:00.123456793Z"), + synced(SECOND_TAG, "2026-10-01T10:00:00.223456793Z"), + ], + ], + [ + "GET", + `/api/tags/${numberedUuid(1)}`, + synced(FIRST_TAG, "2026-10-01T09:00:00.123456793Z"), + ], + [ + "GET", + `/api/tags/${numberedUuid(2)}`, + synced(SECOND_TAG, "2026-10-01T10:00:00.223456793Z"), + ], + [ + "PATCH", + `/api/tags/${numberedUuid(1)}`, + synced(RENAMED_FIRST_TAG, goClock(3, "512345681")), + 200, + { name: "Fragile glass" }, + ], + [ + "GET", + `/api/tags/${numberedUuid(1)}`, + synced(RENAMED_FIRST_TAG, goClock(3, "512345681")), + ], + [ + "PATCH", + `/api/tags/${numberedUuid(2)}`, + synced(RENAMED_SECOND_TAG, goClock(5, "612345683")), + 200, + { name: "Outdoor gear" }, + ], + [ + "GET", + `/api/tags/${numberedUuid(2)}`, + synced(RENAMED_SECOND_TAG, goClock(5, "612345683")), + ], +]; + +/** A clock `seconds` after the first exchange with the fraction digits as given. */ +const clockAt = (seconds: number, fraction: string): string => + bookClock(seconds).replace("Z", `.${fraction}Z`); + +/** The same renames whose later call trims the update clock and shows the synced clock's trailing zero. */ +export const MIXED_WIDTH_TAG_UPDATE_ROWS: readonly TimedCaptureRow[] = [ + ...TWO_CLOCK_TAG_UPDATE_ROWS.slice(0, 5), + [ + "PATCH", + `/api/tags/${numberedUuid(2)}`, + synced(RENAMED_SECOND_TAG, clockAt(5, "612345680")), + 200, + { name: "Outdoor gear" }, + ], + [ + "GET", + `/api/tags/${numberedUuid(2)}`, + synced(RENAMED_SECOND_TAG, clockAt(5, "612345680")), + ], +]; + +const SPARE_TAG = tag( + 5, + "Spare", + goClock(7, "412345671"), + goClock(7, "412345683"), +); + +/** The same creates and a third whose creation clock shows nine digits without a trailing zero. */ +export const LATER_FIXED_SIBLING_TAG_ROWS: readonly TimedCaptureRow[] = [ + ...FIXED_SIBLING_TAG_ROWS.slice(0, 7), + ["POST", "/api/tags", SPARE_TAG, 201, { name: "Spare" }], + ["GET", `/api/tags/${numberedUuid(5)}`, SPARE_TAG], + [ + "GET", + "/api/tags", + [FIRST_TAG, SECOND_TAG, HEAVY_TAG, FIXED_CREATED_TAG, SPARE_TAG], + ], +]; + +/** A held tag whose creation clock keeps a trailing zero while its update clock shows eight digits. */ +const FIXED_HELD_TAG = tag( + 1, + "Fragile", + "2026-10-01T09:00:00.123456780Z", + "2026-10-01T09:00:00.12345679Z", +); + +/** One tag create among held tags that show the update clock trimmed and the creation clock at a fixed width. */ +export const FIXED_HELD_TAG_ROWS: readonly TimedCaptureRow[] = [ + ["GET", "/api/tags", [FIXED_HELD_TAG, SECOND_TAG]], + ["GET", `/api/tags/${numberedUuid(1)}`, FIXED_HELD_TAG], + ["GET", `/api/tags/${numberedUuid(2)}`, SECOND_TAG], + ["POST", "/api/tags", HEAVY_TAG, 201, { name: "Heavy" }], + ["GET", `/api/tags/${numberedUuid(3)}`, HEAVY_TAG], + ["GET", "/api/tags", [FIXED_HELD_TAG, SECOND_TAG, HEAVY_TAG]], +]; + +const GLASS_TAG = { + ...FIRST_TAG, + name: "Fragile glass", + updatedAt: clockAt(3, "123456789"), +}; +const GEAR_TAG = { + ...SECOND_TAG, + name: "Outdoor gear", + updatedAt: clockAt(5, "123456780"), +}; +const GLASSWARE_TAG = { + ...GLASS_TAG, + name: "Fragile glassware", + updatedAt: clockAt(7, "12345678"), +}; + +/** Three tag renames whose update clocks show nine digits, nine ending in a zero, then eight. */ +export const MIDDLE_FIXED_TAG_UPDATE_ROWS: readonly TimedCaptureRow[] = [ + ["GET", "/api/tags", [FIRST_TAG, SECOND_TAG]], + ["GET", `/api/tags/${numberedUuid(1)}`, FIRST_TAG], + ["GET", `/api/tags/${numberedUuid(2)}`, SECOND_TAG], + [ + "PATCH", + `/api/tags/${numberedUuid(1)}`, + GLASS_TAG, + 200, + { name: "Fragile glass" }, + ], + ["GET", `/api/tags/${numberedUuid(1)}`, GLASS_TAG], + [ + "PATCH", + `/api/tags/${numberedUuid(2)}`, + GEAR_TAG, + 200, + { name: "Outdoor gear" }, + ], + ["GET", `/api/tags/${numberedUuid(2)}`, GEAR_TAG], + [ + "PATCH", + `/api/tags/${numberedUuid(1)}`, + GLASSWARE_TAG, + 200, + { name: "Fragile glassware" }, + ], + ["GET", `/api/tags/${numberedUuid(1)}`, GLASSWARE_TAG], +];