mirror of
https://github.com/morluto/rea
synced 2026-10-25 22:08:11 -04:00
aba487eec0
* feat(environment): explain values an application derives from other fields
Prompt-built environments withheld creates whose responses carry values the
application derives deterministically, which the write model kept as
constants or could not match across calls:
- A composed string is literal text plus parts read from the fields other
rules write: a field's value, its slug, or a clock's calendar digits. A
create composes a string only when another entity of the type spells it
the same way and none spells it otherwise, and never from two sources
serving could tell apart; an update composes an unsent change from members
stored as sent and unchanged fields that also spell its value before the
call. A composed field no longer pins the members it reads.
- A trimmed clock format drops trailing fraction zeros, as Go spells
nanoseconds. It is learned from the type's values, a later create or a
later update, and spreads to the plan's other clocks unless a value shows
a fixed width.
- A large integer no entity of the type holds, among distinct ones of the
same length and kind, is a generated unique value, served as a seeded
integer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): record the derived-values verification
Gates, tests, write plans over 69 recorded captures and served write tasks on the baseline and on this branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(environment): learn a create plan from every call
A later create the plan does not take is explained on its own and merged when both calls show the same fields, every field both rule they rule alike, and a field neither rules holds one value: members either call sent are accepted, only those every call sent are required, and an echoed member a call left out takes the value that call stored, kept in the body's defaults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(environment): sorted arrays, referenced entities and nested view keys
- An echo is `sorted` when the application stored the sent strings in
code-point order; serving stores them sorted, and a plain echo and a
sorted one agree in update and create plans.
- A `reference` is an object field holding fields of the entity a sent
member names, as the state holds it. The member is accepted with any
value; replay and serving look the entity up, and serving refuses a call
naming one the environment does not hold.
- A view names in `nested_fields` the keys it always shows of objects it
shows with fewer keys than an entity stores; a stored object it never
recorded renders with them. The view spelling learner moves to its own
module.
- Every later create, not only one the plan refuses, is merged into its
plan when it agrees, keeping the plan's rule order.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): keep clock spellings, linked ids and word boundaries honest
- A later update's clocks take the plan's spelling, sibling-trimmed
clocks included, wherever the call's values fit it, so an update whose
later call trims only one of two clocks is served.
- A held value that ends its fraction in a zero keeps a create's clock
at a fixed width, whatever order the held spellings come in.
- A large integer that another field of the created entity also holds
is linked, as a repeated token is: it is never unique, and the create
is withheld as unexplained-change.
- Composed parts are delimited by whole characters through the shared
joinsBefore/joinsAfter, so an astral letter joins a match to its word.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(environment): keep a same-width fixed clock out of trimming
A held .123456780 beside a held .12345678 keeps the create's clock at
nine digits in either order; the cases fail on 8ce1127b5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): record the create-calls verification
Gates, tests, write plans over 69 recorded captures and served write tasks on the derived-values baseline and on this branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): note the create-calls census on current main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): check composed parts and clock fits before they shape a plan
- A composed string whose part has nothing to spell refuses the create
or update through the apply Result instead of keeping a stale value.
- A composition learned on a later call is rebuilt from each earlier
call's fields, so an earlier call that left the value otherwise
contradicts it.
- An update adopts its fitted clock spellings only after the
unknown-entity and contradiction checks, so a call on an entity an
unmodelled write left unknown no longer trims a served plan's clock.
- Sibling trimming on a later create also checks that call's fields,
so a sibling clock it shows with a trailing zero stays fixed-width.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(environment): split the derived-values view tests by rule
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): keep fixed-width, zone and generated-integer evidence through a plan
- A plan keeps the first value that showed each clock at a fixed width,
from held entities and every accepted create or update. No later fit
or sibling trim widens that field, and a later call that shows it
shorter is withheld.
- Two stamped clocks are one composed source only when they share a
zone offset. A held entity that holds the target field as anything
the parts do not spell, a non-string included, vetoes the
composition. The token index is built once, and only when a string is
left to compose.
- A generated integer that another field repeats as a number at any
depth is linked. Compare treats two integers of one JSON kind and
length under a unique rule as generated values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): keep optional references across create calls and in answers
- A reference a later create added stays while further creates leave its
member out, and agrees with a later call that shows it again.
- A create's answer shows optional reference and echo fields only when the
call wrote them, whichever recorded call the answer came from.
- An update that re-sends the id of the entity a field already repeats keeps
the plan's reference instead of pinning the member.
- An id naming entities of one type in several collections teaches no
reference, and serving or replay refuses a call that names one.
- An echo is not promoted to sorted once a call stored its strings out of
code-point order as sent.
- Nested view keys compare as sets and keep the first recording's order.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(setup): share registration prewarm host preparation
* fix(environment): retain write evidence and match trimmed response clocks
* fix(environment): withhold creates whose generated integer another string repeats
A unique integer whose digits show in another top-level string that no
template or composed rule rebuilds from it (an echoed serialText, or a
token like folder-<id>) withholds the create as unexplained-change,
since serving would seed the integer and keep the string. A templated
copy is still served.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): respect earlier clock widths and left-out members in later creates
- The create merge widens a clock to trimmed only against the value that
showed its width: a held entity or accepted call at a fixed width keeps it
untrimmed.
- A later create that left out a member the plan fills, with a default or a
reference's absence, while its observation shows that field otherwise is
unexplained-change, so the plan cannot serve the stale value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): hold later creates and compare to the composed and integer forms
- A later create whose response spells a composed field otherwise than
the plan rebuilds it is withheld as unexplained-change.
- A unique field learned from a generated integer accepts only an
integer of the same length and kind from a later create, as serving
seeds one.
- Compare counts a composed field as equal when each side's response
spells it from its own fields, which compare under their own rules.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): preserve composed parts and unique generator families
* fix(environment): reject contradictory create references
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
747 lines
27 KiB
TypeScript
747 lines
27 KiB
TypeScript
import type { ListQuery } from "./environmentStateListQuery.js";
|
|
import { canonicalJson } from "../comparisonSemantics.js";
|
|
import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
|
|
import type { JsonValue } from "../jsonValue.js";
|
|
import type { StateWritePlan } from "./environmentStateWriteRules.js";
|
|
import {
|
|
respellClock,
|
|
type ClockFormat,
|
|
} from "./environmentStateWriteValues.js";
|
|
import { ENTRY_BYTES, textBytes } from "./webNetworkRouteTemplates.js";
|
|
import { counterAt, withCounter } from "./environmentStateCounters.js";
|
|
|
|
/** How records are ordered: by recorded start time, or by record order when any start time is missing. */
|
|
export type StateTimeline = "recorded-start-time" | "record-order";
|
|
|
|
/** What extraction reads of one primary-origin record; times are timeline coordinates. */
|
|
export interface StateRecordFacts {
|
|
readonly ordinal: number;
|
|
readonly method: string;
|
|
readonly path: string;
|
|
/** WHATWG query spelling including `?`, or undefined when the request had none. */
|
|
readonly query: string | undefined;
|
|
readonly status: number | null;
|
|
readonly start: number;
|
|
readonly end: number;
|
|
}
|
|
|
|
/** The manifest tool facts extraction uses. */
|
|
export interface StateToolFacts {
|
|
readonly name: string;
|
|
readonly method: string;
|
|
readonly path_template: string;
|
|
readonly record_ordinals: readonly number[];
|
|
}
|
|
|
|
/** A record's response body as JSON, or unusable when absent, cut, not JSON, not exactly representable or carrying a `__proto__` member. */
|
|
export type ParsedBody =
|
|
| { readonly kind: "json"; readonly value: JsonValue }
|
|
| { readonly kind: "unusable" };
|
|
|
|
/** A request distinguishes confirmed absence from content the state cannot interpret. */
|
|
export type ParsedRequestBody = ParsedBody | { readonly kind: "absent" };
|
|
|
|
export const requestJson = (body: ParsedRequestBody): JsonValue | undefined =>
|
|
body.kind === "json" ? body.value : undefined;
|
|
|
|
/** Budgeted access the accumulator lends extraction; null and false mean the budget is exhausted. */
|
|
export interface StateBudget {
|
|
readonly body: (ordinal: number) => ParsedBody | null;
|
|
/** Keep the request's parsing reservation until its consumer finishes. */
|
|
readonly withRequestBody: <T>(
|
|
ordinal: number,
|
|
use: (body: ParsedRequestBody) => T,
|
|
) => T | null;
|
|
/** Whether a record's response had no body. */
|
|
readonly emptyResponse: (ordinal: number) => boolean;
|
|
readonly retain: (bytes: number) => boolean;
|
|
/** Mark the retained bytes; the returned call releases what was retained since, once none of it is held. */
|
|
readonly checkpoint: () => () => void;
|
|
/** Drop the temporary work of the body just read. */
|
|
readonly release: () => void;
|
|
}
|
|
|
|
export type RecordedViewReason =
|
|
| "query-body"
|
|
| "no-init-recording"
|
|
| "body-unusable"
|
|
| "no-identity"
|
|
| "path-varies"
|
|
| "query-varies"
|
|
| "fields-differ"
|
|
| "envelope-varies"
|
|
| "list-differs"
|
|
| "rendered-oversize";
|
|
|
|
/**
|
|
* Recorded response text, and a document value rendered back to JSON, stay within this many UTF-8
|
|
* bytes. A larger recorded body is not parsed; a parsed value whose JSON rendering exceeds it stays
|
|
* a recorded replay (`rendered-oversize`) rather than a document the server would have to refuse.
|
|
*/
|
|
export const STATE_BODY_LIMIT_BYTES = 256 * 1024;
|
|
|
|
export interface StateType {
|
|
readonly collection_template: string;
|
|
readonly id_field: string;
|
|
readonly id_basis: "path-parameter" | "field-name";
|
|
readonly init_before_ordinal: number | null;
|
|
readonly volatile_fields: string[];
|
|
}
|
|
|
|
export type JsonObject = Record<string, JsonValue>;
|
|
|
|
export interface StateEntity {
|
|
readonly type: string;
|
|
readonly collection: string;
|
|
readonly id: string;
|
|
readonly fields: JsonObject;
|
|
readonly view_fields: {
|
|
readonly tool: string;
|
|
readonly fields: JsonObject;
|
|
}[];
|
|
readonly record_ordinals: number[];
|
|
}
|
|
|
|
export interface StateDocument {
|
|
readonly tool: string;
|
|
readonly path: string;
|
|
readonly value: JsonValue;
|
|
readonly record_ordinal: number;
|
|
}
|
|
|
|
interface EntityViewPlan {
|
|
readonly tool: string;
|
|
readonly type: string;
|
|
readonly pointer: string;
|
|
readonly fields: string[];
|
|
readonly required_fields: string[];
|
|
readonly checked_record_ordinals: number[];
|
|
/** How the view spells the clock fields it always spells otherwise than the entities' stored values. */
|
|
readonly clock_formats?: Readonly<Record<string, ClockFormat>> | undefined;
|
|
/** Fields whose string arrays the view lists in code-point order, whatever order they were written in. */
|
|
readonly sorted_arrays?: string[] | undefined;
|
|
/** The keys the view shows of object fields it shows with fewer keys than stored. */
|
|
readonly nested_fields?: Readonly<Record<string, string[]>> | undefined;
|
|
/** Envelope members each response there mints, which checks leave out and serving keeps as recorded. */
|
|
readonly per_response_members?: string[] | undefined;
|
|
}
|
|
|
|
/** A query key that picks out replayed reads: a parameter, a parameter with one value, or a parameter left out. */
|
|
export interface ReplayKey {
|
|
readonly name: string;
|
|
readonly value?: string | undefined;
|
|
readonly absent?: true | undefined;
|
|
}
|
|
|
|
/** Recorded reads a view leaves to replay, and the query keys that pick calls of them out. */
|
|
interface ReplayedReads {
|
|
/** Reads that asked for some fields only, or show entities in another representation than the checked lines. */
|
|
readonly projected_record_ordinals?: number[] | undefined;
|
|
/** A call whose query matches one of these replays, a field selection by its values; absent on views that only left out projections. */
|
|
readonly projected_query_keys?: ReplayKey[] | undefined;
|
|
}
|
|
|
|
export type StateItemView = EntityViewPlan &
|
|
ReplayedReads & {
|
|
readonly shape: "item";
|
|
readonly envelope_ordinal: number;
|
|
};
|
|
|
|
/** A query parameter that asks for some fields only (`fields`, `fields[...]`, `select`, `$select`). */
|
|
export const isProjectionParameter = (name: string): boolean => {
|
|
const lower = name.toLowerCase();
|
|
return (
|
|
lower === "fields" ||
|
|
lower.startsWith("fields[") ||
|
|
lower === "select" ||
|
|
lower === "$select"
|
|
);
|
|
};
|
|
|
|
/** Whether a recorded query string names a projection parameter. */
|
|
export const projectsFields = (query: string | undefined): boolean =>
|
|
query !== undefined &&
|
|
[...new URLSearchParams(query).keys()].some(isProjectionParameter);
|
|
|
|
/** Whether a recorded query asks for some fields only: a projection parameter with a value other than `*`, which asks for all. */
|
|
export const selectsSomeFields = (query: string | undefined): boolean =>
|
|
query !== undefined &&
|
|
[...new URLSearchParams(query)].some(
|
|
([name, value]) => isProjectionParameter(name) && value !== "*",
|
|
);
|
|
|
|
export type StateListView = EntityViewPlan &
|
|
ReplayedReads & {
|
|
readonly shape: "list";
|
|
readonly membership: "all" | "recorded";
|
|
/** With recorded membership, the checked lines that each list their whole collection. */
|
|
readonly all_record_ordinals?: number[] | undefined;
|
|
readonly counters: string[];
|
|
/** The query the view's other lines follow, when one explains any of them. */
|
|
readonly query?: ListQuery | undefined;
|
|
};
|
|
|
|
/** Whether a list view answers on a checked line with every entity of the collection. */
|
|
export const listsAll = (view: StateListView, ordinal: number): boolean =>
|
|
view.membership === "all" ||
|
|
(view.all_record_ordinals?.includes(ordinal) ?? false);
|
|
|
|
/** The checked lines a list view answers with every entity of the collection. */
|
|
export const allLines = (view: StateListView): readonly number[] =>
|
|
view.membership === "all"
|
|
? view.checked_record_ordinals
|
|
: (view.all_record_ordinals ?? []);
|
|
|
|
export type StateView =
|
|
| StateItemView
|
|
| StateListView
|
|
| {
|
|
readonly tool: string;
|
|
readonly shape: "document";
|
|
readonly checked_record_ordinals: number[];
|
|
readonly per_response_members?: string[] | undefined;
|
|
}
|
|
| {
|
|
readonly tool: string;
|
|
readonly shape: "recorded";
|
|
readonly reason: RecordedViewReason;
|
|
};
|
|
|
|
export interface ExtractedState {
|
|
readonly types: StateType[];
|
|
readonly entities: StateEntity[];
|
|
readonly documents: StateDocument[];
|
|
readonly views: StateView[];
|
|
readonly writes: StateWritePlan[];
|
|
/** The instant a served clock starts from: the end of the capture's last timed record, or null without times. */
|
|
readonly write_clock: string | null;
|
|
}
|
|
|
|
const WRITE_METHODS = new Set(["POST", "PUT", "PATCH", "DELETE"]);
|
|
/** A POST whose path names one of these segments queries state rather than changing it. */
|
|
export const QUERY_SEGMENTS = new Set([
|
|
"search",
|
|
"query",
|
|
"list",
|
|
"find",
|
|
"fuzzy",
|
|
"lookup",
|
|
"count",
|
|
"aggregate",
|
|
"stats",
|
|
"_search",
|
|
"_count",
|
|
"_msearch",
|
|
]);
|
|
/** Identity field names tried, in order, when no path parameter proves one. */
|
|
export const IDENTITY_NAMES = [
|
|
"id",
|
|
"uid",
|
|
"uuid",
|
|
"_id",
|
|
"ID",
|
|
"UID",
|
|
"Id",
|
|
"guid",
|
|
"key",
|
|
"slug",
|
|
"name",
|
|
];
|
|
const VERSION_SEGMENT = /^v\d+(?:\.\d+)*$/u;
|
|
|
|
export const segmentsOf = (path: string): string[] => {
|
|
const segments = path.split("/");
|
|
if (segments[0] === "") segments.shift();
|
|
if (segments.at(-1) === "") segments.pop();
|
|
return segments;
|
|
};
|
|
export const isParam = (segment: string): boolean =>
|
|
segment.startsWith("{") && segment.endsWith("}");
|
|
export const isObject = (value: JsonValue | undefined): value is JsonObject =>
|
|
typeof value === "object" && value !== null && !Array.isArray(value);
|
|
export const scalarText = (value: JsonValue | undefined): string | undefined =>
|
|
typeof value === "string"
|
|
? value
|
|
: typeof value === "number"
|
|
? String(value)
|
|
: undefined;
|
|
/** Equal JSON values; object member order carries no meaning. */
|
|
export const same = (
|
|
left: JsonValue | undefined,
|
|
right: JsonValue | undefined,
|
|
): boolean =>
|
|
left === undefined || right === undefined
|
|
? left === right
|
|
: canonicalJson(left, "Environment state") ===
|
|
canonicalJson(right, "Environment state");
|
|
|
|
/** A path segment percent-decoded, or as written when it does not decode. */
|
|
export const decodedSegment = (segment: string): string => {
|
|
try {
|
|
return decodeURIComponent(segment);
|
|
} catch {
|
|
return segment;
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Whether two path spellings name one segment: they are equal, or either decodes once to the other.
|
|
* Decoding both would make `book%2531` match `book1`.
|
|
*/
|
|
export const pathSegmentMatches = (segment: string, value: string): boolean =>
|
|
segment === value ||
|
|
decodedSegment(segment) === value ||
|
|
decodedSegment(value) === segment;
|
|
|
|
/** One segment of a call: a template literal, or a supplied path value. */
|
|
export interface SuppliedSegment {
|
|
readonly text: string;
|
|
readonly supplied: boolean;
|
|
}
|
|
|
|
/** How many supplied segments a held collection matches verbatim, when it matches the call under the path rule. */
|
|
export const collectionMatch = (
|
|
collection: string,
|
|
segments: readonly SuppliedSegment[],
|
|
): number | undefined => {
|
|
const held = segmentsOf(collection);
|
|
if (held.length !== segments.length) return undefined;
|
|
let verbatim = 0;
|
|
for (const [index, { text, supplied }] of segments.entries()) {
|
|
const segment = held[index]!;
|
|
if (supplied ? !pathSegmentMatches(segment, text) : segment !== text)
|
|
return undefined;
|
|
if (supplied && segment === text) verbatim += 1;
|
|
}
|
|
return verbatim;
|
|
};
|
|
|
|
/** The collection a create writes: the candidate with the most verbatim supplied segments, the earliest on a tie, else the call with each supplied segment encoded. */
|
|
export const selectedCollection = (
|
|
segments: readonly SuppliedSegment[],
|
|
candidates: readonly string[],
|
|
): string => {
|
|
let chosen: string | undefined;
|
|
let verbatim = -1;
|
|
for (const collection of candidates) {
|
|
const matched = collectionMatch(collection, segments);
|
|
if (matched !== undefined && matched > verbatim) {
|
|
chosen = collection;
|
|
verbatim = matched;
|
|
}
|
|
}
|
|
return (
|
|
chosen ??
|
|
`/${segments
|
|
.map(({ text, supplied }) => (supplied ? encodeURIComponent(text) : text))
|
|
.join("/")}`
|
|
);
|
|
};
|
|
|
|
/** The decoded final path segment, which an item request names its entity by. */
|
|
export const lastSegmentValue = (path: string): string =>
|
|
decodedSegment(segmentsOf(path).at(-1) ?? "");
|
|
|
|
/** A path's segments, preserving repeated separators; one trailing slash names the same collection. */
|
|
export const normalizedPath = (path: string): string =>
|
|
`/${segmentsOf(path).join("/")}`;
|
|
|
|
/** An item record's collection is its path without the identity segment; a list record's is its path. */
|
|
export const collectionPathOf = (
|
|
record: Pick<StateRecordFacts, "path">,
|
|
item: boolean,
|
|
): string =>
|
|
item
|
|
? `/${segmentsOf(record.path).slice(0, -1).join("/")}`
|
|
: normalizedPath(record.path);
|
|
|
|
/** A POST whose path names a query segment observes state rather than changing it. */
|
|
export const isQueryPost = (method: string, path: string): boolean =>
|
|
method === "POST" &&
|
|
segmentsOf(path).some((segment) => QUERY_SEGMENTS.has(segment.toLowerCase()));
|
|
|
|
/** Whether a request with this method and path may change state: a write method that is not a query POST. */
|
|
export const changesState = (method: string, path: string): boolean =>
|
|
WRITE_METHODS.has(method) && !isQueryPost(method, path);
|
|
|
|
/** Whether a record may change state: a successful or unanswered request that `changesState` names. */
|
|
export const isStateWrite = (record: StateRecordFacts): boolean =>
|
|
changesState(record.method, record.path) &&
|
|
(record.status === null || (record.status >= 200 && record.status < 400));
|
|
|
|
/** Whether any write overlaps the span from one record's start to a later record's end; writes come in timeline order. */
|
|
export const writeSpanCheck = (
|
|
writes: readonly StateRecordFacts[],
|
|
): ((earlier: StateRecordFacts, later: StateRecordFacts) => boolean) => {
|
|
// The latest end among each prefix of writes, so one search answers a span.
|
|
const reach: number[] = [];
|
|
for (const write of writes)
|
|
reach.push(Math.max(reach.at(-1) ?? Number.NEGATIVE_INFINITY, write.end));
|
|
return (earlier, later) => {
|
|
let low = 0;
|
|
let high = writes.length;
|
|
while (low < high) {
|
|
const middle = (low + high) >>> 1;
|
|
if (writes[middle]!.start < later.end) low = middle + 1;
|
|
else high = middle;
|
|
}
|
|
return low > 0 && reach[low - 1]! > earlier.start;
|
|
};
|
|
};
|
|
|
|
/** Literal leading segments shared by every template with the same first segment: each path's API base. */
|
|
export const apiBaseLengths = (
|
|
templates: readonly string[],
|
|
): ReadonlyMap<string, number> => {
|
|
const groups = new Map<string, string[][]>();
|
|
for (const template of templates) {
|
|
const segments = segmentsOf(template);
|
|
const first = segments[0] ?? "";
|
|
const group = groups.get(first);
|
|
if (group === undefined) groups.set(first, [segments]);
|
|
else group.push(segments);
|
|
}
|
|
return new Map(
|
|
[...groups.entries()].map(([first, members]) => {
|
|
const head = members[0] ?? [];
|
|
let length = 0;
|
|
while (
|
|
length < head.length &&
|
|
!isParam(head[length]!) &&
|
|
members.every((segments) => segments[length] === head[length])
|
|
)
|
|
length += 1;
|
|
return [first, length];
|
|
}),
|
|
);
|
|
};
|
|
|
|
/** The path's API base, then any version segments, then one segment: the scope a write may affect. */
|
|
export const rootOf = (
|
|
path: string,
|
|
bases: ReadonlyMap<string, number>,
|
|
): string[] => {
|
|
const segments = segmentsOf(path);
|
|
let end = Math.min(bases.get(segments[0] ?? "") ?? 0, segments.length);
|
|
while (end < segments.length && VERSION_SEGMENT.test(segments[end]!))
|
|
end += 1;
|
|
return segments.slice(0, end + 1);
|
|
};
|
|
|
|
/** Value bytes as retained state, charged before a value is kept. */
|
|
export const stateValueRetentionBytes = (value: JsonValue): number => {
|
|
if (typeof value === "string") return textBytes(value.length);
|
|
if (Array.isArray(value))
|
|
return value.reduce<number>(
|
|
(sum, item) => sum + stateValueRetentionBytes(item),
|
|
ENTRY_BYTES,
|
|
);
|
|
if (isObject(value))
|
|
return Object.entries(value).reduce<number>(
|
|
(sum, [name, item]) =>
|
|
sum + textBytes(name.length) + stateValueRetentionBytes(item),
|
|
ENTRY_BYTES,
|
|
);
|
|
return ENTRY_BYTES;
|
|
};
|
|
|
|
/** Whether a JSON value nests more than `limit` levels below its root. */
|
|
export const deeperThan = (value: JsonValue, limit: number): boolean => {
|
|
const pending: { readonly value: JsonValue; readonly depth: number }[] = [
|
|
{ value, depth: 0 },
|
|
];
|
|
for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
|
|
if (next.depth > limit) return true;
|
|
const children = Array.isArray(next.value)
|
|
? next.value
|
|
: isObject(next.value)
|
|
? Object.values(next.value)
|
|
: [];
|
|
for (const child of children)
|
|
pending.push({ value: child, depth: next.depth + 1 });
|
|
}
|
|
return false;
|
|
};
|
|
|
|
/** JSON parsers keep a `__proto__` member that plain object construction would turn into a prototype. */
|
|
export const hasPrototypeKey = (value: JsonValue): boolean =>
|
|
Array.isArray(value)
|
|
? value.some(hasPrototypeKey)
|
|
: isObject(value) &&
|
|
Object.entries(value).some(
|
|
([name, item]) => name === "__proto__" || hasPrototypeKey(item),
|
|
);
|
|
|
|
export const pointerTo = (name: string): string =>
|
|
`/${name.replaceAll("~", "~0").replaceAll("/", "~1")}`;
|
|
/** The unescaped reference tokens of a pointer. */
|
|
export const pointerTokens = (pointer: string): string[] =>
|
|
pointer === ""
|
|
? []
|
|
: pointer
|
|
.slice(1)
|
|
.split("/")
|
|
.map((token) => token.replaceAll("~1", "/").replaceAll("~0", "~"));
|
|
|
|
const memberOf = (
|
|
body: JsonValue | undefined,
|
|
name: string,
|
|
): JsonValue | undefined =>
|
|
isObject(body) && Object.hasOwn(body, name) ? body[name] : undefined;
|
|
|
|
/** The element of a one-element array. */
|
|
const soleElement = (value: JsonValue | undefined): JsonValue | undefined =>
|
|
Array.isArray(value) && value.length === 1 ? value[0] : undefined;
|
|
|
|
/** The value at a pointer: the root, one member, or element 0 of a one-element array at the root or in one member. */
|
|
export const at = (body: JsonValue, pointer: string): JsonValue | undefined => {
|
|
const [first, second, ...rest] = pointerTokens(pointer);
|
|
if (first === undefined) return body;
|
|
if (second !== undefined)
|
|
return second === "0" && rest.length === 0
|
|
? soleElement(memberOf(body, first))
|
|
: undefined;
|
|
return Array.isArray(body)
|
|
? first === "0"
|
|
? soleElement(body)
|
|
: undefined
|
|
: memberOf(body, first);
|
|
};
|
|
|
|
/** A copy of `body` with the value `at` reads at the pointer replaced; unchanged when it reads none. */
|
|
export const replaceAt = (
|
|
body: JsonValue,
|
|
pointer: string,
|
|
value: JsonValue,
|
|
): JsonValue => {
|
|
const [first, second] = pointerTokens(pointer);
|
|
if (first === undefined) return value;
|
|
if (at(body, pointer) === undefined) return body;
|
|
if (!isObject(body)) return [value];
|
|
const replaced = second === undefined ? value : [value];
|
|
return Object.fromEntries(
|
|
Object.entries(body).map(([key, item]) => [
|
|
key,
|
|
key === first ? replaced : item,
|
|
]),
|
|
);
|
|
};
|
|
|
|
const matchingKeys = (object: JsonObject, value: string): string[] =>
|
|
Object.keys(object).filter((key) => scalarText(object[key]) === value);
|
|
|
|
/** The entity object an item response carries with the keys repeating its path value: at the root or in one wrapper member, else as the element of a one-element array there. */
|
|
export const locateItem = (
|
|
body: JsonValue,
|
|
value: string,
|
|
): { readonly pointer: string; readonly keys: string[] } | undefined => {
|
|
const located = (
|
|
object: JsonValue | undefined,
|
|
pointer: string,
|
|
named = false,
|
|
) => {
|
|
const keys = isObject(object)
|
|
? matchingKeys(object, value).filter(
|
|
(key) => !named || IDENTITY_NAMES.includes(key),
|
|
)
|
|
: [];
|
|
return keys.length > 0 ? [{ pointer, keys }] : [];
|
|
};
|
|
const members = isObject(body) ? Object.entries(body) : [];
|
|
// Objects take precedence; an array element needs an identity name, since a filtered list may
|
|
// answer with one element that repeats its parent's value.
|
|
const tiers = [
|
|
located(body, ""),
|
|
members.flatMap(([name, item]) => located(item, pointerTo(name))),
|
|
located(soleElement(body), "/0", true),
|
|
members.flatMap(([name, item]) =>
|
|
located(soleElement(item), `${pointerTo(name)}/0`, true),
|
|
),
|
|
];
|
|
const chosen = tiers.find((tier) => tier.length > 0);
|
|
return chosen?.length === 1 ? chosen[0] : undefined;
|
|
};
|
|
|
|
/** Whether every item carries `name` as a distinct string or number. */
|
|
export const identifies = (
|
|
items: readonly JsonObject[],
|
|
name: string,
|
|
): boolean => {
|
|
// Stop at the first missing or repeated value, so a sparse list costs one read per field.
|
|
const seen = new Set<string>();
|
|
for (const item of items) {
|
|
const value = scalarText(item[name]);
|
|
if (value === undefined || seen.has(value)) return false;
|
|
seen.add(value);
|
|
}
|
|
return true;
|
|
};
|
|
|
|
/** Fields ending in one of these may identify listed items that no identity name identifies. */
|
|
const IDENTITY_SUFFIX = /(?:Id|_id)$/u;
|
|
|
|
/** Keys of the first item, in its order, that end like an identity field and identify every item. */
|
|
const suffixIdentities = (items: readonly JsonObject[]): string[] =>
|
|
Object.keys(items[0] ?? {}).filter(
|
|
(name) =>
|
|
IDENTITY_SUFFIX.test(name) &&
|
|
!IDENTITY_NAMES.includes(name) &&
|
|
identifies(items, name),
|
|
);
|
|
|
|
/** Fields that identify every listed item, in precedence order: identity names, then suffix-named fields. */
|
|
export const identityCandidates = (items: readonly JsonObject[]): string[] => [
|
|
...IDENTITY_NAMES.filter((name) => identifies(items, name)),
|
|
...suffixIdentities(items),
|
|
];
|
|
|
|
/** The array of entity objects a list response carries, at the root or in one property of an envelope that is not itself an entity. */
|
|
export const locateList = (
|
|
body: JsonValue,
|
|
idField: string | undefined,
|
|
): string | undefined => {
|
|
if (Array.isArray(body)) return body.every(isObject) ? "" : undefined;
|
|
if (
|
|
!isObject(body) ||
|
|
IDENTITY_NAMES.some((name) => scalarText(body[name]) !== undefined)
|
|
)
|
|
return undefined;
|
|
const arrays = Object.entries(body).filter(
|
|
(entry): entry is [string, JsonObject[]] =>
|
|
Array.isArray(entry[1]) && entry[1].every(isObject),
|
|
);
|
|
const carrying = (test: (items: JsonObject[]) => boolean) =>
|
|
arrays.filter(([, items]) => items.length > 0 && test(items));
|
|
// Identity names take precedence over suffix-named fields, which need two items to show
|
|
// distinct values; with neither, the one array of objects.
|
|
const tiers =
|
|
idField === undefined
|
|
? [
|
|
carrying((items) =>
|
|
IDENTITY_NAMES.some((name) => identifies(items, name)),
|
|
),
|
|
carrying(
|
|
(items) => items.length > 1 && suffixIdentities(items).length > 0,
|
|
),
|
|
arrays,
|
|
]
|
|
: [carrying((items) => identifies(items, idField)), arrays];
|
|
const chosen = tiers.find((entries) => entries.length > 0);
|
|
return chosen?.length === 1 ? pointerTo(chosen[0]![0]) : undefined;
|
|
};
|
|
|
|
export const listItems = (
|
|
body: JsonValue,
|
|
pointer: string,
|
|
): JsonObject[] | undefined => {
|
|
const items = at(body, pointer);
|
|
return Array.isArray(items) && items.every(isObject) ? items : undefined;
|
|
};
|
|
|
|
/** An observed item count, retaining the original value separately for rendering. */
|
|
export const recordedItemCount = (
|
|
value: JsonValue | undefined,
|
|
): number | undefined =>
|
|
typeof value === "number"
|
|
? value
|
|
: typeof value === "string" && /^\d+$/u.test(value)
|
|
? Number(value)
|
|
: undefined;
|
|
|
|
/** Spell a changed count with the recorded numeric/string type and digit width. */
|
|
export const formatItemCount = (
|
|
count: number,
|
|
observed: JsonValue,
|
|
): JsonValue =>
|
|
typeof observed === "string"
|
|
? String(count).padStart(observed.length, "0")
|
|
: count;
|
|
|
|
/** One entity as a view shows it: the view's fields it has, view-specific values first; volatile fields take `observed` values when given. */
|
|
export const renderStateEntity = (
|
|
entity: StateEntity,
|
|
view: Pick<
|
|
EntityViewPlan,
|
|
"tool" | "fields" | "clock_formats" | "sorted_arrays" | "nested_fields"
|
|
>,
|
|
volatileFields: ReadonlySet<string>,
|
|
observed?: JsonObject,
|
|
): JsonObject => {
|
|
const own =
|
|
entity.view_fields.find((candidate) => candidate.tool === view.tool)
|
|
?.fields ?? {};
|
|
const formats = view.clock_formats ?? {};
|
|
const nested = view.nested_fields ?? {};
|
|
const sorted = new Set(view.sorted_arrays);
|
|
return Object.fromEntries(
|
|
view.fields.flatMap((name): [string, JsonValue][] => {
|
|
const source =
|
|
observed !== undefined && volatileFields.has(name)
|
|
? observed
|
|
: Object.hasOwn(own, name)
|
|
? own
|
|
: entity.fields;
|
|
if (!Object.hasOwn(source, name)) return [];
|
|
// A stored value the view never recorded, such as a written one, takes the view's spelling and keys.
|
|
const value =
|
|
source !== entity.fields
|
|
? source[name]!
|
|
: Object.hasOwn(formats, name)
|
|
? respellClock(source[name]!, formats[name]!)
|
|
: Object.hasOwn(nested, name)
|
|
? shownKeys(source[name]!, nested[name]!)
|
|
: source[name]!;
|
|
return [[name, sorted.has(name) ? inSetOrder(value) : value]];
|
|
}),
|
|
);
|
|
};
|
|
|
|
/** An object with only the keys a view shows of it, in the view's order; any other value as it is. */
|
|
const shownKeys = (value: JsonValue, keys: readonly string[]): JsonValue =>
|
|
isObject(value)
|
|
? Object.fromEntries(
|
|
keys.flatMap((key) =>
|
|
Object.hasOwn(value, key) ? [[key, value[key]!]] : [],
|
|
),
|
|
)
|
|
: value;
|
|
|
|
/** Whether a value is an array of strings. */
|
|
export const isStringArray = (value: JsonValue): value is string[] =>
|
|
Array.isArray(value) && value.every((item) => typeof item === "string");
|
|
|
|
/** A string array in code-point order, as a view that keeps the field as a set lists it; any other value as it is. */
|
|
const inSetOrder = (value: JsonValue): JsonValue =>
|
|
isStringArray(value) ? value.toSorted(compareUnicodeCodePoints) : value;
|
|
|
|
/** The fields every recording of a view carried that the entity, as serving renders it, lacks; serving refuses to answer with such an entity. */
|
|
export const missingRequiredFields = (
|
|
entity: StateEntity,
|
|
view: Pick<EntityViewPlan, "tool" | "fields" | "required_fields">,
|
|
volatileFields: ReadonlySet<string>,
|
|
): string[] => {
|
|
const shown = renderStateEntity(entity, view, volatileFields);
|
|
return view.required_fields.filter((name) => !Object.hasOwn(shown, name));
|
|
};
|
|
|
|
/**
|
|
* A list response rendered from `base`: its array replaced by `items`, and set to their number each counter that
|
|
* counted the items `base` listed; one that did not, such as a partial page's total, keeps its recorded value.
|
|
*/
|
|
export const renderStateList = (
|
|
view: Pick<StateListView, "pointer" | "counters">,
|
|
base: JsonValue,
|
|
items: readonly JsonObject[],
|
|
): JsonValue => {
|
|
const listed = listItems(base, view.pointer)?.length;
|
|
return view.counters.reduce(
|
|
(body, name) => {
|
|
const observed = counterAt(base, name);
|
|
if (listed === undefined || recordedItemCount(observed) !== listed)
|
|
return body;
|
|
return withCounter(body, name, formatItemCount(items.length, observed!));
|
|
},
|
|
replaceAt(base, view.pointer, [...items]),
|
|
);
|
|
};
|