mirror of
https://github.com/morluto/rea
synced 2026-10-29 07:17:42 -04:00
feat(analysis-view): project rich captures from an Evidence file and template path pairs
inspect_analysis_view accepts a source {kind: "evidence-file", path},
read through the shared JSON file reader, authenticated as complete
Evidence and projected like retained Evidence. A capture is then bounded
by the file reader instead of being charged as inline input at 14 bytes
per string character, which refused a 10.7 MB capture whose response
bodies the environment manifest never reads.
The environment-manifest view gains the opt-in rule sibling-pair-variance:
groups that differ only in two adjacent literals after a plural literal,
such as /repos/{owner}/{repo}/issues, merge when at least three distinct
pairs vary in both positions. Their parameters are named <singular>_namespace
and <singular>_name. A rich Gitea capture otherwise produced one tool per
repository, beyond the 128 tools model providers accept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar Arakelyan
@@ -30,6 +30,13 @@ jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evi
|
||||
rea inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
The request can also name the saved file instead of embedding it:
|
||||
`{"source": {"kind": "evidence-file", "path": "app-evidence.json"}, "view": {"kind": "summary"}}`.
|
||||
The CLI resolves a relative path against the working directory; MCP clients
|
||||
pass an absolute path. REA reads and authenticates the file, then projects it
|
||||
like retained Evidence, so a large file is bounded by the file reader rather
|
||||
than charged as inline input.
|
||||
|
||||
For example, a 334 MB Obsidian application Evidence file yields a summary view
|
||||
of about 10 KB. Replace the view with
|
||||
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
# Plan
|
||||
|
||||
1. Add the `evidence-file` source to `analysisViewSourceSchema`
|
||||
(`src/domain/analysisView/analysisView.ts`).
|
||||
2. Add `inspectAnalysisViewWithFiles` / `inspectAnalysisViewValidatedWithFiles`
|
||||
to `AnalysisViewService`: read with `readJsonFile`, authenticate with
|
||||
`parseEvidence`, then project through the retained path with a lookup that
|
||||
returns the file's Evidence. The synchronous path refuses an unread file
|
||||
source.
|
||||
3. Route the CLI command (with `source.path` resolved against the working
|
||||
directory) and the MCP tool through the file-aware entry point.
|
||||
4. Add `sibling-pair-variance` to `TEMPLATE_RULES`, with its merge pass after
|
||||
`sibling-variance`, its parameter names and its segment-limit limitation.
|
||||
5. Document both in the tool contract, `docs/cli.md` and
|
||||
`docs/web-network-captures.md`.
|
||||
6. Tests: service (file equals inline; a capture refused inline is accepted
|
||||
from a file; refusals), CLI and MCP boundaries, and the pair rule's merges
|
||||
and non-merges.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Rich captures for the environment manifest
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
A capture large enough to hold a rich application state can be projected into
|
||||
an environment manifest whose tool list stays within what model providers
|
||||
accept. Two changes:
|
||||
|
||||
- `inspect_analysis_view` accepts a third source,
|
||||
`{"kind": "evidence-file", "path": "<absolute path>"}`. REA reads one
|
||||
completed Evidence JSON file, authenticates it, and projects it like retained
|
||||
Evidence, so its strings are bounded by the file reader instead of being
|
||||
charged as inline input.
|
||||
- The environment-manifest view gains an opt-in template rule,
|
||||
`sibling-pair-variance`, that templates two adjacent literal segments after a
|
||||
plural literal, such as `/repos/{owner}/{repo}/…`.
|
||||
|
||||
## Requirements
|
||||
|
||||
- An `evidence-file` path must be absolute and free of NUL bytes; the CLI
|
||||
resolves a relative path against the working directory before validation,
|
||||
and MCP clients pass an absolute path.
|
||||
- The file is read through the shared JSON file reader (regular file, heap and
|
||||
string-length guards) and parsed as complete Evidence. A missing or unreadable
|
||||
file is an `EvidenceFileError`; JSON that is not complete Evidence is invalid
|
||||
input reported under `source.path`.
|
||||
- A file source projects exactly what the same Evidence projects inline or
|
||||
retained, with the same parent Evidence ID, view parameters and result.
|
||||
- The synchronous projection API refuses an unread `evidence-file` source with
|
||||
a typed capability error; the CLI and MCP adapters read files first.
|
||||
- `sibling-pair-variance` applies only when listed, after `sibling-variance`. It
|
||||
compares groups equal in method, origin and segment count except at two
|
||||
adjacent literal positions at index 2 or later that follow a plural literal,
|
||||
and merges a bucket only when it holds at least three distinct pairs and both
|
||||
positions vary. Stop words and `v1`-style versions never qualify, and routes
|
||||
of more than 32 segments are not compared, with a limitation saying so.
|
||||
- Its parameters are named from the plural literal, singular plus `_namespace`
|
||||
and then `_name`, and labelled `sibling-pair-variance`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No change to the 96 MiB projection budget or to inline admission.
|
||||
- No change to `sibling-variance`; pairs are a separate rule so every inferred
|
||||
parameter keeps naming the rule that produced it.
|
||||
- No inference of semantic names (`owner`, `repo`) beyond the plural literal.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the outcome, requirements and non-goals.
|
||||
- [x] Add the `evidence-file` source and the file-aware service entry point.
|
||||
- [x] Route the CLI command and the MCP tool through it.
|
||||
- [x] Add the `sibling-pair-variance` rule, its names and its limitation.
|
||||
- [x] Document both; extend the tool contract text.
|
||||
- [x] Add service, CLI, MCP and rule tests.
|
||||
- [ ] Run static, formatting, duplication, documentation and test gates.
|
||||
- [ ] Rebuild a rich capture with both changes and record the results.
|
||||
@@ -300,7 +300,10 @@ exclusions and deduplicated limitations. It scans path spans without splitting
|
||||
literal paths into arrays. Before schema parsing, it reserves the capture clone
|
||||
and validation scratch by walking the parent without cloning it. Inline requests
|
||||
borrow the Evidence value through CLI/MCP envelope validation; the application
|
||||
admits the entire portable envelope before authenticated parsing or hashing.
|
||||
admits the entire portable envelope before authenticated parsing or hashing. An
|
||||
`evidence-file` source is read and authenticated first and then admitted like
|
||||
retained Evidence, so its strings are not charged as inline envelope text: a
|
||||
capture too large to project inline can be projected from its file.
|
||||
The envelope clone, hash scratch and capture clone stay charged during grouping.
|
||||
Temporary URL/path/MIME/query work and retained state must
|
||||
fit together within a 96 MiB projection representation budget. A whole group or
|
||||
@@ -367,11 +370,12 @@ template and `params` carry the four format rules with names `id`, `uuid`,
|
||||
only when listed, each at most once (a repeated rule is an `invalid_request`),
|
||||
in this order:
|
||||
|
||||
| Rule | Applies to | Parameter |
|
||||
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
|
||||
| `after-uid-literal` | The literal segment right after a literal `uid`, unless it is a stop word | `uid` |
|
||||
| `opaque-token` | A lowercase alphanumeric segment of twelve or more characters mixing letters and digits that is not a `v1`-style version | `uid` |
|
||||
| `sibling-variance` | Groups equal in method, origin and segment count that differ in one literal at index 2 or later; the siblings whose literal is identifier-like (a digit or `:`, or three or more hyphenated names under a plural literal) merge, never a stop word, and the other literals stay their own tools | singular previous literal plus `_id`, else `id` |
|
||||
| Rule | Applies to | Parameter |
|
||||
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
|
||||
| `after-uid-literal` | The literal segment right after a literal `uid`, unless it is a stop word | `uid` |
|
||||
| `opaque-token` | A lowercase alphanumeric segment of twelve or more characters mixing letters and digits that is not a `v1`-style version | `uid` |
|
||||
| `sibling-variance` | Groups equal in method, origin and segment count that differ in one literal at index 2 or later; the siblings whose literal is identifier-like (a digit or `:`, or three or more hyphenated names under a plural literal) merge, never a stop word, and the other literals stay their own tools | singular previous literal plus `_id`, else `id` |
|
||||
| `sibling-pair-variance` | Groups equal in method, origin and segment count that differ in two adjacent literals right after a plural literal at index 2 or later, with at least three distinct pairs and both positions varying (`/repos/{owner}/{repo}/issues`); stop words and `v1`-style versions never qualify | singular plural literal plus `_namespace`, then `_name` |
|
||||
|
||||
With `sibling-variance` selected, literal siblings within its comparison scope
|
||||
fold into existing opt-in parameters as observed values unless they are stop words.
|
||||
@@ -388,7 +392,7 @@ cut to their first label, `_by_<params>` appended when parameters exist,
|
||||
sanitized to `[a-z0-9_]`, at most 60 characters and made unique with numeric
|
||||
suffixes: `get_dashboards_by_uid`, `get_plugins_settings_by_plugin_id`.
|
||||
|
||||
`sibling-variance` compares routes of at most 32 segments; longer routes keep
|
||||
`sibling-variance` and `sibling-pair-variance` compare routes of at most 32 segments; longer routes keep
|
||||
their grouped templates and the manifest says so in a limitation. `body_shape`
|
||||
is derived for `POST`, `PUT` and `PATCH` from the tool's earliest record when it
|
||||
carries an `application/json` or `application/*+json` request body: top-level
|
||||
@@ -403,11 +407,14 @@ typed unsupported errors as the api-surface view, including the canonical
|
||||
token `sensitive_values_unretained`.
|
||||
|
||||
The manifest accumulator shares the api-surface admission model: the capture
|
||||
clone, the inline envelope, record pointers, groups, every route part, observed
|
||||
clone, the inline envelope (for an inline source), record pointers, groups, every route part, observed
|
||||
values, query names and body-shape keys are admitted before allocation, and each
|
||||
recorded body is charged as temporary bytes before it is parsed. Retained state and
|
||||
temporary work must fit together within the 96 MiB projection representation
|
||||
budget; exceeding it returns `resource_constraint` without a partial manifest.
|
||||
Name a large capture with `{"kind": "evidence-file", "path": "/abs/capture.json"}`
|
||||
rather than inline: response bodies the manifest never reads are then charged
|
||||
once per value in the capture clone, not per character.
|
||||
|
||||
## Resource and lifecycle boundaries
|
||||
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import {
|
||||
analysisViewCaptureFixture,
|
||||
analysisViewCaptureEvidence,
|
||||
@@ -19,6 +22,7 @@ import {
|
||||
import {
|
||||
inspectAnalysisView,
|
||||
inspectAnalysisViewValidated,
|
||||
inspectAnalysisViewWithFiles,
|
||||
summarizeRetainedAnalysis,
|
||||
} from "./AnalysisViewService.js";
|
||||
import type { Evidence } from "../../domain/evidence.js";
|
||||
@@ -27,6 +31,7 @@ import { err, ok } from "../../domain/result.js";
|
||||
import { EvidenceLedger } from "../investigation/EvidenceLedger.js";
|
||||
import { WEB_NETWORK_CAPTURE_LIMITS } from "../../domain/webNetworkCapture.js";
|
||||
import { isImmutableJsonSnapshot } from "../../domain/immutableJson.js";
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
|
||||
it.each(["layout", "javascript"])(
|
||||
"carries the retained %s source snapshot into repeatable shared views",
|
||||
@@ -860,3 +865,104 @@ it("admits raw_result and parameters before authenticating a portable capture",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe("inspectAnalysisViewWithFiles", () => {
|
||||
const view = { kind: "environment-manifest" } as const;
|
||||
const writeEvidenceFile = async (value: unknown): Promise<string> => {
|
||||
const root = await createTestTempDirectory("rea-analysis-view-file-");
|
||||
const path = join(root, "capture.json");
|
||||
await writeFile(path, JSON.stringify(value));
|
||||
return path;
|
||||
};
|
||||
|
||||
it("projects an Evidence file exactly as the same Evidence inline", async () => {
|
||||
const parent = analysisViewCaptureEvidence(
|
||||
environmentManifestCaptureFixture(),
|
||||
);
|
||||
const fromFile = await inspectAnalysisViewWithFiles({
|
||||
source: { kind: "evidence-file", path: await writeEvidenceFile(parent) },
|
||||
view,
|
||||
});
|
||||
const inline = inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view,
|
||||
});
|
||||
if (!fromFile.ok) throw fromFile.error;
|
||||
if (!inline.ok) throw inline.error;
|
||||
expect(fromFile.value.normalized_result).toEqual(
|
||||
inline.value.normalized_result,
|
||||
);
|
||||
expect(fromFile.value.parameters).toEqual(inline.value.parameters);
|
||||
});
|
||||
|
||||
it("accepts from a file a capture whose response text exceeds the inline budget", async () => {
|
||||
const capture = environmentManifestCaptureFixture();
|
||||
const reported = capture.records[0]!.reported as {
|
||||
response: { content: { text: string } };
|
||||
};
|
||||
// Inline admission charges about 14 bytes per character; this body alone overruns it.
|
||||
reported.response.content.text = "a".repeat(
|
||||
Math.ceil(WEB_NETWORK_CAPTURE_LIMITS.outputBytes / 14) + 1024,
|
||||
);
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
expect(
|
||||
inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view,
|
||||
}),
|
||||
).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError", resource: "memory" },
|
||||
});
|
||||
const fromFile = await inspectAnalysisViewWithFiles({
|
||||
source: { kind: "evidence-file", path: await writeEvidenceFile(parent) },
|
||||
view,
|
||||
});
|
||||
if (!fromFile.ok) throw fromFile.error;
|
||||
expect(fromFile.value.parameters).toMatchObject({
|
||||
parent_evidence_id: parent.evidence_id,
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses a relative path, a missing file and JSON that is not Evidence", async () => {
|
||||
expect(
|
||||
await inspectAnalysisViewWithFiles({
|
||||
source: { kind: "evidence-file", path: "capture.json" },
|
||||
view,
|
||||
}),
|
||||
).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
|
||||
const missing = join(
|
||||
await createTestTempDirectory("rea-analysis-view-missing-"),
|
||||
"absent.json",
|
||||
);
|
||||
expect(
|
||||
await inspectAnalysisViewWithFiles({
|
||||
source: { kind: "evidence-file", path: missing },
|
||||
view,
|
||||
}),
|
||||
).toMatchObject({ ok: false, error: { _tag: "EvidenceFileError" } });
|
||||
const notEvidence = await inspectAnalysisViewWithFiles({
|
||||
source: {
|
||||
kind: "evidence-file",
|
||||
path: await writeEvidenceFile({ hello: 1 }),
|
||||
},
|
||||
view,
|
||||
});
|
||||
expect(notEvidence).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisInputError" },
|
||||
});
|
||||
});
|
||||
|
||||
it("leaves file sources to the file-aware entry point", () => {
|
||||
expect(
|
||||
inspectAnalysisView({
|
||||
source: { kind: "evidence-file", path: "/tmp/capture.json" },
|
||||
view,
|
||||
}),
|
||||
).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCapabilityUnavailableError" },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -29,6 +29,7 @@ import { err, ok, type Result } from "../../domain/result.js";
|
||||
import { WebNetworkApiSurfaceAccumulator } from "../../domain/analysisView/WebNetworkApiSurfaceAccumulator.js";
|
||||
import { EnvironmentManifestAccumulator } from "../../domain/analysisView/EnvironmentManifestAccumulator.js";
|
||||
import { admitAnalysisViewData } from "../../domain/analysisView/analysisViewDataAdmission.js";
|
||||
import { readJsonFile } from "../JsonFiles.js";
|
||||
|
||||
/** Authenticate a selected-view request and project already completed Evidence. */
|
||||
export const inspectAnalysisView = (
|
||||
@@ -48,6 +49,48 @@ export const inspectAnalysisView = (
|
||||
return inspectAnalysisViewValidated(parsed.data, lookup);
|
||||
};
|
||||
|
||||
/** Validate a request whose source may be an Evidence file, then project it. */
|
||||
export const inspectAnalysisViewWithFiles = async (
|
||||
rawInput: unknown,
|
||||
lookup?: EvidenceLookup,
|
||||
): Promise<Result<Evidence, AnalysisError>> => {
|
||||
const parsed = inspectAnalysisViewInputSchema.safeParse(rawInput);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||
parsed.error.issues,
|
||||
rawInput,
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
return inspectAnalysisViewValidatedWithFiles(parsed.data, lookup);
|
||||
};
|
||||
|
||||
/**
|
||||
* Read and authenticate an Evidence-file source, then project it as retained
|
||||
* Evidence: its strings are bounded by the file reader rather than charged as
|
||||
* inline input. Other sources project unchanged.
|
||||
*/
|
||||
export const inspectAnalysisViewValidatedWithFiles = async (
|
||||
input: InspectAnalysisViewInput,
|
||||
lookup?: EvidenceLookup,
|
||||
): Promise<Result<Evidence, AnalysisError>> => {
|
||||
if (input.source.kind !== "evidence-file")
|
||||
return inspectAnalysisViewValidated(input, lookup);
|
||||
const loaded = await readJsonFile(input.source.path);
|
||||
if (!loaded.ok) return loaded;
|
||||
const parent = authenticateEvidence(loaded.value, ["source", "path"], {
|
||||
source: input.source,
|
||||
});
|
||||
if (!parent.ok) return parent;
|
||||
const id = parent.value.evidence_id;
|
||||
return inspectAnalysisViewValidated(
|
||||
{ ...input, source: { kind: "retained-evidence", evidence_id: id } },
|
||||
(evidenceId) => (evidenceId === id ? parent.value : lookup?.(evidenceId)),
|
||||
);
|
||||
};
|
||||
|
||||
/** Authenticate borrowed inline Evidence after adapter envelope validation. */
|
||||
export const inspectAnalysisViewValidated = (
|
||||
input: InspectAnalysisViewInput,
|
||||
@@ -182,8 +225,28 @@ const resolveAnalysisViewSource = (
|
||||
): Result<Evidence, AnalysisError> => {
|
||||
if (source.kind === "retained-evidence")
|
||||
return resolveEvidenceInput(source, lookup);
|
||||
// Files are read by inspectAnalysisViewValidatedWithFiles, which the CLI and MCP adapters call.
|
||||
if (source.kind === "evidence-file")
|
||||
return err(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
ANALYSIS_VIEW_PROVIDER.id,
|
||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||
"evidence-file sources are read before projection",
|
||||
),
|
||||
);
|
||||
return authenticateEvidence(source.evidence, ["source", "evidence"], {
|
||||
source,
|
||||
});
|
||||
};
|
||||
|
||||
/** Parse complete Evidence, reporting schema issues under the given request path. */
|
||||
const authenticateEvidence = (
|
||||
value: unknown,
|
||||
path: readonly (string | number)[],
|
||||
input: unknown,
|
||||
): Result<Evidence, AnalysisError> => {
|
||||
try {
|
||||
return ok(parseEvidence(source.evidence));
|
||||
return ok(parseEvidence(value));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof z.ZodError)
|
||||
return err(
|
||||
@@ -191,9 +254,9 @@ const resolveAnalysisViewSource = (
|
||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||
cause.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "evidence", ...issue.path],
|
||||
path: [...path, ...issue.path],
|
||||
})),
|
||||
{ source },
|
||||
input,
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { z } from "incur";
|
||||
|
||||
import { inspectAnalysisViewValidated } from "../application/analysisView/AnalysisViewService.js";
|
||||
import { inspectAnalysisViewValidatedWithFiles } from "../application/analysisView/AnalysisViewService.js";
|
||||
import { readEvidenceBundle } from "../application/EvidenceBundleFiles.js";
|
||||
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
|
||||
import {
|
||||
@@ -8,7 +8,7 @@ import {
|
||||
importEvidenceBundleInputSchema,
|
||||
} from "../contracts/sessionToolSchemas.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { parseCliJsonInput } from "../cliJsonInput.js";
|
||||
import { parseCliJsonInput, resolveCliJsonPaths } from "../cliJsonInput.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
@@ -82,7 +82,9 @@ export const registerAnalysisViewCommands = (
|
||||
CLI_COMMANDS.inspectAnalysisView,
|
||||
);
|
||||
if (!input.ok) return input.error;
|
||||
const parsed = inspectAnalysisViewInputSchema.safeParse(input.value);
|
||||
// An evidence-file path is resolved against the operator's working directory.
|
||||
const value = resolveCliJsonPaths(input.value, [["source", "path"]]);
|
||||
const parsed = inspectAnalysisViewInputSchema.safeParse(value);
|
||||
if (!parsed.success)
|
||||
return {
|
||||
error: "Application workflow failed",
|
||||
@@ -90,11 +92,11 @@ export const registerAnalysisViewCommands = (
|
||||
analysisInputErrorFromIssues(
|
||||
CLI_COMMANDS.inspectAnalysisView,
|
||||
parsed.error.issues,
|
||||
input.value,
|
||||
value,
|
||||
),
|
||||
),
|
||||
};
|
||||
const result = inspectAnalysisViewValidated(parsed.data);
|
||||
const result = await inspectAnalysisViewValidatedWithFiles(parsed.data);
|
||||
return result.ok
|
||||
? jsonValueSchema.parse(result.value)
|
||||
: {
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -438,6 +438,22 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumula
|
||||
});
|
||||
}
|
||||
|
||||
/** One limitation per selected sibling rule when a route is too long for it to compare. */
|
||||
#segmentLimitLimitations(toolGroups: readonly ToolGroup[]): string[] {
|
||||
if (
|
||||
!toolGroups.some(
|
||||
(group) => group.parts.length > SIBLING_VARIANCE_SEGMENT_LIMIT,
|
||||
)
|
||||
)
|
||||
return [];
|
||||
return (["sibling-variance", "sibling-pair-variance"] as const)
|
||||
.filter((rule) => this.#rules.has(rule))
|
||||
.map(
|
||||
(rule) =>
|
||||
`Routes with more than ${String(SIBLING_VARIANCE_SEGMENT_LIMIT)} segments are not compared by ${rule}; they keep their grouped templates.`,
|
||||
);
|
||||
}
|
||||
|
||||
/** Project the complete manifest; tools, records and exclusions are sorted deterministically. */
|
||||
finish(): Result<ManifestFields, AnalysisError> {
|
||||
const primary = this.#primaryOrigin();
|
||||
@@ -493,14 +509,7 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumula
|
||||
"A top-level __proto__ key in a recorded JSON body is omitted from body_shape; it cannot be an own property of the published object.",
|
||||
]
|
||||
: []),
|
||||
...(this.#rules.has("sibling-variance") &&
|
||||
toolGroups.some(
|
||||
(group) => group.parts.length > SIBLING_VARIANCE_SEGMENT_LIMIT,
|
||||
)
|
||||
? [
|
||||
`Routes with more than ${String(SIBLING_VARIANCE_SEGMENT_LIMIT)} segments are not compared by sibling-variance; they keep their grouped templates.`,
|
||||
]
|
||||
: []),
|
||||
...this.#segmentLimitLimitations(toolGroups),
|
||||
...(this.#unrepresented > 0
|
||||
? [
|
||||
`${String(this.#unrepresented)} retained records without a usable request line are not represented in the manifest.`,
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
import { isAbsolute } from "node:path";
|
||||
|
||||
import { localPathStringSchema } from "../localPath.js";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
javascriptModuleReferencesRequestSchema,
|
||||
@@ -254,7 +257,7 @@ const inlineEvidenceInputSchema = z.unknown().nonoptional();
|
||||
inlineEvidenceInputSchema._zod.processJSONSchema =
|
||||
evidenceSchema._zod.processJSONSchema;
|
||||
|
||||
/** Unauthenticated inline input or an exact same-session retained reference. */
|
||||
/** Unauthenticated inline input, an exact same-session retained reference, or one local Evidence file. */
|
||||
export const analysisViewSourceSchema = z.discriminatedUnion("kind", [
|
||||
z.strictObject({
|
||||
kind: z.literal("retained-evidence"),
|
||||
@@ -264,6 +267,17 @@ export const analysisViewSourceSchema = z.discriminatedUnion("kind", [
|
||||
kind: z.literal("inline"),
|
||||
evidence: inlineEvidenceInputSchema,
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("evidence-file"),
|
||||
path: localPathStringSchema
|
||||
.refine(isAbsolute, {
|
||||
message:
|
||||
"path must be an absolute local filesystem path (for example /tmp/capture.json or C:\\rea\\capture.json)",
|
||||
})
|
||||
.describe(
|
||||
"Absolute local path of one completed Evidence JSON file. REA reads and authenticates it, then projects it like retained Evidence, so large captures are not charged as inline input.",
|
||||
),
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Public inspect_analysis_view input. */
|
||||
|
||||
@@ -9,6 +9,7 @@ export const TEMPLATE_RULES = [
|
||||
"after-uid-literal",
|
||||
"opaque-token",
|
||||
"sibling-variance",
|
||||
"sibling-pair-variance",
|
||||
] as const;
|
||||
export type TemplateRule = (typeof TEMPLATE_RULES)[number];
|
||||
export type ManifestRule = RouteRule | TemplateRule;
|
||||
@@ -287,6 +288,98 @@ const mergeSiblingVariance = (groups: readonly ToolGroup[]): ToolGroup[] => {
|
||||
return [...current, ...beyondLimit];
|
||||
};
|
||||
|
||||
/** Fewest distinct literal pairs that make two adjacent positions one templated pair. */
|
||||
export const SIBLING_PAIR_MINIMUM = 3;
|
||||
|
||||
/** Both positions are literals that are neither stop words nor versions, right after a plural literal. */
|
||||
const pairCandidate = (group: ToolGroup, position: number): boolean => {
|
||||
const previous = group.parts[position - 1];
|
||||
const first = group.parts[position];
|
||||
const second = group.parts[position + 1];
|
||||
return (
|
||||
previous?.kind === "literal" &&
|
||||
previous.value.endsWith("s") &&
|
||||
!STOP_WORDS.has(previous.value) &&
|
||||
first?.kind === "literal" &&
|
||||
second?.kind === "literal" &&
|
||||
[first.value, second.value].every(
|
||||
(value) => !STOP_WORDS.has(value) && !VERSION_TOKEN.test(value),
|
||||
)
|
||||
);
|
||||
};
|
||||
|
||||
/** Merge one bucket when its pairs are distinct enough and both positions vary; otherwise keep its tools. */
|
||||
const mergePairBucket = (
|
||||
bucket: readonly ToolGroup[],
|
||||
position: number,
|
||||
): ToolGroup[] => {
|
||||
const pairs = new Set(
|
||||
bucket.map((group) =>
|
||||
compositeKey([
|
||||
literalAt(group, position),
|
||||
literalAt(group, position + 1),
|
||||
]),
|
||||
),
|
||||
);
|
||||
const firsts = new Set(bucket.map((group) => literalAt(group, position)));
|
||||
const seconds = new Set(
|
||||
bucket.map((group) => literalAt(group, position + 1)),
|
||||
);
|
||||
if (pairs.size < SIBLING_PAIR_MINIMUM || firsts.size < 2 || seconds.size < 2)
|
||||
return [...bucket];
|
||||
const first = earliest(bucket);
|
||||
const shape = withParts(
|
||||
first,
|
||||
first.parts.map((part, index) =>
|
||||
index === position || index === position + 1
|
||||
? param("sibling-pair-variance", [literalAt(first, index)])
|
||||
: part,
|
||||
),
|
||||
);
|
||||
return [
|
||||
mergeGroups(
|
||||
bucket.map((group) => (group === first ? shape : group)),
|
||||
shape,
|
||||
),
|
||||
];
|
||||
};
|
||||
|
||||
/** Merge groups that differ in two adjacent literals after a plural literal, such as `/repos/{owner}/{repo}/…`. */
|
||||
const mergeSiblingPairs = (groups: readonly ToolGroup[]): ToolGroup[] => {
|
||||
const beyondLimit = groups.filter(
|
||||
(group) => group.parts.length > SIBLING_VARIANCE_SEGMENT_LIMIT,
|
||||
);
|
||||
let current = groups.filter(
|
||||
(group) => group.parts.length <= SIBLING_VARIANCE_SEGMENT_LIMIT,
|
||||
);
|
||||
const longest = Math.max(0, ...current.map((group) => group.parts.length));
|
||||
for (let position = 2; position + 1 < longest; position += 1) {
|
||||
const buckets = new Map<string, ToolGroup[]>();
|
||||
const output: ToolGroup[] = [];
|
||||
for (const group of current) {
|
||||
if (!pairCandidate(group, position)) {
|
||||
output.push(group);
|
||||
continue;
|
||||
}
|
||||
const key = compositeKey([
|
||||
group.method,
|
||||
group.origin,
|
||||
group.parts.length,
|
||||
...group.parts.map((part, index) =>
|
||||
index === position || index === position + 1 ? null : partKey(part),
|
||||
),
|
||||
]);
|
||||
const bucket = buckets.get(key);
|
||||
if (bucket === undefined) buckets.set(key, [group]);
|
||||
else bucket.push(group);
|
||||
}
|
||||
for (const bucket of buckets.values())
|
||||
output.push(...mergePairBucket(bucket, position));
|
||||
current = output;
|
||||
}
|
||||
return [...current, ...beyondLimit];
|
||||
};
|
||||
|
||||
/** Apply the selected opt-in rules in fixed order; every merge is an inference. */
|
||||
export const applyTemplateRules = (
|
||||
groups: readonly ToolGroup[],
|
||||
@@ -303,6 +396,7 @@ export const applyTemplateRules = (
|
||||
);
|
||||
current = rebucket(current);
|
||||
if (rules.has("sibling-variance")) current = mergeSiblingVariance(current);
|
||||
if (rules.has("sibling-pair-variance")) current = mergeSiblingPairs(current);
|
||||
return current;
|
||||
};
|
||||
|
||||
@@ -343,6 +437,15 @@ const paramBaseName = (parts: readonly RoutePart[], index: number): string => {
|
||||
: "";
|
||||
return stem.length > 0 ? `${stem}_id` : "id";
|
||||
}
|
||||
case "sibling-pair-variance": {
|
||||
// The pair's first position follows the plural literal; the second follows the first.
|
||||
const second = parts[index - 1]?.kind === "param";
|
||||
const plural = parts[second ? index - 2 : index - 1];
|
||||
const stem =
|
||||
plural?.kind === "literal" ? sanitizeToken(singular(plural.value)) : "";
|
||||
const suffix = second ? "name" : "namespace";
|
||||
return stem.length > 0 ? `${stem}_${suffix}` : suffix;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -540,6 +540,84 @@ it("merges identifier-like siblings at index two or later and never folds stop w
|
||||
).toEqual(["blue-two", "green-three", "red-one"]);
|
||||
});
|
||||
|
||||
it("templates two adjacent literals after a plural literal as a pair when three pairs vary in both positions", () => {
|
||||
const projected = manifest(
|
||||
["sibling-pair-variance"],
|
||||
syntheticCapture([
|
||||
[
|
||||
"GET",
|
||||
"https://g.test/api/v1/repos/lighthouse-rota/lamp-room-log/issues",
|
||||
],
|
||||
["GET", "https://g.test/api/v1/repos/granite-ledger/ledger-core/issues"],
|
||||
[
|
||||
"GET",
|
||||
"https://g.test/api/v1/repos/blue-heron-kitchen/recipe-book/issues",
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"https://g.test/api/v1/repos/lighthouse-rota/tide-tables/issues/4",
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"https://g.test/api/v1/repos/granite-ledger/audit-trail/issues/9",
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"https://g.test/api/v1/repos/saltmarsh-depot/crate-tracker/issues/2",
|
||||
],
|
||||
["GET", "https://g.test/api/v1/teams/one/alpha/members"],
|
||||
["GET", "https://g.test/api/v1/teams/two/beta/members"],
|
||||
["GET", "https://g.test/api/v1/orgs/acme/x1/hooks"],
|
||||
["GET", "https://g.test/api/v1/orgs/acme/x2/hooks"],
|
||||
["GET", "https://g.test/api/v1/orgs/acme/x3/hooks"],
|
||||
]),
|
||||
);
|
||||
expect(projected.tools.map((tool) => tool.path_template)).toEqual([
|
||||
"/api/v1/orgs/acme/x1/hooks",
|
||||
"/api/v1/orgs/acme/x2/hooks",
|
||||
"/api/v1/orgs/acme/x3/hooks",
|
||||
"/api/v1/repos/{repo_namespace}/{repo_name}/issues",
|
||||
"/api/v1/repos/{repo_namespace}/{repo_name}/issues/{numeric}",
|
||||
"/api/v1/teams/one/alpha/members",
|
||||
"/api/v1/teams/two/beta/members",
|
||||
]);
|
||||
expect(
|
||||
projected.tools.find(
|
||||
(tool) =>
|
||||
tool.path_template ===
|
||||
"/api/v1/repos/{repo_namespace}/{repo_name}/issues",
|
||||
),
|
||||
).toMatchObject({
|
||||
name: "get_repos_issues_by_repo_namespace_repo_name",
|
||||
params: [
|
||||
{
|
||||
name: "repo_namespace",
|
||||
rule: "sibling-pair-variance",
|
||||
observed_values: [
|
||||
"blue-heron-kitchen",
|
||||
"granite-ledger",
|
||||
"lighthouse-rota",
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "repo_name",
|
||||
rule: "sibling-pair-variance",
|
||||
observed_values: ["lamp-room-log", "ledger-core", "recipe-book"],
|
||||
},
|
||||
],
|
||||
record_ordinals: [0, 1, 2],
|
||||
});
|
||||
expect(
|
||||
projected.tools
|
||||
.find((tool) => tool.path_template.endsWith("/issues/{numeric}"))
|
||||
?.params.map((param) => [param.name, param.rule]),
|
||||
).toEqual([
|
||||
["repo_namespace", "sibling-pair-variance"],
|
||||
["repo_name", "sibling-pair-variance"],
|
||||
["id", "numeric"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("never folds a literal sibling into a format-rule parameter", () => {
|
||||
const projected = manifest(
|
||||
ALL_RULES,
|
||||
|
||||
@@ -23,7 +23,7 @@ export const environmentManifestViewRequestSchema = z.strictObject({
|
||||
.meta({ uniqueItems: true })
|
||||
.default([])
|
||||
.describe(
|
||||
"Opt-in template rules applied after the four format rules, each listed at most once: after-uid-literal templates the segment following a literal uid, opaque-token templates long mixed alphanumeric segments, sibling-variance merges groups that differ in one identifier-like segment. Literal sibling folding into existing opt-in parameters also requires sibling-variance. Merged parameters are inferences.",
|
||||
"Opt-in template rules applied after the four format rules, each listed at most once: after-uid-literal templates the segment following a literal uid, opaque-token templates long mixed alphanumeric segments, sibling-variance merges groups that differ in one identifier-like segment, sibling-pair-variance merges groups that differ in two adjacent literals after a plural literal when at least three distinct pairs vary in both. Literal sibling folding into existing opt-in parameters also requires sibling-variance. Merged parameters are inferences.",
|
||||
),
|
||||
include_query_parameter_names: z
|
||||
.boolean()
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
|
||||
|
||||
import { inspectAnalysisView } from "../application/analysisView/AnalysisViewService.js";
|
||||
import { inspectAnalysisViewWithFiles } from "../application/analysisView/AnalysisViewService.js";
|
||||
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
|
||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||
import { toolContract } from "../contracts/toolContracts.js";
|
||||
@@ -24,7 +24,7 @@ export const registerAnalysisViewTool = (
|
||||
const registration = toolRegistrationOptions(contract);
|
||||
const handler = createEvidenceToolHandler(
|
||||
server,
|
||||
(input) => Promise.resolve(inspectAnalysisView(input, evidenceById)),
|
||||
(input) => inspectAnalysisViewWithFiles(input, evidenceById),
|
||||
logger,
|
||||
recordEvidence,
|
||||
withAdmittedAnalysis,
|
||||
|
||||
@@ -547,6 +547,49 @@ cliTest(
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"projects an environment manifest from an Evidence file through the compiled CLI",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-evidence-file-cli-");
|
||||
const capturePath = join(root, "capture.json");
|
||||
const parent = analysisViewCaptureEvidence(
|
||||
environmentManifestCaptureFixture(),
|
||||
);
|
||||
await writeFile(capturePath, JSON.stringify(parent));
|
||||
const requestPath = join(root, "view.json");
|
||||
const view = {
|
||||
kind: "environment-manifest",
|
||||
template_rules: ["opaque-token", "sibling-pair-variance"],
|
||||
} as const;
|
||||
await writeFile(
|
||||
requestPath,
|
||||
JSON.stringify({
|
||||
source: { kind: "evidence-file", path: capturePath },
|
||||
view,
|
||||
}),
|
||||
);
|
||||
const result = await cli.run({
|
||||
arguments: ["inspect-analysis-view", requestPath, "--json"],
|
||||
});
|
||||
expect(result.exitCode, result.stdout).toBe(0);
|
||||
const evidence = parseEvidence(result.json);
|
||||
expect(evidence.parameters).toMatchObject({
|
||||
parent_evidence_id: parent.evidence_id,
|
||||
});
|
||||
// The file source projects exactly what the same Evidence projects inline.
|
||||
const inline = inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view,
|
||||
});
|
||||
if (!inline.ok) throw inline.error;
|
||||
expect(evidence.normalized_result).toEqual(inline.value.normalized_result);
|
||||
expect(evidence.normalized_result).toMatchObject({
|
||||
kind: "environment-manifest",
|
||||
origin: "http://grafana.example",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"projects an environment manifest through the compiled CLI without recorded values",
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -1167,7 +1167,7 @@ it.each(PUBLISHED_EXAMPLES)(
|
||||
},
|
||||
);
|
||||
|
||||
it("projects an environment manifest identically from retained and portable MCP Evidence", async () => {
|
||||
it("projects an environment manifest identically from retained, portable and file MCP Evidence", async () => {
|
||||
const { client, session } = await connect();
|
||||
const { ajv, inputSchema, outputSchema } = await advertisedMcpToolSchemas(
|
||||
client,
|
||||
@@ -1176,6 +1176,11 @@ it("projects an environment manifest identically from retained and portable MCP
|
||||
const capture = environmentManifestCaptureFixture();
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
expect(session.recordEvidence(parent).ok).toBe(true);
|
||||
const evidencePath = join(
|
||||
await createTestTempDirectory("rea-evidence-file-mcp-"),
|
||||
"capture.json",
|
||||
);
|
||||
await writeFile(evidencePath, JSON.stringify(parent));
|
||||
const view = {
|
||||
kind: "environment-manifest",
|
||||
template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"],
|
||||
@@ -1184,6 +1189,7 @@ it("projects an environment manifest identically from retained and portable MCP
|
||||
for (const source of [
|
||||
{ kind: "inline", evidence: parent },
|
||||
{ kind: "retained-evidence", evidence_id: parent.evidence_id },
|
||||
{ kind: "evidence-file", path: evidencePath },
|
||||
]) {
|
||||
expect(ajv.validate(inputSchema, { source, view })).toBe(true);
|
||||
expect(
|
||||
@@ -1224,6 +1230,16 @@ it("projects an environment manifest identically from retained and portable MCP
|
||||
outputs.push(evidence.normalized_result);
|
||||
}
|
||||
expect(outputs[0]).toEqual(outputs[1]);
|
||||
expect(outputs[2]).toEqual(outputs[0]);
|
||||
// JSON Schema cannot express "absolute"; the tool refuses a relative path when called.
|
||||
const relative = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "evidence-file", path: "capture.json" },
|
||||
view,
|
||||
},
|
||||
});
|
||||
expect(parseMcpToolError(relative).error.code).toBe("invalid_request");
|
||||
const hidden = createEvidence(
|
||||
{
|
||||
path: capture.artifact.path,
|
||||
|
||||
Reference in New Issue
Block a user