mirror of
https://github.com/morluto/rea
synced 2026-10-25 22:08:11 -04:00
feat(native): make the function-matching allowance a runtime setting
match_native_functions refused inventories above its fixed 96 MiB representation allowance with "This release has no runtime override", so ordinary 20k-50k function builds could not be matched at all. Add native.matching_representation_bytes (1 byte to 1 GiB, default 96 MiB, next operation) with the REA_NATIVE_MATCHING_REPRESENTATION_BYTES launch seed, copied by rea setup. MCP resolves the accepted operation's override over the launch value on each call; the CLI uses its launch environment. A refusal binds details.limit with the effective value, source and bounds, and its recovery names the setting and the seed. An invalid seed is refused by matching alone, so MCP stays available and a runtime value repairs it. Fixes #6406 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -169,6 +169,7 @@ cannot partially apply. Clients coordinating writes can supply
|
||||
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
|
||||
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
|
||||
| `wasm.command_timeout_ms` | Next WABT operation |
|
||||
| `native.matching_representation_bytes` | Next `match_native_functions` request |
|
||||
|
||||
### Precedence and lifetime
|
||||
|
||||
@@ -243,6 +244,8 @@ Inspection then reports the runtime value while retaining the launch error.
|
||||
Reset exposes the invalid launch state again; it does not repair or replace it
|
||||
with the built-in default. Provider availability and execution must resolve the
|
||||
same effective setting so readiness changes along with repair and reset.
|
||||
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES` is validated the same way, by
|
||||
`match_native_functions` alone.
|
||||
|
||||
## Keep a fixed ceiling honest
|
||||
|
||||
|
||||
@@ -117,7 +117,7 @@ in the JSON for all groups inline. Import the bundle into MCP for views, or
|
||||
extract its matching Evidence record to a JSON file for the existing CLI
|
||||
`inspect-analysis-view` `evidence-file` source. Input bundles are unchanged.
|
||||
|
||||
Representation admission is bounded at 96 MiB before cloning selected observations
|
||||
Representation admission is bounded before cloning selected observations
|
||||
and expanding matching indexes and repeated per-function source links. Each
|
||||
inventory row is charged before parsing for the member it becomes, including the
|
||||
report and Evidence copies that member is serialized into, so an oversized side
|
||||
@@ -126,15 +126,26 @@ charged for the copies their authentication or loading creates. An exact
|
||||
session reference to retained Evidence is borrowed without a copy, so only
|
||||
what matching derives from it is charged; the inventory's own Evidence ID is
|
||||
charged once, not per function. Selected observations are charged in full.
|
||||
Measured with Ghidra-shaped rows, about 20-character names and no
|
||||
Measured under the 96 MiB default with Ghidra-shaped rows, about 20-character names and no
|
||||
observations, about 22,000 rows per side fit through retained
|
||||
`list_procedures` references, and about 10,000 through inline Evidence or CLI
|
||||
`source_bundle` inputs. Work yields to
|
||||
cancellation. No worker/provider concurrency is introduced; this budget is not
|
||||
a bound on total process RSS.
|
||||
|
||||
The allowance defaults to 96 MiB and is the runtime setting
|
||||
`native.matching_representation_bytes`, from 1 byte to 1 GiB. On MCP, call
|
||||
`update_analysis_settings` with that setting and retry; it applies to the next
|
||||
operation without restarting REA. For CLI or future REA processes, set
|
||||
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES` in the launch environment; `rea
|
||||
setup` copies it into client registrations. An invalid launch value is refused
|
||||
by matching only, and a runtime value repairs it. A refusal carries
|
||||
`details.limit` with the effective value, its source and bounds. Raising the
|
||||
allowance lets matching retain more in the REA process; the upper bound keeps
|
||||
the complete Evidence's canonical serialization within one Node string.
|
||||
Summary delivery happens after complete matching Evidence is constructed and
|
||||
retained. It reduces the initial reply, but does not avoid that construction or
|
||||
raise the allowance. There is no runtime override. Repeated provenance remains
|
||||
raise the allowance. Repeated provenance remains
|
||||
part of the complete member contract; splitting inventories into separate matches
|
||||
would change global name/body uniqueness and is not an equivalent workaround.
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ const SELECTED_REGISTRATION_SETTINGS = [
|
||||
"REA_HOPPER_STARTUP_TIMEOUT_MS",
|
||||
"REA_JEB_MCP_TIMEOUT_MS",
|
||||
"REA_MCP_INPUT_SCHEMA_PROFILE",
|
||||
"REA_NATIVE_MATCHING_REPRESENTATION_BYTES",
|
||||
"REA_REGEX_MATCHING_DEADLINE_MS",
|
||||
"REA_REGEX_WORKER_HEAP_REQUEST_MIB",
|
||||
"REA_MCP_INITIALIZE_BUDGET_MS",
|
||||
|
||||
@@ -4,6 +4,7 @@ import { matchNativeFunctions } from "./NativeFunctionMatchingService.js";
|
||||
import { nativeFunctionMatchingSchema } from "../../domain/native/functionMatching.js";
|
||||
import { createEvidence, immutableEvidence } from "../../domain/evidence.js";
|
||||
import { createAnalysisProfile } from "../../domain/analysisProfile.js";
|
||||
import { AnalysisResourceConstraintError } from "../../domain/analysisErrorCore.js";
|
||||
import { functionDossierSchema } from "../../domain/hopperValues.js";
|
||||
import { procedureInfoSchema } from "../../domain/native/procedureInfo.js";
|
||||
import {
|
||||
@@ -919,6 +920,69 @@ it("matches 20,000 retained Ghidra-shaped rows per side and refuses an oversized
|
||||
});
|
||||
});
|
||||
|
||||
it("enforces the caller-resolved representation allowance and names its setting", async () => {
|
||||
const rows = (count: number) =>
|
||||
Array.from({ length: count }, (_, i) => ({
|
||||
address: `0x${(0x1000 + i * 16).toString(16)}`,
|
||||
value: `function_${i}`,
|
||||
}));
|
||||
const small = {
|
||||
left: nativeMatchingInventory("1", rows(100)),
|
||||
right: nativeMatchingInventory("2", rows(100)),
|
||||
};
|
||||
const lowered = await matchNativeFunctions(small, {
|
||||
representationAllowance: { maximumBytes: 64 * 1024, source: "runtime" },
|
||||
});
|
||||
expect(lowered.ok).toBe(false);
|
||||
if (lowered.ok) throw new Error("Lowered allowance admitted the request");
|
||||
expect(lowered.error).toMatchObject({
|
||||
_tag: "AnalysisResourceConstraintError",
|
||||
reportedLimits: { maximum_bytes: 64 * 1024 },
|
||||
retryable: false,
|
||||
limit: {
|
||||
setting: "native.matching_representation_bytes",
|
||||
effective_value: 64 * 1024,
|
||||
source: "runtime",
|
||||
unit: "bytes",
|
||||
minimum: 1,
|
||||
maximum: 1024 * 1024 * 1024,
|
||||
applies_at: "next_operation",
|
||||
agent_editable: true,
|
||||
},
|
||||
});
|
||||
if (!(lowered.error instanceof AnalysisResourceConstraintError))
|
||||
throw lowered.error;
|
||||
expect(lowered.error.remediationAction).toContain(
|
||||
"update_analysis_settings with setting native.matching_representation_bytes",
|
||||
);
|
||||
expect(lowered.error.remediationAction).toContain(
|
||||
"REA_NATIVE_MATCHING_REPRESENTATION_BYTES",
|
||||
);
|
||||
expect((await matchNativeFunctions(small)).ok).toBe(true);
|
||||
// A request refused by the 96 MiB default fits a raised allowance unchanged.
|
||||
const large = {
|
||||
left: nativeMatchingInventory("1", rows(15_000)),
|
||||
right: nativeMatchingInventory("2", rows(15_000)),
|
||||
};
|
||||
const refused = await matchNativeFunctions(large);
|
||||
expect(refused.ok).toBe(false);
|
||||
if (!refused.ok)
|
||||
expect(refused.error).toMatchObject({
|
||||
limit: { effective_value: 96 * 1024 * 1024, source: "default" },
|
||||
});
|
||||
const raised = await matchNativeFunctions(large, {
|
||||
representationAllowance: {
|
||||
maximumBytes: 256 * 1024 * 1024,
|
||||
source: "setting",
|
||||
},
|
||||
});
|
||||
if (!raised.ok) throw raised.error;
|
||||
expect(
|
||||
nativeFunctionMatchingSchema.parse(raised.value.evidence.normalized_result)
|
||||
.summary.matched,
|
||||
).toBe(15_000);
|
||||
});
|
||||
|
||||
it("keeps duplicate candidate storage linear and emits every input occurrence exactly once", async () => {
|
||||
const entries = Array.from({ length: 1000 }, (_, i) => ({
|
||||
address: `0x${i.toString(16)}`,
|
||||
|
||||
@@ -18,9 +18,15 @@ import { err, ok, type Result } from "../../domain/result.js";
|
||||
import type { EvidenceLookup } from "../EvidenceInputResolver.js";
|
||||
import { resolveNativeMatchingSources } from "./NativeMatchingSources.js";
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
import type { NativeMatchingAllowance } from "../../config/nativeMatchingAllowance.js";
|
||||
import { FUNCTION_COMPARISON_PROVIDER } from "../../domain/comparisonEvidenceIdentity.js";
|
||||
|
||||
const OPERATION = "match_native_functions";
|
||||
|
||||
/** Effective admission budget, separate from caller-authored Evidence parameters. */
|
||||
export interface NativeFunctionMatchingOptions extends ExecutionOptions {
|
||||
readonly representationAllowance?: NativeMatchingAllowance;
|
||||
}
|
||||
export interface NativeFunctionMatchingResult {
|
||||
readonly evidence: Evidence;
|
||||
readonly sources: readonly Evidence[];
|
||||
@@ -30,10 +36,14 @@ export interface NativeFunctionMatchingResult {
|
||||
/** Pair existing observations through one source/analysis admission. */
|
||||
export const matchNativeFunctions = async (
|
||||
rawInput: unknown,
|
||||
options?: ExecutionOptions,
|
||||
options?: NativeFunctionMatchingOptions,
|
||||
lookup?: EvidenceLookup,
|
||||
): Promise<Result<NativeFunctionMatchingResult, AnalysisError>> => {
|
||||
const admission = new NativeMatchingAdmission(OPERATION, options?.signal);
|
||||
const admission = new NativeMatchingAdmission(
|
||||
OPERATION,
|
||||
options?.signal,
|
||||
options?.representationAllowance,
|
||||
);
|
||||
try {
|
||||
await setImmediate();
|
||||
await admission.checkpoint();
|
||||
@@ -171,10 +181,14 @@ export const matchNativeFunctionsFromBundles = async (
|
||||
input: MatchNativeFunctionsInput,
|
||||
paths: readonly string[],
|
||||
resources: ArtifactResourceScope,
|
||||
options?: ExecutionOptions,
|
||||
options?: NativeFunctionMatchingOptions,
|
||||
): Promise<Result<NativeFunctionMatchingResult, AnalysisError>> => {
|
||||
try {
|
||||
const admission = new NativeMatchingAdmission(OPERATION, options?.signal);
|
||||
const admission = new NativeMatchingAdmission(
|
||||
OPERATION,
|
||||
options?.signal,
|
||||
options?.representationAllowance,
|
||||
);
|
||||
await admission.admit(input);
|
||||
const loaded = await loadNativeMatchingBundleSources({
|
||||
input,
|
||||
|
||||
@@ -1,32 +1,73 @@
|
||||
import { setImmediate } from "node:timers/promises";
|
||||
import { AnalysisResourceConstraintError } from "../../domain/analysisErrorCore.js";
|
||||
import { fixedNativeAllowance } from "../../domain/native/nativeDeadlines.js";
|
||||
import type { Evidence } from "../../domain/evidence.js";
|
||||
import { analysisSettingLimit } from "../../domain/analysisSettingDefinitions.js";
|
||||
import {
|
||||
DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
} from "../../domain/analysisResourceLimits.js";
|
||||
import {
|
||||
DEFAULT_NATIVE_MATCHING_ALLOWANCE,
|
||||
NATIVE_MATCHING_ALLOWANCE_SETTING,
|
||||
type NativeMatchingAllowance,
|
||||
} from "../../config/nativeMatchingAllowance.js";
|
||||
|
||||
const NATIVE_MATCHING_REPRESENTATION_BYTES = 96 * 1024 * 1024;
|
||||
const MEBIBYTE = 1024 * 1024;
|
||||
const byteAllowance = (bytes: number): string =>
|
||||
Number.isInteger(bytes / MEBIBYTE)
|
||||
? `${String(bytes / MEBIBYTE)} MiB / ${String(bytes)} bytes`
|
||||
: `${String(bytes)} bytes`;
|
||||
|
||||
/** User-visible recovery for the effective matching representation allowance. */
|
||||
const nativeMatchingAllowanceRecovery = (
|
||||
allowance: NativeMatchingAllowance,
|
||||
): string =>
|
||||
[
|
||||
`Native function matching is limited to ${byteAllowance(allowance.maximumBytes)} of representation${allowance.source === "setting" ? ` by ${NATIVE_MATCHING_ALLOWANCE_SETTING}` : ""}.`,
|
||||
`On MCP, call update_analysis_settings with setting native.matching_representation_bytes, action set, and a value in bytes up to ${byteAllowance(MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES)} (default ${byteAllowance(DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES)}), then retry the original request. It applies to the next operation without restarting REA. For CLI or future REA processes, set ${NATIVE_MATCHING_ALLOWANCE_SETTING} in the launch environment.`,
|
||||
"This budget is not a bound on total process RSS; raising it lets matching retain more representation in the REA process.",
|
||||
"Splitting inventories into separate matches changes global name and body uniqueness, so it is not an equivalent workaround.",
|
||||
"Repeating the unchanged request fails the same way.",
|
||||
].join(" ");
|
||||
|
||||
/** Budget the actual selected observations, before cloning or index allocation. */
|
||||
export class NativeMatchingAdmission {
|
||||
#remaining = NATIVE_MATCHING_REPRESENTATION_BYTES;
|
||||
readonly #maximum: number;
|
||||
readonly #source: NativeMatchingAllowance["source"];
|
||||
#remaining: number;
|
||||
#work = 0;
|
||||
readonly #seen = new WeakSet<object>();
|
||||
constructor(
|
||||
readonly operation: string,
|
||||
readonly signal?: AbortSignal,
|
||||
) {}
|
||||
allowance: NativeMatchingAllowance = DEFAULT_NATIVE_MATCHING_ALLOWANCE,
|
||||
) {
|
||||
// Read the accepted operation's value once, so the refusal reports the
|
||||
// budget this request actually ran under.
|
||||
this.#maximum = allowance.maximumBytes;
|
||||
this.#source = allowance.source;
|
||||
this.#remaining = this.#maximum;
|
||||
}
|
||||
readonly checkpoint = async (bytes = 0): Promise<void> => {
|
||||
this.#remaining -= bytes;
|
||||
if (this.#remaining < 0)
|
||||
throw new AnalysisResourceConstraintError(
|
||||
this.operation,
|
||||
"capacity",
|
||||
`Native function matching exceeds its ${String(NATIVE_MATCHING_REPRESENTATION_BYTES)}-byte (96 MiB) representation allowance. This release has no runtime override.`,
|
||||
{ maximum_bytes: NATIVE_MATCHING_REPRESENTATION_BYTES },
|
||||
fixedNativeAllowance(
|
||||
"The native function-matching representation allowance",
|
||||
"bytes",
|
||||
`${String(NATIVE_MATCHING_REPRESENTATION_BYTES)} bytes`,
|
||||
),
|
||||
`Native function matching exceeds its ${byteAllowance(this.#maximum)} representation allowance (native.matching_representation_bytes).`,
|
||||
{ maximum_bytes: this.#maximum },
|
||||
{
|
||||
retryable: false,
|
||||
remediationAction: nativeMatchingAllowanceRecovery({
|
||||
maximumBytes: this.#maximum,
|
||||
source: this.#source,
|
||||
}),
|
||||
limit: analysisSettingLimit(
|
||||
"native.matching_representation_bytes",
|
||||
this.#maximum,
|
||||
this.#source,
|
||||
),
|
||||
},
|
||||
);
|
||||
this.signal?.throwIfAborted();
|
||||
if (++this.#work % 256 === 0)
|
||||
|
||||
+1
-1
@@ -147,7 +147,7 @@ export const createCli = (
|
||||
registerFirmwareCommands(cli, logger, environment);
|
||||
registerBinaryDiagnosticsCommands(cli, logger, environment);
|
||||
registerAnalysisViewCommands(cli, logger);
|
||||
registerNativeFunctionMatchingCommands(cli, logger);
|
||||
registerNativeFunctionMatchingCommands(cli, logger, environment);
|
||||
registerFridaCommands(cli, logger, fridaInstrumentation);
|
||||
registerEvidenceCommands(cli, logger);
|
||||
registerEnvironmentCommands(cli, logger, environment);
|
||||
|
||||
@@ -17,11 +17,13 @@ import {
|
||||
} from "./analysisEvidenceDelivery.js";
|
||||
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { parseNativeMatchingAllowance } from "../config/nativeMatchingAllowance.js";
|
||||
|
||||
/** Resolve portable input bundles, then use the same read-only matching service as MCP. */
|
||||
export const registerNativeFunctionMatchingCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): void => {
|
||||
cli.command(CLI_COMMANDS.matchNativeFunctions, {
|
||||
description:
|
||||
@@ -75,6 +77,8 @@ export const registerNativeFunctionMatchingCommands = (
|
||||
};
|
||||
const invalid = cliEvidenceDeliveryInputError(delivery);
|
||||
if (invalid !== undefined) return projectAnalysisError(invalid);
|
||||
const allowance = parseNativeMatchingAllowance(environment);
|
||||
if (!allowance.ok) return projectAnalysisError(allowance.error);
|
||||
const resources = new ArtifactResourceScope();
|
||||
let sources: readonly Evidence[] = [];
|
||||
const complete = await finishAnalysisCommand(
|
||||
@@ -83,7 +87,7 @@ export const registerNativeFunctionMatchingCommands = (
|
||||
parsed.data,
|
||||
options.source_bundle,
|
||||
resources,
|
||||
{ signal },
|
||||
{ signal, representationAllowance: allowance.value },
|
||||
);
|
||||
if (!result.ok) return result;
|
||||
sources = result.value.sources;
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
import { regexMatchingDeadlineMsSchema } from "./regexMatchingDeadline.js";
|
||||
import { regexWorkerHeapRequestMiBSchema } from "./regexWorkerHeapRequest.js";
|
||||
import { captureDecoderOldGenerationMiBSchema } from "./captureDecoderHeap.js";
|
||||
import { nativeMatchingRepresentationBytesSchema } from "./nativeMatchingAllowance.js";
|
||||
const environmentSchemas: Readonly<Record<string, z.ZodType<number, string>>> =
|
||||
{
|
||||
REA_ASAR_MAX_HEADER_BYTES: asarMaxHeaderBytesSchema,
|
||||
@@ -22,6 +23,8 @@ const environmentSchemas: Readonly<Record<string, z.ZodType<number, string>>> =
|
||||
REA_REGEX_WORKER_HEAP_REQUEST_MIB: regexWorkerHeapRequestMiBSchema,
|
||||
REA_CAPTURE_DECODER_OLD_GENERATION_MIB:
|
||||
captureDecoderOldGenerationMiBSchema,
|
||||
REA_NATIVE_MATCHING_REPRESENTATION_BYTES:
|
||||
nativeMatchingRepresentationBytesSchema,
|
||||
};
|
||||
export const launchAnalysisSetting = (
|
||||
id: AnalysisSettingId,
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
import { expect, it } from "vitest";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { parseEnvironment } from "./environment.js";
|
||||
import { runtimeNativeMatchingAllowance } from "./runtimeAnalysisSettings.js";
|
||||
import {
|
||||
DEFAULT_NATIVE_MATCHING_ALLOWANCE,
|
||||
NATIVE_MATCHING_ALLOWANCE_SETTING,
|
||||
nativeMatchingRepresentationBytesConstraint,
|
||||
parseNativeMatchingAllowance,
|
||||
} from "./nativeMatchingAllowance.js";
|
||||
import { launchAnalysisSetting } from "./analysisSettings.js";
|
||||
import { explicitSetupRegistrationEnvironment } from "../application/SetupRegistrationEnvironment.js";
|
||||
|
||||
it("keeps the 96 MiB allowance when the launch setting is unset", () => {
|
||||
expect(parseNativeMatchingAllowance({})).toEqual({
|
||||
ok: true,
|
||||
value: DEFAULT_NATIVE_MATCHING_ALLOWANCE,
|
||||
});
|
||||
expect(DEFAULT_NATIVE_MATCHING_ALLOWANCE.maximumBytes).toBe(96 * 1024 * 1024);
|
||||
expect(
|
||||
launchAnalysisSetting("native.matching_representation_bytes", {}),
|
||||
).toEqual({
|
||||
ok: true,
|
||||
value: { value: 96 * 1024 * 1024, source: "default" },
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts decimal byte counts from 1 byte to 1 GiB as a launch setting", () => {
|
||||
for (const bytes of [1, 268_435_456, 1024 * 1024 * 1024]) {
|
||||
const environment = { [NATIVE_MATCHING_ALLOWANCE_SETTING]: String(bytes) };
|
||||
expect(parseNativeMatchingAllowance(environment)).toEqual({
|
||||
ok: true,
|
||||
value: { maximumBytes: bytes, source: "setting" },
|
||||
});
|
||||
expect(
|
||||
launchAnalysisSetting(
|
||||
"native.matching_representation_bytes",
|
||||
environment,
|
||||
),
|
||||
).toEqual({ ok: true, value: { value: bytes, source: "setting" } });
|
||||
}
|
||||
// rea setup forwards the launch seed into client registrations.
|
||||
expect(
|
||||
explicitSetupRegistrationEnvironment({
|
||||
[NATIVE_MATCHING_ALLOWANCE_SETTING]: "268435456",
|
||||
UNRELATED: "x",
|
||||
}),
|
||||
).toEqual({ [NATIVE_MATCHING_ALLOWANCE_SETTING]: "268435456" });
|
||||
});
|
||||
|
||||
it("refuses invalid launch values instead of falling back to the default", () => {
|
||||
for (const value of ["0", "-1", "01", "1.5", "96MiB", "", "1073741825"]) {
|
||||
const environment = { [NATIVE_MATCHING_ALLOWANCE_SETTING]: value };
|
||||
const parsed = parseNativeMatchingAllowance(environment);
|
||||
expect(parsed.ok).toBe(false);
|
||||
if (parsed.ok) continue;
|
||||
expect(parsed.error).toBeInstanceOf(ConfigurationError);
|
||||
expect(parsed.error.message).toBe(
|
||||
nativeMatchingRepresentationBytesConstraint,
|
||||
);
|
||||
expect(
|
||||
launchAnalysisSetting("native.matching_representation_bytes", environment)
|
||||
.ok,
|
||||
).toBe(false);
|
||||
// Operation-local: global startup parsing is unaffected, and a runtime
|
||||
// override repairs the effective allowance without editing the seed.
|
||||
expect(parseEnvironment(environment).ok).toBe(true);
|
||||
expect(runtimeNativeMatchingAllowance(environment)().ok).toBe(false);
|
||||
const overridden = runtimeNativeMatchingAllowance(environment, () => ({
|
||||
revision: 1,
|
||||
overrides: { "native.matching_representation_bytes": 4096 },
|
||||
}))();
|
||||
expect(overridden).toEqual({
|
||||
ok: true,
|
||||
value: { maximumBytes: 4096, source: "runtime" },
|
||||
});
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,58 @@
|
||||
import {
|
||||
DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
} from "../domain/analysisResourceLimits.js";
|
||||
import { z } from "zod";
|
||||
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
export const NATIVE_MATCHING_ALLOWANCE_SETTING =
|
||||
"REA_NATIVE_MATCHING_REPRESENTATION_BYTES";
|
||||
|
||||
/** Effective native function-matching representation allowance and its source. */
|
||||
export interface NativeMatchingAllowance {
|
||||
readonly maximumBytes: number;
|
||||
readonly source: "default" | "setting" | "runtime";
|
||||
}
|
||||
|
||||
export const DEFAULT_NATIVE_MATCHING_ALLOWANCE: NativeMatchingAllowance = {
|
||||
maximumBytes: DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
source: "default",
|
||||
};
|
||||
|
||||
export const nativeMatchingRepresentationBytesConstraint = `${NATIVE_MATCHING_ALLOWANCE_SETTING} must be a positive decimal integer of at most ${String(MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES)} bytes (1 GiB); the default is ${String(DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES)} bytes (96 MiB).`;
|
||||
|
||||
/** Optional decimal byte count for the native function-matching allowance. */
|
||||
export const nativeMatchingRepresentationBytesSchema = z
|
||||
.string()
|
||||
.regex(/^[1-9]\d*$/u, {
|
||||
error: nativeMatchingRepresentationBytesConstraint,
|
||||
})
|
||||
.transform(Number)
|
||||
.pipe(
|
||||
z.number().int().max(MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES, {
|
||||
error: nativeMatchingRepresentationBytesConstraint,
|
||||
}),
|
||||
);
|
||||
|
||||
/** Resolve the effective allowance from one launch environment. */
|
||||
export const parseNativeMatchingAllowance = (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
): Result<NativeMatchingAllowance, ConfigurationError> => {
|
||||
const value = environment[NATIVE_MATCHING_ALLOWANCE_SETTING];
|
||||
if (value === undefined) return ok(DEFAULT_NATIVE_MATCHING_ALLOWANCE);
|
||||
const parsed = nativeMatchingRepresentationBytesSchema.safeParse(value);
|
||||
return parsed.success
|
||||
? ok({ maximumBytes: parsed.data, source: "setting" })
|
||||
: err(
|
||||
new ConfigurationError(nativeMatchingRepresentationBytesConstraint, {
|
||||
settings: [
|
||||
{
|
||||
setting: NATIVE_MATCHING_ALLOWANCE_SETTING,
|
||||
constraint: nativeMatchingRepresentationBytesConstraint,
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
};
|
||||
@@ -1,4 +1,6 @@
|
||||
import type { AnalysisSettingsReader } from "../domain/analysisSettings.js";
|
||||
import type { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { ok, type Result } from "../domain/result.js";
|
||||
import type { AppConfig } from "./types.js";
|
||||
import {
|
||||
parseAsarHeaderAdmission,
|
||||
@@ -8,6 +10,10 @@ import {
|
||||
resolveCaptureDecoderHeap,
|
||||
type CaptureDecoderHeap,
|
||||
} from "./captureDecoderHeap.js";
|
||||
import {
|
||||
parseNativeMatchingAllowance,
|
||||
type NativeMatchingAllowance,
|
||||
} from "./nativeMatchingAllowance.js";
|
||||
|
||||
/** Typed views read the enclosing operation snapshot, never the ambient environment. */
|
||||
export const runtimeAnalysisConfig = (
|
||||
@@ -111,3 +117,19 @@ export const runtimeCaptureDecoderHeap = (
|
||||
},
|
||||
};
|
||||
};
|
||||
/**
|
||||
* Resolve the accepted operation's allowance on each call. An invalid launch
|
||||
* value is refused only by the operation it bounds, so MCP stays available
|
||||
* and a runtime override can repair it.
|
||||
*/
|
||||
export const runtimeNativeMatchingAllowance =
|
||||
(
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
read?: AnalysisSettingsReader,
|
||||
): (() => Result<NativeMatchingAllowance, ConfigurationError>) =>
|
||||
() => {
|
||||
const override = read?.().overrides["native.matching_representation_bytes"];
|
||||
return override === undefined
|
||||
? parseNativeMatchingAllowance(environment)
|
||||
: ok({ maximumBytes: override, source: "runtime" });
|
||||
};
|
||||
|
||||
@@ -14,6 +14,19 @@ export const DEFAULT_ASAR_MAX_HEADER_BYTES = 16 * 1024 * 1024;
|
||||
*/
|
||||
export const MAXIMUM_ASAR_MAX_HEADER_BYTES = 256 * 1024 * 1024;
|
||||
|
||||
/** Native function-matching representation admitted when the setting is unset. */
|
||||
export const DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES = 96 * 1024 * 1024;
|
||||
|
||||
/**
|
||||
* Largest configurable matching admission. Complete matching Evidence is
|
||||
* serialized into one canonical string for its digest, about 670 bytes per
|
||||
* emitted member with short names. Each member is charged at least about
|
||||
* 1.9 KiB, so 1 GiB keeps that text well below Node's single-string ceiling
|
||||
* of about 512 Mi code units. A lower value only refuses more requests, so the
|
||||
* setting has no floor.
|
||||
*/
|
||||
export const MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES = 1024 * 1024 * 1024;
|
||||
|
||||
/** Default absolute deadline for Ghidra import, analysis, bridge, and readiness. */
|
||||
export const DEFAULT_GHIDRA_STARTUP_TIMEOUT_MS = 330_000;
|
||||
|
||||
|
||||
@@ -16,6 +16,8 @@ import {
|
||||
CAPTURE_DECODER_SEMI_SPACE_MIB,
|
||||
MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB,
|
||||
DEFAULT_JADX_OPERATION_TIMEOUT_MS,
|
||||
DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
} from "./analysisResourceLimits.js";
|
||||
import {
|
||||
JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS,
|
||||
@@ -192,6 +194,16 @@ export const ANALYSIS_SETTINGS = {
|
||||
operations: ["inspect_wasm_artifact"],
|
||||
parameter: "command_timeout_ms",
|
||||
},
|
||||
"native.matching_representation_bytes": {
|
||||
environment: "REA_NATIVE_MATCHING_REPRESENTATION_BYTES",
|
||||
defaultValue: DEFAULT_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
minimum: 1,
|
||||
maximum: MAXIMUM_NATIVE_MATCHING_REPRESENTATION_BYTES,
|
||||
unit: "bytes",
|
||||
appliesAt: "next_operation",
|
||||
provider: "rea-function-comparison",
|
||||
operations: ["match_native_functions"],
|
||||
},
|
||||
} as const satisfies Record<AnalysisSettingId, SettingDefinition>;
|
||||
export const analysisSettingIds = Object.keys(
|
||||
ANALYSIS_SETTINGS,
|
||||
|
||||
@@ -11,6 +11,7 @@ export interface AnalysisSettingValues {
|
||||
readonly "javascript.analysis_timeout_ms": number;
|
||||
readonly "javascript.max_heap_mb": number;
|
||||
readonly "wasm.command_timeout_ms": number;
|
||||
readonly "native.matching_representation_bytes": number;
|
||||
}
|
||||
export type AnalysisSettingId = keyof AnalysisSettingValues;
|
||||
export interface AnalysisSettingsSnapshot {
|
||||
|
||||
@@ -9,21 +9,34 @@ import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
import { recordEvidenceSources } from "../application/investigation/recordEvidenceSources.js";
|
||||
import { ok } from "../domain/result.js";
|
||||
import type { Logger } from "pino";
|
||||
import { runtimeNativeMatchingAllowance } from "../config/runtimeAnalysisSettings.js";
|
||||
|
||||
/** Bind one named contract to the same retained-Evidence workflow used by CLI. */
|
||||
export const registerNativeFunctionMatchingTool = (
|
||||
server: EvidenceMcpServer,
|
||||
session: BinarySessionPort,
|
||||
logger: Logger,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
): void => {
|
||||
const contract = toolContract("match_native_functions");
|
||||
// Reads each accepted operation's settings snapshot over the launch value.
|
||||
const resolveAllowance = runtimeNativeMatchingAllowance(
|
||||
environment,
|
||||
server.analysisSettings?.read,
|
||||
);
|
||||
const evidenceById = session.evidenceForAnalysis.bind(session);
|
||||
const recordEvidence: BinarySessionPort["recordEvidence"] = (evidence) =>
|
||||
session.recordEvidence(evidence);
|
||||
const handler = createEvidenceToolHandler(
|
||||
server,
|
||||
async (input, options) => {
|
||||
const result = await matchNativeFunctions(input, options, evidenceById);
|
||||
const allowance = resolveAllowance();
|
||||
if (!allowance.ok) return allowance;
|
||||
const result = await matchNativeFunctions(
|
||||
input,
|
||||
{ ...options, representationAllowance: allowance.value },
|
||||
evidenceById,
|
||||
);
|
||||
if (!result.ok) return result;
|
||||
const recorded = recordEvidenceSources(
|
||||
recordEvidence,
|
||||
|
||||
@@ -411,7 +411,12 @@ export const registerSessionTools = (
|
||||
registerProcessComparisonTool(server, session, compareContract);
|
||||
registerArtifactComparisonTool(server, session, compareArtifactsContract);
|
||||
registerFunctionComparisonTool(server, session, compareFunctionsContract);
|
||||
registerNativeFunctionMatchingTool(server, session, logger);
|
||||
registerNativeFunctionMatchingTool(
|
||||
server,
|
||||
session,
|
||||
logger,
|
||||
options.environment,
|
||||
);
|
||||
registerBundleComparisonTool(
|
||||
server,
|
||||
session,
|
||||
|
||||
@@ -7,6 +7,7 @@ import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
|
||||
import { join } from "node:path";
|
||||
import { connectLocalToolsMcp } from "../fixtures/localToolsMcp.js";
|
||||
import { asarHeaderBytes } from "../fixtures/asarHeader.js";
|
||||
import { nativeMatchingNamedInventory } from "../fixtures/nativeFunctionMatching.js";
|
||||
|
||||
it("advertises exact contracts and recovers a full large-index archive over one MCP connection", async () => {
|
||||
const root = await createTestTempDirectory("rea-runtime-settings-");
|
||||
@@ -171,3 +172,117 @@ it.each(["0", "2147483648"])(
|
||||
},
|
||||
30000,
|
||||
);
|
||||
|
||||
it("lowers, restores and seeds the native matching allowance over MCP", async () => {
|
||||
const rows = Array.from({ length: 200 }, (_, i) => ({
|
||||
address: `0x${(0x1000 + i * 16).toString(16)}`,
|
||||
value: `function_${i}`,
|
||||
}));
|
||||
const request = {
|
||||
left: nativeMatchingNamedInventory("1", rows),
|
||||
right: nativeMatchingNamedInventory("2", rows),
|
||||
};
|
||||
const limitOf = (result: {
|
||||
content: readonly { type: string; text?: string }[];
|
||||
}) => {
|
||||
const text = result.content.find((item) => item.type === "text")?.text;
|
||||
if (text === undefined) throw new Error("Missing projected error");
|
||||
return JSON.parse(text).error.details.limit as unknown;
|
||||
};
|
||||
const runtime = await connectLocalToolsMcp();
|
||||
try {
|
||||
expect(
|
||||
(await runtime.call("match_native_functions", request)).isError,
|
||||
).not.toBe(true);
|
||||
await runtime.call("update_analysis_settings", {
|
||||
changes: [
|
||||
{
|
||||
setting: "native.matching_representation_bytes",
|
||||
action: "set",
|
||||
value: 4096,
|
||||
},
|
||||
],
|
||||
});
|
||||
const refused = await runtime.call("match_native_functions", request);
|
||||
expect(refused.isError).toBe(true);
|
||||
expect(limitOf(refused)).toMatchObject({
|
||||
setting: "native.matching_representation_bytes",
|
||||
effective_value: 4096,
|
||||
source: "runtime",
|
||||
agent_editable: true,
|
||||
});
|
||||
await runtime.call("update_analysis_settings", {
|
||||
changes: [
|
||||
{ setting: "native.matching_representation_bytes", action: "reset" },
|
||||
],
|
||||
});
|
||||
expect(
|
||||
(await runtime.call("match_native_functions", request)).isError,
|
||||
).not.toBe(true);
|
||||
} finally {
|
||||
await runtime.client.close();
|
||||
}
|
||||
const seeded = await connectLocalToolsMcp({
|
||||
REA_NATIVE_MATCHING_REPRESENTATION_BYTES: "4096",
|
||||
});
|
||||
try {
|
||||
expect(
|
||||
(
|
||||
await seeded.call("inspect_analysis_settings", {
|
||||
operation: "match_native_functions",
|
||||
})
|
||||
).structuredContent,
|
||||
).toMatchObject({
|
||||
settings: [
|
||||
{
|
||||
setting: "native.matching_representation_bytes",
|
||||
value: 4096,
|
||||
launch_value: 4096,
|
||||
default_value: 100663296,
|
||||
source: "setting",
|
||||
environment: "REA_NATIVE_MATCHING_REPRESENTATION_BYTES",
|
||||
},
|
||||
],
|
||||
});
|
||||
const refused = await seeded.call("match_native_functions", request);
|
||||
expect(refused.isError).toBe(true);
|
||||
expect(limitOf(refused)).toMatchObject({
|
||||
effective_value: 4096,
|
||||
source: "setting",
|
||||
});
|
||||
} finally {
|
||||
await seeded.client.close();
|
||||
}
|
||||
// An invalid seed is refused by matching alone; MCP stays available and a
|
||||
// runtime override repairs the effective allowance.
|
||||
const invalid = await connectLocalToolsMcp({
|
||||
REA_NATIVE_MATCHING_REPRESENTATION_BYTES: "0",
|
||||
});
|
||||
try {
|
||||
expect((await invalid.call("binary_session", {})).isError).not.toBe(true);
|
||||
const refused = await invalid.call("match_native_functions", request);
|
||||
expect(refused.isError).toBe(true);
|
||||
const text = refused.content.find((item) => item.type === "text");
|
||||
if (text?.type !== "text") throw new Error("Missing projected error");
|
||||
expect(JSON.parse(text.text).error).toMatchObject({
|
||||
code: "configuration_invalid",
|
||||
details: {
|
||||
settings: [{ setting: "REA_NATIVE_MATCHING_REPRESENTATION_BYTES" }],
|
||||
},
|
||||
});
|
||||
await invalid.call("update_analysis_settings", {
|
||||
changes: [
|
||||
{
|
||||
setting: "native.matching_representation_bytes",
|
||||
action: "set",
|
||||
value: 100663296,
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(
|
||||
(await invalid.call("match_native_functions", request)).isError,
|
||||
).not.toBe(true);
|
||||
} finally {
|
||||
await invalid.client.close();
|
||||
}
|
||||
}, 30000);
|
||||
|
||||
Reference in New Issue
Block a user