mirror/rea
1
0
Fork 0
mirror of https://github.com/morluto/rea synced 2026-10-25 22:08:11 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(native): preserve endpoint context in function comparisons

Browse Source
This commit is contained in:
morluto
2026-10-24 01:33:24 +08:00
parent 211e3533b3
commit edf7b3d32c
12 changed files with 638 additions and 117 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/mcp-contracts.md
+33 -25
View File
@@ -1164,35 +1164,43 @@ includes all dimension limitations. Equal CFGs establish only equal observed
block sizes and successor relationships. Text from different provider
identities remains unknown.
The `references` dimension preserves each edge's direction, endpoint identity
and available kind facts. When the subject digests match, the current comparator
uses exact raw endpoint coordinates, including the address space; hexadecimal
case and zero padding do not change identity. Resolving an endpoint to a
containing procedure does not replace the raw coordinate. Unparseable
coordinates stay unknown.
The `calls` and `references` dimensions share endpoint identity and name
ambiguity rules. Calls retain procedure entry coordinates and direction;
references retain each edge's raw source and target coordinates, direction and
available kind facts. Resolving a reference endpoint to a containing procedure
does not replace its raw coordinate. Hexadecimal case and zero padding do not
change coordinates; address spaces remain distinct.
Coordinate equality requires equal artifact digests and formats, a known equal
architecture, and matching provider-owned native coordinate commitments. Those
commitments retain the exact selected Mach-O image where available and a digest
of the loader's mapping inputs. They exclude unrelated analysis settings, so
changing an analyzer option alone does not invalidate coordinate correspondence.
Equal coordinates in different slices or mappings cannot establish equality.
Hopper's default executable profile and Ghidra's supported deterministic imports
publish these commitments. Hopper loader overrides and attached databases, IDA
profiles, Ghidra universal-image imports without an exact slice commitment, and
older or imported Evidence without a commitment leave coordinate identity
unknown. Unparseable coordinates also remain unknown.
Across different artifacts, endpoints use nonempty stable procedure names.
Missing or address-generated names remain unknown. A name observed at multiple
procedure entries across either dossier's call and reference facets retains its
label constraint but cannot establish equal endpoints. Same-container records
without a common coordinate context likewise cannot establish equality from
matching labels. Procedure labels do not carry authoritative symbol-origin
facts, so recognition of generated names remains conservative and cannot prove
whether every provider label was generated or explicitly named.
When the subject digests differ, reference endpoints use stable procedure names.
Missing or address-generated names remain unknown. A name observed at multiple procedure
entries retains its label constraint but cannot establish equal endpoints.
Pairing preserves duplicate counts and considers alternative compatible
pairings; array order never identifies an edge. Compatible records with
unknown endpoint identity or kind facts remain `unknown`. Independent known
pairings; array order never identifies an edge. Compatible records with unknown
endpoint identity or reference kind facts remain `unknown`. Independent known
facts that contradict every possible pairing still establish `changed`.
Source Evidence retains the original addresses, names and unavailable facts.
Coordinate identity requires a common executable context: artifact identity,
selected image or architecture slice, address space, and applicable load mapping.
Equal coordinates in different slices of one container do not establish equal
endpoints. The current digest-only selection does not enforce that context and
can report references unchanged across distinct slices
([#6502](https://github.com/morluto/rea/issues/6502)).
The `calls` dimension separately compares direction and procedure names without
retaining endpoint addresses. Distinct same-named procedures can therefore
compare unchanged ([#6853](https://github.com/morluto/rea/issues/6853)). Check
the source dossiers' endpoint identities before using either affected dimension
as a reconstruction proof; another dimension reporting a change does not repair
the selected dimension's false equality.
Reconstruction verification uses the selected dimension's status: a same-name
call retargeting in a common context fails a calls claim; an unresolved slice or
mapping cannot pass it. Empty function labels leave `function_match` ambiguous
and the identity dimension unknown.
### Comparison input shapes
src/domain/functionComparison.test.ts
+31 -4
View File
@@ -4,6 +4,8 @@ import { describe, expect, it } from "vitest";
import { FUNCTION_COMPARISON_EXAMPLE } from "../contracts/functionComparisonExample.js";
import { enhancedInputSchemas } from "../contracts/enhancedInputs.js";
import { functionComparisonResultSchema } from "./functionComparisonSchemas.js";
import { createFunctionCoordinateContext } from "./native/functionCoordinateContext.js";
import { createAnalysisProfile } from "./analysisProfile.js";
import { createEvidence, type Evidence } from "./evidence.js";
import { functionDossierSchema } from "./hopperValues.js";
import { canonicalJson } from "./comparisonSemantics.js";
@@ -115,10 +117,26 @@ const observe = (
path: `/tmp/function-${digit}`,
sha256: digit.repeat(64),
format: "mach-o",
architecture: "arm64",
},
{ id: providerId, name: providerId, version: "1" },
{
operation: "analyze_function",
analysisProfile: createAnalysisProfile(
{ id: providerId, name: providerId, version: "1" },
{
architecture: "arm64",
loader: "auto-from-header",
function_coordinate_context: createFunctionCoordinateContext({
loader: "auto-from-header",
language_id: "auto-from-header",
compiler_spec_id: "auto-default",
load_segment: null,
address_coordinates: null,
entry_point: null,
}),
},
),
parameters: enhancedInputSchemas.analyze_function.parse({
procedure: "main",
}),
@@ -338,7 +356,7 @@ describe("function comparison address-derived call names", () => {
expect(calls).toMatchObject({
status: "unknown",
limitations: expect.arrayContaining([
"Address-derived callee or caller names cannot be matched safely.",
"Unnamed relationship endpoints and generated procedure names have no stable identity outside a common executable coordinate context.",
`Right: ${diagnostic}`,
]),
});
@@ -494,7 +512,12 @@ describe("function collection collation ties", () => {
(name, index) => procedure(`0x${(0x2000 + index).toString(16)}`, name),
);
const canonicalProjection = callees
.map(({ name }) => ({ direction: "out", name }))
.map(({ name }) => ({
direction: "out",
source: null,
target: { kind: "name", name, ambiguous: false },
kind: null,
}))
.sort((left, right) =>
compareUnicodeCodePoints(canonicalJson(left), canonicalJson(right)),
);
@@ -612,17 +635,21 @@ describe("function comparison ordering controls", () => {
functionSnapshotFixture(makeEvidence("e", rightWithoutNativeApi)),
);
for (const dimension of ["calls", "strings_names"] as const)
for (const dimension of ["strings_names"] as const)
expect(
comparison.dimensions.find((item) => item.dimension === dimension),
).toMatchObject({ status: "unchanged" });
// Reference and call facets report the same name at different entries.
expect(
comparison.dimensions.find((item) => item.dimension === "calls"),
).toMatchObject({ status: "unknown" });
// Unnamed endpoints in different artifacts have no stable identity.
expect(
comparison.dimensions.find((item) => item.dimension === "references"),
).toMatchObject({
status: "unknown",
limitations: expect.arrayContaining([
expect.stringContaining("Unnamed reference endpoints"),
expect.stringContaining("Unnamed relationship endpoints"),
]),
});
expect(comparison.status).toBe("unknown");
src/domain/functionComparisonDimensions.ts
+26 -39
View File
@@ -1,12 +1,15 @@
import { canonicalJson } from "./comparisonSemantics.js";
import { compareFunctionReferences } from "./functionReferenceComparison.js";
import {
compareFunctionRelationships,
type FunctionRelationshipComparison,
} from "./functionRelationshipComparison.js";
import { diffLines } from "diff";
import type { FunctionSnapshot } from "./functionDossierEvidence.js";
import {
commentProjection,
identityProjection,
isAutoName,
hasStableFunctionName,
nameAndSwiftProjection,
normalizeCfg,
referencedStringProjection,
@@ -33,6 +36,7 @@ export const compareDimensions = (
right: FunctionSnapshot,
context: ComparisonContext,
): FunctionDimension[] => {
const relationships = compareFunctionRelationships(left, right);
return [
compareIdentity(left, right, context),
compareText(
@@ -48,8 +52,8 @@ export const compareDimensions = (
context,
),
compareComments(left, right, context),
compareCalls(left, right, context),
compareReferences(left, right, context),
compareCalls(left, right, relationships.calls, context),
compareReferences(relationships.references, context),
compareStringsAndNames(left, right, context),
compareCfg(left, right, context),
].map((result) => {
@@ -521,11 +525,9 @@ const swiftLiteralCoverage = (
: "complete";
const compareReferences = (
left: FunctionSnapshot,
right: FunctionSnapshot,
compared: FunctionRelationshipComparison,
context: ComparisonContext,
): FunctionDimension => {
const compared = compareFunctionReferences(left, right);
if (compared.status === "unknown")
return unresolvedDimension({
dimension: "references",
@@ -548,8 +550,8 @@ const compareIdentity = (
context: ComparisonContext,
): FunctionDimension => {
if (
isAutoName(left.dossier.procedure.name) ||
isAutoName(right.dossier.procedure.name)
!hasStableFunctionName(left.dossier.procedure.name) ||
!hasStableFunctionName(right.dossier.procedure.name)
)
return unresolvedDimension({
dimension: "identity",
@@ -557,7 +559,7 @@ const compareIdentity = (
leftCount: null,
rightCount: null,
limitations: [
"Address-derived function names are not stable cross-version identity.",
"Empty or address-derived function names are not stable cross-version identity.",
],
});
return compareValues(
@@ -571,10 +573,9 @@ const compareIdentity = (
const compareCalls = (
left: FunctionSnapshot,
right: FunctionSnapshot,
comparison: FunctionRelationshipComparison,
context: ComparisonContext,
): FunctionDimension => {
const leftValues = callProjection(left);
const rightValues = callProjection(right);
const coverageLimitations = [
...left.callCoverageBlockers.map((reason) => `Left: ${reason}`),
...right.callCoverageBlockers.map((reason) => `Right: ${reason}`),
@@ -587,41 +588,27 @@ const compareCalls = (
return unresolvedDimension({
dimension: "calls",
links: context.links,
leftCount: leftValues.length,
rightCount: rightValues.length,
leftCount: comparison.leftCount,
rightCount: comparison.rightCount,
limitations: [...coverageLimitations, ...coverageDiagnostics],
});
if ([...leftValues, ...rightValues].some(({ name }) => isAutoName(name)))
if (comparison.status === "unknown")
return unresolvedDimension({
dimension: "calls",
links: context.links,
leftCount: leftValues.length,
rightCount: rightValues.length,
limitations: [
"Address-derived callee or caller names cannot be matched safely.",
...coverageDiagnostics,
],
leftCount: comparison.leftCount,
rightCount: comparison.rightCount,
limitations: [...comparison.limitations, ...coverageDiagnostics],
});
return compareValues(
"calls",
sorted(leftValues),
sorted(rightValues),
context,
coverageDiagnostics,
);
return dimensionResult({
dimension: "calls",
...comparison,
...context,
textDelta: null,
limitations: [...comparison.limitations, ...coverageDiagnostics],
});
};
const callProjection = (snapshot: FunctionSnapshot) => [
...snapshot.dossier.callers.map(({ name }) => ({
direction: "in" as const,
name,
})),
...snapshot.dossier.callees.map(({ name }) => ({
direction: "out" as const,
name,
})),
];
const compareCfg = (
left: FunctionSnapshot,
right: FunctionSnapshot,
src/domain/functionComparisonNormalization.ts
+6 -1
View File
@@ -19,7 +19,8 @@ export const functionMatch = (
): FunctionComparisonResult["function_match"] => {
const leftName = left.dossier.procedure.name;
const rightName = right.dossier.procedure.name;
const stableNames = !isAutoName(leftName) && !isAutoName(rightName);
const stableNames =
hasStableFunctionName(leftName) && hasStableFunctionName(rightName);
if (stableNames && leftName === rightName)
return {
status: "matched",
@@ -227,6 +228,10 @@ export const sorted = <T>(values: readonly T[]): readonly T[] =>
export const isAutoName = (name: string): boolean =>
/^(?:sub_[0-9a-f]+|fcn\.[0-9a-f]+|FUN_[0-9a-f]+)$/iu.test(name);
/** A display label must be nonempty before it can constrain named correspondence. */
export const hasStableFunctionName = (name: string): boolean =>
name.length > 0 && !isAutoName(name);
const parseAddress = (value: string) => {
const parsed = nativeAddressCoordinateInputSchema.safeParse(value);
return parsed.success ? parsed.data : null;
src/domain/functionReferenceComparison.ts → src/domain/functionRelationshipComparison.ts
+107 -38
View File
@@ -1,6 +1,11 @@
import { functionCoordinateContextsEqual } from "./native/functionCoordinateContext.js";
import { canonicalJson } from "./comparisonSemantics.js";
import type { FunctionSnapshot } from "./functionDossierEvidence.js";
import { isAutoName, sorted } from "./functionComparisonNormalization.js";
import {
hasStableFunctionName,
isAutoName,
sorted,
} from "./functionComparisonNormalization.js";
import {
referenceFactsCompatible,
type ReferenceCompatibilityRow,
@@ -21,19 +26,23 @@ type Endpoint =
}
| {
readonly kind: "unknown";
readonly reason: "unparsed_address" | "missing_name" | "generated_name";
readonly reason:
| "unparsed_address"
| "missing_name"
| "generated_name"
| "coordinate_context";
readonly reported_address: string;
readonly reported_name: string | null;
};
interface ReferenceFact {
interface RelationshipFact {
readonly direction: "in" | "out";
readonly source: Endpoint;
readonly source: Endpoint | null;
readonly target: Endpoint;
readonly kind: KnownKind | null;
}
type ReferenceComparison = {
export type FunctionRelationshipComparison = {
readonly leftCount: number;
readonly rightCount: number;
readonly limitations: readonly string[];
@@ -41,8 +50,8 @@ type ReferenceComparison = {
| { readonly status: "unknown" }
| {
readonly status: "changed" | "unchanged";
readonly left: readonly ReferenceFact[];
readonly right: readonly ReferenceFact[];
readonly left: readonly RelationshipFact[];
readonly right: readonly RelationshipFact[];
}
);
@@ -66,6 +75,11 @@ const ambiguousProcedureNames = (
entries.set(procedure.name, address.data);
};
observe(snapshot.dossier.procedure);
for (const procedure of [
...snapshot.dossier.callers,
...snapshot.dossier.callees,
])
observe(procedure);
for (const edge of snapshot.dossier.incoming_references) {
observe(edge.source_procedure);
observe(edge.target_procedure);
@@ -81,10 +95,10 @@ const ambiguousProcedureNames = (
const endpoint = (
procedure: Reference["source_procedure"],
address: string,
sameArtifact: boolean,
coordinateContext: "common" | "cross-artifact" | "unknown",
ambiguousNames: ReadonlySet<string>,
): Endpoint => {
if (sameArtifact) {
if (coordinateContext === "common") {
const parsed = nativeAddressInputSchema.safeParse(address);
if (parsed.success) return { kind: "address", address: parsed.data };
return {
@@ -94,11 +108,14 @@ const endpoint = (
reported_name: procedure?.name ?? null,
};
}
if (
procedure === null ||
procedure.name.length === 0 ||
isAutoName(procedure.name)
)
if (coordinateContext === "unknown")
return {
kind: "unknown",
reason: "coordinate_context",
reported_address: address,
reported_name: procedure?.name ?? null,
};
if (procedure === null || !hasStableFunctionName(procedure.name))
return {
kind: "unknown",
reason:
@@ -117,10 +134,30 @@ const endpoint = (
const project = (
snapshot: FunctionSnapshot,
sameArtifact: boolean,
coordinateContext: "common" | "cross-artifact" | "unknown",
ambiguousNames: ReadonlySet<string>,
): ReferenceFact[] => {
const facts: ReferenceFact[] = [];
dimension: "calls" | "references",
): RelationshipFact[] => {
const facts: RelationshipFact[] = [];
if (dimension === "calls") {
for (const [direction, procedures] of [
["in", snapshot.dossier.callers],
["out", snapshot.dossier.callees],
] as const)
for (const procedure of procedures)
facts.push({
direction,
source: null,
target: endpoint(
procedure,
procedure.address,
coordinateContext,
ambiguousNames,
),
kind: null,
});
return facts;
}
const collect = (
references: readonly Reference[],
direction: "in" | "out",
@@ -140,13 +177,13 @@ const project = (
source: endpoint(
edge.source_procedure,
edge.source_address,
sameArtifact,
coordinateContext,
ambiguousNames,
),
target: endpoint(
edge.target_procedure,
edge.target_address,
sameArtifact,
coordinateContext,
ambiguousNames,
),
kind,
@@ -159,8 +196,8 @@ const project = (
};
/** Ambiguity blocks equality, but a known label can still contradict another label. */
const endpointKey = (value: Endpoint): string | null => {
if (value.kind === "unknown") return null;
const endpointKey = (value: Endpoint | null): string | null => {
if (value === null || value.kind === "unknown") return null;
return canonicalJson(
value.kind === "address"
? { address: value.address }
@@ -169,7 +206,7 @@ const endpointKey = (value: Endpoint): string | null => {
};
const constraints = (
facts: readonly ReferenceFact[],
facts: readonly RelationshipFact[],
): ReferenceCompatibilityRow[] =>
facts.map((fact) => ({
direction: fact.direction,
@@ -179,39 +216,71 @@ const constraints = (
}));
/** Compare parsed endpoint and kind facts without replacing partial knowledge with absence. */
export const compareFunctionReferences = (
export const compareFunctionRelationships = (
left: FunctionSnapshot,
right: FunctionSnapshot,
): ReferenceComparison => {
): {
readonly calls: FunctionRelationshipComparison;
readonly references: FunctionRelationshipComparison;
} => {
// Provider coordinate commitments own loader/slice semantics. Missing context is not equality.
const sameArtifact =
left.subject.digest.sha256 === right.subject.digest.sha256;
const ambiguousNames = sameArtifact
? new Set<string>()
: ambiguousProcedureNames([left, right]);
const leftFacts = project(left, sameArtifact, ambiguousNames);
const rightFacts = project(right, sameArtifact, ambiguousNames);
const coordinateContext = !sameArtifact
? "cross-artifact"
: left.subject.format === right.subject.format &&
left.subject.architecture !== null &&
left.subject.architecture === right.subject.architecture &&
functionCoordinateContextsEqual(
left.evidence.analysis_profile,
right.evidence.analysis_profile,
)
? "common"
: "unknown";
const ambiguousNames = ambiguousProcedureNames([left, right]);
const compare = (dimension: "calls" | "references") =>
compareRelationshipFacts(
project(left, coordinateContext, ambiguousNames, dimension),
project(right, coordinateContext, ambiguousNames, dimension),
dimension,
coordinateContext,
);
return { calls: compare("calls"), references: compare("references") };
};
const compareRelationshipFacts = (
leftFacts: readonly RelationshipFact[],
rightFacts: readonly RelationshipFact[],
dimension: "calls" | "references",
coordinateContext: "common" | "cross-artifact" | "unknown",
): FunctionRelationshipComparison => {
let unknownIdentity = false;
let ambiguousIdentity = false;
let unknownKinds = false;
for (const facts of [leftFacts, rightFacts]) {
for (const fact of facts) {
unknownKinds ||= fact.kind === null;
unknownKinds ||= dimension === "references" && fact.kind === null;
for (const value of [fact.source, fact.target]) {
unknownIdentity ||= value.kind === "unknown";
ambiguousIdentity ||= value.kind === "name" && value.ambiguous;
unknownIdentity ||= value !== null && value.kind === "unknown";
ambiguousIdentity ||=
value !== null && value.kind === "name" && value.ambiguous;
}
}
}
const limitations: string[] = [];
if (unknownIdentity)
if (unknownIdentity && coordinateContext !== "unknown")
limitations.push(
sameArtifact
? "Reference endpoint coordinates could not be parsed, so their identity remains unknown."
: "Unnamed reference endpoints and generated procedure names have no stable identity across different artifacts.",
coordinateContext === "common"
? "Relationship endpoint coordinates could not be parsed, so their identity remains unknown."
: "Unnamed relationship endpoints and generated procedure names have no stable identity outside a common executable coordinate context.",
);
if (coordinateContext === "unknown" && (unknownIdentity || ambiguousIdentity))
limitations.push(
"A common executable coordinate context is not established: artifact format, known architecture and provider coordinate commitments must agree.",
);
if (ambiguousIdentity)
limitations.push(
"A reference name does not uniquely identify an observed procedure entry, so matching names do not establish equal endpoints.",
"A relationship name cannot establish a unique comparable procedure entry, so matching names do not establish equal endpoints.",
);
if (unknownKinds)
limitations.push(
@@ -231,7 +300,7 @@ export const compareFunctionReferences = (
left: sorted(leftFacts),
right: sorted(rightFacts),
limitations: [
"Known reference facts cannot be paired without contradicting an endpoint or reference kind.",
"Known relationship facts cannot be paired without contradicting an endpoint or reference kind.",
...limitations,
],
};
src/domain/native/functionCoordinateContext.ts
+64
View File
@@ -0,0 +1,64 @@
import { z } from "zod";
import { digestCanonicalValue } from "../canonicalDigest.js";
import { digestSchema } from "../digests.js";
import type { AnalysisProfileCommitment } from "../analysisProfile.js";
import type { JsonValue } from "../jsonValue.js";
import {
machoTargetSelectionSchema,
sameMachoTargetSelection,
type MachoTargetSelection,
} from "../apple/machoTargetSelection.js";
/** Provider-owned native image selection and deterministic loader mapping. */
const functionCoordinateContextSchema = z.strictObject({
kind: z.literal("native-function-coordinate-context-v1"),
selected_image: machoTargetSelectionSchema.nullable(),
mapping_sha256: digestSchema,
});
/** The provider selects only inputs affecting its executable coordinates. */
export const createFunctionCoordinateContext = (
mapping: Readonly<Record<string, JsonValue>>,
selectedImage: MachoTargetSelection | null = null,
) =>
functionCoordinateContextSchema.parse({
kind: "native-function-coordinate-context-v1",
selected_image: selectedImage,
mapping_sha256: digestCanonicalValue(
mapping,
"Native function coordinate mapping",
),
});
/** Missing or malformed commitments cannot establish coordinate applicability. */
export const functionCoordinateContextsEqual = (
left: AnalysisProfileCommitment | null,
right: AnalysisProfileCommitment | null,
): boolean => {
if (
left === null ||
right === null ||
left.provider.id !== right.provider.id ||
left.provider.name !== right.provider.name ||
left.provider.version !== right.provider.version
)
return false;
const leftContext = functionCoordinateContextSchema.safeParse(
left.parameters.function_coordinate_context,
);
const rightContext = functionCoordinateContextSchema.safeParse(
right.parameters.function_coordinate_context,
);
return (
leftContext.success &&
rightContext.success &&
leftContext.data.mapping_sha256 === rightContext.data.mapping_sha256 &&
(leftContext.data.selected_image === null
? rightContext.data.selected_image === null
: sameMachoTargetSelection(
leftContext.data.selected_image,
rightContext.data.selected_image ?? undefined,
))
);
};
src/domain/native/functionStructuralCandidate.ts
+2 -4
View File
@@ -2,7 +2,7 @@ import type { Evidence } from "../evidence.js";
import type { FunctionDossier } from "../hopperValues.js";
import { digestCanonicalValue } from "../canonicalDigest.js";
import {
isAutoName,
hasStableFunctionName,
normalizeCfg,
} from "../functionComparisonNormalization.js";
import { assessFunctionCallCoverage } from "../functionCallCoverage.js";
@@ -29,9 +29,7 @@ export const functionStructuralCandidate = (
if (
cfg === null ||
cfg.length === 0 ||
dossier.callees.some(
(callee) => isAutoName(callee.name) || callee.name.length === 0,
)
dossier.callees.some((callee) => !hasStableFunctionName(callee.name))
)
return null;
const calls = dossier.callees
src/ghidra/GhidraAnalysisProfile.ts
+29 -2
View File
@@ -3,6 +3,8 @@ import type {
AnalysisProfileResolution,
ProviderIdentity,
} from "../application/AnalysisProvider.js";
import { createFunctionCoordinateContext } from "../domain/native/functionCoordinateContext.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { createAnalysisProfile } from "../domain/analysisProfile.js";
import type { GhidraLanguageOverride } from "../config/ghidraLanguageOverride.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
@@ -105,6 +107,8 @@ export const resolveGhidraAnalysisProfile = async (
: null;
if (pspExtension !== null && !pspExtension.ok) return pspExtension;
const provider = { ...identity, version: installation.providerVersion };
const createProfile = (parameters: Readonly<Record<string, JsonValue>>) =>
createGhidraProfile(provider, parameters, target);
const analyzerParameters =
disabledAnalyzers === undefined || disabledAnalyzers.length === 0
? { analyzer_preset: "ghidra-default" }
@@ -126,7 +130,7 @@ export const resolveGhidraAnalysisProfile = async (
const language = ghidraRawBinaryLanguage(target.architecture);
return Promise.resolve(
ok({
profile: createAnalysisProfile(provider, {
profile: createProfile({
target_kind: target.kind,
target_format: target.format,
architecture: target.architecture,
@@ -208,7 +212,7 @@ export const resolveGhidraAnalysisProfile = async (
);
return Promise.resolve(
ok({
profile: createAnalysisProfile(provider, {
profile: createProfile({
target_kind: target.kind,
target_format: target.format,
...(target.format === "pe"
@@ -375,3 +379,26 @@ export const sameGhidraDisabledAnalyzers = (
? left === right
: left.length === right.length &&
left.every((name, index) => name === right[index]);
/** Keep coordinate-affecting import facts with the provider that interprets them. */
const createGhidraProfile = (
provider: Parameters<typeof createAnalysisProfile>[0],
parameters: Readonly<Record<string, JsonValue>>,
target: BinaryTarget,
) =>
createAnalysisProfile(provider, {
...parameters,
// Ghidra does not currently commit an exact universal-container slice selection.
...(target.kind === "executable" && target.format !== "mach-o-universal"
? {
function_coordinate_context: createFunctionCoordinateContext({
loader: parameters.loader ?? null,
language_id: parameters.language_id ?? null,
compiler_spec_id: parameters.compiler_spec_id ?? null,
load_segment: parameters.load_segment ?? null,
address_coordinates: parameters.address_coordinates ?? null,
entry_point: parameters.entry_point ?? null,
}),
}
: {}),
});
src/ghidra/GhidraProvider.behavior.test.ts
+46
View File
@@ -51,6 +51,10 @@ import { silentLogger } from "../logger.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
import {
createFunctionCoordinateContext,
functionCoordinateContextsEqual,
} from "../domain/native/functionCoordinateContext.js";
import { createAnalysisProfile } from "../domain/analysisProfile.js";
import { createAnalysisExecution } from "../application/AnalysisProvider.js";
import { AnalysisSnapshotCache } from "../application/binary/AnalysisSnapshotCache.js";
@@ -4114,3 +4118,45 @@ it.each([false, true])(
);
},
);
it("commits native coordinates independently of analyzer choices and preserves raw base identity", async () => {
const resolve = async (ghidra: GhidraProvider, target: BinaryTarget) => {
const resolved = await ghidra.resolveAnalysisProfile(target);
if (!resolved.ok) throw resolved.error;
if (resolved.value.profile === null)
throw new Error("expected native profile");
return resolved.value.profile;
};
const defaultProvider = provider();
const configured = configuredProvider({
REA_GHIDRA_DISABLED_ANALYZERS: "Decompiler Switch Analysis",
});
const target = executableTarget("elf", "x86_64");
const baseline = await resolve(defaultProvider, target);
const changedOptions = await resolve(configured, target);
expect(baseline.parameters.function_coordinate_context).toEqual(
createFunctionCoordinateContext({
loader: "auto-from-header",
language_id: "auto-from-header",
compiler_spec_id: "auto-default",
load_segment: null,
address_coordinates: null,
entry_point: null,
}),
);
expect(baseline.digest).not.toBe(changedOptions.digest);
expect(functionCoordinateContextsEqual(baseline, changedOptions)).toBe(true);
const raw = fixtureRawBinaryLoadImage().target;
const rawProfile = await resolve(defaultProvider, raw);
const shifted = await resolve(defaultProvider, {
...raw,
entryPoint: "0x8001000",
entryPointSource: "0x8001000",
});
expect(rawProfile.parameters.function_coordinate_context).toMatchObject({
kind: "native-function-coordinate-context-v1",
selected_image: null,
mapping_sha256: expect.any(String),
});
expect(functionCoordinateContextsEqual(rawProfile, shifted)).toBe(false);
});
src/hopper/HopperAnalysisProfile.test.ts
+20
View File
@@ -1,3 +1,5 @@
import { machoTargetSelectionSchema } from "../domain/apple/machoTargetSelection.js";
import { createFunctionCoordinateContext } from "../domain/native/functionCoordinateContext.js";
import { createHash } from "node:crypto";
import { PassThrough } from "node:stream";
import { writeFile } from "node:fs/promises";
@@ -96,6 +98,11 @@ it("prepares a verified thin arm64e slice for a FAT32 system container", async (
if (!result.ok) throw new Error(result.error.message);
const parameters = result.value.profile?.parameters;
expect(parameters).toMatchObject({
function_coordinate_context: createFunctionCoordinateContext(
{ loader_arguments: ["-l", "Mach-O", "--aarch64"] },
// The same typed selected-image fact is used by the producer and comparator.
machoTargetSelectionSchema.parse(parameters?.target_selection),
),
macho_container: "fat32",
prepared_image: {
method: "fat32_thin_slice",
@@ -157,6 +164,16 @@ it("commits a thin header selection only for derived Hopper loader arguments", a
cpu_subtype: 0x80000002,
container_table_sha256: null,
});
expect(
resolved.value.profile?.parameters.function_coordinate_context,
).toEqual(
createFunctionCoordinateContext(
{ loader_arguments: ["-l", "Mach-O", "--aarch64"] },
machoTargetSelectionSchema.parse(
resolved.value.profile?.parameters.target_selection,
),
),
);
expect(resolved.value.profile?.parameters.prepared_image).toEqual({
method: "thin_verified_copy",
byte_length: thin.length,
@@ -175,6 +192,9 @@ it("commits a thin header selection only for derived Hopper loader arguments", a
"target_selection",
);
expect(overridden.value.profile?.parameters).not.toHaveProperty(
"function_coordinate_context",
);
const changedSameSize = Buffer.from(thin);
changedSameSize.writeUInt8(
changedSameSize.readUInt8(changedSameSize.length - 1) ^ 1,
src/hopper/HopperAnalysisProfile.ts
+11
View File
@@ -6,6 +6,7 @@ import type {
RetainedResourceScope,
ProviderIdentity,
} from "../application/AnalysisProvider.js";
import { createFunctionCoordinateContext } from "../domain/native/functionCoordinateContext.js";
import { createAnalysisProfile } from "../domain/analysisProfile.js";
import {
isMachOFormat,
@@ -136,6 +137,16 @@ const resolveHopperAnalysisProfileInScope = async (
};
return ok({
profile: createAnalysisProfile(provider, {
...(target.kind === "executable" &&
options.loaderArgsOverride.length === 0 &&
(machoContainer === undefined || targetSelection !== undefined)
? {
function_coordinate_context: createFunctionCoordinateContext(
{ loader_arguments: [...loaderArgs] },
targetSelection ?? null,
),
}
: {}),
target_kind: target.kind,
target_format: target.format,
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
tests/boundary/mcp/functionComparisonReferenceFactsMcp.test.ts
+263 -4
View File
@@ -1,7 +1,12 @@
import type { Client } from "@modelcontextprotocol/client";
import { expect, it, onTestFinished } from "vitest";
import { createFunctionCoordinateContext } from "../../../src/domain/native/functionCoordinateContext.js";
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
import {
createAnalysisProfile,
type AnalysisProfileCommitment,
} from "../../../src/domain/analysisProfile.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
import { functionComparisonResultSchema } from "../../../src/domain/functionComparisonSchemas.js";
import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
@@ -12,6 +17,20 @@ import { createTestBinarySession } from "../../fixtures/binarySession.js";
const example = FUNCTION_COMPARISON_EXAMPLE.left;
const dossier = functionDossierSchema.parse(example.normalized_result);
const provider = { ...example.provider, version: "1" };
const mapping = {
loader: "auto-from-header",
language_id: "auto-from-header",
compiler_spec_id: "auto-default",
load_segment: null,
address_coordinates: null,
entry_point: null,
};
const profile = createAnalysisProfile(provider, {
architecture: "arm64",
...mapping,
function_coordinate_context: createFunctionCoordinateContext(mapping),
});
type Reference = (typeof dossier.outgoing_references)[number];
type Classification = "call" | "data" | "unknown";
@@ -90,17 +109,28 @@ const evidence = (
readonly incoming?: readonly Reference[];
readonly procedureAddress?: string;
readonly procedureName?: string;
readonly architecture?: "arm64" | "x86_64" | null;
readonly format?: "mach-o" | "mach-o-universal";
readonly profile?: AnalysisProfileCommitment | null;
readonly callees?: readonly ReturnType<typeof procedure>[];
readonly callers?: readonly ReturnType<typeof procedure>[];
} = {},
) =>
createEvidence(
{
path: `/fixture/function-reference/${id}`,
sha256: digest.repeat(64),
format: "mach-o",
format: options.format ?? "mach-o",
...(options.architecture === null
? {}
: { architecture: options.architecture ?? "arm64" }),
},
example.provider,
provider,
{
operation: "analyze_function",
...(options.profile === null
? {}
: { analysisProfile: options.profile ?? profile }),
parameters: { ...example.parameters, observation: id },
result: jsonValueSchema.parse({
...dossier,
@@ -109,6 +139,8 @@ const evidence = (
address: options.procedureAddress ?? dossier.procedure.address,
name: options.procedureName ?? dossier.procedure.name,
},
callees: [...(options.callees ?? [])],
callers: [...(options.callers ?? [])],
incoming_references: [...(options.incoming ?? [])],
outgoing_references: [...outgoing],
}),
@@ -163,8 +195,9 @@ const verifyReferences = async (
client: Client,
comparisonEvidenceId: string,
status: "pass" | "fail" | "unknown",
dimension: "references" | "calls" | "identity" = "references",
) => {
const claimId = `references-${comparisonEvidenceId}`;
const claimId = `${dimension}-${comparisonEvidenceId}`;
const response = await client.callTool({
name: "verify_reconstruction",
arguments: {
@@ -176,7 +209,7 @@ const verifyReferences = async (
claim_id: claimId,
title: "Reference facts agree",
comparison_evidence_id: comparisonEvidenceId,
dimension: "references",
dimension,
},
],
},
@@ -509,3 +542,229 @@ it("keeps ambiguous duplicate edge kinds unknown independent of input order", as
);
expectReferences(changed, "changed");
});
it("uses shared endpoint identity for selected calls and references claims", async () => {
const { client } = await createHarness();
const observed = (
id: string,
digest: string,
target: string,
name = "helper",
options: Parameters<typeof evidence>[3] = {},
) =>
evidence(
id,
digest,
[reference("0x1000", target, "call", { targetName: name })],
{
callees: [procedure(target, name)],
...options,
},
);
const baseline = observed("identity-baseline", "a", "0x1010");
const cases = [
{
name: "retarget",
right: observed("identity-retarget", "a", "0x1020"),
expected: "fail",
},
{
name: "rename",
right: observed("identity-rename", "a", "0x1010", "renamed"),
expected: "pass",
},
{
name: "relocated-symbol",
right: observed("identity-relocated", "b", "0x2020"),
expected: "pass",
},
{
name: "different-slice",
right: observed("identity-slice", "a", "0x1010", "helper", {
architecture: "x86_64",
profile: createAnalysisProfile(provider, {
...profile.parameters,
architecture: "x86_64",
}),
}),
expected: "unknown",
},
{
name: "unknown-architecture",
right: observed("identity-no-arch", "a", "0x1010", "helper", {
architecture: null,
}),
expected: "unknown",
},
{
name: "rename-without-mapping",
right: observed("identity-rename-no-profile", "a", "0x1010", "renamed", {
profile: null,
}),
expected: "unknown",
},
{
name: "unknown-mapping",
right: observed("identity-no-profile", "a", "0x1010", "helper", {
profile: null,
}),
expected: "unknown",
},
{
name: "unrelated-analysis-options",
right: observed("identity-options", "a", "0x1010", "helper", {
profile: createAnalysisProfile(provider, {
...profile.parameters,
analysis_options: {
disabled_analyzers: ["Decompiler Switch Analysis"],
},
}),
}),
expected: "pass",
},
{
name: "different-mapping",
right: observed("identity-mapping", "a", "0x1010", "helper", {
profile: createAnalysisProfile(provider, {
architecture: "arm64",
...mapping,
function_coordinate_context: createFunctionCoordinateContext({
...mapping,
load_segment: "0x2000",
}),
}),
}),
expected: "unknown",
},
{
name: "duplicate-name",
right: observed("identity-duplicate", "b", "0x1010", "helper", {
callees: [procedure("0x1010", "helper")],
incoming: [
reference("0x3030", "0x1000", "call", {
sourceName: "helper",
targetName: "main",
}),
],
}),
expected: "unknown",
},
{
name: "empty-name",
right: observed("identity-empty", "b", "0x1010", ""),
expected: "unknown",
},
] as const;
for (const scenario of cases) {
const comparison = await compare(client, baseline, scenario.right);
for (const dimension of ["calls", "references"] as const) {
// The duplicate-name reference also adds an edge, a known multiplicity change.
const expected =
scenario.name === "duplicate-name" && dimension === "references"
? "fail"
: scenario.expected;
await verifyReferences(
client,
comparison.evidence.evidence_id,
expected,
dimension,
);
}
}
});
it("keeps empty function labels ambiguous and selected identity unknown", async () => {
const { client } = await createHarness();
const comparison = await compare(
client,
evidence("empty-function-left", "a", [], { procedureName: "" }),
evidence("empty-function-right", "b", [], { procedureName: "" }),
);
expect(comparison.result.function_match.status).toBe("ambiguous");
expect(
comparison.result.dimensions.find(
({ dimension }) => dimension === "identity",
),
).toMatchObject({ status: "unknown" });
await verifyReferences(
client,
comparison.evidence.evidence_id,
"unknown",
"identity",
);
});
it("retains caller entries and independent contradictions beside unknown endpoints", async () => {
const { client } = await createHarness();
const left = evidence("caller-left", "a", [], {
callers: [procedure("0x1010", "caller")],
});
const right = evidence("caller-right", "a", [], {
callers: [procedure("0x1020", "caller")],
});
const retargeted = await compare(client, left, right);
await verifyReferences(
client,
retargeted.evidence.evidence_id,
"fail",
"calls",
);
const knownAndUnknown = (id: string, digest: string, name: string) =>
evidence(id, digest, [], {
callees: [
procedure("0x1010", name),
procedure("0x1010", name),
procedure("0x1020", "sub_1020"),
],
});
const mixed = await compare(
client,
knownAndUnknown("calls-mixed-left", "a", "known_left"),
knownAndUnknown("calls-mixed-right", "b", "known_right"),
);
await verifyReferences(client, mixed.evidence.evidence_id, "fail", "calls");
});
it("keeps equal coordinates in distinct same-architecture universal slices unknown", async () => {
const { client } = await createHarness();
const selection = {
kind: "macho-selected-image-v1" as const,
container: "fat" as const,
offset: 4096,
byte_length: 256,
cpu_type: 0x0100000c,
cpu_subtype: 0,
container_table_sha256: "c".repeat(64),
};
const atSlice = (id: string, selected: typeof selection) =>
evidence(id, "a", [reference("0x1000", "0x1010")], {
format: "mach-o-universal",
callees: [procedure("0x1010", "helper")],
profile: createAnalysisProfile(provider, {
architecture: "arm64",
function_coordinate_context: createFunctionCoordinateContext(
{ loader_arguments: ["-l", "Mach-O", "--aarch64"] },
selected,
),
}),
});
const left = atSlice("slice-arm64-left", selection);
for (const [id, selected, expected] of [
["same", selection, "pass"],
["different", { ...selection, offset: 8192, cpu_subtype: 2 }, "unknown"],
] as const) {
const comparison = await compare(
client,
left,
atSlice(`slice-arm64-${id}`, selected),
);
for (const dimension of ["calls", "references"] as const)
await verifyReferences(
client,
comparison.evidence.evidence_id,
expected,
dimension,
);
}
});