fix(native): admit universal Mach-O files without a host slice (#5501)

* fix(native): admit universal Mach-O files without a host slice

FAT Mach-O admission selected only the host's slice and rejected a valid
container whose slices are all foreign, so list-architectures returned
target_unavailable instead of the container's inventory. A thin foreign
image was already admitted.

Prefer the host's slice and otherwise select the first declared supported
slice. Remove the Ghidra installation probe's re-parse under other host
architectures, which existed only to work around the rejection.

Fixes #5404

* fix(native): select the first valid foreign FAT slice

Without a host slice, FAT selection took the first declared slice even when
it was too short for its Mach-O header, rejecting a container whose later
foreign slice is valid. The Ghidra decompiler probe then reported such a
foreign-only decompiler as unknown rather than incompatible. Select the first
foreign slice that can hold its header; a malformed host slice still fails.

* fix(native): select foreign FAT slices from every declaration

Foreign-slice fallback chose among architecture families after keeping only
each family's first declaration, so an undersized first arm64 subtype hid a
valid later arm64 subtype. Select from all declarations, and report the
deduplicated families separately. The host-slice rule is unchanged. Move the
FAT selection tests into their own block.

* fix(native): admit only unambiguous foreign FAT families

With no host slice, a family declared more than once, such as arm64 and
arm64e, has no provider-neutral choice: IDA analyzes the recorded slice while
Hopper prefers its own subtype. Select only families declared exactly once and
keep ambiguous foreign-only containers rejected, as on main.

* fix(ghidra): decide FAT decompiler compatibility from validated families

Slice selection now refuses a foreign-only FAT table that declares one family
more than once, such as arm64 and arm64e, and the decompiler probe reported that
refusal as unknown, so an x64 host accepted an arm64-only decompiler. When
selection fails, read the table's validated CPU families and report the
decompiler incompatible when they exclude the host. Selection and host
exclusion now have separate owners in binaryTarget.ts.

src/application/BinaryTargetResolver.ts

@@ -79,8 +79,9 @@ export interface BinaryTargetResolverOptions {
 
 /**
  * Resolve and classify a readable local target before a provider is selected.
- * FAT Mach-O inputs select only a host-compatible architecture so setup remains
- * non-interactive; unsupported or ambiguous inputs are returned as typed errors.
+ * FAT Mach-O inputs select the host-compatible architecture, otherwise their
+ * first declared supported slice, so setup remains non-interactive; unsupported
+ * or ambiguous inputs are returned as typed errors.
  */
 export const parseBinaryTarget = async (
   input: string,

src/domain/binaryTarget.test.ts

@@ -94,6 +94,30 @@ describe("executable header parsing", () => {
     });
   });
 
+  it.each([
+    Buffer.alloc(0),
+    Buffer.from([0x7f, 0x45, 0x4c, 0x46]),
+    fat([], 2),
+    // A FAT table declaring only unsupported CPU types has nothing to select.
+    fat([0x00000012]),
+    pe(0xffff),
+  ])("rejects malformed, truncated, or unsupported metadata", (bytes) => {
+    expect(parseExecutableHeader(bytes, "arm64").ok).toBe(false);
+  });
+
+  it("rejects malformed PE optional-header commitments", () => {
+    const wrongMagic = pe(0x8664);
+    wrongMagic.writeUInt16LE(0x10b, 64 + 24);
+    const truncatedDirectories = pe(0x8664);
+    truncatedDirectories.writeUInt16LE(112, 64 + 20);
+    truncatedDirectories.writeUInt32LE(16, 64 + 24 + 108);
+
+    expect(parseExecutableHeader(wrongMagic, "x64").ok).toBe(false);
+    expect(parseExecutableHeader(truncatedDirectories, "x64").ok).toBe(false);
+  });
+});
+
+describe("FAT Mach-O slice selection", () => {
   it("selects the host architecture from a FAT table", () => {
     const bytes = fat([0x01000007, 0x0100000c]);
     const arm = parseExecutableHeader(bytes, "arm64");
@@ -123,30 +147,36 @@ describe("executable header parsing", () => {
     });
   });
 
-  it("rejects FAT files without a host-compatible slice", () => {
+  it("selects the first valid declared slice when none matches the host", () => {
+    const bytes = fat([0x01000007, 0x0100000c]);
     expect(parseExecutableHeader(fat([0x01000007]), "arm64")).toMatchObject({
-      ok: false,
+      ok: true,
+      value: {
+        format: "mach-o-universal",
+        architecture: "x86_64",
+        availableArchitectures: ["x86_64"],
+      },
     });
-  });
-
-  it.each([
-    Buffer.alloc(0),
-    Buffer.from([0x7f, 0x45, 0x4c, 0x46]),
-    fat([], 2),
-    pe(0xffff),
-  ])("rejects malformed, truncated, or unsupported metadata", (bytes) => {
-    expect(parseExecutableHeader(bytes, "arm64").ok).toBe(false);
-  });
-
-  it("rejects malformed PE optional-header commitments", () => {
-    const wrongMagic = pe(0x8664);
-    wrongMagic.writeUInt16LE(0x10b, 64 + 24);
-    const truncatedDirectories = pe(0x8664);
-    truncatedDirectories.writeUInt16LE(112, 64 + 20);
-    truncatedDirectories.writeUInt32LE(16, 64 + 24 + 108);
-
-    expect(parseExecutableHeader(wrongMagic, "x64").ok).toBe(false);
-    expect(parseExecutableHeader(truncatedDirectories, "x64").ok).toBe(false);
+    expect(parseExecutableHeader(bytes, "riscv64")).toMatchObject({
+      value: {
+        architecture: "x86_64",
+        availableArchitectures: ["x86_64", "arm64"],
+      },
+    });
+    // A foreign family declared twice, such as arm64 and arm64e, has no
+    // provider-neutral slice, whether or not one declaration is undersized.
+    const subtypes = fat([0x0100000c, 0x0100000c]);
+    expect(parseExecutableHeader(subtypes, "x64")).toMatchObject({
+      ok: false,
+      error:
+        "FAT binary has no x64 slice and no foreign architecture declared exactly once",
+    });
+    subtypes.writeUInt32BE(31, 8 + 12);
+    expect(parseExecutableHeader(subtypes, "x64").ok).toBe(false);
+    // A unique foreign family is still selected beside an ambiguous one.
+    expect(
+      parseExecutableHeader(fat([0x0100000c, 0x0100000c, 0x01000007]), "arm"),
+    ).toMatchObject({ value: { architecture: "x86_64" } });
   });
 });
 

src/domain/binaryTarget.ts

@@ -99,7 +99,8 @@ export const claimsExecutableHeader = (
 
 /**
  * Parse supported executable headers without I/O. The caller supplies the host
- * architecture used for deterministic FAT Mach-O slice selection.
+ * architecture that FAT Mach-O slice selection prefers; a container without a
+ * host slice selects its first declared supported slice.
  */
 export const parseExecutableHeader = (
   bytes: Buffer,
@@ -168,15 +169,22 @@ const parseThinMachO = (
   });
 };
 
-const parseFatMachO = (
+/** One FAT declaration whose CPU type is a supported Mach-O family. */
+interface FatDeclaration {
+  readonly architecture: HeaderArchitecture;
+  readonly offset: number;
+  readonly size: number;
+}
+
+/** Validate a FAT table and its slice identities without selecting a slice. */
+const validatedFatDeclarations = (
   bytes: Buffer,
-  host: NodeJS.Architecture,
   header: Exclude<
     ReturnType<typeof readFatContainerHeader>,
     { state: "not-fat" }
   >,
   fatEvidence?: FatSliceHeaderEvidence,
-): Result<ExecutableMetadata, string> => {
+): Result<readonly FatDeclaration[], string> => {
   if (header.state === "needs-data")
     return err("truncated or invalid FAT architecture table");
   if (header.state === "malformed") return err(header.reason);
@@ -192,34 +200,93 @@ const parseFatMachO = (
     fatEvidence?.sliceHeadersByOffset,
   );
   if (sliceIdentityError !== null) return err(sliceIdentityError);
-  const architectures: HeaderArchitecture[] = [];
-  const slices = new Map<
-    HeaderArchitecture,
-    { readonly offset: number; readonly size: number }
-  >();
-  for (const declaration of header.slices) {
-    const architecture = machArchitecture(declaration.cpuType);
-    if (architecture === undefined || architectures.includes(architecture))
-      continue;
-    architectures.push(architecture);
-    slices.set(architecture, {
-      offset: declaration.offset,
-      size: declaration.size,
-    });
-  }
+  return ok(
+    header.slices.flatMap((declaration) => {
+      const architecture = machArchitecture(declaration.cpuType);
+      return architecture === undefined
+        ? []
+        : [
+            {
+              architecture,
+              offset: declaration.offset,
+              size: declaration.size,
+            },
+          ];
+    }),
+  );
+};
+
+const declaredFamilies = (
+  declarations: readonly FatDeclaration[],
+): HeaderArchitecture[] => [
+  ...new Set(declarations.map(({ architecture }) => architecture)),
+];
+
+/**
+ * The supported CPU families a validated FAT Mach-O table declares, or null for
+ * a non-FAT input. Unlike header parsing, it selects no slice, so it can show
+ * that a table excludes a host even when no slice is selectable.
+ */
+export const readFatArchitectureFamilies = (
+  bytes: Buffer,
+  fileSize = bytes.length,
+): Result<readonly HeaderArchitecture[], string> | null => {
+  const header = readFatContainerHeader(bytes, fileSize);
+  if (header.state === "not-fat") return null;
+  const declarations = validatedFatDeclarations(bytes, header);
+  return declarations.ok
+    ? ok(declaredFamilies(declarations.value))
+    : declarations;
+};
+
+const parseFatMachO = (
+  bytes: Buffer,
+  host: NodeJS.Architecture,
+  header: Exclude<
+    ReturnType<typeof readFatContainerHeader>,
+    { state: "not-fat" }
+  >,
+  fatEvidence?: FatSliceHeaderEvidence,
+): Result<ExecutableMetadata, string> => {
+  const validated = validatedFatDeclarations(bytes, header, fatEvidence);
+  if (!validated.ok) return validated;
+  const declarations = validated.value;
+  // Prefer the slice the host would load; a malformed host slice is an error.
+  // A container without one still declares its images, so select the first
+  // that can hold its Mach-O header, as a thin foreign image would be. A
+  // family declared more than once, such as arm64 and arm64e, has no
+  // provider-neutral choice and is not selected.
   const preferred = fatHostArchitecture(host);
-  const slice = preferred === undefined ? undefined : slices.get(preferred);
-  if (preferred === undefined || slice === undefined)
-    return err(`FAT binary has no host-compatible ${host} architecture`);
-  const minimumHeaderSize =
-    preferred === "arm64" || preferred === "x86_64" ? 32 : 28;
-  if (slice.size < minimumHeaderSize)
-    return err(`FAT slice for ${preferred} is shorter than its Mach-O header`);
+  const hostSlice = declarations.find(
+    ({ architecture }) => architecture === preferred,
+  );
+  const unambiguous = declarations.filter(
+    ({ architecture }) =>
+      declarations.filter((other) => other.architecture === architecture)
+        .length === 1,
+  );
+  const candidates = hostSlice === undefined ? unambiguous : [hostSlice];
+  const selected = candidates.find(
+    ({ architecture, size }) =>
+      size >= (architecture === "arm64" || architecture === "x86_64" ? 32 : 28),
+  );
+  if (selected === undefined) {
+    const first = candidates[0];
+    if (first !== undefined)
+      return err(
+        `FAT slice for ${first.architecture} is shorter than its Mach-O header`,
+      );
+    return err(
+      declarations.length === 0
+        ? "FAT binary declares no supported Mach-O architecture"
+        : `FAT binary has no ${host} slice and no foreign architecture declared exactly once`,
+    );
+  }
   return ok({
     format: "mach-o-universal",
-    architecture: preferred,
-    availableArchitectures: architectures,
-    universalSlice: { offset: slice.offset, size: slice.size },
+    architecture: selected.architecture,
+    availableArchitectures: declaredFamilies(declarations),
+    universalSlice: { offset: selected.offset, size: selected.size },
   });
 };
 

src/ghidra/GhidraInstallation.test.ts

@@ -1110,6 +1110,32 @@ describe("Ghidra FAT native decompiler admission", () => {
       status: "unavailable",
       detail: "requires mach-o arm64",
     },
+    {
+      name: "foreign-only FAT whose first slice is undersized",
+      bytes: (() => {
+        const bytes = fat([0x01000007, 0x0000000c]);
+        bytes.writeUInt32BE(31, 8 + 12);
+        return bytes;
+      })(),
+      status: "unavailable",
+      detail: "requires mach-o arm64",
+    },
+    {
+      name: "foreign-only FAT declaring one family twice",
+      bytes: fat([0x01000007, 0x01000007]),
+      status: "unavailable",
+      detail: "requires mach-o arm64",
+    },
+    {
+      name: "foreign-only FAT whose first same-family slice is undersized",
+      bytes: (() => {
+        const bytes = fat([0x01000007, 0x01000007]);
+        bytes.writeUInt32BE(31, 8 + 12);
+        return bytes;
+      })(),
+      status: "unavailable",
+      detail: "requires mach-o arm64",
+    },
     {
       name: "matching universal FAT",
       bytes: fat([0x01000007, 0x0100000c]),

src/ghidra/GhidraInstallation.ts

@@ -22,6 +22,7 @@ import { delimiter as pathDelimiter, dirname, posix, win32 } from "node:path";
 import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
 import {
   parseExecutableHeader,
+  readFatArchitectureFamilies,
   type ExecutableMetadata,
 } from "../domain/binaryTarget.js";
 import {
@@ -111,39 +112,37 @@ const assessNativeDecompiler = (
       status: "unknown",
       detail: `${path} is executable, but no native tool format is defined for ${platform}/${architecture}`,
     };
-  let parsed =
+  const parsed =
     probe === undefined
       ? undefined
       : parseExecutableHeader(probe.bytes, architecture, probe.size);
-  if (probe !== undefined && parsed !== undefined && !parsed.ok) {
-    // FAT parsing selects a host slice. A validated foreign slice can still
-    // prove that its complete architecture table excludes this host. If the
-    // host is listed but its slice is malformed, preserve the unknown outcome.
-    for (const { node } of NATIVE_ARCHITECTURES) {
-      if (node === architecture) continue;
-      const alternative = parseExecutableHeader(probe.bytes, node, probe.size);
-      if (
-        alternative.ok &&
-        !alternative.value.availableArchitectures.some(
-          (available) => available === expectedArchitecture,
-        )
-      ) {
-        parsed = alternative;
-        break;
-      }
-    }
-  }
-  if (parsed === undefined || !parsed.ok)
+  // Slice selection can refuse a FAT table that still proves this host is
+  // excluded, such as one declaring only arm64 and arm64e; its validated
+  // families decide. A malformed host slice remains an unknown outcome.
+  const families =
+    probe !== undefined && parsed !== undefined && !parsed.ok
+      ? readFatArchitectureFamilies(probe.bytes, probe.size)
+      : null;
+  const header = parsed?.ok
+    ? parsed.value
+    : families?.ok &&
+        !families.value.some((family) => family === expectedArchitecture)
+      ? {
+          format: "mach-o-universal" as const,
+          architecture: families.value.join("/") || "no supported architecture",
+        }
+      : undefined;
+  if (header === undefined)
     return {
       status: "unknown",
       detail: `${path} is executable, but its executable format for ${platform}/${architecture} could not be established`,
     };
-  const observed = `${parsed.value.format} ${parsed.value.architecture}`;
+  const observed = `${header.format} ${header.architecture}`;
   const expected = `${expectedFormat} ${expectedArchitecture}`;
   const formatMatches = isMachOFormat(expectedFormat)
-    ? isMachOFormat(parsed.value.format)
-    : parsed.value.format === expectedFormat;
-  if (formatMatches && parsed.value.architecture === expectedArchitecture)
+    ? isMachOFormat(header.format)
+    : header.format === expectedFormat;
+  if (formatMatches && header.architecture === expectedArchitecture)
     return {
       status: "compatible",
       detail: `${path} reports ${observed}, which matches ${platform}/${architecture}`,

tests/boundary/cli/foreignUniversalMachoCli.test.ts

@@ -0,0 +1,90 @@
+import { writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+
+import { expect } from "vitest";
+import { z } from "zod";
+
+import { fat } from "../../../src/domain/binaryTarget.fixture.js";
+import { listArchitecturesSchema } from "../../../src/domain/native/nativeInspection.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+const environment = { REA_LOG_LEVEL: "silent" } as const;
+
+// The one slice of each container below is never the host's architecture.
+const foreign =
+  process.arch === "arm64"
+    ? { name: "x86_64", cpuType: 0x01000007 }
+    : { name: "arm64", cpuType: 0x0100000c };
+
+cliTest.skipIf(process.platform !== "darwin")(
+  "inventories a universal Mach-O whose only slice is foreign to the host",
+  async ({ cli, processes }) => {
+    const directory = await createTestTempDirectory("rea-foreign-fat-cli-");
+    const thin = join(directory, "thin");
+    const universal = join(directory, "universal");
+    const compiled = await processes.run(
+      "/usr/bin/xcrun",
+      [
+        "clang",
+        "-arch",
+        foreign.name,
+        resolve("tests/conformance/c/fixture.c"),
+        "-o",
+        thin,
+      ],
+      { timeoutMs: 30_000 },
+    );
+    expect(compiled.exitCode, compiled.stderr).toBe(0);
+    const created = await processes.run(
+      "/usr/bin/xcrun",
+      ["lipo", "-create", thin, "-output", universal],
+      { timeoutMs: 30_000 },
+    );
+    expect(created.exitCode, created.stderr).toBe(0);
+
+    const result = await cli.run({
+      arguments: ["list-architectures", universal, "--json"],
+      environment,
+    });
+
+    expect(result.exitCode, result.stderr).toBe(0);
+    const evidence = z
+      .object({
+        subject: z.object({ format: z.string() }),
+        normalized_result: listArchitecturesSchema,
+      })
+      .parse(result.json);
+    expect(evidence.subject.format).toBe("mach-o-universal");
+    expect(evidence.normalized_result.architectures).toMatchObject({
+      total: 1,
+      exhaustive: true,
+      items: [{ name: foreign.name }],
+    });
+    expect(
+      evidence.normalized_result.architectures.items[0]?.file_offset,
+    ).toBeGreaterThan(0);
+  },
+  60_000,
+);
+
+cliTest.skipIf(process.platform === "darwin")(
+  "admits a universal Mach-O whose only slice is foreign to the host",
+  async ({ cli }) => {
+    const directory = await createTestTempDirectory("rea-foreign-fat-cli-");
+    const universal = join(directory, "universal");
+    await writeFile(universal, fat([foreign.cpuType]));
+
+    const result = await cli.run({
+      arguments: ["list-architectures", universal, "--json"],
+      environment,
+    });
+
+    // The target is admitted; only the macOS-only lipo inventory is refused.
+    expect(result.exitCode).not.toBe(0);
+    expect(result.json).toMatchObject({
+      code: "capability_unavailable",
+      details: { provider_id: "native-macos", operation: "list_architectures" },
+    });
+  },
+);