Convert native directory separators only at Git patch argument production. Exercise fresh, applied and invalid dependency patches in private copies, preserving protectNTFS, and select the regression for Windows CI.
Replace the spelling-based mock inventory with one binding-aware Oxlint rule. Canonical test projects own isolation for both Vitest and lint; external temporary CLI probes avoid concurrent test discovery.
Validated actual local Git and Oxlint execution, resolved-policy parity, CI selection and report tests. Native Windows verification remains with CI. Refs: #7050
Construct the canonical ToolResultDelivery through the native continuation policy factory. Move type and lifecycle spy consumers to the core owner while preserving explicit response budgets and the deferred runtime composition.
Validation: runtime build and 335 focused tests passed; 41 Linux-only firmware cases were skipped on macOS. Tests use authored local fixtures, not real external providers.
Keep the maintained manual evaluator, fixture rubrics and semantic tests together under tests/evaluation/model-evals instead of shipping them in dist. Load the same typed owners from the manual runner and update the architecture and client guide. Grading/report contracts and executable logic are unchanged.
* fix(hopper): keep automated Linux setup off non-x86_64 hosts
Every pinned Linux Hopper package (deb, rpm, Arch) is x86_64-only, but
hopperSetupSupported was derived from os-release alone. Setup therefore
planned and confirmed an amd64 package install on arm64 Ubuntu 24.04 and
the other tested distributions.
Classify installer support with the host architecture where the
distribution is parsed, so setup planning, preflight, installation and
package verification share one decision. Setup now takes its Linux
distribution from the doctor host, as it already does for the macOS
version, and the supported-host remediation text has a single owner.
Fixes#3419
* fix(hopper): install Linux Hopper from the setup host's distribution facts
systemLinuxHopperInstallHost read os-release with process.arch on its own,
while setup planning used the doctor host's injected architecture. Both
agree in production, but a doctor host composed with another architecture
could plan one answer and install with another.
The install host now takes its distribution reader from the caller, and
systemSetupHost passes the doctor host's, so planning, preflight and
installation share one host-fact owner. The unused default host on
installLinuxHopper, the other ambient reader, is removed.
* fix(verify-package): require Hopper readiness only where Linux setup installs it
packageSetupSupport required doctor Hopper readiness on every Linux host
whose host check passed, including installer-untested distributions and,
after the architecture gate, arm64 hosts where setup never provides
Hopper. On Linux, readiness now also requires automated installer
applicability; macOS is unchanged.
This narrows the readiness assertion from #2973 on hosts outside the
installer matrix, such as Kali and ubuntu-24.04-arm.
---------
Co-authored-by: DavidFB-creator <220631175+DavidFB-creator@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Replace fixed Linux Hopper release pins with current vendor feed metadata,
complete detached-PGP package verification under the stable publisher primary
key, and authenticated package evidence for offline admission. Share parsed
publisher facts across approved setup, doctor and the pre-execution guard.
Preserve existing ownership, cancellation, private-display and cleanup rules.
Fixes#3577
Normalize empty XDG settings, preserve backups during approved cache repair,
and retain publisher cleanup through cancellation and shutdown. Exercise cold
offline verification without reusing process-local proof state. Serialize
approved cache publication across processes, restore old evidence on failure,
and share the existing lease owner from process without changing its identities.
Validation: 438 focused tests (six platform-related skips), full static/documentation checks, real Ubuntu
24.04 official Hopper 6.6.0 setup/CLI/MCP, offline ordinary-user admission,
launcher mutation rejection, and cold/warm verification measurements. Official
RPM metadata mismatch is tracked separately in #3578 and fails closed.
BREAKING CHANGE: Older Linux Hopper builds admitted by the fixed allowlist
now require retained publisher-signed package evidence or an approved update
to the current public demo. Update explicit HOPPER_LAUNCHER_PATH values when
migrating; the live feed exposes only its current release.
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
* test(perf): pin genuine Fiddler captures and raw HTTP oracles
Pin two public Microsoft SAZ captures with 13 and 1311 sessions, immutable source/digest/member coverage and predeclared caller-ordered transaction questions. Independently preserve stored HTTP headers, binary/empty bodies and archive coordinates; admit only known corpus bytes before ZIP allocation. Five reader regressions and both genuine selected-record checks pass. No production or performance-gain claim.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* test(perf): measure SAZ selection and retained recovery workflows
Add authenticated complete/ordinal/retained-page SDK experiments, fixed real-Codex prompts and independent cited-byte grading. Preserve actual failed RPCs and exact retained-reference recovery when complete MCP delivery exceeds its receive buffer. Verify selected CLI/MCP facts, parameters and semantic Evidence identity. Keep a genuine partial-message admission-negative probe. Focused 12 tests and 10 actual-Evidence grading cases pass; no formal model or gain claim yet.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
* test(perf): verify resolved SAZ Evidence selection parameters
Distinguish omitted request ordinals from the complete resolved ordinal scope recorded by WebNetworkCaptureService. Share the expected sanitized Evidence representation between SDK checks and model grading; preserve original caller arguments. A focused regression and both saved genuine baseline RPCs confirm the correction. Keep original collector failures and costs unchanged; freeze a new driver before repeated measurements.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
* docs(perf): retain mixed SAZ whole-task costs and failure boundaries
Record both SDK batches, twelve corrected workflows, all four actual Codex attempts, independent byte equivalence and verified payload-phase CPU/memory. Distinguish successful-analysis MCP transport limits from real provider timeouts and negative partial-message admission. Three model questions complete; the only completed pair saves 1.6 percent input but takes 2 percent longer, with no consistent benefit. Keep production APIs/defaults/cache unchanged and preserve all deviations, failed attempts and unknowns.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
---------
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Pin two public sigrok recordings and independently verify raw waveform bytes/sample locations through real CLI/MCP selection workflows. Record all eight actual Codex task costs, including a fixed-plan deviation and mixed whole-task token/time outcomes. Existing annotation controls reduce reply bytes; the cohort does not justify a new production cache/default/API change.
Harden corpus identity, UART data-bit coverage, I2C interval ownership and preallocation ZIP/metadata admission. Focused tests and original Evidence/transcript replays retain observed facts, outcomes, provenance and resource limitations.
Fixes#3131.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Preserve reproducible complete versus summary/page PCAP measurements on two official HTTP recordings and a fixed metadata/URI question. Summary/page uses more than twice the SDK JSON and about 51% more total model input in both real-model pairs, so existing complete defaults and production APIs/cache remain unchanged. These results apply to these captures and question; no general speedup is claimed. Refs #2016.
Authenticate experiment SDK envelopes with the pinned runtime's existing Evidence parser and check coverage, target/view/operation ownership and retained-parent identity. Separate factual and exact-workflow grading, retaining diagnostics and partial facts for malformed replies. Register six comparator CLI scenarios in the discovered Vitest evaluation project and remove the manual suite. The final change is limited to nine experiment files and one discovered evaluation test; production behavior is unchanged.
Validation includes eight actual SDK workflows and four actual Codex CLI tasks, independent raw-frame and CLI/MCP checks, authentication of all 18 original receipts, 34 context/grading regressions and six comparator scenarios. All four stored model answers pass factual and workflow regrading without model reruns. Original measured runtime remains 3ae826ceba6f25b08bce50bcb51001209bc16b57; original measurements and private raw artifacts are unchanged. Serial local checks use one CPU with 3-GiB early-stop/4-GiB hard/no-swap limits. Unknown terminal counters remain explicitly unknown.
Integrate upstream shared fixture owners and Windows verification, removing both superseded local differences. All final experiment/comparator files remain byte-identical to the reviewed sources and the final PR scope is 10 files. Full bounded preflight passes in 73.953 seconds with ~2.23 GiB peak; payload cleanup is verified, zero phase OOM/limit events are observed, and terminal counters remain unknown. Current-head automatic CI 38492231165 passes all 35 selected jobs, including every compiled shard, source/static/build/docs, actual Windows/native/package and real application lanes, Portable reports/inventory, reverse-tools coverage/mutation and CI required. Greptile passes and all review threads are resolved. Actual CI checkout is 4eb2262359cb01f2fb1314db0bc18d3dc9092792, integrated with main 41a2e84647. Passing retries do not establish the root cause or a fix of the earlier main worker cleanup failure; its original evidence is preserved. No release.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Use exact discovered test identities, production MCP sessions and byte-exact setup backups. Add CLI signal cleanup coverage and independent Ghidra, Go and firmware verification assertions; prune redundant test configuration and preserve platform limits.
* fix(ci): restore inherited validation failures
* perf(ci): share trusted caches and fail fast on PR test failures
* fix(ci): bound static cache and hash vitest config for knip
- Bound the rolling .turbo/cache snapshot with 7d/500MB eviction so each
trusted main save cannot accumulate history toward the GitHub limit.
- Hash vitest.config.ts in knip inputs so removing the CiTestSequencer
import invalidates the cached passing check.
- Cover both in cache-policy and task-hash regressions.
* fix(verify-package): read Hopper installer applicability from its owner
packageSetupSupport derived hopperSetupSupported from the general doctor
host check plus a non-Windows condition, so a Debian-family host REA
supports but the Hopper package never tested reported an automated
installer the packaged setup never offers. verifySetupApply then expected
needs_confirmation for the explicit --install-hopper request and failed
at verify-package-setup.mjs:403.
Applicability now comes from readLinuxDistribution() in the built
application, so the verifier follows the predicate the package ships, and
it stays unknown when no distribution can be read instead of collapsing
into a known-unsupported host. Generic agent configuration and idempotence
assertions stay independent of installer applicability, and the expected
status is still derived from the host rather than from the request being
asserted.
Verified on Kali 2026.3, a Debian-family host with an installer-untested
distribution and the simulated Hopper package environment: before the
change npm run verify:package fails at verify-package-setup.mjs:403 with
the packaged setup correctly returning needs_human; after it the same run
passes every stage. Both applicability branches are covered by
tests/evaluation/package/packageSetupSupport.test.ts, which runs in the
source lane without a build. No real Hopper installation was exercised,
so no engine compatibility claim follows.
Refs #2939
* fix(ci): preserve Hopper doctor readiness checks
---------
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* fix(package-verifier): reject overview responses without a procedure count
The analyze and inspect checks compared procedure_count with `< 1`, and an absent
or null normalized_result makes that comparison false, so a binary_overview
response without its required result passed the platform smoke check. Require an
integer procedure count before the positive-count comparison.
* test(package-verifier): cover malformed overview receipts
---------
Co-authored-by: REA Maintenance <76467478+morluto@users.noreply.github.com>
Authenticate completed successful REA Evidence records for citation accounting and shared fixture grading. Preserve genuine structured and complete text-only delivery, including authenticated inventory substeps.
Closes#1728.
Guide task-centered recovery from observed client, registration, PATH and
provider blockers. Install the recovery reference, document exact JavaScript
module selectors, and require clarification before searching for an unnamed
app. Validate public setup in isolated Codex accounts and preserve failed
or unrecognized tool outcomes in the existing evaluator.
Refs: #1587
BREAKING CHANGE: Optional verify:agent reports now use schemaVersion 4.
Consumers must accept expectedFirstTool: null and targetClarificationPassed
for missing-target scenarios. Production CLI/MCP contracts are unchanged.
Deliver complete Evidence directly through MCP with required nullable analysis profiles and immutable per-server delivery settings. Bind selected environments at construction and require typed cleanup outcomes with retained retry ownership.
Remove test-only routing, duplicate session and snapshot metadata, producer-field repairs, and ambiguous provider arguments. Separate artifact byte identity from occurrence classification and retain typed source-map and partial native facts across CLI, MCP, and verifier consumers.
BREAKING CHANGE: obsolete Evidence envelopes, session aliases, incomplete snapshots, and missing normalized producer facts are no longer accepted.
* feat(javascript): recover readable modules with Wakaru
Integrate pinned upstream source behind a typed optional recovery port,
preserve source identity and producer provenance, and publish verified
artifacts for existing JavaScript analysis through CLI and MCP.
Add focused real-tool CI with genuine bundler fixtures and bounded
resource, cancellation, cleanup, schema and packaged-artifact checks.
Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
* fix(javascript): align recovery catalog and browser verification
Share canonical Wakaru provider identity, describe CLI mode options, and
format the new workflow test. Disconnect the real page transport after
its observation readiness progress instead of a fixed delay.
Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
* refactor(javascript): consume the investigation evidence writer
Verify that the SDK recovery handler records complete Evidence through the newly shared investigation owner.
Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
* chore(test): account for isolated recovery fixture compilers
* fix(javascript): validate recovery paths before acquiring the engine
Return field-specific invalid_format diagnostics for relative paths and verify compiled CLI/MCP parity even when the optional engine is unavailable.
Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
morluto