mirror of
https://github.com/morluto/rea
synced 2026-10-28 06:46:47 -04:00
experiment(re): align cross-function question with pinned return-path oracle
Retain both initial real-model answers and whole-task costs. Both prove the core context connection but omit the oracle-required return-code distinctions; clarify that requirement before the next pair without changing the oracle or supplying addresses. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
This commit is contained in:
@@ -54,3 +54,11 @@ added. Older sources used pretty JSON; their smaller/newer encoded sizes must
|
||||
not be attributed to the call exporter. The earlier compact-writer change is a
|
||||
separate experiment. Unsupported old schema graphs remain unavailable with valid
|
||||
C retained, rather than acquiring guessed call records.
|
||||
|
||||
The first paired question runs at `01e5c5955` both explain the primary state flow
|
||||
correctly but omit the oracle's separate return-1/return-18 distinctions. Because
|
||||
the initial question did not explicitly enumerate those codes, the next round
|
||||
clarifies that requirement in `sqlite-call-question.txt` before either lane
|
||||
starts. The required facts and independent oracle stay unchanged. Earlier
|
||||
questions and costs remain pinned to their original commits; neither incomplete
|
||||
sample is counted as a full-oracle success.
|
||||
|
||||
@@ -6,6 +6,8 @@ the corresponding recovered function input. Explain which context field the help
|
||||
updates for opening/closing parentheses, where the caller checks that same field,
|
||||
and how the successful and malformed paths differ. Verify the essential pointer,
|
||||
field and control-transfer claims against selected machine instructions.
|
||||
In the final answer, explicitly distinguish the wrapper's routing for helper
|
||||
return codes 0, 1 and 18, including the malformed and depth-limit diagnostics.
|
||||
|
||||
Keep the answer focused on this cross-function state flow. Give function and
|
||||
instruction addresses and real Evidence references. Distinguish recovered call
|
||||
|
||||
@@ -467,3 +467,51 @@ family CPU 3.619 seconds and memory peak 436.6 MiB. Cleanup is verified, OOM 0.
|
||||
The failed sample and raw receipts remain under `.cache/atomic-codemode/` and
|
||||
are excluded from correctness-equal paired costs. Subsequent attempts use an
|
||||
independent preserved executable and explicit pre/post engine validation.
|
||||
|
||||
## Cross-function call task: initial question pair
|
||||
|
||||
`sqlite-call-codemode-v2` and `sqlite-call-atomic-direct-v2` both run at
|
||||
`01e5c5955`, with engine `64c6b3f...` from an independent preserved executable.
|
||||
The fourteen fact methods, native profile, artifact, requested model/effort and
|
||||
common guidance match; Code Mode runs first. Runtime, engine and target bytes
|
||||
remain unchanged, per-response usage matches runner totals, cleanup is verified
|
||||
and OOM kills are zero. Both fresh target leases use uncontrolled warm OS caches.
|
||||
|
||||
| Measurement | Code Mode call v2 | Atomic direct call v2 |
|
||||
| ------------------------------------ | ----------------: | --------------------: |
|
||||
| MCP requests / transport failures | 10 / 0 | 27 / 0 |
|
||||
| Observed model responses | 13 | 16 |
|
||||
| Client exec calls | 12 | 15 |
|
||||
| Model wall seconds | 168.622 | 156.033 |
|
||||
| Family wall seconds | 169.190 | 156.570 |
|
||||
| Family CPU seconds | 16.096 | 15.854 |
|
||||
| Family memory peak MiB | 514.0 | 608.1 |
|
||||
| Cumulative input tokens | 454,199 | 564,158 |
|
||||
| Cached input tokens | 377,216 | 497,408 |
|
||||
| Uncached input tokens | 76,983 | 66,750 |
|
||||
| Output tokens, including reasoning | 4,987 | 4,838 |
|
||||
| Reasoning tokens, included in output | 1,301 | 935 |
|
||||
|
||||
Both answers correctly connect call `0x65211 -> 0x64060`, caller SSA
|
||||
`v100000118`, passing storage `register:0x38/8`, and callee input `v300000042`.
|
||||
They verify caller LEA, helper pointer preservation, character dispatch,
|
||||
field increment/decrement and the caller's post-success field check against
|
||||
selected machine bytes. They preserve storage-match/alias and runtime
|
||||
limitations. Code Mode actually composes `calls`, `values`, operations and machine
|
||||
reads; direct uses the same primitives, including a bounded slice. All ten/nine
|
||||
cited Evidence sources respectively have matching retained-source SHA-256 and
|
||||
artifact identity.
|
||||
|
||||
Neither final answer includes the original oracle's required separate routing
|
||||
for helper return 1 and 18. The question only explicitly asked to distinguish
|
||||
successful/malformed paths; the oracle was stricter than that wording. Do not
|
||||
retroactively delete this requirement or mark either sample a full-oracle pass.
|
||||
The task wording is now clarified before the next paired round, retaining the
|
||||
same oracle and adding no solution addresses. The table is an observed partial
|
||||
coverage result, not a completed-task efficiency claim. Its mixed costs also
|
||||
do not show a general Code Mode advantage: cumulative input is lower while
|
||||
uncached input and wall time are higher in this sample.
|
||||
|
||||
Raw ledger: `.cache/atomic-codemode/ledger-call-round-01e5c59.json`. Each cell is
|
||||
one stochastic sample. The complete client already provides JavaScript dispatch
|
||||
in both lanes. No new traditional-provider cell is being claimed.
|
||||
|
||||
Reference in New Issue
Block a user