mirror of
https://github.com/morluto/rea
synced 2026-10-25 22:08:11 -04:00
Kaoru0822
8ab5024397
* fix(kuna): stream string scans instead of capping them at 16 MiB `list_strings` and `search_strings` buffered Kuna's whole string JSON and refused anything above 16 MiB. Kuna writes about 290 bytes per row, so every app above roughly 10 MB failed with a non-retryable memory error: a 20 MB Ghostty binary produces 31 MB, Raycast 30 MB and an arm64 Firefox XUL 179 MB. Parse the scan as Kuna writes it. `runOwnedCommand` accepts a stdout sink, and `KunaStringInventoryReader` owns the interpretation: it tokenizes synchronously with stream-json, keeps exact u64 address tokens, validates each row before retaining it, skips members it does not interpret, and admits retained rows against live heap headroom rather than a fixed scan size. The first contradiction stops Kuna instead of waiting for the rest; an incomplete document is judged only after Kuna's own exit status. Diagnostics keep the 8 MiB Kuna budget, now shared with the project export. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): admit string scan text as it streams and drop sinks with their command Review found that packed string tokens were assembled whole before the reader could discard them or check heap headroom, and that a stdout sink could outlive a cancelled or timed-out command. Move the token mechanics into `KunaJsonDocumentReader`: keys and numbers (Kuna's own fields) stay packed, while strings, which carry target-derived text, arrive in chunks. Ignored strings are dropped unassembled and kept ones are admitted against live heap headroom as they grow. Kept text is copied out of the decoded chunk it was sliced from, so retained rows no longer pin whole chunks (XUL: 278 MB retained before, 156 MB after). `KunaStringInventoryReader` keeps only the string-scan semantics. `runOwnedCommand` now ties the stdout pipeline to the command's deadline signal and aborts it when collection fails, destroying the sink. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): type the stream-json tokenizer where it is imported The Reverse tools quality lane type-checks a partial project that reaches KunaJsonStream without the separate streamJsonParser.d.ts augmentation, so jsonParser had no declaration there. Declare the tokenizer's shape at its only import site and remove the augmentation file, so every compilation of the reader is self-contained. Split two helpers out of the token dispatcher and simplify the stdout sink wait in runOwnedCommand, which keeps the added complexity minimal. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): bound key and number assembly in streamed Kuna JSON Review found that packed keys (and packed numbers) were assembled whole by the tokenizer before the reader could discard or constrain them, so a malformed scan with one enormous key could still exhaust the heap. Stream every scalar. Keys are assembled only up to 64 code units, past every field name the reader keeps, so a longer key is skipped unmatched; kept numbers longer than that are rejected, and ignored numbers are dropped unassembled. XUL's 611k-row scan parses in the same time. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): stream the call graph instead of capping it at 32 MiB (#4329) * fix(kuna): stream the call graph instead of capping it at 32 MiB Call relationships buffered `kuna decompile-graph` and refused output above 32 MiB. A 53 MB Raycast binary produces a 157 MB graph (180k functions, 773k edges), so every call query failed with a non-retryable memory error after Kuna's minute-long whole-program walk. `KunaCallGraphReader` validates the graph through the shared streaming document reader. It retains only the facts queries use (126 MB for Raycast) and checks edges against the complete node set once Kuna exits. Unrequested bodies are refused before any of their text is assembled, through new must-be-null row fields; address fields may be null, as `forwardsTo` is. Malformed graphs are now `AnalysisOutputError` like the string scan. `KunaQueryWork.scan` owns the run-and-validate sequence both producers shared: diagnostics budget, early rejection and retention of deterministic overflow. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): index call-graph edges as they stream Review found that the reader kept every parsed edge and built the edge indexes only after streaming, outside any heap admission, so a large graph could exhaust the heap instead of returning a memory constraint. Index each edge when it arrives: duplicate pairs and callee orders are checked per caller, call edges go straight into the caller/callee indexes, and no edge array is kept, so the whole working set grows under the per-row admission. Endpoints and order completeness are checked once all nodes are known, and the validation-only indexes are released then. Raycast's graph now peaks at 346 MB instead of 503 MB and retains 93 MB instead of 126 MB, in the same time. Also drop imports the shared scan made unused. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(kuna): state the graph's retained size after incremental indexing Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>