2 Commits

Include renames

Author	SHA1	Message	Date
4 people	9b66326e72	feat+test: D58 — server.mjs streaming singleflight wiring (ADR 0005 Amendment 8, issue #16) (#37) ... * feat+test: D58 — server.mjs streaming singleflight wiring (ADR 0005 Amendment 8 §§7,11,12, issue #16) Second of three D-days for v1.x roadmap #1. D57 landed the cache-layer primitive (cacheStore.getOrComputeStreaming); D58 wires it into the streaming branch of server.mjs and adds the X-OLP-Streaming-Inflight header. D59 closes issue #16 + polishes README known-limitations. ## What server.mjs — replaced the streaming branch (formerly the peek+spawn pattern at lines 1138-1327) with cacheStore.getOrComputeStreaming(...): - tryAcquireSpawn moves INSIDE sourceFactory closure (§7). Only first caller acquires; attached joiners share the slot. releaseSpawn lives in the source generator try/finally so it fires once on source completion / error / abort. - CONCURRENCY_LIMIT thrown by the factory triggers fallthrough to the buffered path (preserving today's behaviour); any other pre-stream factory error surfaces a 502. - X-OLP-Streaming-Inflight: source | attached header per §11 (cache_hit role omits — X-OLP-Cache: hit already says it). The §11 'solo' value is deferred to a future amendment — observable only post-stream via the streaming_inflight_source_done log event's attached_count. - auditCtx.cache_status: 'miss' for source, 'streaming_attached' for joiners, 'hit' for the TTL-race cache_hit branch. - res.on('close', () => stream.return?.()) propagates client disconnect into the tee's attachedClients accounting (§9). Note: Node 25 emits 'close' on ServerResponse, NOT on IncomingMessage — empirically verified in test 28g. - Cache writes now happen inside the cache layer's tee task on source completion (§4). Server still issues cacheStore.delete on stop-less exhaustion to preserve D16 truncated-not-cached invariant — the cache layer is IR-agnostic and writes accumulatedChunks unconditionally; the IR-aware server deletes the entry if no stop chunk was observed. - The pre-cache-store-acquire and matching releaseSpawn-on-503 branches are gone — they were vestigial once the factory owns acquire+release. lib/audit.mjs — JSDoc cache_status enum extended with 'streaming_attached'. Free-form string at the wire (no schema validator on append); the JSDoc is the source of truth for the consumer enum. test-features.mjs Suite 28 — 7 HTTP integration tests: - 28a single SSE request (source role + cache populated + second request → X-OLP-Cache: hit) - 28b 2 concurrent identical SSE → one spawn, source + attached roles, identical chunk sequences delivered - 28c TOCTOU regression — pre-populated cache + 2 concurrent → both hit buffered replay path, no streaming branch entry, no X-OLP-Streaming-Inflight header - 28d mid-stream join — late joiner receives accumulated burst + live tail - 28f one-of-N disconnect — source NOT aborted; survivor completes - 28g ALL clients disconnect → source aborted; no cache write; subsequent request gets fresh source spawn - 28h CONCURRENCY_LIMIT fallthrough — factory throws at maxConcurrent=1; buffered path's chain-exhausted 502 surfaces (28e backpressure deferred to Suite 27g unit-level coverage.) ## Scope server.mjs + lib/audit.mjs + test-features.mjs. Untouched: cache/store.mjs (D57 frozen), provider plugins, fallback engine, IR. CHANGELOG and package.json bump fires at D59 close. ## Authority - docs/adr/0005-cache-cross-provider.md Amendment 8 §§7, 8, 9, 11, 12 - docs/v1x-roadmap.md #1 - GitHub issue #16 (TOCTOU window; closed in D59) - ADR 0002 Amendment 6 (D38 tryAcquireSpawn/releaseSpawn semantics that §7 now invokes inside the sourceFactory closure) ## Test count 615 → 622 (+7 D58 integration tests). Local: 622/622 pass. ## Iron Rule 10 follow-up notes for the reviewer - res.on('close') vs req.on('close'): switched to res after empirical verification (28g fails on req under Node 25). Comment in code. - Cache-layer write + server-layer delete for stop-less exhaustion is cosmetically inconsistent with streaming_inflight_source_done's cache_written: true log. Functionally correct; flagged for future amendment. - 'solo' header value deferred — would need trailer mechanics or post-stream emission. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: D58 reviewer follow-ups — P2-1 (audit-query gauge drift) + P2-2 (stop-less HTTP test) Fold-in for D58 PR #37 fresh-context opus reviewer findings (APPROVE WITH MINOR — 0 P0/P1, 4 P2). P2-3 (`isFirst` unused) and P2-4 (`solo` not emitted) are design-acceptable per ADR 0005 Amendment 8 §11 and left as-is. P2-1 — `lib/audit-query.mjs` gauge reconciliation. `aggregateRequests` and `cacheHitRateWindow` previously counted `streaming_attached` rows in `total` / `pe.total` without contributing to hit/miss/bypass numerators, breaking the invariant that the cache_status breakdown should sum to the total. Added an explicit `streaming_attached` field to both the global return shape and the `by_provider` shape; the counter is excluded from `hit_rate` numerator AND denominator (joiners did not hit a literal cache so they don't belong in either side of the ratio). Test count is unchanged for D49 suites — they only assert presence + non-negative + reconciliation invariants that the new field preserves; if a test asserted exact value equality on a fixture with NO streaming_attached rows, the new field defaults to 0 and the assertion still passes. P2-2 — Suite 28 stop-less HTTP coverage gap. Test 28i fires an SSE request to a fake provider whose source generator returns WITHOUT a {type:"stop"} chunk; asserts (a) the synthetic truncation marker appears in the body (D26 F19 in-band signal), (b) [DONE] terminator follows, (c) a subsequent identical request triggers a fresh spawn (cache was NOT populated by the truncated stream). Pins the D58 `cacheStore.delete` path at server.mjs:1344-1346 end-to-end. 622 → 623 (+1 D58 follow-up test). 623/623 pass locally. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-25 20:59:54 +10:00
4 people	686794e316	feat+test+docs: D49 — lib/audit-query.mjs (Phase 3 audit aggregate query layer) (#26) ... Second Phase 3 D-day. Implements ADR 0008 § 4 query API. Pure in-memory ndjson scan; cross-file walk over audit.ndjson (live) + audit-YYYY-MM-DD.ndjson (rotated). No server.mjs integration in this D-day (D50 wires the consuming endpoints). NEW lib/audit-query.mjs (~370 lines): 5 public API functions per ADR 0008 § 4.1: - discoverAuditFiles({ olpHome }): filesystem scan; returns Map<date|'live', path>. - readAuditWindow({ startMs, endMs, olpHome, logEvent }): generator over events in half-open window [startMs, endMs). Walks rotated date files + live file. Skips malformed lines + logs warn. - aggregateRequests({ windowMs, olpHome }): counts + status buckets + by_provider + by_owner_tier + by_path + median/p95 latency over rolling window. - topFallbackChains({ windowMs, limit, olpHome }): top-N chains by trigger count from events with fallback_hops > 0. Tied-count tiebreak: ascending first_seen. - spendTrendDaily({ days, olpHome }): daily series ending today with sparse-fill for zero-request days. Per-day request_count + median latency + by_provider breakdown. - cacheHitRateWindow({ windowMs, olpHome }): audit-derived cache hit rate (bypass excluded from denominator); per-provider + overall. PII discipline (ADR 0008 § 4.3): every aggregate function relays only schema fields; never message content. Suite 23g actively asserts the absence of content/message/messages/prompt/response/body keys in every aggregate output. Cross-file walk semantics (ADR 0008 § 4.2): half-open window [startMs, endMs); date-range computed once from window bounds; each rotated date file checked; live audit.ndjson always checked (it covers today regardless of whether the window endpoint is past midnight). spendTrendDaily calendar-date semantics: days: N returns "last N calendar UTC dates ending today" — NOT "events within a rolling N*86400-ms window" (which would span N+1 distinct UTC dates and produce off-by-one buckets at non-midnight call times). Computed via: for (let i = days-1; i >= 0; i--) dates.push(_utcDateFromMs(now - i*86400*1000)); cacheHitRateWindow denominator: hit_rate = hit / (hit + miss). Bypass is intentional non-cacheable (Anthropic cache_control marker), NOT a cache miss; excluding it from the denominator gives a clean cache-effectiveness signal. TESTS — Suite 23, +27 (544 → 571): 23a-1..4: discoverAuditFiles (empty dir / live only / live+rotated / non-audit files ignored) 23b-1..6: readAuditWindow (all-coverage / single-day / half-open exclusivity / empty window / missing files / malformed-skip with warn) 23c-1..4: aggregateRequests (counts + status buckets + by_provider; by_owner_tier; median+p95 latency over realistic distribution; invalid windowMs rejection) 23d-1..4: topFallbackChains (sort desc by count; limit truncation; fallback_hops=0 excluded; first_seen/last_seen carried) 23e-1..3: spendTrendDaily (N-day range correctness — caught off-by- one during local run; populated day breakdown; empty day sparse- fill) 23f-1..3: cacheHitRateWindow (overall + per-provider hit_rate; bypass not in denominator; cache_status=null events excluded) 23g-1..3: PII guard for aggregateRequests / spendTrendDaily / topFallbackChains + cacheHitRateWindow — every output JSON- stringified + scanned for forbidden PII keys DOCUMENTATION: - AGENTS.md: lib/audit-query.mjs new entry; lib/audit.mjs note added that D52 extends with daily rotation. NOT IN D49 scope: - server.mjs endpoints consuming these queries (D50) - dashboard.html (D51) - lib/audit.mjs rotation extension + bin/olp-audit-rotate.mjs (D52) - tried_providers schema fix (D53; D45 P2 deferral) - Phase 3 close → v0.3.0 (D55; maintainer-triggered) Test count: 544 → 571 (+27). Verified locally via npm test. AUTHORITY: - ADR 0008 § 4 (query API surface) + § 5 (rotation file naming pattern) + § 3 (storage layout). - ADR 0007 § 8 (audit ndjson event schema — input data). - CLAUDE.md release_kit overlay phase_rolling_mode — under Unreleased. - Standing autopilot grant. ALIGNMENT.md scope check: this PR adds a new lib/ module. No provider plugin / entry surface / IR change. Rule 5 commit-citation requirements for those scopes do not apply. Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-25 15:51:57 +10:00