taodeng/olp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/olp.git synced 2026-07-19 09:45:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat+test: D58 — server.mjs streaming singleflight wiring (ADR 0005 Amendment 8, issue #16) (#37)

Browse Source 
* feat+test: D58 — server.mjs streaming singleflight wiring (ADR 0005 Amendment 8 §§7,11,12, issue #16)

Second of three D-days for v1.x roadmap #1. D57 landed the cache-layer
primitive (cacheStore.getOrComputeStreaming); D58 wires it into the
streaming branch of server.mjs and adds the X-OLP-Streaming-Inflight
header. D59 closes issue #16 + polishes README known-limitations.

## What

server.mjs — replaced the streaming branch (formerly the peek+spawn
pattern at lines 1138-1327) with cacheStore.getOrComputeStreaming(...):
- tryAcquireSpawn moves INSIDE sourceFactory closure (§7). Only first
  caller acquires; attached joiners share the slot. releaseSpawn lives
  in the source generator try/finally so it fires once on source
  completion / error / abort.
- CONCURRENCY_LIMIT thrown by the factory triggers fallthrough to the
  buffered path (preserving today's behaviour); any other pre-stream
  factory error surfaces a 502.
- X-OLP-Streaming-Inflight: source | attached header per §11 (cache_hit
  role omits — X-OLP-Cache: hit already says it). The §11 'solo' value
  is deferred to a future amendment — observable only post-stream via
  the streaming_inflight_source_done log event's attached_count.
- auditCtx.cache_status: 'miss' for source, 'streaming_attached' for
  joiners, 'hit' for the TTL-race cache_hit branch.
- res.on('close', () => stream.return?.()) propagates client disconnect
  into the tee's attachedClients accounting (§9). Note: Node 25 emits
  'close' on ServerResponse, NOT on IncomingMessage — empirically
  verified in test 28g.
- Cache writes now happen inside the cache layer's tee task on source
  completion (§4). Server still issues cacheStore.delete on stop-less
  exhaustion to preserve D16 truncated-not-cached invariant — the cache
  layer is IR-agnostic and writes accumulatedChunks unconditionally;
  the IR-aware server deletes the entry if no stop chunk was observed.
- The pre-cache-store-acquire and matching releaseSpawn-on-503 branches
  are gone — they were vestigial once the factory owns acquire+release.

lib/audit.mjs — JSDoc cache_status enum extended with 'streaming_attached'.
Free-form string at the wire (no schema validator on append); the JSDoc
is the source of truth for the consumer enum.

test-features.mjs Suite 28 — 7 HTTP integration tests:
- 28a single SSE request (source role + cache populated + second request
  → X-OLP-Cache: hit)
- 28b 2 concurrent identical SSE → one spawn, source + attached roles,
  identical chunk sequences delivered
- 28c TOCTOU regression — pre-populated cache + 2 concurrent → both hit
  buffered replay path, no streaming branch entry, no
  X-OLP-Streaming-Inflight header
- 28d mid-stream join — late joiner receives accumulated burst + live tail
- 28f one-of-N disconnect — source NOT aborted; survivor completes
- 28g ALL clients disconnect → source aborted; no cache write; subsequent
  request gets fresh source spawn
- 28h CONCURRENCY_LIMIT fallthrough — factory throws at maxConcurrent=1;
  buffered path's chain-exhausted 502 surfaces

(28e backpressure deferred to Suite 27g unit-level coverage.)

## Scope

server.mjs + lib/audit.mjs + test-features.mjs. Untouched: cache/store.mjs
(D57 frozen), provider plugins, fallback engine, IR. CHANGELOG and
package.json bump fires at D59 close.

## Authority

- docs/adr/0005-cache-cross-provider.md Amendment 8 §§7, 8, 9, 11, 12
- docs/v1x-roadmap.md #1
- GitHub issue #16 (TOCTOU window; closed in D59)
- ADR 0002 Amendment 6 (D38 tryAcquireSpawn/releaseSpawn semantics that
  §7 now invokes inside the sourceFactory closure)

## Test count

615 → 622 (+7 D58 integration tests). Local: 622/622 pass.

## Iron Rule 10 follow-up notes for the reviewer

- res.on('close') vs req.on('close'): switched to res after empirical
  verification (28g fails on req under Node 25). Comment in code.
- Cache-layer write + server-layer delete for stop-less exhaustion is
  cosmetically inconsistent with streaming_inflight_source_done's
  cache_written: true log. Functionally correct; flagged for future
  amendment.
- 'solo' header value deferred — would need trailer mechanics or
  post-stream emission.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: D58 reviewer follow-ups — P2-1 (audit-query gauge drift) + P2-2 (stop-less HTTP test)

Fold-in for D58 PR #37 fresh-context opus reviewer findings (APPROVE
WITH MINOR — 0 P0/P1, 4 P2). P2-3 (`isFirst` unused) and P2-4 (`solo`
not emitted) are design-acceptable per ADR 0005 Amendment 8 §11 and
left as-is.

P2-1 — `lib/audit-query.mjs` gauge reconciliation. `aggregateRequests`
and `cacheHitRateWindow` previously counted `streaming_attached` rows
in `total` / `pe.total` without contributing to hit/miss/bypass
numerators, breaking the invariant that the cache_status breakdown
should sum to the total. Added an explicit `streaming_attached` field
to both the global return shape and the `by_provider` shape; the
counter is excluded from `hit_rate` numerator AND denominator (joiners
did not hit a literal cache so they don't belong in either side of
the ratio). Test count is unchanged for D49 suites — they only
assert presence + non-negative + reconciliation invariants that the
new field preserves; if a test asserted exact value equality on a
fixture with NO streaming_attached rows, the new field defaults to 0
and the assertion still passes.

P2-2 — Suite 28 stop-less HTTP coverage gap. Test 28i fires an SSE
request to a fake provider whose source generator returns WITHOUT a
{type:"stop"} chunk; asserts (a) the synthetic truncation marker
appears in the body (D26 F19 in-band signal), (b) [DONE] terminator
follows, (c) a subsequent identical request triggers a fresh spawn
(cache was NOT populated by the truncated stream). Pins the D58
`cacheStore.delete` path at server.mjs:1344-1346 end-to-end.

622 → 623 (+1 D58 follow-up test). 623/623 pass locally.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
2026-05-25 20:59:54 +10:00
committed by GitHub
co-authored by taodeng Claude Opus 4.7
parent 1062e88e77
commit 9b66326e72
4 changed files with 750 additions and 175 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/audit-query.mjs
+20 -6
View File
@@ -227,7 +227,7 @@ export function* readAuditWindow({ startMs, endMs, olpHome, logEvent } = {}) {
* {
* window: { startMs, endMs },
* request_count, status_2xx, status_4xx, status_5xx,
* by_provider: { [providerKey]: { count, cache_hit, cache_miss, cache_bypass, fallback_count } },
* by_provider: { [providerKey]: { count, cache_hit, cache_miss, cache_bypass, cache_streaming_attached, fallback_count } },
* by_owner_tier: { owner: N, guest: N, anonymous: N },
* by_path: { '/v1/chat/completions': N, '/v1/models': N, ... },
* median_latency_ms, p95_latency_ms,
@@ -276,12 +276,17 @@ export function aggregateRequests({ windowMs, olpHome, logEvent, _nowFn } = {})
// By provider
if (typeof ev.provider === 'string' && ev.provider.length > 0) {
const p = result.by_provider[ev.provider] ??= {
count: 0, cache_hit: 0, cache_miss: 0, cache_bypass: 0, fallback_count: 0,
count: 0, cache_hit: 0, cache_miss: 0, cache_bypass: 0, cache_streaming_attached: 0, fallback_count: 0,
};
p.count++;
if (ev.cache_status === 'hit') p.cache_hit++;
else if (ev.cache_status === 'miss') p.cache_miss++;
else if (ev.cache_status === 'bypass') p.cache_bypass++;
// D58 — ADR 0005 Amendment 8 §11 + lib/audit.mjs cache_status enum: streaming
// singleflight joiners (attached) share the source spawn but did not hit a
// cache. Tracked separately so `count` and `cache_hit + cache_miss +
// cache_bypass + cache_streaming_attached` reconcile.
else if (ev.cache_status === 'streaming_attached') p.cache_streaming_attached++;
if (typeof ev.fallback_hops === 'number' && ev.fallback_hops > 0) p.fallback_count++;
}
@@ -443,7 +448,12 @@ export function spendTrendDaily({ days, olpHome, logEvent, _nowFn } = {}) {
* `cacheStore.stats()` in server.mjs: that is the live in-process counter;
* this is the audit-side rate scoped to the rolling window.
*
* { window: { startMs, endMs }, total, hit, miss, bypass, hit_rate, by_provider }
* { window: { startMs, endMs }, total, hit, miss, bypass, streaming_attached, hit_rate, by_provider }
*
* `streaming_attached` (D58, ADR 0005 Amendment 8 §11): D58 streaming
* singleflight joiners did not hit a literal cache, so they are excluded
* from both numerator AND denominator of `hit_rate`. Tracked separately
* so the count reconciles with `total = hit + miss + bypass + streaming_attached`.
*
* @param {object} args
* @param {number} args.windowMs
@@ -459,18 +469,22 @@ export function cacheHitRateWindow({ windowMs, olpHome, logEvent, _nowFn } = {})
const startMs = now - windowMs;
const endMs = now;
let total = 0, hit = 0, miss = 0, bypass = 0;
let total = 0, hit = 0, miss = 0, bypass = 0, streaming_attached = 0;
const by_provider = {};
for (const ev of readAuditWindow({ startMs, endMs, olpHome, logEvent })) {
if (ev.cache_status === null || ev.cache_status === undefined) continue;
total++;
const p = typeof ev.provider === 'string' && ev.provider.length > 0 ? ev.provider : '__unknown__';
const pe = by_provider[p] ??= { total: 0, hit: 0, miss: 0, bypass: 0, hit_rate: 0 };
const pe = by_provider[p] ??= { total: 0, hit: 0, miss: 0, bypass: 0, streaming_attached: 0, hit_rate: 0 };
pe.total++;
if (ev.cache_status === 'hit') { hit++; pe.hit++; }
else if (ev.cache_status === 'miss') { miss++; pe.miss++; }
else if (ev.cache_status === 'bypass') { bypass++; pe.bypass++; }
// D58 — ADR 0005 Amendment 8 §11: streaming singleflight joiners.
// Excluded from hit_rate numerator + denominator (they did not hit a
// literal cache); tracked so `total` reconciles.
else if (ev.cache_status === 'streaming_attached') { streaming_attached++; pe.streaming_attached++; }
}
// Compute hit_rate per provider + overall (excludes bypass from denominator
@@ -484,6 +498,6 @@ export function cacheHitRateWindow({ windowMs, olpHome, logEvent, _nowFn } = {})
return {
window: { startMs, endMs },
total, hit, miss, bypass, hit_rate, by_provider,
total, hit, miss, bypass, streaming_attached, hit_rate, by_provider,
};
}
lib/audit.mjs
+14
View File
@@ -174,6 +174,20 @@ export function _maybeRotateAudit(args = {}) {
* fallback_hops, tried_providers, error_code, ir_request_hash, chain_id).
* Caller is responsible for populating fields; missing fields are
* serialized as undefined → omitted by JSON.stringify.
*
* cache_status enum (free-form string; not schema-validated at append):
* 'hit' — served from cache (buffered-replay or streaming
* cache_hit role from ADR 0005 Amendment 8 §1).
* 'miss' — buffered or streaming source path; provider
* spawn fired for this request.
* 'bypass' — D2 cache_control bypass (no cache read/write).
* 'streaming_attached' — D58 / ADR 0005 Amendment 8 §11: client joined
* an in-flight streaming source spawn from
* another caller; this request did NOT spawn a
* provider but also did NOT hit the cache (the
* cache was empty at the inflight Map check).
* null — pre-chain error paths (the cache layer was
* never consulted; e.g. 401, 415, 400 IR).
* @param {object} [opts]
* @param {string} [opts.olpHome] - test override; defaults to ~/.olp
* @param {(level: string, event: string, data?: object) => void} [opts.logEvent]