fix(fallback)+docs(adr-0004): D16 — honor "usable chunks streamed" qualifier on SPAWN_FAILED

cold-audit catch from 2026-05-23

Cold-audit Finding 17 (P2 fallback correctness). ADR 0004 § Trigger
taxonomy Hard triggers bullet 3 says "Provider CLI exit code ≠ 0
**with no usable response chunks streamed**" — the qualifier was
not honored. Pre-D16 code in `collectAllChunks` re-raised SPAWN_FAILED
unconditionally, discarding any partial chunks. Concrete failure:
provider yields 1000 chars of completion then exits non-zero (e.g.,
post-stream cleanup error) → chunks dropped, fallback to next provider,
user pays double spawn cost and loses the original provider's output.

Coordinated change across two layers, single commit per the
ADR-with-code pattern (D11 / D15 precedent):

1. docs/adr/0004-fallback-engine.md — Amendment 1 (top of doc, matching
   D11/D15 placement convention):
   - Documents the "usable chunks streamed" semantics precisely
   - Behavior split: chunks.length > 0 + SPAWN_FAILED → synthesize stop
     + return (Case B); chunks.length === 0 + SPAWN_FAILED → re-throw
     (Case A, hard trigger fires as before)
   - finish_reason='length' rationale (4 reasons documented)
   - Streaming-path note: ADR 0004 first-chunk rule already handles the
     analogous case for D10's real-streaming branch; D16 applies to
     buffered path only
   - Cache behavior: write-through `getOrCompute` (preserves D4 singleflight
     during truncation event) then evict via `set(ttlMs=0)` (so future
     fresh callers re-spawn). `__truncated` non-enumerable marker
     travels with the chunks array for follower visibility

2. server.mjs `collectAllChunks` salvage path:
   - try/catch around the for-await loop
   - On SPAWN_FAILED with chunks.length > 0: synthesize stop chunk
     `{type:'stop', finish_reason:'length'}`, log warn event
     `spawn_failed_after_usable_chunks`, mark chunks array with
     non-enumerable `__truncated`, return (no re-throw)
   - On SPAWN_FAILED with chunks.length === 0 OR any other error:
     re-throw (preserves existing hard-trigger semantics)

3. server.mjs `executeHopFn` cache eviction:
   - After `cacheStore.getOrCompute(...)` returns, check `result.__truncated`
   - If truncated: `cacheStore.set(keyId, hopCacheKey, result, 0)` —
     ttlMs=0 causes `_isAlive` to treat the entry as expired on next
     read (verified in lib/cache/store.mjs)

4. test-features.mjs Suite 13 — 3 new tests:
   - Case A regression: SPAWN_FAILED at iter 0 + 2-hop chain → openai
     serves, X-OLP-Fallback-Hops: 1 (no behavior change)
   - Case B 2-hop: 2 deltas + SPAWN_FAILED → anthropic serves with
     synthesized stop, hops=0, finish_reason='length', content
     concatenates, openai NOT called
   - Case B single-hop: 1 delta + SPAWN_FAILED → HTTP 200 (not 502),
     finish_reason='length', partial content visible

Tests: 297 → 300 (+3). All pass on Node 20.

Pre-commit fold-ins (per evidence-first checkpoint #4 — fold-ins
themselves need second-pass review):

- **Error-chunk-in-chunks fold-in (sonnet flagged)**: pre-D12 code
  pushed error chunks BEFORE throwing. Post-D16's `chunks.length > 0`
  check would incorrectly include an error chunk and trigger Case B
  for a path that's actually Case A. Moved the `type === 'error'`
  check BEFORE the push, restoring the invariant that the chunks
  array contains only delta/stop chunks. Verified all 3 scenarios:
  (1) error at iter 0 → throws before push → length=0 → Case A
  (2) delta×2 + error at iter 3 → throws before push → length=2 → Case B
      with delta×2 + synthesized stop (no error chunk leaks)
  (3) delta + non-zero exit from outside loop → length=1 → Case B

- **ADR doc-code drift fold-in (D16 reviewer flagged)**: original
  Amendment 1 text said the salvaged result "bypasses
  cacheStore.getOrCompute and is returned directly, exactly as the
  cache-bypass path does." This was factually wrong — the code
  write-throughs via getOrCompute then evicts via ttlMs=0. The drift
  was ironic: D16 was about removing doc-code drift in ADR 0004
  bullet 3 itself, and the amendment was about to ship fresh drift.
  Corrected to accurately describe the write-then-evict pattern and
  the rationale (preserving D4 singleflight during truncation events).

Authority:
- ADR 0004 § Trigger taxonomy Hard triggers bullet 3 (the qualifier
  this amendment makes load-bearing)
  https://github.com/dtzp555-max/olp/blob/main/docs/adr/0004-fallback-engine.md
- ADR 0005 § Cache write conditions item 1 — "response completed
  successfully (no truncation, no error mid-stream)"
- OpenAI Chat Completions finish_reason enum (stop|length|tool_calls|
  content_filter|function_call|null)
  https://platform.openai.com/docs/api-reference/chat/object
- ADR 0004 § Fallback safety — first-chunk rule (already governs the
  analogous case in the real-streaming path)
- ALIGNMENT.md Rule 2(c) spirit — ADR amendment + code change land
  in same merge (D11 / D15 precedent)
- CC 开发铁律 v1.6 § 10.x — Cold Audit caught this; diff-review
  passes focused on first-chunk rule for streaming missed the
  buffered path's truncation-vs-fallback decision point

Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): APPROVE_WITH_MINOR. Folded the ADR doc-code drift minor
before commit. Walked all 3 error-chunk scenarios against actual code
to verify the pre-commit fold-in is correct. Analyzed the eviction
race window (sub-ms post-inflight pre-eviction window where a fresh
caller could hit the truncated cache before eviction lands) and
concluded it's structurally bounded — one-shot leak per truncation
event; subsequent callers re-spawn. Acceptable as v0.1.

Follow-up items (reviewer's non-blocking suggestions, NOT in this PR):
- 4th test asserting second identical request triggers fresh spawn
  (defense-in-depth around the eviction; store.mjs ttlMs=0 semantics
  are independently established)
- `cacheStore.delete()` API (cleaner than set-with-ttlMs=0 — leaves
  no dead entry in the namespace map; future PR)
- `cache_evicted_truncated` log event for dashboard observability
- SPAWN_TIMEOUT salvage parity — same architectural argument as
  SPAWN_FAILED (user paid for partial content); deferred as a
  separate cold-audit finding for a future D-stage

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

server.mjs

@@ -368,17 +368,54 @@ async function handleChatCompletions(req, res) {
     // Collect all chunks from this provider, throwing on error chunks.
     // Error semantics: ProviderError thrown here propagates to executeWithFallback
     // which decides whether to advance the chain.
+    //
+    // D16 (ADR 0004 Amendment 1): if the generator throws ProviderError SPAWN_FAILED
+    // after yielding one or more chunks, those chunks are usable content the provider
+    // committed — dropping them and falling back to the next hop is strict waste.
+    // In that case: synthesize a stop chunk with finish_reason='length', return the
+    // partial chunks, and do NOT re-throw (fallback engine sees a success, hops=0).
+    // If chunks.length===0 on SPAWN_FAILED, re-throw as before (hard trigger fires).
+    // Any other error code always re-throws unchanged.
+    //
+    // D16 cache note: truncated responses are NOT cached (ADR 0005 § cache write
+    // conditions item 1 requires no truncation). A non-enumerable __truncated marker
+    // is set on the returned array so executeHopFn can evict the cache entry below.
     async function collectAllChunks() {
       const chunks = [];
-      for await (const irChunk of hopProviderPlugin.spawn(irReq, authContext)) {
-        chunks.push(irChunk);
-        if (irChunk.type === 'error') {
-          throw new ProviderError(
-            irChunk.error ?? 'Provider emitted error chunk',
-            'SPAWN_FAILED',
-          );
+      try {
+        for await (const irChunk of hopProviderPlugin.spawn(irReq, authContext)) {
+          // D16: check error chunks BEFORE pushing — preserves the invariant that
+          // chunks array contains only delta/stop chunks. Without this, the catch
+          // block's `chunks.length > 0` would mistake a single error chunk for
+          // "usable content streamed" (Case B) and synthesize a stop + return,
+          // sending an empty body to the client when the correct behavior is to
+          // re-throw and let the fallback engine advance the chain.
+          if (irChunk.type === 'error') {
+            throw new ProviderError(
+              irChunk.error ?? 'Provider emitted error chunk',
+              'SPAWN_FAILED',
+            );
+          }
+          chunks.push(irChunk);
+          if (irChunk.type === 'stop') break;
         }
-        if (irChunk.type === 'stop') break;
+      } catch (spawnErr) {
+        if (spawnErr instanceof ProviderError && spawnErr.code === 'SPAWN_FAILED' && chunks.length > 0) {
+          // Case B (ADR 0004 Amendment 1): provider emitted usable chunks then exited
+          // non-zero. Synthesize a truncated stop and surface the partial response.
+          chunks.push({ type: 'stop', finish_reason: 'length' });
+          logEvent('warn', 'spawn_failed_after_usable_chunks', {
+            chunks_count: chunks.length - 1, // exclude the synthesized stop
+            provider: hopProvider,
+            model: hopModel,
+          });
+          // Mark as truncated so the caller can evict this entry from cache.
+          Object.defineProperty(chunks, '__truncated', { value: true, enumerable: false });
+          return chunks;
+        }
+        // Case A (SPAWN_FAILED with no chunks) or any other error: re-throw.
+        // Fallback engine fires hard trigger and advances chain as before.
+        throw spawnErr;
       }
       return chunks;
     }
@@ -398,7 +435,21 @@ async function handleChatCompletions(req, res) {
     // D4 singleflight + D1 per-key isolation per ADR 0005.
     // Each hop has its own (provider, model) key — cross-provider contamination
     // is structurally impossible (ADR 0005 § Per-model isolation).
-    return cacheStore.getOrCompute(keyId, hopCacheKey, collectAllChunks);
+    //
+    // D16: after getOrCompute returns, check if collectAllChunks used the salvage
+    // path (chunks.__truncated===true). If so, evict the just-written cache entry
+    // to satisfy ADR 0005 § "Cache write conditions" item 1 (no truncation).
+    // D4 singleflight is preserved: getOrCompute still deduplicates concurrent
+    // requests during the spawn; the eviction only affects persistent caching.
+    const result = await cacheStore.getOrCompute(keyId, hopCacheKey, collectAllChunks);
+    if (result.__truncated) {
+      // Evict the truncated entry so future requests get a fresh spawn.
+      // ADR 0005 § "Cache write conditions" item 1: truncated responses must not
+      // persist in cache. Overwrite with ttlMs=0 so any subsequent get/peek
+      // finds the entry already-expired and deletes it.
+      await cacheStore.set(keyId, hopCacheKey, result, 0);
+    }
+    return result;
   }
 
   // ── Execute with fallback (ADR 0004) ────────────────────────────────────