mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-19 09:45:07 +00:00
feat+docs+test: D39 — D16 follow-ups (issue #3, 4 parts)
D16 reviewer (commit `bafa6d1`) left 4 non-blocking suggestions
batched into issue #3 as a tracker. D39 closes all 4.
**Part 1 — CacheStore.delete(keyId, cacheKey) API** (lib/cache/store.mjs)
D16 originally evicted truncated entries via
`cacheStore.set(keyId, hopCacheKey, result, ttlMs=0)` — a TTL=0
tombstone purged lazily on next access. D39 introduces an explicit
delete primitive that removes the entry immediately.
- Synchronous: `delete(keyId, cacheKey) → boolean`. Returns true if
the entry was present and removed, false if absent. Sync (not
async) for the simplest in-memory Map contract — mirrors clear().
Other CacheStore methods are async to leave room for a Phase 2
file-backed adapter; delete being sync was a deliberate choice.
- Namespace cleanup: when the inner Map becomes empty after delete,
the outer Map's per-keyId entry is also removed (memory hygiene;
mirrors the _activeSpawns cleanup pattern from D38).
- Behavior: peek/get/getOrCompute see no trace after delete; the
subsequent getOrCompute triggers a fresh compute.
**Part 2 — `cache_evicted_truncated` observability log** (server.mjs)
After the D16 eviction call in collectAllChunks, emit:
```js
logEvent('info', 'cache_evicted_truncated', {
provider, model, cache_eviction_hit,
});
```
Dashboard sees salvage frequency per (provider, model). The
cache_eviction_hit boolean distinguishes "we evicted an entry" (true)
from "we tried to evict but it was already gone" (false — race with
concurrent eviction or TTL purge), preserving observability accuracy
under concurrency.
**Part 3 — Sticky-cache regression test** (test-features.mjs)
Defense-in-depth around the eviction path. Two consecutive identical
buffered requests; the first triggers SPAWN_FAILED after partial
chunks → Case B salvage returns `{ chunks..., finish_reason: 'length' }`
to the client and evicts via delete(). The second identical request
must trigger a fresh spawn (NOT serve the salvaged response from a
stale cache entry).
Asserts on BOTH invariants for defense-in-depth:
- Mock provider spawn count == 2 across the 2 identical requests
- Second request's X-OLP-Cache header is 'miss'
If eviction silently breaks in a future regression, both assertions
catch it independently.
**Part 4 — SPAWN_TIMEOUT salvage parity: DOCUMENT ASYMMETRY**
(docs/adr/0004-fallback-engine.md)
Maintainer decision: SPAWN_TIMEOUT is NOT salvaged. Document the
asymmetry rather than implementing parity. ADR 0004 Amendment 1 is
extended with a new section "Why SPAWN_TIMEOUT is excluded from
salvage" with 4-point rationale:
1. SPAWN_FAILED indicates the provider crashed mid-stream — there's
nothing more coming; partial > nothing. Next-hop spawn has no
advantage (same input may crash same way).
2. SPAWN_TIMEOUT indicates the provider was slow (deadline exceeded
per `hints.maxSpawnTimeMs`). Fallback advancement to a DIFFERENT
provider is more likely to give a complete response than salvaging
a partial from a slow provider.
3. The "user paid for partial content" framing from D16 captures only
SPAWN_FAILED. For SPAWN_TIMEOUT the user actually paid for "result
within time T" — partial-at-time-T is not what was paid for;
"full result soon after T" via fallback is closer.
4. Code-level inspection confirms the asymmetry: collectAllChunks
catch matches ONLY `code === 'SPAWN_FAILED'` (server.mjs:563).
SPAWN_TIMEOUT propagates via re-throw and hits evaluateHardTriggers.
v1.x re-evaluation trigger: if real usage shows users want partial-
on-timeout for very long deadlines, add a v1.x design ADR.
Stale comment fix: `lib/providers/anthropic.mjs:369` previously said
"SPAWN_TIMEOUT salvage parity is tracked in issue #3". D39 closes
that issue, so the comment is updated to point at ADR 0004 Amendment 1.
**Tests** (test-features.mjs): 447 → 452 (+5):
- 3 unit tests on CacheStore.delete (Suite 9): present-key true, absent-key
false, namespace cleanup at empty
- 1 D16 integration test: cache_evicted_truncated log fires with
correct fields during salvage
- 1 sticky-cache regression: spawn count 2 across 2 identical requests,
X-OLP-Cache miss on second
Pre-commit fold-ins (per evidence-first checkpoint #4):
- **Reviewer Suggestion #1**: cacheStore.delete() return value was
discarded at the call site → log inflated salvage metric under
concurrent-eviction race. Folded: captured `evicted` boolean and
added to log payload as `cache_eviction_hit`.
- **Reviewer Suggestion #2**: anthropic.mjs:369 stale comment
pointing at now-closed issue #3. Folded: rewrote to point at
ADR 0004 Amendment 1 § "Why SPAWN_TIMEOUT is excluded from
salvage".
- **Reviewer Suggestion #3**: ADR 0004 attribution ambiguity —
parenthetical "(per Amendment 3 — SPAWN_TIMEOUT is one of the 4
live hard-trigger codes alongside SPAWN_FAILED, CLI_NOT_FOUND, and
CONCURRENCY_LIMIT from Amendment 4)" could mis-parse as Amendment 3
covering all four. Folded: split to
"(per Amendment 3: SPAWN_FAILED, CLI_NOT_FOUND, SPAWN_TIMEOUT;
per Amendment 4: CONCURRENCY_LIMIT)".
**CHANGELOG**: D39 sub-entry appended under the existing D38 entry
in Unreleased section. No package.json bump (phase_rolling_mode).
Authority:
- ADR 0005 § Cache layer — CacheStore API extension (Part 1)
- ADR 0004 Amendment 1 update — SPAWN_TIMEOUT asymmetry rationale (Part 4)
- GitHub issue #3 — closed by this commit
- D16 commit bafa6d1 non-blocking suggestions — batched here
- CC 开发铁律 v1.6 § 10.x — fresh-context opus reviewer independent
- CLAUDE.md release_kit_overlay phase_rolling_mode — under Unreleased
Reviewer (fresh-context opus, Iron Rule 10): APPROVE_WITH_MINOR.
Verified: delete() sync + call-site no-await correct; namespace
cleanup guarded on (had && ns.size === 0); SPAWN_TIMEOUT NOT in
salvage catch; ADR section 4-point rationale internally consistent
with code state; hygiene clean; 452/452 tests pass independently.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -52,6 +52,26 @@
|
||||
|
||||
- **Streaming path note:** The D10 real-streaming branch (single-hop, `server.mjs` lines 401–510) already handles the analogous case correctly via ADR 0004's first-chunk rule: once `firstChunkEmitted === true`, any subsequent error truncates the response with `res.end()` (no re-throw, no fallback). This amendment applies specifically to the **buffered path** (`collectAllChunks` + multi-hop fallback chains). The streaming path is not changed by D16.
|
||||
|
||||
#### D39 follow-up — explicit eviction primitive + observability log (2026-05-24, issue #3 Parts 1+2)
|
||||
|
||||
The original D16 cache-eviction implementation used `cacheStore.set(keyId, hopCacheKey, result, ttlMs=0)` to tombstone the just-written truncated entry. The TTL=0 entry survived in the per-keyId namespace `Map` until the next `get`/`peek` lazily purged it via the `_isAlive` check. D39 Part 1 replaces this with an explicit `cacheStore.delete(keyId, cacheKey)` primitive that (a) removes the entry from the namespace `Map` immediately, (b) removes the empty namespace `Map` entry from the outer store when it becomes empty (memory hygiene matching the D38 `_activeSpawns` pattern), and (c) returns `boolean` for caller inspection. D39 Part 2 adds a `cache_evicted_truncated` `info`-level log event with `{ provider, model }` fields immediately after the eviction, giving dashboards visibility into salvage frequency. Neither change alters the salvage semantics established by this Amendment — they are observability + memory-hygiene polish. Test coverage: 3 unit tests on `CacheStore.delete` (present-returns-true, absent-returns-false, empty-namespace-cleanup), 1 HTTP integration test asserting the log event fires with the correct fields, and 1 defense-in-depth regression test asserting two consecutive identical truncated requests both result in fresh spawns (no sticky cache).
|
||||
|
||||
#### Why SPAWN_TIMEOUT is excluded from salvage (D39 Part 4, issue #3 Part 4)
|
||||
|
||||
D16's salvage path is gated on `code === 'SPAWN_FAILED'`. SPAWN_TIMEOUT is **not** salvaged even when partial chunks have accumulated in the buffered path — the timeout error propagates from `collectAllChunks` as-is, the fallback engine fires the SPAWN_TIMEOUT hard trigger, and the chain advances to the next hop. This asymmetry is intentional and is the maintainer's design choice. The four-point rationale:
|
||||
|
||||
1. **SPAWN_FAILED is a terminal signal from this hop.** The provider crashed mid-stream; nothing more is coming from it. Salvaging the partial chunks is strictly better than discarding them (partial > nothing). Advancing the chain in this case offers no advantage: the same input may crash the next hop the same way (when the failure is input-dependent), and even when the next hop succeeds, the salvaged chunks were already paid for in quota — discarding them would be strict waste.
|
||||
|
||||
2. **SPAWN_TIMEOUT is a deadline signal, not a terminal signal.** It indicates the provider was slow (deadline exceeded per `hints.maxSpawnTimeMs`, which the plugin enforces — see the unconditional post-loop `if (spawnTimedOut) throw SPAWN_TIMEOUT` in each provider plugin, e.g. `lib/providers/anthropic.mjs`). The next hop is a *different provider* with different model-speed characteristics, so its full response is plausibly available sooner than the original hop's continuation would have been. Fallback advancement on timeout is more likely to give the user a complete response than salvaging partial-from-slow.
|
||||
|
||||
3. **The "user paid for partial" framing applies only to SPAWN_FAILED.** The D16 reviewer's "user paid for partial content, dropping it is strict waste" captures SPAWN_FAILED correctly: the deadline was honored, the provider died mid-stream, the chunks are real consumed quota. For SPAWN_TIMEOUT the user actually paid for "result within time T" — a partial result delivered *at* time T is not what was paid for. The fallback engine's "full result soon after time T" via a different provider is closer to the contract.
|
||||
|
||||
4. **Code-level inspection confirms the asymmetry (verified post-D38, D39 Part 4).** `collectAllChunks` in `server.mjs` matches only `spawnErr instanceof ProviderError && spawnErr.code === 'SPAWN_FAILED' && chunks.length > 0` for the salvage branch. SPAWN_TIMEOUT propagates through the same catch block via the unconditional re-throw, hits `evaluateHardTriggers` as a hard trigger (per Amendment 3: SPAWN_FAILED, CLI_NOT_FOUND, SPAWN_TIMEOUT; per Amendment 4: CONCURRENCY_LIMIT), and advances the chain. This asymmetry is not an oversight; it is the design.
|
||||
|
||||
**Hard-trigger taxonomy completeness:** The v0.1 hard-trigger code set is enumerated in Amendment 3 (D34 F7) and extended in Amendment 4 (D38, CONCURRENCY_LIMIT). Of those four codes, only SPAWN_FAILED participates in the salvage path. CLI_NOT_FOUND fires before any spawn output is possible (no partial chunks ever exist). CONCURRENCY_LIMIT fires before `provider.spawn(...)` is called (per Amendment 4 § First-chunk safety — zero bytes emitted at rejection moment). SPAWN_TIMEOUT can in principle accumulate partial chunks but is excluded from salvage per the rationale above.
|
||||
|
||||
**v1.x re-evaluation trigger:** If real usage shows users want partial-on-timeout for very long deadlines (e.g., a 5-minute `maxSpawnTimeMs` where the user would rather have whatever streamed in 5 minutes than re-pay quota on a different provider that may take its own 5 minutes), this asymmetry is queued as a future-design question. A v1.x amendment would need to: (a) make salvage-on-timeout opt-in per chain or per provider (default-off preserves v0.1 semantics), (b) extend `collectAllChunks` catch matching to a broader code set, (c) add tests parallel to the D16 Case A/Case B/single-hop trio for the SPAWN_TIMEOUT path. Not a v0.1 issue.
|
||||
|
||||
- **Procedural mechanism:** CC 开发铁律 v1.6 § 10.x (Cold Audit). Diff-review reviewers on earlier passes focused on the first-chunk rule for the real-streaming path; the buffered path has its own truncation-vs-fallback decision point, which the cold-audit pass on 2026-05-23 identified as Finding 17.
|
||||
|
||||
### Amendment 2 — 2026-05-24: Soft triggers deferred to v1.x (D22)
|
||||
|
||||
Reference in New Issue
Block a user