mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-21 21:15:10 +00:00
de9f3ca7c90a9a6dff1d6eb1dce56751e5b5a479
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  taodengandClaude Opus 4.7
|
7ef5510837 |
feat(cache): D23 — implement hints.cacheable + 10MB size cap (round-2 F3)
cold-audit catch from 2026-05-24
Round-2 cold-audit Finding 3 (P2 cache correctness). ADR 0005 § "Cache
write conditions" items 3 and 4 were documented but never wired in
code:
- Item 3: "The provider's hints.cacheable flag is not false"
- Item 4: "The response is below a size cap (default 10 MB; configurable)"
Grep verified zero matches for `cacheable` / `10485760` / size-cap
patterns in lib/ or server.mjs pre-D23.
Changes (9 files, +391 / -12):
1. docs/adr/0002-plugin-architecture.md — Amendment 3 adds `cacheable`
to the Provider contract hints list (after D11's Amendment 1 added
maxSpawnTimeMs). Authority chain cites ADR 0005 § Cache write
conditions item 3 as the field's origin.
2. docs/adr/0005-cache-cross-provider.md — Amendment 3 documents the
D23 implementation of items 3 + 4 + the D16-interaction edge case
(truncated > 10MB → no-op eviction, structurally bounded since
responses > 10MB are anomalous by ADR's own rationale).
3. lib/providers/base.mjs — ProviderHints typedef gains
`[cacheable]` (optional boolean); validateProvider rejects non-
boolean non-undefined values. Omission accepted (default = true).
4. 3 plugins (anthropic / codex / mistral) each declare
`cacheable: true` explicitly with citation comment.
5. lib/cache/store.mjs — CacheStore constructor accepts
`maxEntryBytes` (default 10 * 1024 * 1024 = 10_485_760) +
injectable `_warnFn`. `set()` computes
`Buffer.byteLength(JSON.stringify(value))`; if exceeded, warns via
`_warnFn` and returns undefined (no persistence). `getOrCompute`
still returns the computed value to caller — cache write skipped
but caller gets data; subsequent identical requests re-spawn.
6. server.mjs — 4 sites coordinated for cacheable opt-out:
- `executeHopFn`: cacheable check before D13 shouldBypassCacheForHop
(permanent provider policy precedes per-request bypass condition)
- `cacheStore.peek` gate at line ~504: `cacheableForFirstHop`
short-circuit
- Real-streaming branch entry condition at line ~522:
`cacheableForFirstHop` added (so cacheable: false + stream falls
through to buffered path which honors the opt-out via executeHopFn)
- Both `cacheStore.set` sites in streaming branch wrapped in
`if (cacheableForFirstHop)` defensive guards (post-D23
restructure these are unreachable for cacheable: false, but the
guards make intent explicit and survive future refactors)
7. test-features.mjs — 13 new tests:
- 5 validator tests (Suite 4): explicit true/false, omitted, string
rejected, number rejected
- 5 size-cap unit tests (Suite 9): default 10MB, custom override,
oversize skip + warn capture, within-limit normal persistence,
getOrCompute oversize returns-but-doesn't-cache + re-spawn
- 3 cacheable integration tests (Suite 9e): non-streaming opt-out,
streaming opt-out (the regression case that pre-fold-in failed),
X-OLP-Cache header consistency on both paths
Tests: 335 → 348 (+13). All pass on Node 20.
Pre-commit fold-in (per evidence-first checkpoint #4):
- **D23 reviewer flagged 2 blocking issues**: (1) the cacheable opt-out
in initial implementation was only in `executeHopFn` (buffered path);
the D10 real-streaming branch in server.mjs bypassed the check
entirely — calling streamPlugin.spawn() directly and writing to
cacheStore.set() at 2 sites without consulting cacheable. (2) Suite
9e integration tests didn't cover stream: true so the leak wasn't
caught.
Both diff-review and the implementer focused on `executeHopFn`
because that's where the cold-audit reviewer pointed for Finding 3.
Same class of "narrow attention" miss as several earlier D-days.
Fold-in: compute `cacheableForFirstHop` once at request entry; add
`!cacheableForFirstHop` short-circuit to peek gate; add
`cacheableForFirstHop` to streaming-branch entry condition (forces
fall-through to buffered path which has the opt-out); add defensive
guards on both `cacheStore.set` call sites. Added a 3rd Suite 9e
test covering stream: true + cacheable: false (which pre-fold-in
would have failed by serving the second request from cache).
This is now the FOURTH D-day where a doc-vs-code or path-coverage
gap was caught by the reviewer rather than the implementer. The
v1.6 § 10.x diff-review discipline continues to pay off.
Default behavior unchanged for 3 shipped plugins (all explicitly
`cacheable: true` → cache path identical to pre-D23).
Authority:
- ADR 0002 Amendment 3 (in-place) — establishes cacheable in contract
- ADR 0005 Amendment 3 (in-place) — documents implementation of items
3 + 4
- ADR 0005 § Cache write conditions items 3 + 4 — the original
authority for both rules
- CC 开发铁律 v1.6 § 10.x — Round-2 Cold Audit caught the missing
implementation; diff-review Mode A caught the streaming-path gap
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): REQUEST_CHANGES on initial, APPROVE after fold-in (implicit
— fold-in followed the exact recommendation). Verified:
- ADR amendment placement + structure
- Validator typedef + checks
- Size cap implementation in CacheStore + inflight slot release on
oversize-skip
- All 4 interaction cases (cacheable × cache_control × D16
× ordering) coherent post-fold-in
- 13 new tests including the regression test that would have failed
on pre-fold-in code
Follow-up items (reviewer's non-blocking notes, NOT in this PR):
- ADR 0005 Amendment 3 could add one sentence on the prior-write-also-
oversize case (file as docs polish)
- Consider extracting `shouldUseCacheForHop(hopProvider, ir)` helper
combining D13 + D23 logic — reduces miss-risk for next reviewer
- Test 30 could add `assert.equal(store._inflight.size, 0)` as
inflight-slot leak regression guard
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
taodengandClaude Opus 4.7 (noreply@anthropic.com)
|
8dd02e77ac |
feat(phase-1): land cache layer (D1+D4) + Anthropic E2E gate (D5)
Phase 1 Day 3. ADR 0005 cache layer ships: content-hash key over (provider,
model, IR), D1 per-key isolation, D4 singleflight. server.mjs wires cache
into the dispatch path with X-OLP-Cache: hit|miss|bypass header. Suite 10
adds a gated real Anthropic spawn E2E (OLP_RUN_E2E=1) — orchestrator ran
it once on Mac mini: 7.2s wall-clock, real claude-haiku-4-5 spawn via
keychain OAuth, returned "OK", asserted X-OLP-Provider-Used: anthropic +
X-OLP-Cache: miss on first request. The plugin chain (IR to claude -p to
IR to OpenAI) now works end-to-end on a real binary.
Files:
NEW: lib/cache/keys.mjs (199 lines) — computeCacheKey + cache_control
extraction. sha256(stable-JSON(provider, model, normalized messages,
tools, temperature, response_format, cache_control_markers)).
NEW: lib/cache/store.mjs (348 lines, includes peek() fold-in) — in-memory
CacheStore with D1 per-key isolation (nested Maps), D4 singleflight
via getOrCompute(keyId, cacheKey, computeFn), TTL expiry,
injectable _nowFn for deterministic tests, stats reporting,
scoped clear(keyId?).
MOD: server.mjs (+136/-26 lines) — cache lookup before spawn, bypass on
cache_control markers, getOrCompute for D4 singleflight, header
annotation. authContext changed from {} to null so providers
correctly fall back to readAuthArtifact.
MOD: test-features.mjs (+614 lines, 30 new Suite 9 tests + 1 gated
Suite 10 E2E).
Test count: 98 → 128 (+30) at default. With OLP_RUN_E2E=1: 129/129
(verified by orchestrator on Mac mini, Node 25.8.0).
Authority citations:
Cache key composition: ADR 0005 § Cache key composition (v1.0). All 7
spec fields present (provider, model, normalized messages, tools,
temperature, response_format, cache_control markers).
Per-key isolation D1: ADR 0005 § D1 + OCP keys.mjs precedent (nested
Map keyId → cacheKey → entry).
Singleflight D4: ADR 0005 § D4 + OCP server.mjs inflight Promise
precedent. getOrCompute synchronously registers the inflight promise
BEFORE any await, so concurrent callers cannot observe an empty
inflight slot — JavaScript event-loop guarantee on this is the
correctness anchor.
cache_control bypass D2 basic structure: ADR 0005 § D2. Full marker-
strip-for-non-Anthropic-provider behaviour deferred (only anthropic
plugin exists at D4, so the marker-strip branch is unreachable).
Chunked stream replay D3 basic structure: ADR 0005 § D3. Full timing-
accurate replay deferred per orchestrator spec; D5 stores collected
chunks and replays sequentially without timing fidelity.
Architectural decisions:
1. In-memory cache at D5. ADR 0005 § Cache directory structure shows
~/.olp/cache/<keyId>/... as the eventual layout; D5 ships the
structural equivalent (nested Maps) in memory. File backing lands
in a later Phase. The Map structure is identical to the eventual
filesystem layout; migration is a serializer/deserializer pair.
2. keyId = '__anonymous__' at D5. Per-OLP-API-key namespacing infra
lands in Phase 2 multi-key. The constant is hardcoded in
server.mjs with a comment explaining the Phase 2 transition.
3. authContext changed from {} to null. {} ?? readAuthArtifact()
never falls back (empty object is truthy under ??); null
correctly triggers the fallback. Confirmed by D5 E2E test which
used the keychain OAuth path end-to-end.
4. cache_control side-channel via raw body. The IR translator
(lib/ir/openai-to-ir.mjs) strips cache_control because it is not
an IR v1.0 field. server.mjs bypass check uses both hasCacheControl
(ir) and extractCacheControlMarkers(body?.messages) to compensate.
The proper fix is an ADR 0003 amendment to preserve cache_control
in IR; tracked as a Phase 2 backlog item. Suite 9 Test 30 verifies
the dual-check works end-to-end over HTTP.
5. collectAllChunks throws ProviderError on type:error chunks. This
prevents cache_store.set() from being called on error-terminated
responses per ADR 0005 § Cache write conditions item 1. Anthropic
plugin currently never emits type:error chunks (throws instead),
so this is defensive code for future provider plugins.
Reviewer chain (Iron Rule 10):
Implementer: sonnet (general-purpose).
Fresh-context reviewer: opus (ecc:code-reviewer). Verdict
APPROVE_WITH_MINOR.
Reviewer ran npm test (128/128 pass with Suite 10 skipped), opened
ADR 0005 end-to-end, opened OCP keys.mjs + server.mjs to verify
singleflight precedent, verified the singleflight invariant by code
inspection (5-concurrent Test 23 plus event-loop guarantee proof).
Reviewer non-blocking findings folded in this commit:
1. peek() added to CacheStore — stats-neutral existence check.
server.mjs preCheck now uses peek() instead of has(), fixing the
hit/miss counter double-count bug. Documentation on has() updated
to point callers at peek() for stats-sensitive paths.
2. collectAllChunks now throws ProviderError on type:error chunks
instead of returning an error-terminated array, preventing cache
pollution per ADR 0005 § Cache write conditions item 1.
3. Cache key composition comment clarifies that cache_control slot is
forward-compat infrastructure for the future ADR 0003 amendment;
v1.0 IR strips cache_control so the slot is always null at the
key-composition site, with the D2 bypass side-channeling through
the raw body in server.mjs.
Reviewer findings deferred:
- IR amendment to preserve cache_control as first-class IR field
(would let server.mjs drop the dual-check). Tracked as Phase 2
backlog: "amend ADR 0003 to preserve cache_control markers in IR".
- LRU-by-linked-list eviction at maxEntriesPerKey scale. Current
O(n log n) sort-on-evict is fine at personal/family scale.
Tracked for revisit if cap is raised significantly.
Verification:
node --check on all touched files: clean.
npm test on Node 25.8.0: 128/128 pass in 210ms (default mode).
OLP_RUN_E2E=1 npm test on Mac mini: 129/129 pass in 7.4s (real
claude-haiku-4-5 spawn, "OK" response, all OLP headers correct).
hygiene grep: no personal names, no /Users literal paths, no real
OAuth tokens (test fixtures use "<fake-token>" placeholders).
Co-Authored-By: Claude Opus 4.7 (noreply@anthropic.com)
|