mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-19 09:45:07 +00:00
feat(cache): D23 — implement hints.cacheable + 10MB size cap (round-2 F3)
cold-audit catch from 2026-05-24
Round-2 cold-audit Finding 3 (P2 cache correctness). ADR 0005 § "Cache
write conditions" items 3 and 4 were documented but never wired in
code:
- Item 3: "The provider's hints.cacheable flag is not false"
- Item 4: "The response is below a size cap (default 10 MB; configurable)"
Grep verified zero matches for `cacheable` / `10485760` / size-cap
patterns in lib/ or server.mjs pre-D23.
Changes (9 files, +391 / -12):
1. docs/adr/0002-plugin-architecture.md — Amendment 3 adds `cacheable`
to the Provider contract hints list (after D11's Amendment 1 added
maxSpawnTimeMs). Authority chain cites ADR 0005 § Cache write
conditions item 3 as the field's origin.
2. docs/adr/0005-cache-cross-provider.md — Amendment 3 documents the
D23 implementation of items 3 + 4 + the D16-interaction edge case
(truncated > 10MB → no-op eviction, structurally bounded since
responses > 10MB are anomalous by ADR's own rationale).
3. lib/providers/base.mjs — ProviderHints typedef gains
`[cacheable]` (optional boolean); validateProvider rejects non-
boolean non-undefined values. Omission accepted (default = true).
4. 3 plugins (anthropic / codex / mistral) each declare
`cacheable: true` explicitly with citation comment.
5. lib/cache/store.mjs — CacheStore constructor accepts
`maxEntryBytes` (default 10 * 1024 * 1024 = 10_485_760) +
injectable `_warnFn`. `set()` computes
`Buffer.byteLength(JSON.stringify(value))`; if exceeded, warns via
`_warnFn` and returns undefined (no persistence). `getOrCompute`
still returns the computed value to caller — cache write skipped
but caller gets data; subsequent identical requests re-spawn.
6. server.mjs — 4 sites coordinated for cacheable opt-out:
- `executeHopFn`: cacheable check before D13 shouldBypassCacheForHop
(permanent provider policy precedes per-request bypass condition)
- `cacheStore.peek` gate at line ~504: `cacheableForFirstHop`
short-circuit
- Real-streaming branch entry condition at line ~522:
`cacheableForFirstHop` added (so cacheable: false + stream falls
through to buffered path which honors the opt-out via executeHopFn)
- Both `cacheStore.set` sites in streaming branch wrapped in
`if (cacheableForFirstHop)` defensive guards (post-D23
restructure these are unreachable for cacheable: false, but the
guards make intent explicit and survive future refactors)
7. test-features.mjs — 13 new tests:
- 5 validator tests (Suite 4): explicit true/false, omitted, string
rejected, number rejected
- 5 size-cap unit tests (Suite 9): default 10MB, custom override,
oversize skip + warn capture, within-limit normal persistence,
getOrCompute oversize returns-but-doesn't-cache + re-spawn
- 3 cacheable integration tests (Suite 9e): non-streaming opt-out,
streaming opt-out (the regression case that pre-fold-in failed),
X-OLP-Cache header consistency on both paths
Tests: 335 → 348 (+13). All pass on Node 20.
Pre-commit fold-in (per evidence-first checkpoint #4):
- **D23 reviewer flagged 2 blocking issues**: (1) the cacheable opt-out
in initial implementation was only in `executeHopFn` (buffered path);
the D10 real-streaming branch in server.mjs bypassed the check
entirely — calling streamPlugin.spawn() directly and writing to
cacheStore.set() at 2 sites without consulting cacheable. (2) Suite
9e integration tests didn't cover stream: true so the leak wasn't
caught.
Both diff-review and the implementer focused on `executeHopFn`
because that's where the cold-audit reviewer pointed for Finding 3.
Same class of "narrow attention" miss as several earlier D-days.
Fold-in: compute `cacheableForFirstHop` once at request entry; add
`!cacheableForFirstHop` short-circuit to peek gate; add
`cacheableForFirstHop` to streaming-branch entry condition (forces
fall-through to buffered path which has the opt-out); add defensive
guards on both `cacheStore.set` call sites. Added a 3rd Suite 9e
test covering stream: true + cacheable: false (which pre-fold-in
would have failed by serving the second request from cache).
This is now the FOURTH D-day where a doc-vs-code or path-coverage
gap was caught by the reviewer rather than the implementer. The
v1.6 § 10.x diff-review discipline continues to pay off.
Default behavior unchanged for 3 shipped plugins (all explicitly
`cacheable: true` → cache path identical to pre-D23).
Authority:
- ADR 0002 Amendment 3 (in-place) — establishes cacheable in contract
- ADR 0005 Amendment 3 (in-place) — documents implementation of items
3 + 4
- ADR 0005 § Cache write conditions items 3 + 4 — the original
authority for both rules
- CC 开发铁律 v1.6 § 10.x — Round-2 Cold Audit caught the missing
implementation; diff-review Mode A caught the streaming-path gap
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): REQUEST_CHANGES on initial, APPROVE after fold-in (implicit
— fold-in followed the exact recommendation). Verified:
- ADR amendment placement + structure
- Validator typedef + checks
- Size cap implementation in CacheStore + inflight slot release on
oversize-skip
- All 4 interaction cases (cacheable × cache_control × D16
× ordering) coherent post-fold-in
- 13 new tests including the regression test that would have failed
on pre-fold-in code
Follow-up items (reviewer's non-blocking notes, NOT in this PR):
- ADR 0005 Amendment 3 could add one sentence on the prior-write-also-
oversize case (file as docs polish)
- Consider extracting `shouldUseCacheForHop(hopProvider, ir)` helper
combining D13 + D23 logic — reduces miss-risk for next reviewer
- Test 30 could add `assert.equal(store._inflight.size, 0)` as
inflight-slot leak regression guard
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -7,6 +7,14 @@
|
||||
|
||||
## Amendments
|
||||
|
||||
### Amendment 3 — 2026-05-24: Add `cacheable` to Provider contract hints (D23)
|
||||
|
||||
- **Finding:** Cold-audit round-2 Finding 3 (P2 contract/cache-condition drift) — ADR 0005 § "Cache write conditions" item 3 references `hints.cacheable` but the field was never named in this ADR's Provider contract hints list. `validateProvider` did not enforce it; no plugin declared it; no consumer read it. The field existed only in prose.
|
||||
- **Change:** Add `cacheable: boolean (optional, default true)` to the contract hints. Plugins that omit it are treated as `cacheable: true` (backward-compatible default — preserves v0.1 behavior for the three shipped plugins). A plugin author who explicitly sets `cacheable: false` opts out of OLP's response cache entirely: `executeHopFn` in `server.mjs` skips `cacheStore.getOrCompute` and calls `collectAllChunks` directly; neither `cache.get` nor `cache.set` is called for that hop.
|
||||
- **Rationale:** Some providers may be cheap and stateful enough that caching adds risk without value (e.g., providers with strong intra-session continuity, or providers whose output is intentionally non-deterministic). Giving plugins an explicit opt-out keeps the design honest without imposing a runtime cost on the common case (all three shipped plugins are `cacheable: true`).
|
||||
- **Authority:** ADR 0005 § "Cache write conditions" item 3 — established the field; D23 ratifies it in the contract.
|
||||
- **Procedural mechanism:** CC 开发铁律 v1.6 § 10.x (Round-2 Cold Audit caught it as Finding 3).
|
||||
|
||||
### Amendment 1 — 2026-05-23: Add `maxSpawnTimeMs` to Provider contract hints (retroactive sync)
|
||||
|
||||
- **Finding:** Cold-audit Finding 4 (P2 governance violation) — commit `2cfd0b1` (D10) added `maxSpawnTimeMs` to all three provider plugins (`anthropic.mjs`, `codex.mjs`, `mistral.mjs`) and the fallback engine's spawn-timeout enforcement loop, but the Provider contract documentation in this ADR was not updated in the same merge.
|
||||
@@ -63,11 +71,12 @@ Every provider plugin exports an object conforming to:
|
||||
- `estimateCost: (request) => { inputTokens, outputTokensEstimate, currency, usd }` — best-effort, may return null
|
||||
- `quotaStatus: async (authContext) => { available, percentUsed, resetsAt, pool }` — best-effort, null if unretrievable
|
||||
- `healthCheck: async () => { ok, latencyMs, error? }` — startup and `/health` endpoint use this
|
||||
- `hints: { requiresTTY, concurrentSpawnSafe, maxConcurrent, maxSpawnTimeMs }` — fingerprint, concurrency, and timeout hints:
|
||||
- `hints: { requiresTTY, concurrentSpawnSafe, maxConcurrent, maxSpawnTimeMs, cacheable }` — fingerprint, concurrency, timeout, and cache hints:
|
||||
- `requiresTTY` — boolean; whether the provider CLI requires a TTY to produce non-interactive output (e.g., some CLIs suppress JSON output unless forced with a flag or a TTY is present).
|
||||
- `concurrentSpawnSafe` — boolean; whether the provider CLI is safe to spawn concurrently under the same auth context without rate-limit or session collisions.
|
||||
- `maxConcurrent` — integer; maximum simultaneous spawn count OLP will allow for this provider. **Declarative hint only at v0.1**: the value is type-validated at startup (`lib/providers/base.mjs`) but no runtime enforcement (semaphore / in-flight counter / spawn queue) is wired in `server.mjs` yet. Tracking issue to be filed for a follow-up that lands the runtime guard.
|
||||
- `maxSpawnTimeMs` — optional integer, milliseconds; maximum wall-clock time OLP allows for a single provider spawn before treating it as a hard fallback trigger. Defaults to `600000` (10 minutes) if absent. Used by the fallback engine's spawn-timeout enforcement loop; see ADR 0004 § Trigger taxonomy — Hard triggers bullet 4.
|
||||
- `cacheable` — optional boolean, default `true`; if explicitly set to `false`, the provider opts out of OLP's response cache entirely. `executeHopFn` skips `cacheStore.getOrCompute` and calls `collectAllChunks` directly; no cache read or write occurs for any request to this provider. Omitting the field is equivalent to `cacheable: true`. See ADR 0005 § "Cache write conditions" item 3 and Amendment 3 above. (D23)
|
||||
|
||||
**Loading model.** `lib/providers/index.mjs` is a hand-maintained static enumeration. There is no filesystem scan, no `require.context`, no dynamic discovery. Adding a provider requires:
|
||||
1. Write `lib/providers/<name>.mjs` conforming to the contract.
|
||||
|
||||
Reference in New Issue
Block a user