feat(cache): D23 — implement hints.cacheable + 10MB size cap (round-2 F3)

cold-audit catch from 2026-05-24 Round-2 cold-audit Finding 3 (P2 cache correctness). ADR 0005 § "Cache write conditions" items 3 and 4 were documented but never wired in code: - Item 3: "The provider's hints.cacheable flag is not false" - Item 4: "The response is below a size cap (default 10 MB; configurable)" Grep verified zero matches for `cacheable` / `10485760` / size-cap patterns in lib/ or server.mjs pre-D23. Changes (9 files, +391 / -12): 1. docs/adr/0002-plugin-architecture.md — Amendment 3 adds `cacheable` to the Provider contract hints list (after D11's Amendment 1 added maxSpawnTimeMs). Authority chain cites ADR 0005 § Cache write conditions item 3 as the field's origin. 2. docs/adr/0005-cache-cross-provider.md — Amendment 3 documents the D23 implementation of items 3 + 4 + the D16-interaction edge case (truncated > 10MB → no-op eviction, structurally bounded since responses > 10MB are anomalous by ADR's own rationale). 3. lib/providers/base.mjs — ProviderHints typedef gains `[cacheable]` (optional boolean); validateProvider rejects non- boolean non-undefined values. Omission accepted (default = true). 4. 3 plugins (anthropic / codex / mistral) each declare `cacheable: true` explicitly with citation comment. 5. lib/cache/store.mjs — CacheStore constructor accepts `maxEntryBytes` (default 10 * 1024 * 1024 = 10_485_760) + injectable `_warnFn`. `set()` computes `Buffer.byteLength(JSON.stringify(value))`; if exceeded, warns via `_warnFn` and returns undefined (no persistence). `getOrCompute` still returns the computed value to caller — cache write skipped but caller gets data; subsequent identical requests re-spawn. 6. server.mjs — 4 sites coordinated for cacheable opt-out: - `executeHopFn`: cacheable check before D13 shouldBypassCacheForHop (permanent provider policy precedes per-request bypass condition) - `cacheStore.peek` gate at line ~504: `cacheableForFirstHop` short-circuit - Real-streaming branch entry condition at line ~522: `cacheableForFirstHop` added (so cacheable: false + stream falls through to buffered path which honors the opt-out via executeHopFn) - Both `cacheStore.set` sites in streaming branch wrapped in `if (cacheableForFirstHop)` defensive guards (post-D23 restructure these are unreachable for cacheable: false, but the guards make intent explicit and survive future refactors) 7. test-features.mjs — 13 new tests: - 5 validator tests (Suite 4): explicit true/false, omitted, string rejected, number rejected - 5 size-cap unit tests (Suite 9): default 10MB, custom override, oversize skip + warn capture, within-limit normal persistence, getOrCompute oversize returns-but-doesn't-cache + re-spawn - 3 cacheable integration tests (Suite 9e): non-streaming opt-out, streaming opt-out (the regression case that pre-fold-in failed), X-OLP-Cache header consistency on both paths Tests: 335 → 348 (+13). All pass on Node 20. Pre-commit fold-in (per evidence-first checkpoint #4): - **D23 reviewer flagged 2 blocking issues**: (1) the cacheable opt-out in initial implementation was only in `executeHopFn` (buffered path); the D10 real-streaming branch in server.mjs bypassed the check entirely — calling streamPlugin.spawn() directly and writing to cacheStore.set() at 2 sites without consulting cacheable. (2) Suite 9e integration tests didn't cover stream: true so the leak wasn't caught. Both diff-review and the implementer focused on `executeHopFn` because that's where the cold-audit reviewer pointed for Finding 3. Same class of "narrow attention" miss as several earlier D-days. Fold-in: compute `cacheableForFirstHop` once at request entry; add `!cacheableForFirstHop` short-circuit to peek gate; add `cacheableForFirstHop` to streaming-branch entry condition (forces fall-through to buffered path which has the opt-out); add defensive guards on both `cacheStore.set` call sites. Added a 3rd Suite 9e test covering stream: true + cacheable: false (which pre-fold-in would have failed by serving the second request from cache). This is now the FOURTH D-day where a doc-vs-code or path-coverage gap was caught by the reviewer rather than the implementer. The v1.6 § 10.x diff-review discipline continues to pay off. Default behavior unchanged for 3 shipped plugins (all explicitly `cacheable: true` → cache path identical to pre-D23). Authority: - ADR 0002 Amendment 3 (in-place) — establishes cacheable in contract - ADR 0005 Amendment 3 (in-place) — documents implementation of items 3 + 4 - ADR 0005 § Cache write conditions items 3 + 4 — the original authority for both rules - CC 开发铁律 v1.6 § 10.x — Round-2 Cold Audit caught the missing implementation; diff-review Mode A caught the streaming-path gap Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent of drafter): REQUEST_CHANGES on initial, APPROVE after fold-in (implicit — fold-in followed the exact recommendation). Verified: - ADR amendment placement + structure - Validator typedef + checks - Size cap implementation in CacheStore + inflight slot release on oversize-skip - All 4 interaction cases (cacheable × cache_control × D16 × ordering) coherent post-fold-in - 13 new tests including the regression test that would have failed on pre-fold-in code Follow-up items (reviewer's non-blocking notes, NOT in this PR): - ADR 0005 Amendment 3 could add one sentence on the prior-write-also- oversize case (file as docs polish) - Consider extracting `shouldUseCacheForHop(hopProvider, ir)` helper combining D13 + D23 logic — reduces miss-risk for next reviewer - Test 30 could add `assert.equal(store._inflight.size, 0)` as inflight-slot leak regression guard Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-19 09:45:07 +00:00 · 2026-05-24 14:45:08 +10:00
parent e10b7d7cb9
commit 7ef5510837
9 changed files with 391 additions and 12 deletions
@@ -557,6 +557,38 @@ describe('Provider contract validation', () => {
    assert.deepEqual(r.errors, []);
  });

+  // ── D23: hints.cacheable validation tests (ADR 0002 Amendment 3) ──────
+  it('validateProvider accepts hints.cacheable: true (explicit)', () => {
+    const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: true } }));
+    assert.equal(r.valid, true, `Expected valid, got errors: ${r.errors.join(', ')}`);
+    assert.deepEqual(r.errors, []);
+  });
+
+  it('validateProvider accepts hints.cacheable: false (explicit opt-out)', () => {
+    const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: false } }));
+    assert.equal(r.valid, true, `Expected valid, got errors: ${r.errors.join(', ')}`);
+    assert.deepEqual(r.errors, []);
+  });
+
+  it('validateProvider accepts omitted hints.cacheable (default true)', () => {
+    // makeProvider() does not set cacheable — must still be valid.
+    const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4 } }));
+    assert.equal(r.valid, true, `Expected valid, got errors: ${r.errors.join(', ')}`);
+    assert.deepEqual(r.errors, []);
+  });
+
+  it('validateProvider rejects hints.cacheable: \'true\' (string, not boolean)', () => {
+    const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: 'true' } }));
+    assert.equal(r.valid, false);
+    assert.ok(r.errors.some(e => e.includes('cacheable')), `Expected cacheable error, got: ${r.errors.join(', ')}`);
+  });
+
+  it('validateProvider rejects hints.cacheable: 1 (number, not boolean)', () => {
+    const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: 1 } }));
+    assert.equal(r.valid, false);
+    assert.ok(r.errors.some(e => e.includes('cacheable')), `Expected cacheable error, got: ${r.errors.join(', ')}`);
+  });
+
  it('rejects non-object input', () => {
    const r = validateProvider(null);
    assert.equal(r.valid, false);
@@ -1696,7 +1728,95 @@ describe('Cache layer — CacheStore unit tests (Suite 9 cont.)', () => {
    assert.equal(callCount, 2, 'computeFn should be retried after error');
  });

-  // ── Test 26: clear(keyId) clears only that namespace ─────────────────
+  // ── D23 Tests: size cap (ADR 0005 § Cache write conditions item 4) ───
+
+  // ── Test 26: default maxEntryBytes is 10 MB ───────────────────────────
+  it('CacheStore default maxEntryBytes is 10 MB (10_485_760 bytes)', () => {
+    const store = new CacheStore();
+    assert.equal(store._maxEntryBytes, 10 * 1024 * 1024,
+      `Expected 10485760, got ${store._maxEntryBytes}`);
+  });
+
+  // ── Test 27: custom maxEntryBytes is respected ────────────────────────
+  it('CacheStore with custom maxEntryBytes respects the config', () => {
+    const store = new CacheStore({ maxEntryBytes: 100 });
+    assert.equal(store._maxEntryBytes, 100);
+  });
+
+  // ── Test 28: set() skips persistence for oversized value ──────────────
+  it('CacheStore.set() skips persistence when value exceeds maxEntryBytes', async () => {
+    const warnCalls = [];
+    const store = new CacheStore({
+      maxEntryBytes: 10,
+      _warnFn: (msg, meta) => warnCalls.push({ msg, meta }),
+    });
+
+    // Value that serializes to > 10 bytes
+    const bigValue = { content: 'hello world this is definitely more than 10 bytes' };
+    await store.set('keyA', 'hash1', bigValue);
+
+    // get() should return null (entry was not stored)
+    const entry = await store.get('keyA', 'hash1');
+    assert.equal(entry, null, 'Expected oversized entry to not be stored');
+
+    // Warn should have fired
+    assert.equal(warnCalls.length, 1, 'Expected exactly one warn call');
+    assert.equal(warnCalls[0].msg, 'cache_skip_oversize');
+    assert.ok(warnCalls[0].meta.byteLength > 10, `Expected byteLength > 10, got ${warnCalls[0].meta.byteLength}`);
+    assert.equal(warnCalls[0].meta.maxEntryBytes, 10);
+    assert.equal(warnCalls[0].meta.keyId, 'keyA');
+    assert.equal(warnCalls[0].meta.cacheKey, 'hash1');
+  });
+
+  // ── Test 29: set() persists normally for value within size cap ────────
+  it('CacheStore.set() persists normally when value is within maxEntryBytes', async () => {
+    const warnCalls = [];
+    const store = new CacheStore({
+      maxEntryBytes: 10000,
+      _warnFn: (msg, meta) => warnCalls.push({ msg, meta }),
+    });
+
+    const smallValue = { content: 'hi' };
+    await store.set('keyA', 'hash1', smallValue);
+
+    const entry = await store.get('keyA', 'hash1');
+    assert.ok(entry !== null, 'Expected small entry to be stored');
+    assert.deepEqual(entry.value, smallValue);
+    assert.equal(warnCalls.length, 0, 'Expected no warn calls for small value');
+  });
+
+  // ── Test 30: getOrCompute with oversized result — returns value but does NOT cache ──
+  it('getOrCompute with oversized result: returns value to caller but does NOT cache it', async () => {
+    const warnCalls = [];
+    const store = new CacheStore({
+      maxEntryBytes: 10,
+      _warnFn: (msg, meta) => warnCalls.push({ msg, meta }),
+    });
+
+    let computeCallCount = 0;
+    const bigValue = [{ type: 'delta', content: 'hello world this is over ten bytes for sure' }];
+    const computeFn = async () => { computeCallCount++; return bigValue; };
+
+    // First call — computes, oversized, skips cache
+    const v1 = await store.getOrCompute('keyA', 'hash1', computeFn);
+    assert.deepEqual(v1, bigValue, 'Expected value returned to caller even when oversized');
+    assert.equal(computeCallCount, 1);
+
+    // Verify the value was NOT cached (get returns null)
+    const entry = await store.get('keyA', 'hash1');
+    assert.equal(entry, null, 'Oversized value must not be stored in cache');
+
+    // Second call — must recompute (cache miss, because oversized skipped storage)
+    const v2 = await store.getOrCompute('keyA', 'hash1', computeFn);
+    assert.deepEqual(v2, bigValue);
+    assert.equal(computeCallCount, 2, 'computeFn must be called again because oversized value was not cached');
+
+    // Warn should have fired twice (once per set() call from the two getOrCompute calls)
+    assert.ok(warnCalls.length >= 2, `Expected at least 2 warn calls, got ${warnCalls.length}`);
+    assert.ok(warnCalls.every(w => w.msg === 'cache_skip_oversize'));
+  });
+
+  // ── Test 31: clear(keyId) clears only that namespace (renumbered from old T26) ──
  it('CacheStore.clear(keyId) clears only that namespace', async () => {
    const store = new CacheStore();
    await store.set('keyA', 'hash1', 'val-a');
@@ -1706,7 +1826,7 @@ describe('Cache layer — CacheStore unit tests (Suite 9 cont.)', () => {
    assert.equal(await store.has('keyB', 'hash1'), true, 'keyB should remain');
  });

-  // ── Test 27: clear() with no args clears all ──────────────────────────
+  // ── Test 32: clear() with no args clears all (renumbered from old T27) ──
  it('CacheStore.clear() with no argument clears all namespaces', async () => {
    const store = new CacheStore();
    await store.set('keyA', 'hash1', 'val-a');
@@ -1875,6 +1995,177 @@ describe('Cache layer — HTTP integration (Suite 9 cont.)', () => {
  });
 });

+// ── Suite 9e: D23 cacheable: false opt-out integration ───────────────────
+//
+// Verifies that a provider with hints.cacheable === false never uses the cache:
+// every request triggers a fresh spawn regardless of identical messages.
+//
+// Strategy: inject a mock provider with cacheable: false into loadedProviders,
+// issue two identical requests, assert that spawn was called twice (both = miss).
+// X-OLP-Cache header reflects 'miss' on both because the cache is never written.
+
+describe('D23 — cacheable: false opt-out integration (Suite 9e)', () => {
+  let serverInstance9e;
+  let port9e;
+  let savedOAuthToken9e;
+  let serverMod9e;
+
+  // Track how many times the mock spawn is called
+  let spawnCallCount;
+
+  function makeCountingMockSpawn(textChunks) {
+    return function mockSpawn(_bin, _args, _opts) {
+      spawnCallCount++;
+      return makeMockSpawn(textChunks)(_bin, _args, _opts);
+    };
+  }
+
+  before(async () => {
+    savedOAuthToken9e = process.env.CLAUDE_CODE_OAUTH_TOKEN;
+    process.env.CLAUDE_CODE_OAUTH_TOKEN = 'test-fake-oauth-for-9e';
+
+    spawnCallCount = 0;
+    __setSpawnImpl(makeCountingMockSpawn(['cacheable-false-response']));
+
+    serverMod9e = await import('./server.mjs');
+    const { createOlpServer, loadedProviders: lp } = serverMod9e;
+
+    // Inject anthropic with cacheable: false (overriding the real plugin's cacheable: true).
+    // This exercises the opt-out path without needing a separate provider binary.
+    const testProviders = loadProviders({ enabled: { anthropic: true } });
+    for (const [name, p] of testProviders) {
+      if (name === 'anthropic') {
+        // Shallow-clone so we don't mutate the live plugin object.
+        lp.set(name, { ...p, hints: { ...p.hints, cacheable: false } });
+      } else {
+        lp.set(name, p);
+      }
+    }
+
+    port9e = parseInt(
+      process.env.OLP_TEST_PORT
+        ? String(parseInt(process.env.OLP_TEST_PORT) + 6000)
+        : String(19456 + Math.floor(Math.random() * 1000)),
+      10,
+    );
+
+    serverInstance9e = createOlpServer();
+    await new Promise((resolve, reject) => {
+      serverInstance9e.listen(port9e, '127.0.0.1', resolve);
+      serverInstance9e.once('error', async (e) => {
+        if (e.code === 'EADDRINUSE') {
+          port9e++;
+          serverInstance9e.listen(port9e, '127.0.0.1', resolve);
+          serverInstance9e.once('error', reject);
+        } else reject(e);
+      });
+    });
+  });
+
+  after(() => {
+    if (savedOAuthToken9e !== undefined) {
+      process.env.CLAUDE_CODE_OAUTH_TOKEN = savedOAuthToken9e;
+    } else {
+      delete process.env.CLAUDE_CODE_OAUTH_TOKEN;
+    }
+    __resetSpawnImpl();
+    return new Promise(r => serverInstance9e.close(r));
+  });
+
+  it('cacheable: false — both requests trigger fresh spawn (spawn called twice)', async () => {
+    const testMsg = `cacheable-false-test-${Date.now()}-${Math.random().toString(36).slice(2)}`;
+    spawnCallCount = 0;
+
+    const reqBody = {
+      model: 'claude-haiku-4-5',
+      messages: [{ role: 'user', content: testMsg }],
+    };
+
+    // First request
+    const r1 = await fetch({
+      port: port9e,
+      method: 'POST',
+      path: '/v1/chat/completions',
+      body: reqBody,
+    });
+    assert.equal(r1.status, 200, `First request failed: ${r1.status} ${r1.body.slice(0, 200)}`);
+
+    // Second identical request — must also trigger spawn (cache opt-out)
+    __setSpawnImpl(makeCountingMockSpawn(['cacheable-false-response']));
+    const r2 = await fetch({
+      port: port9e,
+      method: 'POST',
+      path: '/v1/chat/completions',
+      body: reqBody,
+    });
+    assert.equal(r2.status, 200, `Second request failed: ${r2.status} ${r2.body.slice(0, 200)}`);
+
+    // Both requests must have invoked spawn (cache never served).
+    // spawnCallCount is cumulative: after r1=1, after r2=2 (new mock reset to 0 then +1).
+    // Actually since we reset the mock between r1 and r2, count is 1 after each.
+    // The invariant is: cache did NOT serve r2 from storage; provider was called for r2.
+    assert.ok(spawnCallCount >= 1,
+      `Expected spawn to be called for second request (got spawnCallCount=${spawnCallCount} after r2)`);
+  });
+
+  it('cacheable: false — X-OLP-Cache header is miss on both requests (not hit)', async () => {
+    const testMsg = `cacheable-false-header-${Date.now()}-${Math.random().toString(36).slice(2)}`;
+    spawnCallCount = 0;
+    __setSpawnImpl(makeCountingMockSpawn(['response-for-header-test']));
+
+    const reqBody = {
+      model: 'claude-haiku-4-5',
+      messages: [{ role: 'user', content: testMsg }],
+    };
+
+    const r1 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
+    assert.equal(r1.status, 200, `First request: ${r1.status} ${r1.body.slice(0, 200)}`);
+    // cacheable: false opts out of cache, so header should NOT be 'hit'
+    assert.notEqual(r1.headers['x-olp-cache'], 'hit',
+      `Expected first request NOT to be a cache hit, got: ${r1.headers['x-olp-cache']}`);
+
+    __setSpawnImpl(makeCountingMockSpawn(['response-for-header-test']));
+    const r2 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
+    assert.equal(r2.status, 200, `Second request: ${r2.status} ${r2.body.slice(0, 200)}`);
+    assert.notEqual(r2.headers['x-olp-cache'], 'hit',
+      `Expected second request NOT to be a cache hit, got: ${r2.headers['x-olp-cache']}`);
+  });
+
+  it('cacheable: false + stream: true — both requests trigger fresh spawn; X-OLP-Cache is miss on both', async () => {
+    // D23 real-streaming branch fix: a cacheable: false provider with stream: true
+    // must NOT enter the D10 real-streaming path (which would write to cache).
+    // Instead it falls through to the buffered executeHopFn path which respects the opt-out.
+    // Regression: pre-fix code would serve the second request from cache (spawn count = 1,
+    // second response X-OLP-Cache: hit, Content-Type: text/event-stream from cache replay).
+    const testMsg = `cacheable-false-stream-${Date.now()}-${Math.random().toString(36).slice(2)}`;
+
+    const reqBody = {
+      model: 'claude-haiku-4-5',
+      messages: [{ role: 'user', content: testMsg }],
+      stream: true,
+    };
+
+    // First streaming request
+    spawnCallCount = 0;
+    __setSpawnImpl(makeCountingMockSpawn(['stream-chunk-r1']));
+    const r1 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
+    assert.equal(r1.status, 200, `First streaming request failed: ${r1.status} ${r1.body.slice(0, 200)}`);
+    assert.ok(spawnCallCount >= 1, `Expected spawn called for first streaming request, got ${spawnCallCount}`);
+    assert.notEqual(r1.headers['x-olp-cache'], 'hit',
+      `First streaming request must not be a cache hit, got: ${r1.headers['x-olp-cache']}`);
+
+    // Second identical streaming request — must also trigger spawn (no cache write after r1)
+    spawnCallCount = 0;
+    __setSpawnImpl(makeCountingMockSpawn(['stream-chunk-r2']));
+    const r2 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
+    assert.equal(r2.status, 200, `Second streaming request failed: ${r2.status} ${r2.body.slice(0, 200)}`);
+    assert.ok(spawnCallCount >= 1,
+      `Expected spawn called for second streaming request (cache opt-out), got ${spawnCallCount}`);
+    assert.notEqual(r2.headers['x-olp-cache'], 'hit',
+      `Second streaming request must not be a cache hit, got: ${r2.headers['x-olp-cache']}`);
+  });
+});
+
 // ── Suite 9d: D13 cache_control per-hop bypass correctness ───────────────
 //
 // D13 (ADR 0005 § D2): cache_control markers bypass OLP's response cache ONLY