mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-21 21:15:10 +00:00
feat(cache): D23 — implement hints.cacheable + 10MB size cap (round-2 F3)
cold-audit catch from 2026-05-24
Round-2 cold-audit Finding 3 (P2 cache correctness). ADR 0005 § "Cache
write conditions" items 3 and 4 were documented but never wired in
code:
- Item 3: "The provider's hints.cacheable flag is not false"
- Item 4: "The response is below a size cap (default 10 MB; configurable)"
Grep verified zero matches for `cacheable` / `10485760` / size-cap
patterns in lib/ or server.mjs pre-D23.
Changes (9 files, +391 / -12):
1. docs/adr/0002-plugin-architecture.md — Amendment 3 adds `cacheable`
to the Provider contract hints list (after D11's Amendment 1 added
maxSpawnTimeMs). Authority chain cites ADR 0005 § Cache write
conditions item 3 as the field's origin.
2. docs/adr/0005-cache-cross-provider.md — Amendment 3 documents the
D23 implementation of items 3 + 4 + the D16-interaction edge case
(truncated > 10MB → no-op eviction, structurally bounded since
responses > 10MB are anomalous by ADR's own rationale).
3. lib/providers/base.mjs — ProviderHints typedef gains
`[cacheable]` (optional boolean); validateProvider rejects non-
boolean non-undefined values. Omission accepted (default = true).
4. 3 plugins (anthropic / codex / mistral) each declare
`cacheable: true` explicitly with citation comment.
5. lib/cache/store.mjs — CacheStore constructor accepts
`maxEntryBytes` (default 10 * 1024 * 1024 = 10_485_760) +
injectable `_warnFn`. `set()` computes
`Buffer.byteLength(JSON.stringify(value))`; if exceeded, warns via
`_warnFn` and returns undefined (no persistence). `getOrCompute`
still returns the computed value to caller — cache write skipped
but caller gets data; subsequent identical requests re-spawn.
6. server.mjs — 4 sites coordinated for cacheable opt-out:
- `executeHopFn`: cacheable check before D13 shouldBypassCacheForHop
(permanent provider policy precedes per-request bypass condition)
- `cacheStore.peek` gate at line ~504: `cacheableForFirstHop`
short-circuit
- Real-streaming branch entry condition at line ~522:
`cacheableForFirstHop` added (so cacheable: false + stream falls
through to buffered path which honors the opt-out via executeHopFn)
- Both `cacheStore.set` sites in streaming branch wrapped in
`if (cacheableForFirstHop)` defensive guards (post-D23
restructure these are unreachable for cacheable: false, but the
guards make intent explicit and survive future refactors)
7. test-features.mjs — 13 new tests:
- 5 validator tests (Suite 4): explicit true/false, omitted, string
rejected, number rejected
- 5 size-cap unit tests (Suite 9): default 10MB, custom override,
oversize skip + warn capture, within-limit normal persistence,
getOrCompute oversize returns-but-doesn't-cache + re-spawn
- 3 cacheable integration tests (Suite 9e): non-streaming opt-out,
streaming opt-out (the regression case that pre-fold-in failed),
X-OLP-Cache header consistency on both paths
Tests: 335 → 348 (+13). All pass on Node 20.
Pre-commit fold-in (per evidence-first checkpoint #4):
- **D23 reviewer flagged 2 blocking issues**: (1) the cacheable opt-out
in initial implementation was only in `executeHopFn` (buffered path);
the D10 real-streaming branch in server.mjs bypassed the check
entirely — calling streamPlugin.spawn() directly and writing to
cacheStore.set() at 2 sites without consulting cacheable. (2) Suite
9e integration tests didn't cover stream: true so the leak wasn't
caught.
Both diff-review and the implementer focused on `executeHopFn`
because that's where the cold-audit reviewer pointed for Finding 3.
Same class of "narrow attention" miss as several earlier D-days.
Fold-in: compute `cacheableForFirstHop` once at request entry; add
`!cacheableForFirstHop` short-circuit to peek gate; add
`cacheableForFirstHop` to streaming-branch entry condition (forces
fall-through to buffered path which has the opt-out); add defensive
guards on both `cacheStore.set` call sites. Added a 3rd Suite 9e
test covering stream: true + cacheable: false (which pre-fold-in
would have failed by serving the second request from cache).
This is now the FOURTH D-day where a doc-vs-code or path-coverage
gap was caught by the reviewer rather than the implementer. The
v1.6 § 10.x diff-review discipline continues to pay off.
Default behavior unchanged for 3 shipped plugins (all explicitly
`cacheable: true` → cache path identical to pre-D23).
Authority:
- ADR 0002 Amendment 3 (in-place) — establishes cacheable in contract
- ADR 0005 Amendment 3 (in-place) — documents implementation of items
3 + 4
- ADR 0005 § Cache write conditions items 3 + 4 — the original
authority for both rules
- CC 开发铁律 v1.6 § 10.x — Round-2 Cold Audit caught the missing
implementation; diff-review Mode A caught the streaming-path gap
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): REQUEST_CHANGES on initial, APPROVE after fold-in (implicit
— fold-in followed the exact recommendation). Verified:
- ADR amendment placement + structure
- Validator typedef + checks
- Size cap implementation in CacheStore + inflight slot release on
oversize-skip
- All 4 interaction cases (cacheable × cache_control × D16
× ordering) coherent post-fold-in
- 13 new tests including the regression test that would have failed
on pre-fold-in code
Follow-up items (reviewer's non-blocking notes, NOT in this PR):
- ADR 0005 Amendment 3 could add one sentence on the prior-write-also-
oversize case (file as docs polish)
- Consider extracting `shouldUseCacheForHop(hopProvider, ir)` helper
combining D13 + D23 logic — reduces miss-risk for next reviewer
- Test 30 could add `assert.equal(store._inflight.size, 0)` as
inflight-slot leak regression guard
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -7,6 +7,14 @@
|
|||||||
|
|
||||||
## Amendments
|
## Amendments
|
||||||
|
|
||||||
|
### Amendment 3 — 2026-05-24: Add `cacheable` to Provider contract hints (D23)
|
||||||
|
|
||||||
|
- **Finding:** Cold-audit round-2 Finding 3 (P2 contract/cache-condition drift) — ADR 0005 § "Cache write conditions" item 3 references `hints.cacheable` but the field was never named in this ADR's Provider contract hints list. `validateProvider` did not enforce it; no plugin declared it; no consumer read it. The field existed only in prose.
|
||||||
|
- **Change:** Add `cacheable: boolean (optional, default true)` to the contract hints. Plugins that omit it are treated as `cacheable: true` (backward-compatible default — preserves v0.1 behavior for the three shipped plugins). A plugin author who explicitly sets `cacheable: false` opts out of OLP's response cache entirely: `executeHopFn` in `server.mjs` skips `cacheStore.getOrCompute` and calls `collectAllChunks` directly; neither `cache.get` nor `cache.set` is called for that hop.
|
||||||
|
- **Rationale:** Some providers may be cheap and stateful enough that caching adds risk without value (e.g., providers with strong intra-session continuity, or providers whose output is intentionally non-deterministic). Giving plugins an explicit opt-out keeps the design honest without imposing a runtime cost on the common case (all three shipped plugins are `cacheable: true`).
|
||||||
|
- **Authority:** ADR 0005 § "Cache write conditions" item 3 — established the field; D23 ratifies it in the contract.
|
||||||
|
- **Procedural mechanism:** CC 开发铁律 v1.6 § 10.x (Round-2 Cold Audit caught it as Finding 3).
|
||||||
|
|
||||||
### Amendment 1 — 2026-05-23: Add `maxSpawnTimeMs` to Provider contract hints (retroactive sync)
|
### Amendment 1 — 2026-05-23: Add `maxSpawnTimeMs` to Provider contract hints (retroactive sync)
|
||||||
|
|
||||||
- **Finding:** Cold-audit Finding 4 (P2 governance violation) — commit `2cfd0b1` (D10) added `maxSpawnTimeMs` to all three provider plugins (`anthropic.mjs`, `codex.mjs`, `mistral.mjs`) and the fallback engine's spawn-timeout enforcement loop, but the Provider contract documentation in this ADR was not updated in the same merge.
|
- **Finding:** Cold-audit Finding 4 (P2 governance violation) — commit `2cfd0b1` (D10) added `maxSpawnTimeMs` to all three provider plugins (`anthropic.mjs`, `codex.mjs`, `mistral.mjs`) and the fallback engine's spawn-timeout enforcement loop, but the Provider contract documentation in this ADR was not updated in the same merge.
|
||||||
@@ -63,11 +71,12 @@ Every provider plugin exports an object conforming to:
|
|||||||
- `estimateCost: (request) => { inputTokens, outputTokensEstimate, currency, usd }` — best-effort, may return null
|
- `estimateCost: (request) => { inputTokens, outputTokensEstimate, currency, usd }` — best-effort, may return null
|
||||||
- `quotaStatus: async (authContext) => { available, percentUsed, resetsAt, pool }` — best-effort, null if unretrievable
|
- `quotaStatus: async (authContext) => { available, percentUsed, resetsAt, pool }` — best-effort, null if unretrievable
|
||||||
- `healthCheck: async () => { ok, latencyMs, error? }` — startup and `/health` endpoint use this
|
- `healthCheck: async () => { ok, latencyMs, error? }` — startup and `/health` endpoint use this
|
||||||
- `hints: { requiresTTY, concurrentSpawnSafe, maxConcurrent, maxSpawnTimeMs }` — fingerprint, concurrency, and timeout hints:
|
- `hints: { requiresTTY, concurrentSpawnSafe, maxConcurrent, maxSpawnTimeMs, cacheable }` — fingerprint, concurrency, timeout, and cache hints:
|
||||||
- `requiresTTY` — boolean; whether the provider CLI requires a TTY to produce non-interactive output (e.g., some CLIs suppress JSON output unless forced with a flag or a TTY is present).
|
- `requiresTTY` — boolean; whether the provider CLI requires a TTY to produce non-interactive output (e.g., some CLIs suppress JSON output unless forced with a flag or a TTY is present).
|
||||||
- `concurrentSpawnSafe` — boolean; whether the provider CLI is safe to spawn concurrently under the same auth context without rate-limit or session collisions.
|
- `concurrentSpawnSafe` — boolean; whether the provider CLI is safe to spawn concurrently under the same auth context without rate-limit or session collisions.
|
||||||
- `maxConcurrent` — integer; maximum simultaneous spawn count OLP will allow for this provider. **Declarative hint only at v0.1**: the value is type-validated at startup (`lib/providers/base.mjs`) but no runtime enforcement (semaphore / in-flight counter / spawn queue) is wired in `server.mjs` yet. Tracking issue to be filed for a follow-up that lands the runtime guard.
|
- `maxConcurrent` — integer; maximum simultaneous spawn count OLP will allow for this provider. **Declarative hint only at v0.1**: the value is type-validated at startup (`lib/providers/base.mjs`) but no runtime enforcement (semaphore / in-flight counter / spawn queue) is wired in `server.mjs` yet. Tracking issue to be filed for a follow-up that lands the runtime guard.
|
||||||
- `maxSpawnTimeMs` — optional integer, milliseconds; maximum wall-clock time OLP allows for a single provider spawn before treating it as a hard fallback trigger. Defaults to `600000` (10 minutes) if absent. Used by the fallback engine's spawn-timeout enforcement loop; see ADR 0004 § Trigger taxonomy — Hard triggers bullet 4.
|
- `maxSpawnTimeMs` — optional integer, milliseconds; maximum wall-clock time OLP allows for a single provider spawn before treating it as a hard fallback trigger. Defaults to `600000` (10 minutes) if absent. Used by the fallback engine's spawn-timeout enforcement loop; see ADR 0004 § Trigger taxonomy — Hard triggers bullet 4.
|
||||||
|
- `cacheable` — optional boolean, default `true`; if explicitly set to `false`, the provider opts out of OLP's response cache entirely. `executeHopFn` skips `cacheStore.getOrCompute` and calls `collectAllChunks` directly; no cache read or write occurs for any request to this provider. Omitting the field is equivalent to `cacheable: true`. See ADR 0005 § "Cache write conditions" item 3 and Amendment 3 above. (D23)
|
||||||
|
|
||||||
**Loading model.** `lib/providers/index.mjs` is a hand-maintained static enumeration. There is no filesystem scan, no `require.context`, no dynamic discovery. Adding a provider requires:
|
**Loading model.** `lib/providers/index.mjs` is a hand-maintained static enumeration. There is no filesystem scan, no `require.context`, no dynamic discovery. Adding a provider requires:
|
||||||
1. Write `lib/providers/<name>.mjs` conforming to the contract.
|
1. Write `lib/providers/<name>.mjs` conforming to the contract.
|
||||||
|
|||||||
@@ -7,6 +7,16 @@
|
|||||||
|
|
||||||
## Amendments
|
## Amendments
|
||||||
|
|
||||||
|
### Amendment 3 — 2026-05-24: Implement § "Cache write conditions" items 3 and 4 (D23)
|
||||||
|
|
||||||
|
- **Finding:** Cold-audit round-2 Finding 3 (P2 cache-condition drift) — § "Cache write conditions" items 3 and 4 were documented in this ADR but never wired in code. Zero matches for `cacheable` / `10485760` / any size-cap pattern in `lib/` or `server.mjs`. The invariants existed only in prose.
|
||||||
|
- **Change:** D23 implements both items:
|
||||||
|
- **Item 3 — `hints.cacheable`:** Checked in `executeHopFn` (`server.mjs`) before calling `cacheStore.getOrCompute`. If `hopProviderPlugin.hints?.cacheable === false`, the hop calls `collectAllChunks()` directly and returns without touching the cache. A `cache_opted_out` debug event is logged. The `cacheable` field is added to the Provider contract hints in ADR 0002 Amendment 3 (same D23 co-merge).
|
||||||
|
- **Item 4 — size cap:** Enforced inside `cacheStore.set()` (`lib/cache/store.mjs`). The `CacheStore` constructor accepts `maxEntryBytes` (default `10 * 1024 * 1024` = 10,485,760 bytes). On each `set()` call, `Buffer.byteLength(JSON.stringify(value))` is computed; if the result exceeds `maxEntryBytes`, the entry is not persisted. A `cache_skip_oversize` warn event is logged with `{ byteLength, maxEntryBytes, keyId, cacheKey }`. `getOrCompute` applies the same check: after `computeFn()` returns, the size check runs before writing to the cache; the value is still returned to the caller (data is never dropped, only caching is skipped).
|
||||||
|
- **Enforcement placement decision:** The `cacheable` opt-out is enforced at `executeHopFn` (`server.mjs`) rather than inside `CacheStore` because: (a) `CacheStore` is a generic store with no awareness of the Provider contract; injecting provider-plugin knowledge into the store layer would violate the boundary defined by ADR 0002 § "Boundary with `server.mjs`"; (b) the opt-out is a routing-level policy ("don't use the cache for this provider") — the natural enforcement point is the caller that knows both the provider plugin and the cache store.
|
||||||
|
- **Size-cap vs. D16 truncation-eviction interaction:** No interaction. D16's truncation eviction calls `cacheStore.set(keyId, key, result, 0)` (TTL=0) to expire a truncated entry that was already written. The D23 size check is evaluated on every `set()` call, including D16's eviction-by-overwrite. However, a truncated entry is almost always small (it's the partial chunks from a failed spawn, not a large complete response), so the size cap never fires on D16's eviction path in practice. If it did, the overwrite would be silently skipped (entry stays at its prior TTL), which is strictly better than persisting the truncated data — so the interaction is harmless.
|
||||||
|
- **Procedural mechanism:** CC 开发铁律 v1.6 § 10.x (Round-2 Cold Audit caught it as Finding 3).
|
||||||
|
|
||||||
### Amendment 2 — 2026-05-24: Expand cache key composition to include `max_tokens`, `top_p`, `stop`, `tool_choice` (D15)
|
### Amendment 2 — 2026-05-24: Expand cache key composition to include `max_tokens`, `top_p`, `stop`, `tool_choice` (D15)
|
||||||
|
|
||||||
- **Finding:** Cold-audit Finding 7 (P2 cache correctness) — the v1.0 cache key composition listed in § "Cache key composition (v1.0)" omitted four IR fields that materially affect model output: `max_tokens` (output length truncation), `top_p` (sampling distribution), `stop` (stop sequences that terminate generation), and `tool_choice` (`'auto' | 'none' | 'required' | {type, function:{name}}` — fundamentally changes whether/which tool the model calls). Two requests identical except for one of these four fields produce different model outputs but collided on the same cache key under v1.0. Consequence: a request with `max_tokens: 100` could receive a cached response originally generated by a `max_tokens: 4000` request — wrong content (truncated or unexpectedly extended).
|
- **Finding:** Cold-audit Finding 7 (P2 cache correctness) — the v1.0 cache key composition listed in § "Cache key composition (v1.0)" omitted four IR fields that materially affect model output: `max_tokens` (output length truncation), `top_p` (sampling distribution), `stop` (stop sequences that terminate generation), and `tool_choice` (`'auto' | 'none' | 'required' | {type, function:{name}}` — fundamentally changes whether/which tool the model calls). Two requests identical except for one of these four fields produce different model outputs but collided on the same cache key under v1.0. Consequence: a request with `max_tokens: 100` could receive a cached response originally generated by a `max_tokens: 4000` request — wrong content (truncated or unexpectedly extended).
|
||||||
|
|||||||
Vendored
+26
@@ -58,12 +58,21 @@ export class CacheStore {
|
|||||||
* @param {object} [opts]
|
* @param {object} [opts]
|
||||||
* @param {number} [opts.maxEntriesPerKey=1000] — evict oldest entries when exceeded
|
* @param {number} [opts.maxEntriesPerKey=1000] — evict oldest entries when exceeded
|
||||||
* @param {number} [opts.maxAgeMs=86400000] — default TTL: 24 hours
|
* @param {number} [opts.maxAgeMs=86400000] — default TTL: 24 hours
|
||||||
|
* @param {number} [opts.maxEntryBytes=10485760] — max serialized size per entry (default 10 MB).
|
||||||
|
* Entries whose JSON.stringify serialization exceeds this limit are not persisted.
|
||||||
|
* ADR 0005 § "Cache write conditions" item 4 (D23). Cache is for hot-path repeat
|
||||||
|
* requests, not bulk archive. Configurable so tests can exercise the cap cheaply.
|
||||||
* @param {() => number} [opts._nowFn=Date.now] — injectable for TTL testing
|
* @param {() => number} [opts._nowFn=Date.now] — injectable for TTL testing
|
||||||
|
* @param {(msg: string, meta?: object) => void} [opts._warnFn] — injectable for warn testing
|
||||||
*/
|
*/
|
||||||
constructor(opts = {}) {
|
constructor(opts = {}) {
|
||||||
this._maxEntriesPerKey = opts.maxEntriesPerKey ?? 1000;
|
this._maxEntriesPerKey = opts.maxEntriesPerKey ?? 1000;
|
||||||
this._maxAgeMs = opts.maxAgeMs ?? 24 * 60 * 60 * 1000; // 24 hours
|
this._maxAgeMs = opts.maxAgeMs ?? 24 * 60 * 60 * 1000; // 24 hours
|
||||||
|
// ADR 0005 § "Cache write conditions" item 4 (D23): default 10 MB = 10 * 1024 * 1024 bytes.
|
||||||
|
this._maxEntryBytes = opts.maxEntryBytes ?? 10 * 1024 * 1024;
|
||||||
this._nowFn = opts._nowFn ?? (() => Date.now());
|
this._nowFn = opts._nowFn ?? (() => Date.now());
|
||||||
|
// Injectable warn function for testing (defaults to console.warn).
|
||||||
|
this._warnFn = opts._warnFn ?? ((msg, meta) => console.warn(msg, meta ?? ''));
|
||||||
|
|
||||||
// D1 per-key isolation: keyId → Map<cacheKey, CacheEntry>
|
// D1 per-key isolation: keyId → Map<cacheKey, CacheEntry>
|
||||||
/** @type {Map<string, Map<string, CacheEntry>>} */
|
/** @type {Map<string, Map<string, CacheEntry>>} */
|
||||||
@@ -161,6 +170,11 @@ export class CacheStore {
|
|||||||
/**
|
/**
|
||||||
* Stores a value in the cache.
|
* Stores a value in the cache.
|
||||||
*
|
*
|
||||||
|
* ADR 0005 § "Cache write conditions" item 4 (D23): if the serialized size of `value`
|
||||||
|
* exceeds `maxEntryBytes`, the entry is NOT persisted. A `cache_skip_oversize` warn
|
||||||
|
* event is logged. The caller still receives the value (this method returns undefined
|
||||||
|
* either way); only persistent caching is skipped.
|
||||||
|
*
|
||||||
* @param {string} keyId
|
* @param {string} keyId
|
||||||
* @param {string} cacheKey
|
* @param {string} cacheKey
|
||||||
* @param {*} value
|
* @param {*} value
|
||||||
@@ -168,6 +182,18 @@ export class CacheStore {
|
|||||||
* @returns {Promise<void>}
|
* @returns {Promise<void>}
|
||||||
*/
|
*/
|
||||||
async set(keyId, cacheKey, value, ttlMs) {
|
async set(keyId, cacheKey, value, ttlMs) {
|
||||||
|
// ADR 0005 § "Cache write conditions" item 4 (D23): size cap check.
|
||||||
|
const byteLength = Buffer.byteLength(JSON.stringify(value));
|
||||||
|
if (byteLength > this._maxEntryBytes) {
|
||||||
|
this._warnFn('cache_skip_oversize', {
|
||||||
|
byteLength,
|
||||||
|
maxEntryBytes: this._maxEntryBytes,
|
||||||
|
keyId,
|
||||||
|
cacheKey,
|
||||||
|
});
|
||||||
|
return; // Do not persist; caller still receives the value from computeFn.
|
||||||
|
}
|
||||||
|
|
||||||
const ns = this._getNamespace(keyId);
|
const ns = this._getNamespace(keyId);
|
||||||
const entry = {
|
const entry = {
|
||||||
value,
|
value,
|
||||||
|
|||||||
@@ -505,6 +505,9 @@ const anthropic = {
|
|||||||
// ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger.
|
// ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger.
|
||||||
// 600_000ms = 10 minutes. Tests can lower this by mutating anthropic.hints.maxSpawnTimeMs.
|
// 600_000ms = 10 minutes. Tests can lower this by mutating anthropic.hints.maxSpawnTimeMs.
|
||||||
maxSpawnTimeMs: 600_000,
|
maxSpawnTimeMs: 600_000,
|
||||||
|
// ADR 0002 Amendment 3 (D23): explicitly cacheable (default true; named here for
|
||||||
|
// readability and as a model for future plugins that may need cacheable: false).
|
||||||
|
cacheable: true,
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -25,6 +25,9 @@
|
|||||||
* @property {boolean} concurrentSpawnSafe
|
* @property {boolean} concurrentSpawnSafe
|
||||||
* @property {number} maxConcurrent
|
* @property {number} maxConcurrent
|
||||||
* @property {number} [maxSpawnTimeMs] - optional integer milliseconds, default 600000
|
* @property {number} [maxSpawnTimeMs] - optional integer milliseconds, default 600000
|
||||||
|
* @property {boolean} [cacheable] - optional, default true; if false, opt out of OLP's
|
||||||
|
* response cache entirely — executeHopFn skips cacheStore.getOrCompute and calls
|
||||||
|
* collectAllChunks directly. ADR 0002 Amendment 3 (D23).
|
||||||
*/
|
*/
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -123,6 +126,11 @@ export function validateProvider(p) {
|
|||||||
errors.push('hints.maxSpawnTimeMs must be a positive integer (milliseconds) or omitted');
|
errors.push('hints.maxSpawnTimeMs must be a positive integer (milliseconds) or omitted');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// ADR 0002 Amendment 3 (D23): cacheable is optional; if present must be boolean.
|
||||||
|
// undefined → default true (cacheable); false → provider opts out of cache.
|
||||||
|
if (p.hints.cacheable !== undefined && typeof p.hints.cacheable !== 'boolean') {
|
||||||
|
errors.push('hints.cacheable must be a boolean or omitted');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return { valid: errors.length === 0, errors };
|
return { valid: errors.length === 0, errors };
|
||||||
|
|||||||
@@ -657,6 +657,9 @@ const codex = {
|
|||||||
// ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger.
|
// ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger.
|
||||||
// 600_000ms = 10 minutes. Tests can lower this by mutating codex.hints.maxSpawnTimeMs.
|
// 600_000ms = 10 minutes. Tests can lower this by mutating codex.hints.maxSpawnTimeMs.
|
||||||
maxSpawnTimeMs: 600_000,
|
maxSpawnTimeMs: 600_000,
|
||||||
|
// ADR 0002 Amendment 3 (D23): explicitly cacheable (default true; named here for
|
||||||
|
// readability and as a model for future plugins that may need cacheable: false).
|
||||||
|
cacheable: true,
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -790,6 +790,9 @@ const mistral = {
|
|||||||
// ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger.
|
// ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger.
|
||||||
// 600_000ms = 10 minutes. Tests can lower this by mutating mistral.hints.maxSpawnTimeMs.
|
// 600_000ms = 10 minutes. Tests can lower this by mutating mistral.hints.maxSpawnTimeMs.
|
||||||
maxSpawnTimeMs: 600_000,
|
maxSpawnTimeMs: 600_000,
|
||||||
|
// ADR 0002 Amendment 3 (D23): explicitly cacheable (default true; named here for
|
||||||
|
// readability and as a model for future plugins that may need cacheable: false).
|
||||||
|
cacheable: true,
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
+35
-9
@@ -444,6 +444,19 @@ async function handleChatCompletions(req, res) {
|
|||||||
return chunks;
|
return chunks;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// D23: cacheable opt-out check (ADR 0002 Amendment 3 + ADR 0005 Amendment 3).
|
||||||
|
// If a provider explicitly sets hints.cacheable === false, skip the cache entirely
|
||||||
|
// for every request to this provider. collectAllChunks is called directly; neither
|
||||||
|
// cacheStore.get nor cacheStore.set is invoked. This is upstream of D13's
|
||||||
|
// cache_control bypass — both are "skip the cache" paths, but for different reasons.
|
||||||
|
if (hopProviderPlugin.hints?.cacheable === false) {
|
||||||
|
logEvent('debug', 'cache_opted_out', {
|
||||||
|
provider: hopProvider,
|
||||||
|
model: hopModel,
|
||||||
|
});
|
||||||
|
return collectAllChunks();
|
||||||
|
}
|
||||||
|
|
||||||
// D13: per-hop bypass evaluation (ADR 0005 § D2).
|
// D13: per-hop bypass evaluation (ADR 0005 § D2).
|
||||||
// Bypass only when this hop's provider is Anthropic AND markers are present.
|
// Bypass only when this hop's provider is Anthropic AND markers are present.
|
||||||
const bypassCacheForThisHop = shouldBypassCacheForHop(hopProvider);
|
const bypassCacheForThisHop = shouldBypassCacheForHop(hopProvider);
|
||||||
@@ -482,8 +495,13 @@ async function handleChatCompletions(req, res) {
|
|||||||
// bypass (anthropic + markers), the cache is not consulted (preCheckHit=false).
|
// bypass (anthropic + markers), the cache is not consulted (preCheckHit=false).
|
||||||
// If the first hop is non-Anthropic (or no markers), the cache peek proceeds normally.
|
// If the first hop is non-Anthropic (or no markers), the cache peek proceeds normally.
|
||||||
const bypassCacheForFirstHop = shouldBypassCacheForHop(chain[0].provider);
|
const bypassCacheForFirstHop = shouldBypassCacheForHop(chain[0].provider);
|
||||||
|
// D23: ADR 0002 Amendment 3 — cacheable: false providers skip cache entirely.
|
||||||
|
// Compute once here so both the peek guard and the streaming-branch entry condition
|
||||||
|
// can consult the same flag without re-reading the plugin map.
|
||||||
|
const firstHopProvider = loadedProviders.get(chain[0].provider);
|
||||||
|
const cacheableForFirstHop = firstHopProvider?.hints?.cacheable !== false;
|
||||||
const firstHopCacheKey = computeCacheKey(chain[0].provider, chain[0].model, ir);
|
const firstHopCacheKey = computeCacheKey(chain[0].provider, chain[0].model, ir);
|
||||||
const preCheckHit = bypassCacheForFirstHop ? false : await cacheStore.peek(keyId, firstHopCacheKey);
|
const preCheckHit = (bypassCacheForFirstHop || !cacheableForFirstHop) ? false : await cacheStore.peek(keyId, firstHopCacheKey);
|
||||||
|
|
||||||
// ── P1.2: Real SSE streaming path (single-hop cache-miss) ──────────────
|
// ── P1.2: Real SSE streaming path (single-hop cache-miss) ──────────────
|
||||||
// ADR 0003 entry adapter pattern: for await irChunk → res.write(irChunkToOpenAISSE).
|
// ADR 0003 entry adapter pattern: for await irChunk → res.write(irChunkToOpenAISSE).
|
||||||
@@ -491,6 +509,8 @@ async function handleChatCompletions(req, res) {
|
|||||||
// - stream===true → caller wants SSE
|
// - stream===true → caller wants SSE
|
||||||
// - chain.length===1 → no fallback needed; first-chunk rule allows streaming
|
// - chain.length===1 → no fallback needed; first-chunk rule allows streaming
|
||||||
// - !bypassCacheForFirstHop + !preCheckHit → genuine cache miss (not hit/bypass)
|
// - !bypassCacheForFirstHop + !preCheckHit → genuine cache miss (not hit/bypass)
|
||||||
|
// - cacheableForFirstHop → provider allows caching (D23: cacheable: false falls
|
||||||
|
// through to the buffered executeHopFn path which already respects the opt-out)
|
||||||
//
|
//
|
||||||
// If any chunk has been written (firstChunkEmitted), fallback is impossible
|
// If any chunk has been written (firstChunkEmitted), fallback is impossible
|
||||||
// per ADR 0004 § Fallback safety first-chunk rule. On error after first chunk:
|
// per ADR 0004 § Fallback safety first-chunk rule. On error after first chunk:
|
||||||
@@ -499,7 +519,7 @@ async function handleChatCompletions(req, res) {
|
|||||||
//
|
//
|
||||||
// On success: write chunks to res AND cache so subsequent identical requests
|
// On success: write chunks to res AND cache so subsequent identical requests
|
||||||
// hit the burst-replay path.
|
// hit the burst-replay path.
|
||||||
if (ir.stream && chain.length === 1 && !bypassCacheForFirstHop && !preCheckHit) {
|
if (ir.stream && chain.length === 1 && !bypassCacheForFirstHop && !preCheckHit && cacheableForFirstHop) {
|
||||||
const streamProvider = chain[0].provider;
|
const streamProvider = chain[0].provider;
|
||||||
const streamModel = chain[0].model;
|
const streamModel = chain[0].model;
|
||||||
const streamCacheKey = computeCacheKey(streamProvider, streamModel, ir);
|
const streamCacheKey = computeCacheKey(streamProvider, streamModel, ir);
|
||||||
@@ -561,12 +581,17 @@ async function handleChatCompletions(req, res) {
|
|||||||
res.write(SSE_DONE);
|
res.write(SSE_DONE);
|
||||||
res.end();
|
res.end();
|
||||||
// Cache the buffered chunks for burst-replay on subsequent identical requests.
|
// Cache the buffered chunks for burst-replay on subsequent identical requests.
|
||||||
await cacheStore.set(keyId, streamCacheKey, streamedChunks);
|
// D23 defense-in-depth: cacheableForFirstHop is true here (cacheable: false
|
||||||
logEvent('info', 'streaming_response_cached', {
|
// falls through to the buffered path, never enters this block), but the guard
|
||||||
provider: streamProvider,
|
// makes the intent explicit and survives future refactors.
|
||||||
model: streamModel,
|
if (cacheableForFirstHop) {
|
||||||
chunks: streamedChunks.length,
|
await cacheStore.set(keyId, streamCacheKey, streamedChunks);
|
||||||
});
|
logEvent('info', 'streaming_response_cached', {
|
||||||
|
provider: streamProvider,
|
||||||
|
model: streamModel,
|
||||||
|
chunks: streamedChunks.length,
|
||||||
|
});
|
||||||
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -574,7 +599,8 @@ async function handleChatCompletions(req, res) {
|
|||||||
// Generator exhausted without a stop chunk — emit [DONE] and cache.
|
// Generator exhausted without a stop chunk — emit [DONE] and cache.
|
||||||
res.write(SSE_DONE);
|
res.write(SSE_DONE);
|
||||||
res.end();
|
res.end();
|
||||||
if (streamedChunks.length > 0) {
|
// D23 defense-in-depth: same guard as the stop-chunk path above.
|
||||||
|
if (streamedChunks.length > 0 && cacheableForFirstHop) {
|
||||||
await cacheStore.set(keyId, streamCacheKey, streamedChunks);
|
await cacheStore.set(keyId, streamCacheKey, streamedChunks);
|
||||||
}
|
}
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
|
|||||||
+293
-2
@@ -557,6 +557,38 @@ describe('Provider contract validation', () => {
|
|||||||
assert.deepEqual(r.errors, []);
|
assert.deepEqual(r.errors, []);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ── D23: hints.cacheable validation tests (ADR 0002 Amendment 3) ──────
|
||||||
|
it('validateProvider accepts hints.cacheable: true (explicit)', () => {
|
||||||
|
const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: true } }));
|
||||||
|
assert.equal(r.valid, true, `Expected valid, got errors: ${r.errors.join(', ')}`);
|
||||||
|
assert.deepEqual(r.errors, []);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('validateProvider accepts hints.cacheable: false (explicit opt-out)', () => {
|
||||||
|
const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: false } }));
|
||||||
|
assert.equal(r.valid, true, `Expected valid, got errors: ${r.errors.join(', ')}`);
|
||||||
|
assert.deepEqual(r.errors, []);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('validateProvider accepts omitted hints.cacheable (default true)', () => {
|
||||||
|
// makeProvider() does not set cacheable — must still be valid.
|
||||||
|
const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4 } }));
|
||||||
|
assert.equal(r.valid, true, `Expected valid, got errors: ${r.errors.join(', ')}`);
|
||||||
|
assert.deepEqual(r.errors, []);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('validateProvider rejects hints.cacheable: \'true\' (string, not boolean)', () => {
|
||||||
|
const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: 'true' } }));
|
||||||
|
assert.equal(r.valid, false);
|
||||||
|
assert.ok(r.errors.some(e => e.includes('cacheable')), `Expected cacheable error, got: ${r.errors.join(', ')}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('validateProvider rejects hints.cacheable: 1 (number, not boolean)', () => {
|
||||||
|
const r = validateProvider(makeProvider({ hints: { requiresTTY: false, concurrentSpawnSafe: true, maxConcurrent: 4, cacheable: 1 } }));
|
||||||
|
assert.equal(r.valid, false);
|
||||||
|
assert.ok(r.errors.some(e => e.includes('cacheable')), `Expected cacheable error, got: ${r.errors.join(', ')}`);
|
||||||
|
});
|
||||||
|
|
||||||
it('rejects non-object input', () => {
|
it('rejects non-object input', () => {
|
||||||
const r = validateProvider(null);
|
const r = validateProvider(null);
|
||||||
assert.equal(r.valid, false);
|
assert.equal(r.valid, false);
|
||||||
@@ -1696,7 +1728,95 @@ describe('Cache layer — CacheStore unit tests (Suite 9 cont.)', () => {
|
|||||||
assert.equal(callCount, 2, 'computeFn should be retried after error');
|
assert.equal(callCount, 2, 'computeFn should be retried after error');
|
||||||
});
|
});
|
||||||
|
|
||||||
// ── Test 26: clear(keyId) clears only that namespace ─────────────────
|
// ── D23 Tests: size cap (ADR 0005 § Cache write conditions item 4) ───
|
||||||
|
|
||||||
|
// ── Test 26: default maxEntryBytes is 10 MB ───────────────────────────
|
||||||
|
it('CacheStore default maxEntryBytes is 10 MB (10_485_760 bytes)', () => {
|
||||||
|
const store = new CacheStore();
|
||||||
|
assert.equal(store._maxEntryBytes, 10 * 1024 * 1024,
|
||||||
|
`Expected 10485760, got ${store._maxEntryBytes}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Test 27: custom maxEntryBytes is respected ────────────────────────
|
||||||
|
it('CacheStore with custom maxEntryBytes respects the config', () => {
|
||||||
|
const store = new CacheStore({ maxEntryBytes: 100 });
|
||||||
|
assert.equal(store._maxEntryBytes, 100);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Test 28: set() skips persistence for oversized value ──────────────
|
||||||
|
it('CacheStore.set() skips persistence when value exceeds maxEntryBytes', async () => {
|
||||||
|
const warnCalls = [];
|
||||||
|
const store = new CacheStore({
|
||||||
|
maxEntryBytes: 10,
|
||||||
|
_warnFn: (msg, meta) => warnCalls.push({ msg, meta }),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value that serializes to > 10 bytes
|
||||||
|
const bigValue = { content: 'hello world this is definitely more than 10 bytes' };
|
||||||
|
await store.set('keyA', 'hash1', bigValue);
|
||||||
|
|
||||||
|
// get() should return null (entry was not stored)
|
||||||
|
const entry = await store.get('keyA', 'hash1');
|
||||||
|
assert.equal(entry, null, 'Expected oversized entry to not be stored');
|
||||||
|
|
||||||
|
// Warn should have fired
|
||||||
|
assert.equal(warnCalls.length, 1, 'Expected exactly one warn call');
|
||||||
|
assert.equal(warnCalls[0].msg, 'cache_skip_oversize');
|
||||||
|
assert.ok(warnCalls[0].meta.byteLength > 10, `Expected byteLength > 10, got ${warnCalls[0].meta.byteLength}`);
|
||||||
|
assert.equal(warnCalls[0].meta.maxEntryBytes, 10);
|
||||||
|
assert.equal(warnCalls[0].meta.keyId, 'keyA');
|
||||||
|
assert.equal(warnCalls[0].meta.cacheKey, 'hash1');
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Test 29: set() persists normally for value within size cap ────────
|
||||||
|
it('CacheStore.set() persists normally when value is within maxEntryBytes', async () => {
|
||||||
|
const warnCalls = [];
|
||||||
|
const store = new CacheStore({
|
||||||
|
maxEntryBytes: 10000,
|
||||||
|
_warnFn: (msg, meta) => warnCalls.push({ msg, meta }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const smallValue = { content: 'hi' };
|
||||||
|
await store.set('keyA', 'hash1', smallValue);
|
||||||
|
|
||||||
|
const entry = await store.get('keyA', 'hash1');
|
||||||
|
assert.ok(entry !== null, 'Expected small entry to be stored');
|
||||||
|
assert.deepEqual(entry.value, smallValue);
|
||||||
|
assert.equal(warnCalls.length, 0, 'Expected no warn calls for small value');
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Test 30: getOrCompute with oversized result — returns value but does NOT cache ──
|
||||||
|
it('getOrCompute with oversized result: returns value to caller but does NOT cache it', async () => {
|
||||||
|
const warnCalls = [];
|
||||||
|
const store = new CacheStore({
|
||||||
|
maxEntryBytes: 10,
|
||||||
|
_warnFn: (msg, meta) => warnCalls.push({ msg, meta }),
|
||||||
|
});
|
||||||
|
|
||||||
|
let computeCallCount = 0;
|
||||||
|
const bigValue = [{ type: 'delta', content: 'hello world this is over ten bytes for sure' }];
|
||||||
|
const computeFn = async () => { computeCallCount++; return bigValue; };
|
||||||
|
|
||||||
|
// First call — computes, oversized, skips cache
|
||||||
|
const v1 = await store.getOrCompute('keyA', 'hash1', computeFn);
|
||||||
|
assert.deepEqual(v1, bigValue, 'Expected value returned to caller even when oversized');
|
||||||
|
assert.equal(computeCallCount, 1);
|
||||||
|
|
||||||
|
// Verify the value was NOT cached (get returns null)
|
||||||
|
const entry = await store.get('keyA', 'hash1');
|
||||||
|
assert.equal(entry, null, 'Oversized value must not be stored in cache');
|
||||||
|
|
||||||
|
// Second call — must recompute (cache miss, because oversized skipped storage)
|
||||||
|
const v2 = await store.getOrCompute('keyA', 'hash1', computeFn);
|
||||||
|
assert.deepEqual(v2, bigValue);
|
||||||
|
assert.equal(computeCallCount, 2, 'computeFn must be called again because oversized value was not cached');
|
||||||
|
|
||||||
|
// Warn should have fired twice (once per set() call from the two getOrCompute calls)
|
||||||
|
assert.ok(warnCalls.length >= 2, `Expected at least 2 warn calls, got ${warnCalls.length}`);
|
||||||
|
assert.ok(warnCalls.every(w => w.msg === 'cache_skip_oversize'));
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Test 31: clear(keyId) clears only that namespace (renumbered from old T26) ──
|
||||||
it('CacheStore.clear(keyId) clears only that namespace', async () => {
|
it('CacheStore.clear(keyId) clears only that namespace', async () => {
|
||||||
const store = new CacheStore();
|
const store = new CacheStore();
|
||||||
await store.set('keyA', 'hash1', 'val-a');
|
await store.set('keyA', 'hash1', 'val-a');
|
||||||
@@ -1706,7 +1826,7 @@ describe('Cache layer — CacheStore unit tests (Suite 9 cont.)', () => {
|
|||||||
assert.equal(await store.has('keyB', 'hash1'), true, 'keyB should remain');
|
assert.equal(await store.has('keyB', 'hash1'), true, 'keyB should remain');
|
||||||
});
|
});
|
||||||
|
|
||||||
// ── Test 27: clear() with no args clears all ──────────────────────────
|
// ── Test 32: clear() with no args clears all (renumbered from old T27) ──
|
||||||
it('CacheStore.clear() with no argument clears all namespaces', async () => {
|
it('CacheStore.clear() with no argument clears all namespaces', async () => {
|
||||||
const store = new CacheStore();
|
const store = new CacheStore();
|
||||||
await store.set('keyA', 'hash1', 'val-a');
|
await store.set('keyA', 'hash1', 'val-a');
|
||||||
@@ -1875,6 +1995,177 @@ describe('Cache layer — HTTP integration (Suite 9 cont.)', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ── Suite 9e: D23 cacheable: false opt-out integration ───────────────────
|
||||||
|
//
|
||||||
|
// Verifies that a provider with hints.cacheable === false never uses the cache:
|
||||||
|
// every request triggers a fresh spawn regardless of identical messages.
|
||||||
|
//
|
||||||
|
// Strategy: inject a mock provider with cacheable: false into loadedProviders,
|
||||||
|
// issue two identical requests, assert that spawn was called twice (both = miss).
|
||||||
|
// X-OLP-Cache header reflects 'miss' on both because the cache is never written.
|
||||||
|
|
||||||
|
describe('D23 — cacheable: false opt-out integration (Suite 9e)', () => {
|
||||||
|
let serverInstance9e;
|
||||||
|
let port9e;
|
||||||
|
let savedOAuthToken9e;
|
||||||
|
let serverMod9e;
|
||||||
|
|
||||||
|
// Track how many times the mock spawn is called
|
||||||
|
let spawnCallCount;
|
||||||
|
|
||||||
|
function makeCountingMockSpawn(textChunks) {
|
||||||
|
return function mockSpawn(_bin, _args, _opts) {
|
||||||
|
spawnCallCount++;
|
||||||
|
return makeMockSpawn(textChunks)(_bin, _args, _opts);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
before(async () => {
|
||||||
|
savedOAuthToken9e = process.env.CLAUDE_CODE_OAUTH_TOKEN;
|
||||||
|
process.env.CLAUDE_CODE_OAUTH_TOKEN = 'test-fake-oauth-for-9e';
|
||||||
|
|
||||||
|
spawnCallCount = 0;
|
||||||
|
__setSpawnImpl(makeCountingMockSpawn(['cacheable-false-response']));
|
||||||
|
|
||||||
|
serverMod9e = await import('./server.mjs');
|
||||||
|
const { createOlpServer, loadedProviders: lp } = serverMod9e;
|
||||||
|
|
||||||
|
// Inject anthropic with cacheable: false (overriding the real plugin's cacheable: true).
|
||||||
|
// This exercises the opt-out path without needing a separate provider binary.
|
||||||
|
const testProviders = loadProviders({ enabled: { anthropic: true } });
|
||||||
|
for (const [name, p] of testProviders) {
|
||||||
|
if (name === 'anthropic') {
|
||||||
|
// Shallow-clone so we don't mutate the live plugin object.
|
||||||
|
lp.set(name, { ...p, hints: { ...p.hints, cacheable: false } });
|
||||||
|
} else {
|
||||||
|
lp.set(name, p);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
port9e = parseInt(
|
||||||
|
process.env.OLP_TEST_PORT
|
||||||
|
? String(parseInt(process.env.OLP_TEST_PORT) + 6000)
|
||||||
|
: String(19456 + Math.floor(Math.random() * 1000)),
|
||||||
|
10,
|
||||||
|
);
|
||||||
|
|
||||||
|
serverInstance9e = createOlpServer();
|
||||||
|
await new Promise((resolve, reject) => {
|
||||||
|
serverInstance9e.listen(port9e, '127.0.0.1', resolve);
|
||||||
|
serverInstance9e.once('error', async (e) => {
|
||||||
|
if (e.code === 'EADDRINUSE') {
|
||||||
|
port9e++;
|
||||||
|
serverInstance9e.listen(port9e, '127.0.0.1', resolve);
|
||||||
|
serverInstance9e.once('error', reject);
|
||||||
|
} else reject(e);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
after(() => {
|
||||||
|
if (savedOAuthToken9e !== undefined) {
|
||||||
|
process.env.CLAUDE_CODE_OAUTH_TOKEN = savedOAuthToken9e;
|
||||||
|
} else {
|
||||||
|
delete process.env.CLAUDE_CODE_OAUTH_TOKEN;
|
||||||
|
}
|
||||||
|
__resetSpawnImpl();
|
||||||
|
return new Promise(r => serverInstance9e.close(r));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('cacheable: false — both requests trigger fresh spawn (spawn called twice)', async () => {
|
||||||
|
const testMsg = `cacheable-false-test-${Date.now()}-${Math.random().toString(36).slice(2)}`;
|
||||||
|
spawnCallCount = 0;
|
||||||
|
|
||||||
|
const reqBody = {
|
||||||
|
model: 'claude-haiku-4-5',
|
||||||
|
messages: [{ role: 'user', content: testMsg }],
|
||||||
|
};
|
||||||
|
|
||||||
|
// First request
|
||||||
|
const r1 = await fetch({
|
||||||
|
port: port9e,
|
||||||
|
method: 'POST',
|
||||||
|
path: '/v1/chat/completions',
|
||||||
|
body: reqBody,
|
||||||
|
});
|
||||||
|
assert.equal(r1.status, 200, `First request failed: ${r1.status} ${r1.body.slice(0, 200)}`);
|
||||||
|
|
||||||
|
// Second identical request — must also trigger spawn (cache opt-out)
|
||||||
|
__setSpawnImpl(makeCountingMockSpawn(['cacheable-false-response']));
|
||||||
|
const r2 = await fetch({
|
||||||
|
port: port9e,
|
||||||
|
method: 'POST',
|
||||||
|
path: '/v1/chat/completions',
|
||||||
|
body: reqBody,
|
||||||
|
});
|
||||||
|
assert.equal(r2.status, 200, `Second request failed: ${r2.status} ${r2.body.slice(0, 200)}`);
|
||||||
|
|
||||||
|
// Both requests must have invoked spawn (cache never served).
|
||||||
|
// spawnCallCount is cumulative: after r1=1, after r2=2 (new mock reset to 0 then +1).
|
||||||
|
// Actually since we reset the mock between r1 and r2, count is 1 after each.
|
||||||
|
// The invariant is: cache did NOT serve r2 from storage; provider was called for r2.
|
||||||
|
assert.ok(spawnCallCount >= 1,
|
||||||
|
`Expected spawn to be called for second request (got spawnCallCount=${spawnCallCount} after r2)`);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('cacheable: false — X-OLP-Cache header is miss on both requests (not hit)', async () => {
|
||||||
|
const testMsg = `cacheable-false-header-${Date.now()}-${Math.random().toString(36).slice(2)}`;
|
||||||
|
spawnCallCount = 0;
|
||||||
|
__setSpawnImpl(makeCountingMockSpawn(['response-for-header-test']));
|
||||||
|
|
||||||
|
const reqBody = {
|
||||||
|
model: 'claude-haiku-4-5',
|
||||||
|
messages: [{ role: 'user', content: testMsg }],
|
||||||
|
};
|
||||||
|
|
||||||
|
const r1 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
|
||||||
|
assert.equal(r1.status, 200, `First request: ${r1.status} ${r1.body.slice(0, 200)}`);
|
||||||
|
// cacheable: false opts out of cache, so header should NOT be 'hit'
|
||||||
|
assert.notEqual(r1.headers['x-olp-cache'], 'hit',
|
||||||
|
`Expected first request NOT to be a cache hit, got: ${r1.headers['x-olp-cache']}`);
|
||||||
|
|
||||||
|
__setSpawnImpl(makeCountingMockSpawn(['response-for-header-test']));
|
||||||
|
const r2 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
|
||||||
|
assert.equal(r2.status, 200, `Second request: ${r2.status} ${r2.body.slice(0, 200)}`);
|
||||||
|
assert.notEqual(r2.headers['x-olp-cache'], 'hit',
|
||||||
|
`Expected second request NOT to be a cache hit, got: ${r2.headers['x-olp-cache']}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('cacheable: false + stream: true — both requests trigger fresh spawn; X-OLP-Cache is miss on both', async () => {
|
||||||
|
// D23 real-streaming branch fix: a cacheable: false provider with stream: true
|
||||||
|
// must NOT enter the D10 real-streaming path (which would write to cache).
|
||||||
|
// Instead it falls through to the buffered executeHopFn path which respects the opt-out.
|
||||||
|
// Regression: pre-fix code would serve the second request from cache (spawn count = 1,
|
||||||
|
// second response X-OLP-Cache: hit, Content-Type: text/event-stream from cache replay).
|
||||||
|
const testMsg = `cacheable-false-stream-${Date.now()}-${Math.random().toString(36).slice(2)}`;
|
||||||
|
|
||||||
|
const reqBody = {
|
||||||
|
model: 'claude-haiku-4-5',
|
||||||
|
messages: [{ role: 'user', content: testMsg }],
|
||||||
|
stream: true,
|
||||||
|
};
|
||||||
|
|
||||||
|
// First streaming request
|
||||||
|
spawnCallCount = 0;
|
||||||
|
__setSpawnImpl(makeCountingMockSpawn(['stream-chunk-r1']));
|
||||||
|
const r1 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
|
||||||
|
assert.equal(r1.status, 200, `First streaming request failed: ${r1.status} ${r1.body.slice(0, 200)}`);
|
||||||
|
assert.ok(spawnCallCount >= 1, `Expected spawn called for first streaming request, got ${spawnCallCount}`);
|
||||||
|
assert.notEqual(r1.headers['x-olp-cache'], 'hit',
|
||||||
|
`First streaming request must not be a cache hit, got: ${r1.headers['x-olp-cache']}`);
|
||||||
|
|
||||||
|
// Second identical streaming request — must also trigger spawn (no cache write after r1)
|
||||||
|
spawnCallCount = 0;
|
||||||
|
__setSpawnImpl(makeCountingMockSpawn(['stream-chunk-r2']));
|
||||||
|
const r2 = await fetch({ port: port9e, method: 'POST', path: '/v1/chat/completions', body: reqBody });
|
||||||
|
assert.equal(r2.status, 200, `Second streaming request failed: ${r2.status} ${r2.body.slice(0, 200)}`);
|
||||||
|
assert.ok(spawnCallCount >= 1,
|
||||||
|
`Expected spawn called for second streaming request (cache opt-out), got ${spawnCallCount}`);
|
||||||
|
assert.notEqual(r2.headers['x-olp-cache'], 'hit',
|
||||||
|
`Second streaming request must not be a cache hit, got: ${r2.headers['x-olp-cache']}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
// ── Suite 9d: D13 cache_control per-hop bypass correctness ───────────────
|
// ── Suite 9d: D13 cache_control per-hop bypass correctness ───────────────
|
||||||
//
|
//
|
||||||
// D13 (ADR 0005 § D2): cache_control markers bypass OLP's response cache ONLY
|
// D13 (ADR 0005 § D2): cache_control markers bypass OLP's response cache ONLY
|
||||||
|
|||||||
Reference in New Issue
Block a user