mirror of
https://github.com/dtzp555-max/ocp.git
synced 2026-07-19 09:44:07 +00:00
feat(cache): singleflight stampede protection on non-streaming path (#66)
cli.js does not perform proxy-layer stampede protection. The singleflight layer is a value-add proxy operation that exists only inside OCP, between concurrent client requests and the single upstream cli.js spawn. It does not introduce, alter, or remove any endpoint, header, request field, or response field that cli.js emits or expects — no client-observable wire shape change. Justification under ALIGNMENT.md Rule 2: the singleflight Map deduplicates concurrent identical non-streaming cache-miss requests so only one cli.js spawn runs per unique hash window. All followers receive the same resolved (or rejected) content. This is a proxy-internal concurrency optimization, not a wire-level protocol change. Spec: docs/superpowers/specs/2026-05-07-cache-upgrade-design.md (D4) Changes: - keys.mjs: add singleflight(hash, fn) + getInflightStats() exports; in-memory Map cleared via Promise.finally() on each settlement - server.mjs: import singleflight + getInflightStats; wrap non-streaming cache-enabled path through singleflight with inner recheck; add TODO comment in callClaudeStreaming (streaming-path dedup is explicitly out of scope for v3.13.0, see spec D4 streaming caveat); extend /cache/stats to return inflight + requesters fields (additive, no removed fields) - test-features.mjs: 7 new PR-B singleflight tests covering basic dedup, failure fan-out, map cleanup (success + failure), different-hash independence, getInflightStats shape, and sequential-call non-sharing; all 31 tests pass (24 existing + 7 new) Streaming-path singleflight is explicitly out of scope; TODO left in callClaudeStreaming for a future follow-up ticket. Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -371,6 +371,36 @@ export function getCacheStats() {
|
||||
return { entries: total, totalHits, sizeBytes };
|
||||
}
|
||||
|
||||
// ── Singleflight stampede protection ──
|
||||
|
||||
// In-memory singleflight Map: hash → { promise, requesters }
|
||||
// Deduplicates concurrent identical cache-miss flows so only one upstream call runs.
|
||||
// Per ADR 0005 / spec D4: in-process scope only (single Node process per host).
|
||||
const inflightMap = new Map();
|
||||
|
||||
export function singleflight(hash, fn) {
|
||||
const existing = inflightMap.get(hash);
|
||||
if (existing) {
|
||||
existing.requesters++;
|
||||
return existing.promise;
|
||||
}
|
||||
// Wrap fn() in Promise.resolve().then() so synchronous throws don't escape.
|
||||
const promise = Promise.resolve().then(fn).finally(() => {
|
||||
inflightMap.delete(hash);
|
||||
});
|
||||
inflightMap.set(hash, { promise, requesters: 1 });
|
||||
return promise;
|
||||
}
|
||||
|
||||
export function getInflightStats() {
|
||||
let totalRequesters = 0;
|
||||
for (const entry of inflightMap.values()) totalRequesters += entry.requesters;
|
||||
return {
|
||||
inflight: inflightMap.size,
|
||||
requesters: totalRequesters,
|
||||
};
|
||||
}
|
||||
|
||||
// Find a key by id or name (returns { id, name } or null)
|
||||
export function findKey(idOrName) {
|
||||
const d = getDb();
|
||||
|
||||
Reference in New Issue
Block a user