mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-21 21:15:10 +00:00
* feat+test: D57 — cache layer streaming singleflight (ADR 0005 Amendment 8, issue #16) First of three D-days implementing the v1.x roadmap #1 streaming-path singleflight. D57 lands the cache-layer coordination primitive only; server.mjs wiring is D58, docs polish is D59. ## What lib/cache/store.mjs — new method `getOrComputeStreaming(keyId, cacheKey, sourceFactory, opts) → { stream, isFirst, role }`. Three outcomes per Amendment 8 §1: cache_hit (no spawn), attached (joins existing inflight), source (first caller, spawns via factory). Backed by `_streamingInflight` Map keyed by `${keyId}\\0${cacheKey}` with synchronous check+insert per Amendment 8 §1 + §6 atomicity invariant. Internals (Amendment 8 §§2-10, §14): - StreamingInflightEntry + AttachedClient typedefs - Tee fan-out loop: single reader drains source, pushes to accumulatedChunks + every client's queue, fires per-client resolveNext promises - Late-joiner replay buffer (synchronous drain on attach; reject with synthetic STREAM_BACKPRESSURE terminator if drain exceeds cap) - Per-client backpressure (PER_CLIENT_QUEUE_CAP=1MB default, overridable) - Replay buffer cap (ACCUMULATED_REPLAY_CAP=10MB default, overridable; cache write skipped if exceeded) - AbortController propagation: when attachedClients.size === 0 after client iterator return(), source.return() + abort.signal fire - D38 coordination via sourceFactory closure (factory wraps tryAcquireSpawn internally; cache layer just invokes it once) lib/providers/base.mjs — `'STREAM_BACKPRESSURE'` added to PROVIDER_ERROR_CODES per Amendment 8 §8. NOT in HARD_TRIGGER_CODES (engine update lands in D58; whitelist-only map gives correct default). test-features.mjs Suite 27 — 12 new tests (27a-27l) covering: solo stream, 2-concurrent dedup, mid-stream join + post-completion cache_hit, per-client disconnect with other clients continuing, full disconnect → abort, source error propagation, per-client backpressure, replay cap, TTL race during inflight, sourceFactory throw, stats accuracy, composite key isolation. ## Scope Strictly cache layer + base.mjs PROVIDER_ERROR_CODES entry. Untouched: server.mjs, providers/{anthropic,codex,mistral}.mjs, fallback/engine.mjs, IR, dashboard.html, README, CHANGELOG, package.json. D58 will wire server. ## Authority - docs/adr/0005-cache-cross-provider.md Amendment 8 (2026-05-25, design ratified at D42, implementation gated on maintainer "go" — fired 2026-05-25 post-v0.3.1) - docs/v1x-roadmap.md #1 (streaming SF + TOCTOU close) - GitHub issue #16 (round-6 cold-audit F13 sibling TOCTOU window) - ADR 0002 Amendment 6 (D38 tryAcquireSpawn/releaseSpawn — invoked via sourceFactory closure at server layer, not directly by cache) ## Test count 603 → 615 (+12 D57 tests). Local: 615/615 pass. ## Iron Rule 11 (IDR) D57 is the cache-layer minimum reviewable unit. D58 wires server.mjs + adds X-OLP-Streaming-Inflight header + integration tests through HTTP layer. D59 polishes README + closes issue #16. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: D57 reviewer follow-ups — P2-2 (constant cleanup) + P2-3 (join-event deferral note) Fold-in for D57 PR #36 fresh-context opus reviewer findings (APPROVE WITH MINOR — 0 P0/P1, 3 P2). P2-1 is the D58 split (already planned); this commit addresses P2-2 + P2-3. P2-2 (cosmetic) — replace `Object.freeze({ value: X }).value` baroque declaration of PER_CLIENT_QUEUE_CAP_DEFAULT + ACCUMULATED_REPLAY_CAP_DEFAULT with a plain `export const X = 1*1024*1024`. The freeze-then-extract pattern freezes a throwaway wrapper, which the `.value` immediately discards — does nothing useful. Const declaration already gives binding immutability. P2-3 (observability event parity deferral) — ADR 0005 Amendment 8 §11 lists `streaming_inflight_join` as one of four log events. The cache layer cannot emit it correctly because provider/model identity lives in the sourceFactory closure (server-layer concern). Added TODO note in `_attachClient` pointing at D58 server wiring where the event will fire on the consumer of `role: 'attached'`. The other three §11 events (stream_backpressure_disconnect / streaming_inflight_source_done / streaming_inflight_abort) ARE emitted from the cache layer with {client_id, composite_key, ...} payloads; provider/model is enriched at the server-side wrapper. No test-surface change. 615/615 still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
258 lines
10 KiB
JavaScript
258 lines
10 KiB
JavaScript
/**
|
|
* lib/providers/base.mjs — Provider contract definition and shared helpers
|
|
*
|
|
* Authority: ADR 0002 § "Provider contract (v1.0 interface)"
|
|
*
|
|
* This module does NOT implement the Provider contract itself.
|
|
* Provider plugins compose the helpers exported here; they do not inherit
|
|
* from a base class (per ADR 0002 § Consequences/Mitigations: "compose helpers,
|
|
* do not inherit").
|
|
*/
|
|
|
|
// ── Contract typedef ──────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* @typedef {Object} ProviderAuth
|
|
* @property {string} type - e.g. 'subscription', 'api-key', 'oauth'
|
|
* @property {string} storage - e.g. 'cli-managed', 'env', 'keychain'
|
|
* @property {string} path - artifact location hint
|
|
* @property {string|null} refresh - refresh mechanism or null if not applicable
|
|
*/
|
|
|
|
/**
|
|
* @typedef {Object} ProviderHints
|
|
* @property {boolean} requiresTTY
|
|
* @property {boolean} concurrentSpawnSafe
|
|
* @property {number} maxConcurrent
|
|
* @property {number} [maxSpawnTimeMs] - optional integer milliseconds, default 600000
|
|
* @property {boolean} [cacheable] - optional, default true; if false, opt out of OLP's
|
|
* response cache entirely — executeHopFn skips cacheStore.getOrCompute and calls
|
|
* collectAllChunks directly. ADR 0002 Amendment 3 (D23).
|
|
*/
|
|
|
|
/**
|
|
* @typedef {Object} ProviderContractV1
|
|
* @property {string} name - unique lowercase key
|
|
* @property {string} displayName - human-readable name
|
|
* @property {'1.0'} contractVersion - must be '1.0' for v1.0 plugins (D4 fold-in per reviewer F3)
|
|
* @property {string[]} models - model strings this provider serves
|
|
* @property {ProviderAuth} auth
|
|
* @property {function} spawn - async (irRequest, authContext) => AsyncIterator<IRResponseChunk>
|
|
* @property {function} estimateCost - (request) => {inputTokens, outputTokensEstimate, currency, usd}|null
|
|
* @property {function} quotaStatus - async (authContext) => {available, percentUsed, resetsAt, pool}|null
|
|
* @property {function} healthCheck - async () => {ok: boolean, latencyMs: number, error?: string}
|
|
* @property {ProviderHints} hints
|
|
*/
|
|
|
|
// ── Contract validator ────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Validates that a plugin object satisfies the v1.0 Provider contract.
|
|
* Per ADR 0002 § "Loading model", the registry calls this at startup for
|
|
* every registered provider; an invalid provider throws rather than silently
|
|
* degrading.
|
|
*
|
|
* @param {*} p
|
|
* @returns {{ valid: boolean, errors: string[] }}
|
|
*/
|
|
export function validateProvider(p) {
|
|
const errors = [];
|
|
|
|
if (!p || typeof p !== 'object') {
|
|
errors.push('provider must be an object');
|
|
return { valid: false, errors };
|
|
}
|
|
|
|
if (typeof p.name !== 'string' || p.name.trim() === '') {
|
|
errors.push('name must be a non-empty string');
|
|
} else if (!/^[a-z][a-z0-9_-]*$/.test(p.name)) {
|
|
errors.push('name must be lowercase alphanumeric (with _ or -) starting with a letter');
|
|
}
|
|
|
|
if (typeof p.displayName !== 'string' || p.displayName.trim() === '') {
|
|
errors.push('displayName must be a non-empty string');
|
|
}
|
|
|
|
// contractVersion: required to be exactly '1.0' for v1.0 plugins (D4 fold-in per reviewer F3)
|
|
// Per ADR 0002 § Mitigations: "The contract is versioned. v1.0 is the subset in this ADR;
|
|
// future additions require ADR amendment plus a contract-version bump."
|
|
if (p.contractVersion !== '1.0') {
|
|
errors.push(`contractVersion must be '1.0', got ${JSON.stringify(p.contractVersion)}`);
|
|
}
|
|
|
|
if (!Array.isArray(p.models)) {
|
|
errors.push('models must be an array of strings');
|
|
} else if (p.models.some(m => typeof m !== 'string')) {
|
|
errors.push('every entry in models must be a string');
|
|
}
|
|
|
|
if (!p.auth || typeof p.auth !== 'object') {
|
|
errors.push('auth must be an object with { type, storage, path, refresh }');
|
|
} else {
|
|
if (typeof p.auth.type !== 'string') errors.push('auth.type must be a string');
|
|
if (typeof p.auth.storage !== 'string') errors.push('auth.storage must be a string');
|
|
if (typeof p.auth.path !== 'string') errors.push('auth.path must be a string');
|
|
if (p.auth.refresh !== null && typeof p.auth.refresh !== 'string') {
|
|
errors.push('auth.refresh must be a string or null');
|
|
}
|
|
}
|
|
|
|
if (typeof p.spawn !== 'function') {
|
|
errors.push('spawn must be a function');
|
|
}
|
|
|
|
if (typeof p.estimateCost !== 'function') {
|
|
errors.push('estimateCost must be a function');
|
|
}
|
|
|
|
if (typeof p.quotaStatus !== 'function') {
|
|
errors.push('quotaStatus must be a function');
|
|
}
|
|
|
|
if (typeof p.healthCheck !== 'function') {
|
|
errors.push('healthCheck must be a function');
|
|
}
|
|
|
|
if (!p.hints || typeof p.hints !== 'object') {
|
|
errors.push('hints must be an object with { requiresTTY, concurrentSpawnSafe, maxConcurrent } + optional { maxSpawnTimeMs, cacheable }');
|
|
} else {
|
|
if (typeof p.hints.requiresTTY !== 'boolean') errors.push('hints.requiresTTY must be a boolean');
|
|
if (typeof p.hints.concurrentSpawnSafe !== 'boolean') errors.push('hints.concurrentSpawnSafe must be a boolean');
|
|
if (typeof p.hints.maxConcurrent !== 'number' || !Number.isInteger(p.hints.maxConcurrent) || p.hints.maxConcurrent < 0) {
|
|
errors.push('hints.maxConcurrent must be a non-negative integer');
|
|
}
|
|
if (p.hints.maxSpawnTimeMs !== undefined) {
|
|
if (typeof p.hints.maxSpawnTimeMs !== 'number' || !Number.isInteger(p.hints.maxSpawnTimeMs) || p.hints.maxSpawnTimeMs <= 0) {
|
|
errors.push('hints.maxSpawnTimeMs must be a positive integer (milliseconds) or omitted');
|
|
}
|
|
}
|
|
// ADR 0002 Amendment 3 (D23): cacheable is optional; if present must be boolean.
|
|
// undefined → default true (cacheable); false → provider opts out of cache.
|
|
if (p.hints.cacheable !== undefined && typeof p.hints.cacheable !== 'boolean') {
|
|
errors.push('hints.cacheable must be a boolean or omitted');
|
|
}
|
|
}
|
|
|
|
return { valid: errors.length === 0, errors };
|
|
}
|
|
|
|
// ── Error class ───────────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Error codes surfaced by provider plugins.
|
|
*
|
|
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7 and Amendment 4, D38):
|
|
* SPAWN_FAILED, CLI_NOT_FOUND, AUTH_MISSING, SPAWN_TIMEOUT, CONCURRENCY_LIMIT
|
|
*
|
|
* CONCURRENCY_LIMIT (D38, issue #1): synthesized by the orchestration layer
|
|
* (NOT thrown by provider plugins themselves) when a spawn is attempted
|
|
* against a provider already at its hints.maxConcurrent limit. The fallback
|
|
* engine treats this as a hard trigger so the chain advances to the next hop
|
|
* rather than queueing. See ADR 0002 Amendment 6 (runtime enforcement) and
|
|
* ADR 0004 Amendment 4 (CONCURRENCY_LIMIT in hard-trigger taxonomy).
|
|
*
|
|
* QUOTA_EXHAUSTED and RATE_LIMITED were removed (D34 F7): no plugin parses
|
|
* underlying-API HTTP status codes at v0.1, so these codes are never emitted.
|
|
* Re-add via ADR 0004 amendment when a plugin gains HTTP-status parsing.
|
|
* (Previously removed: OUTPUT_PARSE_ERROR, D32 F4.)
|
|
*/
|
|
export const PROVIDER_ERROR_CODES = /** @type {const} */ ([
|
|
'AUTH_MISSING',
|
|
'CLI_NOT_FOUND',
|
|
'SPAWN_FAILED',
|
|
'SPAWN_TIMEOUT', // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
|
|
'CONCURRENCY_LIMIT', // ADR 0002 Amendment 6 / ADR 0004 Amendment 4 (D38, issue #1)
|
|
/* ADR 0005 Amendment 8 §8 (D57): per-client streaming queue overflow.
|
|
* NOT a hard trigger — the source spawned successfully and other attached
|
|
* clients continue to receive chunks; only this client's queue exceeded
|
|
* PER_CLIENT_QUEUE_CAP. The affected client receives a synthetic
|
|
* { type: 'stop', finish_reason: 'length' } + [DONE] terminator.
|
|
* D58 wires server-side header/log surface; HARD_TRIGGER_CODES in
|
|
* lib/fallback/engine.mjs is a whitelist so absence here gives the
|
|
* correct default (no fallback advancement). */
|
|
'STREAM_BACKPRESSURE',
|
|
]);
|
|
|
|
export class ProviderError extends Error {
|
|
/**
|
|
* @param {string} message
|
|
* @param {typeof PROVIDER_ERROR_CODES[number]} code
|
|
*/
|
|
constructor(message, code) {
|
|
super(message);
|
|
this.name = 'ProviderError';
|
|
this.code = code;
|
|
}
|
|
}
|
|
|
|
// ── Shared helpers ────────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Wraps a promise with a timeout. Rejects with a ProviderError if the promise
|
|
* does not settle within `ms` milliseconds.
|
|
*
|
|
* @template T
|
|
* @param {Promise<T>} promise
|
|
* @param {number} ms
|
|
* @param {typeof PROVIDER_ERROR_CODES[number]} errorCode
|
|
* @returns {Promise<T>}
|
|
*/
|
|
export function withTimeout(promise, ms, errorCode) {
|
|
return new Promise((resolve, reject) => {
|
|
const timer = setTimeout(() => {
|
|
reject(new ProviderError(`Operation timed out after ${ms}ms`, errorCode));
|
|
}, ms);
|
|
promise.then(
|
|
v => { clearTimeout(timer); resolve(v); },
|
|
e => { clearTimeout(timer); reject(e); },
|
|
);
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Merges two AsyncIterators into a single ordered stream.
|
|
* Items from whichever source yields first are emitted first.
|
|
* Useful when a provider plugin wants to interleave two internal streams.
|
|
*
|
|
* Not used at D3 (no providers yet) but provided as infrastructure so
|
|
* provider authors don't each implement their own fan-in.
|
|
*
|
|
* @template T
|
|
* @param {AsyncIterator<T>} iter1
|
|
* @param {AsyncIterator<T>} iter2
|
|
* @returns {AsyncGenerator<T>}
|
|
*/
|
|
export async function* mergeStreams(iter1, iter2) {
|
|
// Convert each iterator to a pull-based promise queue
|
|
const done1 = { done: true };
|
|
const done2 = { done: true };
|
|
|
|
let p1 = iter1.next();
|
|
let p2 = iter2.next();
|
|
|
|
while (true) {
|
|
const winner = await Promise.race([
|
|
p1.then(r => ({ r, which: 1 })),
|
|
p2.then(r => ({ r, which: 2 })),
|
|
]);
|
|
|
|
if (winner.which === 1) {
|
|
if (winner.r.done) {
|
|
// iter1 exhausted — drain iter2
|
|
for await (const v of { [Symbol.asyncIterator]: () => iter2 }) yield v;
|
|
return;
|
|
}
|
|
yield winner.r.value;
|
|
p1 = iter1.next();
|
|
} else {
|
|
if (winner.r.done) {
|
|
// iter2 exhausted — drain iter1
|
|
for await (const v of { [Symbol.asyncIterator]: () => iter1 }) yield v;
|
|
return;
|
|
}
|
|
yield winner.r.value;
|
|
p2 = iter2.next();
|
|
}
|
|
}
|
|
}
|