mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-21 21:15:10 +00:00
ADR 0002 Amendment 1 declared `hints.maxConcurrent` declarative-only at
v0.1 — type-validated at startup but with no runtime enforcement.
D38 wires the runtime enforcement via a per-provider in-flight spawn
counter with immediate-advancement on saturation.
Design choice: **immediate-advancement via fallback engine** (queue +
timeout DEFERRED). When a provider is at its maxConcurrent limit, the
spawn call synchronously fails with a new `CONCURRENCY_LIMIT` error
code; the fallback engine treats this as a hard trigger and advances
to the next chain hop. If the entire chain is saturated, the user
sees a chain-exhausted error (existing path).
Rationale for immediate-advancement over queue+timeout:
1. The fallback chain exists precisely for this kind of overflow —
adding a queue layer would duplicate the advancement semantics.
2. Queue + timeout adds new config surface (timeout duration, queue
depth bounds, queue eviction policy) that isn't needed at the
personal/family scale OLP serves.
3. Head-of-line blocking risk: a long-running spawn would stall
queued requests behind it even though other providers in the chain
could serve them immediately.
4. Fail-fast latency aligns with the multi-provider proxy philosophy
("spread risk across providers, not within a provider").
Queue + timeout is deferred to a v1.x design ADR if real usage shows
demand. ADR 0002 Amendment 6 and ADR 0004 Amendment 4 capture the
decision explicitly.
Changes (8 files, +<delta>):
**Code**
1. **lib/providers/base.mjs** — add `CONCURRENCY_LIMIT` to
`PROVIDER_ERROR_CODES`. JSDoc clarifies the code is synthesised by
the orchestration layer, not thrown by provider plugins themselves.
2. **lib/providers/index.mjs** — new semaphore primitives:
- `tryAcquireSpawn(providerName, maxConcurrent)` — atomic
check-then-increment. Returns `true` on success, `false` if at
limit. Atomicity rests on the JS single-threaded invariant
(read + write synchronous, NO `await` between them); module-level
comment block warns future maintainers against breaking this.
- `releaseSpawn(providerName)` — decrement; throws on
under-decrement (defensive bug guard for missing acquire / double
release). Map.delete at zero for clean memory footprint.
- `getActiveSpawnCount(providerName)` — returns current count
(0 for unseen providers). For diagnostics + tests.
- `DEFAULT_MAX_CONCURRENT_SPAWNS = 4` — defense-in-depth fallback
matching the v0.1 plugin defaults (anthropic/codex/mistral all
declare hints.maxConcurrent: 4). Also coerces non-integer / NaN
/ negative inputs to the default.
- `__resetSpawnCounters()` — internal test seam.
3. **lib/fallback/engine.mjs** — `CONCURRENCY_LIMIT: true` added to
`HARD_TRIGGER_CODES`. `classifyTrigger` and `evaluateHardTriggers`
both pick it up via the same lookup. v0.1 live hard-trigger codes
are now 5 (was 4 post-D34): SPAWN_FAILED, CLI_NOT_FOUND,
AUTH_MISSING:false, SPAWN_TIMEOUT, CONCURRENCY_LIMIT.
4. **server.mjs** — gate the spawn call in handleChatCompletions at
BOTH spawn call sites:
- **Buffered path** (executeHopFn → collectAllChunks): acquire
before provider.spawn; on failure synthesise
`ProviderError(CONCURRENCY_LIMIT)` with providerName /
maxConcurrent / activeSpawns diagnostic fields and throw —
fallback engine catches and advances. On success, outer try/
finally wraps the inner D16 truncation-salvage try/catch so
releaseSpawn fires on EVERY exit path (return, D16 salvage
return, re-throw).
- **Streaming path** (single-hop real-SSE branch): acquire BEFORE
the streaming branch entry. If acquire fails, branch is skipped
and request falls through to buffered path (whose own gate
surfaces chain-exhausted for single-hop chains). If acquire
succeeds, existing streaming try/catch gains
`finally { releaseSpawn(streamProvider) }` — slot releases on
stop-chunk completion, generator exhaustion, abort, or any
exception path. `releaseSpawn` fires at END of stream
consumption, not when spawn() returns.
**Tests** (test-features.mjs): 431 → 447 (+16):
Suite 18 — D38 — maxConcurrent runtime enforcement:
- 18a: PROVIDER_ERROR_CODES.CONCURRENCY_LIMIT exists
- 18b: evaluateHardTriggers true for CONCURRENCY_LIMIT
- 18c: AUTH_MISSING regression guard (D38 did NOT flip it to hard)
- 18d.1-18d.4: semaphore unit (increment / saturate-no-increment /
release / map-delete-at-zero)
- 18e: tryAcquireSpawn returns false without side-effect when at limit
- 18f: releaseSpawn throws on under-decrement
- 18g.1-18g.2: DEFAULT_MAX_CONCURRENT_SPAWNS applied for invalid
inputs (undefined / NaN / negative / non-integer)
- 18h: __resetSpawnCounters clears state
- 18i: HTTP integration — 5 concurrent requests to single-hop
chain w/ maxConcurrent=2 → peak in-flight exactly 2, 2 succeed,
3 fail
- 18j: counter releases after buffered request (sequential test)
- 18k: 2-hop chain — saturated primary advances to fallback
- 18l: streaming counter releases at END of stream (not at spawn)
**ADR amendments**
5. **docs/adr/0002-plugin-architecture.md** — Amendment 6 added.
Removes "Declarative hint only at v0.1" caveat from the
`maxConcurrent` description in the Provider contract section.
Adds implementation reference + design-choice rationale (4 points).
Lists all exported symbols.
6. **docs/adr/0004-fallback-engine.md** — Amendment 4 added.
CONCURRENCY_LIMIT added to v0.1 hard-trigger taxonomy. Documents
synthesis-vs-plugin-thrown distinction. Documents first-chunk
safety (acquire before any res.write). v1.x re-evaluation triggers
named.
**CHANGELOG**
7. **CHANGELOG.md** — Unreleased sentinel replaced with proper D38
entry. Per CLAUDE.md release_kit phase_rolling_mode, this lands
under Unreleased; promotion to ## v0.1.1 happens at the v0.1.1
release. The D37 phase_rolling_mode gate now correctly fires if
anyone tags v0.1.x with this content present without promotion —
intentional.
Pre-commit fold-ins (per evidence-first checkpoint #4):
- **Reviewer Suggestion #1 (test 18c name mismatch)**: 18c is named
"classifyTrigger returns hard for CONCURRENCY_LIMIT" but body tests
AUTH_MISSING regression. Renamed header comment to
"AUTH_MISSING regression guard" and clarified that
CONCURRENCY_LIMIT classification is covered by 18b.
- **Reviewer Suggestion #3 (activeSpawns diagnostic field)**: server
.mjs:532 set `concurrencyErr.activeSpawns = maxConcurrent` (the
limit). Technically correct (since acquire just failed, live
count == limit) but confuses future readers. Changed to query
`getActiveSpawnCount(hopProvider)` directly. Added import of the
new symbol to the lib/providers/index.mjs import block.
- **Reviewer Suggestion #5 (ADR overstatement)**: ADR 0002 Amendment 6
said getActiveSpawnCount is "exported for /health, diagnostics,
and tests" but /health integration is not wired at D38. Reworded to
"exported for diagnostics and tests" + "/health integration
deferred — when surfaced there will land at providers.status.<name>
.activeSpawns; not wired at D38." Avoids overstating current state.
Two reviewer suggestions not folded:
- Suggestion #2 (streaming test 18l comment about branch entry
conditions) — low priority; the test passes and the branch is
taken (verified by reviewer). Future polish if confusion arises.
- Suggestion #4 (additional test for streaming-saturation → chain-
exhausted) — code path is straightforward and intentional; 18i
covers the buffered-path version directly. Defer.
Authority:
- ADR 0002 Amendment 1 (declarative-only caveat) — superseded by
Amendment 6
- ADR 0002 Amendment 6 (this commit) — runtime enforcement landed
- ADR 0004 Amendment 4 (this commit) — CONCURRENCY_LIMIT added to
v0.1 hard-trigger taxonomy
- GitHub issue #1 — closed by this commit
- CC 开发铁律 v1.6 § 10.x — independent fresh-context reviewer
- CLAUDE.md release_kit_overlay phase_rolling_mode — no version
bump; Unreleased entry written
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus,
independent of drafter): APPROVE. Critical depth checks:
- Atomicity in tryAcquireSpawn (lines 285-290 read+set with no
intervening await) — confirmed; module-level invariant comment
warns future maintainers
- Release on every exit path: buffered (outer finally wraps inner
try/catch covering D16 salvage return + normal return + re-throw);
streaming (try/finally covers stop-chunk return + loop exhaustion +
catch paths). No double-release path identified.
- Streaming release timing: fires in finally after res.end() at all
exit paths, never before stream consumption completes
- Counter leak: every code path traced — no orphan acquire identified
- 447/447 tests pass in reviewer's independent npm test run
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
249 lines
9.8 KiB
JavaScript
249 lines
9.8 KiB
JavaScript
/**
|
|
* lib/providers/base.mjs — Provider contract definition and shared helpers
|
|
*
|
|
* Authority: ADR 0002 § "Provider contract (v1.0 interface)"
|
|
*
|
|
* This module does NOT implement the Provider contract itself.
|
|
* Provider plugins compose the helpers exported here; they do not inherit
|
|
* from a base class (per ADR 0002 § Consequences/Mitigations: "compose helpers,
|
|
* do not inherit").
|
|
*/
|
|
|
|
// ── Contract typedef ──────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* @typedef {Object} ProviderAuth
|
|
* @property {string} type - e.g. 'subscription', 'api-key', 'oauth'
|
|
* @property {string} storage - e.g. 'cli-managed', 'env', 'keychain'
|
|
* @property {string} path - artifact location hint
|
|
* @property {string|null} refresh - refresh mechanism or null if not applicable
|
|
*/
|
|
|
|
/**
|
|
* @typedef {Object} ProviderHints
|
|
* @property {boolean} requiresTTY
|
|
* @property {boolean} concurrentSpawnSafe
|
|
* @property {number} maxConcurrent
|
|
* @property {number} [maxSpawnTimeMs] - optional integer milliseconds, default 600000
|
|
* @property {boolean} [cacheable] - optional, default true; if false, opt out of OLP's
|
|
* response cache entirely — executeHopFn skips cacheStore.getOrCompute and calls
|
|
* collectAllChunks directly. ADR 0002 Amendment 3 (D23).
|
|
*/
|
|
|
|
/**
|
|
* @typedef {Object} ProviderContractV1
|
|
* @property {string} name - unique lowercase key
|
|
* @property {string} displayName - human-readable name
|
|
* @property {'1.0'} contractVersion - must be '1.0' for v1.0 plugins (D4 fold-in per reviewer F3)
|
|
* @property {string[]} models - model strings this provider serves
|
|
* @property {ProviderAuth} auth
|
|
* @property {function} spawn - async (irRequest, authContext) => AsyncIterator<IRResponseChunk>
|
|
* @property {function} estimateCost - (request) => {inputTokens, outputTokensEstimate, currency, usd}|null
|
|
* @property {function} quotaStatus - async (authContext) => {available, percentUsed, resetsAt, pool}|null
|
|
* @property {function} healthCheck - async () => {ok: boolean, latencyMs: number, error?: string}
|
|
* @property {ProviderHints} hints
|
|
*/
|
|
|
|
// ── Contract validator ────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Validates that a plugin object satisfies the v1.0 Provider contract.
|
|
* Per ADR 0002 § "Loading model", the registry calls this at startup for
|
|
* every registered provider; an invalid provider throws rather than silently
|
|
* degrading.
|
|
*
|
|
* @param {*} p
|
|
* @returns {{ valid: boolean, errors: string[] }}
|
|
*/
|
|
export function validateProvider(p) {
|
|
const errors = [];
|
|
|
|
if (!p || typeof p !== 'object') {
|
|
errors.push('provider must be an object');
|
|
return { valid: false, errors };
|
|
}
|
|
|
|
if (typeof p.name !== 'string' || p.name.trim() === '') {
|
|
errors.push('name must be a non-empty string');
|
|
} else if (!/^[a-z][a-z0-9_-]*$/.test(p.name)) {
|
|
errors.push('name must be lowercase alphanumeric (with _ or -) starting with a letter');
|
|
}
|
|
|
|
if (typeof p.displayName !== 'string' || p.displayName.trim() === '') {
|
|
errors.push('displayName must be a non-empty string');
|
|
}
|
|
|
|
// contractVersion: required to be exactly '1.0' for v1.0 plugins (D4 fold-in per reviewer F3)
|
|
// Per ADR 0002 § Mitigations: "The contract is versioned. v1.0 is the subset in this ADR;
|
|
// future additions require ADR amendment plus a contract-version bump."
|
|
if (p.contractVersion !== '1.0') {
|
|
errors.push(`contractVersion must be '1.0', got ${JSON.stringify(p.contractVersion)}`);
|
|
}
|
|
|
|
if (!Array.isArray(p.models)) {
|
|
errors.push('models must be an array of strings');
|
|
} else if (p.models.some(m => typeof m !== 'string')) {
|
|
errors.push('every entry in models must be a string');
|
|
}
|
|
|
|
if (!p.auth || typeof p.auth !== 'object') {
|
|
errors.push('auth must be an object with { type, storage, path, refresh }');
|
|
} else {
|
|
if (typeof p.auth.type !== 'string') errors.push('auth.type must be a string');
|
|
if (typeof p.auth.storage !== 'string') errors.push('auth.storage must be a string');
|
|
if (typeof p.auth.path !== 'string') errors.push('auth.path must be a string');
|
|
if (p.auth.refresh !== null && typeof p.auth.refresh !== 'string') {
|
|
errors.push('auth.refresh must be a string or null');
|
|
}
|
|
}
|
|
|
|
if (typeof p.spawn !== 'function') {
|
|
errors.push('spawn must be a function');
|
|
}
|
|
|
|
if (typeof p.estimateCost !== 'function') {
|
|
errors.push('estimateCost must be a function');
|
|
}
|
|
|
|
if (typeof p.quotaStatus !== 'function') {
|
|
errors.push('quotaStatus must be a function');
|
|
}
|
|
|
|
if (typeof p.healthCheck !== 'function') {
|
|
errors.push('healthCheck must be a function');
|
|
}
|
|
|
|
if (!p.hints || typeof p.hints !== 'object') {
|
|
errors.push('hints must be an object with { requiresTTY, concurrentSpawnSafe, maxConcurrent } + optional { maxSpawnTimeMs, cacheable }');
|
|
} else {
|
|
if (typeof p.hints.requiresTTY !== 'boolean') errors.push('hints.requiresTTY must be a boolean');
|
|
if (typeof p.hints.concurrentSpawnSafe !== 'boolean') errors.push('hints.concurrentSpawnSafe must be a boolean');
|
|
if (typeof p.hints.maxConcurrent !== 'number' || !Number.isInteger(p.hints.maxConcurrent) || p.hints.maxConcurrent < 0) {
|
|
errors.push('hints.maxConcurrent must be a non-negative integer');
|
|
}
|
|
if (p.hints.maxSpawnTimeMs !== undefined) {
|
|
if (typeof p.hints.maxSpawnTimeMs !== 'number' || !Number.isInteger(p.hints.maxSpawnTimeMs) || p.hints.maxSpawnTimeMs <= 0) {
|
|
errors.push('hints.maxSpawnTimeMs must be a positive integer (milliseconds) or omitted');
|
|
}
|
|
}
|
|
// ADR 0002 Amendment 3 (D23): cacheable is optional; if present must be boolean.
|
|
// undefined → default true (cacheable); false → provider opts out of cache.
|
|
if (p.hints.cacheable !== undefined && typeof p.hints.cacheable !== 'boolean') {
|
|
errors.push('hints.cacheable must be a boolean or omitted');
|
|
}
|
|
}
|
|
|
|
return { valid: errors.length === 0, errors };
|
|
}
|
|
|
|
// ── Error class ───────────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Error codes surfaced by provider plugins.
|
|
*
|
|
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7 and Amendment 4, D38):
|
|
* SPAWN_FAILED, CLI_NOT_FOUND, AUTH_MISSING, SPAWN_TIMEOUT, CONCURRENCY_LIMIT
|
|
*
|
|
* CONCURRENCY_LIMIT (D38, issue #1): synthesized by the orchestration layer
|
|
* (NOT thrown by provider plugins themselves) when a spawn is attempted
|
|
* against a provider already at its hints.maxConcurrent limit. The fallback
|
|
* engine treats this as a hard trigger so the chain advances to the next hop
|
|
* rather than queueing. See ADR 0002 Amendment 6 (runtime enforcement) and
|
|
* ADR 0004 Amendment 4 (CONCURRENCY_LIMIT in hard-trigger taxonomy).
|
|
*
|
|
* QUOTA_EXHAUSTED and RATE_LIMITED were removed (D34 F7): no plugin parses
|
|
* underlying-API HTTP status codes at v0.1, so these codes are never emitted.
|
|
* Re-add via ADR 0004 amendment when a plugin gains HTTP-status parsing.
|
|
* (Previously removed: OUTPUT_PARSE_ERROR, D32 F4.)
|
|
*/
|
|
export const PROVIDER_ERROR_CODES = /** @type {const} */ ([
|
|
'AUTH_MISSING',
|
|
'CLI_NOT_FOUND',
|
|
'SPAWN_FAILED',
|
|
'SPAWN_TIMEOUT', // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
|
|
'CONCURRENCY_LIMIT', // ADR 0002 Amendment 6 / ADR 0004 Amendment 4 (D38, issue #1)
|
|
]);
|
|
|
|
export class ProviderError extends Error {
|
|
/**
|
|
* @param {string} message
|
|
* @param {typeof PROVIDER_ERROR_CODES[number]} code
|
|
*/
|
|
constructor(message, code) {
|
|
super(message);
|
|
this.name = 'ProviderError';
|
|
this.code = code;
|
|
}
|
|
}
|
|
|
|
// ── Shared helpers ────────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Wraps a promise with a timeout. Rejects with a ProviderError if the promise
|
|
* does not settle within `ms` milliseconds.
|
|
*
|
|
* @template T
|
|
* @param {Promise<T>} promise
|
|
* @param {number} ms
|
|
* @param {typeof PROVIDER_ERROR_CODES[number]} errorCode
|
|
* @returns {Promise<T>}
|
|
*/
|
|
export function withTimeout(promise, ms, errorCode) {
|
|
return new Promise((resolve, reject) => {
|
|
const timer = setTimeout(() => {
|
|
reject(new ProviderError(`Operation timed out after ${ms}ms`, errorCode));
|
|
}, ms);
|
|
promise.then(
|
|
v => { clearTimeout(timer); resolve(v); },
|
|
e => { clearTimeout(timer); reject(e); },
|
|
);
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Merges two AsyncIterators into a single ordered stream.
|
|
* Items from whichever source yields first are emitted first.
|
|
* Useful when a provider plugin wants to interleave two internal streams.
|
|
*
|
|
* Not used at D3 (no providers yet) but provided as infrastructure so
|
|
* provider authors don't each implement their own fan-in.
|
|
*
|
|
* @template T
|
|
* @param {AsyncIterator<T>} iter1
|
|
* @param {AsyncIterator<T>} iter2
|
|
* @returns {AsyncGenerator<T>}
|
|
*/
|
|
export async function* mergeStreams(iter1, iter2) {
|
|
// Convert each iterator to a pull-based promise queue
|
|
const done1 = { done: true };
|
|
const done2 = { done: true };
|
|
|
|
let p1 = iter1.next();
|
|
let p2 = iter2.next();
|
|
|
|
while (true) {
|
|
const winner = await Promise.race([
|
|
p1.then(r => ({ r, which: 1 })),
|
|
p2.then(r => ({ r, which: 2 })),
|
|
]);
|
|
|
|
if (winner.which === 1) {
|
|
if (winner.r.done) {
|
|
// iter1 exhausted — drain iter2
|
|
for await (const v of { [Symbol.asyncIterator]: () => iter2 }) yield v;
|
|
return;
|
|
}
|
|
yield winner.r.value;
|
|
p1 = iter1.next();
|
|
} else {
|
|
if (winner.r.done) {
|
|
// iter2 exhausted — drain iter1
|
|
for await (const v of { [Symbol.asyncIterator]: () => iter1 }) yield v;
|
|
return;
|
|
}
|
|
yield winner.r.value;
|
|
p2 = iter2.next();
|
|
}
|
|
}
|
|
}
|