mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-21 21:15:10 +00:00
feat+ci: D38 — maxConcurrent runtime enforcement (issue #1)
ADR 0002 Amendment 1 declared `hints.maxConcurrent` declarative-only at
v0.1 — type-validated at startup but with no runtime enforcement.
D38 wires the runtime enforcement via a per-provider in-flight spawn
counter with immediate-advancement on saturation.
Design choice: **immediate-advancement via fallback engine** (queue +
timeout DEFERRED). When a provider is at its maxConcurrent limit, the
spawn call synchronously fails with a new `CONCURRENCY_LIMIT` error
code; the fallback engine treats this as a hard trigger and advances
to the next chain hop. If the entire chain is saturated, the user
sees a chain-exhausted error (existing path).
Rationale for immediate-advancement over queue+timeout:
1. The fallback chain exists precisely for this kind of overflow —
adding a queue layer would duplicate the advancement semantics.
2. Queue + timeout adds new config surface (timeout duration, queue
depth bounds, queue eviction policy) that isn't needed at the
personal/family scale OLP serves.
3. Head-of-line blocking risk: a long-running spawn would stall
queued requests behind it even though other providers in the chain
could serve them immediately.
4. Fail-fast latency aligns with the multi-provider proxy philosophy
("spread risk across providers, not within a provider").
Queue + timeout is deferred to a v1.x design ADR if real usage shows
demand. ADR 0002 Amendment 6 and ADR 0004 Amendment 4 capture the
decision explicitly.
Changes (8 files, +<delta>):
**Code**
1. **lib/providers/base.mjs** — add `CONCURRENCY_LIMIT` to
`PROVIDER_ERROR_CODES`. JSDoc clarifies the code is synthesised by
the orchestration layer, not thrown by provider plugins themselves.
2. **lib/providers/index.mjs** — new semaphore primitives:
- `tryAcquireSpawn(providerName, maxConcurrent)` — atomic
check-then-increment. Returns `true` on success, `false` if at
limit. Atomicity rests on the JS single-threaded invariant
(read + write synchronous, NO `await` between them); module-level
comment block warns future maintainers against breaking this.
- `releaseSpawn(providerName)` — decrement; throws on
under-decrement (defensive bug guard for missing acquire / double
release). Map.delete at zero for clean memory footprint.
- `getActiveSpawnCount(providerName)` — returns current count
(0 for unseen providers). For diagnostics + tests.
- `DEFAULT_MAX_CONCURRENT_SPAWNS = 4` — defense-in-depth fallback
matching the v0.1 plugin defaults (anthropic/codex/mistral all
declare hints.maxConcurrent: 4). Also coerces non-integer / NaN
/ negative inputs to the default.
- `__resetSpawnCounters()` — internal test seam.
3. **lib/fallback/engine.mjs** — `CONCURRENCY_LIMIT: true` added to
`HARD_TRIGGER_CODES`. `classifyTrigger` and `evaluateHardTriggers`
both pick it up via the same lookup. v0.1 live hard-trigger codes
are now 5 (was 4 post-D34): SPAWN_FAILED, CLI_NOT_FOUND,
AUTH_MISSING:false, SPAWN_TIMEOUT, CONCURRENCY_LIMIT.
4. **server.mjs** — gate the spawn call in handleChatCompletions at
BOTH spawn call sites:
- **Buffered path** (executeHopFn → collectAllChunks): acquire
before provider.spawn; on failure synthesise
`ProviderError(CONCURRENCY_LIMIT)` with providerName /
maxConcurrent / activeSpawns diagnostic fields and throw —
fallback engine catches and advances. On success, outer try/
finally wraps the inner D16 truncation-salvage try/catch so
releaseSpawn fires on EVERY exit path (return, D16 salvage
return, re-throw).
- **Streaming path** (single-hop real-SSE branch): acquire BEFORE
the streaming branch entry. If acquire fails, branch is skipped
and request falls through to buffered path (whose own gate
surfaces chain-exhausted for single-hop chains). If acquire
succeeds, existing streaming try/catch gains
`finally { releaseSpawn(streamProvider) }` — slot releases on
stop-chunk completion, generator exhaustion, abort, or any
exception path. `releaseSpawn` fires at END of stream
consumption, not when spawn() returns.
**Tests** (test-features.mjs): 431 → 447 (+16):
Suite 18 — D38 — maxConcurrent runtime enforcement:
- 18a: PROVIDER_ERROR_CODES.CONCURRENCY_LIMIT exists
- 18b: evaluateHardTriggers true for CONCURRENCY_LIMIT
- 18c: AUTH_MISSING regression guard (D38 did NOT flip it to hard)
- 18d.1-18d.4: semaphore unit (increment / saturate-no-increment /
release / map-delete-at-zero)
- 18e: tryAcquireSpawn returns false without side-effect when at limit
- 18f: releaseSpawn throws on under-decrement
- 18g.1-18g.2: DEFAULT_MAX_CONCURRENT_SPAWNS applied for invalid
inputs (undefined / NaN / negative / non-integer)
- 18h: __resetSpawnCounters clears state
- 18i: HTTP integration — 5 concurrent requests to single-hop
chain w/ maxConcurrent=2 → peak in-flight exactly 2, 2 succeed,
3 fail
- 18j: counter releases after buffered request (sequential test)
- 18k: 2-hop chain — saturated primary advances to fallback
- 18l: streaming counter releases at END of stream (not at spawn)
**ADR amendments**
5. **docs/adr/0002-plugin-architecture.md** — Amendment 6 added.
Removes "Declarative hint only at v0.1" caveat from the
`maxConcurrent` description in the Provider contract section.
Adds implementation reference + design-choice rationale (4 points).
Lists all exported symbols.
6. **docs/adr/0004-fallback-engine.md** — Amendment 4 added.
CONCURRENCY_LIMIT added to v0.1 hard-trigger taxonomy. Documents
synthesis-vs-plugin-thrown distinction. Documents first-chunk
safety (acquire before any res.write). v1.x re-evaluation triggers
named.
**CHANGELOG**
7. **CHANGELOG.md** — Unreleased sentinel replaced with proper D38
entry. Per CLAUDE.md release_kit phase_rolling_mode, this lands
under Unreleased; promotion to ## v0.1.1 happens at the v0.1.1
release. The D37 phase_rolling_mode gate now correctly fires if
anyone tags v0.1.x with this content present without promotion —
intentional.
Pre-commit fold-ins (per evidence-first checkpoint #4):
- **Reviewer Suggestion #1 (test 18c name mismatch)**: 18c is named
"classifyTrigger returns hard for CONCURRENCY_LIMIT" but body tests
AUTH_MISSING regression. Renamed header comment to
"AUTH_MISSING regression guard" and clarified that
CONCURRENCY_LIMIT classification is covered by 18b.
- **Reviewer Suggestion #3 (activeSpawns diagnostic field)**: server
.mjs:532 set `concurrencyErr.activeSpawns = maxConcurrent` (the
limit). Technically correct (since acquire just failed, live
count == limit) but confuses future readers. Changed to query
`getActiveSpawnCount(hopProvider)` directly. Added import of the
new symbol to the lib/providers/index.mjs import block.
- **Reviewer Suggestion #5 (ADR overstatement)**: ADR 0002 Amendment 6
said getActiveSpawnCount is "exported for /health, diagnostics,
and tests" but /health integration is not wired at D38. Reworded to
"exported for diagnostics and tests" + "/health integration
deferred — when surfaced there will land at providers.status.<name>
.activeSpawns; not wired at D38." Avoids overstating current state.
Two reviewer suggestions not folded:
- Suggestion #2 (streaming test 18l comment about branch entry
conditions) — low priority; the test passes and the branch is
taken (verified by reviewer). Future polish if confusion arises.
- Suggestion #4 (additional test for streaming-saturation → chain-
exhausted) — code path is straightforward and intentional; 18i
covers the buffered-path version directly. Defer.
Authority:
- ADR 0002 Amendment 1 (declarative-only caveat) — superseded by
Amendment 6
- ADR 0002 Amendment 6 (this commit) — runtime enforcement landed
- ADR 0004 Amendment 4 (this commit) — CONCURRENCY_LIMIT added to
v0.1 hard-trigger taxonomy
- GitHub issue #1 — closed by this commit
- CC 开发铁律 v1.6 § 10.x — independent fresh-context reviewer
- CLAUDE.md release_kit_overlay phase_rolling_mode — no version
bump; Unreleased entry written
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus,
independent of drafter): APPROVE. Critical depth checks:
- Atomicity in tryAcquireSpawn (lines 285-290 read+set with no
intervening await) — confirmed; module-level invariant comment
warns future maintainers
- Release on every exit path: buffered (outer finally wraps inner
try/catch covering D16 salvage return + normal return + re-throw);
streaming (try/finally covers stop-chunk return + loop exhaustion +
catch paths). No double-release path identified.
- Streaming release timing: fires in finally after res.end() at all
exit paths, never before stream consumption completes
- Counter leak: every code path traced — no orphan acquire identified
- 447/447 tests pass in reviewer's independent npm test run
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
+13
-7
@@ -27,11 +27,16 @@ import { computeIRRequestHash } from '../cache/keys.mjs';
|
||||
/**
|
||||
* Maps ProviderError codes to hard-trigger decisions.
|
||||
*
|
||||
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7):
|
||||
* - SPAWN_FAILED → hard trigger (provider CLI failed)
|
||||
* - CLI_NOT_FOUND → hard trigger (binary missing)
|
||||
* - AUTH_MISSING → NOT a hard trigger (user-config failure; user must fix)
|
||||
* - SPAWN_TIMEOUT → hard trigger (per ADR 0004 § Trigger taxonomy bullet 4)
|
||||
* v0.1 live codes (per ADR 0004 Amendment 3 (D34 F7) + Amendment 4 (D38)):
|
||||
* - SPAWN_FAILED → hard trigger (provider CLI failed)
|
||||
* - CLI_NOT_FOUND → hard trigger (binary missing)
|
||||
* - AUTH_MISSING → NOT a hard trigger (user-config failure; user must fix)
|
||||
* - SPAWN_TIMEOUT → hard trigger (per ADR 0004 § Trigger taxonomy bullet 4)
|
||||
* - CONCURRENCY_LIMIT → hard trigger (D38 / issue #1, ADR 0004 Amendment 4):
|
||||
* synthesized by server.mjs when a provider is at its
|
||||
* hints.maxConcurrent in-flight limit. The chain
|
||||
* advances immediately to the next hop instead of
|
||||
* queueing — design rationale per ADR 0004 Amendment 4.
|
||||
*
|
||||
* QUOTA_EXHAUSTED and RATE_LIMITED removed (D34 F7 / ADR 0004 Amendment 3):
|
||||
* no v0.1 plugin parses underlying-API HTTP status codes, so these codes
|
||||
@@ -44,8 +49,9 @@ import { computeIRRequestHash } from '../cache/keys.mjs';
|
||||
const HARD_TRIGGER_CODES = {
|
||||
SPAWN_FAILED: true,
|
||||
CLI_NOT_FOUND: true,
|
||||
AUTH_MISSING: false, // user config problem — never fall over (ADR 0004 § Decision)
|
||||
SPAWN_TIMEOUT: true, // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
|
||||
AUTH_MISSING: false, // user config problem — never fall over (ADR 0004 § Decision)
|
||||
SPAWN_TIMEOUT: true, // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
|
||||
CONCURRENCY_LIMIT: true, // ADR 0004 Amendment 4 (D38, issue #1): saturation → advance chain
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
+11
-3
@@ -141,8 +141,15 @@ export function validateProvider(p) {
|
||||
/**
|
||||
* Error codes surfaced by provider plugins.
|
||||
*
|
||||
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7):
|
||||
* SPAWN_FAILED, CLI_NOT_FOUND, AUTH_MISSING, SPAWN_TIMEOUT
|
||||
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7 and Amendment 4, D38):
|
||||
* SPAWN_FAILED, CLI_NOT_FOUND, AUTH_MISSING, SPAWN_TIMEOUT, CONCURRENCY_LIMIT
|
||||
*
|
||||
* CONCURRENCY_LIMIT (D38, issue #1): synthesized by the orchestration layer
|
||||
* (NOT thrown by provider plugins themselves) when a spawn is attempted
|
||||
* against a provider already at its hints.maxConcurrent limit. The fallback
|
||||
* engine treats this as a hard trigger so the chain advances to the next hop
|
||||
* rather than queueing. See ADR 0002 Amendment 6 (runtime enforcement) and
|
||||
* ADR 0004 Amendment 4 (CONCURRENCY_LIMIT in hard-trigger taxonomy).
|
||||
*
|
||||
* QUOTA_EXHAUSTED and RATE_LIMITED were removed (D34 F7): no plugin parses
|
||||
* underlying-API HTTP status codes at v0.1, so these codes are never emitted.
|
||||
@@ -153,7 +160,8 @@ export const PROVIDER_ERROR_CODES = /** @type {const} */ ([
|
||||
'AUTH_MISSING',
|
||||
'CLI_NOT_FOUND',
|
||||
'SPAWN_FAILED',
|
||||
'SPAWN_TIMEOUT', // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
|
||||
'SPAWN_TIMEOUT', // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
|
||||
'CONCURRENCY_LIMIT', // ADR 0002 Amendment 6 / ADR 0004 Amendment 4 (D38, issue #1)
|
||||
]);
|
||||
|
||||
export class ProviderError extends Error {
|
||||
|
||||
@@ -209,3 +209,133 @@ export function getProviderByName(loadedProviders, name) {
|
||||
export function listAllProviderNames() {
|
||||
return STATIC_REGISTRY.map(p => p.name);
|
||||
}
|
||||
|
||||
// ── Concurrency semaphore (D38, issue #1) ─────────────────────────────────
|
||||
//
|
||||
// Authority: ADR 0002 Amendment 6 (maxConcurrent runtime enforcement landed in D38)
|
||||
// and ADR 0004 Amendment 4 (CONCURRENCY_LIMIT added to hard-trigger taxonomy).
|
||||
//
|
||||
// Per-provider in-flight spawn counter. The orchestration layer (server.mjs
|
||||
// handleChatCompletions) calls tryAcquireSpawn() before provider.spawn() and
|
||||
// releaseSpawn() after spawn lifecycle completion. On saturation, the caller
|
||||
// synthesises a ProviderError(CONCURRENCY_LIMIT) which the fallback engine
|
||||
// treats as a hard trigger — the chain advances to the next hop. If the
|
||||
// entire chain is saturated, the user receives a chain-exhausted error via
|
||||
// the existing executeWithFallback exhaustion path.
|
||||
//
|
||||
// Design decision (deliberate): immediate-advancement via fallback, NOT
|
||||
// queue+timeout. Rationale (per D38 issue #1 design discussion):
|
||||
// 1. The fallback chain exists precisely for this kind of overflow.
|
||||
// 2. A queue introduces head-of-line blocking + a new timeout config surface.
|
||||
// 3. Immediate-advancement gives fail-fast latency, matching the OLP
|
||||
// multi-provider proxy philosophy.
|
||||
// 4. Queue+timeout is deferred — track via a future issue if real usage
|
||||
// shows need.
|
||||
//
|
||||
// **Atomicity invariant**: JavaScript is single-threaded; the
|
||||
// read-then-write pair inside tryAcquireSpawn() executes synchronously with
|
||||
// NO `await` between the check and the increment. This is the only reason
|
||||
// the semaphore is correct without a Mutex. A future async refactor MUST
|
||||
// preserve this — do NOT introduce an `await` between the limit check and
|
||||
// the count update or the semaphore loses its mutual-exclusion guarantee
|
||||
// (two callers could each read count=limit-1 before either increments).
|
||||
//
|
||||
// Module-level state: lives for the process lifetime; tests that need
|
||||
// isolation should call __resetSpawnCounters() in their teardown.
|
||||
//
|
||||
// @type {Map<string, number>} provider name → current in-flight spawn count
|
||||
const _activeSpawns = new Map();
|
||||
|
||||
/**
|
||||
* Default cap for tryAcquireSpawn when a plugin omits hints.maxConcurrent.
|
||||
*
|
||||
* validateProvider in base.mjs requires hints.maxConcurrent to be a
|
||||
* non-negative integer at startup, so a missing value should not happen in
|
||||
* production. This default is defense-in-depth for callers that pass a
|
||||
* stripped-down provider stub (e.g., in tests) or future plugin paths that
|
||||
* bypass validation. The value (4) matches the v0.1 plugin defaults
|
||||
* (anthropic / codex / mistral all declare hints.maxConcurrent: 4).
|
||||
*/
|
||||
export const DEFAULT_MAX_CONCURRENT_SPAWNS = 4;
|
||||
|
||||
/**
|
||||
* Atomically attempts to reserve a spawn slot for `providerName`.
|
||||
*
|
||||
* If the current in-flight count is below `maxConcurrent`, increments the
|
||||
* counter and returns true. Otherwise returns false WITHOUT incrementing —
|
||||
* the caller is responsible for surfacing the saturation as a
|
||||
* ProviderError(CONCURRENCY_LIMIT) for the fallback engine to consume.
|
||||
*
|
||||
* Atomicity: the check and the increment happen in a single synchronous
|
||||
* block with no `await` in between. See the module-level invariant comment
|
||||
* above for why this is sufficient.
|
||||
*
|
||||
* @param {string} providerName — provider key (e.g. 'anthropic')
|
||||
* @param {number} [maxConcurrent=DEFAULT_MAX_CONCURRENT_SPAWNS] — limit from hints.maxConcurrent
|
||||
* @returns {boolean} true if a slot was acquired, false if at limit
|
||||
*/
|
||||
export function tryAcquireSpawn(providerName, maxConcurrent = DEFAULT_MAX_CONCURRENT_SPAWNS) {
|
||||
// Defensive: coerce undefined/null/non-integer to the default. validateProvider
|
||||
// already enforces this at startup; this guards future plugin paths that
|
||||
// bypass validation.
|
||||
const limit = (typeof maxConcurrent === 'number' && Number.isInteger(maxConcurrent) && maxConcurrent >= 0)
|
||||
? maxConcurrent
|
||||
: DEFAULT_MAX_CONCURRENT_SPAWNS;
|
||||
|
||||
const current = _activeSpawns.get(providerName) ?? 0;
|
||||
// Atomic check-then-increment (no `await` between read and write).
|
||||
if (current >= limit) {
|
||||
return false;
|
||||
}
|
||||
_activeSpawns.set(providerName, current + 1);
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Releases a spawn slot for `providerName`. Must be called exactly once per
|
||||
* successful tryAcquireSpawn() call, regardless of whether the spawn succeeded
|
||||
* or threw. The caller in server.mjs uses a try/finally pattern to guarantee
|
||||
* the release fires on every exit path (success, error, abort, streaming end).
|
||||
*
|
||||
* Throws if the count would go negative — that indicates a bug (a release
|
||||
* without a matching acquire, or a double-release). The throw is loud on
|
||||
* purpose so the bug surfaces in tests rather than silently corrupting the
|
||||
* counter for future requests.
|
||||
*
|
||||
* @param {string} providerName — provider key (e.g. 'anthropic')
|
||||
* @throws {Error} if no slot is currently held for providerName
|
||||
*/
|
||||
export function releaseSpawn(providerName) {
|
||||
const current = _activeSpawns.get(providerName) ?? 0;
|
||||
if (current <= 0) {
|
||||
throw new Error(
|
||||
`releaseSpawn(${providerName}): counter would go negative — release without matching acquire (or double-release)`,
|
||||
);
|
||||
}
|
||||
const next = current - 1;
|
||||
if (next === 0) {
|
||||
_activeSpawns.delete(providerName);
|
||||
} else {
|
||||
_activeSpawns.set(providerName, next);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the current in-flight spawn count for `providerName`. Used by
|
||||
* /health, diagnostics, and tests that need to assert peak concurrency.
|
||||
*
|
||||
* @param {string} providerName
|
||||
* @returns {number} non-negative integer; 0 if no spawns in flight
|
||||
*/
|
||||
export function getActiveSpawnCount(providerName) {
|
||||
return _activeSpawns.get(providerName) ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* @internal — test seam: reset all in-flight spawn counters to zero. Used by
|
||||
* test teardown to ensure a clean state across suites. Production code MUST
|
||||
* NOT call this — it bypasses the acquire/release pairing invariant.
|
||||
*/
|
||||
export function __resetSpawnCounters() {
|
||||
_activeSpawns.clear();
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user