feat+ci: D38 — maxConcurrent runtime enforcement (issue #1)

ADR 0002 Amendment 1 declared `hints.maxConcurrent` declarative-only at
v0.1 — type-validated at startup but with no runtime enforcement.
D38 wires the runtime enforcement via a per-provider in-flight spawn
counter with immediate-advancement on saturation.

Design choice: **immediate-advancement via fallback engine** (queue +
timeout DEFERRED). When a provider is at its maxConcurrent limit, the
spawn call synchronously fails with a new `CONCURRENCY_LIMIT` error
code; the fallback engine treats this as a hard trigger and advances
to the next chain hop. If the entire chain is saturated, the user
sees a chain-exhausted error (existing path).

Rationale for immediate-advancement over queue+timeout:

1. The fallback chain exists precisely for this kind of overflow —
   adding a queue layer would duplicate the advancement semantics.
2. Queue + timeout adds new config surface (timeout duration, queue
   depth bounds, queue eviction policy) that isn't needed at the
   personal/family scale OLP serves.
3. Head-of-line blocking risk: a long-running spawn would stall
   queued requests behind it even though other providers in the chain
   could serve them immediately.
4. Fail-fast latency aligns with the multi-provider proxy philosophy
   ("spread risk across providers, not within a provider").

Queue + timeout is deferred to a v1.x design ADR if real usage shows
demand. ADR 0002 Amendment 6 and ADR 0004 Amendment 4 capture the
decision explicitly.

Changes (8 files, +<delta>):

**Code**

1. **lib/providers/base.mjs** — add `CONCURRENCY_LIMIT` to
   `PROVIDER_ERROR_CODES`. JSDoc clarifies the code is synthesised by
   the orchestration layer, not thrown by provider plugins themselves.

2. **lib/providers/index.mjs** — new semaphore primitives:
   - `tryAcquireSpawn(providerName, maxConcurrent)` — atomic
     check-then-increment. Returns `true` on success, `false` if at
     limit. Atomicity rests on the JS single-threaded invariant
     (read + write synchronous, NO `await` between them); module-level
     comment block warns future maintainers against breaking this.
   - `releaseSpawn(providerName)` — decrement; throws on
     under-decrement (defensive bug guard for missing acquire / double
     release). Map.delete at zero for clean memory footprint.
   - `getActiveSpawnCount(providerName)` — returns current count
     (0 for unseen providers). For diagnostics + tests.
   - `DEFAULT_MAX_CONCURRENT_SPAWNS = 4` — defense-in-depth fallback
     matching the v0.1 plugin defaults (anthropic/codex/mistral all
     declare hints.maxConcurrent: 4). Also coerces non-integer / NaN
     / negative inputs to the default.
   - `__resetSpawnCounters()` — internal test seam.

3. **lib/fallback/engine.mjs** — `CONCURRENCY_LIMIT: true` added to
   `HARD_TRIGGER_CODES`. `classifyTrigger` and `evaluateHardTriggers`
   both pick it up via the same lookup. v0.1 live hard-trigger codes
   are now 5 (was 4 post-D34): SPAWN_FAILED, CLI_NOT_FOUND,
   AUTH_MISSING:false, SPAWN_TIMEOUT, CONCURRENCY_LIMIT.

4. **server.mjs** — gate the spawn call in handleChatCompletions at
   BOTH spawn call sites:
   - **Buffered path** (executeHopFn → collectAllChunks): acquire
     before provider.spawn; on failure synthesise
     `ProviderError(CONCURRENCY_LIMIT)` with providerName /
     maxConcurrent / activeSpawns diagnostic fields and throw —
     fallback engine catches and advances. On success, outer try/
     finally wraps the inner D16 truncation-salvage try/catch so
     releaseSpawn fires on EVERY exit path (return, D16 salvage
     return, re-throw).
   - **Streaming path** (single-hop real-SSE branch): acquire BEFORE
     the streaming branch entry. If acquire fails, branch is skipped
     and request falls through to buffered path (whose own gate
     surfaces chain-exhausted for single-hop chains). If acquire
     succeeds, existing streaming try/catch gains
     `finally { releaseSpawn(streamProvider) }` — slot releases on
     stop-chunk completion, generator exhaustion, abort, or any
     exception path. `releaseSpawn` fires at END of stream
     consumption, not when spawn() returns.

**Tests** (test-features.mjs): 431 → 447 (+16):

Suite 18 — D38 — maxConcurrent runtime enforcement:
- 18a: PROVIDER_ERROR_CODES.CONCURRENCY_LIMIT exists
- 18b: evaluateHardTriggers true for CONCURRENCY_LIMIT
- 18c: AUTH_MISSING regression guard (D38 did NOT flip it to hard)
- 18d.1-18d.4: semaphore unit (increment / saturate-no-increment /
  release / map-delete-at-zero)
- 18e: tryAcquireSpawn returns false without side-effect when at limit
- 18f: releaseSpawn throws on under-decrement
- 18g.1-18g.2: DEFAULT_MAX_CONCURRENT_SPAWNS applied for invalid
  inputs (undefined / NaN / negative / non-integer)
- 18h: __resetSpawnCounters clears state
- 18i: HTTP integration — 5 concurrent requests to single-hop
  chain w/ maxConcurrent=2 → peak in-flight exactly 2, 2 succeed,
  3 fail
- 18j: counter releases after buffered request (sequential test)
- 18k: 2-hop chain — saturated primary advances to fallback
- 18l: streaming counter releases at END of stream (not at spawn)

**ADR amendments**

5. **docs/adr/0002-plugin-architecture.md** — Amendment 6 added.
   Removes "Declarative hint only at v0.1" caveat from the
   `maxConcurrent` description in the Provider contract section.
   Adds implementation reference + design-choice rationale (4 points).
   Lists all exported symbols.

6. **docs/adr/0004-fallback-engine.md** — Amendment 4 added.
   CONCURRENCY_LIMIT added to v0.1 hard-trigger taxonomy. Documents
   synthesis-vs-plugin-thrown distinction. Documents first-chunk
   safety (acquire before any res.write). v1.x re-evaluation triggers
   named.

**CHANGELOG**

7. **CHANGELOG.md** — Unreleased sentinel replaced with proper D38
   entry. Per CLAUDE.md release_kit phase_rolling_mode, this lands
   under Unreleased; promotion to ## v0.1.1 happens at the v0.1.1
   release. The D37 phase_rolling_mode gate now correctly fires if
   anyone tags v0.1.x with this content present without promotion —
   intentional.

Pre-commit fold-ins (per evidence-first checkpoint #4):

- **Reviewer Suggestion #1 (test 18c name mismatch)**: 18c is named
  "classifyTrigger returns hard for CONCURRENCY_LIMIT" but body tests
  AUTH_MISSING regression. Renamed header comment to
  "AUTH_MISSING regression guard" and clarified that
  CONCURRENCY_LIMIT classification is covered by 18b.

- **Reviewer Suggestion #3 (activeSpawns diagnostic field)**: server
  .mjs:532 set `concurrencyErr.activeSpawns = maxConcurrent` (the
  limit). Technically correct (since acquire just failed, live
  count == limit) but confuses future readers. Changed to query
  `getActiveSpawnCount(hopProvider)` directly. Added import of the
  new symbol to the lib/providers/index.mjs import block.

- **Reviewer Suggestion #5 (ADR overstatement)**: ADR 0002 Amendment 6
  said getActiveSpawnCount is "exported for /health, diagnostics,
  and tests" but /health integration is not wired at D38. Reworded to
  "exported for diagnostics and tests" + "/health integration
  deferred — when surfaced there will land at providers.status.<name>
  .activeSpawns; not wired at D38." Avoids overstating current state.

Two reviewer suggestions not folded:
- Suggestion #2 (streaming test 18l comment about branch entry
  conditions) — low priority; the test passes and the branch is
  taken (verified by reviewer). Future polish if confusion arises.
- Suggestion #4 (additional test for streaming-saturation → chain-
  exhausted) — code path is straightforward and intentional; 18i
  covers the buffered-path version directly. Defer.

Authority:
- ADR 0002 Amendment 1 (declarative-only caveat) — superseded by
  Amendment 6
- ADR 0002 Amendment 6 (this commit) — runtime enforcement landed
- ADR 0004 Amendment 4 (this commit) — CONCURRENCY_LIMIT added to
  v0.1 hard-trigger taxonomy
- GitHub issue #1 — closed by this commit
- CC 开发铁律 v1.6 § 10.x — independent fresh-context reviewer
- CLAUDE.md release_kit_overlay phase_rolling_mode — no version
  bump; Unreleased entry written

Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus,
independent of drafter): APPROVE. Critical depth checks:
- Atomicity in tryAcquireSpawn (lines 285-290 read+set with no
  intervening await) — confirmed; module-level invariant comment
  warns future maintainers
- Release on every exit path: buffered (outer finally wraps inner
  try/catch covering D16 salvage return + normal return + re-throw);
  streaming (try/finally covers stop-chunk return + loop exhaustion +
  catch paths). No double-release path identified.
- Streaming release timing: fires in finally after res.end() at all
  exit paths, never before stream consumption completes
- Counter leak: every code path traced — no orphan acquire identified
- 447/447 tests pass in reviewer's independent npm test run

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-24 21:02:40 +10:00
co-authored by Claude Opus 4.7
parent a718d22900
commit 994568a8fb
8 changed files with 797 additions and 47 deletions
+13 -7
View File
@@ -27,11 +27,16 @@ import { computeIRRequestHash } from '../cache/keys.mjs';
/**
* Maps ProviderError codes to hard-trigger decisions.
*
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7):
* - SPAWN_FAILED → hard trigger (provider CLI failed)
* - CLI_NOT_FOUND → hard trigger (binary missing)
* - AUTH_MISSING → NOT a hard trigger (user-config failure; user must fix)
* - SPAWN_TIMEOUT → hard trigger (per ADR 0004 § Trigger taxonomy bullet 4)
* v0.1 live codes (per ADR 0004 Amendment 3 (D34 F7) + Amendment 4 (D38)):
* - SPAWN_FAILED → hard trigger (provider CLI failed)
* - CLI_NOT_FOUND → hard trigger (binary missing)
* - AUTH_MISSING → NOT a hard trigger (user-config failure; user must fix)
* - SPAWN_TIMEOUT → hard trigger (per ADR 0004 § Trigger taxonomy bullet 4)
* - CONCURRENCY_LIMIT → hard trigger (D38 / issue #1, ADR 0004 Amendment 4):
* synthesized by server.mjs when a provider is at its
* hints.maxConcurrent in-flight limit. The chain
* advances immediately to the next hop instead of
* queueing — design rationale per ADR 0004 Amendment 4.
*
* QUOTA_EXHAUSTED and RATE_LIMITED removed (D34 F7 / ADR 0004 Amendment 3):
* no v0.1 plugin parses underlying-API HTTP status codes, so these codes
@@ -44,8 +49,9 @@ import { computeIRRequestHash } from '../cache/keys.mjs';
const HARD_TRIGGER_CODES = {
SPAWN_FAILED: true,
CLI_NOT_FOUND: true,
AUTH_MISSING: false, // user config problem — never fall over (ADR 0004 § Decision)
SPAWN_TIMEOUT: true, // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
AUTH_MISSING: false, // user config problem — never fall over (ADR 0004 § Decision)
SPAWN_TIMEOUT: true, // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
CONCURRENCY_LIMIT: true, // ADR 0004 Amendment 4 (D38, issue #1): saturation → advance chain
};
/**
+11 -3
View File
@@ -141,8 +141,15 @@ export function validateProvider(p) {
/**
* Error codes surfaced by provider plugins.
*
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7):
* SPAWN_FAILED, CLI_NOT_FOUND, AUTH_MISSING, SPAWN_TIMEOUT
* v0.1 live codes (per ADR 0004 Amendment 3, D34 F7 and Amendment 4, D38):
* SPAWN_FAILED, CLI_NOT_FOUND, AUTH_MISSING, SPAWN_TIMEOUT, CONCURRENCY_LIMIT
*
* CONCURRENCY_LIMIT (D38, issue #1): synthesized by the orchestration layer
* (NOT thrown by provider plugins themselves) when a spawn is attempted
* against a provider already at its hints.maxConcurrent limit. The fallback
* engine treats this as a hard trigger so the chain advances to the next hop
* rather than queueing. See ADR 0002 Amendment 6 (runtime enforcement) and
* ADR 0004 Amendment 4 (CONCURRENCY_LIMIT in hard-trigger taxonomy).
*
* QUOTA_EXHAUSTED and RATE_LIMITED were removed (D34 F7): no plugin parses
* underlying-API HTTP status codes at v0.1, so these codes are never emitted.
@@ -153,7 +160,8 @@ export const PROVIDER_ERROR_CODES = /** @type {const} */ ([
'AUTH_MISSING',
'CLI_NOT_FOUND',
'SPAWN_FAILED',
'SPAWN_TIMEOUT', // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
'SPAWN_TIMEOUT', // ADR 0004 § Trigger taxonomy bullet 4: spawn timeout is a hard trigger
'CONCURRENCY_LIMIT', // ADR 0002 Amendment 6 / ADR 0004 Amendment 4 (D38, issue #1)
]);
export class ProviderError extends Error {
+130
View File
@@ -209,3 +209,133 @@ export function getProviderByName(loadedProviders, name) {
export function listAllProviderNames() {
return STATIC_REGISTRY.map(p => p.name);
}
// ── Concurrency semaphore (D38, issue #1) ─────────────────────────────────
//
// Authority: ADR 0002 Amendment 6 (maxConcurrent runtime enforcement landed in D38)
// and ADR 0004 Amendment 4 (CONCURRENCY_LIMIT added to hard-trigger taxonomy).
//
// Per-provider in-flight spawn counter. The orchestration layer (server.mjs
// handleChatCompletions) calls tryAcquireSpawn() before provider.spawn() and
// releaseSpawn() after spawn lifecycle completion. On saturation, the caller
// synthesises a ProviderError(CONCURRENCY_LIMIT) which the fallback engine
// treats as a hard trigger — the chain advances to the next hop. If the
// entire chain is saturated, the user receives a chain-exhausted error via
// the existing executeWithFallback exhaustion path.
//
// Design decision (deliberate): immediate-advancement via fallback, NOT
// queue+timeout. Rationale (per D38 issue #1 design discussion):
// 1. The fallback chain exists precisely for this kind of overflow.
// 2. A queue introduces head-of-line blocking + a new timeout config surface.
// 3. Immediate-advancement gives fail-fast latency, matching the OLP
// multi-provider proxy philosophy.
// 4. Queue+timeout is deferred — track via a future issue if real usage
// shows need.
//
// **Atomicity invariant**: JavaScript is single-threaded; the
// read-then-write pair inside tryAcquireSpawn() executes synchronously with
// NO `await` between the check and the increment. This is the only reason
// the semaphore is correct without a Mutex. A future async refactor MUST
// preserve this — do NOT introduce an `await` between the limit check and
// the count update or the semaphore loses its mutual-exclusion guarantee
// (two callers could each read count=limit-1 before either increments).
//
// Module-level state: lives for the process lifetime; tests that need
// isolation should call __resetSpawnCounters() in their teardown.
//
// @type {Map<string, number>} provider name → current in-flight spawn count
const _activeSpawns = new Map();
/**
* Default cap for tryAcquireSpawn when a plugin omits hints.maxConcurrent.
*
* validateProvider in base.mjs requires hints.maxConcurrent to be a
* non-negative integer at startup, so a missing value should not happen in
* production. This default is defense-in-depth for callers that pass a
* stripped-down provider stub (e.g., in tests) or future plugin paths that
* bypass validation. The value (4) matches the v0.1 plugin defaults
* (anthropic / codex / mistral all declare hints.maxConcurrent: 4).
*/
export const DEFAULT_MAX_CONCURRENT_SPAWNS = 4;
/**
* Atomically attempts to reserve a spawn slot for `providerName`.
*
* If the current in-flight count is below `maxConcurrent`, increments the
* counter and returns true. Otherwise returns false WITHOUT incrementing —
* the caller is responsible for surfacing the saturation as a
* ProviderError(CONCURRENCY_LIMIT) for the fallback engine to consume.
*
* Atomicity: the check and the increment happen in a single synchronous
* block with no `await` in between. See the module-level invariant comment
* above for why this is sufficient.
*
* @param {string} providerName — provider key (e.g. 'anthropic')
* @param {number} [maxConcurrent=DEFAULT_MAX_CONCURRENT_SPAWNS] — limit from hints.maxConcurrent
* @returns {boolean} true if a slot was acquired, false if at limit
*/
export function tryAcquireSpawn(providerName, maxConcurrent = DEFAULT_MAX_CONCURRENT_SPAWNS) {
// Defensive: coerce undefined/null/non-integer to the default. validateProvider
// already enforces this at startup; this guards future plugin paths that
// bypass validation.
const limit = (typeof maxConcurrent === 'number' && Number.isInteger(maxConcurrent) && maxConcurrent >= 0)
? maxConcurrent
: DEFAULT_MAX_CONCURRENT_SPAWNS;
const current = _activeSpawns.get(providerName) ?? 0;
// Atomic check-then-increment (no `await` between read and write).
if (current >= limit) {
return false;
}
_activeSpawns.set(providerName, current + 1);
return true;
}
/**
* Releases a spawn slot for `providerName`. Must be called exactly once per
* successful tryAcquireSpawn() call, regardless of whether the spawn succeeded
* or threw. The caller in server.mjs uses a try/finally pattern to guarantee
* the release fires on every exit path (success, error, abort, streaming end).
*
* Throws if the count would go negative — that indicates a bug (a release
* without a matching acquire, or a double-release). The throw is loud on
* purpose so the bug surfaces in tests rather than silently corrupting the
* counter for future requests.
*
* @param {string} providerName — provider key (e.g. 'anthropic')
* @throws {Error} if no slot is currently held for providerName
*/
export function releaseSpawn(providerName) {
const current = _activeSpawns.get(providerName) ?? 0;
if (current <= 0) {
throw new Error(
`releaseSpawn(${providerName}): counter would go negative — release without matching acquire (or double-release)`,
);
}
const next = current - 1;
if (next === 0) {
_activeSpawns.delete(providerName);
} else {
_activeSpawns.set(providerName, next);
}
}
/**
* Returns the current in-flight spawn count for `providerName`. Used by
* /health, diagnostics, and tests that need to assert peak concurrency.
*
* @param {string} providerName
* @returns {number} non-negative integer; 0 if no spawns in flight
*/
export function getActiveSpawnCount(providerName) {
return _activeSpawns.get(providerName) ?? 0;
}
/**
* @internal — test seam: reset all in-flight spawn counters to zero. Used by
* test teardown to ensure a clean state across suites. Production code MUST
* NOT call this — it bypasses the acquire/release pairing invariant.
*/
export function __resetSpawnCounters() {
_activeSpawns.clear();
}