fix: honor runtime-lowered concurrency limit + cancel queued waiters on disconnect (#149)

* fix: honor runtime-lowered concurrency limit + cancel queued waiters on disconnect

Fixes three findings from an independent concurrency audit of the -p/stream-json
wait-queue (lib/tui/semaphore.mjs, reused by server.mjs as `claudeSemaphore`) and
its acquireClaudeSlot()/callClaudeTui() callers in server.mjs:

F1 (MEDIUM) — release() handed a freed slot straight to the next queued waiter
without re-checking `this.limit`, so a PATCH /settings maxConcurrent decrease was
silently ignored until every already-inflight task happened to finish on its own.
release() now only re-grants when post-decrement inflight is still under the
current limit, and a new setLimit() wakes queued waiters immediately when the
limit is raised instead of only on the next incidental release().

F2 (MEDIUM) — a request queued behind the concurrency limit had no link to its
HTTP connection, so a client that disconnected while still queued would still
get a claude process spawned for it once a slot freed — burning subscription
quota for a dead socket. acquire() now accepts an optional AbortSignal; server.mjs
derives one from the client's res "close" event (closeSignalFor) and passes it
into claudeSemaphore.acquire() / tuiSemaphore.acquire() while queued. On abort the
waiter is spliced out of the queue (not just flagged), so `queued` accounting
stays exact; the same "close" signal is wired into acquireClaudeSlot() (-p path,
non-streaming + streaming + singleflight-wrapped) and callClaudeTui() (TUI path).
If the response is already destroyed by the time we try to queue, we reject
immediately without ever entering the queue.

F8 (cosmetic) — acquireClaudeSlot() set `stats.queued = claudeSemaphore.queued + 1`
BEFORE calling acquire(), over-reporting /health's queued count by 1 whenever a
slot was granted immediately (the common, non-queued case). acquire() already
updates its internal queue synchronously before returning a Promise, so reading
claudeSemaphore.queued right AFTER calling it (instead of guessing "+1" before)
is exact. No /health field was added, removed, or renamed.

ALIGNMENT.md: this PR touches request-handler code (callClaude, callClaudeStreaming,
callClaudeTui, acquireClaudeSlot) but is local concurrency-control/queue-accounting
infrastructure with no cli.js wire analogue — it does not add, rename, or change any
endpoint, header, request field, or response field, and does not touch the /v1/messages
forwarding path or the OAuth bearer machinery (the two Class A surfaces this repo
governs). The /health response shape is unchanged (same field set, same nesting;
only the *value* of the pre-existing `stats.queued` field is corrected). Per
CLAUDE.md hard-requirement #1, a cli.js citation is therefore declared ABSENT:
there is no corresponding cli.js operation to cite because this is not a
cli.js-mirror (Class A) change and not a Class B endpoint-contract change either.

Tests: added 6 unit tests to test-features.mjs against the shared TuiSemaphore
(lowering the limit mid-load does not over-admit; raising the limit wakes queued
waiters up to the new headroom, FIFO; a queued waiter cancelled via AbortSignal
is spliced out and never later acquires; an already-aborted signal never touches
the queue; cancelling one of several queued waiters preserves FIFO for the rest).
238 pre-existing tests remain green; suite is now 244/244.

Verification: `node --check server.mjs && node --check lib/tui/semaphore.mjs && npm test` — 244 passed, 0 failed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: singleflight follower retry on leader disconnect + quiet disconnect handling (review M1/L1/L2)

Addresses the independent reviewer's APPROVE-WITH-CHANGES findings on PR #149:

M1 (MEDIUM, F2 regression) — when a singleflight LEADER disconnected while queued,
its RequestDisconnectedError rejected the SHARED promise, so live followers fell
into respondUpstreamError's generic branch and got a spurious 500 on a healthy
socket. Fix: keys.mjs singleflight() gains an optional follower-side `retryIf`
predicate. When a follower joins an existing flight and the shared promise rejects
with an error retryIf() accepts, the follower does NOT inherit the rejection — it
re-enters singleflight with its OWN fn (the map entry is guaranteed already deleted:
the delete-finally is attached upstream of the promise followers await), becoming
the new leader or joining a retrying sibling's fresh flight. The leader's own
rejection is never retried (it IS that client's disconnect). server.mjs passes
retryIf = (err) => err instanceof RequestDisconnectedError && !res.destroyed, so a
follower whose own client is also gone still propagates quietly. Callers without
retryIf keep byte-for-byte pre-existing share-everything semantics (pinned by the
existing failure-fan-out test).

L1 (LOW) — a disconnect-while-queued on the non-streaming paths was recorded as a
usage FAILURE row and logged as a [proxy] error: metric noise for a non-error.
Both non-streaming catch blocks now early-return on RequestDisconnectedError
without recordUsage(success:false) and without console.error — mirroring the
streaming path, which returns silently. The disconnect remains observable at info
level (concurrency_wait_cancelled, now also emitted with path:"tui" from
callClaudeTui for parity with acquireClaudeSlot's -p log).

L2 (LOW, test gap) — added a unit test for the abort-after-grant race: a waiter
granted its slot whose signal aborts afterward must see no rejection, no queue
corruption, and its slot released exactly once via the normal path (the semaphore
detaches the abort listener at grant; the onAbort idx===-1 guard is the in-dispatch
backstop).

Tests: +3 (2× M1 in the singleflight section, 1× L2 in the F2 section) — suite is
now 247/247 green.

ALIGNMENT.md: unchanged declaration — still local concurrency/dedup infrastructure
with no cli.js wire analogue; no endpoint, header, request field, or response field
added or changed; /health shape untouched. cli.js citation declared ABSENT per
CLAUDE.md hard-requirement #1 (not a Class A mirror change, not a Class B
contract change).

Verification: node --check server.mjs lib/tui/semaphore.mjs keys.mjs && npm test
— 247 passed, 0 failed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

keys.mjs

@@ -382,11 +382,25 @@ export function getCacheStats() {
 // Per ADR 0005 / spec D4: in-process scope only (single Node process per host).
 const inflightMap = new Map();
 
-export function singleflight(hash, fn) {
+// `retryIf` (optional, audit finding M1): a predicate applied on the FOLLOWER path only.
+// When a follower joins an existing flight and the shared promise rejects with an error for
+// which retryIf(err) is true (in practice: the LEADER's client disconnected while queued —
+// an error that is personal to the leader, not a verdict about the upstream), the follower
+// does NOT inherit that rejection. Instead it re-enters singleflight with its OWN fn: it
+// either becomes the new leader (the map entry is already deleted — see the finally below,
+// which runs before any follower's catch because it is attached upstream of the promise the
+// followers await) or joins a flight another retrying follower just created. The leader's
+// own rejection is never retried here — its error belongs to it (leader path returns the
+// bare promise). Callers that pass no retryIf get the exact pre-M1 share-everything behavior.
+export function singleflight(hash, fn, retryIf) {
   const existing = inflightMap.get(hash);
   if (existing) {
     existing.requesters++;
-    return existing.promise;
+    if (!retryIf) return existing.promise;
+    return existing.promise.catch((err) => {
+      if (!retryIf(err)) throw err;
+      return singleflight(hash, fn, retryIf);
+    });
   }
   // Wrap fn() in Promise.resolve().then() so synchronous throws don't escape.
   const promise = Promise.resolve().then(fn).finally(() => {

lib/tui/semaphore.mjs

@@ -20,6 +20,15 @@
 //
 // Pure + importable so test-features.mjs can assert the bound directly (no server boot).
 
+// Thrown by acquire() when the caller-supplied AbortSignal fires before a slot was granted
+// (audit finding F2 — a client that disconnects while queued must never receive a slot; the
+// queue entry is spliced out, not just flagged, so `queued` accounting stays exact). Distinct
+// `name` lets callers (server.mjs acquireClaudeSlot) tell "client went away" apart from
+// "queue is full" without string-matching the message.
+export class SemaphoreAbortError extends Error {
+  constructor(message) { super(message); this.name = "SemaphoreAbortError"; }
+}
+
 export class TuiSemaphore {
   // limit: max concurrent slots. maxQueue: max waiters before run() rejects with backpressure.
   constructor(limit, { maxQueue } = {}) {
@@ -34,9 +43,30 @@ export class TuiSemaphore {
   get inflight() { return this._inflight; }
   get queued() { return this._waiters.length; }
 
+  // Runtime-adjust the concurrency limit (audit finding F1 — a PATCH /settings maxConcurrent
+  // change must actually take effect, not just be ignored until every currently-inflight task
+  // happens to finish). Lowering the limit is handled lazily by release() (see below) — it
+  // simply stops re-granting until inflight drains under the new, lower limit. Raising the
+  // limit has immediate headroom, so we wake as many queued waiters as now fit.
+  setLimit(limit) {
+    this.limit = Math.max(1, parseInt(limit, 10) || 1);
+    while (this._inflight < this.limit && this._waiters.length > 0) {
+      const next = this._waiters.shift();
+      this._inflight++;
+      next();
+    }
+  }
+
   // Acquire a slot. Resolves once a slot is free (immediately if under the limit, otherwise
   // when an in-flight task releases). Rejects synchronously-ish if the wait queue is full.
-  acquire() {
+  // `signal` (optional AbortSignal, F2) lets the caller cancel a QUEUED wait — e.g. wired to
+  // a client's socket "close" event so a request that disconnects before a slot is granted
+  // is removed from the queue instead of eventually being handed a slot for a dead socket.
+  // If `signal` is already aborted, reject immediately without ever touching the queue.
+  acquire(signal) {
+    if (signal?.aborted) {
+      return Promise.reject(new SemaphoreAbortError("acquire aborted before requesting a slot"));
+    }
     if (this._inflight < this.limit) {
       this._inflight++;
       return Promise.resolve();
@@ -46,24 +76,44 @@ export class TuiSemaphore {
         `tui_queue_full: TUI concurrency limit (${this.limit}) reached and wait queue ` +
         `(${this.maxQueue}) is full`));
     }
-    return new Promise((resolve) => { this._waiters.push(resolve); });
+    return new Promise((resolve, reject) => {
+      let waiter; // the FIFO entry — captured so onAbort can find + splice exactly this one
+      const onAbort = () => {
+        const idx = this._waiters.indexOf(waiter);
+        if (idx === -1) return; // already granted a slot (shifted out by release()/setLimit) — too late to cancel
+        this._waiters.splice(idx, 1); // remove, not just flag — keeps `queued` accounting exact
+        reject(new SemaphoreAbortError("acquire aborted while queued"));
+      };
+      waiter = () => {
+        signal?.removeEventListener("abort", onAbort);
+        resolve();
+      };
+      signal?.addEventListener("abort", onAbort, { once: true });
+      this._waiters.push(waiter);
+    });
   }
 
-  // Release a slot. If a waiter is queued, hand the slot directly to it (inflight stays
+  // Release a slot. Always frees the caller's own slot first, then re-grants it to the next
-  // constant across the handoff); otherwise decrement.
+  // waiter ONLY if the (post-decrement) inflight count is still under the current limit (F1
+  // fix). This is what makes a runtime-lowered limit actually bite: if the limit was lowered
+  // while over-subscribed, releases stop re-granting and inflight drains toward the new limit
+  // instead of a freed slot being handed straight back out at the old, higher occupancy.
   release() {
-    const next = this._waiters.shift();
+    if (this._inflight > 0) this._inflight--;
-    if (next) {
+    if (this._inflight < this.limit) {
-      next(); // the woken waiter already "owns" the slot — inflight unchanged
+      const next = this._waiters.shift();
-    } else if (this._inflight > 0) {
+      if (next) {
-      this._inflight--;
+        this._inflight++;
+        next();
+      }
     }
   }
 
   // Run fn() under one slot. Releases in a finally so a throw (PR-A's honesty gates,
   // wallclock truncation, paste-not-landed, tmux spawn failure) NEVER leaks a slot.
-  async run(fn) {
+  // `signal` (optional, F2) is forwarded to acquire() so a queued run() can be cancelled.
-    await this.acquire();
+  async run(fn, signal) {
+    await this.acquire(signal);
     try {
       return await fn();
     } finally {

server.mjs

@@ -43,7 +43,7 @@ import { DEFAULT_PORT } from "./lib/constants.mjs";
 import { isLoopbackBind } from "./lib/net.mjs";
 import { runTuiTurn, reapStaleTuiSessions, resolveTuiHome } from "./lib/tui/session.mjs";
 import { detectTuiUpstreamError } from "./lib/tui/transcript.mjs";
-import { TuiSemaphore, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
+import { TuiSemaphore, SemaphoreAbortError, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
 import { createSerialMutex, createTtlCache, isTokenExpiring, orderLabelsLastGoodFirst } from "./lib/spawn-auth.mjs";
 
 const __dirname = dirname(fileURLToPath(import.meta.url));
@@ -507,16 +507,61 @@ class ConcurrencyOverflowError extends Error {
   constructor(message) { super(message); this.name = "ConcurrencyOverflowError"; this.httpStatus = 429; this.retryAfter = CLAUDE_QUEUE_RETRY_AFTER; }
 }
 
+// Tagged error for audit finding F2: the client disconnected while queued (or was already gone
+// before we even tried to queue it). Distinct from ConcurrencyOverflowError so callers never send
+// a response on this path — there is no socket left to write to.
+class RequestDisconnectedError extends Error {
+  constructor(message) { super(message); this.name = "RequestDisconnectedError"; }
+}
+
+// Build an AbortSignal that fires when `res` (an http.ServerResponse) closes — i.e. the client
+// disconnected. Used to cancel a QUEUED concurrency-slot wait (F2) so a client that gives up
+// before a slot is granted is spliced out of the wait queue instead of eventually spawning a
+// claude process for a dead socket. If `res` has already closed by the time we get here (its
+// underlying stream already torn down), the signal is returned pre-aborted so acquire() rejects
+// immediately without ever touching the queue — the "close already fired before we attach" case.
+// `detach()` MUST be called once the wait settles (granted or rejected) to avoid a listener leak.
+function closeSignalFor(res) {
+  const controller = new AbortController();
+  if (!res || typeof res.on !== "function") return { signal: controller.signal, detach() {} };
+  if (res.destroyed) {
+    controller.abort();
+    return { signal: controller.signal, detach() {} };
+  }
+  const onClose = () => controller.abort();
+  res.on("close", onClose);
+  return { signal: controller.signal, detach() { res.removeListener("close", onClose); } };
+}
+
 // Acquire a -p concurrency slot, queuing if all are busy (up to CLAUDE_MAX_QUEUE). Resolves to a
 // release() fn that MUST be called exactly once on every exit path (wired into ctx.cleanup()).
-// Rejects with ConcurrencyOverflowError when the wait-queue is full. Increments stats.queued while
+// Rejects with ConcurrencyOverflowError when the wait-queue is full, or with
-// waiting (decremented on acquire) and stats.queueRejections on overflow.
+// RequestDisconnectedError when `res` closes before a slot is granted (F2) — the caller must not
-async function acquireClaudeSlot() {
+// spawn claude in that case. `res` is optional (back-compat for any caller without a live response
-  stats.queued = claudeSemaphore.queued + 1; // reflect this waiter before we (maybe) block
+// object); omitting it just means a queued wait can't be cancelled early.
+//
+// F8 fix: stats.queued is set from claudeSemaphore.queued AFTER calling acquire() (not before) —
+// acquire() synchronously updates _inflight/_waiters before its Promise ever resolves, so reading
+// .queued right after the call already reflects reality. The old code set `queued + 1` BEFORE
+// calling acquire() to account for "this waiter", which over-reported by 1 whenever the slot was
+// granted immediately (the common case, not a queue at all).
+async function acquireClaudeSlot(res) {
+  const { signal, detach } = closeSignalFor(res);
+  const slot = claudeSemaphore.acquire(signal);
+  stats.queued = claudeSemaphore.queued; // accurate: acquire() already updated the queue synchronously
   try {
-    await claudeSemaphore.acquire();
+    await slot;
   } catch (e) {
+    detach();
     stats.queued = claudeSemaphore.queued;
+    if (e instanceof SemaphoreAbortError) {
+      // Client-driven cancellation, not backpressure — do NOT count it as a queueRejection or
+      // log it as concurrency_queue_full (that log/counter means "the queue itself is full").
+      logEvent("info", "concurrency_wait_cancelled", {
+        reason: "client_disconnected", inflight: claudeSemaphore.inflight, queued: claudeSemaphore.queued,
+      });
+      throw new RequestDisconnectedError("client disconnected while waiting for a concurrency slot");
+    }
     stats.queueRejections++;
     logEvent("warn", "concurrency_queue_full", {
       limit: claudeSemaphore.limit, maxQueue: claudeSemaphore.maxQueue,
@@ -526,6 +571,7 @@ async function acquireClaudeSlot() {
       `backpressure: concurrency limit (${claudeSemaphore.limit}) reached and wait queue ` +
       `(${claudeSemaphore.maxQueue}) is full — retry shortly`);
   }
+  detach();
   stats.queued = claudeSemaphore.queued;
   let released = false;
   return function releaseClaudeSlot() {
@@ -1135,14 +1181,29 @@ function spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlo
 // We accumulate full text across all content_block_delta events plus the
 // assistant-aggregate fallback, then resolve with the assembled string.
 // Reference: OLP ADR 0009 Amendment 1 + commit 97e7d16.
-async function callClaude(model, messages, conversationId, keyName) {
+// `res` (optional, F2) is the client's http.ServerResponse — passed through so a queued wait
+// can be cancelled the moment the client disconnects, instead of spawning claude for a dead
+// socket once a slot finally frees up.
+async function callClaude(model, messages, conversationId, keyName, res) {
   // FIX ⑥: acquire a concurrency slot first (queues up to CLAUDE_MAX_QUEUE; rejects with a
-  // ConcurrencyOverflowError → 429 when the queue is full). The release fn is passed into the
+  // ConcurrencyOverflowError → 429 when the queue is full, or a RequestDisconnectedError (F2)
-  // spawn so the idempotent cleanup() frees it on every exit path. If the spawn itself throws
+  // if the client goes away first). The release fn is passed into the spawn so the idempotent
-  // synchronously (before cleanup is wired), release here so the slot never leaks.
+  // cleanup() frees it on every exit path. If the spawn itself throws synchronously (before
-  const releaseSlot = await acquireClaudeSlot();
+  // cleanup is wired), release here so the slot never leaks.
-  // F3: resolve the per-spawn HOME/token decision (may serialize on the real-HOME fallback mutex).
+  // F2×F3 composition: the slot acquire comes FIRST and is the cancellable step — a client
-  const spawnDecision = await resolveSpawnDecision();
+  // that disconnects while queued rejects here, BEFORE resolveSpawnDecision() runs, so a
+  // cancelled request can never acquire (or briefly hold) the real-HOME fallback mutex.
+  const releaseSlot = await acquireClaudeSlot(res);
+  // F3: resolve the per-spawn HOME/token decision (may serialize on the real-HOME fallback
+  // mutex). If it throws, release the just-acquired slot before propagating — cleanup() is
+  // not wired yet at this point.
+  let spawnDecision;
+  try {
+    spawnDecision = await resolveSpawnDecision();
+  } catch (err) {
+    releaseSlot();
+    throw err;
+  }
   return new Promise((resolve, reject) => {
     let ctx;
     try {
@@ -1221,28 +1282,50 @@ async function callClaude(model, messages, conversationId, keyName) {
 // flag that could perturb cc_entrypoint classification.
 // Authority: claude CLI v2.1.158 interactive mode (cc_entrypoint=cli).
 // SECURITY: A-path single-user ONLY — home is NOT isolation (see ADR 0007).
-function callClaudeTui(model, messages, _conversationId, _keyName) {
+// `res` (optional, F2) is the client's http.ServerResponse — see closeSignalFor.
+async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
   const cliModel = MODEL_MAP[model] || model;
   const prompt = messagesToPrompt(messages); // includes system as [System] inline
   recordModelRequest(cliModel, prompt.length);
-  // C-4: gate the heavy interactive boot behind the TUI semaphore. run() acquires a slot
+  // C-4: gate the heavy interactive boot behind the TUI semaphore (queuing if all slots are
-  // (queuing if all are busy, up to maxQueue), then releases in a finally so any throw from
+  // busy, up to maxQueue). F2: `signal` (tied to `res` "close") cancels a QUEUED wait the
-  // runTuiTurn (tmux spawn failure, paste-not-landed) OR from the honesty gates below
+  // instant the client disconnects, so a dead socket never triggers a cold-boot tmux+claude
-  // (truncation / error banner) can NEVER leak a slot. tuiSemaphore.inflight feeds /health.
+  // spawn; detach() drops the "close" listener as soon as the wait settles rather than
-  return tuiSemaphore.run(() => runTuiTurn({
+  // holding it for the whole (up to 120s) turn.
-    prompt,
+  const { signal, detach } = closeSignalFor(res);
-    model: cliModel,
+  try {
-    claudeBin: CLAUDE,
+    await tuiSemaphore.acquire(signal);
-    home: TUI_HOME,
+  } catch (err) {
-    realHome: process.env.HOME,
+    detach();
-    cwd: TUI_CWD,
+    if (err instanceof SemaphoreAbortError) {
-    port: PORT, // F7 fix: port-scopes the tmux session name so a sibling OCP instance on a
+      // L1: client-driven cancellation, not an upstream failure — info, not error (mirrors
-                // different port never collides with this instance's reap/kill-server logic.
+      // acquireClaudeSlot's concurrency_wait_cancelled on the -p path).
-    wallclockMs: TUI_WALLCLOCK_MS,
+      logEvent("info", "concurrency_wait_cancelled", {
-    entrypointMode: TUI_ENTRYPOINT,
+        reason: "client_disconnected", path: "tui", inflight: tuiSemaphore.inflight, queued: tuiSemaphore.queued,
-  }).then(({ text, entrypoint, truncated }) => {
+      });
+      throw new RequestDisconnectedError("client disconnected while waiting for a TUI concurrency slot");
+    }
+    throw err;
+  }
+  detach();
+  // release() runs in a finally so any throw from runTuiTurn (tmux spawn failure,
+  // paste-not-landed) OR from the honesty gates below (truncation / error banner) can NEVER
+  // leak a slot. tuiSemaphore.inflight feeds /health.
+  try {
+    const { text, entrypoint, truncated } = await runTuiTurn({
+      prompt,
+      model: cliModel,
+      claudeBin: CLAUDE,
+      home: TUI_HOME,
+      realHome: process.env.HOME,
+      cwd: TUI_CWD,
+      port: PORT, // F7 fix: port-scopes the tmux session name so a sibling OCP instance on a
+                  // different port never collides with this instance's reap/kill-server logic.
+      wallclockMs: TUI_WALLCLOCK_MS,
+      entrypointMode: TUI_ENTRYPOINT,
+    });
     // ── Honesty gates (issue #133) ─ run BEFORE recordModelSuccess / cache write-back.
-    // A throw here propagates to the .catch below (recordModelError + reject), so the
+    // A throw here propagates to the catch below (recordModelError + reject), so the
     // result never reaches the downstream setCachedResponse / singleflight / SUCCESS path.
 
     // C-2: the wall-clock cap hit with partial text and NO terminal marker — the turn
@@ -1277,10 +1360,12 @@ function callClaudeTui(model, messages, _conversationId, _keyName) {
       logEvent("warn", "tui_entrypoint_mismatch", { expected: "cli", got: entrypoint, model: cliModel });
     }
     return text;
-  }).catch((err) => {
+  } catch (err) {
     recordModelError(cliModel, false);
     throw err;
-  }));
+  } finally {
+    tuiSemaphore.release();
+  }
 }
 
 // ── SSE heartbeat (opt-in idle watchdog) ────────────────────────────────
@@ -1326,18 +1411,30 @@ async function callClaudeStreaming(model, messages, conversationId, res, authInf
   // FIX ⑥: acquire a concurrency slot first (queues up to CLAUDE_MAX_QUEUE). On overflow, surface
   // HTTP 429 + Retry-After (NOT 500). Release is wired into cleanup() for every exit path; if the
   // spawn throws synchronously before cleanup is wired, release here.
+  // F2: pass `res` so a queued wait is cancelled the instant this client disconnects — the client
+  // is already gone in that case, so there is no response to send back.
   let releaseSlot;
   try {
-    releaseSlot = await acquireClaudeSlot();
+    releaseSlot = await acquireClaudeSlot(res);
   } catch (err) {
+    if (err instanceof RequestDisconnectedError) return; // client gone — nothing to write to
     if (err instanceof ConcurrencyOverflowError) {
       return jsonResponse(res, 429, { error: { message: sanitizeError(err.message), type: "rate_limit_error" } }, { "Retry-After": String(err.retryAfter) });
     }
     return jsonResponse(res, 500, { error: { message: sanitizeError(err.message), type: "proxy_error" } });
   }
 
-  // F3: resolve the per-spawn HOME/token decision (may serialize on the real-HOME fallback mutex).
+  // F3: resolve the per-spawn HOME/token decision (may serialize on the real-HOME fallback
-  const spawnDecision = await resolveSpawnDecision();
+  // mutex). F2×F3 composition: this runs strictly AFTER the (cancellable) slot acquire, so a
+  // request cancelled while queued never touches the fallback mutex. If it throws, release
+  // the just-acquired slot before responding — cleanup() is not wired yet at this point.
+  let spawnDecision;
+  try {
+    spawnDecision = await resolveSpawnDecision();
+  } catch (err) {
+    releaseSlot();
+    return jsonResponse(res, 500, { error: { message: sanitizeError(err.message), type: "proxy_error" } });
+  }
   let ctx;
   try {
     ctx = spawnClaudeProcess(model, messages, conversationId, authInfo.keyName, releaseSlot, spawnDecision);
@@ -2009,9 +2106,12 @@ function applySettingUpdate(key, value) {
 
   switch (key) {
     case "timeout":          TIMEOUT = value; break;
-    // FIX ⑥: keep the -p wait-queue semaphore's limit in sync with the runtime MAX_CONCURRENT
+    // FIX ⑥ + F1: keep the -p wait-queue semaphore's limit in sync with the runtime MAX_CONCURRENT
-    // so a /settings change to maxConcurrent actually changes how many claude procs run at once.
+    // so a /settings change to maxConcurrent actually changes how many claude procs run at once —
-    case "maxConcurrent":    MAX_CONCURRENT = value; claudeSemaphore.limit = Math.max(1, value); break;
+    // in BOTH directions. setLimit() (not a bare `.limit =` assignment) is required: lowering
+    // needs release() to stop over-granting until inflight drains under the new cap, and raising
+    // needs queued waiters woken immediately to use the new headroom. See lib/tui/semaphore.mjs.
+    case "maxConcurrent":    MAX_CONCURRENT = value; claudeSemaphore.setLimit(value); break;
     case "sessionTTL":       SESSION_TTL = value; break;
     case "maxPromptChars":   MAX_PROMPT_CHARS = value; break;
     case "cacheTTL":         CACHE_TTL = value; break;
@@ -2168,7 +2268,7 @@ async function handleChatCompletions(req, res) {
       const t0TuiStream = Date.now();
       const promptCharsTuiStream = messages.reduce((a, m) => a + contentToText(m.content).length, 0);
       try {
-        const content = await callClaudeTui(model, messages, conversationId, req._authKeyName);
+        const content = await callClaudeTui(model, messages, conversationId, req._authKeyName, res);
         if (CACHE_TTL > 0 && req._cacheHash) {
           try { setCachedResponse(req._cacheHash, model, content); } catch (e) { logEvent("error", "cache_write_failed", { error: e.message }); }
         }
@@ -2205,15 +2305,27 @@ async function handleChatCompletions(req, res) {
         // will re-read the freshly-populated cache entry here rather than spawning.
         const recheck = getCachedResponse(req._cacheHash, CACHE_TTL);
         if (recheck) return recheck.response;
-        const c = await upstreamCall(model, messages, conversationId, req._authKeyName);
+        const c = await upstreamCall(model, messages, conversationId, req._authKeyName, res);
         try { setCachedResponse(req._cacheHash, model, c); } catch (e) { logEvent("error", "cache_write_failed", { error: e.message }); }
         return c;
-      });
+      },
+      // M1: if the LEADER disconnected while queued (F2), its RequestDisconnectedError is
+      // personal to the leader — a live follower must not inherit it as a spurious 500.
+      // retryIf makes this follower re-enter singleflight with its OWN fn (own res, own
+      // disconnect signal), becoming the new leader or joining a retrying sibling's flight —
+      // but only while OUR client is still connected. If our client is also gone, the
+      // rejection propagates and the RDE early-return in the catch below ends it quietly.
+      (err) => err instanceof RequestDisconnectedError && !res.destroyed);
       const id = `chatcmpl-${randomUUID()}`;
       completionResponse(res, id, model, content);
       try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: content.length, elapsedMs: Date.now() - t0Usage, success: true }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
       return;
     } catch (err) {
+      // L1: a client disconnect while queued is NOT an upstream failure — mirror the
+      // streaming path (which returns without recording anything): no usage-failure row,
+      // no [proxy] error log, no error response (the socket is gone). The disconnect is
+      // already logged at info level (concurrency_wait_cancelled) by acquireClaudeSlot.
+      if (err instanceof RequestDisconnectedError) { try { res.end(); } catch {} return; }
       try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: 0, elapsedMs: Date.now() - t0Usage, success: false }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
       console.error(`[proxy] error: ${err.message}`);
       if (res.headersSent || res.writableEnded || res.destroyed) {
@@ -2226,11 +2338,14 @@ async function handleChatCompletions(req, res) {
 
   // Fallback: cache disabled (CACHE_TTL=0) or no _cacheHash — original path untouched.
   try {
-    const content = await upstreamCall(model, messages, conversationId, req._authKeyName);
+    const content = await upstreamCall(model, messages, conversationId, req._authKeyName, res);
     const id = `chatcmpl-${randomUUID()}`;
     completionResponse(res, id, model, content);
     try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: content.length, elapsedMs: Date.now() - t0Usage, success: true }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
   } catch (err) {
+    // L1: disconnect-while-queued — same quiet non-error outcome as the singleflight
+    // path above and the streaming path (see acquireClaudeSlot's info-level log).
+    if (err instanceof RequestDisconnectedError) { try { res.end(); } catch {} return; }
     try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: 0, elapsedMs: Date.now() - t0Usage, success: false }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
     console.error(`[proxy] error: ${err.message}`);
     if (res.headersSent || res.writableEnded || res.destroyed) {

test-features.mjs

@@ -463,6 +463,52 @@ async function runSingleflightTests() {
     assert.equal(r1, 1);
     assert.equal(r2, 2);
   });
+
+  // 7. M1: leader disconnect while queued must not poison live followers. server.mjs passes
+  // retryIf = (err) => err instanceof RequestDisconnectedError && !res.destroyed — here we
+  // model that with a tagged error class. The leader (no retryIf on its own promise — the
+  // rejection is ITS OWN disconnect) sees the error; the live follower re-executes its OWN
+  // fn and gets a real result instead of a spurious inherited failure.
+  await asyncTest("M1: leader disconnects while queued → live follower re-executes and gets a real result", async () => {
+    class FakeDisconnectError extends Error {}
+    const leaderGate = Promise.withResolvers();
+    let leaderRuns = 0;
+    let followerRuns = 0;
+    const leaderFn = async () => { leaderRuns++; await leaderGate.promise; throw new FakeDisconnectError("leader client gone"); };
+    const followerFn = async () => { followerRuns++; return "real-execution"; };
+    const retryIf = (err) => err instanceof FakeDisconnectError;
+
+    const leaderP = singleflight("sf-m1-leader-dc", leaderFn);             // becomes leader
+    const followerP = singleflight("sf-m1-leader-dc", followerFn, retryIf); // joins as follower
+    leaderGate.resolve(); // leader "disconnects" while holding the flight
+
+    await assert.rejects(leaderP, FakeDisconnectError, "the leader itself still sees its own disconnect");
+    assert.equal(await followerP, "real-execution", "follower got a REAL execution, not the leader's disconnect");
+    assert.equal(leaderRuns, 1, "leader fn ran once");
+    assert.equal(followerRuns, 1, "follower re-executed exactly once (as the new leader)");
+    assert.equal(getInflightStats().inflight, 0, "map fully cleaned up after the retry flight settles");
+  });
+
+  // 8. M1 guard: a follower whose retryIf returns false (server.mjs: its OWN client is also
+  // gone) inherits the rejection unchanged — no retry, no masked error. And a follower with
+  // NO retryIf keeps the exact pre-M1 share-everything behavior (test 2 pins the fan-out;
+  // this pins the predicate=false path specifically for the disconnect error).
+  await asyncTest("M1: follower with retryIf=false (own client also gone) inherits the leader's rejection, no retry", async () => {
+    class FakeDisconnectError extends Error {}
+    const gate = Promise.withResolvers();
+    let followerRuns = 0;
+    const leaderFn = async () => { await gate.promise; throw new FakeDisconnectError("leader client gone"); };
+    const followerFn = async () => { followerRuns++; return "should-never-run"; };
+
+    const leaderP = singleflight("sf-m1-both-dc", leaderFn);
+    const followerP = singleflight("sf-m1-both-dc", followerFn, () => false); // own client dead → no retry
+    gate.resolve();
+
+    await assert.rejects(leaderP, FakeDisconnectError);
+    await assert.rejects(followerP, FakeDisconnectError, "rejection propagates unchanged when retryIf says no");
+    assert.equal(followerRuns, 0, "follower fn never executed — no wasted spawn for a dead client");
+    assert.equal(getInflightStats().inflight, 0);
+  });
 }
 
 await runSingleflightTests();
@@ -2005,7 +2051,7 @@ test("resolveTuiHome: explicit OCP_TUI_HOME wins regardless of env token (back-c
 });
 
 // ── TUI concurrency limiter + drift observability (PR-B: audit C-4 / C-5) ──
-import { TuiSemaphore, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
+import { TuiSemaphore, SemaphoreAbortError, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
 
 console.log("\nTUI concurrency limiter (C-4):");
 
@@ -2110,6 +2156,226 @@ await asyncTest("FIX ⑥: slot released on normal completion is immediately reus
   assert.equal(sem.inflight, 0);
 });
 
+// ── Audit F1 — runtime-lowered/raised limit must actually bite ──────────────
+// server.mjs reuses this same TuiSemaphore as `claudeSemaphore`; a PATCH /settings
+// maxConcurrent update now calls `claudeSemaphore.setLimit(value)` (see applySettingUpdate's
+// "maxConcurrent" case). These tests pin the semaphore-level contract that fix depends on.
+console.log("\nF1 — runtime concurrency-limit changes (setLimit / release honoring the current limit):");
+
+await asyncTest("F1: lowering the limit mid-load — release() stops re-granting until inflight drains under the new limit", async () => {
+  const sem = new TuiSemaphore(3, { maxQueue: 16 });
+  const g = [deferred(), deferred(), deferred()];
+  const held = g.map((d) => sem.run(async () => { await d.p; }));
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.inflight, 3, "3 tasks hold the 3 slots");
+  // A 4th arrives while at capacity — it queues.
+  const g4 = deferred();
+  const queued4 = sem.run(async () => { await g4.p; });
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.queued, 1, "4th request queued");
+
+  // Operator lowers maxConcurrent from 3 to 1 while all 3 original slots are still inflight
+  // (mirrors a PATCH /settings maxConcurrent=1 hitting server.mjs mid-burst).
+  sem.setLimit(1);
+  assert.equal(sem.limit, 1);
+
+  // Releasing one of the 3 original holders must NOT hand the freed slot to the queued 4th
+  // request — before the F1 fix, release() handed slots off unconditionally, so inflight
+  // would have stayed pinned at the OLD higher occupancy forever.
+  g[0].resolve();
+  await held[0];
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.inflight, 2, "inflight drains toward the new limit, not re-granted");
+  assert.equal(sem.queued, 1, "4th request is STILL queued — not over-admitted");
+
+  g[1].resolve();
+  await held[1];
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.inflight, 1, "inflight now exactly at the new limit (1)");
+  assert.equal(sem.queued, 1, "still queued — inflight(1) is not < limit(1), so no grant yet");
+
+  // Releasing the LAST original holder finally drops inflight under the new limit — only
+  // now does the queued 4th request get granted.
+  g[2].resolve();
+  await held[2];
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.inflight, 1, "queued 4th request now holds the single slot");
+  assert.equal(sem.queued, 0, "queue drained");
+  g4.resolve();
+  await queued4;
+  assert.equal(sem.inflight, 0);
+});
+
+await asyncTest("F1: raising the limit wakes queued waiters immediately, up to the new headroom", async () => {
+  const sem = new TuiSemaphore(1, { maxQueue: 16 });
+  const g1 = deferred();
+  const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
+  await new Promise((r) => setImmediate(r));
+  const started = [];
+  const g2 = deferred(), g3 = deferred();
+  const t2 = sem.run(async () => { started.push(2); await g2.p; });
+  const t3 = sem.run(async () => { started.push(3); await g3.p; });
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.queued, 2, "both queue behind the single holder");
+  assert.deepEqual(started, [], "neither queued task has started");
+
+  // Operator raises maxConcurrent from 1 to 3 (2 units of new headroom) — BOTH queued
+  // waiters must be woken immediately, without waiting for t1 to release.
+  sem.setLimit(3);
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.inflight, 3, "t1 + both newly-woken waiters now hold slots");
+  assert.equal(sem.queued, 0, "queue drained by the limit raise");
+  assert.deepEqual(started.sort(), [2, 3], "both queued tasks started without waiting for t1's release");
+
+  g1.resolve(); g2.resolve(); g3.resolve();
+  await Promise.all([t1, t2, t3]);
+  assert.equal(sem.inflight, 0);
+});
+
+await asyncTest("F1: raising the limit wakes only as many waiters as the new headroom allows (FIFO)", async () => {
+  const sem = new TuiSemaphore(1, { maxQueue: 16 });
+  const g1 = deferred();
+  const t1 = sem.run(async () => { await g1.p; });
+  await new Promise((r) => setImmediate(r));
+  const started = [];
+  const g2 = deferred(), g3 = deferred();
+  const t2 = sem.run(async () => { started.push(2); await g2.p; });
+  const t3 = sem.run(async () => { started.push(3); await g3.p; });
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.queued, 2);
+
+  sem.setLimit(2); // only 1 unit of new headroom (1 -> 2) — exactly one queued waiter wakes
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.inflight, 2);
+  assert.equal(sem.queued, 1, "one waiter still queued — only one slot of headroom existed");
+  assert.deepEqual(started, [2], "FIFO: the earlier-queued waiter (t2) wakes, not t3");
+
+  // Freeing t1's slot afterward still honors the (now current) limit of 2 via release()'s
+  // normal path — the still-queued t3 gets in once a slot actually frees.
+  g1.resolve();
+  await t1;
+  await new Promise((r) => setImmediate(r));
+  assert.deepEqual(started, [2, 3], "t3 granted once a slot frees, honoring the raised limit");
+  assert.equal(sem.queued, 0);
+
+  g2.resolve(); g3.resolve();
+  await t2; await t3;
+  assert.equal(sem.inflight, 0);
+});
+
+// ── Audit F2 — queued waiters must be cancellable on client disconnect ──────
+// server.mjs wires an AbortSignal derived from the client's res "close" event into
+// claudeSemaphore.acquire()/tuiSemaphore.acquire() (see closeSignalFor + acquireClaudeSlot /
+// callClaudeTui). These tests pin the semaphore-level cancellation contract that depends on.
+console.log("\nF2 — queued-wait cancellation via AbortSignal (client disconnect while queued):");
+
+await asyncTest("F2: aborting a QUEUED waiter rejects with SemaphoreAbortError and SPLICES it out (queued drops immediately, not just flagged)", async () => {
+  const sem = new TuiSemaphore(1, { maxQueue: 16 });
+  const g1 = deferred();
+  const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
+  await new Promise((r) => setImmediate(r));
+  const controller = new AbortController();
+  const acquire2 = sem.acquire(controller.signal); // queues behind t1
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.queued, 1, "second acquire queued");
+
+  controller.abort(); // simulates the client disconnecting while still queued
+  await assert.rejects(acquire2, SemaphoreAbortError, "cancelled waiter rejects with SemaphoreAbortError");
+  assert.equal(sem.queued, 0, "cancelled waiter is REMOVED — queue length drops immediately");
+  assert.equal(sem.inflight, 1, "t1's slot is untouched by the cancellation");
+
+  // Prove the cancelled waiter never later acquires a slot: free t1's slot and confirm
+  // nobody is waiting to receive it (the queue is genuinely empty, not just decremented).
+  g1.resolve();
+  await t1;
+  assert.equal(sem.inflight, 0, "slot freed with nobody queued — the cancelled waiter never got it");
+});
+
+await asyncTest("F2: an already-aborted signal rejects acquire() immediately, never touching the wait queue", async () => {
+  const sem = new TuiSemaphore(1, { maxQueue: 16 });
+  const g1 = deferred();
+  const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
+  await new Promise((r) => setImmediate(r));
+
+  const controller = new AbortController();
+  controller.abort(); // client already gone before this request ever tries to acquire
+  await assert.rejects(sem.acquire(controller.signal), SemaphoreAbortError);
+  assert.equal(sem.queued, 0, "never entered the wait queue at all");
+
+  g1.resolve(); await t1;
+});
+
+await asyncTest("F2: cancelling one queued waiter preserves FIFO order for the others", async () => {
+  const sem = new TuiSemaphore(1, { maxQueue: 16 });
+  const g1 = deferred();
+  const t1 = sem.run(async () => { await g1.p; });
+  await new Promise((r) => setImmediate(r));
+
+  const started = [];
+  const cA = new AbortController();
+  const cB = new AbortController();
+  const accA = sem.acquire(cA.signal).then(() => started.push("A"));
+  const accB = sem.acquire(cB.signal).then(() => started.push("B"));
+  const g3 = deferred();
+  const t3 = sem.run(async () => { started.push("C"); await g3.p; });
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.queued, 3, "A, B, C all queued behind t1");
+
+  cB.abort(); // B (the middle waiter) disconnects
+  await assert.rejects(accB, SemaphoreAbortError);
+  assert.equal(sem.queued, 2, "B removed; A and C remain, in original relative order");
+
+  g1.resolve();
+  await t1;
+  await new Promise((r) => setImmediate(r));
+  assert.deepEqual(started, ["A"], "A (queued first, still present) is granted next — FIFO preserved after B's removal");
+  assert.equal(sem.inflight, 1);
+  assert.equal(sem.queued, 1, "C still waiting");
+
+  sem.release(); // A was acquired directly (not via run()) — free its slot manually
+  await new Promise((r) => setImmediate(r));
+  assert.deepEqual(started, ["A", "C"], "C granted next");
+  g3.resolve();
+  await t3;
+  assert.equal(sem.inflight, 0);
+});
+
+await asyncTest("F2/L2: abort AFTER grant is a no-op — waiter keeps its slot, no rejection, slot released exactly once", async () => {
+  const sem = new TuiSemaphore(1, { maxQueue: 16 });
+  const g1 = deferred();
+  const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
+  await new Promise((r) => setImmediate(r));
+
+  const controller = new AbortController();
+  let granted = false;
+  const acq = sem.acquire(controller.signal).then(() => { granted = true; });
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.queued, 1, "waiter queued behind t1");
+
+  // t1 finishes → release() shifts the waiter out and grants it the slot (waiter() detaches
+  // the abort listener before resolving).
+  g1.resolve();
+  await t1;
+  await acq;
+  assert.equal(granted, true, "waiter was granted the slot");
+  assert.equal(sem.inflight, 1, "granted waiter holds the slot");
+  assert.equal(sem.queued, 0);
+
+  // The client disconnects AFTER the grant — the abort-after-grant race. onAbort must be a
+  // no-op (the waiter is no longer in _waiters; idx===-1 guard): no rejection materializes,
+  // the queue is untouched, and the slot is still owned by the (already-resolved) acquirer.
+  controller.abort();
+  await new Promise((r) => setImmediate(r));
+  assert.equal(sem.inflight, 1, "abort after grant did NOT revoke or double-free the slot");
+  assert.equal(sem.queued, 0, "abort after grant did not corrupt queue accounting");
+
+  // The slot is released exactly once via the normal path and is immediately reusable.
+  sem.release();
+  assert.equal(sem.inflight, 0, "slot released exactly once via the normal path");
+  await sem.run(async () => {}); // prove the semaphore is fully healthy afterward
+  assert.equal(sem.inflight, 0);
+});
+
 console.log("\nTUI drift observability (C-5):");
 
 test("recordTuiEntrypoint: observed 'cli' is NOT a mismatch and sets lastEntrypoint", () => {