feat(tui): warm pane pool — single-use pre-booted panes, opt-in via OCP_TUI_POOL_SIZE (−41%) (#158)

* feat(tui): warm pane pool — single-use pre-booted panes, opt-in via OCP_TUI_POOL_SIZE Backlog item #3 of docs/plans/2026-07-13-tui-latency/README.md. Every TUI request currently cold-boots a tmux+claude pane. This adds an OPT-IN pool of pre-booted panes. Recorded as ADR 0008 (docs/adr/0008-tui-warm-pane-pool.md), which extends ADR 0007. MEASURED (this host, Sonnet 4.6, --effort low, through a real OCP instance; a sample counts only if HTTP 200 AND the body carries the demanded marker): pool off (main code) n= 6 p50 10.17s [9164 9499 9760 10572 10774 11281] pool on, warm hits n=12 p50 6.00s [5286 5289 5520 5584 5621 5969 6040 6098 6280 7846 8036 11053] pool on, warm hits (post- n= 6 p50 5.62s [4729 4753 5236 6004 7548 9548] review-fix re-run) -> -4.17s / -41%. 12 hits / 1 miss / 0 bootFailures over 13 requests (and 6/1/0 on the post-fix re-run). Robust to counting the miss: n=13 p50 -> -40.6%. The plan doc predicted only -1.0s (the boot). It is ~4.2s because the cold path also pays ~2.9s INSIDE the first turn beyond claude's own reported turn_duration — post- input-bar init that an idle pane has already finished. Phase decomposition of the cold path (n=6 medians): prep 2ms | tmux spawn 27ms | boot->input-ready 1232ms | paste 8ms | paste-verify 426ms | submit->terminal 8458ms | teardown 8ms = 10162ms total, vs native turn_duration 5539ms => 4490ms of OCP-side overhead, of which the pool recovers ~1.26s of boot and ~2.9s of in-claude cold start. (The 426ms paste-verify is one 400ms poll tick; a real paste lands in ~80ms. Not addressed here — separate item.) DESIGN - SINGLE-USE panes. A pooled pane serves exactly ONE turn, then is killed and replaced in the background. Each carries its OWN fresh --session-id fixed at boot, so one session still holds one exchange. This is what keeps transcript.mjs's extractLatestAssistantText correct; its warning about a future warm pool reusing a session is answered in-place (comment updated) and left standing for anyone who later wants a second turn on a pane — that would be a cross-request TEXT LEAK and needs user-line scoping in the transcript reader first. - Pool keyed by model; --model is fixed at spawn. A miss falls back to the cold path with zero behaviour change. The pool warms the most recently requested model, so the first request after start (and after a model switch) is always a cold miss. - REAPER COEXISTENCE (the crux). An idle warm pane IS ours, and the periodic sweep runs precisely when we are idle. reapStaleTuiSessions() takes a `spare` set of EXACT live session names, and server.mjs DRAINS the pool immediately before the sweep: 1. a live pooled pane is never reaped — INCLUDING one still BOOTING (see below); 2. an orphaned pooled pane IS still reaped — membership is by exact name from a live in-memory registry, never by name shape, so a pane from a dead process generation has nothing claiming it. Omitting `spare` reaps MORE, never less (fail-safe); 3. kill-server is suppressed while any pane is spared — hence the drain, so the sweep still flushes <defunct> claude zombies (the only mechanism that can). - THE POOL TRACKS ITS IN-FLIGHT BOOT BY NAME, NOT AS A COUNT. bootTuiPane creates the tmux session SYNCHRONOUSLY and only then waits up to POOL_BOOT_MS (20s) for the input bar, so a pooled session can be LIVE for ~20s before its boot resolves. Tracking boots as a count meant the pool could not name that session, which caused two real bugs (found in review, reproduced, fixed, and now regression-tested): * the reap sweep KILLED the booting pane (it could not be spared), left the pool empty with nothing scheduled, and logged the exact tui_pool_boot_failed WARN operators are told to alert on — for a completely healthy drain; * graceful shutdown ORPHANED a live authenticated idle `claude`: gracefulShutdown calls process.exit(0) in the SAME TICK as the drain (TUI panes are tmux children, so activeProcesses is empty and the wait-for-children path exits immediately), so cleanup deferred to a .then() never ran. Fix: the pool mints each pane's identity up front ({sessionId, name}) and holds it in _bootingPane. liveNames() includes it; drain() kills it SYNCHRONOUSLY. A generation counter distinguishes "cancelled by us" from "genuinely failed", so a drain never inflates bootFailures and resume() reliably starts a fresh boot. Deriving the name from the session-id also makes `tmux ls` correlate to the transcript file. - SLOT ACCOUNTING. Refill boots take NO TuiSemaphore slot (those bound real turns and would be starved); they cannot leak one either, since they never hold one. Refills are SERIALIZED, one boot at a time — live at size=2, two cold boots racing an in-flight turn overran the readiness cap and a refill was discarded. A genuinely failed boot does not re-kick the chain (backoff; a broken claude must not respawn forever). Background boots get a more generous readiness cap (POOL_BOOT_MS = 5x BOOT_MS): BOOT_MS is tight because a client is blocked on it, which is not true of a pre-boot. - BOUNDED COST. A warm pane is a LIVE idle claude process held whether or not a request arrives. Peak processes = pool size + OCP_TUI_MAX_CONCURRENT + 1 booting replacement. Size clamped to POOL_MAX_SIZE=4; garbage values disable rather than guess. Panes have a 10-min TTL and a health check at hand-out (dead/degraded pane => miss, never a hang). Missing collaborators throw at CONSTRUCTION, not on a live request (refill() is called synchronously from the request path). DEFAULT OFF (OCP_TUI_POOL_SIZE=0). This is a stable production path and the pool holds standing processes, so the operator opts in. With the pool off, runTuiTurn takes the IDENTICAL code path as before (the `pool ? pool.acquire() : null` branch yields null, and tuiPool is null so no observer is attached and no new log line is emitted) — that is what establishes the default path is unchanged. A pool-off control run (n=6, p50 9.40s) is consistent with the 10.17s baseline but had 2/6 samples >12s, so it is corroboration, NOT proof: n=6 cannot establish "unregressed" on its own. The code-path equivalence can. BANNER: NO SPAWN ARGUMENT CHANGED. buildTuiCmd is byte-identical to main (verified by extracting the function body from both revisions and comparing). Live banner captured from two real POOLED panes anyway: "Sonnet 4.6 with low effort · Claude Max" — the subscription pool, never "API Usage Billing". /health: `tui.pool` added (null when off), incl. `cancelled` (boots WE killed — not a fault; do not alert on it). The tui block is ADR-0007-owned and post-dates ADR 0006's v3.16.4 grandfather snapshot; the addition is purely additive — every pre-existing key keeps a byte-identical value. Authorization recorded in ADR 0008. ALIGNMENT: Class B / ADR 0007 + ADR 0008 (OCP-owned TUI spawn machinery). cli.js does NOT perform this operation — there is no cli.js citation and none is required: this is not an Anthropic API surface, it is OCP's own process management around the claude CLI, exactly as the existing tmux session lifecycle and reaper already are (ALIGNMENT.md Rule 2). TESTS: 294 passed / 0 failed (was 267). +27 covering acquire/hit/miss, single-use (a pane is never handed out twice), bounded + serialized refill, TTL + health-check drops, model retarget, drain/resume, boot-failure backoff, identity linkage, all three reaper invariants incl. post-drain kill-server restoration, and — the coverage gap that let both bugs ship — FIVE mid-boot tests: the booting pane is nameable/spareable, the sweep's drain kills it and resume starts a fresh boot with no bogus WARN, shutdown kills it synchronously (asserted WITHOUT awaiting, since process.exit runs in the same tick), a stale settle cannot clear a newer boot's slot, and a model switch cancels an in-flight boot for the old model. Live verification (temporary 20s reap interval, reverted): sweep drained both panes -> reaped -> refilled with NEW panes; a foreign tmux session survived untouched; with no foreign session kill-server fired and the pool still recovered and served the next request. Both review bugs reproduced against a PRIVATE tmux server (-L pr3repro, so the reaper's internal kill-server could not touch the host) before and after the fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tui): kill a cancelled boot's pane when it settles + make async tests actually count Folds in the independent review's remaining nit — and, in proving the nit's fix, uncovers two defects in the test suite itself. ## The nit (latent M1b, second costume) `_cancelBooting` kills BY NAME, but the tmux session only EXISTS once `bootPane` has run — and `bootPane` is queued on a microtask. So a caller doing `refill()` then `drain()` in the SAME synchronous block leaves `_cancelBooting` with nothing to kill (a no-op); it bumps the generation, and the boot microtask then CREATES the session, succeeds, and — under the old bare `return` on a stale generation — walked away from a LIVE authenticated `claude` that nothing owns. Reproduced: reverted: drain() kills nothing (no session yet) -> boot creates it -> ORPHAN: ['p1'] fixed : drain() kills nothing (no session yet) -> boot creates it -> boot kills it -> [] Not reachable from any current call site, so this is defense-in-depth — but ADR 0008 and the reap-tick comment in server.mjs BOTH explicitly contemplate a boot-time pre-warm, which is exactly the shape that reaches it. Killing an already-dead session is a harmless no-op, so the fix is idempotent whichever way the race lands. ## Defect 1 in the suite: async tests were never awaited (44 of them) Writing the regression guard exposed this. `test()` called `fn()`, got a promise back, and IMMEDIATELY printed ✓ and incremented `passed` — without awaiting it. For all 44 tests written as `test("...", async () => {...})`: - ✓ meant "did not throw SYNCHRONOUSLY", not "passed"; - a failed assertion escaped as an unhandled rejection, crashing the process (CI stays red on the non-zero exit) but never being COUNTED — so the summary could print "0 failed" and be wrong. The suite's headline number was therefore not evidence for ANY async test, including this PR's own M1a/M1b guards. `test()` now settles an async body before counting it, and the summary awaits them. ## Defect 2, exposed the instant defect 1 was fixed: a false guard `"a boot that resolves AFTER a drain kills its own pane ... no orphan process left behind"` asserted `killed.length === 1` — i.e. that kill was CALLED once. But `_cancelBooting`'s kill-by-name on a not-yet-existent session is a NO-OP that still increments that counter. So "kill was called once" and "a live session is orphaned" were both true at the same time: a test named for the absence of an orphan was passing while the orphan was present. Now asserts LIVENESS (`live.size === 0`) — the only honest question. ## Evidence fix present : 295 passed, 0 failed, exit 0 fix reverted: 293 passed, 2 failed <- BOTH liveness guards fire (the old kill-count guard did not) Also: `dropped`'s doc comment now lists `cancelled` (a cancelled in-flight boot lands there via _drop). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VqgWJcjxrjjL9L9SkpZyXR --------- Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 09:44:07 +00:00 · 2026-07-13 16:51:12 +10:00
parent e7ce9899f3
commit 9f5bc3264a
9 changed files with 1371 additions and 72 deletions
@@ -23,9 +23,30 @@ process.env.HOME = homedir(); // ensure consistent
 let passed = 0;
 let failed = 0;

+// Pending promises from tests declared `async` but registered through the SYNC `test()` helper.
+// 44 tests in this file are written that way. Before this, `test()` called fn(), got a promise back,
+// and immediately printed ✓ and incremented `passed` — WITHOUT AWAITING IT. So for every async test:
+//   - ✓ meant "did not throw synchronously", NOT "passed";
+//   - a failed assertion escaped as an unhandled rejection, which crashes the process (CI still goes
+//     red on the non-zero exit) but is NOT counted, so the summary could print "N passed, 0 failed"
+//     and be wrong.
+// The suite's own headline number was therefore not evidence for any async test — including the
+// regression guards in this PR. Collected here and awaited before the summary prints.
+const pendingAsync = [];
+
 function test(name, fn) {
  try {
-    fn();
+    const r = fn();
+    if (r && typeof r.then === "function") {
+      // Async body: settle it before counting. Do NOT print ✓ yet.
+      pendingAsync.push(
+        r.then(
+          () => { passed++; console.log(`  ✓ ${name}`); },
+          (e) => { failed++; console.log(`  ✗ ${name}: ${e.message}`); },
+        ),
+      );
+      return;
+    }
    passed++;
    console.log(`  ✓ ${name}`);
  } catch (e) {
@@ -2027,6 +2048,489 @@ test("reaper with includeLegacy=true still spares a sibling instance's port-scop
  assert.ok(!calls.includes("kill-server"), "sibling instance's live session still blocks kill-server");
 });

+// ── TUI warm pane pool (docs/plans/2026-07-13-tui-latency #3) ────────────
+import { TuiPanePool, resolvePoolSize, POOL_MAX_SIZE, POOL_MAX_AGE_MS } from "./lib/tui/pool.mjs";
+import { poolPaneName as poolName } from "./lib/tui/session.mjs";
+
+// A pool wired to fakes: no tmux, no claude. bootPane resolves on the microtask queue; use
+// `await settle()` after a refill() to let the SERIALIZED boot chain run to target.
+// `live` models the real tmux server: bootTuiPane creates the session SYNCHRONOUSLY and only
+// THEN waits (up to POOL_BOOT_MS) for the input bar, so the fake boot registers the session
+// immediately and only afterwards resolves. `opts.hold` keeps a boot in that mid-flight window
+// so tests can act on a pane that is live-but-not-yet-warm — the state that hid two bugs.
+function makeFakePool(opts = {}) {
+  const killed = [];
+  const booted = [];
+  const live = new Set();   // "tmux sessions" that currently exist
+  let seq = 0;
+  let clock = 1_000_000;
+  const healthy = new Set();
+  // FIFO gate queue — one entry per in-flight held boot. A single `release` slot would be
+  // OVERWRITTEN by a later boot, so releasing "the first boot" would silently release the
+  // second instead (and mask the stale-settle bug this harness exists to test).
+  const gates = [];
+  const pool = new TuiPanePool({
+    size: opts.size ?? 2,
+    maxAgeMs: opts.maxAgeMs ?? POOL_MAX_AGE_MS,
+    now: () => clock,
+    mintPane: () => {
+      const n = ++seq;
+      return { sessionId: `sid-${n}`, name: `ocp-tui-3456-p${String(n).padStart(8, "0")}` };
+    },
+    bootPane: async (model, { sessionId, name }) => {
+      live.add(name);                                     // session exists NOW
+      booted.push({ name, model });
+      if (opts.hold) {
+        await new Promise((r) => gates.push(r));          // ...stuck waiting for readiness
+      }
+      if (opts.bootThrows) { live.delete(name); throw new Error("boom"); }
+      // A pane whose session was killed while booting can never become ready — exactly what
+      // the real bootTuiPane does (it throws tui_pane_not_ready).
+      if (!live.has(name)) throw new Error("tui_pane_not_ready");
+      healthy.add(name);
+      return { name, sessionId, model, bootedAt: clock };
+    },
+    killPane: (name) => { killed.push(name); healthy.delete(name); live.delete(name); },
+    paneHealthy: (name) => healthy.has(name),
+  });
+  return {
+    pool, killed, booted, healthy, live,
+    releaseBoot: () => { const r = gates.shift(); if (r) r(); },  // release the OLDEST held boot
+    advance: (ms) => { clock += ms; }, at: () => clock,
+  };
+}
+const tick = () => new Promise((r) => setImmediate(r));
+// Refills are SERIALIZED (one boot at a time, re-kicked on success), so settling the pool
+// takes a chain of microtask turns, not one. 40 is far more than POOL_MAX_SIZE needs.
+const settle = async () => { for (let i = 0; i < 40; i++) await tick(); };
+
+console.log("\nTUI warm pane pool (acquire / miss / refill / TTL / reaper exemption):");
+
+test("resolvePoolSize: default/garbage/negative disable the pool; size is clamped to POOL_MAX_SIZE", () => {
+  assert.equal(resolvePoolSize(undefined), 0, "unset => off (byte-for-byte today's cold path)");
+  assert.equal(resolvePoolSize("0"), 0);
+  assert.equal(resolvePoolSize("-3"), 0);
+  assert.equal(resolvePoolSize("banana"), 0, "garbage disables rather than guessing a size");
+  assert.equal(resolvePoolSize("2"), 2);
+  assert.equal(resolvePoolSize("99"), POOL_MAX_SIZE, "clamped — never boot an unbounded number of idle claudes");
+});
+
+test("pool size 0 is inert: acquire always misses and refill never boots", async () => {
+  const { pool, booted } = makeFakePool({ size: 0 });
+  assert.equal(pool.enabled, false);
+  assert.equal(pool.acquire("m1"), null, "disabled pool always MISSES → caller cold-boots");
+  pool.refill();
+  await settle();
+  assert.equal(booted.length, 0, "a disabled pool must never spawn a process");
+});
+
+test("acquire MISSES on an empty pool, and the miss refills for the requested model", async () => {
+  const { pool, booted } = makeFakePool({ size: 2 });
+  assert.equal(pool.acquire("sonnet"), null, "first request is always a MISS (no boot-time pre-warm)");
+  assert.equal(pool.misses, 1);
+  pool.refill();
+  await settle();
+  assert.equal(pool.warm, 2, "refilled to target");
+  assert.deepEqual(booted.map((b) => b.model), ["sonnet", "sonnet"], "warmed for the model that missed");
+});
+
+test("acquire HITS a warm pane, hands it out ONCE, and never returns it (single-use)", async () => {
+  const { pool } = makeFakePool({ size: 2 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  assert.equal(pool.warm, 2);
+
+  const a = pool.acquire("sonnet");
+  assert.ok(a && a.name && a.sessionId, "warm pane handed out");
+  assert.equal(pool.hits, 1);
+  assert.equal(pool.warm, 1, "the pane LEAVES the registry when acquired");
+
+  const b = pool.acquire("sonnet");
+  assert.notEqual(b.name, a.name, "a pane is NEVER handed out twice — single-use");
+  assert.notEqual(b.sessionId, a.sessionId, "each pane carries its OWN fresh session-id (transcript.mjs scoping)");
+  assert.equal(pool.warm, 0);
+  assert.equal(pool.acquire("sonnet"), null, "exhausted pool MISSES rather than reusing a pane");
+});
+
+test("refill is bounded: never more than `size` panes, and concurrent refills do not overshoot", async () => {
+  const { pool, booted } = makeFakePool({ size: 2 });
+  pool.acquire("sonnet");
+  pool.refill(); pool.refill(); pool.refill(); // hammer it
+  await settle();
+  assert.equal(pool.warm, 2, "still exactly `size` warm panes");
+  assert.equal(booted.length, 2, "the _booting guard prevented duplicate boots");
+});
+
+// Live finding at size=2: two cold `claude` boots racing an in-flight turn made a refill
+// overrun even the generous pool readiness cap. Boots are therefore SERIALIZED.
+test("refill boots panes ONE AT A TIME (never two claude cold-boots racing each other)", async () => {
+  let concurrent = 0, peak = 0;
+  let seq = 0;
+  const pool = new TuiPanePool({
+    size: 3,
+    mintPane: () => { const n = ++seq; return { sessionId: `s${n}`, name: `p${n}` }; },
+    bootPane: async (model, { sessionId, name }) => {
+      concurrent++; peak = Math.max(peak, concurrent);
+      await new Promise((r) => setImmediate(r)); // simulate boot latency
+      concurrent--;
+      return { name, sessionId, model, bootedAt: Date.now() };
+    },
+    killPane: () => {},
+    paneHealthy: () => true,
+  });
+  pool.acquire("sonnet");
+  pool.refill();
+  await settle();
+  assert.equal(pool.warm, 3, "chain still reaches the target size");
+  assert.equal(peak, 1, "at most ONE boot in flight at any moment");
+});
+
+test("a FAILED boot does not re-kick the chain (backoff — a broken claude must not spin)", async () => {
+  const { pool, booted } = makeFakePool({ size: 3, bootThrows: true });
+  pool.acquire("sonnet");
+  pool.refill();
+  await settle();
+  assert.equal(pool.bootFailures, 1, "counted as a genuine failure (nobody cancelled it)");
+  assert.equal(booted.length, 1, "exactly ONE attempt — a failure stops the chain, it does not respawn forever");
+  assert.equal(pool.warm, 0);
+  assert.equal(pool.booting, 0, "and the booting slot is released, so the next trigger can retry");
+});
+
+test("acquire drops an UNHEALTHY warm pane (kills it) and falls through to a MISS", async () => {
+  const { pool, killed, healthy, booted } = makeFakePool({ size: 1 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  const dead = booted[0].name;
+  healthy.delete(dead); // pane died / stopped being input-ready while idle
+
+  assert.equal(pool.acquire("sonnet"), null, "a dead pane must MISS, never hang a turn");
+  assert.ok(killed.includes(dead), "the dead pane is killed, not leaked");
+  assert.equal(pool.misses, 2);
+});
+
+test("acquire drops an EXPIRED warm pane (older than maxAgeMs)", async () => {
+  const { pool, killed, booted, advance } = makeFakePool({ size: 1, maxAgeMs: 60_000 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  advance(60_001);
+  assert.equal(pool.acquire("sonnet"), null, "a pane past its TTL is not handed out");
+  assert.ok(killed.includes(booted[0].name), "expired pane is killed");
+});
+
+test("a model switch drops the wrong-model panes and retargets the pool (--model is fixed at spawn)", async () => {
+  const { pool, killed, booted } = makeFakePool({ size: 2 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  const sonnetPanes = booted.map((b) => b.name);
+
+  assert.equal(pool.acquire("opus"), null, "different model => MISS (a sonnet pane cannot serve opus)");
+  assert.equal(pool.warm, 0, "sonnet panes dropped");
+  for (const p of sonnetPanes) assert.ok(killed.includes(p), "wrong-model pane killed, not leaked");
+  assert.equal(pool.warmModel, "opus", "pool retargeted to the model actually being asked for");
+
+  pool.refill(); await settle();
+  assert.deepEqual(booted.slice(2).map((b) => b.model), ["opus", "opus"], "refilled for the NEW model");
+});
+
+test("a boot that resolves AFTER a drain kills its own pane instead of enlisting it", async () => {
+  const { pool, live } = makeFakePool({ size: 1 });
+  pool.acquire("sonnet");
+  pool.refill();          // boot is in flight...
+  pool.drain();           // ...pool drained before it resolves (shutdown / reap sweep)
+  await settle();
+  assert.equal(pool.warm, 0, "the late pane must NOT be enlisted into a drained pool");
+  // Assert LIVENESS, not the kill-call COUNT. This assertion used to read
+  // `assert.equal(killed.length, 1)` — and it PASSED while the orphan it is named after was
+  // actually present: _cancelBooting kills BY NAME, and at drain time the tmux session does not
+  // exist yet (bootPane runs on a microtask), so that kill is a NO-OP which still increments the
+  // counter. "kill was called once" and "a live session is orphaned" were both true at the same
+  // time. The only honest question is whether the session is dead.
+  assert.equal(live.size, 0, "it kills itself — no orphan process left behind");
+});
+
+test("bootPane failure is counted, never thrown into the request path, and does not wedge refill", async () => {
+  const { pool } = makeFakePool({ size: 1, bootThrows: true });
+  pool.acquire("sonnet");
+  pool.refill();
+  await settle();
+  assert.equal(pool.warm, 0);
+  assert.equal(pool.bootFailures, 1);
+  assert.equal(pool.booting, 0, "the _booting counter is released on failure (else refill wedges forever)");
+  assert.equal(pool.acquire("sonnet"), null, "and the caller just MISSES → cold path");
+});
+
+test("drain kills every warm pane and pauses refills; resume restarts them", async () => {
+  const { pool, killed } = makeFakePool({ size: 2 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  assert.equal(pool.warm, 2);
+
+  assert.equal(pool.drain(), 2, "drain reports how many it killed");
+  assert.equal(pool.warm, 0);
+  assert.equal(killed.length, 2, "both panes killed — none outlive the drain");
+
+  pool.refill(); await settle();
+  assert.equal(pool.warm, 0, "refill is a NO-OP while drained (paused)");
+
+  pool.resume(); await settle();
+  assert.equal(pool.warm, 2, "resume refills");
+});
+
+// ── The crux: pool ↔ reaper coexistence (POOL/REAPER INVARIANT, lib/tui/session.mjs) ──
+console.log("\nTUI warm pool ↔ session reaper coexistence:");
+
+test("INVARIANT 1: a LIVE pooled pane is NEVER reaped (it is in the spare set)", () => {
+  const killed = [];
+  const live = "ocp-tui-3456-pdeadbeef";
+  const fakeTmux = (args) => {
+    if (args[0] === "list-sessions") return { status: 0, stdout: `${live}\nocp-tui-3456-aaaa\n` };
+    if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
+    return { status: 0, stdout: "" };
+  };
+  const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456, spare: new Set([live]) });
+  assert.equal(n, 1, "only the stale turn session was reaped");
+  assert.ok(!killed.includes(live), "the live warm pane must survive the sweep");
+  assert.ok(killed.includes("ocp-tui-3456-aaaa"), "a genuinely stale own session is still reaped");
+});
+
+test("INVARIANT 2: an ORPHANED pooled pane (pool-shaped but NOT in the spare set) IS reaped", () => {
+  const killed = [];
+  // ocp-tui-3456-porphan01 LOOKS pooled but the live registry does not claim it — e.g. left
+  // behind by a previous process generation, whose in-memory registry died with it.
+  const fakeTmux = (args) => {
+    if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-porphan01\nocp-tui-3456-plive0001\n" };
+    if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
+    return { status: 0, stdout: "" };
+  };
+  const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456, spare: new Set(["ocp-tui-3456-plive0001"]) });
+  assert.equal(n, 1);
+  assert.deepEqual(killed, ["ocp-tui-3456-porphan01"], "exemption is by EXACT NAME, never by name shape");
+});
+
+test("INVARIANT 2b: with NO spare set (the pre-pool call shape) pool-shaped panes are reaped — fail-safe", () => {
+  const killed = [];
+  const fakeTmux = (args) => {
+    if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-pdeadbeef\n" };
+    if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
+    return { status: 0, stdout: "" };
+  };
+  const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
+  assert.equal(n, 1, "omitting `spare` reaps MORE, never less — forgetting it can't leak panes");
+  assert.deepEqual(killed, ["ocp-tui-3456-pdeadbeef"]);
+});
+
+test("INVARIANT 3: kill-server is SUPPRESSED while a live pooled pane is spared", () => {
+  const calls = [];
+  const live = "ocp-tui-3456-plive0001";
+  const fakeTmux = (args) => {
+    calls.push(args.join(" "));
+    if (args[0] === "list-sessions") return { status: 0, stdout: `${live}\nocp-tui-3456-aaaa\n` };
+    return { status: 0, stdout: "" };
+  };
+  reapStaleTuiSessions({ tmux: fakeTmux, port: 3456, spare: new Set([live]) });
+  assert.ok(!calls.includes("kill-server"), "kill-server would kill the live pane (a child of the tmux server)");
+});
+
+test("INVARIANT 3b: after a DRAIN the spare set is empty, so kill-server fires again (zombie reaping preserved)", async () => {
+  // This is the whole reason server.mjs drains BEFORE the periodic sweep: a permanently-full
+  // pool would otherwise permanently suppress the only mechanism that reaps defunct claudes.
+  const { pool } = makeFakePool({ size: 2 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  assert.equal(pool.liveNames().size, 2, "pool is full → the sweep would be suppressed");
+
+  pool.drain();
+  assert.equal(pool.liveNames().size, 0, "drain empties the live registry");
+
+  const calls = [];
+  const fakeTmux = (args) => {
+    calls.push(args.join(" "));
+    if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\n" };
+    return { status: 0, stdout: "" };
+  };
+  reapStaleTuiSessions({ tmux: fakeTmux, port: 3456, spare: pool.liveNames() });
+  assert.ok(calls.includes("kill-server"), "kill-server fires post-drain — defunct zombies still get reaped");
+});
+
+// ── MID-BOOT: the state that hid M1a + M1b ────────────────────────────────────────────
+// bootTuiPane creates the tmux session SYNCHRONOUSLY, then waits up to POOL_BOOT_MS (20s)
+// for the input bar. So a pooled session can be LIVE for ~20s before its boot resolves.
+// Every reaper test above uses a pool that is either full or drained — never mid-boot.
+// That gap is exactly why both bugs shipped past the first round of tests.
+
+test("M1a: a reap tick during an IN-FLIGHT BOOT must not orphan-kill the booting pane", async () => {
+  const { pool, live } = makeFakePool({ size: 1, hold: true });
+  pool.acquire("sonnet");
+  pool.refill();
+  await tick();                                    // boot started; session live; NOT yet warm
+  assert.equal(pool.warm, 0, "not warm yet");
+  assert.equal(pool.booting, 1, "a boot is in flight");
+  assert.equal(live.size, 1, "...and its tmux session ALREADY EXISTS");
+
+  const bootingName = [...live][0];
+  assert.ok(pool.liveNames().has(bootingName),
+    "REGRESSION GUARD: the booting pane MUST be nameable, or the sweep cannot spare it " +
+    "(the pool used to track in-flight boots as a COUNT and this was empty)");
+});
+
+test("M1a: the reap tick's drain kills the booting pane, and resume() starts a FRESH boot", async () => {
+  const warns = [];
+  const { pool, live, releaseBoot } = makeFakePool({ size: 1, hold: true });
+  pool._log = (lvl, ev) => { if (lvl === "warn") warns.push(ev); };
+  pool.acquire("sonnet");
+  pool.refill();
+  await tick();
+  const first = [...live][0];
+
+  // The reap tick, as server.mjs runs it: drain -> reap -> resume.
+  const drained = pool.drain();
+  assert.equal(drained, 1, "drain accounts for the booting pane");
+  assert.equal(live.size, 0, "its tmux session is killed — kill-server can now flush zombies");
+  assert.equal(pool.liveNames().size, 0, "nothing left to spare, so kill-server is not suppressed");
+
+  pool.resume();
+  await tick();
+  assert.equal(pool.booting, 1, "resume() started a FRESH boot — the pool is not left empty with nothing scheduled");
+  assert.notEqual([...live][0], first, "and it is a NEW pane, not the killed one");
+
+  // Now let the ORIGINAL (cancelled) boot settle. It rejects with tui_pane_not_ready because
+  // we killed its session — but that is OUR doing, not a fault.
+  releaseBoot();
+  await settle();
+  assert.equal(pool.bootFailures, 0,
+    "a cancelled boot must NOT be counted as a bootFailure — that is the WARN operators alert on");
+  assert.deepEqual(warns, [], "and it must not log tui_pool_boot_failed for a healthy drain");
+  assert.equal(pool.cancelled, 1, "it is counted as a cancellation instead (counted exactly once)");
+});
+
+test("M1b: shutdown drain kills the booting pane SYNCHRONOUSLY — no orphaned claude", async () => {
+  const { pool, live } = makeFakePool({ size: 1, hold: true });
+  pool.acquire("sonnet");
+  pool.refill();
+  await tick();
+  assert.equal(live.size, 1, "a live pooled session exists");
+
+  // gracefulShutdown: drain() then process.exit(0) IN THE SAME TICK (TUI panes are tmux
+  // children, not node children, so activeProcesses is empty and the exit is immediate).
+  // Nothing scheduled on the microtask queue can run. So we assert WITHOUT awaiting.
+  pool.drain();
+  assert.equal(live.size, 0,
+    "REGRESSION GUARD: the pane must be dead BEFORE any await. A .then()-based cleanup would " +
+    "never run before process.exit and would orphan a live authenticated `claude`.");
+});
+
+// M1b, second costume: drain() in the SAME synchronous block as refill(). The tmux session does
+// not exist yet at drain time (bootPane runs on a microtask), so _cancelBooting's kill-by-name is
+// a no-op — and a `.then` that merely `return`s on a stale generation would then let the boot
+// CREATE the session and walk away from it. Not reachable from any current call site, but ADR 0008
+// and the reap-tick comment both contemplate a boot-time pre-warm, which is exactly this shape.
+// NOTE: deliberately NOT `hold: true`. A held boot never settles, so its `.then` never runs and
+// the guard would vacuously pass — the test must let the boot actually SUCCEED, because the bug is
+// precisely that a SUCCESSFUL boot on a cancelled generation walks away from its live session.
+test("M1b': a boot cancelled BEFORE its session existed is still killed when it settles", async () => {
+  const { pool, live } = makeFakePool({ size: 1 });
+  pool.acquire("sonnet");                 // miss → learns the model
+
+  pool.refill();   // mints the identity; bootPane is queued on a microtask — no session YET
+  pool.drain();    // SAME sync block: kill-by-name finds nothing to kill (no-op), bumps the gen
+  assert.equal(live.size, 0, "precondition: the session genuinely did not exist at cancel time");
+
+  await tick();    // NOW the boot microtask runs, CREATES the session, and settles on a stale gen
+  await tick();
+
+  assert.equal(live.size, 0,
+    "REGRESSION GUARD: a stale-generation boot must KILL its pane, not assume _cancelBooting " +
+    "already did. _cancelBooting kills BY NAME, and the tmux session does not exist until the " +
+    "boot microtask runs — so a cancellation landing first is a no-op, and a bare `return` here " +
+    "orphans a live authenticated `claude` that nothing owns.");
+});
+
+test("a stale boot settling after drain+resume must not clear the NEW boot's slot", async () => {
+  const { pool, releaseBoot } = makeFakePool({ size: 1, hold: true });
+  pool.acquire("sonnet");
+  pool.refill();
+  await tick();
+  pool.drain();                 // cancels boot #1 (generation bumped)
+  pool.resume();
+  await tick();
+  assert.equal(pool.booting, 1, "boot #2 owns the slot");
+  releaseBoot();                // boot #1 finally settles (rejects)
+  await settle();
+  assert.equal(pool.booting, 1, "boot #2 STILL owns the slot — a stale settle must not free it");
+});
+
+test("a model switch cancels an in-flight boot for the OLD model (kills it, frees the slot)", async () => {
+  const { pool, live, killed } = makeFakePool({ size: 1, hold: true });
+  pool.acquire("sonnet");
+  pool.refill();
+  await tick();
+  const sonnetPane = [...live][0];
+
+  pool.acquire("opus");         // retarget mid-boot
+  assert.ok(killed.includes(sonnetPane), "the old model's booting pane is killed, not left to linger");
+  assert.equal(pool.booting, 0, "and its slot is freed immediately, so the new model can boot now");
+  assert.equal(pool.warmModel, "opus");
+
+  pool.refill();
+  await tick();
+  assert.equal(pool.booting, 1, "a boot for the NEW model starts without waiting out the old one");
+});
+
+test("a pane handed out for a turn leaves the spare set immediately (so its teardown is authoritative)", async () => {
+  const { pool } = makeFakePool({ size: 2 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  const taken = pool.acquire("sonnet");
+  assert.ok(!pool.liveNames().has(taken.name),
+    "an acquired pane is the CALLER's — the pool must not also claim it live, or a crashed turn's pane would be spared forever");
+});
+
+test("N1: the pool mints ONE identity — the tmux name's hex is the transcript session-id's hex", async () => {
+  // Without this, `tmux ls` shows a pane whose name has no relation to any transcript file,
+  // so a live pane cannot be correlated to <HOME>/.claude/projects/*/<sessionId>.jsonl.
+  const seen = [];
+  const pool = new TuiPanePool({
+    size: 1,
+    mintPane: () => {
+      const sessionId = "deadbeef-1111-2222-3333-444444444444";
+      return { sessionId, name: poolName(3456, sessionId) };
+    },
+    bootPane: async (model, ident) => {
+      seen.push(ident);
+      return { ...ident, model, bootedAt: Date.now() };
+    },
+    killPane: () => {},
+    paneHealthy: () => true,
+  });
+  pool.acquire("sonnet");
+  pool.refill();
+  await settle();
+  assert.equal(seen.length, 1, "bootPane received the pool-minted identity");
+  assert.equal(seen[0].name, "ocp-tui-3456-pdeadbeef");
+  assert.ok(seen[0].name.endsWith(seen[0].sessionId.slice(0, 8)),
+    "the tmux session name carries the session-id's own hex — `tmux ls` correlates to the transcript");
+  const pane = pool.acquire("sonnet");
+  assert.equal(pane.sessionId, seen[0].sessionId,
+    "and the turn reads the transcript under THAT session-id — one identity end to end");
+});
+
+test("pool pane names are port-scoped (reapable as ours) and never match the legacy shape", () => {
+  const name = poolName(3456, "deadbeef-1111-2222-3333-444444444444");
+  assert.ok(name.startsWith(sessionPrefixForPort(3456)), "pool panes are OURS → reapable when orphaned");
+  assert.equal(name, "ocp-tui-3456-pdeadbeef");
+  assert.ok(!LEGACY_SESSION_NAME_RE.test(name), "must never be mistaken for a legacy bare-prefix session");
+  assert.ok(!poolName(9999, "aaaaaaaa-0000-0000-0000-000000000000").startsWith(sessionPrefixForPort(3456)),
+    "a sibling instance's pool pane is foreign to us");
+});
+
+test("buildTuiHealthBlock reports pool:null when off, and the pool's stats when on", async () => {
+  const st = { lastEntrypoint: "cli", entrypointMismatches: 0 };
+  const sem = { inflight: 0, queued: 0 };
+  const off = buildTuiHealthBlock({ enabled: true, entrypointMode: "cli", maxConcurrent: 2 }, st, sem, null);
+  assert.equal(off.pool, null, "pool disabled → explicit null (stable /health shape)");
+
+  const { pool } = makeFakePool({ size: 2 });
+  pool.acquire("sonnet"); pool.refill(); await settle();
+  const on = buildTuiHealthBlock({ enabled: true, entrypointMode: "cli", maxConcurrent: 2 }, st, sem, pool);
+  assert.equal(on.pool.size, 2);
+  assert.equal(on.pool.warm, 2);
+  assert.equal(on.pool.misses, 1);
+  assert.equal(on.pool.model, "sonnet");
+});
+
 // ── TUI home preparation (scratch vs real) ───────────────────────────────
 import { prepareTuiHome, ensureTuiCwdTrusted } from "./lib/tui/session.mjs";
 import { mkdtempSync as hMkdtemp, mkdirSync as hMkdir, writeFileSync as hWrite, readFileSync as hRead, existsSync as hExists, readlinkSync as hReadlink } from "node:fs";
@@ -2474,8 +2978,13 @@ test("buildTuiHealthBlock: shape + live counters (the additive /health tui block
  const ts = { lastEntrypoint: "cli", entrypointMismatches: 3 };
  const block = buildTuiHealthBlock(
    { enabled: true, entrypointMode: "cli", maxConcurrent: 2 }, ts, sem);
+  // `pool` joined this key set with the warm pane pool. The tui block is ADR-0007-owned (it
+  // did not exist at v3.16.4, so it is outside ADR 0006's grandfather freeze), and the
+  // addition is purely additive: every pre-existing key below still carries a byte-identical
+  // value, and `pool` is null unless the operator opts in via OCP_TUI_POOL_SIZE.
  assert.deepEqual(Object.keys(block).sort(),
-    ["enabled", "entrypointMismatches", "entrypointMode", "inflight", "lastEntrypoint", "maxConcurrent", "queued"]);
+    ["enabled", "entrypointMismatches", "entrypointMode", "inflight", "lastEntrypoint", "maxConcurrent", "pool", "queued"]);
+  assert.equal(block.pool, null, "no pool passed → null (the default, pool disabled)");
  assert.equal(block.enabled, true);
  assert.equal(block.entrypointMode, "cli");
  assert.equal(block.lastEntrypoint, "cli");
@@ -2934,7 +3443,9 @@ async function runAsyncTests() {
 }

 // ── Cleanup ──
-runAsyncTests().then(() => {
+// Settle the async-bodied tests registered through the sync `test()` helper BEFORE summarizing —
+// otherwise their pass/fail is not reflected in the counts (see the `pendingAsync` comment above).
+runAsyncTests().then(() => Promise.all(pendingAsync)).then(() => {
  closeDb();
  console.log(`\n=== Results: ${passed} passed, ${failed} failed ===\n`);
  process.exit(failed > 0 ? 1 : 0);