Compare commits

...
Author SHA1 Message Date
taodengandClaude Fable 5 4239744c86 docs(readme): review nit — 'pre-v3.22' → 'pre-flag' (next version not fixed yet)
Reviewer nit from the Iron Rule 10 independent review of PR #156.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VqgWJcjxrjjL9L9SkpZyXR
2026-07-13 15:15:01 +10:00
taodengandClaude Fable 5 e95a7bb0f2 feat(tui): pin spawn effort via OCP_TUI_EFFORT (default low)
buildTuiCmd never passed --effort, so the pane's claude inherited a
HOME-dependent effortLevel: real-home mode inherits the operator's
~/.claude/settings.json (high/xhigh on typical operator hosts),
env-token scratch mode inherits claude's built-in default — proxied-turn
latency silently depended on which HOME mode resolveTuiHome() picked and
on an unrelated operator setting.

Pass --effort explicitly, from new env var OCP_TUI_EFFORT (default
"low"; allowlist low|medium|high|xhigh|max per `claude --help` 2.1.207;
"inherit" restores the pre-flag argv byte-for-byte; an invalid value
warns and falls back to "low" so a typo can never reach the pane argv).

Not endpoint-touching: no server.mjs change, no wire-level change — the
flag rides the existing interactive spawn (ADR 0007). Billing-pool
safety verified per the docs/plans/2026-07-13-tui-latency banner
protocol: startup banner stays "Claude Max" with "low effort".

Measured through a test OCP instance (:3979, TUI mode, real-home,
claude-sonnet-4-6, n=5+5, same ~1850-token prompt as floor.sh):

  before: median 11.30s  range 9.05-12.38s (spread 3.32s)  banner: high effort - Claude Max
  after:  median  9.55s  range 9.27-9.77s  (spread 0.50s)  banner: low effort - Claude Max
  OCP_TUI_EFFORT=inherit: banner back to "high effort" (pre-flag behavior restored)

README: new row in the Environment Variables table (release_kit
new_feature_doc_expectations: new env var -> README table). Tests: 4 new
buildTuiCmd cases (default, explicit level, inherit, invalid fallback);
suite 267 passed / 0 failed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VqgWJcjxrjjL9L9SkpZyXR
2026-07-13 15:08:50 +10:00
6854075c01 docs(plans): TUI-mode latency floor — measured decomposition + backlog (#155)
* docs(plans): TUI-mode latency floor — measured decomposition + backlog

An external consumer measured OCP's prompt path at TTFT p50 30-32s and excluded
OCP on that basis. This documents where those 30 seconds actually go, with a
reproducible harness (n=15) that bypasses OCP and measures the underlying
subscription path's true first-token time.

Findings:
- boot -> input-ready is only ~1.0s; it is NOT the bottleneck
- true TTFT is 6-10s; the remaining ~20s is runTuiTurn polling the transcript
  until turn_duration (ADR 0007 step 4) — i.e. waiting for the WHOLE turn.
  There is no streaming.
- buildTuiCmd never passes --effort, so the spawned claude inherits the
  operator's global effortLevel (xhigh on this host) — every request runs
  extended thinking. Passing --effort low: TTFT p50 9.70s -> 6.17s (-36%),
  spread 7.85-13.07s -> 5.87-6.44s. Stays on Claude Max.
- ⚠️ --bare SILENTLY drops off the subscription pool (banner flips
  'Claude Max' -> 'API Usage Billing'). It does cut boot to ~0.5s, but defeats
  the entire purpose of ADR 0007. Failure is silent: all 5 --bare samples
  produced no answer at all (no error, no crash, just never a token).
  Anyone optimizing boot MUST diff the banner line.
- Floor after all fixes is ~6s (claude always injects the full CC system prompt
  + tool definitions). TUI mode therefore cannot serve real-time consumers —
  a constraint worth stating in the README.

Backlog ranked by value/effort: (1) OCP_TUI_EFFORT env var, default low;
(2) real streaming instead of turn_duration polling (~20s, the big one);
(3) warm pane pool (~1s); (4) prefill trim (probably not worth it).

Docs-only; no version bump (matches repo convention — bump lands in the
chore(release) commit).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dx5Ncq6wWBrF27vJKHZ9Hr

* docs(plans): address review — restore --bare evidence, qualify effort claim, add banner captures

Reviewer (fresh-context, Iron Rule 10) returned REQUEST_CHANGES. All four technical
conclusions survived independent verification (source-read + live repro); the defects
were in the evidence file, and they were real:

- H-1: measurements.jsonl claimed n=15 but held 10 rows, and the --bare group — the
  basis of this PR's headline warning — had ZERO rows. The author had stripped them
  as 'invalid samples' (ttft_ms:-1) when they were in fact the evidence. Regenerated:
  n=15, three groups × 5, all with tag/extra_args. --bare reproduces exactly (5/5 no
  answer, boot 0.43-0.45s).
- M-1: the effort-inheritance claim was written unconditionally, but it depends on
  resolveTuiHome()'s mode. Real-home (current service config) inherits the operator's
  effortLevel: xhigh; env-token scratch home (~/.ocp-tui/home) has no effortLevel in
  its settings.json and prepareTuiHome() never writes one, so the pane gets claude's
  built-in default. Now documented as a table — and the mode split makes passing
  --effort explicitly MORE valuable, not less.
- M-2: baseline rows were produced by a pre-parameterized script and lacked
  tag/extra_args. Re-run with the committed script. Recomputed effect: -40% (was -36%).
- L-1: documented that the harness suppresses OCP's periodic kill-server tick via the
  othersRemain coexistence guard (by design, resumes next tick).
- L-2: documented that floor.sh's readiness marker differs from OCP's tuiInputReady(),
  so the ~1.0s boot figure is not apples-to-apples with BOOT_MS.
- Direct-API reference figure now explicitly labeled as external (not in this dataset).
- New: billing-banner.txt captures all three configs live, including confirmation that
  --effort low stays on Claude Max (reviewer noted this was asserted but unevidenced).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dx5Ncq6wWBrF27vJKHZ9Hr

* docs(plans): scope the effort claim to TUI mode (currently off), drop nonexistent bin/

Re-review (APPROVE_WITH_MINOR) caught two accuracy defects:
- MIN-1: 'every OCP request runs extended thinking' over-extrapolated. TUI mode is
  currently OFF on this host (CLAUDE_TUI_MODE=false; /health tui.enabled=false), so
  live traffic takes the -p path. The claim is about what happens WHEN TUI mode is
  enabled — now scoped, and the same qualifier applied to the kill-server interaction
  note (that reap tick is itself gated on TUI_MODE).
- NIT-2: the quoted grep included bin/, which does not exist in the repo (exit 2).
  Dropped; the zero-hit result over lib/ + server.mjs is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dx5Ncq6wWBrF27vJKHZ9Hr

---------

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 14:22:17 +10:00
45152d58b0 chore(release): v3.21.1 — concurrency queue + spawn-token + TUI session-scope fixes (#151)
Patch release bundling three merged bug fixes (no new cli.js wire behavior,
no new endpoint/header/env var):

- #148 fix(tui): session prefix + reap/kill-server scoped per-instance by port
- #150 fix(server): serialize -p real-HOME token fallback behind a mutex +
  30s TTL keychain read cache + de-staled isolation decision (new lib/spawn-auth.mjs)
- #149 fix: semaphore honors runtime-lowered maxConcurrent, queued requests
  cancelled on client disconnect, singleflight follower retry on leader
  disconnect, exact queued accounting, quiet disconnect handling

Release-kit walk (CLAUDE.md Iron Rule 5.5): package.json version bump,
CHANGELOG.md entry, one-sentence Troubleshooting note for the genuinely
operator-visible upgrade-overlap caveat from #148. No changes to
models.json / Available Models / API Endpoints / Environment Variables
tables — verified by diffing all three merged commits (2922d68..d96da46);
none add an endpoint, header, or env var.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:09:01 +10:00
d96da46fa0 fix: honor runtime-lowered concurrency limit + cancel queued waiters on disconnect (#149)
* fix: honor runtime-lowered concurrency limit + cancel queued waiters on disconnect

Fixes three findings from an independent concurrency audit of the -p/stream-json
wait-queue (lib/tui/semaphore.mjs, reused by server.mjs as `claudeSemaphore`) and
its acquireClaudeSlot()/callClaudeTui() callers in server.mjs:

F1 (MEDIUM) — release() handed a freed slot straight to the next queued waiter
without re-checking `this.limit`, so a PATCH /settings maxConcurrent decrease was
silently ignored until every already-inflight task happened to finish on its own.
release() now only re-grants when post-decrement inflight is still under the
current limit, and a new setLimit() wakes queued waiters immediately when the
limit is raised instead of only on the next incidental release().

F2 (MEDIUM) — a request queued behind the concurrency limit had no link to its
HTTP connection, so a client that disconnected while still queued would still
get a claude process spawned for it once a slot freed — burning subscription
quota for a dead socket. acquire() now accepts an optional AbortSignal; server.mjs
derives one from the client's res "close" event (closeSignalFor) and passes it
into claudeSemaphore.acquire() / tuiSemaphore.acquire() while queued. On abort the
waiter is spliced out of the queue (not just flagged), so `queued` accounting
stays exact; the same "close" signal is wired into acquireClaudeSlot() (-p path,
non-streaming + streaming + singleflight-wrapped) and callClaudeTui() (TUI path).
If the response is already destroyed by the time we try to queue, we reject
immediately without ever entering the queue.

F8 (cosmetic) — acquireClaudeSlot() set `stats.queued = claudeSemaphore.queued + 1`
BEFORE calling acquire(), over-reporting /health's queued count by 1 whenever a
slot was granted immediately (the common, non-queued case). acquire() already
updates its internal queue synchronously before returning a Promise, so reading
claudeSemaphore.queued right AFTER calling it (instead of guessing "+1" before)
is exact. No /health field was added, removed, or renamed.

ALIGNMENT.md: this PR touches request-handler code (callClaude, callClaudeStreaming,
callClaudeTui, acquireClaudeSlot) but is local concurrency-control/queue-accounting
infrastructure with no cli.js wire analogue — it does not add, rename, or change any
endpoint, header, request field, or response field, and does not touch the /v1/messages
forwarding path or the OAuth bearer machinery (the two Class A surfaces this repo
governs). The /health response shape is unchanged (same field set, same nesting;
only the *value* of the pre-existing `stats.queued` field is corrected). Per
CLAUDE.md hard-requirement #1, a cli.js citation is therefore declared ABSENT:
there is no corresponding cli.js operation to cite because this is not a
cli.js-mirror (Class A) change and not a Class B endpoint-contract change either.

Tests: added 6 unit tests to test-features.mjs against the shared TuiSemaphore
(lowering the limit mid-load does not over-admit; raising the limit wakes queued
waiters up to the new headroom, FIFO; a queued waiter cancelled via AbortSignal
is spliced out and never later acquires; an already-aborted signal never touches
the queue; cancelling one of several queued waiters preserves FIFO for the rest).
238 pre-existing tests remain green; suite is now 244/244.

Verification: `node --check server.mjs && node --check lib/tui/semaphore.mjs && npm test` — 244 passed, 0 failed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: singleflight follower retry on leader disconnect + quiet disconnect handling (review M1/L1/L2)

Addresses the independent reviewer's APPROVE-WITH-CHANGES findings on PR #149:

M1 (MEDIUM, F2 regression) — when a singleflight LEADER disconnected while queued,
its RequestDisconnectedError rejected the SHARED promise, so live followers fell
into respondUpstreamError's generic branch and got a spurious 500 on a healthy
socket. Fix: keys.mjs singleflight() gains an optional follower-side `retryIf`
predicate. When a follower joins an existing flight and the shared promise rejects
with an error retryIf() accepts, the follower does NOT inherit the rejection — it
re-enters singleflight with its OWN fn (the map entry is guaranteed already deleted:
the delete-finally is attached upstream of the promise followers await), becoming
the new leader or joining a retrying sibling's fresh flight. The leader's own
rejection is never retried (it IS that client's disconnect). server.mjs passes
retryIf = (err) => err instanceof RequestDisconnectedError && !res.destroyed, so a
follower whose own client is also gone still propagates quietly. Callers without
retryIf keep byte-for-byte pre-existing share-everything semantics (pinned by the
existing failure-fan-out test).

L1 (LOW) — a disconnect-while-queued on the non-streaming paths was recorded as a
usage FAILURE row and logged as a [proxy] error: metric noise for a non-error.
Both non-streaming catch blocks now early-return on RequestDisconnectedError
without recordUsage(success:false) and without console.error — mirroring the
streaming path, which returns silently. The disconnect remains observable at info
level (concurrency_wait_cancelled, now also emitted with path:"tui" from
callClaudeTui for parity with acquireClaudeSlot's -p log).

L2 (LOW, test gap) — added a unit test for the abort-after-grant race: a waiter
granted its slot whose signal aborts afterward must see no rejection, no queue
corruption, and its slot released exactly once via the normal path (the semaphore
detaches the abort listener at grant; the onAbort idx===-1 guard is the in-dispatch
backstop).

Tests: +3 (2× M1 in the singleflight section, 1× L2 in the F2 section) — suite is
now 247/247 green.

ALIGNMENT.md: unchanged declaration — still local concurrency/dedup infrastructure
with no cli.js wire analogue; no endpoint, header, request field, or response field
added or changed; /health shape untouched. cli.js citation declared ABSENT per
CLAUDE.md hard-requirement #1 (not a Class A mirror change, not a Class B
contract change).

Verification: node --check server.mjs lib/tui/semaphore.mjs keys.mjs && npm test
— 247 passed, 0 failed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:04:13 +10:00
2538233059 fix(server): serialize -p real-HOME token fallback + TTL-cache keychain + de-stale isolation decision (F3/F5/F6) (#150)
* fix(server): serialize -p real-HOME token fallback + TTL-cache keychain + de-stale isolation decision (F3/F5/F6)

Three audit findings in the -p spawn-token resolution + HOME-isolation layer. All are infra/
process changes to how OCP READS and GATES an OAuth token it already holds; none touch the OAuth
wire machinery.

F3 (MEDIUM) — expiry-window fallback herds concurrent -p spawns into real HOME.
When the keychain token is within 5 min of expiry, resolveSpawnToken() returns null and every
concurrent spawn simultaneously falls back to the real HOME; each spawned claude then races a
refresh_token grant against the SAME single-use refresh token — rotating it out from under the
others and the operator's real claude (the credential-fork hazard, #112/#146 class). Fix: a
promise-chain mutex (createSerialMutex) serializes ONLY the real-HOME fallback — one such spawn at
a time. When a serialized waiter is admitted (prior holder torn down → its claude has refreshed the
keychain), it re-runs resolveSpawnToken(): a now-fresh token means it proceeds ISOLATED instead of
real-HOME, so the queue drains to the fast path. Isolated spawns never touch the mutex.

F5 (LOW-MED) — per-spawn double keychain exec on the hot path.
getOAuthCredentials() sync-exec'd `security find-generic-password` up to twice (wrong label first),
worst case 5s×2, blocking the event loop and stalling in-flight SSE streams. Fix: (a) memoize the
last-good keychain label and try it first (orderLabelsLastGoodFirst); (b) a 30s TTL cache of the
read (createTtlCache). This does NOT reintroduce the #146 forever-memoized regression: the TTL
bounds only how often we re-READ the keychain; resolveSpawnToken() still applies the 5-min expiry
gate (isTokenExpiring) to the CACHED creds on EVERY use, so a token expiring within the window is
still rejected → real-HOME fallback. Call sites stay synchronous (no async conversion).

F6 (LOW) — memoized isolation decision goes stale; /health could misreport.
getSpawnHomeMode() memoized the isolated/real-home decision forever: credentials appearing after
startup never enabled isolation; deleting ~/.ocp/spawn-home at runtime ENOENT'd every isolated
spawn until restart; during an expiry stint /health reported isolated:true while spawns ran real-
HOME. Fix: re-evaluate the decision per spawn (cheap now that F5 caches the keychain read);
ensureSpawnHome() re-verifies + re-prepares the scratch dir per isolated spawn; and /health now
reports the EFFECTIVE decision (token presence AND expiry gate). The /health field set is
UNCHANGED — no field added/removed/renamed — only the values are made truthful.

Alignment:
- Class: Not a wire/endpoint change for the spawn-token layer + Class B (B.2) for /health.
- cli.js citation: DECLARED ABSENT. cli.js does NOT perform OCP's spawn-token resolution, HOME
  isolation, keychain caching, or fallback serialization — these are proxy-internal process
  concerns with no cli.js analogue, so no Class A cli.js:NNNN citation exists or is required
  (ALIGNMENT.md Rule 2: this is proxy infra, not an invented forwarded endpoint/header/body).
- The /health change is authorized by ADR 0006 (grandfathered B.2 as of v3.16.4) and is a
  behaviour-preserving contract change: same fields, truthful values.
- OCP still NEVER performs a refresh_token grant itself — that property is preserved; the fix only
  serializes/gates reads of a token refreshed by the spawned or real claude (#112).
- No new endpoint, header, or request/response field. alignment.yml blacklist unaffected.

Tests: extracted the pure primitives to lib/spawn-auth.mjs and added 11 unit tests (mutex
serialization order + idempotent release; TTL cache freshness + null-miss; expiry gate; label
ordering; and the combined invariant that the TTL cache respects the expiry gate). node --check
clean; 249 tests pass (238 + 11).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(server): drain F3 fallback queue immediately — invalidate F5 keychain cache before the serialized re-check

Follow-up to the F3/F5/F6 fix (independent-review observation). F5's 30s keychain TTL cache could
make F3's post-refresh re-check see the stale (expiring) cached creds for up to ~30s, so a waiter
admitted right after the prior real-HOME holder's claude refreshed the keychain would needlessly
fall back to real HOME again instead of proceeding ISOLATED. Serialization safety was never at risk
(still one real-HOME spawn at a time, no double-refresh); only the drain-to-fast-path optimization
lagged.

Fix: invalidateKeychainReadCache() clears the F5 TTL cache; resolveSpawnDecision() calls it under
the fallback mutex, immediately before the re-check, so the admitted waiter reads FRESH keychain
state and drains to the isolated fast path at once. The extra keychain read happens only on the rare
real-HOME fallback path and only under the mutex (serialized, one at a time).

Alignment: unchanged from the parent commit — proxy-internal keychain/HOME-isolation process logic,
no cli.js analogue (cli.js citation DECLARED ABSENT, ALIGNMENT.md Rule 2), no endpoint/header/body,
/health shape unchanged, OCP still never performs a refresh_token grant itself (#112). 249 tests pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:50:52 +10:00
31e5a44099 fix(tui): scope session prefix + reap/kill-server to this instance's port (F7) (#148)
Audit finding F7 (LOW): lib/tui/session.mjs hardcoded SESSION_PREFIX =
"ocp-tui-" as a bare, host-wide constant. The boot-reap and periodic
idle-reap in server.mjs used it to decide which tmux sessions to
kill-session and whether to kill-server (which flushes defunct <claude>
zombies but tears down the WHOLE tmux server, including any live pane).
The coexistence guard only ever spared foreign product prefixes
(olp-tui-*); a SECOND OCP instance on the same host — e.g. a temporary
verification instance stood up alongside production, a real pattern
used during PR #144/#146 verification — was indistinguishable from
"ours" and could have its LIVE sessions reaped/kill-server'd by the
other instance's boot or periodic sweep.

Fix: scope the session-name prefix to this instance's own listen port
(the natural stable per-instance discriminator on one host — two OCP
instances cannot share a port): `ocp-tui-<port>-`. A sibling instance's
`ocp-tui-<otherPort>-*` sessions now fail the own-prefix startsWith
check and fall into the same "othersRemain" bucket as olp-tui-*,
so they are never touched and never used to justify kill-server.

lib/tui/session.mjs:
  - sessionPrefixForPort(port) replaces the bare SESSION_PREFIX export.
  - reapStaleTuiSessions({ tmux, port, includeLegacy }) now requires
    port and computes its own prefix from it.
  - runTuiTurn({ ..., port }) builds the tmux session name from
    sessionPrefixForPort(port) instead of the old bare constant.
  - LEGACY_SESSION_PREFIX / LEGACY_SESSION_NAME_RE (exact
    "ocp-tui-<8-hex>" shape, no port segment) describe the OLD
    pre-fix session-name shape, retained only for the migration below.

Legacy migration rule (chosen + reasoning): a bare-prefix legacy
session cannot be created by any post-fix OCP process, so if one is
seen it is presumed to be an orphaned zombie from THIS instance's own
PRE-fix process generation (left behind across an in-place upgrade),
not a stranger's. reapStaleTuiSessions() therefore accepts an
includeLegacy flag: server.mjs's one-time BOOT reap passes
includeLegacy: true (claims exact-legacy-shape sessions as its own,
enabling cleanup right after an upgrade); the periodic 15-min idle
sweep does NOT set it, so a lingering legacy-shaped session during
steady-state is conservatively treated as foreign and cannot trigger
kill-server on a routine tick. Residual (documented, accepted): a
genuinely-still-running PRE-fix OCP instance coexisting on the host at
the exact moment a new instance boots could have its live legacy
session reaped — the same class of residual risk the audit finding
itself accepts ("no live instance of the new version creates them");
this PR does not regress that scenario, it only removes the far more
common same-version collision that is the actual F7 finding.
LEGACY_SESSION_NAME_RE (`^ocp-tui-[0-9a-f]{8}$`) can never match the
new shape: the new shape always inserts a literal "-" between the
port digits and the 8-hex suffix, which the anchored 8-hex-only legacy
regex cannot satisfy.

server.mjs changes are local TUI session-lifecycle infrastructure
(tmux session naming, boot/periodic reap, kill-server) with no cli.js
wire analogue — verified via `strings` against the compiled claude
CLI 2.1.198 binary (this machine ships cli.js as a Mach-O binary per
ALIGNMENT.md's "OAuth token-host verification" precedent): cli.js
contains only `env.TMUX` detection (whether IT is running inside a
tmux pane) and an unrelated `--remote-control-session-name-prefix`
flag for its own remote-control feature — no session-prefix/reap/
kill-server mechanism of any kind. Per ALIGNMENT.md Rule 2, this is
declared absent: no endpoint, header, request, or response shape
changed; only OCP's own local process-lifecycle bookkeeping. No PORT
literal was hardcoded (CI port-SPOT check) — PORT is threaded through
from the existing server.mjs SPOT (lib/constants.mjs DEFAULT_PORT via
CLAUDE_PROXY_PORT).

test-features.mjs: rewrote the reaper suite's fixture session names to
the new port-scoped shape, added tests for sessionPrefixForPort(),
LEGACY_SESSION_NAME_RE's non-collision with the new shape, a sibling
same-host OCP instance being treated as foreign (F7 regression test),
and the includeLegacy boot-migration behavior (claims legacy zombies,
still spares a sibling instance's port-scoped session).

Verified: node --check server.mjs && node --check lib/tui/session.mjs
&& npm test → 243 passed, 0 failed.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:43:19 +10:00
2922d68842 fix(server): re-resolve -p spawn OAuth token per-spawn, expiry-aware (③ regression) (#146)
The FIX-③ spawn-home isolation memoized the OAuth token at startup. The macOS
keychain access token rotates (~hourly, refreshed by the operator's real claude),
so the startup snapshot went stale and every isolated -p spawn returned upstream
401 'Invalid authentication credentials' — a ~31h Mac-mini outage (PI231/oracle use
static long-lived env tokens, unaffected).

Fix: getSpawnHomeMode() now caches only the isolation DECISION; the token is
re-resolved FRESH per spawn via resolveSpawnToken(), which also returns null when a
known expiry has passed (5-min buffer) so the caller falls back to real HOME — where
the spawned claude refreshes the credential natively and self-heals. OCP still never
refreshes the token itself (a refresh-token grant would consume the single-use token
and log out the operator's real claude — issue #112). Env-token hosts carry no
expiresAt and are never expiry-gated. Infra/process change; no cli.js surface, no new
endpoint/header. 238 tests pass; live-verified sonnet 200 on a temp instance.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
2026-06-26 20:35:21 +10:00
13 changed files with 1499 additions and 132 deletions
+10
View File
@@ -1,5 +1,15 @@
# Changelog
## v3.21.1 — 2026-07-07
Patch release: three bug fixes from an independent concurrency/session-lifecycle audit, each its own PR with a fresh-context reviewer (Iron Rule 10). No new `cli.js` wire behavior, no new endpoint, header, or env var; the `/health` field set is unchanged (only value truthfulness improved).
### Fixed
- **TUI session-scope / boot-reap (#148)** — `lib/tui/session.mjs`'s tmux session prefix is now scoped per-instance by listen port (`ocp-tui-<port>-`) instead of a bare host-wide `ocp-tui-` constant, so a second OCP instance on the same host (e.g. a temporary verification instance) can no longer have its live TUI sessions reaped or `kill-server`'d by another instance's boot/periodic sweep. The one-time boot reap also claims exact-shape legacy `ocp-tui-<8hex>` sessions (pre-fix naming) once, to clean up zombies left behind across an in-place upgrade.
- **`-p` spawn-token mutex + keychain caching (#150)** — the real-HOME token fallback used when the keychain token is within its 5-minute expiry window is now serialized behind a mutex, so concurrent `-p` spawns no longer race the same single-use refresh token against each other (the credential-fork hazard). Added a 30s TTL cache + last-good-label memoization for the keychain read, cutting per-spawn event-loop blocking. The isolation decision (`/health` isolated/real-home reporting) is now re-evaluated per spawn instead of memoized forever, so `/health` no longer misreports a stale decision. New module `lib/spawn-auth.mjs` extracts the pure, unit-testable primitives (mutex, TTL cache, expiry gate, label ordering).
- **Concurrency queue / disconnect handling (#149)** — the shared semaphore now honors a runtime-lowered `maxConcurrent` immediately (previously a decrease was silently ignored until in-flight tasks finished on their own) and wakes queued waiters right away when the limit is raised. Queued `-p`/TUI requests are now linked to the client's HTTP connection via `AbortSignal`; a client that disconnects while queued is spliced out of the queue instead of still spawning `claude` once a slot frees. A singleflight follower whose leader disconnected now retries instead of inheriting a spurious 500, and a queued-then-disconnected request is no longer recorded as a usage failure or logged as an error (quiet disconnect handling).
## v3.21.0 — 2026-06-25
Cleanup + docs release: TUI dead-code removal, docs honesty, and release prep. No new `cli.js` wire behavior; the default path (`CLAUDE_TUI_MODE` unset) is byte-for-byte unchanged.
+5
View File
@@ -894,6 +894,10 @@ node ~/ocp/scripts/sync-openclaw.mjs
This is read-only at startup; the warning never blocks the gateway from running.
### A TUI session vanished right after upgrading OCP
If you ran a pre-3.21.1 OCP instance and a post-3.21.1 instance on the same host at the same time during an upgrade, the new instance's one-time boot reap can, once, kill an old-format (`ocp-tui-<8hex>`) live TUI session belonging to the still-running old instance — restart the affected session (`ocp restart` or re-run your TUI turn) and it will come back under the new instance's port-scoped naming.
### OpenClaw shows old models after `ocp update` (v3.10→v3.11 only)
One-time bootstrap quirk for the v3.10.0 → v3.11.0 jump only — the running shell had the old `cmd_update` cached. Run once manually:
@@ -954,6 +958,7 @@ See [Subscription-pool (TUI) mode](#subscription-pool-tui-mode) and ADR 0007 PR-
| `OCP_TUI_CWD` | `$HOME/.ocp-tui/work` | (TUI-mode) Scratch working directory where interactive claude sessions run. Transcripts land under `<HOME>/.claude/projects/<encoded-cwd>/`. Created automatically. |
| `OCP_TUI_HOME` | *(auto)* | (TUI-mode) `HOME` claude runs under. **When unset, OCP picks it for you:** if `CLAUDE_CODE_OAUTH_TOKEN` is set → a **credential-isolated** scratch home `$HOME/.ocp-tui/home` (no `credentials.json`, env-token auth — **recommended**); if no env token → the operator's real home (legacy shared `credentials.json`). Setting this to an **explicit** path overrides the auto-default. The credential handling at that path still follows the env token: **with** the env token it is credential-free (env-token auth, no `credentials.json` written); **without** the env token (and the path ≠ real home) it uses the legacy symlinked-credentials scratch mode, which carries the credential-fork caveat — see ADR 0007. |
| `OCP_TUI_ENTRYPOINT` | `cli` | (TUI-mode) Billing-classifier labeling: `cli` (default) pins `cc_entrypoint=cli` deterministically; `auto` lets claude self-classify via TTY detection; `off` leaves the inherited env untouched. Honest only when the spawn is a genuine interactive PTY — see ADR 0007. |
| `OCP_TUI_EFFORT` | `low` | (TUI-mode) Effort level passed to the interactive `claude` as an explicit `--effort` flag: `low` (default), `medium`, `high`, `xhigh`, `max`, or `inherit` to omit the flag (the pre-flag behaviour: the pane inherits a HOME-dependent effort — the operator's `~/.claude/settings.json` `effortLevel` in real-home mode, claude's built-in default in env-token scratch mode). Explicit `low` cuts measured TTFT p50 by ~40% and collapses run-to-run variance ~15× versus an inherited `xhigh` (see `docs/plans/2026-07-13-tui-latency/`); proxied requests rarely benefit from extended thinking. Banner-verified to stay on the subscription pool (`· Claude Max`). An invalid value logs a warning and falls back to `low`. |
| `OCP_TUI_MAX_CONCURRENT` | `2` | (TUI-mode) Max concurrent interactive TUI turns. **Independent** of `CLAUDE_MAX_CONCURRENT` (which bounds the `-p`/stream-json path; TUI never uses it). A TUI turn is heavy (per-request cold-boot of tmux+claude + up to `CLAUDE_TUI_WALLCLOCK_MS` wallclock), so the default is low to keep small hosts (e.g. a Pi 4) alive under a burst. Excess turns **queue** (bounded); a full queue yields a 503. See ADR 0007 PR-B amendment. |
| `OCP_SKIP_AUTH_TEST` | *(unset)* | When `=1`, skip the `claude -p` auth probe during `setup.mjs`. After 2026-06-15 this probe draws from the Agent SDK credit pool; set this to avoid burning a metered credit on re-installs or `ocp update` runs. Auth is validated at the first real request. |
| `OCP_TUI_FULL_TOOLS` | *(unset)* | (TUI-mode, **single-user only**) When `=1`, grant the interactive session the **same tool surface as the `-p` path** — `--allowedTools` (+ optional `--mcp-config`, read from `CLAUDE_ALLOWED_TOOLS` / `CLAUDE_MCP_CONFIG`) — instead of the default MCP-walled, built-in-tools-only set. Lets a trusted single-operator TUI deployment run a **tool-using / MCP agent** (e.g. an OpenClaw assistant) on the subscription pool. Safe because TUI **refuses to boot under `AUTH_MODE=multi`** (hard exit) — no guest key can ever reach the TUI path, so this gate cannot expose tools to an untrusted caller. (Under `AUTH_MODE=shared` + `OCP_TUI_ALLOW_LAN=1`, anyone holding the single shared key reaches it — that is the existing TUI trust model, unchanged.) Note: `--dangerously-skip-permissions` / `CLAUDE_SKIP_PERMISSIONS` is **not** supported for TUI — claude v2.1.x shows an interactive bypass-acceptance screen in headless tmux that cannot be answered, bricking the pane. Use scratch-home `settings.json` `additionalDirectories` instead. See [Subscription-pool (TUI) mode](#subscription-pool-tui-mode) and ADR 0007. |
+190
View File
@@ -0,0 +1,190 @@
# TUI-mode latency: measured floor, and the four things worth fixing
**Date**: 2026-07-13
**Status**: findings + backlog (no code changed yet)
**Measured on**: Mac mini / macOS 26.5.2 / Claude Code **v2.1.207** / Sonnet 5 / Claude Max subscription / **real-home mode** (no `CLAUDE_CODE_OAUTH_TOKEN`, no `OCP_TUI_HOME` in the service env)
**Evidence**: [`measurements.jsonl`](measurements.jsonl) — **n=15** (3 configs × 5) · banner captures [`billing-banner.txt`](billing-banner.txt) · harness [`floor.sh`](floor.sh)
## Why this exists
An external consumer (the 知音 AI project) benchmarked OCP's prompt path and measured
**TTFT p50 ≈ 3032 s**, and excluded OCP as a backend on that basis. That number is real,
but it is *not* the model being slow — this document decomposes where the 30 seconds
actually go, and what OCP can do about it.
**The harness deliberately does not go through OCP.** It spawns `tmux` + `claude` directly
(session prefix `zhiyin-floor-`, never `ocp-tui-*`) and polls `tmux capture-pane` for
incremental render, so it measures the **true first-token time** of the underlying
subscription path — the floor OCP could reach if it were perfect.
---
## Measurements
All rows in [`measurements.jsonl`](measurements.jsonl); every number below is recomputable from it.
| Config | n | boot→input-ready (median) | **TTFT (median)** | TTFT range | full answer (median) |
|---|---|---|---|---|---|
| baseline (inherits global `effortLevel: xhigh`) | 5 | 1.07 s | **10.35 s** | 8.32 17.19 s | 11.32 s |
| **`--effort low`** | 5 | 1.03 s | **6.17 s** | **5.87 6.44 s** | 9.98 s |
| `--bare` | 5 | 0.44 s | **no answer at all** (5/5 `ttft_ms: -1`) | — | — |
> **Not from this harness**: the direct Anthropic API reference figure (TTFT 0.841.64 s, n=2)
> comes from the 知音 AI project's own smoke test, not from `measurements.jsonl`. It is quoted
> only to size the gap; do not look for it in the evidence file.
### Where the 30 seconds go
```
~1.0 s spawn → claude's input bar is ready ← NOT the bottleneck
~6-10 s true TTFT (first token rendered in the pane)
~20 s ████ waiting for the whole turn to finish ████ ← this is the 30s
```
`runTuiTurn` blocks on the native transcript until a terminal event (`lib/tui/session.mjs`
"Block on the native transcript … until terminal"; `readTuiTranscript` in
`lib/tui/transcript.mjs`; ADR 0007 step 4) — i.e. it waits for the **entire turn** to complete
before returning anything. There is no streaming path. The ~20 s delta between this harness's
real TTFT and OCP's reported 3032 s is exactly that.
---
## ⚠️ Blocking constraint: `--bare` silently drops you off the subscription pool
Captured live ([`billing-banner.txt`](billing-banner.txt)) — the startup banner is the **only**
reliable indicator:
```
[] | Sonnet 5 with xhigh effort · Claude Max
[--effort low] | Sonnet 5 with low effort · Claude Max
[--bare] | Sonnet 5 with xhigh effort · API Usage Billing ← ❌
```
`--bare` ("skip hooks, LSP, plugin…") **also skips the subscription-credential resolution
path**. It really does cut boot to 0.430.45 s — but you are no longer on the subscription,
which defeats the entire purpose of TUI mode (ADR 0007 exists solely to reach the
subscription pool).
**The failure is silent.** All 5 `--bare` samples reached input-ready (boot 0.430.45 s), were
sent the prompt, and then produced **no answer at all** — 60 s timeout, no error, no crash, the
pane simply never rendered a token (the API-billing account had no credit balance). Nothing in
the transcript or the exit status reveals this.
**Anyone changing spawn flags must diff the banner line before and after.**
---
## Backlog — four items, ranked by value ÷ effort
### 1. Pass `--effort` explicitly on spawn — **do this first**
`buildTuiCmd` (`lib/tui/session.mjs`) does not pass `--effort``grep -rn -- "--effort\|effortLevel" lib/ server.mjs`
returns zero hits. What the pane's `claude` ends up using therefore depends on **which HOME mode
`resolveTuiHome()` picked**:
| mode | HOME | effort the pane gets |
|---|---|---|
| **real-home** (legacy default — *current* service config: no `CLAUDE_CODE_OAUTH_TOKEN`, no `OCP_TUI_HOME`) | `~` | **inherits the operator's `~/.claude/settings.json` → `effortLevel: xhigh` on this host** |
| env-token scratch (`CLAUDE_CODE_OAUTH_TOKEN` set — the direction #146/#150 pushed) | `~/.ocp-tui/home` | that settings.json contains only `permissions.additionalDirectories`; `prepareTuiHome()` never writes `effortLevel`**claude's built-in default** |
**Scope note**: TUI mode is currently *off* on this host (`CLAUDE_TUI_MODE=false`; `/health`
`"tui": {"enabled": false}`), so live traffic takes the `-p` path today. The statement below is
about what happens **when TUI mode is enabled**.
On the current HOME config, **every TUI request would run extended thinking** — pure waste
for the typical "generate this JSON" request, and it makes latency depend on an unrelated global
setting the operator may have changed for their own interactive use. And the mode split means
the effort level silently changes if the operator ever switches to env-token mode.
**Passing `--effort` explicitly fixes both problems at once.**
- **Effect (real-home, measured)**: TTFT p50 **10.35 s → 6.17 s (40 %)**, and the spread
collapses from 8.3217.19 s to **5.876.44 s**. For a proxy, the variance reduction matters
more than the median.
- **Cost**: one flag. Suggested: a new `OCP_TUI_EFFORT` env var (default `low`), documented in
README § "Environment Variables" per `release_kit.new_feature_doc_expectations`.
- **Risk**: none — banner confirms it stays on `Claude Max` (see `billing-banner.txt`).
- ⚠️ Do **not** reach for `--bare` to shave boot: see above.
### 2. Real streaming instead of blocking on turn-terminal — **the big one (~20 s)**
Today `runTuiTurn` blocks on the transcript until the turn is *finished*. The pane is already
rendering tokens incrementally the whole time — this harness proves you can observe first token
at ~6 s by polling `tmux capture-pane`.
- **Effect**: turns a 30 s wall into a ~6 s TTFT with progressive output; enables SSE streaming
on the OCP endpoint instead of a single blob at the end.
- **Cost**: real work. Pane capture is ANSI/redraw-based and lossy for exact text (wrapping,
scrollback, spinner lines). Two candidate sources: (a) incremental reads of the transcript
JSONL, (b) `capture-pane` diffing with a stable start marker. (a) is much cleaner **if it
holds**.
- **Prereq spike (do this before designing anything)**: does the transcript JSONL grow *during*
a turn, or only at the end? If only at the end, (a) is dead and you are stuck with (b).
### 3. Warm pane pool — ~1 s
Every request spawns a fresh tmux session + `claude` (`randomUUID()` + `new-session`, then
`kill-session` in `finally`; `grep -rn "pool\|warm\|reuse" lib/tui/*.mjs` → zero hits). Boot to
input-ready is ~1.0 s, paid on every request. A pool of pre-booted panes (single-use, replaced in
the background) amortizes it to zero for any workload below the pool refill rate.
- **Effect**: 1.0 s.
- **Cost**: moderate; interacts with the session reaper and the per-port prefix scoping added in
#148 — pooled panes must not look like zombies to the sweep.
- Lower priority than #1 and #2: it is the smallest slice.
### 4. Trim the prefill — **probably not worth it; know the floor**
After #1#3, the floor is **~6 s**, and it does not go lower. `claude` always injects the full
Claude Code system prompt + tool definitions (thousands to tens of thousands of prefill tokens)
regardless of what you ask it. `--exclude-dynamic-system-prompt-sections` exists and may shave
some of it — **unmeasured**; worth one spike, but do not expect to reach the direct API's
~1 s.
**Consequence to accept, and to state in the README**: even fully optimized, TUI mode has a
**~6 s TTFT floor**, so it cannot serve real-time / interactive-latency consumers. It remains
appropriate for batch, background, and cost-insensitive-latency use. The 知音 AI project
excluded it on this basis (their prompt-latency budget is 24 s) *independently* of the ToS
question already documented in the README.
---
## Reproduction
```bash
# harness never touches OCP's :3456 service or ocp-tui-* sessions, and never kill-server
bash docs/plans/2026-07-13-tui-latency/floor.sh 5 # baseline
TAG=effort-low EXTRA_ARGS="--effort low" bash .../floor.sh 5 # 40 %
TAG=bare EXTRA_ARGS="--bare" bash .../floor.sh 5 # the trap
# billing-pool check for ANY spawn-flag change — the banner is the only source of truth
tmux new-session -d -s probe -x 200 -y 50 -c "$HOME" \
"claude --model claude-sonnet-5 --session-id $(uuidgen) <your-flags-here>"
sleep 6; tmux capture-pane -p -t probe | grep -E "Claude Max|API Usage Billing"
tmux kill-session -t probe
```
## Interaction with OCP while the harness runs
- **Kill direction is safe both ways**: `reapStaleTuiSessions()` only `kill-session`s names
matching `ocp-tui-<port>-`, which `zhiyin-floor-*` never matches; and the harness only
`kill-session`s its own single session — it contains **no `kill-server`**.
- **One benign interaction** (only when TUI mode is enabled — the reap tick is itself gated on
`TUI_MODE`): OCP's periodic `kill-server` (zombie reaping) is gated on
`othersRemain`*any* foreign-prefixed tmux session suppresses it. So while the harness is
running, that sweep is skipped. This is the coexistence guard working as designed; it resumes
on the next tick.
## Harness caveats (stated so the numbers are not over-trusted)
- **n=5 per config**, single host, single model (Sonnet 5), single prompt size (~1850 tokens).
Enough to separate 6 s from 10 s from 30 s; **not** enough for a p95.
- TTFT is "marker visible in `capture-pane`", which includes tmux render latency (small, but
nonzero) — it is an upper bound on the true first-token time.
- **The harness's readiness marker is not OCP's.** `floor.sh` waits for `│ >||Try "`; OCP's
`tuiInputReady()` matches `/\? for shortcuts/`. These are different events, so the ~1.0 s
boot figure is **not** directly comparable to OCP's `BOOT_MS` gate (default cap 4000 ms). It
does not affect the conclusions (1 s ≪ 6 s TTFT), but it is not apples-to-apples.
- The first version of this harness reported TTFT **0.08 s** — a false positive: the prompt
literally contained the marker string it was grepping for, so the match fired the instant the
prompt was pasted. Fixed by describing the marker instead of spelling it. **The script exited 0
and "successfully" produced 5 samples both times** — exit status proves nothing here.
@@ -0,0 +1,3 @@
[] | ▝▜█████▛▘ Sonnet 5 with xhigh effort · Claude Max
[--effort low] | ▝▜█████▛▘ Sonnet 5 with low effort · Claude Max
[--bare] | ▝▜█████▛▘ Sonnet 5 with xhigh effort · API Usage Billing
+128
View File
@@ -0,0 +1,128 @@
#!/usr/bin/env bash
# OCP TUI-mode latency floor harness — see README.md in this directory.
#
# 目的:回答一个问题——如果把 OCP 现有的两个已知开销砍掉
# (a) 每请求 spawn + boot(可用预热进程池消除)
# (b) 假流式(等 turn_duration 才返回,可用增量读 pane 消除)
# 之后,订阅池路径的**真实 TTFT 地板**是多少?
#
# 判据:地板 ≤ 4s → OCP 作为"省钱选项"可行;> 8s → 死透,不再讨论。
#
# 红线:
# - 不经过生产 OCP 服务(:3456)—— 直接起 tmux+claudeOCP 进程零干扰
# - tmux session 前缀用 zhiyin-floor-**不是** ocp-tui-),避免被 OCP 的
# reaper 当成自己的会话杀掉,也避免我们杀到它的
# - 用 real HOME(凭据)—— scratch HOME + symlink 凭据会 fork OAuth 导致 401
# (见跨机记忆 tui_scratch_home_credential_fork
set -uo pipefail
N=${1:-5}
MODEL=${MODEL:-claude-sonnet-5}
EXTRA_ARGS=${EXTRA_ARGS:-} # 额外 CLI 参数(如 --effort low --bare
TAG=${TAG:-baseline}
OUT=${OUT:-$(dirname "$0")/measurements.jsonl}
PROMPT_FILE=$(mktemp)
PREFIX="zhiyin-floor"
mkdir -p "$(dirname "$OUT")"
# ── 构造提示:~2000 token 的假会议转写 + 明确的起始标记 ────────────────
# 单行(多行会在 tmux send-keys 时提前触发 Enter
build_prompt() {
local seg="Speaker A said the quarterly pipeline is tracking behind plan and the enterprise segment needs a different motion. Speaker B replied that the current onboarding flow loses roughly a third of trial accounts before the first integration is complete. They debated whether the fix belongs in product or in customer success. "
local body=""
for _ in $(seq 1 22); do body+="$seg"; done
printf '%s' "You are a real-time meeting copilot. Meeting transcript so far: $body --- Task: produce ONE prompt card as compact JSON with keys: points (array of 3 short Chinese bullet points), keyline (one English sentence the user can read aloud). IMPORTANT: your reply MUST begin with three hash characters immediately followed by the uppercase word CARD (no space between them), then the JSON. No preamble, no markdown fences." > "$PROMPT_FILE"
}
build_prompt
PROMPT_CHARS=$(wc -c < "$PROMPT_FILE" | tr -d ' ')
now_ms() { python3 -c 'import time;print(int(time.time()*1000))'; }
echo "配置: $TAG 参数: [$EXTRA_ARGS]"
echo "模型: $MODEL 样本: $N 提示长度: ${PROMPT_CHARS} chars (≈$((PROMPT_CHARS/4)) token)"
echo "输出: $OUT"
echo
for i in $(seq 1 "$N"); do
SESS="${PREFIX}-$$-$i"
SID=$(uuidgen)
# ── 冷启动:spawn + 等输入框就绪 ─────────────────────────────────
T_SPAWN=$(now_ms)
tmux new-session -d -s "$SESS" -x 200 -y 50 \
-e CLAUDE_CODE_DISABLE_CLAUDE_MDS=1 \
-e CLAUDE_CODE_DISABLE_AUTO_MEMORY=1 \
-c "$HOME" \
"claude --model $MODEL --session-id $SID --strict-mcp-config --disallowedTools 'mcp__*' $EXTRA_ARGS" 2>/dev/null
if [ $? -ne 0 ]; then echo "[$i] tmux spawn 失败,跳过"; continue; fi
# 轮询输入框就绪(claude TUI 的输入提示符)
READY=0
for _ in $(seq 1 150); do # 上限 15s
PANE=$(tmux capture-pane -p -t "$SESS" 2>/dev/null || true)
if grep -qE '│ >||Try "' <<<"$PANE"; then READY=1; break; fi
sleep 0.1
done
T_READY=$(now_ms)
BOOT_MS=$((T_READY - T_SPAWN))
if [ "$READY" -ne 1 ]; then
echo "[$i] 启动超时(${BOOT_MS}ms),pane 末 3 行:"
tmux capture-pane -p -t "$SESS" 2>/dev/null | tail -3 | sed 's/^/ /'
tmux kill-session -t "$SESS" 2>/dev/null
continue
fi
# ── 热态:粘提示 → 回车 → 量首 token ─────────────────────────────
tmux send-keys -t "$SESS" -l "$(cat "$PROMPT_FILE")" 2>/dev/null
sleep 0.4 # 让粘贴落地(OCP 用 400ms 轮询粒度)
T0=$(now_ms)
tmux send-keys -t "$SESS" Enter 2>/dev/null
TTFT_MS=-1
for _ in $(seq 1 600); do # 上限 60s
if tmux capture-pane -p -t "$SESS" 2>/dev/null | grep -q '###CARD'; then
TTFT_MS=$(( $(now_ms) - T0 )); break
fi
sleep 0.1
done
# ── 完整回答:pane 连续 2s 不再变化 ──────────────────────────────
COMPLETE_MS=-1
if [ "$TTFT_MS" -ge 0 ]; then
LAST=""; STABLE=0
for _ in $(seq 1 900); do # 上限 90s
CUR=$(tmux capture-pane -p -t "$SESS" 2>/dev/null | cksum)
if [ "$CUR" = "$LAST" ]; then
STABLE=$((STABLE+1))
[ "$STABLE" -ge 20 ] && { COMPLETE_MS=$(( $(now_ms) - T0 - 2000 )); break; }
else
STABLE=0; LAST="$CUR"
fi
sleep 0.1
done
fi
printf '{"i":%d,"tag":"%s","model":"%s","extra_args":"%s","prompt_chars":%s,"boot_ms":%d,"ttft_ms":%d,"complete_ms":%d}\n' \
"$i" "$TAG" "$MODEL" "$EXTRA_ARGS" "$PROMPT_CHARS" "$BOOT_MS" "$TTFT_MS" "$COMPLETE_MS" | tee -a "$OUT"
tmux kill-session -t "$SESS" 2>/dev/null
sleep 1
done
rm -f "$PROMPT_FILE"
echo
echo "=== 汇总 ==="
python3 - "$OUT" <<'EOF'
import json,sys,statistics
rows=[json.loads(l) for l in open(sys.argv[1]) if l.strip()]
ok=[r for r in rows if r['ttft_ms']>=0]
if not ok: print("无有效样本"); sys.exit()
def s(k):
v=[r[k] for r in ok if r[k]>=0]
return f"n={len(v)} 中位={statistics.median(v)/1000:.2f}s 最小={min(v)/1000:.2f}s 最大={max(v)/1000:.2f}s" if v else "无"
print(f" 冷启动 boot : {s('boot_ms')} ← 预热进程池可完全消除")
print(f" TTFT(首 token : {s('ttft_ms')} ★ 这就是地板")
print(f" 完整回答 : {s('complete_ms')}")
print(f"\n 失败样本: {len(rows)-len(ok)}/{len(rows)}")
EOF
@@ -0,0 +1,15 @@
{"i": 1, "tag": "effort-low", "model": "claude-sonnet-5", "extra_args": "--effort low", "prompt_chars": 7451, "boot_ms": 1077, "ttft_ms": 6172, "complete_ms": 9929}
{"i": 2, "tag": "effort-low", "model": "claude-sonnet-5", "extra_args": "--effort low", "prompt_chars": 7451, "boot_ms": 1026, "ttft_ms": 6160, "complete_ms": 9996}
{"i": 3, "tag": "effort-low", "model": "claude-sonnet-5", "extra_args": "--effort low", "prompt_chars": 7451, "boot_ms": 1010, "ttft_ms": 6437, "complete_ms": 9977}
{"i": 4, "tag": "effort-low", "model": "claude-sonnet-5", "extra_args": "--effort low", "prompt_chars": 7451, "boot_ms": 1033, "ttft_ms": 5872, "complete_ms": 9944}
{"i": 5, "tag": "effort-low", "model": "claude-sonnet-5", "extra_args": "--effort low", "prompt_chars": 7451, "boot_ms": 1154, "ttft_ms": 6387, "complete_ms": 9993}
{"i":1,"tag":"baseline","model":"claude-sonnet-5","extra_args":"","prompt_chars":7451,"boot_ms":1300,"ttft_ms":8321,"complete_ms":9939}
{"i":2,"tag":"baseline","model":"claude-sonnet-5","extra_args":"","prompt_chars":7451,"boot_ms":1070,"ttft_ms":10347,"complete_ms":11320}
{"i":3,"tag":"baseline","model":"claude-sonnet-5","extra_args":"","prompt_chars":7451,"boot_ms":911,"ttft_ms":13061,"complete_ms":15163}
{"i":4,"tag":"baseline","model":"claude-sonnet-5","extra_args":"","prompt_chars":7451,"boot_ms":1441,"ttft_ms":9981,"complete_ms":11066}
{"i":5,"tag":"baseline","model":"claude-sonnet-5","extra_args":"","prompt_chars":7451,"boot_ms":1036,"ttft_ms":17189,"complete_ms":17985}
{"i":1,"tag":"bare","model":"claude-sonnet-5","extra_args":"--bare","prompt_chars":7451,"boot_ms":429,"ttft_ms":-1,"complete_ms":-1}
{"i":2,"tag":"bare","model":"claude-sonnet-5","extra_args":"--bare","prompt_chars":7451,"boot_ms":437,"ttft_ms":-1,"complete_ms":-1}
{"i":3,"tag":"bare","model":"claude-sonnet-5","extra_args":"--bare","prompt_chars":7451,"boot_ms":444,"ttft_ms":-1,"complete_ms":-1}
{"i":4,"tag":"bare","model":"claude-sonnet-5","extra_args":"--bare","prompt_chars":7451,"boot_ms":446,"ttft_ms":-1,"complete_ms":-1}
{"i":5,"tag":"bare","model":"claude-sonnet-5","extra_args":"--bare","prompt_chars":7451,"boot_ms":441,"ttft_ms":-1,"complete_ms":-1}
+16 -2
View File
@@ -382,11 +382,25 @@ export function getCacheStats() {
// Per ADR 0005 / spec D4: in-process scope only (single Node process per host).
const inflightMap = new Map();
export function singleflight(hash, fn) {
// `retryIf` (optional, audit finding M1): a predicate applied on the FOLLOWER path only.
// When a follower joins an existing flight and the shared promise rejects with an error for
// which retryIf(err) is true (in practice: the LEADER's client disconnected while queued —
// an error that is personal to the leader, not a verdict about the upstream), the follower
// does NOT inherit that rejection. Instead it re-enters singleflight with its OWN fn: it
// either becomes the new leader (the map entry is already deleted — see the finally below,
// which runs before any follower's catch because it is attached upstream of the promise the
// followers await) or joins a flight another retrying follower just created. The leader's
// own rejection is never retried here — its error belongs to it (leader path returns the
// bare promise). Callers that pass no retryIf get the exact pre-M1 share-everything behavior.
export function singleflight(hash, fn, retryIf) {
const existing = inflightMap.get(hash);
if (existing) {
existing.requesters++;
return existing.promise;
if (!retryIf) return existing.promise;
return existing.promise.catch((err) => {
if (!retryIf(err)) throw err;
return singleflight(hash, fn, retryIf);
});
}
// Wrap fn() in Promise.resolve().then() so synchronous throws don't escape.
const promise = Promise.resolve().then(fn).finally(() => {
+67
View File
@@ -0,0 +1,67 @@
// Pure, dependency-injected primitives for the `-p` spawn-token resolution + HOME-isolation
// layer. Extracted from server.mjs (findings F3 / F5 / F6, 2026-07-07) so the concurrency,
// caching and expiry logic is unit-testable WITHOUT booting the server or mocking execFileSync /
// child_process.spawn / fs. server.mjs owns all I/O (macOS keychain exec, process spawn, fs);
// this module owns only pure decision logic.
//
// ALIGNMENT NOTE: none of this touches the OAuth wire machinery (no endpoint / header / body).
// OCP still NEVER performs a refresh_token grant itself — these helpers only READ + GATE a token
// that some other process (the operator's real claude, or a spawned claude under the real HOME)
// refreshes. That property is load-bearing (issue #112) and preserved.
// Promise-chain mutex. `acquire()` resolves to a `release()` fn; the NEXT `acquire()` does not
// resolve until the current holder calls its `release()`. Serializes async critical sections
// without busy-waiting. release() is idempotent.
export function createSerialMutex() {
let tail = Promise.resolve();
return {
acquire() {
let release;
const gate = new Promise((r) => { release = r; });
const prev = tail;
tail = tail.then(() => gate);
// Hand the caller its release fn only after the previous holder has released.
return prev.then(() => {
let released = false;
return function releaseMutex() { if (!released) { released = true; release(); } };
});
},
};
}
// Short-TTL memo. `get(produce, now)` returns the cached value while `now - storedAt < ttlMs`,
// otherwise calls `produce()` and re-stores. A miss that produces null/undefined is STILL stored
// (so a genuinely-absent source is not re-probed on every call within the TTL window). `now` is
// injectable for testing.
export function createTtlCache({ ttlMs }) {
let value;
let at = -Infinity;
let has = false;
return {
get(produce, now = Date.now()) {
if (has && now - at < ttlMs) return value;
value = produce();
at = now;
has = true;
return value;
},
clear() { has = false; value = undefined; at = -Infinity; },
};
}
// Pure expiry gate. Returns true when `creds` carries a known expiry that is at/within `bufferMs`
// of `now`. Creds WITHOUT `expiresAt` (e.g. long-lived env tokens) are never treated as expiring.
// This gate is applied to the CACHED creds on EVERY use — which is precisely why a short-TTL
// keychain cache (createTtlCache) cannot reintroduce the #146 forever-stale-token regression: the
// cache bounds how often we re-READ the keychain, but the expiry decision is recomputed per use.
export function isTokenExpiring(creds, now = Date.now(), bufferMs = 300000) {
return !!(creds && creds.expiresAt && now + bufferMs >= creds.expiresAt);
}
// Order candidate keychain labels so the last-known-good label is tried first (avoids the
// wrong-label miss that doubles the `security` exec count on the hot path). Pure: performs no
// read. Returns a fresh array; input is not mutated.
export function orderLabelsLastGoodFirst(labels, lastGood) {
if (!lastGood || !labels.includes(lastGood)) return labels.slice();
return [lastGood, ...labels.filter((l) => l !== lastGood)];
}
+61 -11
View File
@@ -20,6 +20,15 @@
//
// Pure + importable so test-features.mjs can assert the bound directly (no server boot).
// Thrown by acquire() when the caller-supplied AbortSignal fires before a slot was granted
// (audit finding F2 — a client that disconnects while queued must never receive a slot; the
// queue entry is spliced out, not just flagged, so `queued` accounting stays exact). Distinct
// `name` lets callers (server.mjs acquireClaudeSlot) tell "client went away" apart from
// "queue is full" without string-matching the message.
export class SemaphoreAbortError extends Error {
constructor(message) { super(message); this.name = "SemaphoreAbortError"; }
}
export class TuiSemaphore {
// limit: max concurrent slots. maxQueue: max waiters before run() rejects with backpressure.
constructor(limit, { maxQueue } = {}) {
@@ -34,9 +43,30 @@ export class TuiSemaphore {
get inflight() { return this._inflight; }
get queued() { return this._waiters.length; }
// Runtime-adjust the concurrency limit (audit finding F1 — a PATCH /settings maxConcurrent
// change must actually take effect, not just be ignored until every currently-inflight task
// happens to finish). Lowering the limit is handled lazily by release() (see below) — it
// simply stops re-granting until inflight drains under the new, lower limit. Raising the
// limit has immediate headroom, so we wake as many queued waiters as now fit.
setLimit(limit) {
this.limit = Math.max(1, parseInt(limit, 10) || 1);
while (this._inflight < this.limit && this._waiters.length > 0) {
const next = this._waiters.shift();
this._inflight++;
next();
}
}
// Acquire a slot. Resolves once a slot is free (immediately if under the limit, otherwise
// when an in-flight task releases). Rejects synchronously-ish if the wait queue is full.
acquire() {
// `signal` (optional AbortSignal, F2) lets the caller cancel a QUEUED wait — e.g. wired to
// a client's socket "close" event so a request that disconnects before a slot is granted
// is removed from the queue instead of eventually being handed a slot for a dead socket.
// If `signal` is already aborted, reject immediately without ever touching the queue.
acquire(signal) {
if (signal?.aborted) {
return Promise.reject(new SemaphoreAbortError("acquire aborted before requesting a slot"));
}
if (this._inflight < this.limit) {
this._inflight++;
return Promise.resolve();
@@ -46,24 +76,44 @@ export class TuiSemaphore {
`tui_queue_full: TUI concurrency limit (${this.limit}) reached and wait queue ` +
`(${this.maxQueue}) is full`));
}
return new Promise((resolve) => { this._waiters.push(resolve); });
return new Promise((resolve, reject) => {
let waiter; // the FIFO entry — captured so onAbort can find + splice exactly this one
const onAbort = () => {
const idx = this._waiters.indexOf(waiter);
if (idx === -1) return; // already granted a slot (shifted out by release()/setLimit) — too late to cancel
this._waiters.splice(idx, 1); // remove, not just flag — keeps `queued` accounting exact
reject(new SemaphoreAbortError("acquire aborted while queued"));
};
waiter = () => {
signal?.removeEventListener("abort", onAbort);
resolve();
};
signal?.addEventListener("abort", onAbort, { once: true });
this._waiters.push(waiter);
});
}
// Release a slot. If a waiter is queued, hand the slot directly to it (inflight stays
// constant across the handoff); otherwise decrement.
// Release a slot. Always frees the caller's own slot first, then re-grants it to the next
// waiter ONLY if the (post-decrement) inflight count is still under the current limit (F1
// fix). This is what makes a runtime-lowered limit actually bite: if the limit was lowered
// while over-subscribed, releases stop re-granting and inflight drains toward the new limit
// instead of a freed slot being handed straight back out at the old, higher occupancy.
release() {
const next = this._waiters.shift();
if (next) {
next(); // the woken waiter already "owns" the slot — inflight unchanged
} else if (this._inflight > 0) {
this._inflight--;
if (this._inflight > 0) this._inflight--;
if (this._inflight < this.limit) {
const next = this._waiters.shift();
if (next) {
this._inflight++;
next();
}
}
}
// Run fn() under one slot. Releases in a finally so a throw (PR-A's honesty gates,
// wallclock truncation, paste-not-landed, tmux spawn failure) NEVER leaks a slot.
async run(fn) {
await this.acquire();
// `signal` (optional, F2) is forwarded to acquire() so a queued run() can be cancelled.
async run(fn, signal) {
await this.acquire(signal);
try {
return await fn();
} finally {
+92 -12
View File
@@ -15,14 +15,48 @@ import { tmpdir } from "node:os";
import { randomUUID } from "node:crypto";
import { readTuiTranscript } from "./transcript.mjs";
export const SESSION_PREFIX = "ocp-tui-"; // per-proxy namespace (coexistence rule)
// F7 fix (audit finding, LOW): the prefix used to be a bare, host-wide constant
// ("ocp-tui-"), so a SECOND OCP instance on the same host (e.g. a temporary
// verification instance stood up alongside production — a real pattern used during
// PR #144/#146 verification) would boot-reap and potentially kill-server the OTHER
// instance's LIVE sessions: the coexistence guard below only ever spared foreign
// PRODUCT prefixes (olp-tui-*), never a second ocp-tui-* instance on a different port.
//
// Fix: scope the prefix to the instance's own listen port. The port is the natural
// stable per-instance discriminator on one host (two OCP instances cannot share a
// port), so `ocp-tui-<port>-` uniquely namespaces this instance's sessions and makes
// a same-host sibling OCP instance look exactly like a foreign product (olp-tui-*) to
// the coexistence guard — its `ocp-tui-<otherPort>-*` sessions never match our own
// prefix and are therefore never reaped/kill-server'd by us.
//
// LEGACY_SESSION_PREFIX / LEGACY_SESSION_NAME_RE describe the OLD bare-prefix shape
// (pre-this-fix), retained ONLY for the boot-time legacy-zombie migration handled in
// reapStaleTuiSessions (see comment there). No code path in this version ever CREATES
// a legacy-shaped session name again — sessionPrefixForPort() is the only session-name
// prefix constructor used going forward.
export const LEGACY_SESSION_PREFIX = "ocp-tui-";
// Exact legacy shape: LEGACY_SESSION_PREFIX + sessionId.slice(0, 8), where sessionId is
// a randomUUID() — so the suffix is always exactly 8 lowercase hex characters with NO
// further separator. The new port-scoped shape always inserts a "-" between the port
// digits and the 8-hex suffix (see sessionPrefixForPort), so this regex can never match
// a new-shape name: a new-shape suffix is `<port digits>-<8 hex>` (contains a literal
// "-"), which `[0-9a-f]{8}$` anchored immediately after the prefix cannot satisfy.
export const LEGACY_SESSION_NAME_RE = /^ocp-tui-[0-9a-f]{8}$/;
// Build this instance's own session-name prefix, scoped by its listen port so a
// second OCP instance on the same host (different port) is never mistaken for "ours".
export function sessionPrefixForPort(port) {
return `ocp-tui-${port}-`;
}
const TMUX = process.env.OCP_TUI_TMUX_BIN || "tmux";
const defaultTmux = (args, opts = {}) =>
spawnSync(TMUX, args, { encoding: "utf8", ...opts });
// Kill ONLY our own stale sessions. Scoped to SESSION_PREFIX so a co-hosted
// OLP test instance's `olp-tui-*` sessions are never touched.
// Kill ONLY our own stale sessions. Scoped to sessionPrefixForPort(port) so a co-hosted
// OLP test instance's `olp-tui-*` sessions — AND a co-hosted second OCP instance's
// `ocp-tui-<otherPort>-*` sessions — are never touched (F7 fix).
//
// Defunct-reaping (PI231 incident): the pane's `claude` process is a child of the
// long-lived tmux SERVER daemon, NOT of the OCP node process — `tmux new-session -d`
@@ -36,23 +70,40 @@ const defaultTmux = (args, opts = {}) =>
// merely re-signalling — is to stop the tmux server: when the server exits, the kernel
// reparents its surviving children to init (PID 1), which reaps them immediately.
//
// So after killing our own sessions, if the server has NO sessions left of ANY prefix
// (i.e. nothing we could disrupt — no co-hosted `olp-tui-*` or other instance), we
// `kill-server` to flush the defunct backlog. If ANY non-ocp session remains we leave the
// server running (coexistence rule, ADR 0007) and let the next boot/periodic sweep retry
// once the server is otherwise idle.
export function reapStaleTuiSessions({ tmux = defaultTmux } = {}) {
// `port` (required) is this instance's own listen port (server.mjs's PORT / lib/constants.mjs
// DEFAULT_PORT resolution) — the SPOT for "which sessions are ours."
//
// `includeLegacy` (default false): when true, sessions matching the exact OLD bare-prefix
// shape (LEGACY_SESSION_NAME_RE) are ALSO treated as ours for kill-session purposes. This is
// the boot-time legacy migration: an operator upgrading past this fix could otherwise be left
// with orphaned bare-prefix zombie sessions from the PREVIOUS (pre-fix) process generation of
// this SAME instance, since no live instance of the new version ever creates that shape again
// — a legacy-shaped session found at boot is therefore presumed to be this instance's own
// leftover, not a stranger's. Passed true ONLY from the one-time boot-reap call site in
// server.mjs; the periodic idle-reap sweep does NOT set it, so a lingering legacy session
// during steady-state is conservatively treated as foreign (correctly blocking kill-server)
// rather than assumed to be ours on every 15-minute tick. Residual (accepted, documented):
// if a genuinely-still-running PRE-FIX OCP instance is coexisting on the same host at the
// exact moment a new instance boots, its live legacy-shaped session could be reaped — the
// same class of residual risk the audit finding itself accepts ("no live instance of the new
// version creates them"); this PR does not regress that scenario, it only removes the far
// more common same-version collision (the actual F7 finding).
export function reapStaleTuiSessions({ tmux = defaultTmux, port, includeLegacy = false } = {}) {
const r = tmux(["list-sessions", "-F", "#{session_name}"]);
if (!r || r.status !== 0) return 0; // no tmux server / no sessions
const names = String(r.stdout || "").split("\n").map((s) => s.trim()).filter(Boolean);
const ownPrefix = sessionPrefixForPort(port);
let killed = 0;
let othersRemain = false;
for (const name of names) {
if (name.startsWith(SESSION_PREFIX)) {
const isOwn = name.startsWith(ownPrefix);
const isLegacyOwn = includeLegacy && LEGACY_SESSION_NAME_RE.test(name);
if (isOwn || isLegacyOwn) {
tmux(["kill-session", "-t", name]);
killed++;
} else {
othersRemain = true; // a session we do NOT own (e.g. olp-tui-*) — never kill-server
othersRemain = true; // a session we do NOT own (olp-tui-*, a sibling ocp-tui-<otherPort>-*,
// or — outside includeLegacy — a legacy-shaped name) — never kill-server
}
}
// Reap defunct `claude` zombies: safe ONLY when the server is now ours-only/empty.
@@ -326,12 +377,38 @@ export function buildTuiCmd(claudeBin, model, sessionId, ehome, entrypointMode)
} else {
toolArgs = ["--strict-mcp-config", "--disallowedTools", shq("mcp__*")];
}
// Effort: pass --effort EXPLICITLY. Without it, the pane's claude inherits a
// HOME-dependent effortLevel — real-home mode inherits the operator's
// ~/.claude/settings.json (whatever they set for their own interactive use),
// env-token scratch mode inherits claude's built-in default (prepareTuiHome never
// writes effortLevel) — so latency silently depends on which HOME mode
// resolveTuiHome() picked AND on an unrelated operator setting. Pinning it here
// removes both. Measured (docs/plans/2026-07-13-tui-latency): explicit low cuts
// direct-spawn TTFT p50 10.35s → 6.17s (40%) and collapses the spread ~15×;
// banner-verified to stay on the subscription pool (`· Claude Max`).
// OCP_TUI_EFFORT=inherit restores the pre-flag argv byte-for-byte (no --effort).
// An unknown value falls back to the default rather than reaching claude's argv:
// a typo'd --effort value must not risk a spawn-time usage error in the pane.
const EFFORT_LEVELS = ["low", "medium", "high", "xhigh", "max"]; // claude 2.1.207 --help
const effortRaw = (process.env.OCP_TUI_EFFORT || "low").trim().toLowerCase();
let effortArgs;
if (effortRaw === "inherit") {
effortArgs = [];
} else if (EFFORT_LEVELS.includes(effortRaw)) {
effortArgs = ["--effort", effortRaw];
} else {
console.error(`[tui] invalid OCP_TUI_EFFORT=${JSON.stringify(process.env.OCP_TUI_EFFORT)}; using "low" (valid: ${EFFORT_LEVELS.join("|")}, or "inherit" to omit the flag)`);
effortArgs = ["--effort", "low"];
}
return [
envPrefix,
shq(claudeBin),
"--model", shq(model),
"--session-id", sessionId,
...toolArgs,
...effortArgs,
].join(" ");
}
@@ -358,12 +435,15 @@ export async function runTuiTurn({
home,
realHome,
cwd,
port,
wallclockMs = 120000,
entrypointMode = "cli",
tmux = defaultTmux,
}) {
const sessionId = randomUUID();
const tmuxName = SESSION_PREFIX + sessionId.slice(0, 8);
// Port-scoped session name (F7 fix) — see sessionPrefixForPort / reapStaleTuiSessions
// for why this instance's own listen port is the namespace discriminator.
const tmuxName = sessionPrefixForPort(port) + sessionId.slice(0, 8);
const ehome = home || process.env.HOME; // HOME claude runs under (scratch or real)
const rhome = realHome || process.env.HOME; // real home (OAuth + onboarded config source)
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "open-claude-proxy",
"version": "3.21.0",
"version": "3.21.1",
"description": "OCP (Open Claude Proxy) — use your Claude Pro/Max subscription as an OpenAI-compatible API for any IDE. Works with Cline, OpenCode, Aider, Continue.dev, OpenClaw, and more.",
"type": "module",
"bin": {
+352 -87
View File
@@ -43,7 +43,8 @@ import { DEFAULT_PORT } from "./lib/constants.mjs";
import { isLoopbackBind } from "./lib/net.mjs";
import { runTuiTurn, reapStaleTuiSessions, resolveTuiHome } from "./lib/tui/session.mjs";
import { detectTuiUpstreamError } from "./lib/tui/transcript.mjs";
import { TuiSemaphore, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
import { TuiSemaphore, SemaphoreAbortError, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
import { createSerialMutex, createTtlCache, isTokenExpiring, orderLabelsLastGoodFirst } from "./lib/spawn-auth.mjs";
const __dirname = dirname(fileURLToPath(import.meta.url));
const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8"));
@@ -387,29 +388,104 @@ function prepareSpawnHome(dir = SPAWN_HOME_DIR) {
} catch { /* best effort — spawn will surface a hard error if the dir is truly unusable */ }
}
// Resolve the default-spawn HOME-isolation decision ONCE, lazily + memoized (so it runs after
// getOAuthCredentials is defined regardless of source order, and the token probe happens at most
// once). Returns { isolated, home, token } where:
// Resolve the default-spawn HOME-isolation decision. Returns { isolated, home, reason }:
// - isolated:true → spawn under SPAWN_HOME_DIR with cwd=SPAWN_HOME_DIR + the env token.
// - isolated:false → legacy real-HOME spawn, no cwd override (no token, or kill-switch on).
// NEVER logs/returns the token verbatim to any caller that logs; spawnClaudeProcess uses it only
// to populate the spawn env. token is null when isolation is off.
let _spawnHomeMode = null;
//
// FIX F6 (2026-07-07): this decision is NO LONGER memoized permanently. The previous version
// cached it forever at first call, which meant: (a) credentials appearing after startup never
// enabled isolation; (b) `rm -rf ~/.ocp/spawn-home` at runtime made every isolated spawn ENOENT
// until restart; (c) during a token-expiry stint /health reported isolated:true while spawns
// actually ran real-HOME. Re-evaluating per spawn is cheap because F5's 30s keychain TTL cache
// backs getOAuthCredentials(). This function is the CONFIG-level decision (isolated iff a token
// resolves AND the kill-switch is off) and has NO fs side effects — the per-spawn EFFECTIVE
// decision additionally applies the expiry gate (resolveSpawnDecision), and scratch-HOME dir prep
// moved to ensureSpawnHome() at the isolated spawn site.
//
// The token itself is re-resolved FRESH per spawn via resolveSpawnToken(); a memoized token goes
// stale when its source rotates (the macOS keychain access token rotates ~hourly, refreshed by the
// operator's real claude), which 401'd every isolated spawn for ~31h on 2026-06-26 (#146). OCP
// deliberately does NOT refresh the token itself — a refresh-token grant would consume the
// single-use refresh token and log out the operator's real claude (issue #112).
function getSpawnHomeMode() {
if (_spawnHomeMode) return _spawnHomeMode;
if (SPAWN_REAL_HOME) {
_spawnHomeMode = { isolated: false, home: null, token: null, reason: "kill-switch (OCP_SPAWN_REAL_HOME=1)" };
return _spawnHomeMode;
return { isolated: false, home: null, reason: "kill-switch (OCP_SPAWN_REAL_HOME=1)" };
}
let token = null;
try { token = getOAuthCredentials()?.accessToken || null; } catch { token = null; }
let hasToken = false;
try { hasToken = !!(getOAuthCredentials()?.accessToken); } catch { hasToken = false; }
if (hasToken) return { isolated: true, home: SPAWN_HOME_DIR, reason: "oauth token resolved" };
return { isolated: false, home: null, reason: "no oauth token resolvable" };
}
// FIX F6: re-verify the scratch HOME exists before each isolated spawn and re-create it if it was
// deleted at runtime (it used to be prepared once at startup, so a runtime deletion made every
// isolated spawn fail ENOENT until restart). mkdirSync is recursive+idempotent → cheap to re-run.
function ensureSpawnHome(dir = SPAWN_HOME_DIR) {
if (!existsSync(`${dir}/.claude`)) prepareSpawnHome(dir);
}
// Resolve a FRESH OAuth access token for an isolated spawn. Read-only (keychain / credentials.json
// / env) — NEVER refreshes/rotates (see getSpawnHomeMode note). Returns null if none resolvable OR
// if a known expiry is within the 5-min buffer (isTokenExpiring): a null return makes the caller
// fall back to real HOME, where the spawned claude refreshes the credential natively and self-heals
// (the keychain token is then fresh again → next spawn is fast). The env-token path (Linux) carries
// no expiresAt → never expiry-gated (those tokens are long-lived).
function resolveSpawnToken() {
try {
const creds = getOAuthCredentials();
if (!creds?.accessToken) return null;
if (isTokenExpiring(creds)) return null; // 5-min buffer; applied to the CACHED creds every use
return creds.accessToken;
} catch { return null; }
}
// FIX F3 (2026-07-07): serializes ONLY the real-HOME fallback spawns. Isolated spawns (the common
// fast path) never touch this mutex.
const realHomeFallbackMutex = createSerialMutex();
// Resolve the EFFECTIVE per-spawn HOME/token decision. Returns
// { isolated, home, token, releaseFallback }
// `releaseFallback` is non-null ONLY for a real-HOME fallback holder — the caller MUST call it on
// spawn teardown (wired into cleanup()); it releases the serialization mutex. It is null (no-op)
// for isolated and stable real-HOME (kill-switch / no-token) spawns.
//
// This is async so the real-HOME fallback can `await` the mutex; the keychain reads inside stay
// synchronous (F5 keeps the call sites off async conversion).
async function resolveSpawnDecision() {
const shm = getSpawnHomeMode();
if (!shm.isolated) return { isolated: false, home: null, token: null, releaseFallback: null };
const token = resolveSpawnToken();
if (token) {
prepareSpawnHome(SPAWN_HOME_DIR);
_spawnHomeMode = { isolated: true, home: SPAWN_HOME_DIR, token, reason: "oauth token resolved" };
} else {
_spawnHomeMode = { isolated: false, home: null, token: null, reason: "no oauth token resolvable" };
ensureSpawnHome(shm.home);
return { isolated: true, home: shm.home, token, releaseFallback: null };
}
return _spawnHomeMode;
// Token is present but within the 5-min expiry window → we would fall back to real HOME, where
// the spawned claude refreshes the credential natively. HAZARD PREVENTED: without serialization,
// every concurrent -p spawn inside this window runs claude under the real HOME simultaneously,
// and each spawned claude races a `refresh_token` grant against the SAME single-use refresh
// token — rotating it out from under the others AND the operator's own real claude (the
// credential-fork hazard; #112 / #146 class). Serialize: admit ONE real-HOME spawn at a time.
// When the next waiter is admitted (the prior holder torn down → its claude has had its lifetime
// to refresh the keychain), re-run resolveSpawnToken(): a now-fresh token means we proceed
// ISOLATED and release the mutex immediately, so the queue drains to the fast path instead of
// piling every request into the real HOME.
const release = await realHomeFallbackMutex.acquire();
try {
// Drop the 30s keychain TTL cache so the re-check reads FRESH keychain state — otherwise a
// waiter admitted right after the prior holder's claude refreshed the token could still see the
// stale (expiring) cached creds and needlessly fall back to real HOME again for up to ~30s.
invalidateKeychainReadCache();
const retry = resolveSpawnToken();
if (retry) {
release();
ensureSpawnHome(shm.home);
return { isolated: true, home: shm.home, token: retry, releaseFallback: null };
}
} catch (e) {
release();
throw e;
}
return { isolated: false, home: null, token: null, releaseFallback: release };
}
// ── FIX ⑥ (concurrency): bounded wait-queue for the -p / stream-json path ──────────────
@@ -431,16 +507,61 @@ class ConcurrencyOverflowError extends Error {
constructor(message) { super(message); this.name = "ConcurrencyOverflowError"; this.httpStatus = 429; this.retryAfter = CLAUDE_QUEUE_RETRY_AFTER; }
}
// Tagged error for audit finding F2: the client disconnected while queued (or was already gone
// before we even tried to queue it). Distinct from ConcurrencyOverflowError so callers never send
// a response on this path — there is no socket left to write to.
class RequestDisconnectedError extends Error {
constructor(message) { super(message); this.name = "RequestDisconnectedError"; }
}
// Build an AbortSignal that fires when `res` (an http.ServerResponse) closes — i.e. the client
// disconnected. Used to cancel a QUEUED concurrency-slot wait (F2) so a client that gives up
// before a slot is granted is spliced out of the wait queue instead of eventually spawning a
// claude process for a dead socket. If `res` has already closed by the time we get here (its
// underlying stream already torn down), the signal is returned pre-aborted so acquire() rejects
// immediately without ever touching the queue — the "close already fired before we attach" case.
// `detach()` MUST be called once the wait settles (granted or rejected) to avoid a listener leak.
function closeSignalFor(res) {
const controller = new AbortController();
if (!res || typeof res.on !== "function") return { signal: controller.signal, detach() {} };
if (res.destroyed) {
controller.abort();
return { signal: controller.signal, detach() {} };
}
const onClose = () => controller.abort();
res.on("close", onClose);
return { signal: controller.signal, detach() { res.removeListener("close", onClose); } };
}
// Acquire a -p concurrency slot, queuing if all are busy (up to CLAUDE_MAX_QUEUE). Resolves to a
// release() fn that MUST be called exactly once on every exit path (wired into ctx.cleanup()).
// Rejects with ConcurrencyOverflowError when the wait-queue is full. Increments stats.queued while
// waiting (decremented on acquire) and stats.queueRejections on overflow.
async function acquireClaudeSlot() {
stats.queued = claudeSemaphore.queued + 1; // reflect this waiter before we (maybe) block
// Rejects with ConcurrencyOverflowError when the wait-queue is full, or with
// RequestDisconnectedError when `res` closes before a slot is granted (F2) — the caller must not
// spawn claude in that case. `res` is optional (back-compat for any caller without a live response
// object); omitting it just means a queued wait can't be cancelled early.
//
// F8 fix: stats.queued is set from claudeSemaphore.queued AFTER calling acquire() (not before) —
// acquire() synchronously updates _inflight/_waiters before its Promise ever resolves, so reading
// .queued right after the call already reflects reality. The old code set `queued + 1` BEFORE
// calling acquire() to account for "this waiter", which over-reported by 1 whenever the slot was
// granted immediately (the common case, not a queue at all).
async function acquireClaudeSlot(res) {
const { signal, detach } = closeSignalFor(res);
const slot = claudeSemaphore.acquire(signal);
stats.queued = claudeSemaphore.queued; // accurate: acquire() already updated the queue synchronously
try {
await claudeSemaphore.acquire();
await slot;
} catch (e) {
detach();
stats.queued = claudeSemaphore.queued;
if (e instanceof SemaphoreAbortError) {
// Client-driven cancellation, not backpressure — do NOT count it as a queueRejection or
// log it as concurrency_queue_full (that log/counter means "the queue itself is full").
logEvent("info", "concurrency_wait_cancelled", {
reason: "client_disconnected", inflight: claudeSemaphore.inflight, queued: claudeSemaphore.queued,
});
throw new RequestDisconnectedError("client disconnected while waiting for a concurrency slot");
}
stats.queueRejections++;
logEvent("warn", "concurrency_queue_full", {
limit: claudeSemaphore.limit, maxQueue: claudeSemaphore.maxQueue,
@@ -450,6 +571,7 @@ async function acquireClaudeSlot() {
`backpressure: concurrency limit (${claudeSemaphore.limit}) reached and wait queue ` +
`(${claudeSemaphore.maxQueue}) is full — retry shortly`);
}
detach();
stats.queued = claudeSemaphore.queued;
let released = false;
return function releaseClaudeSlot() {
@@ -652,7 +774,13 @@ const TUI_REAP_INTERVAL_MS = 15 * 60 * 1000;
const tuiReapInterval = TUI_MODE ? setInterval(() => {
if (tuiSemaphore.inflight > 0 || tuiSemaphore.queued > 0) return; // a turn is live — defer
try {
const n = reapStaleTuiSessions();
// F7 fix: scope to THIS instance's own port; a sibling ocp-tui-<otherPort>-* session
// (a second OCP instance on the same host) is treated as foreign, same as olp-tui-*.
// includeLegacy is NOT set here — see reapStaleTuiSessions' comment: the periodic sweep
// conservatively treats any lingering bare-prefix legacy session as foreign so it can
// never trigger kill-server on a steady-state tick; only the one-time boot reap below
// claims legacy-shaped zombies.
const n = reapStaleTuiSessions({ port: PORT });
if (n) logEvent("info", "tui_reaped_stale_sessions", { count: n, trigger: "periodic" });
} catch (e) { logEvent("error", "tui_periodic_reap_failed", { error: e.message }); }
}, TUI_REAP_INTERVAL_MS) : null;
@@ -906,7 +1034,7 @@ function getModelTier(cliModel) {
// budget. releaseSlot is wired into the idempotent cleanup() so the slot is freed on EVERY exit
// path (close/error/timeout/abort). Back-compat: releaseSlot defaults to a no-op so any future
// internal caller that does its own gating still works.
function spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlot = () => {}) {
function spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlot = () => {}, spawnDecision = null) {
const cliModel = MODEL_MAP[model] || model;
// Circuit breaker: disabled (see comment at top of breaker section)
@@ -943,22 +1071,24 @@ function spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlo
env.CLAUDE_CODE_DISABLE_AUTO_MEMORY = "1";
}
// FIX ③ (latency): default-path spawn-home isolation. When a token is resolvable (and the
// OCP_SPAWN_REAL_HOME kill-switch is off), run claude under a credential-free minimal HOME
// with cwd = that same neutral dir, so it loads NONE of the operator's global ~/.claude
// (plugins/skills/hooks) or the ~/ocp project CLAUDE.md/skills — the measured 1028s → 37s
// latency win. The env token is authoritative for `-p` (unlike interactive claude). When no
// token is resolvable, falls back to real HOME + inherited cwd (zero regression). See
// getSpawnHomeMode() / prepareSpawnHome() above. The DISABLE_CLAUDE_MDS / AUTO_MEMORY flags
// are set unconditionally in isolated mode (belt-and-braces; mirrors the TUI path).
const spawnHome = getSpawnHomeMode();
// FIX ③ (latency) + F3 (concurrency): apply the pre-resolved per-spawn HOME/token decision.
// The decision is resolved ASYNC in the caller (resolveSpawnDecision) so the real-HOME fallback
// serialization can await its mutex; here we only apply the result. When isolated, run claude
// under a credential-free minimal HOME with cwd = that same neutral dir, so it loads NONE of the
// operator's global ~/.claude (plugins/skills/hooks) or the ~/ocp project CLAUDE.md/skills — the
// measured 1028s → 37s latency win. The env token is authoritative for `-p` (unlike
// interactive claude). When no fresh token is resolvable, decision.isolated is false → real HOME
// + inherited cwd (zero regression), and the spawned claude resolves+refreshes credentials
// natively. The DISABLE_CLAUDE_MDS / AUTO_MEMORY flags are set unconditionally in isolated mode
// (belt-and-braces; mirrors the TUI path).
const decision = spawnDecision || { isolated: false, releaseFallback: null };
const spawnOpts = { env, stdio: ["pipe", "pipe", "pipe"] };
if (spawnHome.isolated) {
env.HOME = spawnHome.home;
env.CLAUDE_CODE_OAUTH_TOKEN = spawnHome.token; // env token is authoritative for -p
if (decision.isolated && decision.token) {
env.HOME = decision.home;
env.CLAUDE_CODE_OAUTH_TOKEN = decision.token; // env token is authoritative for -p
env.CLAUDE_CODE_DISABLE_CLAUDE_MDS = "1";
env.CLAUDE_CODE_DISABLE_AUTO_MEMORY = "1";
spawnOpts.cwd = spawnHome.home; // neutral cwd: no project CLAUDE.md/skills
spawnOpts.cwd = decision.home; // neutral cwd: no project CLAUDE.md/skills
}
const proc = spawn(CLAUDE, cliArgs, spawnOpts);
@@ -977,6 +1107,11 @@ function spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlo
// and cleanup() is guarded by `cleaned`, so the slot is released exactly once on the first
// exit path reached (proc 'exit' fires before 'close'; 'error' covers spawn failure).
try { releaseSlot(); } catch { /* never let release throw out of cleanup */ }
// F3: release the real-HOME fallback serialization mutex (no-op for isolated/normal spawns).
// By now this spawn's claude has had its lifetime to refresh the keychain token, so the next
// queued fallback waiter re-checks resolveSpawnToken() and proceeds ISOLATED with the now-fresh
// token instead of piling into the real HOME. Idempotent; cleanup() is guarded by `cleaned`.
try { if (decision.releaseFallback) decision.releaseFallback(); } catch { /* never throw out of cleanup */ }
}
// Guarantee slot release on ANY exit path (normal close, error, timeout kill,
@@ -1046,18 +1181,37 @@ function spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlo
// We accumulate full text across all content_block_delta events plus the
// assistant-aggregate fallback, then resolve with the assembled string.
// Reference: OLP ADR 0009 Amendment 1 + commit 97e7d16.
async function callClaude(model, messages, conversationId, keyName) {
// `res` (optional, F2) is the client's http.ServerResponse — passed through so a queued wait
// can be cancelled the moment the client disconnects, instead of spawning claude for a dead
// socket once a slot finally frees up.
async function callClaude(model, messages, conversationId, keyName, res) {
// FIX ⑥: acquire a concurrency slot first (queues up to CLAUDE_MAX_QUEUE; rejects with a
// ConcurrencyOverflowError → 429 when the queue is full). The release fn is passed into the
// spawn so the idempotent cleanup() frees it on every exit path. If the spawn itself throws
// synchronously (before cleanup is wired), release here so the slot never leaks.
const releaseSlot = await acquireClaudeSlot();
// ConcurrencyOverflowError → 429 when the queue is full, or a RequestDisconnectedError (F2)
// if the client goes away first). The release fn is passed into the spawn so the idempotent
// cleanup() frees it on every exit path. If the spawn itself throws synchronously (before
// cleanup is wired), release here so the slot never leaks.
// F2×F3 composition: the slot acquire comes FIRST and is the cancellable step — a client
// that disconnects while queued rejects here, BEFORE resolveSpawnDecision() runs, so a
// cancelled request can never acquire (or briefly hold) the real-HOME fallback mutex.
const releaseSlot = await acquireClaudeSlot(res);
// F3: resolve the per-spawn HOME/token decision (may serialize on the real-HOME fallback
// mutex). If it throws, release the just-acquired slot before propagating — cleanup() is
// not wired yet at this point.
let spawnDecision;
try {
spawnDecision = await resolveSpawnDecision();
} catch (err) {
releaseSlot();
throw err;
}
return new Promise((resolve, reject) => {
let ctx;
try {
ctx = spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlot);
ctx = spawnClaudeProcess(model, messages, conversationId, keyName, releaseSlot, spawnDecision);
} catch (err) {
releaseSlot();
// Spawn threw before cleanup() was wired → release the fallback mutex here so it never leaks.
try { spawnDecision.releaseFallback?.(); } catch { /* best effort */ }
return reject(err);
}
@@ -1128,26 +1282,50 @@ async function callClaude(model, messages, conversationId, keyName) {
// flag that could perturb cc_entrypoint classification.
// Authority: claude CLI v2.1.158 interactive mode (cc_entrypoint=cli).
// SECURITY: A-path single-user ONLY — home is NOT isolation (see ADR 0007).
function callClaudeTui(model, messages, _conversationId, _keyName) {
// `res` (optional, F2) is the client's http.ServerResponse — see closeSignalFor.
async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
const cliModel = MODEL_MAP[model] || model;
const prompt = messagesToPrompt(messages); // includes system as [System] inline
recordModelRequest(cliModel, prompt.length);
// C-4: gate the heavy interactive boot behind the TUI semaphore. run() acquires a slot
// (queuing if all are busy, up to maxQueue), then releases in a finally so any throw from
// runTuiTurn (tmux spawn failure, paste-not-landed) OR from the honesty gates below
// (truncation / error banner) can NEVER leak a slot. tuiSemaphore.inflight feeds /health.
return tuiSemaphore.run(() => runTuiTurn({
prompt,
model: cliModel,
claudeBin: CLAUDE,
home: TUI_HOME,
realHome: process.env.HOME,
cwd: TUI_CWD,
wallclockMs: TUI_WALLCLOCK_MS,
entrypointMode: TUI_ENTRYPOINT,
}).then(({ text, entrypoint, truncated }) => {
// C-4: gate the heavy interactive boot behind the TUI semaphore (queuing if all slots are
// busy, up to maxQueue). F2: `signal` (tied to `res` "close") cancels a QUEUED wait the
// instant the client disconnects, so a dead socket never triggers a cold-boot tmux+claude
// spawn; detach() drops the "close" listener as soon as the wait settles rather than
// holding it for the whole (up to 120s) turn.
const { signal, detach } = closeSignalFor(res);
try {
await tuiSemaphore.acquire(signal);
} catch (err) {
detach();
if (err instanceof SemaphoreAbortError) {
// L1: client-driven cancellation, not an upstream failure — info, not error (mirrors
// acquireClaudeSlot's concurrency_wait_cancelled on the -p path).
logEvent("info", "concurrency_wait_cancelled", {
reason: "client_disconnected", path: "tui", inflight: tuiSemaphore.inflight, queued: tuiSemaphore.queued,
});
throw new RequestDisconnectedError("client disconnected while waiting for a TUI concurrency slot");
}
throw err;
}
detach();
// release() runs in a finally so any throw from runTuiTurn (tmux spawn failure,
// paste-not-landed) OR from the honesty gates below (truncation / error banner) can NEVER
// leak a slot. tuiSemaphore.inflight feeds /health.
try {
const { text, entrypoint, truncated } = await runTuiTurn({
prompt,
model: cliModel,
claudeBin: CLAUDE,
home: TUI_HOME,
realHome: process.env.HOME,
cwd: TUI_CWD,
port: PORT, // F7 fix: port-scopes the tmux session name so a sibling OCP instance on a
// different port never collides with this instance's reap/kill-server logic.
wallclockMs: TUI_WALLCLOCK_MS,
entrypointMode: TUI_ENTRYPOINT,
});
// ── Honesty gates (issue #133) ─ run BEFORE recordModelSuccess / cache write-back.
// A throw here propagates to the .catch below (recordModelError + reject), so the
// A throw here propagates to the catch below (recordModelError + reject), so the
// result never reaches the downstream setCachedResponse / singleflight / SUCCESS path.
// C-2: the wall-clock cap hit with partial text and NO terminal marker — the turn
@@ -1182,10 +1360,12 @@ function callClaudeTui(model, messages, _conversationId, _keyName) {
logEvent("warn", "tui_entrypoint_mismatch", { expected: "cli", got: entrypoint, model: cliModel });
}
return text;
}).catch((err) => {
} catch (err) {
recordModelError(cliModel, false);
throw err;
}));
} finally {
tuiSemaphore.release();
}
}
// ── SSE heartbeat (opt-in idle watchdog) ────────────────────────────────
@@ -1231,23 +1411,39 @@ async function callClaudeStreaming(model, messages, conversationId, res, authInf
// FIX ⑥: acquire a concurrency slot first (queues up to CLAUDE_MAX_QUEUE). On overflow, surface
// HTTP 429 + Retry-After (NOT 500). Release is wired into cleanup() for every exit path; if the
// spawn throws synchronously before cleanup is wired, release here.
// F2: pass `res` so a queued wait is cancelled the instant this client disconnects — the client
// is already gone in that case, so there is no response to send back.
let releaseSlot;
try {
releaseSlot = await acquireClaudeSlot();
releaseSlot = await acquireClaudeSlot(res);
} catch (err) {
if (err instanceof RequestDisconnectedError) return; // client gone — nothing to write to
if (err instanceof ConcurrencyOverflowError) {
return jsonResponse(res, 429, { error: { message: sanitizeError(err.message), type: "rate_limit_error" } }, { "Retry-After": String(err.retryAfter) });
}
return jsonResponse(res, 500, { error: { message: sanitizeError(err.message), type: "proxy_error" } });
}
let ctx;
// F3: resolve the per-spawn HOME/token decision (may serialize on the real-HOME fallback
// mutex). F2×F3 composition: this runs strictly AFTER the (cancellable) slot acquire, so a
// request cancelled while queued never touches the fallback mutex. If it throws, release
// the just-acquired slot before responding — cleanup() is not wired yet at this point.
let spawnDecision;
try {
ctx = spawnClaudeProcess(model, messages, conversationId, authInfo.keyName, releaseSlot);
spawnDecision = await resolveSpawnDecision();
} catch (err) {
releaseSlot();
return jsonResponse(res, 500, { error: { message: sanitizeError(err.message), type: "proxy_error" } });
}
let ctx;
try {
ctx = spawnClaudeProcess(model, messages, conversationId, authInfo.keyName, releaseSlot, spawnDecision);
} catch (err) {
releaseSlot();
// Spawn threw before cleanup() was wired → release the fallback mutex here so it never leaks.
try { spawnDecision.releaseFallback?.(); } catch { /* best effort */ }
return jsonResponse(res, 500, { error: { message: sanitizeError(err.message), type: "proxy_error" } });
}
const { proc, cliModel, conversationId: convId, t0, cleanup, clearOverallTimer, handleSessionFailure, markFirstByte } = ctx;
let stderr = "";
@@ -1517,6 +1713,51 @@ const OAUTH_REFRESH_MIN_BACKOFF = 60 * 1000;
const OAUTH_REFRESH_MAX_BACKOFF = 3600 * 1000;
let oauthRefreshBackoff = { nextAttemptAt: 0, currentDelay: OAUTH_REFRESH_MIN_BACKOFF };
// FIX F5 (2026-07-07): the macOS keychain read (`security find-generic-password`, up to 5s × 2
// labels when the first label misses) ran on EVERY -p spawn's hot path, blocking the event loop
// (worst case 10s) and stalling all in-flight SSE streams. Two minimal, sync-preserving mitigations:
// (a) memoize the last-good keychain label and try it FIRST → one exec instead of two on the
// steady-state path (orderLabelsLastGoodFirst);
// (b) a short (30s) TTL cache of the keychain read result (createTtlCache).
// SAFETY vs the #146 regression: #146 was a token memoized FOREVER at startup that went stale and
// 401'd. This is a 30s TTL (not forever), AND resolveSpawnToken() re-applies the 5-min expiry gate
// (isTokenExpiring) to the CACHED creds on EVERY use — the creds object carries `expiresAt`, so a
// token expiring within the cache window is still rejected → real-HOME fallback. A short TTL bounds
// how often we re-READ the keychain; it does NOT bound how often we re-DECIDE expiry. This is why a
// short-TTL keychain cache + a per-use expiry check does not reintroduce the forever-stale bug.
const KEYCHAIN_LABELS = ["claude-code-credentials", "Claude Code-credentials"];
const KEYCHAIN_CACHE_TTL_MS = 30 * 1000;
const _keychainCache = createTtlCache({ ttlMs: KEYCHAIN_CACHE_TTL_MS });
let _lastGoodKeychainLabel = null;
// Read the macOS keychain credentials, label-memoized + short-TTL cached (F5). Sync (execFileSync);
// returns the `claudeAiOauth` creds object or null.
function readKeychainCreds() {
return _keychainCache.get(() => {
for (const label of orderLabelsLastGoodFirst(KEYCHAIN_LABELS, _lastGoodKeychainLabel)) {
try {
const raw = execFileSync("security", [
"find-generic-password", "-s", label, "-w"
], { encoding: "utf8", timeout: 5000 }).trim();
const creds = JSON.parse(raw);
if (creds?.claudeAiOauth?.accessToken) {
_lastGoodKeychainLabel = label; // remember the winner → try it first next time
return creds.claudeAiOauth;
}
} catch { /* try next label */ }
}
return null;
});
}
// F3 drain helper: drop the F5 keychain TTL cache so the NEXT getOAuthCredentials() re-reads the
// keychain from scratch. Called under the real-HOME fallback mutex just before the re-check, so a
// waiter admitted after the prior holder's claude refreshed the keychain sees the FRESH token
// immediately (and proceeds ISOLATED) instead of waiting out the ≤30s TTL on the stale creds.
function invalidateKeychainReadCache() {
_keychainCache.clear();
}
function getOAuthCredentials() {
// 1. Env var fallback — highest precedence for explicit overrides.
if (process.env.CLAUDE_CODE_OAUTH_TOKEN) {
@@ -1530,17 +1771,8 @@ function getOAuthCredentials() {
if (creds?.claudeAiOauth?.accessToken) return creds.claudeAiOauth;
} catch { /* fall through to macOS keychain */ }
// 3. macOS keychain (both label formats)
for (const label of ["claude-code-credentials", "Claude Code-credentials"]) {
try {
const raw = execFileSync("security", [
"find-generic-password", "-s", label, "-w"
], { encoding: "utf8", timeout: 5000 }).trim();
const creds = JSON.parse(raw);
if (creds?.claudeAiOauth?.accessToken) return creds.claudeAiOauth;
} catch { /* try next */ }
}
return null;
// 3. macOS keychain (both label formats) — F5: label-memoized + 30s TTL cached (see above).
return readKeychainCreds();
}
async function refreshOAuthToken(refreshToken) {
@@ -1874,9 +2106,12 @@ function applySettingUpdate(key, value) {
switch (key) {
case "timeout": TIMEOUT = value; break;
// FIX ⑥: keep the -p wait-queue semaphore's limit in sync with the runtime MAX_CONCURRENT
// so a /settings change to maxConcurrent actually changes how many claude procs run at once.
case "maxConcurrent": MAX_CONCURRENT = value; claudeSemaphore.limit = Math.max(1, value); break;
// FIX ⑥ + F1: keep the -p wait-queue semaphore's limit in sync with the runtime MAX_CONCURRENT
// so a /settings change to maxConcurrent actually changes how many claude procs run at once
// in BOTH directions. setLimit() (not a bare `.limit =` assignment) is required: lowering
// needs release() to stop over-granting until inflight drains under the new cap, and raising
// needs queued waiters woken immediately to use the new headroom. See lib/tui/semaphore.mjs.
case "maxConcurrent": MAX_CONCURRENT = value; claudeSemaphore.setLimit(value); break;
case "sessionTTL": SESSION_TTL = value; break;
case "maxPromptChars": MAX_PROMPT_CHARS = value; break;
case "cacheTTL": CACHE_TTL = value; break;
@@ -2033,7 +2268,7 @@ async function handleChatCompletions(req, res) {
const t0TuiStream = Date.now();
const promptCharsTuiStream = messages.reduce((a, m) => a + contentToText(m.content).length, 0);
try {
const content = await callClaudeTui(model, messages, conversationId, req._authKeyName);
const content = await callClaudeTui(model, messages, conversationId, req._authKeyName, res);
if (CACHE_TTL > 0 && req._cacheHash) {
try { setCachedResponse(req._cacheHash, model, content); } catch (e) { logEvent("error", "cache_write_failed", { error: e.message }); }
}
@@ -2070,15 +2305,27 @@ async function handleChatCompletions(req, res) {
// will re-read the freshly-populated cache entry here rather than spawning.
const recheck = getCachedResponse(req._cacheHash, CACHE_TTL);
if (recheck) return recheck.response;
const c = await upstreamCall(model, messages, conversationId, req._authKeyName);
const c = await upstreamCall(model, messages, conversationId, req._authKeyName, res);
try { setCachedResponse(req._cacheHash, model, c); } catch (e) { logEvent("error", "cache_write_failed", { error: e.message }); }
return c;
});
},
// M1: if the LEADER disconnected while queued (F2), its RequestDisconnectedError is
// personal to the leader — a live follower must not inherit it as a spurious 500.
// retryIf makes this follower re-enter singleflight with its OWN fn (own res, own
// disconnect signal), becoming the new leader or joining a retrying sibling's flight —
// but only while OUR client is still connected. If our client is also gone, the
// rejection propagates and the RDE early-return in the catch below ends it quietly.
(err) => err instanceof RequestDisconnectedError && !res.destroyed);
const id = `chatcmpl-${randomUUID()}`;
completionResponse(res, id, model, content);
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: content.length, elapsedMs: Date.now() - t0Usage, success: true }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
return;
} catch (err) {
// L1: a client disconnect while queued is NOT an upstream failure — mirror the
// streaming path (which returns without recording anything): no usage-failure row,
// no [proxy] error log, no error response (the socket is gone). The disconnect is
// already logged at info level (concurrency_wait_cancelled) by acquireClaudeSlot.
if (err instanceof RequestDisconnectedError) { try { res.end(); } catch {} return; }
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: 0, elapsedMs: Date.now() - t0Usage, success: false }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
console.error(`[proxy] error: ${err.message}`);
if (res.headersSent || res.writableEnded || res.destroyed) {
@@ -2091,11 +2338,14 @@ async function handleChatCompletions(req, res) {
// Fallback: cache disabled (CACHE_TTL=0) or no _cacheHash — original path untouched.
try {
const content = await upstreamCall(model, messages, conversationId, req._authKeyName);
const content = await upstreamCall(model, messages, conversationId, req._authKeyName, res);
const id = `chatcmpl-${randomUUID()}`;
completionResponse(res, id, model, content);
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: content.length, elapsedMs: Date.now() - t0Usage, success: true }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
} catch (err) {
// L1: disconnect-while-queued — same quiet non-error outcome as the singleflight
// path above and the streaming path (see acquireClaudeSlot's info-level log).
if (err instanceof RequestDisconnectedError) { try { res.end(); } catch {} return; }
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: 0, elapsedMs: Date.now() - t0Usage, success: false }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
console.error(`[proxy] error: ${err.message}`);
if (res.headersSent || res.writableEnded || res.destroyed) {
@@ -2273,11 +2523,22 @@ const server = createServer(async (req, res) => {
spawn: (() => {
if (TUI_MODE) return { mode: "tui (default -p path unused)", isolated: false, home: null };
const shm = getSpawnHomeMode();
// FIX F6: report the EFFECTIVE current decision, not just token PRESENCE. During the
// 5-min pre-expiry window the token exists (shm.isolated=true) but resolveSpawnToken()
// returns null and spawns actually run real-HOME — so `isolated` MUST also reflect the
// expiry gate, or /health lies. The field SET is unchanged (grandfathered B.2 contract,
// ADR 0006 — HARD CONSTRAINT: no field add/remove/rename); only the VALUES are made
// truthful. resolveSpawnToken() is read-only + backed by F5's 30s keychain cache → cheap.
const effIsolated = shm.isolated && resolveSpawnToken() !== null;
return {
mode: shm.isolated ? "isolated-scratch-home" : "real-home",
isolated: shm.isolated,
home: shm.isolated ? shm.home : null,
reason: shm.reason,
mode: effIsolated ? "isolated-scratch-home" : "real-home",
isolated: effIsolated,
home: effIsolated ? shm.home : null,
reason: effIsolated
? shm.reason
: (shm.isolated
? "oauth token within 5-min expiry window → real-HOME fallback (self-heals on next refresh)"
: shm.reason),
};
})(),
// ── FIX ⑥ -p concurrency wait-queue surface — ADDITIVE ──
@@ -2606,7 +2867,11 @@ server.listen(PORT, BIND_ADDRESS, () => {
: "credentials.json (no CLAUDE_CODE_OAUTH_TOKEN — see Troubleshooting #401)";
console.log(` TUI-mode: ON home=${TUI_HOME} cwd=${TUI_CWD} auth=${tuiAuth} wallclock=${TUI_WALLCLOCK_MS}ms maxConcurrent=${TUI_MAX_CONCURRENT}`);
try {
const n = reapStaleTuiSessions();
// F7 fix: scope to THIS instance's own port (see reapStaleTuiSessions). includeLegacy:
// true ONLY here — the one-time boot reap is the designated point to claim orphaned
// bare-prefix ("ocp-tui-<uuid8>") zombie sessions left by a PRE-fix process generation
// of this same instance (no live post-fix instance ever creates that shape again).
const n = reapStaleTuiSessions({ port: PORT, includeLegacy: true });
if (n) logEvent("info", "tui_reaped_stale_sessions", { count: n });
} catch {}
}
+559 -19
View File
@@ -5,6 +5,7 @@
*/
import { getDb, createKey, listKeys, validateKey, recordUsage, checkQuota, updateKeyQuota, getKeyQuota, findKey, cacheHash, getCachedResponse, setCachedResponse, clearCache, getCacheStats, closeDb, hasCacheControl, singleflight, getInflightStats } from "./keys.mjs";
import { isLoopbackBind } from "./lib/net.mjs";
import { createSerialMutex, createTtlCache, isTokenExpiring, orderLabelsLastGoodFirst } from "./lib/spawn-auth.mjs";
import { createHash } from "node:crypto";
import { strict as assert } from "node:assert";
import { unlinkSync } from "node:fs";
@@ -33,6 +34,17 @@ function test(name, fn) {
}
}
async function testAsync(name, fn) {
try {
await fn();
passed++;
console.log(`${name}`);
} catch (e) {
failed++;
console.log(`${name}: ${e.message}`);
}
}
console.log("\n=== OCP Feature Tests (Quota + Cache) ===\n");
// Initialize DB
@@ -451,6 +463,52 @@ async function runSingleflightTests() {
assert.equal(r1, 1);
assert.equal(r2, 2);
});
// 7. M1: leader disconnect while queued must not poison live followers. server.mjs passes
// retryIf = (err) => err instanceof RequestDisconnectedError && !res.destroyed — here we
// model that with a tagged error class. The leader (no retryIf on its own promise — the
// rejection is ITS OWN disconnect) sees the error; the live follower re-executes its OWN
// fn and gets a real result instead of a spurious inherited failure.
await asyncTest("M1: leader disconnects while queued → live follower re-executes and gets a real result", async () => {
class FakeDisconnectError extends Error {}
const leaderGate = Promise.withResolvers();
let leaderRuns = 0;
let followerRuns = 0;
const leaderFn = async () => { leaderRuns++; await leaderGate.promise; throw new FakeDisconnectError("leader client gone"); };
const followerFn = async () => { followerRuns++; return "real-execution"; };
const retryIf = (err) => err instanceof FakeDisconnectError;
const leaderP = singleflight("sf-m1-leader-dc", leaderFn); // becomes leader
const followerP = singleflight("sf-m1-leader-dc", followerFn, retryIf); // joins as follower
leaderGate.resolve(); // leader "disconnects" while holding the flight
await assert.rejects(leaderP, FakeDisconnectError, "the leader itself still sees its own disconnect");
assert.equal(await followerP, "real-execution", "follower got a REAL execution, not the leader's disconnect");
assert.equal(leaderRuns, 1, "leader fn ran once");
assert.equal(followerRuns, 1, "follower re-executed exactly once (as the new leader)");
assert.equal(getInflightStats().inflight, 0, "map fully cleaned up after the retry flight settles");
});
// 8. M1 guard: a follower whose retryIf returns false (server.mjs: its OWN client is also
// gone) inherits the rejection unchanged — no retry, no masked error. And a follower with
// NO retryIf keeps the exact pre-M1 share-everything behavior (test 2 pins the fan-out;
// this pins the predicate=false path specifically for the disconnect error).
await asyncTest("M1: follower with retryIf=false (own client also gone) inherits the leader's rejection, no retry", async () => {
class FakeDisconnectError extends Error {}
const gate = Promise.withResolvers();
let followerRuns = 0;
const leaderFn = async () => { await gate.promise; throw new FakeDisconnectError("leader client gone"); };
const followerFn = async () => { followerRuns++; return "should-never-run"; };
const leaderP = singleflight("sf-m1-both-dc", leaderFn);
const followerP = singleflight("sf-m1-both-dc", followerFn, () => false); // own client dead → no retry
gate.resolve();
await assert.rejects(leaderP, FakeDisconnectError);
await assert.rejects(followerP, FakeDisconnectError, "rejection propagates unchanged when retryIf says no");
assert.equal(followerRuns, 0, "follower fn never executed — no wasted spawn for a dead client");
assert.equal(getInflightStats().inflight, 0);
});
}
await runSingleflightTests();
@@ -1652,12 +1710,23 @@ await asyncTest("readTuiTranscript throws when no text and cap elapses", async (
});
// ── TUI session reaper ───────────────────────────────────────────────────
import { reapStaleTuiSessions, SESSION_PREFIX, buildTuiCmd } from "./lib/tui/session.mjs";
import { reapStaleTuiSessions, sessionPrefixForPort, LEGACY_SESSION_PREFIX, LEGACY_SESSION_NAME_RE, buildTuiCmd } from "./lib/tui/session.mjs";
console.log("\nTUI session reaper:");
test("SESSION_PREFIX is ocp-tui-", () => {
assert.equal(SESSION_PREFIX, "ocp-tui-");
// F7 fix: the session prefix is instance-scoped by listen port so a second OCP
// instance on the same host (different port) is never mistaken for "ours".
test("sessionPrefixForPort embeds the port (F7 instance scoping)", () => {
assert.equal(sessionPrefixForPort(3456), "ocp-tui-3456-");
assert.equal(sessionPrefixForPort(4000), "ocp-tui-4000-");
assert.notEqual(sessionPrefixForPort(3456), sessionPrefixForPort(4000));
});
test("LEGACY_SESSION_NAME_RE matches only the exact old bare-prefix shape, never the new shape", () => {
assert.ok(LEGACY_SESSION_NAME_RE.test(`${LEGACY_SESSION_PREFIX}a1b2c3d4`), "legacy 8-hex shape matches");
assert.ok(!LEGACY_SESSION_NAME_RE.test("ocp-tui-3456-a1b2c3d4"), "new port-scoped shape must NOT match legacy regex");
assert.ok(!LEGACY_SESSION_NAME_RE.test("ocp-tui-a1b2c3"), "too-short suffix must not match");
assert.ok(!LEGACY_SESSION_NAME_RE.test("ocp-tui-a1b2c3d4extra"), "trailing extra chars must not match");
});
console.log("\nTUI command construction (proxy-purity / #4):");
@@ -1726,6 +1795,65 @@ test("buildTuiCmd shq-escapes a token containing shell metacharacters (no inject
}
});
// OCP_TUI_EFFORT (TUI latency, docs/plans/2026-07-13-tui-latency): the pane's claude
// must get an EXPLICIT --effort so its effort never depends on which HOME mode
// resolveTuiHome() picked (real-home inherits the operator's settings.json effortLevel;
// env-token scratch inherits claude's built-in default).
test("buildTuiCmd passes --effort low by default (OCP_TUI_EFFORT unset)", () => {
const save = process.env.OCP_TUI_EFFORT;
try {
delete process.env.OCP_TUI_EFFORT;
const cmd = buildTuiCmd("/usr/bin/claude", "m", "sid-eff1", "/home/u", "cli");
assert.ok(cmd.includes("--effort low"), "default must pin --effort low");
} finally {
if (save === undefined) delete process.env.OCP_TUI_EFFORT;
else process.env.OCP_TUI_EFFORT = save;
}
});
test("buildTuiCmd honors an explicit OCP_TUI_EFFORT level (case/space-normalized)", () => {
const save = process.env.OCP_TUI_EFFORT;
try {
process.env.OCP_TUI_EFFORT = " XHigh ";
const cmd = buildTuiCmd("/usr/bin/claude", "m", "sid-eff2", "/home/u", "cli");
assert.ok(cmd.includes("--effort xhigh"), "explicit level must be passed, normalized");
assert.ok(!cmd.includes("--effort low"), "default must not also appear");
} finally {
if (save === undefined) delete process.env.OCP_TUI_EFFORT;
else process.env.OCP_TUI_EFFORT = save;
}
});
test("buildTuiCmd OCP_TUI_EFFORT=inherit omits --effort entirely (pre-flag argv)", () => {
const save = process.env.OCP_TUI_EFFORT;
try {
process.env.OCP_TUI_EFFORT = "inherit";
const cmd = buildTuiCmd("/usr/bin/claude", "m", "sid-eff3", "/home/u", "cli");
assert.ok(!/--effort/.test(cmd), "inherit must not add --effort");
} finally {
if (save === undefined) delete process.env.OCP_TUI_EFFORT;
else process.env.OCP_TUI_EFFORT = save;
}
});
test("buildTuiCmd falls back to --effort low on an invalid OCP_TUI_EFFORT (never reaches argv)", () => {
const save = process.env.OCP_TUI_EFFORT;
const savedErr = console.error;
try {
process.env.OCP_TUI_EFFORT = "ludicrous'; rm -rf /;'";
let warned = "";
console.error = (...a) => { warned = a.join(" "); };
const cmd = buildTuiCmd("/usr/bin/claude", "m", "sid-eff4", "/home/u", "cli");
assert.ok(cmd.includes("--effort low"), "invalid value must fall back to low");
assert.ok(!cmd.includes("ludicrous"), "invalid raw value must NOT reach the shell string");
assert.ok(/invalid OCP_TUI_EFFORT/.test(warned), "must log a warning");
} finally {
console.error = savedErr;
if (save === undefined) delete process.env.OCP_TUI_EFFORT;
else process.env.OCP_TUI_EFFORT = save;
}
});
test("buildTuiCmd OCP_TUI_FULL_TOOLS=1 grants -p-equivalent tool surface (single-user opt-in)", () => {
const save = { ...process.env };
const restore = () => {
@@ -1764,22 +1892,40 @@ test("buildTuiCmd OCP_TUI_FULL_TOOLS=1 grants -p-equivalent tool surface (single
}
});
test("reaper kills ONLY ocp-tui- sessions, never olp-tui-", () => {
test("reaper kills ONLY this instance's own port-scoped sessions, never olp-tui-", () => {
const killed = [];
const fakeTmux = (args) => {
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-aaaa\nolp-tui-bbbb\nmisc\nocp-tui-cccc\n" };
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\nolp-tui-bbbb\nmisc\nocp-tui-3456-cccc\n" };
if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux });
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.equal(n, 2);
assert.equal(killed.join(","), "ocp-tui-aaaa,ocp-tui-cccc");
assert.equal(killed.join(","), "ocp-tui-3456-aaaa,ocp-tui-3456-cccc");
assert.ok(!killed.includes("olp-tui-bbbb"), "olp-tui-bbbb must never be killed");
});
// F7 fix: a second OCP instance on the same host (different port) must be treated exactly
// like a foreign product prefix — never reaped, never allowed to trigger kill-server.
test("reaper treats a sibling OCP instance on a DIFFERENT port as foreign (F7)", () => {
const killed = [];
const calls = [];
const fakeTmux = (args) => {
calls.push(args.join(" "));
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\nocp-tui-9999-bbbb\n" };
if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.equal(n, 1, "killed only the own-port session");
assert.equal(killed.join(","), "ocp-tui-3456-aaaa");
assert.ok(!killed.includes("ocp-tui-9999-bbbb"), "sibling instance's session (port 9999) must NEVER be killed");
assert.ok(!calls.includes("kill-server"), "kill-server MUST NOT fire — sibling instance's session still live");
});
test("reaper returns 0 when tmux status !== 0 (no server)", () => {
const fakeTmux = (_args) => ({ status: 1, stdout: "" });
const n = reapStaleTuiSessions({ tmux: fakeTmux });
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.equal(n, 0);
});
@@ -1790,7 +1936,7 @@ test("reaper returns 0 for empty session list", () => {
if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux });
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.equal(n, 0);
assert.equal(killed.length, 0);
});
@@ -1803,10 +1949,10 @@ test("reaper kill-servers when the server is ours-only (flush defunct claude zom
const calls = [];
const fakeTmux = (args) => {
calls.push(args.join(" "));
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-aaaa\nocp-tui-bbbb\n" };
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\nocp-tui-3456-bbbb\n" };
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux });
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.equal(n, 2, "killed both of our sessions");
assert.ok(calls.includes("kill-server"), "kill-server fired — reaps the defunct backlog");
});
@@ -1815,10 +1961,10 @@ test("reaper does NOT kill-server when a foreign (non-ocp) session remains (coex
const calls = [];
const fakeTmux = (args) => {
calls.push(args.join(" "));
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-aaaa\nolp-tui-bbbb\n" };
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\nolp-tui-bbbb\n" };
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux });
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.equal(n, 1, "killed only our own session");
assert.ok(!calls.includes("kill-server"), "kill-server MUST NOT fire — would disrupt olp-tui-*");
});
@@ -1826,10 +1972,61 @@ test("reaper does NOT kill-server when a foreign (non-ocp) session remains (coex
test("reaper does NOT kill-server when there is no server (status !== 0)", () => {
const calls = [];
const fakeTmux = (args) => { calls.push(args.join(" ")); return { status: 1, stdout: "" }; };
reapStaleTuiSessions({ tmux: fakeTmux });
reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.ok(!calls.includes("kill-server"), "no server → no kill-server (early return)");
});
// Legacy migration (F7): pre-fix versions created bare-prefix `ocp-tui-<uuid8>` sessions with
// no port segment. includeLegacy is the boot-only opt-in that claims these as our own leftover
// zombies; the periodic sweep never sets it, so a lingering legacy session cannot trigger
// kill-server on a routine 15-minute tick.
console.log("\nTUI legacy-prefix migration (boot-only reap, F7):");
test("reaper leaves legacy bare-prefix sessions untouched by default (includeLegacy unset)", () => {
const killed = [];
const calls = [];
const fakeTmux = (args) => {
calls.push(args.join(" "));
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\nocp-tui-deadbeef\n" };
if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456 });
assert.equal(n, 1, "killed only the own-port session");
assert.ok(!killed.includes("ocp-tui-deadbeef"), "legacy session must NOT be reaped without includeLegacy");
assert.ok(!calls.includes("kill-server"), "legacy session blocks kill-server when not claimed");
});
test("reaper claims legacy bare-prefix sessions when includeLegacy=true (boot-time migration)", () => {
const killed = [];
const calls = [];
const fakeTmux = (args) => {
calls.push(args.join(" "));
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\nocp-tui-deadbeef\n" };
if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456, includeLegacy: true });
assert.equal(n, 2, "both own-port and legacy sessions reaped");
assert.ok(killed.includes("ocp-tui-deadbeef"), "legacy session claimed as our own leftover");
assert.ok(calls.includes("kill-server"), "kill-server fires once no foreign/unclaimed session remains");
});
test("reaper with includeLegacy=true still spares a sibling instance's port-scoped session", () => {
const killed = [];
const calls = [];
const fakeTmux = (args) => {
calls.push(args.join(" "));
if (args[0] === "list-sessions") return { status: 0, stdout: "ocp-tui-3456-aaaa\nocp-tui-deadbeef\nocp-tui-9999-zzzz\n" };
if (args[0] === "kill-session") { killed.push(args[args.indexOf("-t") + 1]); return { status: 0 }; }
return { status: 0, stdout: "" };
};
const n = reapStaleTuiSessions({ tmux: fakeTmux, port: 3456, includeLegacy: true });
assert.equal(n, 2, "own-port + legacy reaped, sibling instance untouched");
assert.ok(!killed.includes("ocp-tui-9999-zzzz"), "sibling instance session must never be claimed as legacy");
assert.ok(!calls.includes("kill-server"), "sibling instance's live session still blocks kill-server");
});
// ── TUI home preparation (scratch vs real) ───────────────────────────────
import { prepareTuiHome, ensureTuiCwdTrusted } from "./lib/tui/session.mjs";
import { mkdtempSync as hMkdtemp, mkdirSync as hMkdir, writeFileSync as hWrite, readFileSync as hRead, existsSync as hExists, readlinkSync as hReadlink } from "node:fs";
@@ -1913,7 +2110,7 @@ test("resolveTuiHome: explicit OCP_TUI_HOME wins regardless of env token (back-c
});
// ── TUI concurrency limiter + drift observability (PR-B: audit C-4 / C-5) ──
import { TuiSemaphore, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
import { TuiSemaphore, SemaphoreAbortError, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
console.log("\nTUI concurrency limiter (C-4):");
@@ -2018,6 +2215,226 @@ await asyncTest("FIX ⑥: slot released on normal completion is immediately reus
assert.equal(sem.inflight, 0);
});
// ── Audit F1 — runtime-lowered/raised limit must actually bite ──────────────
// server.mjs reuses this same TuiSemaphore as `claudeSemaphore`; a PATCH /settings
// maxConcurrent update now calls `claudeSemaphore.setLimit(value)` (see applySettingUpdate's
// "maxConcurrent" case). These tests pin the semaphore-level contract that fix depends on.
console.log("\nF1 — runtime concurrency-limit changes (setLimit / release honoring the current limit):");
await asyncTest("F1: lowering the limit mid-load — release() stops re-granting until inflight drains under the new limit", async () => {
const sem = new TuiSemaphore(3, { maxQueue: 16 });
const g = [deferred(), deferred(), deferred()];
const held = g.map((d) => sem.run(async () => { await d.p; }));
await new Promise((r) => setImmediate(r));
assert.equal(sem.inflight, 3, "3 tasks hold the 3 slots");
// A 4th arrives while at capacity — it queues.
const g4 = deferred();
const queued4 = sem.run(async () => { await g4.p; });
await new Promise((r) => setImmediate(r));
assert.equal(sem.queued, 1, "4th request queued");
// Operator lowers maxConcurrent from 3 to 1 while all 3 original slots are still inflight
// (mirrors a PATCH /settings maxConcurrent=1 hitting server.mjs mid-burst).
sem.setLimit(1);
assert.equal(sem.limit, 1);
// Releasing one of the 3 original holders must NOT hand the freed slot to the queued 4th
// request — before the F1 fix, release() handed slots off unconditionally, so inflight
// would have stayed pinned at the OLD higher occupancy forever.
g[0].resolve();
await held[0];
await new Promise((r) => setImmediate(r));
assert.equal(sem.inflight, 2, "inflight drains toward the new limit, not re-granted");
assert.equal(sem.queued, 1, "4th request is STILL queued — not over-admitted");
g[1].resolve();
await held[1];
await new Promise((r) => setImmediate(r));
assert.equal(sem.inflight, 1, "inflight now exactly at the new limit (1)");
assert.equal(sem.queued, 1, "still queued — inflight(1) is not < limit(1), so no grant yet");
// Releasing the LAST original holder finally drops inflight under the new limit — only
// now does the queued 4th request get granted.
g[2].resolve();
await held[2];
await new Promise((r) => setImmediate(r));
assert.equal(sem.inflight, 1, "queued 4th request now holds the single slot");
assert.equal(sem.queued, 0, "queue drained");
g4.resolve();
await queued4;
assert.equal(sem.inflight, 0);
});
await asyncTest("F1: raising the limit wakes queued waiters immediately, up to the new headroom", async () => {
const sem = new TuiSemaphore(1, { maxQueue: 16 });
const g1 = deferred();
const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
await new Promise((r) => setImmediate(r));
const started = [];
const g2 = deferred(), g3 = deferred();
const t2 = sem.run(async () => { started.push(2); await g2.p; });
const t3 = sem.run(async () => { started.push(3); await g3.p; });
await new Promise((r) => setImmediate(r));
assert.equal(sem.queued, 2, "both queue behind the single holder");
assert.deepEqual(started, [], "neither queued task has started");
// Operator raises maxConcurrent from 1 to 3 (2 units of new headroom) — BOTH queued
// waiters must be woken immediately, without waiting for t1 to release.
sem.setLimit(3);
await new Promise((r) => setImmediate(r));
assert.equal(sem.inflight, 3, "t1 + both newly-woken waiters now hold slots");
assert.equal(sem.queued, 0, "queue drained by the limit raise");
assert.deepEqual(started.sort(), [2, 3], "both queued tasks started without waiting for t1's release");
g1.resolve(); g2.resolve(); g3.resolve();
await Promise.all([t1, t2, t3]);
assert.equal(sem.inflight, 0);
});
await asyncTest("F1: raising the limit wakes only as many waiters as the new headroom allows (FIFO)", async () => {
const sem = new TuiSemaphore(1, { maxQueue: 16 });
const g1 = deferred();
const t1 = sem.run(async () => { await g1.p; });
await new Promise((r) => setImmediate(r));
const started = [];
const g2 = deferred(), g3 = deferred();
const t2 = sem.run(async () => { started.push(2); await g2.p; });
const t3 = sem.run(async () => { started.push(3); await g3.p; });
await new Promise((r) => setImmediate(r));
assert.equal(sem.queued, 2);
sem.setLimit(2); // only 1 unit of new headroom (1 -> 2) — exactly one queued waiter wakes
await new Promise((r) => setImmediate(r));
assert.equal(sem.inflight, 2);
assert.equal(sem.queued, 1, "one waiter still queued — only one slot of headroom existed");
assert.deepEqual(started, [2], "FIFO: the earlier-queued waiter (t2) wakes, not t3");
// Freeing t1's slot afterward still honors the (now current) limit of 2 via release()'s
// normal path — the still-queued t3 gets in once a slot actually frees.
g1.resolve();
await t1;
await new Promise((r) => setImmediate(r));
assert.deepEqual(started, [2, 3], "t3 granted once a slot frees, honoring the raised limit");
assert.equal(sem.queued, 0);
g2.resolve(); g3.resolve();
await t2; await t3;
assert.equal(sem.inflight, 0);
});
// ── Audit F2 — queued waiters must be cancellable on client disconnect ──────
// server.mjs wires an AbortSignal derived from the client's res "close" event into
// claudeSemaphore.acquire()/tuiSemaphore.acquire() (see closeSignalFor + acquireClaudeSlot /
// callClaudeTui). These tests pin the semaphore-level cancellation contract that depends on.
console.log("\nF2 — queued-wait cancellation via AbortSignal (client disconnect while queued):");
await asyncTest("F2: aborting a QUEUED waiter rejects with SemaphoreAbortError and SPLICES it out (queued drops immediately, not just flagged)", async () => {
const sem = new TuiSemaphore(1, { maxQueue: 16 });
const g1 = deferred();
const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
await new Promise((r) => setImmediate(r));
const controller = new AbortController();
const acquire2 = sem.acquire(controller.signal); // queues behind t1
await new Promise((r) => setImmediate(r));
assert.equal(sem.queued, 1, "second acquire queued");
controller.abort(); // simulates the client disconnecting while still queued
await assert.rejects(acquire2, SemaphoreAbortError, "cancelled waiter rejects with SemaphoreAbortError");
assert.equal(sem.queued, 0, "cancelled waiter is REMOVED — queue length drops immediately");
assert.equal(sem.inflight, 1, "t1's slot is untouched by the cancellation");
// Prove the cancelled waiter never later acquires a slot: free t1's slot and confirm
// nobody is waiting to receive it (the queue is genuinely empty, not just decremented).
g1.resolve();
await t1;
assert.equal(sem.inflight, 0, "slot freed with nobody queued — the cancelled waiter never got it");
});
await asyncTest("F2: an already-aborted signal rejects acquire() immediately, never touching the wait queue", async () => {
const sem = new TuiSemaphore(1, { maxQueue: 16 });
const g1 = deferred();
const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
await new Promise((r) => setImmediate(r));
const controller = new AbortController();
controller.abort(); // client already gone before this request ever tries to acquire
await assert.rejects(sem.acquire(controller.signal), SemaphoreAbortError);
assert.equal(sem.queued, 0, "never entered the wait queue at all");
g1.resolve(); await t1;
});
await asyncTest("F2: cancelling one queued waiter preserves FIFO order for the others", async () => {
const sem = new TuiSemaphore(1, { maxQueue: 16 });
const g1 = deferred();
const t1 = sem.run(async () => { await g1.p; });
await new Promise((r) => setImmediate(r));
const started = [];
const cA = new AbortController();
const cB = new AbortController();
const accA = sem.acquire(cA.signal).then(() => started.push("A"));
const accB = sem.acquire(cB.signal).then(() => started.push("B"));
const g3 = deferred();
const t3 = sem.run(async () => { started.push("C"); await g3.p; });
await new Promise((r) => setImmediate(r));
assert.equal(sem.queued, 3, "A, B, C all queued behind t1");
cB.abort(); // B (the middle waiter) disconnects
await assert.rejects(accB, SemaphoreAbortError);
assert.equal(sem.queued, 2, "B removed; A and C remain, in original relative order");
g1.resolve();
await t1;
await new Promise((r) => setImmediate(r));
assert.deepEqual(started, ["A"], "A (queued first, still present) is granted next — FIFO preserved after B's removal");
assert.equal(sem.inflight, 1);
assert.equal(sem.queued, 1, "C still waiting");
sem.release(); // A was acquired directly (not via run()) — free its slot manually
await new Promise((r) => setImmediate(r));
assert.deepEqual(started, ["A", "C"], "C granted next");
g3.resolve();
await t3;
assert.equal(sem.inflight, 0);
});
await asyncTest("F2/L2: abort AFTER grant is a no-op — waiter keeps its slot, no rejection, slot released exactly once", async () => {
const sem = new TuiSemaphore(1, { maxQueue: 16 });
const g1 = deferred();
const t1 = sem.run(async () => { await g1.p; }); // holds the only slot
await new Promise((r) => setImmediate(r));
const controller = new AbortController();
let granted = false;
const acq = sem.acquire(controller.signal).then(() => { granted = true; });
await new Promise((r) => setImmediate(r));
assert.equal(sem.queued, 1, "waiter queued behind t1");
// t1 finishes → release() shifts the waiter out and grants it the slot (waiter() detaches
// the abort listener before resolving).
g1.resolve();
await t1;
await acq;
assert.equal(granted, true, "waiter was granted the slot");
assert.equal(sem.inflight, 1, "granted waiter holds the slot");
assert.equal(sem.queued, 0);
// The client disconnects AFTER the grant — the abort-after-grant race. onAbort must be a
// no-op (the waiter is no longer in _waiters; idx===-1 guard): no rejection materializes,
// the queue is untouched, and the slot is still owned by the (already-resolved) acquirer.
controller.abort();
await new Promise((r) => setImmediate(r));
assert.equal(sem.inflight, 1, "abort after grant did NOT revoke or double-free the slot");
assert.equal(sem.queued, 0, "abort after grant did not corrupt queue accounting");
// The slot is released exactly once via the normal path and is immediately reusable.
sem.release();
assert.equal(sem.inflight, 0, "slot released exactly once via the normal path");
await sem.run(async () => {}); // prove the semaphore is fully healthy afterward
assert.equal(sem.inflight, 0);
});
console.log("\nTUI drift observability (C-5):");
test("recordTuiEntrypoint: observed 'cli' is NOT a mismatch and sets lastEntrypoint", () => {
@@ -2398,8 +2815,131 @@ test("isLoopbackBind: '100.64.0.1' → false (Tailscale IP)", () => {
assert.equal(isLoopbackBind("100.64.0.1"), false);
});
// ── Cleanup ──
closeDb();
// ── Spawn-auth primitives (F3 / F5 / F6, lib/spawn-auth.mjs) ──
// Pure, dependency-injected primitives extracted from server.mjs so the spawn-token concurrency /
// caching / expiry logic is testable without booting the server or mocking execFileSync/spawn.
console.log("\nSpawn-auth (F3 mutex / F5 TTL cache + label memo / F6 expiry gate):");
console.log(`\n=== Results: ${passed} passed, ${failed} failed ===\n`);
process.exit(failed > 0 ? 1 : 0);
// F5: expiry gate — the load-bearing invariant that lets a short-TTL keychain cache stay safe.
test("isTokenExpiring: creds within 5-min buffer → true", () => {
assert.equal(isTokenExpiring({ expiresAt: 1000 }, 1000 - 300000, 300000), true); // exactly at buffer edge
assert.equal(isTokenExpiring({ expiresAt: 1000 }, 900, 300000), true); // past the edge
});
test("isTokenExpiring: creds well beyond buffer → false", () => {
assert.equal(isTokenExpiring({ expiresAt: 10_000_000 }, 0, 300000), false);
});
test("isTokenExpiring: no expiresAt (long-lived env token) → never expiring", () => {
assert.equal(isTokenExpiring({ accessToken: "x" }, Date.now(), 300000), false);
assert.equal(isTokenExpiring(null, Date.now(), 300000), false);
});
// F5: last-good label ordering — one exec instead of two on the steady-state keychain path.
test("orderLabelsLastGoodFirst: last-good label is tried first", () => {
const labels = ["A", "B"];
assert.deepEqual(orderLabelsLastGoodFirst(labels, "B"), ["B", "A"]);
});
test("orderLabelsLastGoodFirst: null/unknown last-good → original order, fresh array", () => {
const labels = ["A", "B"];
assert.deepEqual(orderLabelsLastGoodFirst(labels, null), ["A", "B"]);
assert.deepEqual(orderLabelsLastGoodFirst(labels, "Z"), ["A", "B"]);
assert.notEqual(orderLabelsLastGoodFirst(labels, null), labels); // does not mutate/alias input
});
// F5: TTL cache — bounds how often we RE-READ the keychain (not how often we re-decide expiry).
test("createTtlCache: serves cached value within TTL, re-produces after TTL", () => {
const cache = createTtlCache({ ttlMs: 30000 });
let calls = 0;
const produce = () => { calls++; return `v${calls}`; };
assert.equal(cache.get(produce, 0), "v1");
assert.equal(cache.get(produce, 10000), "v1"); // within TTL → cached, producer NOT called
assert.equal(calls, 1);
assert.equal(cache.get(produce, 40000), "v2"); // past TTL → re-produced
assert.equal(calls, 2);
});
test("createTtlCache: caches a null miss (absent source not re-probed within TTL)", () => {
const cache = createTtlCache({ ttlMs: 30000 });
let calls = 0;
const produce = () => { calls++; return null; };
assert.equal(cache.get(produce, 0), null);
assert.equal(cache.get(produce, 5000), null);
assert.equal(calls, 1); // the null was cached, not re-probed
});
// F5 core safety property: a short-TTL cache CANNOT reintroduce the #146 forever-stale bug because
// the expiry gate is applied to the CACHED creds on every use. The cache keeps returning the same
// creds object, but isTokenExpiring flips to true the moment the clock crosses the expiry buffer.
test("TTL cache respects expiry gate: cached creds still rejected once clock passes expiry", () => {
const cache = createTtlCache({ ttlMs: 30000 });
const creds = { accessToken: "tok", expiresAt: 1_000_000 };
// t=980_000: cached AND not yet within the 5-min (300_000) buffer → usable.
const c1 = cache.get(() => creds, 980_000 - 300_000 - 1);
assert.equal(isTokenExpiring(c1, 980_000 - 300_000 - 1, 300000), false);
// t=800_000 later: SAME cached object returned (within TTL of the second read window), but now
// within the expiry buffer → gate rejects it → caller falls back to real HOME. No forever-stale.
const c2 = cache.get(() => creds, 990_000);
assert.equal(c2, c1, "cache returns the same creds object");
assert.equal(isTokenExpiring(c2, 990_000, 300000), true, "expiry gate still fires on cached creds");
});
// ── Async: F3 real-HOME fallback serialization mutex ──
async function runAsyncTests() {
await testAsync("createSerialMutex: second waiter blocks until first holder releases", async () => {
const mutex = createSerialMutex();
const order = [];
const rel1 = await mutex.acquire();
order.push("h1-enter");
let secondEntered = false;
const p2 = mutex.acquire().then((rel2) => { secondEntered = true; order.push("h2-enter"); return rel2; });
await new Promise((r) => setTimeout(r, 15));
assert.equal(secondEntered, false, "second waiter must NOT enter while first holds the mutex");
order.push("h1-release");
rel1();
const rel2 = await p2;
assert.equal(secondEntered, true, "second waiter enters only after release");
rel2();
assert.deepEqual(order, ["h1-enter", "h1-release", "h2-enter"]);
});
await testAsync("createSerialMutex: N acquires run strictly in FIFO order, never overlapping", async () => {
const mutex = createSerialMutex();
const events = [];
let active = 0;
async function critical(id) {
const rel = await mutex.acquire();
active++;
assert.equal(active, 1, `only one holder at a time (id=${id})`);
events.push(`start${id}`);
await new Promise((r) => setTimeout(r, 5));
events.push(`end${id}`);
active--;
rel();
}
await Promise.all([critical(1), critical(2), critical(3)]);
assert.deepEqual(events, ["start1", "end1", "start2", "end2", "start3", "end3"]);
});
await testAsync("createSerialMutex: release() is idempotent (double-release does not double-admit)", async () => {
const mutex = createSerialMutex();
const rel1 = await mutex.acquire();
rel1();
rel1(); // second call must be a no-op
const rel2 = await mutex.acquire(); // should acquire cleanly, exactly once
let thirdEntered = false;
const p3 = mutex.acquire().then((r) => { thirdEntered = true; return r; });
await new Promise((r) => setTimeout(r, 15));
assert.equal(thirdEntered, false, "double-release must not have leaked an extra admit slot");
rel2();
(await p3)();
});
}
// ── Cleanup ──
runAsyncTests().then(() => {
closeDb();
console.log(`\n=== Results: ${passed} passed, ${failed} failed ===\n`);
process.exit(failed > 0 ? 1 : 0);
}).catch((e) => {
console.error("async test runner crashed:", e);
closeDb();
process.exit(1);
});