mirror of
https://github.com/dtzp555-max/ocp.git
synced 2026-07-19 09:44:07 +00:00
feat(tui): real SSE streaming via claude's MessageDisplay hook (OCP_TUI_STREAM, default off) (#159)
* feat(tui): real SSE streaming via claude's MessageDisplay hook (OCP_TUI_STREAM, default off) Backlog #2. TUI-mode `stream:true` turns can now emit real SSE `delta.content` chunks as `claude` generates them, instead of buffering the turn and replaying it with streamStringAsSSE. Opt-in: with OCP_TUI_STREAM unset/0 the spawn argv, the SSE bytes and the cache behaviour are byte-for-byte unchanged (asserted by test). This PR does NOT mirror any cli.js function, so no `cli.js:NNNN` citation applies, and per CLAUDE.md's hard requirement #1 that is stated explicitly here rather than left implicit: - We consume claude's OWN `MessageDisplay` hook surface AS EMITTED — forwarding, not inventing. No new endpoint, no fabricated protocol, no new field. - The TUI spawn is OCP-owned surface: ADR 0007 owns it, not cli.js. - The SSE wire shapes are the OpenAI chat/completions streaming spec, adopted by ADR 0006. Every frame emitted here (role chunk, content-delta chunk, stop chunk, `[DONE]`, and the post-header {error:{message,type}} frame) is COPIED from callClaudeStreaming, the -p path. /health gains additive fields only (streamEnabled + 4 counters) — same grandfathered B.2 rationale as the existing tui block (ADR 0006). Existing keys are untouched. `claude` fires MessageDisplay per rendered block, handing the hook the RAW MARKDOWN SOURCE of an incremental delta on stdin. The hook is registered with `--settings` on the ordinary interactive spawn (no -p, no --bare) — verified to leave the billing pool alone. Sink: a static sh hook script appends each payload to `<streamDir>/<session_id>.jsonl`; OCP polls that file and forwards deltas as SSE. The per-session-id keying is MANDATORY, not an optimization — OCP_TUI_MAX_CONCURRENT defaults to 2, so two claude panes already run at once and a shared sink would splice one client's deltas into another's stream. Warm-pool compatible (a separate in-flight PR depends on this): the hook script AND the settings file are static — nothing request-specific is baked in at spawn time. The sink path reaches the pane through its own env (OCP_TUI_STREAM_FILE) and derives from the session-id, which a pre-booted pane fixes at boot. The hook is SYNCHRONOUS (forceSyncExecution: claude blocks on it), so the script writes and exits: one `cat` append, nothing else. Measured p50 7.2ms / p90 14.7ms per fire, ~50ms across a whole turn — noise against a 6-10s turn. It remains the terminal-turn signal, the source of the returned/cached text T, and the input to the honesty gates. The delta stream is a low-latency MIRROR, never a replacement: - the truncation gate (C-2) and auth-banner gate (C-1, issue #133) run BEFORE anything is committed or flushed, unchanged; - at end of turn the streamed bytes are asserted against T. Equal -> serve. A strict PREFIX of T -> top up from the transcript so the client still receives exactly T (counted). NOT a prefix -> REFUSE the turn: SSE error frame, no cache, no success, streamDivergences++. Serving text the transcript disagrees with is the failure class ALIGNMENT.md exists to prevent, so this fails loud rather than degrading quietly; - only T is ever cached — never the concatenated deltas. The auth banner needs prevention, not just detection (SSE deltas cannot be un-sent), so the first OCP_TUI_STREAM_HOLDBACK (100) chars are withheld: the default banner detector cannot match a message longer than 100 chars, so releasing past that provably cannot leak a banner. A custom CLAUDE_TUI_ERROR_PATTERNS has no such bound — OCP warns at boot. - BANNER, before/after the spawn change: `Sonnet 4.6 with low effort · Claude Max` both, including on the pane the server itself spawns. Never `API Usage Billing`. Transcript entrypoint stays "cli". --settings is not a --bare-class flag. - --settings MERGES with <HOME>/.claude/settings.json rather than clobbering it (the user-level settings' `env` block still reached the hook), so the isolated-HOME settings story (permissions / additionalDirectories) survives. - EXACTNESS: 8/8 varied prompts (short, long, markdown, code fence, multilingual, JSON, table, unicode) byte-exact vs transcript T, streamed AND buffered. 0 top-ups, 0 divergences over 15 streamed turns. - TTFT: buffered delivers NOTHING until the turn ends (TTFB == total, 7.5-15.8s). Streamed sends headers at ~25ms (heartbeat covers the pre-first-delta silence) and first content mid-generation, e.g. markdown 7.9s first chunk / 12.8s total; long 9.7s / 17.4s. - CONCURRENCY DEMUX: two concurrent streamed turns (ALPHA/BRAVO), tui.inflight peaked at 2, each read its own session-keyed transcript, ZERO cross-contamination. - AUTH-BANNER GATE under streaming, both layers: a short banner-like turn reached the client as 0 content chunks + an SSE error frame (never emitted); a long one was streamed but still ended on an error frame, not finish_reason:"stop", and was not cached. - DISCONNECT mid-turn: pane torn down and semaphore slot released within 1s (info-logged, not booked as a model error). - THINKING: not leaked. Opus 4.8 + xhigh turns carry a thinking block with a signature but `thinking:""` (the reasoning text is not persisted in interactive mode), both MessageDisplay text-extraction sites in the 2.1.207 bundle filter type==="text", and no reasoning prose appeared in any delta; concat===T held exactly on the single-message turn. - npm test: 282 passed, 0 failed (was 267 on main; +15). The transcript keeps only the model's LAST assistant message. A turn where the model narrates before calling a tool therefore has two messages, and T is only the second. If the narration exceeds the holdback it has already been streamed and cannot be retracted -> the turn is REFUSED. Reproduced live: Opus narrated 475 chars before a Bash call. The assembler discards a prior message's text when nothing has been emitted yet (so short narration is handled correctly and stays exact), and raising OCP_TUI_STREAM_HOLDBACK above the narration length rescues the turn — verified on that exact transcript: holdback>=500 -> served, exact=true. Documented in README and ADR 0007; this is why streaming is opt-in and off by default. ADR 0007 line 59 ("no real token streaming — deliberate") is amended, not silently contradicted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tui): re-integrate streaming onto the warm-pane pool (#158) — install the hook at BOOT Rebasing backlog #2 (streaming) onto #158 (warm pane pool) is not a textual merge: #158 split the monolithic runTuiTurn into bootTuiPane + runTuiTurn, and streaming had patched the monolith. Re-integrating it in the OLD shape would have compiled, passed every existing test, and been WRONG. The bug that shape would have shipped: the sink was derived at TURN time from a streamDir argument. But a POOLED pane is pre-booted long before any request exists — so on a pool HIT runTuiTurn never cold-boots, no hook was ever registered on that pane, and the turn would silently serve BUFFERED. Every miss streams, every hit does not; no error, no failing test. The operator sees "streaming does nothing in production" and has nothing to grep for. Fix — install the hook where the pane is born: - bootTuiPane({ streamDir }) registers the MessageDisplay hook at spawn and returns the pane's own sink (pane.streamFile), keyed by the pane's own --session-id. The hook script and settings file are STATIC (one pair per streamDir); the only per-turn thing is the sink path, and it is fixed at boot. So nothing request-specific is baked into a spawn. - runTuiTurn reads pane.streamFile — never recomputes it — so a warm pane and a cold pane stream through byte-for-byte the same path. - server.mjs threads the same streamDir into the pool's bootPane closure, so pre-booted panes carry the hook too. TUI_STREAM/TUI_STREAM_DIR now declare before the pool needs them. Three regression guards added (test-features.mjs), and the third was MUTATION-TESTED: with the fix reverted to the turn-time shape it fails ("the pooled pane's deltas must reach the client"), with the fix in place it passes. A guard nobody has watched fail is not a guard. /health: pool + stream* fields are now a union — the shape assertion asserts CONTAINMENT of the seven grandfathered keys plus an exact added-set, so a future field that silently REPLACED an original key cannot pass. Class B (ADR 0007, OCP-owned TUI spawn) — cli.js does NOT perform this operation. npm test: 313 passed, 0 failed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VqgWJcjxrjjL9L9SkpZyXR * fix(tui): close the streaming auth-banner leak + 6 further review findings (PR #159) Independent review (Iron Rule 10) found a HIGH bug by EXECUTING the code, not reading it. All seven findings fixed. F1 and F3 were merge-blocking. F1 (HIGH) — the auth-banner holdback was bypassed after the first release. TuiDeltaAssembler.released was set once and never reset at a message_id boundary, so the holdback + detectError predicate guarded only the FIRST message of a turn. In production's own configuration (OCP_TUI_FULL_TOOLS=1, where multi-message tool-using turns are the norm): the model narrates past the holdback before a tool call -> released; credentials expire mid-turn -> claude renders the 401 as ordinary assistant TEXT as a NEW message -> push() took the `if (this.released)` branch and handed the banner verbatim to the client. That is precisely the silent-error case the C-1 gate exists to prevent. Detection survived (the turn was still refused at finalize) but PREVENTION did not. Fix: once a message boundary follows an emit, the turn is already unrecoverable — finalize() will refuse it — so push() now emits NOTHING further for the rest of the turn. Second hole in the same predicate: detectTuiUpstreamError() trims before applying its <=100-char rule, so 101 whitespace chars trimmed to "" -> detector had nothing to classify -> returned null -> release fired having screened nothing. Release now gates on the TRIMMED length, so both sides of the check talk about the same string. F2 — the "provably safe" claim in stream.mjs, ADR 0007 and README was unsound as written. Restated with both required halves: (i) nothing is emitted until the trimmed accumulation exceeds the detector's max banner length, AND (ii) no emission at all once a message boundary follows an emit. Half (i) alone only ever covered a turn's first message. F3 (blocker) — prepareStreamHook was write-if-missing, so md-hook.sh could never be updated OR repaired: a host that booted once under an older version was stuck on that HOOK_SCRIPT forever, and a non-atomic write interrupted mid-flight left a TRUNCATED script that existsSync() called fine — on a hook claude BLOCKS on synchronously. Now written unconditionally via tmp+renameSync (the pattern already used by ensureTuiCwdTrusted). F4 — the two spawn paths differed for non-streaming requests: the pool installed the hook whenever OCP_TUI_STREAM was on (correct — a pre-booted pane cannot know what request it will serve), but the cold path gated it on this turn's onDelta. So one stream:false request got --settings on a pool HIT and not on a MISS: two spawn argvs for the identical request, on this project's billing-classification surface. Both paths now gate on TUI_STREAM alone; whether the sink is POLLED remains correctly gated on onDelta. F5 — pool._drop() killed the pane but orphaned its sink file; the reap tick drains the whole pool, so sinks accumulated with no GC path. Now removed best-effort on every drop path. F6 — /health counters did not measure what they documented: streamTurns was incremented only AFTER the honesty gates, hiding exactly the turns an operator most wants to see (and making streamDivergences/streamTurns a meaningless ratio); streamDeltas counted every fire while claiming to count forwarded ones. Counters and docs now agree. F7 — total hook failure was silent: zero fires per turn still yields ok:true/exact:false and a normal, fully-buffered answer. Only streamTopUps moved, which the code itself calls benign. Added streamZeroDeltaTurns (+ a tui_stream_zero_deltas warning) to separate "the hook is dead" from "one fire was dropped". Tests: 316 passed, 0 failed (was 313). Every new guard MUTATION-TESTED — with each fix reverted the guard named for it fails, and passes with the fix restored: - drop the restartedAfterEmit guard -> 2 failed (incl. the strengthened old test) - revert trim() in the release gate -> 1 failed - revert F3 to write-if-missing -> 1 failed The pre-existing test "new message_id AFTER an emit" asserted finalize().ok === false but never checked what push() RETURNED — so it passed while F1 was live, documenting the leak instead of catching it. Strengthened to assert the emission, not just the verdict. Class B (ADR 0007, OCP-owned TUI spawn) — cli.js does NOT perform this operation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VqgWJcjxrjjL9L9SkpZyXR --------- Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
+254
-2
@@ -47,6 +47,7 @@ import { runTuiTurn, reapStaleTuiSessions, resolveTuiHome, bootTuiPane, tuiPaneH
|
||||
import { detectTuiUpstreamError } from "./lib/tui/transcript.mjs";
|
||||
import { TuiSemaphore, SemaphoreAbortError, recordTuiEntrypoint, buildTuiHealthBlock } from "./lib/tui/semaphore.mjs";
|
||||
import { TuiPanePool, resolvePoolSize, POOL_MAX_SIZE } from "./lib/tui/pool.mjs";
|
||||
import { TuiDeltaAssembler, DEFAULT_HOLDBACK_CHARS } from "./lib/tui/stream.mjs";
|
||||
import { createSerialMutex, createTtlCache, isTokenExpiring, orderLabelsLastGoodFirst } from "./lib/spawn-auth.mjs";
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
@@ -352,8 +353,48 @@ const tuiSemaphore = new TuiSemaphore(TUI_MAX_CONCURRENT);
|
||||
const tuiStats = {
|
||||
lastEntrypoint: null, // last observed cc_entrypoint from the transcript ("cli" | "sdk-cli" | null)
|
||||
entrypointMismatches: 0, // count of cli-expected-but-got-other turns
|
||||
streamTurns: 0, // streamed TUI turns ATTEMPTED (counted before the honesty gates — F6)
|
||||
streamDeltas: 0, // MessageDisplay hook fires OBSERVED (forwarded + held-back — F6)
|
||||
streamTopUps: 0, // turns where the delta stream != T but was a safe PREFIX of it
|
||||
streamDivergences: 0, // turns REFUSED: emitted bytes were not a prefix of T
|
||||
streamZeroDeltaTurns: 0, // streamed turns where the hook fired ZERO times (F7 — the hook is
|
||||
// dead, not just one fire dropped; distinct from streamTopUps)
|
||||
};
|
||||
|
||||
// ── TUI real streaming (backlog #2) — opt-in; default OFF ────────────────
|
||||
// When ON *and* TUI_MODE is on *and* the client asked for stream:true, the turn is emitted
|
||||
// as real SSE delta.content chunks as claude renders them, sourced from claude's own
|
||||
// MessageDisplay hook (lib/tui/stream.mjs). When OFF, the buffered
|
||||
// callClaudeTui → streamStringAsSSE path below is byte-for-byte unchanged — the spawn does
|
||||
// not even get --settings. Opt-in is deliberate: the buffered path is stable production.
|
||||
//
|
||||
// Honest expectation (docs/plans/2026-07-13-tui-latency/streaming-spike.md): this moves the
|
||||
// FIRST byte, not the last. A consumer that must parse a complete reply gains nothing; a
|
||||
// progressively-rendering chat UI gains the ~4s between first delta and last. It does not
|
||||
// move the ~6s TTFT floor of TUI mode.
|
||||
const TUI_STREAM = process.env.OCP_TUI_STREAM === "1";
|
||||
const TUI_STREAM_DIR = process.env.OCP_TUI_STREAM_DIR || `${process.env.HOME}/.ocp-tui/stream`;
|
||||
// First-bytes holdback — the auth-banner gate's (C-1) survival mechanism under streaming.
|
||||
// See TuiDeltaAssembler: nothing is emitted for a message until its TRIMMED accumulation
|
||||
// exceeds this, which puts it out of the default banner detector's <=100-char reach — the
|
||||
// FIRST of the two halves of the guarantee (see the assembler's class comment for the second:
|
||||
// no further emission at all once a message boundary follows an emit). Only raise it.
|
||||
const TUI_STREAM_HOLDBACK = parseInt(process.env.OCP_TUI_STREAM_HOLDBACK || String(DEFAULT_HOLDBACK_CHARS), 10);
|
||||
if (TUI_MODE && TUI_STREAM && process.env.CLAUDE_TUI_ERROR_PATTERNS != null && TUI_STREAM_HOLDBACK <= DEFAULT_HOLDBACK_CHARS) {
|
||||
// The holdback's FIRST-MESSAGE half (see TuiDeltaAssembler) is sound for the DEFAULT
|
||||
// auth-banner detector (which cannot match a message longer than 100 chars). An
|
||||
// operator-supplied pattern set has no such bound, so a banner longer than the holdback
|
||||
// could reach the client before the terminal gate rejects the turn. (The second half — no
|
||||
// further emission once a message boundary follows an emit — holds regardless of the
|
||||
// detector; this warning is only about the first-message case.)
|
||||
console.error(
|
||||
`[tui] WARNING: OCP_TUI_STREAM=1 with a custom CLAUDE_TUI_ERROR_PATTERNS and holdback=${TUI_STREAM_HOLDBACK}.\n` +
|
||||
" The streaming holdback's first-message coverage is sound only against the DEFAULT banner\n" +
|
||||
" detector (<=100 chars). Raise OCP_TUI_STREAM_HOLDBACK above your longest custom banner, or\n" +
|
||||
" the first chars of one could be streamed before the end-of-turn gate refuses the turn."
|
||||
);
|
||||
}
|
||||
|
||||
// ── Warm pane pool (docs/plans/2026-07-13-tui-latency #3) — opt-in; default OFF ─────────
|
||||
// OCP_TUI_POOL_SIZE=0 (default) => tuiPool is null => runTuiTurn's cold-boot path is
|
||||
// byte-for-byte unchanged. Set it to N (clamped to POOL_MAX_SIZE) to keep N pre-booted
|
||||
@@ -389,6 +430,14 @@ const tuiPool = TUI_POOL_SIZE > 0
|
||||
name: ident.name,
|
||||
requireReady: true, // a pane that never reached its input bar must not be enlisted
|
||||
bootMs: POOL_BOOT_MS, // background pre-boot — no client is blocked, so be patient
|
||||
// Warm panes must carry the MessageDisplay hook too, or every pool HIT would
|
||||
// silently fall back to buffered while every MISS streamed — the two paths have to
|
||||
// spawn identically (F4). Gated on TUI_STREAM, the deployment-wide switch — NOT on any
|
||||
// particular request's stream:true/false, which does not exist yet at pre-boot time.
|
||||
// The runTuiTurn cold-boot call site (callClaudeTui, below) mirrors this exact gate for
|
||||
// the same reason. bootTuiPane derives the sink from the pane's own session-id, which is
|
||||
// minted above, so nothing request-specific is baked in at pre-boot time.
|
||||
streamDir: TUI_STREAM ? TUI_STREAM_DIR : null,
|
||||
}),
|
||||
killPane: (name) => { try { spawnSync(process.env.OCP_TUI_TMUX_BIN || "tmux", ["kill-session", "-t", name]); } catch { /* already gone */ } },
|
||||
paneHealthy: (name) => tuiPaneHealthy((args) => spawnSync(process.env.OCP_TUI_TMUX_BIN || "tmux", args, { encoding: "utf8" }), name),
|
||||
@@ -1354,7 +1403,14 @@ async function callClaude(model, messages, conversationId, keyName, res) {
|
||||
// Authority: claude CLI v2.1.158 interactive mode (cc_entrypoint=cli).
|
||||
// SECURITY: A-path single-user ONLY — home is NOT isolation (see ADR 0007).
|
||||
// `res` (optional, F2) is the client's http.ServerResponse — see closeSignalFor.
|
||||
async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
|
||||
//
|
||||
// `streamCtx` (optional, OCP_TUI_STREAM): { emit(text), signal } — when present the turn is
|
||||
// ALSO streamed live via claude's MessageDisplay hook. The contract is unchanged: this still
|
||||
// returns the TRANSCRIPT's text (T), the honesty gates still run on T before anything is
|
||||
// committed, and the cache still stores T — never the concatenated deltas. streamCtx.emit is
|
||||
// the SSE sink; streamCtx.signal is the client's disconnect signal, which tears the pane down
|
||||
// mid-turn instead of holding the semaphore slot for a dead socket.
|
||||
async function callClaudeTui(model, messages, _conversationId, _keyName, res, streamCtx = null) {
|
||||
const cliModel = MODEL_MAP[model] || model;
|
||||
const prompt = messagesToPrompt(messages); // includes system as [System] inline
|
||||
recordModelRequest(cliModel, prompt.length);
|
||||
@@ -1382,6 +1438,23 @@ async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
|
||||
// release() runs in a finally so any throw from runTuiTurn (tmux spawn failure,
|
||||
// paste-not-landed) OR from the honesty gates below (truncation / error banner) can NEVER
|
||||
// leak a slot. tuiSemaphore.inflight feeds /health.
|
||||
// Streaming assembler (null when OCP_TUI_STREAM is off — then runTuiTurn gets no onDelta,
|
||||
// spawns no hook, and behaves byte-for-byte as before). It owns the auth-banner holdback
|
||||
// and the message scoping; see lib/tui/stream.mjs.
|
||||
const assembler = streamCtx ? new TuiDeltaAssembler({ holdbackChars: TUI_STREAM_HOLDBACK }) : null;
|
||||
// F6: counted here — the moment a streamed turn is ATTEMPTED — not after the honesty gates
|
||||
// below. A turn refused by the truncation or auth-banner gate is exactly the turn an operator
|
||||
// most wants visible in streamTurns; counting only turns that reached the gates made
|
||||
// streamDivergences/streamTurns silently exclude its own worst cases from the denominator.
|
||||
if (assembler) tuiStats.streamTurns++;
|
||||
const onDelta = assembler
|
||||
? (payload) => {
|
||||
const out = assembler.push(payload);
|
||||
tuiStats.streamDeltas++; // every hook fire OBSERVED, not just forwarded ones — see the
|
||||
// /health field doc in lib/tui/semaphore.mjs (F6)
|
||||
if (out) streamCtx.emit(out); // released past the holdback — safe to show the client
|
||||
}
|
||||
: null;
|
||||
try {
|
||||
const { text, entrypoint, truncated } = await runTuiTurn({
|
||||
prompt,
|
||||
@@ -1403,6 +1476,18 @@ async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
|
||||
? ({ warm }) => logEvent("info", warm ? "tui_pool_hit" : "tui_pool_miss",
|
||||
{ model: cliModel, warmRemaining: tuiPool.warm })
|
||||
: null,
|
||||
onDelta,
|
||||
// Gated on TUI_STREAM (the deployment-wide switch), NOT on `assembler` (this REQUEST's
|
||||
// stream:true/false) — F4 fix. The pool's bootPane closure above installs the hook on
|
||||
// every warm pane whenever TUI_STREAM is on, regardless of what any given future request
|
||||
// asks for (a pre-booted pane cannot know that yet); the cold path must match, or a
|
||||
// stream:false request gets --settings on a pool HIT and not on a pool MISS — two
|
||||
// different spawn argvs for the identical request, which this project's alignment/billing
|
||||
// posture cannot tolerate. Whether the hook's OUTPUT is actually consumed for THIS turn is
|
||||
// decided downstream by `onDelta` (null when assembler is null), so a non-streaming
|
||||
// request still never polls or emits — it just spawns identically either way.
|
||||
streamDir: TUI_STREAM ? TUI_STREAM_DIR : null,
|
||||
abortSignal: streamCtx ? streamCtx.signal : null,
|
||||
});
|
||||
// ── Honesty gates (issue #133) ─ run BEFORE recordModelSuccess / cache write-back.
|
||||
// A throw here propagates to the catch below (recordModelError + reject), so the
|
||||
@@ -1427,6 +1512,59 @@ async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
|
||||
throw new Error("tui_upstream_error: claude CLI returned an in-session error banner instead of an answer");
|
||||
}
|
||||
|
||||
// ── Streaming safety net — the transcript is the authority, the deltas are the mirror.
|
||||
// Runs AFTER the two gates above (so a truncated turn or an auth banner is never
|
||||
// reconciled, let alone flushed) and BEFORE recordModelSuccess / the caller's cache
|
||||
// write. Three outcomes:
|
||||
// exact — concat(deltas) === T. The invariant held; emit whatever is still held
|
||||
// back (a short answer never passes the holdback, so this is its whole text).
|
||||
// top-up — what we emitted is a strict PREFIX of T but the deltas did not add up to
|
||||
// it (a dropped/late fire). We serve exactly T by emitting the missing tail;
|
||||
// the client still gets the right answer. Counted, and visible on /health.
|
||||
// divergence— we already emitted bytes that are NOT a prefix of T. The client is holding
|
||||
// text the transcript disagrees with and it cannot be retracted. REFUSE the
|
||||
// turn: throw → SSE error frame, no cache, no success. Serving on would be
|
||||
// exactly the "silently serve wrong text" failure this gate exists to stop.
|
||||
// (Known trigger: a tool-using turn whose pre-tool prose exceeded the
|
||||
// holdback — the transcript keeps only the LAST assistant message, so the
|
||||
// prose we streamed is text T does not contain.)
|
||||
if (assembler) {
|
||||
// F7: a total hook failure (a claude version bump stops honoring --settings, or a
|
||||
// truncated md-hook.sh per F3) produces zero fires for every turn, finalize() still
|
||||
// reports ok:true/exact:false (the transcript alone carries the whole answer), and the
|
||||
// turn succeeds NORMALLY — degrading to buffered with no error, no divergence, nothing
|
||||
// but streamTopUps climbing (which the comment above calls "benign"). That is
|
||||
// indistinguishable from one late fire dropped unless it is counted separately.
|
||||
if (assembler.deltas === 0) {
|
||||
tuiStats.streamZeroDeltaTurns++;
|
||||
logEvent("warn", "tui_stream_zero_deltas", { model: cliModel });
|
||||
}
|
||||
const rec = assembler.finalize(text);
|
||||
if (!rec.ok) {
|
||||
tuiStats.streamDivergences++;
|
||||
logEvent("error", "tui_stream_divergence", {
|
||||
model: cliModel,
|
||||
// The dominant cause in practice: a TOOL-USING turn whose pre-tool prose exceeded the
|
||||
// holdback and was already streamed. The transcript keeps only the LAST assistant
|
||||
// message, so that prose is text T does not contain. Remedy for such a deployment:
|
||||
// raise OCP_TUI_STREAM_HOLDBACK above the model's typical narration length (later first
|
||||
// chunk, but the prose stays held back and is then correctly discarded), or leave
|
||||
// OCP_TUI_STREAM off. See README + ADR 0007 (2026-07-13 amendment).
|
||||
reason: assembler.restartedAfterEmit ? "multi_message_after_emit (tool-use turn?)" : "delta_transcript_mismatch",
|
||||
emittedChars: rec.emitted, transcriptChars: rec.transcript,
|
||||
deltas: assembler.deltas, messages: assembler.messages,
|
||||
});
|
||||
throw new Error("tui_stream_divergence: streamed text is not a prefix of the transcript; refusing to serve it");
|
||||
}
|
||||
if (!rec.exact) {
|
||||
tuiStats.streamTopUps++;
|
||||
logEvent("warn", "tui_stream_topup", {
|
||||
model: cliModel, emittedChars: rec.emitted, transcriptChars: rec.transcript, deltas: assembler.deltas,
|
||||
});
|
||||
}
|
||||
if (rec.tail) streamCtx.emit(rec.tail);
|
||||
}
|
||||
|
||||
recordModelSuccess(cliModel, 0); // elapsed not measurable here; wallclock at reader level
|
||||
// Assert the subscription-pool classification. TUI exists to keep cc_entrypoint=cli
|
||||
// (subscription pool); a silent degrade to sdk-cli (metered Agent SDK pool) would still
|
||||
@@ -1441,6 +1579,14 @@ async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
|
||||
}
|
||||
return text;
|
||||
} catch (err) {
|
||||
// A mid-turn client disconnect (streaming path only — abortSignal) is NOT an upstream
|
||||
// failure: runTuiTurn's finally already tore the pane down, and this finally releases the
|
||||
// slot. Mirror the queued-disconnect handling above (info, no recordModelError, no
|
||||
// response) rather than booking a phantom model error against the socket going away.
|
||||
if (err && err.name === "TuiAbortError") {
|
||||
logEvent("info", "tui_turn_aborted", { reason: "client_disconnected", model: cliModel });
|
||||
throw new RequestDisconnectedError("client disconnected mid-turn; TUI pane torn down");
|
||||
}
|
||||
recordModelError(cliModel, false);
|
||||
throw err;
|
||||
} finally {
|
||||
@@ -1448,6 +1594,102 @@ async function callClaudeTui(model, messages, _conversationId, _keyName, res) {
|
||||
}
|
||||
}
|
||||
|
||||
// ── TUI-mode REAL streaming (OCP_TUI_STREAM=1) ──────────────────────────
|
||||
// The stream:true + TUI_MODE + OCP_TUI_STREAM=1 path. Emits the turn as it is generated,
|
||||
// from claude's own MessageDisplay hook, instead of buffering it and replaying it with
|
||||
// streamStringAsSSE.
|
||||
//
|
||||
// WIRE SHAPES: every frame below is COPIED from callClaudeStreaming (the -p path) — the role
|
||||
// chunk, the content-delta chunk, the stop chunk, `[DONE]`, and the post-header
|
||||
// {error:{message,type}} frame. No new fields, no new shapes. (ALIGNMENT.md Rule 2 / Class B:
|
||||
// the authority for the wire format is the OpenAI chat/completions streaming spec, adopted by
|
||||
// ADR 0006; the authority for the TUI spawn is ADR 0007. No cli.js citation applies — see the
|
||||
// commit body.)
|
||||
//
|
||||
// HEADERS ARE SENT EAGERLY, exactly as the -p path does, so the existing heartbeat
|
||||
// (CLAUDE_HEARTBEAT_INTERVAL) covers the ~6s of silence before the first delta. The cost is
|
||||
// the same one the -p path already pays: after the headers are out, an upstream failure can
|
||||
// no longer be a JSON 500, so it is surfaced as the SSE error frame instead (issue #110).
|
||||
async function callClaudeTuiStreaming(model, messages, conversationId, res, authInfo = {}) {
|
||||
const id = `chatcmpl-${randomUUID()}`;
|
||||
const created = Math.floor(Date.now() / 1000);
|
||||
const t0 = Date.now();
|
||||
const promptChars = messages.reduce((a, m) => a + contentToText(m.content).length, 0);
|
||||
let headersSent = false;
|
||||
|
||||
function ensureHeaders() {
|
||||
if (res.writableEnded || res.destroyed) return false;
|
||||
if (headersSent) return true;
|
||||
headersSent = true;
|
||||
res.writeHead(200, {
|
||||
"Content-Type": "text/event-stream",
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"X-Accel-Buffering": "no",
|
||||
});
|
||||
sendSSE(res, {
|
||||
id, object: "chat.completion.chunk", created, model,
|
||||
choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }],
|
||||
});
|
||||
return true;
|
||||
}
|
||||
|
||||
ensureHeaders();
|
||||
const hb = startHeartbeat(res, HEARTBEAT_INTERVAL, conversationId);
|
||||
// Held for the WHOLE turn (not just the queue wait): a disconnect must abort the transcript
|
||||
// wait so runTuiTurn tears the pane down and callClaudeTui's finally frees the slot.
|
||||
const { signal, detach } = closeSignalFor(res);
|
||||
|
||||
const streamCtx = {
|
||||
signal,
|
||||
emit(text) {
|
||||
if (!text) return;
|
||||
if (!ensureHeaders()) return; // client vanished — drop the write, the turn still unwinds
|
||||
sendSSE(res, {
|
||||
id, object: "chat.completion.chunk", created, model,
|
||||
choices: [{ index: 0, delta: { content: text }, finish_reason: null }],
|
||||
}, hb);
|
||||
},
|
||||
};
|
||||
|
||||
try {
|
||||
// callClaudeTui returns the TRANSCRIPT text T after its honesty gates + the streaming
|
||||
// reconciliation. Everything the client should see has been emitted by then.
|
||||
const content = await callClaudeTui(model, messages, conversationId, authInfo.keyName, res, streamCtx);
|
||||
// Cache T — never the concatenated deltas (mirrors the buffered TUI path).
|
||||
if (CACHE_TTL > 0 && authInfo.cacheHash) {
|
||||
try { setCachedResponse(authInfo.cacheHash, model, content); } catch (e) { logEvent("error", "cache_write_failed", { error: e.message }); }
|
||||
}
|
||||
if (!res.writableEnded && !res.destroyed) {
|
||||
sendSSE(res, {
|
||||
id, object: "chat.completion.chunk", created, model,
|
||||
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
|
||||
}, hb);
|
||||
res.write("data: [DONE]\n\n");
|
||||
res.end();
|
||||
}
|
||||
try { recordUsage({ keyId: authInfo.keyId, keyName: authInfo.keyName, model, promptChars, responseChars: content.length, elapsedMs: Date.now() - t0, success: true }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
|
||||
} catch (err) {
|
||||
// Client walked away (queued OR mid-turn): nothing to write to, nothing to record —
|
||||
// same quiet outcome as every other disconnect path (L1 / F2).
|
||||
if (err instanceof RequestDisconnectedError) { try { res.end(); } catch {} return; }
|
||||
try { recordUsage({ keyId: authInfo.keyId, keyName: authInfo.keyName, model, promptChars, responseChars: 0, elapsedMs: Date.now() - t0, success: false }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
|
||||
console.error(`[proxy] error: ${err.message}`);
|
||||
// Headers are already out (eager, above), so — exactly like the -p path — the failure is
|
||||
// surfaced as an SSE error frame, NOT a success-looking finish_reason:"stop". This is what
|
||||
// keeps a truncated turn, an auth banner, or a stream divergence from being served as an
|
||||
// answer: the client sees an error, and nothing was cached.
|
||||
if (!res.writableEnded && !res.destroyed) {
|
||||
sendSSE(res, { error: { message: sanitizeError(err.message), type: "provider_error" } }, hb);
|
||||
res.write("data: [DONE]\n\n");
|
||||
res.end();
|
||||
}
|
||||
} finally {
|
||||
hb.stop();
|
||||
detach();
|
||||
}
|
||||
}
|
||||
|
||||
// ── SSE heartbeat (opt-in idle watchdog) ────────────────────────────────
|
||||
// Emits `: keepalive\n\n` SSE comment frames during silent windows on the
|
||||
// streaming response. Design: docs/superpowers/specs/2026-04-25-47-sse-heartbeat-design.md
|
||||
@@ -2340,6 +2582,11 @@ async function handleChatCompletions(req, res) {
|
||||
}
|
||||
|
||||
if (stream) {
|
||||
if (TUI_MODE && TUI_STREAM) {
|
||||
// TUI-mode REAL streaming (opt-in): emit delta.content chunks as claude renders them,
|
||||
// via its MessageDisplay hook. The transcript remains authoritative (gates + cache).
|
||||
return callClaudeTuiStreaming(model, messages, conversationId, res, { keyId: req._authKeyId, keyName: req._authKeyName, cacheHash: req._cacheHash });
|
||||
}
|
||||
if (TUI_MODE) {
|
||||
// TUI-mode: no real token stream — buffer the full turn via callClaudeTui,
|
||||
// optionally write-back to cache, then replay as chunked SSE.
|
||||
@@ -2641,8 +2888,13 @@ const server = createServer(async (req, res) => {
|
||||
// `pool` is a NEW nested field inside the (already additive) tui block: null when the
|
||||
// warm pool is off (the default), so the disabled shape is unchanged apart from one
|
||||
// explicit null. Lets the operator confirm hit rate + standing process cost.
|
||||
//
|
||||
// streamEnabled + the stream* counters are likewise ADDITIVE (new fields only, same
|
||||
// grandfathered B.2 rationale — ADR 0006). streamDivergences is the one an operator
|
||||
// must watch: a non-zero value means a streamed turn was REFUSED because the deltas
|
||||
// disagreed with the transcript, which is the streaming path's only correctness risk.
|
||||
tui: buildTuiHealthBlock(
|
||||
{ enabled: TUI_MODE, entrypointMode: TUI_ENTRYPOINT, maxConcurrent: TUI_MAX_CONCURRENT },
|
||||
{ enabled: TUI_MODE, entrypointMode: TUI_ENTRYPOINT, maxConcurrent: TUI_MAX_CONCURRENT, streamEnabled: TUI_MODE && TUI_STREAM },
|
||||
tuiStats, tuiSemaphore, tuiPool,
|
||||
),
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user