fix(tui): honest error/truncation handling + version-robust entrypoint + short-prompt paste (PR-A) (#137)

TUI-mode (CLAUDE_TUI_MODE=true) is an OCP-owned subscription-pool bridge for the
2026-06-15 Anthropic billing split. This PR fixes four honesty/robustness defects
confirmed by a prior audit and live-reproduced on PI231 (2026-06-10).

C-1 (P1) — upstream AUTH-FAILURE banner returned/cached as a real answer.
  The interactive claude CLI renders in-session errors as ordinary assistant text.
  The R-1 case C-1 exists to catch is expired/invalid credentials, where EVERY turn
  comes back as the same one-line auth-failure banner, e.g. the two live PI231
  banners (2026-06-10):
    "Please run /login · API Error: 401 Invalid authentication credentials"  (69 chars)
    "Failed to authenticate. API Error: 401 Invalid authentication credentials" (73 chars)
  callClaudeTui returned that banner verbatim → OCP cached it, shared it via
  singleflight, and recorded a model SUCCESS. New detectTuiUpstreamError()
  (lib/tui/transcript.mjs) flags such a turn; callClaudeTui throws tui_upstream_error
  + logEvent("error", …) BEFORE recordModelSuccess/cache write-back, so the error
  never enters the cache; the client gets a 5xx.

  C-1 NARROWING (false-positive probe, 2026-06-10). An earlier generalised default
  rule — ^<short auth-failure prefix>?API Error: <3-digit> <detail>$ — was TOO BROAD:
  the unbounded ".*" detail tail let any short prefix + "API Error: NNN" + an
  arbitrarily long sentence match, so it KILLED legitimate long answers that merely
  DISCUSS an API error (e.g. "API Error: 500 happened because the server was
  overloaded. To fix this, retry with exponential backoff …"). A false-positive costs
  the user a missing answer AND a double-burn retry — strictly worse than the rare
  false-negative (caching one transient error for the 5-min TTL). C-1 is therefore
  reframed from "detect any API error" to "detect a claude-CLI AUTHENTICATION-FAILURE
  banner", and is CONSERVATIVE: when unsure it PASSES. The narrowed default detector
  flags a turn only when ALL of the following hold over the WHOLE trimmed text
  (a conjunction; any one failing => PASS):
    1. SHORT whole-message — length ≤ 100 (live banners are 69/73; cap gives headroom
       while rejecting multi-sentence prose; a 226-char auth answer is dropped on
       length alone).
    2. Contains "API Error: 4\d{2}" — auth failures are 4xx (401/403); transient 5xx
       and bare "HTTP 401 means unauthorized." (no API-Error core) are excluded.
    3. Contains an auth keyword — authenticat | /login | credential (case-insensitive);
       rejects "To debug a 401 … API Error: 401 Unauthorized …" (authoriz-, not
       authenticat-).
    4. Contains NO backtick/quote char (` ' ") — a real banner is plain text; quoted
       text signals an answer QUOTING the error, e.g. "You'll see `API Error: 401` …
       run /login to fix it." (short + 4xx + /login, excluded only by this signal).
  Full required matrix (2 KILL + 7 PASS) encoded as tests; npm test green.
  CLAUDE_TUI_ERROR_PATTERNS still overrides (a non-empty value REPLACES the default
  with operator regexes; empty/whitespace DISABLES detection). New fixture
  lib/tui/fixtures/error-401-failauth.jsonl + positive/negative tests retained.

C-2 (P1) — wallclock-truncated partial text returned as silent success.
  readTuiTranscript returned {text, entrypoint} identically on the terminal-marker
  path and the cap-with-partial path, so a cut-off turn was cached + returned as
  finish_reason:stop. It now returns truncated:false on terminal-marker and
  truncated:true on the cap-with-partial path (additive field; no-text cap path
  still throws). callClaudeTui throws tui_wallclock_truncated on truncated.

C-3 (P1) — verifyEntrypoint only read the turn_duration line.
  Some claude builds don't emit turn_duration (Mac mini: absent; PI231/2.1.104:
  present), while the entrypoint field is on ordinary lines on BOTH. Reading only
  turn_duration made the server.mjs tui_entrypoint_mismatch assertion get got:null
  every turn on non-emitting builds. verifyEntrypoint now PREFERS the turn_duration
  entrypoint and FALLS BACK to the entrypoint field on any line.

C-4 (P2) — short prompts 100% failed paste-landing.
  tuiPromptLanded required needle.length >= 3, so a 1–2 char first line ("hi","ok")
  never matched and 5s-failed with tui_paste_not_landed every time (live-repro:
  "hi"). Threshold lowered 3 → 2; the input box starts empty (placeholder excluded
  by the affirmative-signal design) so a 2-char needle present in the pane is the
  prompt. Kept >=2 (not >=1) to avoid collisions with claude's chrome glyphs.

Tests: extended test-features.mjs with unit coverage for each fix + three fixtures
(lib/tui/fixtures/error-401.jsonl, error-401-failauth.jsonl, no-turn-duration.jsonl).
The C-1 block now encodes the full narrowed auth-banner matrix (2 must-kill + 7
must-pass + supporting regression guards incl. a length-cap-load-bearing test).
npm test: 224 passed, 0 failed. Default path (CLAUDE_TUI_MODE unset →
upstreamCall=callClaude) is byte-identical: the only server.mjs change is one import
+ the callClaudeTui body, and callClaudeTui is unreachable when TUI_MODE is off.

ALIGNMENT: Class B (OCP-owned compatibility surface). cli.js does not perform this
operation (TUI is an OCP-owned subscription-pool bridge); scope authorized by ADR
0007 (Class B). No Class A path touched (callClaude / handleUsage / OAuth unchanged);
no change to .github/workflows/alignment.yml or any blacklisted token.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude <claude-opus> <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
2026-06-10 21:39:01 +10:00
committed by GitHub
co-authored by taodeng Claude <claude-opus> <noreply@anthropic.com>
parent 6d4751f983
commit a37ff713d9
7 changed files with 402 additions and 18 deletions
@@ -0,0 +1,2 @@
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb2222-3333-4444-5555-666677778888","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"What is 2 + 2?"}]}}
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb2222-3333-4444-5555-666677778888","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"Failed to authenticate. API Error: 401 Invalid authentication credentials"}]}}
+2
View File
@@ -0,0 +1,2 @@
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"aaaa1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"What is 2 + 2?"}]}}
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"aaaa1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"Please run /login · API Error: 401 Invalid authentication credentials"}]}}
+2
View File
@@ -0,0 +1,2 @@
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"Say PONG and nothing else."}]}}
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"PONG"}]}}
+10 -1
View File
@@ -68,7 +68,16 @@ function tuiPromptLanded(pane, prompt) {
if (flatPane.includes("[Pasted text")) return true; if (flatPane.includes("[Pasted text")) return true;
const firstLine = String(prompt).split("\n").map(s => s.trim()).find(Boolean) || ""; const firstLine = String(prompt).split("\n").map(s => s.trim()).find(Boolean) || "";
const needle = firstLine.replace(/\s+/g, " ").slice(0, 24); const needle = firstLine.replace(/\s+/g, " ").slice(0, 24);
return needle.length >= 3 && flatPane.includes(needle); // C-4/#133: threshold lowered 3 → 2. A prompt whose first non-blank line is 12
// chars ("hi", "ok") previously NEVER matched (needle.length >= 3) and never
// surfaced "[Pasted text", so EVERY short prompt 5s-failed with tui_paste_not_landed
// (live-reproduced: "hi"). The input box starts EMPTY (the curly-quote placeholder
// is excluded by the affirmative-signal design above), so a >=2-char needle present
// in the pane is the pasted prompt, not placeholder noise — false-positive risk is
// low. We keep >=2 rather than >=1 because a single visible char is more likely to
// collide with incidental glyphs in claude's chrome (borders, the "" prompt mark);
// 2 chars is the floor that lands real prompts while staying conservative.
return needle.length >= 2 && flatPane.includes(needle);
} }
async function pollUntil(fn, { timeoutMs, intervalMs }) { async function pollUntil(fn, { timeoutMs, intervalMs }) {
+161 -12
View File
@@ -100,24 +100,169 @@ export function extractLatestAssistantText(events) {
return text; return text;
} }
// Returns the entrypoint string from the turn_duration line (e.g. "cli"), // Returns the entrypoint string (e.g. "cli") used for the billing-pool assertion,
// or null if absent. Lets callers assert the subscription-classified path. // or null if absent. Lets callers assert the subscription-classified path.
// Fixture-confirmed: entrypoint field lives directly on the turn_duration line. //
// Resolution order (C-3, issue #133):
// 1. PREFER the turn_duration system line's `entrypoint` — the authoritative
// end-of-turn classifier emitted by builds that produce turn_duration
// (e.g. claude-2.1.104/2.1.157 on PI231).
// 2. FALL BACK to the `entrypoint` field on ANY ordinary transcript line
// (assistant / user / attachment / system) — present on BOTH emitting and
// non-emitting builds. Some claude builds (e.g. certain Mac mini transcripts)
// do NOT emit a turn_duration line at all; reading ONLY turn_duration made the
// caller's tui_entrypoint_mismatch assertion (server.mjs) get got:null every
// turn and go blind. The entrypoint value is identical across line types within
// a single interactive session (fixture-confirmed: every line in
// complete-haiku.jsonl carrying `entrypoint` reads "cli"), so the fallback
// yields the same classifier. Last-writer-wins on the fallback.
export function verifyEntrypoint(events) { export function verifyEntrypoint(events) {
let fallback = null;
for (const ev of events) { for (const ev of events) {
if (ev && ev.type === "system" && ev.subtype === "turn_duration") { if (!ev || typeof ev !== "object") continue;
return ev.entrypoint != null ? ev.entrypoint : null; if (ev.type === "system" && ev.subtype === "turn_duration" && ev.entrypoint != null) {
return ev.entrypoint; // authoritative — short-circuit
} }
if (ev.entrypoint != null) fallback = ev.entrypoint;
}
return fallback;
}
// ── C-1: honest AUTH-FAILURE banner detection (issue #133) ───────────────
// When the interactive `claude` CLI hits an in-session error it does NOT crash —
// it renders the error as ordinary assistant text in the transcript. The specific
// failure C-1 exists to catch is R-1: EXPIRED / INVALID credentials, where every
// turn comes back as the same one-line auth-failure banner and OCP, none the wiser,
// caches that banner (server.mjs setCachedResponse), shares it via singleflight, and
// records a model SUCCESS — so a hard auth error is silently served (and cached for
// the 5-min TTL) as a real answer. The two live-reproduced banners on PI231
// (2026-06-10) are:
// "Please run /login · API Error: 401 Invalid authentication credentials" (69 chars)
// "Failed to authenticate. API Error: 401 Invalid authentication credentials" (73 chars)
//
// WHY THE SCOPE IS NARROW (conservatism — the load-bearing design choice).
// An earlier generalised rule (^<short-prefix>?API Error:\s*\d{3}\b.*$) was TOO
// BROAD: its unbounded `.*` tail let any short prefix + "API Error: NNN" + an
// arbitrarily long sentence match, so it KILLED legitimate long answers that merely
// DISCUSS an API error (e.g. "API Error: 500 happened because the server was
// overloaded. To fix this, retry with exponential backoff …"). That is the worst
// outcome: a false-positive costs the user a missing answer AND a double-burn retry,
// whereas the rare false-negative (caching one transient error for the 5-min TTL) is
// cheap and self-healing. So C-1 is reframed from "detect ANY API error" to "detect
// a claude-CLI AUTHENTICATION-FAILURE banner", and when unsure it PASSES (does not
// kill). Transient 5xx server errors are deliberately NOT detected — they are not the
// R-1 case and the conservative choice is to let them through.
//
// THE SIGNAL — a turn is an auth-failure banner only if ALL of these hold over the
// WHOLE trimmed assistant text (a conjunction; any one failing => PASS):
// 1. SHORT whole-message. Real banners are one short line (the two live samples are
// 69 and 73 chars). Cap = TUI_ERR_MAX_LEN (100) — headroom over 73 for a
// slightly longer future banner, while still rejecting multi-sentence prose. A
// long answer that happens to discuss auth (no code chars, e.g. 226 chars) is
// rejected on length alone.
// 2. Contains "API Error: 4\d{2}" — auth failures are 4xx (401/403). This rejects
// transient 5xx ("API Error: 500/503 …") and bare "HTTP 401 means unauthorized."
// (no "API Error:" core).
// 3. Contains an auth KEYWORD — authenticat | /login | credential (case-insensitive).
// This rejects answers that quote a 4xx but are not auth banners, e.g.
// "To debug a 401: the server returns API Error: 401 Unauthorized …"
// ("Unauthorized" is authoriz-, not authenticat-; no /login, no credential).
// 4. Contains NO backtick or quote char (` ' "). A real CLI banner is plain text;
// backticked/quoted text signals an answer that is QUOTING the error rather than
// being the banner, e.g. "You'll see `API Error: 401` … run /login to fix it."
// (75 chars — passes 1-3 but is excluded here). This is the conservative tie-
// breaker for short instructional answers.
//
// Worked matrix (all required cases pass — see test-features.mjs C-1 block):
// KILL: "Please run /login · API Error: 401 Invalid authentication credentials"
// KILL: "Failed to authenticate. API Error: 401 Invalid authentication credentials"
// PASS: "API Error: 500 happened because the server was overloaded. …" (not 4xx)
// PASS: "Failed to parse the config. Here are the API Error: 401 details …" (too long + no auth-kw)
// PASS: "To debug a 401: … API Error: 401 Unauthorized, then you refresh …" (no auth-kw)
// PASS: "Here is the handler … It logs the string API Error: 503 …" (not 4xx)
// PASS: "You'll see `API Error: 401` … run /login to fix it." (has backtick)
// PASS: "HTTP 401 means unauthorized." (no API Error core)
// PASS: "The capital of France is Paris." (nothing matches)
//
// OPERATOR OVERRIDE (unchanged): CLAUDE_TUI_ERROR_PATTERNS lets an operator REPLACE
// the default auth-banner detector with their own newline- or `||`-separated JS regex
// source strings (each auto-anchored ^…$ over the trimmed text, case-insensitive). A
// non-empty override uses ONLY those regexes (the narrowed default is bypassed); an
// empty / whitespace-only override DISABLES detection entirely (escape hatch).
// Whole-message length cap for the default auth-banner detector. Real banners are
// 69/73 chars; 100 gives headroom while still rejecting multi-sentence prose.
const TUI_ERR_MAX_LEN = 100;
// 4xx "API Error:" core — auth failures are 4xx (401/403), never 5xx.
const TUI_ERR_4XX = /API Error:\s*4\d{2}\b/i;
// Auth keyword — the message must be about authentication, not just quote a 4xx.
const TUI_ERR_AUTH_KW = /authenticat|\/login|credential/i;
// Code/quote chars — their presence signals prose QUOTING an error, not the banner.
const TUI_ERR_CODE_CHAR = /[`'"]/;
// Default detector: returns true iff `trimmed` IS a claude-CLI auth-failure banner
// (all four signals above). Conservative — any signal failing => false (PASS).
function isDefaultAuthFailureBanner(trimmed) {
if (trimmed.length > TUI_ERR_MAX_LEN) return false; // 1. short whole-message
if (!TUI_ERR_4XX.test(trimmed)) return false; // 2. 4xx API Error core
if (!TUI_ERR_AUTH_KW.test(trimmed)) return false; // 3. auth keyword
if (TUI_ERR_CODE_CHAR.test(trimmed)) return false; // 4. no code/quote chars
return true;
}
// Compile an OPERATOR-SUPPLIED pattern set (override path only). Each source is
// anchored ^…$ over the trimmed text and matched case-insensitively (`s` so `.` spans
// a multi-line banner). A pattern that fails to compile is skipped (never throws into
// the request path).
function compileTuiErrorPatterns(raw) {
const sources = String(raw).split(/\r?\n|\|\|/).map((s) => s.trim()).filter(Boolean);
const out = [];
for (const src of sources) {
try { out.push(new RegExp(`^(?:${src})$`, "is")); } catch { /* skip bad pattern */ }
}
return out;
}
// Returns the matched banner text (the trimmed assistant text) if `text` IS a claude-
// CLI auth-failure banner in its entirety, else null. `patternsRaw` defaults to
// process.env.CLAUDE_TUI_ERROR_PATTERNS:
// - undefined → narrowed default auth-banner detector (isDefaultAuthFailureBanner).
// - non-empty → operator regex override REPLACES the default.
// - empty/ws → detection disabled (escape hatch).
export function detectTuiUpstreamError(text, patternsRaw = process.env.CLAUDE_TUI_ERROR_PATTERNS) {
if (typeof text !== "string") return null;
const trimmed = text.trim();
if (!trimmed) return null;
if (patternsRaw == null) {
return isDefaultAuthFailureBanner(trimmed) ? trimmed : null;
}
// Operator override path: empty/whitespace disables; otherwise use only their regexes.
const patterns = compileTuiErrorPatterns(patternsRaw);
if (patterns.length === 0) return null;
for (const re of patterns) {
if (re.test(trimmed)) return trimmed;
} }
return null; return null;
} }
// Block until the session transcript is terminal (turn_duration) or // Block until the session transcript is terminal (turn_duration / final
// the wall-clock cap elapses, polling the file (no fs.watch — robust over NFS / // stop_reason) or the wall-clock cap elapses, polling the file (no fs.watch —
// editors). Returns { text, entrypoint } where text is the latest assistant text // robust over NFS / editors). Returns { text, entrypoint, truncated }:
// and entrypoint is the billing-pool classifier from the turn_duration line (e.g. // - text: latest assistant text.
// "cli"), or null if not yet present. On cap with text, returns the partial result; // - entrypoint: billing-pool classifier (see verifyEntrypoint), or null.
// on cap with no text at all, throws. // - truncated: FALSE when a terminal marker was reached (the turn completed);
// TRUE when the wall-clock cap was hit with partial text but NO
// terminal marker (the turn is INCOMPLETE — what we have is a
// cut-off prefix). (C-2, issue #133.)
//
// Why `truncated` matters: previously the terminal-marker path and the
// cap-with-partial-text path BOTH returned `{text, entrypoint}` identically, so
// callClaudeTui could not tell a complete answer from a truncated one and cached +
// returned the partial as finish_reason:stop (silent success). The caller now
// throws on `truncated` so a cut-off turn is neither cached nor counted as success.
// The field is additive — existing call sites that ignore it keep working.
//
// On cap with NO text at all, still throws (unchanged) — there is nothing to return.
// //
// No quiescence heuristic by design: a long Opus thinking turn stalls transcript // No quiescence heuristic by design: a long Opus thinking turn stalls transcript
// growth and a "file stable for N s" rule would false-abort it (spec §4.3). // growth and a "file stable for N s" rule would false-abort it (spec §4.3).
@@ -135,10 +280,14 @@ export async function readTuiTranscript({ transcriptPath: p, home, sessionId, wa
lastText = extractLatestAssistantText(events) || lastText; lastText = extractLatestAssistantText(events) || lastText;
const ep = verifyEntrypoint(events); const ep = verifyEntrypoint(events);
if (ep != null) lastEntrypoint = ep; if (ep != null) lastEntrypoint = ep;
if (events.some(isTerminalLine)) return { text: lastText, entrypoint: lastEntrypoint }; // Terminal marker reached → the turn is COMPLETE.
if (events.some(isTerminalLine)) return { text: lastText, entrypoint: lastEntrypoint, truncated: false };
} }
await sleep(pollMs); await sleep(pollMs);
} }
if (lastText) return { text: lastText, entrypoint: lastEntrypoint }; // Cap elapsed with no terminal marker. If we have partial text, flag it truncated
// so the caller rejects it (don't cache / don't count as success). No text at all
// → throw (nothing to return).
if (lastText) return { text: lastText, entrypoint: lastEntrypoint, truncated: true };
throw new Error("tui_transcript_timeout: no assistant text within wallclock cap"); throw new Error("tui_transcript_timeout: no assistant text within wallclock cap");
} }
+25 -1
View File
@@ -38,6 +38,7 @@ import { validateKey, recordUsage, getUsageByKey, getUsageTimeline, getRecentUsa
import { DEFAULT_PORT } from "./lib/constants.mjs"; import { DEFAULT_PORT } from "./lib/constants.mjs";
import { isLoopbackBind } from "./lib/net.mjs"; import { isLoopbackBind } from "./lib/net.mjs";
import { runTuiTurn, reapStaleTuiSessions } from "./lib/tui/session.mjs"; import { runTuiTurn, reapStaleTuiSessions } from "./lib/tui/session.mjs";
import { detectTuiUpstreamError } from "./lib/tui/transcript.mjs";
const __dirname = dirname(fileURLToPath(import.meta.url)); const __dirname = dirname(fileURLToPath(import.meta.url));
const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8")); const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8"));
@@ -926,7 +927,30 @@ function callClaudeTui(model, messages, _conversationId, _keyName) {
cwd: TUI_CWD, cwd: TUI_CWD,
wallclockMs: TUI_WALLCLOCK_MS, wallclockMs: TUI_WALLCLOCK_MS,
entrypointMode: TUI_ENTRYPOINT, entrypointMode: TUI_ENTRYPOINT,
}).then(({ text, entrypoint }) => { }).then(({ text, entrypoint, truncated }) => {
// ── Honesty gates (issue #133) ─ run BEFORE recordModelSuccess / cache write-back.
// A throw here propagates to the .catch below (recordModelError + reject), so the
// result never reaches the downstream setCachedResponse / singleflight / SUCCESS path.
// C-2: the wall-clock cap hit with partial text and NO terminal marker — the turn
// is INCOMPLETE. Returning the cut-off prefix would cache it and report it as
// finish_reason:stop (a truncated answer served as a complete one). Reject instead.
if (truncated) {
logEvent("error", "tui_wallclock_truncated", { model: cliModel, chars: (text || "").length, wallclockMs: TUI_WALLCLOCK_MS });
throw new Error("tui_wallclock_truncated: turn hit the wall-clock cap before completing; partial text dropped");
}
// C-1: the interactive claude CLI renders in-session errors (expired/invalid
// credentials, transient API failure) as ordinary assistant text. Returning that
// banner would cache an error AS an answer and record a model SUCCESS. Detect a
// known error banner (anchored whole-text match — see detectTuiUpstreamError) and
// reject so it does NOT enter the cache and the client gets a 5xx.
const banner = detectTuiUpstreamError(text);
if (banner) {
logEvent("error", "tui_upstream_error", { model: cliModel, banner: banner.slice(0, 200) });
throw new Error("tui_upstream_error: claude CLI returned an in-session error banner instead of an answer");
}
recordModelSuccess(cliModel, 0); // elapsed not measurable here; wallclock at reader level recordModelSuccess(cliModel, 0); // elapsed not measurable here; wallclock at reader level
// Assert the subscription-pool classification. TUI exists to keep cc_entrypoint=cli // Assert the subscription-pool classification. TUI exists to keep cc_entrypoint=cli
// (subscription pool); a silent degrade to sdk-cli (metered Agent SDK pool) would still // (subscription pool); a silent degrade to sdk-cli (metered Agent SDK pool) would still
+200 -4
View File
@@ -1340,7 +1340,7 @@ test("streamStringAsSSE empty content: role + stop + [DONE] only", () => {
}); });
// ── Suite: TUI transcript reader ──────────────────────────────────────── // ── Suite: TUI transcript reader ────────────────────────────────────────
import { encodeCwd, transcriptPath, findTranscriptPath, parseTranscriptLines, isTerminalLine, extractLatestAssistantText, verifyEntrypoint } from "./lib/tui/transcript.mjs"; import { encodeCwd, transcriptPath, findTranscriptPath, parseTranscriptLines, isTerminalLine, extractLatestAssistantText, verifyEntrypoint, detectTuiUpstreamError } from "./lib/tui/transcript.mjs";
import { readFileSync as tuiReadFileSync, mkdtempSync as tuiMkdtemp0, mkdirSync as tuiMkdir0, writeFileSync as tuiWrite0 } from "node:fs"; import { readFileSync as tuiReadFileSync, mkdtempSync as tuiMkdtemp0, mkdirSync as tuiMkdir0, writeFileSync as tuiWrite0 } from "node:fs";
import { tmpdir as tuiTmp0 } from "node:os"; import { tmpdir as tuiTmp0 } from "node:os";
@@ -1436,6 +1436,173 @@ test("real complete fixture: verifyEntrypoint returns 'cli'", () => {
assert.equal(verifyEntrypoint(evs), "cli"); assert.equal(verifyEntrypoint(evs), "cli");
}); });
// ── C-3 (#133): verifyEntrypoint is version-robust ───────────────────────
// Some claude builds do NOT emit a turn_duration line; entrypoint lives on
// ordinary lines on BOTH emitting and non-emitting builds. Reading ONLY
// turn_duration made the server.mjs tui_entrypoint_mismatch assertion get null
// every turn on non-emitting builds. verifyEntrypoint must fall back to ANY line.
console.log("\nTUI transcript — verifyEntrypoint version-robustness (C-3, #133):");
test("verifyEntrypoint PREFERS the turn_duration line's entrypoint", () => {
// turn_duration says "cli"; an earlier ordinary line says "sdk-cli" — the
// authoritative turn_duration value must win, not last-writer-wins on the fallback.
const evs = [
{ type: "assistant", entrypoint: "sdk-cli", message: { content: [{ type: "text", text: "x" }] } },
{ type: "system", subtype: "turn_duration", entrypoint: "cli" },
];
assert.equal(verifyEntrypoint(evs), "cli");
});
test("verifyEntrypoint falls back to entrypoint on an ordinary assistant line when no turn_duration", () => {
const evs = [
{ type: "user", entrypoint: "cli", message: { content: "hi" } },
{ type: "assistant", entrypoint: "cli", message: { stop_reason: "end_turn", content: [{ type: "text", text: "ok" }] } },
];
assert.equal(verifyEntrypoint(evs), "cli");
});
test("verifyEntrypoint returns null when NO line carries an entrypoint", () => {
const evs = [
{ type: "assistant", message: { stop_reason: "end_turn", content: [{ type: "text", text: "ok" }] } },
];
assert.equal(verifyEntrypoint(evs), null);
});
test("real no-turn_duration fixture: verifyEntrypoint still resolves 'cli' (was null before C-3)", () => {
const evs = parseTranscriptLines(tuiReadFileSync("./lib/tui/fixtures/no-turn-duration.jsonl", "utf8"));
// Sanity: the fixture genuinely lacks a turn_duration line (so this exercises the fallback).
assert.ok(!evs.some((e) => e && e.type === "system" && e.subtype === "turn_duration"), "fixture must NOT emit turn_duration");
assert.equal(verifyEntrypoint(evs), "cli");
});
// ── C-1 (#133): honest AUTH-FAILURE banner detection ─────────────────────
// The interactive claude CLI renders in-session errors as ordinary assistant text.
// C-1 catches the R-1 case: expired/invalid creds, where EVERY turn returns the same
// one-line auth-failure banner and OCP would cache it as a real answer. The detector
// is deliberately NARROW/conservative: a false-positive (killing a real long answer
// that merely DISCUSSES an API error) costs the user a missing answer + a double-burn
// retry, which is worse than the rare false-negative (caching one transient error for
// the 5-min TTL). Signal = ALL of: SHORT whole-message (≤100; live samples 69/73) AND
// "API Error: 4xx" AND an auth keyword (authenticat | /login | credential) AND NO
// backtick/quote char. When unsure → PASS. The earlier generalised rule
// (^<short-prefix>?API Error:\d{3}.*$) was TOO BROAD: its unbounded .* tail killed
// legit long answers; this block encodes the full narrowed matrix.
console.log("\nTUI transcript — auth-failure banner detection (C-1, #133):");
// ---- Required matrix: MUST detect (kill) ----
test("C-1 KILL: live /login 401 auth banner", () => {
const banner = "Please run /login · API Error: 401 Invalid authentication credentials";
assert.equal(detectTuiUpstreamError(banner), banner);
});
test("C-1 KILL: live 'Failed to authenticate.' 401 banner variant", () => {
// Second real PI231 banner: a different short auth-failure prefix before the same
// "API Error: 4xx" core. Still short, still 4xx, still has 'authenticate'/'credentials'.
const banner = "Failed to authenticate. API Error: 401 Invalid authentication credentials";
assert.equal(detectTuiUpstreamError(banner), banner);
});
// ---- Required matrix: MUST NOT kill (pass) ----
test("C-1 PASS: long answer discussing a 500 (not 4xx, too long)", () => {
// The exact false-positive the over-broad .* rule produced. 166 chars; 5xx.
const legit = "API Error: 500 happened because the server was overloaded. To fix this, retry with exponential backoff and verify your rate limits before resending the request again.";
assert.equal(detectTuiUpstreamError(legit), null);
});
test("C-1 PASS: long answer with 'API Error: 401 details' (too long, no auth keyword)", () => {
// 142 chars; the literal word 'authenticate'/'credential'/'/login' never appears, and
// it is far over the length cap — rejected on length AND keyword.
const legit = "Failed to parse the config. Here are the API Error: 401 details you asked about: the token expired and must be refreshed before the next call.";
assert.equal(detectTuiUpstreamError(legit), null);
});
test("C-1 PASS: 'To debug a 401 … API Error: 401 Unauthorized' (no auth keyword)", () => {
// 91 chars (short!) and 4xx, but 'Unauthorized' is authoriz-, not authenticat-, and
// there is no /login or credential — the auth-keyword signal rejects it.
const legit = "To debug a 401: the server returns API Error: 401 Unauthorized, then you refresh the token.";
assert.equal(detectTuiUpstreamError(legit), null);
});
test("C-1 PASS: handler answer logging 'API Error: 503' (not 4xx)", () => {
const legit = "Here is the handler you asked for. It logs the string API Error: 503 on failure and retries.";
assert.equal(detectTuiUpstreamError(legit), null);
});
test("C-1 PASS: short instructional answer quoting `API Error: 401` + /login (has backtick)", () => {
// 75 chars: short, 4xx, has '/login' — passes signals 1-3. Rejected ONLY by the
// backtick/quote constraint: it QUOTES the error in code formatting, it is not the banner.
const legit = "You'll see `API Error: 401` when your token expires — run /login to fix it.";
assert.equal(detectTuiUpstreamError(legit), null);
});
test("C-1 PASS: bare HTTP-status sentence (no 'API Error:' core)", () => {
assert.equal(detectTuiUpstreamError("HTTP 401 means unauthorized."), null);
});
test("C-1 PASS: plain unrelated answer", () => {
assert.equal(detectTuiUpstreamError("The capital of France is Paris."), null);
});
// ---- Supporting / regression coverage ----
test("C-1 PASS: transient 5xx banner is NOT detected (narrowed to 4xx auth only)", () => {
// The old rule flagged any 3-digit code; the narrowed detector is 4xx-only by design
// (5xx is transient/server-side, not the R-1 auth case). Accepted false-negative.
assert.equal(detectTuiUpstreamError("API Error: 500 Internal Server Error"), null);
});
test("C-1 PASS: bare 4xx with no auth keyword is NOT detected", () => {
// 'API Error: 403 Forbidden' alone — 4xx and short, but no authenticat/login/credential.
assert.equal(detectTuiUpstreamError("API Error: 403 Forbidden"), null);
});
test("detectTuiUpstreamError trims surrounding whitespace before matching", () => {
const out = detectTuiUpstreamError("\n\n Please run /login · API Error: 401 credential boom \n");
assert.equal(out, "Please run /login · API Error: 401 credential boom");
});
test("detectTuiUpstreamError is case-insensitive on the banner keywords", () => {
// lower-cased: /login + api error: 401 + 'credential' keyword, short, no code char.
assert.ok(detectTuiUpstreamError("please run /login · api error: 401 bad credential") !== null);
});
test("detectTuiUpstreamError does NOT match prose that mentions an API error mid-paragraph (#133 regression guard)", () => {
// A long, legit answer that merely discusses an API error — rejected on length alone.
const para = "When integrating with the upstream service you may occasionally hit an API Error: 401 response if the bearer token has lapsed; the recommended remediation is to re-run the login flow and retry the request with a fresh credential, after which the 401 should clear.";
assert.equal(detectTuiUpstreamError(para), null);
});
test("detectTuiUpstreamError does NOT match a long plain-text auth answer with NO code chars (length cap is load-bearing)", () => {
// 226 chars, no backtick/quote, has 4xx + /login + credential + authenticate — passes
// signals 2-4. ONLY the length cap rejects it. Guards against dropping the cap.
const para = "If you call the endpoint without a bearer token the API Error: 401 response tells you the credential is missing; just authenticate again with /login and the request will succeed on the next attempt without any further changes.";
assert.equal(detectTuiUpstreamError(para), null);
});
test("detectTuiUpstreamError returns null on empty / whitespace / non-string", () => {
assert.equal(detectTuiUpstreamError(""), null);
assert.equal(detectTuiUpstreamError(" \n "), null);
assert.equal(detectTuiUpstreamError(null), null);
assert.equal(detectTuiUpstreamError(undefined), null);
assert.equal(detectTuiUpstreamError(42), null);
});
test("detectTuiUpstreamError respects CLAUDE_TUI_ERROR_PATTERNS override (custom banner)", () => {
// Override with a single custom pattern; the default 401 banner no longer matches,
// but the custom one does (anchored whole-text).
assert.equal(detectTuiUpstreamError("Please run /login · API Error: 401 x", "Session expired, please re-auth"), null);
assert.equal(detectTuiUpstreamError("Session expired, please re-auth", "Session expired, please re-auth"), "Session expired, please re-auth");
});
test("detectTuiUpstreamError with an empty override disables detection (escape hatch)", () => {
assert.equal(detectTuiUpstreamError("API Error: 500 boom", ""), null);
assert.equal(detectTuiUpstreamError("API Error: 500 boom", " "), null);
});
test("detectTuiUpstreamError override accepts '||'-separated patterns", () => {
const raw = "First banner||Second banner";
assert.equal(detectTuiUpstreamError("First banner", raw), "First banner");
assert.equal(detectTuiUpstreamError("Second banner", raw), "Second banner");
assert.equal(detectTuiUpstreamError("Third", raw), null);
});
test("real error fixture: latest assistant text IS the banner and detectTuiUpstreamError flags it", () => {
const evs = parseTranscriptLines(tuiReadFileSync("./lib/tui/fixtures/error-401.jsonl", "utf8"));
const text = extractLatestAssistantText(evs);
assert.equal(text, "Please run /login · API Error: 401 Invalid authentication credentials");
assert.ok(detectTuiUpstreamError(text) !== null, "error fixture's final turn must be flagged as an upstream error");
});
test("real error fixture (Failed-to-authenticate variant): final turn is flagged (#133 runtime gap)", () => {
const evs = parseTranscriptLines(tuiReadFileSync("./lib/tui/fixtures/error-401-failauth.jsonl", "utf8"));
const text = extractLatestAssistantText(evs);
assert.equal(text, "Failed to authenticate. API Error: 401 Invalid authentication credentials");
assert.ok(detectTuiUpstreamError(text) !== null, "Failed-to-authenticate banner must be flagged as an upstream error");
});
test("real complete fixture: final answer is NOT flagged as an upstream error", () => {
const evs = parseTranscriptLines(tuiReadFileSync("./lib/tui/fixtures/complete-haiku.jsonl", "utf8"));
const text = extractLatestAssistantText(evs);
assert.equal(detectTuiUpstreamError(text), null);
});
// ── TUI transcript — polling reader (async) ────────────────────────────── // ── TUI transcript — polling reader (async) ──────────────────────────────
import { readTuiTranscript } from "./lib/tui/transcript.mjs"; import { readTuiTranscript } from "./lib/tui/transcript.mjs";
import { mkdtempSync as tuiMkdtemp, writeFileSync as tuiWriteFile } from "node:fs"; import { mkdtempSync as tuiMkdtemp, writeFileSync as tuiWriteFile } from "node:fs";
@@ -1455,12 +1622,30 @@ await asyncTest("readTuiTranscript returns assistant text when terminal marker p
assert.equal(out.entrypoint, "cli"); assert.equal(out.entrypoint, "cli");
}); });
await asyncTest("readTuiTranscript honours wall-clock cap and returns partial text", async () => { // C-2 (#133): the terminal-marker path must signal a COMPLETE turn.
await asyncTest("readTuiTranscript signals truncated:false when a terminal marker is hit (complete turn)", async () => {
const dir = tuiMkdtemp(`${tuiTmpdir()}/tui-`); const dir = tuiMkdtemp(`${tuiTmpdir()}/tui-`);
const p = `${dir}/s.jsonl`; const p = `${dir}/s.jsonl`;
tuiWriteFile(p, [
JSON.stringify({ type: "assistant", message: { content: [{ type: "text", text: "done" }] } }),
JSON.stringify({ type: "system", subtype: "turn_duration", durationMs: 1200, entrypoint: "cli" }),
].join("\n") + "\n");
const out = await readTuiTranscript({ transcriptPath: p, wallclockMs: 2000, pollMs: 50 });
assert.equal(out.truncated, false);
});
// C-2 (#133): cap-with-partial-text must be DISTINGUISHABLE from a complete turn.
// Previously both returned {text, entrypoint} identically and the partial was cached
// + returned as finish_reason:stop. The cap path now returns truncated:true so the
// caller (callClaudeTui) can throw instead of serving a cut-off answer.
await asyncTest("readTuiTranscript honours wall-clock cap and flags partial text truncated:true", async () => {
const dir = tuiMkdtemp(`${tuiTmpdir()}/tui-`);
const p = `${dir}/s.jsonl`;
// No terminal marker → reader will spin to the cap then return the partial.
tuiWriteFile(p, JSON.stringify({ type: "assistant", message: { content: [{ type: "text", text: "partial" }] } }) + "\n"); tuiWriteFile(p, JSON.stringify({ type: "assistant", message: { content: [{ type: "text", text: "partial" }] } }) + "\n");
const out = await readTuiTranscript({ transcriptPath: p, wallclockMs: 300, pollMs: 50 }); const out = await readTuiTranscript({ transcriptPath: p, wallclockMs: 300, pollMs: 50 });
assert.equal(out.text, "partial"); assert.equal(out.text, "partial");
assert.equal(out.truncated, true);
}); });
await asyncTest("readTuiTranscript against real fixture: entrypoint is 'cli'", async () => { await asyncTest("readTuiTranscript against real fixture: entrypoint is 'cli'", async () => {
@@ -1699,7 +1884,7 @@ function _tuiPromptLanded(pane, prompt) {
if (flatPane.includes("[Pasted text")) return true; if (flatPane.includes("[Pasted text")) return true;
const firstLine = String(prompt).split("\n").map(s => s.trim()).find(Boolean) || ""; const firstLine = String(prompt).split("\n").map(s => s.trim()).find(Boolean) || "";
const needle = firstLine.replace(/\s+/g, " ").slice(0, 24); const needle = firstLine.replace(/\s+/g, " ").slice(0, 24);
return needle.length >= 3 && flatPane.includes(needle); return needle.length >= 2 && flatPane.includes(needle); // C-4 (#133): 3 → 2 (see lib/tui/session.mjs)
} }
// Real captured pane samples (empirically confirmed via live capture-pane on PI231, // Real captured pane samples (empirically confirmed via live capture-pane on PI231,
@@ -1731,12 +1916,23 @@ test("tuiPromptLanded(READY_PANE, 'Reply with exactly: PONG_TEST') === false (s
test("tuiPromptLanded(LANDED_PANE, 'Reply with exactly: PONG_TEST') === true (prompt prefix visible)", () => { test("tuiPromptLanded(LANDED_PANE, 'Reply with exactly: PONG_TEST') === true (prompt prefix visible)", () => {
assert.equal(_tuiPromptLanded(TUI_LANDED_PANE, "Reply with exactly: PONG_TEST"), true); assert.equal(_tuiPromptLanded(TUI_LANDED_PANE, "Reply with exactly: PONG_TEST"), true);
}); });
test("tuiPromptLanded(READY_PANE, 'ping') === false (needle <3 chars, placeholder present)", () => { test("tuiPromptLanded(READY_PANE, 'ping') === false (prompt text absent from placeholder pane)", () => {
assert.equal(_tuiPromptLanded(TUI_READY_PANE, "ping"), false); assert.equal(_tuiPromptLanded(TUI_READY_PANE, "ping"), false);
}); });
test("tuiPromptLanded(' ping\\n ? for shortcuts', 'ping') === true (needle present, no placeholder)", () => { test("tuiPromptLanded(' ping\\n ? for shortcuts', 'ping') === true (needle present, no placeholder)", () => {
assert.equal(_tuiPromptLanded(" ping\n ? for shortcuts", "ping"), true); assert.equal(_tuiPromptLanded(" ping\n ? for shortcuts", "ping"), true);
}); });
// C-4 (#133): short prompts (12 char first line) MUST be able to land. Threshold
// lowered 3 → 2. A 2-char prompt ("hi") present in the pane now lands instead of
// 5s-failing with tui_paste_not_landed every time (live-reproduced: "hi").
test("tuiPromptLanded(' hi\\n ? for shortcuts', 'hi') === true (2-char prompt lands — C-4)", () => {
assert.equal(_tuiPromptLanded(" hi\n ? for shortcuts", "hi"), true);
});
// False-positive guard for the lowered threshold: a 2-char needle ABSENT from the
// still-empty placeholder pane must NOT land (no spurious Enter into an empty box).
test("tuiPromptLanded(READY_PANE, 'hi') === false (2-char prompt not yet visible — no false positive)", () => {
assert.equal(_tuiPromptLanded(TUI_READY_PANE, "hi"), false);
});
// issue #130 root cause: a big bracketed paste shows "[Pasted text #N +M lines]" — must be landed. // issue #130 root cause: a big bracketed paste shows "[Pasted text #N +M lines]" — must be landed.
test("tuiPromptLanded(bracketed-paste pane, big prompt) === true", () => { test("tuiPromptLanded(bracketed-paste pane, big prompt) === true", () => {
assert.equal(_tuiPromptLanded(" [Pasted text #1 +301 lines]\n ? for shortcuts", "[System] Context 0."), true); assert.equal(_tuiPromptLanded(" [Pasted text #1 +301 lines]\n ? for shortcuts", "[System] Context 0."), true);