fix(tui): honest error/truncation handling + version-robust entrypoint + short-prompt paste (PR-A) (#137)

TUI-mode (CLAUDE_TUI_MODE=true) is an OCP-owned subscription-pool bridge for the
2026-06-15 Anthropic billing split. This PR fixes four honesty/robustness defects
confirmed by a prior audit and live-reproduced on PI231 (2026-06-10).

C-1 (P1) — upstream AUTH-FAILURE banner returned/cached as a real answer.
  The interactive claude CLI renders in-session errors as ordinary assistant text.
  The R-1 case C-1 exists to catch is expired/invalid credentials, where EVERY turn
  comes back as the same one-line auth-failure banner, e.g. the two live PI231
  banners (2026-06-10):
    "Please run /login · API Error: 401 Invalid authentication credentials"  (69 chars)
    "Failed to authenticate. API Error: 401 Invalid authentication credentials" (73 chars)
  callClaudeTui returned that banner verbatim → OCP cached it, shared it via
  singleflight, and recorded a model SUCCESS. New detectTuiUpstreamError()
  (lib/tui/transcript.mjs) flags such a turn; callClaudeTui throws tui_upstream_error
  + logEvent("error", …) BEFORE recordModelSuccess/cache write-back, so the error
  never enters the cache; the client gets a 5xx.

  C-1 NARROWING (false-positive probe, 2026-06-10). An earlier generalised default
  rule — ^<short auth-failure prefix>?API Error: <3-digit> <detail>$ — was TOO BROAD:
  the unbounded ".*" detail tail let any short prefix + "API Error: NNN" + an
  arbitrarily long sentence match, so it KILLED legitimate long answers that merely
  DISCUSS an API error (e.g. "API Error: 500 happened because the server was
  overloaded. To fix this, retry with exponential backoff …"). A false-positive costs
  the user a missing answer AND a double-burn retry — strictly worse than the rare
  false-negative (caching one transient error for the 5-min TTL). C-1 is therefore
  reframed from "detect any API error" to "detect a claude-CLI AUTHENTICATION-FAILURE
  banner", and is CONSERVATIVE: when unsure it PASSES. The narrowed default detector
  flags a turn only when ALL of the following hold over the WHOLE trimmed text
  (a conjunction; any one failing => PASS):
    1. SHORT whole-message — length ≤ 100 (live banners are 69/73; cap gives headroom
       while rejecting multi-sentence prose; a 226-char auth answer is dropped on
       length alone).
    2. Contains "API Error: 4\d{2}" — auth failures are 4xx (401/403); transient 5xx
       and bare "HTTP 401 means unauthorized." (no API-Error core) are excluded.
    3. Contains an auth keyword — authenticat | /login | credential (case-insensitive);
       rejects "To debug a 401 … API Error: 401 Unauthorized …" (authoriz-, not
       authenticat-).
    4. Contains NO backtick/quote char (` ' ") — a real banner is plain text; quoted
       text signals an answer QUOTING the error, e.g. "You'll see `API Error: 401` …
       run /login to fix it." (short + 4xx + /login, excluded only by this signal).
  Full required matrix (2 KILL + 7 PASS) encoded as tests; npm test green.
  CLAUDE_TUI_ERROR_PATTERNS still overrides (a non-empty value REPLACES the default
  with operator regexes; empty/whitespace DISABLES detection). New fixture
  lib/tui/fixtures/error-401-failauth.jsonl + positive/negative tests retained.

C-2 (P1) — wallclock-truncated partial text returned as silent success.
  readTuiTranscript returned {text, entrypoint} identically on the terminal-marker
  path and the cap-with-partial path, so a cut-off turn was cached + returned as
  finish_reason:stop. It now returns truncated:false on terminal-marker and
  truncated:true on the cap-with-partial path (additive field; no-text cap path
  still throws). callClaudeTui throws tui_wallclock_truncated on truncated.

C-3 (P1) — verifyEntrypoint only read the turn_duration line.
  Some claude builds don't emit turn_duration (Mac mini: absent; PI231/2.1.104:
  present), while the entrypoint field is on ordinary lines on BOTH. Reading only
  turn_duration made the server.mjs tui_entrypoint_mismatch assertion get got:null
  every turn on non-emitting builds. verifyEntrypoint now PREFERS the turn_duration
  entrypoint and FALLS BACK to the entrypoint field on any line.

C-4 (P2) — short prompts 100% failed paste-landing.
  tuiPromptLanded required needle.length >= 3, so a 1–2 char first line ("hi","ok")
  never matched and 5s-failed with tui_paste_not_landed every time (live-repro:
  "hi"). Threshold lowered 3 → 2; the input box starts empty (placeholder excluded
  by the affirmative-signal design) so a 2-char needle present in the pane is the
  prompt. Kept >=2 (not >=1) to avoid collisions with claude's chrome glyphs.

Tests: extended test-features.mjs with unit coverage for each fix + three fixtures
(lib/tui/fixtures/error-401.jsonl, error-401-failauth.jsonl, no-turn-duration.jsonl).
The C-1 block now encodes the full narrowed auth-banner matrix (2 must-kill + 7
must-pass + supporting regression guards incl. a length-cap-load-bearing test).
npm test: 224 passed, 0 failed. Default path (CLAUDE_TUI_MODE unset →
upstreamCall=callClaude) is byte-identical: the only server.mjs change is one import
+ the callClaudeTui body, and callClaudeTui is unreachable when TUI_MODE is off.

ALIGNMENT: Class B (OCP-owned compatibility surface). cli.js does not perform this
operation (TUI is an OCP-owned subscription-pool bridge); scope authorized by ADR
0007 (Class B). No Class A path touched (callClaude / handleUsage / OAuth unchanged);
no change to .github/workflows/alignment.yml or any blacklisted token.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude <claude-opus> <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
2026-06-10 21:39:01 +10:00
committed by GitHub
co-authored by taodeng Claude <claude-opus> <noreply@anthropic.com>
parent 6d4751f983
commit a37ff713d9
7 changed files with 402 additions and 18 deletions
@@ -0,0 +1,2 @@
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb2222-3333-4444-5555-666677778888","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"What is 2 + 2?"}]}}
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb2222-3333-4444-5555-666677778888","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"Failed to authenticate. API Error: 401 Invalid authentication credentials"}]}}
+2
View File
@@ -0,0 +1,2 @@
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"aaaa1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"What is 2 + 2?"}]}}
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"aaaa1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"Please run /login · API Error: 401 Invalid authentication credentials"}]}}
+2
View File
@@ -0,0 +1,2 @@
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"Say PONG and nothing else."}]}}
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"PONG"}]}}
+10 -1
View File
@@ -68,7 +68,16 @@ function tuiPromptLanded(pane, prompt) {
if (flatPane.includes("[Pasted text")) return true;
const firstLine = String(prompt).split("\n").map(s => s.trim()).find(Boolean) || "";
const needle = firstLine.replace(/\s+/g, " ").slice(0, 24);
return needle.length >= 3 && flatPane.includes(needle);
// C-4/#133: threshold lowered 3 → 2. A prompt whose first non-blank line is 12
// chars ("hi", "ok") previously NEVER matched (needle.length >= 3) and never
// surfaced "[Pasted text", so EVERY short prompt 5s-failed with tui_paste_not_landed
// (live-reproduced: "hi"). The input box starts EMPTY (the curly-quote placeholder
// is excluded by the affirmative-signal design above), so a >=2-char needle present
// in the pane is the pasted prompt, not placeholder noise — false-positive risk is
// low. We keep >=2 rather than >=1 because a single visible char is more likely to
// collide with incidental glyphs in claude's chrome (borders, the "" prompt mark);
// 2 chars is the floor that lands real prompts while staying conservative.
return needle.length >= 2 && flatPane.includes(needle);
}
async function pollUntil(fn, { timeoutMs, intervalMs }) {
+161 -12
View File
@@ -100,24 +100,169 @@ export function extractLatestAssistantText(events) {
return text;
}
// Returns the entrypoint string from the turn_duration line (e.g. "cli"),
// Returns the entrypoint string (e.g. "cli") used for the billing-pool assertion,
// or null if absent. Lets callers assert the subscription-classified path.
// Fixture-confirmed: entrypoint field lives directly on the turn_duration line.
//
// Resolution order (C-3, issue #133):
// 1. PREFER the turn_duration system line's `entrypoint` — the authoritative
// end-of-turn classifier emitted by builds that produce turn_duration
// (e.g. claude-2.1.104/2.1.157 on PI231).
// 2. FALL BACK to the `entrypoint` field on ANY ordinary transcript line
// (assistant / user / attachment / system) — present on BOTH emitting and
// non-emitting builds. Some claude builds (e.g. certain Mac mini transcripts)
// do NOT emit a turn_duration line at all; reading ONLY turn_duration made the
// caller's tui_entrypoint_mismatch assertion (server.mjs) get got:null every
// turn and go blind. The entrypoint value is identical across line types within
// a single interactive session (fixture-confirmed: every line in
// complete-haiku.jsonl carrying `entrypoint` reads "cli"), so the fallback
// yields the same classifier. Last-writer-wins on the fallback.
export function verifyEntrypoint(events) {
let fallback = null;
for (const ev of events) {
if (ev && ev.type === "system" && ev.subtype === "turn_duration") {
return ev.entrypoint != null ? ev.entrypoint : null;
if (!ev || typeof ev !== "object") continue;
if (ev.type === "system" && ev.subtype === "turn_duration" && ev.entrypoint != null) {
return ev.entrypoint; // authoritative — short-circuit
}
if (ev.entrypoint != null) fallback = ev.entrypoint;
}
return fallback;
}
// ── C-1: honest AUTH-FAILURE banner detection (issue #133) ───────────────
// When the interactive `claude` CLI hits an in-session error it does NOT crash —
// it renders the error as ordinary assistant text in the transcript. The specific
// failure C-1 exists to catch is R-1: EXPIRED / INVALID credentials, where every
// turn comes back as the same one-line auth-failure banner and OCP, none the wiser,
// caches that banner (server.mjs setCachedResponse), shares it via singleflight, and
// records a model SUCCESS — so a hard auth error is silently served (and cached for
// the 5-min TTL) as a real answer. The two live-reproduced banners on PI231
// (2026-06-10) are:
// "Please run /login · API Error: 401 Invalid authentication credentials" (69 chars)
// "Failed to authenticate. API Error: 401 Invalid authentication credentials" (73 chars)
//
// WHY THE SCOPE IS NARROW (conservatism — the load-bearing design choice).
// An earlier generalised rule (^<short-prefix>?API Error:\s*\d{3}\b.*$) was TOO
// BROAD: its unbounded `.*` tail let any short prefix + "API Error: NNN" + an
// arbitrarily long sentence match, so it KILLED legitimate long answers that merely
// DISCUSS an API error (e.g. "API Error: 500 happened because the server was
// overloaded. To fix this, retry with exponential backoff …"). That is the worst
// outcome: a false-positive costs the user a missing answer AND a double-burn retry,
// whereas the rare false-negative (caching one transient error for the 5-min TTL) is
// cheap and self-healing. So C-1 is reframed from "detect ANY API error" to "detect
// a claude-CLI AUTHENTICATION-FAILURE banner", and when unsure it PASSES (does not
// kill). Transient 5xx server errors are deliberately NOT detected — they are not the
// R-1 case and the conservative choice is to let them through.
//
// THE SIGNAL — a turn is an auth-failure banner only if ALL of these hold over the
// WHOLE trimmed assistant text (a conjunction; any one failing => PASS):
// 1. SHORT whole-message. Real banners are one short line (the two live samples are
// 69 and 73 chars). Cap = TUI_ERR_MAX_LEN (100) — headroom over 73 for a
// slightly longer future banner, while still rejecting multi-sentence prose. A
// long answer that happens to discuss auth (no code chars, e.g. 226 chars) is
// rejected on length alone.
// 2. Contains "API Error: 4\d{2}" — auth failures are 4xx (401/403). This rejects
// transient 5xx ("API Error: 500/503 …") and bare "HTTP 401 means unauthorized."
// (no "API Error:" core).
// 3. Contains an auth KEYWORD — authenticat | /login | credential (case-insensitive).
// This rejects answers that quote a 4xx but are not auth banners, e.g.
// "To debug a 401: the server returns API Error: 401 Unauthorized …"
// ("Unauthorized" is authoriz-, not authenticat-; no /login, no credential).
// 4. Contains NO backtick or quote char (` ' "). A real CLI banner is plain text;
// backticked/quoted text signals an answer that is QUOTING the error rather than
// being the banner, e.g. "You'll see `API Error: 401` … run /login to fix it."
// (75 chars — passes 1-3 but is excluded here). This is the conservative tie-
// breaker for short instructional answers.
//
// Worked matrix (all required cases pass — see test-features.mjs C-1 block):
// KILL: "Please run /login · API Error: 401 Invalid authentication credentials"
// KILL: "Failed to authenticate. API Error: 401 Invalid authentication credentials"
// PASS: "API Error: 500 happened because the server was overloaded. …" (not 4xx)
// PASS: "Failed to parse the config. Here are the API Error: 401 details …" (too long + no auth-kw)
// PASS: "To debug a 401: … API Error: 401 Unauthorized, then you refresh …" (no auth-kw)
// PASS: "Here is the handler … It logs the string API Error: 503 …" (not 4xx)
// PASS: "You'll see `API Error: 401` … run /login to fix it." (has backtick)
// PASS: "HTTP 401 means unauthorized." (no API Error core)
// PASS: "The capital of France is Paris." (nothing matches)
//
// OPERATOR OVERRIDE (unchanged): CLAUDE_TUI_ERROR_PATTERNS lets an operator REPLACE
// the default auth-banner detector with their own newline- or `||`-separated JS regex
// source strings (each auto-anchored ^…$ over the trimmed text, case-insensitive). A
// non-empty override uses ONLY those regexes (the narrowed default is bypassed); an
// empty / whitespace-only override DISABLES detection entirely (escape hatch).
// Whole-message length cap for the default auth-banner detector. Real banners are
// 69/73 chars; 100 gives headroom while still rejecting multi-sentence prose.
const TUI_ERR_MAX_LEN = 100;
// 4xx "API Error:" core — auth failures are 4xx (401/403), never 5xx.
const TUI_ERR_4XX = /API Error:\s*4\d{2}\b/i;
// Auth keyword — the message must be about authentication, not just quote a 4xx.
const TUI_ERR_AUTH_KW = /authenticat|\/login|credential/i;
// Code/quote chars — their presence signals prose QUOTING an error, not the banner.
const TUI_ERR_CODE_CHAR = /[`'"]/;
// Default detector: returns true iff `trimmed` IS a claude-CLI auth-failure banner
// (all four signals above). Conservative — any signal failing => false (PASS).
function isDefaultAuthFailureBanner(trimmed) {
if (trimmed.length > TUI_ERR_MAX_LEN) return false; // 1. short whole-message
if (!TUI_ERR_4XX.test(trimmed)) return false; // 2. 4xx API Error core
if (!TUI_ERR_AUTH_KW.test(trimmed)) return false; // 3. auth keyword
if (TUI_ERR_CODE_CHAR.test(trimmed)) return false; // 4. no code/quote chars
return true;
}
// Compile an OPERATOR-SUPPLIED pattern set (override path only). Each source is
// anchored ^…$ over the trimmed text and matched case-insensitively (`s` so `.` spans
// a multi-line banner). A pattern that fails to compile is skipped (never throws into
// the request path).
function compileTuiErrorPatterns(raw) {
const sources = String(raw).split(/\r?\n|\|\|/).map((s) => s.trim()).filter(Boolean);
const out = [];
for (const src of sources) {
try { out.push(new RegExp(`^(?:${src})$`, "is")); } catch { /* skip bad pattern */ }
}
return out;
}
// Returns the matched banner text (the trimmed assistant text) if `text` IS a claude-
// CLI auth-failure banner in its entirety, else null. `patternsRaw` defaults to
// process.env.CLAUDE_TUI_ERROR_PATTERNS:
// - undefined → narrowed default auth-banner detector (isDefaultAuthFailureBanner).
// - non-empty → operator regex override REPLACES the default.
// - empty/ws → detection disabled (escape hatch).
export function detectTuiUpstreamError(text, patternsRaw = process.env.CLAUDE_TUI_ERROR_PATTERNS) {
if (typeof text !== "string") return null;
const trimmed = text.trim();
if (!trimmed) return null;
if (patternsRaw == null) {
return isDefaultAuthFailureBanner(trimmed) ? trimmed : null;
}
// Operator override path: empty/whitespace disables; otherwise use only their regexes.
const patterns = compileTuiErrorPatterns(patternsRaw);
if (patterns.length === 0) return null;
for (const re of patterns) {
if (re.test(trimmed)) return trimmed;
}
return null;
}
// Block until the session transcript is terminal (turn_duration) or
// the wall-clock cap elapses, polling the file (no fs.watch — robust over NFS /
// editors). Returns { text, entrypoint } where text is the latest assistant text
// and entrypoint is the billing-pool classifier from the turn_duration line (e.g.
// "cli"), or null if not yet present. On cap with text, returns the partial result;
// on cap with no text at all, throws.
// Block until the session transcript is terminal (turn_duration / final
// stop_reason) or the wall-clock cap elapses, polling the file (no fs.watch —
// robust over NFS / editors). Returns { text, entrypoint, truncated }:
// - text: latest assistant text.
// - entrypoint: billing-pool classifier (see verifyEntrypoint), or null.
// - truncated: FALSE when a terminal marker was reached (the turn completed);
// TRUE when the wall-clock cap was hit with partial text but NO
// terminal marker (the turn is INCOMPLETE — what we have is a
// cut-off prefix). (C-2, issue #133.)
//
// Why `truncated` matters: previously the terminal-marker path and the
// cap-with-partial-text path BOTH returned `{text, entrypoint}` identically, so
// callClaudeTui could not tell a complete answer from a truncated one and cached +
// returned the partial as finish_reason:stop (silent success). The caller now
// throws on `truncated` so a cut-off turn is neither cached nor counted as success.
// The field is additive — existing call sites that ignore it keep working.
//
// On cap with NO text at all, still throws (unchanged) — there is nothing to return.
//
// No quiescence heuristic by design: a long Opus thinking turn stalls transcript
// growth and a "file stable for N s" rule would false-abort it (spec §4.3).
@@ -135,10 +280,14 @@ export async function readTuiTranscript({ transcriptPath: p, home, sessionId, wa
lastText = extractLatestAssistantText(events) || lastText;
const ep = verifyEntrypoint(events);
if (ep != null) lastEntrypoint = ep;
if (events.some(isTerminalLine)) return { text: lastText, entrypoint: lastEntrypoint };
// Terminal marker reached → the turn is COMPLETE.
if (events.some(isTerminalLine)) return { text: lastText, entrypoint: lastEntrypoint, truncated: false };
}
await sleep(pollMs);
}
if (lastText) return { text: lastText, entrypoint: lastEntrypoint };
// Cap elapsed with no terminal marker. If we have partial text, flag it truncated
// so the caller rejects it (don't cache / don't count as success). No text at all
// → throw (nothing to return).
if (lastText) return { text: lastText, entrypoint: lastEntrypoint, truncated: true };
throw new Error("tui_transcript_timeout: no assistant text within wallclock cap");
}