mirror of
https://github.com/dtzp555-max/ocp.git
synced 2026-07-21 21:15:09 +00:00
fix(tui): honest error/truncation handling + version-robust entrypoint + short-prompt paste (PR-A) (#137)
TUI-mode (CLAUDE_TUI_MODE=true) is an OCP-owned subscription-pool bridge for the
2026-06-15 Anthropic billing split. This PR fixes four honesty/robustness defects
confirmed by a prior audit and live-reproduced on PI231 (2026-06-10).
C-1 (P1) — upstream AUTH-FAILURE banner returned/cached as a real answer.
The interactive claude CLI renders in-session errors as ordinary assistant text.
The R-1 case C-1 exists to catch is expired/invalid credentials, where EVERY turn
comes back as the same one-line auth-failure banner, e.g. the two live PI231
banners (2026-06-10):
"Please run /login · API Error: 401 Invalid authentication credentials" (69 chars)
"Failed to authenticate. API Error: 401 Invalid authentication credentials" (73 chars)
callClaudeTui returned that banner verbatim → OCP cached it, shared it via
singleflight, and recorded a model SUCCESS. New detectTuiUpstreamError()
(lib/tui/transcript.mjs) flags such a turn; callClaudeTui throws tui_upstream_error
+ logEvent("error", …) BEFORE recordModelSuccess/cache write-back, so the error
never enters the cache; the client gets a 5xx.
C-1 NARROWING (false-positive probe, 2026-06-10). An earlier generalised default
rule — ^<short auth-failure prefix>?API Error: <3-digit> <detail>$ — was TOO BROAD:
the unbounded ".*" detail tail let any short prefix + "API Error: NNN" + an
arbitrarily long sentence match, so it KILLED legitimate long answers that merely
DISCUSS an API error (e.g. "API Error: 500 happened because the server was
overloaded. To fix this, retry with exponential backoff …"). A false-positive costs
the user a missing answer AND a double-burn retry — strictly worse than the rare
false-negative (caching one transient error for the 5-min TTL). C-1 is therefore
reframed from "detect any API error" to "detect a claude-CLI AUTHENTICATION-FAILURE
banner", and is CONSERVATIVE: when unsure it PASSES. The narrowed default detector
flags a turn only when ALL of the following hold over the WHOLE trimmed text
(a conjunction; any one failing => PASS):
1. SHORT whole-message — length ≤ 100 (live banners are 69/73; cap gives headroom
while rejecting multi-sentence prose; a 226-char auth answer is dropped on
length alone).
2. Contains "API Error: 4\d{2}" — auth failures are 4xx (401/403); transient 5xx
and bare "HTTP 401 means unauthorized." (no API-Error core) are excluded.
3. Contains an auth keyword — authenticat | /login | credential (case-insensitive);
rejects "To debug a 401 … API Error: 401 Unauthorized …" (authoriz-, not
authenticat-).
4. Contains NO backtick/quote char (` ' ") — a real banner is plain text; quoted
text signals an answer QUOTING the error, e.g. "You'll see `API Error: 401` …
run /login to fix it." (short + 4xx + /login, excluded only by this signal).
Full required matrix (2 KILL + 7 PASS) encoded as tests; npm test green.
CLAUDE_TUI_ERROR_PATTERNS still overrides (a non-empty value REPLACES the default
with operator regexes; empty/whitespace DISABLES detection). New fixture
lib/tui/fixtures/error-401-failauth.jsonl + positive/negative tests retained.
C-2 (P1) — wallclock-truncated partial text returned as silent success.
readTuiTranscript returned {text, entrypoint} identically on the terminal-marker
path and the cap-with-partial path, so a cut-off turn was cached + returned as
finish_reason:stop. It now returns truncated:false on terminal-marker and
truncated:true on the cap-with-partial path (additive field; no-text cap path
still throws). callClaudeTui throws tui_wallclock_truncated on truncated.
C-3 (P1) — verifyEntrypoint only read the turn_duration line.
Some claude builds don't emit turn_duration (Mac mini: absent; PI231/2.1.104:
present), while the entrypoint field is on ordinary lines on BOTH. Reading only
turn_duration made the server.mjs tui_entrypoint_mismatch assertion get got:null
every turn on non-emitting builds. verifyEntrypoint now PREFERS the turn_duration
entrypoint and FALLS BACK to the entrypoint field on any line.
C-4 (P2) — short prompts 100% failed paste-landing.
tuiPromptLanded required needle.length >= 3, so a 1–2 char first line ("hi","ok")
never matched and 5s-failed with tui_paste_not_landed every time (live-repro:
"hi"). Threshold lowered 3 → 2; the input box starts empty (placeholder excluded
by the affirmative-signal design) so a 2-char needle present in the pane is the
prompt. Kept >=2 (not >=1) to avoid collisions with claude's chrome glyphs.
Tests: extended test-features.mjs with unit coverage for each fix + three fixtures
(lib/tui/fixtures/error-401.jsonl, error-401-failauth.jsonl, no-turn-duration.jsonl).
The C-1 block now encodes the full narrowed auth-banner matrix (2 must-kill + 7
must-pass + supporting regression guards incl. a length-cap-load-bearing test).
npm test: 224 passed, 0 failed. Default path (CLAUDE_TUI_MODE unset →
upstreamCall=callClaude) is byte-identical: the only server.mjs change is one import
+ the callClaudeTui body, and callClaudeTui is unreachable when TUI_MODE is off.
ALIGNMENT: Class B (OCP-owned compatibility surface). cli.js does not perform this
operation (TUI is an OCP-owned subscription-pool bridge); scope authorized by ADR
0007 (Class B). No Class A path touched (callClaude / handleUsage / OAuth unchanged);
no change to .github/workflows/alignment.yml or any blacklisted token.
Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude <claude-opus> <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb2222-3333-4444-5555-666677778888","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"What is 2 + 2?"}]}}
|
||||
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb2222-3333-4444-5555-666677778888","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"Failed to authenticate. API Error: 401 Invalid authentication credentials"}]}}
|
||||
@@ -0,0 +1,2 @@
|
||||
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"aaaa1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"What is 2 + 2?"}]}}
|
||||
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"aaaa1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"Please run /login · API Error: 401 Invalid authentication credentials"}]}}
|
||||
@@ -0,0 +1,2 @@
|
||||
{"type":"user","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"user","content":[{"type":"text","text":"Say PONG and nothing else."}]}}
|
||||
{"type":"assistant","entrypoint":"cli","cwd":"/tmp/tui-test","sessionId":"bbbb1111-2222-3333-4444-555566667777","version":"2.1.104","message":{"role":"assistant","model":"claude-haiku-4-5-20251001","stop_reason":"end_turn","content":[{"type":"text","text":"PONG"}]}}
|
||||
+10
-1
@@ -68,7 +68,16 @@ function tuiPromptLanded(pane, prompt) {
|
||||
if (flatPane.includes("[Pasted text")) return true;
|
||||
const firstLine = String(prompt).split("\n").map(s => s.trim()).find(Boolean) || "";
|
||||
const needle = firstLine.replace(/\s+/g, " ").slice(0, 24);
|
||||
return needle.length >= 3 && flatPane.includes(needle);
|
||||
// C-4/#133: threshold lowered 3 → 2. A prompt whose first non-blank line is 1–2
|
||||
// chars ("hi", "ok") previously NEVER matched (needle.length >= 3) and never
|
||||
// surfaced "[Pasted text", so EVERY short prompt 5s-failed with tui_paste_not_landed
|
||||
// (live-reproduced: "hi"). The input box starts EMPTY (the curly-quote placeholder
|
||||
// is excluded by the affirmative-signal design above), so a >=2-char needle present
|
||||
// in the pane is the pasted prompt, not placeholder noise — false-positive risk is
|
||||
// low. We keep >=2 rather than >=1 because a single visible char is more likely to
|
||||
// collide with incidental glyphs in claude's chrome (borders, the "❯" prompt mark);
|
||||
// 2 chars is the floor that lands real prompts while staying conservative.
|
||||
return needle.length >= 2 && flatPane.includes(needle);
|
||||
}
|
||||
|
||||
async function pollUntil(fn, { timeoutMs, intervalMs }) {
|
||||
|
||||
+161
-12
@@ -100,24 +100,169 @@ export function extractLatestAssistantText(events) {
|
||||
return text;
|
||||
}
|
||||
|
||||
// Returns the entrypoint string from the turn_duration line (e.g. "cli"),
|
||||
// Returns the entrypoint string (e.g. "cli") used for the billing-pool assertion,
|
||||
// or null if absent. Lets callers assert the subscription-classified path.
|
||||
// Fixture-confirmed: entrypoint field lives directly on the turn_duration line.
|
||||
//
|
||||
// Resolution order (C-3, issue #133):
|
||||
// 1. PREFER the turn_duration system line's `entrypoint` — the authoritative
|
||||
// end-of-turn classifier emitted by builds that produce turn_duration
|
||||
// (e.g. claude-2.1.104/2.1.157 on PI231).
|
||||
// 2. FALL BACK to the `entrypoint` field on ANY ordinary transcript line
|
||||
// (assistant / user / attachment / system) — present on BOTH emitting and
|
||||
// non-emitting builds. Some claude builds (e.g. certain Mac mini transcripts)
|
||||
// do NOT emit a turn_duration line at all; reading ONLY turn_duration made the
|
||||
// caller's tui_entrypoint_mismatch assertion (server.mjs) get got:null every
|
||||
// turn and go blind. The entrypoint value is identical across line types within
|
||||
// a single interactive session (fixture-confirmed: every line in
|
||||
// complete-haiku.jsonl carrying `entrypoint` reads "cli"), so the fallback
|
||||
// yields the same classifier. Last-writer-wins on the fallback.
|
||||
export function verifyEntrypoint(events) {
|
||||
let fallback = null;
|
||||
for (const ev of events) {
|
||||
if (ev && ev.type === "system" && ev.subtype === "turn_duration") {
|
||||
return ev.entrypoint != null ? ev.entrypoint : null;
|
||||
if (!ev || typeof ev !== "object") continue;
|
||||
if (ev.type === "system" && ev.subtype === "turn_duration" && ev.entrypoint != null) {
|
||||
return ev.entrypoint; // authoritative — short-circuit
|
||||
}
|
||||
if (ev.entrypoint != null) fallback = ev.entrypoint;
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
// ── C-1: honest AUTH-FAILURE banner detection (issue #133) ───────────────
|
||||
// When the interactive `claude` CLI hits an in-session error it does NOT crash —
|
||||
// it renders the error as ordinary assistant text in the transcript. The specific
|
||||
// failure C-1 exists to catch is R-1: EXPIRED / INVALID credentials, where every
|
||||
// turn comes back as the same one-line auth-failure banner and OCP, none the wiser,
|
||||
// caches that banner (server.mjs setCachedResponse), shares it via singleflight, and
|
||||
// records a model SUCCESS — so a hard auth error is silently served (and cached for
|
||||
// the 5-min TTL) as a real answer. The two live-reproduced banners on PI231
|
||||
// (2026-06-10) are:
|
||||
// "Please run /login · API Error: 401 Invalid authentication credentials" (69 chars)
|
||||
// "Failed to authenticate. API Error: 401 Invalid authentication credentials" (73 chars)
|
||||
//
|
||||
// WHY THE SCOPE IS NARROW (conservatism — the load-bearing design choice).
|
||||
// An earlier generalised rule (^<short-prefix>?API Error:\s*\d{3}\b.*$) was TOO
|
||||
// BROAD: its unbounded `.*` tail let any short prefix + "API Error: NNN" + an
|
||||
// arbitrarily long sentence match, so it KILLED legitimate long answers that merely
|
||||
// DISCUSS an API error (e.g. "API Error: 500 happened because the server was
|
||||
// overloaded. To fix this, retry with exponential backoff …"). That is the worst
|
||||
// outcome: a false-positive costs the user a missing answer AND a double-burn retry,
|
||||
// whereas the rare false-negative (caching one transient error for the 5-min TTL) is
|
||||
// cheap and self-healing. So C-1 is reframed from "detect ANY API error" to "detect
|
||||
// a claude-CLI AUTHENTICATION-FAILURE banner", and when unsure it PASSES (does not
|
||||
// kill). Transient 5xx server errors are deliberately NOT detected — they are not the
|
||||
// R-1 case and the conservative choice is to let them through.
|
||||
//
|
||||
// THE SIGNAL — a turn is an auth-failure banner only if ALL of these hold over the
|
||||
// WHOLE trimmed assistant text (a conjunction; any one failing => PASS):
|
||||
// 1. SHORT whole-message. Real banners are one short line (the two live samples are
|
||||
// 69 and 73 chars). Cap = TUI_ERR_MAX_LEN (100) — headroom over 73 for a
|
||||
// slightly longer future banner, while still rejecting multi-sentence prose. A
|
||||
// long answer that happens to discuss auth (no code chars, e.g. 226 chars) is
|
||||
// rejected on length alone.
|
||||
// 2. Contains "API Error: 4\d{2}" — auth failures are 4xx (401/403). This rejects
|
||||
// transient 5xx ("API Error: 500/503 …") and bare "HTTP 401 means unauthorized."
|
||||
// (no "API Error:" core).
|
||||
// 3. Contains an auth KEYWORD — authenticat | /login | credential (case-insensitive).
|
||||
// This rejects answers that quote a 4xx but are not auth banners, e.g.
|
||||
// "To debug a 401: the server returns API Error: 401 Unauthorized …"
|
||||
// ("Unauthorized" is authoriz-, not authenticat-; no /login, no credential).
|
||||
// 4. Contains NO backtick or quote char (` ' "). A real CLI banner is plain text;
|
||||
// backticked/quoted text signals an answer that is QUOTING the error rather than
|
||||
// being the banner, e.g. "You'll see `API Error: 401` … run /login to fix it."
|
||||
// (75 chars — passes 1-3 but is excluded here). This is the conservative tie-
|
||||
// breaker for short instructional answers.
|
||||
//
|
||||
// Worked matrix (all required cases pass — see test-features.mjs C-1 block):
|
||||
// KILL: "Please run /login · API Error: 401 Invalid authentication credentials"
|
||||
// KILL: "Failed to authenticate. API Error: 401 Invalid authentication credentials"
|
||||
// PASS: "API Error: 500 happened because the server was overloaded. …" (not 4xx)
|
||||
// PASS: "Failed to parse the config. Here are the API Error: 401 details …" (too long + no auth-kw)
|
||||
// PASS: "To debug a 401: … API Error: 401 Unauthorized, then you refresh …" (no auth-kw)
|
||||
// PASS: "Here is the handler … It logs the string API Error: 503 …" (not 4xx)
|
||||
// PASS: "You'll see `API Error: 401` … run /login to fix it." (has backtick)
|
||||
// PASS: "HTTP 401 means unauthorized." (no API Error core)
|
||||
// PASS: "The capital of France is Paris." (nothing matches)
|
||||
//
|
||||
// OPERATOR OVERRIDE (unchanged): CLAUDE_TUI_ERROR_PATTERNS lets an operator REPLACE
|
||||
// the default auth-banner detector with their own newline- or `||`-separated JS regex
|
||||
// source strings (each auto-anchored ^…$ over the trimmed text, case-insensitive). A
|
||||
// non-empty override uses ONLY those regexes (the narrowed default is bypassed); an
|
||||
// empty / whitespace-only override DISABLES detection entirely (escape hatch).
|
||||
|
||||
// Whole-message length cap for the default auth-banner detector. Real banners are
|
||||
// 69/73 chars; 100 gives headroom while still rejecting multi-sentence prose.
|
||||
const TUI_ERR_MAX_LEN = 100;
|
||||
// 4xx "API Error:" core — auth failures are 4xx (401/403), never 5xx.
|
||||
const TUI_ERR_4XX = /API Error:\s*4\d{2}\b/i;
|
||||
// Auth keyword — the message must be about authentication, not just quote a 4xx.
|
||||
const TUI_ERR_AUTH_KW = /authenticat|\/login|credential/i;
|
||||
// Code/quote chars — their presence signals prose QUOTING an error, not the banner.
|
||||
const TUI_ERR_CODE_CHAR = /[`'"]/;
|
||||
|
||||
// Default detector: returns true iff `trimmed` IS a claude-CLI auth-failure banner
|
||||
// (all four signals above). Conservative — any signal failing => false (PASS).
|
||||
function isDefaultAuthFailureBanner(trimmed) {
|
||||
if (trimmed.length > TUI_ERR_MAX_LEN) return false; // 1. short whole-message
|
||||
if (!TUI_ERR_4XX.test(trimmed)) return false; // 2. 4xx API Error core
|
||||
if (!TUI_ERR_AUTH_KW.test(trimmed)) return false; // 3. auth keyword
|
||||
if (TUI_ERR_CODE_CHAR.test(trimmed)) return false; // 4. no code/quote chars
|
||||
return true;
|
||||
}
|
||||
|
||||
// Compile an OPERATOR-SUPPLIED pattern set (override path only). Each source is
|
||||
// anchored ^…$ over the trimmed text and matched case-insensitively (`s` so `.` spans
|
||||
// a multi-line banner). A pattern that fails to compile is skipped (never throws into
|
||||
// the request path).
|
||||
function compileTuiErrorPatterns(raw) {
|
||||
const sources = String(raw).split(/\r?\n|\|\|/).map((s) => s.trim()).filter(Boolean);
|
||||
const out = [];
|
||||
for (const src of sources) {
|
||||
try { out.push(new RegExp(`^(?:${src})$`, "is")); } catch { /* skip bad pattern */ }
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// Returns the matched banner text (the trimmed assistant text) if `text` IS a claude-
|
||||
// CLI auth-failure banner in its entirety, else null. `patternsRaw` defaults to
|
||||
// process.env.CLAUDE_TUI_ERROR_PATTERNS:
|
||||
// - undefined → narrowed default auth-banner detector (isDefaultAuthFailureBanner).
|
||||
// - non-empty → operator regex override REPLACES the default.
|
||||
// - empty/ws → detection disabled (escape hatch).
|
||||
export function detectTuiUpstreamError(text, patternsRaw = process.env.CLAUDE_TUI_ERROR_PATTERNS) {
|
||||
if (typeof text !== "string") return null;
|
||||
const trimmed = text.trim();
|
||||
if (!trimmed) return null;
|
||||
if (patternsRaw == null) {
|
||||
return isDefaultAuthFailureBanner(trimmed) ? trimmed : null;
|
||||
}
|
||||
// Operator override path: empty/whitespace disables; otherwise use only their regexes.
|
||||
const patterns = compileTuiErrorPatterns(patternsRaw);
|
||||
if (patterns.length === 0) return null;
|
||||
for (const re of patterns) {
|
||||
if (re.test(trimmed)) return trimmed;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Block until the session transcript is terminal (turn_duration) or
|
||||
// the wall-clock cap elapses, polling the file (no fs.watch — robust over NFS /
|
||||
// editors). Returns { text, entrypoint } where text is the latest assistant text
|
||||
// and entrypoint is the billing-pool classifier from the turn_duration line (e.g.
|
||||
// "cli"), or null if not yet present. On cap with text, returns the partial result;
|
||||
// on cap with no text at all, throws.
|
||||
// Block until the session transcript is terminal (turn_duration / final
|
||||
// stop_reason) or the wall-clock cap elapses, polling the file (no fs.watch —
|
||||
// robust over NFS / editors). Returns { text, entrypoint, truncated }:
|
||||
// - text: latest assistant text.
|
||||
// - entrypoint: billing-pool classifier (see verifyEntrypoint), or null.
|
||||
// - truncated: FALSE when a terminal marker was reached (the turn completed);
|
||||
// TRUE when the wall-clock cap was hit with partial text but NO
|
||||
// terminal marker (the turn is INCOMPLETE — what we have is a
|
||||
// cut-off prefix). (C-2, issue #133.)
|
||||
//
|
||||
// Why `truncated` matters: previously the terminal-marker path and the
|
||||
// cap-with-partial-text path BOTH returned `{text, entrypoint}` identically, so
|
||||
// callClaudeTui could not tell a complete answer from a truncated one and cached +
|
||||
// returned the partial as finish_reason:stop (silent success). The caller now
|
||||
// throws on `truncated` so a cut-off turn is neither cached nor counted as success.
|
||||
// The field is additive — existing call sites that ignore it keep working.
|
||||
//
|
||||
// On cap with NO text at all, still throws (unchanged) — there is nothing to return.
|
||||
//
|
||||
// No quiescence heuristic by design: a long Opus thinking turn stalls transcript
|
||||
// growth and a "file stable for N s" rule would false-abort it (spec §4.3).
|
||||
@@ -135,10 +280,14 @@ export async function readTuiTranscript({ transcriptPath: p, home, sessionId, wa
|
||||
lastText = extractLatestAssistantText(events) || lastText;
|
||||
const ep = verifyEntrypoint(events);
|
||||
if (ep != null) lastEntrypoint = ep;
|
||||
if (events.some(isTerminalLine)) return { text: lastText, entrypoint: lastEntrypoint };
|
||||
// Terminal marker reached → the turn is COMPLETE.
|
||||
if (events.some(isTerminalLine)) return { text: lastText, entrypoint: lastEntrypoint, truncated: false };
|
||||
}
|
||||
await sleep(pollMs);
|
||||
}
|
||||
if (lastText) return { text: lastText, entrypoint: lastEntrypoint };
|
||||
// Cap elapsed with no terminal marker. If we have partial text, flag it truncated
|
||||
// so the caller rejects it (don't cache / don't count as success). No text at all
|
||||
// → throw (nothing to return).
|
||||
if (lastText) return { text: lastText, entrypoint: lastEntrypoint, truncated: true };
|
||||
throw new Error("tui_transcript_timeout: no assistant text within wallclock cap");
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user