feat(server): SPOT-derived prompt budget — MAX_PROMPT_CHARS default follows models.json (ADR 0009)

Maintainer directive (2026-07-18): the hand-set 150,000-char default (~37.5k English
tokens) is obsolete in the long-context era. Instead of a new constant that would rot
the same way, the default now derives from the SPOT:

  MAX_PROMPT_CHARS (default) = max(models.json contextWindow) x 3 chars/token
                             = 200000 x 3 = 600,000 chars today (~150-200k tokens)

x3 is the CJK-safe multiplier: English runs ~4 chars/token, CJK ~1-1.5, so a
1M-token-derived char cap would let CJK text sail past the model's real window into
an upstream rejection; at x3 the cap fires at roughly the model's true window and OCP
truncates gracefully (tail-first) instead. Pure derivePromptCharBudget() in
lib/prompt.mjs with a 150k floor guarding degenerate SPOT states (empty models[],
absent contextWindow) - a zero budget would truncate every request to nothing.

CLAUDE_MAX_PROMPT_CHARS (env) and the runtime settings API remain ABSOLUTE overrides;
derivation applies only when neither is set. If models.json ever advertises a larger
window (e.g. 1M for the 1M-native models), the budget scales automatically - that
advertisement is a separate deliberate decision (quota burn, OpenClaw compaction, TUI
paste limits) explicitly NOT made here; see ADR 0009.

Behavior change (intended): requests between 150k and 600k chars previously truncated
now pass through whole - longer TTFT + higher quota use for those requests. Truncation
mechanism/logging unchanged; only the default's provenance changed.

ALIGNMENT.md Rule 2: no cli.js citation applies - the truncation guard is OCP-internal
prompt shaping; no endpoint, header, or wire field changes.

Tests: +4, doubly mutation-proven (max->min fails the largest-window test; dropping
the floor fails the floor test). Suite 347 passed / 0 failed. ADR 0009 + index row +
README env-table row included (release_kit: env var default change documented).

Co-Authored-By: Claude <claude-opus-4-8> <noreply@anthropic.com>
This commit is contained in:
2026-07-18 10:03:58 +10:00
co-authored by Claude <claude-opus-4-8> <noreply@anthropic.com>
parent 17038b56e6
commit 8405e30d0e
6 changed files with 120 additions and 4 deletions
+10 -2
View File
@@ -49,7 +49,7 @@ import { TuiSemaphore, SemaphoreAbortError, recordTuiEntrypoint, buildTuiHealthB
import { TuiPanePool, resolvePoolSize, POOL_MAX_SIZE } from "./lib/tui/pool.mjs";
import { TuiDeltaAssembler, DEFAULT_HOLDBACK_CHARS, resolveStreamHoldback } from "./lib/tui/stream.mjs";
import { createSerialMutex, createTtlCache, isTokenExpiring, orderLabelsLastGoodFirst } from "./lib/spawn-auth.mjs";
import { appendOperatorPrompt } from "./lib/prompt.mjs";
import { appendOperatorPrompt, derivePromptCharBudget } from "./lib/prompt.mjs";
const __dirname = dirname(fileURLToPath(import.meta.url));
const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8"));
@@ -1148,7 +1148,15 @@ function buildCliArgs(cliModel, systemPrompt) {
// Truncation guard: if total chars exceed MAX_PROMPT_CHARS, keep the system
// message(s) + first user message + last N messages, dropping the middle.
// This prevents runaway context from gateway-side conversation accumulation.
let MAX_PROMPT_CHARS = parseInt(process.env.CLAUDE_MAX_PROMPT_CHARS || "150000", 10);
//
// Default is SPOT-DERIVED (ADR 0009): max(models.json contextWindow) × 3 chars/token —
// currently 200000 × 3 = 600,000 chars — instead of the old hand-set 150000 (≈37.5k
// English tokens), which silently under-delivered the advertised window by ~5×. The env
// var (and the runtime settings API below) remain absolute operator overrides. If
// models.json ever advertises a bigger window, this budget follows automatically.
let MAX_PROMPT_CHARS = process.env.CLAUDE_MAX_PROMPT_CHARS != null
? parseInt(process.env.CLAUDE_MAX_PROMPT_CHARS, 10)
: derivePromptCharBudget(modelsConfig.models);
// Flatten OpenAI content (string | array of parts) to plain text for the prompt.
// Array content: concatenate text parts; replace non-text parts (e.g. image_url)