feat(server): SPOT-derived prompt budget — MAX_PROMPT_CHARS follows models.json (ADR 0009) (#179)

* feat(server): SPOT-derived prompt budget — MAX_PROMPT_CHARS default follows models.json (ADR 0009)

Maintainer directive (2026-07-18): the hand-set 150,000-char default (~37.5k English
tokens) is obsolete in the long-context era. Instead of a new constant that would rot
the same way, the default now derives from the SPOT:

  MAX_PROMPT_CHARS (default) = max(models.json contextWindow) x 3 chars/token
                             = 200000 x 3 = 600,000 chars today (~150-200k tokens)

x3 is the CJK-safe multiplier: English runs ~4 chars/token, CJK ~1-1.5, so a
1M-token-derived char cap would let CJK text sail past the model's real window into
an upstream rejection; at x3 the cap fires at roughly the model's true window and OCP
truncates gracefully (tail-first) instead. Pure derivePromptCharBudget() in
lib/prompt.mjs with a 150k floor guarding degenerate SPOT states (empty models[],
absent contextWindow) - a zero budget would truncate every request to nothing.

CLAUDE_MAX_PROMPT_CHARS (env) and the runtime settings API remain ABSOLUTE overrides;
derivation applies only when neither is set. If models.json ever advertises a larger
window (e.g. 1M for the 1M-native models), the budget scales automatically - that
advertisement is a separate deliberate decision (quota burn, OpenClaw compaction, TUI
paste limits) explicitly NOT made here; see ADR 0009.

Behavior change (intended): requests between 150k and 600k chars previously truncated
now pass through whole - longer TTFT + higher quota use for those requests. Truncation
mechanism/logging unchanged; only the default's provenance changed.

ALIGNMENT.md Rule 2: no cli.js citation applies - the truncation guard is OCP-internal
prompt shaping; no endpoint, header, or wire field changes.

Tests: +4, doubly mutation-proven (max->min fails the largest-window test; dropping
the floor fails the floor test). Suite 347 passed / 0 failed. ADR 0009 + index row +
README env-table row included (release_kit: env var default change documented).

Co-Authored-By: Claude <claude-opus-4-8> <noreply@anthropic.com>

* fix(server): empty CLAUDE_MAX_PROMPT_CHARS falls back to derived default (PR #179 review)

Reviewer regression: `!= null` treated an EMPTY env value ("CLAUDE_MAX_PROMPT_CHARS="
in an EnvironmentFile/.env) as explicit -> parseInt("") = NaN -> guard disabled + a
false "[System] Note: 0 older messages were truncated" injected into every prompt.
Extracted resolvePromptCharBudget() (truthiness contract, matching the old
`parseInt(env || default)` behavior) into lib/prompt.mjs so the semantics are
mutation-tested: switching back to != null fails the empty-string test. +2 tests, 349/0.

Co-Authored-By: Claude <claude-opus-4-8> <noreply@anthropic.com>

---------

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude <claude-opus-4-8> <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
2026-07-18 10:11:38 +10:00
committed by GitHub
co-authored by taodeng Claude <claude-opus-4-8> <noreply@anthropic.com>
parent 17038b56e6
commit fe12419386
6 changed files with 144 additions and 4 deletions
+37
View File
@@ -15,3 +15,40 @@ export function appendOperatorPrompt(base, operatorAppend) {
const op = typeof operatorAppend === "string" ? operatorAppend.trim() : "";
return op ? `${base}\n\n${op}` : base;
}
// Derive the default prompt-char budget from the models.json SPOT (ADR 0009).
//
// The old default was a hand-set constant (150000 chars ≈ 37.5k English tokens) from the
// 200k-window era — silently far below what the advertised contextWindow promises. Instead
// of picking a new constant that will also rot, the default now FOLLOWS the SPOT:
//
// budget = max(models[].contextWindow) × charsPerToken
//
// charsPerToken = 3 is deliberately conservative: English runs ~4 chars/token, CJK ~11.5.
// At ×3, a 200k-token window yields 600,000 chars — full window for English, and CJK text
// hits the model's real window at roughly the same point the cap fires, so we truncate
// (graceful, tail-first) rather than let the upstream reject the request outright.
//
// The floor guards the degenerate cases (empty/missing models[], absent contextWindow):
// fall back to the historical constant rather than 0 — a zero budget would truncate every
// request to nothing, which is fail-OPEN in the "serve garbage" sense. CLAUDE_MAX_PROMPT_CHARS
// remains an absolute operator override at the call site (server.mjs); this function is only
// the unset-env default.
export function derivePromptCharBudget(models, { charsPerToken = 3, floor = 150000 } = {}) {
const windows = (Array.isArray(models) ? models : [])
.map(m => m?.contextWindow)
.filter(w => Number.isFinite(w) && w > 0);
if (windows.length === 0) return floor;
return Math.max(floor, Math.max(...windows) * charsPerToken);
}
// Resolve the effective budget from the env var + SPOT. TRUTHINESS (not != null) on the env
// value deliberately: an EMPTY value ("CLAUDE_MAX_PROMPT_CHARS=" in a systemd EnvironmentFile
// or .env) must mean "use the default" — exactly the old `parseInt(env || "150000")` contract.
// Treating "" as explicit gives parseInt("") = NaN, and a NaN cap silently DISABLES the
// runaway-context guard while injecting a false "[System] Note: 0 older messages were
// truncated" line into every prompt (caught in PR #179 review). Non-empty garbage still
// parses to NaN — the pre-existing class, slated for parseIntEnv routing in PR #154.
export function resolvePromptCharBudget(rawEnv, models, opts) {
return rawEnv ? parseInt(rawEnv, 10) : derivePromptCharBudget(models, opts);
}