taodeng/ocp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/ocp.git synced 2026-07-21 21:15:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(cache): fold a boot-config epoch into the response-cache key (#176) (#177)

Browse Source 
The cache key hashed model + keyId + sampling params + raw messages, but the
ANSWER also depends on boot-time server config that shapes the composed prompt
and tool surface: CLAUDE_SYSTEM_PROMPT (newly load-bearing since #175),
OCP_SYSTEM_PROMPT_WRAPPER, CLAUDE_ALLOWED_TOOLS, and CLAUDE_NO_CONTEXT. The
cache store is SQLite-backed and survives restarts, so an operator who changed
any of these and restarted kept serving answers composed under the OLD config
until TTL expiry (found by the #175 independent reviewer).

Fix: server.mjs computes CONFIG_EPOCH once at boot — a 16-hex sha256 digest of
the four values — and passes it to cacheHash, which folds `ce:<epoch>|` into the
key. Any config change = instant whole-cache invalidation (the honest behavior).
Callers that omit configEpoch (tests, any older path) hash byte-identically to
before — asserted by test. Runtime-mutable settings (maxPromptChars via the
settings API) are deliberately excluded: a const epoch cannot track them, and
truncation drops context rather than changing the instruction set (noted in the
code comment).

One-time side effect on upgrade: existing cache entries no longer match (keys
now carry the epoch). Cache is off by default and TTL-bounded; a one-time miss
storm is the cost of never serving stale-config answers again.

ALIGNMENT.md Rule 2: no cli.js citation applies — cache-key composition is
OCP-internal (Class B); no endpoint, header, or wire field changes.

Tests: +2, mutation-proven (dropping the fold fails both). Suite 343/0.

Closes #176

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude <claude-opus-4-8> <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
2026-07-17 19:31:31 +10:00
committed by GitHub
co-authored by taodeng Claude <claude-opus-4-8> <noreply@anthropic.com>
parent e6f1a6aac1
commit 73314e6698
3 changed files with 39 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server.mjs
+17 -3
View File
@@ -35,7 +35,7 @@
*/
import { createServer } from "node:http";
import { spawn, execFileSync, spawnSync } from "node:child_process";
import { randomUUID, timingSafeEqual } from "node:crypto";
import { randomUUID, timingSafeEqual, createHash as cryptoCreateHash } from "node:crypto";
import { readFileSync, readdirSync, accessSync, existsSync, constants, chmodSync, statSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
@@ -346,6 +346,19 @@ const BREAKER_HALF_OPEN_MAX = parseInt(process.env.CLAUDE_BREAKER_HALF_OPEN_MAX
const HEARTBEAT_INTERVAL = parseInt(process.env.CLAUDE_HEARTBEAT_INTERVAL || "0", 10);
const BIND_ADDRESS = process.env.CLAUDE_BIND || "127.0.0.1";
const NO_CONTEXT = process.env.CLAUDE_NO_CONTEXT === "true";
// Config epoch for the response cache (issue #176). The cache key hashes model + messages +
// sampling params, but the ANSWER also depends on boot-time server config that shapes the
// composed prompt / tool surface: the operator system prompt (#175), the OCP wrapper text,
// the allowed-tools set, and NO_CONTEXT. The cache store is SQLite-backed and survives
// restarts, so without this an operator who changes any of these and restarts keeps serving
// answers composed under the OLD config until TTL expiry. Folding a digest of the four into
// every cache key makes any change an instant, whole-cache invalidation — the honest behavior.
// Deliberately boot-time-only: runtime-mutable settings (e.g. maxPromptChars via the settings
// API) are excluded because a const epoch cannot track them; truncation also only drops
// context rather than changing the instruction set.
const CONFIG_EPOCH = cryptoCreateHash("sha256")
.update(JSON.stringify([SYSTEM_PROMPT, OCP_SYSTEM_PROMPT_WRAPPER, ALLOWED_TOOLS, NO_CONTEXT]))
.digest("hex").slice(0, 16);
// Kill-switch for the FIX-③ default-path spawn-home isolation (see resolveSpawnHome /
// spawnHomeMode below). When "1", the -p/stream-json spawn always runs in the operator's
// real HOME with no cwd override — byte-for-byte the pre-isolation behaviour — even if an
@@ -2629,8 +2642,9 @@ async function handleChatCompletions(req, res) {
req._cacheHash = null;
logEvent("info", "cache_skipped", { reason: "cache_control_present" });
} else {
// D1: include keyId in hash to isolate per-key cache pools (v2 format)
const hash = cacheHash(model, messages, { keyId: req._authKeyId, temperature: parsed.temperature, max_tokens: parsed.max_tokens, top_p: parsed.top_p });
// D1: include keyId in hash to isolate per-key cache pools (v2 format).
// configEpoch (#176): any boot-config change that shapes answers invalidates the cache.
const hash = cacheHash(model, messages, { keyId: req._authKeyId, temperature: parsed.temperature, max_tokens: parsed.max_tokens, top_p: parsed.top_p, configEpoch: CONFIG_EPOCH });
req._cacheHash = hash; // store for later write-back
try {
const cached = getCachedResponse(hash, CACHE_TTL);