From 9494fd6c69035fd972e4baa7650dcc8361125166 Mon Sep 17 00:00:00 2001 From: dtzp555-max Date: Thu, 7 May 2026 17:19:44 +1000 Subject: [PATCH] =?UTF-8?q?chore(release):=20v3.13.0=20=E2=80=94=20cache?= =?UTF-8?q?=20layer=20hardening=20(per-key=20isolation=20+=20bypass=20+=20?= =?UTF-8?q?chunked=20replay=20+=20singleflight)=20(#67)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per release_kit overlay (CLAUDE.md § Iron Rule 5.5): - package.json bumped 3.12.0 → 3.13.0 - CHANGELOG.md updated with v3.13.0 entry - README.md § Response Cache updated to document the four hardening features This is a release preparation commit. Tag push to v3.13.0 will trigger .github/workflows/release.yml which auto-creates the GitHub Release. cli.js does not perform proxy-layer response caching, stampede protection, or replay; this release only ships internal cache-layer correctness and concurrency improvements that do not change the OpenAI-compatible wire surface visible to clients. No client-observable wire shape change. Co-authored-by: dtzp555 Co-authored-by: Claude Opus 4.7 --- CHANGELOG.md | 22 ++++++++++++++++++++++ README.md | 13 +++++++++---- package.json | 2 +- 3 files changed, 32 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 01fee26..3e5f4d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,27 @@ # Changelog +## v3.13.0 — 2026-05-07 + +### Features (cache layer hardening) + +- **Per-key cache isolation** (D1) — the cache key now includes the API key id, so distinct keys never share cache entries. Anonymous/unauthenticated callers share one `anon` pool. Hash format upgraded to `v2`; legacy v1-format rows orphan and are reaped by the existing TTL cleanup interval (no migration script). +- **`cache_control` bypass** (D2) — when a request carries an Anthropic `cache_control` annotation (top-level or nested in a content array), OCP skips its own cache entirely. The caller is using Anthropic-side prompt caching deliberately, and OCP must not interfere. A `cache_skipped{reason: cache_control_present}` log line is emitted on bypass. +- **Chunked stream replay** (D3) — when a streaming request hits the cache, the cached content is now emitted as multiple SSE chunks (80 codepoints/chunk, codepoint-safe via `Array.from()`) instead of a single large delta. Multibyte characters (CJK / emoji) stay intact. +- **Singleflight stampede protection** (D4) — concurrent identical cache-miss requests now share one upstream `cli.js` spawn instead of spawning N processes. Followers receive byte-identical responses to what the leader returns. All-or-nothing failure semantics: if the leader errors, all followers receive the same error. Streaming-path singleflight is explicitly out of scope (TODO left for follow-up). + +### Behavior changes + +- `/cache/stats` response now includes additive fields `inflight` and `requesters` (current in-flight singleflight entries and total waiting callers). Existing fields `entries`, `totalHits`, `sizeBytes` are preserved unchanged. + +### Governance + +- New ADR [`docs/adr/0005-no-multi-provider.md`](docs/adr/0005-no-multi-provider.md): OCP stays single-provider (Anthropic via `cli.js` spawn). Multi-provider gateway refactor explicitly out of scope; cache improvements are explicitly in scope. +- Design spec for this release: [`docs/superpowers/specs/2026-05-07-cache-upgrade-design.md`](docs/superpowers/specs/2026-05-07-cache-upgrade-design.md). + +### No new env vars / no public API surface change + +This release adds no new env vars or endpoints. All four improvements are internal correctness/concurrency upgrades to the existing `CLAUDE_CACHE_TTL`-gated cache layer. No client-observable wire shape change. + ## v3.12.0 — 2026-04-25 ### Features diff --git a/README.md b/README.md index dec9bfd..8bc0b72 100644 --- a/README.md +++ b/README.md @@ -472,17 +472,20 @@ ocp settings cacheTTL 300000 ``` **How it works:** -- Cache key = SHA-256 of `model` + `messages` + `temperature` + `max_tokens` + `top_p` +- Cache key = SHA-256 of `v2||model + messages + temperature + max_tokens + top_p` +- **Per-key isolation** — different API keys never share cache entries; anonymous callers share one `anon` pool - Cache hits return instantly — no Claude CLI process spawned -- Works for both streaming and non-streaming requests +- **Streaming hits** are replayed as multiple SSE chunks (80 codepoints each), not one large delta — incremental render preserved +- **`cache_control` bypass** — if a request carries an Anthropic `cache_control` annotation (top-level or nested in `content[]`), OCP skips its own cache entirely so it doesn't interfere with Anthropic-side prompt caching +- **Singleflight stampede protection** — concurrent identical cache-miss requests share one upstream `cli.js` spawn; followers receive byte-identical responses to the leader's call. Non-streaming path only (streaming-path singleflight is a known TODO) - Multi-turn conversations (with `session_id`) are never cached - Expired entries are cleaned up automatically every 10 minutes **Management:** ```bash -# View cache stats +# View cache stats (now includes singleflight in-flight counts) curl http://127.0.0.1:3456/cache/stats -# → { "entries": 42, "totalHits": 156, "sizeBytes": 284000 } +# → { "entries": 42, "totalHits": 156, "sizeBytes": 284000, "inflight": 0, "requesters": 0 } # Clear all cached responses curl -X DELETE http://127.0.0.1:3456/cache @@ -493,6 +496,8 @@ ocp settings cacheTTL 0 Cache is **disabled by default** (`CLAUDE_CACHE_TTL=0`). All data is stored locally in `~/.ocp/ocp.db`. +**Hash format upgrade in v3.13.0:** legacy `v1` cache rows from earlier versions don't match new `v2`-format lookups; they orphan and are reaped by the TTL cleanup interval within one window. No migration script required. + ## How It Works ``` diff --git a/package.json b/package.json index 6a6a63d..301c877 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "open-claude-proxy", - "version": "3.12.0", + "version": "3.13.0", "description": "OCP (Open Claude Proxy) — use your Claude Pro/Max subscription as an OpenAI-compatible API for any IDE. Works with Cline, OpenCode, Aider, Continue.dev, OpenClaw, and more.", "type": "module", "bin": {