taodeng/ocp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/ocp.git synced 2026-07-19 09:44:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(release): v3.13.0 — cache layer hardening (per-key isolation + bypass + chunked replay + singleflight) (#67)

Browse Source 
Per release_kit overlay (CLAUDE.md § Iron Rule 5.5):
- package.json bumped 3.12.0 → 3.13.0
- CHANGELOG.md updated with v3.13.0 entry
- README.md § Response Cache updated to document the four hardening features

This is a release preparation commit. Tag push to v3.13.0 will trigger
.github/workflows/release.yml which auto-creates the GitHub Release.

cli.js does not perform proxy-layer response caching, stampede protection,
or replay; this release only ships internal cache-layer correctness and
concurrency improvements that do not change the OpenAI-compatible wire
surface visible to clients. No client-observable wire shape change.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
2026-05-07 17:19:44 +10:00
committed by GitHub
co-authored by taodeng Claude Opus 4.7
parent 5ff30ac9b6
commit 9494fd6c69
3 changed files with 32 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+9 -4
View File
@@ -472,17 +472,20 @@ ocp settings cacheTTL 300000
```
**How it works:**
- Cache key = SHA-256 of `model` + `messages` + `temperature` + `max_tokens` + `top_p`
- Cache key = SHA-256 of `v2|<keyId or "anon">|model + messages + temperature + max_tokens + top_p`
- **Per-key isolation** — different API keys never share cache entries; anonymous callers share one `anon` pool
- Cache hits return instantly — no Claude CLI process spawned
- Works for both streaming and non-streaming requests
- **Streaming hits** are replayed as multiple SSE chunks (80 codepoints each), not one large delta — incremental render preserved
- **`cache_control` bypass** — if a request carries an Anthropic `cache_control` annotation (top-level or nested in `content[]`), OCP skips its own cache entirely so it doesn't interfere with Anthropic-side prompt caching
- **Singleflight stampede protection** — concurrent identical cache-miss requests share one upstream `cli.js` spawn; followers receive byte-identical responses to the leader's call. Non-streaming path only (streaming-path singleflight is a known TODO)
- Multi-turn conversations (with `session_id`) are never cached
- Expired entries are cleaned up automatically every 10 minutes
**Management:**
```bash
# View cache stats
# View cache stats (now includes singleflight in-flight counts)
curl http://127.0.0.1:3456/cache/stats
# → { "entries": 42, "totalHits": 156, "sizeBytes": 284000 }
# → { "entries": 42, "totalHits": 156, "sizeBytes": 284000, "inflight": 0, "requesters": 0 }
# Clear all cached responses
curl -X DELETE http://127.0.0.1:3456/cache
@@ -493,6 +496,8 @@ ocp settings cacheTTL 0
Cache is **disabled by default** (`CLAUDE_CACHE_TTL=0`). All data is stored locally in `~/.ocp/ocp.db`.
**Hash format upgrade in v3.13.0:** legacy `v1` cache rows from earlier versions don't match new `v2`-format lookups; they orphan and are reaped by the TTL cleanup interval within one window. No migration script required.
## How It Works
```