Compare commits

...
Author SHA1 Message Date
taodengandClaude Sonnet 4.6 eb6cd09bed fix(setup): remove duplicate server spawn; verify health post-install
## Dogfood evidence (Pi231, 2026-05-08)

A user ran `node setup.mjs --bind 0.0.0.0 --auth-mode multi` and got:
- setup.mjs exit 0
- /health responded with authMode:"none" and server bound to 127.0.0.1 only
- ps showed two server.mjs processes: one orphan from setup (wrong config),
  one systemd child restart-looping on EADDRINUSE

## Root cause (two-step conflict)

Step 6 (the deleted block) called `execSync('bash "${startPath}"')` which ran
start.sh's `nohup node server.mjs &` — spawning the server WITHOUT exporting
CLAUDE_BIND or CLAUDE_AUTH_MODE. That server ran with default bind=127.0.0.1
and authMode=none, ignoring the user's CLI flags.

Step 7 then wrote the systemd unit/launchd plist WITH the correct env vars and
bootstrapped the service — but port 3456 was already taken by Step 6's spawn,
causing EADDRINUSE and a silent restart loop. setup.mjs exited 0 because Step 6
had "succeeded" (a server was running, just the wrong one).

## What changed

1. Deleted Step 6 entirely (the `execSync('bash "${startPath}"')` block).
   The systemd/launchd service installed in Step 7 is now the sole authoritative
   start path. start.sh is unchanged and still available for manual non-systemd use.

2. Added Step 8 inside the `if (!DRY_RUN)` block: after Step 7 bootstrap,
   waits 3 s, then GETs http://127.0.0.1:${PORT}/health with a 5 s timeout.
   - On 200 OK: logs version, authMode, and bind socket (best-effort).
   - On failure: prints clear error pointing to service logs, exits 1.
   - Skipped when --no-start is set (existing flag).

Identified during PR #71 dogfood testing on Pi231 (RPi4 / Debian Bookworm).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 14:52:27 +10:00
cce0110253 docs(why-ocp): reorder bullets + soften alignment framing + visceral hooks (#70)
The "Why OCP?" section previously led with technical/governance bullets
(SSE heartbeat, alignment, models.json) and buried the most relatable
selling points (LAN multi-user keys, ocp-connect IDE auto-config, cache).
First-time readers stopped reading before reaching the bullets that would
have sold them on the project.

Changes (README.md only, line 19-26):

1. Reorder: human-relatable benefits first, governance discipline last.
   New order: LAN multi-user → ocp-connect → cache → quota → SSE heartbeat
   → alignment → models.json. Quota is split out from the old combined
   "quota + cache" bullet so each gets its own one-liner.

2. Soften the alignment bullet's framing. The previous prose flagged
   "Other Claude proxies have shipped exactly that" and was deleted —
   no need to call out competitors. Replaced with a measured "LLM-assisted
   code drifts easily — it's tempting to invent plausible-looking endpoints
   that cli.js doesn't actually use" plus a deliberately understated
   payoff: "your setup keeps working when cli.js ships its next minor."

3. Add visceral hooks where bullets benefit from them:
   - SSE heartbeat: "If you've ever watched your IDE die at the 60s idle
     mark during a long Claude tool-use pause — that's nginx/Cloudflare
     default behavior" (frames the problem in user-felt terms before
     describing the fix).
   - Per-key quota: concrete example "set a kid's iPad to 20/day, a
     partner's laptop to 100/week" (replaces abstract "limits per key").

4. Cache bullet now explicitly states the per-key isolation guarantee:
   "cross-user pollution is impossible by hash construction, not by
   application logic" — this addresses the most common pre-adoption
   concern (and reflects the v3.13.0 D1 design).

5. Total length compressed ~20% despite added context — old bullets had
   redundant "what" descriptions; new bullets lead with the "what for".

Doc-only change. server.mjs not touched. ALIGNMENT.md Rule 5 (cli.js
citation requirement) does not apply. Same pattern as #68, #69.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 13:57:06 +10:00
40391791a1 docs(funding): raise sponsorship visibility — top badges + intro CTA + expanded support section (#69)
Context: Buy Me a Coffee + Stripe onboarding now fully live (verified
buymeacoffee.com/dtzp555 returns og:type=profile with Support CTA, default
$3 price tier, membership tier active). Previous §Support OCP at line 709
was effectively buried — last section before License, missed by most readers.

Changes (README.md only, server.mjs untouched):

1. Top of file — three shields.io badges (License MIT, latest release,
   Buy Me a Coffee) under H1, before tagline. Standard OSS pattern
   (Vue / Vite / Tailwind), high visibility, doesn't disrupt prose.

2. Just under tagline — one-line italic personal note pointing to
   §Support OCP, with inline  link as fallback for users who don't
   scroll. Quotes the spirit of the longer section without duplicating it.

3. §Support OCP expanded — adds the "open source from day one" framing
   (not freemium, not commercial-turned-open), the "my family uses it
   daily" angle, and an explicit feedback / issues invitation. The
   debugging-history paragraph is preserved verbatim.

Doc-only change. ALIGNMENT.md Rule 5 (cli.js citation) does not apply —
server.mjs is not modified. Same pattern as #68.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 13:29:58 +10:00
342a0a44f5 docs(funding): add Buy Me a Coffee link + GitHub FUNDING.yml (#68)
- .github/FUNDING.yml — enables GitHub's native "Sponsor" button on the
  repo page, pointing to buymeacoffee.com/dtzp555. Other platforms
  (GitHub Sponsors, Ko-fi) are commented out and can be enabled later
  by uncommenting + filling in handles.
- README.md § Support OCP — new section just before License. States the
  free-and-open-source commitment, lists the kinds of work that don't
  show up in commits (multi-machine debugging, IDE validation, drift
  incidents, concurrency leaks), and offers a single  link for users
  who want to support continued maintenance. Explicitly disclaims paid
  tiers / premium features so the open-source posture stays unambiguous.

server.mjs is not modified; this commit is doc-only and therefore exempt
from the cli.js citation requirement (ALIGNMENT.md Rule 5 applies only
to commits that touch server.mjs).

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 03:29:27 +10:00
9494fd6c69 chore(release): v3.13.0 — cache layer hardening (per-key isolation + bypass + chunked replay + singleflight) (#67)
Per release_kit overlay (CLAUDE.md § Iron Rule 5.5):
- package.json bumped 3.12.0 → 3.13.0
- CHANGELOG.md updated with v3.13.0 entry
- README.md § Response Cache updated to document the four hardening features

This is a release preparation commit. Tag push to v3.13.0 will trigger
.github/workflows/release.yml which auto-creates the GitHub Release.

cli.js does not perform proxy-layer response caching, stampede protection,
or replay; this release only ships internal cache-layer correctness and
concurrency improvements that do not change the OpenAI-compatible wire
surface visible to clients. No client-observable wire shape change.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 17:19:44 +10:00
5ff30ac9b6 feat(cache): singleflight stampede protection on non-streaming path (#66)
cli.js does not perform proxy-layer stampede protection. The singleflight
layer is a value-add proxy operation that exists only inside OCP, between
concurrent client requests and the single upstream cli.js spawn. It does
not introduce, alter, or remove any endpoint, header, request field, or
response field that cli.js emits or expects — no client-observable wire
shape change.

Justification under ALIGNMENT.md Rule 2: the singleflight Map deduplicates
concurrent identical non-streaming cache-miss requests so only one cli.js
spawn runs per unique hash window. All followers receive the same resolved
(or rejected) content. This is a proxy-internal concurrency optimization,
not a wire-level protocol change.

Spec: docs/superpowers/specs/2026-05-07-cache-upgrade-design.md (D4)

Changes:
- keys.mjs: add singleflight(hash, fn) + getInflightStats() exports;
  in-memory Map cleared via Promise.finally() on each settlement
- server.mjs: import singleflight + getInflightStats; wrap non-streaming
  cache-enabled path through singleflight with inner recheck; add TODO
  comment in callClaudeStreaming (streaming-path dedup is explicitly out
  of scope for v3.13.0, see spec D4 streaming caveat); extend /cache/stats
  to return inflight + requesters fields (additive, no removed fields)
- test-features.mjs: 7 new PR-B singleflight tests covering basic dedup,
  failure fan-out, map cleanup (success + failure), different-hash
  independence, getInflightStats shape, and sequential-call non-sharing;
  all 31 tests pass (24 existing + 7 new)

Streaming-path singleflight is explicitly out of scope; TODO left in
callClaudeStreaming for a future follow-up ticket.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 17:16:46 +10:00
8 changed files with 289 additions and 27 deletions
+9
View File
@@ -0,0 +1,9 @@
# GitHub recognizes this file and shows a "Sponsor" button on the repo page.
# Add other platforms here as they get set up. Empty / commented-out entries
# are skipped silently.
buy_me_a_coffee: dtzp555
# github: [dtzp555-max] # uncomment after GitHub Sponsors enrollment is approved
# ko_fi: dtzp555
# custom: ["https://example.com/donate"]
+22
View File
@@ -1,5 +1,27 @@
# Changelog
## v3.13.0 — 2026-05-07
### Features (cache layer hardening)
- **Per-key cache isolation** (D1) — the cache key now includes the API key id, so distinct keys never share cache entries. Anonymous/unauthenticated callers share one `anon` pool. Hash format upgraded to `v2`; legacy v1-format rows orphan and are reaped by the existing TTL cleanup interval (no migration script).
- **`cache_control` bypass** (D2) — when a request carries an Anthropic `cache_control` annotation (top-level or nested in a content array), OCP skips its own cache entirely. The caller is using Anthropic-side prompt caching deliberately, and OCP must not interfere. A `cache_skipped{reason: cache_control_present}` log line is emitted on bypass.
- **Chunked stream replay** (D3) — when a streaming request hits the cache, the cached content is now emitted as multiple SSE chunks (80 codepoints/chunk, codepoint-safe via `Array.from()`) instead of a single large delta. Multibyte characters (CJK / emoji) stay intact.
- **Singleflight stampede protection** (D4) — concurrent identical cache-miss requests now share one upstream `cli.js` spawn instead of spawning N processes. Followers receive byte-identical responses to what the leader returns. All-or-nothing failure semantics: if the leader errors, all followers receive the same error. Streaming-path singleflight is explicitly out of scope (TODO left for follow-up).
### Behavior changes
- `/cache/stats` response now includes additive fields `inflight` and `requesters` (current in-flight singleflight entries and total waiting callers). Existing fields `entries`, `totalHits`, `sizeBytes` are preserved unchanged.
### Governance
- New ADR [`docs/adr/0005-no-multi-provider.md`](docs/adr/0005-no-multi-provider.md): OCP stays single-provider (Anthropic via `cli.js` spawn). Multi-provider gateway refactor explicitly out of scope; cache improvements are explicitly in scope.
- Design spec for this release: [`docs/superpowers/specs/2026-05-07-cache-upgrade-design.md`](docs/superpowers/specs/2026-05-07-cache-upgrade-design.md).
### No new env vars / no public API surface change
This release adds no new env vars or endpoints. All four improvements are internal correctness/concurrency upgrades to the existing `CLAUDE_CACHE_TTL`-gated cache layer. No client-observable wire shape change.
## v3.12.0 — 2026-04-25
### Features
+39 -11
View File
@@ -1,7 +1,11 @@
# OCP — Open Claude Proxy
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) [![GitHub release](https://img.shields.io/github/v/release/dtzp555-max/ocp)](https://github.com/dtzp555-max/ocp/releases) [![Buy Me a Coffee](https://img.shields.io/badge/Buy_Me_a_Coffee-ffdd00?logo=buy-me-a-coffee&logoColor=black)](https://buymeacoffee.com/dtzp555)
> **Already paying for Claude Pro/Max? Use your subscription as an OpenAI-compatible API — $0 extra cost.**
*Open source from day one, used daily by my family, maintained on nights and weekends. If OCP saves you money too, you can [☕ buy me a coffee](https://buymeacoffee.com/dtzp555) — [full story below](#support-ocp).*
OCP turns your Claude Pro/Max subscription into a standard OpenAI-compatible API on localhost. Any tool that speaks the OpenAI protocol can use it — no separate API key, no extra billing.
```
@@ -16,14 +20,15 @@ One proxy. Multiple IDEs. All models. **$0 API cost.**
## Why OCP?
There are several Claude proxy projects. OCP picks a specific lane: **align tightly with what `cli.js` actually does, observe + multiplex what's already there, don't extend the protocol.** Concretely:
There are several Claude proxy projects. OCP picks a specific lane: **align tightly with what `cli.js` actually does, observe + multiplex what's already there, don't extend the protocol.** What you get:
- **SSE heartbeat on streaming** ([v3.12.0](https://github.com/dtzp555-max/ocp/releases/tag/v3.12.0), opt-in via `CLAUDE_HEARTBEAT_INTERVAL`). Long Claude reasoning or tool-use pauses can sit silent for minutes; downstream load balancers and reverse proxies often kill the connection at 60s idle. OCP emits an SSE comment frame during silent windows so the connection stays alive without polluting the response. ([PR #49](https://github.com/dtzp555-max/ocp/pull/49))
- **Alignment constitution + CI guardrail.** [`ALIGNMENT.md`](./ALIGNMENT.md) is the binding spec: every endpoint OCP exposes must correspond to something `cli.js` actually does, with a line-number citation. The [`alignment.yml`](./.github/workflows/alignment.yml) workflow auto-blocks PRs that introduce known-hallucinated tokens (`api/oauth/usage`, `api/usage`, etc). Hard to drift, even with LLM-assisted contributions.
- **`models.json` single source of truth** (v3.11.0). Adding a model is one file edit; both `/v1/models` and the OpenClaw bootstrap derive from it. No more drift between server and installer. ([PR #30](https://github.com/dtzp555-max/ocp/pull/30))
- **Multi anonymous-key distribution** (v3.7.0). Share one OCP instance with friends/family/devices without exposing your OAuth session — each gets a per-user key, with usage tracking and revocation.
- **Per-key quota + response cache** (v3.8.0). Daily/weekly/monthly request limits per key. Optional SHA-256 prompt cache for development loops. ([PR #18](https://github.com/dtzp555-max/ocp/pull/18))
- **`ocp-connect` IDE auto-config.** Detects Claude Code, Cursor, Cline, Continue.dev, opencode, and OpenClaw on the client machine and configures them in one command.
- **LAN multi-user keys** (v3.7.0) — share one Claude Pro/Max subscription with family, friends, or your own devices. Each user gets a per-key API token (no OAuth session leak), with independent usage tracking and one-line revocation.
- **`ocp-connect` one-shot IDE setup** — one command on the client machine detects and configures Claude Code, Cursor, Cline, Continue.dev, OpenCode, and OpenClaw. No pasting `OPENAI_BASE_URL` six times.
- **Response cache with per-key isolation + singleflight** (v3.13.0). Optional SHA-256 prompt cache, isolated per API key (cross-user pollution is impossible by hash construction, not by application logic), with stampede protection on concurrent identical prompts. Off by default. ([PR #65](https://github.com/dtzp555-max/ocp/pull/65), [PR #66](https://github.com/dtzp555-max/ocp/pull/66))
- **Per-key request quotas** (v3.8.0). Daily / weekly / monthly limits per key — set a kid's iPad to 20/day, a partner's laptop to 100/week. ([PR #18](https://github.com/dtzp555-max/ocp/pull/18))
- **SSE heartbeat for long reasoning** ([v3.12.0](https://github.com/dtzp555-max/ocp/releases/tag/v3.12.0), opt-in). If you've ever watched your IDE die at the 60s idle mark during a long Claude tool-use pause — that's nginx/Cloudflare default behavior. OCP emits an SSE comment frame to keep the connection alive without polluting the response. ([PR #49](https://github.com/dtzp555-max/ocp/pull/49))
- **`cli.js` alignment + CI guardrail.** LLM-assisted code drifts easily — it's tempting to invent plausible-looking endpoints that `cli.js` doesn't actually use. [`ALIGNMENT.md`](./ALIGNMENT.md) is binding: every endpoint OCP exposes must cite a `cli.js` line. The [`alignment.yml`](./.github/workflows/alignment.yml) CI workflow blocks PRs that introduce known-hallucinated tokens. The payoff is boring: your setup keeps working when `cli.js` ships its next minor.
- **`models.json` single source of truth** (v3.11.0). Adding a model is one file edit; both `/v1/models` and the OpenClaw bootstrap derive from it. ([PR #30](https://github.com/dtzp555-max/ocp/pull/30))
### Comparison
@@ -472,17 +477,20 @@ ocp settings cacheTTL 300000
```
**How it works:**
- Cache key = SHA-256 of `model` + `messages` + `temperature` + `max_tokens` + `top_p`
- Cache key = SHA-256 of `v2|<keyId or "anon">|model + messages + temperature + max_tokens + top_p`
- **Per-key isolation** — different API keys never share cache entries; anonymous callers share one `anon` pool
- Cache hits return instantly — no Claude CLI process spawned
- Works for both streaming and non-streaming requests
- **Streaming hits** are replayed as multiple SSE chunks (80 codepoints each), not one large delta — incremental render preserved
- **`cache_control` bypass** — if a request carries an Anthropic `cache_control` annotation (top-level or nested in `content[]`), OCP skips its own cache entirely so it doesn't interfere with Anthropic-side prompt caching
- **Singleflight stampede protection** — concurrent identical cache-miss requests share one upstream `cli.js` spawn; followers receive byte-identical responses to the leader's call. Non-streaming path only (streaming-path singleflight is a known TODO)
- Multi-turn conversations (with `session_id`) are never cached
- Expired entries are cleaned up automatically every 10 minutes
**Management:**
```bash
# View cache stats
# View cache stats (now includes singleflight in-flight counts)
curl http://127.0.0.1:3456/cache/stats
# → { "entries": 42, "totalHits": 156, "sizeBytes": 284000 }
# → { "entries": 42, "totalHits": 156, "sizeBytes": 284000, "inflight": 0, "requesters": 0 }
# Clear all cached responses
curl -X DELETE http://127.0.0.1:3456/cache
@@ -493,6 +501,8 @@ ocp settings cacheTTL 0
Cache is **disabled by default** (`CLAUDE_CACHE_TTL=0`). All data is stored locally in `~/.ocp/ocp.db`.
**Hash format upgrade in v3.13.0:** legacy `v1` cache rows from earlier versions don't match new `v2`-format lookups; they orphan and are reaped by the TTL cleanup interval within one window. No migration script required.
## How It Works
```
@@ -701,6 +711,24 @@ OCP runs under a small set of binding documents so contributions stay aligned wi
If you want to contribute: read `ALIGNMENT.md` first, search `cli.js` for the operation you're proposing, and cite the line number in your PR.
## Support OCP
OCP has been **open source from day one** — not a freemium tool, not a commercial product turned open, just open. It will stay that way forever. No paid tiers, no premium features, no "Pro" version locked behind a paywall.
I built it because my family and I needed it. We use OCP every day across our own machines and IDEs — keeping one Claude Pro/Max subscription powering everything, saving the per-token API cost we'd otherwise pay. It's been quietly heartwarming to hear from users online who say OCP has saved them money the same way it saves ours. That's the whole point.
Behind every version are hundreds of hours that don't show up in commits: building it from scratch, adding new features as the Claude Code ecosystem evolves, debugging across Mac / Windows / Linux machines, validating against half a dozen IDEs (Claude Code, Cursor, Cline, OpenCode, Aider, Continue.dev, OpenClaw), tracking down `cli.js` drift, OAuth refresh edge cases, SSE streaming quirks, concurrency leaks, and the occasional incident that turns into a multi-day investigation (the [2026-04-11 alignment drift](./docs/adr/0002-alignment-constitution.md), the [v3.11.1 concurrency leak](./CHANGELOG.md), the v3.12 SSE replay regression).
**The commitment**: this project will keep being updated, keep getting new features, and will stay open source as long as I'm able to maintain it.
**Please try it.** If something breaks or could be better, [open an issue](https://github.com/dtzp555-max/ocp/issues) — feedback is genuinely what keeps the project moving.
And if OCP saves you (or your team, or your family) real money and you'd like to chip in toward the next debugging session:
-**[Buy me a coffee](https://buymeacoffee.com/dtzp555)**
Donations directly fund the time it takes to keep OCP saving the community money.
## License
MIT — see [`LICENSE`](LICENSE).
+30
View File
@@ -371,6 +371,36 @@ export function getCacheStats() {
return { entries: total, totalHits, sizeBytes };
}
// ── Singleflight stampede protection ──
// In-memory singleflight Map: hash → { promise, requesters }
// Deduplicates concurrent identical cache-miss flows so only one upstream call runs.
// Per ADR 0005 / spec D4: in-process scope only (single Node process per host).
const inflightMap = new Map();
export function singleflight(hash, fn) {
const existing = inflightMap.get(hash);
if (existing) {
existing.requesters++;
return existing.promise;
}
// Wrap fn() in Promise.resolve().then() so synchronous throws don't escape.
const promise = Promise.resolve().then(fn).finally(() => {
inflightMap.delete(hash);
});
inflightMap.set(hash, { promise, requesters: 1 });
return promise;
}
export function getInflightStats() {
let totalRequesters = 0;
for (const entry of inflightMap.values()) totalRequesters += entry.requesters;
return {
inflight: inflightMap.size,
requesters: totalRequesters,
};
}
// Find a key by id or name (returns { id, name } or null)
export function findKey(idOrName) {
const d = getDb();
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "open-claude-proxy",
"version": "3.12.0",
"version": "3.13.0",
"description": "OCP (Open Claude Proxy) — use your Claude Pro/Max subscription as an OpenAI-compatible API for any IDE. Works with Cline, OpenCode, Aider, Continue.dev, OpenClaw, and more.",
"type": "module",
"bin": {
+39 -7
View File
@@ -34,7 +34,7 @@ import { readFileSync, accessSync, existsSync, constants } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { homedir } from "node:os";
import { validateKey, recordUsage, getUsageByKey, getUsageTimeline, getRecentUsage, createKey, listKeys, revokeKey, closeDb, checkQuota, updateKeyQuota, getKeyQuota, findKey, cacheHash, getCachedResponse, setCachedResponse, clearCache, getCacheStats, hasCacheControl } from "./keys.mjs";
import { validateKey, recordUsage, getUsageByKey, getUsageTimeline, getRecentUsage, createKey, listKeys, revokeKey, closeDb, checkQuota, updateKeyQuota, getKeyQuota, findKey, cacheHash, getCachedResponse, setCachedResponse, clearCache, getCacheStats, hasCacheControl, singleflight, getInflightStats } from "./keys.mjs";
const __dirname = dirname(fileURLToPath(import.meta.url));
const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8"));
@@ -590,6 +590,7 @@ function startHeartbeat(res, intervalMs, sessionId) {
// ── Call claude CLI (real streaming) ─────────────────────────────────────
// Pipes stdout from the claude process directly to SSE chunks as they arrive.
// Each data chunk becomes a proper SSE event with delta content in real time.
// TODO(cache-singleflight-stream): streaming-path singleflight is out of scope for v3.13.0; see spec D4 streaming caveat.
function callClaudeStreaming(model, messages, conversationId, res, authInfo = {}) {
const id = `chatcmpl-${randomUUID()}`;
const created = Math.floor(Date.now() / 1000);
@@ -1265,14 +1266,45 @@ async function handleChatCompletions(req, res) {
const t0Usage = Date.now();
const promptChars = messages.reduce((a, m) => a + (typeof m.content === "string" ? m.content.length : JSON.stringify(m.content).length), 0);
// Non-streaming path with stampede protection: wrap the upstream call in singleflight
// when cache is enabled and a hash is present. Concurrent identical requests share
// one upstream spawn; followers receive the same promise. Streaming-path dedup is
// explicitly out of scope (see TODO comment above callClaudeStreaming).
if (CACHE_TTL > 0 && req._cacheHash) {
try {
const content = await singleflight(req._cacheHash, async () => {
// Re-check cache inside the singleflight: a follower that enters before the
// leader finishes will wait on the shared promise (not reach here), but a
// request that races in just after the previous singleflight cleared the map
// will re-read the freshly-populated cache entry here rather than spawning.
const recheck = getCachedResponse(req._cacheHash, CACHE_TTL);
if (recheck) return recheck.response;
const c = await callClaude(model, messages, conversationId);
try { setCachedResponse(req._cacheHash, model, c); } catch (e) { logEvent("error", "cache_write_failed", { error: e.message }); }
return c;
});
const id = `chatcmpl-${randomUUID()}`;
completionResponse(res, id, model, content);
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: content.length, elapsedMs: Date.now() - t0Usage, success: true }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
return;
} catch (err) {
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: 0, elapsedMs: Date.now() - t0Usage, success: false }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
console.error(`[proxy] error: ${err.message}`);
if (res.headersSent || res.writableEnded || res.destroyed) {
try { res.end(); } catch {}
return;
}
const safeMessage = (err.message || "Internal error").replace(/\/[\w/.\-]+/g, "[path]");
return jsonResponse(res, 500, { error: { message: safeMessage, type: "proxy_error" } });
}
}
// Fallback: cache disabled (CACHE_TTL=0) or no _cacheHash — original path untouched.
try {
const content = await callClaude(model, messages, conversationId);
const id = `chatcmpl-${randomUUID()}`;
completionResponse(res, id, model, content);
// Write to cache
if (CACHE_TTL > 0 && req._cacheHash) {
try { setCachedResponse(req._cacheHash, model, content); } catch (e) { logEvent("error", "cache_write_failed", { error: e.message }); }
}
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: content.length, elapsedMs: Date.now() - t0Usage, success: true }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
} catch (err) {
try { recordUsage({ keyId: req._authKeyId, keyName: req._authKeyName, model, promptChars, responseChars: 0, elapsedMs: Date.now() - t0Usage, success: false }); } catch (e) { logEvent("error", "usage_record_failed", { error: e.message }); }
@@ -1551,10 +1583,10 @@ const server = createServer(async (req, res) => {
});
}
// GET /cache/stats — cache statistics
// GET /cache/stats — cache statistics (entries, hits, size, inflight singleflight count)
if (pathname === "/cache/stats" && req.method === "GET") {
if (!isAdmin) return jsonResponse(res, 403, { error: "Admin access required" });
return jsonResponse(res, 200, getCacheStats());
return jsonResponse(res, 200, { ...getCacheStats(), ...getInflightStats() });
}
// DELETE /cache — clear cache
+48 -7
View File
@@ -264,13 +264,6 @@ console.log(`
╚══════════════════════════════════════════════════════════════╝
`);
// ── Step 6: Optionally start ────────────────────────────────────────────
if (!SKIP_START && !DRY_RUN) {
try {
execSync(`bash "${startPath}"`, { stdio: "inherit" });
} catch { /* ignore */ }
}
// ── Step 7: Install auto-start on boot ──────────────────────────────────
if (!DRY_RUN) {
console.log("\n🔄 Installing auto-start on login...\n");
@@ -405,4 +398,52 @@ WantedBy=default.target
}
console.log("\n✅ Auto-start installed — proxy will start automatically on login\n");
// ── Step 8: Post-install health verification ───────────────────────────
if (!SKIP_START) {
console.log("⏳ Waiting for server to bind...\n");
await new Promise(r => setTimeout(r, 3000));
const healthUrl = `http://127.0.0.1:${PORT}/health`;
let verified = false;
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 5000);
const res = await fetch(healthUrl, { signal: controller.signal });
clearTimeout(timer);
if (res.ok) {
const body = await res.json().catch(() => ({}));
console.log(` ✓ Health check passed (${healthUrl})`);
console.log(` version: ${body.version ?? "unknown"}`);
console.log(` authMode: ${body.authMode ?? "unknown"}`);
// Verify bind socket
try {
const bindCheck = process.platform === "linux"
? execSync(`ss -tlnp 2>/dev/null | grep ':${PORT}'`, { encoding: "utf-8" }).trim()
: execSync(`lsof -nP -iTCP:${PORT} -sTCP:LISTEN 2>/dev/null`, { encoding: "utf-8" }).trim();
if (bindCheck) {
console.log(` bind: ${bindCheck.split("\n")[0]}`);
}
} catch { /* bind check is best-effort */ }
verified = true;
} else {
warn(`Health check returned HTTP ${res.status} — service may not have started cleanly`);
}
} catch (e) {
const isTimeout = e.name === "AbortError" || (e.cause && e.cause.code === "UND_ERR_CONNECT_TIMEOUT");
warn(`Health check failed: ${isTimeout ? "timeout (5s)" : e.message}`);
}
if (!verified) {
const logHint = process.platform === "linux"
? "journalctl --user -u ocp-proxy -n 50"
: `tail -n 100 ~/.ocp/logs/proxy.log`;
console.error(`\n ✗ Server did not respond on port ${PORT} within 5 seconds.`);
console.error(` Check service logs:\n ${logHint}\n`);
process.exit(1);
}
}
}
+101 -1
View File
@@ -3,7 +3,7 @@
* Integration test for Quota + Cache features.
* Tests database layer functions directly — no server needed.
*/
import { getDb, createKey, listKeys, validateKey, recordUsage, checkQuota, updateKeyQuota, getKeyQuota, findKey, cacheHash, getCachedResponse, setCachedResponse, clearCache, getCacheStats, closeDb, hasCacheControl } from "./keys.mjs";
import { getDb, createKey, listKeys, validateKey, recordUsage, checkQuota, updateKeyQuota, getKeyQuota, findKey, cacheHash, getCachedResponse, setCachedResponse, clearCache, getCacheStats, closeDb, hasCacheControl, singleflight, getInflightStats } from "./keys.mjs";
import { createHash } from "node:crypto";
import { strict as assert } from "node:assert";
import { unlinkSync } from "node:fs";
@@ -354,6 +354,106 @@ test("D3: chunked replay uses Array.from — multibyte codepoints stay intact",
}
});
// ── PR-B Singleflight tests (async) ──
async function asyncTest(name, fn) {
try {
await fn();
passed++;
console.log(`${name}`);
} catch (e) {
failed++;
console.log(`${name}: ${e.message}`);
}
}
async function runSingleflightTests() {
console.log("\nPR-B Singleflight:");
// 1. Basic dedup: 10 concurrent calls with same hash execute fn only once.
await asyncTest("basic dedup: 10 concurrent callers execute fn only once", async () => {
let callCount = 0;
const fn = () => new Promise(resolve => {
callCount++;
setTimeout(() => resolve("result-A"), 20);
});
const results = await Promise.all(Array.from({ length: 10 }, () => singleflight("sf-dedup-1", fn)));
assert.equal(callCount, 1, `fn called ${callCount} times, expected 1`);
assert.ok(results.every(r => r === "result-A"), "all 10 callers should receive the same return value");
});
// 2. Failure fan-out: all followers reject when leader rejects.
await asyncTest("failure fan-out: all followers reject with leader error", async () => {
let callCount = 0;
const fn = () => new Promise((_, reject) => {
callCount++;
setTimeout(() => reject(new Error("upstream-fail")), 20);
});
const promises = Array.from({ length: 10 }, () => singleflight("sf-fail-1", fn));
const results = await Promise.allSettled(promises);
assert.equal(callCount, 1, `fn called ${callCount} times, expected 1`);
assert.ok(results.every(r => r.status === "rejected"), "all 10 should be rejected");
assert.ok(results.every(r => r.reason?.message === "upstream-fail"), "all should share the same error message");
});
// 3a. Map cleanup after success: inflight count returns to 0 after promise resolves.
await asyncTest("map cleanup after success: inflight=0 after promise settles", async () => {
const fn = () => new Promise(resolve => setTimeout(() => resolve("done"), 10));
await singleflight("sf-cleanup-ok", fn);
const stats = getInflightStats();
assert.equal(stats.inflight, 0, `expected inflight=0 after settlement, got ${stats.inflight}`);
});
// 3b. Map cleanup after failure: inflight count returns to 0 after promise rejects.
await asyncTest("map cleanup after failure: inflight=0 after promise rejects", async () => {
const fn = () => new Promise((_, reject) => setTimeout(() => reject(new Error("fail")), 10));
try { await singleflight("sf-cleanup-fail", fn); } catch {}
const stats = getInflightStats();
assert.equal(stats.inflight, 0, `expected inflight=0 after rejection, got ${stats.inflight}`);
});
// 4. Different hashes don't share: two parallel calls with distinct hashes both execute.
await asyncTest("different hashes do not share a singleflight entry", async () => {
let countA = 0;
let countB = 0;
const fnA = () => new Promise(resolve => { countA++; setTimeout(() => resolve("A"), 20); });
const fnB = () => new Promise(resolve => { countB++; setTimeout(() => resolve("B"), 20); });
const [rA, rB] = await Promise.all([singleflight("sf-hash-A", fnA), singleflight("sf-hash-B", fnB)]);
assert.equal(countA, 1);
assert.equal(countB, 1);
assert.equal(rA, "A");
assert.equal(rB, "B");
});
// 5. getInflightStats shape: returns { inflight: number, requesters: number }.
await asyncTest("getInflightStats returns correct shape", async () => {
// Verify shape against a settled state (inflight=0 is still the right shape).
const stats = getInflightStats();
assert.equal(typeof stats.inflight, "number", "inflight should be a number");
assert.equal(typeof stats.requesters, "number", "requesters should be a number");
// Also verify live counts: start a pending fn, check inflight>0, then resolve.
const { promise: blocker, resolve: resolveBlocker } = Promise.withResolvers();
const fn = () => blocker;
const p = singleflight("sf-stats-shape", fn);
const liveStats = getInflightStats();
assert.ok(liveStats.inflight >= 1, `expected inflight>=1, got ${liveStats.inflight}`);
resolveBlocker("ok");
await p;
});
// 6. Sequential calls don't share: singleflight is for concurrent dedup only.
await asyncTest("sequential calls with same hash each execute fn independently", async () => {
let callCount = 0;
const fn = () => new Promise(resolve => { callCount++; setTimeout(() => resolve(callCount), 10); });
const r1 = await singleflight("sf-sequential", fn);
const r2 = await singleflight("sf-sequential", fn);
assert.equal(callCount, 2, `fn should have been called twice, got ${callCount}`);
assert.equal(r1, 1);
assert.equal(r2, 2);
});
}
await runSingleflightTests();
// ── Cleanup ──
closeDb();