Compare commits

...
3 Commits
Author SHA1 Message Date
taodengandClaude Opus 4.6 7434f6adc0 docs: v2.5.0 release notes — sliding-window circuit breaker incident writeup
Document the 2026-03-22 multi-agent cascading timeout incident, root cause
analysis, and all new/changed defaults. Update env vars table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 19:48:23 +10:00
taodengandClaude Opus 4.6 8a7951134e feat: v2.5.0 — sliding-window circuit breaker, graduated backoff, increased timeouts
Emergency fix for multi-agent (ClawTeam) burst scenario that caused cascading
failures: when multiple Opus agents ran concurrently, the old consecutive-count
breaker (threshold=3) tripped within seconds, blocking ALL subsequent requests
globally — including non-ClawTeam agents and new sessions.

Changes:
- Circuit breaker now uses a sliding time window (default 5min) instead of
  consecutive failure count. 6 failures in 5min triggers open, vs old 3-in-a-row.
- Half-open state allows 2 concurrent probe requests (was 1), improving recovery
  speed when API comes back.
- Graduated backoff: cooldown doubles on each re-open (120s→240s→300s cap),
  resets fully on first success.
- Increased default timeouts: Opus first-byte 60s→90s, Sonnet 45s→60s,
  overall 120s→300s, max concurrent 5→8.
- /health endpoint now exposes per-model breaker state, window stats, and
  marks status as "degraded" when any breaker is open.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 19:47:23 +10:00
47434a8ba7 fix: security hardening + real streaming (#4)
* Add graceful shutdown handling for SIGTERM and SIGINT

On receiving SIGTERM or SIGINT, the server now:
1. Stops accepting new connections (server.close)
2. Clears session cleanup and auth check intervals
3. Sends SIGTERM to all active child processes
4. Waits up to 5s for processes to exit, then SIGKILL + exit(1)
5. Exits cleanly with exit(0) once all processes are gone

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: security hardening — bind localhost, validate models, limit body size

- Bind to 127.0.0.1 instead of 0.0.0.0 to prevent network exposure
- Restrict CORS Access-Control-Allow-Origin to http://127.0.0.1
- Add 10MB request body size limit (413 on exceed)
- Validate model parameter against known MODEL_MAP keys (400 on unknown)
- Remove catch-all POST route; only /v1/chat/completions accepted
- Use crypto.timingSafeEqual for API key comparison
- Sanitize error messages to strip internal file paths

Bumps version to 2.4.1.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace fake streaming with real stdout-piped SSE streaming

Previously, streaming requests waited for the entire claude CLI process
to complete, then split the output into artificial 500-char chunks sent
as SSE events. This defeated the purpose of streaming and added latency.

Now, stdout from the claude child process is piped directly to SSE
chunks as data arrives. Each `data` event from the process stdout
becomes an immediate SSE delta event, giving clients real incremental
output. The shared process setup logic is extracted into
spawnClaudeProcess() to avoid duplication between streaming and
non-streaming paths. Client disconnect detection kills the child process
to free resources.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: tighten CORS to dynamic localhost-only and reduce body limit to 5MB

- CORS now dynamically matches localhost/127.0.0.1 origins instead of
  static http://127.0.0.1, preventing cross-origin abuse while
  supporting any localhost port
- Body size limit reduced from 10MB to 5MB to limit OOM risk

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Oracle Public Cloud User <opc@instance-20230820-1333.subnet07301351.vcn07301351.oraclevcn.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 18:46:36 +10:00
3 changed files with 555 additions and 235 deletions
+41 -2
View File
@@ -4,6 +4,40 @@
A lightweight, zero-dependency proxy that lets [OpenClaw](https://github.com/openclaw/openclaw) agents talk to Claude through your existing subscription. One command to set up, one file to run. A lightweight, zero-dependency proxy that lets [OpenClaw](https://github.com/openclaw/openclaw) agents talk to Claude through your existing subscription. One command to set up, one file to run.
## v2.5.0 — Emergency Fix: Sliding-Window Circuit Breaker
**Incident (2026-03-22):** Multi-agent burst (ClawTeam with 5+ Opus agents) caused cascading timeout failure. The old consecutive-count circuit breaker (threshold=3) tripped within seconds, blocking ALL requests globally — including unrelated agents and new sessions. With fallback models removed, this resulted in complete service outage ("LLM request timed out." on every message).
**Root cause:** v2.4.0's circuit breaker counted consecutive failures per model. When ClawTeam spawned multiple concurrent Opus agents and Claude API had moderate latency, 3 quick timeouts opened the breaker for the entire model. With `fallbacks: []`, the gateway had no alternative path.
**What's new in v2.5.0:**
- **Sliding-window circuit breaker** — counts failures in a 5-minute window (default: 6 failures) instead of 3 consecutive. Multi-agent bursts no longer trip the breaker instantly.
- **Graduated backoff** — cooldown doubles on each re-open (120s → 240s → 300s cap), resets fully on first success. Prevents oscillation between open/half-open during extended API issues.
- **Multi-probe half-open** — allows 2 concurrent probe requests in half-open state (was 1), improving recovery speed.
- **Increased default timeouts** — Opus first-byte 60s→90s, Sonnet 45s→60s, overall 120s→300s, max concurrent 5→8. Designed for large agent system prompts (30K+ chars).
- **Health endpoint shows breaker state** — `/health` now exposes per-model breaker state (window failures, cooldown, reopen count). Status is "degraded" when any breaker is open.
**New env vars:**
| Variable | Default | Description |
|----------|---------|-------------|
| `CLAUDE_BREAKER_WINDOW` | `300000` | Sliding window duration (ms) for failure counting |
| `CLAUDE_BREAKER_HALF_OPEN_MAX` | `2` | Max concurrent probe requests in half-open state |
**Updated defaults:**
| Variable | Old Default | New Default |
|----------|-------------|-------------|
| `CLAUDE_TIMEOUT` | `120000` | `300000` |
| `CLAUDE_FIRST_BYTE_TIMEOUT` | `45000` | `90000` |
| `CLAUDE_MAX_CONCURRENT` | `5` | `8` |
| `CLAUDE_BREAKER_THRESHOLD` | `3` | `6` |
| `CLAUDE_BREAKER_COOLDOWN` | `60000` | `120000` |
**Upgrade:** Pull latest and restart the proxy. The new defaults take effect immediately. If you have custom env vars set in your plist/service file, review and adjust them.
---
## v2.0.0 — Major Upgrade ## v2.0.0 — Major Upgrade
**What's new:** **What's new:**
@@ -174,13 +208,18 @@ openclaw gateway restart
|----------|---------|-------------| |----------|---------|-------------|
| `CLAUDE_PROXY_PORT` | `3456` | Listen port | | `CLAUDE_PROXY_PORT` | `3456` | Listen port |
| `CLAUDE_BIN` | *(auto-detect)* | Path to claude binary | | `CLAUDE_BIN` | *(auto-detect)* | Path to claude binary |
| `CLAUDE_TIMEOUT` | `300000` | Request timeout (ms) | | `CLAUDE_TIMEOUT` | `300000` | Overall request timeout (ms) |
| `CLAUDE_FIRST_BYTE_TIMEOUT` | `90000` | Base first-byte timeout (ms), adaptive by model tier |
| `CLAUDE_ALLOWED_TOOLS` | `Bash,Read,...,Agent` | Comma-separated tools to pre-approve | | `CLAUDE_ALLOWED_TOOLS` | `Bash,Read,...,Agent` | Comma-separated tools to pre-approve |
| `CLAUDE_SKIP_PERMISSIONS` | `false` | Set `true` to bypass all permission checks | | `CLAUDE_SKIP_PERMISSIONS` | `false` | Set `true` to bypass all permission checks |
| `CLAUDE_SYSTEM_PROMPT` | *(empty)* | System prompt appended to all requests | | `CLAUDE_SYSTEM_PROMPT` | *(empty)* | System prompt appended to all requests |
| `CLAUDE_MCP_CONFIG` | *(empty)* | Path to MCP server config JSON file | | `CLAUDE_MCP_CONFIG` | *(empty)* | Path to MCP server config JSON file |
| `CLAUDE_SESSION_TTL` | `3600000` | Session expiry in ms (default: 1 hour) | | `CLAUDE_SESSION_TTL` | `3600000` | Session expiry in ms (default: 1 hour) |
| `CLAUDE_MAX_CONCURRENT` | `5` | Max concurrent claude processes | | `CLAUDE_MAX_CONCURRENT` | `8` | Max concurrent claude processes |
| `CLAUDE_BREAKER_THRESHOLD` | `6` | Failures in window before circuit opens |
| `CLAUDE_BREAKER_COOLDOWN` | `120000` | Base cooldown (ms) before half-open (graduates on re-open) |
| `CLAUDE_BREAKER_WINDOW` | `300000` | Sliding window duration (ms) for failure counting |
| `CLAUDE_BREAKER_HALF_OPEN_MAX` | `2` | Max concurrent probe requests in half-open state |
| `PROXY_API_KEY` | *(unset)* | Bearer token for API authentication | | `PROXY_API_KEY` | *(unset)* | Bearer token for API authentication |
## API Endpoints ## API Endpoints
+2 -2
View File
@@ -1,7 +1,7 @@
{ {
"name": "openclaw-claude-proxy", "name": "openclaw-claude-proxy",
"version": "2.4.0", "version": "2.5.0",
"description": "OpenAI-compatible proxy for Claude CLI v2 — per-model circuit breaker, adaptive first-byte timeout, structured logging", "description": "OpenAI-compatible proxy for Claude CLI v2 — sliding-window circuit breaker, adaptive first-byte timeout, structured logging",
"type": "module", "type": "module",
"bin": { "bin": {
"openclaw-claude-proxy": "./server.mjs" "openclaw-claude-proxy": "./server.mjs"
+512 -231
View File
@@ -1,10 +1,19 @@
#!/usr/bin/env node #!/usr/bin/env node
/** /**
* openclaw-claude-proxy v2.4.0 — OpenAI-compatible proxy for Claude CLI * openclaw-claude-proxy v2.5.0 — OpenAI-compatible proxy for Claude CLI
* *
* Translates OpenAI chat/completions requests into `claude -p` CLI calls, * Translates OpenAI chat/completions requests into `claude -p` CLI calls,
* letting you use your Claude Pro/Max subscription as an OpenClaw model provider. * letting you use your Claude Pro/Max subscription as an OpenClaw model provider.
* *
* v2.5.0:
* - Sliding-window circuit breaker: uses time-windowed failure rate instead of
* consecutive-count, preventing multi-agent burst scenarios from tripping the
* breaker too aggressively. Half-open state allows configurable probe requests.
* - Graduated backoff: cooldown doubles on each re-open (capped at 5min),
* resets fully on success.
* - Health endpoint now exposes per-model breaker state and sliding window stats.
* - Increased default timeout tiers for Opus/Sonnet to handle large agent prompts.
*
* v2.4.0: * v2.4.0:
* - Per-model circuit breaker: consecutive timeouts temporarily mark a model as degraded * - Per-model circuit breaker: consecutive timeouts temporarily mark a model as degraded
* - Adaptive first-byte timeout: scales by model tier + prompt size * - Adaptive first-byte timeout: scales by model tier + prompt size
@@ -12,23 +21,25 @@
* - On-demand spawning (no pool), session management, full tool access * - On-demand spawning (no pool), session management, full tool access
* *
* Env vars: * Env vars:
* CLAUDE_PROXY_PORT — listen port (default: 3456) * CLAUDE_PROXY_PORT — listen port (default: 3456)
* CLAUDE_BIN — path to claude binary (default: auto-detect) * CLAUDE_BIN — path to claude binary (default: auto-detect)
* CLAUDE_TIMEOUT — per-request timeout in ms (default: 120000) * CLAUDE_TIMEOUT — per-request timeout in ms (default: 300000)
* CLAUDE_FIRST_BYTE_TIMEOUT — base first-byte timeout in ms (default: 45000) * CLAUDE_FIRST_BYTE_TIMEOUT — base first-byte timeout in ms (default: 90000)
* CLAUDE_ALLOWED_TOOLS — comma-separated tools to allow (default: expanded set) * CLAUDE_ALLOWED_TOOLS — comma-separated tools to allow (default: expanded set)
* CLAUDE_SKIP_PERMISSIONS — "true" to bypass all permission checks (default: false) * CLAUDE_SKIP_PERMISSIONS — "true" to bypass all permission checks (default: false)
* CLAUDE_SYSTEM_PROMPT — system prompt appended to all requests * CLAUDE_SYSTEM_PROMPT — system prompt appended to all requests
* CLAUDE_MCP_CONFIG — path to MCP server config JSON file * CLAUDE_MCP_CONFIG — path to MCP server config JSON file
* CLAUDE_SESSION_TTL — session TTL in ms (default: 3600000 = 1h) * CLAUDE_SESSION_TTL — session TTL in ms (default: 3600000 = 1h)
* CLAUDE_MAX_CONCURRENT — max concurrent claude processes (default: 5) * CLAUDE_MAX_CONCURRENT — max concurrent claude processes (default: 8)
* CLAUDE_BREAKER_THRESHOLD — consecutive timeouts before circuit opens (default: 3) * CLAUDE_BREAKER_THRESHOLD — failures in window before circuit opens (default: 6)
* CLAUDE_BREAKER_COOLDOWN ms to wait before retrying after circuit opens (default: 60000) * CLAUDE_BREAKER_COOLDOWN — base ms to wait before retrying after circuit opens (default: 120000)
* PROXY_API_KEY — Bearer token for API auth (optional) * CLAUDE_BREAKER_WINDOW — sliding window duration in ms (default: 300000 = 5min)
* CLAUDE_BREAKER_HALF_OPEN_MAX — max concurrent probes in half-open state (default: 2)
* PROXY_API_KEY — Bearer token for API auth (optional)
*/ */
import { createServer } from "node:http"; import { createServer } from "node:http";
import { spawn, execFileSync } from "node:child_process"; import { spawn, execFileSync } from "node:child_process";
import { randomUUID } from "node:crypto"; import { randomUUID, timingSafeEqual } from "node:crypto";
import { readFileSync, accessSync, constants } from "node:fs"; import { readFileSync, accessSync, constants } from "node:fs";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path"; import { dirname, join } from "node:path";
@@ -76,8 +87,8 @@ function resolveClaude() {
// ── Configuration ─────────────────────────────────────────────────────── // ── Configuration ───────────────────────────────────────────────────────
const PORT = parseInt(process.env.CLAUDE_PROXY_PORT || "3456", 10); const PORT = parseInt(process.env.CLAUDE_PROXY_PORT || "3456", 10);
const CLAUDE = resolveClaude(); const CLAUDE = resolveClaude();
const TIMEOUT = parseInt(process.env.CLAUDE_TIMEOUT || "120000", 10); const TIMEOUT = parseInt(process.env.CLAUDE_TIMEOUT || "300000", 10);
const BASE_FIRST_BYTE_TIMEOUT = parseInt(process.env.CLAUDE_FIRST_BYTE_TIMEOUT || "45000", 10); const BASE_FIRST_BYTE_TIMEOUT = parseInt(process.env.CLAUDE_FIRST_BYTE_TIMEOUT || "90000", 10);
const PROXY_API_KEY = process.env.PROXY_API_KEY || ""; const PROXY_API_KEY = process.env.PROXY_API_KEY || "";
const SKIP_PERMISSIONS = process.env.CLAUDE_SKIP_PERMISSIONS === "true"; const SKIP_PERMISSIONS = process.env.CLAUDE_SKIP_PERMISSIONS === "true";
const ALLOWED_TOOLS = (process.env.CLAUDE_ALLOWED_TOOLS || const ALLOWED_TOOLS = (process.env.CLAUDE_ALLOWED_TOOLS ||
@@ -86,9 +97,11 @@ const ALLOWED_TOOLS = (process.env.CLAUDE_ALLOWED_TOOLS ||
const SYSTEM_PROMPT = process.env.CLAUDE_SYSTEM_PROMPT || ""; const SYSTEM_PROMPT = process.env.CLAUDE_SYSTEM_PROMPT || "";
const MCP_CONFIG = process.env.CLAUDE_MCP_CONFIG || ""; const MCP_CONFIG = process.env.CLAUDE_MCP_CONFIG || "";
const SESSION_TTL = parseInt(process.env.CLAUDE_SESSION_TTL || "3600000", 10); const SESSION_TTL = parseInt(process.env.CLAUDE_SESSION_TTL || "3600000", 10);
const MAX_CONCURRENT = parseInt(process.env.CLAUDE_MAX_CONCURRENT || "5", 10); const MAX_CONCURRENT = parseInt(process.env.CLAUDE_MAX_CONCURRENT || "8", 10);
const BREAKER_THRESHOLD = parseInt(process.env.CLAUDE_BREAKER_THRESHOLD || "3", 10); const BREAKER_THRESHOLD = parseInt(process.env.CLAUDE_BREAKER_THRESHOLD || "6", 10);
const BREAKER_COOLDOWN = parseInt(process.env.CLAUDE_BREAKER_COOLDOWN || "60000", 10); const BREAKER_COOLDOWN = parseInt(process.env.CLAUDE_BREAKER_COOLDOWN || "120000", 10);
const BREAKER_WINDOW = parseInt(process.env.CLAUDE_BREAKER_WINDOW || "300000", 10);
const BREAKER_HALF_OPEN_MAX = parseInt(process.env.CLAUDE_BREAKER_HALF_OPEN_MAX || "2", 10);
const VERSION = _pkg.version; const VERSION = _pkg.version;
const START_TIME = Date.now(); const START_TIME = Date.now();
@@ -103,47 +116,128 @@ function logEvent(level, event, data = {}) {
} }
} }
// ── Per-model circuit breaker ─────────────────────────────────────────── // ── Per-model sliding-window circuit breaker ─────────────────────────────
// Tracks consecutive timeouts per model. When threshold is reached, the // Uses a time-windowed failure rate instead of consecutive-count. This prevents
// model is marked "open" (degraded) for BREAKER_COOLDOWN ms. During that // multi-agent burst scenarios (e.g. ClawTeam spawning 5+ Opus agents) from
// window, requests for this model fail fast with a clear error instead of // tripping the breaker after just 3 quick timeouts.
// waiting for yet another timeout that would block the gateway. //
const breakers = new Map(); // cliModel → { failures, state, openedAt } // States: closed → open → half-open → closed (on success) or open (on failure)
// Half-open allows up to BREAKER_HALF_OPEN_MAX concurrent probes (not just 1).
// Cooldown uses graduated backoff: doubles on each re-open, resets on success.
const breakers = new Map(); // cliModel → BreakerState
function newBreakerState() {
return {
state: "closed", // closed | open | half-open
failureTimestamps: [], // timestamps of failures within the sliding window
successCount: 0, // successes within window (for rate calculation)
openedAt: 0,
currentCooldown: BREAKER_COOLDOWN, // graduates on repeated opens
reopenCount: 0, // how many times breaker has re-opened without a full reset
halfOpenProbes: 0, // active probe requests in half-open state
};
}
function pruneWindow(b) {
const cutoff = Date.now() - BREAKER_WINDOW;
b.failureTimestamps = b.failureTimestamps.filter(ts => ts > cutoff);
}
function getBreakerState(cliModel) { function getBreakerState(cliModel) {
if (!breakers.has(cliModel)) { if (!breakers.has(cliModel)) {
breakers.set(cliModel, { failures: 0, state: "closed", openedAt: 0 }); breakers.set(cliModel, newBreakerState());
} }
const b = breakers.get(cliModel); const b = breakers.get(cliModel);
// Auto-recover: if cooldown has elapsed, transition to half-open // Auto-recover: if cooldown has elapsed, transition to half-open
if (b.state === "open" && Date.now() - b.openedAt >= BREAKER_COOLDOWN) { if (b.state === "open" && Date.now() - b.openedAt >= b.currentCooldown) {
b.state = "half-open"; b.state = "half-open";
logEvent("info", "breaker_half_open", { model: cliModel, cooldownMs: BREAKER_COOLDOWN }); b.halfOpenProbes = 0;
logEvent("info", "breaker_half_open", { model: cliModel, cooldownMs: b.currentCooldown, reopenCount: b.reopenCount });
} }
return b; return b;
} }
function breakerRecordSuccess(cliModel) { function breakerRecordSuccess(cliModel) {
const b = getBreakerState(cliModel); const b = getBreakerState(cliModel);
if (b.failures > 0 || b.state !== "closed") { b.successCount++;
logEvent("info", "breaker_reset", { model: cliModel, previousFailures: b.failures, previousState: b.state });
if (b.state === "half-open") {
b.halfOpenProbes = Math.max(0, b.halfOpenProbes - 1);
}
if (b.state !== "closed") {
logEvent("info", "breaker_reset", {
model: cliModel,
previousFailures: b.failureTimestamps.length,
previousState: b.state,
reopenCount: b.reopenCount,
});
// Full reset on success — graduated backoff resets too
b.state = "closed";
b.openedAt = 0;
b.currentCooldown = BREAKER_COOLDOWN;
b.reopenCount = 0;
b.halfOpenProbes = 0;
b.failureTimestamps = [];
b.successCount = 0;
} }
b.failures = 0;
b.state = "closed";
b.openedAt = 0;
} }
function breakerRecordTimeout(cliModel) { function breakerRecordTimeout(cliModel) {
const b = getBreakerState(cliModel); const b = getBreakerState(cliModel);
b.failures++; const now = Date.now();
logEvent("warn", "breaker_failure", { model: cliModel, consecutiveFailures: b.failures, threshold: BREAKER_THRESHOLD }); b.failureTimestamps.push(now);
pruneWindow(b);
if (b.failures >= BREAKER_THRESHOLD && b.state !== "open") { if (b.state === "half-open") {
b.state = "open"; b.halfOpenProbes = Math.max(0, b.halfOpenProbes - 1);
b.openedAt = Date.now();
logEvent("error", "breaker_open", { model: cliModel, failures: b.failures, cooldownMs: BREAKER_COOLDOWN });
} }
const windowFailures = b.failureTimestamps.length;
logEvent("warn", "breaker_failure", {
model: cliModel,
windowFailures,
threshold: BREAKER_THRESHOLD,
windowMs: BREAKER_WINDOW,
state: b.state,
});
if (windowFailures >= BREAKER_THRESHOLD && b.state !== "open") {
b.state = "open";
b.openedAt = now;
b.halfOpenProbes = 0;
// Graduated backoff: double cooldown on each re-open, cap at 5 min
if (b.reopenCount > 0) {
b.currentCooldown = Math.min(b.currentCooldown * 2, 300000);
}
b.reopenCount++;
logEvent("error", "breaker_open", {
model: cliModel,
windowFailures,
cooldownMs: b.currentCooldown,
reopenCount: b.reopenCount,
});
}
}
// Expose breaker snapshot for /health endpoint
function getBreakerSnapshot() {
const snapshot = {};
for (const [model, b] of breakers) {
pruneWindow(b);
snapshot[model] = {
state: b.state,
windowFailures: b.failureTimestamps.length,
threshold: BREAKER_THRESHOLD,
windowMs: BREAKER_WINDOW,
currentCooldown: b.currentCooldown,
reopenCount: b.reopenCount,
halfOpenProbes: b.halfOpenProbes,
...(b.openedAt ? { openedAt: new Date(b.openedAt).toISOString() } : {}),
};
}
return snapshot;
} }
// ── Model mapping ─────────────────────────────────────────────────────── // ── Model mapping ───────────────────────────────────────────────────────
@@ -171,7 +265,7 @@ const MODELS = [
// Enables --resume for multi-turn conversations, reducing token waste. // Enables --resume for multi-turn conversations, reducing token waste.
const sessions = new Map(); // conversationId → { uuid, messageCount, lastUsed, model } const sessions = new Map(); // conversationId → { uuid, messageCount, lastUsed, model }
setInterval(() => { const sessionCleanupInterval = setInterval(() => {
const now = Date.now(); const now = Date.now();
for (const [id, s] of sessions) { for (const [id, s] of sessions) {
if (now - s.lastUsed > SESSION_TTL) { if (now - s.lastUsed > SESSION_TTL) {
@@ -181,6 +275,9 @@ setInterval(() => {
} }
}, 60000); }, 60000);
// ── Active child process tracking ────────────────────────────────────────
const activeProcesses = new Set();
// ── Stats & diagnostics ───────────────────────────────────────────────── // ── Stats & diagnostics ─────────────────────────────────────────────────
const stats = { const stats = {
totalRequests: 0, totalRequests: 0,
@@ -220,7 +317,7 @@ async function checkAuth() {
// Check auth on start and every 10 minutes // Check auth on start and every 10 minutes
checkAuth(); checkAuth();
setInterval(checkAuth, 600000); const authCheckInterval = setInterval(checkAuth, 600000);
// ── Build CLI arguments ───────────────────────────────────────────────── // ── Build CLI arguments ─────────────────────────────────────────────────
function buildCliArgs(cliModel, sessionInfo) { function buildCliArgs(cliModel, sessionInfo) {
@@ -268,8 +365,8 @@ function messagesToPrompt(messages) {
// Model tier multipliers for first-byte timeout. // Model tier multipliers for first-byte timeout.
// Opus is much slower to produce first token, especially with large contexts. // Opus is much slower to produce first token, especially with large contexts.
const MODEL_TIMEOUT_TIERS = { const MODEL_TIMEOUT_TIERS = {
"opus": { base: 60000, perPromptChar: 0.00015 }, // 60s base + ~15s per 100k chars "opus": { base: 90000, perPromptChar: 0.00020 }, // 90s base + ~20s per 100k chars
"sonnet": { base: 45000, perPromptChar: 0.00008 }, // 45s base + ~8s per 100k chars "sonnet": { base: 60000, perPromptChar: 0.00010 }, // 60s base + ~10s per 100k chars
"haiku": { base: 30000, perPromptChar: 0.00005 }, // 30s base + ~5s per 100k chars "haiku": { base: 30000, perPromptChar: 0.00005 }, // 30s base + ~5s per 100k chars
}; };
@@ -285,162 +382,293 @@ function computeFirstByteTimeout(cliModel, promptLength) {
return Math.min(timeout, Math.max(TIMEOUT - 5000, 10000)); return Math.min(timeout, Math.max(TIMEOUT - 5000, 10000));
} }
// ── Call claude CLI ───────────────────────────────────────────────────── // ── Spawn claude CLI (shared setup) ─────────────────────────────────────
// On-demand spawning: each request spawns a fresh `claude -p` process. // Resolves session logic, builds CLI args, spawns the process, and sets up
// No pool = no crash loops, no stale workers, no degraded states. // timeouts. Returns context object or throws synchronously.
// Stdin is written immediately so there's no 3s stdin timeout issue. function spawnClaudeProcess(model, messages, conversationId) {
function callClaude(model, messages, conversationId) { if (stats.activeRequests >= MAX_CONCURRENT) {
return new Promise((resolve, reject) => { throw new Error(`concurrency limit reached (${stats.activeRequests}/${MAX_CONCURRENT})`);
if (stats.activeRequests >= MAX_CONCURRENT) { }
return reject(new Error(`concurrency limit reached (${stats.activeRequests}/${MAX_CONCURRENT})`));
const cliModel = MODEL_MAP[model] || model;
// Circuit breaker check: fail fast if model is in open state
const breaker = getBreakerState(cliModel);
if (breaker.state === "open") {
const remainingMs = breaker.currentCooldown - (Date.now() - breaker.openedAt);
logEvent("warn", "breaker_rejected", { model: cliModel, remainingCooldownMs: remainingMs, reopenCount: breaker.reopenCount });
throw new Error(`circuit breaker open for ${cliModel}: ${breaker.failureTimestamps.length} timeouts in window, retry in ${Math.ceil(remainingMs / 1000)}s`);
}
// Half-open: allow limited probe requests
if (breaker.state === "half-open" && breaker.halfOpenProbes >= BREAKER_HALF_OPEN_MAX) {
logEvent("warn", "breaker_half_open_full", { model: cliModel, activeProbes: breaker.halfOpenProbes, max: BREAKER_HALF_OPEN_MAX });
throw new Error(`circuit breaker half-open for ${cliModel}: ${breaker.halfOpenProbes}/${BREAKER_HALF_OPEN_MAX} probe slots in use, wait for probe result`);
}
if (breaker.state === "half-open") {
breaker.halfOpenProbes++;
logEvent("info", "breaker_probe", { model: cliModel, activeProbes: breaker.halfOpenProbes, max: BREAKER_HALF_OPEN_MAX });
}
stats.activeRequests++;
stats.totalRequests++;
let sessionInfo = null;
let prompt;
// ── Session logic ──
if (conversationId && sessions.has(conversationId)) {
const session = sessions.get(conversationId);
session.lastUsed = Date.now();
sessionInfo = { uuid: session.uuid, resume: true };
stats.sessionHits++;
const lastUserMsg = [...messages].reverse().find((m) => m.role === "user");
prompt = lastUserMsg
? (typeof lastUserMsg.content === "string" ? lastUserMsg.content : JSON.stringify(lastUserMsg.content))
: "";
session.messageCount = messages.length;
console.log(`[session] resume conv=${conversationId.slice(0, 12)}... uuid=${session.uuid.slice(0, 8)}... msgs=${messages.length} prompt_chars=${prompt.length}`);
} else if (conversationId) {
const uuid = randomUUID();
sessions.set(conversationId, { uuid, messageCount: messages.length, lastUsed: Date.now(), model: cliModel });
sessionInfo = { uuid, resume: false };
stats.sessionMisses++;
prompt = messagesToPrompt(messages);
console.log(`[session] new conv=${conversationId.slice(0, 12)}... uuid=${uuid.slice(0, 8)}... msgs=${messages.length}`);
} else {
stats.oneOffRequests++;
prompt = messagesToPrompt(messages);
}
const cliArgs = buildCliArgs(cliModel, sessionInfo);
const env = { ...process.env };
delete env.CLAUDECODE;
delete env.ANTHROPIC_API_KEY;
delete env.ANTHROPIC_BASE_URL;
delete env.ANTHROPIC_AUTH_TOKEN;
const proc = spawn(CLAUDE, cliArgs, { env, stdio: ["pipe", "pipe", "pipe"] });
activeProcesses.add(proc);
const t0 = Date.now();
const firstByteTimeoutMs = computeFirstByteTimeout(cliModel, prompt.length);
let gotFirstByte = false;
let cleaned = false;
function cleanup() {
if (cleaned) return;
cleaned = true;
clearTimeout(overallTimer);
clearTimeout(firstByteTimer);
stats.activeRequests--;
}
function handleSessionFailure() {
if (sessionInfo?.resume && conversationId) {
console.warn(`[session] resume failed for ${conversationId.slice(0, 12)}..., removing stale session`);
sessions.delete(conversationId);
} }
}
const cliModel = MODEL_MAP[model] || model; function markFirstByte() {
if (!gotFirstByte) {
// Circuit breaker check: fail fast if model is in open state gotFirstByte = true;
const breaker = getBreakerState(cliModel);
if (breaker.state === "open") {
const remainingMs = BREAKER_COOLDOWN - (Date.now() - breaker.openedAt);
logEvent("warn", "breaker_rejected", { model: cliModel, remainingCooldownMs: remainingMs });
return reject(new Error(`circuit breaker open for ${cliModel}: ${breaker.failures} consecutive timeouts, retry in ${Math.ceil(remainingMs / 1000)}s`));
}
stats.activeRequests++;
stats.totalRequests++;
let sessionInfo = null;
let prompt;
// ── Session logic ──
if (conversationId && sessions.has(conversationId)) {
// Resume existing session: only send the latest user message
const session = sessions.get(conversationId);
session.lastUsed = Date.now();
sessionInfo = { uuid: session.uuid, resume: true };
stats.sessionHits++;
const lastUserMsg = [...messages].reverse().find((m) => m.role === "user");
prompt = lastUserMsg
? (typeof lastUserMsg.content === "string" ? lastUserMsg.content : JSON.stringify(lastUserMsg.content))
: "";
session.messageCount = messages.length;
console.log(`[session] resume conv=${conversationId.slice(0, 12)}... uuid=${session.uuid.slice(0, 8)}... msgs=${messages.length} prompt_chars=${prompt.length}`);
} else if (conversationId) {
// New session: send all messages, persist session for future --resume
const uuid = randomUUID();
sessions.set(conversationId, { uuid, messageCount: messages.length, lastUsed: Date.now(), model: cliModel });
sessionInfo = { uuid, resume: false };
stats.sessionMisses++;
prompt = messagesToPrompt(messages);
console.log(`[session] new conv=${conversationId.slice(0, 12)}... uuid=${uuid.slice(0, 8)}... msgs=${messages.length}`);
} else {
// One-off request, no session
stats.oneOffRequests++;
prompt = messagesToPrompt(messages);
}
const cliArgs = buildCliArgs(cliModel, sessionInfo);
const env = { ...process.env };
delete env.CLAUDECODE;
delete env.ANTHROPIC_API_KEY;
delete env.ANTHROPIC_BASE_URL;
delete env.ANTHROPIC_AUTH_TOKEN;
const proc = spawn(CLAUDE, cliArgs, { env, stdio: ["pipe", "pipe", "pipe"] });
let stdout = "";
let stderr = "";
const t0 = Date.now();
const firstByteTimeoutMs = computeFirstByteTimeout(cliModel, prompt.length);
let settled = false;
let gotFirstByte = false;
function settle(err, result) {
if (settled) return;
settled = true;
clearTimeout(timer);
clearTimeout(firstByteTimer); clearTimeout(firstByteTimer);
stats.activeRequests--; console.log(`[claude] first-byte model=${cliModel} elapsed=${Date.now() - t0}ms`);
if (err) {
trackError(err.message || String(err));
// If session resume failed, remove session so next request starts fresh
if (sessionInfo?.resume && conversationId) {
console.warn(`[session] resume failed for ${conversationId.slice(0, 12)}..., removing stale session`);
sessions.delete(conversationId);
}
reject(err);
} else {
resolve(result);
}
} }
}
proc.stdout.on("data", (d) => { // Write prompt to stdin immediately
if (!gotFirstByte) { proc.stdin.write(prompt);
gotFirstByte = true; proc.stdin.end();
clearTimeout(firstByteTimer);
console.log(`[claude] first-byte model=${cliModel} elapsed=${Date.now() - t0}ms`);
}
stdout += d;
});
proc.stderr.on("data", (d) => (stderr += d));
proc.on("close", (code, signal) => { logEvent("info", "claude_spawned", { model: cliModel, promptChars: prompt.length, firstByteTimeout: firstByteTimeoutMs, tier: getModelTier(cliModel), session: conversationId ? conversationId.slice(0, 12) + "..." : "none" });
const elapsed = Date.now() - t0;
if (settled) {
logEvent("warn", "late_close", { model: cliModel, code, signal: signal || "none", elapsed });
return;
}
if (code !== 0) {
logEvent("error", "claude_exit", { model: cliModel, code, signal: signal || "none", elapsed, stderr: stderr.slice(0, 300) });
settle(new Error(stderr.slice(0, 300) || stdout.slice(0, 300) || `claude exit ${code}`));
} else {
breakerRecordSuccess(cliModel);
logEvent("info", "claude_ok", { model: cliModel, chars: stdout.length, elapsed, session: conversationId ? conversationId.slice(0, 12) + "..." : "none" });
settle(null, stdout.trim());
}
});
proc.on("error", (err) => { // First-byte timeout
console.error(`[claude] spawn error: ${err.message}`); const firstByteTimer = setTimeout(() => {
settle(err); if (!gotFirstByte && !cleaned) {
}); stats.timeouts++;
breakerRecordTimeout(cliModel);
logEvent("error", "first_byte_timeout", { model: cliModel, timeoutMs: firstByteTimeoutMs, promptChars: prompt.length });
try { proc.kill("SIGTERM"); } catch {}
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
}
}, firstByteTimeoutMs);
// Write prompt to stdin immediately — no idle timeout issue // Overall request timeout
proc.stdin.write(prompt); const overallTimer = setTimeout(() => {
proc.stdin.end(); if (!cleaned) {
logEvent("info", "claude_spawned", { model: cliModel, promptChars: prompt.length, firstByteTimeout: firstByteTimeoutMs, tier: getModelTier(cliModel), session: conversationId ? conversationId.slice(0, 12) + "..." : "none" });
// First-byte timeout: abort early if Claude CLI produces no output
const firstByteTimer = setTimeout(() => {
if (!gotFirstByte && !settled) {
stats.timeouts++;
breakerRecordTimeout(cliModel);
logEvent("error", "first_byte_timeout", { model: cliModel, timeoutMs: firstByteTimeoutMs, promptChars: prompt.length });
try { proc.kill("SIGTERM"); } catch {}
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
settle(new Error(`first-byte timeout after ${firstByteTimeoutMs}ms`));
}
}, firstByteTimeoutMs);
// Overall request timeout with graceful kill
const timer = setTimeout(() => {
if (settled) return;
stats.timeouts++; stats.timeouts++;
breakerRecordTimeout(cliModel); breakerRecordTimeout(cliModel);
logEvent("error", "request_timeout", { model: cliModel, timeoutMs: TIMEOUT }); logEvent("error", "request_timeout", { model: cliModel, timeoutMs: TIMEOUT });
try { proc.kill("SIGTERM"); } catch {} try { proc.kill("SIGTERM"); } catch {}
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000); setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
settle(new Error(`timeout after ${TIMEOUT}ms`)); }
}, TIMEOUT); }, TIMEOUT);
return { proc, cliModel, conversationId, t0, cleanup, handleSessionFailure, markFirstByte };
}
// ── Call claude CLI (non-streaming) ─────────────────────────────────────
// On-demand spawning: each request spawns a fresh `claude -p` process.
// No pool = no crash loops, no stale workers, no degraded states.
// Stdin is written immediately so there's no 3s stdin timeout issue.
function callClaude(model, messages, conversationId) {
return new Promise((resolve, reject) => {
let ctx;
try {
ctx = spawnClaudeProcess(model, messages, conversationId);
} catch (err) {
return reject(err);
}
const { proc, cliModel, conversationId: convId, t0, cleanup, handleSessionFailure, markFirstByte } = ctx;
let stdout = "";
let stderr = "";
proc.stdout.on("data", (d) => {
markFirstByte();
stdout += d;
});
proc.stderr.on("data", (d) => (stderr += d));
proc.on("close", (code, signal) => {
activeProcesses.delete(proc);
const elapsed = Date.now() - t0;
cleanup();
if (code !== 0) {
logEvent("error", "claude_exit", { model: cliModel, code, signal: signal || "none", elapsed, stderr: stderr.slice(0, 300) });
trackError(stderr.slice(0, 300) || stdout.slice(0, 300) || `claude exit ${code}`);
handleSessionFailure();
reject(new Error(stderr.slice(0, 300) || stdout.slice(0, 300) || `claude exit ${code}`));
} else {
breakerRecordSuccess(cliModel);
logEvent("info", "claude_ok", { model: cliModel, chars: stdout.length, elapsed, session: convId ? convId.slice(0, 12) + "..." : "none" });
resolve(stdout.trim());
}
});
proc.on("error", (err) => {
console.error(`[claude] spawn error: ${err.message}`);
cleanup();
trackError(err.message);
handleSessionFailure();
reject(err);
});
});
}
// ── Call claude CLI (real streaming) ─────────────────────────────────────
// Pipes stdout from the claude process directly to SSE chunks as they arrive.
// Each data chunk becomes a proper SSE event with delta content in real time.
function callClaudeStreaming(model, messages, conversationId, res) {
const id = `chatcmpl-${randomUUID()}`;
const created = Math.floor(Date.now() / 1000);
let ctx;
try {
ctx = spawnClaudeProcess(model, messages, conversationId);
} catch (err) {
return jsonResponse(res, 500, { error: { message: err.message, type: "proxy_error" } });
}
const { proc, cliModel, conversationId: convId, t0, cleanup, handleSessionFailure, markFirstByte } = ctx;
let stderr = "";
let headersSent = false;
let totalChars = 0;
function ensureHeaders() {
if (headersSent || res.writableEnded || res.destroyed) return false;
headersSent = true;
res.writeHead(200, {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
});
// Send initial role chunk
sendSSE(res, {
id, object: "chat.completion.chunk", created, model,
choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }],
});
return true;
}
proc.stdout.on("data", (d) => {
markFirstByte();
const text = d.toString();
totalChars += text.length;
if (!ensureHeaders()) return;
// Stream each chunk as it arrives from the CLI process
sendSSE(res, {
id, object: "chat.completion.chunk", created, model,
choices: [{ index: 0, delta: { content: text }, finish_reason: null }],
});
});
proc.stderr.on("data", (d) => (stderr += d));
proc.on("close", (code, signal) => {
activeProcesses.delete(proc);
cleanup();
const elapsed = Date.now() - t0;
if (code !== 0) {
logEvent("error", "claude_exit", { model: cliModel, code, signal: signal || "none", elapsed, stderr: stderr.slice(0, 300) });
trackError(stderr.slice(0, 300) || `claude exit ${code}`);
handleSessionFailure();
if (!headersSent && !res.writableEnded && !res.destroyed) {
// No output was sent yet — return a JSON error
jsonResponse(res, 500, { error: { message: stderr.slice(0, 300) || `claude exit ${code}`, type: "proxy_error" } });
} else if (!res.writableEnded && !res.destroyed) {
// Already streaming — close the stream gracefully
sendSSE(res, {
id, object: "chat.completion.chunk", created, model,
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
});
res.write("data: [DONE]\n\n");
res.end();
}
} else {
breakerRecordSuccess(cliModel);
logEvent("info", "claude_ok", { model: cliModel, chars: totalChars, elapsed, session: convId ? convId.slice(0, 12) + "..." : "none" });
if (!headersSent) ensureHeaders();
if (!res.writableEnded && !res.destroyed) {
sendSSE(res, {
id, object: "chat.completion.chunk", created, model,
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
});
res.write("data: [DONE]\n\n");
res.end();
}
}
});
proc.on("error", (err) => {
console.error(`[claude] spawn error: ${err.message}`);
cleanup();
trackError(err.message);
handleSessionFailure();
if (!headersSent && !res.writableEnded && !res.destroyed) {
jsonResponse(res, 500, { error: { message: err.message, type: "proxy_error" } });
} else if (!res.writableEnded && !res.destroyed) {
res.end();
}
});
// If client disconnects, kill the process to free resources
res.on("close", () => {
if (!proc.killed) {
try { proc.kill("SIGTERM"); } catch {}
}
}); });
} }
@@ -455,32 +683,6 @@ function sendSSE(res, data) {
res.write(`data: ${JSON.stringify(data)}\n\n`); res.write(`data: ${JSON.stringify(data)}\n\n`);
} }
function streamResponse(res, id, model, content) {
if (res.headersSent || res.writableEnded || res.destroyed) return;
res.writeHead(200, {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
});
const created = Math.floor(Date.now() / 1000);
sendSSE(res, {
id, object: "chat.completion.chunk", created, model,
choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }],
});
for (let i = 0; i < content.length; i += 500) {
sendSSE(res, {
id, object: "chat.completion.chunk", created, model,
choices: [{ index: 0, delta: { content: content.slice(i, i + 500) }, finish_reason: null }],
});
}
sendSSE(res, {
id, object: "chat.completion.chunk", created, model,
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
});
res.write("data: [DONE]\n\n");
res.end();
}
function completionResponse(res, id, model, content) { function completionResponse(res, id, model, content) {
jsonResponse(res, 200, { jsonResponse(res, 200, {
id, object: "chat.completion", id, object: "chat.completion",
@@ -492,9 +694,19 @@ function completionResponse(res, id, model, content) {
} }
// ── Handle chat completions ───────────────────────────────────────────── // ── Handle chat completions ─────────────────────────────────────────────
const MAX_BODY_SIZE = 5 * 1024 * 1024; // 5 MB
// Set of all valid model identifiers (canonical IDs + aliases)
const VALID_MODELS = new Set(Object.keys(MODEL_MAP));
async function handleChatCompletions(req, res) { async function handleChatCompletions(req, res) {
let body = ""; let body = "";
for await (const chunk of req) body += chunk; for await (const chunk of req) {
body += chunk;
if (body.length > MAX_BODY_SIZE) {
return jsonResponse(res, 413, { error: { message: "Request body too large (max 5MB)", type: "invalid_request_error" } });
}
}
let parsed; let parsed;
try { parsed = JSON.parse(body); } catch { return jsonResponse(res, 400, { error: "Invalid JSON" }); } try { parsed = JSON.parse(body); } catch { return jsonResponse(res, 400, { error: "Invalid JSON" }); }
@@ -503,33 +715,43 @@ async function handleChatCompletions(req, res) {
const model = parsed.model || "claude-sonnet-4-6"; const model = parsed.model || "claude-sonnet-4-6";
const stream = parsed.stream; const stream = parsed.stream;
// Validate model against known models
if (!VALID_MODELS.has(model)) {
return jsonResponse(res, 400, { error: { message: `Unknown model: ${model}. Valid models: ${[...VALID_MODELS].join(", ")}`, type: "invalid_request_error" } });
}
// Session ID: from request body, header, or null (one-off) // Session ID: from request body, header, or null (one-off)
const conversationId = parsed.session_id || parsed.conversation_id || req.headers["x-session-id"] || req.headers["x-conversation-id"] || null; const conversationId = parsed.session_id || parsed.conversation_id || req.headers["x-session-id"] || req.headers["x-conversation-id"] || null;
if (!messages?.length) return jsonResponse(res, 400, { error: "messages required" }); if (!messages?.length) return jsonResponse(res, 400, { error: "messages required" });
if (stream) {
// Real streaming: pipe stdout from claude process directly as SSE chunks
return callClaudeStreaming(model, messages, conversationId, res);
}
try { try {
const content = await callClaude(model, messages, conversationId); const content = await callClaude(model, messages, conversationId);
const id = `chatcmpl-${randomUUID()}`; const id = `chatcmpl-${randomUUID()}`;
completionResponse(res, id, model, content);
if (stream) {
streamResponse(res, id, model, content);
} else {
completionResponse(res, id, model, content);
}
} catch (err) { } catch (err) {
console.error(`[proxy] error: ${err.message}`); console.error(`[proxy] error: ${err.message}`);
if (res.headersSent || res.writableEnded || res.destroyed) { if (res.headersSent || res.writableEnded || res.destroyed) {
try { res.end(); } catch {} try { res.end(); } catch {}
return; return;
} }
jsonResponse(res, 500, { error: { message: err.message, type: "proxy_error" } }); // Sanitize error: strip internal file paths before sending to client
const safeMessage = (err.message || "Internal error").replace(/\/[\w/.\-]+/g, "[path]");
jsonResponse(res, 500, { error: { message: safeMessage, type: "proxy_error" } });
} }
} }
// ── HTTP server ───────────────────────────────────────────────────────── // ── HTTP server ─────────────────────────────────────────────────────────
const server = createServer(async (req, res) => { const server = createServer(async (req, res) => {
res.setHeader("Access-Control-Allow-Origin", "*"); // Dynamic CORS: only allow localhost origins
const origin = req.headers["origin"] || "";
const isLocalhost = /^https?:\/\/(127\.0\.0\.1|localhost)(:\d+)?$/.test(origin);
res.setHeader("Access-Control-Allow-Origin", isLocalhost ? origin : `http://127.0.0.1:${PORT}`);
res.setHeader("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS"); res.setHeader("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS");
res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Session-Id, X-Conversation-Id"); res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Session-Id, X-Conversation-Id");
if (req.method === "OPTIONS") { res.writeHead(204); res.end(); return; } if (req.method === "OPTIONS") { res.writeHead(204); res.end(); return; }
@@ -538,7 +760,9 @@ const server = createServer(async (req, res) => {
if (PROXY_API_KEY && req.url !== "/health") { if (PROXY_API_KEY && req.url !== "/health") {
const auth = req.headers["authorization"] || ""; const auth = req.headers["authorization"] || "";
const token = auth.startsWith("Bearer ") ? auth.slice(7) : ""; const token = auth.startsWith("Bearer ") ? auth.slice(7) : "";
if (token !== PROXY_API_KEY) { const tokenBuf = Buffer.from(token);
const keyBuf = Buffer.from(PROXY_API_KEY);
if (tokenBuf.length !== keyBuf.length || !timingSafeEqual(tokenBuf, keyBuf)) {
return jsonResponse(res, 401, { error: { message: "Unauthorized: invalid or missing Bearer token", type: "auth_error" } }); return jsonResponse(res, 401, { error: { message: "Unauthorized: invalid or missing Bearer token", type: "auth_error" } });
} }
} }
@@ -575,8 +799,11 @@ const server = createServer(async (req, res) => {
}); });
} }
const breakerState = getBreakerSnapshot();
const anyBreakerOpen = Object.values(breakerState).some(b => b.state === "open");
return jsonResponse(res, 200, { return jsonResponse(res, 200, {
status: binaryOk && authStatus.ok !== false ? "ok" : "degraded", status: binaryOk && authStatus.ok !== false && !anyBreakerOpen ? "ok" : "degraded",
version: VERSION, version: VERSION,
architecture: "on-demand (v2)", architecture: "on-demand (v2)",
uptime: uptimeMs, uptime: uptimeMs,
@@ -589,11 +816,16 @@ const server = createServer(async (req, res) => {
firstByteTimeout: BASE_FIRST_BYTE_TIMEOUT, firstByteTimeout: BASE_FIRST_BYTE_TIMEOUT,
maxConcurrent: MAX_CONCURRENT, maxConcurrent: MAX_CONCURRENT,
sessionTTL: SESSION_TTL, sessionTTL: SESSION_TTL,
breakerThreshold: BREAKER_THRESHOLD,
breakerCooldown: BREAKER_COOLDOWN,
breakerWindow: BREAKER_WINDOW,
breakerHalfOpenMax: BREAKER_HALF_OPEN_MAX,
allowedTools: SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS, allowedTools: SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS,
systemPrompt: SYSTEM_PROMPT ? `${SYSTEM_PROMPT.slice(0, 50)}...` : "(none)", systemPrompt: SYSTEM_PROMPT ? `${SYSTEM_PROMPT.slice(0, 50)}...` : "(none)",
mcpConfig: MCP_CONFIG || "(none)", mcpConfig: MCP_CONFIG || "(none)",
}, },
stats, stats,
breakers: breakerState,
sessions: sessionList, sessions: sessionList,
recentErrors: recentErrors.slice(-5), recentErrors: recentErrors.slice(-5),
}); });
@@ -615,21 +847,70 @@ const server = createServer(async (req, res) => {
return jsonResponse(res, 200, { sessions: list }); return jsonResponse(res, 200, { sessions: list });
} }
// Catch-all POST
if (req.method === "POST") {
return handleChatCompletions(req, res);
}
jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET|DELETE /sessions" }); jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET|DELETE /sessions" });
}); });
// ── Graceful shutdown ────────────────────────────────────────────────────
let shuttingDown = false;
function gracefulShutdown(signal) {
if (shuttingDown) return;
shuttingDown = true;
logEvent("info", "shutdown_start", { signal });
// 1. Stop accepting new connections
server.close(() => {
logEvent("info", "shutdown_server_closed", {});
});
// 2. Clear intervals/timers
clearInterval(sessionCleanupInterval);
clearInterval(authCheckInterval);
// 3. Kill all active child processes
for (const proc of activeProcesses) {
try { proc.kill("SIGTERM"); } catch {}
}
// Force-kill any remaining processes after 5s, then exit
const forceExitTimer = setTimeout(() => {
for (const proc of activeProcesses) {
try { proc.kill("SIGKILL"); } catch {}
}
logEvent("warn", "shutdown_forced", { remainingProcesses: activeProcesses.size });
process.exit(1);
}, 5000);
forceExitTimer.unref();
// If no active processes, exit immediately
if (activeProcesses.size === 0) {
logEvent("info", "shutdown_complete", {});
process.exit(0);
}
// Wait for active processes to finish
const checkDone = setInterval(() => {
if (activeProcesses.size === 0) {
clearInterval(checkDone);
logEvent("info", "shutdown_complete", {});
process.exit(0);
}
}, 200);
checkDone.unref();
}
process.on("SIGTERM", () => gracefulShutdown("SIGTERM"));
process.on("SIGINT", () => gracefulShutdown("SIGINT"));
// ── Start ─────────────────────────────────────────────────────────────── // ── Start ───────────────────────────────────────────────────────────────
server.listen(PORT, "0.0.0.0", () => { server.listen(PORT, "127.0.0.1", () => {
console.log(`openclaw-claude-proxy v${VERSION} listening on http://0.0.0.0:${PORT}`); console.log(`openclaw-claude-proxy v${VERSION} listening on http://127.0.0.1:${PORT}`);
console.log(`Architecture: on-demand spawning (no pool)`); console.log(`Architecture: on-demand spawning (no pool)`);
console.log(`Models: ${MODELS.map((m) => m.id).join(", ")}`); console.log(`Models: ${MODELS.map((m) => m.id).join(", ")}`);
console.log(`Claude binary: ${CLAUDE}`); console.log(`Claude binary: ${CLAUDE}`);
console.log(`Timeout: ${TIMEOUT}ms (base first-byte: ${BASE_FIRST_BYTE_TIMEOUT}ms, adaptive by model/prompt) | Max concurrent: ${MAX_CONCURRENT}`); console.log(`Timeout: ${TIMEOUT}ms (base first-byte: ${BASE_FIRST_BYTE_TIMEOUT}ms, adaptive by model/prompt) | Max concurrent: ${MAX_CONCURRENT}`);
console.log(`Circuit breaker: threshold=${BREAKER_THRESHOLD} in ${BREAKER_WINDOW/1000}s window, cooldown=${BREAKER_COOLDOWN/1000}s (graduated), half-open probes=${BREAKER_HALF_OPEN_MAX}`);
console.log(`Tools: ${SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS.join(", ")}`); console.log(`Tools: ${SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS.join(", ")}`);
console.log(`Sessions: TTL=${SESSION_TTL / 1000}s`); console.log(`Sessions: TTL=${SESSION_TTL / 1000}s`);
if (SYSTEM_PROMPT) console.log(`System prompt: "${SYSTEM_PROMPT.slice(0, 80)}..."`); if (SYSTEM_PROMPT) console.log(`System prompt: "${SYSTEM_PROMPT.slice(0, 80)}..."`);