mirror of
https://github.com/dtzp555-max/ocp.git
synced 2026-07-22 13:35:08 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7434f6adc0 | ||
|
|
8a7951134e | ||
|
|
47434a8ba7 |
@@ -4,6 +4,40 @@
|
|||||||
|
|
||||||
A lightweight, zero-dependency proxy that lets [OpenClaw](https://github.com/openclaw/openclaw) agents talk to Claude through your existing subscription. One command to set up, one file to run.
|
A lightweight, zero-dependency proxy that lets [OpenClaw](https://github.com/openclaw/openclaw) agents talk to Claude through your existing subscription. One command to set up, one file to run.
|
||||||
|
|
||||||
|
## v2.5.0 — Emergency Fix: Sliding-Window Circuit Breaker
|
||||||
|
|
||||||
|
**Incident (2026-03-22):** Multi-agent burst (ClawTeam with 5+ Opus agents) caused cascading timeout failure. The old consecutive-count circuit breaker (threshold=3) tripped within seconds, blocking ALL requests globally — including unrelated agents and new sessions. With fallback models removed, this resulted in complete service outage ("LLM request timed out." on every message).
|
||||||
|
|
||||||
|
**Root cause:** v2.4.0's circuit breaker counted consecutive failures per model. When ClawTeam spawned multiple concurrent Opus agents and Claude API had moderate latency, 3 quick timeouts opened the breaker for the entire model. With `fallbacks: []`, the gateway had no alternative path.
|
||||||
|
|
||||||
|
**What's new in v2.5.0:**
|
||||||
|
- **Sliding-window circuit breaker** — counts failures in a 5-minute window (default: 6 failures) instead of 3 consecutive. Multi-agent bursts no longer trip the breaker instantly.
|
||||||
|
- **Graduated backoff** — cooldown doubles on each re-open (120s → 240s → 300s cap), resets fully on first success. Prevents oscillation between open/half-open during extended API issues.
|
||||||
|
- **Multi-probe half-open** — allows 2 concurrent probe requests in half-open state (was 1), improving recovery speed.
|
||||||
|
- **Increased default timeouts** — Opus first-byte 60s→90s, Sonnet 45s→60s, overall 120s→300s, max concurrent 5→8. Designed for large agent system prompts (30K+ chars).
|
||||||
|
- **Health endpoint shows breaker state** — `/health` now exposes per-model breaker state (window failures, cooldown, reopen count). Status is "degraded" when any breaker is open.
|
||||||
|
|
||||||
|
**New env vars:**
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `CLAUDE_BREAKER_WINDOW` | `300000` | Sliding window duration (ms) for failure counting |
|
||||||
|
| `CLAUDE_BREAKER_HALF_OPEN_MAX` | `2` | Max concurrent probe requests in half-open state |
|
||||||
|
|
||||||
|
**Updated defaults:**
|
||||||
|
|
||||||
|
| Variable | Old Default | New Default |
|
||||||
|
|----------|-------------|-------------|
|
||||||
|
| `CLAUDE_TIMEOUT` | `120000` | `300000` |
|
||||||
|
| `CLAUDE_FIRST_BYTE_TIMEOUT` | `45000` | `90000` |
|
||||||
|
| `CLAUDE_MAX_CONCURRENT` | `5` | `8` |
|
||||||
|
| `CLAUDE_BREAKER_THRESHOLD` | `3` | `6` |
|
||||||
|
| `CLAUDE_BREAKER_COOLDOWN` | `60000` | `120000` |
|
||||||
|
|
||||||
|
**Upgrade:** Pull latest and restart the proxy. The new defaults take effect immediately. If you have custom env vars set in your plist/service file, review and adjust them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## v2.0.0 — Major Upgrade
|
## v2.0.0 — Major Upgrade
|
||||||
|
|
||||||
**What's new:**
|
**What's new:**
|
||||||
@@ -174,13 +208,18 @@ openclaw gateway restart
|
|||||||
|----------|---------|-------------|
|
|----------|---------|-------------|
|
||||||
| `CLAUDE_PROXY_PORT` | `3456` | Listen port |
|
| `CLAUDE_PROXY_PORT` | `3456` | Listen port |
|
||||||
| `CLAUDE_BIN` | *(auto-detect)* | Path to claude binary |
|
| `CLAUDE_BIN` | *(auto-detect)* | Path to claude binary |
|
||||||
| `CLAUDE_TIMEOUT` | `300000` | Request timeout (ms) |
|
| `CLAUDE_TIMEOUT` | `300000` | Overall request timeout (ms) |
|
||||||
|
| `CLAUDE_FIRST_BYTE_TIMEOUT` | `90000` | Base first-byte timeout (ms), adaptive by model tier |
|
||||||
| `CLAUDE_ALLOWED_TOOLS` | `Bash,Read,...,Agent` | Comma-separated tools to pre-approve |
|
| `CLAUDE_ALLOWED_TOOLS` | `Bash,Read,...,Agent` | Comma-separated tools to pre-approve |
|
||||||
| `CLAUDE_SKIP_PERMISSIONS` | `false` | Set `true` to bypass all permission checks |
|
| `CLAUDE_SKIP_PERMISSIONS` | `false` | Set `true` to bypass all permission checks |
|
||||||
| `CLAUDE_SYSTEM_PROMPT` | *(empty)* | System prompt appended to all requests |
|
| `CLAUDE_SYSTEM_PROMPT` | *(empty)* | System prompt appended to all requests |
|
||||||
| `CLAUDE_MCP_CONFIG` | *(empty)* | Path to MCP server config JSON file |
|
| `CLAUDE_MCP_CONFIG` | *(empty)* | Path to MCP server config JSON file |
|
||||||
| `CLAUDE_SESSION_TTL` | `3600000` | Session expiry in ms (default: 1 hour) |
|
| `CLAUDE_SESSION_TTL` | `3600000` | Session expiry in ms (default: 1 hour) |
|
||||||
| `CLAUDE_MAX_CONCURRENT` | `5` | Max concurrent claude processes |
|
| `CLAUDE_MAX_CONCURRENT` | `8` | Max concurrent claude processes |
|
||||||
|
| `CLAUDE_BREAKER_THRESHOLD` | `6` | Failures in window before circuit opens |
|
||||||
|
| `CLAUDE_BREAKER_COOLDOWN` | `120000` | Base cooldown (ms) before half-open (graduates on re-open) |
|
||||||
|
| `CLAUDE_BREAKER_WINDOW` | `300000` | Sliding window duration (ms) for failure counting |
|
||||||
|
| `CLAUDE_BREAKER_HALF_OPEN_MAX` | `2` | Max concurrent probe requests in half-open state |
|
||||||
| `PROXY_API_KEY` | *(unset)* | Bearer token for API authentication |
|
| `PROXY_API_KEY` | *(unset)* | Bearer token for API authentication |
|
||||||
|
|
||||||
## API Endpoints
|
## API Endpoints
|
||||||
|
|||||||
+2
-2
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "openclaw-claude-proxy",
|
"name": "openclaw-claude-proxy",
|
||||||
"version": "2.4.0",
|
"version": "2.5.0",
|
||||||
"description": "OpenAI-compatible proxy for Claude CLI v2 — per-model circuit breaker, adaptive first-byte timeout, structured logging",
|
"description": "OpenAI-compatible proxy for Claude CLI v2 — sliding-window circuit breaker, adaptive first-byte timeout, structured logging",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"bin": {
|
"bin": {
|
||||||
"openclaw-claude-proxy": "./server.mjs"
|
"openclaw-claude-proxy": "./server.mjs"
|
||||||
|
|||||||
+512
-231
@@ -1,10 +1,19 @@
|
|||||||
#!/usr/bin/env node
|
#!/usr/bin/env node
|
||||||
/**
|
/**
|
||||||
* openclaw-claude-proxy v2.4.0 — OpenAI-compatible proxy for Claude CLI
|
* openclaw-claude-proxy v2.5.0 — OpenAI-compatible proxy for Claude CLI
|
||||||
*
|
*
|
||||||
* Translates OpenAI chat/completions requests into `claude -p` CLI calls,
|
* Translates OpenAI chat/completions requests into `claude -p` CLI calls,
|
||||||
* letting you use your Claude Pro/Max subscription as an OpenClaw model provider.
|
* letting you use your Claude Pro/Max subscription as an OpenClaw model provider.
|
||||||
*
|
*
|
||||||
|
* v2.5.0:
|
||||||
|
* - Sliding-window circuit breaker: uses time-windowed failure rate instead of
|
||||||
|
* consecutive-count, preventing multi-agent burst scenarios from tripping the
|
||||||
|
* breaker too aggressively. Half-open state allows configurable probe requests.
|
||||||
|
* - Graduated backoff: cooldown doubles on each re-open (capped at 5min),
|
||||||
|
* resets fully on success.
|
||||||
|
* - Health endpoint now exposes per-model breaker state and sliding window stats.
|
||||||
|
* - Increased default timeout tiers for Opus/Sonnet to handle large agent prompts.
|
||||||
|
*
|
||||||
* v2.4.0:
|
* v2.4.0:
|
||||||
* - Per-model circuit breaker: consecutive timeouts temporarily mark a model as degraded
|
* - Per-model circuit breaker: consecutive timeouts temporarily mark a model as degraded
|
||||||
* - Adaptive first-byte timeout: scales by model tier + prompt size
|
* - Adaptive first-byte timeout: scales by model tier + prompt size
|
||||||
@@ -12,23 +21,25 @@
|
|||||||
* - On-demand spawning (no pool), session management, full tool access
|
* - On-demand spawning (no pool), session management, full tool access
|
||||||
*
|
*
|
||||||
* Env vars:
|
* Env vars:
|
||||||
* CLAUDE_PROXY_PORT — listen port (default: 3456)
|
* CLAUDE_PROXY_PORT — listen port (default: 3456)
|
||||||
* CLAUDE_BIN — path to claude binary (default: auto-detect)
|
* CLAUDE_BIN — path to claude binary (default: auto-detect)
|
||||||
* CLAUDE_TIMEOUT — per-request timeout in ms (default: 120000)
|
* CLAUDE_TIMEOUT — per-request timeout in ms (default: 300000)
|
||||||
* CLAUDE_FIRST_BYTE_TIMEOUT — base first-byte timeout in ms (default: 45000)
|
* CLAUDE_FIRST_BYTE_TIMEOUT — base first-byte timeout in ms (default: 90000)
|
||||||
* CLAUDE_ALLOWED_TOOLS — comma-separated tools to allow (default: expanded set)
|
* CLAUDE_ALLOWED_TOOLS — comma-separated tools to allow (default: expanded set)
|
||||||
* CLAUDE_SKIP_PERMISSIONS — "true" to bypass all permission checks (default: false)
|
* CLAUDE_SKIP_PERMISSIONS — "true" to bypass all permission checks (default: false)
|
||||||
* CLAUDE_SYSTEM_PROMPT — system prompt appended to all requests
|
* CLAUDE_SYSTEM_PROMPT — system prompt appended to all requests
|
||||||
* CLAUDE_MCP_CONFIG — path to MCP server config JSON file
|
* CLAUDE_MCP_CONFIG — path to MCP server config JSON file
|
||||||
* CLAUDE_SESSION_TTL — session TTL in ms (default: 3600000 = 1h)
|
* CLAUDE_SESSION_TTL — session TTL in ms (default: 3600000 = 1h)
|
||||||
* CLAUDE_MAX_CONCURRENT — max concurrent claude processes (default: 5)
|
* CLAUDE_MAX_CONCURRENT — max concurrent claude processes (default: 8)
|
||||||
* CLAUDE_BREAKER_THRESHOLD — consecutive timeouts before circuit opens (default: 3)
|
* CLAUDE_BREAKER_THRESHOLD — failures in window before circuit opens (default: 6)
|
||||||
* CLAUDE_BREAKER_COOLDOWN — ms to wait before retrying after circuit opens (default: 60000)
|
* CLAUDE_BREAKER_COOLDOWN — base ms to wait before retrying after circuit opens (default: 120000)
|
||||||
* PROXY_API_KEY — Bearer token for API auth (optional)
|
* CLAUDE_BREAKER_WINDOW — sliding window duration in ms (default: 300000 = 5min)
|
||||||
|
* CLAUDE_BREAKER_HALF_OPEN_MAX — max concurrent probes in half-open state (default: 2)
|
||||||
|
* PROXY_API_KEY — Bearer token for API auth (optional)
|
||||||
*/
|
*/
|
||||||
import { createServer } from "node:http";
|
import { createServer } from "node:http";
|
||||||
import { spawn, execFileSync } from "node:child_process";
|
import { spawn, execFileSync } from "node:child_process";
|
||||||
import { randomUUID } from "node:crypto";
|
import { randomUUID, timingSafeEqual } from "node:crypto";
|
||||||
import { readFileSync, accessSync, constants } from "node:fs";
|
import { readFileSync, accessSync, constants } from "node:fs";
|
||||||
import { fileURLToPath } from "node:url";
|
import { fileURLToPath } from "node:url";
|
||||||
import { dirname, join } from "node:path";
|
import { dirname, join } from "node:path";
|
||||||
@@ -76,8 +87,8 @@ function resolveClaude() {
|
|||||||
// ── Configuration ───────────────────────────────────────────────────────
|
// ── Configuration ───────────────────────────────────────────────────────
|
||||||
const PORT = parseInt(process.env.CLAUDE_PROXY_PORT || "3456", 10);
|
const PORT = parseInt(process.env.CLAUDE_PROXY_PORT || "3456", 10);
|
||||||
const CLAUDE = resolveClaude();
|
const CLAUDE = resolveClaude();
|
||||||
const TIMEOUT = parseInt(process.env.CLAUDE_TIMEOUT || "120000", 10);
|
const TIMEOUT = parseInt(process.env.CLAUDE_TIMEOUT || "300000", 10);
|
||||||
const BASE_FIRST_BYTE_TIMEOUT = parseInt(process.env.CLAUDE_FIRST_BYTE_TIMEOUT || "45000", 10);
|
const BASE_FIRST_BYTE_TIMEOUT = parseInt(process.env.CLAUDE_FIRST_BYTE_TIMEOUT || "90000", 10);
|
||||||
const PROXY_API_KEY = process.env.PROXY_API_KEY || "";
|
const PROXY_API_KEY = process.env.PROXY_API_KEY || "";
|
||||||
const SKIP_PERMISSIONS = process.env.CLAUDE_SKIP_PERMISSIONS === "true";
|
const SKIP_PERMISSIONS = process.env.CLAUDE_SKIP_PERMISSIONS === "true";
|
||||||
const ALLOWED_TOOLS = (process.env.CLAUDE_ALLOWED_TOOLS ||
|
const ALLOWED_TOOLS = (process.env.CLAUDE_ALLOWED_TOOLS ||
|
||||||
@@ -86,9 +97,11 @@ const ALLOWED_TOOLS = (process.env.CLAUDE_ALLOWED_TOOLS ||
|
|||||||
const SYSTEM_PROMPT = process.env.CLAUDE_SYSTEM_PROMPT || "";
|
const SYSTEM_PROMPT = process.env.CLAUDE_SYSTEM_PROMPT || "";
|
||||||
const MCP_CONFIG = process.env.CLAUDE_MCP_CONFIG || "";
|
const MCP_CONFIG = process.env.CLAUDE_MCP_CONFIG || "";
|
||||||
const SESSION_TTL = parseInt(process.env.CLAUDE_SESSION_TTL || "3600000", 10);
|
const SESSION_TTL = parseInt(process.env.CLAUDE_SESSION_TTL || "3600000", 10);
|
||||||
const MAX_CONCURRENT = parseInt(process.env.CLAUDE_MAX_CONCURRENT || "5", 10);
|
const MAX_CONCURRENT = parseInt(process.env.CLAUDE_MAX_CONCURRENT || "8", 10);
|
||||||
const BREAKER_THRESHOLD = parseInt(process.env.CLAUDE_BREAKER_THRESHOLD || "3", 10);
|
const BREAKER_THRESHOLD = parseInt(process.env.CLAUDE_BREAKER_THRESHOLD || "6", 10);
|
||||||
const BREAKER_COOLDOWN = parseInt(process.env.CLAUDE_BREAKER_COOLDOWN || "60000", 10);
|
const BREAKER_COOLDOWN = parseInt(process.env.CLAUDE_BREAKER_COOLDOWN || "120000", 10);
|
||||||
|
const BREAKER_WINDOW = parseInt(process.env.CLAUDE_BREAKER_WINDOW || "300000", 10);
|
||||||
|
const BREAKER_HALF_OPEN_MAX = parseInt(process.env.CLAUDE_BREAKER_HALF_OPEN_MAX || "2", 10);
|
||||||
|
|
||||||
const VERSION = _pkg.version;
|
const VERSION = _pkg.version;
|
||||||
const START_TIME = Date.now();
|
const START_TIME = Date.now();
|
||||||
@@ -103,47 +116,128 @@ function logEvent(level, event, data = {}) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Per-model circuit breaker ───────────────────────────────────────────
|
// ── Per-model sliding-window circuit breaker ─────────────────────────────
|
||||||
// Tracks consecutive timeouts per model. When threshold is reached, the
|
// Uses a time-windowed failure rate instead of consecutive-count. This prevents
|
||||||
// model is marked "open" (degraded) for BREAKER_COOLDOWN ms. During that
|
// multi-agent burst scenarios (e.g. ClawTeam spawning 5+ Opus agents) from
|
||||||
// window, requests for this model fail fast with a clear error instead of
|
// tripping the breaker after just 3 quick timeouts.
|
||||||
// waiting for yet another timeout that would block the gateway.
|
//
|
||||||
const breakers = new Map(); // cliModel → { failures, state, openedAt }
|
// States: closed → open → half-open → closed (on success) or open (on failure)
|
||||||
|
// Half-open allows up to BREAKER_HALF_OPEN_MAX concurrent probes (not just 1).
|
||||||
|
// Cooldown uses graduated backoff: doubles on each re-open, resets on success.
|
||||||
|
const breakers = new Map(); // cliModel → BreakerState
|
||||||
|
|
||||||
|
function newBreakerState() {
|
||||||
|
return {
|
||||||
|
state: "closed", // closed | open | half-open
|
||||||
|
failureTimestamps: [], // timestamps of failures within the sliding window
|
||||||
|
successCount: 0, // successes within window (for rate calculation)
|
||||||
|
openedAt: 0,
|
||||||
|
currentCooldown: BREAKER_COOLDOWN, // graduates on repeated opens
|
||||||
|
reopenCount: 0, // how many times breaker has re-opened without a full reset
|
||||||
|
halfOpenProbes: 0, // active probe requests in half-open state
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function pruneWindow(b) {
|
||||||
|
const cutoff = Date.now() - BREAKER_WINDOW;
|
||||||
|
b.failureTimestamps = b.failureTimestamps.filter(ts => ts > cutoff);
|
||||||
|
}
|
||||||
|
|
||||||
function getBreakerState(cliModel) {
|
function getBreakerState(cliModel) {
|
||||||
if (!breakers.has(cliModel)) {
|
if (!breakers.has(cliModel)) {
|
||||||
breakers.set(cliModel, { failures: 0, state: "closed", openedAt: 0 });
|
breakers.set(cliModel, newBreakerState());
|
||||||
}
|
}
|
||||||
const b = breakers.get(cliModel);
|
const b = breakers.get(cliModel);
|
||||||
|
|
||||||
// Auto-recover: if cooldown has elapsed, transition to half-open
|
// Auto-recover: if cooldown has elapsed, transition to half-open
|
||||||
if (b.state === "open" && Date.now() - b.openedAt >= BREAKER_COOLDOWN) {
|
if (b.state === "open" && Date.now() - b.openedAt >= b.currentCooldown) {
|
||||||
b.state = "half-open";
|
b.state = "half-open";
|
||||||
logEvent("info", "breaker_half_open", { model: cliModel, cooldownMs: BREAKER_COOLDOWN });
|
b.halfOpenProbes = 0;
|
||||||
|
logEvent("info", "breaker_half_open", { model: cliModel, cooldownMs: b.currentCooldown, reopenCount: b.reopenCount });
|
||||||
}
|
}
|
||||||
return b;
|
return b;
|
||||||
}
|
}
|
||||||
|
|
||||||
function breakerRecordSuccess(cliModel) {
|
function breakerRecordSuccess(cliModel) {
|
||||||
const b = getBreakerState(cliModel);
|
const b = getBreakerState(cliModel);
|
||||||
if (b.failures > 0 || b.state !== "closed") {
|
b.successCount++;
|
||||||
logEvent("info", "breaker_reset", { model: cliModel, previousFailures: b.failures, previousState: b.state });
|
|
||||||
|
if (b.state === "half-open") {
|
||||||
|
b.halfOpenProbes = Math.max(0, b.halfOpenProbes - 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (b.state !== "closed") {
|
||||||
|
logEvent("info", "breaker_reset", {
|
||||||
|
model: cliModel,
|
||||||
|
previousFailures: b.failureTimestamps.length,
|
||||||
|
previousState: b.state,
|
||||||
|
reopenCount: b.reopenCount,
|
||||||
|
});
|
||||||
|
// Full reset on success — graduated backoff resets too
|
||||||
|
b.state = "closed";
|
||||||
|
b.openedAt = 0;
|
||||||
|
b.currentCooldown = BREAKER_COOLDOWN;
|
||||||
|
b.reopenCount = 0;
|
||||||
|
b.halfOpenProbes = 0;
|
||||||
|
b.failureTimestamps = [];
|
||||||
|
b.successCount = 0;
|
||||||
}
|
}
|
||||||
b.failures = 0;
|
|
||||||
b.state = "closed";
|
|
||||||
b.openedAt = 0;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function breakerRecordTimeout(cliModel) {
|
function breakerRecordTimeout(cliModel) {
|
||||||
const b = getBreakerState(cliModel);
|
const b = getBreakerState(cliModel);
|
||||||
b.failures++;
|
const now = Date.now();
|
||||||
logEvent("warn", "breaker_failure", { model: cliModel, consecutiveFailures: b.failures, threshold: BREAKER_THRESHOLD });
|
b.failureTimestamps.push(now);
|
||||||
|
pruneWindow(b);
|
||||||
|
|
||||||
if (b.failures >= BREAKER_THRESHOLD && b.state !== "open") {
|
if (b.state === "half-open") {
|
||||||
b.state = "open";
|
b.halfOpenProbes = Math.max(0, b.halfOpenProbes - 1);
|
||||||
b.openedAt = Date.now();
|
|
||||||
logEvent("error", "breaker_open", { model: cliModel, failures: b.failures, cooldownMs: BREAKER_COOLDOWN });
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const windowFailures = b.failureTimestamps.length;
|
||||||
|
logEvent("warn", "breaker_failure", {
|
||||||
|
model: cliModel,
|
||||||
|
windowFailures,
|
||||||
|
threshold: BREAKER_THRESHOLD,
|
||||||
|
windowMs: BREAKER_WINDOW,
|
||||||
|
state: b.state,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (windowFailures >= BREAKER_THRESHOLD && b.state !== "open") {
|
||||||
|
b.state = "open";
|
||||||
|
b.openedAt = now;
|
||||||
|
b.halfOpenProbes = 0;
|
||||||
|
// Graduated backoff: double cooldown on each re-open, cap at 5 min
|
||||||
|
if (b.reopenCount > 0) {
|
||||||
|
b.currentCooldown = Math.min(b.currentCooldown * 2, 300000);
|
||||||
|
}
|
||||||
|
b.reopenCount++;
|
||||||
|
logEvent("error", "breaker_open", {
|
||||||
|
model: cliModel,
|
||||||
|
windowFailures,
|
||||||
|
cooldownMs: b.currentCooldown,
|
||||||
|
reopenCount: b.reopenCount,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Expose breaker snapshot for /health endpoint
|
||||||
|
function getBreakerSnapshot() {
|
||||||
|
const snapshot = {};
|
||||||
|
for (const [model, b] of breakers) {
|
||||||
|
pruneWindow(b);
|
||||||
|
snapshot[model] = {
|
||||||
|
state: b.state,
|
||||||
|
windowFailures: b.failureTimestamps.length,
|
||||||
|
threshold: BREAKER_THRESHOLD,
|
||||||
|
windowMs: BREAKER_WINDOW,
|
||||||
|
currentCooldown: b.currentCooldown,
|
||||||
|
reopenCount: b.reopenCount,
|
||||||
|
halfOpenProbes: b.halfOpenProbes,
|
||||||
|
...(b.openedAt ? { openedAt: new Date(b.openedAt).toISOString() } : {}),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return snapshot;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Model mapping ───────────────────────────────────────────────────────
|
// ── Model mapping ───────────────────────────────────────────────────────
|
||||||
@@ -171,7 +265,7 @@ const MODELS = [
|
|||||||
// Enables --resume for multi-turn conversations, reducing token waste.
|
// Enables --resume for multi-turn conversations, reducing token waste.
|
||||||
const sessions = new Map(); // conversationId → { uuid, messageCount, lastUsed, model }
|
const sessions = new Map(); // conversationId → { uuid, messageCount, lastUsed, model }
|
||||||
|
|
||||||
setInterval(() => {
|
const sessionCleanupInterval = setInterval(() => {
|
||||||
const now = Date.now();
|
const now = Date.now();
|
||||||
for (const [id, s] of sessions) {
|
for (const [id, s] of sessions) {
|
||||||
if (now - s.lastUsed > SESSION_TTL) {
|
if (now - s.lastUsed > SESSION_TTL) {
|
||||||
@@ -181,6 +275,9 @@ setInterval(() => {
|
|||||||
}
|
}
|
||||||
}, 60000);
|
}, 60000);
|
||||||
|
|
||||||
|
// ── Active child process tracking ────────────────────────────────────────
|
||||||
|
const activeProcesses = new Set();
|
||||||
|
|
||||||
// ── Stats & diagnostics ─────────────────────────────────────────────────
|
// ── Stats & diagnostics ─────────────────────────────────────────────────
|
||||||
const stats = {
|
const stats = {
|
||||||
totalRequests: 0,
|
totalRequests: 0,
|
||||||
@@ -220,7 +317,7 @@ async function checkAuth() {
|
|||||||
|
|
||||||
// Check auth on start and every 10 minutes
|
// Check auth on start and every 10 minutes
|
||||||
checkAuth();
|
checkAuth();
|
||||||
setInterval(checkAuth, 600000);
|
const authCheckInterval = setInterval(checkAuth, 600000);
|
||||||
|
|
||||||
// ── Build CLI arguments ─────────────────────────────────────────────────
|
// ── Build CLI arguments ─────────────────────────────────────────────────
|
||||||
function buildCliArgs(cliModel, sessionInfo) {
|
function buildCliArgs(cliModel, sessionInfo) {
|
||||||
@@ -268,8 +365,8 @@ function messagesToPrompt(messages) {
|
|||||||
// Model tier multipliers for first-byte timeout.
|
// Model tier multipliers for first-byte timeout.
|
||||||
// Opus is much slower to produce first token, especially with large contexts.
|
// Opus is much slower to produce first token, especially with large contexts.
|
||||||
const MODEL_TIMEOUT_TIERS = {
|
const MODEL_TIMEOUT_TIERS = {
|
||||||
"opus": { base: 60000, perPromptChar: 0.00015 }, // 60s base + ~15s per 100k chars
|
"opus": { base: 90000, perPromptChar: 0.00020 }, // 90s base + ~20s per 100k chars
|
||||||
"sonnet": { base: 45000, perPromptChar: 0.00008 }, // 45s base + ~8s per 100k chars
|
"sonnet": { base: 60000, perPromptChar: 0.00010 }, // 60s base + ~10s per 100k chars
|
||||||
"haiku": { base: 30000, perPromptChar: 0.00005 }, // 30s base + ~5s per 100k chars
|
"haiku": { base: 30000, perPromptChar: 0.00005 }, // 30s base + ~5s per 100k chars
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -285,162 +382,293 @@ function computeFirstByteTimeout(cliModel, promptLength) {
|
|||||||
return Math.min(timeout, Math.max(TIMEOUT - 5000, 10000));
|
return Math.min(timeout, Math.max(TIMEOUT - 5000, 10000));
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Call claude CLI ─────────────────────────────────────────────────────
|
// ── Spawn claude CLI (shared setup) ─────────────────────────────────────
|
||||||
// On-demand spawning: each request spawns a fresh `claude -p` process.
|
// Resolves session logic, builds CLI args, spawns the process, and sets up
|
||||||
// No pool = no crash loops, no stale workers, no degraded states.
|
// timeouts. Returns context object or throws synchronously.
|
||||||
// Stdin is written immediately so there's no 3s stdin timeout issue.
|
function spawnClaudeProcess(model, messages, conversationId) {
|
||||||
function callClaude(model, messages, conversationId) {
|
if (stats.activeRequests >= MAX_CONCURRENT) {
|
||||||
return new Promise((resolve, reject) => {
|
throw new Error(`concurrency limit reached (${stats.activeRequests}/${MAX_CONCURRENT})`);
|
||||||
if (stats.activeRequests >= MAX_CONCURRENT) {
|
}
|
||||||
return reject(new Error(`concurrency limit reached (${stats.activeRequests}/${MAX_CONCURRENT})`));
|
|
||||||
|
const cliModel = MODEL_MAP[model] || model;
|
||||||
|
|
||||||
|
// Circuit breaker check: fail fast if model is in open state
|
||||||
|
const breaker = getBreakerState(cliModel);
|
||||||
|
if (breaker.state === "open") {
|
||||||
|
const remainingMs = breaker.currentCooldown - (Date.now() - breaker.openedAt);
|
||||||
|
logEvent("warn", "breaker_rejected", { model: cliModel, remainingCooldownMs: remainingMs, reopenCount: breaker.reopenCount });
|
||||||
|
throw new Error(`circuit breaker open for ${cliModel}: ${breaker.failureTimestamps.length} timeouts in window, retry in ${Math.ceil(remainingMs / 1000)}s`);
|
||||||
|
}
|
||||||
|
// Half-open: allow limited probe requests
|
||||||
|
if (breaker.state === "half-open" && breaker.halfOpenProbes >= BREAKER_HALF_OPEN_MAX) {
|
||||||
|
logEvent("warn", "breaker_half_open_full", { model: cliModel, activeProbes: breaker.halfOpenProbes, max: BREAKER_HALF_OPEN_MAX });
|
||||||
|
throw new Error(`circuit breaker half-open for ${cliModel}: ${breaker.halfOpenProbes}/${BREAKER_HALF_OPEN_MAX} probe slots in use, wait for probe result`);
|
||||||
|
}
|
||||||
|
if (breaker.state === "half-open") {
|
||||||
|
breaker.halfOpenProbes++;
|
||||||
|
logEvent("info", "breaker_probe", { model: cliModel, activeProbes: breaker.halfOpenProbes, max: BREAKER_HALF_OPEN_MAX });
|
||||||
|
}
|
||||||
|
|
||||||
|
stats.activeRequests++;
|
||||||
|
stats.totalRequests++;
|
||||||
|
|
||||||
|
let sessionInfo = null;
|
||||||
|
let prompt;
|
||||||
|
|
||||||
|
// ── Session logic ──
|
||||||
|
if (conversationId && sessions.has(conversationId)) {
|
||||||
|
const session = sessions.get(conversationId);
|
||||||
|
session.lastUsed = Date.now();
|
||||||
|
sessionInfo = { uuid: session.uuid, resume: true };
|
||||||
|
stats.sessionHits++;
|
||||||
|
|
||||||
|
const lastUserMsg = [...messages].reverse().find((m) => m.role === "user");
|
||||||
|
prompt = lastUserMsg
|
||||||
|
? (typeof lastUserMsg.content === "string" ? lastUserMsg.content : JSON.stringify(lastUserMsg.content))
|
||||||
|
: "";
|
||||||
|
session.messageCount = messages.length;
|
||||||
|
|
||||||
|
console.log(`[session] resume conv=${conversationId.slice(0, 12)}... uuid=${session.uuid.slice(0, 8)}... msgs=${messages.length} prompt_chars=${prompt.length}`);
|
||||||
|
|
||||||
|
} else if (conversationId) {
|
||||||
|
const uuid = randomUUID();
|
||||||
|
sessions.set(conversationId, { uuid, messageCount: messages.length, lastUsed: Date.now(), model: cliModel });
|
||||||
|
sessionInfo = { uuid, resume: false };
|
||||||
|
stats.sessionMisses++;
|
||||||
|
prompt = messagesToPrompt(messages);
|
||||||
|
|
||||||
|
console.log(`[session] new conv=${conversationId.slice(0, 12)}... uuid=${uuid.slice(0, 8)}... msgs=${messages.length}`);
|
||||||
|
|
||||||
|
} else {
|
||||||
|
stats.oneOffRequests++;
|
||||||
|
prompt = messagesToPrompt(messages);
|
||||||
|
}
|
||||||
|
|
||||||
|
const cliArgs = buildCliArgs(cliModel, sessionInfo);
|
||||||
|
|
||||||
|
const env = { ...process.env };
|
||||||
|
delete env.CLAUDECODE;
|
||||||
|
delete env.ANTHROPIC_API_KEY;
|
||||||
|
delete env.ANTHROPIC_BASE_URL;
|
||||||
|
delete env.ANTHROPIC_AUTH_TOKEN;
|
||||||
|
|
||||||
|
const proc = spawn(CLAUDE, cliArgs, { env, stdio: ["pipe", "pipe", "pipe"] });
|
||||||
|
activeProcesses.add(proc);
|
||||||
|
|
||||||
|
const t0 = Date.now();
|
||||||
|
const firstByteTimeoutMs = computeFirstByteTimeout(cliModel, prompt.length);
|
||||||
|
let gotFirstByte = false;
|
||||||
|
let cleaned = false;
|
||||||
|
|
||||||
|
function cleanup() {
|
||||||
|
if (cleaned) return;
|
||||||
|
cleaned = true;
|
||||||
|
clearTimeout(overallTimer);
|
||||||
|
clearTimeout(firstByteTimer);
|
||||||
|
stats.activeRequests--;
|
||||||
|
}
|
||||||
|
|
||||||
|
function handleSessionFailure() {
|
||||||
|
if (sessionInfo?.resume && conversationId) {
|
||||||
|
console.warn(`[session] resume failed for ${conversationId.slice(0, 12)}..., removing stale session`);
|
||||||
|
sessions.delete(conversationId);
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const cliModel = MODEL_MAP[model] || model;
|
function markFirstByte() {
|
||||||
|
if (!gotFirstByte) {
|
||||||
// Circuit breaker check: fail fast if model is in open state
|
gotFirstByte = true;
|
||||||
const breaker = getBreakerState(cliModel);
|
|
||||||
if (breaker.state === "open") {
|
|
||||||
const remainingMs = BREAKER_COOLDOWN - (Date.now() - breaker.openedAt);
|
|
||||||
logEvent("warn", "breaker_rejected", { model: cliModel, remainingCooldownMs: remainingMs });
|
|
||||||
return reject(new Error(`circuit breaker open for ${cliModel}: ${breaker.failures} consecutive timeouts, retry in ${Math.ceil(remainingMs / 1000)}s`));
|
|
||||||
}
|
|
||||||
|
|
||||||
stats.activeRequests++;
|
|
||||||
stats.totalRequests++;
|
|
||||||
|
|
||||||
let sessionInfo = null;
|
|
||||||
let prompt;
|
|
||||||
|
|
||||||
// ── Session logic ──
|
|
||||||
if (conversationId && sessions.has(conversationId)) {
|
|
||||||
// Resume existing session: only send the latest user message
|
|
||||||
const session = sessions.get(conversationId);
|
|
||||||
session.lastUsed = Date.now();
|
|
||||||
sessionInfo = { uuid: session.uuid, resume: true };
|
|
||||||
stats.sessionHits++;
|
|
||||||
|
|
||||||
const lastUserMsg = [...messages].reverse().find((m) => m.role === "user");
|
|
||||||
prompt = lastUserMsg
|
|
||||||
? (typeof lastUserMsg.content === "string" ? lastUserMsg.content : JSON.stringify(lastUserMsg.content))
|
|
||||||
: "";
|
|
||||||
session.messageCount = messages.length;
|
|
||||||
|
|
||||||
console.log(`[session] resume conv=${conversationId.slice(0, 12)}... uuid=${session.uuid.slice(0, 8)}... msgs=${messages.length} prompt_chars=${prompt.length}`);
|
|
||||||
|
|
||||||
} else if (conversationId) {
|
|
||||||
// New session: send all messages, persist session for future --resume
|
|
||||||
const uuid = randomUUID();
|
|
||||||
sessions.set(conversationId, { uuid, messageCount: messages.length, lastUsed: Date.now(), model: cliModel });
|
|
||||||
sessionInfo = { uuid, resume: false };
|
|
||||||
stats.sessionMisses++;
|
|
||||||
prompt = messagesToPrompt(messages);
|
|
||||||
|
|
||||||
console.log(`[session] new conv=${conversationId.slice(0, 12)}... uuid=${uuid.slice(0, 8)}... msgs=${messages.length}`);
|
|
||||||
|
|
||||||
} else {
|
|
||||||
// One-off request, no session
|
|
||||||
stats.oneOffRequests++;
|
|
||||||
prompt = messagesToPrompt(messages);
|
|
||||||
}
|
|
||||||
|
|
||||||
const cliArgs = buildCliArgs(cliModel, sessionInfo);
|
|
||||||
|
|
||||||
const env = { ...process.env };
|
|
||||||
delete env.CLAUDECODE;
|
|
||||||
delete env.ANTHROPIC_API_KEY;
|
|
||||||
delete env.ANTHROPIC_BASE_URL;
|
|
||||||
delete env.ANTHROPIC_AUTH_TOKEN;
|
|
||||||
|
|
||||||
const proc = spawn(CLAUDE, cliArgs, { env, stdio: ["pipe", "pipe", "pipe"] });
|
|
||||||
|
|
||||||
let stdout = "";
|
|
||||||
let stderr = "";
|
|
||||||
const t0 = Date.now();
|
|
||||||
const firstByteTimeoutMs = computeFirstByteTimeout(cliModel, prompt.length);
|
|
||||||
let settled = false;
|
|
||||||
let gotFirstByte = false;
|
|
||||||
|
|
||||||
function settle(err, result) {
|
|
||||||
if (settled) return;
|
|
||||||
settled = true;
|
|
||||||
clearTimeout(timer);
|
|
||||||
clearTimeout(firstByteTimer);
|
clearTimeout(firstByteTimer);
|
||||||
stats.activeRequests--;
|
console.log(`[claude] first-byte model=${cliModel} elapsed=${Date.now() - t0}ms`);
|
||||||
|
|
||||||
if (err) {
|
|
||||||
trackError(err.message || String(err));
|
|
||||||
|
|
||||||
// If session resume failed, remove session so next request starts fresh
|
|
||||||
if (sessionInfo?.resume && conversationId) {
|
|
||||||
console.warn(`[session] resume failed for ${conversationId.slice(0, 12)}..., removing stale session`);
|
|
||||||
sessions.delete(conversationId);
|
|
||||||
}
|
|
||||||
|
|
||||||
reject(err);
|
|
||||||
} else {
|
|
||||||
resolve(result);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
proc.stdout.on("data", (d) => {
|
// Write prompt to stdin immediately
|
||||||
if (!gotFirstByte) {
|
proc.stdin.write(prompt);
|
||||||
gotFirstByte = true;
|
proc.stdin.end();
|
||||||
clearTimeout(firstByteTimer);
|
|
||||||
console.log(`[claude] first-byte model=${cliModel} elapsed=${Date.now() - t0}ms`);
|
|
||||||
}
|
|
||||||
stdout += d;
|
|
||||||
});
|
|
||||||
proc.stderr.on("data", (d) => (stderr += d));
|
|
||||||
|
|
||||||
proc.on("close", (code, signal) => {
|
logEvent("info", "claude_spawned", { model: cliModel, promptChars: prompt.length, firstByteTimeout: firstByteTimeoutMs, tier: getModelTier(cliModel), session: conversationId ? conversationId.slice(0, 12) + "..." : "none" });
|
||||||
const elapsed = Date.now() - t0;
|
|
||||||
if (settled) {
|
|
||||||
logEvent("warn", "late_close", { model: cliModel, code, signal: signal || "none", elapsed });
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (code !== 0) {
|
|
||||||
logEvent("error", "claude_exit", { model: cliModel, code, signal: signal || "none", elapsed, stderr: stderr.slice(0, 300) });
|
|
||||||
settle(new Error(stderr.slice(0, 300) || stdout.slice(0, 300) || `claude exit ${code}`));
|
|
||||||
} else {
|
|
||||||
breakerRecordSuccess(cliModel);
|
|
||||||
logEvent("info", "claude_ok", { model: cliModel, chars: stdout.length, elapsed, session: conversationId ? conversationId.slice(0, 12) + "..." : "none" });
|
|
||||||
settle(null, stdout.trim());
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
proc.on("error", (err) => {
|
// First-byte timeout
|
||||||
console.error(`[claude] spawn error: ${err.message}`);
|
const firstByteTimer = setTimeout(() => {
|
||||||
settle(err);
|
if (!gotFirstByte && !cleaned) {
|
||||||
});
|
stats.timeouts++;
|
||||||
|
breakerRecordTimeout(cliModel);
|
||||||
|
logEvent("error", "first_byte_timeout", { model: cliModel, timeoutMs: firstByteTimeoutMs, promptChars: prompt.length });
|
||||||
|
try { proc.kill("SIGTERM"); } catch {}
|
||||||
|
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
|
||||||
|
}
|
||||||
|
}, firstByteTimeoutMs);
|
||||||
|
|
||||||
// Write prompt to stdin immediately — no idle timeout issue
|
// Overall request timeout
|
||||||
proc.stdin.write(prompt);
|
const overallTimer = setTimeout(() => {
|
||||||
proc.stdin.end();
|
if (!cleaned) {
|
||||||
|
|
||||||
logEvent("info", "claude_spawned", { model: cliModel, promptChars: prompt.length, firstByteTimeout: firstByteTimeoutMs, tier: getModelTier(cliModel), session: conversationId ? conversationId.slice(0, 12) + "..." : "none" });
|
|
||||||
|
|
||||||
// First-byte timeout: abort early if Claude CLI produces no output
|
|
||||||
const firstByteTimer = setTimeout(() => {
|
|
||||||
if (!gotFirstByte && !settled) {
|
|
||||||
stats.timeouts++;
|
|
||||||
breakerRecordTimeout(cliModel);
|
|
||||||
logEvent("error", "first_byte_timeout", { model: cliModel, timeoutMs: firstByteTimeoutMs, promptChars: prompt.length });
|
|
||||||
try { proc.kill("SIGTERM"); } catch {}
|
|
||||||
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
|
|
||||||
settle(new Error(`first-byte timeout after ${firstByteTimeoutMs}ms`));
|
|
||||||
}
|
|
||||||
}, firstByteTimeoutMs);
|
|
||||||
|
|
||||||
// Overall request timeout with graceful kill
|
|
||||||
const timer = setTimeout(() => {
|
|
||||||
if (settled) return;
|
|
||||||
stats.timeouts++;
|
stats.timeouts++;
|
||||||
breakerRecordTimeout(cliModel);
|
breakerRecordTimeout(cliModel);
|
||||||
logEvent("error", "request_timeout", { model: cliModel, timeoutMs: TIMEOUT });
|
logEvent("error", "request_timeout", { model: cliModel, timeoutMs: TIMEOUT });
|
||||||
try { proc.kill("SIGTERM"); } catch {}
|
try { proc.kill("SIGTERM"); } catch {}
|
||||||
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
|
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
|
||||||
settle(new Error(`timeout after ${TIMEOUT}ms`));
|
}
|
||||||
}, TIMEOUT);
|
}, TIMEOUT);
|
||||||
|
|
||||||
|
return { proc, cliModel, conversationId, t0, cleanup, handleSessionFailure, markFirstByte };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Call claude CLI (non-streaming) ─────────────────────────────────────
|
||||||
|
// On-demand spawning: each request spawns a fresh `claude -p` process.
|
||||||
|
// No pool = no crash loops, no stale workers, no degraded states.
|
||||||
|
// Stdin is written immediately so there's no 3s stdin timeout issue.
|
||||||
|
function callClaude(model, messages, conversationId) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
let ctx;
|
||||||
|
try {
|
||||||
|
ctx = spawnClaudeProcess(model, messages, conversationId);
|
||||||
|
} catch (err) {
|
||||||
|
return reject(err);
|
||||||
|
}
|
||||||
|
|
||||||
|
const { proc, cliModel, conversationId: convId, t0, cleanup, handleSessionFailure, markFirstByte } = ctx;
|
||||||
|
let stdout = "";
|
||||||
|
let stderr = "";
|
||||||
|
|
||||||
|
proc.stdout.on("data", (d) => {
|
||||||
|
markFirstByte();
|
||||||
|
stdout += d;
|
||||||
|
});
|
||||||
|
proc.stderr.on("data", (d) => (stderr += d));
|
||||||
|
|
||||||
|
proc.on("close", (code, signal) => {
|
||||||
|
activeProcesses.delete(proc);
|
||||||
|
const elapsed = Date.now() - t0;
|
||||||
|
cleanup();
|
||||||
|
if (code !== 0) {
|
||||||
|
logEvent("error", "claude_exit", { model: cliModel, code, signal: signal || "none", elapsed, stderr: stderr.slice(0, 300) });
|
||||||
|
trackError(stderr.slice(0, 300) || stdout.slice(0, 300) || `claude exit ${code}`);
|
||||||
|
handleSessionFailure();
|
||||||
|
reject(new Error(stderr.slice(0, 300) || stdout.slice(0, 300) || `claude exit ${code}`));
|
||||||
|
} else {
|
||||||
|
breakerRecordSuccess(cliModel);
|
||||||
|
logEvent("info", "claude_ok", { model: cliModel, chars: stdout.length, elapsed, session: convId ? convId.slice(0, 12) + "..." : "none" });
|
||||||
|
resolve(stdout.trim());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
proc.on("error", (err) => {
|
||||||
|
console.error(`[claude] spawn error: ${err.message}`);
|
||||||
|
cleanup();
|
||||||
|
trackError(err.message);
|
||||||
|
handleSessionFailure();
|
||||||
|
reject(err);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Call claude CLI (real streaming) ─────────────────────────────────────
|
||||||
|
// Pipes stdout from the claude process directly to SSE chunks as they arrive.
|
||||||
|
// Each data chunk becomes a proper SSE event with delta content in real time.
|
||||||
|
function callClaudeStreaming(model, messages, conversationId, res) {
|
||||||
|
const id = `chatcmpl-${randomUUID()}`;
|
||||||
|
const created = Math.floor(Date.now() / 1000);
|
||||||
|
|
||||||
|
let ctx;
|
||||||
|
try {
|
||||||
|
ctx = spawnClaudeProcess(model, messages, conversationId);
|
||||||
|
} catch (err) {
|
||||||
|
return jsonResponse(res, 500, { error: { message: err.message, type: "proxy_error" } });
|
||||||
|
}
|
||||||
|
|
||||||
|
const { proc, cliModel, conversationId: convId, t0, cleanup, handleSessionFailure, markFirstByte } = ctx;
|
||||||
|
let stderr = "";
|
||||||
|
let headersSent = false;
|
||||||
|
let totalChars = 0;
|
||||||
|
|
||||||
|
function ensureHeaders() {
|
||||||
|
if (headersSent || res.writableEnded || res.destroyed) return false;
|
||||||
|
headersSent = true;
|
||||||
|
res.writeHead(200, {
|
||||||
|
"Content-Type": "text/event-stream",
|
||||||
|
"Cache-Control": "no-cache",
|
||||||
|
"Connection": "keep-alive",
|
||||||
|
});
|
||||||
|
// Send initial role chunk
|
||||||
|
sendSSE(res, {
|
||||||
|
id, object: "chat.completion.chunk", created, model,
|
||||||
|
choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }],
|
||||||
|
});
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
proc.stdout.on("data", (d) => {
|
||||||
|
markFirstByte();
|
||||||
|
const text = d.toString();
|
||||||
|
totalChars += text.length;
|
||||||
|
|
||||||
|
if (!ensureHeaders()) return;
|
||||||
|
|
||||||
|
// Stream each chunk as it arrives from the CLI process
|
||||||
|
sendSSE(res, {
|
||||||
|
id, object: "chat.completion.chunk", created, model,
|
||||||
|
choices: [{ index: 0, delta: { content: text }, finish_reason: null }],
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
proc.stderr.on("data", (d) => (stderr += d));
|
||||||
|
|
||||||
|
proc.on("close", (code, signal) => {
|
||||||
|
activeProcesses.delete(proc);
|
||||||
|
cleanup();
|
||||||
|
const elapsed = Date.now() - t0;
|
||||||
|
|
||||||
|
if (code !== 0) {
|
||||||
|
logEvent("error", "claude_exit", { model: cliModel, code, signal: signal || "none", elapsed, stderr: stderr.slice(0, 300) });
|
||||||
|
trackError(stderr.slice(0, 300) || `claude exit ${code}`);
|
||||||
|
handleSessionFailure();
|
||||||
|
|
||||||
|
if (!headersSent && !res.writableEnded && !res.destroyed) {
|
||||||
|
// No output was sent yet — return a JSON error
|
||||||
|
jsonResponse(res, 500, { error: { message: stderr.slice(0, 300) || `claude exit ${code}`, type: "proxy_error" } });
|
||||||
|
} else if (!res.writableEnded && !res.destroyed) {
|
||||||
|
// Already streaming — close the stream gracefully
|
||||||
|
sendSSE(res, {
|
||||||
|
id, object: "chat.completion.chunk", created, model,
|
||||||
|
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
|
||||||
|
});
|
||||||
|
res.write("data: [DONE]\n\n");
|
||||||
|
res.end();
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
breakerRecordSuccess(cliModel);
|
||||||
|
logEvent("info", "claude_ok", { model: cliModel, chars: totalChars, elapsed, session: convId ? convId.slice(0, 12) + "..." : "none" });
|
||||||
|
|
||||||
|
if (!headersSent) ensureHeaders();
|
||||||
|
if (!res.writableEnded && !res.destroyed) {
|
||||||
|
sendSSE(res, {
|
||||||
|
id, object: "chat.completion.chunk", created, model,
|
||||||
|
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
|
||||||
|
});
|
||||||
|
res.write("data: [DONE]\n\n");
|
||||||
|
res.end();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
proc.on("error", (err) => {
|
||||||
|
console.error(`[claude] spawn error: ${err.message}`);
|
||||||
|
cleanup();
|
||||||
|
trackError(err.message);
|
||||||
|
handleSessionFailure();
|
||||||
|
if (!headersSent && !res.writableEnded && !res.destroyed) {
|
||||||
|
jsonResponse(res, 500, { error: { message: err.message, type: "proxy_error" } });
|
||||||
|
} else if (!res.writableEnded && !res.destroyed) {
|
||||||
|
res.end();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// If client disconnects, kill the process to free resources
|
||||||
|
res.on("close", () => {
|
||||||
|
if (!proc.killed) {
|
||||||
|
try { proc.kill("SIGTERM"); } catch {}
|
||||||
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -455,32 +683,6 @@ function sendSSE(res, data) {
|
|||||||
res.write(`data: ${JSON.stringify(data)}\n\n`);
|
res.write(`data: ${JSON.stringify(data)}\n\n`);
|
||||||
}
|
}
|
||||||
|
|
||||||
function streamResponse(res, id, model, content) {
|
|
||||||
if (res.headersSent || res.writableEnded || res.destroyed) return;
|
|
||||||
res.writeHead(200, {
|
|
||||||
"Content-Type": "text/event-stream",
|
|
||||||
"Cache-Control": "no-cache",
|
|
||||||
"Connection": "keep-alive",
|
|
||||||
});
|
|
||||||
const created = Math.floor(Date.now() / 1000);
|
|
||||||
sendSSE(res, {
|
|
||||||
id, object: "chat.completion.chunk", created, model,
|
|
||||||
choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }],
|
|
||||||
});
|
|
||||||
for (let i = 0; i < content.length; i += 500) {
|
|
||||||
sendSSE(res, {
|
|
||||||
id, object: "chat.completion.chunk", created, model,
|
|
||||||
choices: [{ index: 0, delta: { content: content.slice(i, i + 500) }, finish_reason: null }],
|
|
||||||
});
|
|
||||||
}
|
|
||||||
sendSSE(res, {
|
|
||||||
id, object: "chat.completion.chunk", created, model,
|
|
||||||
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
|
|
||||||
});
|
|
||||||
res.write("data: [DONE]\n\n");
|
|
||||||
res.end();
|
|
||||||
}
|
|
||||||
|
|
||||||
function completionResponse(res, id, model, content) {
|
function completionResponse(res, id, model, content) {
|
||||||
jsonResponse(res, 200, {
|
jsonResponse(res, 200, {
|
||||||
id, object: "chat.completion",
|
id, object: "chat.completion",
|
||||||
@@ -492,9 +694,19 @@ function completionResponse(res, id, model, content) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ── Handle chat completions ─────────────────────────────────────────────
|
// ── Handle chat completions ─────────────────────────────────────────────
|
||||||
|
const MAX_BODY_SIZE = 5 * 1024 * 1024; // 5 MB
|
||||||
|
|
||||||
|
// Set of all valid model identifiers (canonical IDs + aliases)
|
||||||
|
const VALID_MODELS = new Set(Object.keys(MODEL_MAP));
|
||||||
|
|
||||||
async function handleChatCompletions(req, res) {
|
async function handleChatCompletions(req, res) {
|
||||||
let body = "";
|
let body = "";
|
||||||
for await (const chunk of req) body += chunk;
|
for await (const chunk of req) {
|
||||||
|
body += chunk;
|
||||||
|
if (body.length > MAX_BODY_SIZE) {
|
||||||
|
return jsonResponse(res, 413, { error: { message: "Request body too large (max 5MB)", type: "invalid_request_error" } });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
let parsed;
|
let parsed;
|
||||||
try { parsed = JSON.parse(body); } catch { return jsonResponse(res, 400, { error: "Invalid JSON" }); }
|
try { parsed = JSON.parse(body); } catch { return jsonResponse(res, 400, { error: "Invalid JSON" }); }
|
||||||
@@ -503,33 +715,43 @@ async function handleChatCompletions(req, res) {
|
|||||||
const model = parsed.model || "claude-sonnet-4-6";
|
const model = parsed.model || "claude-sonnet-4-6";
|
||||||
const stream = parsed.stream;
|
const stream = parsed.stream;
|
||||||
|
|
||||||
|
// Validate model against known models
|
||||||
|
if (!VALID_MODELS.has(model)) {
|
||||||
|
return jsonResponse(res, 400, { error: { message: `Unknown model: ${model}. Valid models: ${[...VALID_MODELS].join(", ")}`, type: "invalid_request_error" } });
|
||||||
|
}
|
||||||
|
|
||||||
// Session ID: from request body, header, or null (one-off)
|
// Session ID: from request body, header, or null (one-off)
|
||||||
const conversationId = parsed.session_id || parsed.conversation_id || req.headers["x-session-id"] || req.headers["x-conversation-id"] || null;
|
const conversationId = parsed.session_id || parsed.conversation_id || req.headers["x-session-id"] || req.headers["x-conversation-id"] || null;
|
||||||
|
|
||||||
if (!messages?.length) return jsonResponse(res, 400, { error: "messages required" });
|
if (!messages?.length) return jsonResponse(res, 400, { error: "messages required" });
|
||||||
|
|
||||||
|
if (stream) {
|
||||||
|
// Real streaming: pipe stdout from claude process directly as SSE chunks
|
||||||
|
return callClaudeStreaming(model, messages, conversationId, res);
|
||||||
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const content = await callClaude(model, messages, conversationId);
|
const content = await callClaude(model, messages, conversationId);
|
||||||
const id = `chatcmpl-${randomUUID()}`;
|
const id = `chatcmpl-${randomUUID()}`;
|
||||||
|
completionResponse(res, id, model, content);
|
||||||
if (stream) {
|
|
||||||
streamResponse(res, id, model, content);
|
|
||||||
} else {
|
|
||||||
completionResponse(res, id, model, content);
|
|
||||||
}
|
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error(`[proxy] error: ${err.message}`);
|
console.error(`[proxy] error: ${err.message}`);
|
||||||
if (res.headersSent || res.writableEnded || res.destroyed) {
|
if (res.headersSent || res.writableEnded || res.destroyed) {
|
||||||
try { res.end(); } catch {}
|
try { res.end(); } catch {}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
jsonResponse(res, 500, { error: { message: err.message, type: "proxy_error" } });
|
// Sanitize error: strip internal file paths before sending to client
|
||||||
|
const safeMessage = (err.message || "Internal error").replace(/\/[\w/.\-]+/g, "[path]");
|
||||||
|
jsonResponse(res, 500, { error: { message: safeMessage, type: "proxy_error" } });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── HTTP server ─────────────────────────────────────────────────────────
|
// ── HTTP server ─────────────────────────────────────────────────────────
|
||||||
const server = createServer(async (req, res) => {
|
const server = createServer(async (req, res) => {
|
||||||
res.setHeader("Access-Control-Allow-Origin", "*");
|
// Dynamic CORS: only allow localhost origins
|
||||||
|
const origin = req.headers["origin"] || "";
|
||||||
|
const isLocalhost = /^https?:\/\/(127\.0\.0\.1|localhost)(:\d+)?$/.test(origin);
|
||||||
|
res.setHeader("Access-Control-Allow-Origin", isLocalhost ? origin : `http://127.0.0.1:${PORT}`);
|
||||||
res.setHeader("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS");
|
res.setHeader("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS");
|
||||||
res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Session-Id, X-Conversation-Id");
|
res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Session-Id, X-Conversation-Id");
|
||||||
if (req.method === "OPTIONS") { res.writeHead(204); res.end(); return; }
|
if (req.method === "OPTIONS") { res.writeHead(204); res.end(); return; }
|
||||||
@@ -538,7 +760,9 @@ const server = createServer(async (req, res) => {
|
|||||||
if (PROXY_API_KEY && req.url !== "/health") {
|
if (PROXY_API_KEY && req.url !== "/health") {
|
||||||
const auth = req.headers["authorization"] || "";
|
const auth = req.headers["authorization"] || "";
|
||||||
const token = auth.startsWith("Bearer ") ? auth.slice(7) : "";
|
const token = auth.startsWith("Bearer ") ? auth.slice(7) : "";
|
||||||
if (token !== PROXY_API_KEY) {
|
const tokenBuf = Buffer.from(token);
|
||||||
|
const keyBuf = Buffer.from(PROXY_API_KEY);
|
||||||
|
if (tokenBuf.length !== keyBuf.length || !timingSafeEqual(tokenBuf, keyBuf)) {
|
||||||
return jsonResponse(res, 401, { error: { message: "Unauthorized: invalid or missing Bearer token", type: "auth_error" } });
|
return jsonResponse(res, 401, { error: { message: "Unauthorized: invalid or missing Bearer token", type: "auth_error" } });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -575,8 +799,11 @@ const server = createServer(async (req, res) => {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const breakerState = getBreakerSnapshot();
|
||||||
|
const anyBreakerOpen = Object.values(breakerState).some(b => b.state === "open");
|
||||||
|
|
||||||
return jsonResponse(res, 200, {
|
return jsonResponse(res, 200, {
|
||||||
status: binaryOk && authStatus.ok !== false ? "ok" : "degraded",
|
status: binaryOk && authStatus.ok !== false && !anyBreakerOpen ? "ok" : "degraded",
|
||||||
version: VERSION,
|
version: VERSION,
|
||||||
architecture: "on-demand (v2)",
|
architecture: "on-demand (v2)",
|
||||||
uptime: uptimeMs,
|
uptime: uptimeMs,
|
||||||
@@ -589,11 +816,16 @@ const server = createServer(async (req, res) => {
|
|||||||
firstByteTimeout: BASE_FIRST_BYTE_TIMEOUT,
|
firstByteTimeout: BASE_FIRST_BYTE_TIMEOUT,
|
||||||
maxConcurrent: MAX_CONCURRENT,
|
maxConcurrent: MAX_CONCURRENT,
|
||||||
sessionTTL: SESSION_TTL,
|
sessionTTL: SESSION_TTL,
|
||||||
|
breakerThreshold: BREAKER_THRESHOLD,
|
||||||
|
breakerCooldown: BREAKER_COOLDOWN,
|
||||||
|
breakerWindow: BREAKER_WINDOW,
|
||||||
|
breakerHalfOpenMax: BREAKER_HALF_OPEN_MAX,
|
||||||
allowedTools: SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS,
|
allowedTools: SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS,
|
||||||
systemPrompt: SYSTEM_PROMPT ? `${SYSTEM_PROMPT.slice(0, 50)}...` : "(none)",
|
systemPrompt: SYSTEM_PROMPT ? `${SYSTEM_PROMPT.slice(0, 50)}...` : "(none)",
|
||||||
mcpConfig: MCP_CONFIG || "(none)",
|
mcpConfig: MCP_CONFIG || "(none)",
|
||||||
},
|
},
|
||||||
stats,
|
stats,
|
||||||
|
breakers: breakerState,
|
||||||
sessions: sessionList,
|
sessions: sessionList,
|
||||||
recentErrors: recentErrors.slice(-5),
|
recentErrors: recentErrors.slice(-5),
|
||||||
});
|
});
|
||||||
@@ -615,21 +847,70 @@ const server = createServer(async (req, res) => {
|
|||||||
return jsonResponse(res, 200, { sessions: list });
|
return jsonResponse(res, 200, { sessions: list });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Catch-all POST
|
|
||||||
if (req.method === "POST") {
|
|
||||||
return handleChatCompletions(req, res);
|
|
||||||
}
|
|
||||||
|
|
||||||
jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET|DELETE /sessions" });
|
jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET|DELETE /sessions" });
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
||||||
|
// ── Graceful shutdown ────────────────────────────────────────────────────
|
||||||
|
let shuttingDown = false;
|
||||||
|
|
||||||
|
function gracefulShutdown(signal) {
|
||||||
|
if (shuttingDown) return;
|
||||||
|
shuttingDown = true;
|
||||||
|
logEvent("info", "shutdown_start", { signal });
|
||||||
|
|
||||||
|
// 1. Stop accepting new connections
|
||||||
|
server.close(() => {
|
||||||
|
logEvent("info", "shutdown_server_closed", {});
|
||||||
|
});
|
||||||
|
|
||||||
|
// 2. Clear intervals/timers
|
||||||
|
clearInterval(sessionCleanupInterval);
|
||||||
|
clearInterval(authCheckInterval);
|
||||||
|
|
||||||
|
// 3. Kill all active child processes
|
||||||
|
for (const proc of activeProcesses) {
|
||||||
|
try { proc.kill("SIGTERM"); } catch {}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Force-kill any remaining processes after 5s, then exit
|
||||||
|
const forceExitTimer = setTimeout(() => {
|
||||||
|
for (const proc of activeProcesses) {
|
||||||
|
try { proc.kill("SIGKILL"); } catch {}
|
||||||
|
}
|
||||||
|
logEvent("warn", "shutdown_forced", { remainingProcesses: activeProcesses.size });
|
||||||
|
process.exit(1);
|
||||||
|
}, 5000);
|
||||||
|
forceExitTimer.unref();
|
||||||
|
|
||||||
|
// If no active processes, exit immediately
|
||||||
|
if (activeProcesses.size === 0) {
|
||||||
|
logEvent("info", "shutdown_complete", {});
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for active processes to finish
|
||||||
|
const checkDone = setInterval(() => {
|
||||||
|
if (activeProcesses.size === 0) {
|
||||||
|
clearInterval(checkDone);
|
||||||
|
logEvent("info", "shutdown_complete", {});
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
}, 200);
|
||||||
|
checkDone.unref();
|
||||||
|
}
|
||||||
|
|
||||||
|
process.on("SIGTERM", () => gracefulShutdown("SIGTERM"));
|
||||||
|
process.on("SIGINT", () => gracefulShutdown("SIGINT"));
|
||||||
|
|
||||||
// ── Start ───────────────────────────────────────────────────────────────
|
// ── Start ───────────────────────────────────────────────────────────────
|
||||||
server.listen(PORT, "0.0.0.0", () => {
|
server.listen(PORT, "127.0.0.1", () => {
|
||||||
console.log(`openclaw-claude-proxy v${VERSION} listening on http://0.0.0.0:${PORT}`);
|
console.log(`openclaw-claude-proxy v${VERSION} listening on http://127.0.0.1:${PORT}`);
|
||||||
console.log(`Architecture: on-demand spawning (no pool)`);
|
console.log(`Architecture: on-demand spawning (no pool)`);
|
||||||
console.log(`Models: ${MODELS.map((m) => m.id).join(", ")}`);
|
console.log(`Models: ${MODELS.map((m) => m.id).join(", ")}`);
|
||||||
console.log(`Claude binary: ${CLAUDE}`);
|
console.log(`Claude binary: ${CLAUDE}`);
|
||||||
console.log(`Timeout: ${TIMEOUT}ms (base first-byte: ${BASE_FIRST_BYTE_TIMEOUT}ms, adaptive by model/prompt) | Max concurrent: ${MAX_CONCURRENT}`);
|
console.log(`Timeout: ${TIMEOUT}ms (base first-byte: ${BASE_FIRST_BYTE_TIMEOUT}ms, adaptive by model/prompt) | Max concurrent: ${MAX_CONCURRENT}`);
|
||||||
|
console.log(`Circuit breaker: threshold=${BREAKER_THRESHOLD} in ${BREAKER_WINDOW/1000}s window, cooldown=${BREAKER_COOLDOWN/1000}s (graduated), half-open probes=${BREAKER_HALF_OPEN_MAX}`);
|
||||||
console.log(`Tools: ${SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS.join(", ")}`);
|
console.log(`Tools: ${SKIP_PERMISSIONS ? "all (skip-permissions)" : ALLOWED_TOOLS.join(", ")}`);
|
||||||
console.log(`Sessions: TTL=${SESSION_TTL / 1000}s`);
|
console.log(`Sessions: TTL=${SESSION_TTL / 1000}s`);
|
||||||
if (SYSTEM_PROMPT) console.log(`System prompt: "${SYSTEM_PROMPT.slice(0, 80)}..."`);
|
if (SYSTEM_PROMPT) console.log(`System prompt: "${SYSTEM_PROMPT.slice(0, 80)}..."`);
|
||||||
|
|||||||
Reference in New Issue
Block a user