diff --git a/PLAN-v4.md b/PLAN-v4.md deleted file mode 100644 index 1aa14de..0000000 --- a/PLAN-v4.md +++ /dev/null @@ -1,439 +0,0 @@ -# OCP v4.0 开发计划 — Agent 成本控制器 - -## 背景 - -OCP 当前架构:`Tool → OCP (localhost:3456) → Claude CLI → Anthropic` - -核心风险:Claude CLI proxy 属于灰色地带,Anthropic 随时可能封堵。 - -核心资产(不依赖 CLI 的部分): -- OpenAI 兼容 API 层 -- Session 管理(会话隔离、TTL、复用) -- Usage 监控(plan limits、per-model stats、实时查询) -- 运行时调参(timeout、并发、prompt 限制) -- `/ocp` 命令集(Telegram/Discord/CLI) -- 多实例并发控制 - -## 定位 - -> OCP is a cost-control and model-routing layer for AI agents, optimized for subscription-based usage. - -**不是 proxy,不是 gateway,不是 LiteLLM clone。** -**是:让 Agent 用模型这件事变得可控 — 尤其是成本。** - -核心差异点:OCP 有 agent 上下文,知道是谁在请求、在做什么任务、已经花了多少。LiteLLM 只看到 HTTP 请求。 - -## 目标 - -将 OCP 从 "Claude CLI proxy" 重构为 "Agent 成本控制器",支持可扩展的多后端,保留全部管理能力。 - ---- - -## ⚠️ 第零原则:不动现有功能 - -OCP 作为 Claude CLI proxy 是当前的核心价值和用户吸引力。所有 v4 重构必须遵守: - -- **外部 API 不变** — `/v1/chat/completions`、`/usage`、`/health`、`/settings` 签名不变 -- **行为不变** — 现有用户升级后零感知,所有 session/timeout/并发逻辑保持原样 -- **逐步替换** — 先抽模块,跑稳了再切;任何时候回归测试不过就回滚 -- **新 backend 是增量** — 加 OpenAI/Ollama 是新能力,不改动 Claude CLI 路径 - -Phase 1 的定义就是:**拆完之后跟没拆一样。** 拆出问题就说明拆法有误。 - ---- - -## 架构关键决策(三轮 Design Review 定稿) - -### 决策 1: 路由分两层 — Model Registry + Agent Policy - -**三轮讨论结论:** -- 第一轮建议"删掉 model routing" → 错误,非 agent 客户端(Cline/Aider)只带 model 名 -- 反驳:model-to-backend mapping 是基础设施,不是 routing rule -- 校准:物理拆成两个模块,不要混在一起 - -**最终设计 — 两层分离:** - -``` -🧱 Layer 1: Model Registry(基础设施层) - model-registry.mjs - 静态映射,无条件,必须存在 - "claude-sonnet-4-6" → claude-cli backend - "gpt-4o" → openai backend - "qwen3" → ollama backend - -🧠 Layer 2: Agent Policy(策略层) - agent-router.mjs - 动态决策,可覆盖,可选 - agent "coder" → override to claude-cli - agent "interpreter" → override to ollama -``` - -**路由决策顺序(写死):** -``` -1. Agent policy override → 有 agent identity 且有配置?用它 -2. Model registry → 模型属于哪个 backend?路由过去 -3. Default backend → 都没匹配?全局默认 -4. Fallback chain → 上面选的挂了?降级 -``` - -### 决策 2: 统一 Streaming 协议 - -**所有 adapter 必须输出统一的内部 chunk 格式:** - -```javascript -{ - type: "delta" | "done" | "error", - content?: string, - model: string, - backend: string, - usage?: { promptTokens: number, completionTokens: number } -} -``` - -server.mjs 只处理统一协议 → 转换为 OpenAI SSE 输出。 - -### 决策 3: 成本标记 — actual / estimated / free - -```javascript -{ - cost: { - value: 0.23, - type: "actual" | "estimated" | "free" - } -} -``` - -| Backend | cost type | 来源 | -|---------|-----------|------| -| OpenAI API | actual | API 返回 usage | -| Claude CLI | estimated | prompt chars × 估价 | -| Ollama | free | 本地运行 | - -`/ocp cost` 输出明确标注类型,不误导用户。 - -### 决策 4: Fallback — 只在 First-Byte 前触发 - -```javascript -const allowFallback = !hasStartedStreaming; -``` - -Phase 3 严格限制:已输出 ≥1 个 delta → 不 fallback,直接报错。 -flag 预留,未来扩展 streaming fallback 只改判断条件。 - -### 决策 5: Agent Identity — 结构预留,实现从简 - -**优先级:** -``` -1. x-ocp-agent header → OpenClaw 自动附加 -2. Session metadata → 首次请求绑定,后续复用 -3. (future) Inferred → 按使用模式推断(预留接口) -4. default → 兜底 -``` - -**现在只实现 1 + 2 + 4。** 但接口预留 inferred 扩展点。 - -为什么重要:未来 Cline coding mode 和 chat mode 可能需要不同路由,如果没有 identity hook 以后很难加。 - ---- - -## Backend Adapter 设计 - -### 接口 - -```javascript -class BackendAdapter { - get id() // "claude-cli" | "openai-api" | "ollama" - get displayName() - get models() - get costType() // "actual" | "estimated" | "free" - get tier() // "core" | "community" - - async healthCheck() → { ok, message, latencyMs } - async *chatCompletion(request) → AsyncGenerator - async initialize(config) - async shutdown() -} -``` - -### Backend Tier 体系 - -OCP provides a pluggable backend adapter interface. Core adapters are maintained officially; community adapters are contributed by users. - -``` -/ocp backends 输出示例: - -claude-cli ✓ healthy 12ms (core) -openai ✓ healthy 89ms (core) -ollama ✓ healthy 5ms (core) -deepseek ✓ healthy 120ms (community) -``` - -架构不限制 backend 数量。我们主动维护 3 个 core adapter(Claude CLI、OpenAI 兼容、Ollama),接口开放,社区可贡献更多。 - -OpenAI 兼容的服务(DeepSeek、Groq、Together 等)直接复用 `OpenAiApiAdapter`,只需换 baseUrl + apiKey,不算新 adapter。 - ---- - -## 分阶段实施 - -### Phase 1: 后端抽象 + Model Registry(v4.0-alpha) -**目标:** 不改变外部行为,内部重构 - -- [ ] 定义 `BackendAdapter` 接口 + `UnifiedChunk` 类型 -- [ ] `model-registry.mjs` — 模型到 backend 的静态映射 - - `getBackendForModel(model) → backendId` -- [ ] `agent-router.mjs` — 路由决策引擎 - - `resolveBackend({ agent, model }) → backendId` - - 实现决策顺序:agent override → model registry → default → fallback -- [ ] 将 Claude CLI 逻辑封装为 `ClaudeCliAdapter` - - stdout → UnifiedChunk 转换 - - costType = "estimated" -- [ ] 抽取 `SessionManager`(从 server.mjs) -- [ ] 抽取 `StatsCollector`(per-agent, per-backend) -- [ ] Config v4 格式(带 version + validation): - -```json -{ - "version": "4.0", - "backends": { - "claude-cli": { - "type": "claude-cli", - "tier": "core", - "enabled": true, - "models": ["claude-sonnet-4-6", "claude-opus-4-6", "claude-haiku-4-5"], - "maxConcurrent": 4, - "timeout": { "firstByte": 120000, "overall": 300000 }, - "estimatedCostPerMTok": { "input": 3.0, "output": 15.0 } - } - }, - "agents": { - "default": { "preferred": "claude-cli", "fallback": null } - } -} -``` - -- [ ] 启动时 config validation -- [ ] `/ocp backends` 命令 -- [ ] 所有现有测试通过,行为不变 - -**验收标准:** 现有用户升级后零感知变化 - -### Phase 2: OpenAI API 后端(v4.0-beta) -**目标:** 第二个 backend,验证 adapter 抽象 - -- [ ] `OpenAiApiAdapter` - - SSE → UnifiedChunk - - costType = "actual" - - 支持任何 OpenAI 兼容端点(baseUrl + apiKey) -- [ ] Model registry 自动注册新 backend 的 models -- [ ] Agent routing 生效: - -```json -{ - "agents": { - "default": { "preferred": "claude-cli", "fallback": "openai" }, - "interpreter": { "preferred": "openai/gpt-4o-mini", "fallback": null } - } -} -``` - -- [ ] Non-streaming fallback(`allowFallback = !hasStartedStreaming`) -- [ ] Stats 按 backend + agent 分组 - -**验收标准:** claude-cli 挂了 → default agent 自动 fallback 到 openai - -### Phase 3: Ollama + Agent Routing 稳定化(v4.0-rc) -**目标:** 摆脱付费依赖,agent routing 跑稳 - -- [ ] `OllamaAdapter` - - 自动发现模型 - - costType = "free" -- [ ] Agent routing 完整覆盖所有 agent: - -```json -{ - "agents": { - "main": { "preferred": "claude-cli/opus", "fallback": "openai" }, - "tech_geek": { "preferred": "claude-cli/sonnet", "fallback": "ollama/qwen3" }, - "interpreter": { "preferred": "ollama/qwen3", "fallback": null } - } -} -``` - -- [ ] `/ocp routing` — 显示每个 agent 当前走哪个 backend -- [ ] 端到端测试:Claude CLI 下线 → agent 自动 fallback → 恢复后自动回切 - -**验收标准:** 三个 backend 同时运行,每个 agent 走不同路径,稳定无错 - -### Phase 4: Budget + Cost Control(v4.1) -**目标:** 核心差异化 — agent-aware 成本控制 - -**前置条件:** Phase 3 routing 稳定(先确定流量怎么走 → 再控制花多少钱) - -**安全原则:** 没有显式配置 budget 的 agent → 行为等同于 v3(不限制、不降级、不告警)。Budget 是 opt-in,不是 opt-out。 - -- [ ] Per-agent 预算: - -```json -{ - "agents": { - "main": { - "preferred": "claude-cli/opus", - "fallback": "openai", - "dailyBudget": { "limit": 3.00, "onExhausted": "downgrade" } - } - }, - "budget": { - "global": { "daily": 10.00, "onExhausted": "warn" } - } -} -``` - -- [ ] 超预算行为:downgrade | block | warn -- [ ] `/ocp cost` — per-agent 成本(标注 actual/estimated/free) -- [ ] Telegram 告警(预算 80%、backend 连续失败) - -### Phase 5: Anthropic 官方 API(v4.2,等时机) -- [ ] `AnthropicApiAdapter`(官方 SDK) -- [ ] 迁移指南:一行配置切换 -- [ ] costType = "actual" - ---- - -## 文件结构(目标) - -``` -claude-proxy/ -├── server.mjs → HTTP 层(统一 chunk → SSE 输出) -├── model-registry.mjs → 模型注册表(model → backend 映射) -├── agent-router.mjs → Agent 路由引擎(策略层) -├── session-manager.mjs → 会话管理 -├── stats-collector.mjs → 统计 + 成本追踪 -├── config.mjs → 配置加载 + 版本验证 -├── unified-chunk.mjs → UnifiedChunk 类型定义 -├── backends/ -│ ├── base.mjs → BackendAdapter 接口 -│ ├── claude-cli.mjs → Claude CLI(core) -│ ├── openai-api.mjs → OpenAI 兼容(core) -│ └── ollama.mjs → Ollama 本地(core) -├── setup.mjs → 安装向导 -└── ocp-plugin/ → Gateway 命令插件 -``` - ---- - -## 配置完整示例(目标状态) - -```json -{ - "version": "4.0", - "port": 3456, - - "backends": { - "claude-cli": { - "type": "claude-cli", - "tier": "core", - "enabled": true, - "models": ["claude-sonnet-4-6", "claude-opus-4-6"], - "maxConcurrent": 4, - "timeout": { "firstByte": 120000, "overall": 300000 }, - "estimatedCostPerMTok": { "input": 3.0, "output": 15.0 } - }, - "openai": { - "type": "openai-api", - "tier": "core", - "enabled": true, - "baseUrl": "https://openrouter.ai/api/v1", - "apiKey": "${OPENROUTER_API_KEY}", - "models": ["anthropic/claude-3.5-sonnet", "google/gemini-2.0-flash"], - "maxConcurrent": 10, - "timeout": { "firstByte": 30000, "overall": 120000 } - }, - "ollama": { - "type": "ollama", - "tier": "core", - "enabled": true, - "baseUrl": "http://localhost:11434", - "models": "auto" - } - }, - - "agents": { - "default": { - "preferred": "claude-cli", - "fallback": "openai" - }, - "main": { - "preferred": "claude-cli/claude-opus-4-6", - "fallback": "openai", - "dailyBudget": { "limit": 3.00, "onExhausted": "downgrade" } - }, - "tech_geek": { - "preferred": "claude-cli/claude-sonnet-4-6", - "fallback": "ollama/qwen3", - "dailyBudget": { "limit": 2.00, "onExhausted": "fallback" } - }, - "interpreter": { - "preferred": "ollama/qwen3", - "fallback": null - } - }, - - "fallback": { - "onlyBeforeFirstByte": true, - "triggers": ["timeout", "rate-limit", "connection-error"], - "maxRetries": 1 - }, - - "budget": { - "global": { "daily": 10.00, "onExhausted": "warn" }, - "alerts": { "threshold": 0.8, "channel": "telegram" } - }, - - "sessions": { "ttl": 3600000, "maxPromptChars": 150000 }, - "monitoring": { "retainHours": 72 } -} -``` - ---- - -## 风险与应对 - -| 风险 | 应对 | -|------|------| -| Claude CLI 被封 | 切换 backend,上层不变 | -| Anthropic 出订阅 API | 实现 anthropic-api adapter,成本控制层继续有价值 | -| LiteLLM 竞争 | 不竞争。OCP 做 agent-aware 成本控制,赛道不同 | -| 成本数据不准 | cost type 标记 actual/estimated/free | -| Streaming fallback 拼接 | Phase 3 只做 non-streaming fallback | -| 过度工程 | 只维护 3 个 core backend,phase 严格拆分 | - -## 优先级 - -| Phase | 时间 | 内容 | 交付物 | -|-------|------|------|--------| -| 1 | 本周 | 重构:adapter 接口 + model registry + agent router | 行为不变,架构就位 | -| 2 | 1-2 周 | OpenAI backend | 两个 backend 跑通 | -| 3 | 2-3 周 | Ollama + Agent routing 稳定化 | 三 backend + per-agent 路由 | -| 4 | 按需 | Budget + Cost control | 成本可控 | -| 5 | 等时机 | Anthropic 官方 API | 逃生通道 | - -## 成功标准 - -1. Claude CLI 被封的那天,改一行配置,所有 agent 继续运行 -2. `/ocp cost` 每个 agent 花了多少(actual/estimated/free) -3. 翻译 agent 自动用 Ollama($0),coding agent 用 Claude -4. 超预算自动降级,不需人工干预 -5. 新 backend 只需实现 BackendAdapter 接口 + 一个文件 - -## 设计审核记录 - -| 轮次 | 审核方 | 关键结论 | -|------|--------|----------| -| 第一轮 | AI-A | 定位从 proxy 转向成本控制器;识别 5 个 P0 问题 | -| 第二轮 | Claude(本项目开发者) | 反驳:不能删 model routing(基础设施层);backend 数量是运营决策不是架构约束;budget 拆到 Phase 4 | -| 第三轮 | AI-A | 校准:model registry 与 routing 物理拆开两个模块;agent identity 结构预留 inferred 扩展点;backend 加 tier 标记 | - -## 一句话总结 - -**短期靠套利活(Claude CLI proxy),长期靠控制力活(Agent 成本控制器)。两条腿走路,CLI 死了上层不死。** diff --git a/agent-router.mjs b/agent-router.mjs deleted file mode 100644 index 5fda1b4..0000000 --- a/agent-router.mjs +++ /dev/null @@ -1,171 +0,0 @@ -/** - * Agent Router — Layer 2 of OCP routing. - * - * Routing decision order (written in stone): - * 1. Agent policy override → agent has explicit backend config? Use it. - * 2. Model registry → model maps to a backend? Route there. - * 3. Default backend → global fallback. - * 4. Fallback chain → chosen backend is down? Try next. - * - * Agent identity sources (priority): - * 1. x-ocp-agent header - * 2. Session metadata (bound on first request) - * 3. (future) Inferred from usage patterns - * 4. "default" - */ - -export class AgentRouter { - /** - * @param {Object} options - * @param {import("./model-registry.mjs").ModelRegistry} options.registry - * @param {Map} options.backends - * @param {Object} options.agentConfig — agent routing rules from config - * @param {string} options.defaultBackend — fallback backend ID - */ - constructor({ registry, backends, agentConfig = {}, defaultBackend = "claude-cli" }) { - this._registry = registry; - this._backends = backends; - this._agentConfig = agentConfig; - this._defaultBackend = defaultBackend; - } - - /** - * Identify the agent from a request. - * - * @param {Object} req - HTTP request - * @param {Object} [sessionMeta] - Session metadata (may contain agent binding) - * @returns {string} Agent name - */ - identifyAgent(req, sessionMeta = null) { - // 1. Explicit header - const headerAgent = req.headers?.["x-ocp-agent"]; - if (headerAgent) return headerAgent; - - // 2. Session binding - if (sessionMeta?.agent) return sessionMeta.agent; - - // 3. (future) Inferred — not implemented yet - - // 4. Default - return "default"; - } - - /** - * Resolve which backend + model to use for a request. - * - * @param {Object} params - * @param {string} params.agent - Agent name (from identifyAgent) - * @param {string} params.model - Requested model ID - * @returns {{backendId: string, model: string, reason: string}} - */ - resolve({ agent, model }) { - // Step 1: Agent policy override - const agentRule = this._agentConfig[agent]; - if (agentRule?.preferred) { - const { backendId, modelId } = this._parsePreferred(agentRule.preferred); - const backend = this._backends.get(backendId); - if (backend?.enabled) { - const resolvedModel = modelId || model; - return { - backendId, - model: resolvedModel, - reason: `agent-policy(${agent})`, - }; - } - // Agent preferred backend is down — try fallback - if (agentRule.fallback) { - const fb = this._parseFallback(agentRule.fallback, model); - if (fb) return { ...fb, reason: `agent-fallback(${agent})` }; - } - } - - // Step 2: Model registry - const registryEntry = this._registry.resolve(model); - if (registryEntry) { - const backend = this._backends.get(registryEntry.backendId); - if (backend?.enabled) { - return { - backendId: registryEntry.backendId, - model: registryEntry.canonical, - reason: "model-registry", - }; - } - } - - // Step 2b: Check adapters for alias support - for (const [id, backend] of this._backends) { - if (backend.enabled && backend.supportsModel(model)) { - const resolved = typeof backend.resolveModel === "function" - ? backend.resolveModel(model) - : model; - return { - backendId: id, - model: resolved, - reason: `adapter-match(${id})`, - }; - } - } - - // Step 3: Default backend - const defaultAgent = this._agentConfig.default; - if (defaultAgent?.preferred) { - const { backendId } = this._parsePreferred(defaultAgent.preferred); - if (this._backends.get(backendId)?.enabled) { - return { backendId, model, reason: "default-agent-policy" }; - } - } - - if (this._backends.get(this._defaultBackend)?.enabled) { - return { - backendId: this._defaultBackend, - model, - reason: "default-backend", - }; - } - - // Step 4: Fallback — try any enabled backend - for (const [id, backend] of this._backends) { - if (backend.enabled) { - return { backendId: id, model, reason: `last-resort(${id})` }; - } - } - - return { backendId: null, model, reason: "no-backend-available" }; - } - - /** - * Parse a preferred string like "claude-cli" or "claude-cli/claude-opus-4-6" - */ - _parsePreferred(preferred) { - if (preferred.includes("/")) { - const [backendId, modelId] = preferred.split("/", 2); - return { backendId, modelId }; - } - return { backendId: preferred, modelId: null }; - } - - /** - * Parse a fallback value and return routing info. - */ - _parseFallback(fallback, model) { - if (!fallback) return null; - const { backendId, modelId } = this._parsePreferred(fallback); - const backend = this._backends.get(backendId); - if (!backend?.enabled) return null; - return { backendId, model: modelId || model }; - } - - /** - * Get routing info for display (e.g. /ocp routing command). - */ - getRoutingTable() { - const table = {}; - for (const [agent, config] of Object.entries(this._agentConfig)) { - table[agent] = { - preferred: config.preferred || null, - fallback: config.fallback || null, - }; - } - return table; - } -} diff --git a/backends/base.mjs b/backends/base.mjs deleted file mode 100644 index aa4c3f5..0000000 --- a/backends/base.mjs +++ /dev/null @@ -1,90 +0,0 @@ -/** - * BackendAdapter — base class for all OCP backends. - * - * Each backend adapter MUST: - * 1. Implement chatCompletion() as an async generator yielding UnifiedChunks - * 2. Implement healthCheck() - * 3. Report its costType (actual | estimated | free) - * 4. Report its tier (core | community) - * - * Adapters do NOT handle: - * - Session management (SessionManager does that) - * - Stats collection (StatsCollector does that) - * - Routing decisions (agent-router does that) - */ - -export class BackendAdapter { - /** - * @param {Object} config - Backend-specific configuration - */ - constructor(config = {}) { - this._config = config; - this._enabled = config.enabled !== false; - } - - /** @returns {string} Unique backend identifier */ - get id() { throw new Error("BackendAdapter.id must be implemented"); } - - /** @returns {string} Human-readable name */ - get displayName() { throw new Error("BackendAdapter.displayName must be implemented"); } - - /** @returns {string[]} List of supported model IDs */ - get models() { throw new Error("BackendAdapter.models must be implemented"); } - - /** @returns {"actual"|"estimated"|"free"} */ - get costType() { throw new Error("BackendAdapter.costType must be implemented"); } - - /** @returns {"core"|"community"} */ - get tier() { return "core"; } - - /** @returns {boolean} */ - get enabled() { return this._enabled; } - - /** - * Initialize the backend (called once at startup). - * Override to do async setup (e.g. validate binary, check API key). - */ - async initialize() {} - - /** - * Shutdown the backend (called on graceful shutdown). - * Override to clean up resources. - */ - async shutdown() {} - - /** - * Check if this backend is healthy and reachable. - * @returns {Promise<{ok: boolean, message: string, latencyMs: number}>} - */ - async healthCheck() { - return { ok: false, message: "healthCheck not implemented", latencyMs: 0 }; - } - - /** - * Execute a chat completion request. - * - * MUST yield UnifiedChunk objects: - * { type: "delta", content: "...", model, backend } — streaming content - * { type: "done", model, backend, usage?, cost? } — stream finished - * { type: "error", error: "...", model, backend } — error occurred - * - * @param {Object} request - * @param {string} request.model - Canonical model ID - * @param {Array} request.messages - OpenAI-format messages array - * @param {Object} [request.session] - Session info { uuid, resume } - * @param {Object} [request.options] - Additional options (temperature, etc.) - * @yields {import("../unified-chunk.mjs").UnifiedChunk} - */ - async *chatCompletion(request) { - throw new Error("BackendAdapter.chatCompletion must be implemented"); - } - - /** - * Check if this backend supports a given model. - * Default: exact match against models list. - * Override for pattern matching (e.g. "gpt-*" → openai). - */ - supportsModel(model) { - return this.models.includes(model); - } -} diff --git a/backends/claude-cli.mjs b/backends/claude-cli.mjs deleted file mode 100644 index 366b699..0000000 --- a/backends/claude-cli.mjs +++ /dev/null @@ -1,393 +0,0 @@ -/** - * ClaudeCliAdapter — OCP backend for Claude via local CLI binary. - * - * Spawns `claude -p` processes for each request. Converts CLI stdout - * into UnifiedChunk streaming protocol. - * - * costType: "estimated" (no actual cost data from CLI) - */ - -import { spawn, execFileSync } from "node:child_process"; -import { accessSync, constants } from "node:fs"; -import { join } from "node:path"; -import { BackendAdapter } from "./base.mjs"; -import { delta, done, error } from "../unified-chunk.mjs"; - -// Default models exposed by this backend -const DEFAULT_MODELS = [ - "claude-opus-4-6", - "claude-sonnet-4-6", - "claude-haiku-4-5-20251001", -]; - -// Alias → canonical model mapping -const MODEL_ALIASES = { - "claude-opus-4": "claude-opus-4-6", - "claude-haiku-4": "claude-haiku-4-5-20251001", - "claude-haiku-4-5": "claude-haiku-4-5-20251001", - "opus": "claude-opus-4-6", - "sonnet": "claude-sonnet-4-6", - "haiku": "claude-haiku-4-5-20251001", -}; - -export class ClaudeCliAdapter extends BackendAdapter { - constructor(config = {}) { - super(config); - this._claudeBin = null; - this._activeProcesses = new Set(); - this._maxConcurrent = config.maxConcurrent || 8; - this._timeout = config.timeout?.overall || 300000; - this._skipPermissions = config.skipPermissions || false; - this._allowedTools = config.allowedTools || [ - "Bash", "Read", "Write", "Edit", "Glob", "Grep", - "WebSearch", "WebFetch", "Agent", - ]; - this._systemPrompt = config.systemPrompt || ""; - this._mcpConfig = config.mcpConfig || ""; - this._modelList = config.models || DEFAULT_MODELS; - - // Timeout tiers: per-model base + per-prompt-char scaling - this._timeoutTiers = config.timeoutTiers || { - opus: { base: 150000, perPromptChar: 0.00050 }, - sonnet: { base: 120000, perPromptChar: 0.00050 }, - haiku: { base: 45000, perPromptChar: 0.00010 }, - }; - - // Estimated cost per MTok (for cost tracking) - this._estimatedCost = config.estimatedCostPerMTok || { - input: 3.0, - output: 15.0, - }; - } - - get id() { return "claude-cli"; } - get displayName() { return "Claude CLI"; } - get models() { return this._modelList; } - get costType() { return "estimated"; } - get tier() { return "core"; } - get activeProcessCount() { return this._activeProcesses.size; } - - /** - * Resolve a model ID, handling aliases. - * Returns canonical CLI model ID or the input if no alias found. - */ - resolveModel(model) { - return MODEL_ALIASES[model] || model; - } - - supportsModel(model) { - const canonical = this.resolveModel(model); - return this._modelList.includes(canonical) || model in MODEL_ALIASES; - } - - async initialize() { - this._claudeBin = this._resolveBinary(); - } - - async shutdown() { - for (const proc of this._activeProcesses) { - try { proc.kill("SIGTERM"); } catch {} - } - // Force kill after 5s - if (this._activeProcesses.size > 0) { - await new Promise(r => setTimeout(r, 5000)); - for (const proc of this._activeProcesses) { - try { proc.kill("SIGKILL"); } catch {} - } - } - } - - async healthCheck() { - const t0 = Date.now(); - try { - const env = this._cleanEnv(); - execFileSync(this._claudeBin, ["auth", "status"], { - encoding: "utf8", timeout: 10000, env, - }); - return { ok: true, message: "authenticated", latencyMs: Date.now() - t0 }; - } catch (e) { - return { - ok: false, - message: (e.stderr || e.message || "").slice(0, 200), - latencyMs: Date.now() - t0, - }; - } - } - - /** - * Execute a chat completion via Claude CLI. - * Yields UnifiedChunks as stdout data arrives. - */ - async *chatCompletion(request) { - const { model, messages, session, promptText } = request; - const cliModel = this.resolveModel(model); - - if (this._activeProcesses.size >= this._maxConcurrent) { - yield error( - `concurrency limit reached (${this._activeProcesses.size}/${this._maxConcurrent})`, - cliModel, this.id, - ); - return; - } - - const prompt = promptText || this._messagesToPrompt(messages); - const cliArgs = this._buildCliArgs(cliModel, session); - const env = this._cleanEnv(); - - const proc = spawn(this._claudeBin, cliArgs, { - env, stdio: ["pipe", "pipe", "pipe"], - }); - this._activeProcesses.add(proc); - - const t0 = Date.now(); - const firstByteTimeoutMs = this._computeFirstByteTimeout(cliModel, prompt.length); - let gotFirstByte = false; - let totalChars = 0; - let stderr = ""; - let finished = false; - - // Create a promise-based wrapper for the process - try { - yield* await new Promise((resolve, reject) => { - const chunks = []; - let streamResolve = null; - let streamDone = false; - - // We can't directly yield from inside callbacks, so we collect - // and use a different approach — return an async iterable. - // Actually, let's use a simpler pattern with a queue. - reject(new Error("USE_QUEUE_PATTERN")); - }); - } catch { - // Fall through to queue-based pattern below - } - - // Queue-based async generator pattern - const queue = []; - let queueResolve = null; - let queueDone = false; - - function enqueue(chunk) { - if (queueDone) return; - if (queueResolve) { - const r = queueResolve; - queueResolve = null; - r(chunk); - } else { - queue.push(chunk); - } - } - - function waitForChunk() { - if (queue.length > 0) return Promise.resolve(queue.shift()); - if (queueDone) return Promise.resolve(null); - return new Promise(r => { queueResolve = r; }); - } - - // Write prompt to stdin - proc.stdin.write(prompt); - proc.stdin.end(); - - proc.stdout.on("data", (d) => { - if (!gotFirstByte) { - gotFirstByte = true; - clearTimeout(firstByteTimer); - } - const text = d.toString(); - totalChars += text.length; - enqueue(delta(text, cliModel, this.id)); - }); - - proc.stderr.on("data", (d) => { stderr += d; }); - - proc.on("close", (code, signal) => { - this._activeProcesses.delete(proc); - clearTimeout(firstByteTimer); - clearTimeout(overallTimer); - const elapsed = Date.now() - t0; - - if (code !== 0) { - const msg = stderr.slice(0, 300) || `claude exit ${code}`; - enqueue(error(msg, cliModel, this.id)); - } else { - // Estimate cost based on prompt + completion chars - const estimatedInputTokens = Math.ceil(prompt.length / 4); - const estimatedOutputTokens = Math.ceil(totalChars / 4); - enqueue(done(cliModel, this.id, { - promptTokens: estimatedInputTokens, - completionTokens: estimatedOutputTokens, - }, { - value: (estimatedInputTokens * this._estimatedCost.input + - estimatedOutputTokens * this._estimatedCost.output) / 1000000, - type: "estimated", - })); - } - queueDone = true; - if (queueResolve) { - const r = queueResolve; - queueResolve = null; - r(null); - } - }); - - proc.on("error", (err) => { - this._activeProcesses.delete(proc); - clearTimeout(firstByteTimer); - clearTimeout(overallTimer); - enqueue(error(err.message, cliModel, this.id)); - queueDone = true; - if (queueResolve) { - const r = queueResolve; - queueResolve = null; - r(null); - } - }); - - // First-byte timeout - const firstByteTimer = setTimeout(() => { - if (!gotFirstByte) { - try { proc.kill("SIGTERM"); } catch {} - setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000); - enqueue(error(`first-byte timeout after ${firstByteTimeoutMs}ms`, cliModel, this.id)); - queueDone = true; - } - }, firstByteTimeoutMs); - - // Overall timeout - const overallTimer = setTimeout(() => { - try { proc.kill("SIGTERM"); } catch {} - setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000); - enqueue(error(`request timeout after ${this._timeout}ms`, cliModel, this.id)); - queueDone = true; - }, this._timeout); - - // Yield chunks as they arrive - while (true) { - const chunk = await waitForChunk(); - if (chunk === null) break; - yield chunk; - if (chunk.type === "done" || chunk.type === "error") break; - } - } - - // ── Internal helpers ────────────────────────────────────────────────── - - _resolveBinary() { - if (process.env.CLAUDE_BIN) { - try { - accessSync(process.env.CLAUDE_BIN, constants.X_OK); - return process.env.CLAUDE_BIN; - } catch { - throw new Error(`CLAUDE_BIN="${process.env.CLAUDE_BIN}" is set but not executable.`); - } - } - - const candidates = [ - "/opt/homebrew/bin/claude", - "/usr/local/bin/claude", - "/usr/bin/claude", - join(process.env.HOME || "", ".local/bin/claude"), - ]; - for (const p of candidates) { - try { accessSync(p, constants.X_OK); return p; } catch {} - } - - try { - const resolved = execFileSync("which", ["claude"], { - encoding: "utf8", timeout: 5000, - }).trim(); - if (resolved) return resolved; - } catch {} - - throw new Error( - "claude binary not found. Set CLAUDE_BIN or ensure claude is in PATH." - ); - } - - _cleanEnv() { - const env = { ...process.env }; - delete env.CLAUDECODE; - delete env.ANTHROPIC_API_KEY; - delete env.ANTHROPIC_BASE_URL; - delete env.ANTHROPIC_AUTH_TOKEN; - return env; - } - - _buildCliArgs(cliModel, session) { - const args = ["-p", "--model", cliModel, "--output-format", "text"]; - - if (session?.resume) { - args.push("--resume", session.uuid); - } else if (session?.uuid) { - args.push("--session-id", session.uuid); - } else { - args.push("--no-session-persistence"); - } - - if (this._skipPermissions) { - args.push("--dangerously-skip-permissions"); - } else if (this._allowedTools.length > 0) { - args.push("--allowedTools", ...this._allowedTools); - } - - if (this._systemPrompt) { - args.push("--append-system-prompt", this._systemPrompt); - } - - if (this._mcpConfig) { - args.push("--mcp-config", this._mcpConfig); - } - - return args; - } - - _getModelTier(cliModel) { - if (cliModel.includes("opus")) return "opus"; - if (cliModel.includes("haiku")) return "haiku"; - return "sonnet"; - } - - _computeFirstByteTimeout(cliModel, promptLength) { - const tierName = this._getModelTier(cliModel); - const tier = this._timeoutTiers[tierName] || this._timeoutTiers.sonnet; - const timeout = tier.base + Math.floor(promptLength * tier.perPromptChar); - return Math.min(timeout, Math.max(this._timeout - 5000, 10000)); - } - - /** - * Convert OpenAI-format messages to a plain text prompt for Claude CLI. - * Includes truncation guard for oversized prompts. - */ - _messagesToPrompt(messages, maxChars = 150000) { - const full = messages.map((m) => { - const text = typeof m.content === "string" ? m.content : JSON.stringify(m.content); - if (m.role === "system") return `[System] ${text}`; - if (m.role === "assistant") return `[Assistant] ${text}`; - return text; - }); - - const joined = full.join("\n\n"); - if (joined.length <= maxChars) return joined; - - // Truncation: keep system + recent messages - const system = []; - const rest = []; - for (let i = 0; i < full.length; i++) { - if (messages[i].role === "system") system.push(full[i]); - else rest.push(full[i]); - } - - const systemText = system.join("\n\n"); - const budget = maxChars - systemText.length - 200; - const kept = []; - let used = 0; - for (let i = rest.length - 1; i >= 0; i--) { - if (used + rest[i].length + 2 > budget) break; - kept.unshift(rest[i]); - used += rest[i].length + 2; - } - - const truncNote = `[System] Note: ${rest.length - kept.length} older messages were truncated to fit context limit.`; - return [systemText, truncNote, ...kept].filter(Boolean).join("\n\n"); - } -} diff --git a/model-registry.mjs b/model-registry.mjs deleted file mode 100644 index 9ef8b26..0000000 --- a/model-registry.mjs +++ /dev/null @@ -1,99 +0,0 @@ -/** - * Model Registry — Layer 1 of OCP routing. - * - * Static mapping: model ID → backend ID. - * This is infrastructure, not policy. Every model must map to exactly one backend. - * - * Non-agent callers (Cline, Aider, etc.) only send a model name — this module - * is the only thing that tells OCP where to route the request. - */ - -export class ModelRegistry { - constructor() { - /** @type {Map} */ - this._models = new Map(); - - /** @type {Map} model id → display metadata */ - this._metadata = new Map(); - } - - /** - * Register models from a backend adapter. - * Called during startup for each enabled backend. - * - * @param {import("./backends/base.mjs").BackendAdapter} adapter - */ - registerBackend(adapter) { - for (const modelId of adapter.models) { - this._models.set(modelId, { - backendId: adapter.id, - canonical: modelId, - }); - this._metadata.set(modelId, { - backendId: adapter.id, - backendName: adapter.displayName, - tier: adapter.tier, - costType: adapter.costType, - }); - } - - // Also register aliases if the adapter supports resolveModel - if (typeof adapter.resolveModel === "function") { - // We don't auto-register aliases here — they're handled by - // supportsModel() on the adapter. The registry only knows canonical IDs. - } - } - - /** - * Look up which backend serves a given model. - * - * @param {string} model — model ID (could be canonical or alias) - * @returns {{backendId: string, canonical: string} | null} - */ - resolve(model) { - // Direct lookup first - const direct = this._models.get(model); - if (direct) return direct; - - // No alias resolution here — that's the adapter's job. - // The router will ask each adapter if it supports the model. - return null; - } - - /** - * Get all registered models with metadata. - * Used by GET /v1/models endpoint. - */ - listModels() { - const result = []; - for (const [id, meta] of this._metadata) { - result.push({ id, ...meta }); - } - return result; - } - - /** - * Check if a model is known to the registry. - */ - has(model) { - return this._models.has(model); - } - - /** - * Get the backend ID for a model, or null. - */ - getBackendId(model) { - return this._models.get(model)?.backendId || null; - } - - /** - * Get all models for a specific backend. - */ - getModelsForBackend(backendId) { - const models = []; - for (const [id, entry] of this._models) { - if (entry.backendId === backendId) models.push(id); - } - return models; - } -} diff --git a/package.json b/package.json index 8debaa3..421df26 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "openclaw-claude-proxy", - "version": "3.0.1", + "version": "3.0.0", "description": "OpenAI-compatible proxy for Claude CLI — plan usage monitoring, runtime settings, prompt truncation, OCP CLI", "type": "module", "bin": { @@ -24,6 +24,6 @@ }, "repository": { "type": "git", - "url": "https://github.com/dtzp555-max/ocp" + "url": "https://github.com/dtzp555-max/openclaw-claude-proxy" } } diff --git a/server.mjs b/server.mjs index 1291b92..354ebf1 100644 --- a/server.mjs +++ b/server.mjs @@ -1,17 +1,11 @@ #!/usr/bin/env node /** - * OCP (OpenClaw Control Plane) v4.0-alpha + * openclaw-claude-proxy v2.5.0 — OpenAI-compatible proxy for Claude CLI * - * Cost-control and model-routing layer for AI agents. - * Currently supports Claude CLI backend; architecture supports multiple backends. + * Translates OpenAI chat/completions requests into `claude -p` CLI calls, + * letting you use your Claude Pro/Max subscription as an OpenClaw model provider. * - * v4.0-alpha: - * - Internal modular architecture: BackendAdapter, ModelRegistry, AgentRouter, - * SessionManager, StatsCollector - * - External API unchanged from v2.5.0 - * - Claude CLI backend extracted to ClaudeCliAdapter - * - * v2.5.0 (legacy notes): + * v2.5.0: * - Sliding-window circuit breaker: uses time-windowed failure rate instead of * consecutive-count, preventing multi-agent burst scenarios from tripping the * breaker too aggressively. Half-open state allows configurable probe requests. @@ -53,54 +47,6 @@ import { dirname, join } from "node:path"; const __dirname = dirname(fileURLToPath(import.meta.url)); const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8")); -// ── v4 Modular Architecture (Step 1: load + initialize, hot path unchanged) ── -import { ClaudeCliAdapter } from "./backends/claude-cli.mjs"; -import { ModelRegistry } from "./model-registry.mjs"; -import { AgentRouter } from "./agent-router.mjs"; -import { SessionManager as SessionMgr } from "./session-manager.mjs"; -import { StatsCollector } from "./stats-collector.mjs"; - -// These modules are initialized at startup but NOT yet wired into the request -// path. Step 2 will replace the legacy callClaude/callClaudeStreaming with -// adapter-based calls. For now, they run in parallel for validation. -let _v4Backend = null; -let _v4Registry = null; -let _v4Router = null; -let _v4Sessions = null; -let _v4Stats = null; - -async function initV4Modules() { - try { - _v4Stats = new StatsCollector(); - _v4Sessions = new SessionMgr({ ttl: SESSION_TTL }); - _v4Registry = new ModelRegistry(); - - _v4Backend = new ClaudeCliAdapter({ - maxConcurrent: MAX_CONCURRENT, - timeout: { overall: TIMEOUT }, - skipPermissions: SKIP_PERMISSIONS, - allowedTools: ALLOWED_TOOLS, - systemPrompt: SYSTEM_PROMPT, - mcpConfig: MCP_CONFIG, - timeoutTiers: MODEL_TIMEOUT_TIERS, - }); - await _v4Backend.initialize(); - _v4Registry.registerBackend(_v4Backend); - - const backends = new Map([["claude-cli", _v4Backend]]); - _v4Router = new AgentRouter({ - registry: _v4Registry, - backends, - agentConfig: { default: { preferred: "claude-cli", fallback: null } }, - defaultBackend: "claude-cli", - }); - - console.log(`[v4] Modules initialized: backend=${_v4Backend.id}, models=${_v4Backend.models.length}, registry=${_v4Registry.listModels().length} models`); - } catch (err) { - console.error(`[v4] Module init failed (non-fatal, legacy path active): ${err.message}`); - } -} - // ── Resolve claude binary ─────────────────────────────────────────────── // Priority: CLAUDE_BIN env > well-known paths > which lookup // Fail-fast if not found — never start with an unresolvable binary. @@ -740,57 +686,20 @@ function completionResponse(res, id, model, content) { let usageCache = { data: null, fetchedAt: 0 }; const USAGE_CACHE_TTL = 300000; // 5 min -function getOAuthCredentials() { +function getOAuthToken() { try { const raw = execFileSync("security", [ "find-generic-password", "-s", "Claude Code-credentials", "-w" ], { encoding: "utf8", timeout: 5000 }).trim(); - return JSON.parse(raw)?.claudeAiOauth || null; + const creds = JSON.parse(raw); + return creds?.claudeAiOauth?.accessToken || null; } catch { return null; } } -async function refreshOAuthToken(_creds) { - // Trigger Claude Code to refresh the token by running a minimal command. - // Claude Code automatically refreshes expired tokens on startup. - try { - execFileSync("claude", ["-p", "--model", "haiku", "--max-turns", "1", "hi"], { - encoding: "utf8", - timeout: 30000, - stdio: ["pipe", "pipe", "pipe"], - }); - // Re-read the now-refreshed token from keychain - const refreshed = getOAuthCredentials(); - if (refreshed?.accessToken && refreshed.expiresAt > Date.now()) { - console.log("[ocp] OAuth token refreshed via Claude CLI"); - return refreshed.accessToken; - } - return null; - } catch (e) { - console.error("[ocp] OAuth refresh via CLI failed:", e.message); - return null; - } -} - -async function getOAuthToken() { - const creds = getOAuthCredentials(); - if (!creds?.accessToken) return null; - - // Check if token is expired or about to expire (5 min buffer) - const expiresAt = creds.expiresAt || 0; - if (expiresAt > Date.now() + 300000) { - return creds.accessToken; - } - - // Token expired or expiring soon — try refresh - console.log("[ocp] OAuth token expired, attempting refresh..."); - const newToken = await refreshOAuthToken(creds); - return newToken || creds.accessToken; // fallback to old token (might still work briefly) -} - async function fetchUsageFromApi() { - const token = await getOAuthToken(); + const token = getOAuthToken(); if (!token) { return { error: "No OAuth token found in keychain" }; } @@ -1269,36 +1178,7 @@ const server = createServer(async (req, res) => { return handleSettings(req, res); } - // GET /backends — v4 backend status (new endpoint) - if (req.url === "/backends" && req.method === "GET") { - if (!_v4Backend) { - return jsonResponse(res, 503, { error: "v4 modules not initialized" }); - } - const health = await _v4Backend.healthCheck(); - return jsonResponse(res, 200, { - backends: [{ - id: _v4Backend.id, - displayName: _v4Backend.displayName, - tier: _v4Backend.tier, - costType: _v4Backend.costType, - enabled: _v4Backend.enabled, - models: _v4Backend.models, - activeProcesses: _v4Backend.activeProcessCount, - health, - }], - routing: _v4Router?.getRoutingTable() || {}, - }); - } - - // GET /routing — v4 agent routing table (new endpoint) - if (req.url === "/routing" && req.method === "GET") { - return jsonResponse(res, 200, { - table: _v4Router?.getRoutingTable() || {}, - registry: _v4Registry?.listModels() || [], - }); - } - - jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET /usage, GET /status, GET /logs, GET|PATCH /settings, GET|DELETE /sessions, GET /backends, GET /routing" }); + jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET /usage, GET /status, GET /logs, GET|PATCH /settings, GET|DELETE /sessions" }); }); @@ -1319,10 +1199,6 @@ function gracefulShutdown(signal) { clearInterval(sessionCleanupInterval); clearInterval(authCheckInterval); - // 2b. Shutdown v4 modules - if (_v4Sessions) _v4Sessions.shutdown(); - if (_v4Backend) _v4Backend.shutdown().catch(() => {}); - // 3. Kill all active child processes for (const proc of activeProcesses) { try { proc.kill("SIGTERM"); } catch {} @@ -1359,11 +1235,8 @@ process.on("SIGTERM", () => gracefulShutdown("SIGTERM")); process.on("SIGINT", () => gracefulShutdown("SIGINT")); // ── Start ─────────────────────────────────────────────────────────────── -// Initialize v4 modules before listening -initV4Modules().catch(err => console.error(`[v4] init error: ${err.message}`)); - server.listen(PORT, "127.0.0.1", () => { - console.log(`OCP (OpenClaw Control Plane) v${VERSION} listening on http://127.0.0.1:${PORT}`); + console.log(`openclaw-claude-proxy v${VERSION} listening on http://127.0.0.1:${PORT}`); console.log(`Architecture: on-demand spawning (no pool)`); console.log(`Models: ${MODELS.map((m) => m.id).join(", ")}`); console.log(`Claude binary: ${CLAUDE}`); diff --git a/session-manager.mjs b/session-manager.mjs deleted file mode 100644 index 68e3458..0000000 --- a/session-manager.mjs +++ /dev/null @@ -1,123 +0,0 @@ -/** - * SessionManager — manages conversation sessions across requests. - * - * Maps caller-provided conversation IDs to backend-specific session state. - * For Claude CLI, this means mapping to CLI session UUIDs for --resume. - */ - -import { randomUUID } from "node:crypto"; - -export class SessionManager { - /** - * @param {Object} options - * @param {number} [options.ttl=3600000] — Session TTL in ms (default 1h) - */ - constructor({ ttl = 3600000 } = {}) { - this._ttl = ttl; - /** @type {Map} */ - this._sessions = new Map(); - - // Cleanup expired sessions every 60s - this._cleanupInterval = setInterval(() => this._cleanup(), 60000); - } - - get ttl() { return this._ttl; } - set ttl(val) { this._ttl = val; } - get size() { return this._sessions.size; } - - /** - * Get or create a session for a conversation. - * - * @param {string|null} conversationId - * @param {string} model — model being used - * @param {string} [agent] — agent identity - * @returns {{session: {uuid: string, resume: boolean}, isNew: boolean, isOneOff: boolean}} - */ - resolve(conversationId, model, agent = "default") { - if (!conversationId) { - return { session: null, isNew: false, isOneOff: true }; - } - - if (this._sessions.has(conversationId)) { - const entry = this._sessions.get(conversationId); - entry.lastUsed = Date.now(); - entry.messageCount++; - return { - session: { uuid: entry.uuid, resume: true }, - isNew: false, - isOneOff: false, - }; - } - - // New session - const uuid = randomUUID(); - this._sessions.set(conversationId, { - uuid, - model, - agent, - messageCount: 1, - lastUsed: Date.now(), - createdAt: Date.now(), - }); - - return { - session: { uuid, resume: false }, - isNew: true, - isOneOff: false, - }; - } - - /** - * Remove a session (e.g. on resume failure). - */ - delete(conversationId) { - this._sessions.delete(conversationId); - } - - /** - * Clear all sessions. - * @returns {number} Number of sessions cleared - */ - clear() { - const count = this._sessions.size; - this._sessions.clear(); - return count; - } - - /** - * List all active sessions (for /sessions endpoint). - */ - list() { - const result = []; - for (const [id, s] of this._sessions) { - result.push({ - id, - uuid: s.uuid, - model: s.model, - agent: s.agent, - messages: s.messageCount, - lastUsed: new Date(s.lastUsed).toISOString(), - idleMs: Date.now() - s.lastUsed, - }); - } - return result; - } - - /** - * Shutdown — clear interval. - */ - shutdown() { - clearInterval(this._cleanupInterval); - } - - // ── Internal ────────────────────────────────────────────────────────── - - _cleanup() { - const now = Date.now(); - for (const [id, s] of this._sessions) { - if (now - s.lastUsed > this._ttl) { - this._sessions.delete(id); - } - } - } -} diff --git a/stats-collector.mjs b/stats-collector.mjs deleted file mode 100644 index 0cee712..0000000 --- a/stats-collector.mjs +++ /dev/null @@ -1,186 +0,0 @@ -/** - * StatsCollector — tracks request metrics per-model, per-backend, per-agent. - * - * Replaces the flat stats + modelStats from server.mjs with structured, - * multi-dimensional tracking. Backward compatible: still exposes the same - * data for /health, /usage, /status endpoints. - */ - -export class StatsCollector { - constructor() { - // Global counters (backward compatible) - this.totalRequests = 0; - this.activeRequests = 0; - this.errors = 0; - this.timeouts = 0; - this.sessionHits = 0; - this.sessionMisses = 0; - this.oneOffRequests = 0; - - // Recent errors ring buffer - this._recentErrors = []; - this._maxErrors = 20; - - /** @type {Map} cliModel → stats */ - this._modelStats = new Map(); - - /** @type {Map} backendId → stats */ - this._backendStats = new Map(); - - /** @type {Map} agentName → stats */ - this._agentStats = new Map(); - } - - /** - * Record a new request starting. - */ - recordRequest(model, backend, agent, promptChars) { - this.totalRequests++; - this.activeRequests++; - this._getOrCreate(this._modelStats, model).recordRequest(promptChars); - this._getOrCreate(this._backendStats, backend).recordRequest(promptChars); - this._getOrCreate(this._agentStats, agent).recordRequest(promptChars); - } - - /** - * Record a successful completion. - */ - recordSuccess(model, backend, agent, elapsedMs, cost = null) { - this.activeRequests = Math.max(0, this.activeRequests - 1); - this._getOrCreate(this._modelStats, model).recordSuccess(elapsedMs, cost); - this._getOrCreate(this._backendStats, backend).recordSuccess(elapsedMs, cost); - this._getOrCreate(this._agentStats, agent).recordSuccess(elapsedMs, cost); - } - - /** - * Record an error. - */ - recordError(model, backend, agent, isTimeout, message = "") { - this.activeRequests = Math.max(0, this.activeRequests - 1); - this.errors++; - if (isTimeout) this.timeouts++; - - this._getOrCreate(this._modelStats, model).recordError(isTimeout); - this._getOrCreate(this._backendStats, backend).recordError(isTimeout); - this._getOrCreate(this._agentStats, agent).recordError(isTimeout); - - this._recentErrors.push({ - time: new Date().toISOString(), - message: String(message).slice(0, 200), - model, backend, agent, - }); - if (this._recentErrors.length > this._maxErrors) this._recentErrors.shift(); - } - - /** - * Record session hit/miss/one-off. - */ - recordSessionHit() { this.sessionHits++; } - recordSessionMiss() { this.sessionMisses++; } - recordOneOff() { this.oneOffRequests++; } - - /** - * Get per-model stats snapshot (backward compatible with v2). - */ - getModelStatsSnapshot() { - return this._snapshot(this._modelStats); - } - - /** - * Get per-backend stats snapshot. - */ - getBackendStatsSnapshot() { - return this._snapshot(this._backendStats); - } - - /** - * Get per-agent stats snapshot. - */ - getAgentStatsSnapshot() { - return this._snapshot(this._agentStats); - } - - /** - * Get recent errors. - */ - getRecentErrors(n = 5) { - return this._recentErrors.slice(-n); - } - - /** - * Get global stats (backward compatible). - */ - getGlobalStats() { - return { - totalRequests: this.totalRequests, - activeRequests: this.activeRequests, - errors: this.errors, - timeouts: this.timeouts, - sessionHits: this.sessionHits, - sessionMisses: this.sessionMisses, - oneOffRequests: this.oneOffRequests, - }; - } - - // ── Internal ────────────────────────────────────────────────────────── - - _getOrCreate(map, key) { - if (!map.has(key)) map.set(key, new ModelStats()); - return map.get(key); - } - - _snapshot(map) { - const result = {}; - for (const [key, stats] of map) { - result[key] = stats.toJSON(); - } - return result; - } -} - -/** Per-entity stats tracker */ -class ModelStats { - constructor() { - this.requests = 0; - this.successes = 0; - this.errors = 0; - this.timeouts = 0; - this.totalElapsed = 0; - this.maxElapsed = 0; - this.totalPromptChars = 0; - this.maxPromptChars = 0; - this.totalCost = 0; // accumulated estimated/actual cost - } - - recordRequest(promptChars) { - this.requests++; - this.totalPromptChars += promptChars; - if (promptChars > this.maxPromptChars) this.maxPromptChars = promptChars; - } - - recordSuccess(elapsedMs, cost = null) { - this.successes++; - this.totalElapsed += elapsedMs; - if (elapsedMs > this.maxElapsed) this.maxElapsed = elapsedMs; - if (cost?.value) this.totalCost += cost.value; - } - - recordError(isTimeout) { - this.errors++; - if (isTimeout) this.timeouts++; - } - - toJSON() { - return { - requests: this.requests, - successes: this.successes, - errors: this.errors, - timeouts: this.timeouts, - avgElapsed: this.successes > 0 ? Math.round(this.totalElapsed / this.successes) : 0, - maxElapsed: this.maxElapsed, - avgPromptChars: this.requests > 0 ? Math.round(this.totalPromptChars / this.requests) : 0, - maxPromptChars: this.maxPromptChars, - totalCost: Math.round(this.totalCost * 10000) / 10000, // 4 decimal places - }; - } -} diff --git a/unified-chunk.mjs b/unified-chunk.mjs deleted file mode 100644 index 00db874..0000000 --- a/unified-chunk.mjs +++ /dev/null @@ -1,42 +0,0 @@ -/** - * UnifiedChunk — the internal streaming protocol for OCP. - * - * All backend adapters MUST convert their native output into this format. - * server.mjs only handles UnifiedChunks → OpenAI SSE conversion. - */ - -/** - * @typedef {Object} UnifiedChunk - * @property {"delta"|"done"|"error"} type - * @property {string} [content] — text content (for "delta") - * @property {string} model — canonical model id - * @property {string} backend — backend id (e.g. "claude-cli") - * @property {Object} [usage] — token usage (for "done") - * @property {number} [usage.promptTokens] - * @property {number} [usage.completionTokens] - * @property {Object} [cost] — cost info (for "done") - * @property {number} [cost.value] - * @property {"actual"|"estimated"|"free"} [cost.type] - * @property {string} [error] — error message (for "error") - */ - -/** - * Create a delta chunk (streaming content) - */ -export function delta(content, model, backend) { - return { type: "delta", content, model, backend }; -} - -/** - * Create a done chunk (stream finished) - */ -export function done(model, backend, usage = null, cost = null) { - return { type: "done", model, backend, ...(usage && { usage }), ...(cost && { cost }) }; -} - -/** - * Create an error chunk - */ -export function error(message, model, backend) { - return { type: "error", error: message, model, backend }; -}