mirror of
https://github.com/dtzp555-max/ocp.git
synced 2026-07-21 21:15:09 +00:00
revert: rollback Phase 1 and OAuth fix — broke plan usage functionality
Revertsb5be1c0and1f1b387. These changes broke the OAuth token authentication for fetching plan usage data from Anthropic API, causing /ocp usage to show 0% with "unknown" reset times. Root cause: code changes inadvertently altered the fetchUsageFromApi flow, breaking the existing working OAuth x-api-key authentication. This restores server.mjs to the last known working state (f6f4f68). Phase 1 modular files removed — will be re-implemented in a separate branch with proper isolation testing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.6
-439
@@ -1,439 +0,0 @@
|
|||||||
# OCP v4.0 开发计划 — Agent 成本控制器
|
|
||||||
|
|
||||||
## 背景
|
|
||||||
|
|
||||||
OCP 当前架构：`Tool → OCP (localhost:3456) → Claude CLI → Anthropic`
|
|
||||||
|
|
||||||
核心风险：Claude CLI proxy 属于灰色地带，Anthropic 随时可能封堵。
|
|
||||||
|
|
||||||
核心资产（不依赖 CLI 的部分）：
|
|
||||||
- OpenAI 兼容 API 层
|
|
||||||
- Session 管理（会话隔离、TTL、复用）
|
|
||||||
- Usage 监控（plan limits、per-model stats、实时查询）
|
|
||||||
- 运行时调参（timeout、并发、prompt 限制）
|
|
||||||
- `/ocp` 命令集（Telegram/Discord/CLI）
|
|
||||||
- 多实例并发控制
|
|
||||||
|
|
||||||
## 定位
|
|
||||||
|
|
||||||
> OCP is a cost-control and model-routing layer for AI agents, optimized for subscription-based usage.
|
|
||||||
|
|
||||||
**不是 proxy，不是 gateway，不是 LiteLLM clone。**
|
|
||||||
**是：让 Agent 用模型这件事变得可控 — 尤其是成本。**
|
|
||||||
|
|
||||||
核心差异点：OCP 有 agent 上下文，知道是谁在请求、在做什么任务、已经花了多少。LiteLLM 只看到 HTTP 请求。
|
|
||||||
|
|
||||||
## 目标
|
|
||||||
|
|
||||||
将 OCP 从 "Claude CLI proxy" 重构为 "Agent 成本控制器"，支持可扩展的多后端，保留全部管理能力。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ⚠️ 第零原则：不动现有功能
|
|
||||||
|
|
||||||
OCP 作为 Claude CLI proxy 是当前的核心价值和用户吸引力。所有 v4 重构必须遵守：
|
|
||||||
|
|
||||||
- **外部 API 不变** — `/v1/chat/completions`、`/usage`、`/health`、`/settings` 签名不变
|
|
||||||
- **行为不变** — 现有用户升级后零感知，所有 session/timeout/并发逻辑保持原样
|
|
||||||
- **逐步替换** — 先抽模块，跑稳了再切；任何时候回归测试不过就回滚
|
|
||||||
- **新 backend 是增量** — 加 OpenAI/Ollama 是新能力，不改动 Claude CLI 路径
|
|
||||||
|
|
||||||
Phase 1 的定义就是：**拆完之后跟没拆一样。** 拆出问题就说明拆法有误。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 架构关键决策（三轮 Design Review 定稿）
|
|
||||||
|
|
||||||
### 决策 1: 路由分两层 — Model Registry + Agent Policy
|
|
||||||
|
|
||||||
**三轮讨论结论：**
|
|
||||||
- 第一轮建议"删掉 model routing" → 错误，非 agent 客户端（Cline/Aider）只带 model 名
|
|
||||||
- 反驳：model-to-backend mapping 是基础设施，不是 routing rule
|
|
||||||
- 校准：物理拆成两个模块，不要混在一起
|
|
||||||
|
|
||||||
**最终设计 — 两层分离：**
|
|
||||||
|
|
||||||
```
|
|
||||||
🧱 Layer 1: Model Registry（基础设施层）
|
|
||||||
model-registry.mjs
|
|
||||||
静态映射，无条件，必须存在
|
|
||||||
"claude-sonnet-4-6" → claude-cli backend
|
|
||||||
"gpt-4o" → openai backend
|
|
||||||
"qwen3" → ollama backend
|
|
||||||
|
|
||||||
🧠 Layer 2: Agent Policy（策略层）
|
|
||||||
agent-router.mjs
|
|
||||||
动态决策，可覆盖，可选
|
|
||||||
agent "coder" → override to claude-cli
|
|
||||||
agent "interpreter" → override to ollama
|
|
||||||
```
|
|
||||||
|
|
||||||
**路由决策顺序（写死）：**
|
|
||||||
```
|
|
||||||
1. Agent policy override → 有 agent identity 且有配置？用它
|
|
||||||
2. Model registry → 模型属于哪个 backend？路由过去
|
|
||||||
3. Default backend → 都没匹配？全局默认
|
|
||||||
4. Fallback chain → 上面选的挂了？降级
|
|
||||||
```
|
|
||||||
|
|
||||||
### 决策 2: 统一 Streaming 协议
|
|
||||||
|
|
||||||
**所有 adapter 必须输出统一的内部 chunk 格式：**
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
{
|
|
||||||
type: "delta" | "done" | "error",
|
|
||||||
content?: string,
|
|
||||||
model: string,
|
|
||||||
backend: string,
|
|
||||||
usage?: { promptTokens: number, completionTokens: number }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
server.mjs 只处理统一协议 → 转换为 OpenAI SSE 输出。
|
|
||||||
|
|
||||||
### 决策 3: 成本标记 — actual / estimated / free
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
{
|
|
||||||
cost: {
|
|
||||||
value: 0.23,
|
|
||||||
type: "actual" | "estimated" | "free"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
| Backend | cost type | 来源 |
|
|
||||||
|---------|-----------|------|
|
|
||||||
| OpenAI API | actual | API 返回 usage |
|
|
||||||
| Claude CLI | estimated | prompt chars × 估价 |
|
|
||||||
| Ollama | free | 本地运行 |
|
|
||||||
|
|
||||||
`/ocp cost` 输出明确标注类型，不误导用户。
|
|
||||||
|
|
||||||
### 决策 4: Fallback — 只在 First-Byte 前触发
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const allowFallback = !hasStartedStreaming;
|
|
||||||
```
|
|
||||||
|
|
||||||
Phase 3 严格限制：已输出 ≥1 个 delta → 不 fallback，直接报错。
|
|
||||||
flag 预留，未来扩展 streaming fallback 只改判断条件。
|
|
||||||
|
|
||||||
### 决策 5: Agent Identity — 结构预留，实现从简
|
|
||||||
|
|
||||||
**优先级：**
|
|
||||||
```
|
|
||||||
1. x-ocp-agent header → OpenClaw 自动附加
|
|
||||||
2. Session metadata → 首次请求绑定，后续复用
|
|
||||||
3. (future) Inferred → 按使用模式推断（预留接口）
|
|
||||||
4. default → 兜底
|
|
||||||
```
|
|
||||||
|
|
||||||
**现在只实现 1 + 2 + 4。** 但接口预留 inferred 扩展点。
|
|
||||||
|
|
||||||
为什么重要：未来 Cline coding mode 和 chat mode 可能需要不同路由，如果没有 identity hook 以后很难加。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Backend Adapter 设计
|
|
||||||
|
|
||||||
### 接口
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
class BackendAdapter {
|
|
||||||
get id() // "claude-cli" | "openai-api" | "ollama"
|
|
||||||
get displayName()
|
|
||||||
get models()
|
|
||||||
get costType() // "actual" | "estimated" | "free"
|
|
||||||
get tier() // "core" | "community"
|
|
||||||
|
|
||||||
async healthCheck() → { ok, message, latencyMs }
|
|
||||||
async *chatCompletion(request) → AsyncGenerator<UnifiedChunk>
|
|
||||||
async initialize(config)
|
|
||||||
async shutdown()
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backend Tier 体系
|
|
||||||
|
|
||||||
OCP provides a pluggable backend adapter interface. Core adapters are maintained officially; community adapters are contributed by users.
|
|
||||||
|
|
||||||
```
|
|
||||||
/ocp backends 输出示例：
|
|
||||||
|
|
||||||
claude-cli ✓ healthy 12ms (core)
|
|
||||||
openai ✓ healthy 89ms (core)
|
|
||||||
ollama ✓ healthy 5ms (core)
|
|
||||||
deepseek ✓ healthy 120ms (community)
|
|
||||||
```
|
|
||||||
|
|
||||||
架构不限制 backend 数量。我们主动维护 3 个 core adapter（Claude CLI、OpenAI 兼容、Ollama），接口开放，社区可贡献更多。
|
|
||||||
|
|
||||||
OpenAI 兼容的服务（DeepSeek、Groq、Together 等）直接复用 `OpenAiApiAdapter`，只需换 baseUrl + apiKey，不算新 adapter。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 分阶段实施
|
|
||||||
|
|
||||||
### Phase 1: 后端抽象 + Model Registry（v4.0-alpha）
|
|
||||||
**目标：** 不改变外部行为，内部重构
|
|
||||||
|
|
||||||
- [ ] 定义 `BackendAdapter` 接口 + `UnifiedChunk` 类型
|
|
||||||
- [ ] `model-registry.mjs` — 模型到 backend 的静态映射
|
|
||||||
- `getBackendForModel(model) → backendId`
|
|
||||||
- [ ] `agent-router.mjs` — 路由决策引擎
|
|
||||||
- `resolveBackend({ agent, model }) → backendId`
|
|
||||||
- 实现决策顺序：agent override → model registry → default → fallback
|
|
||||||
- [ ] 将 Claude CLI 逻辑封装为 `ClaudeCliAdapter`
|
|
||||||
- stdout → UnifiedChunk 转换
|
|
||||||
- costType = "estimated"
|
|
||||||
- [ ] 抽取 `SessionManager`（从 server.mjs）
|
|
||||||
- [ ] 抽取 `StatsCollector`（per-agent, per-backend）
|
|
||||||
- [ ] Config v4 格式（带 version + validation）：
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"version": "4.0",
|
|
||||||
"backends": {
|
|
||||||
"claude-cli": {
|
|
||||||
"type": "claude-cli",
|
|
||||||
"tier": "core",
|
|
||||||
"enabled": true,
|
|
||||||
"models": ["claude-sonnet-4-6", "claude-opus-4-6", "claude-haiku-4-5"],
|
|
||||||
"maxConcurrent": 4,
|
|
||||||
"timeout": { "firstByte": 120000, "overall": 300000 },
|
|
||||||
"estimatedCostPerMTok": { "input": 3.0, "output": 15.0 }
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"agents": {
|
|
||||||
"default": { "preferred": "claude-cli", "fallback": null }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] 启动时 config validation
|
|
||||||
- [ ] `/ocp backends` 命令
|
|
||||||
- [ ] 所有现有测试通过，行为不变
|
|
||||||
|
|
||||||
**验收标准：** 现有用户升级后零感知变化
|
|
||||||
|
|
||||||
### Phase 2: OpenAI API 后端（v4.0-beta）
|
|
||||||
**目标：** 第二个 backend，验证 adapter 抽象
|
|
||||||
|
|
||||||
- [ ] `OpenAiApiAdapter`
|
|
||||||
- SSE → UnifiedChunk
|
|
||||||
- costType = "actual"
|
|
||||||
- 支持任何 OpenAI 兼容端点（baseUrl + apiKey）
|
|
||||||
- [ ] Model registry 自动注册新 backend 的 models
|
|
||||||
- [ ] Agent routing 生效：
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"agents": {
|
|
||||||
"default": { "preferred": "claude-cli", "fallback": "openai" },
|
|
||||||
"interpreter": { "preferred": "openai/gpt-4o-mini", "fallback": null }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] Non-streaming fallback（`allowFallback = !hasStartedStreaming`）
|
|
||||||
- [ ] Stats 按 backend + agent 分组
|
|
||||||
|
|
||||||
**验收标准：** claude-cli 挂了 → default agent 自动 fallback 到 openai
|
|
||||||
|
|
||||||
### Phase 3: Ollama + Agent Routing 稳定化（v4.0-rc）
|
|
||||||
**目标：** 摆脱付费依赖，agent routing 跑稳
|
|
||||||
|
|
||||||
- [ ] `OllamaAdapter`
|
|
||||||
- 自动发现模型
|
|
||||||
- costType = "free"
|
|
||||||
- [ ] Agent routing 完整覆盖所有 agent：
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"agents": {
|
|
||||||
"main": { "preferred": "claude-cli/opus", "fallback": "openai" },
|
|
||||||
"tech_geek": { "preferred": "claude-cli/sonnet", "fallback": "ollama/qwen3" },
|
|
||||||
"interpreter": { "preferred": "ollama/qwen3", "fallback": null }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] `/ocp routing` — 显示每个 agent 当前走哪个 backend
|
|
||||||
- [ ] 端到端测试：Claude CLI 下线 → agent 自动 fallback → 恢复后自动回切
|
|
||||||
|
|
||||||
**验收标准：** 三个 backend 同时运行，每个 agent 走不同路径，稳定无错
|
|
||||||
|
|
||||||
### Phase 4: Budget + Cost Control（v4.1）
|
|
||||||
**目标：** 核心差异化 — agent-aware 成本控制
|
|
||||||
|
|
||||||
**前置条件：** Phase 3 routing 稳定（先确定流量怎么走 → 再控制花多少钱）
|
|
||||||
|
|
||||||
**安全原则：** 没有显式配置 budget 的 agent → 行为等同于 v3（不限制、不降级、不告警）。Budget 是 opt-in，不是 opt-out。
|
|
||||||
|
|
||||||
- [ ] Per-agent 预算：
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"agents": {
|
|
||||||
"main": {
|
|
||||||
"preferred": "claude-cli/opus",
|
|
||||||
"fallback": "openai",
|
|
||||||
"dailyBudget": { "limit": 3.00, "onExhausted": "downgrade" }
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"budget": {
|
|
||||||
"global": { "daily": 10.00, "onExhausted": "warn" }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] 超预算行为：downgrade | block | warn
|
|
||||||
- [ ] `/ocp cost` — per-agent 成本（标注 actual/estimated/free）
|
|
||||||
- [ ] Telegram 告警（预算 80%、backend 连续失败）
|
|
||||||
|
|
||||||
### Phase 5: Anthropic 官方 API（v4.2，等时机）
|
|
||||||
- [ ] `AnthropicApiAdapter`（官方 SDK）
|
|
||||||
- [ ] 迁移指南：一行配置切换
|
|
||||||
- [ ] costType = "actual"
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 文件结构（目标）
|
|
||||||
|
|
||||||
```
|
|
||||||
claude-proxy/
|
|
||||||
├── server.mjs → HTTP 层（统一 chunk → SSE 输出）
|
|
||||||
├── model-registry.mjs → 模型注册表（model → backend 映射）
|
|
||||||
├── agent-router.mjs → Agent 路由引擎（策略层）
|
|
||||||
├── session-manager.mjs → 会话管理
|
|
||||||
├── stats-collector.mjs → 统计 + 成本追踪
|
|
||||||
├── config.mjs → 配置加载 + 版本验证
|
|
||||||
├── unified-chunk.mjs → UnifiedChunk 类型定义
|
|
||||||
├── backends/
|
|
||||||
│ ├── base.mjs → BackendAdapter 接口
|
|
||||||
│ ├── claude-cli.mjs → Claude CLI（core）
|
|
||||||
│ ├── openai-api.mjs → OpenAI 兼容（core）
|
|
||||||
│ └── ollama.mjs → Ollama 本地（core）
|
|
||||||
├── setup.mjs → 安装向导
|
|
||||||
└── ocp-plugin/ → Gateway 命令插件
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 配置完整示例（目标状态）
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"version": "4.0",
|
|
||||||
"port": 3456,
|
|
||||||
|
|
||||||
"backends": {
|
|
||||||
"claude-cli": {
|
|
||||||
"type": "claude-cli",
|
|
||||||
"tier": "core",
|
|
||||||
"enabled": true,
|
|
||||||
"models": ["claude-sonnet-4-6", "claude-opus-4-6"],
|
|
||||||
"maxConcurrent": 4,
|
|
||||||
"timeout": { "firstByte": 120000, "overall": 300000 },
|
|
||||||
"estimatedCostPerMTok": { "input": 3.0, "output": 15.0 }
|
|
||||||
},
|
|
||||||
"openai": {
|
|
||||||
"type": "openai-api",
|
|
||||||
"tier": "core",
|
|
||||||
"enabled": true,
|
|
||||||
"baseUrl": "https://openrouter.ai/api/v1",
|
|
||||||
"apiKey": "${OPENROUTER_API_KEY}",
|
|
||||||
"models": ["anthropic/claude-3.5-sonnet", "google/gemini-2.0-flash"],
|
|
||||||
"maxConcurrent": 10,
|
|
||||||
"timeout": { "firstByte": 30000, "overall": 120000 }
|
|
||||||
},
|
|
||||||
"ollama": {
|
|
||||||
"type": "ollama",
|
|
||||||
"tier": "core",
|
|
||||||
"enabled": true,
|
|
||||||
"baseUrl": "http://localhost:11434",
|
|
||||||
"models": "auto"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
"agents": {
|
|
||||||
"default": {
|
|
||||||
"preferred": "claude-cli",
|
|
||||||
"fallback": "openai"
|
|
||||||
},
|
|
||||||
"main": {
|
|
||||||
"preferred": "claude-cli/claude-opus-4-6",
|
|
||||||
"fallback": "openai",
|
|
||||||
"dailyBudget": { "limit": 3.00, "onExhausted": "downgrade" }
|
|
||||||
},
|
|
||||||
"tech_geek": {
|
|
||||||
"preferred": "claude-cli/claude-sonnet-4-6",
|
|
||||||
"fallback": "ollama/qwen3",
|
|
||||||
"dailyBudget": { "limit": 2.00, "onExhausted": "fallback" }
|
|
||||||
},
|
|
||||||
"interpreter": {
|
|
||||||
"preferred": "ollama/qwen3",
|
|
||||||
"fallback": null
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
"fallback": {
|
|
||||||
"onlyBeforeFirstByte": true,
|
|
||||||
"triggers": ["timeout", "rate-limit", "connection-error"],
|
|
||||||
"maxRetries": 1
|
|
||||||
},
|
|
||||||
|
|
||||||
"budget": {
|
|
||||||
"global": { "daily": 10.00, "onExhausted": "warn" },
|
|
||||||
"alerts": { "threshold": 0.8, "channel": "telegram" }
|
|
||||||
},
|
|
||||||
|
|
||||||
"sessions": { "ttl": 3600000, "maxPromptChars": 150000 },
|
|
||||||
"monitoring": { "retainHours": 72 }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 风险与应对
|
|
||||||
|
|
||||||
| 风险 | 应对 |
|
|
||||||
|------|------|
|
|
||||||
| Claude CLI 被封 | 切换 backend，上层不变 |
|
|
||||||
| Anthropic 出订阅 API | 实现 anthropic-api adapter，成本控制层继续有价值 |
|
|
||||||
| LiteLLM 竞争 | 不竞争。OCP 做 agent-aware 成本控制，赛道不同 |
|
|
||||||
| 成本数据不准 | cost type 标记 actual/estimated/free |
|
|
||||||
| Streaming fallback 拼接 | Phase 3 只做 non-streaming fallback |
|
|
||||||
| 过度工程 | 只维护 3 个 core backend，phase 严格拆分 |
|
|
||||||
|
|
||||||
## 优先级
|
|
||||||
|
|
||||||
| Phase | 时间 | 内容 | 交付物 |
|
|
||||||
|-------|------|------|--------|
|
|
||||||
| 1 | 本周 | 重构：adapter 接口 + model registry + agent router | 行为不变，架构就位 |
|
|
||||||
| 2 | 1-2 周 | OpenAI backend | 两个 backend 跑通 |
|
|
||||||
| 3 | 2-3 周 | Ollama + Agent routing 稳定化 | 三 backend + per-agent 路由 |
|
|
||||||
| 4 | 按需 | Budget + Cost control | 成本可控 |
|
|
||||||
| 5 | 等时机 | Anthropic 官方 API | 逃生通道 |
|
|
||||||
|
|
||||||
## 成功标准
|
|
||||||
|
|
||||||
1. Claude CLI 被封的那天，改一行配置，所有 agent 继续运行
|
|
||||||
2. `/ocp cost` 每个 agent 花了多少（actual/estimated/free）
|
|
||||||
3. 翻译 agent 自动用 Ollama（$0），coding agent 用 Claude
|
|
||||||
4. 超预算自动降级，不需人工干预
|
|
||||||
5. 新 backend 只需实现 BackendAdapter 接口 + 一个文件
|
|
||||||
|
|
||||||
## 设计审核记录
|
|
||||||
|
|
||||||
| 轮次 | 审核方 | 关键结论 |
|
|
||||||
|------|--------|----------|
|
|
||||||
| 第一轮 | AI-A | 定位从 proxy 转向成本控制器；识别 5 个 P0 问题 |
|
|
||||||
| 第二轮 | Claude（本项目开发者） | 反驳：不能删 model routing（基础设施层）；backend 数量是运营决策不是架构约束；budget 拆到 Phase 4 |
|
|
||||||
| 第三轮 | AI-A | 校准：model registry 与 routing 物理拆开两个模块；agent identity 结构预留 inferred 扩展点；backend 加 tier 标记 |
|
|
||||||
|
|
||||||
## 一句话总结
|
|
||||||
|
|
||||||
**短期靠套利活（Claude CLI proxy），长期靠控制力活（Agent 成本控制器）。两条腿走路，CLI 死了上层不死。**
|
|
||||||
@@ -1,171 +0,0 @@
|
|||||||
/**
|
|
||||||
* Agent Router — Layer 2 of OCP routing.
|
|
||||||
*
|
|
||||||
* Routing decision order (written in stone):
|
|
||||||
* 1. Agent policy override → agent has explicit backend config? Use it.
|
|
||||||
* 2. Model registry → model maps to a backend? Route there.
|
|
||||||
* 3. Default backend → global fallback.
|
|
||||||
* 4. Fallback chain → chosen backend is down? Try next.
|
|
||||||
*
|
|
||||||
* Agent identity sources (priority):
|
|
||||||
* 1. x-ocp-agent header
|
|
||||||
* 2. Session metadata (bound on first request)
|
|
||||||
* 3. (future) Inferred from usage patterns
|
|
||||||
* 4. "default"
|
|
||||||
*/
|
|
||||||
|
|
||||||
export class AgentRouter {
|
|
||||||
/**
|
|
||||||
* @param {Object} options
|
|
||||||
* @param {import("./model-registry.mjs").ModelRegistry} options.registry
|
|
||||||
* @param {Map<string, import("./backends/base.mjs").BackendAdapter>} options.backends
|
|
||||||
* @param {Object} options.agentConfig — agent routing rules from config
|
|
||||||
* @param {string} options.defaultBackend — fallback backend ID
|
|
||||||
*/
|
|
||||||
constructor({ registry, backends, agentConfig = {}, defaultBackend = "claude-cli" }) {
|
|
||||||
this._registry = registry;
|
|
||||||
this._backends = backends;
|
|
||||||
this._agentConfig = agentConfig;
|
|
||||||
this._defaultBackend = defaultBackend;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Identify the agent from a request.
|
|
||||||
*
|
|
||||||
* @param {Object} req - HTTP request
|
|
||||||
* @param {Object} [sessionMeta] - Session metadata (may contain agent binding)
|
|
||||||
* @returns {string} Agent name
|
|
||||||
*/
|
|
||||||
identifyAgent(req, sessionMeta = null) {
|
|
||||||
// 1. Explicit header
|
|
||||||
const headerAgent = req.headers?.["x-ocp-agent"];
|
|
||||||
if (headerAgent) return headerAgent;
|
|
||||||
|
|
||||||
// 2. Session binding
|
|
||||||
if (sessionMeta?.agent) return sessionMeta.agent;
|
|
||||||
|
|
||||||
// 3. (future) Inferred — not implemented yet
|
|
||||||
|
|
||||||
// 4. Default
|
|
||||||
return "default";
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Resolve which backend + model to use for a request.
|
|
||||||
*
|
|
||||||
* @param {Object} params
|
|
||||||
* @param {string} params.agent - Agent name (from identifyAgent)
|
|
||||||
* @param {string} params.model - Requested model ID
|
|
||||||
* @returns {{backendId: string, model: string, reason: string}}
|
|
||||||
*/
|
|
||||||
resolve({ agent, model }) {
|
|
||||||
// Step 1: Agent policy override
|
|
||||||
const agentRule = this._agentConfig[agent];
|
|
||||||
if (agentRule?.preferred) {
|
|
||||||
const { backendId, modelId } = this._parsePreferred(agentRule.preferred);
|
|
||||||
const backend = this._backends.get(backendId);
|
|
||||||
if (backend?.enabled) {
|
|
||||||
const resolvedModel = modelId || model;
|
|
||||||
return {
|
|
||||||
backendId,
|
|
||||||
model: resolvedModel,
|
|
||||||
reason: `agent-policy(${agent})`,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Agent preferred backend is down — try fallback
|
|
||||||
if (agentRule.fallback) {
|
|
||||||
const fb = this._parseFallback(agentRule.fallback, model);
|
|
||||||
if (fb) return { ...fb, reason: `agent-fallback(${agent})` };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 2: Model registry
|
|
||||||
const registryEntry = this._registry.resolve(model);
|
|
||||||
if (registryEntry) {
|
|
||||||
const backend = this._backends.get(registryEntry.backendId);
|
|
||||||
if (backend?.enabled) {
|
|
||||||
return {
|
|
||||||
backendId: registryEntry.backendId,
|
|
||||||
model: registryEntry.canonical,
|
|
||||||
reason: "model-registry",
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 2b: Check adapters for alias support
|
|
||||||
for (const [id, backend] of this._backends) {
|
|
||||||
if (backend.enabled && backend.supportsModel(model)) {
|
|
||||||
const resolved = typeof backend.resolveModel === "function"
|
|
||||||
? backend.resolveModel(model)
|
|
||||||
: model;
|
|
||||||
return {
|
|
||||||
backendId: id,
|
|
||||||
model: resolved,
|
|
||||||
reason: `adapter-match(${id})`,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 3: Default backend
|
|
||||||
const defaultAgent = this._agentConfig.default;
|
|
||||||
if (defaultAgent?.preferred) {
|
|
||||||
const { backendId } = this._parsePreferred(defaultAgent.preferred);
|
|
||||||
if (this._backends.get(backendId)?.enabled) {
|
|
||||||
return { backendId, model, reason: "default-agent-policy" };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._backends.get(this._defaultBackend)?.enabled) {
|
|
||||||
return {
|
|
||||||
backendId: this._defaultBackend,
|
|
||||||
model,
|
|
||||||
reason: "default-backend",
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 4: Fallback — try any enabled backend
|
|
||||||
for (const [id, backend] of this._backends) {
|
|
||||||
if (backend.enabled) {
|
|
||||||
return { backendId: id, model, reason: `last-resort(${id})` };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return { backendId: null, model, reason: "no-backend-available" };
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Parse a preferred string like "claude-cli" or "claude-cli/claude-opus-4-6"
|
|
||||||
*/
|
|
||||||
_parsePreferred(preferred) {
|
|
||||||
if (preferred.includes("/")) {
|
|
||||||
const [backendId, modelId] = preferred.split("/", 2);
|
|
||||||
return { backendId, modelId };
|
|
||||||
}
|
|
||||||
return { backendId: preferred, modelId: null };
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Parse a fallback value and return routing info.
|
|
||||||
*/
|
|
||||||
_parseFallback(fallback, model) {
|
|
||||||
if (!fallback) return null;
|
|
||||||
const { backendId, modelId } = this._parsePreferred(fallback);
|
|
||||||
const backend = this._backends.get(backendId);
|
|
||||||
if (!backend?.enabled) return null;
|
|
||||||
return { backendId, model: modelId || model };
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get routing info for display (e.g. /ocp routing command).
|
|
||||||
*/
|
|
||||||
getRoutingTable() {
|
|
||||||
const table = {};
|
|
||||||
for (const [agent, config] of Object.entries(this._agentConfig)) {
|
|
||||||
table[agent] = {
|
|
||||||
preferred: config.preferred || null,
|
|
||||||
fallback: config.fallback || null,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
return table;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
/**
|
|
||||||
* BackendAdapter — base class for all OCP backends.
|
|
||||||
*
|
|
||||||
* Each backend adapter MUST:
|
|
||||||
* 1. Implement chatCompletion() as an async generator yielding UnifiedChunks
|
|
||||||
* 2. Implement healthCheck()
|
|
||||||
* 3. Report its costType (actual | estimated | free)
|
|
||||||
* 4. Report its tier (core | community)
|
|
||||||
*
|
|
||||||
* Adapters do NOT handle:
|
|
||||||
* - Session management (SessionManager does that)
|
|
||||||
* - Stats collection (StatsCollector does that)
|
|
||||||
* - Routing decisions (agent-router does that)
|
|
||||||
*/
|
|
||||||
|
|
||||||
export class BackendAdapter {
|
|
||||||
/**
|
|
||||||
* @param {Object} config - Backend-specific configuration
|
|
||||||
*/
|
|
||||||
constructor(config = {}) {
|
|
||||||
this._config = config;
|
|
||||||
this._enabled = config.enabled !== false;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @returns {string} Unique backend identifier */
|
|
||||||
get id() { throw new Error("BackendAdapter.id must be implemented"); }
|
|
||||||
|
|
||||||
/** @returns {string} Human-readable name */
|
|
||||||
get displayName() { throw new Error("BackendAdapter.displayName must be implemented"); }
|
|
||||||
|
|
||||||
/** @returns {string[]} List of supported model IDs */
|
|
||||||
get models() { throw new Error("BackendAdapter.models must be implemented"); }
|
|
||||||
|
|
||||||
/** @returns {"actual"|"estimated"|"free"} */
|
|
||||||
get costType() { throw new Error("BackendAdapter.costType must be implemented"); }
|
|
||||||
|
|
||||||
/** @returns {"core"|"community"} */
|
|
||||||
get tier() { return "core"; }
|
|
||||||
|
|
||||||
/** @returns {boolean} */
|
|
||||||
get enabled() { return this._enabled; }
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Initialize the backend (called once at startup).
|
|
||||||
* Override to do async setup (e.g. validate binary, check API key).
|
|
||||||
*/
|
|
||||||
async initialize() {}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Shutdown the backend (called on graceful shutdown).
|
|
||||||
* Override to clean up resources.
|
|
||||||
*/
|
|
||||||
async shutdown() {}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if this backend is healthy and reachable.
|
|
||||||
* @returns {Promise<{ok: boolean, message: string, latencyMs: number}>}
|
|
||||||
*/
|
|
||||||
async healthCheck() {
|
|
||||||
return { ok: false, message: "healthCheck not implemented", latencyMs: 0 };
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Execute a chat completion request.
|
|
||||||
*
|
|
||||||
* MUST yield UnifiedChunk objects:
|
|
||||||
* { type: "delta", content: "...", model, backend } — streaming content
|
|
||||||
* { type: "done", model, backend, usage?, cost? } — stream finished
|
|
||||||
* { type: "error", error: "...", model, backend } — error occurred
|
|
||||||
*
|
|
||||||
* @param {Object} request
|
|
||||||
* @param {string} request.model - Canonical model ID
|
|
||||||
* @param {Array} request.messages - OpenAI-format messages array
|
|
||||||
* @param {Object} [request.session] - Session info { uuid, resume }
|
|
||||||
* @param {Object} [request.options] - Additional options (temperature, etc.)
|
|
||||||
* @yields {import("../unified-chunk.mjs").UnifiedChunk}
|
|
||||||
*/
|
|
||||||
async *chatCompletion(request) {
|
|
||||||
throw new Error("BackendAdapter.chatCompletion must be implemented");
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if this backend supports a given model.
|
|
||||||
* Default: exact match against models list.
|
|
||||||
* Override for pattern matching (e.g. "gpt-*" → openai).
|
|
||||||
*/
|
|
||||||
supportsModel(model) {
|
|
||||||
return this.models.includes(model);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,393 +0,0 @@
|
|||||||
/**
|
|
||||||
* ClaudeCliAdapter — OCP backend for Claude via local CLI binary.
|
|
||||||
*
|
|
||||||
* Spawns `claude -p` processes for each request. Converts CLI stdout
|
|
||||||
* into UnifiedChunk streaming protocol.
|
|
||||||
*
|
|
||||||
* costType: "estimated" (no actual cost data from CLI)
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { spawn, execFileSync } from "node:child_process";
|
|
||||||
import { accessSync, constants } from "node:fs";
|
|
||||||
import { join } from "node:path";
|
|
||||||
import { BackendAdapter } from "./base.mjs";
|
|
||||||
import { delta, done, error } from "../unified-chunk.mjs";
|
|
||||||
|
|
||||||
// Default models exposed by this backend
|
|
||||||
const DEFAULT_MODELS = [
|
|
||||||
"claude-opus-4-6",
|
|
||||||
"claude-sonnet-4-6",
|
|
||||||
"claude-haiku-4-5-20251001",
|
|
||||||
];
|
|
||||||
|
|
||||||
// Alias → canonical model mapping
|
|
||||||
const MODEL_ALIASES = {
|
|
||||||
"claude-opus-4": "claude-opus-4-6",
|
|
||||||
"claude-haiku-4": "claude-haiku-4-5-20251001",
|
|
||||||
"claude-haiku-4-5": "claude-haiku-4-5-20251001",
|
|
||||||
"opus": "claude-opus-4-6",
|
|
||||||
"sonnet": "claude-sonnet-4-6",
|
|
||||||
"haiku": "claude-haiku-4-5-20251001",
|
|
||||||
};
|
|
||||||
|
|
||||||
export class ClaudeCliAdapter extends BackendAdapter {
|
|
||||||
constructor(config = {}) {
|
|
||||||
super(config);
|
|
||||||
this._claudeBin = null;
|
|
||||||
this._activeProcesses = new Set();
|
|
||||||
this._maxConcurrent = config.maxConcurrent || 8;
|
|
||||||
this._timeout = config.timeout?.overall || 300000;
|
|
||||||
this._skipPermissions = config.skipPermissions || false;
|
|
||||||
this._allowedTools = config.allowedTools || [
|
|
||||||
"Bash", "Read", "Write", "Edit", "Glob", "Grep",
|
|
||||||
"WebSearch", "WebFetch", "Agent",
|
|
||||||
];
|
|
||||||
this._systemPrompt = config.systemPrompt || "";
|
|
||||||
this._mcpConfig = config.mcpConfig || "";
|
|
||||||
this._modelList = config.models || DEFAULT_MODELS;
|
|
||||||
|
|
||||||
// Timeout tiers: per-model base + per-prompt-char scaling
|
|
||||||
this._timeoutTiers = config.timeoutTiers || {
|
|
||||||
opus: { base: 150000, perPromptChar: 0.00050 },
|
|
||||||
sonnet: { base: 120000, perPromptChar: 0.00050 },
|
|
||||||
haiku: { base: 45000, perPromptChar: 0.00010 },
|
|
||||||
};
|
|
||||||
|
|
||||||
// Estimated cost per MTok (for cost tracking)
|
|
||||||
this._estimatedCost = config.estimatedCostPerMTok || {
|
|
||||||
input: 3.0,
|
|
||||||
output: 15.0,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
get id() { return "claude-cli"; }
|
|
||||||
get displayName() { return "Claude CLI"; }
|
|
||||||
get models() { return this._modelList; }
|
|
||||||
get costType() { return "estimated"; }
|
|
||||||
get tier() { return "core"; }
|
|
||||||
get activeProcessCount() { return this._activeProcesses.size; }
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Resolve a model ID, handling aliases.
|
|
||||||
* Returns canonical CLI model ID or the input if no alias found.
|
|
||||||
*/
|
|
||||||
resolveModel(model) {
|
|
||||||
return MODEL_ALIASES[model] || model;
|
|
||||||
}
|
|
||||||
|
|
||||||
supportsModel(model) {
|
|
||||||
const canonical = this.resolveModel(model);
|
|
||||||
return this._modelList.includes(canonical) || model in MODEL_ALIASES;
|
|
||||||
}
|
|
||||||
|
|
||||||
async initialize() {
|
|
||||||
this._claudeBin = this._resolveBinary();
|
|
||||||
}
|
|
||||||
|
|
||||||
async shutdown() {
|
|
||||||
for (const proc of this._activeProcesses) {
|
|
||||||
try { proc.kill("SIGTERM"); } catch {}
|
|
||||||
}
|
|
||||||
// Force kill after 5s
|
|
||||||
if (this._activeProcesses.size > 0) {
|
|
||||||
await new Promise(r => setTimeout(r, 5000));
|
|
||||||
for (const proc of this._activeProcesses) {
|
|
||||||
try { proc.kill("SIGKILL"); } catch {}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async healthCheck() {
|
|
||||||
const t0 = Date.now();
|
|
||||||
try {
|
|
||||||
const env = this._cleanEnv();
|
|
||||||
execFileSync(this._claudeBin, ["auth", "status"], {
|
|
||||||
encoding: "utf8", timeout: 10000, env,
|
|
||||||
});
|
|
||||||
return { ok: true, message: "authenticated", latencyMs: Date.now() - t0 };
|
|
||||||
} catch (e) {
|
|
||||||
return {
|
|
||||||
ok: false,
|
|
||||||
message: (e.stderr || e.message || "").slice(0, 200),
|
|
||||||
latencyMs: Date.now() - t0,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Execute a chat completion via Claude CLI.
|
|
||||||
* Yields UnifiedChunks as stdout data arrives.
|
|
||||||
*/
|
|
||||||
async *chatCompletion(request) {
|
|
||||||
const { model, messages, session, promptText } = request;
|
|
||||||
const cliModel = this.resolveModel(model);
|
|
||||||
|
|
||||||
if (this._activeProcesses.size >= this._maxConcurrent) {
|
|
||||||
yield error(
|
|
||||||
`concurrency limit reached (${this._activeProcesses.size}/${this._maxConcurrent})`,
|
|
||||||
cliModel, this.id,
|
|
||||||
);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const prompt = promptText || this._messagesToPrompt(messages);
|
|
||||||
const cliArgs = this._buildCliArgs(cliModel, session);
|
|
||||||
const env = this._cleanEnv();
|
|
||||||
|
|
||||||
const proc = spawn(this._claudeBin, cliArgs, {
|
|
||||||
env, stdio: ["pipe", "pipe", "pipe"],
|
|
||||||
});
|
|
||||||
this._activeProcesses.add(proc);
|
|
||||||
|
|
||||||
const t0 = Date.now();
|
|
||||||
const firstByteTimeoutMs = this._computeFirstByteTimeout(cliModel, prompt.length);
|
|
||||||
let gotFirstByte = false;
|
|
||||||
let totalChars = 0;
|
|
||||||
let stderr = "";
|
|
||||||
let finished = false;
|
|
||||||
|
|
||||||
// Create a promise-based wrapper for the process
|
|
||||||
try {
|
|
||||||
yield* await new Promise((resolve, reject) => {
|
|
||||||
const chunks = [];
|
|
||||||
let streamResolve = null;
|
|
||||||
let streamDone = false;
|
|
||||||
|
|
||||||
// We can't directly yield from inside callbacks, so we collect
|
|
||||||
// and use a different approach — return an async iterable.
|
|
||||||
// Actually, let's use a simpler pattern with a queue.
|
|
||||||
reject(new Error("USE_QUEUE_PATTERN"));
|
|
||||||
});
|
|
||||||
} catch {
|
|
||||||
// Fall through to queue-based pattern below
|
|
||||||
}
|
|
||||||
|
|
||||||
// Queue-based async generator pattern
|
|
||||||
const queue = [];
|
|
||||||
let queueResolve = null;
|
|
||||||
let queueDone = false;
|
|
||||||
|
|
||||||
function enqueue(chunk) {
|
|
||||||
if (queueDone) return;
|
|
||||||
if (queueResolve) {
|
|
||||||
const r = queueResolve;
|
|
||||||
queueResolve = null;
|
|
||||||
r(chunk);
|
|
||||||
} else {
|
|
||||||
queue.push(chunk);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function waitForChunk() {
|
|
||||||
if (queue.length > 0) return Promise.resolve(queue.shift());
|
|
||||||
if (queueDone) return Promise.resolve(null);
|
|
||||||
return new Promise(r => { queueResolve = r; });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Write prompt to stdin
|
|
||||||
proc.stdin.write(prompt);
|
|
||||||
proc.stdin.end();
|
|
||||||
|
|
||||||
proc.stdout.on("data", (d) => {
|
|
||||||
if (!gotFirstByte) {
|
|
||||||
gotFirstByte = true;
|
|
||||||
clearTimeout(firstByteTimer);
|
|
||||||
}
|
|
||||||
const text = d.toString();
|
|
||||||
totalChars += text.length;
|
|
||||||
enqueue(delta(text, cliModel, this.id));
|
|
||||||
});
|
|
||||||
|
|
||||||
proc.stderr.on("data", (d) => { stderr += d; });
|
|
||||||
|
|
||||||
proc.on("close", (code, signal) => {
|
|
||||||
this._activeProcesses.delete(proc);
|
|
||||||
clearTimeout(firstByteTimer);
|
|
||||||
clearTimeout(overallTimer);
|
|
||||||
const elapsed = Date.now() - t0;
|
|
||||||
|
|
||||||
if (code !== 0) {
|
|
||||||
const msg = stderr.slice(0, 300) || `claude exit ${code}`;
|
|
||||||
enqueue(error(msg, cliModel, this.id));
|
|
||||||
} else {
|
|
||||||
// Estimate cost based on prompt + completion chars
|
|
||||||
const estimatedInputTokens = Math.ceil(prompt.length / 4);
|
|
||||||
const estimatedOutputTokens = Math.ceil(totalChars / 4);
|
|
||||||
enqueue(done(cliModel, this.id, {
|
|
||||||
promptTokens: estimatedInputTokens,
|
|
||||||
completionTokens: estimatedOutputTokens,
|
|
||||||
}, {
|
|
||||||
value: (estimatedInputTokens * this._estimatedCost.input +
|
|
||||||
estimatedOutputTokens * this._estimatedCost.output) / 1000000,
|
|
||||||
type: "estimated",
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
queueDone = true;
|
|
||||||
if (queueResolve) {
|
|
||||||
const r = queueResolve;
|
|
||||||
queueResolve = null;
|
|
||||||
r(null);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
proc.on("error", (err) => {
|
|
||||||
this._activeProcesses.delete(proc);
|
|
||||||
clearTimeout(firstByteTimer);
|
|
||||||
clearTimeout(overallTimer);
|
|
||||||
enqueue(error(err.message, cliModel, this.id));
|
|
||||||
queueDone = true;
|
|
||||||
if (queueResolve) {
|
|
||||||
const r = queueResolve;
|
|
||||||
queueResolve = null;
|
|
||||||
r(null);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// First-byte timeout
|
|
||||||
const firstByteTimer = setTimeout(() => {
|
|
||||||
if (!gotFirstByte) {
|
|
||||||
try { proc.kill("SIGTERM"); } catch {}
|
|
||||||
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
|
|
||||||
enqueue(error(`first-byte timeout after ${firstByteTimeoutMs}ms`, cliModel, this.id));
|
|
||||||
queueDone = true;
|
|
||||||
}
|
|
||||||
}, firstByteTimeoutMs);
|
|
||||||
|
|
||||||
// Overall timeout
|
|
||||||
const overallTimer = setTimeout(() => {
|
|
||||||
try { proc.kill("SIGTERM"); } catch {}
|
|
||||||
setTimeout(() => { try { proc.kill("SIGKILL"); } catch {} }, 5000);
|
|
||||||
enqueue(error(`request timeout after ${this._timeout}ms`, cliModel, this.id));
|
|
||||||
queueDone = true;
|
|
||||||
}, this._timeout);
|
|
||||||
|
|
||||||
// Yield chunks as they arrive
|
|
||||||
while (true) {
|
|
||||||
const chunk = await waitForChunk();
|
|
||||||
if (chunk === null) break;
|
|
||||||
yield chunk;
|
|
||||||
if (chunk.type === "done" || chunk.type === "error") break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Internal helpers ──────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_resolveBinary() {
|
|
||||||
if (process.env.CLAUDE_BIN) {
|
|
||||||
try {
|
|
||||||
accessSync(process.env.CLAUDE_BIN, constants.X_OK);
|
|
||||||
return process.env.CLAUDE_BIN;
|
|
||||||
} catch {
|
|
||||||
throw new Error(`CLAUDE_BIN="${process.env.CLAUDE_BIN}" is set but not executable.`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const candidates = [
|
|
||||||
"/opt/homebrew/bin/claude",
|
|
||||||
"/usr/local/bin/claude",
|
|
||||||
"/usr/bin/claude",
|
|
||||||
join(process.env.HOME || "", ".local/bin/claude"),
|
|
||||||
];
|
|
||||||
for (const p of candidates) {
|
|
||||||
try { accessSync(p, constants.X_OK); return p; } catch {}
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const resolved = execFileSync("which", ["claude"], {
|
|
||||||
encoding: "utf8", timeout: 5000,
|
|
||||||
}).trim();
|
|
||||||
if (resolved) return resolved;
|
|
||||||
} catch {}
|
|
||||||
|
|
||||||
throw new Error(
|
|
||||||
"claude binary not found. Set CLAUDE_BIN or ensure claude is in PATH."
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
_cleanEnv() {
|
|
||||||
const env = { ...process.env };
|
|
||||||
delete env.CLAUDECODE;
|
|
||||||
delete env.ANTHROPIC_API_KEY;
|
|
||||||
delete env.ANTHROPIC_BASE_URL;
|
|
||||||
delete env.ANTHROPIC_AUTH_TOKEN;
|
|
||||||
return env;
|
|
||||||
}
|
|
||||||
|
|
||||||
_buildCliArgs(cliModel, session) {
|
|
||||||
const args = ["-p", "--model", cliModel, "--output-format", "text"];
|
|
||||||
|
|
||||||
if (session?.resume) {
|
|
||||||
args.push("--resume", session.uuid);
|
|
||||||
} else if (session?.uuid) {
|
|
||||||
args.push("--session-id", session.uuid);
|
|
||||||
} else {
|
|
||||||
args.push("--no-session-persistence");
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._skipPermissions) {
|
|
||||||
args.push("--dangerously-skip-permissions");
|
|
||||||
} else if (this._allowedTools.length > 0) {
|
|
||||||
args.push("--allowedTools", ...this._allowedTools);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._systemPrompt) {
|
|
||||||
args.push("--append-system-prompt", this._systemPrompt);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._mcpConfig) {
|
|
||||||
args.push("--mcp-config", this._mcpConfig);
|
|
||||||
}
|
|
||||||
|
|
||||||
return args;
|
|
||||||
}
|
|
||||||
|
|
||||||
_getModelTier(cliModel) {
|
|
||||||
if (cliModel.includes("opus")) return "opus";
|
|
||||||
if (cliModel.includes("haiku")) return "haiku";
|
|
||||||
return "sonnet";
|
|
||||||
}
|
|
||||||
|
|
||||||
_computeFirstByteTimeout(cliModel, promptLength) {
|
|
||||||
const tierName = this._getModelTier(cliModel);
|
|
||||||
const tier = this._timeoutTiers[tierName] || this._timeoutTiers.sonnet;
|
|
||||||
const timeout = tier.base + Math.floor(promptLength * tier.perPromptChar);
|
|
||||||
return Math.min(timeout, Math.max(this._timeout - 5000, 10000));
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Convert OpenAI-format messages to a plain text prompt for Claude CLI.
|
|
||||||
* Includes truncation guard for oversized prompts.
|
|
||||||
*/
|
|
||||||
_messagesToPrompt(messages, maxChars = 150000) {
|
|
||||||
const full = messages.map((m) => {
|
|
||||||
const text = typeof m.content === "string" ? m.content : JSON.stringify(m.content);
|
|
||||||
if (m.role === "system") return `[System] ${text}`;
|
|
||||||
if (m.role === "assistant") return `[Assistant] ${text}`;
|
|
||||||
return text;
|
|
||||||
});
|
|
||||||
|
|
||||||
const joined = full.join("\n\n");
|
|
||||||
if (joined.length <= maxChars) return joined;
|
|
||||||
|
|
||||||
// Truncation: keep system + recent messages
|
|
||||||
const system = [];
|
|
||||||
const rest = [];
|
|
||||||
for (let i = 0; i < full.length; i++) {
|
|
||||||
if (messages[i].role === "system") system.push(full[i]);
|
|
||||||
else rest.push(full[i]);
|
|
||||||
}
|
|
||||||
|
|
||||||
const systemText = system.join("\n\n");
|
|
||||||
const budget = maxChars - systemText.length - 200;
|
|
||||||
const kept = [];
|
|
||||||
let used = 0;
|
|
||||||
for (let i = rest.length - 1; i >= 0; i--) {
|
|
||||||
if (used + rest[i].length + 2 > budget) break;
|
|
||||||
kept.unshift(rest[i]);
|
|
||||||
used += rest[i].length + 2;
|
|
||||||
}
|
|
||||||
|
|
||||||
const truncNote = `[System] Note: ${rest.length - kept.length} older messages were truncated to fit context limit.`;
|
|
||||||
return [systemText, truncNote, ...kept].filter(Boolean).join("\n\n");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,99 +0,0 @@
|
|||||||
/**
|
|
||||||
* Model Registry — Layer 1 of OCP routing.
|
|
||||||
*
|
|
||||||
* Static mapping: model ID → backend ID.
|
|
||||||
* This is infrastructure, not policy. Every model must map to exactly one backend.
|
|
||||||
*
|
|
||||||
* Non-agent callers (Cline, Aider, etc.) only send a model name — this module
|
|
||||||
* is the only thing that tells OCP where to route the request.
|
|
||||||
*/
|
|
||||||
|
|
||||||
export class ModelRegistry {
|
|
||||||
constructor() {
|
|
||||||
/** @type {Map<string, {backendId: string, canonical: string}>} */
|
|
||||||
this._models = new Map();
|
|
||||||
|
|
||||||
/** @type {Map<string, Object>} model id → display metadata */
|
|
||||||
this._metadata = new Map();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Register models from a backend adapter.
|
|
||||||
* Called during startup for each enabled backend.
|
|
||||||
*
|
|
||||||
* @param {import("./backends/base.mjs").BackendAdapter} adapter
|
|
||||||
*/
|
|
||||||
registerBackend(adapter) {
|
|
||||||
for (const modelId of adapter.models) {
|
|
||||||
this._models.set(modelId, {
|
|
||||||
backendId: adapter.id,
|
|
||||||
canonical: modelId,
|
|
||||||
});
|
|
||||||
this._metadata.set(modelId, {
|
|
||||||
backendId: adapter.id,
|
|
||||||
backendName: adapter.displayName,
|
|
||||||
tier: adapter.tier,
|
|
||||||
costType: adapter.costType,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// Also register aliases if the adapter supports resolveModel
|
|
||||||
if (typeof adapter.resolveModel === "function") {
|
|
||||||
// We don't auto-register aliases here — they're handled by
|
|
||||||
// supportsModel() on the adapter. The registry only knows canonical IDs.
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Look up which backend serves a given model.
|
|
||||||
*
|
|
||||||
* @param {string} model — model ID (could be canonical or alias)
|
|
||||||
* @returns {{backendId: string, canonical: string} | null}
|
|
||||||
*/
|
|
||||||
resolve(model) {
|
|
||||||
// Direct lookup first
|
|
||||||
const direct = this._models.get(model);
|
|
||||||
if (direct) return direct;
|
|
||||||
|
|
||||||
// No alias resolution here — that's the adapter's job.
|
|
||||||
// The router will ask each adapter if it supports the model.
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get all registered models with metadata.
|
|
||||||
* Used by GET /v1/models endpoint.
|
|
||||||
*/
|
|
||||||
listModels() {
|
|
||||||
const result = [];
|
|
||||||
for (const [id, meta] of this._metadata) {
|
|
||||||
result.push({ id, ...meta });
|
|
||||||
}
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if a model is known to the registry.
|
|
||||||
*/
|
|
||||||
has(model) {
|
|
||||||
return this._models.has(model);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get the backend ID for a model, or null.
|
|
||||||
*/
|
|
||||||
getBackendId(model) {
|
|
||||||
return this._models.get(model)?.backendId || null;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get all models for a specific backend.
|
|
||||||
*/
|
|
||||||
getModelsForBackend(backendId) {
|
|
||||||
const models = [];
|
|
||||||
for (const [id, entry] of this._models) {
|
|
||||||
if (entry.backendId === backendId) models.push(id);
|
|
||||||
}
|
|
||||||
return models;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+2
-2
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "openclaw-claude-proxy",
|
"name": "openclaw-claude-proxy",
|
||||||
"version": "3.0.1",
|
"version": "3.0.0",
|
||||||
"description": "OpenAI-compatible proxy for Claude CLI — plan usage monitoring, runtime settings, prompt truncation, OCP CLI",
|
"description": "OpenAI-compatible proxy for Claude CLI — plan usage monitoring, runtime settings, prompt truncation, OCP CLI",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"bin": {
|
"bin": {
|
||||||
@@ -24,6 +24,6 @@
|
|||||||
},
|
},
|
||||||
"repository": {
|
"repository": {
|
||||||
"type": "git",
|
"type": "git",
|
||||||
"url": "https://github.com/dtzp555-max/ocp"
|
"url": "https://github.com/dtzp555-max/openclaw-claude-proxy"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+10
-137
@@ -1,17 +1,11 @@
|
|||||||
#!/usr/bin/env node
|
#!/usr/bin/env node
|
||||||
/**
|
/**
|
||||||
* OCP (OpenClaw Control Plane) v4.0-alpha
|
* openclaw-claude-proxy v2.5.0 — OpenAI-compatible proxy for Claude CLI
|
||||||
*
|
*
|
||||||
* Cost-control and model-routing layer for AI agents.
|
* Translates OpenAI chat/completions requests into `claude -p` CLI calls,
|
||||||
* Currently supports Claude CLI backend; architecture supports multiple backends.
|
* letting you use your Claude Pro/Max subscription as an OpenClaw model provider.
|
||||||
*
|
*
|
||||||
* v4.0-alpha:
|
* v2.5.0:
|
||||||
* - Internal modular architecture: BackendAdapter, ModelRegistry, AgentRouter,
|
|
||||||
* SessionManager, StatsCollector
|
|
||||||
* - External API unchanged from v2.5.0
|
|
||||||
* - Claude CLI backend extracted to ClaudeCliAdapter
|
|
||||||
*
|
|
||||||
* v2.5.0 (legacy notes):
|
|
||||||
* - Sliding-window circuit breaker: uses time-windowed failure rate instead of
|
* - Sliding-window circuit breaker: uses time-windowed failure rate instead of
|
||||||
* consecutive-count, preventing multi-agent burst scenarios from tripping the
|
* consecutive-count, preventing multi-agent burst scenarios from tripping the
|
||||||
* breaker too aggressively. Half-open state allows configurable probe requests.
|
* breaker too aggressively. Half-open state allows configurable probe requests.
|
||||||
@@ -53,54 +47,6 @@ import { dirname, join } from "node:path";
|
|||||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||||
const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8"));
|
const _pkg = JSON.parse(readFileSync(join(__dirname, "package.json"), "utf8"));
|
||||||
|
|
||||||
// ── v4 Modular Architecture (Step 1: load + initialize, hot path unchanged) ──
|
|
||||||
import { ClaudeCliAdapter } from "./backends/claude-cli.mjs";
|
|
||||||
import { ModelRegistry } from "./model-registry.mjs";
|
|
||||||
import { AgentRouter } from "./agent-router.mjs";
|
|
||||||
import { SessionManager as SessionMgr } from "./session-manager.mjs";
|
|
||||||
import { StatsCollector } from "./stats-collector.mjs";
|
|
||||||
|
|
||||||
// These modules are initialized at startup but NOT yet wired into the request
|
|
||||||
// path. Step 2 will replace the legacy callClaude/callClaudeStreaming with
|
|
||||||
// adapter-based calls. For now, they run in parallel for validation.
|
|
||||||
let _v4Backend = null;
|
|
||||||
let _v4Registry = null;
|
|
||||||
let _v4Router = null;
|
|
||||||
let _v4Sessions = null;
|
|
||||||
let _v4Stats = null;
|
|
||||||
|
|
||||||
async function initV4Modules() {
|
|
||||||
try {
|
|
||||||
_v4Stats = new StatsCollector();
|
|
||||||
_v4Sessions = new SessionMgr({ ttl: SESSION_TTL });
|
|
||||||
_v4Registry = new ModelRegistry();
|
|
||||||
|
|
||||||
_v4Backend = new ClaudeCliAdapter({
|
|
||||||
maxConcurrent: MAX_CONCURRENT,
|
|
||||||
timeout: { overall: TIMEOUT },
|
|
||||||
skipPermissions: SKIP_PERMISSIONS,
|
|
||||||
allowedTools: ALLOWED_TOOLS,
|
|
||||||
systemPrompt: SYSTEM_PROMPT,
|
|
||||||
mcpConfig: MCP_CONFIG,
|
|
||||||
timeoutTiers: MODEL_TIMEOUT_TIERS,
|
|
||||||
});
|
|
||||||
await _v4Backend.initialize();
|
|
||||||
_v4Registry.registerBackend(_v4Backend);
|
|
||||||
|
|
||||||
const backends = new Map([["claude-cli", _v4Backend]]);
|
|
||||||
_v4Router = new AgentRouter({
|
|
||||||
registry: _v4Registry,
|
|
||||||
backends,
|
|
||||||
agentConfig: { default: { preferred: "claude-cli", fallback: null } },
|
|
||||||
defaultBackend: "claude-cli",
|
|
||||||
});
|
|
||||||
|
|
||||||
console.log(`[v4] Modules initialized: backend=${_v4Backend.id}, models=${_v4Backend.models.length}, registry=${_v4Registry.listModels().length} models`);
|
|
||||||
} catch (err) {
|
|
||||||
console.error(`[v4] Module init failed (non-fatal, legacy path active): ${err.message}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Resolve claude binary ───────────────────────────────────────────────
|
// ── Resolve claude binary ───────────────────────────────────────────────
|
||||||
// Priority: CLAUDE_BIN env > well-known paths > which lookup
|
// Priority: CLAUDE_BIN env > well-known paths > which lookup
|
||||||
// Fail-fast if not found — never start with an unresolvable binary.
|
// Fail-fast if not found — never start with an unresolvable binary.
|
||||||
@@ -740,57 +686,20 @@ function completionResponse(res, id, model, content) {
|
|||||||
let usageCache = { data: null, fetchedAt: 0 };
|
let usageCache = { data: null, fetchedAt: 0 };
|
||||||
const USAGE_CACHE_TTL = 300000; // 5 min
|
const USAGE_CACHE_TTL = 300000; // 5 min
|
||||||
|
|
||||||
function getOAuthCredentials() {
|
function getOAuthToken() {
|
||||||
try {
|
try {
|
||||||
const raw = execFileSync("security", [
|
const raw = execFileSync("security", [
|
||||||
"find-generic-password", "-s", "Claude Code-credentials", "-w"
|
"find-generic-password", "-s", "Claude Code-credentials", "-w"
|
||||||
], { encoding: "utf8", timeout: 5000 }).trim();
|
], { encoding: "utf8", timeout: 5000 }).trim();
|
||||||
return JSON.parse(raw)?.claudeAiOauth || null;
|
const creds = JSON.parse(raw);
|
||||||
|
return creds?.claudeAiOauth?.accessToken || null;
|
||||||
} catch {
|
} catch {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async function refreshOAuthToken(_creds) {
|
|
||||||
// Trigger Claude Code to refresh the token by running a minimal command.
|
|
||||||
// Claude Code automatically refreshes expired tokens on startup.
|
|
||||||
try {
|
|
||||||
execFileSync("claude", ["-p", "--model", "haiku", "--max-turns", "1", "hi"], {
|
|
||||||
encoding: "utf8",
|
|
||||||
timeout: 30000,
|
|
||||||
stdio: ["pipe", "pipe", "pipe"],
|
|
||||||
});
|
|
||||||
// Re-read the now-refreshed token from keychain
|
|
||||||
const refreshed = getOAuthCredentials();
|
|
||||||
if (refreshed?.accessToken && refreshed.expiresAt > Date.now()) {
|
|
||||||
console.log("[ocp] OAuth token refreshed via Claude CLI");
|
|
||||||
return refreshed.accessToken;
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
} catch (e) {
|
|
||||||
console.error("[ocp] OAuth refresh via CLI failed:", e.message);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function getOAuthToken() {
|
|
||||||
const creds = getOAuthCredentials();
|
|
||||||
if (!creds?.accessToken) return null;
|
|
||||||
|
|
||||||
// Check if token is expired or about to expire (5 min buffer)
|
|
||||||
const expiresAt = creds.expiresAt || 0;
|
|
||||||
if (expiresAt > Date.now() + 300000) {
|
|
||||||
return creds.accessToken;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Token expired or expiring soon — try refresh
|
|
||||||
console.log("[ocp] OAuth token expired, attempting refresh...");
|
|
||||||
const newToken = await refreshOAuthToken(creds);
|
|
||||||
return newToken || creds.accessToken; // fallback to old token (might still work briefly)
|
|
||||||
}
|
|
||||||
|
|
||||||
async function fetchUsageFromApi() {
|
async function fetchUsageFromApi() {
|
||||||
const token = await getOAuthToken();
|
const token = getOAuthToken();
|
||||||
if (!token) {
|
if (!token) {
|
||||||
return { error: "No OAuth token found in keychain" };
|
return { error: "No OAuth token found in keychain" };
|
||||||
}
|
}
|
||||||
@@ -1269,36 +1178,7 @@ const server = createServer(async (req, res) => {
|
|||||||
return handleSettings(req, res);
|
return handleSettings(req, res);
|
||||||
}
|
}
|
||||||
|
|
||||||
// GET /backends — v4 backend status (new endpoint)
|
jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET /usage, GET /status, GET /logs, GET|PATCH /settings, GET|DELETE /sessions" });
|
||||||
if (req.url === "/backends" && req.method === "GET") {
|
|
||||||
if (!_v4Backend) {
|
|
||||||
return jsonResponse(res, 503, { error: "v4 modules not initialized" });
|
|
||||||
}
|
|
||||||
const health = await _v4Backend.healthCheck();
|
|
||||||
return jsonResponse(res, 200, {
|
|
||||||
backends: [{
|
|
||||||
id: _v4Backend.id,
|
|
||||||
displayName: _v4Backend.displayName,
|
|
||||||
tier: _v4Backend.tier,
|
|
||||||
costType: _v4Backend.costType,
|
|
||||||
enabled: _v4Backend.enabled,
|
|
||||||
models: _v4Backend.models,
|
|
||||||
activeProcesses: _v4Backend.activeProcessCount,
|
|
||||||
health,
|
|
||||||
}],
|
|
||||||
routing: _v4Router?.getRoutingTable() || {},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// GET /routing — v4 agent routing table (new endpoint)
|
|
||||||
if (req.url === "/routing" && req.method === "GET") {
|
|
||||||
return jsonResponse(res, 200, {
|
|
||||||
table: _v4Router?.getRoutingTable() || {},
|
|
||||||
registry: _v4Registry?.listModels() || [],
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
jsonResponse(res, 404, { error: "Not found. Endpoints: GET /v1/models, POST /v1/chat/completions, GET /health, GET /usage, GET /status, GET /logs, GET|PATCH /settings, GET|DELETE /sessions, GET /backends, GET /routing" });
|
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
||||||
@@ -1319,10 +1199,6 @@ function gracefulShutdown(signal) {
|
|||||||
clearInterval(sessionCleanupInterval);
|
clearInterval(sessionCleanupInterval);
|
||||||
clearInterval(authCheckInterval);
|
clearInterval(authCheckInterval);
|
||||||
|
|
||||||
// 2b. Shutdown v4 modules
|
|
||||||
if (_v4Sessions) _v4Sessions.shutdown();
|
|
||||||
if (_v4Backend) _v4Backend.shutdown().catch(() => {});
|
|
||||||
|
|
||||||
// 3. Kill all active child processes
|
// 3. Kill all active child processes
|
||||||
for (const proc of activeProcesses) {
|
for (const proc of activeProcesses) {
|
||||||
try { proc.kill("SIGTERM"); } catch {}
|
try { proc.kill("SIGTERM"); } catch {}
|
||||||
@@ -1359,11 +1235,8 @@ process.on("SIGTERM", () => gracefulShutdown("SIGTERM"));
|
|||||||
process.on("SIGINT", () => gracefulShutdown("SIGINT"));
|
process.on("SIGINT", () => gracefulShutdown("SIGINT"));
|
||||||
|
|
||||||
// ── Start ───────────────────────────────────────────────────────────────
|
// ── Start ───────────────────────────────────────────────────────────────
|
||||||
// Initialize v4 modules before listening
|
|
||||||
initV4Modules().catch(err => console.error(`[v4] init error: ${err.message}`));
|
|
||||||
|
|
||||||
server.listen(PORT, "127.0.0.1", () => {
|
server.listen(PORT, "127.0.0.1", () => {
|
||||||
console.log(`OCP (OpenClaw Control Plane) v${VERSION} listening on http://127.0.0.1:${PORT}`);
|
console.log(`openclaw-claude-proxy v${VERSION} listening on http://127.0.0.1:${PORT}`);
|
||||||
console.log(`Architecture: on-demand spawning (no pool)`);
|
console.log(`Architecture: on-demand spawning (no pool)`);
|
||||||
console.log(`Models: ${MODELS.map((m) => m.id).join(", ")}`);
|
console.log(`Models: ${MODELS.map((m) => m.id).join(", ")}`);
|
||||||
console.log(`Claude binary: ${CLAUDE}`);
|
console.log(`Claude binary: ${CLAUDE}`);
|
||||||
|
|||||||
@@ -1,123 +0,0 @@
|
|||||||
/**
|
|
||||||
* SessionManager — manages conversation sessions across requests.
|
|
||||||
*
|
|
||||||
* Maps caller-provided conversation IDs to backend-specific session state.
|
|
||||||
* For Claude CLI, this means mapping to CLI session UUIDs for --resume.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { randomUUID } from "node:crypto";
|
|
||||||
|
|
||||||
export class SessionManager {
|
|
||||||
/**
|
|
||||||
* @param {Object} options
|
|
||||||
* @param {number} [options.ttl=3600000] — Session TTL in ms (default 1h)
|
|
||||||
*/
|
|
||||||
constructor({ ttl = 3600000 } = {}) {
|
|
||||||
this._ttl = ttl;
|
|
||||||
/** @type {Map<string, SessionEntry>} */
|
|
||||||
this._sessions = new Map();
|
|
||||||
|
|
||||||
// Cleanup expired sessions every 60s
|
|
||||||
this._cleanupInterval = setInterval(() => this._cleanup(), 60000);
|
|
||||||
}
|
|
||||||
|
|
||||||
get ttl() { return this._ttl; }
|
|
||||||
set ttl(val) { this._ttl = val; }
|
|
||||||
get size() { return this._sessions.size; }
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get or create a session for a conversation.
|
|
||||||
*
|
|
||||||
* @param {string|null} conversationId
|
|
||||||
* @param {string} model — model being used
|
|
||||||
* @param {string} [agent] — agent identity
|
|
||||||
* @returns {{session: {uuid: string, resume: boolean}, isNew: boolean, isOneOff: boolean}}
|
|
||||||
*/
|
|
||||||
resolve(conversationId, model, agent = "default") {
|
|
||||||
if (!conversationId) {
|
|
||||||
return { session: null, isNew: false, isOneOff: true };
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._sessions.has(conversationId)) {
|
|
||||||
const entry = this._sessions.get(conversationId);
|
|
||||||
entry.lastUsed = Date.now();
|
|
||||||
entry.messageCount++;
|
|
||||||
return {
|
|
||||||
session: { uuid: entry.uuid, resume: true },
|
|
||||||
isNew: false,
|
|
||||||
isOneOff: false,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// New session
|
|
||||||
const uuid = randomUUID();
|
|
||||||
this._sessions.set(conversationId, {
|
|
||||||
uuid,
|
|
||||||
model,
|
|
||||||
agent,
|
|
||||||
messageCount: 1,
|
|
||||||
lastUsed: Date.now(),
|
|
||||||
createdAt: Date.now(),
|
|
||||||
});
|
|
||||||
|
|
||||||
return {
|
|
||||||
session: { uuid, resume: false },
|
|
||||||
isNew: true,
|
|
||||||
isOneOff: false,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Remove a session (e.g. on resume failure).
|
|
||||||
*/
|
|
||||||
delete(conversationId) {
|
|
||||||
this._sessions.delete(conversationId);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Clear all sessions.
|
|
||||||
* @returns {number} Number of sessions cleared
|
|
||||||
*/
|
|
||||||
clear() {
|
|
||||||
const count = this._sessions.size;
|
|
||||||
this._sessions.clear();
|
|
||||||
return count;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* List all active sessions (for /sessions endpoint).
|
|
||||||
*/
|
|
||||||
list() {
|
|
||||||
const result = [];
|
|
||||||
for (const [id, s] of this._sessions) {
|
|
||||||
result.push({
|
|
||||||
id,
|
|
||||||
uuid: s.uuid,
|
|
||||||
model: s.model,
|
|
||||||
agent: s.agent,
|
|
||||||
messages: s.messageCount,
|
|
||||||
lastUsed: new Date(s.lastUsed).toISOString(),
|
|
||||||
idleMs: Date.now() - s.lastUsed,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Shutdown — clear interval.
|
|
||||||
*/
|
|
||||||
shutdown() {
|
|
||||||
clearInterval(this._cleanupInterval);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Internal ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_cleanup() {
|
|
||||||
const now = Date.now();
|
|
||||||
for (const [id, s] of this._sessions) {
|
|
||||||
if (now - s.lastUsed > this._ttl) {
|
|
||||||
this._sessions.delete(id);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,186 +0,0 @@
|
|||||||
/**
|
|
||||||
* StatsCollector — tracks request metrics per-model, per-backend, per-agent.
|
|
||||||
*
|
|
||||||
* Replaces the flat stats + modelStats from server.mjs with structured,
|
|
||||||
* multi-dimensional tracking. Backward compatible: still exposes the same
|
|
||||||
* data for /health, /usage, /status endpoints.
|
|
||||||
*/
|
|
||||||
|
|
||||||
export class StatsCollector {
|
|
||||||
constructor() {
|
|
||||||
// Global counters (backward compatible)
|
|
||||||
this.totalRequests = 0;
|
|
||||||
this.activeRequests = 0;
|
|
||||||
this.errors = 0;
|
|
||||||
this.timeouts = 0;
|
|
||||||
this.sessionHits = 0;
|
|
||||||
this.sessionMisses = 0;
|
|
||||||
this.oneOffRequests = 0;
|
|
||||||
|
|
||||||
// Recent errors ring buffer
|
|
||||||
this._recentErrors = [];
|
|
||||||
this._maxErrors = 20;
|
|
||||||
|
|
||||||
/** @type {Map<string, ModelStats>} cliModel → stats */
|
|
||||||
this._modelStats = new Map();
|
|
||||||
|
|
||||||
/** @type {Map<string, ModelStats>} backendId → stats */
|
|
||||||
this._backendStats = new Map();
|
|
||||||
|
|
||||||
/** @type {Map<string, ModelStats>} agentName → stats */
|
|
||||||
this._agentStats = new Map();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Record a new request starting.
|
|
||||||
*/
|
|
||||||
recordRequest(model, backend, agent, promptChars) {
|
|
||||||
this.totalRequests++;
|
|
||||||
this.activeRequests++;
|
|
||||||
this._getOrCreate(this._modelStats, model).recordRequest(promptChars);
|
|
||||||
this._getOrCreate(this._backendStats, backend).recordRequest(promptChars);
|
|
||||||
this._getOrCreate(this._agentStats, agent).recordRequest(promptChars);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Record a successful completion.
|
|
||||||
*/
|
|
||||||
recordSuccess(model, backend, agent, elapsedMs, cost = null) {
|
|
||||||
this.activeRequests = Math.max(0, this.activeRequests - 1);
|
|
||||||
this._getOrCreate(this._modelStats, model).recordSuccess(elapsedMs, cost);
|
|
||||||
this._getOrCreate(this._backendStats, backend).recordSuccess(elapsedMs, cost);
|
|
||||||
this._getOrCreate(this._agentStats, agent).recordSuccess(elapsedMs, cost);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Record an error.
|
|
||||||
*/
|
|
||||||
recordError(model, backend, agent, isTimeout, message = "") {
|
|
||||||
this.activeRequests = Math.max(0, this.activeRequests - 1);
|
|
||||||
this.errors++;
|
|
||||||
if (isTimeout) this.timeouts++;
|
|
||||||
|
|
||||||
this._getOrCreate(this._modelStats, model).recordError(isTimeout);
|
|
||||||
this._getOrCreate(this._backendStats, backend).recordError(isTimeout);
|
|
||||||
this._getOrCreate(this._agentStats, agent).recordError(isTimeout);
|
|
||||||
|
|
||||||
this._recentErrors.push({
|
|
||||||
time: new Date().toISOString(),
|
|
||||||
message: String(message).slice(0, 200),
|
|
||||||
model, backend, agent,
|
|
||||||
});
|
|
||||||
if (this._recentErrors.length > this._maxErrors) this._recentErrors.shift();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Record session hit/miss/one-off.
|
|
||||||
*/
|
|
||||||
recordSessionHit() { this.sessionHits++; }
|
|
||||||
recordSessionMiss() { this.sessionMisses++; }
|
|
||||||
recordOneOff() { this.oneOffRequests++; }
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get per-model stats snapshot (backward compatible with v2).
|
|
||||||
*/
|
|
||||||
getModelStatsSnapshot() {
|
|
||||||
return this._snapshot(this._modelStats);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get per-backend stats snapshot.
|
|
||||||
*/
|
|
||||||
getBackendStatsSnapshot() {
|
|
||||||
return this._snapshot(this._backendStats);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get per-agent stats snapshot.
|
|
||||||
*/
|
|
||||||
getAgentStatsSnapshot() {
|
|
||||||
return this._snapshot(this._agentStats);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get recent errors.
|
|
||||||
*/
|
|
||||||
getRecentErrors(n = 5) {
|
|
||||||
return this._recentErrors.slice(-n);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get global stats (backward compatible).
|
|
||||||
*/
|
|
||||||
getGlobalStats() {
|
|
||||||
return {
|
|
||||||
totalRequests: this.totalRequests,
|
|
||||||
activeRequests: this.activeRequests,
|
|
||||||
errors: this.errors,
|
|
||||||
timeouts: this.timeouts,
|
|
||||||
sessionHits: this.sessionHits,
|
|
||||||
sessionMisses: this.sessionMisses,
|
|
||||||
oneOffRequests: this.oneOffRequests,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Internal ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_getOrCreate(map, key) {
|
|
||||||
if (!map.has(key)) map.set(key, new ModelStats());
|
|
||||||
return map.get(key);
|
|
||||||
}
|
|
||||||
|
|
||||||
_snapshot(map) {
|
|
||||||
const result = {};
|
|
||||||
for (const [key, stats] of map) {
|
|
||||||
result[key] = stats.toJSON();
|
|
||||||
}
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Per-entity stats tracker */
|
|
||||||
class ModelStats {
|
|
||||||
constructor() {
|
|
||||||
this.requests = 0;
|
|
||||||
this.successes = 0;
|
|
||||||
this.errors = 0;
|
|
||||||
this.timeouts = 0;
|
|
||||||
this.totalElapsed = 0;
|
|
||||||
this.maxElapsed = 0;
|
|
||||||
this.totalPromptChars = 0;
|
|
||||||
this.maxPromptChars = 0;
|
|
||||||
this.totalCost = 0; // accumulated estimated/actual cost
|
|
||||||
}
|
|
||||||
|
|
||||||
recordRequest(promptChars) {
|
|
||||||
this.requests++;
|
|
||||||
this.totalPromptChars += promptChars;
|
|
||||||
if (promptChars > this.maxPromptChars) this.maxPromptChars = promptChars;
|
|
||||||
}
|
|
||||||
|
|
||||||
recordSuccess(elapsedMs, cost = null) {
|
|
||||||
this.successes++;
|
|
||||||
this.totalElapsed += elapsedMs;
|
|
||||||
if (elapsedMs > this.maxElapsed) this.maxElapsed = elapsedMs;
|
|
||||||
if (cost?.value) this.totalCost += cost.value;
|
|
||||||
}
|
|
||||||
|
|
||||||
recordError(isTimeout) {
|
|
||||||
this.errors++;
|
|
||||||
if (isTimeout) this.timeouts++;
|
|
||||||
}
|
|
||||||
|
|
||||||
toJSON() {
|
|
||||||
return {
|
|
||||||
requests: this.requests,
|
|
||||||
successes: this.successes,
|
|
||||||
errors: this.errors,
|
|
||||||
timeouts: this.timeouts,
|
|
||||||
avgElapsed: this.successes > 0 ? Math.round(this.totalElapsed / this.successes) : 0,
|
|
||||||
maxElapsed: this.maxElapsed,
|
|
||||||
avgPromptChars: this.requests > 0 ? Math.round(this.totalPromptChars / this.requests) : 0,
|
|
||||||
maxPromptChars: this.maxPromptChars,
|
|
||||||
totalCost: Math.round(this.totalCost * 10000) / 10000, // 4 decimal places
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,42 +0,0 @@
|
|||||||
/**
|
|
||||||
* UnifiedChunk — the internal streaming protocol for OCP.
|
|
||||||
*
|
|
||||||
* All backend adapters MUST convert their native output into this format.
|
|
||||||
* server.mjs only handles UnifiedChunks → OpenAI SSE conversion.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @typedef {Object} UnifiedChunk
|
|
||||||
* @property {"delta"|"done"|"error"} type
|
|
||||||
* @property {string} [content] — text content (for "delta")
|
|
||||||
* @property {string} model — canonical model id
|
|
||||||
* @property {string} backend — backend id (e.g. "claude-cli")
|
|
||||||
* @property {Object} [usage] — token usage (for "done")
|
|
||||||
* @property {number} [usage.promptTokens]
|
|
||||||
* @property {number} [usage.completionTokens]
|
|
||||||
* @property {Object} [cost] — cost info (for "done")
|
|
||||||
* @property {number} [cost.value]
|
|
||||||
* @property {"actual"|"estimated"|"free"} [cost.type]
|
|
||||||
* @property {string} [error] — error message (for "error")
|
|
||||||
*/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Create a delta chunk (streaming content)
|
|
||||||
*/
|
|
||||||
export function delta(content, model, backend) {
|
|
||||||
return { type: "delta", content, model, backend };
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Create a done chunk (stream finished)
|
|
||||||
*/
|
|
||||||
export function done(model, backend, usage = null, cost = null) {
|
|
||||||
return { type: "done", model, backend, ...(usage && { usage }), ...(cost && { cost }) };
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Create an error chunk
|
|
||||||
*/
|
|
||||||
export function error(message, model, backend) {
|
|
||||||
return { type: "error", error: message, model, backend };
|
|
||||||
}
|
|
||||||
Reference in New Issue
Block a user