taodeng/olp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/olp.git synced 2026-07-21 21:15:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
de9f3ca7c90a9a6dff1d6eb1dce56751e5b5a479
olp/README.md
T
taodengandClaude Opus 4.7 cd391b13ba chore: D25 — round-2 P3 batch (F5/F6/F7/F9/F10/F11/F13 + D22 follow-up) 
cold-audit catch from 2026-05-24

Final round-2 cold-audit cleanup batch. 8 small P3 items, batched per
Iron Rule 11 IDR cleanup-batch convention (precedent: D19 / D20). No
item exceeds ~35 lines; only F9 is a code change, the rest are docs +
registry updates.

Changes (7 files, +140 / -10):

1. ALIGNMENT.md (+8 / -2)
   - **F7**: Authority pin rows for anthropic / codex / mistral updated
     from "TBD on Phase-1 spawn" to the actual citations cited in each
     plugin's header:
     · anthropic — @anthropic-ai/claude-code v2.1.89 (OCP fork audit pin)
     · openai — https://developers.openai.com/codex/cli/reference + features URL
     · mistral — https://docs.mistral.ai/mistral-vibe/terminal/quickstart + config URL
     Plugins remain Candidate (per Provider Inventory) — D25 just removes the
     `TBD` marker; Enabled transition still requires Phase audit.
   - **F6**: New 3rd entry in § One-shot Triggered Audits — "OpenAI Codex ToS
     formal pin (trigger: Phase 2 E2E enable for Codex, OR 2026-12-31)".
     Closes the cross-reference from ADR 0006 § Decision table.

2. README.md (+4 / -2)
   - **F5**: Cache-key bullet (Architecture section) replaced the stale
     7-tuple with a link to ADR 0005 § Cache key composition + the
     post-D15 11-field tuple inlined.
   - **D22 follow-up**: "328-test suite" → "Comprehensive test suite
     covering IR, cache, fallback, and integration paths" (version-less
     to prevent re-drift on every D-day).

3. docs/adr/0002-plugin-architecture.md (+4 / -2) — **F11**: Amendment 1
   wording corrected. The original Authority line + maxSpawnTimeMs
   description said "fallback engine's spawn-timeout enforcement loop"
   — but the enforcement actually lives in each provider plugin's
   `_spawnAndStream` (setTimeout + proc.kill + reject pattern). The
   fallback engine merely treats SPAWN_TIMEOUT as a hard trigger per
   ADR 0004 § Trigger taxonomy bullet 4. Both wording sites updated.

4. docs/adr/0005-cache-cross-provider.md (+1) — **F13**: Amendment 2
   gains a "Note on null-coalescing collisions" paragraph documenting
   that the `?? null` serialization treats undefined / null / [] as
   equivalent cache keys for array-typed fields. Intentional — both
   `tools: []` and `tools` omitted semantically mean "no tools." If a
   future provider distinguishes empty-array vs absent, the serialization
   needs revision.

5. models-registry.json (+35) — **F10**: 5 candidate entries added for
   the providers ALIGNMENT.md names but registry omitted. All five with
   `candidate: true`, `models: []`, tier per ALIGNMENT.md inventory:
   · grok / kimi → Tier C
   · minimax / glm / qwen → Tier B
   Closes the release_kit overlay's "Supported Providers from
   models-registry.json" claim. alignment.yml KNOWN_PROVIDERS validation
   array already includes all 8 names, so registry → workflow validation
   continues to pass.

6. server.mjs (+20 / -6) — **F9**: Streaming success path now distinguishes
   stop-terminated vs exhausted-without-stop. Pre-D25 code unconditionally
   cached after the for-await loop ended, treating any exhaustion as
   "completed." Now: only cache if `lastChunk?.type === 'stop'`. If the
   generator exhausts without emitting stop (truncation), log
   `streaming_no_stop_chunk` warn event and do NOT persist. Mirrors
   D16's buffered-path semantics ("response completed successfully (no
   truncation, no error mid-stream)") with the simpler skip-write
   pattern (streaming path doesn't use getOrCompute → no singleflight
   eviction needed). D23's cacheableForFirstHop guard preserved as the
   outer condition.

7. test-features.mjs (+78) — F9 test 15e in Suite 15:
   - Mock provider whose spawn yields delta chunks then implicit-returns
     without stop (provider-injection pattern same as 15d — the real
     plugins synthesize a stop on clean proc exit so __setSpawnImpl can't
     simulate this case)
   - Asserts response succeeds AND second identical request triggers
     fresh spawn (proves no caching happened) AND X-OLP-Cache: miss on
     both responses

Tests: 348 → 349 (+1 from 15e). All pass on Node 20.

Authority:
- F5 → ADR 0005 § Cache key composition (post-D15 Amendment 2)
- F6 → ALIGNMENT.md self + ADR 0006 self-reference
- F7 → plugin headers (verified during D25 implementation)
- F9 → ADR 0005 § Cache write conditions item 1 + D16 truncation precedent
- F10 → ALIGNMENT.md § Provider Inventory tier classification
- F11 → ADR 0004 § Trigger taxonomy bullet 4 (the actual SPAWN_TIMEOUT
  hard-trigger documentation)
- F13 → ADR 0005 Amendment 2 (extends with the null-coalescing note)
- D22 fu → no spec authority; version-less framing prevents future drift
- CC 开发铁律 v1.6 § 10.x — Round-2 Cold Audit caught all 8 items

Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): APPROVE. Independently verified F7 citations against
plugin headers (anthropic.mjs:5-6, codex.mjs:23/31, mistral.mjs:26/32);
F10 tier classifications against ALIGNMENT.md § Provider Inventory;
F6 cross-reference now self-consistent (ADR 0006 → ALIGNMENT.md);
alignment.yml workflow validation passes; F9 truncation semantics
mirror D16 buffered-path; test 15e correctly uses provider-injection
since real plugin synthesizes stop on clean exit.

Three non-blocking suggestions noted (README cache-key bullet slightly
verbose with both link + inline list; F9 could optionally synthesize
a finish_reason: 'length' stop chunk for client-visible truncation
observability; test 15e single-delta variant could be extended to
multi-delta). None folded in — all genuine polish, not correctness
gaps.

---

**Round-2 cold-audit cleanup complete.** 5 D-days (D21-D25 minus the
already-completed D24) closed all 13 round-2 findings (P2: F1/F2/F3/F4
in D21/D22/D23/D24; P3: F5-F13 distributed across D25 + earlier issues
#2/#3). v0.1 is one cold-audit-round-3 away from being ready to tag.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-24 14:56:05 +10:00

224 lines
13 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# OLP — Open LLM Proxy
A personal- and family-scale multi-provider LLM proxy. One HTTP endpoint, many subscriptions behind it, automatic routing, automatic fallback, content-addressed caching — so your IDEs and family clients keep working as long as *any* of your subscriptions has quota left.
> **Status:** v0.1 — bootstrap. Most of this README is a skeleton; sections marked _placeholder_ land alongside the relevant phase of work (see [phase plan](#phase-plan)).
---
## Why OLP
On 2026-05-14, Anthropic announced (effective 2026-06-15) that `claude -p`, the Agent SDK, and third-party agent traffic move out of the Pro/Max subscription pool into a separate fixed monthly Agent SDK Credit pool. [OCP](https://github.com/dtzp555-max/ocp), OLP's predecessor, was a proxy around a single CLI — its core assumption was *"subscription = unlimited within rate limits"*. That assumption breaks for Anthropic on the effective date.
The structural response is to stop relying on one provider's subscription terms remaining favourable. OLP spreads risk across multiple providers whose subscriptions still include CLI/programmatic use, routes intelligently between them, and caches aggressively so every request that does spawn a CLI counts.
OLP is **not**: a commercial multi-tenant SaaS; an enterprise gateway competing with LiteLLM / OpenCode / CLIProxyAPI on breadth; a model-capability router ("route to the smartest model" — you pick the model); a conversation-state store (your client handles that).
See [`ALIGNMENT.md`](./ALIGNMENT.md) for OLP's constitution and [`docs/adr/`](./docs/adr/) for the founding ADRs.
---
## Quick Start
_placeholder — lands with Phase 1._
Anticipated shape:
```bash
# install
npm install -g @dtzp555-max/olp
# run setup (writes ~/.olp/config.json, asks which providers to enable)
olp setup
# start the proxy (default port 3456 — same as OCP if you migrate)
olp start
# point your IDE at http://localhost:3456/v1/chat/completions with the OLP API key from `olp keys list`.
```
---
## Supported Providers
Source of truth: [`models-registry.json`](./models-registry.json). This table is regenerated from the registry per the [`release_kit`](./CLAUDE.md) overlay; do not edit it out of sync.
OLP distinguishes **Candidate Providers** (declared as intended, not yet pinned) from **Enabled Providers** (authority pin filled + plugin landed + Phase audit passed). The v0.1 founding commit ships **zero Enabled Providers** — enablement is a Phase audit deliverable, not a bootstrap claim. See [`ALIGNMENT.md` § Provider Inventory](./ALIGNMENT.md) for the transition gate.
### Candidate Providers
| Provider key | CLI | Subscription / auth | Anticipated Tier | Anticipated Phase |
|---|---|---|---|---|
| `anthropic` | `claude -p` | Pro / Max OAuth (pre-2026-06-15); Agent SDK Credit pool after | D (re-eval post-2026-06-15) | Phase 1 |
| `openai` | `codex exec --json` | ChatGPT Pro OAuth or API key | D | Phase 2 |
| `mistral` | `vibe --prompt --output json` | Le Chat Pro API key | D | Phase 3 |
| `grok` | `grok -p --output-format streaming-json` | xAI Build `xai-...` API key | C | Phase 8+ |
| `kimi` | `kimi -p --output-format stream-json` | Moonshot Kimi API key | C | Phase 8+ |
| `minimax` | TBD | MiniMax Token Plan (¥29+/mo) | B | Phase 8+ |
| `glm` | TBD | Zhipu Coding Plan ($10+/mo) | B | Phase 8+ |
| `qwen` | TBD | Alibaba Coding Plan ($50/mo) | B | Phase 8+ |
**Risk tier guide.** D = permissive / safe (eligible for default-enabled); C = tightening signal, no enforcement history (opt-in); B = service-level key revocation risk (opt-in + consent); A = excluded by default (cannot be opt-in enabled). Tier B providers prompt for explicit consent on first enable and record consent in `~/.olp/config.json`. See [`ALIGNMENT.md` § Risk Tier Framework](./ALIGNMENT.md#risk-tier-framework).
**Excluded by default (Tier A — evidence-backed, pending primary-source pin).** Google Antigravity. See [ADR 0006](./docs/adr/0006-provider-inclusion.md) for the named-prohibition + no-cost-advantage + reinstatement-friction rationale, and for the primary-source pinning follow-up that may force a Tier reconsideration if the Google FAQ language cannot be sourced within 90 days of 2026-05-23.
---
## Configuration
_placeholder — full configuration reference lands with Phase 4 (fallback engine)._
OLP reads its config from `~/.olp/config.json`. The minimum useful shape:
```json
{
"routing": {
"chains": {
"<requested-model>": [
{ "provider": "<key>", "model": "<provider-model-id>" },
{ "provider": "<key>", "model": "<provider-model-id>" }
]
},
"soft_triggers": {
"<provider-key>": { "<trigger>": <threshold> }
}
}
}
```
> **Note:** `routing.soft_triggers` thresholds are parsed and stored but have **no runtime effect at v0.1** — the quota polling path (`quotaStatus()` per hop) is deferred to v1.x per [ADR 0004 Amendment 2](./docs/adr/0004-fallback-engine.md#amendment-2--2026-05-24-soft-triggers-deferred-to-v1x-d22). The evaluation logic exists and is tested; only the production data ingestion path is deferred.
Trigger types, fallback safety, idempotency rules, and the full example config land here when Phase 4 ships. See [ADR 0004 (Fallback Engine Semantics & Safety)](./docs/adr/0004-fallback-engine.md) for the design.
---
## API Endpoints
_placeholder — full table lands as each endpoint lands._
| Endpoint | Method | Phase | Description |
|---|---|---|---|
| `/v1/chat/completions` | POST | 1 | OpenAI-compatible Chat Completions entry. Internally normalized to IR, dispatched to a provider plugin, response shape converted back. |
| `/v1/models` | GET | 1 | Lists models from `models-registry.json`. |
| `/health` | GET | 1 | Per-provider health snapshot (owner-only). |
| `/cache/stats` | GET | 5 | Cache hit rate, by-provider breakdown. |
| `/v0/management/quota` | GET | 6 | Per-provider quota / credit pool status (best-effort). |
| `/dashboard` | GET | 6 | Owner-only dashboard (localhost-bound by default). |
---
## Environment Variables
_placeholder — full table lands per-phase as variables are introduced._
| Variable | Default | Description |
|---|---|---|
| `OLP_PORT` | `3456` | HTTP listener port. |
| `OLP_HOME` | `~/.olp` | Config, providers, keys, cache, logs root. |
| `OLP_LOG_LEVEL` | `info` | One of `error`, `warn`, `info`, `debug`. |
Further variables (per-provider auth path overrides, cache size limits, fallback-engine knobs) land with the relevant phase.
---
## Response Headers
Every response served through OLP carries:
- `X-OLP-Provider-Used: <provider-key>` — which provider's plugin served the request.
- `X-OLP-Model-Used: <model-id>` — which model the served provider used.
- `X-OLP-Fallback-Hops: <n>` — number of fallback hops (`0` if served by the primary chain entry).
- `X-OLP-Cache: hit | miss | bypass` — cache layer outcome.
- `X-OLP-Latency-Ms: <ms>` — end-to-end latency observed at the proxy.
If a fallback chain is exhausted, `X-OLP-Fallback-Exhausted` lists the tried providers in order.
---
## Implementation status (as of 2026-05-24)
Phase 1 is in progress. This table reflects what is currently shipped vs. what is designed for later phases.
| File / artifact | Status | Notes |
|---|---|---|
| `server.mjs` | ✅ Shipped | HTTP listener + dispatcher |
| `lib/ir/` | ✅ Shipped | IR definition + serializers (ADR 0003) |
| `lib/providers/anthropic.mjs` | ✅ Shipped | `claude -p` spawn-binary plugin |
| `lib/providers/codex.mjs` | ✅ Shipped | `codex exec --json` plugin |
| `lib/providers/mistral.mjs` | ✅ Shipped | `vibe --prompt` plugin |
| `lib/cache/keys.mjs` | ✅ Shipped | Content-addressed key computation |
| `lib/cache/store.mjs` | ✅ Shipped | In-memory Map (file-backed layout: 📋 Phase 2 storage adapter) |
| `lib/fallback/engine.mjs` | ✅ Shipped | Trigger evaluation + chain advancement (ADR 0004) |
| Soft trigger data path (`quotaStatus()` polling) | 📋 Planned (v1.x) | Evaluation logic shipped + tested; data ingestion deferred per ADR 0004 Amendment 2 |
| `models-registry.json` | ✅ Shipped | SPOT for `(provider, model)` metadata |
| `test-features.mjs` | ✅ Shipped | Comprehensive test suite covering IR, cache, fallback, and integration paths (CI: `test.yml`) |
| `lib/keys.mjs` | 📋 Planned (Phase 2) | Multi-key auth, per-key namespacing, audit log |
| `dashboard.html` | 📋 Planned (Phase 6) | Owner-only multi-provider dashboard |
| `docs/provider-caveats.md` | 📋 Planned (Phase 3+) | Lossy-translation reference; for now documented inline in each plugin header |
| `docs/openai-spec-pin.md` | 📋 Planned (Phase 1 gate) | OpenAI spec snapshot for annual audit; deferred from v0.1 bootstrap |
| `docs/alignment-audits/` | 📋 Planned | Output directory for annual alignment audits (first audit: 2027-05-14) |
| `scripts/migrate-from-ocp.mjs` | 📋 Planned (Phase 7) | OCP → OLP migration tool |
| `setup.mjs` | 📋 Planned | Setup wizard / initial config |
---
## Architecture
OLP is a Node.js (ESM, `.mjs`) HTTP proxy with no build step and minimal dependencies. The high-level shape:
- **Entry surface** — `server.mjs` handles `/v1/chat/completions` and the administrative endpoints. Governed by OpenAI's `/v1/chat/completions` specification as the wire authority. See [`ALIGNMENT.md` § Authorities](./ALIGNMENT.md#authorities).
- **Intermediate Representation (IR)** — `lib/ir/` normalizes between the entry surface and provider-native shapes. The IR is the lingua franca; any extension is an [ADR 0003](./docs/adr/0003-intermediate-representation.md) amendment.
- **Provider plugins** — `lib/providers/<name>.mjs`. Each plugin implements the contract in [ADR 0002 (Plugin Architecture for Providers)](./docs/adr/0002-plugin-architecture.md), spawns its CLI, and translates between IR and provider-native IO.
- **Cache layer** — `lib/cache/` is a content-addressed cache keyed on the 11-field tuple defined in [ADR 0005 § "Cache key composition (v1.0)"](./docs/adr/0005-cache-cross-provider.md#decision) (provider, model, messages, tools, temperature, response_format, cache_control, max_tokens, top_p, stop, tool_choice — as of Amendment 2/D15). Per-key isolation, prompt-caching bypass, chunked stream replay, and singleflight. See [ADR 0005 (Cache Layer Cross-Provider Design)](./docs/adr/0005-cache-cross-provider.md). Current backing store is an in-memory Map; file-backed storage is planned for Phase 2.
- **Fallback engine** — `lib/fallback/` advances a configured chain one provider at a time on configured triggers, never retrying after the first response chunk has been emitted to the client. See [ADR 0004](./docs/adr/0004-fallback-engine.md).
- **Multi-key auth** — `lib/keys.mjs` (📋 planned, Phase 2) will carry OCP's per-OLP-key namespace isolation forward. Each OLP API key will have independent quota, cache namespace, and audit log; each key declares which providers it can access.
Read the ADRs in `docs/adr/` in order before proposing structural changes.
---
## Phase plan
OLP lands in phases. Each phase has its own PR series and Iron-Rule-10 reviewer; this README's placeholders are filled per-phase via the [`release_kit`](./CLAUDE.md) overlay.
- Phase 0 — Repo bootstrap, `ALIGNMENT.md`, founding ADRs, CI workflows, PR template. **(current)**
- Phase 1 — `server.mjs` skeleton, IR, Anthropic plugin, cache D1+D4. Port from OCP.
- Phase 2 — OpenAI Codex plugin.
- Phase 3 — Mistral Vibe plugin.
- Phase 4 — Fallback engine + routing chains config + quota poll worker.
- Phase 5 — Cache cross-provider hardening (D2+D3).
- Phase 6 — Dashboard + observability (`/v0/management/quota`).
- Phase 7 — Release v0.1, OCP enters maintenance.
- Phase 8+ — Optional Grok / Kimi / tier-2 plugins; provider-native protocol endpoints; deterministic triggers.
Full spec (decision rationale, open questions, risks): `~/.cc-rules/memory/projects/olp_v0_1_spec.md` on the maintainer's workstations.
---
## Migration from OCP
_placeholder — `scripts/migrate-from-ocp.mjs` lands with Phase 7 (📋 planned, not yet authored)._
Anticipated user-facing flow (target: <5 minutes):
1. Stop OCP (`launchctl bootout` the OCP service or `ocp stop`).
2. Install OLP.
3. Run `olp migrate-from-ocp` — copies `~/.ocp/keys/` to `~/.olp/keys/` and points provider plugins at OCP's existing auth artifacts where applicable.
4. Start OLP. Clients pointing at port 3456 keep working; their existing OLP API keys remain valid.
OCP's cache directory is *not* migrated: OLP's cache key format includes provider+model and warms cold naturally. OCP enters maintenance mode (stability fixes only) when OLP v0.1 ships; new development happens in OLP.
---
## License
MIT.
---
## Acknowledgements
OLP evolved from [OCP (Open Claude Proxy)](https://github.com/dtzp555-max/ocp). OCP's per-key isolation model, cache-layer design (D1–D4), dashboard, and alignment-constitution discipline are all carried forward. The structural generalization from single-CLI to multi-provider is what makes this a new project rather than an OCP minor version — see [`ALIGNMENT.md` § Reference: How OCP's `cli.js` discipline maps to OLP](./ALIGNMENT.md#reference-how-ocps-clijs-discipline-maps-to-olp).
Authors: project maintainer (with AI drafting assistance).
Reference in New Issue View Git Blame Copy Permalink