From 0041fb10179e9d14876d01bc225798d606f86f69 Mon Sep 17 00:00:00 2001 From: dtzp555 Date: Sat, 23 May 2026 15:46:56 +1000 Subject: [PATCH] =?UTF-8?q?chore:=20bootstrap=20OLP=20v0.1=20=E2=80=94=20m?= =?UTF-8?q?ulti-provider=20LLM=20proxy?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Initial release. OLP (Open LLM Proxy) is a personal- and family-scale multi-provider LLM proxy that supersedes OCP (Open Claude Proxy). Trigger: Anthropic's 2026-05-14 announcement (effective 2026-06-15) moves `claude -p` / Agent SDK / third-party agent traffic out of the Pro/Max subscription pool into a separate fixed monthly Agent SDK Credit pool. OCP's foundational assumption ("subscription = unlimited within rate limits") breaks for Anthropic on that date. Spreading risk across multiple providers is the structural response. Phase 0 lands: - ALIGNMENT.md (constitution: 5 Rules, 3 Authorities, 4-tier Risk Framework, 8-provider inventory) - AGENTS.md (multi-tool agent guidelines; inherits cc-rules) - CLAUDE.md (Claude-Code session instructions + release_kit overlay) - README.md (phase-aware skeleton) - docs/adr/0001-0006 (Founding ADRs: project founding / plugin architecture / IR design / fallback engine / cross-provider cache / provider inclusion + risk-tier framework) - .github/PULL_REQUEST_TEMPLATE.md (8-radio Change Type + per-type Authority Evidence + Iron Rule 10 reviewer checklist) - .github/workflows/alignment.yml (blacklist + Antigravity exclusion enforcement + models-registry validator + commit-citation soft check) - .github/workflows/release.yml (auto-release on tag with version match check per Iron Rule 5) - .github/workflows/test.yml (Node 20/24 matrix, bootstrap-tolerant) - package.json, .gitignore, LICENSE (MIT), CHANGELOG.md Provider inventory at bootstrap: Tier D (default-enabled): anthropic, openai, mistral Tier C (opt-in): grok, kimi Tier B (opt-in + consent): minimax, glm, qwen Tier A (permanently excluded): google-antigravity Supersedes OCP ADR 0005 (No Multi-Provider) per OLP ADR 0001. OCP will enter maintenance mode when OLP v0.1 ships per Phase 7 plan. Iron Rule 10 gate: fresh-context independent opus reviewer audited all 15 governance files against OLP v0.1 spec + OCP precedent. Verdict: APPROVE_WITH_MINOR. Two minor findings folded in: 1. alignment.yml heredoc EOF moved to column 0 (was indented; bash parse failed silently on real blacklist hits, printing a cryptic "syntax error" instead of the structured ALIGNMENT GUARDRAIL FAILURE banner). 2. AGENTS.md clarified that the SPOT discipline for models-registry.json will be codified by a Phase-1 ADR (OLP ADR 0003 is currently the IR design, not a SPOT codification; OCP's ADR 0003 is the precedent but OLP's registry shape differs and warrants its own ADR). Co-Authored-By: Claude Opus 4.7 --- .github/PULL_REQUEST_TEMPLATE.md | 125 +++++++++++ .github/workflows/alignment.yml | 213 +++++++++++++++++++ .github/workflows/release.yml | 73 +++++++ .github/workflows/test.yml | 53 +++++ .gitignore | 25 +++ AGENTS.md | 85 ++++++++ ALIGNMENT.md | 168 +++++++++++++++ CHANGELOG.md | 43 ++++ CLAUDE.md | 109 ++++++++++ LICENSE | 21 ++ README.md | 190 +++++++++++++++++ docs/adr/0001-project-founding.md | 69 ++++++ docs/adr/0002-plugin-architecture.md | 101 +++++++++ docs/adr/0003-intermediate-representation.md | 109 ++++++++++ docs/adr/0004-fallback-engine.md | 108 ++++++++++ docs/adr/0005-cache-cross-provider.md | 104 +++++++++ docs/adr/0006-provider-inclusion.md | 113 ++++++++++ docs/adr/README.md | 37 ++++ package.json | 30 +++ 19 files changed, 1776 insertions(+) create mode 100644 .github/PULL_REQUEST_TEMPLATE.md create mode 100644 .github/workflows/alignment.yml create mode 100644 .github/workflows/release.yml create mode 100644 .github/workflows/test.yml create mode 100644 .gitignore create mode 100644 AGENTS.md create mode 100644 ALIGNMENT.md create mode 100644 CHANGELOG.md create mode 100644 CLAUDE.md create mode 100644 LICENSE create mode 100644 README.md create mode 100644 docs/adr/0001-project-founding.md create mode 100644 docs/adr/0002-plugin-architecture.md create mode 100644 docs/adr/0003-intermediate-representation.md create mode 100644 docs/adr/0004-fallback-engine.md create mode 100644 docs/adr/0005-cache-cross-provider.md create mode 100644 docs/adr/0006-provider-inclusion.md create mode 100644 docs/adr/README.md create mode 100644 package.json diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..61bc3e7 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,125 @@ +# Pull Request + +## Summary + + + +## Change Type (REQUIRED) + +Pick the most specific applicable type. PRs touching multiple types must fill the evidence section for each (Hybrid covers spans of changes across types): + +- [ ] **Provider Plugin** — change to `lib/providers/.mjs`. Specify which: + - [ ] `anthropic` + - [ ] `openai` + - [ ] `mistral` + - [ ] `grok` + - [ ] `kimi` + - [ ] `minimax` + - [ ] `glm` + - [ ] `qwen` +- [ ] **Entry Surface** — change to a request handler in `server.mjs`. Specify which endpoint: + - [ ] `/v1/chat/completions` + - [ ] `/health` + - [ ] `/v1/models` + - [ ] `/cache/stats` + - [ ] `/v0/management/quota` + - [ ] other (specify in Summary) +- [ ] **IR** — change to `lib/ir/*`. Requires an ADR amendment (existing or co-merged in this PR). +- [ ] **Fallback Engine** — change to `lib/fallback/*`. Cite ADR 0004. +- [ ] **Cache Layer** — change to `lib/cache/*`. Cite ADR 0005. +- [ ] **Governance / docs / CI** — change to `ALIGNMENT.md`, `AGENTS.md`, `CLAUDE.md`, `docs/`, or `.github/workflows/*`. +- [ ] **Not source-touching** — test infra, scripts that do not touch any of the above, dependency hygiene. +- [ ] **Hybrid** — spans multiple types above. Fill every applicable evidence section. + +## Authority Evidence (REQUIRED) + +PRs with the relevant evidence section blank or unchecked will receive a `request changes` review and cannot be merged. Per `ALIGNMENT.md` Rules 1 and 5. + +### If Provider Plugin + +- [ ] **Provider CLI documentation citation.** I have identified the provider CLI documentation page (URL) or observed-behaviour transcript that this change reflects. CLI version audited: + + +- [ ] **Behaviour the change reflects.** I have described in one sentence what the provider CLI actually does that this PR matches. + + +- [ ] **If the provider CLI does not perform this operation**, I have stated this explicitly below and justified the scope under `ALIGNMENT.md` Rule 2. (Note: in almost all cases this means the PR should be closed, not merged. Provider plugins do not invent CLI behaviour. If the desired behaviour belongs at the entry surface or IR layer, switch the Change Type above accordingly.) + + +- [ ] **Commit message citations.** Every "Provider X uses Y" or " CLI uses Y" assertion in every commit of this PR is immediately followed by a citation (CLI version + section, docs URL, or observed-transcript reference). I have verified this by rereading each commit message. + +### If Entry Surface + +- [ ] **OpenAI spec citation.** I have linked to the relevant section of OpenAI's `/v1/chat/completions` specification, including the specific field, parameter, or behaviour being implemented. + + +- [ ] **No invention beyond the spec.** I confirm this PR does not introduce any field or behaviour not present in OpenAI's spec for the endpoint. If something the user actually wants is not in the spec, the right answer is to close this PR and propose an upstream spec change or move the behaviour to the IR with an ADR. + +### If IR + +- [ ] **Authorizing ADR.** Cite the ADR that authorizes the IR change. If this PR introduces a new IR field or alters semantics, the amending ADR must land in this PR or before it. + + +### If Fallback Engine + +- [ ] **ADR 0004 citation.** Cite the specific section of ADR 0004 (Fallback Engine Semantics & Safety) the change implements or amends. + + +- [ ] **Trigger documentation.** If this PR adds a new trigger type, document it in `README.md § Configuration` and add a test case. + +### If Cache Layer + +- [ ] **ADR 0005 citation.** Cite the specific section of ADR 0005 (Cache Layer Cross-Provider Design) the change implements or amends. + + +- [ ] **Cache-key impact.** Confirm whether this PR changes the cache key formula. If yes, document in CHANGELOG that existing cache entries will warm-cold-start. + +## Type of change + +- [ ] Bug fix (alignment with existing provider CLI behaviour, OpenAI spec, or cited ADR) +- [ ] Feature (new provider plugin / new IR field / new fallback trigger / new endpoint already in OpenAI's spec) +- [ ] Refactor (no wire-level behaviour change) +- [ ] Deletion (unalignable change removal per `ALIGNMENT.md` Unalignable Policy) +- [ ] Documentation / governance + +## Reviewer checklist (Iron Rule 10) + +Reviewers: this section is for you, not the author. Do not approve until every box is checked. + +- [ ] I opened the cited authority and confirmed the change matches: + - Provider Plugin → opened the provider CLI doc page / verified the observed-behaviour transcript at the pinned CLI version. + - Entry Surface → opened the OpenAI spec at the cited section. + - IR → opened the cited ADR. + - Fallback / Cache → opened the cited ADR section. +- [ ] I ran (or confirmed CI ran) `.github/workflows/alignment.yml` and it passed. +- [ ] I am not the commit author of any commit in this PR (Iron Rule 10). +- [ ] If the PR asserts scope without a canonical citation, I confirmed the justification is sound per `ALIGNMENT.md` Rule 2. +- [ ] If the PR adds a new provider plugin, I confirmed the ADR 0006 risk-tier classification is in the same merge or before. +- [ ] If the PR adds a new IR field, I confirmed the amending ADR lands in the same merge or before. + +## Related + +- `ALIGNMENT.md` Rule(s) invoked: +- Authority cited: +- Related issue / prior PR: + +### User-visible change self-check (release_kit § new_feature_doc_expectations) + +- [ ] This PR has user-visible changes → README has corresponding documentation (paste diff link or line range) +- [ ] This PR adds a new provider plugin → README "Supported Providers" entry present + ADR 0006 risk-tier classification entry present +- [ ] This PR adds a new fallback trigger → README "Configuration" entry present + test added +- [ ] This PR adds a new env var → README "Environment Variables" table entry present +- [ ] This PR adds a new endpoint → README "API Endpoints" table entry present +- [ ] This PR has no user-visible changes → stated "no user-visible change" in summary above + +Reviewers: if any "user-visible" box is checked but README diff is empty, block merge. + +### Privacy self-check (for PUBLIC repos) + +- [ ] This PR does not introduce real names, nicknames, or handles that identify specific individuals. All references use role-based terms (`project maintainer`, `contributor`, `user`, `reviewer`). +- [ ] This PR does not introduce literal personal paths (`/Users//`, `/home//`). Uses `$HOME/` or `~/` instead. +- [ ] This PR does not introduce personal machine hostnames. Uses role-based names or generic descriptors. +- [ ] This PR does not introduce personal email addresses beyond automated placeholders like `noreply@.com`. +- [ ] This PR does not introduce literal credentials, tokens, or API keys (even expired ones). + +Reviewers: if any of the above is violated and the repo is PUBLIC, block merge and request scrub. diff --git a/.github/workflows/alignment.yml b/.github/workflows/alignment.yml new file mode 100644 index 0000000..e2c0449 --- /dev/null +++ b/.github/workflows/alignment.yml @@ -0,0 +1,213 @@ +name: Alignment Guardrail + +on: + pull_request: + paths: + - 'server.mjs' + - 'setup.mjs' + - 'lib/**' + - 'scripts/**' + - 'models-registry.json' + - '.github/workflows/alignment.yml' + push: + branches: [main] + paths: + - 'server.mjs' + - 'setup.mjs' + - 'lib/**' + - 'scripts/**' + - 'models-registry.json' + - '.github/workflows/alignment.yml' + +jobs: + blacklist: + name: source blacklist (hard fail) + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Scan source for hallucinated tokens + shell: bash + run: | + set -euo pipefail + + # Known-hallucinated tokens. Extend only via an ALIGNMENT.md amendment PR. + # Each token is matched as a fixed string against the OLP source tree + # (excluding docs/, CHANGELOG, README, this workflow, and tests that + # may legitimately reference the historical token as a guardrail). + # + # Inherited transitively from OCP's 2026-04-11 drift: + # - api.anthropic.com/api/oauth/usage : fabricated Anthropic OAuth + # usage endpoint. Does not appear in any shipped @anthropic-ai/ + # claude-code cli.js. Carried forward as a transitive guardrail. + # + # OLP-native entries: added as drift incidents accumulate. + BLACKLIST=( + "api.anthropic.com/api/oauth/usage" + ) + + # Provider keys that may appear in source (positive list — present in + # ALIGNMENT.md provider inventory). Any provider key in source that + # is NOT in this list is suspicious and flagged below. + KNOWN_PROVIDERS=( + "anthropic" + "openai" + "mistral" + "grok" + "kimi" + "minimax" + "glm" + "qwen" + ) + + # Source files in scope. Exclude docs, CHANGELOG, README, the + # workflow itself, and the test file (which may pin historical + # strings intentionally). + SOURCE_FILES="$(git ls-files \ + | grep -E '\.(mjs|js|ts|json)$' \ + | grep -v -E '^(docs/|CHANGELOG\.md|README\.md|test-features\.mjs|\.github/workflows/alignment\.yml)')" + + FAIL=0 + + # 1. Blacklist scan + for token in "${BLACKLIST[@]}"; do + if echo "$SOURCE_FILES" | xargs grep -n -F "$token" 2>/dev/null; then + echo "::error::Blacklisted token '$token' detected in OLP source." + FAIL=1 + fi + done + + # 2. Excluded-provider scan: Google Antigravity is permanently + # excluded per ADR 0006 / ALIGNMENT.md Risk Tier A. Any reference + # to a `google-antigravity` plugin file or provider key in source + # (outside docs which may discuss the exclusion) is a finding. + FORBIDDEN_PROVIDER_TOKENS=( + "google-antigravity" + "antigravity" + ) + for token in "${FORBIDDEN_PROVIDER_TOKENS[@]}"; do + HITS="$(echo "$SOURCE_FILES" | xargs grep -n -F "$token" 2>/dev/null || true)" + if [ -n "$HITS" ]; then + echo "::error::Tier-A-excluded provider token '$token' detected in OLP source. Per ALIGNMENT.md / ADR 0006, this provider is permanently excluded." + echo "$HITS" + FAIL=1 + fi + done + + if [ "$FAIL" -ne 0 ]; then + cat <<'EOF' + + ============================================================ + ALIGNMENT GUARDRAIL FAILURE + ============================================================ + OLP source contains a token on the alignment blacklist or + references a permanently excluded provider. + + Blacklist tokens were introduced by LLM hallucinations and + do not appear in the relevant authority (provider CLI, + OpenAI spec, or ADR). See ALIGNMENT.md and (where the + token is inherited from OCP) the OCP 2026-04-11 drift + record at https://github.com/dtzp555-max/ocp. + + Excluded providers are listed in ALIGNMENT.md § Risk Tier + Framework and ADR 0006. Permanent exclusion means not + bundled, not pluggable, not added via opt-in. + + Required action: + 1. Remove the token from source. + 2. Cite the real authority (provider CLI doc / OpenAI + spec section / ADR) for the operation you intended. + 3. See ALIGNMENT.md Rules 1, 2, and 5. + + Do not add allowlist entries to this workflow without an + amendment PR to ALIGNMENT.md (see Amendment Procedure). + ============================================================ +EOF + exit 1 + fi + + echo "Blacklist + excluded-provider scan clean." + + models-registry: + name: models-registry.json sanity (hard fail) + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Validate models-registry.json + shell: bash + run: | + set -euo pipefail + + if [ ! -f models-registry.json ]; then + echo "models-registry.json not found yet (bootstrap phase). Skipping." + exit 0 + fi + + # Basic JSON validity. + if ! node -e "JSON.parse(require('fs').readFileSync('models-registry.json','utf8'))"; then + echo "::error::models-registry.json is not valid JSON." + exit 1 + fi + + # Provider keys in models-registry.json must match the inventory + # in ALIGNMENT.md. + KNOWN_PROVIDERS='["anthropic","openai","mistral","grok","kimi","minimax","glm","qwen"]' + node -e " + const fs = require('fs'); + const known = ${KNOWN_PROVIDERS}; + const reg = JSON.parse(fs.readFileSync('models-registry.json','utf8')); + const providers = reg.providers || {}; + const bad = Object.keys(providers).filter(p => !known.includes(p)); + if (bad.length > 0) { + console.error('::error::Unknown provider key(s) in models-registry.json: ' + bad.join(', ')); + console.error('Known providers per ALIGNMENT.md: ' + known.join(', ')); + process.exit(1); + } + console.log('models-registry.json provider keys OK: ' + Object.keys(providers).join(', ')); + " + + commit-citation: + name: per-provider commit citation (soft check) + runs-on: ubuntu-latest + continue-on-error: true + steps: + - name: Checkout full history + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Scan PR commits for uncited assertions + if: github.event_name == 'pull_request' + shell: bash + env: + BASE_SHA: ${{ github.event.pull_request.base.sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + run: | + set -euo pipefail + + if [ -z "${BASE_SHA:-}" ] || [ -z "${HEAD_SHA:-}" ]; then + echo "No PR context; skipping." + exit 0 + fi + + WARN=0 + git log --format="%H" "${BASE_SHA}..${HEAD_SHA}" | while read -r sha; do + BODY="$(git log -1 --format=%B "$sha")" + if echo "$BODY" | grep -E -i -q '(provider|claude|codex|vibe|grok|kimi|minimax|glm|qwen|cli)[[:space:]]+(code[[:space:]]+)?uses'; then + if echo "$BODY" | grep -E -i -q '(cli[[:space:]]+v[0-9]+|https?://|ADR[[:space:]]+[0-9]{4})'; then + echo "OK $sha: assertion cited." + else + echo "::warning::Commit $sha asserts 'Provider X uses ...' or ' CLI uses ...' but does not cite a CLI version, docs URL, or ADR number. See CLAUDE.md -> Commit message conventions." + WARN=1 + fi + fi + done + + if [ "$WARN" -ne 0 ]; then + echo "Soft check raised warnings. Reviewer: please enforce per CLAUDE.md." + else + echo "Commit citation soft check clean." + fi diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..5227335 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,73 @@ +name: Auto Release on Tag + +on: + push: + tags: + - 'v*.*.*' + +permissions: + contents: write + +jobs: + release: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Extract version from tag + id: ver + run: echo "version=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT + + - name: Verify package.json version matches tag + shell: bash + run: | + set -euo pipefail + if [ ! -f package.json ]; then + echo "::warning::package.json not found; skipping version-match check." + exit 0 + fi + PKG_VERSION="$(node -p "require('./package.json').version")" + TAG_VERSION="${{ steps.ver.outputs.version }}" + if [ "$PKG_VERSION" != "$TAG_VERSION" ]; then + echo "::error::Tag v${TAG_VERSION} does not match package.json version ${PKG_VERSION}. Bump package.json before tagging (Iron Rule 5)." + exit 1 + fi + echo "Tag v${TAG_VERSION} matches package.json version ${PKG_VERSION}." + + - name: Extract CHANGELOG section + id: notes + shell: bash + run: | + set -euo pipefail + VERSION="${{ steps.ver.outputs.version }}" + if [ ! -f CHANGELOG.md ]; then + echo "No CHANGELOG.md found; using minimal release notes" + echo "Release v${VERSION}" > /tmp/release-notes.md + exit 0 + fi + awk -v ver="v${VERSION}" ' + $0 ~ "^## " ver { found=1; print; next } + found && /^## v/ { exit } + found { print } + ' CHANGELOG.md > /tmp/release-notes.md + if [ ! -s /tmp/release-notes.md ]; then + echo "No matching section in CHANGELOG for v${VERSION}; using minimal notes" + echo "Release v${VERSION}" > /tmp/release-notes.md + fi + + - name: Create GitHub Release + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + VERSION="${{ steps.ver.outputs.version }}" + if gh release view "v${VERSION}" >/dev/null 2>&1; then + echo "Release v${VERSION} already exists - skipping" + exit 0 + fi + gh release create "v${VERSION}" \ + --title "v${VERSION}" \ + --notes-file /tmp/release-notes.md \ + --latest diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml new file mode 100644 index 0000000..9ac44d0 --- /dev/null +++ b/.github/workflows/test.yml @@ -0,0 +1,53 @@ +name: Tests + +on: + push: + branches: [main] + pull_request: + +jobs: + test-features: + name: test-features.mjs (smoke) + runs-on: ubuntu-latest + + strategy: + matrix: + node-version: ['20', '24'] + fail-fast: false + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Node ${{ matrix.node-version }} + uses: actions/setup-node@v4 + with: + node-version: ${{ matrix.node-version }} + + - name: Detect package-lock + id: detect_lock + shell: bash + run: | + if [ -f package-lock.json ]; then + DEPS="$(node -p "Object.keys(require('./package.json').dependencies || {}).length")" + if [ "$DEPS" -gt 0 ]; then + echo "needs_install=true" >> $GITHUB_OUTPUT + else + echo "needs_install=false" >> $GITHUB_OUTPUT + fi + else + echo "needs_install=false" >> $GITHUB_OUTPUT + fi + + - name: npm install (if dependencies present) + if: steps.detect_lock.outputs.needs_install == 'true' + run: npm ci + + - name: Run npm test + shell: bash + run: | + if [ ! -f test-features.mjs ] && ! node -e "process.exit(require('./package.json').scripts && require('./package.json').scripts.test ? 0 : 1)" 2>/dev/null; then + echo "::notice::No test-features.mjs or npm test script yet (bootstrap phase). Skipping." + exit 0 + fi + npm test diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..8cfe824 --- /dev/null +++ b/.gitignore @@ -0,0 +1,25 @@ +node_modules/ +*.log +.DS_Store +.env +.env.local +.env.*.local + +# Local OLP state (config, providers auth, keys, cache, logs) lives under +# ~/.olp/ on user machines, NOT inside this repo. Block any local copy here. +.olp/ + +# Editor / OS +.vscode/ +.idea/ +*.swp +*.swo +*~ + +# Test outputs / coverage +coverage/ +*.lcov + +# Stage / scratch +.staging/ +.scratch/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..47fab4b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,85 @@ +Inherits: @~/.cc-rules/AGENTS.md + +# OLP — Open LLM Proxy — Agent Guidelines + +**Scope**: the `dtzp555-max/olp` repository. +**Audience**: any AI coding agent (Claude Code / Cursor / OpenCode / Copilot / Codex / Gemini) touching OLP source. + +--- + +## What this project is + +OLP (Open LLM Proxy) is a personal- and family-scale multi-provider LLM proxy. It exposes a single OpenAI-compatible HTTP endpoint (`/v1/chat/completions`) and routes each request to one of N provider plugins, each of which spawns the corresponding provider CLI (e.g. `claude -p`, `codex exec --json`, `vibe --prompt`). An IR (Intermediate Representation) normalizes between the entry surface and each provider's native shape. Intelligent fallback chains advance one provider at a time on configured triggers; content-addressed caching minimizes quota consumption. + +OLP supersedes OCP (Open Claude Proxy) as of v1.0. The trigger was the 2026-05-14 Anthropic announcement (effective 2026-06-15) splitting CLI/Agent SDK traffic out of the Pro/Max subscription pool — OCP's foundational assumption ("subscription = unlimited within rate limits") broke for Anthropic on that date. Spreading risk across multiple providers is the structural response. + +OLP is **not** a commercial multi-tenant SaaS, **not** an enterprise gateway competing with LiteLLM/OpenCode/CLIProxyAPI on breadth, **not** a model-capability router ("route to the smartest model"), and **not** a conversation-state store (clients manage their own state). See `ALIGNMENT.md` Core Principle and `docs/adr/0001-project-founding.md`. + +Runtime: Node.js (ESM, `.mjs` throughout). No build step. No bundler. `server.mjs` is the single executable entrypoint. The architecture is spawn-binary: OLP spawns the real provider CLI for every uncached request. + +--- + +## Stack + +- Node.js >=18, native ESM modules +- `http`/`https` built-ins for the proxy core (no Express, no Fastify) +- `models-registry.json` as the single source of truth for `(provider, model) → metadata` mappings (analogous to OCP's `models.json`; SPOT discipline will be codified in a Phase 1 ADR — OLP ADR 0003 is currently the IR design, not the SPOT codification) +- GitHub Actions for CI (`alignment.yml`, `release.yml`, `test.yml`) +- `gh` CLI assumed for PR creation and release automation +- No TypeScript. No test framework beyond `test-features.mjs` (run via `npm test`; CI workflow `.github/workflows/test.yml`). Keep dependencies minimal. + +--- + +## Key files to know + +- `server.mjs` — HTTP listener, entry surface (`/v1/chat/completions`, `/health`, `/v1/models`, etc.), plugin loader, request dispatch. Governed by `ALIGNMENT.md`. +- `lib/providers/` — per-provider plugins. Each file (`anthropic.mjs`, `openai.mjs`, `mistral.mjs`, …) implements the Provider contract documented in ADR 0002. +- `lib/ir/` — Intermediate Representation definition + serializers. Governed by ADR 0003. +- `lib/cache/` — content-addressed cache layer (per-key isolation, `cache_control` bypass, chunked stream replay, singleflight). Governed by ADR 0005. +- `lib/fallback/` — fallback engine (trigger detection, chain advancement, idempotent-failure safety, header annotation). Governed by ADR 0004. +- `lib/keys.mjs` — multi-key auth, per-key namespacing, audit log. Carries OCP's per-key isolation model into OLP. +- `dashboard.html` — owner-only multi-provider dashboard (quota panels, fallback rate, cache hit rate). +- `models-registry.json` — single source of truth for `(provider, model) → metadata`. SPOT. +- `ALIGNMENT.md` — the constitution. Binding for any plugin / entry-surface / IR change. +- `docs/adr/` — Architecture Decision Records. Read the index in `docs/adr/README.md` before proposing governance, SPOT, or contract changes. +- `.github/workflows/alignment.yml` — CI blacklist grep + per-provider citation soft check; fails the build on known-hallucinated tokens. +- `CLAUDE.md` — Claude-Code-specific session instructions + `release_kit` overlay (Iron Rule 5.5). + +--- + +## Project-specific constraints + +- **`ALIGNMENT.md` is binding.** Any PR touching a provider plugin, the entry surface, or the IR must cite the relevant authority (provider CLI documentation / OpenAI spec URL / ADR number) in the commit body and PR description. See `CLAUDE.md` § "Hard requirements for plugin / server.mjs changes" and `ALIGNMENT.md` Rules 1, 2, 5. +- **Alignment CI is not suppressible.** The `alignment.yml` workflow greps for known-hallucinated tokens (currently carrying OCP's `api.anthropic.com/api/oauth/usage` as a transitive guardrail) and runs per-provider sanity checks. Adding new blacklist tokens is done via PR amendment to `alignment.yml`; removing entries requires an `ALIGNMENT.md` amendment PR. +- **No self-approval.** Implementation author cannot merge their own PR (Iron Rule 10). A fresh-context reviewer must open the cited authority and confirm in the review comment. +- **`models-registry.json` is the only place to add/edit `(provider, model)` metadata.** Do not touch hardcoded model maps in `server.mjs`, `lib/providers/*.mjs`, or `setup.mjs`. The OLP ADR that codifies this SPOT discipline lands in Phase 1 (OCP's ADR 0003 — `models.json` SPOT — is the precedent; OLP needs its own SPOT ADR because the registry shape differs). +- **Provider plugins follow the contract in ADR 0002.** A new provider plugin must implement every method on the Provider contract (`name`, `displayName`, `models`, `auth`, `spawn`, `estimateCost`, `quotaStatus`, `healthCheck`, `hints`). Partial implementations are unalignable per `ALIGNMENT.md` Rule 4. +- **No anti-fingerprinting.** OLP is honest about spawning the real CLI. If a provider detects subscription-spawn proxying and bans it, the response is to drop the provider per ADR 0006, not to mask the spawn. +- **No conversation state.** OLP is a stateless proxy. Memory / continuity is the client's responsibility (see ADR 0001 § Non-mission). + +--- + +## Release protocol + +OLP follows the machine-readable `release_kit:` overlay in `CLAUDE.md` (Iron Rule 5.5). Before any version bump or tag push, re-read that YAML block and walk every item in `new_feature_doc_expectations` and `bootstrap_quirk_policy`. Tag push triggers `.github/workflows/release.yml`, which creates the GitHub Release automatically — do not create the release manually. + +Version is sourced from `package.json`; changelog from `CHANGELOG.md`; user-facing docs from `README.md`. The Supported Providers table in `README.md` is sourced from `models-registry.json` per the overlay; do not hand-edit it out of sync. + +--- + +## Handoff expectations + +A fresh session picking up OLP work should read, in order: + +1. This file (`AGENTS.md`). +2. `ALIGNMENT.md` — constitution; non-optional. +3. `CLAUDE.md` — tool-specific instructions and `release_kit` overlay. +4. `docs/adr/` — most recent ADRs first; they explain why the current structure exists. Founding ADRs 0001–0006 are the OLP-bootstrap set. +5. `~/.cc-rules/memory/projects/olp_v0_1_spec.md` — the v0.1 spec (authoritative for OLP scope until v1.0 ships). +6. `~/.cc-rules/memory/auto/MEMORY.md` — cross-machine memory index. + +Only after these should the session touch code. + +--- + +**Authors:** project maintainer (with AI drafting assistance). diff --git a/ALIGNMENT.md b/ALIGNMENT.md new file mode 100644 index 0000000..a58e0b4 --- /dev/null +++ b/ALIGNMENT.md @@ -0,0 +1,168 @@ +# OLP Alignment Constitution + +**Status:** Active. This document is the supreme source of truth for OLP scope decisions. Conflicts with other documents (README, issues, prior commit messages, vendor documentation summaries) resolve in favor of this file. + +**Project:** OLP (Open LLM Proxy) — a personal- and family-scale multi-provider LLM proxy that supersedes OCP (Open Claude Proxy). See `docs/adr/0001-project-founding.md` for the founding decision and the 2026-06-15 Anthropic billing-split trigger that motivated multi-provider scope. + +--- + +## Core Principle + +OLP is a **router-and-cache layer** over a fixed set of independent provider CLIs. It spawns those CLIs, forwards traffic between them and an OpenAI-compatible HTTP entry surface, normalizes shapes via an Intermediate Representation (IR), and caches deterministically. It is **not** an extension layer for any provider, and it is **not** a model-capability router. + +This Core Principle decomposes into three honesty commitments: + +1. **Per-provider plugin honesty.** Each provider plugin must honestly proxy the underlying provider CLI without invention. If the CLI does not perform a given operation, the plugin does not invent one. +2. **OpenAI-compat entry honesty.** The `/v1/chat/completions` entry surface must honestly implement OpenAI's published specification without invention. If OpenAI's spec does not define a field, OLP does not invent one. +3. **IR honesty.** The IR is the internal interop contract between the entry surface and provider plugins. Any extension to the IR requires an Architecture Decision Record (ADR) amendment before it ships. + +OCP's constitution was anchored on a single wire authority (`cli.js`). OLP has **multiple wire authorities, one per provider plugin**, plus the OpenAI spec for the entry surface, plus the IR for internal interop. All three classes are governed uniformly by the Rules below. + +--- + +## Rules + +The five Rules below apply to every PR touching a provider plugin, the entry surface, or the IR. They are written in the language of the multi-authority model: each Rule names the authority the change must be reconciled against. + +1. **Rule 1 (Cite First).** Before adding, renaming, or changing any provider invocation, request field, response field, IR field, or entry-surface behaviour, the author must identify and cite the relevant authority — the provider CLI's documentation or observed behaviour (provider plugin scope), the OpenAI `/v1/chat/completions` specification section (entry-surface scope), or the relevant ADR (IR scope). The citation goes in the commit message and PR body. An absent citation is itself a finding and must be declared. + +2. **Rule 2 (No Invention).** OLP must not introduce: (a) provider CLI flags, invocation patterns, or output-shape assumptions the underlying provider CLI does not itself support; (b) OpenAI-spec fields, parameters, or response shapes that OpenAI's `/v1/chat/completions` specification does not document; (c) IR fields or semantics without an ADR amendment landing in the same merge or before. Speculative "Provider X probably uses Y" or "OpenAI clients probably expect Z" statements are prohibited. + +3. **Rule 3 (Match the Implementation).** When proxying a provider CLI, the plugin must match its semantics on the wire it actually emits: output format, error codes, streaming chunk shape, exit-code conventions, auth artifact location. When implementing OpenAI-compat at the entry surface, OLP must match the spec wire-format. When carrying data through the IR, the IR's documented semantics are normative; provider plugins document any lossy translations. Deviations require an explicit, reviewed exception recorded in this file (Class-specific Exceptions section). + +4. **Rule 4 (Unalignable Plugins / Fields Are Deleted).** A provider plugin that cannot be traced to a real CLI behaviour is removed, not disabled or feature-flagged. An IR field that cannot be traced to its authorizing ADR is removed. An entry-surface field that cannot be traced to OpenAI's spec is removed. The policy is removal, not deprecation. See the Unalignable Policy section. + +5. **Rule 5 (Cite in Commits).** Every commit touching `lib/providers/.mjs` must cite the provider CLI authority in the commit body. Every commit touching the entry surface must cite the OpenAI spec section URL. Every commit touching the IR must cite the authorizing ADR. CI performs a soft check for the citation pattern; reviewers enforce per `CLAUDE.md`. + +--- + +## Authorities + +OLP has three concurrent authorities. Each governs a distinct scope. + +### Authority 1 — Per-provider CLI (provider-plugin scope) + +Each provider plugin in `lib/providers/.mjs` is governed by the underlying provider's CLI as the wire authority for that plugin. The plugin's job is to spawn the CLI and translate between its native IO and the IR. + +**Provider authority pins** (audited at project founding 2026-05-23, re-audited annually per § Annual Audit): + +| Provider key | Provider CLI | Audit pin (TBD on Phase-1 spawn) | Risk Tier (see § Risk Tier Framework) | +|---|---|---|---| +| `anthropic` | `claude -p` from `@anthropic-ai/claude-code` | inherits OCP's `cli.js` 2.1.89 audit pin at fork; OLP-side pin set when Phase 1 lands | D (pre-2026-06-15) / re-evaluate post-2026-06-15 | +| `openai` | `codex exec --json` from OpenAI Codex CLI | TBD at Phase 2 spawn | D | +| `mistral` | `vibe --prompt --output json` from Mistral Vibe CLI | TBD at Phase 3 spawn | D | +| `grok` | `grok -p --output-format streaming-json` (xAI Build) | TBD at Phase 8+ enable | C | +| `kimi` | `kimi -p --output-format stream-json` (Moonshot) | TBD at Phase 8+ enable | C | +| `minimax` | TBD CLI command (MiniMax Token Plan) | TBD at opt-in enable | B | +| `glm` | TBD CLI command (Zhipu Coding Plan) | TBD at opt-in enable | B | +| `qwen` | TBD CLI command (Alibaba Coding Plan) | TBD at opt-in enable | B | + +Citation format for provider-plugin PRs: ` CLI § ` (e.g., `codex CLI v0.118.0 § exec --json output schema`) or a direct URL into the provider's published CLI documentation. If the change is based on observed behaviour rather than published documentation, the citation states that explicitly (`codex CLI v0.118.0 — observed behaviour, transcript attached`). + +### Authority 2 — OpenAI specification (entry-surface scope) + +OLP exposes a single external HTTP surface: `/v1/chat/completions` in OpenAI Chat Completions shape, plus a minimum administrative surface (`/health`, `/v1/models`, etc.). The OpenAI specification at https://platform.openai.com/docs/api-reference/chat is the wire authority for the entry surface. + +Citation format for entry-surface PRs: OpenAI spec URL + the specific field, parameter, or behaviour. Example: `OpenAI Chat Completions, response_format parameter (https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format)`. + +The spec pin lives in `docs/openai-spec-pin.md` (created at Phase 1; not required for v0.1 governance bootstrap to land). + +### Authority 3 — IR contract (internal interop scope) + +The IR is the lingua franca between the entry surface and provider plugins. It is documented in `docs/adr/0003-intermediate-representation.md`. Any IR change — adding a field, removing one, or altering semantics — is an amendment to ADR 0003 and follows the ADR amendment procedure. + +Citation format for IR PRs: `ADR 0003 §
` plus the amending ADR if applicable. + +--- + +## Risk Tier Framework + +Provider inclusion / exclusion is governed by a four-tier risk model. The full rationale lives in **ADR 0006 (Provider Inclusion / Exclusion Criteria)**. + +| Tier | Meaning | OLP treatment | +|---|---|---| +| **A** — vendor AI-platform-wide ban + explicit named prohibition + poor cost/benefit | **Permanently excluded.** Not bundled, not pluggable, not added via opt-in. | +| **B** — service-level key revocation; vendor may extend across AI services | **Optional tier 2.** Default-disabled. Requires one-time explicit consent prompt on enable. README documents the policy clause and realistic revocation outcome. | +| **C** — tightening signal; no documented enforcement history | **Optional tier 1.** Default-disabled. Opt-in via config without consent prompt. | +| **D** — permissive / safe | **Default-enabled.** Bundled in default config. | + +A vendor's documented ToS, FAQ language naming third-party proxy tools, observable enforcement history, and subscription cost/benefit jointly determine tier. Re-classification requires an ADR 0006 amendment. + +--- + +## Provider Inventory (v1.0) + +| Provider key | Tier | Default state | Inclusion source | +|---|---|---|---| +| `anthropic` | D (re-eval post-2026-06-15) | Enabled | ADR 0001 § Mission inheritance; ADR 0006 | +| `openai` | D | Enabled | ADR 0006 | +| `mistral` | D | Enabled | ADR 0006 | +| `grok` | C | Disabled (opt-in) | ADR 0006 | +| `kimi` | C | Disabled (opt-in) | ADR 0006 | +| `minimax` | B | Disabled (consent required) | ADR 0006 | +| `glm` | B | Disabled (consent required) | ADR 0006 | +| `qwen` | B | Disabled (consent required) | ADR 0006 | + +**Excluded permanently (Tier A):** Google Antigravity. See ADR 0006 for the named-prohibition + no-cost-advantage + reinstatement-friction rationale. Re-inclusion requires an ADR 0006 amendment with new evidence of policy change. + +--- + +## Unalignable Policy + +A change is **unalignable** if, after a good-faith search, it cannot be mapped to: + +- a specific provider CLI flag, documented behaviour, or observed transcript (provider-plugin scope), **or** +- a specific OpenAI specification section (entry-surface scope), **or** +- an existing or co-merged ADR (IR scope). + +Unalignable changes are **deleted**, not disabled, not feature-flagged, not deprecated. + +Burden of proof is on the change author. Audit findings that cannot be reconciled trigger an immediate deletion PR. There is no grandfathering for OLP — the project is new and inherits no legacy contracts. + +If a user workflow appears to depend on an unalignable behaviour, the correct remediations are: (a) upstream the behaviour into the relevant provider's CLI (engage that vendor), (b) propose an OpenAI-spec extension and wait for adoption, (c) write an ADR authorizing a new IR field, or (d) move the behaviour out of OLP into a separate tool. OLP does not retain it. + +--- + +## Annual Alignment Audit + +- **Date:** 14 May each year (the anniversary of the Anthropic 2026-05-14 announcement, which is the structural trigger for OLP). +- **Scope (per-provider plugin):** For each enabled provider, re-audit the provider CLI version against the pin in the Authorities table above. Re-verify that every spawn invocation, flag, and output-parser expectation in `lib/providers/.mjs` still matches that CLI version's actual behaviour. Update the pin row in this file. +- **Scope (entry surface):** Snapshot OpenAI's `/v1/chat/completions` specification and diff against the pin in `docs/openai-spec-pin.md`. For each field OLP implements, verify the spec still defines it the same way. Update the pin. +- **Scope (IR):** Diff the IR documented in ADR 0003 against the implementation in `lib/ir/`. Any drift triggers an amendment or a deletion PR. +- **Scope (Risk Tier reclassifications):** For each provider, re-evaluate the risk tier against current ToS, FAQ language, and observed enforcement events from the past year. Reclassifications land as ADR 0006 amendments. +- **Output:** A signed audit note committed to `docs/alignment-audits/YYYY-05-14.md`, updating pins inline in this file as needed. +- **Failure mode:** Any finding that cannot be reconciled triggers an immediate deletion PR per the Unalignable Policy. + +--- + +## Class-specific Exceptions + +(none at project founding) + +Any future Rule 3 deviation lands here as a numbered exception with PR link, reviewer, and rationale. + +--- + +## Amendment Procedure + +This constitution is amended only by a PR that (a) cites the evidence motivating the amendment, (b) is reviewed by an independent reviewer per CC Iron Rule 10, and (c) updates the relevant ADR if the amendment is structural. Amendments never retroactively legitimize previously unalignable changes. + +--- + +## Reference: How OCP's `cli.js` discipline maps to OLP + +OLP did not abandon OCP's discipline — it generalized it. The mapping: + +| OCP concept | OLP equivalent | +|---|---| +| `cli.js` as single golden reference | Per-provider CLI as authority for that plugin; OpenAI spec as authority for entry surface; IR contract as authority for internal interop | +| "Grep `cli.js` before touching `server.mjs`" | "Cite the relevant provider CLI doc or behaviour before touching that provider's plugin; cite OpenAI spec before touching the entry surface; cite the relevant ADR before touching the IR" | +| Class A / Class B endpoint discipline | Per-provider plugin scope + entry-surface scope + IR amendment process — each scope has its own citation requirement | +| `cli.js:NNNN` citation format | ` CLI §
` (provider) / OpenAI spec URL + field (entry) / `ADR NNNN §
` (IR) | +| Single ALIGNMENT.md governing one CLI | One ALIGNMENT.md governing many CLIs uniformly via the plugin contract | +| 2026-04-11 drift lesson | Inherited as institutional memory; no OLP-side incident yet. The blacklist in `.github/workflows/alignment.yml` carries the OCP-era token forward as a transitive guardrail. | + +--- + +**Authors:** project maintainer (with AI drafting assistance). +**First committed:** 2026-05-23 with the OLP repo bootstrap PR. diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..c029427 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,43 @@ +# Changelog + +All notable changes to OLP land here. Per `CLAUDE.md` release_kit overlay, this file is the source of truth for GitHub release notes. + +## v0.1.0-bootstrap — 2026-05-23 + +### Phase 0 — Repo bootstrap + +This is the founding commit of OLP (Open LLM Proxy), a personal- and family-scale multi-provider LLM proxy that supersedes OCP. The trigger was Anthropic's 2026-05-14 announcement (effective 2026-06-15) splitting `claude -p` / Agent SDK / third-party agent traffic out of the Pro/Max subscription pool into a separate fixed monthly Agent SDK Credit pool. + +**What lands in this commit:** + +- `ALIGNMENT.md` — OLP constitution. Three concurrent authorities (per-provider CLI / OpenAI spec / IR contract), 5 Rules, 4-tier Risk Tier Framework, 8-provider inventory. +- `AGENTS.md` — multi-tool agent guidelines (inherits `~/.cc-rules/AGENTS.md`). +- `CLAUDE.md` — Claude-Code-specific session instructions + machine-readable `release_kit` overlay (Iron Rule 5.5). +- `README.md` — phase-aware skeleton with provider inventory, API endpoint table, environment-variables table, response-headers spec, architecture overview, phase plan, migration-from-OCP outline. Placeholder content marked as such per phase. +- `docs/adr/` — 6 founding ADRs: + - `0001-project-founding.md` — Mission, non-mission, and supersession of OCP ADR 0005 (No Multi-Provider). + - `0002-plugin-architecture.md` — `lib/providers/.mjs` plug-in model with the Provider contract (name / models / auth / spawn / estimateCost / quotaStatus / healthCheck / hints). + - `0003-intermediate-representation.md` — OLP-internal canonical IR between OpenAI-compat entry and provider plugins. + - `0004-fallback-engine.md` — Trigger taxonomy (Hard / Soft / Deterministic-deferred / Cost-aware-deferred), idempotent-failure safety, first-chunk rule, chain advancement. + - `0005-cache-cross-provider.md` — Cache key composition over `(provider, model, messages, ...)`, D1+D2+D3+D4 port from OCP v3.13.0. + - `0006-provider-inclusion.md` — 4-tier Risk Framework, 8-provider classification, Antigravity exclusion (named prohibition + no cost advantage + reinstatement friction; *not* whole-account ban — Google AI services tier only per piunikaweb 2026-03-02 OpenClaw exec confirmation). +- `.github/PULL_REQUEST_TEMPLATE.md` — 8-radio Change Type taxonomy + per-type Authority Evidence sections + Iron Rule 10 reviewer checklist. +- `.github/workflows/alignment.yml` — CI blacklist (transitive `api.anthropic.com/api/oauth/usage` from OCP 2026-04-11 drift; Antigravity provider exclusion enforcement) + `models-registry.json` validator + commit-citation soft check. +- `.github/workflows/release.yml` — Auto-release on tag push with `package.json`-vs-tag version match check (Iron Rule 5). +- `.github/workflows/test.yml` — Node 20/24 matrix; tolerates bootstrap-phase absence of `test-features.mjs`. +- `package.json`, `.gitignore`, `LICENSE` (MIT), `CHANGELOG.md` — standard project boilerplate. + +**Provider inventory at bootstrap:** + +| Tier | Providers | +|---|---| +| D (default-enabled) | Anthropic, OpenAI Codex, Mistral Vibe | +| C (opt-in, no consent) | xAI Grok, Moonshot Kimi | +| B (opt-in, explicit consent) | MiniMax, Zhipu GLM, Alibaba Qwen | +| A (permanently excluded) | Google Antigravity | + +**Governance gate at bootstrap:** + +- Fresh-context independent reviewer (opus, Iron Rule 10) audited all 15 governance files against the OLP v0.1 spec and OCP precedent. Verdict: APPROVE_WITH_MINOR. Two minor findings folded in before this commit (alignment.yml heredoc indentation fix; AGENTS.md ADR-0003 reference clarification). + +**Next:** Phase 1 lands `server.mjs` skeleton + IR + Anthropic provider plugin + cache D1+D4 port from OCP. Per the spec §6 phase plan. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..52b0eb3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,109 @@ +@AGENTS.md +@~/.cc-rules/AGENTS.md + +# OLP Project Session Instructions + +> **WARNING — READ BEFORE WRITING ANY CODE IN THIS REPO** +> +> Before touching any provider plugin (`lib/providers/*.mjs`), the entry surface (`server.mjs` request handlers), or the IR (`lib/ir/*`), read [`./ALIGNMENT.md`](./ALIGNMENT.md) in full. The constitution is binding. Non-compliant commits are reverted. + +--- + +## Before starting any task + +1. Read `./ALIGNMENT.md`. Internalize the five Rules and the three-authority model (per-provider CLI / OpenAI spec / IR contract). +2. Run `/dev-start ` to get a pre-flight plan that incorporates the iron rules, `SKILL_ROUTING.md`, this file, and `ALIGNMENT.md`. +3. Locate the provider authority **before** drafting any code: + - Provider-plugin change → identify the provider CLI documentation page or observed behaviour you are matching, and pin the CLI version. + - Entry-surface change → identify the OpenAI `/v1/chat/completions` spec section and URL. + - IR change → identify the ADR you are amending or co-merging. + No code is written ahead of the authority citation. + +--- + +## Hard requirements for plugin / server.mjs / IR changes + +Every PR that modifies a provider plugin, the entry surface in `server.mjs`, or the IR in `lib/ir/` must satisfy all three of the following. A PR missing any one of them is blocked from merge. + +1. **Authority citation.** The commit message and PR body declare the relevant authority and citation: + - Provider plugin → ` CLI § ` plus URL or transcript reference. + - Entry surface → OpenAI spec URL plus the specific field, parameter, or behaviour. + - IR → `ADR NNNN §
` (and the amending ADR if applicable). + If the underlying authority does not perform the operation, the PR must state this explicitly and justify scope under `ALIGNMENT.md` Rule 2 (in practice, this almost always means the PR should be closed). +2. **CI `alignment.yml` pass.** The workflow must pass. It greps for known-hallucinated tokens, validates `models-registry.json`, and soft-checks per-provider commit citations. New blacklist tokens are added via PR amendment to `alignment.yml`; removing entries requires an `ALIGNMENT.md` amendment PR. Do not suppress the workflow. +3. **Independent reviewer (Iron Rule 10).** The implementation author may not self-approve. A separate reviewer — human or a subagent spawned with a fresh context — must read the diff, open the cited authority (provider CLI doc / OpenAI spec / ADR), and explicitly confirm the citation. A review comment that does not confirm the authority was checked is not a valid approval. + +--- + +## Iron rules in force + +This repo operates under the CC Development Iron Rules (CC 开发铁律) v1.4. Three rules are load-bearing for OLP work: + +- **Iron Rule 10 (Code Review).** Every implementation phase has an independent reviewer. Self-review does not count. See hard requirement #3 above. +- **Iron Rule 11 (Incremental Diff Review).** Non-trivial work is split into the minimum reviewable unit — one PR per layer per severity. `ALIGNMENT.md`, this file, the PR template, and the CI workflows are shipped as a single constitutional PR (one layer: governance). Subsequent layers (plugin loader, individual provider plugins, IR serializers, cache layer, fallback engine, dashboard) each land as their own PR. +- **Iron Rule 12 (Pre-Brainstorm Prior-Art Search).** Before proposing any new IR field, fallback trigger, or provider plugin, search the relevant provider's docs, OpenAI's spec, the local `docs/adr/`, and the cross-machine `~/.cc-rules/memory/learnings/`. The provider-specific search is the decisive one: if the provider CLI does not perform the operation, Rule 2 of the constitution applies. + +The full iron rules are at `~/.claude/CC_DEV_IRON_RULES.md` (symlinked from the cc-rules repo on the maintainer's workstations). Load them into session context with `/cc-rules` when needed. + +--- + +## Skills relevant to this repo + +- `/dev-start` — pre-flight planning, always first for non-trivial tasks. +- `/cc-rules` — load the iron rules into context. +- `/agent-dispatch` — pick the correct model (opus for design and review, sonnet for straightforward edits, haiku for mechanical chores) before spawning any subagent. +- `/cc-mem search ` — look up cross-machine memory for prior decisions, especially provider-policy events and CLI-version migrations. + +--- + +## Commit message conventions + +- Subject line uses Conventional Commits (`fix:`, `feat:`, `docs:`, `refactor:`, `chore:`). +- Provider-plugin commits include the citation pattern ` CLI ` or a direct provider docs URL in the body. CI performs a soft check. +- Entry-surface commits include an OpenAI spec URL. +- IR commits cite the authorizing or amending ADR. +- Any assertion of the form "Provider X uses Y" in the body must be immediately followed by a citation (CLI version + section, or docs URL, or observed-transcript reference). CI soft-checks the pattern. +- Co-author trailer is required for LLM-assisted commits (`Co-Authored-By: Claude `). + +--- + +## Project-level escalation + +If a design decision cannot be resolved by reference to the relevant authority (provider CLI / OpenAI spec / ADR) and `ALIGNMENT.md`, escalate to the project maintainer via `/cc-chat` rather than guessing. Silent guessing is what produced OCP's 2026-04-11 drift; OLP inherits that institutional lesson and does not repeat it. + +--- + +## Release kit overlay (CC 开发铁律 第五律 5.5) + +This project's overlay per iron rule v1.4's 5.5. Machine-checkable declaration. + +```yaml +release_kit: + version_source: package.json + changelog: CHANGELOG.md + release_channel: + type: github-release + tag_format: v{semver} + auto_create_on_tag_push: true # via .github/workflows/release.yml + docs_source: README.md + resource_lists: + - name: Supported Providers table + location: README.md § "Supported Providers" + source_of_truth: models-registry.json + - name: Routing chains table + location: README.md § "Configuration" + - name: API Endpoints table + location: README.md § "API Endpoints" + - name: Environment Variables table + location: README.md § "Environment Variables" + new_feature_doc_expectations: + - new provider plugin → README § "Supported Providers" entry + ADR 0006 inclusion entry + risk-tier classification + - new fallback trigger → README § "Configuration" + tests in test-features.mjs + - new IR field → ADR 0003 amendment + README impact note (if user-visible) + - new env var → README § "Environment Variables" table + - new endpoint → README § "API Endpoints" table + relevant Config / Troubleshooting § + - new auto-sync / hook → dedicated §, must document trigger + manual invocation + opt-out + any bootstrap quirk + - new file / SPOT / schema → Architecture or contributor § with link + bootstrap_quirk_policy: + - any first-run migration quirk (e.g., from OCP) → README § "Troubleshooting" + scripts/migrate-from-ocp.mjs if applicable +``` diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..6435b78 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 OLP project maintainer + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..43c6e5e --- /dev/null +++ b/README.md @@ -0,0 +1,190 @@ +# OLP — Open LLM Proxy + +A personal- and family-scale multi-provider LLM proxy. One HTTP endpoint, many subscriptions behind it, automatic routing, automatic fallback, content-addressed caching — so your IDEs and family clients keep working as long as *any* of your subscriptions has quota left. + +> **Status:** v0.1 — bootstrap. Most of this README is a skeleton; sections marked _placeholder_ land alongside the relevant phase of work (see [phase plan](#phase-plan)). + +--- + +## Why OLP + +On 2026-05-14, Anthropic announced (effective 2026-06-15) that `claude -p`, the Agent SDK, and third-party agent traffic move out of the Pro/Max subscription pool into a separate fixed monthly Agent SDK Credit pool. [OCP](https://github.com/dtzp555-max/ocp), OLP's predecessor, was a proxy around a single CLI — its core assumption was *"subscription = unlimited within rate limits"*. That assumption breaks for Anthropic on the effective date. + +The structural response is to stop relying on one provider's subscription terms remaining favourable. OLP spreads risk across multiple providers whose subscriptions still include CLI/programmatic use, routes intelligently between them, and caches aggressively so every request that does spawn a CLI counts. + +OLP is **not**: a commercial multi-tenant SaaS; an enterprise gateway competing with LiteLLM / OpenCode / CLIProxyAPI on breadth; a model-capability router ("route to the smartest model" — you pick the model); a conversation-state store (your client handles that). + +See [`ALIGNMENT.md`](./ALIGNMENT.md) for OLP's constitution and [`docs/adr/`](./docs/adr/) for the founding ADRs. + +--- + +## Quick Start + +_placeholder — lands with Phase 1._ + +Anticipated shape: + +```bash +# install +npm install -g @dtzp555-max/olp + +# run setup (writes ~/.olp/config.json, asks which providers to enable) +olp setup + +# start the proxy (default port 3456 — same as OCP if you migrate) +olp start + +# point your IDE at http://localhost:3456/v1/chat/completions with the OLP API key from `olp keys list`. +``` + +--- + +## Supported Providers + +Source of truth: [`models-registry.json`](./models-registry.json). This table is regenerated from the registry per the [`release_kit`](./CLAUDE.md) overlay; do not edit it out of sync. + +| Provider key | CLI | Subscription / auth | Risk Tier (per [ADR 0006](./docs/adr/0006-provider-inclusion.md)) | Default state | +|---|---|---|---|---| +| `anthropic` | `claude -p` | Pro / Max OAuth (pre-2026-06-15); Agent SDK Credit pool after | D (re-eval post-2026-06-15) | Enabled | +| `openai` | `codex exec --json` | ChatGPT Pro OAuth or API key | D | Enabled | +| `mistral` | `vibe --prompt --output json` | Le Chat Pro API key | D | Enabled | +| `grok` | `grok -p --output-format streaming-json` | xAI Build `xai-...` API key | C | Disabled (opt-in) | +| `kimi` | `kimi -p --output-format stream-json` | Moonshot Kimi API key | C | Disabled (opt-in) | +| `minimax` | TBD | MiniMax Token Plan (¥29+/mo) | B | Disabled (consent required) | +| `glm` | TBD | Zhipu Coding Plan ($10+/mo) | B | Disabled (consent required) | +| `qwen` | TBD | Alibaba Coding Plan ($50/mo) | B | Disabled (consent required) | + +**Risk tier guide.** D = permissive / safe; C = tightening signal, no enforcement history; B = service-level key revocation risk, vendor may extend across that vendor's AI services; A = permanently excluded. Tier B providers prompt for explicit consent on first enable and record consent in `~/.olp/config.json`. See [`ALIGNMENT.md` § Risk Tier Framework](./ALIGNMENT.md#risk-tier-framework). + +**Excluded permanently (Tier A).** Google Antigravity. See [ADR 0006](./docs/adr/0006-provider-inclusion.md) for the named-prohibition + no-cost-advantage + reinstatement-friction rationale. + +--- + +## Configuration + +_placeholder — full configuration reference lands with Phase 4 (fallback engine)._ + +OLP reads its config from `~/.olp/config.json`. The minimum useful shape: + +```json +{ + "routing": { + "chains": { + "": [ + { "provider": "", "model": "" }, + { "provider": "", "model": "" } + ] + }, + "soft_triggers": { + "": { "": } + } + } +} +``` + +Trigger types, fallback safety, idempotency rules, and the full example config land here when Phase 4 ships. See [ADR 0004 (Fallback Engine Semantics & Safety)](./docs/adr/0004-fallback-engine.md) for the design. + +--- + +## API Endpoints + +_placeholder — full table lands as each endpoint lands._ + +| Endpoint | Method | Phase | Description | +|---|---|---|---| +| `/v1/chat/completions` | POST | 1 | OpenAI-compatible Chat Completions entry. Internally normalized to IR, dispatched to a provider plugin, response shape converted back. | +| `/v1/models` | GET | 1 | Lists models from `models-registry.json`. | +| `/health` | GET | 1 | Per-provider health snapshot (owner-only). | +| `/cache/stats` | GET | 5 | Cache hit rate, by-provider breakdown. | +| `/v0/management/quota` | GET | 6 | Per-provider quota / credit pool status (best-effort). | +| `/dashboard` | GET | 6 | Owner-only dashboard (localhost-bound by default). | + +--- + +## Environment Variables + +_placeholder — full table lands per-phase as variables are introduced._ + +| Variable | Default | Description | +|---|---|---| +| `OLP_PORT` | `3456` | HTTP listener port. | +| `OLP_HOME` | `~/.olp` | Config, providers, keys, cache, logs root. | +| `OLP_LOG_LEVEL` | `info` | One of `error`, `warn`, `info`, `debug`. | + +Further variables (per-provider auth path overrides, cache size limits, fallback-engine knobs) land with the relevant phase. + +--- + +## Response Headers + +Every response served through OLP carries: + +- `X-OLP-Provider-Used: ` — which provider's plugin served the request. +- `X-OLP-Model-Used: ` — which model the served provider used. +- `X-OLP-Fallback-Hops: ` — number of fallback hops (`0` if served by the primary chain entry). +- `X-OLP-Cache: hit | miss | bypass` — cache layer outcome. +- `X-OLP-Latency-Ms: ` — end-to-end latency observed at the proxy. + +If a fallback chain is exhausted, `X-OLP-Fallback-Exhausted` lists the tried providers in order. + +--- + +## Architecture + +OLP is a Node.js (ESM, `.mjs`) HTTP proxy with no build step and minimal dependencies. The high-level shape: + +- **Entry surface** — `server.mjs` handles `/v1/chat/completions` and the administrative endpoints. Governed by OpenAI's `/v1/chat/completions` specification as the wire authority. See [`ALIGNMENT.md` § Authorities](./ALIGNMENT.md#authorities). +- **Intermediate Representation (IR)** — `lib/ir/` normalizes between the entry surface and provider-native shapes. The IR is the lingua franca; any extension is an [ADR 0003](./docs/adr/0003-intermediate-representation.md) amendment. +- **Provider plugins** — `lib/providers/.mjs`. Each plugin implements the contract in [ADR 0002 (Plugin Architecture for Providers)](./docs/adr/0002-plugin-architecture.md), spawns its CLI, and translates between IR and provider-native IO. +- **Cache layer** — `lib/cache/` is a content-addressed cache keyed on `(provider, model, messages, tools, temperature, response_format, cache_control)`. Per-key isolation, prompt-caching bypass, chunked stream replay, and singleflight. See [ADR 0005 (Cache Layer Cross-Provider Design)](./docs/adr/0005-cache-cross-provider.md). +- **Fallback engine** — `lib/fallback/` advances a configured chain one provider at a time on configured triggers, never retrying after the first response chunk has been emitted to the client. See [ADR 0004](./docs/adr/0004-fallback-engine.md). +- **Multi-key auth** — `lib/keys.mjs` carries OCP's per-OLP-key namespace isolation forward. Each OLP API key has independent quota, cache namespace, and audit log; each key declares which providers it can access. + +Read the ADRs in `docs/adr/` in order before proposing structural changes. + +--- + +## Phase plan + +OLP lands in phases. Each phase has its own PR series and Iron-Rule-10 reviewer; this README's placeholders are filled per-phase via the [`release_kit`](./CLAUDE.md) overlay. + +- Phase 0 — Repo bootstrap, `ALIGNMENT.md`, founding ADRs, CI workflows, PR template. **(current)** +- Phase 1 — `server.mjs` skeleton, IR, Anthropic plugin, cache D1+D4. Port from OCP. +- Phase 2 — OpenAI Codex plugin. +- Phase 3 — Mistral Vibe plugin. +- Phase 4 — Fallback engine + routing chains config + quota poll worker. +- Phase 5 — Cache cross-provider hardening (D2+D3). +- Phase 6 — Dashboard + observability (`/v0/management/quota`). +- Phase 7 — Release v0.1, OCP enters maintenance. +- Phase 8+ — Optional Grok / Kimi / tier-2 plugins; provider-native protocol endpoints; deterministic triggers. + +Full spec (decision rationale, open questions, risks): `~/.cc-rules/memory/projects/olp_v0_1_spec.md` on the maintainer's workstations. + +--- + +## Migration from OCP + +_placeholder — `scripts/migrate-from-ocp.mjs` lands with Phase 7._ + +Anticipated user-facing flow (target: <5 minutes): + +1. Stop OCP (`launchctl bootout` the OCP service or `ocp stop`). +2. Install OLP. +3. Run `olp migrate-from-ocp` — copies `~/.ocp/keys/` to `~/.olp/keys/` and points provider plugins at OCP's existing auth artifacts where applicable. +4. Start OLP. Clients pointing at port 3456 keep working; their existing OLP API keys remain valid. + +OCP's cache directory is *not* migrated: OLP's cache key format includes provider+model and warms cold naturally. OCP enters maintenance mode (stability fixes only) when OLP v0.1 ships; new development happens in OLP. + +--- + +## License + +MIT. + +--- + +## Acknowledgements + +OLP evolved from [OCP (Open Claude Proxy)](https://github.com/dtzp555-max/ocp). OCP's per-key isolation model, cache-layer design (D1–D4), dashboard, and alignment-constitution discipline are all carried forward. The structural generalization from single-CLI to multi-provider is what makes this a new project rather than an OCP minor version — see [`ALIGNMENT.md` § Reference: How OCP's `cli.js` discipline maps to OLP](./ALIGNMENT.md#reference-how-ocps-clijs-discipline-maps-to-olp). + +Authors: project maintainer (with AI drafting assistance). diff --git a/docs/adr/0001-project-founding.md b/docs/adr/0001-project-founding.md new file mode 100644 index 0000000..2701b38 --- /dev/null +++ b/docs/adr/0001-project-founding.md @@ -0,0 +1,69 @@ +# ADR 0001 — Project Founding: Mission, Non-mission, and Supersession of OCP ADR 0005 + +- **Date:** 2026-05-23 +- **Status:** Accepted (bootstrap) +- **Authors:** project maintainer (with AI drafting assistance) +- **Related:** OLP v0.1 spec §1, §2, §10; OCP ADR 0005 (No Multi-Provider — explicitly superseded by this ADR); Anthropic billing announcement 2026-05-14 + +## Context + +On 2026-05-14, Anthropic announced (effective 2026-06-15) that `claude -p`, the Agent SDK, and third-party agent traffic move out of the Pro/Max subscription pool into a separate fixed monthly Agent SDK Credit pool ($100/month at Max5x, no rollover, then standard per-token API rates). OCP — the Open Claude Proxy — was built on the foundational assumption that *subscription = unlimited within rate limits*. That assumption breaks on 2026-06-15 for the only provider OCP serves. + +This is not a `cli.js` drift in the OCP-ADR-0002 sense. It is not an internal authoring failure. The proxy logic remains aligned to `cli.js` byte-for-byte. What changed is the *commercial contract* sitting underneath the proxy: the subscription the maintainer pays for no longer underwrites the traffic OCP forwards. OCP-as-written continues to work; it just stops being the cost-optimal way to spend the maintainer's LLM budget. + +OCP ADR 0005 (2026-05-06, "No Multi-Provider") explicitly rejected a multi-provider refactor on four grounds: (1) loss of `cli.js` alignment as moat, (2) hybrid-architecture awkwardness, (3) direct competition with funded incumbents, (4) the grayscale problem isn't solved by adding providers. Eight days after that ADR shipped, the assumption underpinning ground 1 — that there *was* an unlimited subscription pool worth aligning to — was invalidated by Anthropic's announcement. The other three grounds remain partially valid but no longer dominate: (2) hybrid awkwardness goes away if a clean multi-provider proxy is built from scratch rather than retrofitted onto OCP's single-provider core; (3) competition with funded incumbents is irrelevant because the new project, like OCP, is a personal-and-family-scale tool not chasing commercial revenue; (4) grayscale is intrinsic to the spawn-binary architecture regardless of how many providers participate. + +The structural response is not to amend OCP ADR 0005 in place — that ADR was correct under its premises, and patching it would muddy the historical record. The structural response is a new project, founded today, that supersedes ADR 0005 by being the codified alternative the ADR explicitly contemplated in its trigger conditions ("[revisit if] a genuine user need emerges … that single-provider OCP cannot serve"). + +## Decision + +Found OLP (Open LLM Proxy) as a new project, separate from OCP, with the following mission and non-mission boundaries: + +**Mission.** OLP is a personal- and family-scale multi-provider LLM proxy that maximizes the value of LLM subscriptions the maintainer already pays for, with intelligent fallback when one provider's quota is exhausted and content-addressed caching to minimize quota consumption. Core value proposition: one HTTP endpoint, multiple subscriptions behind it, automatic routing, automatic fallback, transparent caching — so IDE clients and family clients keep working as long as *any* of the subscriptions has quota left. + +**Non-mission (explicit).** Per spec §1, OLP is **not**: +- A commercial multi-tenant SaaS. Helicone, OpenRouter, LiteLLM, Portkey, Cloudflare AI Gateway already serve that market with funding, SOC2, dashboards, and team features. OLP enters none of those races. +- A generic enterprise AI gateway competing on provider breadth. The provider set is intentionally narrow (currently 3 default + 2 optional tier-1 + 3 optional tier-2) and curated by subscription economics, not by "more is better." +- A model-capability router. OLP does not auto-route to "the smartest model"; the user picks the model explicitly per chain in `config.routing.chains[]`. Capability routing is a separate problem with separate failure modes and is out of scope for v1.0 and beyond. +- A conversation-state store. Memory and continuity are client-side concerns (Memory Continuity, Hermes equivalents, IDE-side context). OLP is a pure stateless proxy. + +**Supersession of OCP ADR 0005.** OCP ADR 0005 ("No Multi-Provider Refactor") is explicitly superseded by OLP's existence. Specifically: +- ADR 0005's premise that *single-provider sufficiency* was the right scope for the maintainer's LLM proxy was correct on 2026-05-06 and is wrong on 2026-05-23, because Anthropic changed the underlying contract. +- ADR 0005's recommendation that "commercial work, if pursued, starts elsewhere" pointed correctly toward "a separate repository, separate name, BYOK from day one, no `cli.js` spawn." OLP is not the commercial work ADR 0005 contemplated, but it is the structural shape ADR 0005 endorsed: a separate repo with multi-provider design baked in from day one. +- OCP itself is not deleted. Per spec §7, OCP enters maintenance mode when OLP v0.1 ships. The two projects do not parallel-run in production (port-3456 conflict, single launchd service slot, one set of credentials per machine). + +OCP ADR 0005 receives a header amendment on merge of this ADR: *"Superseded by OLP — see [OLP repo URL] ADR 0001."* The body of ADR 0005 is otherwise untouched. Future readers should see the original reasoning intact and the supersession marker explicit. + +## Consequences + +**Positive** +- Multi-provider routing solves the post-2026-06-15 quota problem that OCP cannot solve on its own. As long as any one of the configured providers has quota remaining, the maintainer's IDE clients keep working. +- Clean-slate codebase avoids the hybrid-architecture awkwardness ADR 0005 correctly warned about. OLP is multi-provider from day one; there is no single-provider core to retrofit. +- Project scope stays bounded by the non-mission list. Future contributors (including the maintainer) reading this ADR see the explicit "we don't do X" list, which makes scope creep cost an ADR amendment rather than a quiet PR. + +**Negative** +- OCP's `cli.js`-alignment moat (ADR 0002 + 0005) does not transfer. OLP proxies multiple CLIs, each with its own protocol authority; the `cli.js` byte-for-byte discipline becomes per-provider per-CLI discipline (see ADR 0002 and the new OLP ALIGNMENT.md). This is more surface area to keep honest, not less. +- Migration cost for the user: stop OCP, install OLP, point clients at the new port, migrate auth tokens. Spec §7 budgets this at <5 minutes via `scripts/migrate-from-ocp.mjs`. Real-world time will be higher on the first machine. +- The maintainer now operates two repos during OLP's pre-v1.0 phase: OCP in maintenance, OLP under active development. This is intentional — OCP serves real traffic today and cannot be deleted in advance — but doubles the release surface temporarily. + +**Mitigations** +- OCP enters maintenance, not parallel development. The maintainer is not maintaining two evolving codebases; OCP receives only stability fixes after OLP v0.1 ships. +- The non-mission list in this ADR is intentionally specific (commercial SaaS, breadth race, capability routing, conversation state) so "creep into one of these four" requires an explicit ADR amendment. +- Spec §10 (Decision log) records the 2026-05-23 founding decisions; new founding-era decisions are appended to spec §10 with cross-references to the relevant ADR. + +## Alternatives considered + +**(a) Amend OCP ADR 0005 in place and refactor OCP to multi-provider.** Considered and rejected. ADR 0005 explicitly warned against hybrid architecture awkwardness for exactly this case; the maintainer recorded the warning in writing eight days ago. Honoring that warning means starting clean, not retrofitting. The structural cost of two repos during the transition is lower than the structural cost of muddying the constitutional reasoning that protected OCP from earlier scope creep. + +**(b) Wait until 2026-06-15 (effective date) before founding OLP.** Considered and rejected. The Anthropic announcement is dated, the effective date is dated, and the construction time for OLP v0.1 (~16 working days per spec §6 phase plan) puts a 2026-05-23 start at roughly the effective date — exactly when the maintainer's family clients begin seeing the new credit-pool consumption. Waiting until the wall clock turns means OLP ships *after* the problem starts hurting. + +**(c) Add multi-provider behind a feature flag in OCP and ship it as OCP v4.0.** Considered and rejected. Feature flags do not retire the single-provider code path; they double the test matrix and create the exact hybrid path ADR 0005 warned about. The honest engineering move is a separate codebase. + +**(d) Drop the proxy approach entirely and let IDE clients talk to each provider directly.** Considered and rejected. The whole point of OCP/OLP is *one* endpoint that family/IDE clients can be pointed at. N clients × M providers = N×M configuration surface managed by the maintainer, plus zero fallback, zero cross-provider caching, zero unified spend dashboard. The proxy is load-bearing. + +## Sources + +- OLP v0.1 spec §1 (Mission), §2 (Why OLP exists), §7 (Migration from OCP), §10 (Decision log) +- OCP ADR 0005 (2026-05-06) — `~/ocp/docs/adr/0005-no-multi-provider.md` +- Anthropic billing announcement 2026-05-14 — `~/.cc-rules/memory/learnings/anthropic_claude_code_billing_split_2026_06_15.md` +- Memory entry 2026-05-07 (Mac → OCP v3.13.0 cache layer hardening) — records the maintainer's preceding decision that OCP itself will not be commercialized diff --git a/docs/adr/0002-plugin-architecture.md b/docs/adr/0002-plugin-architecture.md new file mode 100644 index 0000000..2f1faa2 --- /dev/null +++ b/docs/adr/0002-plugin-architecture.md @@ -0,0 +1,101 @@ +# ADR 0002 — Plugin Architecture for Providers + +- **Date:** 2026-05-23 +- **Status:** Accepted (bootstrap) +- **Authors:** project maintainer (with AI drafting assistance) +- **Related:** OLP v0.1 spec §4.2; ADR 0001 (project founding); ADR 0003 (IR design); ADR 0006 (provider inclusion framework) + +## Context + +OLP supports a curated set of providers — three default-enabled (Anthropic, OpenAI Codex, Mistral Vibe) plus two optional tier-1 (xAI Grok, Moonshot Kimi) and three optional tier-2 (MiniMax, Zhipu GLM, Alibaba Qwen). Each provider has its own CLI binary, its own auth artifact location, its own request shape, its own response shape, its own quota-reporting endpoint (or none), and its own rate-limit posture. The maintainer's strong prior is that this set grows over the project's lifetime — provider economics will continue to shift, and "the right five providers" in 2027 will not be identical to today's five. + +The naive architecture is a monolithic dispatcher inside `server.mjs`: + +```js +if (provider === 'anthropic') { spawn claude -p ... } +else if (provider === 'openai') { spawn codex exec --json ... } +else if (provider === 'mistral') { spawn vibe --prompt ... } +// ... and so on for every provider, every auth shape, every quirk +``` + +This shape works for two providers, becomes painful at four, and is the structural shape that produced the worst pages of OCP's `server.mjs` (1667 lines, ADR 0005 context paragraph). Worse, it makes the answer to "how does a contributor add a sixth provider?" be "edit eight places inside `server.mjs` and hope you caught them all." That is the exact failure mode `models.json` SPOT (OCP ADR 0003) was designed to prevent for model metadata; the provider equivalent needs the same structural answer. + +The other end of the spectrum is full external plugin discovery — npm-installed plugins, runtime registration, hot-load. That is unambiguously out of scope for v1.0: the provider set is curated for security and ToS-risk reasons (see ADR 0006), and "anyone can install a third-party plugin" violates that curation by design. + +The middle path is a **plugin model with a fixed in-tree provider registry**: each provider is a `.mjs` file under `lib/providers/`, all conforming to a single `Provider` contract, loaded at startup from a static enumeration in `lib/providers/index.mjs`. Adding a provider means writing one file and adding one line to the registry. Disabling an optional provider means a config-file toggle, not a code change. + +## Decision + +Per spec §4.2, OLP uses a plugin-based provider architecture with the following structure: + +**Filesystem layout:** +``` +lib/providers/ + base.mjs # abstract Provider contract + shared helpers + index.mjs # static registry (enumeration of in-tree providers) + anthropic.mjs # spawn `claude -p` — port of OCP server.mjs spawn logic + codex.mjs # spawn `codex exec --json` + vibe.mjs # spawn `vibe --prompt --output json` + grok.mjs # spawn `grok -p --output-format streaming-json` (optional) + kimi.mjs # spawn `kimi -p --output-format stream-json` (optional) + minimax.mjs # tier-2 optional, default-disabled + glm.mjs # tier-2 optional, default-disabled + qwen.mjs # tier-2 optional, default-disabled +``` + +**Provider contract** (v1.0 interface — exact shape per spec §4.2): + +Every provider plugin exports an object conforming to: +- `name: string` — unique key (`anthropic`, `openai`, `mistral`, etc.) +- `displayName: string` — human-readable name for dashboards and consent UX +- `models: string[]` — models this provider serves +- `auth: { type, storage, path, refresh }` — auth-artifact profile +- `spawn: async (normalizedRequest, authContext) => AsyncIterator` — the core invocation +- `estimateCost: (request) => { inputTokens, outputTokensEstimate, currency, usd }` — best-effort, may return null +- `quotaStatus: async (authContext) => { available, percentUsed, resetsAt, pool }` — best-effort, null if unretrievable +- `healthCheck: async () => { ok, latencyMs, error? }` — startup and `/health` endpoint use this +- `hints: { requiresTTY, concurrentSpawnSafe, maxConcurrent }` — fingerprint and concurrency hints + +**Loading model.** `lib/providers/index.mjs` is a hand-maintained static enumeration. There is no filesystem scan, no `require.context`, no dynamic discovery. Adding a provider requires: +1. Write `lib/providers/.mjs` conforming to the contract. +2. Add one import + one entry to `lib/providers/index.mjs`. +3. Add a row to README's "Supported Providers" table. +4. File an inclusion ADR per ADR 0006's framework. + +**Disable model.** Optional providers (tier-1 and tier-2 per ADR 0006) are present in the registry but `enabled: false` by default. Enable is a `~/.olp/config.json` toggle, plus the tier-2 consent flow described in spec §3.1. Disabling a provider does not require touching `server.mjs`. + +**Boundary with `server.mjs`.** `server.mjs` knows about the registry and the contract; it does not know about specific providers. The fallback engine (ADR 0004), the cache layer (ADR 0005), and the dashboard (spec §4.6) all consume providers through the contract, not through provider-specific code paths. + +## Consequences + +**Positive** +- Adding a new provider is a four-step recipe with no `server.mjs` edits required. The recipe is explicit (file + registry + README + ADR), so a future contributor (including a future Claude session) cannot accidentally do steps 1–2 without 3–4. +- The contract is the test surface. A provider plugin can be tested in isolation against a contract conformance suite (`test-features.mjs` extended per spec §6 Phase 1), independent of `server.mjs`. +- `server.mjs` stays generic. There is no "is this Anthropic? then do special thing" path inside the core proxy loop. Provider-specific quirks live inside the provider plugin where they belong. +- Disabling a misbehaving provider (e.g., a ToS change announcement triggers fast-disable per spec §9 risks) is a config flip, not a code revert. The provider quarantine path spec §9 calls for is the existing config mechanism. + +**Negative** +- The contract surface is real governance work. Adding a field to the contract (e.g., a new `streamingMode` or `toolUseShape`) is an ADR amendment per OLP ALIGNMENT.md, not a quick PR. This is intentional — contract drift is the path back to the monolithic-dispatcher problem the contract was built to prevent. +- Provider plugins have non-trivial duplication: every provider re-implements the same SSE-chunk-translation skeleton, the same auth-env-injection, the same spawn-with-timeout. `base.mjs` exists to absorb the truly-shared parts, but resisting the temptation to push provider-specific logic into `base.mjs` requires discipline. +- Some providers (notably Anthropic) have *much* more behavior to encode than others (Mistral Vibe is comparatively spartan). The contract has to be expressive enough for the rich case without being burdensome for the spartan case. Lossy translations are documented per-provider per ADR 0003. + +**Mitigations** +- `base.mjs` provides shared helpers but does not implement the `Provider` contract itself. Provider plugins compose helpers; they do not inherit from a base class. This keeps the "what does this provider do?" question answerable by reading one file. +- The contract is versioned. v1.0 is the subset in this ADR; future additions (e.g., a `cancel()` method for in-flight request termination, or a `costPerToken` snapshot) require ADR amendment plus a contract-version bump. Old provider plugins continue to declare `contractVersion: '1.0'` and the loader handles the version gap. +- The provider inclusion ADR per ADR 0006 doubles as the contract-conformance review gate. A new provider's inclusion ADR must show how it satisfies each field of the contract; that review is the structural counter-measure against contract drift. + +## Alternatives considered + +**(a) Monolithic dispatch inside `server.mjs`.** A single function with `if/else if` per provider, each branch implementing spawn/quota/health inline. Rejected: this is the architectural shape that produced OCP's `server.mjs` length problem at *one* provider, and it does not survive contact with three default + two optional tier-1 + three optional tier-2 providers (eight code paths). Worse, it makes provider-disable a code change, which means fast-quarantine in response to a ToS announcement (spec §9) is a release event rather than a config flip. + +**(b) Full external plugin discovery (npm-installable, runtime-loaded, hot-discoverable).** Plugins are npm packages; OLP scans `node_modules/@olp-providers/*` at startup; users `npm install` to add a provider. Rejected for v1.0 on three grounds: (1) the provider set is curated for ToS-risk reasons (ADR 0006), and "anyone can install any provider" defeats that curation; (2) the discovery layer is itself non-trivial code (manifest validation, version compatibility, security review of third-party plugin code) that does not earn its complexity at three to eight providers; (3) the contract has not stabilized enough — locking it as a stable plugin API before v1.0 ships is premature commitment. + +**(c) Per-provider sub-processes / microservices.** Each provider runs as a separate Node.js process; `server.mjs` is a router that proxies to the right sub-process. Rejected as massive over-engineering for v1.0 traffic levels (family-scale, dozens of requests per hour, not hundreds per second). The spawn-per-request cost is already the dominant latency; sub-process IPC adds latency without buying anything until the maintainer is running OLP at a scale where one Node process is the bottleneck — which is not v1.0's problem. + +**(d) Code generation from a YAML manifest per provider.** Each provider is described in YAML; a generator emits the corresponding `.mjs`. Rejected as a layer that does not pay for itself. The provider plugins are ~150–400 lines of hand-written code each; generating them from YAML would shift complexity from the `.mjs` to the YAML + generator + the round-trip when a generated file needs to be hand-tuned for a quirk. The generator-first approach also makes the `cli.js`-alignment equivalent (per-provider CLI behavior alignment per OLP ALIGNMENT.md) harder to enforce, because the source of truth becomes the YAML, not the spawned-binary's real behavior. + +## Sources + +- OLP v0.1 spec §4.2 (Plugin-based provider system, including the v1.0 Provider contract definition) +- OCP ADR 0003 (`models.json` as SPOT) — informs the "static enumeration, not filesystem scan" loading model +- OCP ADR 0005 — the context paragraph references OCP's `server.mjs` reaching 1667 lines at one provider; the plugin architecture is the structural response to that complexity scaling N× diff --git a/docs/adr/0003-intermediate-representation.md b/docs/adr/0003-intermediate-representation.md new file mode 100644 index 0000000..13eabb9 --- /dev/null +++ b/docs/adr/0003-intermediate-representation.md @@ -0,0 +1,109 @@ +# ADR 0003 — Intermediate Representation (IR) Design + +- **Date:** 2026-05-23 +- **Status:** Accepted (bootstrap) +- **Authors:** project maintainer (with AI drafting assistance) +- **Related:** OLP v0.1 spec §4.1, §4.2 (IR subsection); ADR 0002 (plugin architecture); ADR 0004 (fallback engine — requires shape-stable requests for safe replay) + +## Context + +OLP exposes exactly one external API surface: OpenAI-compatible `/v1/chat/completions` (per spec §4.1). Internally, OLP fans out to N providers, each with its own native protocol — Anthropic Messages API, OpenAI Chat Completions, Mistral La Plateforme, xAI Grok, Moonshot Kimi, etc. The combinatorial fact is unavoidable: a request entering as OpenAI shape must exit as Anthropic shape (or Mistral shape, or whatever the routing chain advances to), and the response on the way back must travel the inverse path. + +The ad-hoc approach is per-provider entry-to-native translation: + +``` +provider.spawn(openAIRequest) { + const native = openAIToProviderNative(openAIRequest); // per-provider impl + ... + const openAIResponse = providerNativeToOpenAI(nativeResponse); // per-provider impl + return openAIResponse; +} +``` + +At three providers, that's six translation functions (three forward + three reverse). At eight providers it's sixteen. None of them share code — Anthropic-to-OpenAI and Mistral-to-OpenAI have different shape mismatches, different tool-calling translations, different streaming-event mappings. The "OpenAI-shape canonical" assumption privileges one provider's protocol as the lingua franca, which is wrong: OpenAI's Chat Completions is OLP's *entry* surface, not OLP's *internal* representation. Anchoring internal logic on the entry surface means every entry-surface change (a new OpenAI field, a deprecated OpenAI field, a new OpenAI streaming-event variant) ripples through every provider plugin. + +The structural answer is an **Intermediate Representation (IR)**: a canonical OLP-internal shape that *all* provider plugins consume (forward) and produce (reverse). The entry adapter translates OpenAI → IR once. Each provider plugin translates IR → native and native → IR. The combinatorial cost goes from N² to 2N: one entry adapter + N provider translators. + +The IR is not OpenAI Chat Completions, and it is not Anthropic Messages. It is a *deliberately-OLP-owned* shape, normalized to be expressible in every provider's native shape with documented lossy fields where a provider's expressiveness is narrower than the IR. Choosing IR shape carefully — neither so rich it cannot round-trip through Mistral, nor so narrow it loses Anthropic's tool_use semantics — is the load-bearing design decision of this ADR. + +The fallback engine (ADR 0004) is a second consumer of IR's shape stability. When a request fails over from Anthropic to OpenAI, the IR that was constructed at the entry point is replayed against the next provider. If the IR were a thin OpenAI wrapper, replay against OpenAI would be trivial but replay against Mistral or xAI would re-run the same shape-translation pain. A properly-OLP-owned IR makes replay symmetric across providers. + +## Decision + +OLP defines an Intermediate Representation (IR) as the canonical internal shape between the OpenAI-compat entry surface and each provider plugin. Per spec §4.2 (IR subsection), the IR v1.0 encodes: + +**Required fields:** +- `messages[]` — each with `role`, `content`, optional `name`, optional `tool_calls`, optional `tool_call_id`. Roles supported: `system`, `user`, `assistant`, `tool`. +- `model` — the user-requested model string. The provider plugin maps this to the provider-native model identifier (e.g., `claude-sonnet-4-6` → `claude-sonnet-4-6-20260301` for Anthropic). +- `stream` — boolean. SSE expected on the entry side iff true. + +**Optional fields:** +- `max_tokens` — integer; provider plugins clamp to provider-supported maxima. +- `temperature` — float [0, 2]; provider plugins map to provider-native range (e.g., Anthropic uses [0, 1]). +- `top_p` — float [0, 1]. +- `stop` — string or string[]; provider plugins handle the single-vs-array shape difference per native protocol. +- `tools[]` + `tool_choice` — function-calling subset. v1.0 supports basic function definitions (`name`, `description`, `parameters` JSON-schema); advanced features (parallel tool use, tool result handling beyond single round-trip) are per-provider documented as caveats per spec §8 Q-C. +- `response_format` — best-effort; provider-specific (e.g., Anthropic does not natively honor a `response_format: json_object` field, so this becomes a system-prompt augmentation on that provider). + +**Translation direction model.** Each provider plugin owns two translation functions: + +```js +// IR → provider-native shape +const nativeRequest = provider.irToNative(irRequest); + +// provider-native response chunk → IR response chunk +const irChunk = provider.nativeToIR(nativeChunk); +``` + +The entry adapter (`server.mjs`'s `/v1/chat/completions` handler) owns the inverse: + +```js +const irRequest = openAIToIR(openAIRequest); +const irResponseStream = await provider.spawn(irRequest, authContext); +for await (const irChunk of irResponseStream) { + res.write(irToOpenAI(irChunk)); +} +``` + +**Lossy-translation documentation requirement.** When a provider plugin's `irToNative` or `nativeToIR` cannot losslessly carry an IR field (e.g., Mistral Vibe does not support `tool_choice: "required"`), the lossy edge is documented in the provider plugin's header comment AND in `docs/provider-caveats.md`. The caveats document is referenced from `/v1/models` response metadata so clients can see provider-specific limits without reading source. + +**IR is versioned.** This ADR ratifies IR v1.0. Future additions (e.g., a `reasoning_effort` field if reasoning-model behavior diverges enough across providers to warrant explicit modeling) require an amendment ADR. Removals are breaking changes and require a major-version bump of OLP. + +**The IR is not exposed externally.** It is OLP-internal. There is no `/v1/ir/*` endpoint, no client SDK in IR shape, no documented external IR contract. The only public contract is OpenAI `/v1/chat/completions`; the IR is an implementation detail that the maintainer (and the maintainer's reviewers) reason about, not a contract with clients. + +## Consequences + +**Positive** +- Adding a provider is N translation functions (one IR→native, one native→IR), not 2N (one openai→native, one native→openai for every provider). The maintenance burden of "OpenAI shipped a new field" is one entry-adapter change, not one change per provider. +- The fallback engine (ADR 0004) replays the same IR object across providers in a chain. Replay correctness depends on IR being a fixed-shape data structure; there is no per-provider state leaking through that the next chain hop has to re-derive. +- Lossy translations are documented per-provider and surfaced to clients via `docs/provider-caveats.md` + `/v1/models` metadata. A client choosing a fallback chain can see "this chain's third hop is Mistral, which does not support tool_choice=required" *before* they hit the failure mode. +- The IR is OLP's source of truth for "what was actually requested" for cache-key generation (ADR 0005). Caching on OpenAI shape would couple cache keys to OpenAI field naming; caching on IR makes the cache key provider-agnostic. + +**Negative** +- Two translation layers per request: openai→IR at entry, IR→native at provider. That's two passes of object construction per request, plus the inverse on the way back. For family-scale traffic this is unmeasurable; at higher scale the overhead would need profiling. +- IR design decisions are sticky. Choosing the wrong shape for `tool_calls` in v1.0 (e.g., privileging OpenAI's parallel-tool-call shape over Anthropic's single-tool shape) would force every provider plugin to work around the bad choice forever. IR v1.0 is intentionally narrow (basic function calling only) to give the maintainer space to expand the IR with informed amendments rather than commit too early. +- The IR is a third concept (alongside OpenAI shape and per-provider native shape) that every contributor must internalize. Onboarding a new contributor means teaching three shapes, not two. The PR template and OLP ALIGNMENT.md call out the IR explicitly so contributors know which shape applies in which layer of code. + +**Mitigations** +- IR v1.0 is deliberately a subset, not a superset, of what providers can do. Where providers diverge in expressive power (tool calling, structured outputs, reasoning modes), the IR encodes the *common subset* in v1.0, with caveats documenting provider extensions. Expanding the IR to cover provider extensions is an explicit amendment, not a silent generalization. +- Each provider plugin includes a `__irRoundTripTest()` export that exercises the plugin's `irToNative` + `nativeToIR` pair against a fixture set in `test-features.mjs`. Round-trip discrepancies fail CI. This is the structural counter-measure against the silent-lossy-translation failure mode. +- The IR has no external surface. If IR v1.0 turns out to be the wrong shape and IR v2.0 is incompatible, the migration is OLP-internal: provider plugins are updated, the entry adapter is updated, no client code changes. The cost of being wrong is bounded by the project's own codebase. + +## Alternatives considered + +**(a) Full pass-through with no normalization — OpenAI request goes straight to each provider plugin.** Provider plugins each implement their own openai→native + native→openai. Rejected: this is the N² shape the IR exists to avoid. Worse, it makes the fallback engine (ADR 0004) harder, because each fallback hop has to redo openai→native; there is no canonical "what was requested" object to replay. The shape-stability invariant the fallback engine depends on becomes per-provider rather than systemic. + +**(b) Per-provider entry endpoints — `/v1/anthropic/messages`, `/v1/openai/chat/completions`, `/v1/mistral/...`.** Each entry endpoint speaks its provider's native shape; no translation. Rejected: this just relocates the N² problem to the client side — each client now needs N integrations (one per provider's wire format), and the unified-endpoint value proposition (spec §1: "one HTTP endpoint, multiple subscriptions behind it") is dead on arrival. Cross-provider fallback also stops being client-transparent: clients have to know about the fallback because they have to switch endpoints when the primary fails. + +**(c) Use Anthropic Messages format as the IR (Anthropic-native as canonical).** This privileges Anthropic's shape as OLP's internal language. Rejected on two grounds: (1) Anthropic is currently one provider in eight; privileging its shape codifies provider preference into the IR, which is the wrong abstraction layer; (2) Anthropic-shape is rich enough to be lossy when translated to spartans like Mistral Vibe, so the IR would have features no other provider could use without elaborate fallback in the translator. The IR should be neutral, with provider extensions documented per-plugin, not Anthropic-shaped with per-plugin de-features. + +**(d) Use OpenAI Chat Completions as the IR (entry-surface as canonical).** Privileges OpenAI shape. Rejected: this is the implicit shape of alternative (a), just lifted into a named layer. The coupling to OpenAI's evolving spec means every OpenAI deprecation / addition is a cross-cutting change. The IR exists precisely to decouple OLP-internal logic from OpenAI's release cadence. + +**(e) Defer IR design and start with two providers tightly coupled (Anthropic + OpenAI), refactor to IR when adding the third.** Rejected: this defers the architectural decision into a code-rewrite-under-pressure moment (adding Mistral while two providers are in production). The pattern of "we'll abstract it later" is what produced the hybrid awkwardness OCP ADR 0005 warned against. Pay the IR cost up front, with v1.0 deliberately narrow, and expand via amendment. + +## Sources + +- OLP v0.1 spec §4.1 (Single-protocol entry: OpenAI-compatible) +- OLP v0.1 spec §4.2 (Plugin-based provider system, including IR subsection) +- OLP v0.1 spec §8 Q-C (Tool-calling translation differences across providers — informs why IR v1.0 keeps tool calling to a basic subset) +- OCP ADR 0003 (`models.json` SPOT) — informs the "OLP-owned canonical shape, not external surface" framing diff --git a/docs/adr/0004-fallback-engine.md b/docs/adr/0004-fallback-engine.md new file mode 100644 index 0000000..e83cbdf --- /dev/null +++ b/docs/adr/0004-fallback-engine.md @@ -0,0 +1,108 @@ +# ADR 0004 — Fallback Engine Semantics and Safety + +- **Date:** 2026-05-23 +- **Status:** Accepted (bootstrap) +- **Authors:** project maintainer (with AI drafting assistance) +- **Related:** OLP v0.1 spec §4.3; ADR 0001 (project founding — fallback is OLP's core value proposition); ADR 0002 (plugin architecture); ADR 0003 (IR — fallback replays IR across chain hops) + +## Context + +A multi-provider proxy is only valuable if it fails over gracefully. Per spec §1, OLP's core value proposition is "your IDEs and family clients keep working as long as *any* of your subscriptions has quota left." If Anthropic returns 529 (overloaded), or OpenAI returns 429 with `insufficient_quota`, or `claude -p` exits non-zero, the client should not see an error; the client should see the next provider in the chain transparently take over. + +Naive fallback is dangerous. The most-natural-looking implementation — "any error, retry next provider" — is catastrophic for any request with side effects. Consider a `/v1/chat/completions` request whose first response chunk contained `{"role": "assistant", "tool_calls": [{"name": "write_file", "arguments": "..."}]}`, and the client's agent loop has already begun executing that tool call when the SSE stream breaks. If OLP retries against the next provider, the second provider's response may contain a *different* tool call — and now the client agent has executed two different write_file operations from what looks to them like one request. This is exactly the "duplicate write" failure mode that makes naive retry a foot-gun in any HTTP system with non-idempotent operations. + +The structural answer is **fallback only on idempotent failures**: if the response has not started streaming back to the client, fallback is safe. If the response has started, the request is not retried — the client sees the truncation and decides what to do (an IDE agent loop typically has its own retry logic with awareness of partial state; OLP cannot reason about that). + +The second axis of fallback design is **trigger taxonomy**. Not every provider response is a fallback signal. Anthropic 401 (auth failure) is a fallback signal only if it persistently fails — otherwise it's a configuration problem the user needs to fix, not a quota problem the proxy should hide. Anthropic 429 with no body might be a transient burst limit; OpenAI 429 with `insufficient_quota` is a hard out-of-credit signal. The fallback engine has to distinguish these. + +The third axis is **chain advancement**. Spec §4.3 calls for one-provider-at-a-time advancement: if the chain is `[anthropic, openai, mistral]` and Anthropic fails, advance to OpenAI; if OpenAI also fails, advance to Mistral; if Mistral also fails, return the *original* error (not Mistral's, not the cascade) with `X-OLP-Fallback-Exhausted` listing tried providers. This preserves the client's ability to debug — the first failure is the load-bearing signal, not the last one. + +The fourth axis is **observability**. Every fallback hop must be loggable and counted. `X-OLP-Fallback-Hops: 0` on a successful primary serve; `X-OLP-Fallback-Hops: 1` on first-fallback success; `X-OLP-Provider-Used: openai` so the client knows which subscription consumed the quota. These headers are user-facing instrumentation, not internal-debug; they are spec'd in §4.7. + +## Decision + +Per spec §4.3, the OLP fallback engine implements the following: + +**Trigger taxonomy.** + +- **Hard triggers (mandatory, non-configurable).** + - HTTP 5xx from provider's underlying API (after spawned CLI surfaces them) + - HTTP 4xx that semantically indicate quota exhaustion (Anthropic 529 overloaded, OpenAI 429 with `insufficient_quota` body, Anthropic post-2026-06-15 credit-pool-exhausted indicator if a body discriminator exists) + - Provider CLI exit code ≠ 0 with no usable response chunks streamed + - Provider CLI spawn timeout (configurable per-provider via `hints.maxSpawnTimeMs`) + +- **Soft triggers (configurable per chain).** + - `credit_pool_percent_threshold` — fall back before hitting Anthropic's 100% Agent SDK Credit consumption, e.g., at 90% (per spec §4.3 example config) + - `daily_request_count_threshold` — fall back after N requests on the primary today + - `five_hour_window_percent_threshold` — fall back based on rolling 5h window quota usage + - Soft triggers are evaluated *before* spawn. If a soft trigger fires for the primary, the proxy advances to the next chain entry without attempting the primary at all. + +- **Deterministic triggers (deferred to v1.x).** Time-of-day routing, request-content routing ("code requests prefer Codex"). Out of scope for v0.1; tracked in spec §8 future work. + +- **Cost-aware triggers (deferred to v1.x).** Per spec §4.3 + §8, fully cost-aware automatic fallback is deferred until provider cost-reporting is reliably retrievable. v0.1 ships without this. + +**Fallback safety — first-chunk rule.** A request is eligible for fallback **only** if no response chunk has been emitted to the client. Specifically: + +```js +if (responseChunksEmitted === 0 && triggerMatches(error)) { + advanceChainAndRetry(); +} else { + surfaceErrorToClient(); // client sees truncation; client decides +} +``` + +This is non-negotiable for v1.0. Post-first-chunk truncations are surfaced to the client with the same error semantics OpenAI uses for streaming interruptions. Tool-using clients (agent loops) typically have their own handling for partial responses; OLP cannot second-guess that handling. + +**Chain advancement — one at a time.** Given a chain `[A, B, C]`: +1. Try A. If A succeeds, return; emit `X-OLP-Fallback-Hops: 0`, `X-OLP-Provider-Used: A`. +2. If A's failure matches a hard or soft trigger AND no chunks emitted: try B. If B succeeds, return; emit `X-OLP-Fallback-Hops: 1`, `X-OLP-Provider-Used: B`. +3. If B also fails: try C. Same logic. +4. If all of A, B, C fail: return **A's original error** (not B's, not C's) with `X-OLP-Fallback-Exhausted: A,B,C` listing the chain order, and per-provider failure detail in a debug header `X-OLP-Fallback-Detail` (JSON, owner-only — gated behind a config flag for non-owner keys). + +**Observability headers (per spec §4.7).** +- `X-OLP-Provider-Used: ` +- `X-OLP-Model-Used: ` +- `X-OLP-Fallback-Hops: ` +- `X-OLP-Cache: hit | miss | bypass` +- `X-OLP-Latency-Ms: ` + +Each fallback hop emits a structured log event with: timestamp, chain id, hop index, failed provider, trigger type, IR request hash, downstream provider that was tried next. + +**No fallback for client-side errors.** HTTP 400, 401 (auth misconfigured), 403, 404, 422 from a provider are *not* fallback triggers. They indicate the request itself is malformed or the user's auth is wrong, and silently masking that by trying the next provider would prevent the user from ever discovering the misconfiguration. These errors are surfaced to the client immediately with provider attribution. + +## Consequences + +**Positive** +- The first-chunk rule eliminates the duplicate-side-effect failure mode at the architectural level. There is no configuration knob, no per-provider override, no "we'll try to be smart about it" — the rule is binary and the safety guarantee is binary. +- The trigger taxonomy is enumerated and bounded. A reviewer reading a PR that proposes a new trigger has a clean checkpoint: "is this hard, soft, deterministic, or cost-aware? cite the spec §4.3 category." Triggers outside the four categories require an ADR amendment. +- Chain advancement returning the *original* error (not the cascade) makes debugging tractable. The user sees what their primary provider said, not a confusing tail-end "Mistral failed" message that obscures the actual problem. +- Observability headers are surface-level. Clients (and the maintainer running `curl -i`) can immediately see which provider served a request and whether fallback fired, without enabling debug logging. + +**Negative** +- Some failures *don't* fall back. Post-first-chunk truncations surface to the client as truncations. This is the correct behavior, but it means OLP cannot achieve "literally never fails as long as one provider is alive" — clients still see truncations when a provider dies mid-response. This is a property of the safety guarantee, not a bug. +- The credit-pool-percent-threshold soft trigger requires per-provider quota retrievability. Per spec §4.2, `quotaStatus` may return null if a provider doesn't expose it. Soft triggers gracefully degrade (treat null as "don't fire") but the value proposition is weaker for providers without retrievable quota. +- The trigger configuration is per-chain, not global. A user with three chains has three places to tune thresholds. The dashboard surface (spec §4.6) makes this navigable; raw `config.json` editing is the source of truth. + +**Mitigations** +- The first-chunk rule applies at the byte level (any data written to the response stream blocks fallback), not at the semantic level (where it would require parsing). Implementation is a single boolean flag tracking whether anything has been written; the rule is auditable in a few lines of code. +- Provider plugins that cannot retrieve quota (`quotaStatus` returns null) are documented in their plugin header and in `docs/provider-caveats.md`. Soft triggers configured against such providers issue a startup warning so the user knows the trigger will never fire. +- The deferred trigger types (deterministic, cost-aware) are tracked in spec §8 with explicit Q-tags. Adding them later is an ADR amendment plus a contract addition to the trigger taxonomy, not a redesign. + +## Alternatives considered + +**(a) Full retry-on-any-error.** Any non-2xx response triggers fallback. Rejected: this is the duplicate-side-effect foot-gun, and it also masks client-side errors (a malformed request would silently parade through every provider in the chain before being surfaced). The safety guarantee OLP is making would be impossible to make. + +**(b) No fallback at all — pure routing only.** OLP picks one provider per request based on chain config but does not retry on failure. Rejected: this defeats the spec §1 value proposition. The whole reason for OLP's existence is "as long as *any* subscription has quota left, things work." No-fallback gives the user a manually-coordinated multi-provider setup, which they could have built with environment-variable swapping in their IDE. + +**(c) Semantic retry — try to detect when retry is safe.** Inspect the in-flight request, the partial response, and the failure mode; if the partial response is reasoning-only (no tool calls emitted yet), retry; if tool calls have been emitted, do not retry. Rejected for v1.0 as too ambitious. Distinguishing "safe to retry" requires parsing the partial IR response, identifying side-effecting tool calls (which means OLP has to know which tools are side-effecting — it cannot), and reasoning about idempotency at the application layer. v1.0 ships with the conservative first-chunk rule; semantic retry is a candidate for a future ADR if real failure modes justify the complexity. + +**(d) Retry-after-N-seconds before fallback.** When a provider transiently fails (e.g., Anthropic 529), wait N seconds and retry the same provider before advancing the chain. Rejected as a separate concern from this ADR. Per-provider retry-with-backoff before chain advancement is a useful refinement, but it's an enhancement to the "try A" step of chain advancement, not a change to the chain advancement rules. Tracked as future work; v1.0 has a single attempt per chain hop. + +**(e) Parallel fan-out — issue requests to all providers in the chain simultaneously, race them.** Rejected: this consumes quota on all providers regardless of which one's response is used, which directly violates the spec §1 value proposition (minimize quota consumption). Quota is the constraint OLP exists to manage; spending it speculatively is the wrong shape. + +## Sources + +- OLP v0.1 spec §4.3 (Fallback engine, including trigger types, fallback safety, example config) +- OLP v0.1 spec §4.7 (Observability — response headers) +- OLP v0.1 spec §8 Q-B, Q-E (Open questions about provider-specific quota and rate-limit-window awareness) +- OCP architecture context (`~/ocp/server.mjs` does no fallback — it forwards to one Anthropic endpoint; the absence of fallback handling in OCP is informative about what greenfield design OLP needs) diff --git a/docs/adr/0005-cache-cross-provider.md b/docs/adr/0005-cache-cross-provider.md new file mode 100644 index 0000000..02c351f --- /dev/null +++ b/docs/adr/0005-cache-cross-provider.md @@ -0,0 +1,104 @@ +# ADR 0005 — Cache Layer Cross-Provider Design + +- **Date:** 2026-05-23 +- **Status:** Accepted (bootstrap) +- **Authors:** project maintainer (with AI drafting assistance) +- **Related:** OLP v0.1 spec §4.4; ADR 0002 (plugin architecture); ADR 0003 (IR — cache keys are computed over IR shape); ADR 0004 (fallback — cross-provider cache misses are correct on fallback) + +## Context + +OCP shipped a four-layer cache hardening (D1+D2+D3+D4) in v3.13.0 (2026-05-07 per the MEMORY.md entry). The four layers: +- **D1 — per-key isolation:** each OCP API key has its own cache namespace, so one user's cache doesn't leak to another's. +- **D2 — `cache_control` bypass:** Anthropic prompt-caching markers in the request bypass OLP's response cache (the prompt cache lives at Anthropic's side; double-caching would shadow Anthropic's TTLs). +- **D3 — chunked stream replay:** SSE streams are replayed from cache without re-spawning the CLI, preserving the chunking pattern the client expects. +- **D4 — singleflight:** concurrent identical requests share one spawn; all callers receive the same response. + +All four are valuable in OLP. None of the four translate directly because OCP's cache key was Anthropic-specific in composition: + +``` +ocp_cache_key = sha256({ messages, tools, temperature, response_format, cache_control }) +``` + +OCP never had a `provider` field in the key because there was only one provider. OLP serves multiple providers, and multiple models per provider. A cache hit between `anthropic/claude-sonnet-4-6` and `openai/gpt-5-codex` would be catastrophic: identical prompts produce *different* outputs on different models, and even on the same model from different providers the response style and tool-calling conventions diverge enough that a cross-provider cache hit is just a wrong answer wearing a correct cache key. + +The simple fix is to include `provider` and `model` in the cache key. That's structurally correct but has a non-obvious consequence: when a fallback chain advances from `anthropic/sonnet` to `openai/codex`, the cache lookup against the new provider misses — even if the *prompt* is identical to a previous successful Anthropic serve. This is the right behavior (different model = different output = different cache entry), but it's worth naming explicitly so future readers don't try to "fix" the cache to share entries across providers. + +A second design point: OLP's cache key is computed over the **IR** (per ADR 0003), not over the raw OpenAI request shape. This matters because OpenAI shape evolves (new fields, deprecated fields) and we don't want every entry-surface change to invalidate every cache entry. IR is OLP-owned and changes only via amendment ADR, so cache-key stability is governed by OLP's own release cadence, not OpenAI's. + +The third design point: `cache_control` (D2) is Anthropic-specific in v1.0. Other providers may grow their own prompt-cache markers over time (OpenAI has prompt caching but the marker semantics differ; Mistral has none as of 2026-05). Per spec §4.4, OLP honors Anthropic `cache_control` markers as bypass signals when the routing target is Anthropic; for non-Anthropic targets, the bypass markers are noop'd (logged once per request at debug level so users can see they were ignored). Future providers' prompt-cache markers extend the bypass logic per-provider; the bypass mechanism is provider-pluggable, not Anthropic-specific. + +## Decision + +Per spec §4.4, OLP's cache layer: + +**Cache key composition (v1.0).** +``` +key = sha256(JSON.stringify({ + provider, // e.g., 'anthropic', 'openai', 'mistral' + model, // the actual model that served the request (provider-native form) + messages, // IR messages[] — normalized form, not OpenAI raw + tools, // IR tools[] if present, null otherwise + temperature, // included if non-default + response_format, // included if present + cache_control, // Anthropic prompt-caching markers (the markers themselves; the bypass logic is separate) +})) +``` + +**Per-model isolation.** Cache entries are keyed on `(provider, model)` pair. `anthropic/claude-sonnet-4-6` and `openai/gpt-5-codex` produce distinct cache entries for identical IR messages. There is no cross-model cache sharing in v1.0; this is intentional and correct. + +**D1 — per-key isolation (ported from OCP v3.13.0).** Each OLP API key has an independent cache namespace. Cache directory structure: +``` +~/.olp/cache///.json +``` +The `` segment ensures per-key isolation; the `` is the first two hex chars of the key for filesystem-fanout sanity at high cache counts. + +**D2 — `cache_control` bypass (ported, scope-expanded).** If the IR request contains Anthropic `cache_control` markers AND the active provider in the current chain hop is Anthropic, the OLP response cache is bypassed (Anthropic's own prompt cache is consulted at the provider). If the active provider is not Anthropic, the `cache_control` markers are stripped from the IR before provider translation (so they don't get passed to providers that wouldn't understand them) and a debug log entry is emitted. Future provider-specific bypass markers extend this rule by adding their own provider-conditional bypass logic. + +**D3 — chunked stream replay (ported).** When a cache hit occurs on a `stream: true` request, the cached response is replayed as SSE chunks with the same chunking granularity as the original spawn. This requires storing the chunk boundaries (not just the concatenated content) in the cache entry. Cache entry format includes a `chunks[]` array, each with `delta` and `timestamp` relative to spawn start; replay throttles to the original timing pattern (configurable: real-time replay vs. burst-replay; default burst-replay because real-time replay adds latency without value for cached requests). + +**D4 — singleflight (ported).** Concurrent requests with identical cache keys share one spawn. The first request triggers the spawn; subsequent identical requests block on a per-key promise until the first completes, then all read the same response. Per the OCP MEMORY.md learning (`learnings/concurrency_dedup_test_signals.md`), singleflight is verified by observing that N concurrent requests return within milliseconds of each other when the cache key matches. + +**Cross-provider fallback cache behavior (new for OLP).** When a request falls back from `anthropic/sonnet` to `openai/codex`, the cache lookup against the new `(openai, codex)` key likely misses. This is correct: even if Anthropic had served the same prompt successfully yesterday, the codex response is a different output and warrants its own cache entry. There is no "cache hit on any provider's prior serve" mode; that would defeat the per-model isolation invariant. + +**Cache write conditions.** A response is cached if and only if: +1. The response completed successfully (no truncation, no error mid-stream). +2. The request did not include `cache_control` bypass markers for the active provider. +3. The provider's `hints.cacheable` flag is not `false` (a provider plugin can opt out of caching entirely for, e.g., real-time use cases — none do in v1.0). +4. The response is below a size cap (default 10 MB; configurable). Cache is for hot-path repeat requests, not bulk archive. + +## Consequences + +**Positive** +- Cross-model contamination is impossible. The `(provider, model)` prefix on every cache key means `claude/sonnet` and `codex/gpt-5` cannot collide even on identical prompts. The cache invariant is auditable in one line of code. +- All four OCP cache hardening layers (D1+D2+D3+D4) port to OLP, preserving the operational properties OCP achieved in v3.13.0. The OCP MEMORY.md learnings about cache-related pitfalls (e.g., not hashing the randomUUID-included full JSON; using content-only sha256) carry over. +- Fallback to a different provider correctly misses cache. The user's quota is consumed on the new provider, but the new provider's response is now cached for future identical requests on the new (provider, model) pair. +- IR-based cache keys (per ADR 0003) decouple cache stability from OpenAI's entry-surface evolution. A new OpenAI field that doesn't change semantics (e.g., a cosmetic field rename) does not invalidate existing cache entries. + +**Negative** +- Cache hit rate is lower than OCP's single-provider rate by construction. Per-model isolation means the cache is partitioned N-ways across N (provider, model) pairs. For the maintainer's typical usage (~70% claude, ~25% codex, ~5% other), the most-used pair retains high hit rate; less-used pairs have effectively cold caches. This is the right behavior, not a bug. +- The D3 chunked-replay format adds complexity to cache entries (storing chunk boundaries, not just concatenated content). Cache entries are larger than they would be with naive content storage. Disk usage scales with chunks count; for typical claude responses this is modest, but it is real. +- `cache_control` markers being silently stripped for non-Anthropic providers is a subtle behavior. Users who configure aggressive Anthropic prompt caching and then fall over to OpenAI will get full-cost OpenAI responses without prompt caching, even on prompts they marked as cacheable. The debug log entry mitigates surprise; the README's caveats section names this explicitly. + +**Mitigations** +- Cache hit rate is monitored via `/cache/stats` (per spec §4.6), broken down by `(provider, model)` pair. Low hit rates on specific pairs are visible to the maintainer and can inform chain configuration (e.g., "this pair has 2% hit rate; is it worth being in this chain?"). +- Cache entry size cap (default 10 MB) prevents pathological growth from the chunked-replay format. Sizes above cap are not cached; logged once. +- The non-Anthropic-provider behavior for `cache_control` markers is tested in `test-features.mjs` to verify the bypass is correctly noop'd, the markers are stripped before provider translation, and the debug log is emitted. The marker-strip behavior is the structural counter-measure against accidentally passing Anthropic-specific markers to providers that wouldn't understand them. + +## Alternatives considered + +**(a) Model-agnostic caching (cache key includes provider but not model).** A cache hit on `anthropic/claude-sonnet-4-6` for a prompt could serve a request for `anthropic/claude-opus-4-7` if the prompt is identical. Rejected: this is cross-model contamination, and the outputs are simply different. The cache becomes a source of wrong answers. + +**(b) Cache disable on fallback paths.** Skip cache lookup entirely for any request that's reached fallback hop ≥ 1. Rejected: this would mean every fallback serve is a fresh spawn even if the new provider has served the same prompt before. Per-model isolation already gives the right behavior here (the fallback hop's cache lookup is against the new (provider, model) and is a miss only if that pair hasn't served before). + +**(c) Cross-provider canonicalization — cache a "model-tier" key (e.g., 'sonnet-equivalent') rather than (provider, model).** Anthropic/sonnet, openai/codex, and mistral/devstral could all share a cache entry tagged "sonnet-equivalent". Rejected on two grounds: (1) cross-model output equivalence is false — the outputs *differ*, even if the inputs are equivalent; (2) defining "sonnet-equivalent" is a capability-routing problem (spec §1 non-mission explicitly excludes capability routing). The cache should reflect real outputs, not assumed equivalence classes. + +**(d) Don't port D3 (chunked stream replay) — replay cache hits as a single chunk.** Simplifies cache entries; loses the chunking-pattern fidelity OCP achieved in v3.13.0. Rejected: some clients (notably OpenClaw and Continue) make UX decisions based on chunking pattern (e.g., "is this thinking, or is this output?"); collapsing to a single chunk breaks those decisions even though the content is correct. D3's complexity earns its keep. + +**(e) Defer caching to a future ADR — ship v1.0 without cache.** Rejected: the OCP cache layer is one of the most successful structural decisions in OCP's history, and the post-2026-06-15 cost picture makes caching *more* valuable, not less (every cache hit is a credit not consumed). Caching is in scope for v1.0. + +## Sources + +- OLP v0.1 spec §4.4 (Cache layer — inherited from OCP, generalized) +- OCP v3.13.0 release context — MEMORY.md entry 2026-05-07 (Mac → OCP v3.13.0 cache layer hardening shipped) +- OCP `learnings/concurrency_dedup_test_signals.md` — informs D4 (singleflight) test methodology +- OCP `keys.mjs` `cacheHash` / `getCachedResponse` / `setCachedResponse` / `clearCache` — the implementation OLP ports diff --git a/docs/adr/0006-provider-inclusion.md b/docs/adr/0006-provider-inclusion.md new file mode 100644 index 0000000..e762900 --- /dev/null +++ b/docs/adr/0006-provider-inclusion.md @@ -0,0 +1,113 @@ +# ADR 0006 — Provider Inclusion / Exclusion Criteria and Risk-Tier Framework + +- **Date:** 2026-05-23 +- **Status:** Accepted (bootstrap) +- **Authors:** project maintainer (with AI drafting assistance) +- **Related:** OLP v0.1 spec §3.1, §10; ADR 0001 (project founding); ADR 0002 (plugin architecture — inclusion criteria gate plugin merges); OCP ADR 0006 (OpenAI Shim Scope — informed the risk-tier framing approach) + +## Context + +OLP's value proposition (spec §1) is multi-provider spreading of LLM subscription quota. The more providers OLP supports, the better its fallback fan-out and the more resilient the overall system. But provider inclusion is not free: each provider's terms of service, enforcement history, and cost-benefit profile differ, and including a provider with poor ToS posture exposes the user to account-revocation or service-termination risk that the user may not have signed up for when they enabled OLP. + +The decision space is not binary "include / exclude." A four-tier framework better matches the actual risk landscape observed in the 2026-05-23 provider deep-dive audit: + +- Some providers (OpenAI Codex, Mistral Vibe, pre-2026-06-15 Anthropic) have permissive ToS with no anti-third-party clauses and healthy ecosystems. These are safe defaults. +- Some providers (xAI Grok, Moonshot Kimi) have *tightened* ToS language recently but no documented enforcement history. The user may reasonably opt in with awareness. +- Some providers (MiniMax, Zhipu GLM, Alibaba Qwen) have explicit anti-automation clauses in their subscription plan terms. Key revocation is a documented risk; account-wide enforcement is not. +- One provider (Google Antigravity) has all three risk factors compounded: explicit named prohibition of third-party tools (OpenClaw, OpenCode, Claude Code by name in Google's FAQ), an active enforcement wave (Feb 2026), and no cost advantage to compensate. Reinstatement is friction-laden (Google Form, unclear SLA). + +The temptation in a permissive design ("include everything that has a CLI") is to treat ToS as the provider's problem, not OLP's. That's wrong: OLP's user is the one whose account gets revoked. The temptation in a conservative design ("include only Tier D — fully safe") is to leave too much value on the table — the Chinese providers (MiniMax/GLM/Qwen) are token-economically attractive for the maintainer's usage patterns even with the consent-required overhead. + +The structural middle path is a tiered framework with explicit consent UX for the optional tiers, modeled on the OCP ADR 0006 "Class A / Class B" precedent of using explicit categorization to manage scope risk. + +One precision point requires emphasis. The Google Antigravity ban is **not** a "whole Google account ban." The verified scope (per the piunikaweb 2026-03-02 OpenClaw exec confirmation cited in spec §3.1) is *Google AI services tier*: Antigravity + Gemini CLI + Cloud Code Private APIs. Gmail, Drive, YouTube, Calendar, and other non-AI Google services are unaffected. Antigravity is excluded from OLP not on ban scope alone — a Google-AI-tier ban would still leave most of the user's Google services intact — but on the *combination* of: (a) explicit named prohibition of third-party tools, (b) no cost advantage versus Codex equivalents, and (c) reinstatement-process friction. Any one of these factors alone would not be sufficient for permanent exclusion; the combination is. + +## Decision + +OLP adopts a four-tier provider classification framework per spec §3.1: + +**Tier A — Excluded.** Vendor's AI suite (note: scoped to AI services, not always whole account) suspended on enforcement; vendor documentation explicitly names third-party proxy tools as prohibited; subscription price does not differentially advantage the provider over Tier D alternatives. Bar is intentionally narrow: **all three conditions** must hold for Tier A exclusion. Single-signal triggers (e.g., a tightening clause with no enforcement) drop to Tier C, not A. + +**Tier B — Optional, default-disabled, explicit consent required.** Service-level key revocation risk; vendor may extend revocation across that vendor's AI surfaces but does not touch non-AI services. ToS contains explicit anti-automation language. Enable requires a one-time consent UX: the consent prompt names the specific provider's policy clause and the realistic outcome (key revocation, not account suspension). Consent is recorded in `~/.olp/config.json` with a timestamp. + +**Tier C — Optional, default-disabled, no consent UX.** ToS language has tightened (recent ToS update with anti-third-party direction) but no enforcement history exists. Enable is a config toggle; no consent prompt because the risk is signal-only rather than documented. + +**Tier D — Default-enabled.** No anti-third-party clauses in subscription ToS; healthy ecosystem; permissive maintainer posture (where verifiable, e.g., maintainer-confirmed via official channel). + +**Current classification table (v0.1):** + +| Provider | Tier | Reason | +|----------|------|--------| +| Anthropic (Claude Code) | D | OCP-inherited. Pre-2026-06-15 subscription model was permissive. Post-2026-06-15 the credit-pool-vs-subscription split is a **quota constraint** within OLP's mission (manage it via fallback), not a **third-party prohibition**. ToS does not name third-party tools. | +| OpenAI Codex | D | Codex maintainer confirmed permissive in Codex GitHub Discussion #8338 (2026 reference). Codex CLI is the official Codex client; subscription auth via ChatGPT account is the intended UX. | +| Mistral Vibe (Le Chat Pro) | D | Usage Policy contains no anti-third-party / anti-automation clauses. Vibe is the official Mistral CLI; Le Chat Pro subscription explicitly includes Vibe. | +| xAI Grok Build | C | Jan 2026 ToS tightening introduced more restrictive automation language; no documented enforcement wave; uses API-key auth (not SuperGrok OAuth) to remain in the lower-risk lane. | +| Moonshot Kimi | C | "3 violations = ban" generic policy; no third-party-CLI-specific enforcement observed; API-key auth (not OAuth). | +| MiniMax | B | Token Plan policy prohibits automated scripts. Key revocation risk documented. Consent: key may be revoked; account remains usable. | +| Zhipu GLM | B | Coding Plan restricted to "officially supported tools." Key revocation risk. Consent UX: key may be revoked; account remains usable. | +| Alibaba Qwen | B | Coding Plan policy prohibits automation. Key revocation risk. Consent UX as above. | +| Google Antigravity | A | Feb 2026 enforcement wave restricted Antigravity + Gemini CLI + Cloud Code Private APIs for affected accounts. **Important scope correction:** the ban is *Google AI services tier* (not whole Google account); Gmail/Drive/YouTube are unaffected per piunikaweb 2026-03-02 OpenClaw exec confirmation. Excluded on the **combination** of (1) Google FAQ explicitly names OpenClaw/OpenCode/Claude Code as prohibited third-party tools — only provider in this survey with named-tool prohibition; (2) AI Pro $20 / Ultra $200 is not differentially cheap vs OpenAI Codex Plus $20 / Pro $200, so no cost advantage to taking on the risk; (3) reinstatement process exists but is via Google Form with unclear SLA. Any single factor would not justify Tier A; the combination does. | + +**Consent UX for Tier B (per spec §3.1).** First-enable triggers a CLI prompt: + +``` +Enable MiniMax provider? + +MiniMax's Token Plan policy prohibits automated scripts. +Realistic outcome of enforcement: your MiniMax API key may be revoked. +Scope: MiniMax service only. Other accounts unaffected. + +This is a one-time consent. Type 'I understand' to enable, anything else to abort: +``` + +On 'I understand', `~/.olp/config.json` records `providers.minimax.consent_at: ` and the provider becomes enabled. Subsequent enables/disables are config toggles without re-prompting unless the consent record is cleared. + +**Future provider additions.** Each new provider plugin (per ADR 0002) requires an accompanying inclusion ADR that: +1. Cites the provider's ToS / Usage Policy and any anti-third-party clauses found. +2. Cites enforcement history (or absence thereof). +3. Compares cost against existing Tier D providers — if no advantage, the addition must justify why the tier risk is worth taking. +4. Assigns a tier (A/B/C/D) using the framework above. +5. If Tier A: the plugin is not built. The ADR documents the exclusion for posterity. +6. If Tier B: the consent UX content is drafted in the ADR. + +The inclusion ADR is a hard prerequisite for the plugin's merge — no ADR, no merge. + +## Consequences + +**Positive** +- Provider risk is enumerated and explicit. Users see in README's "Supported Providers" table which providers are Tier D / C / B / A and what the implications are. Surprise is structurally prevented. +- Tier B consent UX makes the user an active participant in the risk decision. They cannot accidentally enable a key-revocation-risk provider; they have to type 'I understand' literally. +- The framework is forward-compatible. New providers in 2027 / 2028 go through the same four-tier classification; the ADR per provider creates a documented trail of risk decisions over time. +- The Antigravity exclusion is documented at the framework level (not as an ad-hoc "we don't support this"), so future readers see *why* it's excluded — and specifically that the exclusion is on the combination of named-prohibition + no-cost-advantage + reinstatement-friction, not on ban-scope. The piunikaweb 2026-03-02 scope clarification is preserved in the ADR rather than degraded over time into folklore. + +**Negative** +- Tier B's consent UX adds a step to first-enable. Users who already understand the risk perceive friction. The friction is intentional; the consent is the structural mechanism that protects the user. +- The framework is provider-by-provider, not category-by-category. A new Chinese provider similar to MiniMax/GLM/Qwen gets its own ADR even though the pattern is shared. This is intentional — boilerplate-and-paste ADRs are still cheaper than the alternative of "we just classed them together once and now we have a category drift problem when their ToS shifts." +- Tier-A exclusion is *permanent* in the current framework, with no documented reinstatement path. If Google were to walk back the named-tool prohibition and the cost picture changed, Antigravity would need a fresh ADR superseding this one. The framework deliberately makes Tier A "needs a new decision to revisit," not "auto-reconsidered each release." + +**Mitigations** +- The README "Supported Providers" table is the user's first stop and surfaces tiers prominently. The dashboard (spec §4.6) shows tier per enabled provider so users see it on every dashboard load, not only at enable-time. +- Tier B consent prompts include the specific clause cited (e.g., "MiniMax Token Plan policy: 'No automated scripts'"). This avoids vague "this provider has terms" boilerplate that users would tune out. +- The framework's Tier A bar (three conditions, not one) is the structural counter-measure against "any provider with a tightening clause gets banned forever." Single-signal events drop to Tier C; only the combination justifies Tier A. + +## Alternatives considered + +**(a) Blanket "include everything that has a CLI."** Add Antigravity, add anything with a programmatic entry point, leave ToS judgment to the user. Rejected: this is the failure mode where the user's account gets revoked by Google because OLP's defaults said it was fine. OLP's defaults are part of OLP's contract with the user; they have to encode actual risk awareness. + +**(b) Blanket "include only Tier D — fully safe."** Skip MiniMax, GLM, Qwen, Grok, Kimi entirely. Rejected: the Chinese providers are token-economically attractive enough that the consent UX is justified rather than blanket exclusion. The maintainer explicitly identified the Token Plan pricing as a reason to provide an opt-in path. A user choosing to opt in with consent is exercising informed choice; a blanket exclusion forecloses that choice. + +**(c) Two-tier system (safe / risky) instead of four.** Simpler framework. Rejected: it conflates the Grok/Kimi case (signal-only, no enforcement history) with the MiniMax/GLM/Qwen case (explicit anti-automation language, documented revocation risk), which warrant different consent UX. Conflation either over-warns the Grok/Kimi user or under-warns the MiniMax user; four tiers separates the cases. + +**(d) Per-provider ToS auto-check via a scheduled job.** A `bin/olp doctor` style command periodically fetches each provider's ToS and diffs against a stored snapshot, flagging changes. Rejected for v1.0: ToS pages are unstable HTML, anti-bot-protected, and frequently A/B-tested per region. Auto-checking is a project of its own; out of scope for v0.1. Manual review on each provider addition + annual re-review is the v1.0 substitute. + +**(e) Document tier classifications in spec §3.1 only, no per-provider ADR for additions.** Lower-ceremony alternative. Rejected: a per-provider ADR creates the structural review gate (per ADR 0002) that prevents silent provider additions. The spec is updated alongside the ADR (decision log §10); the ADR carries the reasoning detail. Both layers earn their keep. + +## Sources + +- OLP v0.1 spec §3.1 (Scope — tiers, provider classifications, Antigravity exclusion reasoning) +- OLP v0.1 spec §10 (Decision log entries for the tier classifications) +- OCP ADR 0006 (OpenAI Shim Scope) — informs the risk-tier framing methodology of "explicit categorization with named criteria" +- Google FAQ on AI services (Feb 2026 enforcement wave) — referenced via spec §3.1 +- piunikaweb 2026-03-02 OpenClaw exec confirmation on ban scope — referenced via spec §3.1 +- Codex GitHub Discussion #8338 — Codex maintainer confirmation of permissive third-party-client posture +- 2026-05-23 provider deep-dive audit (on file per spec §11) diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 0000000..0d0e1de --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,37 @@ +# Architecture Decision Records — OLP + +This directory holds the OLP Architecture Decision Records (ADRs) — short documents that capture the **why** behind structural choices. + +Read these before proposing governance, plug-in contract, IR, fallback, cache, or provider-inclusion changes. + +## Numbering + +ADRs in OLP start at `0001` (the project's founding decision). OLP did not inherit the `0001` placeholder convention OCP used; the project starts on a fresh numbering line. + +New ADRs increment from the highest existing number. Filenames are `NNNN-.md`. + +## Index + +| ADR | Title | What it covers | +|---|---|---| +| [0001](0001-project-founding.md) | Project Founding | Why OLP exists (the 2026-06-15 Anthropic billing-split trigger), mission + non-mission boundaries, explicit supersession of OCP ADR 0005 "No Multi-Provider". | +| [0002](0002-plugin-architecture.md) | Plugin Architecture for Providers | The `lib/providers/.mjs` plug-in model, the Provider contract (name / models / auth / spawn / estimateCost / quotaStatus / healthCheck / hints), static in-tree registry, why-not full external plugin discovery. | +| [0003](0003-intermediate-representation.md) | Intermediate Representation (IR) Design | The OLP-internal canonical request/response shape between the OpenAI-compat entry surface and each provider plugin. v1.0 IR fields, IR-vs-OpenAI-vs-native-provider three-shape model, IR is not exposed externally. | +| [0004](0004-fallback-engine.md) | Fallback Engine Semantics & Safety | Trigger taxonomy (Hard / Soft / Deterministic-deferred / Cost-aware-deferred), idempotent-failure safety (first-chunk rule), chain advancement one-at-a-time, observability headers. | +| [0005](0005-cache-cross-provider.md) | Cache Layer Cross-Provider Design | Cache key composition over `(provider, model, messages, …)`, per-model isolation, D1+D2+D3+D4 port from OCP v3.13.0, cross-provider fallback cache behaviour (correct miss). | +| [0006](0006-provider-inclusion.md) | Provider Inclusion / Exclusion + Risk-Tier Framework | The 4-tier classification (A excluded / B explicit consent / C opt-in / D default-enabled), current v0.1 inventory, Antigravity exclusion rationale (named prohibition + no cost advantage + reinstatement friction), consent UX, future provider addition procedure. | + +## When to write a new ADR + +Open one whenever: + +- A structural rule is being added or changed (e.g., new IR field, new trigger type, new cache-key composition). +- A new provider plugin is being proposed (per ADR 0002 + ADR 0006 — ADR is a hard prerequisite for plugin merge). +- A decision encodes a lesson from an external event (provider policy change, drift incident). +- A future contributor reading the code alone could plausibly undo or re-litigate the choice. + +Skip ADRs for routine implementation choices (algorithm pick, naming) — those belong in commit messages. + +## Format + +Keep ADRs short — Context / Decision / Consequences / Alternatives is the standard skeleton. Cite incidents, PRs, or commits where useful. Length per ADR ~150–250 lines; longer than that, split. diff --git a/package.json b/package.json new file mode 100644 index 0000000..5c951e0 --- /dev/null +++ b/package.json @@ -0,0 +1,30 @@ +{ + "name": "olp", + "version": "0.1.0-bootstrap", + "description": "Personal multi-provider LLM proxy. Successor to OCP. One HTTP endpoint, multiple subscriptions behind it, automatic routing + fallback + caching.", + "type": "module", + "main": "server.mjs", + "scripts": { + "test": "node test-features.mjs", + "start": "node server.mjs" + }, + "engines": { + "node": ">=18" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/dtzp555-max/olp.git" + }, + "license": "MIT", + "private": true, + "keywords": [ + "llm", + "proxy", + "multi-provider", + "anthropic", + "openai", + "mistral", + "fallback", + "cache" + ] +}