diff --git a/ALIGNMENT.md b/ALIGNMENT.md index ebb79e1..5a50a67 100644 --- a/ALIGNMENT.md +++ b/ALIGNMENT.md @@ -48,9 +48,9 @@ Each provider plugin in `lib/providers/.mjs` is governed by the underlying | Provider key | Provider CLI | Audit pin (TBD on Phase-1 spawn) | Risk Tier (see § Risk Tier Framework) | |---|---|---|---| -| `anthropic` | `claude -p` from `@anthropic-ai/claude-code` | inherits OCP's `cli.js` 2.1.89 audit pin at fork; OLP-side pin set when Phase 1 lands | D (pre-2026-06-15) / re-evaluate post-2026-06-15 | -| `openai` | `codex exec --json` from OpenAI Codex CLI | TBD at Phase 2 spawn | D | -| `mistral` | `vibe --prompt --output json` from Mistral Vibe CLI | TBD at Phase 3 spawn | D | +| `anthropic` | `claude -p` from `@anthropic-ai/claude-code` | inherits OCP's `cli.js` 2.1.89 audit pin at fork; OLP-side pin: `@anthropic-ai/claude-code` v2.1.89 (observed at D4 — `lib/providers/anthropic.mjs` header). Re-evaluate post-2026-06-15 per One-shot Triggered Audit above. | D (pre-2026-06-15) / re-evaluate post-2026-06-15 | +| `openai` | `codex exec --json` from OpenAI Codex CLI | Codex CLI reference page: https://developers.openai.com/codex/cli/reference (retrieved 2026-05-23 — §§ "codex exec [flags] PROMPT", "--json / --experimental-json", "--model, -m"; D6 WebFetch-verified reachable). Secondary authority: https://developers.openai.com/codex/cli/features §§ "Supported Models", "Automation". | D | +| `mistral` | `vibe --prompt --output json` from Mistral Vibe CLI | Mistral Vibe terminal quickstart: https://docs.mistral.ai/mistral-vibe/terminal/quickstart (retrieved 2026-05-23 — § "--prompt flag triggers programmatic mode; --output selects format (text, json, streaming)"; D8 WebFetch-verified reachable). Configuration authority: https://docs.mistral.ai/mistral-vibe/terminal/configuration (§§ auth file `~/.vibe/.env`, `MISTRAL_API_KEY` env var). | D | | `grok` | `grok -p --output-format streaming-json` (xAI Build) | TBD at Phase 8+ enable | C | | `kimi` | `kimi -p --output-format stream-json` (Moonshot) | TBD at Phase 8+ enable | C | | `minimax` | TBD CLI command (MiniMax Token Plan) | TBD at opt-in enable | B | @@ -148,6 +148,8 @@ In addition to the recurring 14 May audit below, the following one-shot audits a - **Antigravity primary-source pin (open-ended, no deadline)** — find a primary-source URL or archival snapshot for the Google FAQ language naming OpenClaw / OpenCode / Claude Code as prohibited. Pin the URL + retrieval timestamp into ADR 0006. If the primary source cannot be located within 90 days of this commit and only secondary reports remain, file an ADR 0006 amendment proposing a re-classification (Tier A may need to soften to Tier C "evidence-backed signal only" if the prohibition cannot be primary-sourced). +- **OpenAI Codex ToS formal pin (trigger: Phase 2 E2E enable for Codex, OR 2026-12-31, whichever comes first)** — ADR 0006 § Classification table row "OpenAI Codex" currently relies on Codex GitHub Discussion #8338 (maintainer posture: "I'm an engineer, not a lawyer") as the authority pin, which is a maintainer statement of permissive intent rather than a formal ToS blessing. Before Codex transitions from Candidate to Enabled (Phase 2 E2E audit), locate and pin a primary-source ToS URL or official OpenAI policy document that explicitly addresses third-party CLI proxying. If no such document exists by 2026-12-31, file an ADR 0006 amendment documenting the absence as the formal conclusion of this audit task. Update ADR 0006 § Classification table row "OpenAI Codex" with the audit outcome. + --- ## Annual Alignment Audit diff --git a/README.md b/README.md index 3d733e7..8017fde 100644 --- a/README.md +++ b/README.md @@ -151,7 +151,7 @@ Phase 1 is in progress. This table reflects what is currently shipped vs. what i | `lib/fallback/engine.mjs` | ✅ Shipped | Trigger evaluation + chain advancement (ADR 0004) | | Soft trigger data path (`quotaStatus()` polling) | 📋 Planned (v1.x) | Evaluation logic shipped + tested; data ingestion deferred per ADR 0004 Amendment 2 | | `models-registry.json` | ✅ Shipped | SPOT for `(provider, model)` metadata | -| `test-features.mjs` | ✅ Shipped | 328-test suite (CI: `test.yml`) | +| `test-features.mjs` | ✅ Shipped | Comprehensive test suite covering IR, cache, fallback, and integration paths (CI: `test.yml`) | | `lib/keys.mjs` | 📋 Planned (Phase 2) | Multi-key auth, per-key namespacing, audit log | | `dashboard.html` | 📋 Planned (Phase 6) | Owner-only multi-provider dashboard | | `docs/provider-caveats.md` | 📋 Planned (Phase 3+) | Lossy-translation reference; for now documented inline in each plugin header | @@ -169,7 +169,7 @@ OLP is a Node.js (ESM, `.mjs`) HTTP proxy with no build step and minimal depende - **Entry surface** — `server.mjs` handles `/v1/chat/completions` and the administrative endpoints. Governed by OpenAI's `/v1/chat/completions` specification as the wire authority. See [`ALIGNMENT.md` § Authorities](./ALIGNMENT.md#authorities). - **Intermediate Representation (IR)** — `lib/ir/` normalizes between the entry surface and provider-native shapes. The IR is the lingua franca; any extension is an [ADR 0003](./docs/adr/0003-intermediate-representation.md) amendment. - **Provider plugins** — `lib/providers/.mjs`. Each plugin implements the contract in [ADR 0002 (Plugin Architecture for Providers)](./docs/adr/0002-plugin-architecture.md), spawns its CLI, and translates between IR and provider-native IO. -- **Cache layer** — `lib/cache/` is a content-addressed cache keyed on `(provider, model, messages, tools, temperature, response_format, cache_control)`. Per-key isolation, prompt-caching bypass, chunked stream replay, and singleflight. See [ADR 0005 (Cache Layer Cross-Provider Design)](./docs/adr/0005-cache-cross-provider.md). Current backing store is an in-memory Map; file-backed storage is planned for Phase 2. +- **Cache layer** — `lib/cache/` is a content-addressed cache keyed on the 11-field tuple defined in [ADR 0005 § "Cache key composition (v1.0)"](./docs/adr/0005-cache-cross-provider.md#decision) (provider, model, messages, tools, temperature, response_format, cache_control, max_tokens, top_p, stop, tool_choice — as of Amendment 2/D15). Per-key isolation, prompt-caching bypass, chunked stream replay, and singleflight. See [ADR 0005 (Cache Layer Cross-Provider Design)](./docs/adr/0005-cache-cross-provider.md). Current backing store is an in-memory Map; file-backed storage is planned for Phase 2. - **Fallback engine** — `lib/fallback/` advances a configured chain one provider at a time on configured triggers, never retrying after the first response chunk has been emitted to the client. See [ADR 0004](./docs/adr/0004-fallback-engine.md). - **Multi-key auth** — `lib/keys.mjs` (📋 planned, Phase 2) will carry OCP's per-OLP-key namespace isolation forward. Each OLP API key will have independent quota, cache namespace, and audit log; each key declares which providers it can access. diff --git a/docs/adr/0002-plugin-architecture.md b/docs/adr/0002-plugin-architecture.md index 9dbd0a2..9be9652 100644 --- a/docs/adr/0002-plugin-architecture.md +++ b/docs/adr/0002-plugin-architecture.md @@ -17,7 +17,7 @@ ### Amendment 1 — 2026-05-23: Add `maxSpawnTimeMs` to Provider contract hints (retroactive sync) -- **Finding:** Cold-audit Finding 4 (P2 governance violation) — commit `2cfd0b1` (D10) added `maxSpawnTimeMs` to all three provider plugins (`anthropic.mjs`, `codex.mjs`, `mistral.mjs`) and the fallback engine's spawn-timeout enforcement loop, but the Provider contract documentation in this ADR was not updated in the same merge. +- **Finding:** Cold-audit Finding 4 (P2 governance violation) — commit `2cfd0b1` (D10) added `maxSpawnTimeMs` to all three provider plugins (`anthropic.mjs`, `codex.mjs`, `mistral.mjs`) — enforcement lives inside each provider plugin's spawn drain loop, which throws `ProviderError(SPAWN_TIMEOUT)` that the fallback engine then treats as a hard trigger (ADR 0004 § Trigger taxonomy bullet 4) — but the Provider contract documentation in this ADR was not updated in the same merge. - **Code already landed:** The corresponding implementation is live in commit `2cfd0b1` (D10). This amendment is a retroactive contract sync to restore documentation–implementation alignment per ALIGNMENT.md Rule 1 (Cite First) and `CLAUDE.md` § "Hard requirements for plugin / server.mjs / IR changes" item 1 (Authority citation) — both of which require contract additions to be authority-cited at landing. ALIGNMENT.md Rule 2(c)'s literal wording covers IR fields; its spirit extends to Provider-contract additions. - **Procedural mechanism:** CC 开发铁律 v1.6 § 10.x (Diff Review vs Cold Audit mode). The diff-review pass that approved D10 missed this omission; the subsequent cold-audit run on 2026-05-23 caught it as Finding 4. This amendment is the required remediation. - **Authority:** ADR 0004 § Trigger taxonomy — Hard triggers bullet 4: "Provider CLI spawn timeout (configurable per-provider via `hints.maxSpawnTimeMs`)." @@ -75,7 +75,7 @@ Every provider plugin exports an object conforming to: - `requiresTTY` — boolean; whether the provider CLI requires a TTY to produce non-interactive output (e.g., some CLIs suppress JSON output unless forced with a flag or a TTY is present). - `concurrentSpawnSafe` — boolean; whether the provider CLI is safe to spawn concurrently under the same auth context without rate-limit or session collisions. - `maxConcurrent` — integer; maximum simultaneous spawn count OLP will allow for this provider. **Declarative hint only at v0.1**: the value is type-validated at startup (`lib/providers/base.mjs`) but no runtime enforcement (semaphore / in-flight counter / spawn queue) is wired in `server.mjs` yet. Tracking issue to be filed for a follow-up that lands the runtime guard. - - `maxSpawnTimeMs` — optional integer, milliseconds; maximum wall-clock time OLP allows for a single provider spawn before treating it as a hard fallback trigger. Defaults to `600000` (10 minutes) if absent. Used by the fallback engine's spawn-timeout enforcement loop; see ADR 0004 § Trigger taxonomy — Hard triggers bullet 4. + - `maxSpawnTimeMs` — optional integer, milliseconds; maximum wall-clock time OLP allows for a single provider spawn before treating it as a hard fallback trigger. Defaults to `600000` (10 minutes) if absent. Enforcement lives inside each provider plugin's spawn drain loop (`_spawnAndStream`), which uses a `setTimeout` / `proc.kill` / `reject` pattern to throw `ProviderError(SPAWN_TIMEOUT)`; the fallback engine then treats this error as a hard trigger (ADR 0004 § Trigger taxonomy — Hard triggers bullet 4). The engine itself does not run the timer loop; it only acts on the thrown error. - `cacheable` — optional boolean, default `true`; if explicitly set to `false`, the provider opts out of OLP's response cache entirely. `executeHopFn` skips `cacheStore.getOrCompute` and calls `collectAllChunks` directly; no cache read or write occurs for any request to this provider. Omitting the field is equivalent to `cacheable: true`. See ADR 0005 § "Cache write conditions" item 3 and Amendment 3 above. (D23) **Loading model.** `lib/providers/index.mjs` is a hand-maintained static enumeration. There is no filesystem scan, no `require.context`, no dynamic discovery. Adding a provider requires: diff --git a/docs/adr/0005-cache-cross-provider.md b/docs/adr/0005-cache-cross-provider.md index 1b5f0ea..2f1196f 100644 --- a/docs/adr/0005-cache-cross-provider.md +++ b/docs/adr/0005-cache-cross-provider.md @@ -23,6 +23,7 @@ - **Change:** Expand cache key composition to include `max_tokens`, `top_p`, `stop`, and `tool_choice` in addition to the existing seven fields. The four new fields are appended after the existing set so that the ordering of existing fields is stable (pre-D15 cache entries are invalidated on first request — a forced miss — but the schema is forward-compatible). Field serialization follows the existing `?? null` null-coalescing pattern: a request with `max_tokens: undefined` serializes as `null` and is treated as equivalent to an absent field; `max_tokens: 100` serializes as `100`. This is consistent with how `temperature` and `response_format` are handled. - **Rationale:** ADR 0005's own stated invariant — "different output → different cache entry" — requires that any IR field affecting output be included in the cache key. The original v1.0 key composition was correct for the seven named fields but provided incomplete coverage of ADR 0003's IR field set. The four omitted fields are all defined in ADR 0003 § Optional fields and are all sourced directly from the OpenAI `/v1/chat/completions` spec parameters that affect generation. - **Forward-looking note:** Future IR field additions (e.g., `seed`, `frequency_penalty`, `presence_penalty`, `logit_bias`, `logprobs`, `n`) must be evaluated for cache-key inclusion at the time they are added to the IR. The default is to **include** the field in the cache key unless an explicit rationale documents why omitting it is safe (e.g., the field has no effect on the provider's output for any value, or it is a purely client-side metadata field). This evaluation must appear in the amending ADR per ALIGNMENT.md Rule 2(c) spirit. +- **Note on null-coalescing collisions:** The `?? null` serialization pattern treats `field: undefined`, `field: null`, and `field: []` (for array fields such as `tools`) as equivalent cache keys. This is intentional — semantically these all mean "field absent or empty." In practice, `tools: []` (explicit empty array) and `tools` omitted (undefined) both mean "no tools available" and produce identical model output, so they correctly share a cache entry. If a future provider distinguishes empty-array from absent (e.g., `tools: []` explicitly meaning "tools disabled" vs `tools` omitted meaning "tools not specified"), the serialization for that field must be revisited and this amendment will need revision. - **Procedural mechanism:** CC 开发铁律 v1.6 § 10.x (Cold Audit caught it). This follows the calibration-loop precedent established by D11 / Amendment 1 of ADR 0002: the diff-review pass that approved the original ADR 0005 did not cross-reference all ADR 0003 optional fields; the cold-audit pass on 2026-05-23 caught the gap as Finding 7. ## Context diff --git a/models-registry.json b/models-registry.json index 3bd087a..d2ba126 100644 --- a/models-registry.json +++ b/models-registry.json @@ -109,6 +109,41 @@ "devstral-small": "devstral-small-2-25-12", "devstral-small-2": "devstral-small-2-25-12" } + }, + "grok": { + "displayName": "xAI Grok", + "tier": "C", + "candidate": true, + "models": [], + "comment": "anticipated Phase 8+; no canonical models pinned yet" + }, + "kimi": { + "displayName": "Moonshot Kimi", + "tier": "C", + "candidate": true, + "models": [], + "comment": "anticipated Phase 8+; no canonical models pinned yet" + }, + "minimax": { + "displayName": "MiniMax", + "tier": "B", + "candidate": true, + "models": [], + "comment": "anticipated Phase 8+; no canonical models pinned yet" + }, + "glm": { + "displayName": "Zhipu GLM", + "tier": "B", + "candidate": true, + "models": [], + "comment": "anticipated Phase 8+; no canonical models pinned yet" + }, + "qwen": { + "displayName": "Alibaba Qwen", + "tier": "B", + "candidate": true, + "models": [], + "comment": "anticipated Phase 8+; no canonical models pinned yet" } } } diff --git a/server.mjs b/server.mjs index 8c44ada..b03ac51 100644 --- a/server.mjs +++ b/server.mjs @@ -596,12 +596,26 @@ async function handleChatCompletions(req, res) { } } - // Generator exhausted without a stop chunk — emit [DONE] and cache. + // Generator exhausted without a stop chunk — emit [DONE] but do NOT cache. + // A stop-less exhaustion means the response is truncated (the generator + // ended without the model signalling completion). Caching a truncated + // response would serve wrong answers to future identical requests. + // Compare: D16's buffered-path truncation eviction explicitly avoids + // persisting truncated entries for the same reason. res.write(SSE_DONE); res.end(); - // D23 defense-in-depth: same guard as the stop-chunk path above. if (streamedChunks.length > 0 && cacheableForFirstHop) { - await cacheStore.set(keyId, streamCacheKey, streamedChunks); + const lastChunk = streamedChunks[streamedChunks.length - 1]; + const hasStopChunk = lastChunk?.type === 'stop'; + if (hasStopChunk) { + await cacheStore.set(keyId, streamCacheKey, streamedChunks); + } else { + logEvent('warn', 'streaming_no_stop_chunk', { + chunks_count: streamedChunks.length, + provider: streamProvider, + model: streamModel, + }); + } } } catch (e) { if (firstChunkEmitted) { diff --git a/test-features.mjs b/test-features.mjs index ac59f4d..079ed9c 100644 --- a/test-features.mjs +++ b/test-features.mjs @@ -5414,6 +5414,84 @@ describe('Streaming cache-miss real-time (Suite 15)', () => { } }); + it('15e: streaming generator exhausted without stop chunk → response delivered but NOT cached (F9)', async () => { + // Authority: ADR 0005 § "Cache write conditions" item 1 — "response completed + // successfully (no truncation, no error mid-stream)". A generator that exhausts + // without emitting a stop chunk is a truncated response and must NOT be cached. + // Compare D16's buffered-path truncation eviction. + // + // Implementation note: __setSpawnImpl cannot produce a no-stop response because + // the anthropic provider plugin synthesizes a stop chunk on clean proc exit + // (lib/providers/anthropic.mjs line ~376: `yield anthropicStopToIR('stop')`). + // Instead we inject a custom provider whose spawn() async generator yields a + // delta chunk and then RETURNS without yielding a stop chunk, bypassing the + // plugin layer entirely. Same injection pattern as 15d. + // + // Verifies: (a) client receives the chunks (200); (b) the response is NOT + // written to cache — a second identical request triggers a fresh spawn (cache + // miss), proving no caching happened after request 1. + + const { loadedProviders: lp15e, __clearCache: cc15e } = await import('./server.mjs'); + const savedAnthropicProvider = lp15e.get('anthropic'); + + let spawnCallCount = 0; + const noStopProvider = { + ...savedAnthropicProvider, + spawn: async function* () { + spawnCallCount++; + // Yield one delta chunk but NO stop chunk — generator exhausts here. + yield { type: 'delta', content: 'no-stop-content', finish_reason: null }; + // (implicit return — no stop chunk emitted) + }, + }; + lp15e.set('anthropic', noStopProvider); + cc15e(); + + const makeRequest = () => new Promise((resolve, reject) => { + const req = httpRequest({ + hostname: '127.0.0.1', + port: port15, + method: 'POST', + path: '/v1/chat/completions', + headers: { 'Content-Type': 'application/json' }, + }, res => { + let data = ''; + res.on('data', d => { data += d.toString(); }); + res.on('end', () => resolve({ status: res.statusCode, body: data, headers: res.headers })); + res.on('error', reject); + }); + req.on('error', reject); + req.write(JSON.stringify({ + model: 'claude-sonnet-4-6', + messages: [{ role: 'user', content: 'no-stop-chunk-f9-test' }], + stream: true, + })); + req.end(); + }); + + try { + // First request: generator exhausts without stop → client receives response (200) + const r1 = await makeRequest(); + assert.equal(r1.status, 200, `First request failed: ${r1.status} ${r1.body.slice(0, 200)}`); + assert.equal(spawnCallCount, 1, 'Spawn called once for first request'); + assert.equal(r1.headers['x-olp-cache'], 'miss', 'First request must be cache miss'); + + // Second identical request: must ALSO be cache miss + fresh spawn (proves no caching after r1) + const r2 = await makeRequest(); + assert.equal(r2.status, 200, `Second request failed: ${r2.status} ${r2.body.slice(0, 200)}`); + assert.equal(spawnCallCount, 2, + `Expected spawn called again for r2 (no-stop response must not be cached), got spawnCallCount=${spawnCallCount}`); + assert.equal(r2.headers['x-olp-cache'], 'miss', 'Second request must also be cache miss (no caching of truncated response)'); + } finally { + // Restore the original anthropic provider + if (savedAnthropicProvider !== undefined) { + lp15e.set('anthropic', savedAnthropicProvider); + } else { + lp15e.delete('anthropic'); + } + } + }); + }); // ── Suite 17: D18 — /v1/models population + X-OLP-* headers on errors ────────