diff --git a/AGENTS.md b/AGENTS.md index f879334..cbdd88b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -49,7 +49,7 @@ Runtime: Node.js (ESM, `.mjs` throughout). No build step. No bundler. `server.mj - `.github/workflows/alignment.yml` β€” CI blacklist grep + per-provider citation soft check; fails the build on known-hallucinated tokens. - `CLAUDE.md` β€” Claude-Code-specific session instructions + `release_kit` overlay (Iron Rule 5.5). -**Implementation status note (as of 2026-05-25):** Files marked πŸ“‹ above are designed and documented but not yet on disk; files marked 🟑 are partially shipped; files marked βœ… are Phase 2 deliverables. The shipped set as of D47 is: `server.mjs` (with Phase 2 auth middleware + audit wire + owner-vs-non-owner gating), `lib/ir/`, `lib/providers/{anthropic,codex,mistral}.mjs`, `lib/cache/{keys,store}.mjs`, `lib/fallback/engine.mjs`, `lib/keys.mjs` (core + loadAuthConfigSync β€” D44 + D45), `lib/audit.mjs` (D45), `bin/olp-keys.mjs` (D47), `models-registry.json`, `test-features.mjs` (Suites 19–22). Phase 2 functional scope is complete; remaining is Phase 2 close β†’ v0.2.0 (maintainer-triggered, explicit per CLAUDE.md `release_kit.phase_close_trigger`). +**Implementation status note (as of 2026-05-27):** Phase 5 (Quota Probes + Dashboard Enrichment) is closed at v0.5.0 + v0.5.1 hotfix. The shipped set includes all Phases 1–5 deliverables. v0.5.1 hotfix (2026-05-27) fixes three codex review findings: F1 (doctor check bypassed backoff by calling `_probeOnce` directly β€” now routes through `quotaStatus()`), F2 (200 with empty `anthropic-ratelimit-*` headers was cached as live β€” minimum-viable-schema gate added), F3 (null collapsed all failure modes β€” `probe_status:'unreachable'` shape + `failure`/`failure_kind`/`backoff_until` fields added). See ADR 0008 Amendment 2 + ADR 0013 Rule 3/5 clarifications. Phase 6 is next (per CLAUDE.md `release_kit.current_phase`). --- diff --git a/CHANGELOG.md b/CHANGELOG.md index 13d8d53..5f548e2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,46 @@ All notable changes to OLP land here. Per `CLAUDE.md` release_kit overlay, this (empty β€” Phase 6 entries land here once Phase 6 opens) +## v0.5.1 β€” 2026-05-27 + +**Hotfix β€” Quota probe cache/backoff/schema-drift correctness (codex review findings F1–F3).** Three production-quality bugs in the v0.5.0 quota probe, reproduced by codex with local mocks, are corrected. 756 β†’ 759 tests (3 new regression tests); 4 existing test assertions updated to reflect the v0.5.1 return-shape contract. + +### Fixes + +- **F1 [P1] β€” Doctor bypass of cache + backoff (ADR 0013 Rule 3).** `anthropic.quota_probe_reachable` doctor check called `_probeOnce(auth)` directly, bypassing the module-level `quotaProbeState.backoffUntil` check. Successive `olp doctor` invocations within a backoff window each hit upstream β€” violating ADR 0013 Rule 3 (60s-3600s exponential backoff is mandatory for all consumers). **Fix:** doctor check now routes through `quotaStatus()`, which enforces cache + backoff. ADR 0013 Rule 3 clarification added: "All consumers of `quotaStatus()`, including `olp doctor` checks, MUST route through `quotaStatus()` and MUST NOT call `_probeOnce()` directly." + +- **F2 [P2] β€” 200 with empty `anthropic-ratelimit-*` headers cached as live data (ADR 0013 Rule 5).** `_probeOnce` treated any 200 OK (regardless of header content) as a successful probe, caching it with `stale: false` even when zero `anthropic-ratelimit-*` headers were present. A proxy stripping headers, a schema change, or a mock returning `{}` would silently appear as "LIVE" on the dashboard with all bars empty. **Fix:** minimum-viable-schema gate requires these 4 fields non-null: `5h-utilization`, `5h-reset`, `7d-utilization`, `7d-reset`. Any absence β†’ `failureKind: 'schema_drift'`, backoff scheduled, result not cached. ADR 0013 Rule 5 updated with the gate specification. + +- **F3 [P2] β€” Dashboard-data loses failure detail (ADR 0013 Rule 6).** `aggregateProviderQuota()` collapsed all non-null failure modes (no credentials, auth failure, rate limit, schema drift, network error) into `status: 'unavailable', reason: 'no public quota api or probe disabled'` β€” the same string as providers with no quota API at all. Operator could not tell what to fix. **Fix:** `quotaStatus()` v0.5.1 return contract: `null` reserved for opt-in-off only; probe failures return `{ probe_status: 'unreachable', failure: { kind, message, backoff_until? } }`. `aggregateProviderQuota()` emits new fields `failure_kind`, `failure`, `backoff_until` per row. `status: 'unreachable'` distinguishes "probe failed" from `status: 'unavailable'` ("no API or disabled"). Dashboard renders `unreachable` with a red border + failure.message + backoff countdown. + +### Backwards-compat notes + +- `quotaStatus()`: `stale: false` β†’ now also includes `probe_status: 'live'` (additive). `stale: true` β†’ now also includes `probe_status: 'stale'` + `failure: {...}` (additive). `null` β†’ NOW RESERVED FOR OPT-IN-OFF ONLY (breaking for callers that relied on `null` to detect "no credentials" or "probe failed" β€” use `probe_status: 'unreachable'` instead). +- `ProviderQuotaEntry.status`: gains `'unreachable'` as a new value (additive). Existing `'live'`, `'stale'`, `'unavailable'` semantics unchanged. +- `ProviderQuotaEntry` gains new fields `failure`, `failure_kind`, `backoff_until` (additive, null when not applicable). +- `dashboard.html`: handles `unreachable` row (no existing row had this status; additive render path). + +### Test changes + +- 38f, 38j, 38l: updated assertions from `null` to `probe_status: 'unreachable'` (F3 shape change). +- 38r: refactored to seed cache + manually expire it + set backoff (F1 β€” doctor now routes through `quotaStatus()`). Added F1-regression assertion: HTTP call counter stays at 1 after two doctor calls within backoff. +- 38g, 38k: added `probe_status` + `failure` assertions (verify new fields present on live/stale shapes). +- **38u** (new): F1 regression β€” successive doctor calls within backoff window β†’ HTTP counter stays at 1. +- **38v** (new): F2 regression β€” 200 + empty ratelimit headers β†’ `probe_status: 'unreachable'` + `failure_kind: 'schema_drift'` + cache stays null. +- **38w** (new): F3 regression β€” `lastError` + `failureKind` propagate through `quotaStatus()` shape for all failure modes (rate_limited / auth_failed / schema_drift / no_credentials). + +### ADR changes + +- **ADR 0013 Rule 3** clarification: doctor checks route through `quotaStatus()`, not `_probeOnce()` directly. +- **ADR 0013 Rule 5** update: minimum-viable-schema gate specification (4 required fields; absence = schema_drift signal). +- **ADR 0008 Amendment 2**: richer `ProviderQuotaEntry` shape with `failure`/`failure_kind`/`backoff_until`; `probe_status` on `quotaStatus()` return; `unreachable` status semantics; `dashboard.html` unreachable rendering. + +### Authority + +ADR 0013 Rules 3, 5, 6 (cache + backoff + schema-drift + failure transparency); ADR 0008 Amendment 2; ADR 0002 Amendment 8 (unchanged); codex review findings F1–F3 (codex PR review on v0.5.0 close PR #57). + +--- + ## v0.5.0 β€” 2026-05-26 **Phase 5 β€” Provider Quota Probes + Dashboard Enrichment.** OLP gains live subscription-quota observability for Anthropic Pro/Max subscribers, surfaced through a Claude.ai-style Plan Usage panel on the owner-only dashboard. The probe is opt-in, READ-ONLY, idempotent on failure, and 5-min-cached with 60sβ†’3600s exponential backoff. Six D-days, seven PRs, zero blocking reviewer findings, no flaky tests; 720 β†’ 756 total tests. diff --git a/README.md b/README.md index c8e93d7..8ecd869 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ A personal- and family-scale multi-provider LLM proxy. One HTTP endpoint, many subscriptions behind it, automatic routing + fallback + content-addressed caching. Your IDEs and family clients keep working as long as **any** of your subscriptions has quota left. -> **Status:** v0.4.3 shipped, 714+ tests. Phase 4 (Operator + Client UX) closed; Phase 5 scope is open. Coming from [OCP](https://github.com/dtzp555-max/ocp)? See [Β§ Migration from OCP](#migration-from-ocp). +> **Status:** v0.5.1 shipped, 759+ tests. Phase 5 (Quota Probes + Dashboard Enrichment) closed; Phase 6 next. Coming from [OCP](https://github.com/dtzp555-max/ocp)? See [Β§ Migration from OCP](#migration-from-ocp). --- @@ -484,7 +484,7 @@ Use a dedicated bot key β€” not the maintainer's personal owner key β€” so revoc ## Implementation status (as of 2026-05-26, post-v0.4.0) -Phase 1 closed at v0.1.1 (multi-provider proxy core + pre-Phase-2 cleanup). Phase 2 closed at v0.2.0 (multi-key auth + audit + owner gating + keygen CLI; ADR 0007 Β§ 10 all 11 acceptance criteria shipped). Phase 3 closed at v0.3.0 (Dashboard + `lib/audit-query.mjs` + daily audit rotation; ADR 0008 Β§ 10 all 15 acceptance criteria shipped). Phase 4 closed at v0.4.0 (Operator + Client UX per ADR 0010: SSE heartbeat + `recentErrors[20]` + `/v0/management/status` / `olp` Node CLI + `olp doctor` framework + ADR 0002 Amendment 7 / `olp-connect` bash + `/health.anonymousKey` + ADR 0011 / `olp-plugin/` Telegram-Discord + 6-IDE integration docs). Phase 5 scope is open β€” candidates per ADR 0010 Β§ Out-of-Phase-4-scope. This table reflects what is currently shipped vs. what is designed for later phases. +Phase 1 closed at v0.1.1 (multi-provider proxy core + pre-Phase-2 cleanup). Phase 2 closed at v0.2.0 (multi-key auth + audit + owner gating + keygen CLI; ADR 0007 Β§ 10 all 11 acceptance criteria shipped). Phase 3 closed at v0.3.0 (Dashboard + `lib/audit-query.mjs` + daily audit rotation; ADR 0008 Β§ 10 all 15 acceptance criteria shipped). Phase 4 closed at v0.4.0 (Operator + Client UX per ADR 0010: SSE heartbeat + `recentErrors[20]` + `/v0/management/status` / `olp` Node CLI + `olp doctor` framework + ADR 0002 Amendment 7 / `olp-connect` bash + `/health.anonymousKey` + ADR 0011 / `olp-plugin/` Telegram-Discord + 6-IDE integration docs). Phase 5 closed at v0.5.0 (Quota Probes + Dashboard Enrichment β€” live Anthropic plan-usage probe + Claude.ai-style dashboard + audit-query aggregateProviderQuota); v0.5.1 hotfix (quota probe cache/backoff/schema-drift correctness β€” codex review findings F1–F3). Phase 6 is next. This table reflects what is currently shipped vs. what is designed for later phases. | File / artifact | Status | Notes | |---|---|---| @@ -573,7 +573,8 @@ The original v0.1 spec (in `~/.cc-rules/memory/projects/olp_v0_1_spec.md` on the - **Phase 1** β€” Multi-provider proxy core: `server.mjs`, IR, three Tier-D provider plugins (Anthropic / OpenAI Codex / Mistral Vibe), cache (D1+D4) + cleanup (D2 bypass / D3 chunked replay / D23 size cap), fallback engine with first-chunk safety + hard triggers + per-hop log observability, IR↔OpenAI translation under Rule 2(b). βœ… Shipped β€” v0.1.0 (2026-05-24) + v0.1.1 cleanup (2026-05-25, D35–D42). - **Phase 2** β€” Multi-key auth (`lib/keys.mjs`) per ADR 0007: opaque OLP API keys, per-key cache namespacing, owner-vs-guest tier for header gating, audit ndjson (`lib/audit.mjs`), `/health` payload trimming + `X-OLP-Fallback-Detail` emission gating, `OLP_OWNER_TOKEN` env override, keygen CLI (`bin/olp-keys.mjs`). βœ… Shipped β€” v0.2.0 (2026-05-25, D43-A β†’ D47). All 11 ADR 0007 Β§ 10 acceptance criteria covered. - **Phase 3** β€” Dashboard + audit query layer + daily audit rotation per ADR 0008: in-memory ndjson aggregate query layer (`lib/audit-query.mjs`), 4 owner-only_block management endpoints (`/dashboard` + `/v0/management/dashboard-data` + `/v0/management/quota` + `/cache/stats`), multi-panel `dashboard.html` with 30s poll, synchronous daily audit rotation + `bin/olp-audit-rotate.mjs` cron tool, `tried_providers` schema fix (D45 P2 deferral). βœ… Shipped β€” v0.3.0 (2026-05-25, D48 β†’ D54). All 15 ADR 0008 Β§ 10 acceptance criteria covered. -- **Phase 4 (planned)** β€” Per-key per-provider auth artifact mapping (ADR 0007 Β§ 12 deferral), audit query rotation/retention policies, SQLite hybrid migration (ADR 0007 Β§ 13 trigger), provider-cost weights for spend trend. +- **Phase 5** β€” Live quota probe (Anthropic Pro/Max OAuth plan-usage via `anthropic-ratelimit-unified-*` headers), Claude.ai-style dashboard enrichment (utilization bars + reset countdowns), audit-query `aggregateProviderQuota()`, per-provider quota_v2 shape in dashboard-data. βœ… Shipped β€” v0.5.0 (2026-05-27). v0.5.1 hotfix (2026-05-27): quota probe cache/backoff/schema-drift correctness (codex review findings F1–F3). +- **Phase 6 (planned)** β€” Per-key per-provider auth artifact mapping (ADR 0007 Β§ 12 deferral), audit query rotation/retention policies, SQLite hybrid migration (ADR 0007 Β§ 13 trigger), provider-cost weights for spend trend. - **Phase 4+ (v1.x roadmap, triggered as needed)** β€” Full deferred-work tracker: [`docs/v1x-roadmap.md`](./docs/v1x-roadmap.md). Includes streaming-path singleflight ([issue #16](https://github.com/dtzp555-max/olp/issues/16) + ADR 0005 Amendment 8 design ratified), soft-trigger reactivation (ADR 0004 Amendment 2), `/health` activeSpawns integration, provider-level `cacheKeyFields` mask, streaming-path SPAWN_FAILED salvage. - **Phase N (opt-in)** β€” Tier-2 / Tier-C provider plugins (Grok / Kimi / MiniMax / GLM / Qwen) per [ADR 0006](./docs/adr/0006-provider-inclusion.md); provider-native protocol endpoints; deterministic triggers. Triggered by tier-2 demand, not on the bootstrap path. diff --git a/dashboard.html b/dashboard.html index e91d3c0..6e739c0 100644 --- a/dashboard.html +++ b/dashboard.html @@ -100,6 +100,7 @@ .provider-row:last-child { margin-bottom: 0; } .provider-row.unavailable { background: #f9fafb; } .provider-row.stale { border-color: #fcd34d; } + .provider-row.unreachable { border-color: #fca5a5; background: #fff5f5; } .provider-row-top { display: flex; @@ -134,6 +135,7 @@ .status-dot.live { background: #10b981; } .status-dot.stale { background: #f59e0b; } .status-dot.unavailable { background: #9ca3af; } + .status-dot.unreachable { background: #ef4444; } .status-chip { display: inline-block; @@ -147,6 +149,7 @@ .status-chip.live { background: #d1fae5; color: #065f46; } .status-chip.stale { background: #fef3c7; color: #92400e; } .status-chip.unavailable { background: #f3f4f6; color: #6b7280; } + .status-chip.unreachable { background: #fee2e2; color: #991b1b; } .chip-sm { display: inline-block; @@ -168,6 +171,12 @@ font-style: italic; padding: 0.25rem 0 0; } + .unreachable-reason { + font-size: 0.875rem; + color: #b91c1c; + font-style: italic; + padding: 0.25rem 0 0; + } .last-fresh-tag { font-size: 0.7rem; color: #9ca3af; @@ -294,7 +303,7 @@
Loading…
- +