From c0b696984f9facaed42377a57dc73b30eca016b3 Mon Sep 17 00:00:00 2001 From: dtzp555-max Date: Mon, 25 May 2026 15:41:26 +1000 Subject: [PATCH] =?UTF-8?q?docs:=20D48=20=E2=80=94=20ADR=200008=20Phase=20?= =?UTF-8?q?3=20design=20draft=20(Dashboard=20+=20audit=20query=20layer)=20?= =?UTF-8?q?(#25)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs: D48 — ADR 0008 Phase 3 design draft (Dashboard + audit query layer, design-only) First Phase 3 D-day. Design-only. Ratifies the storage / query model / rotation / dashboard / refresh / scope decisions ahead of D49+ implementation D-days. Opens ADR 0007 § 12 deferral for Dashboard + audit query layer + rotation. NEW docs/adr/0008-dashboard-and-audit-query.md (~368 lines): 13 sections + Consequences + Authority citations. Per maintainer-pinned lanes A/A/B/A/B from Phase 3 kickoff brief 2026-05-25: Lane 1 (tech stack): A — static HTML + vanilla JS + fetch; no build step; matches OLP "no bundler" ethos. Lane 2 (query model): A — in-memory scan of audit ndjson per request; O(N) per call; family-scale acceptable; defers SQLite hybrid to Option 3 trigger per ADR 0007 § 13 (requires engines bump as separate prior PR). Lane 3 (rotation): B — daily rotation, audit-YYYY-MM-DD.ndjson on first append after UTC midnight + optional bin/olp-audit-rotate.mjs external cron. Lane 4 (refresh): A — 30s page poll (no SSE infra at v0.3.0; pause when document.visibilityState is hidden). Lane 5 (dashboard scope): B — full per spec § 4.6: 4 panels (per- provider quota / 24h request+cache+fallback / 30d spend trend / top-N fallback chains). ADR sections: §1 Context (what Phase 3 closes; what stays out) §2 Decision (5 lanes pinned) §3 Storage layout (~/.olp/logs/ with rotated date files) §4 lib/audit-query.mjs API (readAuditWindow, aggregateRequests, topFallbackChains, spendTrendDaily, cacheHitRateWindow) §5 Audit rotation (first-append-after-UTC-midnight trigger + cron alternative + concurrent-safety per-process lock) §6 Dashboard panels (4 panels per spec § 4.6, refresh + localhost- bound notes) §7 Server endpoints (/dashboard, /v0/management/dashboard-data, /v0/ management/quota, /cache/stats — owner-only gated) §8 Auth gating (reuses ADR 0007 § 7 owner_only_endpoints config) §9 Failure modes + degradation (per-panel error states) §10 Acceptance criteria (15 testable items for D49-D54) §11 Forward path (Phase 4+ — SQLite migration, SSE push, key-mgmt UI) §12 Out of scope (explicitly NOT in Phase 3) §13 Phase 3 sprint shape (D48-D55) CHANGED: - docs/adr/README.md: added ADR 0008 row with one-paragraph summary. - CHANGELOG.md Unreleased: D48 entry per release_kit overlay phase_rolling_mode discipline. Includes Phase 3 sprint shape table. NOT IN D48 scope: - lib/audit-query.mjs (D49) - server.mjs endpoints (D50) - dashboard.html (D51) - lib/audit.mjs rotation extension + bin/olp-audit-rotate.mjs (D52) - tried_providers schema fix (D53; D45 P2 deferral) - Phase 3 close (D55; v0.3.0; maintainer-triggered) Test count: 544 / 544 pass (design-only, no test change). Verified locally via npm test before commit. AUTHORITY: - ADR 0007 § 12 (opens Phase 3 Dashboard + audit query deferral). - ADR 0007 § 13 (rejects SQLite at Phase 3 per Node baseline + documents forward-path trigger). - OLP v0.1 spec § 4.6 + § 4.7 (Dashboard + observability endpoints planning authority). - OCP dashboard.html (prior-art reference for multi-panel HTML structure). - ADR 0002 (Provider.quotaStatus contract — Panel 1 data source). - ADR 0005 § Cache stats (/cache/stats endpoint pre-existing design). - CC 开发铁律 v1.6 § 10 — fresh-context opus reviewer required for design ADR. - Phase 3 kickoff via maintainer "go" 2026-05-25 + standing-autopilot grant (~/.cc-rules/memory/auto/standing_autopilot_phase_2.md in cc-rules bf0ed9a — Phase 3+ requires new authorization; the "go" supplied that). ALIGNMENT.md scope check: this PR introduces a new ADR with full authority citations per Rule 1 (Cite First). No provider plugin / entry surface / IR change in this commit. Co-Authored-By: Claude Opus 4.7 * docs: D48 fold-in — opus reviewer findings (1 P2 + 2 P3, all ADR-text polish) Fresh-context opus reviewer (PR #25) returned APPROVE_WITH_MINOR with 3 findings, all ADR-text polish. No design semantic change beyond the clarifications. P2 — § 8 + § 10 #9 gating-mode wording Original § 8 implied a new "block non-owner identities" behaviour without naming it; § 10 #9 tested only the universal allow_anonymous: false 401 case. Fix: - § 8 now formalizes two gating modes — owner_only_trim (Phase 2 /health pattern) vs owner_only_block (new Phase 3 management- endpoints pattern) — and explains the management endpoints are owner_only_block because the entire payload is sensitive. - § 10 #9 now covers both 401 paths: (a) allow_anonymous: true + no header → anonymous identity → STILL 401 because management endpoints are owner_only_block; (b) allow_anonymous: false + no header → 401 at the authenticate middleware itself. P3 #1 — /cache/stats citation accuracy Original § 7.4 + Authority block cited "ADR 0005 § Cache stats" which is not a real section. Corrected: planning authority is OLP v0.1 spec § 4.6; ADR 0005 references the endpoint in Consequences/Mitigations (~line 279) for the per-(provider, model) cache-hit-rate breakdown surface. P3 #2 — cacheStore.stats() shape gap § 7.4 now explicitly acknowledges the current shape ({ hits, misses, size, inflightCount } global aggregate) lacks the per-(provider, model) breakdown spec § 4.6 implies; Phase 3 Panel 2 sources per-provider counts from aggregateRequests (audit-side) instead. If a future panel needs the breakdown, D50 amends the store shape + an ADR 0005 amendment fires at that time. Phase 3 acceptance criteria do not require the breakdown. CHANGELOG.md Unreleased D48 entry: fold-in bullet added enumerating the 3 fixes. Test count: 544 / 544 pass (design-only, no test change). Verified locally via npm test. Authority: PR #25 fresh-context opus reviewer findings; CLAUDE.md release_kit overlay phase_rolling_mode — under Unreleased. Co-Authored-By: Claude Opus 4.7 --------- Co-authored-by: dtzp555 Co-authored-by: Claude Opus 4.7 --- CHANGELOG.md | 20 +- docs/adr/0008-dashboard-and-audit-query.md | 372 +++++++++++++++++++++ docs/adr/README.md | 1 + 3 files changed, 392 insertions(+), 1 deletion(-) create mode 100644 docs/adr/0008-dashboard-and-audit-query.md diff --git a/CHANGELOG.md b/CHANGELOG.md index dc20666..a64833b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,25 @@ All notable changes to OLP land here. Per `CLAUDE.md` release_kit overlay, this ## Unreleased -(empty — Phase 3 entries land here once Phase 3 opens) +### D48 — ADR 0008 Phase 3 design draft (Dashboard + audit query layer) + +First Phase 3 D-day. Design-only. Ratifies the storage / query model / rotation / dashboard / refresh / scope decisions ahead of D49+ implementation D-days. Opens ADR 0007 § 12 deferral for Dashboard + audit query layer + rotation. + +- **New file `docs/adr/0008-dashboard-and-audit-query.md`** (~390 lines): 13 sections + Consequences + Authority citations. Decisions per maintainer-pinned lanes: + - Lane 1 (tech stack): static HTML + vanilla JS + fetch (no build step; matches OLP "no bundler" ethos) + - Lane 2 (query model): in-memory scan of audit ndjson per request (defers SQLite hybrid per ADR 0007 § 13) + - Lane 3 (rotation): daily rotation, `audit-YYYY-MM-DD.ndjson` on first append after UTC midnight + optional `bin/olp-audit-rotate.mjs` external cron + - Lane 4 (refresh): 30s page poll (no SSE infra at v0.3.0) + - Lane 5 (dashboard scope): full per spec § 4.6 — 4 panels (quota / per-provider 24h counts / 30d spend trend / top fallback chains) +- **`docs/adr/README.md` index**: added ADR 0008 row with one-paragraph summary. +- **CHANGELOG.md** Unreleased: this entry. +- **Phase 3 sprint shape:** D49 `lib/audit-query.mjs` + Suite 23 → D50 `/v0/management/*` endpoints + Suite 24 → D51 `dashboard.html` → D52 daily audit rotation + Suite 25 → D53 `tried_providers` schema fix (D45 P2 deferral) → D54 E2E + docs → D55 Phase 3 close → v0.3.0 (maintainer-triggered). +- **Fold-in (fresh-context opus reviewer findings — 1 P2 + 2 P3, all ADR-text polish):** + - **P2 § 8 + § 10 #9 gating-mode wording** — original § 8 implied a new "block non-owner identities" behaviour without naming it; § 10 #9 tested only the universal `allow_anonymous: false` 401 case. Fix: § 8 now formalizes two gating modes — `owner_only_trim` (Phase 2 /health pattern) vs `owner_only_block` (new Phase 3 management-endpoints pattern) — and explains the management endpoints are `owner_only_block` because the entire payload is sensitive. § 10 #9 now covers both 401 paths (with `allow_anonymous: true` + no header → anonymous identity → still 401 because management endpoints are `owner_only_block`; AND with `allow_anonymous: false` + no header → 401 at the authenticate middleware itself). + - **P3 `/cache/stats` citation accuracy** — original § 7.4 + Authority block cited "ADR 0005 § Cache stats" which is not a real section. Corrected: planning authority is OLP v0.1 spec § 4.6; ADR 0005 references the endpoint in `Consequences/Mitigations` (~line 279) for the per-`(provider, model)` cache-hit-rate breakdown surface. + - **P3 `cacheStore.stats()` shape gap** — § 7.4 now explicitly acknowledges the current shape (`{ hits, misses, size, inflightCount }` global aggregate) lacks the per-`(provider, model)` breakdown spec § 4.6 implies; Phase 3 Panel 2 sources per-provider counts from `aggregateRequests` (audit-side) instead. If a future panel needs the breakdown, D50 amends the store shape + an ADR 0005 amendment fires at that time. Phase 3 acceptance criteria do not require the breakdown. +- **Test count:** 544 → 544 (design-only, no test change). +- **Authority:** ADR 0007 § 12 (opens deferral) + § 13 (rejects SQLite at Phase 3 per Node baseline); v0.1 spec § 4.6 / § 4.7 (Dashboard + observability endpoints planning authority); OCP `dashboard.html` (prior art); CC 开发铁律 v1.6 § 10 — fresh-context opus reviewer required for design ADR; Phase 3 kickoff via maintainer "go" 2026-05-25 + standing-autopilot grant; PR #25 fresh-context opus reviewer findings (3 polish items). ## v0.2.0 — 2026-05-25 diff --git a/docs/adr/0008-dashboard-and-audit-query.md b/docs/adr/0008-dashboard-and-audit-query.md new file mode 100644 index 0000000..f67f8a1 --- /dev/null +++ b/docs/adr/0008-dashboard-and-audit-query.md @@ -0,0 +1,372 @@ +# ADR 0008 — Dashboard + Audit Query Layer (Phase 3) + +- **Date:** 2026-05-25 +- **Status:** Accepted (D48, design-only — implementation D-days D49–D54 follow; Phase 3 close = v0.3.0) +- **Authors:** project maintainer (with AI drafting assistance) +- **Related:** + - OLP v0.1 spec § 4.6 (Dashboard requirements — port from OCP with multi-provider support) and § 4.7 (observability endpoints) + - ADR 0007 § 12 (Phase 3+ out-of-scope: Dashboard, audit query layer, rotation) — this ADR opens those deferrals + - ADR 0007 § 13 (Option 3 hybrid migration to SQLite) — explicitly **NOT** triggered by Phase 3; in-memory ndjson scan is the v0.3.0 query model + - ADR 0007 § 7 (Identity classes) — Dashboard auth gating reuses owner-vs-non-owner pattern (Dashboard is owner-only) + - ADR 0007 § 8 (Audit ndjson schema) — the data source the query layer reads + - ADR 0004 Amendment 2 (soft triggers deferred to v1.x) — quota panel sources from `provider.quotaStatus()` which is a contract method; per-provider returns what it can or `null` + - D45 reviewer P2 deferral on `tried_providers` semantics — addressed in this Phase as D53 (separate D-day; not in ADR 0008 scope) +- **Phase 3 kickoff authority:** maintainer "go" + standing-autopilot grant (`~/.cc-rules/memory/auto/standing_autopilot_phase_2.md` in cc-rules `bf0ed9a` — the grant explicitly excludes Phase 3+ as needing new authorization; the "go" supplied that) +- **Lanes pinned by maintainer 2026-05-25 (Phase 3 kickoff brief):** + - Lane 1 Dashboard tech stack: **A** — static HTML + vanilla JS + fetch (no build step; matches OLP "no bundler" ethos) + - Lane 2 Audit query model: **A** — in-memory scan of audit ndjson per request (O(N) per query; family-scale acceptable; SQLite deferred to Option 3 trigger per ADR 0007 § 13) + - Lane 3 Audit rotation: **B** — daily rotation, files named `audit-YYYY-MM-DD.ndjson` (UTC date) + - Lane 4 Refresh: **A** — page poll every 30s (no SSE infra introduction) + - Lane 5 Dashboard scope: **B** — full per spec § 4.6 (quota + per-provider counts/cache/fallback last 24h + multi-provider spend trend last 30d + top fallback chains by trigger count) + +--- + +## 1. Context + +OLP v0.2.0 ships per-key audit ndjson at `~/.olp/logs/audit.ndjson` (ADR 0007 § 8) but provides no aggregate query surface or visualization. The audit file grows unbounded, and operators must `tail`/`grep` to observe basic facts ("which provider served the most requests today", "what's my cache hit rate", "how often did the chain fall back to OpenAI"). Phase 3 closes this gap with: + +1. **`lib/audit-query.mjs`** — an in-memory aggregator that scans `~/.olp/logs/audit-*.ndjson` files in a configurable time window and returns shaped summaries. +2. **`server.mjs` `/v0/management/*` endpoints** — three owner-only JSON endpoints exposing the aggregate data: `/v0/management/dashboard-data`, `/v0/management/quota`, `/cache/stats`. +3. **`dashboard.html`** — a single static HTML file served from `/dashboard` (owner-only) that fetches the JSON endpoints, renders 4 panels, and polls every 30s. +4. **Daily audit rotation** — at UTC midnight (or on first append after a UTC-date change), the live `audit.ndjson` is renamed to `audit-YYYY-MM-DD.ndjson` and a fresh `audit.ndjson` opens. Cross-file queries handle the rolling 30-day window. + +Phase 3 deliberately **does not** add a build step, a database, or per-key UI write surface. The first three are out of scope (Option 3 hybrid is § 13's forward path; SQLite has a Node-baseline blocker per § 11). The last is a security surface that warrants a separate review pass (Phase 4+). + +Phase 3 is the natural home for OCP's `dashboard.html` port (v0.1 spec § 4.6 — "Port OCP's `dashboard.html` with multi-provider support"). OCP's dashboard was single-provider; OLP's is multi-provider, which is the substantive change. The HTML structure is otherwise lifted. + +--- + +## 2. Decision + +The five lanes above are normative. The deviation lattice they sit in: + +| Lane | Pinned | Rejected (why) | +|---|---|---| +| **1. Tech stack** | Static HTML + vanilla JS + fetch | SSR (Node template literals) — adds server-side render path; CSP harder. SPA — adds build step, violates ethos. | +| **2. Query model** | In-memory scan of `audit-*.ndjson` per request | SQLite indexed mirror — touches ADR 0007 § 13 migration → engines bump prerequisite. In-memory rolling aggregate — complex (per-write incremental + window expiry), correctness risk. | +| **3. Rotation** | Daily UTC rotation (`audit-YYYY-MM-DD.ndjson`) | No rotation — single file grows unbounded at family scale ~10–100 lines/day but a year is ~10–50k lines, still ok but no natural query unit. Size-based (100MB / keep 5) — equivalent complexity, query range less intuitive. | +| **4. Refresh** | Page poll every 30s | SSE push — adds server-side subscriber state; not justified at family scale. Static-at-load — UX too poor. | +| **5. Dashboard scope** | Full per spec § 4.6 | Minimal (quota + cache + fallback only) — spec is already drafted; full is ~1 D-day more for trend + top-chains panels. Plus key-mgmt UI — security surface delta, Phase 4+. | + +**Phase 3 does not** introduce: a database, a build step, an SPA framework, SSE for dashboard, or web-side key management. Each is a future-phase concern (Option 3 hybrid; SSE for dashboard if poll latency becomes pain; key-mgmt UI after a security review pass). + +--- + +## 3. Storage layout (`~/.olp/logs/`) + +``` +~/.olp/logs/ + audit.ndjson — live append target (Phase 2 D45) + audit-2026-05-24.ndjson — yesterday (after rotation) + audit-2026-05-23.ndjson + audit-2026-05-22.ndjson + ... +``` + +Files are append-only (no in-place edits). Rotation atomically renames the live file and opens a fresh one. All files have mode 0600; the `logs/` directory is 0700. + +**Retention policy at v0.3.0:** unbounded by default (operator manages disk). A Phase 3+ amendment may add automatic retention (`olp_audit_max_days` config) once an observed operational need exists. + +--- + +## 4. Audit query layer (`lib/audit-query.mjs`) + +A new module that reads the rotated + live audit files in a date range and returns aggregate summaries. Per-call O(N) where N = total lines in the date range (family scale: thousands per day = trivial). + +### 4.1 Public API + +```js +// Read all audit lines in [startMs, endMs); returns iterator of parsed events. +// Skips malformed lines (logs warn) so a corrupted day doesn't kill the query. +export function* readAuditWindow({ startMs, endMs, olpHome }): Iterator; + +// Aggregate request shape over a window. Returns: +// { +// window: { startMs, endMs }, +// request_count, status_2xx, status_4xx, status_5xx, +// by_provider: { [providerKey]: { count, cache_hit, cache_miss, cache_bypass, fallback_count } }, +// by_owner_tier: { owner: N, guest: N, anonymous: N }, +// by_path: { '/v1/chat/completions': N, '/v1/models': N }, +// median_latency_ms, p95_latency_ms, +// } +export function aggregateRequests({ windowMs, olpHome }): RequestAggregate; + +// Top-N fallback chains by trigger count in window. Returns sorted array: +// [{ chain: ['anthropic', 'openai'], count: 42, first_seen, last_seen }, ...] +export function topFallbackChains({ windowMs, limit, olpHome }): FallbackChainSummary[]; + +// Daily series of request_count + latency_median over N days. Returns sorted array: +// [{ date: '2026-05-22', request_count, median_latency_ms, by_provider }, ...] +export function spendTrendDaily({ days, olpHome }): DailySpendEntry[]; + +// Cache hit rate snapshot (in-memory cacheStore stats + audit-derived numerator). +// Differs from /cache/stats: that returns the live in-memory CacheStore stats; +// this is the audit-side derived rate over the window. +export function cacheHitRateWindow({ windowMs, olpHome }): CacheHitRateSummary; +``` + +### 4.2 Window semantics + +`windowMs` is a duration ending at "now"; `[now - windowMs, now)`. The implementation walks files whose date overlaps that range: today's `audit.ndjson` always; `audit-YYYY-MM-DD.ndjson` for each prior date in range. A line is included only if its `ts` (ISO-8601) falls in the window. + +For the 30-day spend trend, `windowMs = 30 * 86400 * 1000`. The implementation buckets per UTC day and returns one entry per day, including days with zero requests (sparse-fill). + +### 4.3 PII discipline + +Per ADR 0007 § 8, audit events contain no message content, no response content, no raw tokens. The query layer relays only the schema fields. It MUST NOT introduce derived fields that reveal content (e.g., "first 50 chars of prompt"). + +### 4.4 Error handling + +- Missing file → empty iteration (not an error). +- Malformed JSON line → log warn `audit_query_skip_malformed` + skip; continue. +- File-read error (EACCES, etc.) → throw to caller; the dashboard endpoint surfaces 500 with diagnostic message. + +--- + +## 5. Audit rotation + +### 5.1 Trigger + +Rotation fires on the **first append after a UTC date change**. Implementation lives in `lib/audit.mjs` (extended at D49). On each `appendAuditEvent` call: + +1. Compute `today = new Date().toISOString().slice(0, 10)` (e.g., `'2026-05-25'`). +2. Read a module-scoped `_currentDate` cached at startup. +3. If `today !== _currentDate` AND `audit.ndjson` exists AND it is non-empty: + - Rename `audit.ndjson` → `audit-${_currentDate}.ndjson` (the previous day's date). + - Set `_currentDate = today`. + - Continue with the append (new `audit.ndjson` opens via append-create). + +The check is per-call (microsecond cost). The rename is the only filesystem heavy op and fires once per UTC day. + +### 5.2 External-cron alternative (`bin/olp-audit-rotate.mjs`) + +An auxiliary script is shipped at D52 for operators who prefer cron-driven rotation (e.g., to rotate exactly at 00:00:00 UTC rather than "first request after midnight"). The script does the same rename + state-bump logic but can be invoked from a host cron / launchd job. The in-server check remains as a safety net; both can coexist. + +### 5.3 Concurrent-rotation safety + +In-process: the rotation logic is wrapped in a per-process lock (`Map`) so two concurrent `appendAuditEvent` calls don't both attempt the rename. External cron + in-server check: the in-server check sees the rename has already happened (file with today's date already exists if cron beat it); the no-op fallback is "if `audit.ndjson` exists, append; else create + append" — POSIX semantics. + +### 5.4 Renamed-file query path + +The query layer (§ 4) walks `audit-${date}.ndjson` files for any date in the window. Today's file is always `audit.ndjson` (not renamed yet); yesterday + prior are date-suffixed. + +--- + +## 6. Dashboard panels (per spec § 4.6, Lane 5 = B full) + +`dashboard.html` is a single static file served from `/dashboard`. Renders 4 panels in a 2×2 grid: + +### 6.1 Panel 1 — Per-provider quota / credit pool + +For each loaded provider, calls `provider.quotaStatus()` (ADR 0002 Provider contract). Returns whatever the provider can report (e.g., Anthropic Plan limits remaining, Codex credit pool balance) OR `null` (provider opts out — Phase 2 mistral has no quota API). + +Each row shows: +- Provider key + display name +- Quota remaining / quota total (or "n/a" if null) +- Last poll timestamp + +### 6.2 Panel 2 — Per-provider request count + cache hit rate + fallback rate (last 24h) + +Uses `aggregateRequests({ windowMs: 86400 * 1000 })`. One row per provider showing: +- Request count (total served by that provider, regardless of chain position) +- Cache hit rate (% of requests where `cache_status === 'hit'`) +- Fallback rate (% of requests where `fallback_hops > 0`) +- 5xx error rate (% of requests where `status_code >= 500`) + +### 6.3 Panel 3 — Multi-provider unified spend trend (last 30 days) + +Uses `spendTrendDaily({ days: 30 })`. Shows a sparkline-style chart (vanilla SVG, no library) with: +- X axis: 30 daily buckets +- Y axis: request count per day (stacked by provider color) +- Hover tooltip: per-day breakdown by provider + +Note: "spend trend" is the spec's term — at v0.3.0 we don't have provider-side cost integration (Anthropic Plan is flat-rate per Anthropic 2026-06-15 split per the learning memory). So "spend" is proxied by request count. A future ADR may add cost weights per provider when commercial cost-tracking lands. + +### 6.4 Panel 4 — Top fallback chains by trigger count + +Uses `topFallbackChains({ windowMs: 86400 * 1000, limit: 10 })`. Lists top 10 chains: +- Chain shape (e.g., `anthropic → openai`) +- Trigger count +- First / last seen timestamps + +### 6.5 Refresh model (Lane 4 = A) + +The dashboard sets a 30s `setInterval` that calls `fetch('/v0/management/dashboard-data')` + updates DOM in place (no full reload). Initial fetch on page load. The interval pauses when the page is hidden (via `document.visibilityState` listener) to avoid useless background polls. + +### 6.6 Localhost-bound by default + +The dashboard is served from the existing OLP HTTP port (default 3456) which is already bound to `127.0.0.1` per `server.mjs` startup (`server.listen(PORT, '127.0.0.1', ...)`). No additional binding logic. Remote operators access via SSH tunnel; ADR 0007 § 7 owner-only auth provides the per-request gate. + +--- + +## 7. Server endpoints (D50) + +All endpoints are owner-only per ADR 0007 § 7 (owner-tier validation through `authenticate`). Non-owner identities get 401 / 403 per existing patterns; anonymous (when `allow_anonymous: true`) gets 401 (these are management endpoints, not user-facing). + +### 7.1 `GET /dashboard` + +Serves `dashboard.html`. Owner-only gated. Content-Type `text/html; charset=utf-8`. Static file read once at server startup + cached in memory (small, no need to re-read per request). + +### 7.2 `GET /v0/management/dashboard-data` + +Returns the JSON payload the dashboard's 30s poll consumes. Shape: + +```json +{ + "generated_at": "", + "window_24h": , + "quota": [ + { "provider": "anthropic", "quota_remaining": 1234, "quota_total": 5000, "polled_at": "" }, + { "provider": "openai", "quota_remaining": null } + ], + "spend_trend_30d": , + "top_fallback_chains_24h": , + "cache_stats": +} +``` + +### 7.3 `GET /v0/management/quota` + +Returns just the quota array (subset of dashboard-data; useful for scripted monitoring). + +### 7.4 `GET /cache/stats` + +Returns the live in-memory `cacheStore.stats()` shape. Planning authority is **OLP v0.1 spec § 4.6** (which names `/cache/stats` explicitly); ADR 0005's `Consequences/Mitigations` paragraph (~ line 279) references it as the monitoring surface for per-`(provider, model)` cache hit-rate breakdown. + +**Shape gap to resolve at D50.** The current `cacheStore.stats()` in `lib/cache/store.mjs:320-350` returns `{ hits, misses, size, inflightCount }` — global aggregate only, no per-`(provider, model)` breakdown. If D50 reveals the shape is insufficient for Panel 2 (per-provider 24h cache hit rate, currently sourced from `aggregateRequests` audit-side rather than `cacheStore.stats`), the dashboard endpoint is satisfied. If a future panel needs the per-`(provider, model)` breakdown that spec § 4.6 implies, D50 amends the store shape + an ADR 0005 amendment fires at that time. Phase 3 acceptance criteria do not require the breakdown. + +### 7.5 Audit on management endpoints + +All four endpoints append an audit row via the existing `appendAuditEvent` pattern. Path values are `/dashboard` / `/v0/management/dashboard-data` / `/v0/management/quota` / `/cache/stats`. The 30s poll generates 2880 dashboard rows per day per owner — manageable at family scale, but noted as a knob (a future amendment may suppress audit for these paths if they become noise-dominant). + +--- + +## 8. Auth gating + +Reuses ADR 0007 § 7 owner-vs-non-owner model + introduces a second gating mode. + +**Two gating modes (this ADR formalizes the distinction):** + +- **`owner_only_trim`** (Phase 2 / D46 model) — non-owner identities receive a 200 response with a trimmed payload (e.g., `/health` returns `{ ok, version }` only). Used when the endpoint has a baseline payload that is safe to share with all identities and an enriched payload only for owners. +- **`owner_only_block`** (Phase 3 / D48 new) — non-owner identities receive `401 invalid_or_revoked_key` (or `401 auth_required` if no token). Used when the entire payload is sensitive and there is no safe baseline to share (Dashboard quota stats, fallback chains by trigger, etc. all reveal operational behaviour that should not leak to non-owner identities). + +The four new endpoints (`/dashboard`, `/v0/management/dashboard-data`, `/v0/management/quota`, `/cache/stats`) are `owner_only_block`. `/health` remains `owner_only_trim`. + +The owner_only_endpoints config gains four entries; the gating-mode distinction is implementation-side (the handler decides whether to trim or block based on the endpoint). Server startup defaults `owner_only_endpoints` to include `/health` + the four new ones (Phase 3 default; operator can opt-out per-endpoint via config). Pre-Phase-3 deployments with `owner_only_endpoints: ['/health']` continue to work — the new endpoints will 401 for non-owner under that legacy config because the handler is `owner_only_block`-mode regardless of the config list (the config controls /health's trim/full toggle only; the management endpoints are not opt-out-able to a non-401 response). + +`401` shapes match Phase 2 / D45 pattern: JSON `{ error: { message, type } }` with `type: 'auth_required'` or `'invalid_or_revoked_key'`. + +--- + +## 9. Failure modes + graceful degradation + +| Failure | Behavior | +|---|---| +| `audit.ndjson` absent (fresh install) | Empty arrays in all aggregates; dashboard shows "No requests in window" panels | +| `audit-YYYY-MM-DD.ndjson` corrupted (one bad line) | Skip line + log warn; continue; dashboard rendering unaffected | +| Provider `quotaStatus()` throws | Panel shows that row as `quota: "error"`; other providers' rows render normally | +| Dashboard `/v0/management/dashboard-data` query >5s | Dashboard JS shows "Loading…" with a timeout; second poll attempts after 30s | +| `audit.ndjson` rotation fails (rename EACCES) | `appendAuditEvent` warn `audit_rotate_failed` + continues appending to the un-rotated file; next call retries the rotation | +| File handle limit hit during 30-day query (many files open) | Query reads one file at a time (no parallel reads); never opens >2 simultaneously | + +The dashboard degrades visibly (per-panel error states) rather than failing whole-page. + +--- + +## 10. Acceptance criteria + +Implementation D-days (D49+) MUST land tests covering: + +1. **`readAuditWindow`** correctly iterates events from today's `audit.ndjson` + N prior rotated files within the window. +2. **`readAuditWindow`** skips malformed lines without throwing; logs warn for each. +3. **`aggregateRequests`** correctly counts by provider + cache_status + owner_tier + path; correctly computes median + p95 latency. +4. **`topFallbackChains`** returns sorted by count descending; ties broken by first-seen timestamp ascending. +5. **`spendTrendDaily`** sparse-fills zero-request days; window respects UTC day boundaries. +6. **Daily rotation** — writing past UTC midnight renames `audit.ndjson` → `audit-.ndjson` and continues appending to a fresh `audit.ndjson`. +7. **Cross-file query** — a 30-day window with mixed rotated files returns correctly merged results. +8. **Concurrent rotation safety** — N concurrent `appendAuditEvent` calls during a UTC date change result in exactly one rename + all lines append to the correct file. +9. **`GET /dashboard`** returns 200 HTML to owner; 401 to non-owner. This includes the case where `allow_anonymous: true` AND no Authorization header is presented: the authenticate middleware produces an anonymous identity, the `owner_only_block` mode then rejects with 401 (per § 8 — anonymous is non-owner; management endpoints block, do not trim). When `allow_anonymous: false` + no header, 401 fires earlier at the authenticate middleware itself. Test must cover both cases. +10. **`GET /v0/management/dashboard-data`** returns 200 JSON to owner with all required fields populated. +11. **`GET /cache/stats`** returns 200 JSON to owner with the live in-memory cache stats shape. +12. **Dashboard HTML smoke** — fetched via test http client + parsed → has the 4 panel containers + 30s poll script; no JS console errors when loaded in a real browser (manual or playwright; manual is acceptable at Phase 3). +13. **Audit on management endpoints** — calling `/v0/management/dashboard-data` appends an audit row with `path: '/v0/management/dashboard-data'` and `status_code: 200`. +14. **Graceful degradation** — when a provider's `quotaStatus()` throws, the dashboard endpoint still returns 200 with that provider's quota row showing `"quota_remaining": null` and an error indicator. +15. **PII guard** — every aggregate query function asserts at the test level that returned data does NOT include any message content; the `prompt`/`messages`/`response`/`content` fields MUST NEVER appear in any output shape. + +--- + +## 11. Forward path (Phase 4+) + +Items deliberately deferred: + +- **SQLite migration (Option 3 hybrid)** — trigger: query latency >2s on a typical owner session, OR Dashboard usage scales beyond family (>5 owners polling). Preconditions per ADR 0007 § 13: engines bump + CI matrix change as a separate prior PR. +- **SSE push for dashboard live updates** — trigger: 30s poll feels stale, OR operator wants real-time view of streaming requests. Reuses existing streaming infra from ADR 0005 Amendment 8 (v1.x streaming SF when it ships). +- **Key-mgmt UI from dashboard** — owner can create/revoke/edit keys from the web UI rather than CLI. Out of Phase 3 because (a) it adds a write surface to the dashboard requiring careful CSRF handling, (b) security review of the auth flow is non-trivial, (c) the CLI surface from D47 covers the same use cases. +- **Cost weights per provider** — once provider-side cost tracking is feasible, "spend trend" can show actual dollars. At v0.3.0 it's a request-count proxy. +- **Audit retention / max-days policy** — currently unbounded; operator manages disk. A Phase 3+ amendment adds `audit_max_days` config when an operational need emerges. +- **Per-key dashboard views** — owner sees aggregate; per-key drill-down is a future amendment. + +--- + +## 12. Out of scope (explicitly NOT in Phase 3) + +- Per-key per-provider auth artifact mapping (ADR 0007 § 12; Phase 4+). +- `tried_providers` schema semantics fix on `key_no_provider_access` 403 — D45 reviewer P2 deferral. Tracked as a Phase 3 implementation D-day (D53) but NOT part of ADR 0008; documented in ADR 0004 amendment or ADR 0007 § 8 amendment at D53. +- All ADR 0007 § 12 deferrals other than Dashboard + audit query layer + rotation. +- Externally-visible Dashboard (anything bound to 0.0.0.0 / public). Operator SSH-tunnels. + +--- + +## 13. Phase 3 sprint shape + +| D-day | Deliverable | Type | +|---|---|---| +| **D48** | This ADR (0008 draft) | ADR-only | +| **D49** | `lib/audit-query.mjs` + Suite 23 unit tests | impl | +| **D50** | `server.mjs` `/v0/management/*` endpoints + `/dashboard` route + Suite 24 HTTP tests | impl | +| **D51** | `dashboard.html` + render JS + 30s poll | impl | +| **D52** | Audit daily rotation (`lib/audit.mjs` extension + `bin/olp-audit-rotate.mjs` + Suite 25 rotation tests) | impl | +| **D53** | `tried_providers` schema fix (D45 P2 deferral; small) | impl | +| **D54** | E2E browser smoke (manual or playwright) + AGENTS / README polish | tests + docs | +| **D55** | Phase 3 close → v0.3.0 (release-kit PR per `phase_close_trigger`) | release | + +Each D-day = implementor + fresh-context opus reviewer per Iron Rule 10. Estimated wall-clock: similar to Phase 2 cadence (1 intense session per D-day under standing autopilot). + +--- + +## Consequences + +**Positive:** + +- Operators get an at-a-glance view of OLP's behaviour (was a tail/grep exercise pre-Phase-3). +- Audit layer becomes queryable, not just append-only; `lib/audit-query.mjs` is reusable for future CLI tools (`olp-audit search` etc.). +- Daily rotation bounds per-file size + creates a natural archival unit. +- Owner-only gating reuses Phase 2 auth model (no new auth surface). +- v0.1 spec § 4.6 Dashboard requirements are met without introducing a build step / database / framework. + +**Negative / trade-offs:** + +- In-memory ndjson scan is O(N) per query; if audit grows to millions of lines (single-host years), the 30-day query becomes slow. Mitigation: ADR 0007 § 13 Option 3 hybrid is the documented next step; trigger is observed slowness. +- 30s poll generates baseline traffic when dashboard is open (2880 management requests/day per owner). Not a real cost but worth observing. +- Audit rotation is "first append after UTC midnight" which means a server with zero requests overnight rotates lazily (first request of the new day triggers it). External cron at D52 covers the strict-midnight case. +- No automatic audit retention. A multi-year-running server accumulates files; operator manages. + +**Reversibility:** + +- Dashboard is a static file + 4 endpoints. Removable in a future revert PR if Phase 3 retrospectively proves unwanted. +- Audit rotation is additive — disabling reverts to single-file behaviour without code change (operator never invokes the cron + the in-server rotation can be guarded by a config flag). +- The `lib/audit-query.mjs` module is consumed by Dashboard endpoints + can be used standalone; removing it requires unrelated endpoint surgery. + +--- + +## Authority citations + +- **OLP v0.1 spec § 4.6 + § 4.7** (Dashboard + observability endpoints) — at `~/.cc-rules/memory/projects/olp_v0_1_spec.md` on the maintainer's workstations. +- **OCP `dashboard.html`** (prior-art reference for the multi-panel HTML structure) — at `~/ocp/dashboard.html` on the maintainer's workstation; OCP production reference. +- **ADR 0007 §§ 7 / 8 / 12 / 13** (owner-gating model; audit ndjson schema; Phase 3 scope opening; SQLite forward path). +- **ADR 0002** (Provider contract — `quotaStatus` method that Panel 1 consumes). +- **OLP v0.1 spec § 4.6** (planning authority for `/cache/stats` endpoint name + Dashboard requirements). ADR 0005's `Consequences/Mitigations` paragraph references the endpoint as the monitoring surface for per-`(provider, model)` cache hit-rate breakdown; that breakdown is a Phase 4+ amendment trigger if needed (see § 7.4 above). +- **ADR 0004 Amendment 5** (D40 X-OLP-Fallback-Detail — top-fallback-chains panel data shape lineage). +- **Standing-autopilot grant** (`~/.cc-rules/memory/auto/standing_autopilot_phase_2.md` in cc-rules `bf0ed9a`) — Phase 3 kickoff via maintainer "go" + lane pin. +- **Phase 2 kickoff handoff pattern** (`~/.cc-rules/memory/handoffs/2026-05-25-phase-2-kickoff.md` in cc-rules `d9da966`) — this ADR follows the same structure. +- **CLAUDE.md `release_kit.phase_rolling_mode current_phase: Phase 3`** — confirms this ADR lands in the Phase 3 sprint. diff --git a/docs/adr/README.md b/docs/adr/README.md index 85d806f..cf09468 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -21,6 +21,7 @@ New ADRs increment from the highest existing number. Filenames are `NNNN-/manifest.json`) + opaque `olp_<32-byte>` token + SHA-256 hash. Owner / guest / anonymous tier gating with explicit `config.json auth.allow_anonymous` (default false). Bootstrap keygen command surface + `OLP_OWNER_TOKEN` env override with stable synthetic `key_id`. Audit ndjson append-only at `~/.olp/logs/audit.ndjson`, warn+1-retry on append failure. Rejects direct SQLite port at v0.2.0 due to Node baseline (`engines >=18` + CI 20/24 vs `node:sqlite` added 22.5.0 / RC); Option 3 hybrid documented as forward path when Phase 3+ Dashboard / SQL-aggregate quota arrives. | +| [0008](0008-dashboard-and-audit-query.md) | Dashboard + Audit Query Layer | Phase 3 design ADR (D48, 2026-05-25). Static HTML dashboard + vanilla JS + fetch (no build step). In-memory ndjson scan for aggregate queries (O(N) per call; family-scale acceptable; defers SQLite migration to Option 3 hybrid trigger). Daily audit rotation `audit-YYYY-MM-DD.ndjson` on first append after UTC midnight; cross-file query layer for rolling 30-day windows. Owner-only gating on `/dashboard` + 3 `/v0/management/*` JSON endpoints reusing ADR 0007 § 7 auth model. 30s page poll (no SSE infra). Panels: per-provider quota / 24h request+cache+fallback / 30d spend trend / top-N fallback chains per spec § 4.6. Opens ADR 0007 § 12 Phase 3 deferral (Dashboard + audit query + rotation). | ## When to write a new ADR