mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-19 09:45:07 +00:00
feat+test+docs: D45 — server.mjs auth integration + lib/audit.mjs (#21)
* feat+test+docs: D45 — server.mjs auth integration + lib/audit.mjs (Phase 2 wire-up) Second Phase 2 implementation D-day. Wires the D44 lib/keys.mjs identity layer into the request flow + lands lib/audit.mjs per ADR 0007 § 6.2 + § 8. Closes ADR § 10 acceptance criteria #1 (per-key cache isolation), #2 (anonymous prod-default off), #3 (anonymous dev-mode on), #6 (post-revoke 401 within next request — full coverage with D45), #8 (audit ndjson round-trip), #10 (OLP_OWNER_TOKEN env override — full server-side coverage), #11 (providers_enabled 403 scope). Owner-vs-guest gating for /health + X-OLP-Fallback-Detail (criteria #4, #5) remains in D46 scope. NEW lib/audit.mjs (~110 lines): - appendAuditEvent(event, opts): one JSON event per line to ~/.olp/logs/audit.ndjson (file 0600, dir 0700). § 6.2 retry: warn + 1 retry; per-process drop counter + warn on second failure; NEVER throws. Per-call OLP_HOME env resolution (matches lib/keys.mjs). - getAuditDropCount(): for future /health surface. lib/keys.mjs extended: - loadAuthConfigSync({ olpHome }): reads auth block from ~/.olp/config.json with ADR § 7.2 defaults (allow_anonymous: false, owner_only_endpoints: ['/health'], fallback_detail_header_policy: 'owner_only'). Never throws; missing file / malformed JSON falls back to defaults. - _resolveOlpHome(opts): precedence opts.olpHome → process.env.OLP_HOME → ~/.olp. Per-call resolution so tests + operator deployments can redirect without code edits. server.mjs auth middleware integration: - extractToken(req): parses Authorization Bearer / x-api-key. - authenticate(req): validateKey + 401 paths (auth_required vs invalid_or_revoked_key). - isProviderEnabled(olpIdentity, providerKey): '*' = all; else array allowlist. - _authConfig loaded at startup; warn auth_allow_anonymous_enabled when true. Test seams __setAuthConfig / __resetAuthConfig. - handleChatCompletions + handleModels both gated by authenticate at top. Audit ctx built throughout; res.on('finish') appends row + fires touchLastUsed async. - IDENTITY-VS-CREDENTIALS SEPARATION: olpIdentity (new validated identity) consumed for cache namespacing + providers_enabled + audit; authContext passed to provider.spawn() REMAINS null so providers continue their own credential discovery (env / keychain / file). Per-provider per-key credential mapping is Phase 3+ per ADR § 12. - handleChatCompletions chain filtered by isProviderEnabled; empty result returns 403 key_no_provider_access. - keyId = olpIdentity.keyId (replacing hardcoded '__anonymous__'). - Audit captures fields throughout: post-auth, post-IR, post-chain (success or exhausted). Status + latency populated on res.on('finish'). TESTS — Suite 20, +15 (499 → 514): 20a-d: header parsing + valid key happy paths (Bearer / x-api-key / invalid → 401) 20e: revoked key 401 (criterion #6 end-to-end) 20f: OLP_OWNER_TOKEN env override returns 200 (criterion #10 full) 20g: allow_anonymous=true + no header returns 200 (criterion #3) 20h + 20h-extra: providers_enabled=['mistral'] for anthropic model → 403; '*' baseline returns 200 (criterion #11) 20i: per-key cache namespace isolation (criterion #1 end-to-end) 20j + 20j-401: audit.ndjson written with § 8 schema fields + PII guard; 401 path also appends (criterion #8) 20k: filesystem key last_used_at populated post-request (D45 touch wire) 20l + 20l-200: /v1/models also enforces auth TEST-MODE SETUP (test-features.mjs): - process.env.OLP_HOME = mkdtempSync(...) at module load so audit + key writes don't pollute ~/.olp/. - __setAuthConfig({ allow_anonymous: true }) after server.mjs imports so pre-D45 HTTP integration tests (Suite 18 etc.) continue to pass. - Suite 20 explicitly overrides __setAuthConfig per-case to exercise production-default-off coverage. DOCUMENTATION: - AGENTS.md: lib/keys.mjs 🟡 marker updated + NEW lib/audit.mjs entry; Implementation-status-note + shipped-set updated. - README.md: Implementation Status table gains lib/audit.mjs row + lib/keys.mjs row updated; Known limitations Multi-key auth note rewritten to reflect D45 ship + D46 follow-up; new env-vars (OLP_HOME, OLP_OWNER_TOKEN) and auth config block surfaced. - CHANGELOG.md: D45 entry under Unreleased per release_kit overlay phase_rolling_mode discipline. AUTHORITY: - ADR 0007 (multi-key auth — §§ 5/6.2/7/9.4 implementation contracts + § 10 criteria #1/#2/#3/#6/#8/#10/#11 covered). - CLAUDE.md release_kit overlay phase_rolling_mode — under Unreleased. - Phase 2 kickoff handoff (~/.cc-rules/memory/handoffs/ 2026-05-25-phase-2-kickoff.md in cc-rules d9da966). - Standing autopilot grant (~/.cc-rules/memory/auto/ standing_autopilot_phase_2.md in cc-rules bf0ed9a). Verified: 514/514 pass via npm test (no regression in 499 existing tests; 15 new Suite 20 tests all green). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix+test+docs: D45 fold-in — CI fail recovery + opus reviewer P1/P2/P3 Fresh-context opus reviewer (PR #21) returned APPROVE_WITH_MINOR with 4 findings; CI Node 24 separately reported 9 Suite 20 failures (all 200-expecting tests). Root cause of CI: Suite 20 setup did not stub CLAUDE_CODE_OAUTH_TOKEN before mock spawn, so anthropic.mjs AUTH_MISSING pre-check fired and tests 502'd. (Local Node 22 had the env from the maintainer's claude install — masked the gap.) CI FIX — Suite 20 OAuth env stub Added ensureSuite20FakeOAuth / restoreSuite20OAuth helpers in makeSuite20Server / teardownSuite20. Matches the existing pattern in Suite 9 line ~2154 (test-fake-oauth-token-for-cache-tests). P1 — Real-streaming path audit fidelity Single-hop streaming success (server.mjs ~L1050, the most common deployed shape) did not populate auditCtx.provider / tried_providers / cache_status. Audit rows for streaming requests carried provider: null. Fixed by stamping these at the top of the streaming branch and amending error_code on the two streaming failure exit paths (streaming_error_after_first_chunk + streaming_error_before_first_chunk). New regression test 20j-stream: streaming request asserts the audit row's provider, cache_status, and tried_providers fields are populated. P2 — Global test tmpdir cleanup process.env.OLP_HOME = mkdtempSync(...) at test-features.mjs module load left /var/folders/.../olp-test-home-* leak per npm test run. Fixed by process.on('exit', () => rmSync(_GLOBAL_TEST_OLP_HOME)). Best-effort; swallows errors so exit handler never throws. P3 — handleModels 401 lacks OLP diagnostic headers handleChatCompletions 401 passes olpErrorHeaders({ startMs }); handleModels 401 did not. Aligned. DEFERRED — P2 tried_providers semantics on 403 Reviewer noted that key_no_provider_access 403 stamps original chain in tried_providers, but the field name implies hops actually dispatched. Either ADR § 8 amendment or D46+ semantic fix. Marked in CHANGELOG; not in this fold-in scope. Test count: 514 → 515 (+1 streaming-audit regression test 20j-stream; 14 existing Suite 20 tests still pass). Verified locally via npm test. CI Node 24 recovery via the OAuth env stub. Authority: PR #21 fresh-context opus reviewer findings; CI Node 24 run 26382758946 failure logs; CLAUDE.md release_kit overlay phase_rolling_mode — under Unreleased. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
+110
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* lib/audit.mjs — OLP audit ndjson append (Phase 2 / D45)
|
||||
*
|
||||
* Authority: ADR 0007 § 6.2 (audit append semantics) + § 8 (event schema).
|
||||
*
|
||||
* Behaviour:
|
||||
* - One JSON event per line; newline-terminated; UTF-8.
|
||||
* - Append to ~/.olp/logs/audit.ndjson (chmod 0600 file, 0700 dir).
|
||||
* - On append failure: log a warn ('audit_append_failed_once') + retry
|
||||
* once synchronously.
|
||||
* - On second-failure: increment per-process drop counter + log warn
|
||||
* ('audit_append_dropped'); NEVER throw to the caller (audit is
|
||||
* observability, not authorization). Per § 6.2.
|
||||
* - No memory buffer at Phase 2 (forward-path note in ADR § 13).
|
||||
*
|
||||
* Atomicity note: Node's `fs.appendFileSync` opens with O_APPEND which is
|
||||
* POSIX-atomic for writes <= PIPE_BUF (typically 4096 bytes). Our event
|
||||
* payloads (§ 8 schema with hash + shape fields, no PII / no message
|
||||
* content) are well under that limit, so concurrent in-process appends
|
||||
* are line-atomic without explicit locking.
|
||||
*
|
||||
* No PII: § 8 explicitly excludes request body, response body, and IR
|
||||
* message content. Hash + shape only.
|
||||
*/
|
||||
|
||||
import { appendFileSync, mkdirSync, chmodSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { homedir } from 'node:os';
|
||||
|
||||
const DEFAULT_OLP_HOME = join(homedir(), '.olp');
|
||||
const OLP_HOME_ENV = 'OLP_HOME';
|
||||
const RETRY_COUNT = 1; // § 6.2: warn + 1 retry
|
||||
|
||||
let _dropCounter = 0;
|
||||
|
||||
/**
|
||||
* Resolve OLP home dir (matches lib/keys.mjs precedence): opts.olpHome →
|
||||
* process.env.OLP_HOME → ~/.olp. Resolved per call so tests setting the
|
||||
* env mid-run take effect.
|
||||
*/
|
||||
function _resolveOlpHome(opts) {
|
||||
if (opts?.olpHome) return opts.olpHome;
|
||||
if (process.env[OLP_HOME_ENV]) return process.env[OLP_HOME_ENV];
|
||||
return DEFAULT_OLP_HOME;
|
||||
}
|
||||
|
||||
/**
|
||||
* Append a single audit event to ~/.olp/logs/audit.ndjson.
|
||||
*
|
||||
* @param {object} event - § 8 schema fields (ts, key_id, owner_tier,
|
||||
* method, path, provider, model, status_code, latency_ms, cache_status,
|
||||
* fallback_hops, tried_providers, error_code, ir_request_hash, chain_id).
|
||||
* Caller is responsible for populating fields; missing fields are
|
||||
* serialized as undefined → omitted by JSON.stringify.
|
||||
* @param {object} [opts]
|
||||
* @param {string} [opts.olpHome] - test override; defaults to ~/.olp
|
||||
* @param {(level: string, event: string, data?: object) => void} [opts.logEvent]
|
||||
* - injectable structured logger; defaults to console.warn with JSON line
|
||||
*/
|
||||
export function appendAuditEvent(event, opts = {}) {
|
||||
const olpHome = _resolveOlpHome(opts);
|
||||
const logsDir = join(olpHome, 'logs');
|
||||
const path = join(logsDir, 'audit.ndjson');
|
||||
const line = JSON.stringify(event) + '\n';
|
||||
const logEvent = opts.logEvent ?? ((level, ev, data) => {
|
||||
const entry = { ts: new Date().toISOString(), level, event: ev, ...(data ?? {}) };
|
||||
process.stderr.write(JSON.stringify(entry) + '\n');
|
||||
});
|
||||
|
||||
for (let attempt = 0; attempt <= RETRY_COUNT; attempt++) {
|
||||
try {
|
||||
mkdirSync(logsDir, { recursive: true, mode: 0o700 });
|
||||
// Tighten dir mode in case it already existed with broader permissions.
|
||||
try { chmodSync(logsDir, 0o700); } catch { /* tolerate EPERM */ }
|
||||
appendFileSync(path, line, { mode: 0o600 });
|
||||
return;
|
||||
} catch (err) {
|
||||
if (attempt < RETRY_COUNT) {
|
||||
logEvent('warn', 'audit_append_failed_once', {
|
||||
path,
|
||||
error: err?.message ?? String(err),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
_dropCounter++;
|
||||
logEvent('warn', 'audit_append_dropped', {
|
||||
path,
|
||||
error: err?.message ?? String(err),
|
||||
drop_count: _dropCounter,
|
||||
});
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-process count of audit events dropped due to repeated append failure.
|
||||
* Useful for /health observability surface (D46).
|
||||
*/
|
||||
export function getAuditDropCount() {
|
||||
return _dropCounter;
|
||||
}
|
||||
|
||||
/**
|
||||
* Test-only: reset the drop counter to zero. Suite 20 uses this between
|
||||
* cases to assert independent failure-handling counts.
|
||||
*/
|
||||
export function __resetAuditDropCount() {
|
||||
_dropCounter = 0;
|
||||
}
|
||||
+61
-1
@@ -45,6 +45,20 @@ export const ENV_OWNER_KEY_ID = '__env_owner__';
|
||||
export const ENV_OWNER_VAR = 'OLP_OWNER_TOKEN';
|
||||
|
||||
const DEFAULT_OLP_HOME = join(homedir(), '.olp');
|
||||
export const OLP_HOME_ENV = 'OLP_HOME';
|
||||
|
||||
/**
|
||||
* Resolve the OLP home directory. Precedence:
|
||||
* 1. `opts.olpHome` (explicit caller override — tests, CLI flags)
|
||||
* 2. `process.env.OLP_HOME` (operator / CI env override)
|
||||
* 3. `~/.olp` (default per ADR 0007 § 3)
|
||||
* Resolved dynamically per call so tests setting OLP_HOME mid-run take effect.
|
||||
*/
|
||||
function _resolveOlpHome(opts) {
|
||||
if (opts?.olpHome) return opts.olpHome;
|
||||
if (process.env[OLP_HOME_ENV]) return process.env[OLP_HOME_ENV];
|
||||
return DEFAULT_OLP_HOME;
|
||||
}
|
||||
|
||||
// ── Internal state ────────────────────────────────────────────────────────
|
||||
|
||||
@@ -60,7 +74,7 @@ let _touchInterleaveHook = async () => {};
|
||||
|
||||
// ── Path helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
function _olpHome(opts) { return opts?.olpHome ?? DEFAULT_OLP_HOME; }
|
||||
function _olpHome(opts) { return _resolveOlpHome(opts); }
|
||||
function _keysDir(opts) { return join(_olpHome(opts), 'keys'); }
|
||||
function _keyDir(id, opts) { return join(_keysDir(opts), id); }
|
||||
function _manifestPath(id, opts) { return join(_keyDir(id, opts), 'manifest.json'); }
|
||||
@@ -431,6 +445,52 @@ export async function touchLastUsed(id, opts = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Auth config loader (§ 7.2) ────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Read the `auth` block from ~/.olp/config.json. All fields defaulted
|
||||
* so partial / absent config is safe.
|
||||
*
|
||||
* Defaults per ADR § 7.2:
|
||||
* - allow_anonymous: false (production-off default)
|
||||
* - owner_only_endpoints: ['/health'] (D46 consumes; D45 only loads)
|
||||
* - fallback_detail_header_policy: 'owner_only' (D46 consumes; D45 only loads)
|
||||
*
|
||||
* Returns the auth config object. Never throws — missing file / parse
|
||||
* error / missing `auth` key all fall back to defaults.
|
||||
*
|
||||
* @param {object} [opts]
|
||||
* @param {string} [opts.olpHome] - test override; defaults to ~/.olp
|
||||
* @returns {{ allow_anonymous: boolean, owner_only_endpoints: string[], fallback_detail_header_policy: 'owner_only'|'all'|'none' }}
|
||||
*/
|
||||
export function loadAuthConfigSync(opts = {}) {
|
||||
const olpHome = _resolveOlpHome(opts);
|
||||
const path = join(olpHome, 'config.json');
|
||||
const DEFAULTS = {
|
||||
allow_anonymous: false,
|
||||
owner_only_endpoints: ['/health'],
|
||||
fallback_detail_header_policy: 'owner_only',
|
||||
};
|
||||
if (!existsSync(path)) return { ...DEFAULTS };
|
||||
try {
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const cfg = JSON.parse(raw);
|
||||
const auth = (cfg && typeof cfg === 'object' && cfg.auth && typeof cfg.auth === 'object')
|
||||
? cfg.auth
|
||||
: {};
|
||||
return {
|
||||
allow_anonymous: typeof auth.allow_anonymous === 'boolean' ? auth.allow_anonymous : DEFAULTS.allow_anonymous,
|
||||
owner_only_endpoints: Array.isArray(auth.owner_only_endpoints) ? auth.owner_only_endpoints : DEFAULTS.owner_only_endpoints,
|
||||
fallback_detail_header_policy: ['owner_only', 'all', 'none'].includes(auth.fallback_detail_header_policy)
|
||||
? auth.fallback_detail_header_policy
|
||||
: DEFAULTS.fallback_detail_header_policy,
|
||||
};
|
||||
} catch {
|
||||
// Malformed JSON / unreadable file → safe defaults
|
||||
return { ...DEFAULTS };
|
||||
}
|
||||
}
|
||||
|
||||
// ── Test-only hooks ───────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user