feat+test+docs: D52 — daily audit rotation (lib/audit.mjs + bin/olp-audit-rotate.mjs) (#29)

Fifth Phase 3 D-day. Adds daily UTC-aware rotation to lib/audit.mjs
per ADR 0008 § 5 + ships an external cron tool. Rotation is
SYNCHRONOUS at v0.3.0 — synchronous design eliminates the race that
an async wrapper would create between date-change-detection and the
append.

lib/audit.mjs EXTENSIONS:

  - New _maybeRotateAudit({ olpHome, logEvent }) (sync): probes live
    audit.ndjson; if it holds events from past UTC date, renames it
    to audit-YYYY-MM-DD.ndjson. Idempotent. If target file exists
    (cron beat in-server check), logs warn + skips per § 5.3.

  - appendAuditEvent extended: cheap fast-path date check via module-
    cached _lastSeenUtcDate. On date change, calls _maybeRotateAudit
    synchronously BEFORE appendFileSync — so old-date events land in
    the rotated file and new-date events land in the fresh live file.
    No event straddles the boundary.

  - Why SYNCHRONOUS: an async wrapper would let the sync
    appendFileSync race the not-yet-completed renameSync, landing
    today's event in the about-to-be-renamed file. Sync rotation is
    the only correct ordering at the append-fired-from-many-routes
    scale OLP runs. (Test 26b-1 caught this during local run; the
    initial async-wrapper implementation failed because the live
    file at assertion time didn't exist.)

  - New exports: _maybeRotateAudit (sync), getAuditRotateCount,
    getAuditRotateFailCount, __resetAuditRotateState,
    __setLastSeenUtcDateForTesting.

  - First-event-date discovery: when probing the live file's date,
    reads only the first ndjson line + parses its ts. Falls back to
    file mtime if events absent (corrupt/empty edge).

bin/olp-audit-rotate.mjs (~95 lines): external cron tool per § 5.2.

  Calls _maybeRotateAudit once + reports outcome. Exit codes 0
  (success or no-op), 1 (bad usage), 2 (rotation failed). Installed
  via package.json bin so `npx olp-audit-rotate [--olp-home=<path>]`
  works. Example cron line in file header.

CONCURRENT-SAFETY (§ 5.3):

  In-process sequential appends after the first date-change detection
  short-circuit via the updated _lastSeenUtcDate cache → exactly 1
  rename even under N sequential appends. Cross-process (cron + server)
  coexistence handled by the "target already exists → skip + warn"
  branch.

TESTS — Suite 26, +12 (588 → 600):

  26a-1..5: _maybeRotateAudit (no live file / today already /
    yesterday→rotate / idempotent re-call / cron-race target-exists
    warn)
  26b-1: appendAuditEvent past UTC date change triggers sync rotation
    + append lands in fresh live file
  26c-1: 10 sequential appendAuditEvent across date change → exactly
    1 rotation + all 10 events in new live file
  26d-1..4: bin/olp-audit-rotate.mjs CLI (--help / no-live-file /
    yesterday-file-rotates / unknown-flag exit 1)
  26e-1: rotated files queryable via lib/audit-query.mjs
    discoverAuditFiles + readAuditWindow cross-file read

package.json: bin.olp-audit-rotate + scripts.olp-audit-rotate entries
added.

DOCUMENTATION:

  - AGENTS.md: lib/audit.mjs marker promoted  (D45 append + D52
    rotation both shipped); new bin/olp-audit-rotate.mjs entry.
  - CHANGELOG.md: D52 entry under Unreleased per release_kit overlay.

NOT IN D52:

  - tried_providers schema fix (D53; D45 P2 deferral)
  - E2E + docs polish (D54)
  - Phase 3 close → v0.3.0 (D55; maintainer-triggered)

Test count: 588 → 600 (+12). Verified locally via npm test.

AUTHORITY:

  - ADR 0008 § 5.1 (first-append-after-UTC-midnight trigger).
  - ADR 0008 § 5.2 (external cron alternative).
  - ADR 0008 § 5.3 (concurrent-rotation safety + cron-coexistence).
  - ADR 0008 § 5.4 (renamed-file query path consumed by D49 lib/
    audit-query.mjs).
  - CLAUDE.md release_kit overlay phase_rolling_mode — under
    Unreleased.
  - Standing autopilot grant.

ALIGNMENT.md scope check: extends lib/audit.mjs (a Phase 2 internal
module) + adds new bin/ CLI + small package.json bin/scripts entries.
No provider plugin / entry surface / IR change.

Co-authored-by: dtzp555 <dtzp555@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
dtzp555-max
2026-05-25 16:27:26 +10:00
committed by GitHub
co-authored by taodeng Claude Opus 4.7
parent 251b578114
commit 408d5a839a
6 changed files with 530 additions and 5 deletions
+2 -1
View File
@@ -39,7 +39,8 @@ Runtime: Node.js (ESM, `.mjs` throughout). No build step. No bundler. `server.mj
- `lib/fallback/` — fallback engine (trigger detection, chain advancement, idempotent-failure safety, header annotation). Governed by ADR 0004. - `lib/fallback/` — fallback engine (trigger detection, chain advancement, idempotent-failure safety, header annotation). Governed by ADR 0004.
- `lib/keys.mjs` — multi-key auth, per-key namespacing, identity layer. Carries OCP's per-key isolation model into OLP. **✅ Phase 2 — D44 core + D45 server integration + D46 owner gating shipped (validateKey on every /v1/* + /health; chain filtered by providers_enabled; touchLastUsed fires post-response; /health payload trimmed for non-owner; X-OLP-Fallback-Detail gated by fallback_detail_header_policy).** - `lib/keys.mjs` — multi-key auth, per-key namespacing, identity layer. Carries OCP's per-key isolation model into OLP. **✅ Phase 2 — D44 core + D45 server integration + D46 owner gating shipped (validateKey on every /v1/* + /health; chain filtered by providers_enabled; touchLastUsed fires post-response; /health payload trimmed for non-owner; X-OLP-Fallback-Detail gated by fallback_detail_header_policy).**
- `bin/olp-keys.mjs` — keygen CLI bootstrap surface per ADR 0007 § 9.1. **✅ Shipped at D47.** Subcommands: `keygen [--owner|--name=X|--providers=csv|--force]`, `list [--owner-only|--include-revoked]`, `revoke --id=X`. Plaintext token printed once on keygen. Installed via `package.json bin` so `npx olp-keys ...` works (also `npm run olp-keys ...`). - `bin/olp-keys.mjs` — keygen CLI bootstrap surface per ADR 0007 § 9.1. **✅ Shipped at D47.** Subcommands: `keygen [--owner|--name=X|--providers=csv|--force]`, `list [--owner-only|--include-revoked]`, `revoke --id=X`. Plaintext token printed once on keygen. Installed via `package.json bin` so `npx olp-keys ...` works (also `npm run olp-keys ...`).
- `lib/audit.mjs` — append-only ndjson audit per ADR 0007 § 6.2 + § 8. **🟡 D45 appendAuditEvent + getAuditDropCount shipped. Fires per /v1/chat/completions + /v1/models request including 401/403/5xx paths. Warn+1-retry on append failure; no memory buffer at Phase 2 (forward path).** **D52 will extend with daily rotation per ADR 0008 § 5.** - `lib/audit.mjs` — append-only ndjson audit per ADR 0007 § 6.2 + § 8 + daily rotation per ADR 0008 § 5. ** D45 (append) + D52 (rotation) shipped. `appendAuditEvent` fires per /v1/chat/completions + /v1/models + /v0/management/* request (warn + 1 retry; no memory buffer). `_maybeRotateAudit` (sync) is called BEFORE the append when the UTC date changes; renames live → `audit-YYYY-MM-DD.ndjson`. Optional external cron tool `bin/olp-audit-rotate.mjs` for exact-at-midnight rotation.**
- `bin/olp-audit-rotate.mjs` — external audit rotation cron tool per ADR 0008 § 5.2. **✅ D52 — `runCli(argv, { out, err })` invocable + main-guard for direct execution. Installed via `package.json bin` so `npx olp-audit-rotate` works (also `npm run olp-audit-rotate`). Idempotent + safe alongside in-server first-append trigger.**
- `lib/audit-query.mjs` — audit ndjson aggregate query layer per ADR 0008 § 4. **🟡 D49 — discoverAuditFiles + readAuditWindow + aggregateRequests + topFallbackChains + spendTrendDaily + cacheHitRateWindow shipped. Cross-file walk over `audit.ndjson` (live) + `audit-YYYY-MM-DD.ndjson` (rotated). PII guard: aggregate shapes never include message content. In-memory scan per request (ADR 0008 Lane 2 = A; SQLite hybrid deferred to ADR 0007 § 13 trigger).** - `lib/audit-query.mjs` — audit ndjson aggregate query layer per ADR 0008 § 4. **🟡 D49 — discoverAuditFiles + readAuditWindow + aggregateRequests + topFallbackChains + spendTrendDaily + cacheHitRateWindow shipped. Cross-file walk over `audit.ndjson` (live) + `audit-YYYY-MM-DD.ndjson` (rotated). PII guard: aggregate shapes never include message content. In-memory scan per request (ADR 0008 Lane 2 = A; SQLite hybrid deferred to ADR 0007 § 13 trigger).**
- `dashboard.html` — owner-only multi-provider dashboard (4 panels: per-provider quota / 24h request+cache+fallback / 30d spend trend SVG sparkline / top-10 fallback chains). **✅ D51 — full UI shipped at repo root per ADR 0008 § 6. Vanilla HTML + JS + fetch (no build step, no framework, no CDN). 30s page poll with `document.visibilityState` pause/resume. Served by `/dashboard` route in server.mjs owner-only_block. Cached in memory at first request via `_loadDashboardHtml`.** - `dashboard.html` — owner-only multi-provider dashboard (4 panels: per-provider quota / 24h request+cache+fallback / 30d spend trend SVG sparkline / top-10 fallback chains). **✅ D51 — full UI shipped at repo root per ADR 0008 § 6. Vanilla HTML + JS + fetch (no build step, no framework, no CDN). 30s page poll with `document.visibilityState` pause/resume. Served by `/dashboard` route in server.mjs owner-only_block. Cached in memory at first request via `_loadDashboardHtml`.**
- `models-registry.json` — single source of truth for `(provider, model) → metadata`. SPOT. - `models-registry.json` — single source of truth for `(provider, model) → metadata`. SPOT.
+23
View File
@@ -4,6 +4,29 @@ All notable changes to OLP land here. Per `CLAUDE.md` release_kit overlay, this
## Unreleased ## Unreleased
### D52 — Daily audit rotation (`lib/audit.mjs` extension + `bin/olp-audit-rotate.mjs`)
Fifth Phase 3 D-day. Adds daily UTC-aware rotation to `lib/audit.mjs` per ADR 0008 § 5 + ships an external cron tool. Rotation is **synchronous** at v0.3.0 (Lane 3 = B daily rotation; synchronous design eliminates the race that an async wrapper would create between date-change-detection and the append).
- **`lib/audit.mjs` extended**:
- New `_maybeRotateAudit({ olpHome, logEvent })` (synchronous): probes the live `audit.ndjson`; if it holds events from a past UTC date, renames it to `audit-YYYY-MM-DD.ndjson`. Idempotent. If the target file already exists (cron beat the in-server check), logs warn + skips per ADR 0008 § 5.3 race safety.
- `appendAuditEvent` extended: cheap fast-path date check via module-cached `_lastSeenUtcDate`. On date change, calls `_maybeRotateAudit` synchronously BEFORE `appendFileSync` — so old-date events land in the rotated file and new-date events land in the fresh live file. No event straddles the boundary.
- Why synchronous instead of async: an async wrapper would let the sync `appendFileSync` race the not-yet-completed `renameSync`, landing today's event in the about-to-be-renamed file. Sync rotation is the only correct ordering at the append-fired-from-many-routes scale OLP runs.
- New exports: `_maybeRotateAudit` (sync), `getAuditRotateCount`, `getAuditRotateFailCount`, `__resetAuditRotateState`, `__setLastSeenUtcDateForTesting`.
- First-event-date discovery: when probing the live file's date, reads only the first ndjson line + parses its `ts`. Falls back to file mtime if events absent (corrupt/empty edge).
- **`bin/olp-audit-rotate.mjs`** (~95 lines): external cron tool per ADR 0008 § 5.2. Calls `_maybeRotateAudit` once + reports outcome. Exit codes 0 (success or no-op), 1 (bad usage), 2 (rotation failed). Installed via `package.json bin` so `npx olp-audit-rotate [--olp-home=<path>]` works. Example cron line documented in the file header.
- **Concurrent-safety semantics** (ADR 0008 § 5.3): in-process sequential appends after the first date-change detection short-circuit via the updated `_lastSeenUtcDate` cache → exactly 1 rename even under N sequential appends. Cross-process (cron + server) coexistence handled by the "target already exists → skip + warn" branch.
- **Test surface (Suite 26, +12 tests — 588 → 600):**
- 26a-1..5: `_maybeRotateAudit` (no live file / today already / yesterday→rotate / idempotent re-call / cron-race target-exists warn)
- 26b-1: `appendAuditEvent` past UTC date change triggers sync rotation + append lands in fresh live file
- 26c-1: 10 sequential `appendAuditEvent` across date change → exactly 1 rotation + all 10 events in new live file
- 26d-1..4: `bin/olp-audit-rotate.mjs` CLI (--help / no-live-file / yesterday-file-rotates / unknown-flag exit 1)
- 26e-1: rotated files queryable via `lib/audit-query.mjs` `discoverAuditFiles` + `readAuditWindow` cross-file read
- **`package.json`**: `bin.olp-audit-rotate` + `scripts.olp-audit-rotate` entries added.
- **Documentation:** AGENTS.md `lib/audit.mjs` marker promoted to ✅ (D45 append + D52 rotation both shipped); new `bin/olp-audit-rotate.mjs` entry.
- **Test count:** 588 → 600 (+12 D52 tests in Suite 26).
- **Authority:** ADR 0008 § 5.1 (first-append-after-UTC-midnight trigger), § 5.2 (external cron alternative), § 5.3 (concurrent-rotation safety + cron-coexistence semantics), § 5.4 (renamed-file query path consumed by D49 lib/audit-query.mjs); CLAUDE.md `release_kit overlay phase_rolling_mode` — under Unreleased; standing autopilot grant.
### D51 — `dashboard.html` full multi-panel UI (Phase 3) ### D51 — `dashboard.html` full multi-panel UI (Phase 3)
Fourth Phase 3 D-day. Replaces the D50 `dashboard.html` placeholder with the full 4-panel UI per ADR 0008 § 6. Vanilla HTML + JS + fetch — no build step, no framework, no CDN (Lane 1 = A). 30s page poll with `document.visibilityState` pause/resume (Lane 4 = A). Fourth Phase 3 D-day. Replaces the D50 `dashboard.html` placeholder with the full 4-panel UI per ADR 0008 § 6. Vanilla HTML + JS + fetch — no build step, no framework, no CDN (Lane 1 = A). 30s page poll with `document.visibilityState` pause/resume (Lane 4 = A).
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env node
/**
* bin/olp-audit-rotate.mjs — External audit rotation cron tool (Phase 3 / D52)
*
* Authority: ADR 0008 § 5.2 (external cron alternative to in-server first-
* append-after-UTC-midnight trigger).
*
* Use case: operators who want exact-at-UTC-midnight rotation rather than
* "first request after midnight". Invoke from a host cron / launchd job.
*
* Idempotent + safe to run alongside the in-server check (both detect the
* date condition; whichever fires first does the rename; the other no-ops).
*
* Usage:
* olp-audit-rotate [--olp-home=<path>]
*
* Exit codes:
* 0 = success (rotation performed OR no rotation needed)
* 1 = bad usage (unknown flag)
* 2 = rotation attempted + failed (e.g., EACCES)
*
* Example cron line (UTC midnight):
* 1 0 * * * /usr/local/bin/node /path/to/bin/olp-audit-rotate.mjs >> /var/log/olp-audit-rotate.log 2>&1
*/
import { _maybeRotateAudit } from '../lib/audit.mjs';
function parseArgv(argv) {
const flags = {};
for (const arg of argv) {
if (arg.startsWith('--')) {
const eq = arg.indexOf('=');
if (eq > 0) flags[arg.slice(2, eq)] = arg.slice(eq + 1);
else flags[arg.slice(2)] = true;
}
}
return flags;
}
const USAGE = `OLP audit rotation cron tool
Usage:
olp-audit-rotate [--olp-home=<path>]
Triggers a rotation check: if the live audit.ndjson holds events from a
past UTC date, rename it to audit-YYYY-MM-DD.ndjson. Idempotent; safe to
run alongside the in-server first-append-after-UTC-midnight trigger.
Authority: ADR 0008 § 5.2.`;
export async function runCli(argv, opts = {}) {
const ioOut = opts.out ?? (s => process.stdout.write(s));
const ioErr = opts.err ?? (s => process.stderr.write(s));
if (argv.includes('--help') || argv.includes('-h')) {
ioOut(USAGE + '\n');
return 0;
}
const flags = parseArgv(argv);
const allowed = new Set(['olp-home', 'help', 'h']);
for (const k of Object.keys(flags)) {
if (!allowed.has(k)) {
ioErr(`Error: unknown flag --${k}\n${USAGE}\n`);
return 1;
}
}
const olpHome = typeof flags['olp-home'] === 'string' ? flags['olp-home'] : undefined;
try {
// _maybeRotateAudit is synchronous at v0.3.0 (D52) — rotation must
// complete BEFORE the next append so no event straddles the boundary.
const result = _maybeRotateAudit({ olpHome });
if (result.rotated) {
ioOut(`Rotated ${result.fromPath} -> ${result.toPath} (dateUsed=${result.dateUsed}).\n`);
} else {
ioOut('No rotation needed (live audit is current or absent).\n');
}
return 0;
} catch (err) {
ioErr(`Error: rotation failed: ${err?.message ?? err}\n`);
return 2;
}
}
// Main guard
const isMain = (() => {
try { return import.meta.url === `file://${process.argv[1]}`; }
catch { return false; }
})();
if (isMain) {
runCli(process.argv.slice(2)).then(code => process.exit(code));
}
+174 -2
View File
@@ -23,15 +23,23 @@
* message content. Hash + shape only. * message content. Hash + shape only.
*/ */
import { appendFileSync, mkdirSync, chmodSync } from 'node:fs'; import { appendFileSync, mkdirSync, chmodSync, renameSync, existsSync, statSync, readFileSync } from 'node:fs';
import { join } from 'node:path'; import { join } from 'node:path';
import { homedir } from 'node:os'; import { homedir } from 'node:os';
const DEFAULT_OLP_HOME = join(homedir(), '.olp'); const DEFAULT_OLP_HOME = join(homedir(), '.olp');
const OLP_HOME_ENV = 'OLP_HOME'; const OLP_HOME_ENV = 'OLP_HOME';
const RETRY_COUNT = 1; // § 6.2: warn + 1 retry const RETRY_COUNT = 1; // § 6.2: warn + 1 retry
const LIVE_AUDIT_FILE = 'audit.ndjson';
const ROTATED_FILE_PREFIX = 'audit-';
const ROTATED_FILE_SUFFIX = '.ndjson';
let _dropCounter = 0; let _dropCounter = 0;
let _rotateCounter = 0; // observability + test assertion target
let _rotateFailCounter = 0;
// Module-cached "last UTC date we saw at append time" so we don't read
// disk metadata on every append just to check for rotation.
let _lastSeenUtcDate = _utcDateNow();
/** /**
* Resolve OLP home dir (matches lib/keys.mjs precedence): opts.olpHome → * Resolve OLP home dir (matches lib/keys.mjs precedence): opts.olpHome →
@@ -44,6 +52,120 @@ function _resolveOlpHome(opts) {
return DEFAULT_OLP_HOME; return DEFAULT_OLP_HOME;
} }
/**
* Current UTC date as YYYY-MM-DD. Module-private; reused by rotation logic.
*/
function _utcDateNow() {
return new Date().toISOString().slice(0, 10);
}
/**
* Read the first event's `ts` from the live audit file to discover the
* date it was opened on (for stale-cache recovery when the process is
* restarted and the in-memory `_lastSeenUtcDate` doesn't match). Returns
* the YYYY-MM-DD string, or null if the file is missing / empty /
* malformed.
*/
function _firstEventDateInLiveFile(livePath) {
try {
if (!existsSync(livePath)) return null;
// Read the file + slice off the first ndjson line. For audit ndjson the
// first line is at most a few hundred bytes; this is fine for the rare
// "process restart with a stale live file" recovery path. (Family-scale
// single-file audit is bounded; multi-MB scans are not a real concern
// until Phase 4+ when Option 3 SQLite migration would kick in anyway.)
const raw = readFileSync(livePath, 'utf-8');
const nl = raw.indexOf('\n');
const firstLine = nl === -1 ? raw : raw.slice(0, nl);
if (!firstLine) return null;
const obj = JSON.parse(firstLine);
if (typeof obj?.ts !== 'string') return null;
return obj.ts.slice(0, 10);
} catch {
return null;
}
}
/**
* Rotate the live audit file (if any) to its UTC-date suffix. Idempotent:
* if a rotated file with that name already exists (e.g., cron beat us to
* it), this skips the rename.
*
* Per ADR 0008 § 5.1 + § 5.3. SYNCHRONOUS so callers (appendAuditEvent +
* external cron) can rely on it completing before the next IO operation.
* Sync rotation eliminates the race that an async wrapper would create
* between the date-change-detection and the append (where the append could
* land in the old un-rotated file).
*
* Concurrent in-process invocations: Node's single-threaded event loop
* serializes synchronous calls within a tick. Cross-tick concurrency
* uses the `_lastSeenUtcDate` cache as the gate — once set, subsequent
* appends short-circuit the rotation check.
*
* @param {object} args - { olpHome, logEvent, _nowFn (test injection) }
* @returns {{ rotated: boolean, fromPath?: string, toPath?: string, dateUsed?: string }}
*/
export function _maybeRotateAudit(args = {}) {
const olpHome = _resolveOlpHome(args);
const logEvent = args.logEvent ?? ((level, ev, data) => {
const entry = { ts: new Date().toISOString(), level, event: ev, ...(data ?? {}) };
process.stderr.write(JSON.stringify(entry) + '\n');
});
const nowFn = args._nowFn ?? (() => new Date());
const logsDir = join(olpHome, 'logs');
const livePath = join(logsDir, LIVE_AUDIT_FILE);
if (!existsSync(livePath)) {
// No live file yet (first append ever); no rotation needed.
_lastSeenUtcDate = nowFn().toISOString().slice(0, 10);
return { rotated: false };
}
// Determine the "date the live file holds" — use the first event's ts.
// Fall back to file mtime if events absent (corrupt / empty file edge).
let fileDate = _firstEventDateInLiveFile(livePath);
if (fileDate === null) {
try {
fileDate = statSync(livePath).mtime.toISOString().slice(0, 10);
} catch {
return { rotated: false };
}
}
const today = nowFn().toISOString().slice(0, 10);
if (fileDate === today) {
_lastSeenUtcDate = today;
return { rotated: false };
}
// Rotate: rename live → audit-<fileDate>.ndjson.
const rotatedName = `${ROTATED_FILE_PREFIX}${fileDate}${ROTATED_FILE_SUFFIX}`;
const rotatedPath = join(logsDir, rotatedName);
if (existsSync(rotatedPath)) {
// Cron or another writer beat us; the live file holding fileDate's
// events must be merged manually. Per ADR 0008 § 5.3 we log + skip.
logEvent('warn', 'audit_rotate_target_exists', {
livePath, rotatedPath,
message: 'rotation target exists — concurrent rotator beat in-process check; manual merge required if events overlap',
});
_lastSeenUtcDate = today;
return { rotated: false };
}
try {
renameSync(livePath, rotatedPath);
_rotateCounter++;
_lastSeenUtcDate = today;
logEvent('info', 'audit_rotated', { fromPath: livePath, toPath: rotatedPath, dateUsed: fileDate });
return { rotated: true, fromPath: livePath, toPath: rotatedPath, dateUsed: fileDate };
} catch (err) {
_rotateFailCounter++;
logEvent('warn', 'audit_rotate_failed', {
livePath, rotatedPath,
error: err?.message ?? String(err),
});
// Don't update _lastSeenUtcDate so the next append retries.
return { rotated: false };
}
}
/** /**
* Append a single audit event to ~/.olp/logs/audit.ndjson. * Append a single audit event to ~/.olp/logs/audit.ndjson.
* *
@@ -60,13 +182,28 @@ function _resolveOlpHome(opts) {
export function appendAuditEvent(event, opts = {}) { export function appendAuditEvent(event, opts = {}) {
const olpHome = _resolveOlpHome(opts); const olpHome = _resolveOlpHome(opts);
const logsDir = join(olpHome, 'logs'); const logsDir = join(olpHome, 'logs');
const path = join(logsDir, 'audit.ndjson'); const path = join(logsDir, LIVE_AUDIT_FILE);
const line = JSON.stringify(event) + '\n'; const line = JSON.stringify(event) + '\n';
const logEvent = opts.logEvent ?? ((level, ev, data) => { const logEvent = opts.logEvent ?? ((level, ev, data) => {
const entry = { ts: new Date().toISOString(), level, event: ev, ...(data ?? {}) }; const entry = { ts: new Date().toISOString(), level, event: ev, ...(data ?? {}) };
process.stderr.write(JSON.stringify(entry) + '\n'); process.stderr.write(JSON.stringify(entry) + '\n');
}); });
// D52: cheap fast-path date check. If the module-cached date matches the
// current UTC date, skip the rotation probe entirely (no disk I/O). If
// the date has changed, synchronously rotate BEFORE the append so the
// append lands in the (post-rotation) new live file rather than the
// about-to-rotate-away old one.
// Per ADR 0008 § 5.1: rotation fires "on the first append after a UTC
// date change." Synchronous rotation ensures no append straddles the
// boundary — old-date events land in the rotated file; new-date events
// land in the fresh live file. The cache flip happens INSIDE
// _maybeRotateAudit so concurrent in-process re-triggers short-circuit.
const today = _utcDateNow();
if (today !== _lastSeenUtcDate) {
_maybeRotateAudit({ olpHome, logEvent });
}
for (let attempt = 0; attempt <= RETRY_COUNT; attempt++) { for (let attempt = 0; attempt <= RETRY_COUNT; attempt++) {
try { try {
mkdirSync(logsDir, { recursive: true, mode: 0o700 }); mkdirSync(logsDir, { recursive: true, mode: 0o700 });
@@ -108,3 +245,38 @@ export function getAuditDropCount() {
export function __resetAuditDropCount() { export function __resetAuditDropCount() {
_dropCounter = 0; _dropCounter = 0;
} }
/**
* Per-process count of successful rotations performed. Test + future
* observability surface.
*/
export function getAuditRotateCount() {
return _rotateCounter;
}
/**
* Per-process count of failed rotation attempts (rename threw, target
* existed, etc.). Test + future observability surface.
*/
export function getAuditRotateFailCount() {
return _rotateFailCounter;
}
/**
* Test-only: reset the rotation counters + the in-process "last seen UTC
* date" cache so each test exercises a fresh code path.
*/
export function __resetAuditRotateState() {
_rotateCounter = 0;
_rotateFailCounter = 0;
_lastSeenUtcDate = _utcDateNow();
}
/**
* Test-only: force the cached "last seen UTC date" to a specific value
* so the next appendAuditEvent observes a date change and triggers
* rotation deterministically.
*/
export function __setLastSeenUtcDateForTesting(dateStr) {
_lastSeenUtcDate = dateStr;
}
+4 -2
View File
@@ -5,12 +5,14 @@
"type": "module", "type": "module",
"main": "server.mjs", "main": "server.mjs",
"bin": { "bin": {
"olp-keys": "./bin/olp-keys.mjs" "olp-keys": "./bin/olp-keys.mjs",
"olp-audit-rotate": "./bin/olp-audit-rotate.mjs"
}, },
"scripts": { "scripts": {
"start": "node server.mjs", "start": "node server.mjs",
"test": "node test-features.mjs", "test": "node test-features.mjs",
"olp-keys": "node bin/olp-keys.mjs" "olp-keys": "node bin/olp-keys.mjs",
"olp-audit-rotate": "node bin/olp-audit-rotate.mjs"
}, },
"engines": { "engines": {
"node": ">=18" "node": ">=18"
+233
View File
@@ -11726,3 +11726,236 @@ describe('Suite 25 — D51 dashboard.html UI smoke (Phase 3, ADR 0008 § 6)', ()
assert.match(r.body, /401.*owner-tier/i, 'dashboard error banner must mention owner-tier in 401 case'); assert.match(r.body, /401.*owner-tier/i, 'dashboard error banner must mention owner-tier in 401 case');
}); });
}); });
// ── Suite 26: D52 audit rotation (Phase 3, ADR 0008 § 5) ──────────────────
//
// Tests for daily audit rotation:
// - First append after UTC date change triggers rename
// - Concurrent appends during rotation produce exactly one rename
// - External cron tool (bin/olp-audit-rotate.mjs) coexists with in-server
// - Rotated files readable by lib/audit-query.mjs after rotation
import {
appendAuditEvent as appendAuditEventS26,
_maybeRotateAudit,
getAuditRotateCount,
__resetAuditRotateState,
__setLastSeenUtcDateForTesting,
} from './lib/audit.mjs';
import { runCli as runAuditRotateCli } from './bin/olp-audit-rotate.mjs';
describe('Suite 26 — D52 audit rotation (Phase 3, ADR 0008 § 5)', () => {
function writeAuditFile(tmp, filename, events) {
mkdirSync(_pathJoinForSetup(tmp, 'logs'), { recursive: true, mode: 0o700 });
const path = _pathJoinForSetup(tmp, 'logs', filename);
const lines = events.map(ev => JSON.stringify(ev)).join('\n') + '\n';
fsWriteFileSyncForS23(path, lines, { mode: 0o600 });
}
function makeEvent(ts) {
return {
ts,
key_id: 'k1',
owner_tier: 'owner',
method: 'POST',
path: '/v1/chat/completions',
provider: 'anthropic',
model: 'claude-sonnet-4-6',
status_code: 200,
latency_ms: 100,
cache_status: 'miss',
fallback_hops: 0,
tried_providers: ['anthropic'],
};
}
describe('26a — _maybeRotateAudit', () => {
let TMP;
before(() => { TMP = _mkdtempSyncForSetup(_pathJoinForSetup(_tmpdirForSetup(), 'olp-test-26a-')); });
after(() => { rmSync(TMP, { recursive: true, force: true }); __resetAuditRotateState(); });
it('26a-1: no live file → no rotation, returns { rotated: false }', () => {
__resetAuditRotateState();
const r = _maybeRotateAudit({ olpHome: TMP });
assert.equal(r.rotated, false);
});
it('26a-2: live file already on today → no rotation', () => {
__resetAuditRotateState();
const today = new Date().toISOString().slice(0, 10);
writeAuditFile(TMP, 'audit.ndjson', [makeEvent(`${today}T10:00:00Z`)]);
const r = _maybeRotateAudit({ olpHome: TMP });
assert.equal(r.rotated, false);
});
it('26a-3: live file holds yesterday\'s events → rotated to audit-YYYY-MM-DD.ndjson', () => {
__resetAuditRotateState();
const livePath = _pathJoinForSetup(TMP, 'logs', 'audit.ndjson');
rmSync(livePath, { force: true });
const yesterday = new Date(Date.now() - 86400 * 1000).toISOString().slice(0, 10);
writeAuditFile(TMP, 'audit.ndjson', [makeEvent(`${yesterday}T15:00:00Z`)]);
const r = _maybeRotateAudit({ olpHome: TMP });
assert.equal(r.rotated, true);
assert.equal(r.dateUsed, yesterday);
const rotatedPath = _pathJoinForSetup(TMP, 'logs', `audit-${yesterday}.ndjson`);
assert.ok(fsExistsSync(rotatedPath));
assert.ok(!fsExistsSync(livePath));
assert.equal(getAuditRotateCount(), 1);
});
it('26a-4: idempotent — second call after rotation is a no-op', () => {
const r = _maybeRotateAudit({ olpHome: TMP });
assert.equal(r.rotated, false);
assert.equal(getAuditRotateCount(), 1);
});
it('26a-5: rotation target already exists → skip + warn (cron-race safety)', () => {
__resetAuditRotateState();
const livePath = _pathJoinForSetup(TMP, 'logs', 'audit.ndjson');
rmSync(livePath, { force: true });
const yesterday = new Date(Date.now() - 86400 * 1000).toISOString().slice(0, 10);
writeAuditFile(TMP, `audit-${yesterday}.ndjson`, [makeEvent(`${yesterday}T08:00:00Z`)]);
writeAuditFile(TMP, 'audit.ndjson', [makeEvent(`${yesterday}T20:00:00Z`)]);
let warned = false;
const r = _maybeRotateAudit({
olpHome: TMP,
logEvent: (level, ev) => { if (level === 'warn' && ev === 'audit_rotate_target_exists') warned = true; },
});
assert.equal(r.rotated, false);
assert.ok(warned, 'must warn when rotation target already exists');
assert.ok(fsExistsSync(livePath));
});
});
describe('26b — appendAuditEvent triggers rotation on date change', () => {
let TMP;
before(() => { TMP = _mkdtempSyncForSetup(_pathJoinForSetup(_tmpdirForSetup(), 'olp-test-26b-')); });
after(() => { rmSync(TMP, { recursive: true, force: true }); __resetAuditRotateState(); });
it('26b-1: appending past a UTC date change triggers sync rotation + append lands in new file', () => {
__resetAuditRotateState();
const livePath = _pathJoinForSetup(TMP, 'logs', 'audit.ndjson');
rmSync(livePath, { force: true });
const yesterday = new Date(Date.now() - 86400 * 1000).toISOString().slice(0, 10);
writeAuditFile(TMP, 'audit.ndjson', [makeEvent(`${yesterday}T10:00:00Z`)]);
__setLastSeenUtcDateForTesting(yesterday);
const todayTs = new Date().toISOString();
appendAuditEventS26(makeEvent(todayTs), { olpHome: TMP });
const rotatedPath = _pathJoinForSetup(TMP, 'logs', `audit-${yesterday}.ndjson`);
assert.ok(fsExistsSync(rotatedPath), `yesterday's rotated file must exist (${rotatedPath})`);
assert.ok(fsExistsSync(livePath), 'live file must exist (with today\'s new event)');
const liveContent = fsReadFileSync(livePath, 'utf-8');
assert.ok(liveContent.includes(todayTs), 'live file must contain today\'s appended event');
assert.ok(!liveContent.includes(`${yesterday}T10:00:00Z`), 'live file must NOT contain yesterday\'s content (it rotated)');
assert.equal(getAuditRotateCount(), 1);
});
});
describe('26c — concurrent appends during rotation', () => {
let TMP;
before(() => { TMP = _mkdtempSyncForSetup(_pathJoinForSetup(_tmpdirForSetup(), 'olp-test-26c-')); });
after(() => { rmSync(TMP, { recursive: true, force: true }); __resetAuditRotateState(); });
it('26c-1: N sequential appendAuditEvent during date change → exactly 1 rotation + all events land in live', () => {
__resetAuditRotateState();
const livePath = _pathJoinForSetup(TMP, 'logs', 'audit.ndjson');
rmSync(livePath, { force: true });
const yesterday = new Date(Date.now() - 86400 * 1000).toISOString().slice(0, 10);
writeAuditFile(TMP, 'audit.ndjson', [makeEvent(`${yesterday}T10:00:00Z`)]);
__setLastSeenUtcDateForTesting(yesterday);
// 10 sync appends in quick succession. The first triggers rotation
// (date change detected); subsequent appends short-circuit the date
// check because _lastSeenUtcDate was updated inside the first call's
// _maybeRotateAudit. Result: exactly 1 rotation + all 10 events in
// the new live file.
const todayTs = new Date().toISOString();
for (let i = 0; i < 10; i++) {
appendAuditEventS26(makeEvent(todayTs), { olpHome: TMP });
}
assert.equal(getAuditRotateCount(), 1, 'exactly 1 rotation despite 10 sequential appends past date change');
const rotatedPath = _pathJoinForSetup(TMP, 'logs', `audit-${yesterday}.ndjson`);
assert.ok(fsExistsSync(rotatedPath));
const liveContent = fsReadFileSync(livePath, 'utf-8');
const liveLines = liveContent.split('\n').filter(Boolean);
assert.equal(liveLines.length, 10, 'live file has all 10 appended events');
});
});
describe('26d — bin/olp-audit-rotate.mjs CLI', () => {
let TMP;
before(() => { TMP = _mkdtempSyncForSetup(_pathJoinForSetup(_tmpdirForSetup(), 'olp-test-26d-')); });
after(() => { rmSync(TMP, { recursive: true, force: true }); __resetAuditRotateState(); });
it('26d-1: --help → exit 0 with usage', async () => {
let out = '';
const code = await runAuditRotateCli(['--help'], { out: s => { out += s; }, err: () => {} });
assert.equal(code, 0);
assert.match(out, /OLP audit rotation cron tool/);
});
it('26d-2: no live file → exit 0 "no rotation needed"', async () => {
__resetAuditRotateState();
let out = '';
const code = await runAuditRotateCli([`--olp-home=${TMP}`], { out: s => { out += s; }, err: () => {} });
assert.equal(code, 0);
assert.match(out, /No rotation needed/);
});
it('26d-3: live file with yesterday events → CLI rotates + exit 0', async () => {
__resetAuditRotateState();
const yesterday = new Date(Date.now() - 86400 * 1000).toISOString().slice(0, 10);
writeAuditFile(TMP, 'audit.ndjson', [makeEvent(`${yesterday}T10:00:00Z`)]);
let out = '';
const code = await runAuditRotateCli([`--olp-home=${TMP}`], { out: s => { out += s; }, err: () => {} });
assert.equal(code, 0);
assert.match(out, /Rotated.*dateUsed=/);
const rotatedPath = _pathJoinForSetup(TMP, 'logs', `audit-${yesterday}.ndjson`);
assert.ok(fsExistsSync(rotatedPath));
});
it('26d-4: unknown flag → exit 1', async () => {
let err = '';
const code = await runAuditRotateCli([`--olp-home=${TMP}`, '--unknown'], {
out: () => {}, err: s => { err += s; },
});
assert.equal(code, 1);
assert.match(err, /unknown flag --unknown/);
});
});
describe('26e — rotated files queryable via lib/audit-query.mjs', () => {
let TMP;
before(() => { TMP = _mkdtempSyncForSetup(_pathJoinForSetup(_tmpdirForSetup(), 'olp-test-26e-')); });
after(() => { rmSync(TMP, { recursive: true, force: true }); __resetAuditRotateState(); });
it('26e-1: after rotation, discoverAuditFiles + readAuditWindow see both live + rotated', () => {
__resetAuditRotateState();
const yesterday = new Date(Date.now() - 86400 * 1000).toISOString().slice(0, 10);
const today = new Date().toISOString().slice(0, 10);
writeAuditFile(TMP, 'audit.ndjson', [
makeEvent(`${yesterday}T10:00:00Z`),
makeEvent(`${yesterday}T11:00:00Z`),
]);
__setLastSeenUtcDateForTesting(yesterday);
// Sync rotation: completes before append returns
appendAuditEventS26(makeEvent(`${today}T05:00:00Z`), { olpHome: TMP });
const files = discoverAuditFiles({ olpHome: TMP });
assert.ok(files.has('live'), 'live file present');
assert.ok(files.has(yesterday), `rotated file for ${yesterday} present`);
const startMs = Date.parse(`${yesterday}T00:00:00Z`);
const endMs = Date.parse(`${today}T23:59:59Z`);
const events = [...readAuditWindow({ startMs, endMs, olpHome: TMP })];
assert.equal(events.length, 3, 'cross-file read yields yesterday(2) + today(1) = 3 events');
});
});
});