mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-19 09:45:07 +00:00
release(v0.5.1): hotfix — quota probe cache/backoff/schema-drift correctness (codex review) (#58)
* release(v0.5.1): hotfix — quota probe cache/backoff/schema-drift correctness (codex review) Addresses three production-quality findings from codex's post-v0.5.0 review (codex review on PR #57, reproduced with local mocks): F1 [P1] — Doctor bypass of cache + backoff (ADR 0013 Rule 3) `anthropic.quota_probe_reachable` called `_probeOnce(auth)` directly, bypassing `quotaProbeState.backoffUntil`. Successive `olp doctor` invocations within a backoff window each hit upstream — violating ADR 0013 Rule 3 (backoff is mandatory for ALL consumers). Fix: doctor now routes through `quotaStatus()`. ADR 0013 Rule 3 clarified: "All consumers of `quotaStatus()`, including `olp doctor` checks, MUST route through `quotaStatus()` and MUST NOT call `_probeOnce()` directly." F2 [P2] — 200 with empty ratelimit headers cached as live data (ADR 0013 Rule 5) A 200 OK with zero `anthropic-ratelimit-*` headers was cached as `stale: false` (live). Minimum-viable-schema gate added to `_probeOnce`: requires 5h-utilization + 5h-reset + 7d-utilization + 7d-reset present; absence → `failureKind: 'schema_drift'`, backoff scheduled, result NOT cached. ADR 0013 Rule 5 updated with the gate specification. F3 [P2] — Dashboard-data loses failure detail (ADR 0013 Rule 6) `aggregateProviderQuota()` collapsed all failure modes into `status: 'unavailable'` ("no public quota api or probe disabled") — same as providers with no API at all. Fix: `quotaStatus()` v0.5.1 contract — `null` ONLY for opt-in-off; failures return `{ probe_status: 'unreachable', failure: { kind, message, backoff_until? } }`. New `failure_kind` enum: no_credentials | auth_failed | rate_limited | schema_drift | network | other. Dashboard renders `unreachable` with red border + failure detail. Authority: ADR 0013 Rules 3, 5, 6 (cache + backoff + schema-drift + failure transparency) ADR 0008 Amendment 2 (richer quota_v2 shape; new unreachable status) ADR 0002 Amendment 8 unchanged (constitutional permission for the probe) Codex review findings F1–F3 (codex on PR #57) Changes: - lib/providers/anthropic.mjs: quotaProbeState gains lastError + failureKind; _probeOnce: min-field gate + failureKind population; quotaStatus(): v0.5.1 contract (null=disabled only; probe_status:live/stale/unreachable); doctorChecks routes through quotaStatus(); reset functions updated - lib/audit-query.mjs: _normalizeAnthropicQuota handles probe_status field; aggregateProviderQuota emits failure/failure_kind/backoff_until; unreachable status - dashboard.html: unreachable CSS classes + render path + footer v0.5.1 - test-features.mjs: 38f/j/l updated for v0.5.1 shape; 38r refactored for F1; 38g/k gain probe_status assertions; 38u/v/w new regression tests; 756→759 tests - docs/adr/0008: Amendment 2 (richer ProviderQuotaEntry + quotaStatus contract) - docs/adr/0013: Rule 3 clarification (doctor must use quotaStatus); Rule 5 min-viable-schema gate specification - package.json: 0.5.0 → 0.5.1 - CHANGELOG.md: v0.5.1 hotfix entry promoted from Unreleased - README.md / AGENTS.md: Phase 5 closed at v0.5.1; Phase 6 next Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs+code: PR #58 fold-in — reviewer Nits #1 + #2 (ADR doc-drift + opt_in_off enum) Fresh-context reviewer (PR #58) verdict: APPROVE_WITH_MINOR, 0 blocking, 4 nits. Folding in #1 + #2 (both 1-line cosmetic-but-correct fixes). Deferring #3 (test-only edge case in module-level seam restore) and #4 (pre-existing codex F4 — bin/olp.mjs + olp-plugin still read legacy quota shape; separate PR planned). Nit #1 — ADR 0002 Amendment 8 documentation drift. Amendment 8 at v0.5.0 line 20 said "the function returns `null` rather than throwing", and line 33 said "stale-cache-on-failure (`null` is returned only when no cache entry exists; if a stale entry exists it's returned with a `stale: true` marker)". v0.5.1 refined this contract: `null` is now reserved STRICTLY for opt-in-off, and all failure modes return `{ probe_status: 'unreachable' | 'stale', failure: {...} }`. The substantive idempotent-failure constraint (no throw to caller) is unchanged. The operational description in Amendment 8 was stale — fixed to cross-reference ADR 0008 Amendment 2 + ADR 0013 Rule 6 for the v0.5.1 contract refinement. Also references Suite 38 (38u/38v/38w) as the regression coverage producing the new shape. Nit #2 — `failureKind: 'opt_in_off'` declared but never produced. The enum value was listed in both the code comment (anthropic.mjs:250) and ADR 0008 Amendment 2 (line 30) but never actually assigned — because when opt-in is off, `quotaStatus()` returns the literal `null` BEFORE any state mutation happens. The enum value was dead. Fix: removed `opt_in_off` from both enum declarations + added an inline note explaining that consumers (audit-query, doctor) distinguish opt-in-off by checking `quotaStatus() === null`, not via failureKind. Deferred: - Nit #3 (test-seam restore edge case): if a caller pre-sets `_quotaAuthReadFnForTest` AND passes a non-default `_authReadFn`, the finally block restores to null clobbering pre-set value. Test-only impact, no production risk. Pure hygiene; defer. - Nit #4 (codex F4): bin/olp.mjs cmdUsage and olp-plugin/index.js still read legacy `body.quota` field, never consume `quota_v2`. Reviewer confirmed neither crashes — both gracefully fall through to "no quota api" branch. Out of scope for this hotfix per the hotfix dispatch contract; separate PR will migrate them. Tests: 759/759 still pass post-fold-in. No test changes needed. Authority: PR #58 review thread + ADR 0013 Rule 6 (failure transparency) + ADR 0008 Amendment 2 (ProviderQuotaEntry v0.5.1 shape). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+34
-7
@@ -449,19 +449,26 @@ export function spendTrendDaily({ days, olpHome, logEvent, _nowFn } = {}) {
|
||||
* Provider-specific: called only for 'anthropic'. Returns null if the raw
|
||||
* shape is absent or malformed.
|
||||
*
|
||||
* v0.5.1: handles probe_status field (F3 — ADR 0013 Rule 6).
|
||||
* Accepts both old shape (stale: boolean) and new shape (probe_status: string).
|
||||
*
|
||||
* @internal — used by aggregateProviderQuota()
|
||||
*/
|
||||
function _normalizeAnthropicQuota(raw) {
|
||||
if (!raw || typeof raw !== 'object') return null;
|
||||
const f = raw.fields ?? {};
|
||||
// v0.5.1: probe_status field (new) takes precedence; fall back to stale bool for compat.
|
||||
const probeStatus = raw.probe_status ?? (raw.stale === true ? 'stale' : 'live');
|
||||
return {
|
||||
schema_version: raw.schemaVersion ?? null,
|
||||
last_fresh_at: raw.stale ? (raw.last_fresh_at ?? null) : (raw.probedAt ?? null),
|
||||
utilization: {
|
||||
last_fresh_at: (probeStatus === 'stale')
|
||||
? (raw.last_fresh_at ?? null)
|
||||
: (raw.probedAt ?? null),
|
||||
utilization: probeStatus === 'unreachable' ? null : {
|
||||
'5h': f.utilization_5h ?? null,
|
||||
'7d': f.utilization_7d ?? null,
|
||||
},
|
||||
reset: {
|
||||
reset: probeStatus === 'unreachable' ? null : {
|
||||
'5h': f.reset_5h ?? null,
|
||||
'7d': f.reset_7d ?? null,
|
||||
overall: f.reset ?? null,
|
||||
@@ -469,11 +476,15 @@ function _normalizeAnthropicQuota(raw) {
|
||||
},
|
||||
representative_claim: f.representative_claim ?? null,
|
||||
fallback_percentage: f.fallback_percentage ?? null,
|
||||
overage: {
|
||||
overage: probeStatus === 'unreachable' ? null : {
|
||||
status: f.overage_status ?? null,
|
||||
disabled_reason: f.overage_disabled_reason ?? null,
|
||||
},
|
||||
raw_available: (typeof raw.raw === 'object' && raw.raw !== null),
|
||||
// v0.5.1 (F3 — ADR 0013 Rule 6): failure detail for operator diagnostics
|
||||
failure: raw.failure ?? null,
|
||||
failure_kind: raw.failure?.kind ?? null,
|
||||
backoff_until: raw.failure?.backoff_until ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -556,7 +567,8 @@ export async function aggregateProviderQuota({
|
||||
}
|
||||
|
||||
if (rawResult === null || rawResult === undefined) {
|
||||
// Plugin returned null: either disabled, no quota API, or no credentials.
|
||||
// Plugin returned null: opt-in disabled (the ONLY case per v0.5.1 contract)
|
||||
// or providers with no quota API at all (codex, mistral).
|
||||
results.push({
|
||||
provider: name,
|
||||
status: 'unavailable',
|
||||
@@ -569,14 +581,18 @@ export async function aggregateProviderQuota({
|
||||
fallback_percentage: null,
|
||||
overage: null,
|
||||
raw_available: false,
|
||||
failure: null,
|
||||
failure_kind: null,
|
||||
backoff_until: null,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
// quotaStatus() returned a non-null shape — normalize.
|
||||
// v0.5.1: handle probe_status field (live/stale/unreachable).
|
||||
// Currently only 'anthropic' returns a structured shape; other providers
|
||||
// returning structured data will work if their shape is compatible.
|
||||
const isStale = rawResult.stale === true;
|
||||
const probeStatus = rawResult.probe_status ?? (rawResult.stale === true ? 'stale' : 'live');
|
||||
const normalized = _normalizeAnthropicQuota(rawResult);
|
||||
|
||||
if (normalized === null) {
|
||||
@@ -593,13 +609,24 @@ export async function aggregateProviderQuota({
|
||||
fallback_percentage: null,
|
||||
overage: null,
|
||||
raw_available: false,
|
||||
failure: null,
|
||||
failure_kind: null,
|
||||
backoff_until: null,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
// Map probe_status to output status:
|
||||
// 'live' → 'live'
|
||||
// 'stale' → 'stale'
|
||||
// 'unreachable' → 'unreachable' (new in v0.5.1; dashboard renders with red border)
|
||||
const outputStatus = probeStatus === 'unreachable' ? 'unreachable'
|
||||
: probeStatus === 'stale' ? 'stale'
|
||||
: 'live';
|
||||
|
||||
results.push({
|
||||
provider: name,
|
||||
status: isStale ? 'stale' : 'live',
|
||||
status: outputStatus,
|
||||
...normalized,
|
||||
});
|
||||
}
|
||||
|
||||
+244
-81
@@ -182,6 +182,9 @@ export function _resetQuotaProbeStateForTest() {
|
||||
quotaProbeState.cache = null;
|
||||
quotaProbeState.backoffUntil = 0;
|
||||
quotaProbeState.backoffMs = QUOTA_BACKOFF_MIN_MS;
|
||||
// v0.5.1 (F3): reset failure-tracking fields
|
||||
quotaProbeState.lastError = null;
|
||||
quotaProbeState.failureKind = null;
|
||||
}
|
||||
|
||||
// Convenience: reset state only, leaving URL overrides in place.
|
||||
@@ -190,6 +193,9 @@ export function _resetQuotaStateOnlyForTest() {
|
||||
quotaProbeState.cache = null;
|
||||
quotaProbeState.backoffUntil = 0;
|
||||
quotaProbeState.backoffMs = QUOTA_BACKOFF_MIN_MS;
|
||||
// v0.5.1 (F3): reset failure-tracking fields
|
||||
quotaProbeState.lastError = null;
|
||||
quotaProbeState.failureKind = null;
|
||||
}
|
||||
|
||||
// Returns a direct reference to the internal quotaProbeState object.
|
||||
@@ -234,10 +240,18 @@ function _resolveSchemaVersion() {
|
||||
|
||||
// Module-level probe state — one instance per process (not per request).
|
||||
// Ported from OCP server.mjs:852 (usageCache) + 862 (oauthRefreshBackoff).
|
||||
// v0.5.1: added lastError + failureKind for F3 failure-transparency (ADR 0013 Rule 6).
|
||||
const quotaProbeState = {
|
||||
cache: null, // { fetchedAt: <ms>, data: <quotaShape> } when fresh; null when not
|
||||
backoffUntil: 0, // ms epoch: when next refresh attempt is allowed
|
||||
backoffMs: QUOTA_BACKOFF_MIN_MS, // current backoff window; doubled on failure, reset on success
|
||||
// v0.5.1 (F3 — ADR 0013 Rule 6 failure transparency):
|
||||
lastError: null, // { kind, message, statusCode?, attemptedAt } | null
|
||||
failureKind: null, // 'no_credentials'|'auth_failed'|'rate_limited'|'schema_drift'|'network'|'other' | null
|
||||
// Note: 'opt_in_off' is NOT in this enum — when the probe is opted out
|
||||
// quotaStatus() returns the literal null BEFORE any state mutation, so
|
||||
// failureKind is never produced for the disabled case. Consumers
|
||||
// (audit-query, doctor) distinguish opt-in-off by quotaStatus() === null.
|
||||
};
|
||||
|
||||
// ── Config reader: providers.<name>.<field> ───────────────────────────────
|
||||
@@ -468,21 +482,56 @@ async function _probeOnce(creds) {
|
||||
}
|
||||
|
||||
// Non-2xx AND no ratelimit headers → treat as failure
|
||||
// Classify failure kind for F3 (ADR 0013 Rule 6 failure transparency).
|
||||
if (statusCode >= 400 && Object.keys(rlHeaders).length === 0) {
|
||||
// Schedule backoff on failure
|
||||
const now = Date.now();
|
||||
const kind = (statusCode === 401 || statusCode === 403) ? 'auth_failed'
|
||||
: statusCode === 429 ? 'rate_limited'
|
||||
: 'other';
|
||||
quotaProbeState.lastError = { kind, message: `HTTP ${statusCode}`, statusCode, attemptedAt: now };
|
||||
quotaProbeState.failureKind = kind;
|
||||
_scheduleBackoff();
|
||||
return null;
|
||||
}
|
||||
|
||||
// Success (or non-2xx with headers present — headers still valid quota data)
|
||||
// Reset backoff on successful probe
|
||||
// v0.5.1 F2 (ADR 0013 Rule 5 minimum-viable-schema gate):
|
||||
// Require at least these 4 load-bearing fields (5h-utilization, 5h-reset,
|
||||
// 7d-utilization, 7d-reset). A 200 OK with zero ratelimit headers (or with
|
||||
// headers but missing these 4) is treated as schema-drift and classified as
|
||||
// failure. The other 9 fields are tolerated as missing.
|
||||
//
|
||||
// Authority: ADR 0013 Rule 5 + codex review finding F2 (v0.5.1 hotfix).
|
||||
const MIN_FIELDS = [
|
||||
'anthropic-ratelimit-unified-5h-utilization',
|
||||
'anthropic-ratelimit-unified-5h-reset',
|
||||
'anthropic-ratelimit-unified-7d-utilization',
|
||||
'anthropic-ratelimit-unified-7d-reset',
|
||||
];
|
||||
const missingMinFields = MIN_FIELDS.filter(f => rlHeaders[f] == null);
|
||||
if (missingMinFields.length > 0) {
|
||||
const now = Date.now();
|
||||
const msg = `schema_drift: missing minimum-viable fields: ${missingMinFields.join(', ')}`;
|
||||
quotaProbeState.lastError = { kind: 'schema_drift', message: msg, statusCode, attemptedAt: now };
|
||||
quotaProbeState.failureKind = 'schema_drift';
|
||||
_scheduleBackoff();
|
||||
return null;
|
||||
}
|
||||
|
||||
// Success (or non-2xx with headers present and passing min-field gate —
|
||||
// headers still valid quota data). Reset backoff on successful probe.
|
||||
quotaProbeState.backoffMs = QUOTA_BACKOFF_MIN_MS;
|
||||
quotaProbeState.backoffUntil = 0;
|
||||
quotaProbeState.lastError = null;
|
||||
quotaProbeState.failureKind = null;
|
||||
|
||||
const fields = _parseRateLimitHeaders(rlHeaders);
|
||||
return { fields, raw: rlHeaders };
|
||||
|
||||
} catch {
|
||||
} catch (err) {
|
||||
const now = Date.now();
|
||||
const msg = err?.message ?? String(err);
|
||||
quotaProbeState.lastError = { kind: 'network', message: msg, attemptedAt: now };
|
||||
quotaProbeState.failureKind = 'network';
|
||||
_scheduleBackoff();
|
||||
return null;
|
||||
}
|
||||
@@ -817,46 +866,71 @@ export function estimateCost(request) {
|
||||
// Port source: OCP server.mjs:842-1109 (usageCache, refreshOAuthToken, fetchUsageFromApi,
|
||||
// parseRateLimitHeaders, handleUsage)
|
||||
//
|
||||
// ADR 0013 Rule 4 (opt-in): returns null unless config.providers.anthropic.quota_probe_enabled
|
||||
// is explicitly true in ~/.olp/config.json.
|
||||
// ADR 0013 Rule 4 (opt-in): returns null ONLY when quota_probe_enabled is false.
|
||||
// All other failure paths return a structured failure shape instead of null.
|
||||
//
|
||||
// Return shape (when enabled + successful):
|
||||
// {
|
||||
// probedAt: <unix-ms>,
|
||||
// source: 'anthropic-ratelimit-unified-headers',
|
||||
// schemaVersion: '2026-05-26',
|
||||
// stale: false,
|
||||
// fields: { ...13 fields... },
|
||||
// raw: { ...response-headers-object... }
|
||||
// }
|
||||
// Returns null when: disabled, no credentials, probe failure with no stale cache.
|
||||
// Returns { ...shape, stale: true, last_fresh_at } when probe failed but cache exists.
|
||||
// v0.5.1 return contract (F1+F3 — ADR 0013 Rules 3+6 + codex review):
|
||||
//
|
||||
// null → ONLY when quota_probe_enabled: false (opt-in off)
|
||||
//
|
||||
// { probe_status: 'live', probedAt, source, schemaVersion,
|
||||
// stale: false, fields, raw }
|
||||
// → probe succeeded recently (within QUOTA_CACHE_TTL_MS)
|
||||
//
|
||||
// { probe_status: 'stale', ...cachedShape, stale: true, last_fresh_at,
|
||||
// failure: { kind, message, backoff_until } }
|
||||
// → probe failed but cache exists (backoff active)
|
||||
//
|
||||
// { probe_status: 'unreachable', source, schemaVersion,
|
||||
// failure: { kind, message, backoff_until? } }
|
||||
// → probe failed AND no cache (includes no_credentials case when
|
||||
// creds absent after opt-in, also in-backoff-no-cache)
|
||||
//
|
||||
// Backwards-compat note: stale:false shape gains probe_status:'live' (additive).
|
||||
// stale:true shape gains probe_status:'stale' (additive). 'unreachable' replaces
|
||||
// the prior null for failure cases (breaking only if caller checked null for
|
||||
// "disabled" — now null strictly means disabled only).
|
||||
export async function quotaStatus(_authContext) {
|
||||
// ADR 0013 Rule 4 — opt-in check
|
||||
// ADR 0013 Rule 4 — opt-in check; null means "disabled", not "failed"
|
||||
const providerCfg = _readProviderConfig('anthropic');
|
||||
if (providerCfg.quota_probe_enabled !== true) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const schemaVersion = _resolveSchemaVersion();
|
||||
const now = Date.now();
|
||||
|
||||
// Cache hit: return cached value if within TTL (ADR 0013 Rule 3)
|
||||
if (quotaProbeState.cache !== null &&
|
||||
(now - quotaProbeState.cache.fetchedAt) < QUOTA_CACHE_TTL_MS) {
|
||||
return quotaProbeState.cache.data;
|
||||
// Cache is fresh — return probe_status:'live' shape
|
||||
return { ...quotaProbeState.cache.data, probe_status: 'live' };
|
||||
}
|
||||
|
||||
// Backoff check: still in exponential backoff window — return stale cache or null
|
||||
// Backoff check: still in exponential backoff window — return stale cache or unreachable
|
||||
if (now < quotaProbeState.backoffUntil) {
|
||||
const failureInfo = {
|
||||
kind: quotaProbeState.failureKind ?? 'other',
|
||||
message: quotaProbeState.lastError?.message ?? 'probe failed (in backoff)',
|
||||
backoff_until: quotaProbeState.backoffUntil,
|
||||
};
|
||||
if (quotaProbeState.cache !== null) {
|
||||
// Return stale cache marked as stale (ADR 0013 Rule 3)
|
||||
return {
|
||||
...quotaProbeState.cache.data,
|
||||
probe_status: 'stale',
|
||||
stale: true,
|
||||
last_fresh_at: quotaProbeState.cache.fetchedAt,
|
||||
failure: failureInfo,
|
||||
};
|
||||
}
|
||||
return null; // no stale cache
|
||||
// In backoff but no cache — unreachable
|
||||
return {
|
||||
probe_status: 'unreachable',
|
||||
source: 'anthropic-ratelimit-unified-headers',
|
||||
schemaVersion,
|
||||
failure: failureInfo,
|
||||
};
|
||||
}
|
||||
|
||||
// Read auth artifact (ADR 0013 Rule 1 — same credentials as spawn path).
|
||||
@@ -864,8 +938,25 @@ export async function quotaStatus(_authContext) {
|
||||
const _authReadFn = _quotaAuthReadFnForTest ?? readAuthArtifact;
|
||||
const creds = _authReadFn();
|
||||
if (!creds?.accessToken) {
|
||||
// No credentials at all — return stale cache or null; do not hammer API with 401s
|
||||
return null;
|
||||
// No credentials — set failureKind so doctor/aggregator can distinguish this
|
||||
quotaProbeState.lastError = { kind: 'no_credentials', message: 'no OAuth credential found', attemptedAt: now };
|
||||
quotaProbeState.failureKind = 'no_credentials';
|
||||
if (quotaProbeState.cache !== null) {
|
||||
// Stale cache still available — return it with failure info
|
||||
return {
|
||||
...quotaProbeState.cache.data,
|
||||
probe_status: 'stale',
|
||||
stale: true,
|
||||
last_fresh_at: quotaProbeState.cache.fetchedAt,
|
||||
failure: { kind: 'no_credentials', message: 'no OAuth credential found' },
|
||||
};
|
||||
}
|
||||
return {
|
||||
probe_status: 'unreachable',
|
||||
source: 'anthropic-ratelimit-unified-headers',
|
||||
schemaVersion,
|
||||
failure: { kind: 'no_credentials', message: 'no OAuth credential found' },
|
||||
};
|
||||
}
|
||||
|
||||
// Perform the probe
|
||||
@@ -878,7 +969,8 @@ export async function quotaStatus(_authContext) {
|
||||
source: 'anthropic-ratelimit-unified-headers',
|
||||
// D81: schema_version sourced from models-registry.json with QUOTA_SCHEMA_VERSION
|
||||
// as fallback per ADR 0013 Rule 5 + D81 ADR 0008 Amendment requirement.
|
||||
schemaVersion: _resolveSchemaVersion(),
|
||||
schemaVersion,
|
||||
probe_status: 'live',
|
||||
stale: false,
|
||||
fields: probeResult.fields,
|
||||
raw: probeResult.raw,
|
||||
@@ -887,16 +979,29 @@ export async function quotaStatus(_authContext) {
|
||||
return shape;
|
||||
}
|
||||
|
||||
// Probe failed: _probeOnce already called _scheduleBackoff().
|
||||
// Return stale cache if present, else null.
|
||||
// Probe failed: _probeOnce already called _scheduleBackoff() and set lastError/failureKind.
|
||||
const failureInfo = {
|
||||
kind: quotaProbeState.failureKind ?? 'other',
|
||||
message: quotaProbeState.lastError?.message ?? 'probe failed',
|
||||
backoff_until: quotaProbeState.backoffUntil,
|
||||
};
|
||||
|
||||
if (quotaProbeState.cache !== null) {
|
||||
return {
|
||||
...quotaProbeState.cache.data,
|
||||
probe_status: 'stale',
|
||||
stale: true,
|
||||
last_fresh_at: quotaProbeState.cache.fetchedAt,
|
||||
failure: failureInfo,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
// No cache — return unreachable shape (F3: operator can distinguish failure modes)
|
||||
return {
|
||||
probe_status: 'unreachable',
|
||||
source: 'anthropic-ratelimit-unified-headers',
|
||||
schemaVersion,
|
||||
failure: failureInfo,
|
||||
};
|
||||
}
|
||||
|
||||
// ── healthCheck ───────────────────────────────────────────────────────────
|
||||
@@ -999,73 +1104,131 @@ export function doctorChecks({ _binaryExistsFn, _authReadFn } = {}) {
|
||||
{
|
||||
id: 'anthropic.quota_probe_reachable',
|
||||
category: 'provider',
|
||||
// D80 / ADR 0013 Rule 6 — failure transparency for olp doctor.
|
||||
// Only runs if quota_probe_enabled: true in ~/.olp/config.json.
|
||||
// Status mapping:
|
||||
// ok → probe returned fresh data
|
||||
// warn → stale cache returned (probe failed but cache present)
|
||||
// fail → no cache AND probe failed; fix_commands recipe provided
|
||||
// v0.5.1 (F1 — ADR 0013 Rule 3 + codex review):
|
||||
// Routes through quotaStatus() so it respects cache + backoff.
|
||||
// Bypassing _probeOnce() directly (old code) allowed successive `olp doctor`
|
||||
// invocations to each hit upstream even when backoffUntil was in the future.
|
||||
//
|
||||
// Status mapping based on quotaStatus() probe_status:
|
||||
// null (opt-in off) → ok + advisory
|
||||
// probe_status: 'live' → ok + utilization data in message
|
||||
// probe_status: 'stale' → warn + age info + failure kind in message
|
||||
// probe_status: 'unreachable' (no_credentials) → fail + human_steps (re-login)
|
||||
// probe_status: 'unreachable' (auth_failed/rate_limited/schema_drift/network/other)
|
||||
// → fail + human_steps + fix_commands
|
||||
async run() {
|
||||
const providerCfg = _readProviderConfig('anthropic');
|
||||
if (providerCfg.quota_probe_enabled !== true) {
|
||||
return { status: 'ok', message: 'quota probe disabled (opt-in via config.providers.anthropic.quota_probe_enabled)' };
|
||||
// Inject the test auth override so tests can seed the quotaAuthReadFn.
|
||||
// In production _quotaAuthReadFnForTest is null so quotaStatus() reads
|
||||
// credentials normally. Tests can set it via _setQuotaAuthReadFnForTest.
|
||||
if (authRead !== readAuthArtifact) {
|
||||
_quotaAuthReadFnForTest = authRead;
|
||||
}
|
||||
try {
|
||||
const result = await quotaStatus(null);
|
||||
|
||||
// Attempt a fresh probe (bypassing module-level cache for doctor diagnostics)
|
||||
const auth = authRead();
|
||||
if (!auth?.accessToken) {
|
||||
// null → opt-in off
|
||||
if (result === null) {
|
||||
return { status: 'ok', message: 'quota probe disabled (opt-in via config.providers.anthropic.quota_probe_enabled)' };
|
||||
}
|
||||
|
||||
const probeStatus = result.probe_status;
|
||||
|
||||
if (probeStatus === 'live') {
|
||||
const util5h = result.fields?.utilization_5h;
|
||||
const util7d = result.fields?.utilization_7d;
|
||||
const msg = `quota probe OK — 5h utilization: ${util5h !== null && util5h !== undefined ? `${Math.round(util5h * 100)}%` : 'n/a'}, 7d utilization: ${util7d !== null && util7d !== undefined ? `${Math.round(util7d * 100)}%` : 'n/a'}`;
|
||||
return { status: 'ok', message: msg };
|
||||
}
|
||||
|
||||
if (probeStatus === 'stale') {
|
||||
const ageMin = result.last_fresh_at
|
||||
? Math.round((Date.now() - result.last_fresh_at) / 60_000)
|
||||
: null;
|
||||
const failKind = result.failure?.kind ?? 'other';
|
||||
const ageStr = ageMin !== null ? ` from ${ageMin}min ago` : '';
|
||||
return {
|
||||
status: 'warn',
|
||||
message: `quota probe failed (${failKind}); returning stale cache${ageStr}; check network or token expiry`,
|
||||
evidence: {
|
||||
fix_commands: [
|
||||
'olp doctor # re-run after verifying OAuth credentials are valid',
|
||||
],
|
||||
human_steps: [
|
||||
'run: claude (if token may have expired, re-login)',
|
||||
],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// probe_status === 'unreachable' — distinguish failure kind
|
||||
const failKind = result.failure?.kind ?? 'other';
|
||||
const failMsg = result.failure?.message ?? 'probe failed';
|
||||
|
||||
if (failKind === 'no_credentials') {
|
||||
return {
|
||||
status: 'fail',
|
||||
message: `quota probe enabled but no OAuth credential — ${failMsg}`,
|
||||
evidence: {
|
||||
human_steps: [
|
||||
'run: claude (first interactive launch prompts browser OAuth login)',
|
||||
],
|
||||
reference: 'https://docs.anthropic.com/en/docs/claude-code/setup#authentication',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
if (failKind === 'auth_failed') {
|
||||
return {
|
||||
status: 'fail',
|
||||
message: `quota probe auth failure (${failMsg}); OAuth token may be expired`,
|
||||
evidence: {
|
||||
human_steps: [
|
||||
'if OAuth token is expired: run: claude (re-login)',
|
||||
'to disable probe: set providers.anthropic.quota_probe_enabled = false in ~/.olp/config.json',
|
||||
],
|
||||
reference: 'https://docs.anthropic.com/en/docs/claude-code/setup#authentication',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
if (failKind === 'schema_drift') {
|
||||
return {
|
||||
status: 'fail',
|
||||
message: `quota probe schema drift detected (${failMsg}); schema may have changed`,
|
||||
evidence: {
|
||||
human_steps: [
|
||||
'check for OLP updates: the anthropic-ratelimit-unified-* header schema may have changed',
|
||||
'file an issue if the schema has genuinely drifted (ADR 0013 Rule 5)',
|
||||
],
|
||||
fix_commands: [
|
||||
'node -e "const {_getQuotaProbeStateForTest: s}=await import(\'./lib/providers/anthropic.mjs\');console.log(JSON.stringify(s().lastError))"',
|
||||
],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// rate_limited / network / other
|
||||
return {
|
||||
status: 'fail',
|
||||
message: 'quota probe enabled but no OAuth credential — probe cannot run',
|
||||
evidence: {
|
||||
human_steps: [
|
||||
'run: claude (first interactive launch prompts browser OAuth login)',
|
||||
],
|
||||
reference: 'https://docs.anthropic.com/en/docs/claude-code/setup#authentication',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const result = await _probeOnce(auth);
|
||||
if (result !== null) {
|
||||
const util5h = result.fields.utilization_5h;
|
||||
const util7d = result.fields.utilization_7d;
|
||||
const msg = `quota probe OK — 5h utilization: ${util5h !== null ? `${Math.round(util5h * 100)}%` : 'n/a'}, 7d utilization: ${util7d !== null ? `${Math.round(util7d * 100)}%` : 'n/a'}`;
|
||||
return { status: 'ok', message: msg };
|
||||
}
|
||||
|
||||
// Probe failed — check if stale cache exists
|
||||
if (quotaProbeState.cache !== null) {
|
||||
const ageMin = Math.round((Date.now() - quotaProbeState.cache.fetchedAt) / 60_000);
|
||||
return {
|
||||
status: 'warn',
|
||||
message: `quota probe failed (returning stale cache from ${ageMin}min ago); check network or token expiry`,
|
||||
message: `quota probe failed and no cached data available (${failKind}: ${failMsg})`,
|
||||
evidence: {
|
||||
fix_commands: [
|
||||
'olp doctor # re-run after verifying OAuth credentials are valid',
|
||||
'node -e "const { readAuthArtifact } = await import(\'./lib/providers/anthropic.mjs\'); console.log(JSON.stringify(readAuthArtifact()?.accessToken ? \'token_present\' : \'no_token\'))"',
|
||||
],
|
||||
human_steps: [
|
||||
'run: claude (if token may have expired, re-login)',
|
||||
'verify network access: curl -s -o /dev/null -w "%{http_code}" https://api.anthropic.com/v1/messages',
|
||||
'if OAuth token is expired: run: claude (re-login)',
|
||||
'to disable probe: set providers.anthropic.quota_probe_enabled = false in ~/.olp/config.json',
|
||||
],
|
||||
},
|
||||
};
|
||||
} finally {
|
||||
// Restore auth override so this doctor run does not permanently affect
|
||||
// the module-level state beyond this call.
|
||||
if (authRead !== readAuthArtifact) {
|
||||
_quotaAuthReadFnForTest = null;
|
||||
}
|
||||
}
|
||||
|
||||
// No cache at all — hard failure
|
||||
return {
|
||||
status: 'fail',
|
||||
message: 'quota probe failed and no cached data available; check OAuth credentials and network access to api.anthropic.com',
|
||||
evidence: {
|
||||
fix_commands: [
|
||||
'node -e "const { readAuthArtifact } = await import(\'./lib/providers/anthropic.mjs\'); console.log(JSON.stringify(readAuthArtifact()?.accessToken ? \'token_present\' : \'no_token\'))"',
|
||||
],
|
||||
human_steps: [
|
||||
'verify network access: curl -s -o /dev/null -w "%{http_code}" https://api.anthropic.com/v1/messages',
|
||||
'if OAuth token is expired: run: claude (re-login)',
|
||||
'to disable probe: set providers.anthropic.quota_probe_enabled = false in ~/.olp/config.json',
|
||||
],
|
||||
},
|
||||
};
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
Reference in New Issue
Block a user