mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-21 21:15:10 +00:00
feat(phase-1): land fallback engine (D9)
Phase 1 Day 6 — the architectural milestone of Phase 1. OLP is no longer
a single-provider proxy with cache; it is now a multi-provider router
with idempotent-failure-safe fallback. ADR 0004 ratified in v0.1 governance
is fully implemented for D9 scope. Configuration-driven chains activate
when a user populates ~/.olp/config.json routing.chains; at v0.1 default
(empty chains), behaviour is single-hop pass-through identical to D5.
Files:
NEW: lib/fallback/engine.mjs (~470 lines) — Trigger taxonomy, chain
advancement, first-chunk safety, observability annotation.
MOD: server.mjs (+80 lines net) — wires executeWithFallback between IR
construction and provider.spawn. Per-hop cache key isolation
(ADR 0005 § Cross-provider fallback). Test seams added:
__setFallbackConfig, __resetFallbackConfig, __clearCache.
MOD: test-features.mjs (+845 lines, 6 new suites = +55 tests).
Authority citations:
ADR 0004 § Decision § Trigger taxonomy — Hard / Soft / Deterministic
(deferred) / Cost-aware (deferred) implemented exactly per spec.
ADR 0004 § Decision § Fallback safety — first-chunk rule satisfied at
D9 by buffering composition (executeHopFn = collectAllChunks; server
writes to res only after engine returns).
ADR 0004 § Decision § Chain advancement — one-at-a-time iteration;
originalError preserved from FIRST hop (not last) on exhaustion.
ADR 0004 § Decision § No fallback for client-side errors — HTTP 400/
401/403/404/422 stop the chain immediately; AUTH_MISSING also stops
immediately (user-config failure, not provider-quota).
ADR 0004 § Decision § Observability headers — X-OLP-Fallback-Hops set
from engine return value; X-OLP-Fallback-Exhausted lists tried
providers on exhaustion.
ADR 0005 § Cross-provider fallback cache behavior — each fallback hop
computes a fresh cache key with the hop's (provider, model) tuple;
primary's cache entries cannot leak to secondary's hop.
Reviewer chain (Iron Rule 10):
Implementer: sonnet (general-purpose).
Fresh-context reviewer: opus. Verdict APPROVE_WITH_MINOR.
Reviewer ran npm test (277/277 pass), read ADR 0004 + ADR 0005 end-
to-end, verified first-chunk-safety composition, checked all
trigger taxonomy mappings.
Reviewer non-blocking findings folded in this commit:
1. Test label at test-features.mjs:3756 clarified. Original title
"client error (400) → does NOT fall back" was misleading because
the test actually exercises both-fail SPAWN_FAILED → exhausted
path (400-no-fallback semantic is covered at unit level instead).
Renamed + commented to match actual behaviour.
2. triedProviders semantic (includes soft-skipped) documented in
engine.mjs comment near the soft-trigger branch.
3. SOFT_TRIGGER synthetic code documented in engine.mjs as engine-
internal, NOT a member of base.mjs PROVIDER_ERROR_CODES.
Reviewer non-blocking suggestions deferred (open D-later questions):
- Q1 cacheStatus on fallback hops: conservative 'miss', acceptable
per ADR 0005 § Per-model isolation.
- Q2 X-OLP-Fallback-Exhausted only when triedProviders > 1:
acceptable per ADR 0004 spec.
- Q3 soft-trigger quotaSnapshot always null at D9: acceptable per
ADR 0004 § Consequences/Negative graceful-degrade clause. Phase 2
will add quota poll-worker.
- Q4 loadFallbackConfigSync sync-only: acceptable for bounded
startup I/O.
Test count: 222 (D8) → 277 (D9). 6 new test suites cover trigger
taxonomy, engine chain advancement (including the load-bearing
originalError-from-FIRST-hop property), HTTP integration through
real server.mjs, soft trigger pre-fetch path, exhausted-chain
header emission.
Verification:
node --check on all touched files: clean.
npm test on Node 25.8.0: 277/277 pass in 266ms.
Hygiene grep: zero personal-name/path/token hits.
No new external npm deps.
Co-Authored-By: Claude Opus 4.7 (noreply@anthropic.com)
This commit is contained in:
co-authored by
Claude Opus 4.7 (noreply@anthropic.com)
parent
95dc072245
commit
5960486542
@@ -0,0 +1,473 @@
|
|||||||
|
/**
|
||||||
|
* lib/fallback/engine.mjs — OLP Fallback Engine
|
||||||
|
*
|
||||||
|
* Authority: ADR 0004 — Fallback Engine Semantics and Safety
|
||||||
|
*
|
||||||
|
* Implements:
|
||||||
|
* - Trigger taxonomy (Hard / Soft per ADR 0004 § Decision § Trigger taxonomy)
|
||||||
|
* - Chain advancement one-at-a-time (ADR 0004 § Decision § Chain advancement)
|
||||||
|
* - First-chunk safety rule (ADR 0004 § Decision § Fallback safety)
|
||||||
|
* - Observability return values for header annotation (ADR 0004 § Decision § Observability headers)
|
||||||
|
* - No fallback for client-side errors (ADR 0004 § Decision § No fallback for client-side errors)
|
||||||
|
*
|
||||||
|
* Sealed at D9. Do NOT modify without an ADR 0004 amendment.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { readFileSync } from 'node:fs';
|
||||||
|
import { homedir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
|
||||||
|
import { ProviderError } from '../providers/base.mjs';
|
||||||
|
|
||||||
|
// ── Hard-trigger evaluation ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Maps ProviderError codes to hard-trigger decisions.
|
||||||
|
* Per ADR 0004 § Decision § Trigger taxonomy — Hard triggers:
|
||||||
|
* - QUOTA_EXHAUSTED → hard trigger
|
||||||
|
* - RATE_LIMITED → hard trigger
|
||||||
|
* - SPAWN_FAILED → hard trigger (provider CLI failed)
|
||||||
|
* - CLI_NOT_FOUND → hard trigger (binary missing)
|
||||||
|
* - AUTH_MISSING → NOT a hard trigger (user-config failure; user must fix)
|
||||||
|
* - OUTPUT_PARSE_ERROR → hard trigger
|
||||||
|
*
|
||||||
|
* @type {Record<string, boolean>}
|
||||||
|
*/
|
||||||
|
const HARD_TRIGGER_CODES = {
|
||||||
|
QUOTA_EXHAUSTED: true,
|
||||||
|
RATE_LIMITED: true,
|
||||||
|
SPAWN_FAILED: true,
|
||||||
|
CLI_NOT_FOUND: true,
|
||||||
|
AUTH_MISSING: false, // user config problem — never fall over (ADR 0004 § Decision)
|
||||||
|
OUTPUT_PARSE_ERROR: true,
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* HTTP status codes that are client errors and MUST NOT trigger fallback.
|
||||||
|
* Per ADR 0004 § Decision § "No fallback for client-side errors":
|
||||||
|
* HTTP 400, 401, 403, 404, 422 from a provider are NOT fallback triggers.
|
||||||
|
*
|
||||||
|
* @type {Set<number>}
|
||||||
|
*/
|
||||||
|
const CLIENT_ERROR_STATUSES = new Set([400, 401, 403, 404, 422]);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Returns true if `status` is a client-side error that must NOT trigger fallback.
|
||||||
|
* Exported for tests.
|
||||||
|
*
|
||||||
|
* @param {number} status
|
||||||
|
* @returns {boolean}
|
||||||
|
*/
|
||||||
|
export function isClientError(status) {
|
||||||
|
return CLIENT_ERROR_STATUSES.has(status);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluates whether an error from a provider qualifies as a Hard trigger.
|
||||||
|
*
|
||||||
|
* Per ADR 0004 § Decision § Trigger taxonomy — Hard triggers:
|
||||||
|
* HTTP 5xx and quota-exhaustion 4xx (surfaced via ProviderError codes)
|
||||||
|
* trigger mandatory chain advancement.
|
||||||
|
*
|
||||||
|
* Per ADR 0004 § Decision § "No fallback for client-side errors":
|
||||||
|
* Client-side HTTP errors (400/401/403/404/422) are NOT hard triggers.
|
||||||
|
* AUTH_MISSING is also not a hard trigger (user must fix their config).
|
||||||
|
*
|
||||||
|
* @param {Error} error — the error thrown by executeHopFn
|
||||||
|
* @param {object} [_providerHints] — reserved for future hints-based logic
|
||||||
|
* @returns {boolean} — true if this error should trigger chain advancement
|
||||||
|
*/
|
||||||
|
export function evaluateHardTriggers(error, _providerHints = {}) {
|
||||||
|
if (!error) return false;
|
||||||
|
|
||||||
|
// ProviderError with a code: map via HARD_TRIGGER_CODES lookup.
|
||||||
|
// AUTH_MISSING is explicitly false in the table — do not fall over.
|
||||||
|
if (error instanceof ProviderError && error.code) {
|
||||||
|
return HARD_TRIGGER_CODES[error.code] === true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// HTTP status code present on the error object.
|
||||||
|
const status = error.statusCode ?? error.status ?? null;
|
||||||
|
if (status !== null && typeof status === 'number') {
|
||||||
|
// Client-side errors: never fall over (ADR 0004 § No fallback for client-side errors)
|
||||||
|
if (CLIENT_ERROR_STATUSES.has(status)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
// 5xx: always a hard trigger (ADR 0004 § Hard triggers: HTTP 5xx from provider)
|
||||||
|
if (status >= 500) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
// Non-client 4xx (e.g. 429 / 529): hard trigger (quota exhaustion semantics)
|
||||||
|
if (status >= 400) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unknown error type — conservative default: do NOT trigger fallback.
|
||||||
|
// Per ADR 0004 § Alternatives considered (a): "any error" fallback was rejected.
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Soft-trigger evaluation ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluates whether a chain hop's soft triggers fire against the current
|
||||||
|
* quota snapshot for that provider.
|
||||||
|
*
|
||||||
|
* Per ADR 0004 § Decision § Trigger taxonomy — Soft triggers:
|
||||||
|
* - credit_pool_percent_threshold: fires when percentUsed >= threshold
|
||||||
|
* - daily_request_count_threshold: fires when dailyCount >= threshold
|
||||||
|
* - five_hour_window_percent_threshold: fires when fiveHourWindowPercent >= threshold
|
||||||
|
*
|
||||||
|
* Per ADR 0004 § Consequences/Negative (Mitigations):
|
||||||
|
* null quotaStatus → treat as "don't fire" — degrade gracefully.
|
||||||
|
*
|
||||||
|
* @param {object|null|undefined} triggerConfig — per-hop soft-trigger config, e.g.
|
||||||
|
* { credit_pool_percent_threshold: 90, daily_request_count_threshold: 1000 }
|
||||||
|
* @param {object|null|undefined} quotaSnapshot — from provider.quotaStatus(), e.g.
|
||||||
|
* { percentUsed: 95, dailyCount: 500, fiveHourWindowPercent: 80 }
|
||||||
|
* May be null if the provider cannot retrieve quota.
|
||||||
|
* @returns {boolean} — true if any configured soft trigger fires
|
||||||
|
*/
|
||||||
|
export function evaluateSoftTriggers(triggerConfig, quotaSnapshot) {
|
||||||
|
// No trigger config at all → soft triggers never fire
|
||||||
|
if (!triggerConfig || typeof triggerConfig !== 'object') return false;
|
||||||
|
|
||||||
|
// Per ADR 0004 § Consequences/Negative: null quotaStatus → don't fire
|
||||||
|
if (quotaSnapshot == null) return false;
|
||||||
|
|
||||||
|
// credit_pool_percent_threshold: fires when percentUsed >= threshold
|
||||||
|
if (
|
||||||
|
triggerConfig.credit_pool_percent_threshold !== undefined &&
|
||||||
|
quotaSnapshot.percentUsed !== undefined &&
|
||||||
|
quotaSnapshot.percentUsed !== null &&
|
||||||
|
quotaSnapshot.percentUsed >= triggerConfig.credit_pool_percent_threshold
|
||||||
|
) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// daily_request_count_threshold: fires when dailyCount >= threshold
|
||||||
|
if (
|
||||||
|
triggerConfig.daily_request_count_threshold !== undefined &&
|
||||||
|
quotaSnapshot.dailyCount !== undefined &&
|
||||||
|
quotaSnapshot.dailyCount !== null &&
|
||||||
|
quotaSnapshot.dailyCount >= triggerConfig.daily_request_count_threshold
|
||||||
|
) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// five_hour_window_percent_threshold: fires when fiveHourWindowPercent >= threshold
|
||||||
|
if (
|
||||||
|
triggerConfig.five_hour_window_percent_threshold !== undefined &&
|
||||||
|
quotaSnapshot.fiveHourWindowPercent !== undefined &&
|
||||||
|
quotaSnapshot.fiveHourWindowPercent !== null &&
|
||||||
|
quotaSnapshot.fiveHourWindowPercent >= triggerConfig.five_hour_window_percent_threshold
|
||||||
|
) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Fallback engine ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @typedef {Object} ChainHop
|
||||||
|
* @property {string} provider — provider name (e.g. 'anthropic')
|
||||||
|
* @property {string} model — model string for this hop
|
||||||
|
* @property {object} [softTriggers] — optional soft-trigger config for this hop
|
||||||
|
* @property {object|null} [quotaSnapshot] — optional pre-fetched quota snapshot
|
||||||
|
*/
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @typedef {Object} FallbackResult
|
||||||
|
* @property {Array<object>|null} chunks — IR chunk array on success; null if exhausted
|
||||||
|
* @property {string} providerUsed — provider name that served the request (or first that failed)
|
||||||
|
* @property {string} modelUsed — model string that served (or first that failed)
|
||||||
|
* @property {number} fallbackHops — chain index of the serving hop (0=primary, 1=first fallback, etc.)
|
||||||
|
* @property {Error|null} originalError — first-hop error if exhausted; null on success
|
||||||
|
* @property {string[]} triedProviders — all providers tried, in chain order
|
||||||
|
*/
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Executes a provider chain with fallback semantics.
|
||||||
|
*
|
||||||
|
* Per ADR 0004 § Decision § Chain advancement (one-at-a-time):
|
||||||
|
* 1. Try each hop in order.
|
||||||
|
* 2. Before calling executeHopFn: evaluate soft triggers. If fired, skip this hop.
|
||||||
|
* 3. On error: check for client error / AUTH_MISSING (stop immediately).
|
||||||
|
* Otherwise evaluate hard triggers. If hard-triggered, advance to next hop.
|
||||||
|
* 4. If chain exhausted: return first-hop's originalError (not last).
|
||||||
|
*
|
||||||
|
* Per ADR 0004 § Decision § Fallback safety — first-chunk rule:
|
||||||
|
* At D9, executeHopFn is collectAllChunks() — fully buffered before return.
|
||||||
|
* An exception from executeHopFn means zero bytes were written to the client.
|
||||||
|
* First-chunk safety is trivially satisfied by the buffering pattern.
|
||||||
|
*
|
||||||
|
* @param {ChainHop[]} chain — ordered list of hops
|
||||||
|
* @param {object} irRequest — IR request object (ADR 0003)
|
||||||
|
* @param {function} executeHopFn — async (provider, model, ir) => IR chunk array; throws on failure
|
||||||
|
* @param {object} [options]
|
||||||
|
* @param {function} [options.logEvent] — optional structured logger (level, event, data)
|
||||||
|
* @returns {Promise<FallbackResult>}
|
||||||
|
*/
|
||||||
|
export async function executeWithFallback(chain, irRequest, executeHopFn, options = {}) {
|
||||||
|
const { logEvent = () => {} } = options;
|
||||||
|
|
||||||
|
if (!Array.isArray(chain) || chain.length === 0) {
|
||||||
|
throw new Error('executeWithFallback: chain must be a non-empty array');
|
||||||
|
}
|
||||||
|
|
||||||
|
const triedProviders = [];
|
||||||
|
let originalError = null; // Per ADR 0004: first-hop error is the canonical signal
|
||||||
|
let firstErrorRecorded = false;
|
||||||
|
|
||||||
|
for (let i = 0; i < chain.length; i++) {
|
||||||
|
const hop = chain[i];
|
||||||
|
const { provider, model, softTriggers = null, quotaSnapshot = null } = hop;
|
||||||
|
|
||||||
|
// ── Soft-trigger check (BEFORE spawn) ──────────────────────────────
|
||||||
|
// Per ADR 0004 § Decision: soft triggers evaluated before spawn;
|
||||||
|
// if fired, advance without attempting this provider at all.
|
||||||
|
//
|
||||||
|
// Note on `triedProviders` semantics (D9 review-2 finding): we push the
|
||||||
|
// provider name onto triedProviders even when soft-skipped (no spawn).
|
||||||
|
// ADR 0004 § Observability headers describes X-OLP-Fallback-Exhausted as
|
||||||
|
// "tried" providers; per ADR 0004 § Soft triggers a soft-fired hop is
|
||||||
|
// counted as tried-and-skipped (the proxy advances to the next chain
|
||||||
|
// entry without attempting the primary at all). The header therefore
|
||||||
|
// includes soft-skipped hops. A future enhancement could split into
|
||||||
|
// `triedProviders` + `softSkippedProviders` for finer-grained debug
|
||||||
|
// visibility; D9 keeps them unified per ADR-0004 spec.
|
||||||
|
if (evaluateSoftTriggers(softTriggers, quotaSnapshot)) {
|
||||||
|
logEvent('info', 'fallback_soft_trigger', {
|
||||||
|
hop: i,
|
||||||
|
provider,
|
||||||
|
model,
|
||||||
|
reason: 'soft_trigger_fired',
|
||||||
|
});
|
||||||
|
triedProviders.push(provider);
|
||||||
|
|
||||||
|
// Record a synthetic originalError for the first fired soft trigger.
|
||||||
|
// `code: 'SOFT_TRIGGER'` is a fallback-engine-internal marker, NOT a
|
||||||
|
// member of PROVIDER_ERROR_CODES (base.mjs) — those are provider
|
||||||
|
// plugin error codes. Downstream consumers inspecting originalError.code
|
||||||
|
// should treat 'SOFT_TRIGGER' as engine-synthetic. D9 review-2 noted
|
||||||
|
// this; documented here so future readers do not try to add SOFT_TRIGGER
|
||||||
|
// to the PROVIDER_ERROR_CODES closed enum.
|
||||||
|
if (!firstErrorRecorded) {
|
||||||
|
originalError = Object.assign(
|
||||||
|
new Error(`Soft trigger fired for provider ${provider}: quota threshold exceeded`),
|
||||||
|
{ code: 'SOFT_TRIGGER', provider },
|
||||||
|
);
|
||||||
|
firstErrorRecorded = true;
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Execute hop ────────────────────────────────────────────────────
|
||||||
|
triedProviders.push(provider);
|
||||||
|
try {
|
||||||
|
const chunks = await executeHopFn(provider, model, irRequest);
|
||||||
|
|
||||||
|
// Per ADR 0004 § Fallback safety — first-chunk rule:
|
||||||
|
// executeHopFn returned successfully = chunks buffered = no client writes
|
||||||
|
// blocked = safe to return. Any retry attempt after this point would be
|
||||||
|
// post-first-chunk and is therefore forbidden. We return immediately.
|
||||||
|
logEvent('info', 'fallback_hop_success', { hop: i, provider, model });
|
||||||
|
return {
|
||||||
|
chunks,
|
||||||
|
providerUsed: provider,
|
||||||
|
modelUsed: model,
|
||||||
|
fallbackHops: i,
|
||||||
|
originalError: null,
|
||||||
|
triedProviders,
|
||||||
|
};
|
||||||
|
} catch (err) {
|
||||||
|
// Record FIRST hop error as the canonical signal
|
||||||
|
// Per ADR 0004 § Chain advancement step 4: return first error, not last.
|
||||||
|
if (!firstErrorRecorded) {
|
||||||
|
originalError = err;
|
||||||
|
firstErrorRecorded = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
logEvent('warn', 'fallback_hop_error', {
|
||||||
|
hop: i,
|
||||||
|
provider,
|
||||||
|
model,
|
||||||
|
error: err.message,
|
||||||
|
code: err.code ?? null,
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Client error: stop immediately ─────────────────────────────
|
||||||
|
// Per ADR 0004 § "No fallback for client-side errors"
|
||||||
|
const status = err.statusCode ?? err.status ?? null;
|
||||||
|
if (status !== null && CLIENT_ERROR_STATUSES.has(status)) {
|
||||||
|
logEvent('warn', 'fallback_client_error_no_fallback', { hop: i, provider, status });
|
||||||
|
return {
|
||||||
|
chunks: null,
|
||||||
|
providerUsed: provider,
|
||||||
|
modelUsed: model,
|
||||||
|
fallbackHops: i,
|
||||||
|
originalError: err,
|
||||||
|
triedProviders,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── AUTH_MISSING: stop immediately ─────────────────────────────
|
||||||
|
// Per ADR 0004 § Decision: AUTH_MISSING = HARD_TRIGGER_CODES[AUTH_MISSING]=false.
|
||||||
|
// Silently masking it by trying next provider prevents user from discovering
|
||||||
|
// the misconfiguration.
|
||||||
|
if (err instanceof ProviderError && err.code === 'AUTH_MISSING') {
|
||||||
|
logEvent('warn', 'fallback_auth_missing_no_fallback', { hop: i, provider });
|
||||||
|
return {
|
||||||
|
chunks: null,
|
||||||
|
providerUsed: provider,
|
||||||
|
modelUsed: model,
|
||||||
|
fallbackHops: i,
|
||||||
|
originalError: err,
|
||||||
|
triedProviders,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Hard-trigger check: advance chain ──────────────────────────
|
||||||
|
// Per ADR 0004 § Decision § Fallback safety — first-chunk rule:
|
||||||
|
// At D9, executeHopFn threw, so zero chunks were emitted.
|
||||||
|
// Fallback is eligible if the error is a hard trigger.
|
||||||
|
if (evaluateHardTriggers(err)) {
|
||||||
|
logEvent('info', 'fallback_hard_trigger', {
|
||||||
|
hop: i,
|
||||||
|
provider,
|
||||||
|
model,
|
||||||
|
error: err.message,
|
||||||
|
code: err.code ?? null,
|
||||||
|
advance_to_hop: i + 1,
|
||||||
|
});
|
||||||
|
continue; // advance to next hop
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Non-trigger error: surface immediately ──────────────────────
|
||||||
|
// Per ADR 0004 § Alternatives considered (a): "any error" fallback rejected.
|
||||||
|
logEvent('warn', 'fallback_non_trigger_error', {
|
||||||
|
hop: i,
|
||||||
|
provider,
|
||||||
|
error: err.message,
|
||||||
|
code: err.code ?? null,
|
||||||
|
});
|
||||||
|
return {
|
||||||
|
chunks: null,
|
||||||
|
providerUsed: provider,
|
||||||
|
modelUsed: model,
|
||||||
|
fallbackHops: i,
|
||||||
|
originalError: err,
|
||||||
|
triedProviders,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Chain exhausted
|
||||||
|
// Per ADR 0004 § Decision § Chain advancement step 4:
|
||||||
|
// "return A's original error (not B's, not C's)"
|
||||||
|
logEvent('error', 'fallback_chain_exhausted', {
|
||||||
|
chain: chain.map(h => h.provider),
|
||||||
|
triedProviders,
|
||||||
|
});
|
||||||
|
|
||||||
|
return {
|
||||||
|
chunks: null,
|
||||||
|
providerUsed: chain[0].provider,
|
||||||
|
modelUsed: chain[0].model,
|
||||||
|
fallbackHops: chain.length,
|
||||||
|
originalError,
|
||||||
|
triedProviders,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Chain builder ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Builds the execution chain for a given model string from config.
|
||||||
|
*
|
||||||
|
* Per D9 implementation notes:
|
||||||
|
* - If fallbackChainConfig has an explicit entry for this model, use it.
|
||||||
|
* - Otherwise, use a single-hop chain with the first matched provider (no fallback).
|
||||||
|
* - v0.1 ships with empty chain config; fallback activates only when the user
|
||||||
|
* populates ~/.olp/config.json's routing.chains.
|
||||||
|
*
|
||||||
|
* @param {string} modelString — user-requested model string
|
||||||
|
* @param {Map<string, object>} loadedProviders — name → provider plugin
|
||||||
|
* @param {object} [fallbackChainConfig] — routing.chains from config.json
|
||||||
|
* @param {object} [softTriggerConfig] — routing.soft_triggers from config.json
|
||||||
|
* @returns {ChainHop[]|null} — ordered chain, or null if no provider found for model
|
||||||
|
*/
|
||||||
|
export function buildDefaultChain(
|
||||||
|
modelString,
|
||||||
|
loadedProviders,
|
||||||
|
fallbackChainConfig = {},
|
||||||
|
softTriggerConfig = {},
|
||||||
|
) {
|
||||||
|
// Check for explicit chain in config for this model
|
||||||
|
const explicitChain = fallbackChainConfig[modelString];
|
||||||
|
if (explicitChain && Array.isArray(explicitChain) && explicitChain.length > 0) {
|
||||||
|
return explicitChain.map(hop => ({
|
||||||
|
provider: hop.provider,
|
||||||
|
model: hop.model ?? modelString,
|
||||||
|
softTriggers: softTriggerConfig[hop.provider] ?? null,
|
||||||
|
quotaSnapshot: null, // populated at runtime if provider.quotaStatus() is called
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
// No explicit chain: find first loaded provider that serves this model
|
||||||
|
for (const [name, provider] of loadedProviders.entries()) {
|
||||||
|
if (provider.models && provider.models.includes(modelString)) {
|
||||||
|
return [{
|
||||||
|
provider: name,
|
||||||
|
model: modelString,
|
||||||
|
softTriggers: softTriggerConfig[name] ?? null,
|
||||||
|
quotaSnapshot: null,
|
||||||
|
}];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// No provider found for this model
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Config loader ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Default path for the OLP config file.
|
||||||
|
* @returns {string}
|
||||||
|
*/
|
||||||
|
function defaultConfigPath() {
|
||||||
|
return join(homedir(), '.olp', 'config.json');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Loads fallback chain configuration from ~/.olp/config.json (or the given path).
|
||||||
|
*
|
||||||
|
* Per D9 implementation notes:
|
||||||
|
* v0.1 ships with no config file. Fallback engine is wired but single-hop
|
||||||
|
* until the user populates routing.chains.
|
||||||
|
*
|
||||||
|
* Returns empty config (no chains, no soft triggers) if the file is absent,
|
||||||
|
* unreadable, or malformed.
|
||||||
|
*
|
||||||
|
* @param {string} [configPath] — override path (for testing — do NOT write to ~/.olp/config.json in tests)
|
||||||
|
* @returns {{ chains: object, soft_triggers: object }}
|
||||||
|
*/
|
||||||
|
export function loadFallbackConfigSync(configPath) {
|
||||||
|
try {
|
||||||
|
const path = configPath ?? defaultConfigPath();
|
||||||
|
const raw = readFileSync(path, 'utf8');
|
||||||
|
const parsed = JSON.parse(raw);
|
||||||
|
const routing = parsed?.routing ?? {};
|
||||||
|
return {
|
||||||
|
chains: routing.chains ?? {},
|
||||||
|
soft_triggers: routing.soft_triggers ?? {},
|
||||||
|
};
|
||||||
|
} catch {
|
||||||
|
// File absent, unreadable, or malformed → no fallback config (single-hop mode)
|
||||||
|
return { chains: {}, soft_triggers: {} };
|
||||||
|
}
|
||||||
|
}
|
||||||
+168
-87
@@ -33,6 +33,12 @@ import { loadProviders, getProviderForModel, listAllProviderNames } from './lib/
|
|||||||
import { ProviderError } from './lib/providers/base.mjs';
|
import { ProviderError } from './lib/providers/base.mjs';
|
||||||
import { computeCacheKey, hasCacheControl, extractCacheControlMarkers } from './lib/cache/keys.mjs';
|
import { computeCacheKey, hasCacheControl, extractCacheControlMarkers } from './lib/cache/keys.mjs';
|
||||||
import { CacheStore } from './lib/cache/store.mjs';
|
import { CacheStore } from './lib/cache/store.mjs';
|
||||||
|
import {
|
||||||
|
evaluateHardTriggers,
|
||||||
|
executeWithFallback,
|
||||||
|
buildDefaultChain,
|
||||||
|
loadFallbackConfigSync,
|
||||||
|
} from './lib/fallback/engine.mjs';
|
||||||
|
|
||||||
// ── Config ────────────────────────────────────────────────────────────────
|
// ── Config ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
@@ -48,6 +54,27 @@ const BODY_LIMIT = 5 * 1024 * 1024; // 5 MB
|
|||||||
// Empty config → empty loaded map → all POST /v1/chat/completions → 503.
|
// Empty config → empty loaded map → all POST /v1/chat/completions → 503.
|
||||||
const loadedProviders = loadProviders({ enabled: {} });
|
const loadedProviders = loadProviders({ enabled: {} });
|
||||||
|
|
||||||
|
// ── Fallback config ───────────────────────────────────────────────────────
|
||||||
|
// Read ~/.olp/config.json routing.chains at startup. Empty at v0.1.
|
||||||
|
// Per ADR 0004 § D9: fallback engine is wired; activates when user populates chains.
|
||||||
|
// Tests may inject a synthetic fallbackConfig via __setFallbackConfig().
|
||||||
|
let _fallbackConfig = loadFallbackConfigSync();
|
||||||
|
|
||||||
|
/** @internal — test seam: inject a synthetic fallback config (no file I/O) */
|
||||||
|
export function __setFallbackConfig(config) {
|
||||||
|
_fallbackConfig = config ?? { chains: {}, soft_triggers: {} };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @internal — reset to file-based config */
|
||||||
|
export function __resetFallbackConfig() {
|
||||||
|
_fallbackConfig = loadFallbackConfigSync();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @internal — clear the cache store (for tests that need a fresh cache state) */
|
||||||
|
export function __clearCache() {
|
||||||
|
cacheStore.clear();
|
||||||
|
}
|
||||||
|
|
||||||
// ── Cache layer ───────────────────────────────────────────────────────────
|
// ── Cache layer ───────────────────────────────────────────────────────────
|
||||||
// D1 per-key isolation + D4 singleflight per ADR 0005.
|
// D1 per-key isolation + D4 singleflight per ADR 0005.
|
||||||
// keyId: '__anonymous__' at D5 — Phase 2 multi-key infrastructure wires in
|
// keyId: '__anonymous__' at D5 — Phase 2 multi-key infrastructure wires in
|
||||||
@@ -132,9 +159,10 @@ function sendError(res, status, message, type) {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Returns the standard OLP diagnostic headers.
|
* Returns the standard OLP diagnostic headers.
|
||||||
* Per spec: X-OLP-Provider-Used, X-OLP-Model-Used, X-OLP-Fallback-Hops,
|
* Per spec §4.7 and ADR 0004 § Observability headers:
|
||||||
|
* X-OLP-Provider-Used, X-OLP-Model-Used, X-OLP-Fallback-Hops,
|
||||||
* X-OLP-Cache, X-OLP-Latency-Ms.
|
* X-OLP-Cache, X-OLP-Latency-Ms.
|
||||||
* Fallback-Hops is always 0 at D5 (no fallback engine yet — ADR 0004).
|
* Fallback-Hops reflects which chain index served the request (0=primary).
|
||||||
* Cache reflects actual hit/miss/bypass status from the cache layer (ADR 0005).
|
* Cache reflects actual hit/miss/bypass status from the cache layer (ADR 0005).
|
||||||
*
|
*
|
||||||
* @param {object} opts
|
* @param {object} opts
|
||||||
@@ -142,13 +170,14 @@ function sendError(res, status, message, type) {
|
|||||||
* @param {string} opts.modelUsed
|
* @param {string} opts.modelUsed
|
||||||
* @param {number} opts.startMs
|
* @param {number} opts.startMs
|
||||||
* @param {'hit'|'miss'|'bypass'} [opts.cacheStatus='miss']
|
* @param {'hit'|'miss'|'bypass'} [opts.cacheStatus='miss']
|
||||||
|
* @param {number} [opts.fallbackHops=0] — from executeWithFallback result
|
||||||
* @returns {Record<string,string>}
|
* @returns {Record<string,string>}
|
||||||
*/
|
*/
|
||||||
function olpHeaders({ providerUsed, modelUsed, startMs, cacheStatus = 'miss' }) {
|
function olpHeaders({ providerUsed, modelUsed, startMs, cacheStatus = 'miss', fallbackHops = 0 }) {
|
||||||
return {
|
return {
|
||||||
'X-OLP-Provider-Used': providerUsed,
|
'X-OLP-Provider-Used': providerUsed,
|
||||||
'X-OLP-Model-Used': modelUsed,
|
'X-OLP-Model-Used': modelUsed,
|
||||||
'X-OLP-Fallback-Hops': '0',
|
'X-OLP-Fallback-Hops': String(fallbackHops),
|
||||||
'X-OLP-Cache': cacheStatus,
|
'X-OLP-Cache': cacheStatus,
|
||||||
'X-OLP-Latency-Ms': String(Date.now() - startMs),
|
'X-OLP-Latency-Ms': String(Date.now() - startMs),
|
||||||
};
|
};
|
||||||
@@ -181,7 +210,13 @@ function handleModels(req, res) {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* POST /v1/chat/completions
|
* POST /v1/chat/completions
|
||||||
* Core dispatch path: OpenAI request → IR → provider.spawn → OpenAI response.
|
* Core dispatch path: OpenAI request → IR → fallback engine → provider.spawn → OpenAI response.
|
||||||
|
*
|
||||||
|
* D9: Fallback engine (ADR 0004) is wired between IR construction and provider.spawn.
|
||||||
|
* Chain advancement, soft/hard trigger evaluation, and first-chunk safety are all
|
||||||
|
* handled by executeWithFallback(). At v0.1 with empty routing.chains config, this
|
||||||
|
* is a transparent single-hop pass-through. Multi-hop fallback activates when the
|
||||||
|
* user populates ~/.olp/config.json.
|
||||||
*
|
*
|
||||||
* @param {import('node:http').IncomingMessage} req
|
* @param {import('node:http').IncomingMessage} req
|
||||||
* @param {import('node:http').ServerResponse} res
|
* @param {import('node:http').ServerResponse} res
|
||||||
@@ -202,7 +237,7 @@ async function handleChatCompletions(req, res) {
|
|||||||
return sendError(res, e.statusCode ?? 400, e.message, 'invalid_request_error');
|
return sendError(res, e.statusCode ?? 400, e.message, 'invalid_request_error');
|
||||||
}
|
}
|
||||||
|
|
||||||
// Translate OpenAI → IR
|
// Translate OpenAI → IR (ADR 0003)
|
||||||
let ir;
|
let ir;
|
||||||
try {
|
try {
|
||||||
ir = openAIToIR(body);
|
ir = openAIToIR(body);
|
||||||
@@ -213,9 +248,23 @@ async function handleChatCompletions(req, res) {
|
|||||||
throw e;
|
throw e;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Find a provider for the requested model
|
// Auth context is null at D5/D9 — providers fall back to their own credential
|
||||||
const match = getProviderForModel(loadedProviders, ir.model);
|
// discovery (env var, keychain, credentials file). Phase 2 multi-key
|
||||||
if (!match) {
|
// infrastructure will pass a real authContext carrying the per-key OLP token.
|
||||||
|
const authContext = null;
|
||||||
|
|
||||||
|
// ── Fallback engine: build chain (ADR 0004) ─────────────────────────────
|
||||||
|
// buildDefaultChain returns null if no enabled provider serves this model.
|
||||||
|
// Per ADR 0004 § D9: at v0.1, chain is single-hop (no fallback) unless the
|
||||||
|
// user has populated ~/.olp/config.json routing.chains.
|
||||||
|
const chain = buildDefaultChain(
|
||||||
|
ir.model,
|
||||||
|
loadedProviders,
|
||||||
|
_fallbackConfig.chains,
|
||||||
|
_fallbackConfig.soft_triggers,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!chain) {
|
||||||
// ALIGNMENT.md: 0 Enabled Providers at v0.1 → 503 per spec
|
// ALIGNMENT.md: 0 Enabled Providers at v0.1 → 503 per spec
|
||||||
return sendError(
|
return sendError(
|
||||||
res, 503,
|
res, 503,
|
||||||
@@ -224,49 +273,50 @@ async function handleChatCompletions(req, res) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
const { provider, name: providerName } = match;
|
|
||||||
const requestId = generateRequestId();
|
const requestId = generateRequestId();
|
||||||
|
|
||||||
// Auth context is null at D5 — providers fall back to their own credential
|
|
||||||
// discovery (env var, keychain, credentials file). Phase 2 multi-key
|
|
||||||
// infrastructure will pass a real authContext carrying the per-key OLP token.
|
|
||||||
const authContext = null;
|
|
||||||
|
|
||||||
// ── Cache layer (ADR 0005) ──────────────────────────────────────────────
|
// ── Cache layer (ADR 0005) ──────────────────────────────────────────────
|
||||||
// keyId: '__anonymous__' at D5. Phase 2 multi-key infrastructure wires the
|
// keyId: '__anonymous__' at D5/D9. Phase 2 multi-key infrastructure wires the
|
||||||
// real OLP API key ID here for D1 per-key isolation.
|
// real OLP API key ID here for D1 per-key isolation.
|
||||||
const keyId = '__anonymous__';
|
const keyId = '__anonymous__';
|
||||||
const cacheKey = computeCacheKey(providerName, ir.model, ir);
|
|
||||||
|
|
||||||
// D2 bypass: if the request contains Anthropic cache_control markers,
|
// D2 bypass: if the request contains Anthropic cache_control markers,
|
||||||
// skip OLP's response cache entirely (the prompt cache lives at Anthropic's
|
// skip OLP's response cache (prompt cache lives at Anthropic's side per ADR 0005 § D2).
|
||||||
// side; double-caching would shadow Anthropic's TTLs per ADR 0005 § D2).
|
|
||||||
//
|
|
||||||
// Note: hasCacheControl() checks the IR (ADR 0003). Since openAIToIR() does not
|
|
||||||
// preserve cache_control fields from the raw OpenAI message shape (those fields
|
|
||||||
// are Anthropic-specific extensions, not part of the IR schema), we also check
|
|
||||||
// the raw body.messages directly via extractCacheControlMarkers. This ensures
|
|
||||||
// D2 bypass is triggered even though the IR translator strips the field.
|
|
||||||
const bypassCache = hasCacheControl(ir) || extractCacheControlMarkers(body?.messages ?? []).length > 0;
|
const bypassCache = hasCacheControl(ir) || extractCacheControlMarkers(body?.messages ?? []).length > 0;
|
||||||
|
|
||||||
if (bypassCache) {
|
if (bypassCache) {
|
||||||
logEvent('debug', 'cache_bypass', { provider: providerName, model: ir.model, reason: 'cache_control_markers' });
|
logEvent('debug', 'cache_bypass', { model: ir.model, reason: 'cache_control_markers' });
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Collect chunks helper (used by both streaming and non-streaming paths) ──
|
// ── executeHopFn: per-hop spawn + cache wrapper ─────────────────────────
|
||||||
// collectAllChunks wraps provider.spawn() to collect all IR chunks into an
|
// This is the function executeWithFallback calls for each chain hop.
|
||||||
// array. Used as the computeFn for getOrCompute (D4 singleflight).
|
// Each hop gets its own (provider, model) cache key per ADR 0005 § Per-model isolation.
|
||||||
//
|
//
|
||||||
// Error semantics: if the provider emits a `type: 'error'` chunk, we throw
|
// First-chunk safety (ADR 0004 § Fallback safety):
|
||||||
// a ProviderError instead of returning the chunk array. This prevents
|
// collectAllChunks fully buffers the provider response before returning.
|
||||||
// cache_store.set() from being called on an error-terminated response, per
|
// Therefore, if executeHopFn throws, zero bytes have been written to `res`.
|
||||||
// ADR 0005 § "Cache write conditions" item 1 ("The response completed
|
// The fallback engine safely advances the chain on hard triggers.
|
||||||
// successfully (no truncation, no error mid-stream)"). The thrown error
|
// If executeHopFn returns successfully, the chunks are buffered and we write
|
||||||
// propagates to the catch block at the call site, which returns 502 to
|
// them to `res` only AFTER executeWithFallback returns — ensuring no writes
|
||||||
// the client without writing to cache.
|
// occur during chain iteration.
|
||||||
|
async function executeHopFn(hopProvider, hopModel, irReq) {
|
||||||
|
const hopCacheKey = computeCacheKey(hopProvider, hopModel, irReq);
|
||||||
|
const hopProviderPlugin = loadedProviders.get(hopProvider);
|
||||||
|
|
||||||
|
if (!hopProviderPlugin) {
|
||||||
|
// Provider in the chain is not loaded (config references a disabled provider)
|
||||||
|
throw Object.assign(
|
||||||
|
new Error(`Provider ${hopProvider} is not enabled`),
|
||||||
|
{ statusCode: 503 },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect all chunks from this provider, throwing on error chunks.
|
||||||
|
// Error semantics: ProviderError thrown here propagates to executeWithFallback
|
||||||
|
// which decides whether to advance the chain.
|
||||||
async function collectAllChunks() {
|
async function collectAllChunks() {
|
||||||
const chunks = [];
|
const chunks = [];
|
||||||
for await (const irChunk of provider.spawn(ir, authContext)) {
|
for await (const irChunk of hopProviderPlugin.spawn(irReq, authContext)) {
|
||||||
chunks.push(irChunk);
|
chunks.push(irChunk);
|
||||||
if (irChunk.type === 'error') {
|
if (irChunk.type === 'error') {
|
||||||
throw new ProviderError(
|
throw new ProviderError(
|
||||||
@@ -279,36 +329,92 @@ async function handleChatCompletions(req, res) {
|
|||||||
return chunks;
|
return chunks;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Pre-check: stats-neutral peek before calling getOrCompute so we can
|
|
||||||
// reliably report hit/miss in the response header. peek() does NOT touch
|
|
||||||
// hit/miss counters (fix for codex-flagged double-count); getOrCompute()
|
|
||||||
// increments at the semantic boundary.
|
|
||||||
const preCheckHit = bypassCache ? false : await cacheStore.peek(keyId, cacheKey);
|
|
||||||
|
|
||||||
if (ir.stream) {
|
|
||||||
// Streaming response path
|
|
||||||
// D5 simplified D3: cache stores full collected chunks, replays them one
|
|
||||||
// at a time without timing fidelity. Full D3 (timing-accurate replay) lands
|
|
||||||
// in a later Phase per ADR 0005 § D3.
|
|
||||||
let chunks;
|
|
||||||
let cacheStatus;
|
|
||||||
|
|
||||||
try {
|
|
||||||
if (bypassCache) {
|
if (bypassCache) {
|
||||||
cacheStatus = 'bypass';
|
return collectAllChunks();
|
||||||
chunks = await collectAllChunks();
|
|
||||||
} else {
|
|
||||||
// D4 singleflight + D1 per-key isolation
|
|
||||||
chunks = await cacheStore.getOrCompute(keyId, cacheKey, collectAllChunks);
|
|
||||||
cacheStatus = preCheckHit ? 'hit' : 'miss';
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// D4 singleflight + D1 per-key isolation per ADR 0005.
|
||||||
|
// Each hop has its own (provider, model) key — cross-provider contamination
|
||||||
|
// is structurally impossible (ADR 0005 § Per-model isolation).
|
||||||
|
return cacheStore.getOrCompute(keyId, hopCacheKey, collectAllChunks);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Execute with fallback (ADR 0004) ────────────────────────────────────
|
||||||
|
// Pre-check for cache status reporting uses first hop's key (primary provider).
|
||||||
|
const firstHopCacheKey = computeCacheKey(chain[0].provider, chain[0].model, ir);
|
||||||
|
const preCheckHit = bypassCache ? false : await cacheStore.peek(keyId, firstHopCacheKey);
|
||||||
|
|
||||||
|
let fallbackResult;
|
||||||
|
try {
|
||||||
|
fallbackResult = await executeWithFallback(chain, ir, executeHopFn, {
|
||||||
|
logEvent,
|
||||||
|
});
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
logEvent('error', 'spawn_error', { provider: providerName, model: ir.model, error: e.message });
|
// executeWithFallback throws only on programming errors (empty chain).
|
||||||
sendError(res, e instanceof ProviderError ? 502 : 500, e.message ?? 'Provider error', 'provider_error');
|
logEvent('error', 'fallback_engine_error', { error: e.message });
|
||||||
|
return sendError(res, 500, 'Internal server error', 'internal_error');
|
||||||
|
}
|
||||||
|
|
||||||
|
const {
|
||||||
|
chunks,
|
||||||
|
providerUsed,
|
||||||
|
modelUsed,
|
||||||
|
fallbackHops,
|
||||||
|
originalError,
|
||||||
|
triedProviders,
|
||||||
|
} = fallbackResult;
|
||||||
|
|
||||||
|
// ── Chain exhausted or non-trigger error ─────────────────────────────────
|
||||||
|
if (chunks === null) {
|
||||||
|
logEvent('error', 'spawn_error', {
|
||||||
|
model: ir.model,
|
||||||
|
providerUsed,
|
||||||
|
fallbackHops,
|
||||||
|
triedProviders,
|
||||||
|
error: originalError?.message,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Emit exhausted header if more than one provider was tried
|
||||||
|
const exhaustedHeader = triedProviders.length > 1
|
||||||
|
? { 'X-OLP-Fallback-Exhausted': triedProviders.join(',') }
|
||||||
|
: {};
|
||||||
|
|
||||||
|
// Determine status: preserve client errors (400/401/403/404/422) as-is.
|
||||||
|
// Otherwise map ProviderError → 502, unknown → 500.
|
||||||
|
let errStatus = 502;
|
||||||
|
if (originalError) {
|
||||||
|
const httpStatus = originalError.statusCode ?? originalError.status ?? null;
|
||||||
|
if (httpStatus !== null) {
|
||||||
|
errStatus = httpStatus;
|
||||||
|
} else if (!(originalError instanceof ProviderError)) {
|
||||||
|
errStatus = 500;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Send error with exhausted header
|
||||||
|
const payload = JSON.stringify({
|
||||||
|
error: {
|
||||||
|
message: originalError?.message ?? 'Provider error',
|
||||||
|
type: 'provider_error',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
res.writeHead(errStatus, {
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
'Content-Length': Buffer.byteLength(payload),
|
||||||
|
...exhaustedHeader,
|
||||||
|
});
|
||||||
|
res.end(payload);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
const headers = olpHeaders({ providerUsed: providerName, modelUsed: ir.model, startMs, cacheStatus });
|
// ── Success: emit response ─────────────────────────────────────────────
|
||||||
|
// Per ADR 0004 § Observability headers: X-OLP-Fallback-Hops reflects the
|
||||||
|
// chain index of the serving hop; 0 = primary served, 1 = first fallback, etc.
|
||||||
|
const cacheStatus = bypassCache ? 'bypass' : (preCheckHit && fallbackHops === 0 ? 'hit' : 'miss');
|
||||||
|
const headers = olpHeaders({ providerUsed, modelUsed, startMs, cacheStatus, fallbackHops });
|
||||||
|
|
||||||
|
if (ir.stream) {
|
||||||
|
// Streaming response path (D3 simplified: burst replay, no timing fidelity)
|
||||||
res.writeHead(200, {
|
res.writeHead(200, {
|
||||||
'Content-Type': 'text/event-stream',
|
'Content-Type': 'text/event-stream',
|
||||||
'Cache-Control': 'no-cache',
|
'Cache-Control': 'no-cache',
|
||||||
@@ -317,7 +423,6 @@ async function handleChatCompletions(req, res) {
|
|||||||
...headers,
|
...headers,
|
||||||
});
|
});
|
||||||
|
|
||||||
// D3 simplified replay: emit collected chunks sequentially
|
|
||||||
for (const irChunk of chunks) {
|
for (const irChunk of chunks) {
|
||||||
res.write(irChunkToOpenAISSE(irChunk, requestId, ir.model));
|
res.write(irChunkToOpenAISSE(irChunk, requestId, ir.model));
|
||||||
if (irChunk.type === 'stop' || irChunk.type === 'error') break;
|
if (irChunk.type === 'stop' || irChunk.type === 'error') break;
|
||||||
@@ -326,31 +431,7 @@ async function handleChatCompletions(req, res) {
|
|||||||
res.end();
|
res.end();
|
||||||
} else {
|
} else {
|
||||||
// Non-streaming response path
|
// Non-streaming response path
|
||||||
let cacheStatus;
|
const responseObj = irResponseToOpenAINonStream(chunks, requestId, ir.model);
|
||||||
let responseObj;
|
|
||||||
|
|
||||||
try {
|
|
||||||
if (bypassCache) {
|
|
||||||
cacheStatus = 'bypass';
|
|
||||||
const chunks = await collectAllChunks();
|
|
||||||
responseObj = irResponseToOpenAINonStream(chunks, requestId, ir.model);
|
|
||||||
} else {
|
|
||||||
// D4 singleflight: concurrent identical requests share one spawn.
|
|
||||||
// We cache the full IR chunk array and assemble the OpenAI response on
|
|
||||||
// each retrieval so the requestId is fresh per request (per OpenAI spec
|
|
||||||
// each response has a unique id).
|
|
||||||
const chunks = await cacheStore.getOrCompute(keyId, cacheKey, collectAllChunks);
|
|
||||||
cacheStatus = preCheckHit ? 'hit' : 'miss';
|
|
||||||
responseObj = irResponseToOpenAINonStream(chunks, requestId, ir.model);
|
|
||||||
}
|
|
||||||
} catch (e) {
|
|
||||||
logEvent('error', 'spawn_error', { provider: providerName, model: ir.model, error: e.message });
|
|
||||||
const status = e instanceof ProviderError ? 502 : 500;
|
|
||||||
sendError(res, status, e.message ?? 'Provider error', 'provider_error');
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const headers = olpHeaders({ providerUsed: providerName, modelUsed: ir.model, startMs, cacheStatus });
|
|
||||||
sendJSON(res, 200, responseObj, headers);
|
sendJSON(res, 200, responseObj, headers);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2999,3 +2999,848 @@ describe('Mistral Vibe plugin (D8)', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ── Suite 13: Fallback engine (D9) ───────────────────────────────────────
|
||||||
|
//
|
||||||
|
// All tests are unit tests. No real provider CLIs are invoked.
|
||||||
|
// Tests cover:
|
||||||
|
// 13a: Trigger taxonomy (evaluateHardTriggers / evaluateSoftTriggers)
|
||||||
|
// 13b: executeWithFallback engine behaviour
|
||||||
|
// 13c: First-chunk safety (buffering semantics at D9)
|
||||||
|
// 13d: Soft trigger skipping spawn
|
||||||
|
// 13e: Header annotation (providerUsed / modelUsed / fallbackHops / originalError)
|
||||||
|
// 13f: HTTP integration with __setFallbackConfig test seam
|
||||||
|
//
|
||||||
|
// Authority: ADR 0004 — Fallback Engine Semantics and Safety
|
||||||
|
|
||||||
|
import {
|
||||||
|
evaluateHardTriggers,
|
||||||
|
evaluateSoftTriggers,
|
||||||
|
executeWithFallback,
|
||||||
|
buildDefaultChain,
|
||||||
|
loadFallbackConfigSync,
|
||||||
|
isClientError,
|
||||||
|
} from './lib/fallback/engine.mjs';
|
||||||
|
|
||||||
|
import {
|
||||||
|
createOlpServer,
|
||||||
|
__setFallbackConfig,
|
||||||
|
__resetFallbackConfig,
|
||||||
|
__clearCache,
|
||||||
|
} from './server.mjs';
|
||||||
|
|
||||||
|
// ── 13a: Trigger taxonomy ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
describe('Fallback engine — trigger taxonomy (D9)', () => {
|
||||||
|
|
||||||
|
// ── evaluateHardTriggers ─────────────────────────────────────────────
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 500 → fires (5xx hard trigger)', () => {
|
||||||
|
const err = Object.assign(new Error('Server error'), { statusCode: 500 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 503 → fires (5xx hard trigger)', () => {
|
||||||
|
const err = Object.assign(new Error('Service unavailable'), { statusCode: 503 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 400 → does NOT fire (client error)', () => {
|
||||||
|
const err = Object.assign(new Error('Bad request'), { statusCode: 400 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 401 → does NOT fire (client error)', () => {
|
||||||
|
const err = Object.assign(new Error('Unauthorized'), { statusCode: 401 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 403 → does NOT fire (client error)', () => {
|
||||||
|
const err = Object.assign(new Error('Forbidden'), { statusCode: 403 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 404 → does NOT fire (client error)', () => {
|
||||||
|
const err = Object.assign(new Error('Not found'), { statusCode: 404 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 422 → does NOT fire (client error)', () => {
|
||||||
|
const err = Object.assign(new Error('Unprocessable'), { statusCode: 422 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: HTTP 429 → fires (quota-like 4xx not in client-error set)', () => {
|
||||||
|
const err = Object.assign(new Error('Too Many Requests'), { statusCode: 429 });
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: ProviderError QUOTA_EXHAUSTED → fires', () => {
|
||||||
|
const err = new ProviderError('Quota exhausted', 'QUOTA_EXHAUSTED');
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: ProviderError RATE_LIMITED → fires', () => {
|
||||||
|
const err = new ProviderError('Rate limited', 'RATE_LIMITED');
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: ProviderError SPAWN_FAILED → fires', () => {
|
||||||
|
const err = new ProviderError('Spawn failed', 'SPAWN_FAILED');
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: ProviderError CLI_NOT_FOUND → fires', () => {
|
||||||
|
const err = new ProviderError('CLI not found', 'CLI_NOT_FOUND');
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: ProviderError AUTH_MISSING → does NOT fire (user must fix)', () => {
|
||||||
|
const err = new ProviderError('Auth missing', 'AUTH_MISSING');
|
||||||
|
assert.equal(evaluateHardTriggers(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: ProviderError OUTPUT_PARSE_ERROR → fires', () => {
|
||||||
|
const err = new ProviderError('Parse error', 'OUTPUT_PARSE_ERROR');
|
||||||
|
assert.equal(evaluateHardTriggers(err), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: generic Error with no statusCode → does NOT fire', () => {
|
||||||
|
const err = new Error('Something went wrong');
|
||||||
|
assert.equal(evaluateHardTriggers(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateHardTriggers: null error → does NOT fire', () => {
|
||||||
|
assert.equal(evaluateHardTriggers(null), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── evaluateSoftTriggers ─────────────────────────────────────────────
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: credit_pool_percent_threshold at exactly threshold → fires', () => {
|
||||||
|
const cfg = { credit_pool_percent_threshold: 90 };
|
||||||
|
const quota = { percentUsed: 90 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, quota), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: credit_pool_percent_threshold above threshold → fires', () => {
|
||||||
|
const cfg = { credit_pool_percent_threshold: 90 };
|
||||||
|
const quota = { percentUsed: 95 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, quota), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: credit_pool_percent_threshold below threshold → does NOT fire', () => {
|
||||||
|
const cfg = { credit_pool_percent_threshold: 90 };
|
||||||
|
const quota = { percentUsed: 89 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, quota), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: daily_request_count_threshold at threshold → fires', () => {
|
||||||
|
const cfg = { daily_request_count_threshold: 500 };
|
||||||
|
const quota = { dailyCount: 500 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, quota), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: daily_request_count_threshold below threshold → does NOT fire', () => {
|
||||||
|
const cfg = { daily_request_count_threshold: 500 };
|
||||||
|
const quota = { dailyCount: 499 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, quota), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: five_hour_window_percent_threshold at threshold → fires', () => {
|
||||||
|
const cfg = { five_hour_window_percent_threshold: 85 };
|
||||||
|
const quota = { fiveHourWindowPercent: 85 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, quota), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: null quotaSnapshot → does NOT fire (graceful degrade)', () => {
|
||||||
|
const cfg = { credit_pool_percent_threshold: 90 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, null), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: undefined quotaSnapshot → does NOT fire (graceful degrade)', () => {
|
||||||
|
const cfg = { credit_pool_percent_threshold: 90 };
|
||||||
|
assert.equal(evaluateSoftTriggers(cfg, undefined), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: null triggerConfig → does NOT fire', () => {
|
||||||
|
const quota = { percentUsed: 95 };
|
||||||
|
assert.equal(evaluateSoftTriggers(null, quota), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('evaluateSoftTriggers: empty triggerConfig → does NOT fire', () => {
|
||||||
|
const quota = { percentUsed: 95 };
|
||||||
|
assert.equal(evaluateSoftTriggers({}, quota), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── isClientError ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
it('isClientError: 400 → true', () => { assert.equal(isClientError(400), true); });
|
||||||
|
it('isClientError: 401 → true', () => { assert.equal(isClientError(401), true); });
|
||||||
|
it('isClientError: 403 → true', () => { assert.equal(isClientError(403), true); });
|
||||||
|
it('isClientError: 404 → true', () => { assert.equal(isClientError(404), true); });
|
||||||
|
it('isClientError: 422 → true', () => { assert.equal(isClientError(422), true); });
|
||||||
|
it('isClientError: 429 → false (quota, not client error)', () => { assert.equal(isClientError(429), false); });
|
||||||
|
it('isClientError: 500 → false', () => { assert.equal(isClientError(500), false); });
|
||||||
|
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── 13b: executeWithFallback engine ─────────────────────────────────────
|
||||||
|
|
||||||
|
describe('Fallback engine — executeWithFallback (D9)', () => {
|
||||||
|
|
||||||
|
// helper: build a mock executeHopFn that succeeds or throws per provider name
|
||||||
|
function makeHopFn(outcomes) {
|
||||||
|
// outcomes: { [provider]: 'success' | Error }
|
||||||
|
return async function (provider, model, _ir) {
|
||||||
|
const outcome = outcomes[provider];
|
||||||
|
if (!outcome || outcome === 'success') {
|
||||||
|
return [{ type: 'delta', role: 'assistant', content: `response from ${provider}` },
|
||||||
|
{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
}
|
||||||
|
throw outcome;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const dummyIR = makeIR({ model: 'test-model' });
|
||||||
|
|
||||||
|
it('empty chain → throws Error', async () => {
|
||||||
|
let caught = null;
|
||||||
|
try {
|
||||||
|
await executeWithFallback([], dummyIR, makeHopFn({}));
|
||||||
|
} catch (e) {
|
||||||
|
caught = e;
|
||||||
|
}
|
||||||
|
assert.ok(caught instanceof Error, 'Expected Error for empty chain');
|
||||||
|
assert.ok(caught.message.includes('chain'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('single-hop chain with success → returns chunks + fallbackHops=0', async () => {
|
||||||
|
const chain = [{ provider: 'anthropic', model: 'claude-sonnet-4-6' }];
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, makeHopFn({ anthropic: 'success' }));
|
||||||
|
assert.ok(Array.isArray(result.chunks), 'Expected chunks array');
|
||||||
|
assert.equal(result.fallbackHops, 0);
|
||||||
|
assert.equal(result.providerUsed, 'anthropic');
|
||||||
|
assert.equal(result.originalError, null);
|
||||||
|
assert.deepEqual(result.triedProviders, ['anthropic']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('single-hop chain with hard-triggered error → exhausted, returns originalError', async () => {
|
||||||
|
const err = new ProviderError('Quota exhausted', 'QUOTA_EXHAUSTED');
|
||||||
|
const chain = [{ provider: 'anthropic', model: 'claude-sonnet-4-6' }];
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, makeHopFn({ anthropic: err }));
|
||||||
|
assert.equal(result.chunks, null, 'Expected null chunks on exhausted chain');
|
||||||
|
assert.equal(result.originalError, err);
|
||||||
|
assert.equal(result.fallbackHops, 1); // chain.length = 1, all exhausted
|
||||||
|
assert.deepEqual(result.triedProviders, ['anthropic']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('two-hop chain, primary fails with hard trigger → falls back to secondary, fallbackHops=1', async () => {
|
||||||
|
const err = new ProviderError('Spawn failed', 'SPAWN_FAILED');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
];
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, makeHopFn({ anthropic: err, openai: 'success' }));
|
||||||
|
assert.ok(Array.isArray(result.chunks), 'Expected chunks from secondary');
|
||||||
|
assert.equal(result.fallbackHops, 1);
|
||||||
|
assert.equal(result.providerUsed, 'openai');
|
||||||
|
assert.equal(result.originalError, null);
|
||||||
|
assert.deepEqual(result.triedProviders, ['anthropic', 'openai']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('three-hop chain, primary + secondary fail → tertiary returns chunks, fallbackHops=2', async () => {
|
||||||
|
const errA = new ProviderError('Rate limited', 'RATE_LIMITED');
|
||||||
|
const errB = Object.assign(new Error('Service unavailable'), { statusCode: 503 });
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
{ provider: 'mistral', model: 'devstral-2' },
|
||||||
|
];
|
||||||
|
const result = await executeWithFallback(
|
||||||
|
chain, dummyIR,
|
||||||
|
makeHopFn({ anthropic: errA, openai: errB, mistral: 'success' }),
|
||||||
|
);
|
||||||
|
assert.ok(Array.isArray(result.chunks));
|
||||||
|
assert.equal(result.fallbackHops, 2);
|
||||||
|
assert.equal(result.providerUsed, 'mistral');
|
||||||
|
assert.equal(result.originalError, null);
|
||||||
|
assert.deepEqual(result.triedProviders, ['anthropic', 'openai', 'mistral']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('three-hop chain, all fail → exhausted, originalError is from FIRST hop (not last)', async () => {
|
||||||
|
const errA = new ProviderError('Rate limited', 'RATE_LIMITED');
|
||||||
|
const errB = new ProviderError('Spawn failed', 'SPAWN_FAILED');
|
||||||
|
const errC = new ProviderError('CLI not found', 'CLI_NOT_FOUND');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
{ provider: 'mistral', model: 'devstral-2' },
|
||||||
|
];
|
||||||
|
const result = await executeWithFallback(
|
||||||
|
chain, dummyIR,
|
||||||
|
makeHopFn({ anthropic: errA, openai: errB, mistral: errC }),
|
||||||
|
);
|
||||||
|
assert.equal(result.chunks, null);
|
||||||
|
assert.equal(result.originalError, errA, 'Should be first hop error, not last');
|
||||||
|
assert.equal(result.fallbackHops, 3); // chain.length = 3
|
||||||
|
assert.deepEqual(result.triedProviders, ['anthropic', 'openai', 'mistral']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('client error (400) on primary → does NOT fall back, surfaces error immediately', async () => {
|
||||||
|
const err = Object.assign(new Error('Bad request'), { statusCode: 400 });
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
];
|
||||||
|
let openaiCalled = false;
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'anthropic') throw err;
|
||||||
|
openaiCalled = true;
|
||||||
|
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(result.chunks, null);
|
||||||
|
assert.equal(result.originalError, err);
|
||||||
|
assert.equal(openaiCalled, false, 'Second provider must NOT be called on client error');
|
||||||
|
assert.deepEqual(result.triedProviders, ['anthropic']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('AUTH_MISSING on primary → does NOT fall back (user must fix config)', async () => {
|
||||||
|
const err = new ProviderError('Auth missing', 'AUTH_MISSING');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
];
|
||||||
|
let openaiCalled = false;
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'anthropic') throw err;
|
||||||
|
openaiCalled = true;
|
||||||
|
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(result.chunks, null);
|
||||||
|
assert.equal(result.originalError, err);
|
||||||
|
assert.equal(openaiCalled, false, 'Second provider must NOT be called on AUTH_MISSING');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('non-trigger error (generic) on primary → does NOT fall back', async () => {
|
||||||
|
const err = new Error('Unexpected internal error');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
];
|
||||||
|
let openaiCalled = false;
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'anthropic') throw err;
|
||||||
|
openaiCalled = true;
|
||||||
|
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(result.chunks, null);
|
||||||
|
assert.equal(openaiCalled, false, 'Must not fall back on non-trigger error');
|
||||||
|
});
|
||||||
|
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── 13c: First-chunk safety ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
describe('Fallback engine — first-chunk safety (D9)', () => {
|
||||||
|
|
||||||
|
// At D9, executeHopFn = collectAllChunks() which is fully buffered.
|
||||||
|
// Safety is enforced by construction: if executeHopFn returns successfully,
|
||||||
|
// chunks are already in memory (none written to res yet). Once chunks are
|
||||||
|
// returned, fallback is no longer attempted — we return immediately.
|
||||||
|
// If executeHopFn throws, zero bytes went to the client.
|
||||||
|
|
||||||
|
it('successful executeHopFn returns chunks immediately without retrying secondary', async () => {
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
];
|
||||||
|
let callCount = 0;
|
||||||
|
const hopFn = async (_provider) => {
|
||||||
|
callCount++;
|
||||||
|
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, makeIR({ model: 'test' }), hopFn);
|
||||||
|
assert.equal(result.fallbackHops, 0, 'Primary served — no fallback');
|
||||||
|
assert.equal(callCount, 1, 'executeHopFn called exactly once (first-chunk safety)');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('error from executeHopFn (hard trigger) means zero chunks emitted — advance is safe', async () => {
|
||||||
|
// Simulate: anthropic throws (no bytes emitted), openai succeeds
|
||||||
|
const err = new ProviderError('Rate limited', 'RATE_LIMITED');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
];
|
||||||
|
let writtenToClient = false;
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'anthropic') {
|
||||||
|
// Throw BEFORE any "write" — simulates first-chunk rule
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
// openai succeeds — simulate "writtenToClient" only AFTER returning
|
||||||
|
// (in real server, chunks are written after executeWithFallback returns)
|
||||||
|
writtenToClient = true;
|
||||||
|
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, makeIR({ model: 'test' }), hopFn);
|
||||||
|
// openai served, which means writtenToClient is set (simulates post-return write)
|
||||||
|
assert.equal(result.fallbackHops, 1, 'Fell back to openai');
|
||||||
|
// Crucially: the "write" happened AFTER executeHopFn returned (not during chain iteration)
|
||||||
|
assert.equal(writtenToClient, true, 'Client write happens post-return, not during chain iteration');
|
||||||
|
});
|
||||||
|
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── 13d: Soft triggers skipping spawn ────────────────────────────────────
|
||||||
|
|
||||||
|
describe('Fallback engine — soft trigger skipping (D9)', () => {
|
||||||
|
|
||||||
|
const dummyIR = makeIR({ model: 'test-model' });
|
||||||
|
|
||||||
|
it('chain [A, B], A soft trigger fires → engine skips A entirely, calls B; fallbackHops=1', async () => {
|
||||||
|
let aCalled = false;
|
||||||
|
let bCalled = false;
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'a') { aCalled = true; return [{ type: 'stop', finish_reason: 'stop' }]; }
|
||||||
|
if (provider === 'b') { bCalled = true; return [{ type: 'stop', finish_reason: 'stop' }]; }
|
||||||
|
throw new Error('Unexpected provider');
|
||||||
|
};
|
||||||
|
const chain = [
|
||||||
|
{
|
||||||
|
provider: 'a',
|
||||||
|
model: 'model-a',
|
||||||
|
softTriggers: { credit_pool_percent_threshold: 90 },
|
||||||
|
quotaSnapshot: { percentUsed: 95 }, // fires the trigger
|
||||||
|
},
|
||||||
|
{ provider: 'b', model: 'model-b' },
|
||||||
|
];
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(aCalled, false, 'A must NOT be called (soft trigger skipped it)');
|
||||||
|
assert.equal(bCalled, true, 'B must be called');
|
||||||
|
assert.equal(result.fallbackHops, 1, 'B served as fallback hop 1');
|
||||||
|
assert.equal(result.providerUsed, 'b');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('chain [A, B], neither soft trigger fires → A is called; if A succeeds, B never called', async () => {
|
||||||
|
let aCalled = false;
|
||||||
|
let bCalled = false;
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'a') { aCalled = true; return [{ type: 'stop', finish_reason: 'stop' }]; }
|
||||||
|
if (provider === 'b') { bCalled = true; return [{ type: 'stop', finish_reason: 'stop' }]; }
|
||||||
|
throw new Error('Unexpected provider');
|
||||||
|
};
|
||||||
|
const chain = [
|
||||||
|
{
|
||||||
|
provider: 'a',
|
||||||
|
model: 'model-a',
|
||||||
|
softTriggers: { credit_pool_percent_threshold: 90 },
|
||||||
|
quotaSnapshot: { percentUsed: 85 }, // below threshold, trigger does NOT fire
|
||||||
|
},
|
||||||
|
{ provider: 'b', model: 'model-b' },
|
||||||
|
];
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(aCalled, true, 'A must be called');
|
||||||
|
assert.equal(bCalled, false, 'B must NOT be called (A succeeded)');
|
||||||
|
assert.equal(result.fallbackHops, 0);
|
||||||
|
assert.equal(result.providerUsed, 'a');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('chain [A], A soft trigger fires (null quotaSnapshot) → trigger does NOT fire, A called', async () => {
|
||||||
|
// Per ADR 0004: null quotaStatus → treat as "don't fire"
|
||||||
|
let aCalled = false;
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'a') { aCalled = true; return [{ type: 'stop', finish_reason: 'stop' }]; }
|
||||||
|
throw new Error('Unexpected provider');
|
||||||
|
};
|
||||||
|
const chain = [
|
||||||
|
{
|
||||||
|
provider: 'a',
|
||||||
|
model: 'model-a',
|
||||||
|
softTriggers: { credit_pool_percent_threshold: 90 },
|
||||||
|
quotaSnapshot: null, // null → trigger never fires
|
||||||
|
},
|
||||||
|
];
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(aCalled, true, 'A must be called (null quota → trigger does not fire)');
|
||||||
|
assert.equal(result.fallbackHops, 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── 13e: Header annotation ───────────────────────────────────────────────
|
||||||
|
|
||||||
|
describe('Fallback engine — observability / header annotation (D9)', () => {
|
||||||
|
|
||||||
|
const dummyIR = makeIR({ model: 'test-model' });
|
||||||
|
|
||||||
|
it('success on primary: providerUsed + modelUsed match primary hop', async () => {
|
||||||
|
const chain = [{ provider: 'anthropic', model: 'claude-sonnet-4-6' }];
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, async () => [{ type: 'stop', finish_reason: 'stop' }]);
|
||||||
|
assert.equal(result.providerUsed, 'anthropic');
|
||||||
|
assert.equal(result.modelUsed, 'claude-sonnet-4-6');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('success on fallback: providerUsed + modelUsed match the serving hop', async () => {
|
||||||
|
const err = new ProviderError('Rate limited', 'RATE_LIMITED');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
];
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'anthropic') throw err;
|
||||||
|
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(result.providerUsed, 'openai');
|
||||||
|
assert.equal(result.modelUsed, 'gpt-5.5');
|
||||||
|
assert.equal(result.fallbackHops, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('chain exhausted: originalError is from FIRST hop, not second or third', async () => {
|
||||||
|
const errA = new ProviderError('Rate limited', 'RATE_LIMITED');
|
||||||
|
const errB = new ProviderError('Spawn failed', 'SPAWN_FAILED');
|
||||||
|
const errC = new ProviderError('CLI not found', 'CLI_NOT_FOUND');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'a', model: 'model-a' },
|
||||||
|
{ provider: 'b', model: 'model-b' },
|
||||||
|
{ provider: 'c', model: 'model-c' },
|
||||||
|
];
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'a') throw errA;
|
||||||
|
if (provider === 'b') throw errB;
|
||||||
|
if (provider === 'c') throw errC;
|
||||||
|
throw new Error('Unexpected');
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.equal(result.originalError, errA, 'originalError must be first-hop error');
|
||||||
|
assert.notEqual(result.originalError, errB, 'Must NOT be second-hop error');
|
||||||
|
assert.notEqual(result.originalError, errC, 'Must NOT be third-hop error');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('triedProviders lists all attempted hops in chain order', async () => {
|
||||||
|
const err = new ProviderError('Rate limited', 'RATE_LIMITED');
|
||||||
|
const chain = [
|
||||||
|
{ provider: 'a', model: 'model-a' },
|
||||||
|
{ provider: 'b', model: 'model-b' },
|
||||||
|
{ provider: 'c', model: 'model-c' },
|
||||||
|
];
|
||||||
|
const hopFn = async (provider) => {
|
||||||
|
if (provider === 'a') throw err;
|
||||||
|
if (provider === 'b') throw err;
|
||||||
|
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||||
|
};
|
||||||
|
const result = await executeWithFallback(chain, dummyIR, hopFn);
|
||||||
|
assert.deepEqual(result.triedProviders, ['a', 'b', 'c']);
|
||||||
|
assert.equal(result.fallbackHops, 2);
|
||||||
|
assert.equal(result.providerUsed, 'c');
|
||||||
|
});
|
||||||
|
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── 13f: HTTP integration ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
describe('Fallback engine — HTTP integration (D9)', () => {
|
||||||
|
let server;
|
||||||
|
let port;
|
||||||
|
|
||||||
|
before(async () => {
|
||||||
|
__resetSpawnImpl();
|
||||||
|
codexResetSpawnImpl();
|
||||||
|
mistralResetSpawnImpl();
|
||||||
|
__resetFallbackConfig();
|
||||||
|
|
||||||
|
// Start test server with all 3 providers enabled
|
||||||
|
const testProviders = loadProviders({ enabled: { anthropic: true, openai: true, mistral: true } });
|
||||||
|
const { loadedProviders: lp, cacheStore: cs } = await import('./server.mjs');
|
||||||
|
// Inject test providers
|
||||||
|
for (const [name, p] of testProviders) {
|
||||||
|
lp.set(name, p);
|
||||||
|
}
|
||||||
|
cs.clear();
|
||||||
|
|
||||||
|
server = createOlpServer();
|
||||||
|
port = 20456 + Math.floor(Math.random() * 500);
|
||||||
|
await new Promise((resolve, reject) => {
|
||||||
|
server.listen(port, '127.0.0.1', resolve);
|
||||||
|
server.once('error', (e) => {
|
||||||
|
if (e.code === 'EADDRINUSE') {
|
||||||
|
port++;
|
||||||
|
server.listen(port, '127.0.0.1', resolve);
|
||||||
|
server.once('error', reject);
|
||||||
|
} else reject(e);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
after(() => {
|
||||||
|
__resetFallbackConfig();
|
||||||
|
__resetSpawnImpl();
|
||||||
|
codexResetSpawnImpl();
|
||||||
|
mistralResetSpawnImpl();
|
||||||
|
if (!server) return;
|
||||||
|
return new Promise(r => server.close(r));
|
||||||
|
});
|
||||||
|
|
||||||
|
// Shared mock spawn builder that returns a successful response
|
||||||
|
function makeSuccessSpawn(content = 'Hello from mock') {
|
||||||
|
return function mockSpawnImpl(_bin, _args, _opts) {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
proc.stdout = new EventEmitter();
|
||||||
|
proc.stderr = new EventEmitter();
|
||||||
|
proc.stdin = {
|
||||||
|
write: () => {},
|
||||||
|
end: () => {
|
||||||
|
setImmediate(() => {
|
||||||
|
// Emit a minimal SSE-like response that anthropic plugin parses
|
||||||
|
// (Anthropic mock format: JSON lines per the real anthropic.mjs parser)
|
||||||
|
const lines = [
|
||||||
|
JSON.stringify({ type: 'content_block_delta', index: 0, delta: { type: 'text_delta', text: content } }),
|
||||||
|
JSON.stringify({ type: 'message_stop' }),
|
||||||
|
];
|
||||||
|
for (const line of lines) {
|
||||||
|
proc.stdout.emit('data', Buffer.from(`data: ${line}\n\n`));
|
||||||
|
}
|
||||||
|
proc.stdout.emit('end');
|
||||||
|
proc.stderr.emit('end');
|
||||||
|
proc.emit('close', 0, null);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
};
|
||||||
|
proc.killed = false;
|
||||||
|
proc.kill = () => {};
|
||||||
|
return proc;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
it('no fallback config: POST with no enabled provider → 503', async () => {
|
||||||
|
// Remove all providers temporarily
|
||||||
|
const { loadedProviders: lp } = await import('./server.mjs');
|
||||||
|
const savedMap = new Map(lp);
|
||||||
|
lp.clear();
|
||||||
|
try {
|
||||||
|
const r = await fetch({
|
||||||
|
port,
|
||||||
|
method: 'POST',
|
||||||
|
path: '/v1/chat/completions',
|
||||||
|
body: { model: 'unknown-model', messages: [{ role: 'user', content: 'Hi' }] },
|
||||||
|
});
|
||||||
|
assert.equal(r.status, 503);
|
||||||
|
} finally {
|
||||||
|
for (const [name, p] of savedMap) lp.set(name, p);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('no fallback config + mock anthropic: POST claude-sonnet-4-6 → 200 + X-OLP-Fallback-Hops: 0', async () => {
|
||||||
|
__setSpawnImpl(makeSuccessSpawn('Test response'));
|
||||||
|
try {
|
||||||
|
const r = await fetch({
|
||||||
|
port,
|
||||||
|
method: 'POST',
|
||||||
|
path: '/v1/chat/completions',
|
||||||
|
body: {
|
||||||
|
model: 'claude-sonnet-4-6',
|
||||||
|
messages: [{ role: 'user', content: 'Hi' }],
|
||||||
|
max_tokens: 10,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
assert.equal(r.status, 200, `Expected 200, got ${r.status}: ${r.body.slice(0, 200)}`);
|
||||||
|
assert.equal(r.headers['x-olp-fallback-hops'], '0');
|
||||||
|
assert.equal(r.headers['x-olp-provider-used'], 'anthropic');
|
||||||
|
} finally {
|
||||||
|
__resetSpawnImpl();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('fallback config: mock anthropic fails (SPAWN_FAILED), chain [anthropic→openai], openai mock succeeds → 200 + X-OLP-Fallback-Hops: 1 + X-OLP-Provider-Used: openai', async () => {
|
||||||
|
// Clear cache to prevent cache-hit from previous test polluting this one.
|
||||||
|
// ADR 0005: each (provider, model) pair is independently cached; a hit
|
||||||
|
// from the previous test would cause executeHopFn to skip the spawn and
|
||||||
|
// return cached chunks, masking the SPAWN_FAILED mock.
|
||||||
|
__clearCache();
|
||||||
|
|
||||||
|
// Provide fake codex auth so AUTH_MISSING doesn't stop the chain before
|
||||||
|
// the codex mock can respond. Codex checks readAuthArtifact() before spawnImpl.
|
||||||
|
// Write a temp file and set OPENAI_CODEX_AUTH_PATH to point at it.
|
||||||
|
const { writeFileSync, unlinkSync } = await import('node:fs');
|
||||||
|
const { tmpdir } = await import('node:os');
|
||||||
|
const { join: pathJoin } = await import('node:path');
|
||||||
|
const tmpAuthFile = pathJoin(tmpdir(), `olp-test-codex-auth-${Date.now()}.json`);
|
||||||
|
writeFileSync(tmpAuthFile, JSON.stringify({ accessToken: 'fake-test-token-for-codex' }), 'utf8');
|
||||||
|
const savedCodexAuthPath = process.env.OPENAI_CODEX_AUTH_PATH;
|
||||||
|
process.env.OPENAI_CODEX_AUTH_PATH = tmpAuthFile;
|
||||||
|
|
||||||
|
// Set up a 2-hop chain: anthropic → openai for claude-sonnet-4-6
|
||||||
|
__setFallbackConfig({
|
||||||
|
chains: {
|
||||||
|
'claude-sonnet-4-6': [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
soft_triggers: {},
|
||||||
|
});
|
||||||
|
|
||||||
|
// Anthropic mock: always exits with code 1 → SPAWN_FAILED (hard trigger)
|
||||||
|
__setSpawnImpl(function (_bin, _args, _opts) {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
proc.stdout = new EventEmitter();
|
||||||
|
proc.stderr = new EventEmitter();
|
||||||
|
proc.stdin = {
|
||||||
|
write: () => {},
|
||||||
|
end: () => {
|
||||||
|
setImmediate(() => {
|
||||||
|
proc.stderr.emit('data', Buffer.from('spawn failed\n'));
|
||||||
|
proc.stdout.emit('end');
|
||||||
|
proc.stderr.emit('end');
|
||||||
|
proc.emit('close', 1, null); // non-zero exit → SPAWN_FAILED
|
||||||
|
});
|
||||||
|
},
|
||||||
|
};
|
||||||
|
proc.killed = false;
|
||||||
|
proc.kill = () => {};
|
||||||
|
return proc;
|
||||||
|
});
|
||||||
|
|
||||||
|
// Codex (openai) mock: succeeds with a delta+stop response
|
||||||
|
codexSetSpawnImpl(function (_bin, _args, _opts) {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
proc.stdout = new EventEmitter();
|
||||||
|
proc.stderr = new EventEmitter();
|
||||||
|
proc.stdin = {
|
||||||
|
write: () => {},
|
||||||
|
end: () => {
|
||||||
|
setImmediate(() => {
|
||||||
|
proc.stdout.emit('data', Buffer.from('{"content":"Fallback response from openai"}\n'));
|
||||||
|
proc.stdout.emit('data', Buffer.from('{"type":"stop"}\n'));
|
||||||
|
proc.stdout.emit('end');
|
||||||
|
proc.stderr.emit('end');
|
||||||
|
proc.emit('close', 0, null);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
};
|
||||||
|
proc.killed = false;
|
||||||
|
proc.kill = () => {};
|
||||||
|
return proc;
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
const r = await fetch({
|
||||||
|
port,
|
||||||
|
method: 'POST',
|
||||||
|
path: '/v1/chat/completions',
|
||||||
|
body: {
|
||||||
|
model: 'claude-sonnet-4-6',
|
||||||
|
messages: [{ role: 'user', content: 'Hi' }],
|
||||||
|
max_tokens: 10,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
assert.equal(r.status, 200, `Expected 200 on fallback, got ${r.status}: ${r.body.slice(0, 300)}`);
|
||||||
|
assert.equal(r.headers['x-olp-fallback-hops'], '1', `Expected fallback-hops=1, got: ${r.headers['x-olp-fallback-hops']}`);
|
||||||
|
assert.equal(r.headers['x-olp-provider-used'], 'openai', `Expected provider-used=openai, got: ${r.headers['x-olp-provider-used']}`);
|
||||||
|
} finally {
|
||||||
|
__resetFallbackConfig();
|
||||||
|
__resetSpawnImpl();
|
||||||
|
codexResetSpawnImpl();
|
||||||
|
if (savedCodexAuthPath !== undefined) {
|
||||||
|
process.env.OPENAI_CODEX_AUTH_PATH = savedCodexAuthPath;
|
||||||
|
} else {
|
||||||
|
delete process.env.OPENAI_CODEX_AUTH_PATH;
|
||||||
|
}
|
||||||
|
try { unlinkSync(tmpAuthFile); } catch { /* ignore */ }
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('fallback config: two-hop chain with both providers failing SPAWN_FAILED → exhausted-chain path → error response with X-OLP-Fallback-Exhausted header', async () => {
|
||||||
|
// NOTE on this test's renaming (D9 review-2): the original sonnet draft
|
||||||
|
// titled this as a "client error (400) → no fallback" HTTP test. But the
|
||||||
|
// anthropic plugin surfaces non-zero exit codes as ProviderError
|
||||||
|
// SPAWN_FAILED (a HARD trigger), not as a typed-400 client error. So at
|
||||||
|
// the HTTP integration layer we can't easily inject a synthetic 400
|
||||||
|
// without monkey-patching executeHopFn. The "client error stops chain
|
||||||
|
// immediately" semantic IS covered at the UNIT level in
|
||||||
|
// "Engine: two-hop chain primary client error 400 → secondary NOT
|
||||||
|
// called" (line 3289+ in this file). This HTTP test now exercises the
|
||||||
|
// complementary path: both hops fail with SPAWN_FAILED → chain
|
||||||
|
// exhausted → originalError surfaces with X-OLP-Fallback-Exhausted
|
||||||
|
// header listing both providers.
|
||||||
|
__clearCache();
|
||||||
|
|
||||||
|
// Set up a 2-hop chain
|
||||||
|
__setFallbackConfig({
|
||||||
|
chains: {
|
||||||
|
'claude-sonnet-4-6': [
|
||||||
|
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||||
|
{ provider: 'openai', model: 'gpt-5.5' },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
soft_triggers: {},
|
||||||
|
});
|
||||||
|
|
||||||
|
// Both providers fail with SPAWN_FAILED
|
||||||
|
__setSpawnImpl(function (_bin, _args, _opts) {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
proc.stdout = new EventEmitter();
|
||||||
|
proc.stderr = new EventEmitter();
|
||||||
|
proc.stdin = {
|
||||||
|
write: () => {},
|
||||||
|
end: () => {
|
||||||
|
setImmediate(() => {
|
||||||
|
proc.stdout.emit('end');
|
||||||
|
proc.stderr.emit('end');
|
||||||
|
proc.emit('close', 1, null);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
};
|
||||||
|
proc.killed = false;
|
||||||
|
proc.kill = () => {};
|
||||||
|
return proc;
|
||||||
|
});
|
||||||
|
|
||||||
|
codexSetSpawnImpl(function (_bin, _args, _opts) {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
proc.stdout = new EventEmitter();
|
||||||
|
proc.stderr = new EventEmitter();
|
||||||
|
proc.stdin = {
|
||||||
|
write: () => {},
|
||||||
|
end: () => {
|
||||||
|
setImmediate(() => {
|
||||||
|
proc.stdout.emit('end');
|
||||||
|
proc.stderr.emit('end');
|
||||||
|
proc.emit('close', 1, null);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
};
|
||||||
|
proc.killed = false;
|
||||||
|
proc.kill = () => {};
|
||||||
|
return proc;
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
const r = await fetch({
|
||||||
|
port,
|
||||||
|
method: 'POST',
|
||||||
|
path: '/v1/chat/completions',
|
||||||
|
body: {
|
||||||
|
model: 'claude-sonnet-4-6',
|
||||||
|
messages: [{ role: 'user', content: 'Hi' }],
|
||||||
|
max_tokens: 10,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
// Both providers fail → exhausted chain → 502 error response with exhausted header
|
||||||
|
assert.ok(r.status >= 400 && r.status < 600, `Expected error status, got ${r.status}`);
|
||||||
|
// X-OLP-Fallback-Exhausted header should be present since both providers tried
|
||||||
|
assert.ok(
|
||||||
|
r.headers['x-olp-fallback-exhausted'] !== undefined,
|
||||||
|
`Expected X-OLP-Fallback-Exhausted header, got headers: ${JSON.stringify(r.headers)}`,
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
__resetFallbackConfig();
|
||||||
|
__resetSpawnImpl();
|
||||||
|
codexResetSpawnImpl();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user