mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-19 09:45:07 +00:00
feat+docs+test: D40 — X-OLP-Fallback-Detail header (issue #7)
ADR 0004 § Decision § Chain advancement step 4 promised a per-hop
failure detail debug header `X-OLP-Fallback-Detail`. From D9 through
D39 the engine logged per-hop events (D28 added correlation fields)
but never surfaced the failure trail on the response. D40 fulfills
the promise via Option A — ungated v0.1 emission. Phase 2 will
re-introduce owner-vs-non-owner gating when `lib/keys.mjs` lands.
**Per-hop tuple collection** (lib/fallback/engine.mjs)
executeWithFallback now collects `fallbackDetail` — an array of per-hop
tuples — and returns it on EVERY return shape (success / client-error /
AUTH_MISSING / non-trigger / chain-exhausted). Soft-trigger skipped
hops are also recorded with `trigger_type: 'soft'` (currently dead-
code-by-config per ADR 0004 Amendment 2; shape forward-compatible).
Tuple shape (reuses D28 log-event field shapes so logs and the header
pivot on the same keys):
```
{
hop: <0-indexed hop number>,
provider: <provider name>,
model: <model name from IR>,
code: <ProviderError code, engine-synthetic SOFT_TRIGGER, or 'UNKNOWN'>,
error_message: <truncated to 200 chars with ellipsis on overflow>,
trigger_type: 'hard' | 'soft' | 'client_error' | 'auth_missing' | 'unclassified'
}
```
**Header serialiser** (server.mjs)
- `FALLBACK_DETAIL_BYTE_CAP = 4096` (UTF-8 bytes).
- `serializeFallbackDetailHeader(fallbackDetail)` exported. Returns
`null` for empty/null/undefined → caller omits the header.
- `jsonStringifyAscii` escapes every non-ASCII code point as `\uXXXX`
to satisfy RFC 7230 §3.2.6 field-vchar (Node's HTTP header validator
rejects multi-byte UTF-8). The D38 CONCURRENCY_LIMIT synthesised
message contains a U+2014 em dash — without this escape, every
CONCURRENCY_LIMIT response crashed at writeHead with
"Invalid character in header content". The regression test pins
this exact string.
- 4KB cap algorithm: builds candidates as `[...slice(0, kept), sentinel]`
and measures the FULL serialised length (including sentinel) before
comparing against the cap, so the result is guaranteed under cap.
Tail tuples dropped one at a time. Sentinel form:
`{ truncated: true, omitted_hops: N }`.
**Header emission** (server.mjs)
- `withFallbackDetailHeader(base, fallbackDetail)` wraps the base
header object; emits the header only when serialiser returns
non-null.
- Emitted on chain-exhausted, non-trigger-error, client-error,
AUTH_MISSING, and success-with-prior-failure paths.
- ABSENT on clean primary success — verified by a dedicated HTTP
integration test.
**Tests** (test-features.mjs): 452 → 468 (+16):
- Engine-level tuple shape: 2-hop/exhausted, 2-hop/success-with-prior-
failure, 1-hop/success (empty array), 1-hop/fail, non-ProviderError-
yields-UNKNOWN, 500-char-message → 200-char-with-ellipsis, client
error → 1 tuple + client_error trigger type
- Serialiser: empty/null → null, small array round-trip, >4KB cap
with `{truncated:true,omitted_hops:N}` sentinel, RFC 7230 newline/CR
escaping, non-ASCII escaping (em dash regression guard for the D38
CONCURRENCY_LIMIT synthesised message)
- HTTP integration: clean-1-hop-success (header absent), 2-hop-
exhausted (2 tuples on the wire), 2-hop-success-with-prior-failure
(1 tuple on the wire)
Pre-commit fold-in (per evidence-first checkpoint #4):
- **Reviewer Suggestion #2**: `jsonStringifyAscii` regex character
class `[U+0080-U+FFFF]` contains an invisible U+0080 boundary marker
that editors render as nothing, making the line easy to misread as
the empty class `[-...]`. Folded in a 5-line comment block above
the function body explaining the literal byte range and citing
RFC 7230 §3.2.6.
Two reviewer suggestions not folded:
- **Suggestion #1**: AUTH_MISSING tuple path lacks a dedicated D40
test. Code is structurally correct (tuple pushed before early-
return); low priority. Future polish.
- **Suggestion #3**: defensive `err.code != null` guard. Extremely
low priority — PROVIDER_ERROR_CODES is a closed enum with no
falsy values.
**ADR amendments** (docs/adr/0004-fallback-engine.md)
- Amendment 5 added at top of amendments stack — full tuple schema,
cap behavior, RFC 7230 hygiene, ungated v0.1 rationale, Phase 2
follow-up.
- § Chain advancement step 4 updated to remove TBD / not-yet-
implemented qualifiers and cross-reference Amendment 5.
- § Observability headers section gains the X-OLP-Fallback-Detail
schema as IMPLEMENTED at v0.1.
**CHANGELOG.md** — D40 sub-entry appended under existing D38/D39
entries in Unreleased section. No package.json bump per
phase_rolling_mode.
Authority:
- ADR 0004 § Decision § Chain advancement step 4 — D40 fulfils
the promise
- ADR 0004 Amendment 5 (this commit) — implementation contract
- ADR 0004 § Observability headers — updated to IMPLEMENTED state
- D18 (5 standard X-OLP-* headers) — D40 builds on this convention
- D28 (per-hop structured log fields) — D40 reuses the field shapes
- GitHub issue #7 — closed by this commit
- CC 开发铁律 v1.6 § 10.x — fresh-context opus reviewer independent
- CLAUDE.md release_kit_overlay phase_rolling_mode — no version bump
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus,
independent of drafter): APPROVE. Critical depth checks:
- Tuple pushed once per hop, BEFORE client-error / AUTH_MISSING
early-return branches — both paths include the failing tuple
- fallbackDetail returned on all 5 return paths (verified line by line)
- Cap algorithm measures FULL serialised length (with sentinel)
before comparing — no overshoot
- RFC 7230 compliance verified end-to-end: imported the function
in Node, confirmed em-dash → —, unescaped form throws
"Invalid character in header content"
- Serialiser handles null / undefined / [] gracefully → header
absent on clean primary success (HTTP test pins this)
- Hygiene: 0 hits for personal markers, home paths, tokens
- 468/468 tests pass in reviewer's independent npm test run
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.7
@@ -17,6 +17,16 @@ All notable changes to OLP land here. Per `CLAUDE.md` release_kit overlay, this
|
||||
- **Authority:** ADR 0005 § Cache layer / CacheStore API extension (Part 1); ADR 0004 Amendment 1 (Part 4); GitHub issue #3 — closed by this commit; D16 commit `bafa6d1` non-blocking suggestions — batched here.
|
||||
- **Test count:** 447 → 452 (3 unit tests for `CacheStore.delete` + 1 log-event integration test + 1 sticky-cache regression test).
|
||||
|
||||
### D40 — `X-OLP-Fallback-Detail` header (issue #7)
|
||||
|
||||
- **New debug header on responses with a non-empty failure trail** — `lib/fallback/engine.mjs#executeWithFallback` now returns a `fallbackDetail` array of per-hop tuples on every code path. `server.mjs` emits `X-OLP-Fallback-Detail: <JSON-stringified array>` on any response where at least one hop failed before the chain resolved or exhausted (chain-exhausted, non-trigger-error, client-error, AUTH_MISSING, and success-with-prior-failure paths). Header is absent on clean primary success (no failure trail to report).
|
||||
- **Tuple schema** — `{ hop, provider, model, code, error_message, trigger_type }` per failed hop. `code` is the `ProviderError` code or `'UNKNOWN'` for non-`ProviderError` exceptions; `error_message` is truncated to 200 chars with a U+2026 ellipsis on truncation; `trigger_type` matches D28's `classifyTrigger` output (`'hard'` / `'soft'` / `'auth_missing'` / `'client_error'` / `'non_trigger'`). Field shapes reuse D28's per-hop structured log event keys so logs and the header pivot on the same surface.
|
||||
- **4KB UTF-8 byte cap** — if the JSON-stringified array exceeds 4096 bytes, tail tuples are dropped and a `{ truncated: true, omitted_hops: N }` sentinel is appended such that the total fits under the cap. Cap calculation uses `Buffer.byteLength('utf8')`, not string length.
|
||||
- **RFC 7230 hygiene** — non-ASCII code points (e.g. the em dash in the D38 `CONCURRENCY_LIMIT` synthesised error message) are escaped as `\uXXXX` so the header value is pure ASCII. Node's HTTP header validator rejects multi-byte UTF-8 in field values; without this step, em-dash-bearing error messages would crash `res.writeHead`. `JSON.parse` round-trips the escaped form correctly.
|
||||
- **Gating posture — ungated at v0.1** — the original ADR 0004 § Chain advancement step 4 specified owner-only gating. Per the maintainer decision in issue #7, v0.1 ships the header **ungated** (single-tenant family-scale per ALIGNMENT.md; no PII risk in error details). **Phase 2 will re-introduce owner-vs-non-owner gating when `lib/keys.mjs` lands** — explicit follow-up tracked in AGENTS.md § Key files to know and ADR 0004 Amendment 5.
|
||||
- **Authority:** ADR 0004 § Decision § Chain advancement step 4 (original promise — D40 fulfils it); ADR 0004 Amendment 5 (D40 ratification); D18 (5 standard X-OLP-* headers; D40 builds on the convention); D28 (per-hop structured log fields; D40 reuses the field shapes); GitHub issue #7 — closed by this commit.
|
||||
- **Test count:** 452 → 468 (7 engine-level tuple-shape tests + 6 serialiser unit tests including the 4KB cap + non-ASCII regression + 3 HTTP integration tests).
|
||||
|
||||
## v0.1.0 — 2026-05-24
|
||||
|
||||
### Phase 1 Close — Multi-provider proxy core
|
||||
|
||||
@@ -7,6 +7,23 @@
|
||||
|
||||
## Amendments
|
||||
|
||||
### Amendment 5 — 2026-05-24: `X-OLP-Fallback-Detail` header shipped as ungated v0.1 (D40, issue #7)
|
||||
|
||||
- **Finding:** Step 4 of § Decision § Chain advancement (below) promised "per-provider failure detail in a debug header `X-OLP-Fallback-Detail` (JSON, owner-only — gated behind a config flag for non-owner keys)." From D9 through D39 the engine logged per-hop failure events via `fallback_hop_error` / `fallback_hard_trigger` / `fallback_client_error_no_fallback` / `fallback_auth_missing_no_fallback` / `fallback_non_trigger_error` (D28 added the `chain_id` / `trigger_type` / `ir_request_hash` / `next_provider` correlation fields), but the per-hop failure trail was not surfaced on the response. GitHub issue #7 tracked the gap.
|
||||
- **Change (D40):**
|
||||
- `lib/fallback/engine.mjs#executeWithFallback` now collects per-hop failure tuples in a new `fallbackDetail` array on the returned `FallbackResult`. Tuple shape reuses D28 log-event field shapes so logs and the header pivot on the same keys:
|
||||
`{ hop, provider, model, code, error_message, trigger_type }`. `code` is the `ProviderError` code, or any string `err.code` (including the engine-synthetic `SOFT_TRIGGER`), or `'UNKNOWN'` for non-`ProviderError` exceptions. `error_message` is truncated to 200 chars (single-character ellipsis `…` appended on truncation). `trigger_type` is the same classification surfaced in the D28 log events.
|
||||
- `server.mjs` emits the new header `X-OLP-Fallback-Detail: <JSON-stringified array>` on any response where `fallbackDetail` is non-empty — i.e., chain-exhausted, non-trigger-error, client-error, AUTH_MISSING, and success-with-prior-failure paths. Header is absent on clean primary success (semantically: no failure trail to report).
|
||||
- 4KB UTF-8 byte cap on the header value: if the serialised array exceeds 4096 bytes, tail tuples are dropped one at a time and a `{ truncated: true, omitted_hops: N }` sentinel is appended such that the total fits under the cap.
|
||||
- Non-ASCII characters in tuple fields (e.g. the em dash in the synthesised `CONCURRENCY_LIMIT` error message) are escaped as `\uXXXX` to satisfy RFC 7230 §3.2.6 `field-vchar` (Node's HTTP header validator rejects multi-byte UTF-8). `JSON.parse` round-trips the escaped form correctly.
|
||||
- **Gating — Option A (ungated v0.1):** The original promise specified owner-only gating. Per the maintainer decision recorded in issue #7, v0.1 ships the header **ungated**: the failure detail is surfaced on every response regardless of API key identity. Rationale: OLP v0.1 is single-tenant family-scale (per ALIGNMENT.md § What this project is); no PII risk in error details. **Phase 2 will re-introduce owner-vs-non-owner gating when `lib/keys.mjs` lands** — this is an explicit follow-up tracked in AGENTS.md § Key files to know and in the lib/keys.mjs Phase 2 planning. Until then, the header is informational on every response and operators should not assume per-key visibility differs.
|
||||
- **Authority:** § Decision § Chain advancement step 4 (original promise — D40 fulfils it); D18 (5 standard X-OLP-* headers; D40 builds on this convention); D28 (per-hop structured log fields; D40 reuses the field shapes); GitHub issue #7 — closed by this commit.
|
||||
- **Tests (test-features.mjs):** New describe block "D40 — X-OLP-Fallback-Detail header (issue #7)" covers: engine-level tuple shape on 2-hop/exhausted, 2-hop/success-with-prior-failure, 1-hop/success (empty array), 1-hop/fail, non-ProviderError-yields-`UNKNOWN`, 500-char-message → 200-char-with-ellipsis, client error → 1 tuple + `client_error` trigger type; serialiser-level empty/null → null, small-array round-trip, >4KB cap with `{truncated:true,omitted_hops:N}` sentinel, RFC 7230 newline/CR escaping, and non-ASCII escaping (em dash regression guard for the D38 `CONCURRENCY_LIMIT` synthesised message); HTTP integration covers clean-1-hop-success (header absent), 2-hop-exhausted (2 tuples on the wire), and 2-hop-success-with-prior-failure (1 tuple on the wire). Test count 452 → 468 (16 new tests).
|
||||
- **v1.x re-evaluation triggers:**
|
||||
- When `lib/keys.mjs` lands (Phase 2), re-introduce owner-vs-non-owner gating. Update this amendment + § Observability headers below + AGENTS.md.
|
||||
- If a future debug-header field becomes useful (e.g., `attempts`, `cache_eviction_count`, `last_chunk_index`), add to the tuple schema documented above + bump this amendment + extend the test schema assertions.
|
||||
- **Procedural mechanism:** CC 开发铁律 v1.6 § 10.x (D40 issue #7 implementation; fresh-context opus reviewer to follow per Iron Rule 10).
|
||||
|
||||
### Amendment 4 — 2026-05-24: Add `CONCURRENCY_LIMIT` to v0.1 hard-trigger code taxonomy (D38, issue #1)
|
||||
|
||||
- **Finding:** ADR 0002 Amendment 1 (2026-05-23) ratified `hints.maxConcurrent` into the Provider contract as **declarative-only at v0.1** — no runtime enforcement. GitHub issue #1 tracked the gap. D38 lands runtime enforcement (see ADR 0002 Amendment 6 for the implementation details and design rationale). Once a saturation event occurs, the orchestration layer must communicate "this hop is at capacity — advance the chain" to the fallback engine using a code that fits the existing hard-trigger taxonomy in `evaluateHardTriggers`.
|
||||
@@ -141,7 +158,7 @@ This is non-negotiable for v1.0. Post-first-chunk truncations are surfaced to th
|
||||
1. Try A. If A succeeds, return; emit `X-OLP-Fallback-Hops: 0`, `X-OLP-Provider-Used: A`.
|
||||
2. If A's failure matches a hard or soft trigger AND no chunks emitted: try B. If B succeeds, return; emit `X-OLP-Fallback-Hops: 1`, `X-OLP-Provider-Used: B`.
|
||||
3. If B also fails: try C. Same logic.
|
||||
4. If all of A, B, C fail: return **A's original error** (not B's, not C's) with `X-OLP-Fallback-Exhausted: A,B,C` listing the chain order, and per-provider failure detail in a debug header `X-OLP-Fallback-Detail` (JSON, owner-only — gated behind a config flag for non-owner keys).
|
||||
4. If all of A, B, C fail: return **A's original error** (not B's, not C's) with `X-OLP-Fallback-Exhausted: A,B,C` listing the chain order, and per-provider failure detail in a debug header `X-OLP-Fallback-Detail` (JSON, ungated at v0.1 per Amendment 5 / D40, issue #7; owner-vs-non-owner gating planned for Phase 2 when `lib/keys.mjs` lands). The header is also emitted on success-with-prior-failure paths (e.g., A fails + B succeeds → response carries the 1-tuple failure trail for A). See § Observability headers below for the tuple schema and cap behaviour.
|
||||
|
||||
**Observability headers (per spec §4.7).**
|
||||
- `X-OLP-Provider-Used: <provider-name>`
|
||||
@@ -149,6 +166,12 @@ This is non-negotiable for v1.0. Post-first-chunk truncations are surfaced to th
|
||||
- `X-OLP-Fallback-Hops: <integer ≥ 0>`
|
||||
- `X-OLP-Cache: hit | miss | bypass`
|
||||
- `X-OLP-Latency-Ms: <integer>`
|
||||
- `X-OLP-Fallback-Exhausted: <comma-separated provider list>` — emitted only when multiple providers were tried (D18; chain-exhaustion path).
|
||||
- `X-OLP-Fallback-Detail: <JSON array>` — **shipped as IMPLEMENTED at v0.1, ungated** per Amendment 5 (D40, issue #7). Emitted on any response where at least one prior hop failed before the chain resolved or exhausted; absent on clean primary success.
|
||||
- **Tuple schema (per failed hop):** `{ hop: <0-indexed integer>, provider: <string>, model: <string>, code: <ProviderError.code or 'UNKNOWN'>, error_message: <string truncated to 200 chars with U+2026 ellipsis on truncation>, trigger_type: 'hard' | 'soft' | 'auth_missing' | 'client_error' | 'non_trigger' }`. Field shapes reuse D28's per-hop log event keys.
|
||||
- **4KB UTF-8 byte cap:** if the JSON-stringified array exceeds 4096 bytes, tail tuples are dropped and a `{ truncated: true, omitted_hops: <count> }` sentinel is appended so the total fits under the cap.
|
||||
- **RFC 7230 hygiene:** all non-ASCII code points are escaped as `\uXXXX` so the header value is pure ASCII (Node's HTTP header validator rejects multi-byte UTF-8). `JSON.parse` still round-trips the escaped form to the original Unicode.
|
||||
- **Phase 2 follow-up:** owner-vs-non-owner gating is planned for when `lib/keys.mjs` lands. Until then, the header is informational on every response. See Amendment 5 for the full rationale and the gating re-introduction trigger.
|
||||
|
||||
Each fallback hop emits a structured log event with: timestamp, chain id, hop index, failed provider, trigger type, IR request hash, downstream provider that was tried next.
|
||||
|
||||
|
||||
+97
-7
@@ -227,6 +227,18 @@ export function evaluateSoftTriggers(triggerConfig, quotaSnapshot) {
|
||||
* @property {object|null} [quotaSnapshot] — optional pre-fetched quota snapshot
|
||||
*/
|
||||
|
||||
/**
|
||||
* @typedef {Object} FallbackDetailTuple — per-hop failure detail tuple emitted in X-OLP-Fallback-Detail.
|
||||
* D40 (issue #7) — Option A (ungated v0.1). Shapes reuse D28 log event fields so future readers
|
||||
* can grep both surfaces consistently.
|
||||
* @property {number} hop — 0-indexed hop number
|
||||
* @property {string} provider — provider name at this hop
|
||||
* @property {string} model — model string at this hop (from chain hop, which carries IR model)
|
||||
* @property {string} code — ProviderError code, or 'UNKNOWN' for non-ProviderError exceptions
|
||||
* @property {string} error_message — error message, truncated to 200 chars
|
||||
* @property {string} trigger_type — classifyTrigger() output: 'hard' | 'auth_missing' | 'client_error' | 'non_trigger' | 'soft' for engine-synthesized soft skips
|
||||
*/
|
||||
|
||||
/**
|
||||
* @typedef {Object} FallbackResult
|
||||
* @property {Array<object>|null} chunks — IR chunk array on success; null if exhausted
|
||||
@@ -235,8 +247,65 @@ export function evaluateSoftTriggers(triggerConfig, quotaSnapshot) {
|
||||
* @property {number} fallbackHops — chain index of the serving hop (0=primary, 1=first fallback, etc.)
|
||||
* @property {Error|null} originalError — first-hop error if exhausted; null on success
|
||||
* @property {string[]} triedProviders — all providers tried, in chain order
|
||||
* @property {FallbackDetailTuple[]} fallbackDetail — per-hop failure tuples (D40, issue #7).
|
||||
* On success, contains the failing hops before the serving hop (may be empty).
|
||||
* On exhausted/non-trigger/client-error/auth-missing return paths, contains every failed hop.
|
||||
* Server.mjs emits X-OLP-Fallback-Detail when this array is non-empty.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Truncates an error message to at most 200 characters. If truncation occurs,
|
||||
* the result ends with a single-character ellipsis (U+2026 '…') to signal
|
||||
* the cut. Used to keep X-OLP-Fallback-Detail tuples readable in dashboards.
|
||||
*
|
||||
* D40 (issue #7) — see ADR 0004 § Observability headers.
|
||||
*
|
||||
* @param {unknown} message
|
||||
* @returns {string}
|
||||
*/
|
||||
function truncateErrorMessage(message) {
|
||||
const s = typeof message === 'string' ? message : String(message ?? '');
|
||||
if (s.length <= 200) return s;
|
||||
return s.slice(0, 199) + '…';
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the per-hop failure tuple emitted in X-OLP-Fallback-Detail.
|
||||
* Field shapes reuse D28's structured log event values so the header and
|
||||
* the log line are pivotable on the same keys.
|
||||
*
|
||||
* D40 (issue #7) — see ADR 0004 § Observability headers.
|
||||
*
|
||||
* @param {number} hop
|
||||
* @param {string} provider
|
||||
* @param {string} model
|
||||
* @param {Error} err
|
||||
* @param {'hard'|'soft'|'auth_missing'|'client_error'|'non_trigger'|null} triggerType
|
||||
* @returns {FallbackDetailTuple}
|
||||
*/
|
||||
function makeFallbackDetailTuple(hop, provider, model, err, triggerType) {
|
||||
let code;
|
||||
if (err instanceof ProviderError && err.code) {
|
||||
code = err.code;
|
||||
} else if (typeof err?.code === 'string') {
|
||||
// Carries err.code from soft-trigger synthesized errors (code: 'SOFT_TRIGGER')
|
||||
// or any custom error class that uses string codes. Non-string err.code
|
||||
// falls through to 'UNKNOWN' so a numeric Node errno (e.g. ECONNREFUSED's
|
||||
// numeric system errno) does not get mis-typed.
|
||||
code = err.code;
|
||||
} else {
|
||||
code = 'UNKNOWN';
|
||||
}
|
||||
return {
|
||||
hop,
|
||||
provider,
|
||||
model,
|
||||
code,
|
||||
error_message: truncateErrorMessage(err?.message),
|
||||
trigger_type: triggerType ?? 'non_trigger',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Executes a provider chain with fallback semantics.
|
||||
*
|
||||
@@ -276,6 +345,12 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
let originalError = null; // Per ADR 0004: first-hop error is the canonical signal
|
||||
let firstErrorRecorded = false;
|
||||
|
||||
// D40 (issue #7) — per-hop failure tuples for X-OLP-Fallback-Detail.
|
||||
// Reuses D28 log event field shapes; emitted by server.mjs on any response
|
||||
// where this array is non-empty. ADR 0004 § Observability headers.
|
||||
/** @type {FallbackDetailTuple[]} */
|
||||
const fallbackDetail = [];
|
||||
|
||||
for (let i = 0; i < chain.length; i++) {
|
||||
const hop = chain[i];
|
||||
const { provider, model, softTriggers = null, quotaSnapshot = null } = hop;
|
||||
@@ -313,13 +388,17 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
// should treat 'SOFT_TRIGGER' as engine-synthetic. D9 review-2 noted
|
||||
// this; documented here so future readers do not try to add SOFT_TRIGGER
|
||||
// to the PROVIDER_ERROR_CODES closed enum.
|
||||
const softErr = Object.assign(
|
||||
new Error(`Soft trigger fired for provider ${provider}: quota threshold exceeded`),
|
||||
{ code: 'SOFT_TRIGGER', provider },
|
||||
);
|
||||
if (!firstErrorRecorded) {
|
||||
originalError = Object.assign(
|
||||
new Error(`Soft trigger fired for provider ${provider}: quota threshold exceeded`),
|
||||
{ code: 'SOFT_TRIGGER', provider },
|
||||
);
|
||||
originalError = softErr;
|
||||
firstErrorRecorded = true;
|
||||
}
|
||||
// D40: record soft-skipped hop in fallbackDetail. trigger_type='soft' lets
|
||||
// downstream readers distinguish a skipped hop from a spawned-and-failed one.
|
||||
fallbackDetail.push(makeFallbackDetailTuple(i, provider, model, softErr, 'soft'));
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -348,6 +427,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
fallbackHops: i,
|
||||
originalError: null,
|
||||
triedProviders,
|
||||
fallbackDetail, // D40: failing hops that came BEFORE this success
|
||||
};
|
||||
} catch (err) {
|
||||
// Record FIRST hop error as the canonical signal
|
||||
@@ -357,6 +437,12 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
firstErrorRecorded = true;
|
||||
}
|
||||
|
||||
// D40 (issue #7): classify once and record the per-hop tuple. The same
|
||||
// trigger_type value flows into the log event below (consistency between
|
||||
// logs and X-OLP-Fallback-Detail).
|
||||
const errTriggerType = classifyTrigger(err);
|
||||
fallbackDetail.push(makeFallbackDetailTuple(i, provider, model, err, errTriggerType));
|
||||
|
||||
logEvent('warn', 'fallback_hop_error', {
|
||||
chain_id: chainId,
|
||||
hop: i,
|
||||
@@ -364,7 +450,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
model,
|
||||
error: err.message,
|
||||
code: err.code ?? null,
|
||||
trigger_type: classifyTrigger(err),
|
||||
trigger_type: errTriggerType,
|
||||
ir_request_hash: irRequestHash,
|
||||
next_provider: chain[i + 1]?.provider ?? null,
|
||||
});
|
||||
@@ -389,6 +475,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
fallbackHops: i,
|
||||
originalError: err,
|
||||
triedProviders,
|
||||
fallbackDetail, // D40
|
||||
};
|
||||
}
|
||||
|
||||
@@ -412,6 +499,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
fallbackHops: i,
|
||||
originalError: err,
|
||||
triedProviders,
|
||||
fallbackDetail, // D40
|
||||
};
|
||||
}
|
||||
|
||||
@@ -428,7 +516,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
error: err.message,
|
||||
code: err.code ?? null,
|
||||
advance_to_hop: i + 1,
|
||||
trigger_type: classifyTrigger(err),
|
||||
trigger_type: errTriggerType,
|
||||
ir_request_hash: irRequestHash,
|
||||
next_provider: chain[i + 1]?.provider ?? null,
|
||||
});
|
||||
@@ -443,7 +531,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
provider,
|
||||
error: err.message,
|
||||
code: err.code ?? null,
|
||||
trigger_type: classifyTrigger(err),
|
||||
trigger_type: errTriggerType,
|
||||
ir_request_hash: irRequestHash,
|
||||
next_provider: null,
|
||||
});
|
||||
@@ -454,6 +542,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
fallbackHops: i,
|
||||
originalError: err,
|
||||
triedProviders,
|
||||
fallbackDetail, // D40
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -477,6 +566,7 @@ export async function executeWithFallback(chain, irRequest, executeHopFn, option
|
||||
fallbackHops: chain.length,
|
||||
originalError,
|
||||
triedProviders,
|
||||
fallbackDetail, // D40 (issue #7): per-hop failure tuples; every attempted hop on exhaustion
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
+123
-2
@@ -272,6 +272,115 @@ function olpErrorHeaders({ startMs, model }) {
|
||||
};
|
||||
}
|
||||
|
||||
// ── X-OLP-Fallback-Detail (D40, issue #7) ─────────────────────────────────
|
||||
|
||||
/**
|
||||
* 4KB UTF-8 byte cap on the X-OLP-Fallback-Detail header value. Per RFC 7230 §3.2.5
|
||||
* intermediaries are not obligated to forward arbitrarily large header values;
|
||||
* 4KB matches the conservative upper bound (well below the 8KB total-header
|
||||
* default of common reverse proxies — nginx `large_client_header_buffers`,
|
||||
* Apache `LimitRequestFieldSize`). Tuples beyond the cap are dropped and
|
||||
* a sentinel { truncated:true, omitted_hops:N } is appended.
|
||||
*/
|
||||
export const FALLBACK_DETAIL_BYTE_CAP = 4096;
|
||||
|
||||
/**
|
||||
* Serialises per-hop fallback failure tuples into the X-OLP-Fallback-Detail
|
||||
* header value. Returns null if the input array is empty/missing (header
|
||||
* should not be emitted in that case).
|
||||
*
|
||||
* D40 (issue #7) — see ADR 0004 § Observability headers.
|
||||
*
|
||||
* Cap behaviour:
|
||||
* - JSON.stringify the tuples; if Buffer.byteLength <= 4096, return as-is.
|
||||
* - Otherwise, drop tuples from the tail one at a time until the array PLUS
|
||||
* a trailing { truncated:true, omitted_hops:N } sentinel fits under the cap.
|
||||
* - If even a single tuple + sentinel cannot fit (extremely long error_message
|
||||
* beyond engine truncation, e.g. very long provider/model names), return
|
||||
* just the sentinel { truncated:true, omitted_hops:<all> } — never produce
|
||||
* a value > 4096 bytes.
|
||||
*
|
||||
* RFC 7230 hygiene: JSON.stringify already escapes raw newlines (\n → \\n),
|
||||
* carriage returns, and other control characters. In addition, we escape all
|
||||
* non-ASCII code points to \uXXXX sequences because Node's HTTP header
|
||||
* validator rejects multi-byte UTF-8 in field values (and RFC 7230 §3.2.6
|
||||
* limits `field-vchar` to ASCII VCHAR / obs-text). Without this step, an
|
||||
* em dash (U+2014) in a synthesised error message — e.g. the CONCURRENCY_LIMIT
|
||||
* message produced in server.mjs `collectAllChunks` — would trigger
|
||||
* `Invalid character in header content` from `res.writeHead`.
|
||||
*
|
||||
* @param {Array<object>|null|undefined} fallbackDetail — from FallbackResult.fallbackDetail
|
||||
* @returns {string|null} — header value, or null to skip emission
|
||||
*/
|
||||
export function serializeFallbackDetailHeader(fallbackDetail) {
|
||||
if (!Array.isArray(fallbackDetail) || fallbackDetail.length === 0) {
|
||||
return null;
|
||||
}
|
||||
const full = jsonStringifyAscii(fallbackDetail);
|
||||
if (Buffer.byteLength(full, 'utf8') <= FALLBACK_DETAIL_BYTE_CAP) {
|
||||
return full;
|
||||
}
|
||||
|
||||
// Cap exceeded — drop tail tuples until [...kept, sentinel] fits.
|
||||
// Linear scan from the full array down to 0 kept tuples. Worst case O(n^2)
|
||||
// on serialisation length, but n is bounded by chain length (small) so this
|
||||
// is fine in practice.
|
||||
for (let kept = fallbackDetail.length - 1; kept >= 0; kept--) {
|
||||
const omitted = fallbackDetail.length - kept;
|
||||
const sentinel = { truncated: true, omitted_hops: omitted };
|
||||
const candidate = jsonStringifyAscii([...fallbackDetail.slice(0, kept), sentinel]);
|
||||
if (Buffer.byteLength(candidate, 'utf8') <= FALLBACK_DETAIL_BYTE_CAP) {
|
||||
return candidate;
|
||||
}
|
||||
}
|
||||
|
||||
// Even an array containing only the sentinel exceeds the cap — produce the
|
||||
// shortest possible valid sentinel value. This branch should be unreachable
|
||||
// for any realistic chain (the sentinel itself is ~45 bytes for omitted_hops
|
||||
// up to 9999).
|
||||
return jsonStringifyAscii([{ truncated: true, omitted_hops: fallbackDetail.length }]);
|
||||
}
|
||||
|
||||
/**
|
||||
* JSON.stringify wrapper that escapes every non-ASCII code point as \uXXXX
|
||||
* so the result is safe to embed in an HTTP header value (RFC 7230 §3.2.6
|
||||
* field-vchar). JSON itself accepts both literal Unicode and \uXXXX escapes,
|
||||
* so JSON.parse round-trips correctly.
|
||||
*
|
||||
* Surrogate-pair handling: characters above U+FFFF (emoji etc.) are already
|
||||
* emitted as JS surrogate pairs by the string iterator; each surrogate is
|
||||
* a code unit in range 0xD800–0xDFFF, which our >= 0x80 guard catches.
|
||||
*
|
||||
* @param {unknown} value
|
||||
* @returns {string}
|
||||
*/
|
||||
function jsonStringifyAscii(value) {
|
||||
// The replace pattern is the UTF-8 literal byte range for code points
|
||||
// U+0080..U+FFFF (every non-ASCII BMP character). U+0080 is non-printable,
|
||||
// so the source line can render as the empty character class "[-...]" in
|
||||
// editors that hide it — the range is intentional and load-bearing for
|
||||
// RFC 7230 §3.2.6 compliance.
|
||||
return JSON.stringify(value).replace(/[-]/g, (ch) => {
|
||||
return '\\u' + ch.charCodeAt(0).toString(16).padStart(4, '0');
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Merges X-OLP-Fallback-Detail into a base header object when the per-hop
|
||||
* failure tuples are non-empty. Returns the base object unchanged otherwise.
|
||||
*
|
||||
* D40 (issue #7).
|
||||
*
|
||||
* @param {Record<string,string>} baseHeaders
|
||||
* @param {Array<object>|null|undefined} fallbackDetail
|
||||
* @returns {Record<string,string>}
|
||||
*/
|
||||
function withFallbackDetailHeader(baseHeaders, fallbackDetail) {
|
||||
const value = serializeFallbackDetailHeader(fallbackDetail);
|
||||
if (value === null) return baseHeaders;
|
||||
return { ...baseHeaders, 'X-OLP-Fallback-Detail': value };
|
||||
}
|
||||
|
||||
// ── Route handlers ────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
@@ -894,6 +1003,7 @@ async function handleChatCompletions(req, res) {
|
||||
fallbackHops,
|
||||
originalError,
|
||||
triedProviders,
|
||||
fallbackDetail, // D40 (issue #7): per-hop failure tuples for X-OLP-Fallback-Detail
|
||||
} = fallbackResult;
|
||||
|
||||
// ── Chain exhausted or non-trigger error ─────────────────────────────────
|
||||
@@ -943,18 +1053,23 @@ async function handleChatCompletions(req, res) {
|
||||
fallbackHops: fallbackHops ?? 0,
|
||||
});
|
||||
|
||||
// Send error with standard OLP headers + optional exhausted header
|
||||
// Send error with standard OLP headers + optional exhausted header +
|
||||
// D40 X-OLP-Fallback-Detail (when any hop attempted to spawn failed).
|
||||
// D40 (issue #7) — ungated v0.1 per maintainer decision; owner-vs-non-owner
|
||||
// gating planned for Phase 2 with lib/keys.mjs.
|
||||
const payload = JSON.stringify({
|
||||
error: {
|
||||
message: originalError?.message ?? 'Provider error',
|
||||
type: 'provider_error',
|
||||
},
|
||||
});
|
||||
const detailHeader = withFallbackDetailHeader({}, fallbackDetail);
|
||||
res.writeHead(errStatus, {
|
||||
'Content-Type': 'application/json',
|
||||
'Content-Length': Buffer.byteLength(payload),
|
||||
...errorOlpHeaders,
|
||||
...exhaustedHeader,
|
||||
...detailHeader,
|
||||
});
|
||||
res.end(payload);
|
||||
return;
|
||||
@@ -974,7 +1089,13 @@ async function handleChatCompletions(req, res) {
|
||||
const cacheStatus = bypassCacheForServingHop ? 'bypass'
|
||||
: (lastHopWasCached || (preCheckHit && fallbackHops === 0)) ? 'hit'
|
||||
: 'miss';
|
||||
const headers = olpHeaders({ providerUsed, modelUsed, startMs, cacheStatus, fallbackHops });
|
||||
// D40 (issue #7): when at least one prior hop failed before this success,
|
||||
// surface X-OLP-Fallback-Detail with the failure trail. Header is omitted
|
||||
// when fallbackDetail is empty (single-hop success).
|
||||
const headers = withFallbackDetailHeader(
|
||||
olpHeaders({ providerUsed, modelUsed, startMs, cacheStatus, fallbackHops }),
|
||||
fallbackDetail,
|
||||
);
|
||||
|
||||
if (ir.stream) {
|
||||
// Streaming response path: burst replay from buffered chunks.
|
||||
|
||||
@@ -5586,6 +5586,219 @@ describe('Fallback engine — HTTP integration (D9)', () => {
|
||||
}
|
||||
});
|
||||
|
||||
// ── D40 HTTP integration tests (issue #7): X-OLP-Fallback-Detail on the wire ─
|
||||
|
||||
it('D40: 1-hop chain succeeds → X-OLP-Fallback-Detail header is absent (no failures to report)', async () => {
|
||||
__clearCache();
|
||||
__setSpawnImpl(makeSuccessSpawn('D40 clean success'));
|
||||
try {
|
||||
const r = await fetch({
|
||||
port,
|
||||
method: 'POST',
|
||||
path: '/v1/chat/completions',
|
||||
body: {
|
||||
model: 'claude-sonnet-4-6',
|
||||
messages: [{ role: 'user', content: 'Hi' }],
|
||||
max_tokens: 10,
|
||||
},
|
||||
});
|
||||
assert.equal(r.status, 200, `Expected 200, got ${r.status}: ${r.body.slice(0, 200)}`);
|
||||
assert.equal(r.headers['x-olp-fallback-hops'], '0');
|
||||
assert.equal(
|
||||
r.headers['x-olp-fallback-detail'],
|
||||
undefined,
|
||||
`Header must be absent on clean primary success, got: ${r.headers['x-olp-fallback-detail']}`,
|
||||
);
|
||||
} finally {
|
||||
__resetSpawnImpl();
|
||||
}
|
||||
});
|
||||
|
||||
it('D40: 2-hop chain, both fail → response carries X-OLP-Fallback-Detail with 2 tuples', async () => {
|
||||
__clearCache();
|
||||
__setFallbackConfig({
|
||||
chains: {
|
||||
'claude-sonnet-4-6': [
|
||||
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||
{ provider: 'openai', model: 'gpt-5.5' },
|
||||
],
|
||||
},
|
||||
soft_triggers: {},
|
||||
});
|
||||
|
||||
// Anthropic mock: exits non-zero with stderr message → SPAWN_FAILED
|
||||
__setSpawnImpl(function (_bin, _args, _opts) {
|
||||
const proc = new EventEmitter();
|
||||
proc.stdout = new EventEmitter();
|
||||
proc.stderr = new EventEmitter();
|
||||
proc.stdin = {
|
||||
write: () => {},
|
||||
end: () => {
|
||||
setImmediate(() => {
|
||||
proc.stderr.emit('data', Buffer.from('anthropic-stderr\n'));
|
||||
proc.stdout.emit('end');
|
||||
proc.stderr.emit('end');
|
||||
proc.emit('close', 1, null);
|
||||
});
|
||||
},
|
||||
};
|
||||
proc.killed = false;
|
||||
proc.kill = () => {};
|
||||
return proc;
|
||||
});
|
||||
// Codex (openai) mock: also fails → exhausted chain
|
||||
codexSetSpawnImpl(function (_bin, _args, _opts) {
|
||||
const proc = new EventEmitter();
|
||||
proc.stdout = new EventEmitter();
|
||||
proc.stderr = new EventEmitter();
|
||||
proc.stdin = {
|
||||
write: () => {},
|
||||
end: () => {
|
||||
setImmediate(() => {
|
||||
proc.stderr.emit('data', Buffer.from('codex-stderr\n'));
|
||||
proc.stdout.emit('end');
|
||||
proc.stderr.emit('end');
|
||||
proc.emit('close', 1, null);
|
||||
});
|
||||
},
|
||||
};
|
||||
proc.killed = false;
|
||||
proc.kill = () => {};
|
||||
return proc;
|
||||
});
|
||||
|
||||
try {
|
||||
const r = await fetch({
|
||||
port,
|
||||
method: 'POST',
|
||||
path: '/v1/chat/completions',
|
||||
body: {
|
||||
model: 'claude-sonnet-4-6',
|
||||
messages: [{ role: 'user', content: 'Hi' }],
|
||||
max_tokens: 10,
|
||||
},
|
||||
});
|
||||
assert.ok(r.status >= 400 && r.status < 600, `Expected error status, got ${r.status}`);
|
||||
assert.ok(
|
||||
r.headers['x-olp-fallback-detail'] !== undefined,
|
||||
`Expected X-OLP-Fallback-Detail header on exhausted chain, headers: ${JSON.stringify(r.headers)}`,
|
||||
);
|
||||
const parsed = JSON.parse(r.headers['x-olp-fallback-detail']);
|
||||
assert.ok(Array.isArray(parsed), 'Header must JSON-parse to array');
|
||||
assert.equal(parsed.length, 2, `Expected 2 tuples on 2-hop exhaustion, got ${parsed.length}`);
|
||||
assert.equal(parsed[0].hop, 0);
|
||||
assert.equal(parsed[0].provider, 'anthropic');
|
||||
assert.equal(parsed[0].code, 'SPAWN_FAILED');
|
||||
assert.equal(parsed[0].trigger_type, 'hard');
|
||||
assert.equal(parsed[1].hop, 1);
|
||||
assert.equal(parsed[1].provider, 'openai');
|
||||
} finally {
|
||||
__resetFallbackConfig();
|
||||
__resetSpawnImpl();
|
||||
codexResetSpawnImpl();
|
||||
}
|
||||
});
|
||||
|
||||
it('D40: 2-hop chain, primary fails + secondary succeeds → success response carries X-OLP-Fallback-Detail with 1 tuple (the failed primary)', async () => {
|
||||
__clearCache();
|
||||
|
||||
const { writeFileSync, unlinkSync } = await import('node:fs');
|
||||
const { tmpdir } = await import('node:os');
|
||||
const { join: pathJoin } = await import('node:path');
|
||||
const tmpAuthFile = pathJoin(tmpdir(), `olp-test-d40-success-with-detail-${Date.now()}.json`);
|
||||
writeFileSync(tmpAuthFile, JSON.stringify({ accessToken: 'fake-d40-codex-token' }), 'utf8');
|
||||
const savedCodexAuthPath = process.env.OPENAI_CODEX_AUTH_PATH;
|
||||
process.env.OPENAI_CODEX_AUTH_PATH = tmpAuthFile;
|
||||
|
||||
__setFallbackConfig({
|
||||
chains: {
|
||||
'claude-sonnet-4-6': [
|
||||
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||
{ provider: 'openai', model: 'gpt-5.5' },
|
||||
],
|
||||
},
|
||||
soft_triggers: {},
|
||||
});
|
||||
|
||||
// Anthropic mock: SPAWN_FAILED (no chunks)
|
||||
__setSpawnImpl(function (_bin, _args, _opts) {
|
||||
const proc = new EventEmitter();
|
||||
proc.stdout = new EventEmitter();
|
||||
proc.stderr = new EventEmitter();
|
||||
proc.stdin = {
|
||||
write: () => {},
|
||||
end: () => {
|
||||
setImmediate(() => {
|
||||
proc.stdout.emit('end');
|
||||
proc.stderr.emit('end');
|
||||
proc.emit('close', 1, null);
|
||||
});
|
||||
},
|
||||
};
|
||||
proc.killed = false;
|
||||
proc.kill = () => {};
|
||||
return proc;
|
||||
});
|
||||
|
||||
// Codex (openai) mock: succeeds
|
||||
codexSetSpawnImpl(function (_bin, _args, _opts) {
|
||||
const proc = new EventEmitter();
|
||||
proc.stdout = new EventEmitter();
|
||||
proc.stderr = new EventEmitter();
|
||||
proc.stdin = {
|
||||
write: () => {},
|
||||
end: () => {
|
||||
setImmediate(() => {
|
||||
proc.stdout.emit('data', Buffer.from('{"content":"D40 success-after-fail"}\n'));
|
||||
proc.stdout.emit('data', Buffer.from('{"type":"stop"}\n'));
|
||||
proc.stdout.emit('end');
|
||||
proc.stderr.emit('end');
|
||||
proc.emit('close', 0, null);
|
||||
});
|
||||
},
|
||||
};
|
||||
proc.killed = false;
|
||||
proc.kill = () => {};
|
||||
return proc;
|
||||
});
|
||||
|
||||
try {
|
||||
const r = await fetch({
|
||||
port,
|
||||
method: 'POST',
|
||||
path: '/v1/chat/completions',
|
||||
body: {
|
||||
model: 'claude-sonnet-4-6',
|
||||
messages: [{ role: 'user', content: 'Hi' }],
|
||||
max_tokens: 10,
|
||||
},
|
||||
});
|
||||
assert.equal(r.status, 200, `Expected 200 from fallback success, got ${r.status}: ${r.body.slice(0, 300)}`);
|
||||
assert.equal(r.headers['x-olp-fallback-hops'], '1');
|
||||
assert.equal(r.headers['x-olp-provider-used'], 'openai');
|
||||
assert.ok(
|
||||
r.headers['x-olp-fallback-detail'] !== undefined,
|
||||
`Expected X-OLP-Fallback-Detail on success-with-prior-failure, got headers: ${JSON.stringify(r.headers)}`,
|
||||
);
|
||||
const parsed = JSON.parse(r.headers['x-olp-fallback-detail']);
|
||||
assert.equal(parsed.length, 1, `Expected 1 tuple (the failed primary), got ${parsed.length}`);
|
||||
assert.equal(parsed[0].hop, 0);
|
||||
assert.equal(parsed[0].provider, 'anthropic');
|
||||
assert.equal(parsed[0].code, 'SPAWN_FAILED');
|
||||
assert.equal(parsed[0].trigger_type, 'hard');
|
||||
} finally {
|
||||
__resetFallbackConfig();
|
||||
__resetSpawnImpl();
|
||||
codexResetSpawnImpl();
|
||||
if (savedCodexAuthPath !== undefined) {
|
||||
process.env.OPENAI_CODEX_AUTH_PATH = savedCodexAuthPath;
|
||||
} else {
|
||||
delete process.env.OPENAI_CODEX_AUTH_PATH;
|
||||
}
|
||||
try { unlinkSync(tmpAuthFile); } catch { /* ignore */ }
|
||||
}
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
// ── Suite 13g: D28 round-3 F2 — structured log observability fields ─────────
|
||||
@@ -5901,6 +6114,222 @@ describe('Fallback engine — D28 observability fields (chain_id / ir_request_ha
|
||||
|
||||
});
|
||||
|
||||
// ── D40: X-OLP-Fallback-Detail header (issue #7) ─────────────────────────
|
||||
//
|
||||
// Tests for D40 fold the per-hop fallback failure detail into a response
|
||||
// header. Authority: ADR 0004 § Decision § Chain advancement step 4 +
|
||||
// § Observability headers.
|
||||
|
||||
import {
|
||||
serializeFallbackDetailHeader,
|
||||
FALLBACK_DETAIL_BYTE_CAP,
|
||||
} from './server.mjs';
|
||||
|
||||
describe('D40 — X-OLP-Fallback-Detail header (issue #7)', () => {
|
||||
|
||||
// ── Engine: fallbackDetail returned in FallbackResult ─────────────────
|
||||
|
||||
it('engine: 2-hop chain, both fail → fallbackDetail has 2 tuples with correct shape', async () => {
|
||||
const errA = new ProviderError('Anthropic spawn failed', 'SPAWN_FAILED');
|
||||
const errB = new ProviderError('Codex spawn timeout', 'SPAWN_TIMEOUT');
|
||||
const chain = [
|
||||
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||
{ provider: 'openai', model: 'gpt-5.5' },
|
||||
];
|
||||
const hopFn = async (provider) => {
|
||||
if (provider === 'anthropic') throw errA;
|
||||
throw errB;
|
||||
};
|
||||
const result = await executeWithFallback(chain, makeIR({ model: 'claude-sonnet-4-6' }), hopFn);
|
||||
assert.equal(result.chunks, null, 'Expected exhausted chain');
|
||||
assert.ok(Array.isArray(result.fallbackDetail), 'fallbackDetail must be array');
|
||||
assert.equal(result.fallbackDetail.length, 2);
|
||||
assert.deepEqual(result.fallbackDetail[0], {
|
||||
hop: 0, provider: 'anthropic', model: 'claude-sonnet-4-6',
|
||||
code: 'SPAWN_FAILED', error_message: 'Anthropic spawn failed', trigger_type: 'hard',
|
||||
});
|
||||
assert.deepEqual(result.fallbackDetail[1], {
|
||||
hop: 1, provider: 'openai', model: 'gpt-5.5',
|
||||
code: 'SPAWN_TIMEOUT', error_message: 'Codex spawn timeout', trigger_type: 'hard',
|
||||
});
|
||||
});
|
||||
|
||||
it('engine: 2-hop chain, primary fails + secondary succeeds → fallbackDetail has 1 tuple (the failed primary)', async () => {
|
||||
const err = new ProviderError('Spawn failed', 'SPAWN_FAILED');
|
||||
const chain = [
|
||||
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||
{ provider: 'openai', model: 'gpt-5.5' },
|
||||
];
|
||||
const hopFn = async (provider) => {
|
||||
if (provider === 'anthropic') throw err;
|
||||
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||
};
|
||||
const result = await executeWithFallback(chain, makeIR({ model: 'claude-sonnet-4-6' }), hopFn);
|
||||
assert.ok(Array.isArray(result.chunks), 'Expected success chunks');
|
||||
assert.equal(result.fallbackHops, 1);
|
||||
assert.equal(result.fallbackDetail.length, 1);
|
||||
assert.equal(result.fallbackDetail[0].hop, 0);
|
||||
assert.equal(result.fallbackDetail[0].provider, 'anthropic');
|
||||
assert.equal(result.fallbackDetail[0].code, 'SPAWN_FAILED');
|
||||
assert.equal(result.fallbackDetail[0].trigger_type, 'hard');
|
||||
});
|
||||
|
||||
it('engine: 1-hop chain succeeds → fallbackDetail is empty array', async () => {
|
||||
const chain = [{ provider: 'anthropic', model: 'claude-sonnet-4-6' }];
|
||||
const hopFn = async () => [{ type: 'stop', finish_reason: 'stop' }];
|
||||
const result = await executeWithFallback(chain, makeIR({ model: 'claude-sonnet-4-6' }), hopFn);
|
||||
assert.ok(Array.isArray(result.chunks));
|
||||
assert.deepEqual(result.fallbackDetail, [], 'Empty fallbackDetail on clean primary success');
|
||||
});
|
||||
|
||||
it('engine: 1-hop chain fails → fallbackDetail has 1 tuple', async () => {
|
||||
const err = new ProviderError('Spawn failed', 'SPAWN_FAILED');
|
||||
const chain = [{ provider: 'anthropic', model: 'claude-sonnet-4-6' }];
|
||||
const hopFn = async () => { throw err; };
|
||||
const result = await executeWithFallback(chain, makeIR({ model: 'claude-sonnet-4-6' }), hopFn);
|
||||
assert.equal(result.chunks, null);
|
||||
assert.equal(result.fallbackDetail.length, 1);
|
||||
assert.equal(result.fallbackDetail[0].hop, 0);
|
||||
assert.equal(result.fallbackDetail[0].code, 'SPAWN_FAILED');
|
||||
});
|
||||
|
||||
it('engine: non-ProviderError exception → tuple code is "UNKNOWN"', async () => {
|
||||
const err = new Error('Something unexpected'); // no .code field
|
||||
const chain = [{ provider: 'anthropic', model: 'claude-sonnet-4-6' }];
|
||||
const hopFn = async () => { throw err; };
|
||||
const result = await executeWithFallback(chain, makeIR({ model: 'claude-sonnet-4-6' }), hopFn);
|
||||
assert.equal(result.fallbackDetail.length, 1);
|
||||
assert.equal(result.fallbackDetail[0].code, 'UNKNOWN');
|
||||
assert.equal(result.fallbackDetail[0].error_message, 'Something unexpected');
|
||||
assert.equal(result.fallbackDetail[0].trigger_type, 'non_trigger');
|
||||
});
|
||||
|
||||
it('engine: 500-char error message → tuple.error_message is truncated to 200 chars ending in ellipsis', async () => {
|
||||
const longMsg = 'x'.repeat(500);
|
||||
const err = new ProviderError(longMsg, 'SPAWN_FAILED');
|
||||
const chain = [{ provider: 'anthropic', model: 'claude-sonnet-4-6' }];
|
||||
const hopFn = async () => { throw err; };
|
||||
const result = await executeWithFallback(chain, makeIR({ model: 'claude-sonnet-4-6' }), hopFn);
|
||||
const tuple = result.fallbackDetail[0];
|
||||
assert.equal(tuple.error_message.length, 200, 'Truncated message must be exactly 200 chars');
|
||||
assert.equal(tuple.error_message.slice(-1), '…', 'Truncated message must end with ellipsis');
|
||||
assert.equal(tuple.error_message.slice(0, 199), 'x'.repeat(199), 'First 199 chars preserved');
|
||||
});
|
||||
|
||||
it('engine: client error 400 → fallbackDetail has 1 tuple with trigger_type "client_error" (no advance)', async () => {
|
||||
const err = Object.assign(new Error('Bad request'), { statusCode: 400 });
|
||||
const chain = [
|
||||
{ provider: 'anthropic', model: 'claude-sonnet-4-6' },
|
||||
{ provider: 'openai', model: 'gpt-5.5' },
|
||||
];
|
||||
const hopFn = async (provider) => {
|
||||
if (provider === 'anthropic') throw err;
|
||||
return [{ type: 'stop', finish_reason: 'stop' }];
|
||||
};
|
||||
const result = await executeWithFallback(chain, makeIR({ model: 'claude-sonnet-4-6' }), hopFn);
|
||||
assert.equal(result.fallbackDetail.length, 1, 'Client error stops chain — only 1 tuple');
|
||||
assert.equal(result.fallbackDetail[0].trigger_type, 'client_error');
|
||||
});
|
||||
|
||||
// ── serializeFallbackDetailHeader: cap + RFC 7230 hygiene ─────────────
|
||||
|
||||
it('serialize: empty array → null (no header emitted)', () => {
|
||||
assert.equal(serializeFallbackDetailHeader([]), null);
|
||||
});
|
||||
|
||||
it('serialize: null/undefined → null (no header emitted)', () => {
|
||||
assert.equal(serializeFallbackDetailHeader(null), null);
|
||||
assert.equal(serializeFallbackDetailHeader(undefined), null);
|
||||
});
|
||||
|
||||
it('serialize: small array → JSON.stringified array under cap', () => {
|
||||
const detail = [{
|
||||
hop: 0, provider: 'anthropic', model: 'claude-sonnet-4-6',
|
||||
code: 'SPAWN_FAILED', error_message: 'oops', trigger_type: 'hard',
|
||||
}];
|
||||
const v = serializeFallbackDetailHeader(detail);
|
||||
assert.ok(typeof v === 'string');
|
||||
assert.ok(Buffer.byteLength(v, 'utf8') <= FALLBACK_DETAIL_BYTE_CAP);
|
||||
const parsed = JSON.parse(v);
|
||||
assert.deepEqual(parsed, detail);
|
||||
});
|
||||
|
||||
it('serialize: array exceeding 4KB → truncated with { truncated:true, omitted_hops:N } sentinel; total stays <= 4096 bytes', () => {
|
||||
// Build many tuples each just under the cap individually but cumulatively
|
||||
// over the cap. Each tuple here is roughly 250 bytes serialised.
|
||||
const baseMsg = 'x'.repeat(180); // leaves ~20 chars for JSON braces/key padding
|
||||
const tuples = [];
|
||||
for (let i = 0; i < 50; i++) {
|
||||
tuples.push({
|
||||
hop: i,
|
||||
provider: 'anthropic',
|
||||
model: 'claude-sonnet-4-6',
|
||||
code: 'SPAWN_FAILED',
|
||||
error_message: baseMsg,
|
||||
trigger_type: 'hard',
|
||||
});
|
||||
}
|
||||
const v = serializeFallbackDetailHeader(tuples);
|
||||
assert.ok(typeof v === 'string');
|
||||
assert.ok(
|
||||
Buffer.byteLength(v, 'utf8') <= FALLBACK_DETAIL_BYTE_CAP,
|
||||
`Header value must be <= ${FALLBACK_DETAIL_BYTE_CAP} bytes, got ${Buffer.byteLength(v, 'utf8')}`,
|
||||
);
|
||||
const parsed = JSON.parse(v);
|
||||
assert.ok(Array.isArray(parsed), 'Must JSON-parse to array');
|
||||
const tail = parsed[parsed.length - 1];
|
||||
assert.equal(tail.truncated, true, 'Last entry must be the truncation sentinel');
|
||||
assert.equal(typeof tail.omitted_hops, 'number');
|
||||
assert.ok(tail.omitted_hops > 0, 'omitted_hops must be positive when truncation fired');
|
||||
assert.equal(
|
||||
parsed.length - 1 + tail.omitted_hops,
|
||||
tuples.length,
|
||||
'kept tuples + omitted_hops must equal total original tuples',
|
||||
);
|
||||
});
|
||||
|
||||
it('serialize: JSON.stringify escapes newlines in error_message (RFC 7230 hygiene — no raw CR/LF in header)', () => {
|
||||
const detail = [{
|
||||
hop: 0, provider: 'anthropic', model: 'claude-sonnet-4-6',
|
||||
code: 'UNKNOWN', error_message: 'line1\nline2\rline3', trigger_type: 'non_trigger',
|
||||
}];
|
||||
const v = serializeFallbackDetailHeader(detail);
|
||||
assert.ok(typeof v === 'string');
|
||||
// Raw \n / \r in a header value would break RFC 7230. JSON.stringify escapes them.
|
||||
assert.equal(v.indexOf('\n'), -1, 'Header value must not contain raw newline');
|
||||
assert.equal(v.indexOf('\r'), -1, 'Header value must not contain raw CR');
|
||||
// Parsed back, the original message round-trips correctly
|
||||
assert.equal(JSON.parse(v)[0].error_message, 'line1\nline2\rline3');
|
||||
});
|
||||
|
||||
it('serialize: non-ASCII characters in error_message (e.g. em dash U+2014) → escaped as \\uXXXX so Node HTTP header validator accepts the value', () => {
|
||||
// The D38 CONCURRENCY_LIMIT synthesised error message in server.mjs contains
|
||||
// an em dash. Without \uXXXX escaping, res.writeHead would throw
|
||||
// "Invalid character in header content" because Node's HTTP layer rejects
|
||||
// multi-byte UTF-8 in header values (RFC 7230 §3.2.6 field-vchar = ASCII VCHAR).
|
||||
const detail = [{
|
||||
hop: 0, provider: 'anthropic', model: 'claude-sonnet-4-6',
|
||||
code: 'CONCURRENCY_LIMIT',
|
||||
error_message: 'provider anthropic at maxConcurrent (2) — advancing to next hop',
|
||||
trigger_type: 'hard',
|
||||
}];
|
||||
const v = serializeFallbackDetailHeader(detail);
|
||||
assert.ok(typeof v === 'string');
|
||||
// Every byte must be ASCII (0x00-0x7F) — verified via Buffer round-trip equality.
|
||||
const utf8Len = Buffer.byteLength(v, 'utf8');
|
||||
assert.equal(utf8Len, v.length, 'Header value must be pure ASCII (1 byte per char)');
|
||||
for (let i = 0; i < v.length; i++) {
|
||||
assert.ok(v.charCodeAt(i) < 0x80, `Char at ${i} (${v[i]}) must be ASCII`);
|
||||
}
|
||||
// JSON.parse must still round-trip the original em dash.
|
||||
assert.equal(
|
||||
JSON.parse(v)[0].error_message,
|
||||
'provider anthropic at maxConcurrent (2) — advancing to next hop',
|
||||
);
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
// ── Suite 14: providers.enabled config wiring ──────────────────────────────
|
||||
//
|
||||
// Tests that:
|
||||
|
||||
Reference in New Issue
Block a user