olp/docs/adr/0003-intermediate-representation.md

ADR 0003 — Intermediate Representation (IR) Design

• Date: 2026-05-23
• Status: Accepted (bootstrap)
• Authors: project maintainer (with AI drafting assistance)
• Related: OLP v0.1 spec §4.1, §4.2 (IR subsection); ADR 0002 (plugin architecture); ADR 0004 (fallback engine — requires shape-stable requests for safe replay)

Amendments

Amendment 1 — 2026-05-24: Remove __irRoundTripTest() claim; document substitute test strategy (D31 F5)

• Finding: Round-3 cold-audit F5 (P2 ADR-claim vs implementation drift) — § Mitigations names a __irRoundTripTest() export on every provider plugin as the structural counter-measure against silent lossy translation. ZERO plugins export this function; the export-name was an early-design aspiration that does not fit the actual plugin shape (spawn-wrappers with asymmetric request→CLI-args+stdin and response-stream→IR, not symmetric irToNative+nativeToIR functions that could be tested in isolation via a round-trip fixture).
• Decision: Remove the __irRoundTripTest() export-name claim from § Mitigations. The structural counter-measure against silent lossy translation is preserved via the existing test strategy documented below; the goal (CI-visible lossy-translation coverage) is met; only the mechanism name was wrong.
• Substitute test strategy (live at v0.1):
  • Entry-surface IR↔OpenAI round-trip — Suite 3 (irChunkToOpenAISSE format + openai-to-ir tests in test-features.mjs) covers the OpenAI ↔ IR direction at the entry surface.
  • Per-plugin spawn-with-mock tests in each plugin's test block (anthropic, codex, and mistral spawn unit tests) exercise the IR → CLI args → IR-chunk round-trip end-to-end.
  • Lossy-translation edges are documented inline in each plugin's header comment; mistral.mjs's explicit "DOCS-N" UNPINNED markers are the most thorough example; anthropic.mjs and codex.mjs carry equivalent lossy-field tables in their headers.
  • Future plugins MUST include both: (a) a spawn-with-mock test block, and (b) header documentation of any lossy IR fields. These two requirements are the enforcement mechanism replacing the __irRoundTripTest() export-name aspiration.
• Original § Mitigations update: The __irRoundTripTest() sentence in § Mitigations below is replaced with a pointer to this amendment. See § Mitigations.
• Procedural mechanism: CC 开发铁律 v1.6 § 10.x (Round-3 Cold Audit caught it as F5).

Context

OLP exposes exactly one external API surface: OpenAI-compatible /v1/chat/completions (per spec §4.1). Internally, OLP fans out to N providers, each with its own native protocol — Anthropic Messages API, OpenAI Chat Completions, Mistral La Plateforme, xAI Grok, Moonshot Kimi, etc. The combinatorial fact is unavoidable: a request entering as OpenAI shape must exit as Anthropic shape (or Mistral shape, or whatever the routing chain advances to), and the response on the way back must travel the inverse path.

The ad-hoc approach is per-provider entry-to-native translation:

provider.spawn(openAIRequest) {
  const native = openAIToProviderNative(openAIRequest);  // per-provider impl
  ...
  const openAIResponse = providerNativeToOpenAI(nativeResponse);  // per-provider impl
  return openAIResponse;
}

At three providers, that's six translation functions (three forward + three reverse). At eight providers it's sixteen. None of them share code — Anthropic-to-OpenAI and Mistral-to-OpenAI have different shape mismatches, different tool-calling translations, different streaming-event mappings. The "OpenAI-shape canonical" assumption privileges one provider's protocol as the lingua franca, which is wrong: OpenAI's Chat Completions is OLP's entry surface, not OLP's internal representation. Anchoring internal logic on the entry surface means every entry-surface change (a new OpenAI field, a deprecated OpenAI field, a new OpenAI streaming-event variant) ripples through every provider plugin.

The structural answer is an Intermediate Representation (IR): a canonical OLP-internal shape that all provider plugins consume (forward) and produce (reverse). The entry adapter translates OpenAI → IR once. Each provider plugin translates IR → native and native → IR. The combinatorial cost goes from N² to 2N: one entry adapter + N provider translators.

The IR is not OpenAI Chat Completions, and it is not Anthropic Messages. It is a deliberately-OLP-owned shape, normalized to be expressible in every provider's native shape with documented lossy fields where a provider's expressiveness is narrower than the IR. Choosing IR shape carefully — neither so rich it cannot round-trip through Mistral, nor so narrow it loses Anthropic's tool_use semantics — is the load-bearing design decision of this ADR.

The fallback engine (ADR 0004) is a second consumer of IR's shape stability. When a request fails over from Anthropic to OpenAI, the IR that was constructed at the entry point is replayed against the next provider. If the IR were a thin OpenAI wrapper, replay against OpenAI would be trivial but replay against Mistral or xAI would re-run the same shape-translation pain. A properly-OLP-owned IR makes replay symmetric across providers.

Decision

OLP defines an Intermediate Representation (IR) as the canonical internal shape between the OpenAI-compat entry surface and each provider plugin. Per spec §4.2 (IR subsection), the IR v1.0 encodes:

Required fields:

• messages[] — each with role, content, optional name, optional tool_calls, optional tool_call_id. Roles supported: system, user, assistant, tool.
• model — the user-requested model string. The provider plugin maps this to the provider-native model identifier (e.g., claude-sonnet-4-6 → claude-sonnet-4-6-20260301 for Anthropic).
• stream — boolean. SSE expected on the entry side iff true.

Optional fields:

• max_tokens — integer; provider plugins clamp to provider-supported maxima.
• temperature — float [0, 2]; provider plugins map to provider-native range (e.g., Anthropic uses [0, 1]).
• top_p — float [0, 1].
• stop — string or string[]; provider plugins handle the single-vs-array shape difference per native protocol.
• tools[] + tool_choice — function-calling subset. v1.0 supports basic function definitions (name, description, parameters JSON-schema); advanced features (parallel tool use, tool result handling beyond single round-trip) are per-provider documented as caveats per spec §8 Q-C.
• response_format — best-effort; provider-specific (e.g., Anthropic does not natively honor a response_format: json_object field, so this becomes a system-prompt augmentation on that provider).

Translation direction model. Each provider plugin owns two translation functions:

// IR → provider-native shape
const nativeRequest = provider.irToNative(irRequest);

// provider-native response chunk → IR response chunk
const irChunk = provider.nativeToIR(nativeChunk);

The entry adapter (server.mjs's /v1/chat/completions handler) owns the inverse:

const irRequest = openAIToIR(openAIRequest);
const irResponseStream = await provider.spawn(irRequest, authContext);
for await (const irChunk of irResponseStream) {
  res.write(irToOpenAI(irChunk));
}

Lossy-translation documentation requirement. When a provider plugin's irToNative or nativeToIR cannot losslessly carry an IR field (e.g., Mistral Vibe does not support tool_choice: "required"), the lossy edge is documented in the provider plugin's header comment AND in docs/provider-caveats.md (📋 planned — not yet authored; until it exists, lossy translations are recorded only in the provider plugin header comments). The caveats document is referenced from /v1/models response metadata so clients can see provider-specific limits without reading source.

IR is versioned. This ADR ratifies IR v1.0. Future additions (e.g., a reasoning_effort field if reasoning-model behavior diverges enough across providers to warrant explicit modeling) require an amendment ADR. Removals are breaking changes and require a major-version bump of OLP.

The IR is not exposed externally. It is OLP-internal. There is no /v1/ir/* endpoint, no client SDK in IR shape, no documented external IR contract. The only public contract is OpenAI /v1/chat/completions; the IR is an implementation detail that the maintainer (and the maintainer's reviewers) reason about, not a contract with clients.

Consequences

Positive

• Adding a provider is N translation functions (one IR→native, one native→IR), not 2N (one openai→native, one native→openai for every provider). The maintenance burden of "OpenAI shipped a new field" is one entry-adapter change, not one change per provider.
• The fallback engine (ADR 0004) replays the same IR object across providers in a chain. Replay correctness depends on IR being a fixed-shape data structure; there is no per-provider state leaking through that the next chain hop has to re-derive.
• Lossy translations are documented per-provider and surfaced to clients via docs/provider-caveats.md (📋 planned) + /v1/models metadata. A client choosing a fallback chain can see "this chain's third hop is Mistral, which does not support tool_choice=required" before they hit the failure mode.
• The IR is OLP's source of truth for "what was actually requested" for cache-key generation (ADR 0005). Caching on OpenAI shape would couple cache keys to OpenAI field naming; caching on IR makes the cache key provider-agnostic.

Negative

• Two translation layers per request: openai→IR at entry, IR→native at provider. That's two passes of object construction per request, plus the inverse on the way back. For family-scale traffic this is unmeasurable; at higher scale the overhead would need profiling.
• IR design decisions are sticky. Choosing the wrong shape for tool_calls in v1.0 (e.g., privileging OpenAI's parallel-tool-call shape over Anthropic's single-tool shape) would force every provider plugin to work around the bad choice forever. IR v1.0 is intentionally narrow (basic function calling only) to give the maintainer space to expand the IR with informed amendments rather than commit too early.
• The IR is a third concept (alongside OpenAI shape and per-provider native shape) that every contributor must internalize. Onboarding a new contributor means teaching three shapes, not two. The PR template and OLP ALIGNMENT.md call out the IR explicitly so contributors know which shape applies in which layer of code.

Mitigations

• IR v1.0 is deliberately a subset, not a superset, of what providers can do. Where providers diverge in expressive power (tool calling, structured outputs, reasoning modes), the IR encodes the common subset in v1.0, with caveats documenting provider extensions. Expanding the IR to cover provider extensions is an explicit amendment, not a silent generalization.
• (The __irRoundTripTest() export-name claim that originally appeared here was removed by Amendment 1 (D31 F5). The structural counter-measure against silent lossy translation is the substitute test strategy described in Amendment 1: entry-surface round-trip tests in Suite 3 of test-features.mjs + per-plugin spawn-with-mock test blocks + inline lossy-field header documentation. Future plugins must satisfy both requirements.)
• The IR has no external surface. If IR v1.0 turns out to be the wrong shape and IR v2.0 is incompatible, the migration is OLP-internal: provider plugins are updated, the entry adapter is updated, no client code changes. The cost of being wrong is bounded by the project's own codebase.

Alternatives considered

(a) Full pass-through with no normalization — OpenAI request goes straight to each provider plugin. Provider plugins each implement their own openai→native + native→openai. Rejected: this is the N² shape the IR exists to avoid. Worse, it makes the fallback engine (ADR 0004) harder, because each fallback hop has to redo openai→native; there is no canonical "what was requested" object to replay. The shape-stability invariant the fallback engine depends on becomes per-provider rather than systemic.

(b) Per-provider entry endpoints — /v1/anthropic/messages, /v1/openai/chat/completions, /v1/mistral/.... Each entry endpoint speaks its provider's native shape; no translation. Rejected: this just relocates the N² problem to the client side — each client now needs N integrations (one per provider's wire format), and the unified-endpoint value proposition (spec §1: "one HTTP endpoint, multiple subscriptions behind it") is dead on arrival. Cross-provider fallback also stops being client-transparent: clients have to know about the fallback because they have to switch endpoints when the primary fails.

(c) Use Anthropic Messages format as the IR (Anthropic-native as canonical). This privileges Anthropic's shape as OLP's internal language. Rejected on two grounds: (1) Anthropic is currently one provider in eight; privileging its shape codifies provider preference into the IR, which is the wrong abstraction layer; (2) Anthropic-shape is rich enough to be lossy when translated to spartans like Mistral Vibe, so the IR would have features no other provider could use without elaborate fallback in the translator. The IR should be neutral, with provider extensions documented per-plugin, not Anthropic-shaped with per-plugin de-features.

(d) Use OpenAI Chat Completions as the IR (entry-surface as canonical). Privileges OpenAI shape. Rejected: this is the implicit shape of alternative (a), just lifted into a named layer. The coupling to OpenAI's evolving spec means every OpenAI deprecation / addition is a cross-cutting change. The IR exists precisely to decouple OLP-internal logic from OpenAI's release cadence.

(e) Defer IR design and start with two providers tightly coupled (Anthropic + OpenAI), refactor to IR when adding the third. Rejected: this defers the architectural decision into a code-rewrite-under-pressure moment (adding Mistral while two providers are in production). The pattern of "we'll abstract it later" is what produced the hybrid awkwardness OCP ADR 0005 warned against. Pay the IR cost up front, with v1.0 deliberately narrow, and expand via amendment.

Sources

• OLP v0.1 spec §4.1 (Single-protocol entry: OpenAI-compatible)
• OLP v0.1 spec §4.2 (Plugin-based provider system, including IR subsection)
• OLP v0.1 spec §8 Q-C (Tool-calling translation differences across providers — informs why IR v1.0 keeps tool calling to a basic subset)
• OCP ADR 0003 (models.json SPOT) — informs the "OLP-owned canonical shape, not external surface" framing