feat(chat): forward OpenAI image_url parts to Claude (multimodal vision) (#154)

* feat(chat): forward OpenAI image_url parts to Claude (multimodal vision)

`POST /v1/chat/completions` previously flattened every message to plain text
via contentToText(), replacing image_url parts with "[non-text content
omitted]" — so images were silently dropped (issue #110). This adds real
multimodal support: OpenAI `image_url` content parts are translated to
Anthropic image blocks and fed to the Claude CLI over
`--input-format stream-json`, keeping subscription auth (the reason OCP routes
through the CLI rather than the API).

Class B.1 (OpenAI-compatibility surface), authorized by ADR 0006. Request
shape follows OpenAI's published vision / chat-completions spec
(https://platform.openai.com/docs/guides/vision and the chat/completions
`content` image_url part) — no field is introduced beyond OpenAI's shape. The
CLI's `--input-format stream-json` is the transport for this Class B endpoint,
not a forwarded cli.js operation, so there is no Class A cli.js citation to
make; scope is justified under the ALIGNMENT.md Class B mapping of Rule 2
(no invention beyond the cited OpenAI spec).

Mechanism (verified empirically against the installed CLI, v2.1.206): a user
message whose `content` is an Anthropic block array including
`{type:"image",source:{type:"base64",media_type,data}}` fed to
`claude -p --input-format stream-json` is correctly described by the model.
Confirmed live end-to-end through this endpoint (a base64 PNG returns the
correct color).

Design:
- New pure module `lib/multimodal.mjs` (mirrors the lib/*.mjs pattern; unit-
  testable without a live server): hasImageContent, buildImageBlocks,
  buildStreamJsonInput, MultimodalError.
- server.mjs: text path is byte-for-byte unchanged. Only when a request carries
  an image_url part does spawnClaudeProcess switch stdin to a stream-json user
  envelope and buildCliArgs add `--input-format stream-json`. Image parsing runs
  before any stats mutation so a validation failure never leaks counters/slots.
- Images bypass the text char budget (CLAUDE_MAX_PROMPT_CHARS) and are bounded
  by explicit byte/count caps with clear 4xx errors (413 for size/count, 400
  for malformed/unsupported/disabled-remote), never a silent drop.

Scope decisions (v1):
- Base64 data URIs supported by default (image/jpeg,png,gif,webp).
- Remote http(s) image URLs OFF by default behind CLAUDE_IMAGE_ALLOW_URL; when
  enabled they are passed through as an Anthropic url-source (OCP never fetches
  the URL itself, so no OCP-side SSRF surface).
- Audio/file parts deferred: existing placeholder behavior preserved.
- Images anywhere in multi-turn history, not just the last message.

New env vars (documented in README Environment Variables table):
CLAUDE_IMAGE_ALLOW_URL, CLAUDE_MAX_IMAGE_BYTES, CLAUDE_MAX_IMAGES,
CLAUDE_MAX_IMAGE_TOTAL_BYTES, CLAUDE_MAX_BODY_SIZE (now configurable; default
5 MB unchanged).

Tests: 26 unit tests in test-features.mjs covering data-URI parse, multiple
images, text/image ordering, multi-turn history images, malformed/oversized/
too-many handling, remote-URL policy, and text-path parity. `npm test` green
(289 passed). `node --check` clean. No alignment-blacklist tokens added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(chat): address PR #154 review blockers — TUI guard, fail-closed caps, text budget

Remediates the three merge-blocking findings from the maintainer's review of the
multimodal vision PR. Class B.1 (OpenAI-compat surface): request shape per OpenAI
vision spec (image_url content parts), authorized by ADR 0006. No new wire shape
and no cli.js surface change — the stream-json image contract is the CLI's native
input format (already cited in the base commit); these are correctness fixes on the
OCP-owned validation/dispatch layer.

F1 — TUI mode silently dropped images and returned 200. callClaudeTui() renders
every non-text part as "[non-text content omitted]", so a vision request in
CLAUDE_TUI_MODE=true was answered about an image the model never saw. Now
handleChatCompletions fails loudly with 400 images_unsupported_in_tui_mode instead
of a silent drop (ALIGNMENT.md forbids serving text the model did not mean).
Documented in README § Images / Multimodal.

F3 — NaN env parsing failed open. CLAUDE_MAX_BODY_SIZE=unlimited -> NaN ->
`body.length > NaN` always false -> body cap gone (OOM DoS); =5MB -> 5 bytes ->
proxy bricked; same on CLAUDE_MAX_IMAGES / _IMAGE_BYTES / _IMAGE_TOTAL_BYTES.
Added lib/env.mjs parsePositiveInt (pure, fail-closed) + a thin parseIntEnv warn
wrapper; a malformed cap now keeps the safe default and warns at startup.

F2 — images let unbounded text bypass MAX_PROMPT_CHARS. buildImageBlocks only
counted textChars and never truncated; the budget was never passed in. Threaded
maxTextChars (= MAX_PROMPT_CHARS) into the multimodal transform, which now
truncates text tail-first (mirroring messagesToPrompt) while preserving image
blocks, and logs prompt_truncated.

Tests: +11 in test-features.mjs (all pure-module, per the repo's no-server-import
pattern) covering the text-budget enforcement and the fail-closed cap parsing,
including the exact F2 (500k chars + 1 image) and F3 (unlimited/5MB/0/20.5)
scenarios. 300 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(server): address PR #154 review round 2 — MAX_PROMPT_CHARS fail-closed + system-only image guard

Closes the two residual gaps from the round-2 review, both traced to the same root as the
already-fixed blockers. Class B.1 (OpenAI-compat vision): authorized by ADR 0006; request
shape is the OpenAI `image_url` content part. No new wire shape — the Anthropic image block
over `--input-format stream-json` is the CLI's native contract (cli.js buildStreamJsonInput
path, verified live in round 1). MAX_PROMPT_CHARS is OCP's own truncation guard, not a cli.js
operation.

Gap (a) — MAX_PROMPT_CHARS was left on the raw parseInt while every other cap moved to the
fail-closed helper. `let MAX_PROMPT_CHARS = parseInt(env||"150000",10)` sat five lines above
the parseIntEnv helper this PR added, so CLAUDE_MAX_PROMPT_CHARS=unlimited → NaN →
enforceTextBudget's `!(NaN > 0)` early-return → 500k chars passed unbounded, truncated:false,
silently defeating F2's text-budget guarantee under a plausible operator config. Fix: hoist
parseIntEnv above the declaration and derive MAX_PROMPT_CHARS through it (keeps `let` for the
settings API). A misconfigured value now keeps the 150k default and warns, like the other caps.

Gap (b) — an image present ONLY in a system message silently dropped in non-TUI mode.
Detection runs on the full message list, but extraction/spawn filter role==="system" out, so a
system-only image was detected as multimodal, survived no filter, fell to the text path, and
rendered as "[non-text content omitted]" → 200 with a hallucinated answer — the one silent-drop
outcome F1 exists to forbid. Fix: after filtering, if hasImageContent(full) is true but no image
survives, return `400 images_unsupported_in_system_messages`. Narrow (OpenAI disallows images in
the system role) so no legitimate request is rejected. Documented in README § Images.

Tests: +4 (all pure-module) — parsePositiveInt('unlimited') keeps the 150k default (gap a) and a
valid override is honored; hasImageContent proves the guard predicate fires for a system-only
image (true on full list, false after the system filter) and does NOT fire for a user-message
image. 304 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: vvlasy-openclaw <vvlasy-openclaw@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: vvlasy-openclaw <vvlasy@gmail.com>
Co-authored-by: dtzp555 <dtzp555@gmail.com>

lib/env.mjs

@@ -0,0 +1,29 @@
+// OCP env-var parsing helpers.
+//
+// Fail-closed positive-integer parsing for numeric caps (body size, image
+// byte/count limits). A misconfigured cap must NEVER silently disable a guard:
+// `parseInt("unlimited", 10)` is NaN and `x > NaN` is always false, so a naive
+// parse of CLAUDE_MAX_BODY_SIZE=unlimited would remove the body-size limit
+// entirely (unbounded body → OOM). Likewise CLAUDE_MAX_BODY_SIZE=5MB naively
+// parses to 5 (bytes) and bricks the proxy. So a present-but-invalid value is
+// REJECTED (default kept, caller warns), not accepted. (PR #154 review F3.)
+//
+// Pure (no env access, no IO) so it is unit-testable without a live server.
+
+// Parse `raw` as a strictly-positive base-10 integer of bytes/count (no unit
+// suffix). Returns { value, ok, reason }:
+//   - missing/empty        → { value: def, ok: true }               (use default)
+//   - valid positive int   → { value: n,  ok: true }
+//   - anything else        → { value: def, ok: false, reason }      (fail closed)
+// Rejects: NaN ("unlimited"), non-positive ("0", "-1"), unit-suffixed ("5MB"),
+// and fractional/ambiguous ("20.5", "0x10") values — String(n) !== trimmed catches
+// any input parseInt only partially consumed.
+export function parsePositiveInt(raw, def) {
+  if (raw === undefined || raw === null || raw === "") return { value: def, ok: true };
+  const trimmed = String(raw).trim();
+  const n = parseInt(trimmed, 10);
+  if (!Number.isFinite(n) || n <= 0 || String(n) !== trimmed) {
+    return { value: def, ok: false, reason: "not a strictly-positive integer (bytes/count, no unit suffix)" };
+  }
+  return { value: n, ok: true };
+}

lib/multimodal.mjs

@@ -0,0 +1,278 @@
+// OCP multimodal helpers — OpenAI `image_url` content parts → Anthropic image
+// blocks fed to `claude -p --input-format stream-json`. (issue #110)
+//
+// Class B.1 (OpenAI-compatibility surface). Protocol authority is OpenAI's
+// chat/completions spec — the multimodal `content` parts shape
+// (https://platform.openai.com/docs/guides/vision and
+// https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages,
+// `image_url` part with `image_url.url` = data URI or http(s) URL). Authorized
+// by ADR 0006. This module introduces NO field beyond OpenAI's published shape:
+// the OpenAI-side vocabulary read here is `type:"image_url"` +
+// `image_url:{url, detail?}`; the Anthropic-side vocabulary written here
+// (`type:"image", source:{type:"base64"|"url", ...}`) is the CLI's native
+// stream-json input contract, not an OCP invention.
+//
+// Kept as a pure module (no I/O, no network, no process state) mirroring the
+// lib/*.mjs pattern so it is unit-testable without a live server. server.mjs is
+// the only consumer; it owns spawning, caps configuration, and HTTP status.
+
+// Anthropic vision-supported image media types. A data URI whose media type is
+// outside this set is rejected with a clear 4xx rather than forwarded (the API
+// would reject it anyway; failing early gives a better error).
+export const SUPPORTED_IMAGE_TYPES = new Set([
+  "image/jpeg",
+  "image/png",
+  "image/gif",
+  "image/webp",
+]);
+
+// Default caps. server.mjs overrides these from env; they live here so the
+// pure transform is self-contained and testable.
+export const DEFAULT_MULTIMODAL_OPTS = {
+  allowRemoteUrl: false,        // http(s) image URLs are OFF by default (v1: data URIs only)
+  maxImageBytes: 5 * 1024 * 1024,   // per-image decoded-byte cap
+  maxImages: 20,                // max image parts across the whole request
+  maxTotalImageBytes: 20 * 1024 * 1024, // aggregate decoded-byte cap
+  maxTextChars: Infinity,       // text-char budget (server passes MAX_PROMPT_CHARS); Infinity = no truncation
+};
+
+// Typed error so server.mjs can map to the right HTTP status + OpenAI-shaped
+// error body. `status` is the HTTP code; `type` is the OpenAI error `type`.
+export class MultimodalError extends Error {
+  constructor(code, status, message) {
+    super(message);
+    this.name = "MultimodalError";
+    this.code = code;
+    this.status = status;
+    this.type = "invalid_request_error"; // OpenAI error `type` for 4xx client errors
+  }
+}
+
+// True if any message carries an OpenAI `image_url` content part. Cheap guard so
+// the byte-for-byte text path is only left when an image is genuinely present.
+export function hasImageContent(messages) {
+  if (!Array.isArray(messages)) return false;
+  for (const m of messages) {
+    if (m && Array.isArray(m.content)) {
+      for (const part of m.content) {
+        if (part && part.type === "image_url") return true;
+      }
+    }
+  }
+  return false;
+}
+
+// Extract the URL string from an OpenAI image_url part. Spec form is
+// `{type:"image_url", image_url:{url, detail?}}`; many OpenAI-compatible clients
+// also send `image_url` as a bare string. Accept both (input leniency — no new
+// output field). `detail` (auto|low|high) is OpenAI-only and has no Anthropic
+// analogue, so it is read-and-ignored.
+function imageUrlOf(part) {
+  const iu = part.image_url;
+  if (typeof iu === "string") return iu;
+  if (iu && typeof iu.url === "string") return iu.url;
+  return null;
+}
+
+// Parse a base64 data URI: `data:[<media_type>][;base64],<data>`.
+// Returns { mediaType, data (base64), bytes (decoded size) } or throws MultimodalError.
+function parseDataUri(uri) {
+  const comma = uri.indexOf(",");
+  if (comma === -1) {
+    throw new MultimodalError("invalid_data_uri", 400, "Malformed image data URI (no comma).");
+  }
+  const meta = uri.slice(5, comma); // strip leading "data:"
+  const segs = meta.split(";");
+  const mediaType = (segs[0] || "").trim().toLowerCase();
+  const isBase64 = segs.slice(1).some((s) => s.trim().toLowerCase() === "base64");
+  if (!isBase64) {
+    throw new MultimodalError("invalid_data_uri", 400, "Only base64-encoded image data URIs are supported.");
+  }
+  if (!SUPPORTED_IMAGE_TYPES.has(mediaType)) {
+    throw new MultimodalError(
+      "unsupported_image_type",
+      400,
+      `Unsupported image media type '${mediaType || "(none)"}'. Supported: ${[...SUPPORTED_IMAGE_TYPES].join(", ")}.`
+    );
+  }
+  // Strip incidental whitespace/newlines some encoders insert into data URIs.
+  const data = uri.slice(comma + 1).replace(/\s/g, "");
+  if (data.length === 0 || !/^[A-Za-z0-9+/]+={0,2}$/.test(data)) {
+    throw new MultimodalError("invalid_data_uri", 400, "Image data URI payload is not valid base64.");
+  }
+  // Decoded size from base64 length (minus padding); avoids decoding the buffer
+  // just to measure it.
+  const padding = data.endsWith("==") ? 2 : data.endsWith("=") ? 1 : 0;
+  const bytes = Math.floor((data.length * 3) / 4) - padding;
+  return { mediaType, data, bytes };
+}
+
+// Convert a single OpenAI image_url part to an Anthropic image block, enforcing
+// caps via the mutable `acc` accumulator ({ images, bytes }). Throws MultimodalError.
+function imagePartToBlock(part, opts, acc) {
+  const url = imageUrlOf(part);
+  if (!url) {
+    throw new MultimodalError("invalid_image_url", 400, "image_url part is missing a URL.");
+  }
+
+  acc.images += 1;
+  if (acc.images > opts.maxImages) {
+    throw new MultimodalError("too_many_images", 413, `Too many images in request (max ${opts.maxImages}).`);
+  }
+
+  if (url.startsWith("data:")) {
+    const { mediaType, data, bytes } = parseDataUri(url);
+    if (bytes > opts.maxImageBytes) {
+      throw new MultimodalError("image_too_large", 413, `Image exceeds per-image size limit (${opts.maxImageBytes} bytes).`);
+    }
+    acc.bytes += bytes;
+    if (acc.bytes > opts.maxTotalImageBytes) {
+      throw new MultimodalError("images_too_large", 413, `Total image payload exceeds limit (${opts.maxTotalImageBytes} bytes).`);
+    }
+    return { type: "image", source: { type: "base64", media_type: mediaType, data } };
+  }
+
+  if (/^https?:\/\//i.test(url)) {
+    if (!opts.allowRemoteUrl) {
+      throw new MultimodalError(
+        "remote_url_disabled",
+        400,
+        "Remote image URLs are disabled. Enable CLAUDE_IMAGE_ALLOW_URL=1 to allow http(s) image URLs, or pass the image as a base64 data URI."
+      );
+    }
+    // Passthrough as an Anthropic url-source block. OCP does NOT fetch the URL
+    // itself (no OCP-side SSRF surface); the fetch is performed upstream by the
+    // Anthropic API. Best-effort: unreachable/blocked URLs surface as an API error.
+    return { type: "image", source: { type: "url", url } };
+  }
+
+  throw new MultimodalError("unsupported_url_scheme", 400, "image_url must be a base64 data URI or an http(s) URL.");
+}
+
+// Role prefix mirrors messagesToPrompt()'s text-path labeling so a multi-turn
+// conversation reads the same whether or not it carries images. System messages
+// are handled by the caller via --system-prompt and never reach here.
+function rolePrefix(role) {
+  if (role === "assistant") return "[Assistant] ";
+  return ""; // user / tool / anything else: verbatim, as in the text path
+}
+
+// Build the Anthropic content-block array for a single stream-json user
+// envelope. Mirrors the text path's "collapse the whole conversation into one
+// turn passed via stdin" model (OCP runs stateless, full context per spawn), but
+// preserves image position relative to text and keeps images out of the text
+// char budget entirely. Returns { blocks, stats } or throws MultimodalError.
+export function buildImageBlocks(messages, opts = {}) {
+  const o = { ...DEFAULT_MULTIMODAL_OPTS, ...opts };
+  const blocks = [];
+  const acc = { images: 0, bytes: 0 };
+  let textChars = 0;
+  let firstMessage = true;
+
+  const pushText = (text) => {
+    if (!text) return;
+    blocks.push({ type: "text", text });
+    textChars += text.length;
+  };
+
+  for (const m of messages) {
+    const prefix = rolePrefix(m.role);
+    // Separate messages with a blank line, matching messagesToPrompt's "\n\n" join.
+    const sep = firstMessage ? "" : "\n\n";
+    firstMessage = false;
+    let prefixEmitted = false;
+    const emitPrefixWith = (t) => {
+      if (prefixEmitted) return t;
+      prefixEmitted = true;
+      return sep + prefix + t;
+    };
+
+    if (typeof m.content === "string") {
+      pushText(emitPrefixWith(m.content));
+      continue;
+    }
+    if (!Array.isArray(m.content)) {
+      // null / object content: mirror contentToText's fallback.
+      const t = m.content == null ? "" : JSON.stringify(m.content);
+      pushText(emitPrefixWith(t));
+      continue;
+    }
+
+    for (const part of m.content) {
+      if (part && part.type === "text" && typeof part.text === "string") {
+        pushText(emitPrefixWith(part.text));
+      } else if (part && part.type === "image_url") {
+        // Ensure the role prefix isn't lost when a message leads with an image.
+        if (!prefixEmitted && prefix) pushText(emitPrefixWith(""));
+        blocks.push(imagePartToBlock(part, o, acc));
+      } else {
+        // audio / file / unknown parts: preserve the existing placeholder
+        // behavior (issue #110) — deferred to a future version.
+        pushText(emitPrefixWith("[non-text content omitted]"));
+      }
+    }
+  }
+
+  // Defensive: a stream-json user turn must have at least one content block.
+  if (blocks.length === 0) blocks.push({ type: "text", text: "" });
+
+  // Enforce the text-char budget (PR #154 review F2). Without this, attaching a
+  // single tiny image would let unbounded text bypass the gateway's runaway-context
+  // guard entirely — messagesToPrompt truncates the text path, so the multimodal
+  // path must too. Image blocks are preserved and are NOT counted (they are bounded
+  // by the byte/count caps above).
+  const budgeted = enforceTextBudget(blocks, o.maxTextChars);
+
+  return {
+    blocks: budgeted.blocks,
+    stats: {
+      imageCount: acc.images,
+      totalImageBytes: acc.bytes,
+      textChars: budgeted.textChars,
+      originalTextChars: budgeted.originalTextChars,
+      truncated: budgeted.truncated,
+    },
+  };
+}
+
+// Enforce a text-char budget over already-built content blocks, mirroring
+// messagesToPrompt's "keep the tail, drop the oldest" truncation. Image blocks are
+// preserved in place; only text blocks count against the budget. A truncation note
+// is prepended when anything is dropped. Returns
+// { blocks, truncated, originalTextChars, textChars }.
+function enforceTextBudget(blocks, maxTextChars) {
+  let originalTextChars = 0;
+  for (const b of blocks) if (b.type === "text") originalTextChars += b.text.length;
+  if (!(maxTextChars > 0) || originalTextChars <= maxTextChars) {
+    return { blocks, truncated: false, originalTextChars, textChars: originalTextChars };
+  }
+  // Keep the most recent text up to the budget; trim within the boundary block and
+  // drop older text blocks. Non-text (image) blocks always survive, in order.
+  let budget = maxTextChars;
+  const out = [];
+  for (let i = blocks.length - 1; i >= 0; i--) {
+    const b = blocks[i];
+    if (b.type !== "text") { out.unshift(b); continue; }
+    if (budget <= 0) continue; // older text fully dropped
+    if (b.text.length <= budget) {
+      out.unshift(b);
+      budget -= b.text.length;
+    } else {
+      out.unshift({ type: "text", text: b.text.slice(b.text.length - budget) });
+      budget = 0;
+    }
+  }
+  out.unshift({ type: "text", text: "[System] Note: older text content was truncated to fit the context limit." });
+  let textChars = 0;
+  for (const b of out) if (b.type === "text") textChars += b.text.length;
+  return { blocks: out, truncated: true, originalTextChars, textChars };
+}
+
+// Serialize the non-system conversation to a single newline-terminated
+// stream-json user message for `claude -p --input-format stream-json` stdin.
+// Returns { payload, stats } or throws MultimodalError.
+export function buildStreamJsonInput(messages, opts = {}) {
+  const { blocks, stats } = buildImageBlocks(messages, opts);
+  const envelope = { type: "user", message: { role: "user", content: blocks } };
+  return { payload: JSON.stringify(envelope) + "\n", stats };
+}