Files
ocp/lib/env.mjs
ac81badda1 feat(chat): forward OpenAI image_url parts to Claude (multimodal vision) (#154)
* feat(chat): forward OpenAI image_url parts to Claude (multimodal vision)

`POST /v1/chat/completions` previously flattened every message to plain text
via contentToText(), replacing image_url parts with "[non-text content
omitted]" — so images were silently dropped (issue #110). This adds real
multimodal support: OpenAI `image_url` content parts are translated to
Anthropic image blocks and fed to the Claude CLI over
`--input-format stream-json`, keeping subscription auth (the reason OCP routes
through the CLI rather than the API).

Class B.1 (OpenAI-compatibility surface), authorized by ADR 0006. Request
shape follows OpenAI's published vision / chat-completions spec
(https://platform.openai.com/docs/guides/vision and the chat/completions
`content` image_url part) — no field is introduced beyond OpenAI's shape. The
CLI's `--input-format stream-json` is the transport for this Class B endpoint,
not a forwarded cli.js operation, so there is no Class A cli.js citation to
make; scope is justified under the ALIGNMENT.md Class B mapping of Rule 2
(no invention beyond the cited OpenAI spec).

Mechanism (verified empirically against the installed CLI, v2.1.206): a user
message whose `content` is an Anthropic block array including
`{type:"image",source:{type:"base64",media_type,data}}` fed to
`claude -p --input-format stream-json` is correctly described by the model.
Confirmed live end-to-end through this endpoint (a base64 PNG returns the
correct color).

Design:
- New pure module `lib/multimodal.mjs` (mirrors the lib/*.mjs pattern; unit-
  testable without a live server): hasImageContent, buildImageBlocks,
  buildStreamJsonInput, MultimodalError.
- server.mjs: text path is byte-for-byte unchanged. Only when a request carries
  an image_url part does spawnClaudeProcess switch stdin to a stream-json user
  envelope and buildCliArgs add `--input-format stream-json`. Image parsing runs
  before any stats mutation so a validation failure never leaks counters/slots.
- Images bypass the text char budget (CLAUDE_MAX_PROMPT_CHARS) and are bounded
  by explicit byte/count caps with clear 4xx errors (413 for size/count, 400
  for malformed/unsupported/disabled-remote), never a silent drop.

Scope decisions (v1):
- Base64 data URIs supported by default (image/jpeg,png,gif,webp).
- Remote http(s) image URLs OFF by default behind CLAUDE_IMAGE_ALLOW_URL; when
  enabled they are passed through as an Anthropic url-source (OCP never fetches
  the URL itself, so no OCP-side SSRF surface).
- Audio/file parts deferred: existing placeholder behavior preserved.
- Images anywhere in multi-turn history, not just the last message.

New env vars (documented in README Environment Variables table):
CLAUDE_IMAGE_ALLOW_URL, CLAUDE_MAX_IMAGE_BYTES, CLAUDE_MAX_IMAGES,
CLAUDE_MAX_IMAGE_TOTAL_BYTES, CLAUDE_MAX_BODY_SIZE (now configurable; default
5 MB unchanged).

Tests: 26 unit tests in test-features.mjs covering data-URI parse, multiple
images, text/image ordering, multi-turn history images, malformed/oversized/
too-many handling, remote-URL policy, and text-path parity. `npm test` green
(289 passed). `node --check` clean. No alignment-blacklist tokens added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(chat): address PR #154 review blockers — TUI guard, fail-closed caps, text budget

Remediates the three merge-blocking findings from the maintainer's review of the
multimodal vision PR. Class B.1 (OpenAI-compat surface): request shape per OpenAI
vision spec (image_url content parts), authorized by ADR 0006. No new wire shape
and no cli.js surface change — the stream-json image contract is the CLI's native
input format (already cited in the base commit); these are correctness fixes on the
OCP-owned validation/dispatch layer.

F1 — TUI mode silently dropped images and returned 200. callClaudeTui() renders
every non-text part as "[non-text content omitted]", so a vision request in
CLAUDE_TUI_MODE=true was answered about an image the model never saw. Now
handleChatCompletions fails loudly with 400 images_unsupported_in_tui_mode instead
of a silent drop (ALIGNMENT.md forbids serving text the model did not mean).
Documented in README § Images / Multimodal.

F3 — NaN env parsing failed open. CLAUDE_MAX_BODY_SIZE=unlimited -> NaN ->
`body.length > NaN` always false -> body cap gone (OOM DoS); =5MB -> 5 bytes ->
proxy bricked; same on CLAUDE_MAX_IMAGES / _IMAGE_BYTES / _IMAGE_TOTAL_BYTES.
Added lib/env.mjs parsePositiveInt (pure, fail-closed) + a thin parseIntEnv warn
wrapper; a malformed cap now keeps the safe default and warns at startup.

F2 — images let unbounded text bypass MAX_PROMPT_CHARS. buildImageBlocks only
counted textChars and never truncated; the budget was never passed in. Threaded
maxTextChars (= MAX_PROMPT_CHARS) into the multimodal transform, which now
truncates text tail-first (mirroring messagesToPrompt) while preserving image
blocks, and logs prompt_truncated.

Tests: +11 in test-features.mjs (all pure-module, per the repo's no-server-import
pattern) covering the text-budget enforcement and the fail-closed cap parsing,
including the exact F2 (500k chars + 1 image) and F3 (unlimited/5MB/0/20.5)
scenarios. 300 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(server): address PR #154 review round 2 — MAX_PROMPT_CHARS fail-closed + system-only image guard

Closes the two residual gaps from the round-2 review, both traced to the same root as the
already-fixed blockers. Class B.1 (OpenAI-compat vision): authorized by ADR 0006; request
shape is the OpenAI `image_url` content part. No new wire shape — the Anthropic image block
over `--input-format stream-json` is the CLI's native contract (cli.js buildStreamJsonInput
path, verified live in round 1). MAX_PROMPT_CHARS is OCP's own truncation guard, not a cli.js
operation.

Gap (a) — MAX_PROMPT_CHARS was left on the raw parseInt while every other cap moved to the
fail-closed helper. `let MAX_PROMPT_CHARS = parseInt(env||"150000",10)` sat five lines above
the parseIntEnv helper this PR added, so CLAUDE_MAX_PROMPT_CHARS=unlimited → NaN →
enforceTextBudget's `!(NaN > 0)` early-return → 500k chars passed unbounded, truncated:false,
silently defeating F2's text-budget guarantee under a plausible operator config. Fix: hoist
parseIntEnv above the declaration and derive MAX_PROMPT_CHARS through it (keeps `let` for the
settings API). A misconfigured value now keeps the 150k default and warns, like the other caps.

Gap (b) — an image present ONLY in a system message silently dropped in non-TUI mode.
Detection runs on the full message list, but extraction/spawn filter role==="system" out, so a
system-only image was detected as multimodal, survived no filter, fell to the text path, and
rendered as "[non-text content omitted]" → 200 with a hallucinated answer — the one silent-drop
outcome F1 exists to forbid. Fix: after filtering, if hasImageContent(full) is true but no image
survives, return `400 images_unsupported_in_system_messages`. Narrow (OpenAI disallows images in
the system role) so no legitimate request is rejected. Documented in README § Images.

Tests: +4 (all pure-module) — parsePositiveInt('unlimited') keeps the 150k default (gap a) and a
valid override is honored; hasImageContent proves the guard predicate fires for a system-only
image (true on full list, false after the system filter) and does NOT fire for a user-message
image. 304 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: vvlasy-openclaw <vvlasy-openclaw@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: vvlasy-openclaw <vvlasy@gmail.com>
Co-authored-by: dtzp555 <dtzp555@gmail.com>
2026-07-20 07:50:17 +10:00

30 lines
1.6 KiB
JavaScript

// OCP env-var parsing helpers.
//
// Fail-closed positive-integer parsing for numeric caps (body size, image
// byte/count limits). A misconfigured cap must NEVER silently disable a guard:
// `parseInt("unlimited", 10)` is NaN and `x > NaN` is always false, so a naive
// parse of CLAUDE_MAX_BODY_SIZE=unlimited would remove the body-size limit
// entirely (unbounded body → OOM). Likewise CLAUDE_MAX_BODY_SIZE=5MB naively
// parses to 5 (bytes) and bricks the proxy. So a present-but-invalid value is
// REJECTED (default kept, caller warns), not accepted. (PR #154 review F3.)
//
// Pure (no env access, no IO) so it is unit-testable without a live server.
// Parse `raw` as a strictly-positive base-10 integer of bytes/count (no unit
// suffix). Returns { value, ok, reason }:
// - missing/empty → { value: def, ok: true } (use default)
// - valid positive int → { value: n, ok: true }
// - anything else → { value: def, ok: false, reason } (fail closed)
// Rejects: NaN ("unlimited"), non-positive ("0", "-1"), unit-suffixed ("5MB"),
// and fractional/ambiguous ("20.5", "0x10") values — String(n) !== trimmed catches
// any input parseInt only partially consumed.
export function parsePositiveInt(raw, def) {
if (raw === undefined || raw === null || raw === "") return { value: def, ok: true };
const trimmed = String(raw).trim();
const n = parseInt(trimmed, 10);
if (!Number.isFinite(n) || n <= 0 || String(n) !== trimmed) {
return { value: def, ok: false, reason: "not a strictly-positive integer (bytes/count, no unit suffix)" };
}
return { value: n, ok: true };
}