taodeng/ocp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/ocp.git synced 2026-07-21 21:15:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
217 Commits 52 Branches 44 Tags
ac81badda12a2267e7177b09b65b16d831425032
Commit Graph
3 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
openclaw.vvlasy.czGitHubvvlasy-openclawClaude Opus 4.8vvlasy-openclawtaodeng
 		ac81badda1 feat(chat): forward OpenAI image_url parts to Claude (multimodal vision) (#154) 
* feat(chat): forward OpenAI image_url parts to Claude (multimodal vision)

`POST /v1/chat/completions` previously flattened every message to plain text
via contentToText(), replacing image_url parts with "[non-text content
omitted]" — so images were silently dropped (issue #110). This adds real
multimodal support: OpenAI `image_url` content parts are translated to
Anthropic image blocks and fed to the Claude CLI over
`--input-format stream-json`, keeping subscription auth (the reason OCP routes
through the CLI rather than the API).

Class B.1 (OpenAI-compatibility surface), authorized by ADR 0006. Request
shape follows OpenAI's published vision / chat-completions spec
(https://platform.openai.com/docs/guides/vision and the chat/completions
`content` image_url part) — no field is introduced beyond OpenAI's shape. The
CLI's `--input-format stream-json` is the transport for this Class B endpoint,
not a forwarded cli.js operation, so there is no Class A cli.js citation to
make; scope is justified under the ALIGNMENT.md Class B mapping of Rule 2
(no invention beyond the cited OpenAI spec).

Mechanism (verified empirically against the installed CLI, v2.1.206): a user
message whose `content` is an Anthropic block array including
`{type:"image",source:{type:"base64",media_type,data}}` fed to
`claude -p --input-format stream-json` is correctly described by the model.
Confirmed live end-to-end through this endpoint (a base64 PNG returns the
correct color).

Design:
- New pure module `lib/multimodal.mjs` (mirrors the lib/*.mjs pattern; unit-
  testable without a live server): hasImageContent, buildImageBlocks,
  buildStreamJsonInput, MultimodalError.
- server.mjs: text path is byte-for-byte unchanged. Only when a request carries
  an image_url part does spawnClaudeProcess switch stdin to a stream-json user
  envelope and buildCliArgs add `--input-format stream-json`. Image parsing runs
  before any stats mutation so a validation failure never leaks counters/slots.
- Images bypass the text char budget (CLAUDE_MAX_PROMPT_CHARS) and are bounded
  by explicit byte/count caps with clear 4xx errors (413 for size/count, 400
  for malformed/unsupported/disabled-remote), never a silent drop.

Scope decisions (v1):
- Base64 data URIs supported by default (image/jpeg,png,gif,webp).
- Remote http(s) image URLs OFF by default behind CLAUDE_IMAGE_ALLOW_URL; when
  enabled they are passed through as an Anthropic url-source (OCP never fetches
  the URL itself, so no OCP-side SSRF surface).
- Audio/file parts deferred: existing placeholder behavior preserved.
- Images anywhere in multi-turn history, not just the last message.

New env vars (documented in README Environment Variables table):
CLAUDE_IMAGE_ALLOW_URL, CLAUDE_MAX_IMAGE_BYTES, CLAUDE_MAX_IMAGES,
CLAUDE_MAX_IMAGE_TOTAL_BYTES, CLAUDE_MAX_BODY_SIZE (now configurable; default
5 MB unchanged).

Tests: 26 unit tests in test-features.mjs covering data-URI parse, multiple
images, text/image ordering, multi-turn history images, malformed/oversized/
too-many handling, remote-URL policy, and text-path parity. `npm test` green
(289 passed). `node --check` clean. No alignment-blacklist tokens added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(chat): address PR #154 review blockers — TUI guard, fail-closed caps, text budget

Remediates the three merge-blocking findings from the maintainer's review of the
multimodal vision PR. Class B.1 (OpenAI-compat surface): request shape per OpenAI
vision spec (image_url content parts), authorized by ADR 0006. No new wire shape
and no cli.js surface change — the stream-json image contract is the CLI's native
input format (already cited in the base commit); these are correctness fixes on the
OCP-owned validation/dispatch layer.

F1 — TUI mode silently dropped images and returned 200. callClaudeTui() renders
every non-text part as "[non-text content omitted]", so a vision request in
CLAUDE_TUI_MODE=true was answered about an image the model never saw. Now
handleChatCompletions fails loudly with 400 images_unsupported_in_tui_mode instead
of a silent drop (ALIGNMENT.md forbids serving text the model did not mean).
Documented in README § Images / Multimodal.

F3 — NaN env parsing failed open. CLAUDE_MAX_BODY_SIZE=unlimited -> NaN ->
`body.length > NaN` always false -> body cap gone (OOM DoS); =5MB -> 5 bytes ->
proxy bricked; same on CLAUDE_MAX_IMAGES / _IMAGE_BYTES / _IMAGE_TOTAL_BYTES.
Added lib/env.mjs parsePositiveInt (pure, fail-closed) + a thin parseIntEnv warn
wrapper; a malformed cap now keeps the safe default and warns at startup.

F2 — images let unbounded text bypass MAX_PROMPT_CHARS. buildImageBlocks only
counted textChars and never truncated; the budget was never passed in. Threaded
maxTextChars (= MAX_PROMPT_CHARS) into the multimodal transform, which now
truncates text tail-first (mirroring messagesToPrompt) while preserving image
blocks, and logs prompt_truncated.

Tests: +11 in test-features.mjs (all pure-module, per the repo's no-server-import
pattern) covering the text-budget enforcement and the fail-closed cap parsing,
including the exact F2 (500k chars + 1 image) and F3 (unlimited/5MB/0/20.5)
scenarios. 300 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(server): address PR #154 review round 2 — MAX_PROMPT_CHARS fail-closed + system-only image guard

Closes the two residual gaps from the round-2 review, both traced to the same root as the
already-fixed blockers. Class B.1 (OpenAI-compat vision): authorized by ADR 0006; request
shape is the OpenAI `image_url` content part. No new wire shape — the Anthropic image block
over `--input-format stream-json` is the CLI's native contract (cli.js buildStreamJsonInput
path, verified live in round 1). MAX_PROMPT_CHARS is OCP's own truncation guard, not a cli.js
operation.

Gap (a) — MAX_PROMPT_CHARS was left on the raw parseInt while every other cap moved to the
fail-closed helper. `let MAX_PROMPT_CHARS = parseInt(env||"150000",10)` sat five lines above
the parseIntEnv helper this PR added, so CLAUDE_MAX_PROMPT_CHARS=unlimited → NaN →
enforceTextBudget's `!(NaN > 0)` early-return → 500k chars passed unbounded, truncated:false,
silently defeating F2's text-budget guarantee under a plausible operator config. Fix: hoist
parseIntEnv above the declaration and derive MAX_PROMPT_CHARS through it (keeps `let` for the
settings API). A misconfigured value now keeps the 150k default and warns, like the other caps.

Gap (b) — an image present ONLY in a system message silently dropped in non-TUI mode.
Detection runs on the full message list, but extraction/spawn filter role==="system" out, so a
system-only image was detected as multimodal, survived no filter, fell to the text path, and
rendered as "[non-text content omitted]" → 200 with a hallucinated answer — the one silent-drop
outcome F1 exists to forbid. Fix: after filtering, if hasImageContent(full) is true but no image
survives, return `400 images_unsupported_in_system_messages`. Narrow (OpenAI disallows images in
the system role) so no legitimate request is rejected. Documented in README § Images.

Tests: +4 (all pure-module) — parsePositiveInt('unlimited') keeps the 150k default (gap a) and a
valid override is honored; hasImageContent proves the guard predicate fires for a system-only
image (true on full list, false after the system filter) and does NOT fire for a user-message
image. 304 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: vvlasy-openclaw <vvlasy-openclaw@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: vvlasy-openclaw <vvlasy@gmail.com>
Co-authored-by: dtzp555 <dtzp555@gmail.com>
 2026-07-20 07:50:17 +10:00
openclaw.vvlasy.czGitHubvvlasy-openclawClaude Opus 4.8taodeng
 		0c3e42b2e4 feat(models): repoint default sonnet alias to claude-sonnet-5 (#168) 
* feat(models): add Claude Sonnet 5 to models.json SPOT

Adds `claude-sonnet-5` (the latest Sonnet, supported by claude CLI >= 2.1.206)
to models.json — the single source of truth (ADR 0003). Both the /v1/models
endpoint and setup.mjs OpenClaw registration derive from it automatically.

- New model entry `claude-sonnet-5` (reasoning, 200k ctx, 16k max tokens),
  mirroring the existing Sonnet entry shape.
- Point the `sonnet` alias at `claude-sonnet-5` (newest Sonnet), consistent
  with `opus` -> `claude-opus-4-8`. Previous `claude-sonnet-4-6` is retained
  for pinning.
- README "Available Models" table updated (release-kit 5.3).
- Update the aliases.sonnet SPOT test to the new default.

Endpoint class: B.1 (/v1/models), data-only via the models.json SPOT.
Authorized by ADR 0006 (OpenAI shim scope) + ADR 0003 (models.json SPOT).
Verified: `claude --model claude-sonnet-5 -p` returns a valid response on a
current subscription CLI (2.1.206); npm test green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(models): make PR #152 purely additive + close ocp-connect drift + real SPOT test

Addresses the maintainer's review. Rescopes this PR to the additive change only —
adding claude-sonnet-5 to models.json — and defers the `sonnet` alias repoint to
its own PR per Iron Rule 11 (the alias is the default for every request that omits
`model`; repointing it is a behavior change that deserves separate review + a
CHANGELOG entry). No server.mjs change, so no cli.js citation required.

Metadata confirmed unchanged: contextWindow 200000 / maxTokens 16384 stay, per the
maintainer's correction (OCP truncates at MAX_PROMPT_CHARS, and contextWindow feeds
OpenClaw's compaction budget — advertising a larger window than OCP delivers just
makes OpenClaw overshoot).

Fixes vs review:

1. Reverted `aliases.sonnet` back to claude-sonnet-4-6 — this PR only *adds* the
   model; the repoint ships separately. README updated to match (5 is available by
   full ID; 4-6 remains the alias default).

2. Replaced the tautological SPOT test. The old assertion read a literal out of
   models.json and asserted it equalled the same literal — it passed even with a
   dangling alias. Added referential-integrity tests: every aliases/legacyAliases
   value must resolve to a real models[].id, plus an explicit assertion that
   claude-sonnet-5 exists in models[]. This is the guard that actually catches an
   alias pointing at a non-existent model (VALID_MODELS keys on alias names, never
   targets, so nothing else checks this).

3. Fixed ocp-connect classification drift. Its prefix table pinned "claude-sonnet-4",
   which misses "claude-sonnet-5" and falls through to the non-reasoning / 8k-output
   default. Broadened both the model_meta and alias_prefixes tables to family
   prefixes (claude-opus / claude-sonnet / claude-haiku) so any future versioned ID
   classifies correctly with no per-model edit. /v1/models does not expose
   reasoning/maxTokens (OpenAI /v1/models schema has no such fields — adding them
   would be a Rule 2 invention), so family classification stays in ocp-connect.
   primary_model stays claude-sonnet-4-6, matching the (unchanged) sonnet alias — it
   moves with the alias in the repoint PR.

Tests: 266 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(models): repoint default `sonnet` alias to claude-sonnet-5

Split out from #152 per Iron Rule 11: the additive model entry (#152) lands the
claude-sonnet-5 metadata; this PR makes the behavior change — moving the default
`sonnet` alias from claude-sonnet-4-6 to claude-sonnet-5.

`aliases.sonnet` is the model used for every /v1/chat/completions request that omits
`model` (server.mjs default) and, via ocp-connect, OpenClaw's OCP primary. Repointing
it changes behavior for every such client, so it gets its own PR + CHANGELOG entry
separate from the additive entry.

- models.json: aliases.sonnet -> claude-sonnet-5 (claude-sonnet-4-6 kept by full ID
  for pinning). Both are pricing tier_3_15 — no cost regression.
- ocp-connect: primary_model now prefers claude-sonnet-5 (falls back to 4-6, then
  first model), tracking the alias default so OpenClaw's primary matches.
- README: swap the "default for sonnet alias" annotation onto claude-sonnet-5.
- CHANGELOG: Unreleased § Changed entry documenting the default change + how to pin.
- test: SPOT assertion updated to claude-sonnet-5; referential-integrity tests from
  #152 continue to guard that the alias target actually exists in models[].

No server.mjs change, so no cli.js citation required.

Depends on #152 (needs the claude-sonnet-5 models[] entry to exist, else the
referential-integrity test fails). Rebase/merge after #152 lands.

Tests: 266 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: vvlasy-openclaw <vvlasy@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: dtzp555 <dtzp555@gmail.com>
 2026-07-17 08:10:33 +10:00
openclaw.vvlasy.czGitHubvvlasy-openclawClaude Opus 4.8
 		27216646c8 feat(models): add Claude Sonnet 5 to models.json SPOT (#152) 
* feat(models): add Claude Sonnet 5 to models.json SPOT

Adds `claude-sonnet-5` (the latest Sonnet, supported by claude CLI >= 2.1.206)
to models.json — the single source of truth (ADR 0003). Both the /v1/models
endpoint and setup.mjs OpenClaw registration derive from it automatically.

- New model entry `claude-sonnet-5` (reasoning, 200k ctx, 16k max tokens),
  mirroring the existing Sonnet entry shape.
- Point the `sonnet` alias at `claude-sonnet-5` (newest Sonnet), consistent
  with `opus` -> `claude-opus-4-8`. Previous `claude-sonnet-4-6` is retained
  for pinning.
- README "Available Models" table updated (release-kit 5.3).
- Update the aliases.sonnet SPOT test to the new default.

Endpoint class: B.1 (/v1/models), data-only via the models.json SPOT.
Authorized by ADR 0006 (OpenAI shim scope) + ADR 0003 (models.json SPOT).
Verified: `claude --model claude-sonnet-5 -p` returns a valid response on a
current subscription CLI (2.1.206); npm test green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(models): make PR #152 purely additive + close ocp-connect drift + real SPOT test

Addresses the maintainer's review. Rescopes this PR to the additive change only —
adding claude-sonnet-5 to models.json — and defers the `sonnet` alias repoint to
its own PR per Iron Rule 11 (the alias is the default for every request that omits
`model`; repointing it is a behavior change that deserves separate review + a
CHANGELOG entry). No server.mjs change, so no cli.js citation required.

Metadata confirmed unchanged: contextWindow 200000 / maxTokens 16384 stay, per the
maintainer's correction (OCP truncates at MAX_PROMPT_CHARS, and contextWindow feeds
OpenClaw's compaction budget — advertising a larger window than OCP delivers just
makes OpenClaw overshoot).

Fixes vs review:

1. Reverted `aliases.sonnet` back to claude-sonnet-4-6 — this PR only *adds* the
   model; the repoint ships separately. README updated to match (5 is available by
   full ID; 4-6 remains the alias default).

2. Replaced the tautological SPOT test. The old assertion read a literal out of
   models.json and asserted it equalled the same literal — it passed even with a
   dangling alias. Added referential-integrity tests: every aliases/legacyAliases
   value must resolve to a real models[].id, plus an explicit assertion that
   claude-sonnet-5 exists in models[]. This is the guard that actually catches an
   alias pointing at a non-existent model (VALID_MODELS keys on alias names, never
   targets, so nothing else checks this).

3. Fixed ocp-connect classification drift. Its prefix table pinned "claude-sonnet-4",
   which misses "claude-sonnet-5" and falls through to the non-reasoning / 8k-output
   default. Broadened both the model_meta and alias_prefixes tables to family
   prefixes (claude-opus / claude-sonnet / claude-haiku) so any future versioned ID
   classifies correctly with no per-model edit. /v1/models does not expose
   reasoning/maxTokens (OpenAI /v1/models schema has no such fields — adding them
   would be a Rule 2 invention), so family classification stays in ocp-connect.
   primary_model stays claude-sonnet-4-6, matching the (unchanged) sonnet alias — it
   moves with the alias in the repoint PR.

Tests: 266 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: vvlasy-openclaw <vvlasy@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-17 07:35:42 +10:00