From cb87b82935cdc65c35eb07a899bd8f54091a9fa8 Mon Sep 17 00:00:00 2001 From: dtzp555 Date: Mon, 27 Jul 2026 12:16:01 +1000 Subject: [PATCH] =?UTF-8?q?fix(ocp-connect):=20revert=20the=20unknown-id?= =?UTF-8?q?=20fallback=20to=208192=20=E2=80=94=2032000=20over-advertised?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit raised ocp-connect's unknown-id fallback from 8192 to 32000 on the stated grounds that 32000 was "the LOWEST max_output_tokens.default in the CLI 2.1.220 registry". That is false, and the change it justified made this PR violate its own thesis. Pairing-free enumeration of the compiled registry (17 occurrences, no grep-window artifacts): 7 max_output_tokens:{default:64000,upper:128000} 5 max_output_tokens:{default:32000,upper:64000} 2 max_output_tokens:{default:8192,upper:8192} 2 max_output_tokens:{default:32000,upper:32000} 1 max_output_tokens:{default:32000,upper:128000} distinct defaults: 8192 / 32000 / 64000 GLOBAL MINIMUM: 8192 The floor is 8192, held by claude-3-5-haiku and claude-3-5-sonnet — and those are exactly the ids that reach the fallback, since "claude-3-5-*" matches none of the family prefixes. So the raise over-advertised the only two real models the branch serves, by 4x, in a PR whose entire claim is that advertised metadata should be true. Verified by exercising the real classifier (verbatim source slice, not a replica): claude-3-5-haiku 8192 FALLBACK registry 8192 exact claude-3-5-sonnet 8192 FALLBACK registry 8192 exact claude-opus-5 64000 family registry 64000 exact claude-sonnet-4-6 32000 family registry 32000 exact claude-haiku-4-5-20251001 32000 family registry 32000 exact claude-sonnet-5 32000 family registry 64000 under (safe) Also in this commit: - The family table's comment said "FAMILY FLOOR of the CLI registry". It is the floor over each family's members CURRENTLY IN models.json, not over the registry family: claude-opus-4-0 / -4-5 are opus-family at 32000, so the opus row would over-advertise them 2x. Unreachable today (ocp-connect derives ids from /v1/models, i.e. models.json), but the comment now states the real scope and what to do if a legacy opus is added. - The test table keyed haiku on "claude-haiku-4-5-20251001" while telling the reader to extract values id-anchored. That id has ZERO id-anchored hits: the registry record is id:"claude-haiku-4-5" and the dated string appears only as its provider_ids.first_party (10 bare-string hits). Following the instruction on that row returns nothing, which is precisely what pushes a reader back to the bare-string search this test exists to prevent. The row now carries the registry id. Reviewer credit: the reviewer that found this also retracted its own earlier finding, which is what produced the 32000 fallback in the first place. Both directions verified independently here before acting. server.mjs: unchanged by this commit (ALIGNMENT.md requires no cli.js citation). Suite: 462 passed, 0 failed. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8 --- CHANGELOG.md | 2 +- ocp-connect | 26 ++++++++++++++++---------- test-features.mjs | 12 ++++++++++-- 3 files changed, 27 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 44eaa6a..2efe66f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ ### Changed -- **`maxTokens` now matches the CLI registry instead of a uniform 16384 (#195).** Every Opus entry and `claude-sonnet-5` go to **64000**, `claude-sonnet-4-6` and `claude-haiku-4-5` to **32000** — the `max_output_tokens.default` each model declares in the compiled CLI 2.1.220 registry. This corrects **advertised metadata only**: `models.json` is the SPOT (ADR 0003) and every value in it should be the truth about the model. **It changes nothing about how OCP behaves.** OCP never enforces `maxTokens` — `buildCliArgs` passes no output-token flag to the CLI at all — and OpenClaw addresses a local OCP over `openai-completions`, whose request field (`max_completion_tokens`) appears nowhere in this repo. The value is consumed only by clients that choose to honour it, via `setup.mjs` / `scripts/sync-openclaw.mjs` / `ocp-connect`. **Expect no change in answer length or quota burn.** `ocp-connect`'s independent family table moves to the family floor (opus 64000, sonnet 32000, haiku 32000) since prefixes cannot distinguish versions. +- **`maxTokens` now matches the CLI registry instead of a uniform 16384 (#195).** Every Opus entry and `claude-sonnet-5` go to **64000**, `claude-sonnet-4-6` and `claude-haiku-4-5` to **32000** — the `max_output_tokens.default` each model declares in the compiled CLI 2.1.220 registry. This corrects **advertised metadata only**: `models.json` is the SPOT (ADR 0003) and every value in it should be the truth about the model. **It changes nothing about how OCP behaves.** OCP never enforces `maxTokens` — `buildCliArgs` passes no output-token flag to the CLI at all — and OpenClaw addresses a local OCP over `openai-completions`, whose request field (`max_completion_tokens`) appears nowhere in this repo. The value is consumed only by clients that choose to honour it, via `setup.mjs` / `scripts/sync-openclaw.mjs` / `ocp-connect`. **Expect no change in answer length or quota burn.** `ocp-connect`'s independent family table moves to the floor over each family's current `models.json` members (opus 64000, sonnet 32000, haiku 32000), since prefixes cannot distinguish versions. Its unknown-id fallback stays at **8192** — the registry's global minimum, held by `claude-3-5-haiku` and `claude-3-5-sonnet`, which are the only real ids that reach it (`claude-3-5-*` matches no family prefix). ## v3.25.0 — 2026-07-27 diff --git a/ocp-connect b/ocp-connect index 15cc7b3..b71e093 100755 --- a/ocp-connect +++ b/ocp-connect @@ -129,11 +129,14 @@ provider = { # re-trip it. Family prefixes classify any versioned ID correctly with no per-model # edit. (ADR 0003: models.json is the SPOT for model existence; /v1/models does not # expose reasoning/maxTokens, so family classification stays here.) -# maxTokens values are the FAMILY FLOOR of the CLI registry max_output_tokens.default -# (opus family 64000; sonnet 32000 because sonnet-4-6 is 32000 while sonnet-5 is 64000; -# haiku 32000). Family prefixes cannot distinguish versions, so the floor is used -# deliberately: under-advertising caps a client lower than the model allows, which is safe, -# whereas over-advertising would promise capacity a member of the family does not have. +# maxTokens values are the floor of the CLI registry's max_output_tokens.default over the +# family members CURRENTLY IN models.json (opus 64000; sonnet 32000, because sonnet-4-6 is +# 32000 while sonnet-5 is 64000; haiku 32000) — NOT over the whole registry family. The +# registry also holds claude-opus-4-0 / -4-5 at 32000, so should a legacy opus ever be added +# to models.json, the opus row here would over-advertise it and must drop to 32000. +# Family prefixes cannot distinguish versions, so a floor is used deliberately: +# under-advertising caps a client lower than the model allows, which is safe, whereas +# over-advertising would promise capacity a member of the family does not have. # models.json is authoritative per ADR 0003; this table only exists because /v1/models does # not expose maxTokens. model_meta = { @@ -147,11 +150,14 @@ def get_model_meta(mid): for prefix, meta in sorted(model_meta.items(), key=lambda x: -len(x[0])): if mid.startswith(prefix): return meta - # Unknown id = most likely a model newer than this script. 32000 is the LOWEST - # max_output_tokens.default in the CLI 2.1.220 registry (haiku/sonnet-4-6), so it is the - # safe floor: never over-advertises a real model, and no longer contradicts the rest of - # this table by sitting below every family in it. - return {"name": mid + " (OCP)", "reasoning": False, "maxTokens": 32000} + # Unknown id. 8192 is the GLOBAL MINIMUM max_output_tokens.default across all 17 records in + # the CLI 2.1.220 registry (distinct values: 8192 / 32000 / 64000). The two records holding + # it — claude-3-5-haiku and claude-3-5-sonnet, both default:8192, upper:8192 — are precisely + # the ones that reach this line, because "claude-3-5-*" matches none of the family prefixes + # above. Raising this to a family-table value would over-advertise those two by 4x. Keep 8192: + # under-advertising caps a client lower than a model allows, which is safe; over-advertising + # promises capacity that does not exist. + return {"name": mid + " (OCP)", "reasoning": False, "maxTokens": 8192} for mid in model_ids: meta = get_model_meta(mid) diff --git a/test-features.mjs b/test-features.mjs index 6eb9b69..3ff4e8b 100644 --- a/test-features.mjs +++ b/test-features.mjs @@ -4310,9 +4310,17 @@ test("models.json: every aliases value resolves to a real models[].id (referenti // value above the bar and still call itself "registry-aligned" — which is the actual claim. Adding // a model means adding a row here, and that is the point: the row is where you record what the // registry said when you checked. -const _spotRegistryMaxTokens = { // CLI 2.1.220, id-anchored max_output_tokens.default +// Keys are models.json ids; values are the CLI 2.1.220 registry's max_output_tokens.default, +// each extracted id-anchored (grep 'id:""' + the following bytes) — never by bare-string +// search, which matches cross-references inside OTHER models' records and silently attributes +// the wrong number. ONE KEY IS NOT A REGISTRY ID: models.json carries the dated haiku id, but +// the registry record is id:"claude-haiku-4-5" (the dated string appears only as that record's +// provider_ids.first_party). Anchor the haiku row on the SHORT id; anchoring on the dated one +// returns nothing, which is what tempts the next reader back into a bare-string search. +const _spotRegistryMaxTokens = { "claude-opus-5": 64000, "claude-opus-4-8": 64000, "claude-opus-4-7": 64000, "claude-opus-4-6": 64000, - "claude-sonnet-5": 64000, "claude-sonnet-4-6": 32000, "claude-haiku-4-5-20251001": 32000, + "claude-sonnet-5": 64000, "claude-sonnet-4-6": 32000, + "claude-haiku-4-5-20251001": 32000, // registry id: claude-haiku-4-5 }; test("models.json: every maxTokens equals the CLI registry's max_output_tokens.default (#195)", () => { for (const m of _spotModels.models) {