Compare commits

..
Author SHA1 Message Date
taodengandClaude Opus 5 79656b485f ci: drop the alignment.yml paths change — it would have been a green check for a check that never ran
Reverting my own half of this PR. The reviewer proved the alignment.yml change
was theatre, and my issue #198 was filed on a wrong premise.

What I verified myself before reverting:
  - the alignment job BODY references models.json zero times; only the paths
    filter I added mentioned it
  - test.yml is `pull_request:` with NO paths filter, so it already runs on
    every PR, models.json-only ones included
  - on a deliberately corrupted models.json (contextWindow 1000000, a typo'd
    openclawName), `npm test` catches it: 455 passed, 2 failed

So the real guard on models.json already existed and always ran. Adding the
path would only have made a job execute that inspects nothing about
models.json, and then reported a green "Alignment Guardrail" — implying a check
that did not happen. That is strictly worse than the job not running, which is
precisely the reviewer's point.

#198's premise was also wrong. The blacklist greps server.mjs for known
hallucinated tokens. A models.json-only PR cannot introduce a token into
server.mjs, so CLAUDE.md hard-requirement #2 is INAPPLICABLE on such a PR, not
vacuous. I read "the guardrail didn't run" as "coverage gap" without checking
what the guardrail actually inspects.

The release.yml half is unaffected and stays: that one is a real, reproduced
failure (the no-CHANGELOG branch never wrote the file the create step consumes).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8
2026-07-27 09:05:33 +10:00
taodengandClaude Opus 5 66096cdcab ci: cover models.json in the alignment guardrail; fix release.yml's no-CHANGELOG path
Two CI-layer fixes found during the v3.25.0 review. Same layer, both small, so
they land together (Iron Rule 11).

#198 — alignment.yml did not run on a models.json-only PR.
The `paths:` filter listed server.mjs / setup.mjs / scripts / lib / ocp /
ocp-connect, but not models.json. That made CLAUDE.md hard-requirement #2 ("CI
blacklist pass") vacuous for exactly the PRs that change model routing:
confirmed on #192, where only gitleaks and test-features ran.

models.json is not inert data. It drives MODEL_MAP and VALID_MODELS, the
default request model, and — since ADR 0009 — the global MAX_PROMPT_CHARS
truncation budget. A models.json-only PR can therefore change server.mjs's
runtime behavior with the guardrail never running. models.schema.json is
included for the same reason one level up: loosening the schema is what would
let a malformed models.json through.

Adding paths only widens coverage; the blacklist grep still scans server.mjs,
so this costs nothing and closes the gap. Per CLAUDE.md this is a PR amendment
to alignment.yml, which is the sanctioned way to change it (removing entries
would need an ALIGNMENT.md amendment; nothing is removed here).

#202 — release.yml's no-CHANGELOG fallback would have failed the release job.
The branch wrote an output named `notes` and exited WITHOUT creating
/tmp/release-notes.md, while the create step hard-codes
`--notes-file /tmp/release-notes.md`. So the one path that exists to "degrade
to minimal notes" instead produced a failed release.

Verified both directions by extracting the step's actual script from the YAML
and running it, rather than reading it:

  PRE-FIX,  no CHANGELOG -> exit 0, GITHUB_OUTPUT="notes=Release v3.25.0",
                            /tmp/release-notes.md MISSING
                            -> gh release create --notes-file WOULD FAIL
  POST-FIX, no CHANGELOG -> exit 0, notes_file set, file present ("Release v3.25.0")
  POST-FIX, with CHANGELOG -> file present, first line "## v3.25.0 — 2026-07-27"

Fixed by writing the file in that branch, and by removing the hard-coded-path
coupling entirely: both branches now set `notes_file` and the create step
consumes `steps.notes.outputs.notes_file`. That output existed already and was
never read — the two only agreed by accident.

Also added `set -euo pipefail` (the step previously ran unset-tolerant) and an
echo of the resolved notes before creating the release, so a wrong-looking body
is visible in the job log instead of only on the published release.

NOT changed: the empty-extraction guard. My issue text suggested adding one,
but `if [ ! -s "$NOTES" ]` was already there and already handles a heading
mismatch. Correcting that claim on #202 rather than taking credit for it.

Both workflows re-validated as YAML.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8
2026-07-27 08:51:09 +10:00
5 changed files with 23 additions and 47 deletions
+13 -6
View File
@@ -21,24 +21,31 @@ jobs:
- name: Extract CHANGELOG section
id: notes
run: |
set -euo pipefail
VERSION="${{ steps.ver.outputs.version }}"
NOTES=/tmp/release-notes.md
# Extract section for this version from CHANGELOG.md
# Pattern: "## v${VERSION}" through the next "## " or EOF
if [ ! -f CHANGELOG.md ]; then
echo "No CHANGELOG.md found; using minimal release notes"
echo "notes=Release v${VERSION}" >> $GITHUB_OUTPUT
# MUST write the file, not just an output: the create step consumes a FILE, so an
# early exit here used to leave --notes-file pointing at a path that never existed,
# turning "degrade to minimal notes" into a failed release job (#202).
echo "Release v${VERSION}" > "$NOTES"
echo "notes_file=$NOTES" >> $GITHUB_OUTPUT
exit 0
fi
awk -v ver="v${VERSION}" '
$0 ~ "^## " ver { found=1; print; next }
found && /^## v/ { exit }
found { print }
' CHANGELOG.md > /tmp/release-notes.md
if [ ! -s /tmp/release-notes.md ]; then
' CHANGELOG.md > "$NOTES"
if [ ! -s "$NOTES" ]; then
echo "No matching section in CHANGELOG for v${VERSION}; using minimal notes"
echo "Release v${VERSION}" > /tmp/release-notes.md
echo "Release v${VERSION}" > "$NOTES"
fi
echo "notes_file=/tmp/release-notes.md" >> $GITHUB_OUTPUT
echo "--- release notes (${VERSION}) ---"; cat "$NOTES"
echo "notes_file=$NOTES" >> $GITHUB_OUTPUT
- name: Create GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -49,5 +56,5 @@ jobs:
fi
gh release create "v${{ steps.ver.outputs.version }}" \
--title "v${{ steps.ver.outputs.version }}" \
--notes-file /tmp/release-notes.md \
--notes-file "${{ steps.notes.outputs.notes_file }}" \
--latest
-6
View File
@@ -1,11 +1,5 @@
# Changelog
## Unreleased
### Changed
- **`maxTokens` now matches the CLI registry instead of a uniform 16384 (#195).** Every Opus entry and `claude-sonnet-5` go to **64000**, `claude-sonnet-4-6` and `claude-haiku-4-5` to **32000** — the `max_output_tokens.default` each model actually declares in the compiled CLI 2.1.220 registry. OCP never enforced this value; it is propagated to OpenClaw, where it **caps the outbound request**: `maxTokens = Math.min(baseMaxTokens + thinkingBudget, modelMaxTokens)`. OpenClaw's reasoning budgets are medium 8192 / high 16384, and when `maxTokens <= thinkingBudget` it clamps thinking to `maxTokens - 1024` — so a model declared at 16384 running at **high** reasoning spent ~15360 on thinking and left **~1024 tokens for the actual answer**. `ocp-connect`'s independent family table moves to the family floor (opus 64000, sonnet 32000, haiku 32000) since prefixes cannot distinguish versions. Expect longer answers and correspondingly higher per-request quota burn on long generations.
## v3.25.0 — 2026-07-27
Minor release. Headline: **Claude Opus 5** joins the model list and the `opus` alias now resolves to it. Alongside it, two `server.mjs` correctness fixes found by review rather than by users — a monotonic in-flight-counter leak, and cache keys that were hashing the alias string instead of the model it resolves to. The three TUI/prompt fixes that landed after v3.24.0 are included here too.
+7 -7
View File
@@ -8,7 +8,7 @@
"openclawName": "Claude Opus 5 (via CLI)",
"reasoning": true,
"contextWindow": 200000,
"maxTokens": 64000
"maxTokens": 16384
},
{
"id": "claude-opus-4-8",
@@ -16,7 +16,7 @@
"openclawName": "Claude Opus 4.8 (via CLI)",
"reasoning": true,
"contextWindow": 200000,
"maxTokens": 64000
"maxTokens": 16384
},
{
"id": "claude-opus-4-7",
@@ -24,7 +24,7 @@
"openclawName": "Claude Opus 4.7 (via CLI)",
"reasoning": true,
"contextWindow": 200000,
"maxTokens": 64000
"maxTokens": 16384
},
{
"id": "claude-opus-4-6",
@@ -32,7 +32,7 @@
"openclawName": "Claude Opus 4.6 (via CLI)",
"reasoning": true,
"contextWindow": 200000,
"maxTokens": 64000
"maxTokens": 16384
},
{
"id": "claude-sonnet-5",
@@ -40,7 +40,7 @@
"openclawName": "Claude Sonnet 5 (via CLI)",
"reasoning": true,
"contextWindow": 200000,
"maxTokens": 64000
"maxTokens": 16384
},
{
"id": "claude-sonnet-4-6",
@@ -48,7 +48,7 @@
"openclawName": "Claude Sonnet 4.6 (via CLI)",
"reasoning": true,
"contextWindow": 200000,
"maxTokens": 32000
"maxTokens": 16384
},
{
"id": "claude-haiku-4-5-20251001",
@@ -56,7 +56,7 @@
"openclawName": "Claude Haiku 4.5 (via CLI)",
"reasoning": false,
"contextWindow": 200000,
"maxTokens": 32000
"maxTokens": 8192
}
],
"aliases": {
+3 -10
View File
@@ -129,17 +129,10 @@ provider = {
# re-trip it. Family prefixes classify any versioned ID correctly with no per-model
# edit. (ADR 0003: models.json is the SPOT for model existence; /v1/models does not
# expose reasoning/maxTokens, so family classification stays here.)
# maxTokens values are the FAMILY FLOOR of the CLI registry max_output_tokens.default
# (opus family 64000; sonnet 32000 because sonnet-4-6 is 32000 while sonnet-5 is 64000;
# haiku 32000). Family prefixes cannot distinguish versions, so the floor is used
# deliberately: under-advertising caps a client lower than the model allows, which is safe,
# whereas over-advertising would promise capacity a member of the family does not have.
# models.json is authoritative per ADR 0003; this table only exists because /v1/models does
# not expose maxTokens.
model_meta = {
"claude-opus": {"name": "Claude Opus (OCP)", "reasoning": True, "maxTokens": 64000},
"claude-sonnet": {"name": "Claude Sonnet (OCP)", "reasoning": True, "maxTokens": 32000},
"claude-haiku": {"name": "Claude Haiku (OCP)", "reasoning": False, "maxTokens": 32000},
"claude-opus": {"name": "Claude Opus (OCP)", "reasoning": True, "maxTokens": 16384},
"claude-sonnet": {"name": "Claude Sonnet (OCP)", "reasoning": True, "maxTokens": 16384},
"claude-haiku": {"name": "Claude Haiku (OCP)", "reasoning": False, "maxTokens": 8192},
}
def get_model_meta(mid):
-18
View File
@@ -4299,24 +4299,6 @@ test("models.json: every aliases value resolves to a real models[].id (referenti
}
});
// maxTokens must leave room for a visible answer once a client's thinking budget is taken out
// (#195). OCP never enforces maxTokens itself — it is propagated to OpenClaw (setup.mjs,
// scripts/sync-openclaw.mjs) where it CAPS the outbound request:
// maxTokens = Math.min(baseMaxTokens + thinkingBudget, modelMaxTokens)
// OpenClaw's reasoning budgets are medium 8192 / high 16384, and when maxTokens <= thinkingBudget
// it clamps thinking to maxTokens - 1024. So a model declared at 16384 running at `high` spent
// ~15360 on thinking and left ~1024 for the actual answer — which is what this repo shipped until
// v3.25.0. Asserting the PRINCIPLE rather than pinning per-model numbers: a value test would need
// editing every time a model is added, and would not say why.
const _spotHighThinkingBudget = 16384; // OpenClaw's `high` reasoning budget
test("models.json: every maxTokens exceeds the high thinking budget, leaving room for output (#195)", () => {
for (const m of _spotModels.models) {
assert.ok(m.maxTokens > _spotHighThinkingBudget,
`${m.id} declares maxTokens=${m.maxTokens} <= ${_spotHighThinkingBudget}: at OpenClaw's 'high' reasoning ` +
`level the thinking budget would consume all but ~1024 tokens of the response`);
}
});
test("models.json: every legacyAliases value resolves to a real models[].id (referential integrity)", () => {
for (const [name, target] of Object.entries(_spotModels.legacyAliases || {})) {
assert.ok(_spotModelIds.has(target), `legacyAliases.${name} -> '${target}' is a dangling alias (no matching models[].id)`);