Both review findings taken.
MED — the schema declared `$schema: draft/2020-12`, which promises full draft
semantics, while the validator that actually runs it enforces only a subset.
Someone adding `"minimum": 1` to contextWindow would get SILENT no-op with a
green suite — a trap made worse by the file looking authoritative. The
description now enumerates exactly what is enforced (type, required, const,
enum, additionalProperties in both forms, items, minItems/maxItems,
anyOf/allOf/oneOf, $ref, nullable) and names what is ignored (minimum,
maxLength, pattern, uniqueItems, propertyNames, not), with the instruction to
add such a constraint as a test instead.
LOW — three corruptions neither the schema nor any sibling test caught:
duplicate models[].id, empty-string id/displayName/openclawName, and
non-positive contextWindow/maxTokens. They are structurally NOT expressible
with the supported keyword set (no uniqueItems, no minLength, no minimum), so
they are asserted directly rather than by pretending the schema covers them.
Mutation-proven, all three previously slipped through:
duplicate id -> CAUGHT
empty displayName -> CAUGHT
zero contextWindow -> CAUGHT
Tests: 461 passed, 0 failed (460 + 1).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8
models.json has declared `"$schema": "./models.schema.json"` since v3.11.0, but
that file was never committed — `git log --all -- models.schema.json` is empty.
So the SPOT that ADR 0003 makes canonical had NO structural validation: a
missing contextWindow, a typo'd openclawName, or a boolean written as a string
would pass every existing test and only surface downstream — in OpenClaw's
registry, or silently inside a truncation budget.
Validated with the repo's OWN validator (lib/structured-output.mjs, shipped for
#153) rather than adding ajv. AGENTS.md requires minimal dependencies and the
repo currently has ZERO; this keeps it that way and exercises that validator on
a second real input. Capability-probed first rather than assumed: it enforces
type / required / enum / const / additionalProperties (both the boolean and the
schema form) / items / minItems, which covers everything the SPOT needs.
Three tests:
- the $schema reference resolves to a committed file (the #196 bug itself)
- models.json validates against models.schema.json, strict
- the schema actually REJECTS 8 corruption shapes — missing required field,
wrong scalar type, non-integer window, unknown extra field, alias mapped to
a non-string, empty models array, wrong document version, unknown
top-level key
That third test is the guard-on-the-guard: without it a vacuous schema (`{}` or
a typo'd `properties`) would make the second test pass on anything.
Mutation-proven in three directions:
schema replaced with {} -> "schema failed to reject: missing required field"
additionalProperties:false removed -> guard fails (loosened constraint caught)
models.json corrupted (drop a field) -> strict validation fails
What the schema deliberately does NOT encode: referential integrity — that
every aliases/legacyAliases target exists as a models[].id. That is not a JSON
Schema concept; it is already covered by two dedicated tests, and the schema
says so in its description so the next reader doesn't assume it is covered.
The field descriptions carry the non-obvious consequences that have bitten this
repo before: contextWindow is a GLOBAL lever (max across all entries x 3 sets
MAX_PROMPT_CHARS for every model, ADR 0009), maxTokens is advertising only and
is enforced by OpenClaw rather than OCP, and models[] order is load-bearing
because ocp-connect uses model_ids[0] as a fallback.
Documented per release_kit (new file/SPOT/schema -> contributor section):
AGENTS.md "Key files to know" and README's Available Models section.
Tests: 460 passed, 0 failed (457 + 3).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8
taodengandClaude Opus 5