feat(models): commit models.schema.json and validate the SPOT against it in CI (#196)

models.json has declared `"$schema": "./models.schema.json"` since v3.11.0, but
that file was never committed — `git log --all -- models.schema.json` is empty.
So the SPOT that ADR 0003 makes canonical had NO structural validation: a
missing contextWindow, a typo'd openclawName, or a boolean written as a string
would pass every existing test and only surface downstream — in OpenClaw's
registry, or silently inside a truncation budget.

Validated with the repo's OWN validator (lib/structured-output.mjs, shipped for
#153) rather than adding ajv. AGENTS.md requires minimal dependencies and the
repo currently has ZERO; this keeps it that way and exercises that validator on
a second real input. Capability-probed first rather than assumed: it enforces
type / required / enum / const / additionalProperties (both the boolean and the
schema form) / items / minItems, which covers everything the SPOT needs.

Three tests:
  - the $schema reference resolves to a committed file (the #196 bug itself)
  - models.json validates against models.schema.json, strict
  - the schema actually REJECTS 8 corruption shapes — missing required field,
    wrong scalar type, non-integer window, unknown extra field, alias mapped to
    a non-string, empty models array, wrong document version, unknown
    top-level key

That third test is the guard-on-the-guard: without it a vacuous schema (`{}` or
a typo'd `properties`) would make the second test pass on anything.

Mutation-proven in three directions:
  schema replaced with {}              -> "schema failed to reject: missing required field"
  additionalProperties:false removed   -> guard fails (loosened constraint caught)
  models.json corrupted (drop a field) -> strict validation fails

What the schema deliberately does NOT encode: referential integrity — that
every aliases/legacyAliases target exists as a models[].id. That is not a JSON
Schema concept; it is already covered by two dedicated tests, and the schema
says so in its description so the next reader doesn't assume it is covered.

The field descriptions carry the non-obvious consequences that have bitten this
repo before: contextWindow is a GLOBAL lever (max across all entries x 3 sets
MAX_PROMPT_CHARS for every model, ADR 0009), maxTokens is advertising only and
is enforced by OpenClaw rather than OCP, and models[] order is load-bearing
because ocp-connect uses model_ids[0] as a fallback.

Documented per release_kit (new file/SPOT/schema -> contributor section):
AGENTS.md "Key files to know" and README's Available Models section.

Tests: 460 passed, 0 failed (457 + 3).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8
This commit is contained in:
2026-07-27 08:48:49 +10:00
co-authored by Claude Opus 5
parent c998370b8a
commit 544cc3dad9
4 changed files with 113 additions and 1 deletions
+65
View File
@@ -0,0 +1,65 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/dtzp555-max/ocp/blob/main/models.schema.json",
"title": "OCP models.json",
"description": "Schema for models.json, the single source of truth for model metadata (ADR 0003). Enforced in CI by test-features.mjs, which validates models.json against this file using the repo's own validateJsonSchema (lib/structured-output.mjs) — no new dependency. NOTE: referential integrity (every aliases/legacyAliases target must exist as a models[].id) is NOT expressible here and is covered by separate tests.",
"type": "object",
"required": ["version", "models", "aliases", "legacyAliases"],
"additionalProperties": false,
"properties": {
"$schema": {
"type": "string",
"description": "Relative path to this file. Editors use it for completion; CI does not read it."
},
"version": {
"const": 1,
"description": "Schema version of this document. Bump only alongside a shape change and an ADR."
},
"models": {
"type": "array",
"minItems": 1,
"description": "Every model OCP advertises on /v1/models, newest first. Order is load-bearing: ocp-connect uses model_ids[0] as a primary fallback.",
"items": {
"type": "object",
"required": ["id", "displayName", "openclawName", "reasoning", "contextWindow", "maxTokens"],
"additionalProperties": false,
"properties": {
"id": {
"type": "string",
"description": "Canonical CLI model id, passed verbatim to `claude --model`. Must match the id in the compiled CLI registry."
},
"displayName": {
"type": "string",
"description": "Human label. Also used as the OpenClaw agent alias by scripts/sync-openclaw.mjs."
},
"openclawName": {
"type": "string",
"description": "Label written into OpenClaw's model registry."
},
"reasoning": {
"type": "boolean",
"description": "Whether OpenClaw should treat the model as reasoning-capable."
},
"contextWindow": {
"type": "integer",
"description": "Advertised context window. GLOBAL side effect: MAX_PROMPT_CHARS derives from max(contextWindow) x 3 across ALL entries (lib/prompt.mjs derivePromptCharBudget), so raising this on ONE model raises the truncation ceiling for EVERY model. See ADR 0009. Also feeds OpenClaw's compaction budget."
},
"maxTokens": {
"type": "integer",
"description": "Advertised output cap. OCP does not enforce it; it is propagated to OpenClaw (via setup.mjs / scripts/sync-openclaw.mjs) where it bounds request and compaction budgets."
}
}
}
},
"aliases": {
"type": "object",
"description": "Short name -> canonical models[].id. Client-addressable, so a repoint changes routing for every request using the alias. Cache keys resolve these before hashing (server.mjs cacheModel).",
"additionalProperties": { "type": "string" }
},
"legacyAliases": {
"type": "object",
"description": "Retired ids kept resolvable for backward compatibility -> canonical models[].id. Also client-addressable and also resolved in cache keys.",
"additionalProperties": { "type": "string" }
}
}
}