mirror of
https://github.com/dtzp555-max/ocp.git
synced 2026-07-27 16:05:07 +00:00
f2f9058db4
* feat(models): commit models.schema.json and validate the SPOT against it in CI (#196) models.json has declared `"$schema": "./models.schema.json"` since v3.11.0, but that file was never committed — `git log --all -- models.schema.json` is empty. So the SPOT that ADR 0003 makes canonical had NO structural validation: a missing contextWindow, a typo'd openclawName, or a boolean written as a string would pass every existing test and only surface downstream — in OpenClaw's registry, or silently inside a truncation budget. Validated with the repo's OWN validator (lib/structured-output.mjs, shipped for #153) rather than adding ajv. AGENTS.md requires minimal dependencies and the repo currently has ZERO; this keeps it that way and exercises that validator on a second real input. Capability-probed first rather than assumed: it enforces type / required / enum / const / additionalProperties (both the boolean and the schema form) / items / minItems, which covers everything the SPOT needs. Three tests: - the $schema reference resolves to a committed file (the #196 bug itself) - models.json validates against models.schema.json, strict - the schema actually REJECTS 8 corruption shapes — missing required field, wrong scalar type, non-integer window, unknown extra field, alias mapped to a non-string, empty models array, wrong document version, unknown top-level key That third test is the guard-on-the-guard: without it a vacuous schema (`{}` or a typo'd `properties`) would make the second test pass on anything. Mutation-proven in three directions: schema replaced with {} -> "schema failed to reject: missing required field" additionalProperties:false removed -> guard fails (loosened constraint caught) models.json corrupted (drop a field) -> strict validation fails What the schema deliberately does NOT encode: referential integrity — that every aliases/legacyAliases target exists as a models[].id. That is not a JSON Schema concept; it is already covered by two dedicated tests, and the schema says so in its description so the next reader doesn't assume it is covered. The field descriptions carry the non-obvious consequences that have bitten this repo before: contextWindow is a GLOBAL lever (max across all entries x 3 sets MAX_PROMPT_CHARS for every model, ADR 0009), maxTokens is advertising only and is enforced by OpenClaw rather than OCP, and models[] order is load-bearing because ocp-connect uses model_ids[0] as a fallback. Documented per release_kit (new file/SPOT/schema -> contributor section): AGENTS.md "Key files to know" and README's Available Models section. Tests: 460 passed, 0 failed (457 + 3). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8 * fix(schema): warn that only a subset of draft 2020-12 is enforced; cover the gaps it cannot Both review findings taken. MED — the schema declared `$schema: draft/2020-12`, which promises full draft semantics, while the validator that actually runs it enforces only a subset. Someone adding `"minimum": 1` to contextWindow would get SILENT no-op with a green suite — a trap made worse by the file looking authoritative. The description now enumerates exactly what is enforced (type, required, const, enum, additionalProperties in both forms, items, minItems/maxItems, anyOf/allOf/oneOf, $ref, nullable) and names what is ignored (minimum, maxLength, pattern, uniqueItems, propertyNames, not), with the instruction to add such a constraint as a test instead. LOW — three corruptions neither the schema nor any sibling test caught: duplicate models[].id, empty-string id/displayName/openclawName, and non-positive contextWindow/maxTokens. They are structurally NOT expressible with the supported keyword set (no uniqueItems, no minLength, no minimum), so they are asserted directly rather than by pretending the schema covers them. Mutation-proven, all three previously slipped through: duplicate id -> CAUGHT empty displayName -> CAUGHT zero contextWindow -> CAUGHT Tests: 461 passed, 0 failed (460 + 1). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8 * test(schema): close the fourth gap — uniqueness on all three name fields, not just id Review approved the delta and named this exact follow-up as non-blocking: "extend the same test to assert uniqueness of displayName/openclawName and use m[f] === m[f].trim()". Implemented as specified. Why it matters rather than being tidiness: scripts/sync-openclaw.mjs maps `claude-local/<id>` -> { alias: displayName } and writes openclawName as the registry label, so a duplicate in EITHER field collapses two models onto one OpenClaw entry — the same defect class as a duplicate id, which the previous version covered while leaving its two siblings open. The trim check also changed shape. It was `m[f].trim().length > 0`, which a PADDED id passes by construction — and that id is handed VERBATIM to `claude --model`, so " claude-opus-5" would have failed upstream instead of here. Now asserts `m[f] === m[f].trim()`. Mutation-proven, all three previously slipped through: duplicate displayName -> CAUGHT duplicate openclawName -> CAUGHT whitespace-padded id -> CAUGHT Tests: 461 passed, 0 failed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017gbqUZ8HfBZpjjbzQ85oH8 --------- Co-authored-by: dtzp555 <dtzp555@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
66 lines
3.8 KiB
JSON
66 lines
3.8 KiB
JSON
{
|
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
"$id": "https://github.com/dtzp555-max/ocp/blob/main/models.schema.json",
|
|
"title": "OCP models.json",
|
|
"description": "Schema for models.json, the single source of truth for model metadata (ADR 0003). Enforced in CI by test-features.mjs, which validates models.json against this file using the repo's own validateJsonSchema (lib/structured-output.mjs) — no new dependency. ⚠ ONLY A SUBSET OF DRAFT 2020-12 IS ENFORCED: type, required, const, enum, additionalProperties (boolean and schema forms), items, minItems, maxItems, anyOf/allOf/oneOf, $ref, nullable. Anything else — minimum, maxLength, pattern, uniqueItems, propertyNames, not — is SILENTLY IGNORED by that validator, so adding one here buys nothing and misleads the next reader. Add such a constraint as a test in test-features.mjs instead. NOTE: referential integrity (every aliases/legacyAliases target must exist as a models[].id) is likewise not expressible here and is covered by separate tests.",
|
|
"type": "object",
|
|
"required": ["version", "models", "aliases", "legacyAliases"],
|
|
"additionalProperties": false,
|
|
"properties": {
|
|
"$schema": {
|
|
"type": "string",
|
|
"description": "Relative path to this file. Editors use it for completion; CI does not read it."
|
|
},
|
|
"version": {
|
|
"const": 1,
|
|
"description": "Schema version of this document. Bump only alongside a shape change and an ADR."
|
|
},
|
|
"models": {
|
|
"type": "array",
|
|
"minItems": 1,
|
|
"description": "Every model OCP advertises on /v1/models, newest first. Order is load-bearing: ocp-connect uses model_ids[0] as a primary fallback.",
|
|
"items": {
|
|
"type": "object",
|
|
"required": ["id", "displayName", "openclawName", "reasoning", "contextWindow", "maxTokens"],
|
|
"additionalProperties": false,
|
|
"properties": {
|
|
"id": {
|
|
"type": "string",
|
|
"description": "Canonical CLI model id, passed verbatim to `claude --model`. Must match the id in the compiled CLI registry."
|
|
},
|
|
"displayName": {
|
|
"type": "string",
|
|
"description": "Human label. Also used as the OpenClaw agent alias by scripts/sync-openclaw.mjs."
|
|
},
|
|
"openclawName": {
|
|
"type": "string",
|
|
"description": "Label written into OpenClaw's model registry."
|
|
},
|
|
"reasoning": {
|
|
"type": "boolean",
|
|
"description": "Whether OpenClaw should treat the model as reasoning-capable."
|
|
},
|
|
"contextWindow": {
|
|
"type": "integer",
|
|
"description": "Advertised context window. GLOBAL side effect: MAX_PROMPT_CHARS derives from max(contextWindow) x 3 across ALL entries (lib/prompt.mjs derivePromptCharBudget), so raising this on ONE model raises the truncation ceiling for EVERY model. See ADR 0009. Also feeds OpenClaw's compaction budget."
|
|
},
|
|
"maxTokens": {
|
|
"type": "integer",
|
|
"description": "Advertised output cap. OCP does not enforce it; it is propagated to OpenClaw (via setup.mjs / scripts/sync-openclaw.mjs) where it bounds request and compaction budgets."
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"aliases": {
|
|
"type": "object",
|
|
"description": "Short name -> canonical models[].id. Client-addressable, so a repoint changes routing for every request using the alias. Cache keys resolve these before hashing (server.mjs cacheModel).",
|
|
"additionalProperties": { "type": "string" }
|
|
},
|
|
"legacyAliases": {
|
|
"type": "object",
|
|
"description": "Retired ids kept resolvable for backward compatibility -> canonical models[].id. Also client-addressable and also resolved in cache keys.",
|
|
"additionalProperties": { "type": "string" }
|
|
}
|
|
}
|
|
}
|