Files
olp/.github/workflows/alignment.yml
T
taodengandClaude Opus 4.7 (noreply@anthropic.com) dff428f3d0 docs(governance): fold in codex round-2 review findings (6 issues)
External Codex CLI review pass 2 surfaced 6 substantive issues that
round 1 fold-in missed — the self-consistency trap recurred when fold-in
was scoped only to files codex explicitly named in round 1. This commit
closes round 2 in full.

1. ADR 0002 contradicted ALIGNMENT.md (P1, codex round 2 finding 1)
   ADR 0002 still said "three default-enabled (Anthropic, OpenAI Codex,
   Mistral Vibe)" while ALIGNMENT.md (post round 1) said v0.1 ships zero
   Enabled Providers. Accepted ADR contradicted constitution.
   Fix: ADR 0002 + ADR 0001 + docs/adr/README.md index rewritten to
   Candidate framing.

2. release.yml would publish stale v0.1.0-bootstrap notes (P1, round 2
   finding 2)
   The "Unreleased" amendments would have been silently dropped on tag
   push because release.yml extracts only the matching version section.
   Fix: CHANGELOG restructured so the amended state IS the v0.1.0-
   bootstrap section. Full review history (opus + 2 codex rounds)
   captured inline.

3. package.json advertised non-existent entrypoints (P2, round 2
   finding 3)
   main/scripts.test/scripts.start pointed to files that do not exist.
   Local npm test and npm start failed; CI masked.
   Fix: remove all three from package.json. They return in Phase 1
   alongside the real files. test.yml bootstrap-tolerance updated to
   also skip when scripts.test is absent.

4. models-registry.json missing despite SPOT claim (P2, round 2
   finding 4)
   Fix: minimal stub committed (version + empty providers map).
   alignment.yml validator now actually runs.

5. alignment.yml commit-citation soft check Bash subshell trap (P2,
   round 2 finding 5)
   git log ... while read ... WARN=1 — the while loop ran in a subshell
   because of the pipe, so WARN never propagated out. The post-loop
   check always reported "clean" even when warnings fired.
   Fix: process substitution done less than less than (git log ...).

6. Tier A "permanent" wording inconsistent across ADR 0006 + alignment.
   yml workflow text (P3, round 2 finding 6)
   Fix: unified to "Excluded by default with no routine reinstatement
   path; re-inclusion requires ADR 0006 supersession or amendment with
   new primary-source evidence."

Reviewer: OpenAI Codex CLI (external, fresh-context, pass 2). Iron Rule
10 satisfied — round 2 reviewer was not the implementer of round 1
fold-in.

Memory learning updated: the self-consistency trap recurs in the fold-in
step. Future fold-ins must grep the entire repo for the concept, not
only edit files the reviewer named. See learnings/ai_reviewer_self_
consistency_trap.md in cross-machine memory.

Co-Authored-By: Claude Opus 4.7 (noreply@anthropic.com)
2026-05-23 16:35:23 +10:00

220 lines
8.8 KiB
YAML

name: Alignment Guardrail
on:
pull_request:
paths:
- 'server.mjs'
- 'setup.mjs'
- 'lib/**'
- 'scripts/**'
- 'models-registry.json'
- '.github/workflows/alignment.yml'
push:
branches: [main]
paths:
- 'server.mjs'
- 'setup.mjs'
- 'lib/**'
- 'scripts/**'
- 'models-registry.json'
- '.github/workflows/alignment.yml'
jobs:
blacklist:
name: source blacklist (hard fail)
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Scan source for hallucinated tokens
shell: bash
run: |
set -euo pipefail
# Known-hallucinated tokens. Extend only via an ALIGNMENT.md amendment PR.
# Each token is matched as a fixed string against the OLP source tree
# (excluding docs/, CHANGELOG, README, this workflow, and tests that
# may legitimately reference the historical token as a guardrail).
#
# Inherited transitively from OCP's 2026-04-11 drift:
# - api.anthropic.com/api/oauth/usage : fabricated Anthropic OAuth
# usage endpoint. Does not appear in any shipped @anthropic-ai/
# claude-code cli.js. Carried forward as a transitive guardrail.
#
# OLP-native entries: added as drift incidents accumulate.
BLACKLIST=(
"api.anthropic.com/api/oauth/usage"
)
# Provider keys that may appear in source (positive list — present in
# ALIGNMENT.md provider inventory). Any provider key in source that
# is NOT in this list is suspicious and flagged below.
KNOWN_PROVIDERS=(
"anthropic"
"openai"
"mistral"
"grok"
"kimi"
"minimax"
"glm"
"qwen"
)
# Source files in scope. Exclude docs, CHANGELOG, README, the
# workflow itself, and the test file (which may pin historical
# strings intentionally).
SOURCE_FILES="$(git ls-files \
| grep -E '\.(mjs|js|ts|json)$' \
| grep -v -E '^(docs/|CHANGELOG\.md|README\.md|test-features\.mjs|\.github/workflows/alignment\.yml)')"
FAIL=0
# 1. Blacklist scan
for token in "${BLACKLIST[@]}"; do
if echo "$SOURCE_FILES" | xargs grep -n -F "$token" 2>/dev/null; then
echo "::error::Blacklisted token '$token' detected in OLP source."
FAIL=1
fi
done
# 2. Excluded-provider scan: Google Antigravity is permanently
# excluded per ADR 0006 / ALIGNMENT.md Risk Tier A. Any reference
# to a `google-antigravity` plugin file or provider key in source
# (outside docs which may discuss the exclusion) is a finding.
FORBIDDEN_PROVIDER_TOKENS=(
"google-antigravity"
"antigravity"
)
for token in "${FORBIDDEN_PROVIDER_TOKENS[@]}"; do
HITS="$(echo "$SOURCE_FILES" | xargs grep -n -F "$token" 2>/dev/null || true)"
if [ -n "$HITS" ]; then
echo "::error::Tier-A-excluded provider token '$token' detected in OLP source. Per ALIGNMENT.md / ADR 0006, this provider is excluded by default; re-inclusion requires ADR 0006 supersession/amendment with new primary-source evidence."
echo "$HITS"
FAIL=1
fi
done
if [ "$FAIL" -ne 0 ]; then
echo ""
echo "============================================================"
echo "ALIGNMENT GUARDRAIL FAILURE"
echo "============================================================"
echo "OLP source contains a token on the alignment blacklist or"
echo "references a Tier-A-excluded provider."
echo ""
echo "Blacklist tokens were introduced by LLM hallucinations and"
echo "do not appear in the relevant authority (provider CLI,"
echo "OpenAI spec, or ADR). See ALIGNMENT.md and (where the"
echo "token is inherited from OCP) the OCP 2026-04-11 drift"
echo "record at https://github.com/dtzp555-max/ocp."
echo ""
echo "Excluded providers are listed in ALIGNMENT.md \xc2\xa7 Risk Tier"
echo "Framework and ADR 0006. Tier-A exclusion means not"
echo "bundled, not pluggable, not added via opt-in;"
echo "re-inclusion requires ADR 0006 amendment with new"
echo "primary-source evidence."
echo ""
echo "Required action:"
echo " 1. Remove the token from source."
echo " 2. Cite the real authority (provider CLI doc / OpenAI"
echo " spec section / ADR) for the operation you intended."
echo " 3. See ALIGNMENT.md Rules 1, 2, and 5."
echo ""
echo "Do not add allowlist entries to this workflow without an"
echo "amendment PR to ALIGNMENT.md (see Amendment Procedure)."
echo "============================================================"
exit 1
fi
echo "Blacklist + excluded-provider scan clean."
models-registry:
name: models-registry.json sanity (hard fail)
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Validate models-registry.json
shell: bash
run: |
set -euo pipefail
if [ ! -f models-registry.json ]; then
echo "models-registry.json not found yet (bootstrap phase). Skipping."
exit 0
fi
# Basic JSON validity.
if ! node -e "JSON.parse(require('fs').readFileSync('models-registry.json','utf8'))"; then
echo "::error::models-registry.json is not valid JSON."
exit 1
fi
# Provider keys in models-registry.json must match the inventory
# in ALIGNMENT.md.
KNOWN_PROVIDERS='["anthropic","openai","mistral","grok","kimi","minimax","glm","qwen"]'
node -e "
const fs = require('fs');
const known = ${KNOWN_PROVIDERS};
const reg = JSON.parse(fs.readFileSync('models-registry.json','utf8'));
const providers = reg.providers || {};
const bad = Object.keys(providers).filter(p => !known.includes(p));
if (bad.length > 0) {
console.error('::error::Unknown provider key(s) in models-registry.json: ' + bad.join(', '));
console.error('Known providers per ALIGNMENT.md: ' + known.join(', '));
process.exit(1);
}
console.log('models-registry.json provider keys OK: ' + Object.keys(providers).join(', '));
"
commit-citation:
name: per-provider commit citation (soft check)
runs-on: ubuntu-latest
continue-on-error: true
steps:
- name: Checkout full history
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Scan PR commits for uncited assertions
if: github.event_name == 'pull_request'
shell: bash
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
if [ -z "${BASE_SHA:-}" ] || [ -z "${HEAD_SHA:-}" ]; then
echo "No PR context; skipping."
exit 0
fi
# Use process substitution `< <(...)` rather than piping into the
# while loop. A piped while runs in a subshell, so `WARN=1` would
# never propagate back out to this scope — the if-check below would
# always report "clean" even when warnings were emitted. Classic
# Bash subshell trap; see commit history for the codex review that
# caught this.
WARN=0
while read -r sha; do
BODY="$(git log -1 --format=%B "$sha")"
if echo "$BODY" | grep -E -i -q '(provider|claude|codex|vibe|grok|kimi|minimax|glm|qwen|cli)[[:space:]]+(code[[:space:]]+)?uses'; then
if echo "$BODY" | grep -E -i -q '(cli[[:space:]]+v[0-9]+|https?://|ADR[[:space:]]+[0-9]{4})'; then
echo "OK $sha: assertion cited."
else
echo "::warning::Commit $sha asserts 'Provider X uses ...' or '<provider> CLI uses ...' but does not cite a CLI version, docs URL, or ADR number. See CLAUDE.md -> Commit message conventions."
WARN=1
fi
fi
done < <(git log --format="%H" "${BASE_SHA}..${HEAD_SHA}")
if [ "$WARN" -ne 0 ]; then
echo "Soft check raised warnings. Reviewer: please enforce per CLAUDE.md."
else
echo "Commit citation soft check clean."
fi