taodeng/olp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/olp.git synced 2026-07-19 09:45:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(governance): fold in codex round-2 review findings (6 issues)

Browse Source 
External Codex CLI review pass 2 surfaced 6 substantive issues that
round 1 fold-in missed — the self-consistency trap recurred when fold-in
was scoped only to files codex explicitly named in round 1. This commit
closes round 2 in full.

1. ADR 0002 contradicted ALIGNMENT.md (P1, codex round 2 finding 1)
   ADR 0002 still said "three default-enabled (Anthropic, OpenAI Codex,
   Mistral Vibe)" while ALIGNMENT.md (post round 1) said v0.1 ships zero
   Enabled Providers. Accepted ADR contradicted constitution.
   Fix: ADR 0002 + ADR 0001 + docs/adr/README.md index rewritten to
   Candidate framing.

2. release.yml would publish stale v0.1.0-bootstrap notes (P1, round 2
   finding 2)
   The "Unreleased" amendments would have been silently dropped on tag
   push because release.yml extracts only the matching version section.
   Fix: CHANGELOG restructured so the amended state IS the v0.1.0-
   bootstrap section. Full review history (opus + 2 codex rounds)
   captured inline.

3. package.json advertised non-existent entrypoints (P2, round 2
   finding 3)
   main/scripts.test/scripts.start pointed to files that do not exist.
   Local npm test and npm start failed; CI masked.
   Fix: remove all three from package.json. They return in Phase 1
   alongside the real files. test.yml bootstrap-tolerance updated to
   also skip when scripts.test is absent.

4. models-registry.json missing despite SPOT claim (P2, round 2
   finding 4)
   Fix: minimal stub committed (version + empty providers map).
   alignment.yml validator now actually runs.

5. alignment.yml commit-citation soft check Bash subshell trap (P2,
   round 2 finding 5)
   git log ... while read ... WARN=1 — the while loop ran in a subshell
   because of the pipe, so WARN never propagated out. The post-loop
   check always reported "clean" even when warnings fired.
   Fix: process substitution done less than less than (git log ...).

6. Tier A "permanent" wording inconsistent across ADR 0006 + alignment.
   yml workflow text (P3, round 2 finding 6)
   Fix: unified to "Excluded by default with no routine reinstatement
   path; re-inclusion requires ADR 0006 supersession or amendment with
   new primary-source evidence."

Reviewer: OpenAI Codex CLI (external, fresh-context, pass 2). Iron Rule
10 satisfied — round 2 reviewer was not the implementer of round 1
fold-in.

Memory learning updated: the self-consistency trap recurs in the fold-in
step. Future fold-ins must grep the entire repo for the concept, not
only edit files the reviewer named. See learnings/ai_reviewer_self_
consistency_trap.md in cross-machine memory.

Co-Authored-By: Claude Opus 4.7 (noreply@anthropic.com)
This commit is contained in:
taodeng
2026-05-23 16:35:23 +10:00
co-authored by Claude Opus 4.7 (noreply@anthropic.com)
parent 91223ee9ab
commit dff428f3d0
9 changed files with 78 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+48 -37
View File
@@ -2,57 +2,68 @@
All notable changes to OLP land here. Per `CLAUDE.md` release_kit overlay, this file is the source of truth for GitHub release notes.
## Unreleased — 2026-05-23 (governance amendments per external codex review)
External AI review (OpenAI Codex CLI) of the bootstrap governance surfaced six substantive findings beyond what the internal opus reviewer caught. All six folded in this turn:
1. **Provider Inventory split into Candidate vs Enabled** — ALIGNMENT.md previously listed `anthropic` / `openai` / `mistral` as Tier D default-enabled while their Authority pins were still `TBD at Phase N spawn`. This violated Rule 1 (Cite First) and Rule 3 (Match the Implementation). The v0.1 founding commit now ships **zero Enabled Providers**; all 8 providers are Candidate. Enablement is a Phase audit deliverable, not a bootstrap claim. (ALIGNMENT.md § Provider Inventory; README.md § Supported Providers.)
2. **Antigravity Tier A evidence strength downgraded to "evidence-backed, pending primary-source pin"** — ADR 0006 previously framed the Antigravity exclusion as a closed case based on secondary reports. The reports actually disagree on blast radius (piunikaweb 03-02 says AI-tier only; piunikaweb 02-23, OpenClaw issue #14203, VentureBeat say broader). The Google FAQ language naming OpenClaw/OpenCode/Claude Code is cited from secondary sources only; the original FAQ URL or archival snapshot has not been primary-source-pinned. The exclusion is still active by default, but the constitutional weight now matches the evidence: secondary-sourced. Primary-source pinning is tracked as an open one-shot audit task with a 90-day Tier-reconsideration trigger.
3. **ADR 0001 supersession scope honesty** — ADR 0001 previously claimed OLP is "the structural shape ADR 0005 endorsed: a separate repo with multi-provider design baked in from day one." But ADR 0005's separate-repo recommendation came with two qualifiers OLP rejects: "BYOK from day one" and "no `cli.js` spawn." OLP rejects both — it is non-commercial and explicitly spawn-binary. The supersession is now narrowly scoped to "single-provider-sufficiency premise only"; the BYOK / no-spawn parts of ADR 0005 are not inherited.
4. **Anthropic post-2026-06-15 one-shot audit scheduled** — ALIGNMENT.md's annual 14 May audit would have left the Anthropic Tier re-evaluation almost a year late. Added a one-shot audit for 2026-06-16 (or first Anthropic billing-cycle close) that verifies the post-effective-date behaviour and updates ADR 0006 + Authority pin.
5. **Tier A "permanent" language unified across docs** — ALIGNMENT.md and ADR 0006 disagreed: one said "permanently excluded," the other said "amendment-procedure-revisable." Unified as "Excluded by default. Cannot be re-included unless ADR 0006 is superseded or amended with new primary-source evidence." The amendment procedure remains available; "Tier A" sets the bar for re-inclusion at constitutional-amendment level, not at routine PR level.
6. **OpenAI Tier D wording softened** — ADR 0006 previously described Codex Discussion #8338 as "maintainer confirmed permissive." The discussion is actually a maintainer posture statement ("OSS projects like OpenCode are doing things similar") with an explicit "I'm an engineer, not a lawyer" caveat. Now described as "maintainer signal indicates low risk; formal ToS pin pending" with the pin tracked as a follow-up audit task.
Files changed: `ALIGNMENT.md`, `README.md`, `docs/adr/0001-project-founding.md`, `docs/adr/0006-provider-inclusion.md`. No code, no CI workflow, no PR template change.
Reviewer for this amendment: OpenAI Codex CLI (external, fresh-context). Iron Rule 10 satisfied — the internal opus reviewer was not the source of the findings, and the maintainer is not the author of the underlying critique. The amendment's substantive changes are direct fold-ins of the reviewer's six findings; the internal opus reviewer's earlier APPROVE_WITH_MINOR verdict is therefore narrowed retroactively to "APPROVE conditional on these amendments" for the purposes of the v0.1 governance bootstrap.
## v0.1.0-bootstrap — 2026-05-23
### Phase 0 — Repo bootstrap
### Phase 0 — Repo bootstrap (founding + post-codex-review hardening)
This is the founding commit of OLP (Open LLM Proxy), a personal- and family-scale multi-provider LLM proxy that supersedes OCP. The trigger was Anthropic's 2026-05-14 announcement (effective 2026-06-15) splitting `claude -p` / Agent SDK / third-party agent traffic out of the Pro/Max subscription pool into a separate fixed monthly Agent SDK Credit pool.
This is the founding commit set of OLP (Open LLM Proxy), a personal- and family-scale multi-provider LLM proxy that supersedes OCP. The trigger was Anthropic's 2026-05-14 announcement (effective 2026-06-15) splitting `claude -p` / Agent SDK / third-party agent traffic out of the Pro/Max subscription pool into a separate fixed monthly Agent SDK Credit pool.
**What lands in this commit:**
**What lands at v0.1.0-bootstrap (final state on `main` as of 2026-05-23):**
- `ALIGNMENT.md` — OLP constitution. Three concurrent authorities (per-provider CLI / OpenAI spec / IR contract), 5 Rules, 4-tier Risk Tier Framework, 8-provider inventory.
- `ALIGNMENT.md` — OLP constitution. Three concurrent authorities (per-provider CLI / OpenAI spec / IR contract), 5 Rules, 4-tier Risk Tier Framework, Candidate-vs-Enabled provider inventory, one-shot triggered audits (2026-06-16 Anthropic post-split; 90-day Antigravity primary-source pin).
- `AGENTS.md` — multi-tool agent guidelines (inherits `~/.cc-rules/AGENTS.md`).
- `CLAUDE.md` — Claude-Code-specific session instructions + machine-readable `release_kit` overlay (Iron Rule 5.5).
- `README.md` — phase-aware skeleton with provider inventory, API endpoint table, environment-variables table, response-headers spec, architecture overview, phase plan, migration-from-OCP outline. Placeholder content marked as such per phase.
- `README.md` — phase-aware skeleton with Candidate-vs-Enabled provider tables, API endpoint table, environment-variables table, response-headers spec, architecture overview, phase plan, migration-from-OCP outline. Placeholder content marked as such per phase.
- `docs/adr/` — 6 founding ADRs:
- `0001-project-founding.md` — Mission, non-mission, and supersession of OCP ADR 0005 (No Multi-Provider).
- `0002-plugin-architecture.md` — `lib/providers/<name>.mjs` plug-in model with the Provider contract (name / models / auth / spawn / estimateCost / quotaStatus / healthCheck / hints).
- `0001-project-founding.md` — Mission, non-mission, narrow-scope supersession of OCP ADR 0005 (single-provider-sufficiency premise only; BYOK / no-spawn parts of ADR 0005 not inherited).
- `0002-plugin-architecture.md` — `lib/providers/<name>.mjs` plug-in model with the Provider contract (name / models / auth / spawn / estimateCost / quotaStatus / healthCheck / hints). 8 candidate providers declared, 0 Enabled at v0.1.
- `0003-intermediate-representation.md` — OLP-internal canonical IR between OpenAI-compat entry and provider plugins.
- `0004-fallback-engine.md` — Trigger taxonomy (Hard / Soft / Deterministic-deferred / Cost-aware-deferred), idempotent-failure safety, first-chunk rule, chain advancement.
- `0004-fallback-engine.md` — Trigger taxonomy (Hard / Soft / Deterministic-deferred / Cost-aware-deferred), idempotent-failure safety (first-chunk rule), chain advancement one-at-a-time, observability headers.
- `0005-cache-cross-provider.md` — Cache key composition over `(provider, model, messages, ...)`, D1+D2+D3+D4 port from OCP v3.13.0.
- `0006-provider-inclusion.md` — 4-tier Risk Framework, 8-provider classification, Antigravity exclusion (named prohibition + no cost advantage + reinstatement friction; *not* whole-account ban — Google AI services tier only per piunikaweb 2026-03-02 OpenClaw exec confirmation).
- `0006-provider-inclusion.md` — 4-tier Risk Framework, Candidate-vs-Enabled distinction, 8-provider candidate classification, Antigravity Tier A (evidence-backed, pending primary-source pin) — exclusion rests on (named prohibition + no cost advantage + reinstatement friction) combination; primary-source URL not yet pinned, follow-up tracked.
- `.github/PULL_REQUEST_TEMPLATE.md` — 8-radio Change Type taxonomy + per-type Authority Evidence sections + Iron Rule 10 reviewer checklist.
- `.github/workflows/alignment.yml` — CI blacklist (transitive `api.anthropic.com/api/oauth/usage` from OCP 2026-04-11 drift; Antigravity provider exclusion enforcement) + `models-registry.json` validator + commit-citation soft check.
- `.github/workflows/alignment.yml` — CI blacklist (transitive `api.anthropic.com/api/oauth/usage` from OCP 2026-04-11 drift; Antigravity provider exclusion enforcement) + `models-registry.json` validator + commit-citation soft check (process-substitution form, no Bash subshell trap).
- `.github/workflows/release.yml` — Auto-release on tag push with `package.json`-vs-tag version match check (Iron Rule 5).
- `.github/workflows/test.yml` — Node 20/24 matrix; tolerates bootstrap-phase absence of `test-features.mjs`.
- `package.json`, `.gitignore`, `LICENSE` (MIT), `CHANGELOG.md` — standard project boilerplate.
- `.github/workflows/test.yml` — Node 20/24 matrix; tolerates bootstrap-phase absence of `test-features.mjs` AND `scripts.test`.
- `models-registry.json` — minimal v0.1 stub with empty `providers: {}`, matching the 0-Enabled posture; populated by Phase audits as providers transition Candidate → Enabled.
- `package.json` — minimal: no `main`, no `scripts.test`, no `scripts.start` (those entries land alongside the real files in Phase 1).
- `.gitignore`, `LICENSE` (MIT), `CHANGELOG.md` — standard project boilerplate.
**Provider inventory at bootstrap:**
**Provider posture at v0.1.0-bootstrap (per ALIGNMENT.md § Provider Inventory):**
| Tier | Providers |
|---|---|
| D (default-enabled) | Anthropic, OpenAI Codex, Mistral Vibe |
| C (opt-in, no consent) | xAI Grok, Moonshot Kimi |
| B (opt-in, explicit consent) | MiniMax, Zhipu GLM, Alibaba Qwen |
| A (permanently excluded) | Google Antigravity |
| Tier | Anticipated providers | v0.1 default state |
|---|---|---|
| D (eligible-for-default-enabled) | Anthropic, OpenAI Codex, Mistral Vibe | Candidate (transition gate: authority pin + plugin + Phase audit) |
| C (opt-in) | xAI Grok, Moonshot Kimi | Candidate |
| B (opt-in + consent) | MiniMax, Zhipu GLM, Alibaba Qwen | Candidate |
| A (excluded by default; constitutional-amendment-only re-inclusion) | Google Antigravity | Excluded; pending primary-source pin |
**Governance gate at bootstrap:**
**Total Enabled at v0.1.0-bootstrap: 0.** Enablement is a Phase audit deliverable, not a bootstrap claim. This explicit zero is intentional and codified — a constitution that names providers as "default-enabled" while their CLI versions, output shapes, auth artifacts, and exit-code semantics are still TBD would violate Rules 1 (Cite First) and 3 (Match the Implementation).
- Fresh-context independent reviewer (opus, Iron Rule 10) audited all 15 governance files against the OLP v0.1 spec and OCP precedent. Verdict: APPROVE_WITH_MINOR. Two minor findings folded in before this commit (alignment.yml heredoc indentation fix; AGENTS.md ADR-0003 reference clarification).
**Review history for this version:**
**Next:** Phase 1 lands `server.mjs` skeleton + IR + Anthropic provider plugin + cache D1+D4 port from OCP. Per the spec §6 phase plan.
1. **Initial internal review (Claude Opus, fresh-context, Iron Rule 10).** Verdict: APPROVE_WITH_MINOR — 2 minor items (alignment.yml heredoc indent breaking bash parse on failure path; AGENTS.md cross-reference to ADR 0003 imprecise). Both folded in before the founding commit.
2. **External review #1 (OpenAI Codex CLI, no spec framing).** Verdict: 6 substantive findings beyond internal review.
- Provider Inventory split into Candidate vs Enabled (the v0.1 constitution had declared `anthropic` / `openai` / `mistral` as Tier D default-enabled while their Authority pins were still `TBD at Phase N spawn` — direct violation of Rule 1 / Rule 3 against the constitution's own text).
- Antigravity Tier A downgraded to "evidence-backed, pending primary-source pin" (secondary reports disagree on blast radius; Google FAQ URL not yet primary-source-pinned).
- ADR 0001 supersession scope narrowed (OLP rejects ADR 0005's "BYOK + no spawn" qualifiers, which originally applied to a commercial pivot; OLP is non-commercial and spawn-binary by design).
- Anthropic post-2026-06-15 one-shot audit scheduled (annual May 14 audit would leave Anthropic re-eval ~year late after the split takes effect).
- Tier A "permanent" language unified across docs (constitution and ADR 0006 had disagreed).
- OpenAI Tier D wording softened ("maintainer signal indicates low risk; formal ToS pin pending" — Discussion #8338 is a posture statement, not a formal ToS blessing).
3. **External review #2 (OpenAI Codex CLI, second pass after review #1 fold-in).** Verdict: 6 additional substantive findings — the self-consistency trap recurred when fold-in of review #1 was scoped only to files codex explicitly named. Round #2 caught:
- ADR 0002 still claimed "three default-enabled" while ALIGNMENT.md said zero Enabled — accepted ADR contradicting constitution.
- `release.yml` would publish stale `## v0.1.0-bootstrap` notes that ignored the "Unreleased" amendments — fixed by consolidating amendments into the v0.1.0-bootstrap section (this entry).
- `package.json` advertised `main` / `scripts.test` / `scripts.start` for files that don't exist — `npm test` / `npm start` failed locally. Removed all three; will return in Phase 1 alongside the real files.
- `models-registry.json` documented as SPOT but missing — minimal stub added.
- `alignment.yml` commit-citation soft check had a Bash subshell trap (`while` in pipe loses `WARN=1` mutation) — fixed via process substitution `< <(...)`.
- Tier A "permanent" wording still inconsistent across `alignment.yml` workflow text, ADR 0006 Consequences section, and the rest of the docs — unified throughout.
All 6 round-#2 findings folded in this consolidated v0.1.0-bootstrap state.
**Reviewer framing learning (recorded permanently in `~/.cc-rules/memory/learnings/ai_reviewer_self_consistency_trap.md`):** Internal AI reviewers framed on a shared source-of-truth miss bugs in the source-of-truth itself. The self-consistency trap recurred during the fold-in of round #1 — when an external reviewer surfaces findings, the fold-in must grep the entire repo for the same concept, not only edit the files the reviewer named. Round #2 caught what round #1's fold-in missed for exactly this reason. Both lessons updated in the cross-machine memory.
**Iron Rule 10 status:** Satisfied. Initial reviewer = internal opus (independent from drafters). Round #1 reviewer = external codex (independent from drafters and from internal opus). Round #2 reviewer = external codex (independent from the round #1 fold-in implementer). The maintainer's role across all three reviews was approver, not author. The drafting agents and fold-in agents were never the same as the reviewers for any of the three passes.
**Next:** Phase 1 lands `server.mjs` skeleton + IR + Anthropic provider plugin + cache D1+D4 port from OCP. At that point, `package.json` regains `main` + `scripts.test` + `scripts.start`, `test-features.mjs` lands, `models-registry.json` populates its first `providers.anthropic` entry, and Anthropic transitions Candidate → Enabled. Per spec §6 phase plan.