mirror of
https://github.com/dtzp555-max/olp.git
synced 2026-07-21 21:15:10 +00:00
f8348adb3b8428ba37765366a3823db158908379
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  taodengandClaude Opus 4.7
|
f8348adb3b |
fix(providers): D24 — close spawn-timeout race when timer fires during yield-suspension (round-2 F4)
cold-audit catch from 2026-05-24
Round-2 cold-audit Finding 4 (P2 concurrency bug). All 3 provider plugins
(anthropic / codex / mistral) had a race in spawn-timeout enforcement:
The timer's rejection branch is `if (rejectNext) { rejectNext(SPAWN_TIMEOUT) }`.
`rejectNext` is only set while the drain loop is AWAITING an empty-queue
promise. If the timer fires while the generator is suspended at a `yield`
(rejectNext === null because we're not currently awaiting), the rejection
is silently skipped. The loop then drains queued items, sees `close`
(from the SIGTERM), breaks normally. Post-loop guards on `!spawnTimedOut`
both skip: SPAWN_FAILED throw + yield-stop emit. Generator returns
normally with N partial chunks. Consumer (collectAllChunks) sees a
"successful" return. Fallback never advances. Truncated response gets
cached.
This violates BOTH:
- ADR 0004 § Trigger taxonomy bullet 4 (hard trigger never fires)
- ADR 0005 § Cache write conditions item 1 (truncated response cached)
The bug escaped FOUR reviewers: D10 diff-review, round-1 cold-audit, D11
reviewer, D14 reviewer. Round-2 cold-audit caught it because it was
unprimed and walked the timer/drain-loop race manually.
Fix (3 plugins, parallel race):
Each plugin gains an unconditional post-loop check, placed AFTER the
existing `try { drain loop } finally { clearTimeout(timer) }` block,
but BEFORE the existing `!spawnTimedOut`-gated guards:
```js
if (spawnTimedOut) {
throw new ProviderError(
`<cli-name> spawn timed out after ${maxSpawnTimeMs}ms`,
'SPAWN_TIMEOUT',
);
}
```
This closes the race surface: the inner-loop guard (unchanged, lines
~319/461/583) handles the rejectNext-set path; the new post-loop guard
handles the rejectNext-null path. Together they cover both halves —
SPAWN_TIMEOUT now reliably surfaces as a hard trigger regardless of
which path the timer fire took.
Changes (4 files, +154 / -0):
- lib/providers/anthropic.mjs: post-loop guard at lines 360-365 (after
clearTimeout at line 348-350, before existing SPAWN_FAILED guard at 368)
- lib/providers/codex.mjs: post-loop guard at lines 521-526 (parallel)
- lib/providers/mistral.mjs: post-loop guard at lines 643-648 (parallel)
- test-features.mjs: new Suite 16 test 16e — race reproduction (109 lines)
Test 16e race reproduction (deterministic):
Exploits Node event-loop ordering. Mock spawn schedules data1 + data2 +
close via a single setImmediate. setImmediate runs before timers in the
same tick, so all 3 events queue before the timer can fire. Generator
yields data1 then suspends at yield with rejectNext null. Consumer
pauses 25ms (> 10ms timer). Timer fires during yield-suspension:
spawnTimedOut=true, rejectNext null → branch skipped, SIGTERM sent.
Consumer resumes, drain processes data2 + close, breaks normally.
Post-loop: D24 guard catches spawnTimedOut → throws SPAWN_TIMEOUT.
Pre-D24 behavior: generator returns normally (truncated, silently
cacheable). Test would FAIL.
Post-D24: SPAWN_TIMEOUT throws → test PASSES.
Reviewer empirically validated the test's diagnostic power by stripping
D24 from anthropic.mjs and observing exactly 16e fail (only 16e — 16a
through 16d still pass on the orthogonal rejectNext-set path), then
restored.
Timing flake risk: very low. Race depends only on relative ordering of
two setTimeout callbacks (10ms vs 25ms) which Node guarantees regardless
of absolute drift. Reviewer ran 7 npm test passes; 16e timing spread
26.43-27.70ms (1.3ms variance).
Tests: 334 → 335 (+1). 335/335 pass on Node 20.
Out of scope (filed in issue #3):
- SPAWN_TIMEOUT salvage parity: this fix discards partial chunks (matches
current SPAWN_TIMEOUT semantics, doesn't introduce asymmetry vs
SPAWN_FAILED-no-chunks). Whether to add SPAWN_TIMEOUT-with-usable-chunks
salvage (analogous to D16's SPAWN_FAILED salvage) is a separate design
decision tracked in issue #3.
Authority:
- ADR 0004 § Trigger taxonomy — Hard triggers bullet 4: "Provider CLI
spawn timeout (configurable per-provider via hints.maxSpawnTimeMs)"
https://github.com/dtzp555-max/olp/blob/main/docs/adr/0004-fallback-engine.md
- ADR 0005 § Cache write conditions item 1 (no truncation)
- engine.mjs HARD_TRIGGER_CODES['SPAWN_TIMEOUT'] = true (D10 P1.3,
unchanged)
- CC 开发铁律 v1.6 § 10.x — Round-2 Cold Audit caught this
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): APPROVE. Verified guard position in all 3 plugins; read
push() to confirm rejectNext-null mechanism; empirically validated test
16e's diagnostic specificity (strip-revert-restore); 7 test runs
confirmed timing stability. No blocking issues.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
taodengandClaude Opus 4.7
|
ed82e65859 |
chore: D19 — cleanup batch (Findings 8 + 14 + 15 + D17 dead import)
cold-audit catch from 2026-05-23
Batched 4 small P3 mechanical cleanups per Iron Rule 11 IDR cleanup-batch
convention (all P3, all small, no semantic feature changes beyond defensive
validation).
Changes (7 files):
1. lib/ir/ir-to-openai.mjs (+30 / -3) — Finding 8 defensive validator:
- Added OPENAI_FINISH_REASON_ENUM Set with the 6 spec-allowed values
(stop / length / tool_calls / content_filter / function_call / null)
- Added normalizeFinishReason(value) helper that returns value unchanged
if in enum, else 'stop'
- Routed both irChunkToOpenAISSE (streaming path) and
irResponseToOpenAINonStream (non-stream path) through the helper
- Bonus tightening (in-scope, same finish_reason concept):
irResponseToOpenAINonStream's gate changed from `if (chunk.finish_reason)`
(truthy check) to `if (chunk.finish_reason !== undefined)` so an
explicit null (valid spec value meaning "still in progress") is no
longer silently dropped by the truthy guard
- Note: undefined → null collapse via `?? null` is unreachable in the
current codebase (all provider plugins explicitly set finish_reason
on stop chunks); defensive only against a future plugin that omits
the field — documented inline
2. .github/workflows/alignment.yml (-16) — Finding 14 dead CI cleanup:
- Removed `setup.mjs` from path triggers (push + pull_request) — the
file does not exist in the repo
- Removed the dead `KNOWN_PROVIDERS=(...)` bash array from job 1 and
its comment block — no later step iterated over it, so the array
was abandoned
- LEFT untouched: the Node.js inline KNOWN_PROVIDERS array in the
models-registry validation job — that one is actively consumed by
the schema validation script
3. lib/providers/anthropic.mjs / codex.mjs / mistral.mjs (3 × 1 line) —
Finding 15: removed unused `PROVIDER_ERROR_CODES` from import lines.
Each line went from `import { ProviderError, PROVIDER_ERROR_CODES } from
'./base.mjs';` to `import { ProviderError } from './base.mjs';`. The
constant remains exported from base.mjs (its declaration site, where
it IS used for validation).
4. server.mjs (1 line) — D17 reviewer's observation: removed unused
`getProviderForModel` from the import line. The function is only
called by lib/fallback/engine.mjs which imports it directly from
lib/providers/index.mjs. server.mjs's import was dead (the routing
SPOT lives in engine.mjs after D17 — server.mjs uses buildDefaultChain
exclusively).
5. test-features.mjs (+44) — Suite 3 (irChunkToOpenAISSE format) extended
with 4 new finish_reason normalization tests:
- Test 1: non-spec streaming finish_reason ('timeout', 'overloaded',
'cancelled') → mapped to 'stop'
- Test 2: spec-enum streaming finish_reason (all 6 incl. null) preserved
- Test 3: non-spec non-stream finish_reason → mapped to 'stop'
- Test 4: spec-enum non-stream finish_reason preserved (null
intentionally omitted — documented inline)
Tests: 324 → 328 (+4). All pass on Node 20.
Pre-commit fold-ins (per evidence-first checkpoint #4):
- **D19 reviewer suggestion #1**: added inline comment to
normalizeFinishReason explaining the unreachable `undefined → null`
branch (defensive only, no current plugin omits the field). Cheap
future-reader clarity.
- **D19 reviewer suggestion #2**: added inline comment to Test 4
explaining why null is intentionally omitted from the spec-enum list
(non-stream `!== undefined` gate enters with null and overwrites
default 'stop' to null — semantically odd but spec-valid).
Reviewer suggestion #3 (consider stricter `undefined → 'stop'` on
streaming-stop path vs `null → null` on delta path) explicitly marked
out of scope by reviewer — would require call-site context awareness;
filed mentally as potential future work, not tracked as an issue
since no current path triggers it.
Authority:
- ALIGNMENT.md Rule 2(b) — only spec-defined fields in OpenAI responses
- OpenAI Chat Completions spec finish_reason enum
https://platform.openai.com/docs/api-reference/chat/object#finish_reason
- CC 开发铁律 v1.6 § 10.x — Cold Audit Findings 8 / 14 / 15
Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): APPROVE. Verified the unreachable `undefined → null` branch
claim by grep-checking all 3 provider plugins (none emit undefined);
verified the two KNOWN_PROVIDERS arrays were correctly distinguished
(only the dead bash one removed); ran npm test independently to confirm
328/328.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
taodengandClaude Opus 4.7
|
2cfd0b194e |
feat(phase-1): D10 P1 hardening — providers.enabled wiring + real streaming + spawn timeout
Folds in three production-blocking defects from external Codex review
round-3 of Phase 1 (D5+D6+D8+D9 already merged). Without these, OLP
returns 503 on every request even when `npm start` succeeds, streams
arrive as a single buffered burst after the spawn completes, and hung
CLIs block the engine forever despite ADR 0004 declaring spawn-timeout
a hard trigger.
P1.1 providers.enabled config wiring (ADR 0002 § Disable model):
- loadFallbackConfigSync() returns tri-field {chains, soft_triggers,
providersEnabled}; server.mjs reads _startupConfig.providersEnabled
and passes to loadProviders() at startup
- Empty / missing config → 0 enabled providers → 503 no_enabled_provider
(matches v0.1 0-Enabled posture per ALIGNMENT.md § Provider Inventory)
- __setProvidersEnabled / __resetProvidersEnabled test seams; in-place
Map mutation preserves existing direct-mutation patterns in Suite 13
P1.2 Real SSE streaming on single-hop cache-miss (ADR 0003 entry adapter
pattern, OpenAI /v1/chat/completions stream=true contract):
- New handleChatCompletions branch when ir.stream === true && chain.length
=== 1 && !bypassCache && !preCheckHit
- for await (const irChunk of provider.spawn(...)) writes SSE per chunk
via res.write(irChunkToOpenAISSE(...)); accumulates streamedChunks for
cacheStore.set on stop
- First-chunk rule preserved: error-before-first-chunk → sendError(502);
error-after-first-chunk → truncated res.end(), no fallback
- Multi-hop chains (chain.length > 1) continue to use buffered
executeWithFallback to keep fallback safety semantics
P1.3 Spawn timeout hard trigger (ADR 0004 § Trigger taxonomy bullet 4):
- SPAWN_TIMEOUT added to PROVIDER_ERROR_CODES (lib/providers/base.mjs)
and HARD_TRIGGER_CODES (lib/fallback/engine.mjs)
- All three plugins (anthropic.mjs / codex.mjs / mistral.mjs) wrap drain
loop with setTimeout (default 600_000ms, configurable via
hints.maxSpawnTimeMs); on fire: proc.kill('SIGTERM') + reject pending
drain promise with ProviderError(..., 'SPAWN_TIMEOUT')
- Timer cleared in finally; resolveNext/rejectNext atomically nulled in
push() + timer-fire path to prevent late-fire double-settle
Tests 277 → 288 (+11). Suite 14 (4 providers.enabled), Suite 15
(3 streaming cache-miss real-time, including arrival-count >= 2
assertion that architecturally proves real streaming), Suite 16
(4 spawn timeout, including 2-hop chain advancement from timed-out
primary). 288/288 pass on Node 20.20.2 + Node 25.8.0.
Authorities:
- ADR 0002 § Disable model — config toggle, not plugin-removal
https://github.com/dtzp555-max/olp/blob/main/docs/adr/0002-plugin-architecture.md
- ADR 0003 § Translation direction model — entry adapter for await pattern
https://github.com/dtzp555-max/olp/blob/main/docs/adr/0003-intermediate-representation.md
- ADR 0004 § Trigger taxonomy + § Fallback safety (first-chunk rule)
https://github.com/dtzp555-max/olp/blob/main/docs/adr/0004-fallback-engine.md
- OpenAI /v1/chat/completions stream=true (server-sent events,
data: {chunk} per delta, terminator data: [DONE])
https://platform.openai.com/docs/api-reference/chat/streaming
Reviewer (Iron Rule 10): fresh-context opus, independent of drafter.
Verdict: APPROVE_WITH_MINOR. Folded the one cheap minor before commit
(Suite 15a arrival-count assertion strengthened from chunks.length > 0
to arrivalTimestamps.length >= 2 — the prior assertion would have
admitted a buffered impl). Two remaining non-blocking notes deferred:
optional writeHead deferral (low value; single-hop guard makes pre-
content 200 + empty body safe), and version bump (Phase 1 ships as
v0.1.0 aggregate when D11–D16 land).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
taodengandClaude Opus 4.7 (noreply@anthropic.com)
|
ea9184d2e8 |
feat(phase-1): land OpenAI Codex provider plugin (D6)
Phase 1 Day 4. OpenAI Codex provider plugin code lands as Candidate.
Anthropic and Codex now both in STATIC_REGISTRY length 2. Codex CLI is
NOT installed on the orchestrator machine, so D6 ships with a docs-only
authority pin; D7 will install the binary, probe real behaviour, and
fix any docs-vs-reality divergences (A3 access-token field, A4 NDJSON
event schema, possibly keyring storage support).
Files:
NEW: lib/providers/codex.mjs (586 lines initially, expanded ~10 lines
via reviewer fold-ins) — Codex provider implementing the v1.0
contract. spawns `codex exec --json --model <id> [PROMPT|-]` per
canonical Codex docs.
MOD: lib/providers/index.mjs — STATIC_REGISTRY now [anthropic, codex],
listAllProviderNames() returns 2 entries.
MOD: models-registry.json — providers.openai populated with five
documented model IDs (gpt-5.5, gpt-5.4, gpt-5.4-mini,
gpt-5.3-codex, gpt-5.3-codex-spark) and four aliases.
MOD: test-features.mjs — Suite 11 added with 46 tests covering contract
conformance, IR translation, mock-spawn behaviour, healthCheck,
estimateCost, registry length.
Authority citations (all WebFetched and verified during reviewer pass):
CLI reference: https://developers.openai.com/codex/cli/reference
Source for `codex exec` subcommand syntax, --json flag, --model -m
flag, and PROMPT positional including the `-` form for stdin piping.
Features: https://developers.openai.com/codex/cli/features
Reference for the supported-models list.
Auth: https://developers.openai.com/codex/auth/
Canonical pin for `~/.codex/auth.json` plaintext credential file
and `cli_auth_credentials_store = keyring` OS credential store option.
Models: https://developers.openai.com/codex/models
Canonical pin for the five documented model IDs (each shown as a
`codex -m <id>` example on the page).
ChatGPT plan: https://help.openai.com/en/articles/11369540 — Codex
runs against ChatGPT subscription budget when OAuth-authenticated;
OPENAI_API_KEY env path is for `codex login --with-api-key` only,
not `codex exec` runtime.
Architectural decisions:
1. Mirror D4 anthropic.mjs structure: file header, lossy translation
docs, default export = provider object, named exports include
__setSpawnImpl / __resetSpawnImpl for test injection.
2. Stdin path uses `args.push('-')` per documented CLI behaviour.
(Original D6 sonnet draft omitted the positional entirely and wrote
stdin directly — D6 reviewer pass 2 caught this; corrected before
commit. D7 E2E confirms.)
3. Auth artifact path `~/.codex/auth.json` is now documented in the
header as CONFIRMED per canonical auth doc, not assumed.
4. Access-token field name remains a defensive 3-name try-order
(access_token / token / accessToken) because the auth doc does
not enumerate field names. D7 captures real auth.json post-login.
5. OPENAI_API_KEY env injection during spawn is intentionally NOT
done. The auth doc clarifies OPENAI_API_KEY is a login-time input,
not a runtime override. codex exec reads its own auth artifact.
6. Codex stays Candidate. loadProviders({}) returns empty Map; only
loadProviders({ enabled: { openai: true } }) loads it. POST /v1/
chat/completions gpt-5.5 etc still returns 503 until config flag
is set + E2E audit passes.
Reviewer chain (Iron Rule 10):
Implementer: sonnet (general-purpose).
Fresh-context reviewer: opus (ecc:code-reviewer). Verdict
APPROVE_WITH_MINOR.
Reviewer ran npm test (174/174 pass with Suite 10 skipped), WebFetched
all four canonical Codex docs URLs, and discovered two additional
docs pages (auth + models) that sonnet had missed. Reviewer
independently verified the documentation citations rather than
trusting sonnet quotes — Rule 2 (No Invention) is the load-bearing
check for D6 because no local binary exists to ground-truth the
plugin.
Reviewer non-blocking findings folded in this commit:
1. Stdin path corrected: docs explicitly state `Use - to pipe the
prompt from stdin`. The original draft assumed "no positional →
stdin"; docs require literal `-`. Fixed in irToCodex; test
"irToCodex: multiline prompt uses stdin path (useStdin=true) with
- positional" updated to assert args.includes('-').
2. Model registry expanded from 3 to 5 entries per canonical models
doc. Added gpt-5.4-mini and gpt-5.3-codex-spark. Removed the
misread Rule 2 comment that justified omitting -spark suffix —
the docs literally show `codex -m gpt-5.3-codex-spark`, so the
-spark variant is a separate model not a -codex normalization.
New aliases: codex-spark, gpt5-mini.
3. File header A2 upgraded from "assumed" to "CONFIRMED" with the
canonical auth doc URL cited.
4. File header now cites both auth and models canonical URLs at the
top, alongside reference and features.
Reviewer findings deferred to D7:
- OS credential store / keyring support. Codex docs mention
cli_auth_credentials_store = keyring as an alternative to file
storage. The Anthropic plugin supports macOS keychain via security
find-generic-password; Codex equivalent unknown without inspecting
a real install. D7 will install codex, run codex login, see what
keyring entry codex creates (if any), and mirror the Anthropic
keychain support pattern.
- Real NDJSON event schema (field names). Defensive 4-shape parser
handles the most common conventions; D7 captures real stdout and
pins the schema.
- access_token field name in auth.json. D7 captures the real auth
artifact and removes unused fallback names.
Test count: 128 (after D5) → 174 (after D6).
Verification:
node --check on all touched files: clean.
npm test on Node 25.8.0: 174/174 pass in 210ms with Suite 10 skipped.
Test "codex.models contains all 5 docs-listed model IDs" passes.
Test "irToCodex: multiline prompt uses stdin path with - positional"
passes (asserts args.includes('-')).
Hygiene grep: zero personal-name/path/token hits.
Co-Authored-By: Claude Opus 4.7 (noreply@anthropic.com)
|