taodeng/olp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/olp.git synced 2026-07-19 09:45:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

test: D29 — eliminate port-collision flake class (16 sites, listen(0))

Browse Source 
cold-audit catch from 2026-05-24 (round 3 — D27 flake follow-up)

Pre-existing fragility from D18-era test design: Suite 17 + several
other test sites used overlapping random port ranges (e.g., 25456-25855
and 25460-25859 in Suite 17) with `Math.floor(Math.random() * 400)`
allocation. Sequential close-then-open on Linux Node 20 with TIME_WAIT
timing caused intermittent ECONNRESET on subsequent fetches.

Concrete failure: D27 first CI run failed Suite 17 test 17b with
`socket hang up / ECONNRESET` 1.3ms into the fetch. CI re-run on the
same commit was green — confirming flake, not logic bug. D27's code
changes were unrelated; the flake-prone pattern just happened to
collide on that run.

Fix: switch from random-port-in-fixed-range to OS-assigned port via
`listen(0, ...)`. Each test gets a unique OS-assigned port; collision
is structurally impossible.

Pattern:
```js
const s = createXX();
await new Promise((resolve, reject) => {
  s.listen(0, '127.0.0.1', resolve);
  s.once('error', reject);
});
const p = s.address().port;
// ... use p in fetch
```

Changes (test-features.mjs +32 / -32):

16 sites updated, organized by Suite:
- Suite 14: 14c (no providers), 14d (anthropic enabled)
- Suite 15 `before()`: shared `port15` for 15a-15d (assigned post-listen
  in before-hook)
- Suite 17 (D18-era): 17a, 17b, 17c, 17d (4 /v1/models tests) +
  17e, 17f, 17g (3 error-header tests)
- F19 `before()`: shared `portF19` (assigned post-listen in before-hook)
- D27 F15 a/b/c/d/e (5 alias-surfacing tests)

Out-of-scope sites intentionally LEFT alone (test-features.mjs lines
1253, 1882, 2048, 2212, 3048, 4382): they use wider random ranges
(500-1000) AND already have EADDRINUSE retry logic, so collision risk
is structurally bounded. Not the same defect class.

Stability verification:
- 5 consecutive local `npm test` runs (sonnet) — all 400/400 green
- 3 additional consecutive runs (reviewer) — all 400/400 green
- Durations 596-620ms; no Suite 17/15/F19 flakes observed

Pure mechanical refactor:
- No assertion changes (`grep -cE "^[+-].*assert\." diff` returns 0)
- No production code modified — only test-features.mjs
- No behavior contract change — tests assert the same things

Reviewer (Iron Rule v1.6 § 10.x Mode A, fresh-context opus, independent
of drafter): APPROVE. Verified each of 16 sites uses correct order
(listen before address().port read), Suite 15 + F19 before-block
variables correctly assigned AFTER listen-promise resolves, out-of-scope
sites unchanged, hygiene clean. 3/3 local runs green.

Note for future test additions: prefer `listen(0, '127.0.0.1', resolve)
→ s.address().port` over any fixed-range random port allocation. The
OS-assigned approach makes port collision structurally impossible.

Authority:
- Node.js HTTP server documentation:
  https://nodejs.org/api/net.html#serverlisten — port 0 OS assignment
- D27 CI failure on Suite 17 test 17b (workflow run 26353907900,
  Tests step — first run failed, re-run green)
- CC 开发铁律 v1.6 § 10.x — Round-3 Cold Audit pattern continued:
  D27's flake was a known evidence-first checkpoint violation
  (per ~/.cc-rules/memory/feedback/evidence_first_under_speed_pressure.md
  "host-environment trap" corollary). D29 closes the underlying
  fragility class so the same flake can't reoccur on subsequent D-days.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
taodeng
2026-05-24 16:48:16 +10:00
co-authored by Claude Opus 4.7
parent 4a238c9cc4
commit de9f3ca7c9
1 changed files with 32 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test-features.mjs
+32 -32
View File
@@ -5310,11 +5310,11 @@ describe('providers.enabled config wiring (Suite 14)', () => {
const { createOlpServer: createServer14 } = await import('./server.mjs'); const { createOlpServer: createServer14 } = await import('./server.mjs');
__setProvidersEnabled({}); __setProvidersEnabled({});
const s = createServer14(); const s = createServer14();
const p = 22456 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ const r = await fetch({
port: p, port: p,
@@ -5358,11 +5358,11 @@ describe('providers.enabled config wiring (Suite 14)', () => {
const { createOlpServer: createServer14d, __clearCache: clearCache14d } = await import('./server.mjs'); const { createOlpServer: createServer14d, __clearCache: clearCache14d } = await import('./server.mjs');
clearCache14d(); clearCache14d();
const s = createServer14d(); const s = createServer14d();
const p = 22860 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ const r = await fetch({
@@ -5407,11 +5407,11 @@ describe('Streaming cache-miss real-time (Suite 15)', () => {
const { createOlpServer: s15, __clearCache: cc15 } = await import('./server.mjs'); const { createOlpServer: s15, __clearCache: cc15 } = await import('./server.mjs');
cc15(); cc15();
server15 = s15(); server15 = s15();
port15 = 23456 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
server15.listen(port15, '127.0.0.1', resolve); server15.listen(0, '127.0.0.1', resolve);
server15.once('error', reject); server15.once('error', reject);
}); });
port15 = server15.address().port;
}); });
after(async () => { after(async () => {
@@ -5844,11 +5844,11 @@ describe('/v1/models population + X-OLP-* error headers (Suite 17)', () => {
it('17a: /v1/models with anthropic enabled → 200 + 7 entries (3 canonical + 4 aliases) with owned_by="anthropic"', async () => { it('17a: /v1/models with anthropic enabled → 200 + 7 entries (3 canonical + 4 aliases) with owned_by="anthropic"', async () => {
setProviders17({ anthropic: true }); setProviders17({ anthropic: true });
const s = createServer17(); const s = createServer17();
const p = 25456 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -5871,11 +5871,11 @@ describe('/v1/models population + X-OLP-* error headers (Suite 17)', () => {
it('17b: /v1/models with no providers enabled → 200 + data:[]', async () => { it('17b: /v1/models with no providers enabled → 200 + data:[]', async () => {
setProviders17({}); setProviders17({});
const s = createServer17(); const s = createServer17();
const p = 25460 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -5894,11 +5894,11 @@ describe('/v1/models population + X-OLP-* error headers (Suite 17)', () => {
it('17c: /v1/models returns canonical IDs and alias IDs for loaded providers', async () => { it('17c: /v1/models returns canonical IDs and alias IDs for loaded providers', async () => {
setProviders17({ anthropic: true }); setProviders17({ anthropic: true });
const s = createServer17(); const s = createServer17();
const p = 25464 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -5928,11 +5928,11 @@ describe('/v1/models population + X-OLP-* error headers (Suite 17)', () => {
it('17d: /v1/models entries match OpenAI spec shape (id/object/created/owned_by)', async () => { it('17d: /v1/models entries match OpenAI spec shape (id/object/created/owned_by)', async () => {
setProviders17({ anthropic: true }); setProviders17({ anthropic: true });
const s = createServer17(); const s = createServer17();
const p = 25468 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -6012,11 +6012,11 @@ describe('/v1/models population + X-OLP-* error headers (Suite 17)', () => {
codexSetSpawnImpl(makeFailSpawn17()); codexSetSpawnImpl(makeFailSpawn17());
const s = createServer17(); const s = createServer17();
const p = 25472 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ const r = await fetch({
@@ -6105,11 +6105,11 @@ describe('/v1/models population + X-OLP-* error headers (Suite 17)', () => {
codexSetSpawnImpl(makeFailSpawn17f()); codexSetSpawnImpl(makeFailSpawn17f());
const s = createServer17(); const s = createServer17();
const p = 25876 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ const r = await fetch({
@@ -6154,11 +6154,11 @@ describe('/v1/models population + X-OLP-* error headers (Suite 17)', () => {
// is the minimum viable observability for those paths. // is the minimum viable observability for those paths.
setProviders17({}); setProviders17({});
const s = createServer17(); const s = createServer17();
const p = 25880 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const result = await new Promise((resolve, reject) => { const result = await new Promise((resolve, reject) => {
const req = httpRequest({ const req = httpRequest({
@@ -6758,11 +6758,11 @@ describe('D26 F19 — streaming truncation marker on stop-less exhaustion', () =
const { createOlpServer: s19, __clearCache: cc19 } = await import('./server.mjs'); const { createOlpServer: s19, __clearCache: cc19 } = await import('./server.mjs');
cc19(); cc19();
serverF19 = s19(); serverF19 = s19();
portF19 = 29456 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
serverF19.listen(portF19, '127.0.0.1', resolve); serverF19.listen(0, '127.0.0.1', resolve);
serverF19.once('error', reject); serverF19.once('error', reject);
}); });
portF19 = serverF19.address().port;
}); });
after(async () => { after(async () => {
@@ -7078,11 +7078,11 @@ describe('D27 F15 — /v1/models alias surfacing', () => {
it('F15a: /v1/models with anthropic enabled contains all canonical IDs and all 4 anthropic aliases', async () => { it('F15a: /v1/models with anthropic enabled contains all canonical IDs and all 4 anthropic aliases', async () => {
setProviders27({ anthropic: true }); setProviders27({ anthropic: true });
const s = createServer27(); const s = createServer27();
const p = 27100 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -7105,11 +7105,11 @@ describe('D27 F15 — /v1/models alias surfacing', () => {
it('F15b: each alias entry has owned_by equal to its canonical target provider', async () => { it('F15b: each alias entry has owned_by equal to its canonical target provider', async () => {
setProviders27({ anthropic: true }); setProviders27({ anthropic: true });
const s = createServer27(); const s = createServer27();
const p = 27200 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -7131,11 +7131,11 @@ describe('D27 F15 — /v1/models alias surfacing', () => {
it('F15c: /v1/models with no providers enabled returns empty data (no aliases)', async () => { it('F15c: /v1/models with no providers enabled returns empty data (no aliases)', async () => {
setProviders27({}); setProviders27({});
const s = createServer27(); const s = createServer27();
const p = 27300 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -7150,11 +7150,11 @@ describe('D27 F15 — /v1/models alias surfacing', () => {
it('F15d: /v1/models with anthropic+mistral enabled contains both providers\' canonicals and aliases', async () => { it('F15d: /v1/models with anthropic+mistral enabled contains both providers\' canonicals and aliases', async () => {
setProviders27({ anthropic: true, mistral: true }); setProviders27({ anthropic: true, mistral: true });
const s = createServer27(); const s = createServer27();
const p = 27400 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);
@@ -7180,11 +7180,11 @@ describe('D27 F15 — /v1/models alias surfacing', () => {
// Only anthropic enabled — codex aliases (codex, codex-spark, gpt5, gpt5-mini) must NOT appear // Only anthropic enabled — codex aliases (codex, codex-spark, gpt5, gpt5-mini) must NOT appear
setProviders27({ anthropic: true }); setProviders27({ anthropic: true });
const s = createServer27(); const s = createServer27();
const p = 27500 + Math.floor(Math.random() * 400);
await new Promise((resolve, reject) => { await new Promise((resolve, reject) => {
s.listen(p, '127.0.0.1', resolve); s.listen(0, '127.0.0.1', resolve);
s.once('error', reject); s.once('error', reject);
}); });
const p = s.address().port;
try { try {
const r = await fetch({ port: p, method: 'GET', path: '/v1/models' }); const r = await fetch({ port: p, method: 'GET', path: '/v1/models' });
assert.equal(r.status, 200); assert.equal(r.status, 200);