taodeng/ocp
1
0
Fork 0
mirror of https://github.com/dtzp555-max/ocp.git synced 2026-07-21 21:15:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: REVERSE the streaming verdict — MessageDisplay hook makes it achievable

Browse Source 
The previous commit on this branch concluded TUI streaming was impossible. That was
WRONG, and this corrects it before it could be merged.

The adversarial reviewer commissioned to refute the claim found, on a second pass while
verifying the fold-in, that its OWN first-pass hook enumeration had been truncated by a
400-char grep cap: it reported 21 hook events; the shipped 2.1.207 bundle has 30.
Event #30 is MessageDisplay.

Independently reproduced before acting on it (30 events confirmed via `strings` on the
binary; payload shape `hook_event_name:"MessageDisplay",turn_id,message_id,index,final,
delta`), then live-tested with a MessageDisplay command hook registered via --settings on
a PLAIN INTERACTIVE TUI spawn (no -p, no --bare), claude-sonnet-4-6, --effort low:

  banner: "Sonnet 4.6 with low effort · Claude Max"   ← subscription pool, verified

  7 fires, mid-turn, spread across generation:
    index=0 final=false  '## Mutex\n\n'
    index=1 final=false  'A **mutual exclusion lock** prevents concurrent access to a shar…'
    index=4 final=false  'let counter = 0;\n\nasync function increment() {\n  const release =…'
    index=6 final=true   '```'

  concat(deltas) === T (transcript-authoritative)  ->  TRUE  (579 == 579 bytes)
  T.startsWith(S) at EVERY step                    ->  TRUE  (prefix-stable)
  '## ' / '**' / '```javascript' present in deltas ->  raw markdown SOURCE, not rendered

This satisfies every invariant the previous version declared unobtainable: byte-faithful,
incremental, prefix-stable, no -p, subscription pool. Granularity is block-level (~5-7
chunks/answer), not token-level — which is all an SSE delta.content needs.

Backlog #2 REOPENS and should be built. Implementer caveat recorded: the hook's source
sets forceSyncExecution -> claude BLOCKS on it, so the hook must write and exit
immediately (FIFO/socket), never work inline. Only text blocks fire it (thinking excluded).
ALIGNMENT: consumes claude's OWN hook surface as emitted — forwarding, not inventing
(Class B / ADR 0007; no cli.js citation applies).

Everything still true is kept, and the dead ends are kept as dead ends (they document what
NOT to build): the pane is a rendered view whose source markers are irrecoverable
(capture-pane -e emits IDENTICAL SGR 1 for an H2 and a bold span — a provably non-unique
inverse); the transcript is event-granular; --debug-file carries timing but no payload;
--output-format stream-json requires -p (the metered pool). Also kept: the ~4s (n=1
same-turn) overhead correction, backlog #4's null result with its mechanistic single-user
reason, and the honest value framing — streaming moves the FIRST byte, not the last, so
the complete-answer consumer that motivated this work gains nothing from it.

The wrong conclusion and its refutation are both preserved in the doc. "We checked, it's
impossible" is the most expensive claim to get wrong: it closes a door nobody re-opens.

Docs-only. No code change, no version bump.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VqgWJcjxrjjL9L9SkpZyXR
This commit is contained in:
taodeng
2026-07-13 16:16:12 +10:00
co-authored by Claude Fable 5
parent fc63b8a49a
commit ddaea4df17
4 changed files with 205 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+6 -4
View File
@@ -1033,7 +1033,7 @@ Then restart OCP. At boot you will see (with the env token set, isolated home au
### What changes / what doesn't
- **Callers see no API change.** The response is a normal OpenAI completion object or chunked SSE — identical wire format.
- **No real token streaming — and it cannot be added.** TUI-mode buffers the full response then replays it as chunked SSE. You will see a delay then the complete response rather than real-time tokens. This is **structural, not a missing feature**: interactive `claude` exposes no byte-faithful incremental output. Its three observables are a *rendered* terminal (the pane's text has markdown source markers — `## `, `**`, code fences — stripped by rendering, so it is not the model's text), a transcript written at *event* granularity (the whole answer lands in one line ~0.3 s before the turn ends), and a debug log that reports *when* tokens arrive but never *what* they are (its mid-turn events carry no text payload at any verbosity). The one interface that emits token deltas, `--output-format stream-json`, requires `-p` — the metered-billing path TUI mode exists to avoid. Evidence: [`docs/plans/2026-07-13-tui-latency/streaming-spike.md`](docs/plans/2026-07-13-tui-latency/streaming-spike.md).
- **No real token streaming *today* — but it is achievable, and planned.** TUI-mode currently buffers the full response then replays it as chunked SSE: you see a delay, then the complete response. This is a limitation of the current implementation, **not** of the path — `claude` fires a `MessageDisplay` hook carrying incremental, byte-faithful `delta`s of the raw reply (they concatenate exactly to the final text, and stay prefix-stable), on the subscription pool, without `-p`. Wiring it into OCP's SSE is tracked as backlog item #2. What is *not* available is token-by-token granularity (the hook fires per rendered block, ~5–7 chunks per answer) — which is plenty for SSE. Evidence: [`docs/plans/2026-07-13-tui-latency/streaming-spike.md`](docs/plans/2026-07-13-tui-latency/streaming-spike.md).
- **Cache and singleflight work normally.** TUI-mode writes the buffered response to the cache on success; cache-hits skip the interactive turn entirely.
- **The host's `CLAUDE.md` / auto-memory is never injected.** OCP is a proxy — the proxied client (OpenClaw / your IDE) owns its own context and memory. TUI-mode always runs `claude` with `CLAUDE_CODE_DISABLE_CLAUDE_MDS` + `CLAUDE_CODE_DISABLE_AUTO_MEMORY`, so a `CLAUDE.md` on the OCP host can never leak into proxied turns (verified live; see #4). Built-in tool schemas + the interactive system prompt remain (the inherent ~20–35K context floor of interactive mode); MCP is hard-disabled.
- **Authenticate via `CLAUDE_CODE_OAUTH_TOKEN` in a credential-isolated home (recommended).** tmux does not forward the parent process's env to the pane, so OCP sets the token explicitly on the spawned `claude` when `CLAUDE_CODE_OAUTH_TOKEN` is present. But passing the token is **not enough on its own**: interactive `claude` *prefers* `~/.claude/.credentials.json` over the env var (unlike the `-p` path), so a stale `credentials.json` would shadow the token. With the env token set and `OCP_TUI_HOME` unset, OCP therefore runs claude in a **credential-isolated home** (`$HOME/.ocp-tui/home`) that has **no `credentials.json`** — so the env token is the only credential and is authoritative, and claude never runs the token-refresh path (so the single-use refresh token can't be corrupted by the spawn/teardown cycle). On a long-running host the credentials.json path produced a permanent `Please run /login · API Error: 401` that re-login could not fix (the next spawn re-corrupted it); the isolated home ends that at the root. Transcripts land under the same isolated home, so the answer-reader is unaffected. Without the env token, claude falls back to the real home's `credentials.json` (byte-for-byte the previous behaviour). (The token is visible in `ps` on the pane command — acceptable for the single-user A-path; the multi-user B-path is refused at boot.) See ADR 0007 PR-C / PR-D amendments.
@@ -1059,9 +1059,11 @@ its tool definitions before your prompt, on every turn, no matter what you ask.
thinking is *not* the cause — `OCP_TUI_EFFORT` already defaults to `low`, which is what cuts a
formerly-inherited `xhigh` down to this floor and collapses its variance.
On top of the floor you pay the model's generation time (a function of output length), and there is
**no progressive output** — see "No real token streaming" above; it is structural and cannot be
fixed. So a turn returns as one blob after the full generation completes.
On top of the floor you pay the model's generation time (a function of output length). Progressive
output is not wired up **yet** (see "No real token streaming" above — it is achievable and planned),
so today a turn returns as one blob once generation completes. Note that streaming, when it lands,
will move the *first* byte earlier — it does **not** shorten the turn, and a consumer that needs the
complete answer gains nothing from it.
**Use TUI mode for**: batch, background, and latency-insensitive work where the subscription pool is
the point. **Do not use it for**: anything a person is waiting on interactively, or any consumer with
docs/plans/2026-07-13-tui-latency/README.md
+20 -15
View File
@@ -119,23 +119,28 @@ the effort level silently changes if the operator ever switches to env-token mod
- **Risk**: none — banner confirms it stays on `Claude Max` (see `billing-banner.txt`).
- ⚠️ Do **not** reach for `--bare` to shave boot: see above.
### 2. Real streaming instead of blocking on turn-terminal — ~~the big one (~20 s)~~ → **DEAD**
### 2. Real streaming instead of blocking on turn-terminal — **ACHIEVABLE → [`streaming-spike.md`](streaming-spike.md)**
> **2026-07-13 update — the prereq spike below was run, and it kills this item.** The transcript
> grows at *event* granularity (the whole answer lands in one line, ~0.3 s before terminal), and the
> pane is a **rendered** view whose `capture-pane` text no longer contains the answer's source bytes
> (`## `, `**`, code fences are gone) — so no pane-derived stream can be reconciled with the
> authoritative text. A third source (`--debug-file`) does emit mid-turn stream events at
> `CLAUDE_CODE_DEBUG_LOG_LEVEL=verbose` — but they carry **timing only, no text payload** (0
> `text_delta` at any verbosity); its only byte-exact text is the end-of-turn `Stop` hook payload.
> `--output-format stream-json`, the only interface that emits token deltas, requires `-p` — the
> metered-billing path that TUI mode exists to avoid. **True token streaming is not achievable on the
> TUI path**; OCP's TUI SSE is replay-only. Note also that streaming would never have shortened a
> turn — only its first byte — so a consumer that needs the *complete* answer (the JSON-card case
> that motivated this investigation) gains nothing from it.
> **2026-07-13 update — the prereq spike was run. The answer is YES, but not from either source this
> item guessed at.** (a) The transcript grows at *event* granularity (the whole answer lands in one
> line, ~0.3 s before terminal) — dead. (b) The pane is a **rendered** view whose `capture-pane` text
> no longer contains the answer's source bytes (`## `, `**`, code fences are gone) — dead, and worse
> than "lossy": it is *not the model's text*. **But there is a third source neither this backlog nor
> the first spike considered: `claude` fires a `MessageDisplay` hook carrying incremental,
> byte-faithful `delta`s of the raw reply.** Verified live on a plain interactive TUI spawn (no `-p`),
> banner `· Claude Max`: 7 fires spread across generation, `concat(deltas) === T` **byte-exactly**
> (579 == 579), `T.startsWith(S)` true at every step, `## ` / `**` / ```` ```javascript ```` all
> present in the deltas. Granularity is block-level (~5–7 chunks/answer), not token-level — plenty for
> SSE. **Build it.**
>
> Full evidence, the value re-assessment, and the maintainer's options: **[`streaming-spike.md`](streaming-spike.md)**.
> The original framing is preserved below for the record.
> ⚠️ Two corrections to this item as written: the **"~20 s" is wrong** (inferred from an external
> report, never measured through OCP — the same-turn decomposition puts OCP's own overhead at **~4 s**,
> n=1), and **streaming moves the first byte, not the last** — so a consumer needing the *complete*
> answer (the JSON-card case that motivated this) gains **nothing** from it. Build it for
> progressively-rendering consumers, not as a throughput win.
>
> Full evidence + implementer caveats (the hook is `forceSyncExecution` — claude BLOCKS on it):
> **[`streaming-spike.md`](streaming-spike.md)**. Original framing preserved below.
Today `runTuiTurn` blocks on the transcript until the turn is *finished*. The pane is already
rendering tokens incrementally the whole time — this harness proves you can observe first token
docs/plans/2026-07-13-tui-latency/messagedisplay-deltas.jsonl
+7
View File
@@ -0,0 +1,7 @@
{"hook_event_name": "MessageDisplay", "index": 0, "final": false, "delta": "## Mutex\n\n"}
{"hook_event_name": "MessageDisplay", "index": 1, "final": false, "delta": "A **mutual exclusion lock** prevents concurrent access to a shared resource, ensuring only one thread runs the critical section at a time.\n\n"}
{"hook_event_name": "MessageDisplay", "index": 2, "final": false, "delta": "- Acquiring a locked mutex blocks the caller until the current holder releases it.\n"}
{"hook_event_name": "MessageDisplay", "index": 3, "final": false, "delta": "- Failing to release a mutex causes a deadlock, freezing all waiting threads.\n\n```javascript\nconst { Mutex } = require('async-mutex');\n\nconst mutex = new Mutex();\n"}
{"hook_event_name": "MessageDisplay", "index": 4, "final": false, "delta": "let counter = 0;\n\nasync function increment() {\n const release = await mutex.acquire();\n try {\n"}
{"hook_event_name": "MessageDisplay", "index": 5, "final": false, "delta": " counter++; // only one caller here at a time\n } finally {\n release();\n }\n}\n"}
{"hook_event_name": "MessageDisplay", "index": 6, "final": true, "delta": "```"}
docs/plans/2026-07-13-tui-latency/streaming-spike.md
+172 -184
View File
@@ -1,249 +1,237 @@
# Backlog #2 (real streaming): the prereq spike says **no**
# Backlog #2 (real streaming): **achievable** — via the `MessageDisplay` hook
**Date**: 2026-07-13
**Status**: negative result — evidence, not a decision. The maintainer chooses what to do with it.
**Scope**: this answers the prereq spike that [`README.md`](README.md) § "Backlog #2" demanded *before* any streaming design:
**Status**: prereq-spike result. **Streaming IS achievable on the TUI path**, byte-faithfully, on the
subscription pool. Three obvious sources are dead ends; a fourth one works.
**Scope**: answers the prereq spike that [`README.md`](README.md) § "Backlog #2" demanded *before* any
streaming design:
> **Prereq spike (do this before designing anything)**: does the transcript JSONL grow *during* a
> turn, or only at the end? If only at the end, (a) is dead and you are stuck with (b).
The spike was run. **Both (a) and (b) are dead**, and so is a third candidate the original backlog
did not consider. The conclusion is stronger than "streaming is hard": **the interactive `claude`
CLI does not expose its token stream to any observer.** It renders tokens to a terminal and records
the *finished* text. There is no byte-faithful incremental source for a proxy to forward.
The answer: **(a) is dead, (b) is dead — and you are not stuck with either.** The CLI exposes its own
streaming interface as a **hook**, which the backlog did not consider.
**Measured on**: Mac mini / Claude Code **v2.1.207** / Sonnet 5 + Sonnet 4.6 / Claude Max /
**Measured on**: Mac mini / Claude Code **v2.1.207** / Sonnet 4.6 + Sonnet 5 / Claude Max /
real-home mode. Every claim below is reproducible from the commands given.
> **Honesty note on how this document was produced.** Its first version concluded the exact opposite —
> "streaming is not achievable; the CLI exposes no byte-faithful incremental source" — and was **wrong**.
> An adversarial reviewer, commissioned specifically to *refute* it, found `MessageDisplay` on a second
> pass; its own first pass had enumerated the hook registry with a truncated grep (it reported 21
> events — there are **30**). Both the wrong conclusion and its refutation are preserved here, because
> "we checked, it's impossible" is the most expensive kind of claim to get wrong: it closes a door and
> nobody re-opens it.
---
## The three candidate sources, and how each one dies
## ✅ The source that works: the `MessageDisplay` hook
`claude` fires a **`MessageDisplay`** hook as it renders each block of the assistant's reply. The
payload carries the **raw markdown source** of an incremental `delta`, plus a monotonic `index` and a
`final` flag:
```json
{ "hook_event_name": "MessageDisplay",
"turn_id": "6cb31d21-…", "message_id": "84ab9832-…",
"index": 0, "final": false, "delta": "## Mutex\n\n" }
```
*(payload also carries `session_id`, `transcript_path`, `prompt_id`, `cwd`)*
Registered as an ordinary command hook via `--settings` on a **plain interactive TUI spawn** (no `-p`,
no `--bare`), `claude-sonnet-4-6`, `--effort low`. Banner verified:
`▝▜█████▛▘ Sonnet 4.6 with low effort · Claude Max` — **subscription pool, not metered billing**.
One live turn — 7 fires, spread across generation:
```
index=0 final=false len= 10 '## Mutex\n\n'
index=1 final=false len= 140 'A **mutual exclusion lock** prevents concurrent access to a shar…'
index=2 final=false len= 83 '- Acquiring a locked mutex blocks the caller until the current h…'
index=3 final=false len= 163 '- Failing to release a mutex causes a deadlock, freezing all wai…'
index=4 final=false len= 96 'let counter = 0;\n\nasync function increment() {\n const release =…'
index=5 final=false len= 84 ' counter++; // only one caller here at a time\n } finally {\n …'
index=6 final=true len= 3 '```'
```
**Every invariant a proxy needs — all hold:**
| requirement | result |
|---|---|
| **byte-faithful** — deltas are the model's *source*, not the rendered pane | ✅ `## `, `**`, ```` ```javascript ```` all present in the deltas |
| **exactness** — `concat(deltas) === T` (the transcript-authoritative text) | ✅ **true**, 579 == 579 bytes |
| **prefix-stable** — `T.startsWith(concat(deltas[0..n]))` at every n | ✅ **true at all 7 steps** |
| **incremental** — arrives during generation, not at the end | ✅ 7 fires spread across the turn |
| **no `-p`** — stays out of the metered `sdk-cli` pool | ✅ plain interactive TUI |
| **subscription pool** | ✅ banner `· Claude Max` |
This is exactly the contract a streaming design needs: deltas forward straight into SSE
`delta.content` chunks, and the transcript's final text `T` stays a cheap end-of-turn assertion
(`concat === T`) instead of a reconciliation problem.
### Caveats for the implementer
- **Block-level granularity, not token-level** — 5–7 chunks for a ~600-byte answer, not one per token.
Plenty for SSE (`delta.content` has no minimum size), but do not promise token-by-token output.
- **⚠️ `forceSyncExecution: true` in the hook's source — `claude` BLOCKS on the hook.** A slow hook
adds latency to *every* delta. The hook must write and exit immediately (e.g. write to a FIFO / unix
socket that OCP reads; never work inline). **Measure the added per-delta latency.**
- Only `text` blocks fire it (`content.map(c => c.type === "text" ? c.text : "")`) — **thinking blocks
are excluded**, which is what OCP wants.
- OCP already owns the spawn (isolated HOME, its own flags), so injecting `--settings` with a
`MessageDisplay` hook sits inside the existing architecture.
- **`ALIGNMENT.md`**: this consumes `claude`'s **own** hook surface as emitted — forwarding, not
inventing. Not a new endpoint, not a fabricated protocol. (Class B / ADR 0007 — the TUI spawn is
OCP-owned; no `cli.js` citation applies.)
### Reproduce in 60 seconds
```bash
# hook script: append the payload (arrives on stdin) and exit immediately
printf '#!/bin/bash\ncat >> "$MD_LOG"; printf "\\n" >> "$MD_LOG"; exit 0\n' > /tmp/h.sh && chmod +x /tmp/h.sh
echo '{"hooks":{"MessageDisplay":[{"hooks":[{"type":"command","command":"MD_LOG=/tmp/deltas.jsonl /tmp/h.sh"}]}]}}' > /tmp/s.json
# plain interactive claude in tmux (prefix NOT ocp-tui-*, and never kill-server)
tmux new-session -d -s md-probe -x 220 -y 50 \
"claude --model claude-sonnet-4-6 --effort low --session-id $(uuidgen) --settings /tmp/s.json"
# …wait for '? for shortcuts', paste a markdown-producing prompt, press Enter…
jq -r '"\(.index) \(.final) \(.delta|@json)"' /tmp/deltas.jsonl # incremental raw-markdown deltas
# then assert: concat(deltas) == extractLatestAssistantText(<transcript>.jsonl)
```
---
## The three dead ends (still worth knowing — they say what NOT to build)
### (a) Incremental transcript reads — **dead: event granularity, not token granularity**
The transcript JSONL *does* grow during a turn, but it grows one **whole event at a time**, and the
assistant's text event is written as **one complete line** — it appears only ~0.3 s before the
terminal `turn_duration` event.
The transcript JSONL *does* grow during a turn, but one **whole event at a time**; the assistant's text
event is written as **one complete line**, appearing only ~0.3 s before the terminal `turn_duration`.
Observed (session `efd5b161`, `turn_duration: 7319 ms`, polling the file every 0.5 s):
Observed (session `efd5b161`, `turn_duration: 7319 ms`):
```
bytes=64347 lines=13 → 64479 lines=14 → 92192 lines=21 (during the turn)
#6 t+0.0s type=user (the prompt)
#15 t+4.7s type=assistant blocks=thinking
#16 t+7.0s type=assistant blocks=text ← the ENTIRE answer, in one line
#21 t+7.3s type=system subtype=turn_duration ← terminal
```
Reading the transcript incrementally therefore buys ~0.3 s, not a token stream. The "clean option"
the backlog hoped for does not exist.
Cross-checked at **20 ms polling + `fs.watch`** (25× finer): a partial line **never touches disk** —
one write, `+1` line, carrying the complete answer. Also forced with the undocumented
`CLAUDE_CODE_INCLUDE_PARTIAL_MESSAGES=1`: still 1 assistant event, 0 partials (interactive mode has no
stream-json *sink* for it to write to).
**The transcript is still needed** — as the terminal-turn signal, as the authoritative `concat === T`
check, and as the input to the existing honesty gates (auth-banner detection, `truncated`). It is just
not the *streaming* source.
### (b) `tmux capture-pane` diffing — **dead: the pane is a RENDERED view, not the text**
This is the one the backlog expected to fall back to, noting it is "lossy for exact text (wrapping,
scrollback, spinner lines)". The loss is **worse than formatting noise: the pane does not contain
the answer's source bytes at all.** claude's TUI *renders* markdown, and `capture-pane -p` strips
the ANSI styling that rendering produced — so the source markers are gone, irrecoverably.
The backlog expected to fall back to this, calling it "lossy … (wrapping, scrollback, spinner lines)".
The loss is far worse than formatting noise: **the pane does not contain the answer's source bytes at
all.** The TUI *renders* markdown, and `capture-pane -p` strips the ANSI that rendering produced.
Same turn, same lines — authoritative transcript text vs. what the pane holds
(`capture-pane -p -J -S -500`, pane width 220):
Same turn, same lines:
```
TRANSCRIPT (authoritative T, via extractLatestAssistantText):
'## Semaphore'
''
'A **semaphore** is a synchronization primitive that controls access to a shared resource by …'
''
'- Tracks available "permits" and decrements the count when a thread acquires access, …'
PANE (the same turn):
''
'⏺ Semaphore' ← '## ' heading became a bullet glyph
''
' A semaphore is a synchronization primitive that controls access to a shared resource by …'
'' ← '**' bold markers GONE; every line indented 2 sp
' - Tracks available "permits" and decrements the count when a thread acquires access, …'
TRANSCRIPT (authoritative T): PANE (capture-pane -p -J -S -500):
'## Semaphore' '⏺ Semaphore' ← heading marker gone
'' ''
'A **semaphore** is a synchro…' ' A semaphore is a synchro…' ← bold markers gone, indented
```
Token-presence check **over the pane's answer region** (the prompt echo above it is excluded — it
does contain a literal `**`, because the prompt itself asked for bold):
| token in the answer | in transcript `T` | in pane's answer region |
| token in the answer | in `T` | in the pane's answer region |
|---|---|---|
| `## ` (ATX heading) | yes | **no** — rendered as `⏺` |
| `**` (bold markers) | yes | **no** — rendered as ANSI bold, then stripped by `-p` |
| ` ```javascript ` (fence + language) | yes | **no** — fence gone, language tag gone |
| `**` (bold markers) | yes | **no** — rendered to ANSI bold, then stripped by `-p` |
| ` ```javascript ` (fence + language) | yes | **no** — fence and language tag both gone |
| `- ` (list item) | yes | yes |
**`capture-pane -e` (keep the ANSI) does not rescue it — the inverse is provably non-unique.**
Minimal case, transcript `T` = ``"## Alpha\n\n**bravo**\n\n```javascript\nlet x=1;\n```"``:
*(A literal `**` does appear elsewhere in the pane — in the **prompt echo**, because the prompt asked
for bold. Not in the answer.)*
**`capture-pane -e` (keeping the ANSI) does not rescue it — the inverse is provably non-unique.**
With `T` = ``"## Alpha\n\n**bravo**\n\n```javascript\nlet x=1;\n```"``:
```
⏺\e[39m \e[1mAlpha\n\n\e[0m \e[1mbravo\n\n\e[0m \e[34mlet\e[39m x=\e[32m1\e[39m;
```
`## Alpha` → **SGR 1 (bold)**. `**bravo**` → **SGR 1 (bold)**. *Identical ANSI* — an H2 and a bold
span are indistinguishable, never mind `**` vs `__`. The fence and its `javascript` tag are consumed
by the syntax highlighter into colours (`\e[34mlet`); recovering the tag would mean inverting a
highlighter, and `let x=1;` is valid in several languages. Marker check on the `-e` capture: `## `,
`**`, ```` ``` ````, ```` ```javascript ```` — **all absent**.
`## Alpha` → **SGR 1 (bold)**. `**bravo**` → **SGR 1 (bold)**. *Identical ANSI* — an H2 and a bold span
are indistinguishable, never mind `**` vs `__`. The fence and its `javascript` tag are consumed by the
syntax highlighter into colours; recovering the tag would mean inverting a highlighter, and
`let x=1;` is valid in several languages.
Consequence for any design built on pane diffing: with `S` = text streamed from the pane and `T` =
the transcript's authoritative text, the prefix invariant **`T.startsWith(S)` is false** — both raw
and with the 2-space indent stripped. Not "sometimes, on redraw" — **on essentially every answer
containing markdown**, which is most answers.
And it cannot be repaired:
- `capture-pane -e` (keep ANSI) recovers *styling*, never the *source spelling*: you cannot tell
`**bold**` from `__bold__`, recover a fence's language tag, or distinguish an H2 from bold text.
- Reconstructing markdown from rendered output is a lossy inverse with no unique solution, and it
would be pinned to one claude TUI version's theme.
A proxy that streams pane text is streaming **something the model did not say**. For OCP that is not
a quality trade-off, it is a correctness violation (`ALIGNMENT.md` Core Principle: forward what
`cli.js` emits; do not invent).
So `T.startsWith(paneText)` is **false** — raw and indent-stripped, on essentially every markdown
answer. A proxy streaming pane text would be streaming **something the model did not say**. With
`MessageDisplay` available there is no reason to go near it.
### (c) `--debug-file` — **dead: it logs stream *timing*, never stream *content***
Not considered in the original backlog; checked because it is the only other observable claude writes.
Worth stating precisely, because a casual check misleads in **both** directions here.
**There ARE mid-turn stream events in the debug log** — this is the one place a casual check
misleads, so it is worth stating precisely. The default log level is `debug`, which **suppresses
every `verbose` site**. Raise it and per-chunk lines appear, spread across the generation:
The default log level is `debug`, which **suppresses every `verbose` site**. Raise it and per-chunk
lines *do* appear, spread across generation:
```bash
CLAUDE_CODE_DEBUG_LOG_LEVEL=verbose claude --debug-file /tmp/d.log …
```
```
05:51:11.088 [VERBOSE] [shoji-engine] yield stream_event/- ← 16 of these, mid-turn,
05:51:11.537 [VERBOSE] [shoji-engine] yield stream_event/- spread over ~3.9 s of generation
…
05:51:14.910 [VERBOSE] [shoji-engine] yield assistant/-
05:51:11.537 [VERBOSE] [shoji-engine] yield stream_event/- over ~3.9 s of generation
05:51:15.192 [DEBUG] [shoji-engine] turn 1 end (usage in=575 out=255 api=6736ms stop=end_turn resultLen=857)
```
**But they carry no payload.** The format is `yield <type>/<subtype>` — a bare presence marker. At
verbose level, with every category filter, the log contains:
**But they carry no payload** — the format is `yield <type>/<subtype>`, a bare presence marker. Run with
no category filter (i.e. all categories) at verbose level: `content_block_delta` = **0**, `text_delta` =
**0**, `content_block_start` / `message_start` = **0**. The only byte-exact text in the log is the
end-of-turn `Stop` hook payload (`"last_assistant_message":"## Title\n\n**alpha bravo charlie**"`) —
transcript granularity. The log tells you **when** tokens arrive, never **what** they are. It is also
~2.7 MB per turn.
| event | count |
### Also checked, also not the answer
| candidate | outcome |
|---|---|
| `content_block_delta` | **0** |
| `text_delta` | **0** |
| `content_block_start` / `message_start` | **0** |
The only byte-exact text in the log is the **end-of-turn `Stop` hook payload**
(`"last_assistant_message":"## Title\n\n**alpha bravo charlie**"` — note `## ` and `**` survive, so
this *is* the real text), which is the same granularity as the transcript. So the debug log gives you
**when** tokens arrive, never **what** they are. It is also ~2.7 MB per turn.
*(An incremental "a token arrived" signal with no token in it cannot feed an SSE `delta.content`.)*
### Everything else that could plausibly carry tokens — also checked, also dead
An adversarial second pass (independent reviewer, tasked with *refuting* this document) enumerated
the rest of the search space rather than sampling it. Nothing survived:
| candidate | how it dies |
|---|---|
| **Hooks** — is there a per-message/per-chunk hook? | The hook registry was enumerated from the shipped binary: `PreToolUse, PostToolUse, PostToolUseFailure, PostToolBatch, Notification, UserPromptSubmit, UserPromptExpansion, SessionStart, SessionEnd, Stop, StopFailure, SubagentStart, SubagentStop, PreCompact, PostCompact, PermissionRequest, PermissionDenied, Setup, TeammateIdle, TaskCreated, TaskCompleted`. **No streaming/per-chunk hook exists.** |
| `CLAUDE_CODE_INCLUDE_PARTIAL_MESSAGES=1` (undocumented env twin of the flag; bypasses flag validation) | Ran live in a TUI turn: transcript still had **1** assistant event, **0** partials. Interactive mode has no stream-json *sink* for it to write to. Banner stayed `· Claude Max`. |
| `--output-format stream-json` (the one interface that emits `text_delta`) | **requires `--print`/`-p`** → `cc_entrypoint=sdk-cli` → the **metered** credit pool, which is exactly what TUI mode exists to avoid. Reproduced live. |
| `--input-format stream-json` | `Error: --input-format=stream-json requires output-format=stream-json` → same gate. |
| `CLAUDE_CODE_INCLUDE_PARTIAL_MESSAGES=1` (undocumented) | No stream-json sink in interactive mode → no partials. Banner stayed `· Claude Max`. |
| `sessionMirror` (undocumented) | Gated on `outputFormat === "stream-json"` → the `-p` family. |
| `--sdk-url` (hidden) | Forces stream-json + non-interactive → `sdk-cli` (metered pool). |
| `--input-format stream-json` | Live: `Error: --input-format=stream-json requires output-format=stream-json` → which requires `--print`. And it is an *input* format regardless. |
| `~/.claude/sessions/<pid>.json` (modified mid-turn) | Registry metadata only: `{pid, sessionId, cwd, status, version, entrypoint:"cli", kind:"interactive"}`. **No assistant text.** (Its `entrypoint:"cli"` is incidental independent confirmation that the TUI path stays on the subscription pool.) |
| `~/.claude/history.jsonl` (modified mid-turn) | Keys: `[display, pastedContents, timestamp, project, sessionId]` — **user prompts only**; the answer text is absent. |
| `~/.claude/settings.json` | No key for partial/streaming output. |
| **Ask the model to emit plain text** (so the pane render is faithful) | Would require OCP to **mutate the caller's prompt** — a correctness violation for a proxy (`ALIGNMENT.md`: forward what `cli.js` emits; do not invent), it would corrupt callers who legitimately want markdown or JSON, and it *still* would not be byte-faithful (the TUI keeps wrapping and 2-space indenting). Rejected. |
| `--sdk-url` (hidden) | Forces stream-json + non-interactive → `sdk-cli`. *(inferred from the minified bundle; not banner-tested)* |
| `~/.claude/sessions/<pid>.json` | Registry metadata only (`{pid, sessionId, cwd, status, version, entrypoint:"cli", kind:"interactive"}`). No assistant text. *(Its `entrypoint:"cli"` incidentally confirms the TUI path stays on the subscription pool.)* |
| `~/.claude/history.jsonl` | User prompts only; the answer text is absent. |
| Asking the model to emit plain text (so the pane renders faithfully) | Would mean **mutating the caller's prompt** — a correctness violation for a proxy, and still not byte-faithful (wrapping + indent remain). Rejected. |
---
## What this means
## Value: what streaming actually buys (read before building)
**True token streaming is not achievable on the TUI path**, at any effort, with any flag combination
available in claude 2.1.207. The three observables are: a rendered terminal (lossy beyond
repair), a transcript written at event granularity (the answer lands whole, end-of-turn), and a debug
log that reports *when* tokens arrive but never *what* they are.
`--output-format stream-json` — the one interface that *does* emit `text_delta` events — works
**only with `--print`/`-p`**, and the `-p` path is precisely what TUI mode exists to avoid (it is
classified `cc_entrypoint=sdk-cli` → the metered credit pool, not the subscription pool). The
constraint is structural, not a missing feature.
Streaming is *possible*. Whether it is *worth it* depends on the consumer, and the honest answer is
uncomfortable:
**Therefore OCP's SSE on the TUI path is, and remains, replay-only**: the answer is buffered until
the turn is terminal and then replayed as SSE chunks (`server.mjs`, the `streamStringAsSSE` branch).
It is wire-valid OpenAI SSE; it is not progressive.
- **Streaming never makes the answer arrive sooner. It moves the *first* byte, not the *last*.** The
final token lands at the same wall-clock moment either way.
- So a consumer that must have the **complete** answer before it can act — e.g. one parsing a structured
JSON reply, **which is exactly the 知音 AI use case that motivated this entire investigation** — gains
**nothing at all**. Only a **progressively-rendering** consumer (a chat UI) gains.
### What streaming would have bought — smaller than the backlog implied
And the number the backlog attached to this item was wrong:
Worth stating plainly, because it changes the value calculus:
- The backlog's "~20 s" was inferred from an external 30–32 s report, **never measured through OCP**.
Measured through a real OCP instance (TUI mode, `claude-sonnet-4-6`, ~1850-token prompt, n=5):
**median 11.30 s** before [#156](https://github.com/dtzp555-max/ocp/pull/156), **9.55 s** after.
- **Same-turn decomposition** (baseline row `i=5`): **11.563 s** wall through OCP vs `turn_duration:
7.319 s` of CLI-internal time on that same turn → **OCP's own overhead ≈ 4.2 s** (n=1, baseline
`effort=high` config). *Caveats*: n=1; and `turn_duration` is the CLI's internal duration of an
**OCP-driven** turn, not a separate "native" baseline. Do **not** subtract this `effort=high` 7.3 s
from the `effort=low` 9.55 s median — a low-effort turn generates faster, so mixing them
*understates* the overhead.
- So OCP's own overhead is **single-digit seconds**, not ~20 s. The rest of any large number is the
model generating a long answer — which streaming hides but does not shorten.
- **Streaming never makes the answer arrive sooner.** It makes the *first byte* arrive sooner. A
consumer that needs the *complete* answer before it can act — e.g. a JSON-card consumer parsing a
structured reply, which is exactly the 知音 AI use case that motivated this whole investigation —
gains **nothing at all** from streaming. Only a progressively-rendering consumer (a chat UI) gains.
- The backlog's "~20 s" for item #2 was inferred from an external report of 30–32 s, not measured
through OCP. Measured through a real OCP instance (TUI mode, `claude-sonnet-4-6`, ~1850-token
prompt, n=5): **median 11.30 s** before PR #156, **9.55 s** after.
**Same-turn decomposition** (the honest form — one turn, both numbers): baseline row `i=5` took
**11.563 s** end-to-end through OCP, and that same turn's transcript records
`turn_duration: 7.319 s` of CLI-internal time. → **OCP's own overhead ≈ 4.2 s** (n=1, baseline
`effort=high` config). Not 20 s.
Two caveats, stated so the number is not over-trusted: this is **n=1**, and `turn_duration` is the
*CLI's* internal duration on an **OCP-driven** turn (it is not a "native, non-OCP" baseline — the
same interactive `claude` is doing the work either way). Do **not** subtract this 7.3 s
(`effort=high`) from the 9.55 s `effort=low` median: a low-effort turn generates faster, so its own
`turn_duration` would be lower, and mixing the two would *understate* the overhead. There is no
`turn_duration` sample for the `effort=low` config.
Either way the direction is settled: OCP's overhead is **single-digit seconds**, not the ~20 s the
plan assumed. The rest of any large number is the model generating a long answer — which streaming
hides but does not shorten. (A 30 s turn is a 30 s turn either way; the last token lands at the same
wall-clock moment.)
The honest remaining levers on the TUI path are therefore: the **~6 s TTFT floor** (immovable — see
[`README.md`](README.md)), the **generation time** (a function of output length — not OCP's to
optimize), and OCP's **own ~4 s of overhead** (n=1 same-turn decomposition; per-request cold boot ≈1 s → backlog #3 warm pool;
plus readiness/paste/transcript poll granularity).
---
## Options for the maintainer (this document does not choose)
1. **Accept and document (recommended).** TUI mode cannot stream; its SSE is replay-only. Record the
constraint next to the ~6 s TTFT floor already in `README.md`, and close backlog #2 as *not
achievable honestly*. Redirect the effort to the overhead that *is* recoverable (#3).
2. **Ship lossy "rendered-text" streaming behind an explicit opt-in.** Technically possible; the
client would receive the TUI's rendered view — headings as `⏺`, bold markers stripped, code
fences gone, every line indented. **Recommended against**: it delivers bytes the model did not
produce, silently corrupts structured replies (the JSON/marker protocols proxy consumers actually
use), and pins OCP's output to a claude TUI theme. It also cannot be reconciled with the
transcript afterwards — OpenAI SSE deltas cannot be un-sent.
3. **Reconstruct markdown from `capture-pane -e` ANSI styling.** High effort, brittle across claude
versions, and *still* not byte-exact (no unique inverse). Recommended against.
## Reproduction
```bash
# (a) transcript growth granularity — poll a live turn's JSONL
# watch bytes/lines while a TUI turn runs; dump event types + timestamps afterwards
# (b) pane vs transcript — the decisive one
# spawn claude in tmux (prefix NOT ocp-tui-*, never kill-server), ask for markdown,
# capture `tmux capture-pane -p -J -S -500` frames during the turn, then diff the pane's
# answer region against extractLatestAssistantText() over the session's transcript JSONL.
# Expect: T.startsWith(paneText) === false, '## ' and '**' and '```' absent from the pane.
# (c) debug log — NOTE: default level is `debug`, which SUPPRESSES the verbose stream sites.
# Raise it, or you will wrongly conclude there are no mid-turn events at all.
CLAUDE_CODE_DEBUG_LOG_LEVEL=verbose \
claude --model claude-sonnet-5 --session-id "$(uuidgen)" --effort low --debug-file /tmp/d.log
grep -c 'yield stream_event' /tmp/d.log # → 16 (mid-turn — timing exists!)
grep -cE 'content_block_delta|text_delta' /tmp/d.log # → 0 (…but no text payload, ever)
grep -o 'last_assistant_message":"[^"]*' /tmp/d.log # → byte-exact text, only at Stop-hook time
```
Banner check (mandatory for every spawn-flag change, per [`billing-banner.txt`](billing-banner.txt)):
every configuration used in this spike was verified to stay on `· Claude Max`.
**Recommendation**: build it — the contract is clean and the cost is small — but size the expectation
honestly. It is a *perceived-latency* feature for progressively-rendering consumers, not a throughput
win, and it does not move the **~6 s TTFT floor** ([`README.md`](README.md)) that rules TUI mode out for
interactive-latency consumers regardless.