mirror of
https://github.com/dtzp555-max/ocp.git
synced 2026-07-21 21:15:09 +00:00
docs: fold in adversarial review — correct the debug-log reasoning + the overhead number
Independent adversarial reviewer (tasked with REFUTING this doc) confirmed the central claim — no byte-faithful incremental source exists on the TUI path — but found four factual defects in the prose. A negative claim that will be cited for years has to be right in its reasoning, not just its conclusion. 1. --debug-file: the "written at end-of-turn" reasoning was WRONG. The default log level is `debug`, which suppresses every `verbose` site; the original probe therefore ran with the stream logging OFF. At CLAUDE_CODE_DEBUG_LOG_LEVEL=verbose there ARE 16 mid-turn `[shoji-engine] yield stream_event/-` lines spread over ~3.9s of generation. The conclusion survives because those lines carry TIMING ONLY, no text payload (content_block_delta / text_delta / content_block_start / message_start = 0 at any verbosity or category filter). Reasoning rewritten: "logs when tokens arrive, never what they are" — as written before, the doc was falsifiable in 30 seconds. 2. The 7.319s `turn_duration` is NOT a "native" (non-OCP) baseline: it comes from an OCP-driven turn (cwd .ocp-tui/work, same 7451-char prompt, same 204-char answer as pr1 baseline row i=5, elapsed 11563ms). Reframed as what it actually is — a SAME-TURN decomposition, 11.563s wall - 7.319s CLI-internal = ~4.2s OCP overhead (n=1), which is a cleaner comparison than the doc originally claimed. 3. Dropped the "~2-4s" range: its low end mixed an effort-HIGH turn_duration with the effort-LOW wall-clock median, which understates overhead (a low-effort turn generates faster, so its own turn_duration would be lower). No turn_duration sample exists for the effort-low config. Now stated as ~4s (n=1, baseline config), with both caveats. 4. Softened "ZERO marginal benefit" (backlog #4) to "no benefit detectable at n=5" — n=5 cannot prove zero — and added the mechanistic reason the reviewer supplied, which is far stronger than the empirical null: `--help` says the flag improves cross-user prompt-cache REUSE, and OCP is single-user, so there is no cross-user cache to share. Also folded in the reviewer's independent sweep, which closes the search space rather than sampling it: the hook registry was enumerated from the shipped binary (no per-chunk / streaming hook exists among the 21 events); `capture-pane -e` was tested and shown to be a provably non-unique inverse (an H2 and a bold span emit IDENTICAL SGR 1); and sessions/<pid>.json, history.jsonl, CLAUDE_CODE_INCLUDE_PARTIAL_MESSAGES (undocumented), sessionMirror, --sdk-url and --input-format stream-json were each checked and each dies (contentless, or gated behind --output-format stream-json -> --print -> the metered sdk-cli pool). Prompt-mutation (asking the model for plain text) is named and rejected on ALIGNMENT grounds so it is not re-litigated later. Docs-only. No code change, no version bump. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VqgWJcjxrjjL9L9SkpZyXR
This commit is contained in:
@@ -1033,7 +1033,7 @@ Then restart OCP. At boot you will see (with the env token set, isolated home au
|
||||
### What changes / what doesn't
|
||||
|
||||
- **Callers see no API change.** The response is a normal OpenAI completion object or chunked SSE — identical wire format.
|
||||
- **No real token streaming — and it cannot be added.** TUI-mode buffers the full response then replays it as chunked SSE. You will see a delay then the complete response rather than real-time tokens. This is **structural, not a missing feature**: interactive `claude` exposes no byte-faithful incremental output. Its three observables are a *rendered* terminal (the pane's text has markdown source markers — `## `, `**`, code fences — stripped by rendering, so it is not the model's text), a transcript written at *event* granularity (the whole answer lands in one line ~0.3 s before the turn ends), and a debug log also written at end-of-turn. The one interface that emits token deltas, `--output-format stream-json`, requires `-p` — the metered-billing path TUI mode exists to avoid. Evidence: [`docs/plans/2026-07-13-tui-latency/streaming-spike.md`](docs/plans/2026-07-13-tui-latency/streaming-spike.md).
|
||||
- **No real token streaming — and it cannot be added.** TUI-mode buffers the full response then replays it as chunked SSE. You will see a delay then the complete response rather than real-time tokens. This is **structural, not a missing feature**: interactive `claude` exposes no byte-faithful incremental output. Its three observables are a *rendered* terminal (the pane's text has markdown source markers — `## `, `**`, code fences — stripped by rendering, so it is not the model's text), a transcript written at *event* granularity (the whole answer lands in one line ~0.3 s before the turn ends), and a debug log that reports *when* tokens arrive but never *what* they are (its mid-turn events carry no text payload at any verbosity). The one interface that emits token deltas, `--output-format stream-json`, requires `-p` — the metered-billing path TUI mode exists to avoid. Evidence: [`docs/plans/2026-07-13-tui-latency/streaming-spike.md`](docs/plans/2026-07-13-tui-latency/streaming-spike.md).
|
||||
- **Cache and singleflight work normally.** TUI-mode writes the buffered response to the cache on success; cache-hits skip the interactive turn entirely.
|
||||
- **The host's `CLAUDE.md` / auto-memory is never injected.** OCP is a proxy — the proxied client (OpenClaw / your IDE) owns its own context and memory. TUI-mode always runs `claude` with `CLAUDE_CODE_DISABLE_CLAUDE_MDS` + `CLAUDE_CODE_DISABLE_AUTO_MEMORY`, so a `CLAUDE.md` on the OCP host can never leak into proxied turns (verified live; see #4). Built-in tool schemas + the interactive system prompt remain (the inherent ~20–35K context floor of interactive mode); MCP is hard-disabled.
|
||||
- **Authenticate via `CLAUDE_CODE_OAUTH_TOKEN` in a credential-isolated home (recommended).** tmux does not forward the parent process's env to the pane, so OCP sets the token explicitly on the spawned `claude` when `CLAUDE_CODE_OAUTH_TOKEN` is present. But passing the token is **not enough on its own**: interactive `claude` *prefers* `~/.claude/.credentials.json` over the env var (unlike the `-p` path), so a stale `credentials.json` would shadow the token. With the env token set and `OCP_TUI_HOME` unset, OCP therefore runs claude in a **credential-isolated home** (`$HOME/.ocp-tui/home`) that has **no `credentials.json`** — so the env token is the only credential and is authoritative, and claude never runs the token-refresh path (so the single-use refresh token can't be corrupted by the spawn/teardown cycle). On a long-running host the credentials.json path produced a permanent `Please run /login · API Error: 401` that re-login could not fix (the next spawn re-corrupted it); the isolated home ends that at the root. Transcripts land under the same isolated home, so the answer-reader is unaffected. Without the env token, claude falls back to the real home's `credentials.json` (byte-for-byte the previous behaviour). (The token is visible in `ps` on the pane command — acceptable for the single-user A-path; the multi-user B-path is refused at boot.) See ADR 0007 PR-C / PR-D amendments.
|
||||
@@ -1050,7 +1050,7 @@ path, stated plainly so you can rule it out before building on it:
|
||||
|---|---|
|
||||
| **TTFT floor (first token)** | **≈ 6 s** — immovable |
|
||||
| cold boot → input bar ready | ~1 s (per request; not the bottleneck) |
|
||||
| OCP's own overhead above the CLI | ~2–4 s |
|
||||
| OCP's own overhead above the CLI | ~4 s (n=1 same-turn decomposition) |
|
||||
| direct Anthropic API, same prompt (for scale) | 0.84–1.64 s |
|
||||
|
||||
The ~6 s floor is the `claude` CLI itself: it always injects the full Claude Code system prompt plus
|
||||
|
||||
Reference in New Issue
Block a user