ocp/docs/runbooks/615-canary.md

2026-06-15 Canary Runbook

Purpose: Confirm that a TUI-mode turn is billed to the Pro/Max subscription pool (not the Agent SDK credit pool) after Anthropic's 2026-06-15 billing split activates.

The billing classifier reading cli is necessary but NOT sufficient proof. (Note the naming: the value is stored in the JSONL transcript under the field name entrypoint, and sent to Anthropic on the wire as the cc_entrypoint header — they carry the same value after claude's startup classification. The commands below grep the transcript, so they match entrypoint.) A cli label tells you OCP sent the right classification; it does not tell you Anthropic billed the right pool. The only authoritative test is to observe whether the Agent SDK credit balance moves or not before and after the canary turn.

---

Prerequisites

• CLAUDE_TUI_MODE=true already set and OCP restarted (see TUI-mode setup)
• tmux installed on the host
• No other OCP traffic during the canary (quiesce — see below)
• Access to your Anthropic account billing page (manual step — see below)

---

Step 1 — Quiesce the host

Stop any IDE or client that is actively sending requests through this OCP instance.

Confirm the proxy is idle:

curl -s http://127.0.0.1:3456/health | python3 -m json.tool | grep activeRequests
# Expected: "activeRequests": 0

Wait until activeRequests is 0 before proceeding. If you cannot quiesce (e.g. family members are actively using it), run the canary on a separate OCP instance or during a quiet window.

---

Step 2 — Read the Agent SDK credit balance BEFORE the canary

Manual step — no programmatic API available.

OCP's /usage endpoint reads anthropic-ratelimit-unified-* response headers from the Pro/Max plan quota (5-hour and 7-day subscription windows). These headers report subscription usage, not the Agent SDK credit pool balance. There is no known programmatic API to query the Agent SDK credit pool balance from outside the Anthropic web app.

To read the balance:

1. Open https://claude.ai/settings/billing (or your Anthropic Console billing page) in a browser.
2. Find the Agent SDK Credits section (sometimes labeled "API Credits" or "Agent SDK usage").
3. Note the current balance (e.g. $18.43 remaining of $20.00).

Write the value down — you will compare it after the canary turn.

---

Step 3 — Send the canary turn

With TUI-mode on and the host quiesced, send exactly one small request:

curl -s -X POST http://127.0.0.1:3456/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-haiku-4-5-20251001",
    "messages": [{"role": "user", "content": "Reply with the single word: pong"}],
    "max_tokens": 10
  }' | python3 -m json.tool

Use Haiku (the cheapest model) to minimize any hypothetical impact if the canary turns red.

Wait for the response to arrive completely (TUI-mode buffers the full response before returning — you will see a delay of several seconds, then the full reply).

---

Step 4 — Confirm the transcript shows entrypoint:"cli"

After the canary turn completes, inspect the most recent JSONL transcript for the billing-classifier label:

# The canary was run quiesced (Step 1), so the most recent JSONL across ALL project
# dirs IS the canary turn. We glob every projects subdir instead of recomputing
# claude's cwd-encoding rule (it maps every "/" AND "." to "-", e.g. ~/.ocp-tui/work
# => projects/-home-<user>--ocp-tui-work/; see lib/tui/transcript.mjs encodeCwd) —
# a glob is robust even if that encoding changes in a future claude build.
LATEST=$(ls -t "$HOME"/.claude/projects/*/*.jsonl 2>/dev/null | head -1)
echo "Transcript: $LATEST"
grep -o '"entrypoint":"[^"]*"' "$LATEST" | tail -1
# Expected: "entrypoint":"cli"

If the output shows "entrypoint":"cli", the billing-classifier label is correct. If it shows "entrypoint":"sdk-cli", the spawn did not get a real PTY — stop immediately and do not re-enable TUI-mode without investigation. Check tmux new-session manually and review ADR 0007 § spawn/PTY gate. (If the grep returns nothing, the transcript may not yet be flushed — re-run after a second, or confirm the turn completed.)

Reminder: an entrypoint:cli label (the cc_entrypoint=cli wire header) is necessary but not sufficient. It tells you OCP sent the right label to Anthropic. You must still check the credit balance in Step 5.

---

Step 5 — Re-read the Agent SDK credit balance AFTER the canary

Return to https://claude.ai/settings/billing and reload the page. Note the current balance again.

---

Step 6 — Green/Red decision

Green (balance unchanged)

The Agent SDK credit balance did not decrease. The turn billed against the Pro/Max subscription pool as expected. TUI-mode is working correctly.

Actions:

• Keep CLAUDE_TUI_MODE=true on this host.
• Monitor the balance periodically for the first week to catch any delayed attribution.
• Resume normal traffic.

Red (Agent SDK credit balance decreased)

The Agent SDK credit balance decreased. The subscription pool is not being used for TUI-mode turns on this host, despite cc_entrypoint=cli being set. This may indicate a backend routing change on Anthropic's side, a TTY detection failure, or a policy change.

Actions — immediate:

1. Unset CLAUDE_TUI_MODE (or set to any value other than "true") in the service unit:
   • systemd: edit /etc/ocp/ocp.env (or the unit's Environment= line), then sudo systemctl daemon-reload && sudo systemctl restart ocp.service
   • launchd: edit the plist EnvironmentVariables section, then launchctl bootout gui/$(id -u)/dev.ocp.proxy && launchctl bootstrap gui/$(id -u) <plist-path>
2. Restart OCP and confirm the /health response no longer shows TUI-mode active.
3. If you share this OCP with family or other Max users: freeze their access temporarily until you understand the billing impact.
4. Consider pivoting to OLP multi-provider (see OLP) which can spread load across other providers to avoid the Agent SDK credit drain.

Per ALIGNMENT.md Rule 2 / ADR 0007 § Kill-switch: "Per the constitution, the response is to drop the Anthropic provider rather than escalate spoofing."

---

Ongoing monitoring — self-classification mini-canary

To detect future drift (e.g. a claude CLI upgrade that changes TTY-detection behavior), you can run a periodic one-liner that sends a tiny TUI turn with OCP_TUI_ENTRYPOINT=auto (so claude self-classifies rather than having OCP pin the value) and alerts if the transcript self-classification is not cli:

# Run with OCP temporarily configured OCP_TUI_ENTRYPOINT=auto
# Then check the most recent transcript:
# Glob the most recent transcript across all project dirs (robust to claude's
# cwd-encoding rule; run this right after the auto-mode mini-canary turn).
LATEST=$(ls -t "$HOME"/.claude/projects/*/*.jsonl 2>/dev/null | head -1)
RESULT=$(grep -o '"entrypoint":"[^"]*"' "$LATEST" | tail -1)
echo "Self-classified entrypoint: $RESULT"
if echo "$RESULT" | grep -q '"entrypoint":"cli"'; then
  echo "OK — subscription pool"
else
  echo "ALERT — not cli; check TTY and billing"
fi

Run this after any major claude CLI upgrade. The auto mode lets the CLI's own t$A startup function determine the value from the actual TTY state (see ADR 0007 § Billing-classifier labeling).

---

Related

• Flip/rollback runbook — how to set and unset CLAUDE_TUI_MODE on systemd and launchd hosts
• ADR 0007 — TUI-mode architecture and governing rules
• Subscription-pool (TUI) mode