docs: pivot memory continuity to lifecycle plugin design

references/context-engine-design.md

@@ -1,18 +1,30 @@
-# Memory Continuity ContextEngine Design
+# Memory Continuity Plugin Design
 
 ## Status
 Design draft only. No plugin implementation yet.
 
-## Why a ContextEngine version exists
+## Current architectural choice
+Memory continuity should **not** use the `ContextEngine` slot as its primary v1 architecture.
+
+Reason:
+- `contextEngine` is an **exclusive slot** in OpenClaw
+- users should not be forced to choose between memory continuity and context engines such as `lossless-claw`
+- context compression is a broad baseline need; working-state recovery is an additional capability
+
+Therefore the main path is:
+- **skill + ordinary lifecycle plugin** as the primary architecture
+- **ContextEngine integration** kept as a future option, not the default implementation target
+
+## Why a plugin version exists
 The current `memory-continuity` skill is useful, but it depends too much on agent cooperation:
 - the agent must notice recovery conditions
 - the agent must keep `memory/CURRENT_STATE.md` updated
 - the agent often benefits from `read`
 
-OpenClaw now exposes a formal `ContextEngine` extension point. That gives memory continuity a runtime-aligned path forward without modifying OpenClaw core.
+A lifecycle plugin gives a runtime-aligned way to improve reliability without modifying OpenClaw core and without consuming the exclusive ContextEngine slot.
 
 ## Product strategy
-Keep two product forms:
+Keep three forms with clear roles:
 
 ### A. Skill version
 Role:
@@ -21,13 +33,19 @@ Role:
 - template discipline
 - compatibility with environments that do not install plugins
 
-### B. ContextEngine plugin version
+### B. Lifecycle plugin version (primary runtime path)
 Role:
-- runtime-backed recovery
+- runtime-assisted recovery
-- continuity snapshot injection without depending on `read`
+- automatic checkpointing at key lifecycle points
-- stronger compaction and subagent continuity
+- better startup and `/new` continuity
+- coexistence with `lossless-claw` and other context engines
 
-These two forms should complement each other, not compete.
+### C. ContextEngine version (future option)
+Role:
+- more powerful prompt-time snapshot injection via `assemble` / `systemPromptAddition`
+- only worth pursuing later if slot tradeoffs are acceptable or composite engine support exists
+
+These forms should complement each other, not compete.
 
 ## Source of truth
 Primary durable checkpoint file:
@@ -35,7 +53,7 @@ Primary durable checkpoint file:
 
 The plugin should treat this file as the editable, human-readable source of truth for working state.
 
-The plugin may derive an internal lightweight snapshot from it, but should not replace it with an opaque database-first design.
+The plugin may derive a lighter runtime snapshot from it, but should not replace it with an opaque database-first design.
 
 ## Non-goals
 The plugin version should **not**:
@@ -43,20 +61,23 @@ The plugin version should **not**:
 - replace daily notes in `memory/YYYY-MM-DD.md`
 - replace native OpenClaw compaction summaries
 - replace native `memoryFlush`
+- replace session transcript memory search
 - persist a full transcript mirror
 - inject large recovery payloads into every turn
+- attempt real-time bidirectional state synchronization in v1
 
 ## Core runtime idea
-Use the ContextEngine lifecycle to ensure that short-term working state remains available across:
+Use standard lifecycle hooks to ensure that short-term working state remains available across:
 - `/new`
 - reset/restart
 - compaction
-- subagent boundaries
+- session end / restart-like boundaries
+- limited subagent handoff scenarios when supported by available hooks
 
 The plugin should prefer deterministic, structured recovery over free-form recollection.
 
 ## Desired user-visible property
-Even if an agent lacks `read`, the session should still have a compact continuity hint when there is active work worth recovering.
+Even if an agent lacks `read`, the session should still recover a compact continuity hint whenever there is meaningful active work to recover.
 
 ## Checkpoint schema
 The checkpoint file should retain a stable, minimal structure:
@@ -92,7 +113,7 @@ Potential future additions if truly needed:
 But the default should stay compact.
 
 ## Runtime snapshot shape
-The plugin should inject a much smaller summary than the raw file.
+The plugin should derive a much smaller summary than the raw checkpoint file.
 
 Target content:
 - Objective
@@ -100,126 +121,149 @@ Target content:
 - Last Confirmed Result or Key Decision(s)
 - Next Action
 - Blockers
+- Unsurfaced Results
 - Freshness / Updated At
 
 ### Draft example
 ```text
 CONTINUITY SNAPSHOT
 Objective: Verify memory continuity for Telegram subagents.
-Current Step: Tools fixed; validating runtime-backed recovery design.
+Current Step: Tools fixed; validating plugin-backed recovery design.
-Key Decision: Use skill as fallback, ContextEngine as long-term path.
+Key Decision: Use lifecycle plugin as primary path; ContextEngine stays optional.
-Next Action: Finalize plugin scope and hook responsibilities.
+Next Action: Finalize hook mapping and update the skill docs.
 Blockers: None.
+Unsurfaced Results: None.
 Updated: 2026-03-12T22:00:00+10:00
 ```
 
 ### Size target
 - preferred: ~150-300 tokens
-- avoid large raw file injection on every turn
+- avoid large raw checkpoint injection on every turn
 - skip injection entirely when there is no meaningful active state
 
-## ContextEngine lifecycle mapping
+### V1 rule for “meaningful active work”
+Treat work as active when:
+- `Objective` is non-empty
+- and `Objective` is not placeholder text such as `None`, `idle`, `n/a`, or an empty template marker
 
-### 1. `bootstrap`
+This rule can be refined later, but v1 should use a simple deterministic threshold.
+
+## Primary lifecycle hook mapping
+
+### 1. Startup hook (`before_agent_start` / closest available startup hook)
 Purpose:
-- initialize plugin-managed continuity state for a session
+- establish whether recovery state exists
-- verify whether a checkpoint file exists
+- load a compact continuity summary for startup recovery behavior
-- record whether recovery state is available
+- ensure recovery can happen even when the agent does not explicitly call `read`
 
 Should do:
 - check for `memory/CURRENT_STATE.md`
 - perform lightweight validation
-- avoid heavy prompt injection here
+- derive a compact startup continuity hint when active work exists
-
+- make recovery state available through the startup/lifecycle hook path supported by OpenClaw
-Should not do:
-- inject large content directly
-- rewrite the checkpoint file unnecessarily
-
-Reasoning:
-`bootstrap` should establish availability, not spend tokens.
-
----
-
-### 2. `assemble`
-Purpose:
-- the main recovery injection point
-
-Should do:
-- load/derive a very small continuity snapshot
-- return it via `systemPromptAddition`
-- inject only when the checkpoint indicates meaningful active work
-- favor stable, structured wording
 
 Should avoid:
-- injecting the full checkpoint file by default
+- rewriting the checkpoint unnecessarily
-- injecting stale or placeholder content
+- injecting large raw file content
-- adding snapshot text when objective/current step are empty or obviously idle
+- treating placeholder/idle state as active recovery material
 
-This is the key mechanism that enables baseline continuity **without requiring `read`**.
+Notes:
+- exact injection mechanism depends on the standard plugin hook surface available in OpenClaw
+- this design intentionally does **not** assume access to ContextEngine-only `systemPromptAddition`
 
 ---
 
-### 3. `afterTurn`
+### 2. `/new` hook (`command:new` or equivalent)
 Purpose:
-- opportunistic maintenance of checkpoint state after a completed turn
+- create a reliable checkpoint right before the user deliberately resets conversational continuity
+
+Should do:
+- save a final overwrite-style checkpoint before reset
+- optionally archive the outgoing checkpoint if that remains part of the skill design
+- ensure the next session can recover active work from a deterministic file state
+
+Should avoid:
+- expensive archival behavior for trivial idle sessions
+- losing unsurfaced results at reset boundaries
+
+---
+
+### 3. Session end / run-end hook (`agent_end` or closest available end hook)
+Purpose:
+- checkpoint work at natural lifecycle boundaries
+
+Should do:
+- persist the latest working-state checkpoint when a meaningful state change occurred
+- preserve unsurfaced results
+- act as a safety net when the agent followed the protocol imperfectly during the turn
+
+Should avoid:
+- noisy writes on obviously trivial/no-op turns
+- assuming this hook alone is enough for correctness
+
+---
+
+### 4. Compaction hooks (`session:compact:before` / equivalent)
+Purpose:
+- hard safety checkpoint before compaction removes detailed older context
+
+Should do:
+- force a final continuity checkpoint before compaction
+- preserve current objective / step / blockers / unsurfaced results
+- ensure recovery remains possible after compaction
+
+Critical requirement:
+- checkpoint writing in the compaction path must complete **synchronously** before compaction proceeds
+- if the hook cannot provide that guarantee, this risk must be documented explicitly
+
+This is the strongest required protection point.
+
+---
+
+### 5. Post-turn maintenance hook (when available)
+Purpose:
+- opportunistic checkpoint maintenance after substantive turns
 
 Should do:
 - update checkpoint when meaningful state changes are detected
 - optionally use a configurable cadence (for example every N substantive turns)
 - keep writes overwrite-oriented rather than append-heavy
 
+V1 definition of **substantive turn**:
+A turn counts as substantive when it includes at least one of:
+- a tool result that materially changes work state
+- a user confirmation of a decision or direction
+- an agent statement that a concrete step was completed
+- a newly discovered blocker or newly surfaced result
+
+Non-substantive examples:
+- greetings
+- acknowledgements
+- short clarifications without state change
+- filler chatter
+
 Should avoid:
 - writing on every trivial turn
 - producing noisy I/O for idle chat
 - becoming the only write path
 
 Practical stance:
-- `afterTurn` is a maintenance path, not the only safety net
+- post-turn maintenance is useful, but not sufficient alone
 - correctness should not rely entirely on semantic heuristics
 
----
+## Subagent continuity stance for v1
+V1 should stay conservative.
 
-### 4. `compact`
+### Allowed in v1
-Purpose:
+- parent → child: minimal seed/handoff when a suitable hook/path exists
-- hard safety checkpoint before compaction loses older detailed history
+- child → parent: limited recovery of `Unsurfaced Results`
 
-Should do:
+### Explicitly out of scope in v1
-- force a final continuity checkpoint before compaction
+- continuous bidirectional synchronization
-- preserve current objective / step / blockers / unsurfaced results
+- real-time merge of parent and child working state
-- ensure recovery remains possible after compaction
+- multi-worker consensus state
 
-This is the strongest required protection point.
+This keeps the first implementation tractable and reduces process risk.
-
-If any lifecycle hook must be treated as mandatory continuity insurance, this is the one.
-
----
-
-### 5. `prepareSubagentSpawn`
-Purpose:
-- seed child continuity with the minimum parent working-state context
-
-First implementation should stay simple:
-- prepare a lightweight child seed from parent objective + current step
-- avoid over-copying parent state
-- prefer a minimal handoff
-
-Initial scope suggestion:
-- include only what the child needs to start coherently
-- do not attempt full bidirectional synchronization in v1
-
----
-
-### 6. `onSubagentEnded`
-Purpose:
-- reclaim continuity-relevant child outputs when the child lifecycle ends
-
-First implementation should stay simple:
-- inspect whether child state contains meaningful `Unsurfaced Results`
-- optionally merge a compact result summary back into parent continuity state
-
-Caution:
-- parent/child sync can become complex quickly
-- v1 should prefer conservative merge behavior over ambitious automation
 
 ## Interaction with native OpenClaw systems
 
@@ -239,18 +283,36 @@ Memory continuity should complement that by providing:
 - a stable recovery surface
 - explicit next-step / blocker / unsurfaced-result fields
 
+### With session transcript memory search
+Session memory search can help retrieve prior conversational material.
+
+Memory continuity is different:
+- memory search helps answer “what did we discuss?”
+- continuity helps answer “what were we doing, and what should happen next?”
+
 ### With tools like `read`
 `read` remains valuable for enhanced recovery and debugging.
 
-But baseline continuity should not require `read` once the plugin injects a small snapshot during `assemble`.
+But baseline continuity should not depend on `read` once the lifecycle plugin can expose recovery state through startup/runtime hooks.
+
+### With ContextEngine plugins such as `lossless-claw`
+This is the main architectural reason the lifecycle-plugin path is preferred.
+
+Because `contextEngine` is an exclusive slot, making memory continuity a ContextEngine by default would force users to choose between:
+- context compression / context assembly plugins
+- working-state continuity
+
+V1 should avoid creating that conflict.
 
 ## Open design questions
-1. How should stale checkpoints be detected and labeled?
+1. Which exact standard hook surface is best for startup recovery injection on current OpenClaw releases?
-2. Should the plugin compute confidence/freshness automatically?
+2. How should stale checkpoints be detected and labeled?
-3. What is the best trigger for "active work exists"?
+3. Should the plugin compute confidence/freshness automatically?
-4. How much of `CURRENT_STATE.md` should be normalized vs preserved verbatim?
+4. How should the plugin expose a startup continuity hint without relying on ContextEngine-only `systemPromptAddition`?
 5. Should plugin writes go directly to `memory/CURRENT_STATE.md`, or stage then atomically replace?
-6. How should main/subagent continuity boundaries behave when multiple workers are active?
+6. How should compaction-hook guarantees be validated in practice?
+7. What is the safest minimal parent/child handoff path under current OpenClaw hook support?
+8. Under what future conditions would a ContextEngine variant become worth the slot tradeoff?
 
 ## Recommended implementation phases
 
@@ -260,31 +322,36 @@ But baseline continuity should not require `read` once the plugin injects a smal
 - clarify scope vs non-goals
 - improve validation / doctor behavior
 
-### Phase 2 — Minimal plugin MVP
+### Phase 2 — Minimal lifecycle plugin MVP
-- register context engine
+- register a standard plugin
-- implement `bootstrap`
+- implement startup recovery hook behavior
-- implement `assemble` with `systemPromptAddition`
+- implement `/new` checkpoint behavior
-- implement `compact` forced checkpoint
+- implement end-of-run checkpoint behavior
-- leave subagent lifecycle hooks minimal or no-op initially
+- implement compaction-path checkpoint behavior if the hook guarantees are sufficient
 
 ### Phase 3 — Reliability improvements
-- add controlled `afterTurn` checkpointing
+- add controlled post-turn checkpointing
 - add freshness/confidence labeling
 - improve stale-state handling
-- tune injection length
+- tune snapshot length and injection behavior
 
-### Phase 4 — Subagent continuity
+### Phase 4 — Conservative subagent support
-- implement minimal `prepareSubagentSpawn`
+- add minimal parent → child seed behavior when safe
-- implement conservative `onSubagentEnded`
+- add conservative child → parent unsurfaced-result recovery
-- validate parent/child merge behavior in real workflows
+- validate handoff behavior in real workflows
+
+### Phase 5 — Future option evaluation
+- reassess whether a ContextEngine variant is worth building
+- only pursue if slot tradeoffs are acceptable or composite-engine support exists
 
 ## Success criteria
 The plugin version is successful when:
 - reset/new sessions recover active work without depending on `read`
 - compaction no longer destroys actionable in-flight state
-- subagent continuity improves without excessive prompt bloat
+- the agent does not lose unsurfaced results at reset-like boundaries
-- the snapshot remains small enough to be practical on every assembled turn
+- the runtime path coexists with `lossless-claw` and similar context engines
-- behavior aligns with OpenClaw’s official plugin/context-engine model
+- startup recovery improves without excessive prompt bloat
+- behavior aligns with OpenClaw’s official plugin and hook model
 
 ## Short summary
-The ContextEngine plugin version should become the **runtime-backed continuity layer**, while the existing skill remains the **human-readable protocol and fallback behavior contract**.
+The primary long-term implementation should be a **standard lifecycle plugin** that improves continuity without consuming the exclusive ContextEngine slot, while the existing skill remains the **human-readable protocol and fallback behavior contract**. A ContextEngine variant remains a future option, not the default architecture.

references/scope.md

@@ -20,7 +20,7 @@ The canonical working-state record is a Markdown checkpoint file:
 
 Longer-term direction:
 - keep the checkpoint file as source of truth
-- add a runtime-delivered continuity snapshot derived from that file
+- add runtime-assisted continuity delivery derived from that file
 - keep the file readable/editable by humans and agents
 
 ## Responsibilities
@@ -38,10 +38,11 @@ Memory continuity is **not responsible for**:
 2. Replacing `MEMORY.md` or daily notes
 3. Replacing OpenClaw compaction summaries
 4. Replacing OpenClaw `memoryFlush`
-5. Acting as a project-management database
+5. Replacing session transcript memory search
-6. Acting as a full conversation transcript
+6. Acting as a project-management database
-7. Storing every detail of recent chat history
+7. Acting as a full conversation transcript
-8. Guaranteeing perfect semantic recall of arbitrary facts from all prior turns
+8. Storing every detail of recent chat history
+9. Guaranteeing perfect semantic recall of arbitrary facts from all prior turns
 
 ## Relationship to native OpenClaw systems
 ### Native OpenClaw handles
@@ -51,6 +52,7 @@ Memory continuity is **not responsible for**:
 - transcript persistence
 - tools, sessions, and runtime orchestration
 - context engine selection and plugin lifecycle
+- session transcript recall via session-aware memory search
 
 ### Memory continuity adds
 - a **structured checkpoint** for working state
@@ -58,6 +60,18 @@ Memory continuity is **not responsible for**:
 - explicit fields for `Objective`, `Current Step`, `Next Action`, `Blockers`, and `Unsurfaced Results`
 - stronger short-term recovery for in-flight work than generic compaction summaries alone
 
+### Boundary with session memory search
+Session memory search can help answer questions like:
+- what did we discuss before?
+- what decision was mentioned in a prior session?
+
+Memory continuity is for a different question:
+- what are we doing **right now**, where did we stop, and what should happen next?
+
+In short:
+- session memory search is good at **recalling prior conversation material**
+- memory continuity is good at **recovering active working state**
+
 ## Product forms
 ### 1. Skill version (current / fallback version)
 Purpose:
@@ -76,17 +90,26 @@ What it cannot guarantee:
 - recovery without correct tool/config support
 - automatic runtime injection on every turn
 
-### 2. ContextEngine plugin version (target architecture)
+### 2. Lifecycle plugin version (target architecture)
 Purpose:
-- runtime-backed continuity guarantees
+- runtime-assisted continuity guarantees without taking the exclusive ContextEngine slot
 - reduced dependence on `read`
-- better compaction and subagent continuity
+- better startup, `/new`, and compaction continuity
+- coexistence with context engines such as `lossless-claw`
 
 What it should do:
-- inject a tiny continuity snapshot through `assemble`
+- use ordinary lifecycle hooks to checkpoint and recover working state
-- checkpoint state before compaction
+- improve startup recovery behavior
-- optionally maintain/update state after turns
+- checkpoint before destructive context transitions when hooks permit it
-- support parent/child continuity hooks
+- support minimal parent/child continuity handoff without requiring full bidirectional sync
+
+### 3. ContextEngine version (future option, not v1)
+Purpose:
+- more powerful prompt-time continuity injection when the ecosystem tradeoff is worth it
+
+Why it is not the current primary path:
+- `contextEngine` is an exclusive plugin slot
+- users should not be forced to choose between memory continuity and widely useful context engines such as `lossless-claw`
 
 ## Design principles
 1. **Files remain source of truth**
@@ -96,6 +119,7 @@ What it should do:
 5. **Continuity complements native OpenClaw memory; it does not replace it**
 6. **Working-state recovery must prefer truth over confident guessing**
 7. **User-visible recovery should prioritize current task state over generic greetings when continuity is clearly requested**
+8. **Ecosystem compatibility matters: continuity should not unnecessarily block other high-value plugins**
 
 ## Minimal recovery fields
 Any continuity implementation should preserve, at minimum:
@@ -129,10 +153,12 @@ The continuity layer is considered insufficient if, after a reset-like event, th
 - cannot identify the next action
 - hides completed but unsurfaced results
 - hallucinates prior work instead of expressing uncertainty
+- when `CURRENT_STATE.md` exists and contains active work, opens with generic greeting/chit-chat instead of first surfacing the recovered state in a recovery scenario
 
 ## Current roadmap stance
 - **Short term:** strengthen the existing skill + file discipline version
-- **Medium term:** implement a ContextEngine plugin version aligned with OpenClaw’s official extension model
+- **Medium term:** implement a standard lifecycle plugin version aligned with OpenClaw’s official hook model
-- **Long term:** keep both forms
+- **Long term:** keep multiple compatible forms
   - skill = fallback + behavior contract
-  - plugin = runtime-backed reliability layer
+  - lifecycle plugin = primary runtime-assisted reliability layer
+  - context-engine variant = optional future path when slot tradeoffs are acceptable