memory-continuity/docs/superpowers/plans/2026-03-31-v3.1-short-term-memory.md

v3.1 Short-Term Memory Layer Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add session logging, smart tail protection during compaction, CJK token awareness, and noise filtering to the Memory Continuity plugin.

Architecture: Four independent features added to the existing lifecycle hook system. Session logs write to memory/sessions/YYYY-MM-DD.md (one file per day, append-only). Tail protection injects recent critical messages into before_compaction. CJK token estimation uses a helper function for line-count decisions. Ignore patterns filter out cron/subagent noise at agent_end.

Tech Stack: Node.js (ES modules), pure filesystem, zero dependencies

---

File Structure

File	Action	Responsibility
index.js	Modify	Add session logging in agent_end, enhance before_compaction with tail protection, add CJK helper, add ignore-pattern filtering
openclaw.plugin.json	Modify	Add new config keys: sessionLogging, tailProtectCount, ignorePatterns
mc-plugin/index.js	Modify	Add /mc sessions subcommand, update help text
mc-plugin/openclaw.plugin.json	No change	—

---

Task 1: CJK Token Estimation Helper

Files:

• Modify: index.js (after line 15, the PLACEHOLDER_VALUES block)

• Step 1: Add estimateTokens helper function to index.js

Add this after the PLACEHOLDER_VALUES constant (line 15):

/**
 * Estimate token count with CJK awareness.
 * CJK characters ≈ 1.5 tokens each; Latin words ≈ 1 token per ~4 chars.
 */
function estimateTokens(text) {
  if (!text) return 0;
  // Count CJK characters (CJK Unified Ideographs + common CJK ranges)
  const cjkCount = (text.match(/[\u4e00-\u9fff\u3400-\u4dbf\uf900-\ufaff\u3000-\u303f\uff00-\uffef]/g) || []).length;
  // Remove CJK chars, count remaining as ~1 token per 4 chars
  const nonCjk = text.replace(/[\u4e00-\u9fff\u3400-\u4dbf\uf900-\ufaff\u3000-\u303f\uff00-\uffef]/g, "");
  const latinTokens = Math.ceil(nonCjk.length / 4);
  return Math.ceil(cjkCount * 1.5) + latinTokens;
}

• Step 2: Update truncate in extractStateFromMessages to use token-aware truncation

In extractStateFromMessages (around line 175), replace the existing truncate function:

  // Token-aware truncation
  const truncate = (s, maxTokens = 200) => {
    if (estimateTokens(s) <= maxTokens) return s;
    // Binary search for the right cut point
    let lo = 0, hi = s.length;
    while (lo < hi) {
      const mid = (lo + hi + 1) >> 1;
      if (estimateTokens(s.slice(0, mid)) <= maxTokens) lo = mid;
      else hi = mid - 1;
    }
    return s.slice(0, lo) + "...";
  };

• Step 3: Verify no syntax errors

Run:

cd /Users/taodeng/.openclaw/projects/memory-continuity && node -c index.js

Expected: no output (syntax OK)

• Step 4: Commit

git add index.js
git commit -m "feat: add CJK token-aware estimation helper (v3.1)"

---

Task 2: Ignore Patterns — Filter Cron/Subagent Noise

Files:

• Modify: index.js (agent_end hook, around line 295)

• Modify: openclaw.plugin.json (add ignorePatterns config)

• Step 1: Add ignorePatterns to openclaw.plugin.json config schema

Add after the maxArchiveCount property block (after line 29):

      "ignorePatterns": {
        "type": "array",
        "default": [],
        "description": "Regex patterns to ignore sessions (e.g. cron jobs, subagent noise). Matched against first user message."
      }

And add corresponding uiHints after the maxArchiveCount hint (after line 47):

    "ignorePatterns": {
      "label": "Ignore patterns",
      "help": "Skip state extraction for sessions matching these patterns (regex)"
    }

• Step 2: Add ignore-pattern check in agent_end hook in index.js

In the agent_end hook, after the realUserMsgs filtering block (after line 321), add:

      // Check ignore patterns — skip sessions matching cron/subagent noise
      const ignorePatterns = (config.ignorePatterns || [])
        .map(p => { try { return new RegExp(p, "i"); } catch { return null; } })
        .filter(Boolean);

      if (ignorePatterns.length > 0 && realUserMsgs.length > 0) {
        const firstMsg = realUserMsgs[0];
        if (ignorePatterns.some(re => re.test(firstMsg))) {
          log.info?.("[memory-continuity] Session matches ignorePattern, skipping");
          return;
        }
      }

• Step 3: Verify no syntax errors

Run:

cd /Users/taodeng/.openclaw/projects/memory-continuity && node -c index.js

Expected: no output (syntax OK)

• Step 4: Commit

git add index.js openclaw.plugin.json
git commit -m "feat: add ignorePatterns to filter cron/subagent noise (v3.1)"

---

Task 3: Session Logging

Files:

• Modify: index.js (add writeSessionLog helper + call it in agent_end)

• Step 1: Add writeSessionLog helper function to index.js

Add after the cleanupMemoryDir function (after line 140):

/**
 * Append a session summary to the daily session log.
 * File: memory/sessions/YYYY-MM-DD.md (one per day, append-only)
 */
function writeSessionLog(workspaceDir, messages, config = {}) {
  if (config.sessionLogging === false) return;
  if (!messages || messages.length === 0) return;

  const now = new Date();
  const pad = (n) => String(n).padStart(2, "0");
  const dateStr = `${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}`;
  const timeStr = `${pad(now.getHours())}:${pad(now.getMinutes())}`;

  const sessionsDir = path.join(workspaceDir, "memory", "sessions");
  fs.mkdirSync(sessionsDir, { recursive: true });

  const logFile = path.join(sessionsDir, `${dateStr}.md`);

  // Extract first meaningful user message as topic
  let topic = "(no topic)";
  for (const msg of messages) {
    if (msg?.role !== "user") continue;
    const text = typeof msg.content === "string"
      ? msg.content
      : Array.isArray(msg.content)
        ? msg.content.filter(b => b?.type === "text").map(b => b.text).join("\n")
        : "";
    const cleaned = text
      .replace(/^Conversation info \(untrusted metadata\):[\s\S]*?\n\n/m, "")
      .replace(/^Sender \(untrusted metadata\):[\s\S]*?\n\n/m, "")
      .trim();
    if (cleaned.length > 10) {
      topic = cleaned.split("\n")[0].slice(0, 120);
      break;
    }
  }

  // Count messages by role
  const userCount = messages.filter(m => m?.role === "user").length;
  const assistantCount = messages.filter(m => m?.role === "assistant").length;
  const totalTokens = estimateTokens(
    messages.map(m => typeof m?.content === "string" ? m.content : "").join("")
  );

  // Build log entry
  const entry = [
    `### ${timeStr}`,
    `- **Topic:** ${topic}`,
    `- **Messages:** ${userCount} user / ${assistantCount} assistant`,
    `- **Est. tokens:** ~${totalTokens}`,
    "",
  ].join("\n");

  // Append to daily log (create with header if new)
  if (!fs.existsSync(logFile)) {
    const header = `# Session Log — ${dateStr}\n\n`;
    fs.writeFileSync(logFile, header + entry, "utf8");
  } else {
    fs.appendFileSync(logFile, entry, "utf8");
  }
}

• Step 2: Add sessionLogging config to openclaw.plugin.json

Add after the ignorePatterns property in the config schema:

      "sessionLogging": {
        "type": "boolean",
        "default": true,
        "description": "Write session summaries to memory/sessions/ daily logs"
      }

And add corresponding uiHints:

    "sessionLogging": {
      "label": "Session logging",
      "help": "Append session summaries to daily markdown logs"
    }

• Step 3: Call writeSessionLog from agent_end hook

In the agent_end hook, right before the const existing = readFile(statePath); line (around line 324), add:

      // Write session log entry
      writeSessionLog(ws, messages, config);

• Step 4: Verify no syntax errors

Run:

cd /Users/taodeng/.openclaw/projects/memory-continuity && node -c index.js

Expected: no output (syntax OK)

• Step 5: Commit

git add index.js openclaw.plugin.json
git commit -m "feat: add session logging to memory/sessions/ daily files (v3.1)"

---

Task 4: Smart Tail Protection in before_compaction

Files:

• Modify: index.js (enhance before_compaction hook, around line 254)

• Modify: openclaw.plugin.json (add tailProtectCount config)

• Step 1: Add tailProtectCount config to openclaw.plugin.json

Add after the sessionLogging property in the config schema:

      "tailProtectCount": {
        "type": "number",
        "default": 3,
        "description": "Number of recent critical message pairs to protect during compaction"
      }

And add corresponding uiHints:

    "tailProtectCount": {
      "label": "Tail protect count",
      "help": "Keep N recent user/assistant pairs visible after compaction"
    }

• Step 2: Add extractTailMessages helper to index.js

Add after the writeSessionLog function:

/**
 * Extract the last N meaningful user/assistant exchange pairs from messages.
 * Returns a formatted string for injection into compaction context.
 */
function extractTailMessages(messages, count = 3) {
  if (!messages || messages.length === 0) return null;

  // Walk backwards, collect up to `count` user+assistant pairs
  const pairs = [];
  let i = messages.length - 1;

  while (i >= 0 && pairs.length < count) {
    // Find assistant message
    while (i >= 0 && messages[i]?.role !== "assistant") i--;
    if (i < 0) break;
    const assistantMsg = messages[i];
    i--;

    // Find preceding user message
    while (i >= 0 && messages[i]?.role !== "user") i--;
    if (i < 0) break;
    const userMsg = messages[i];
    i--;

    const getText = (msg) => {
      if (typeof msg?.content === "string") return msg.content;
      if (Array.isArray(msg?.content)) {
        return msg.content.filter(b => b?.type === "text").map(b => b.text).join("\n");
      }
      return "";
    };

    const userText = getText(userMsg).trim();
    const assistantText = getText(assistantMsg).trim();

    // Skip trivial exchanges
    if (userText.length < 10 && assistantText.length < 20) continue;

    // Token-aware truncation per message
    const maxPerMsg = 150;
    const truncMsg = (s) => {
      if (estimateTokens(s) <= maxPerMsg) return s;
      let lo = 0, hi = s.length;
      while (lo < hi) {
        const mid = (lo + hi + 1) >> 1;
        if (estimateTokens(s.slice(0, mid)) <= maxPerMsg) lo = mid;
        else hi = mid - 1;
      }
      return s.slice(0, lo) + "...";
    };

    pairs.unshift({ user: truncMsg(userText), assistant: truncMsg(assistantText) });
  }

  if (pairs.length === 0) return null;

  const lines = ["=== RECENT EXCHANGE (protected) ==="];
  for (const p of pairs) {
    lines.push(`User: ${p.user}`);
    lines.push(`Assistant: ${p.assistant}`);
    lines.push("---");
  }
  lines.push("=== END RECENT EXCHANGE ===");
  return lines.join("\n");
}

• Step 3: Enhance before_compaction hook to inject tail messages

Replace the existing before_compaction hook body (lines 254-270) with:

    api.on("before_compaction", async (_event, _ctx) => {
      const ws = _ctx?.workspaceDir;
      const config = getConfig();
      const statePath = resolveStatePath(ws);
      if (!statePath) return;

      const md = readFile(statePath);
      if (!md) return;

      const snapshot = buildSnapshot(md);
      if (!snapshot) return;

      log.info?.("[memory-continuity] Injecting state before compaction");

      // Smart tail protection: also inject recent critical messages
      const tailCount = config.tailProtectCount ?? 3;
      const messages = _event?.messages;
      const tail = tailCount > 0 ? extractTailMessages(messages, tailCount) : null;

      const parts = [snapshot];
      if (tail) {
        parts.push(tail);
        log.info?.("[memory-continuity] Tail protection: injected recent exchanges");
      }

      return {
        prependSystemContext: parts.join("\n\n"),
      };
    }, { priority: 10 });

• Step 4: Verify no syntax errors

Run:

cd /Users/taodeng/.openclaw/projects/memory-continuity && node -c index.js

Expected: no output (syntax OK)

• Step 5: Commit

git add index.js openclaw.plugin.json
git commit -m "feat: smart tail protection during compaction (v3.1)"

---

Task 5: /mc sessions Command

Files:

• Modify: mc-plugin/index.js (add cmdSessions function + wire into switch)

• Step 1: Add cmdSessions handler to mc-plugin/index.js

Add after the cmdExport function (before cmdHelp, around line 437):

function cmdSessions(args) {
  const parts = (args || "").trim().split(/\s+/);
  const agent = parts.find(p => !p.startsWith("-") && !p.startsWith("2")) || "main";
  const dateArg = parts.find(p => /^\d{4}-\d{2}-\d{2}$/.test(p));

  const memDir = resolveMemDir(agent);
  if (!memDir) return `Agent "${agent}" not found.`;

  const sessionsDir = path.join(memDir, "sessions");
  let files;
  try { files = fs.readdirSync(sessionsDir).filter(f => f.endsWith(".md")).sort().reverse(); }
  catch { return `No session logs for "${agent}".`; }

  if (!files.length) return `No session logs for "${agent}".`;

  // If date specified, show that day's log
  if (dateArg) {
    const target = `${dateArg}.md`;
    const content = readFile(path.join(sessionsDir, target));
    if (!content) return `No session log for ${dateArg}.`;
    // Truncate to last 50 lines to stay compact
    const lines = content.split("\n");
    const shown = lines.length > 50 ? lines.slice(-50) : lines;
    let out = shown.join("\n");
    if (lines.length > 50) out = `... (${lines.length - 50} earlier lines omitted)\n\n` + out;
    return out;
  }

  // List recent session logs
  let out = `Session Logs: ${agent} (${files.length} days)\n`;
  out += "─────────────────────────────\n";

  for (const f of files.slice(0, 14)) {
    const date = f.replace(".md", "");
    const content = readFile(path.join(sessionsDir, f));
    // Count session entries (### HH:MM headers)
    const sessionCount = content ? (content.match(/^### \d{2}:\d{2}/gm) || []).length : 0;
    out += `${date}   ${String(sessionCount).padStart(3)} session(s)\n`;
  }

  if (files.length > 14) out += `\n  ... and ${files.length - 14} more days`;
  out += `\n\nUse /mc sessions <YYYY-MM-DD> to view a specific day.`;
  return out.trimEnd();
}

• Step 2: Wire sessions into the command switch

In the switch statement (around line 471), add a new case before help:

          case "sessions": text = cmdSessions(subargs || null); break;

• Step 3: Update cmdHelp to include sessions

Add this line in the help text after the /mc state --all line:

/mc sessions [date]     Session logs (daily activity)

• Step 4: Verify no syntax errors

Run:

cd /Users/taodeng/.openclaw/projects/memory-continuity && node -c mc-plugin/index.js

Expected: no output (syntax OK)

• Step 5: Commit

git add mc-plugin/index.js
git commit -m "feat: add /mc sessions command for daily session logs (v3.1)"

---

Task 6: Version Bump and Final Verification

Files:

• Modify: openclaw.plugin.json (version 3.0.0 → 3.1.0)

• Modify: mc-plugin/openclaw.plugin.json (version if present)

• Modify: package.json (version if present)

• Step 1: Bump version in openclaw.plugin.json to 3.1.0

Change "version": "3.0.0" to "version": "3.1.0".

• Step 2: Bump version in package.json to 3.1.0

Change "version" value to "3.1.0".

• Step 3: Verify both plugins parse correctly

Run:

cd /Users/taodeng/.openclaw/projects/memory-continuity && node -c index.js && node -c mc-plugin/index.js && node -e "JSON.parse(require('fs').readFileSync('openclaw.plugin.json','utf8'))" && echo "All OK"

Expected: All OK

• Step 4: Commit and tag

git add openclaw.plugin.json package.json
git commit -m "release: v3.1.0 — short-term memory layer"

• Step 5: Push to GitHub

git push

---

Summary of New Config Keys

Key	Type	Default	Purpose
ignorePatterns	string[]	[]	Regex patterns to skip sessions (cron/subagent)
sessionLogging	boolean	true	Enable daily session log files
tailProtectCount	number	3	Recent exchange pairs to protect during compaction

Summary of New/Changed Hooks

Hook	Change
agent_end	+session logging, +ignore pattern filtering, +CJK-aware truncation
before_compaction	+tail message protection

New Command

Command	Purpose
/mc sessions [date]	List daily session logs or view specific day