auto-dream

Background memory consolidation cycle. Scans memory files, finds stale/duplicate/conflicting entries, consolidates, synthesizes cross-session insights, builds an injection-ready payload for next session start, and writes a dated dream report.

When to invoke

• User says "run dream", "consolidate memories", "clean up memories", "memory maintenance", "deduplicate memories"
• Cron job at 2 AM nightly via wrapper script: scripts/auto-dream-cron.sh --execute
• Manual trigger for testing: ./scripts/auto-dream-cron.sh (dry-run by default)

Reference Loading Table

Signal	Load These Files	Why
Debugging failed cron run, silent failure, empty log, wrong exit code	headless-cron-patterns.md	Routes to the matching deep reference
Setting up or modifying wrapper script (flock, --permission-mode, envsubst, PIPESTATUS)	headless-cron-patterns.md	Routes to the matching deep reference
Budget cap, --max-budget-usd, unattended Claude invocation	headless-cron-patterns.md	Routes to the matching deep reference
Writing, updating, or archiving memory files	memory-file-operations.md	Routes to the matching deep reference
Updating MEMORY.md index, atomic write, .tmp rename	memory-file-operations.md	Routes to the matching deep reference
Staleness detection, duplicate merging, conflict flagging	memory-file-operations.md	Routes to the matching deep reference
YAML frontmatter structure, merged_from, memory file format	memory-file-operations.md	Routes to the matching deep reference
Testing the dream cycle safely, dry-run validation, output file verification	dream-cycle-testing.md	Routes to the matching deep reference
Inspecting graduation candidates, snapshot testing, PIPESTATUS in test wrappers	dream-cycle-testing.md	Routes to the matching deep reference
Reading and interpreting cron run logs, detecting silent failures	logging-patterns.md	Routes to the matching deep reference
Log rotation, log directory structure, phase completion markers in logs	logging-patterns.md	Routes to the matching deep reference
last-dream.md stale, missing injection payload, cron log empty	logging-patterns.md	Routes to the matching deep reference
Concurrent dream runs, lockfile already held, duplicate cron invocations	concurrency.md	Routes to the matching deep reference
MEMORY.md.tmp left behind, partial write recovery, atomic rename failure	concurrency.md	Routes to the matching deep reference
database is locked, SQLite WAL mode, busy_timeout, concurrent DB access	concurrency.md	Routes to the matching deep reference
local changes would be overwritten, git stash before GRADUATE branch switch	concurrency.md	Routes to the matching deep reference

Instructions

When invoked interactively (not via cron), read skills/auto-dream/dream-prompt.md and execute its phases directly. The prompt is self-contained — it describes the full seven-phase cycle including safety constraints, file paths, and output formats.

For cron invocation: the dream prompt is passed directly to claude -p and runs as a standalone headless session with no CLAUDE.md, no hooks, no project context. All instructions are embedded in the prompt.

Phases

1. SCAN — Read all memory files, query learning.db sessions (last 7 days), read recent git log. Write scan document to ~/.claude/state/dream-scan-{date}.md.
2. ANALYZE — Identify stale, duplicate, conflicting memories and cross-session patterns. Write analysis to ~/.claude/state/dream-analysis-{date}.md.
3. CONSOLIDATE — Apply consolidation actions (max 5 changes). Archive stale/merged files, update MEMORY.md atomically.
4. SYNTHESIZE — Create insight memories from cross-session patterns (max 2 new memories per cycle).
5. GRADUATE — Promote mature learning DB entries (confidence >= 0.9, 3+ observations) into agent/skill files as permanent anti-patterns. Commits on dream/graduate-YYYY-MM-DD branch for human review. Max 3 per cycle. (ADR-159)
6. SELECT — Build injection-ready payload for session start. Write to ~/.claude/state/dream-injection-{project-hash}.md.
7. REPORT — Write dream summary to ~/.claude/state/last-dream.md.

Safety constraints (always enforced)

• Never delete files — archive to memory/archive/, never rm
• Write the REPORT before executing any CONSOLIDATE filesystem operations
• Maximum 5 memory changes per cycle — excess items deferred to next cycle
• Flag conflicts for human review, never auto-resolve
• Preserve YAML frontmatter when merging; use merged_from field for provenance
• In dry-run mode (the default), CONSOLIDATE, SYNTHESIZE, and GRADUATE describe proposed changes only — no filesystem writes or git operations. The wrapper script sets DREAM_DRY_RUN_MODE=yes which is substituted into the prompt at runtime.
• GRADUATE commits on a feature branch (dream/graduate-*), never on main — user reviews and merges
• Maximum 3 graduations per cycle — only entries with confidence >= 0.9 and 3+ observations

Testing

# Dry run (read-only, no filesystem changes — dry-run is the default)
./scripts/auto-dream-cron.sh

# Full run (execute consolidation)
./scripts/auto-dream-cron.sh --execute

# Check output
cat ~/.claude/state/last-dream.md

# Check graduation candidates (what dream would graduate)
python3 -c "
import sys; sys.path.insert(0, 'hooks/lib')
from learning_db_v2 import query_graduation_candidates
import json
candidates = query_graduation_candidates()
print(json.dumps(candidates, indent=2))
"

# Check if a graduation branch exists
git branch --list 'dream/graduate-*'

# Verify cron registration
python3 ~/.claude/scripts/crontab-manager.py list

Cost estimate

~$0.09 per nightly run with 50 memory files (~20-30K input tokens at Sonnet pricing). ~$33/year for automated overnight operation. Budget capped at $3.00/run via wrapper script.

Cron setup

Use crontab-manager.py (not raw crontab -e) to install. The wrapper script handles PATH, lockfile, logging, budget cap, and dry-run/execute toggle.

# Preview the cron entry
python3 ~/.claude/scripts/crontab-manager.py add \
  --tag "auto-dream" \
  --schedule "7 2 * * *" \
  --command "/home/feedgen/claude-code-toolkit/scripts/auto-dream-cron.sh --execute >> /home/feedgen/claude-code-toolkit/cron-logs/auto-dream/cron.log 2>&1" \
  --dry-run

# Install (after dry-run testing passes)
python3 ~/.claude/scripts/crontab-manager.py add \
  --tag "auto-dream" \
  --schedule "7 2 * * *" \
  --command "/home/feedgen/claude-code-toolkit/scripts/auto-dream-cron.sh --execute >> /home/feedgen/claude-code-toolkit/cron-logs/auto-dream/cron.log 2>&1"

# Verify
python3 ~/.claude/scripts/crontab-manager.py verify --tag auto-dream

Note: schedule uses 2:07 AM (off-minute) per cron best practice — avoids load spikes from jobs firing at :00.

Wrapper script details

scripts/auto-dream-cron.sh follows the established headless cron pattern (see scripts/reddit-automod-cron.sh):

• flock lockfile prevents concurrent runs
• --permission-mode auto (never --dangerously-skip-permissions)
• --max-budget-usd 3.00 caps spend per run
• --no-session-persistence for clean headless operation
• envsubst templates dream-prompt.md with project-specific paths at runtime
• tee to timestamped per-run log file
• Dry-run by default, --execute for live runs
• Exit code propagation via PIPESTATUS[0]

Reference Loading

Load these references when the task matches the signal:

Signal / Task	Reference File
Debugging failed cron run, silent failure, empty log, wrong exit code	references/headless-cron-patterns.md
Setting up or modifying wrapper script (flock, --permission-mode, envsubst, PIPESTATUS)	references/headless-cron-patterns.md
Budget cap, --max-budget-usd, unattended Claude invocation	references/headless-cron-patterns.md
Writing, updating, or archiving memory files	references/memory-file-operations.md
Updating MEMORY.md index, atomic write, .tmp rename	references/memory-file-operations.md
Staleness detection, duplicate merging, conflict flagging	references/memory-file-operations.md
YAML frontmatter structure, merged_from, memory file format	references/memory-file-operations.md
Testing the dream cycle safely, dry-run validation, output file verification	references/dream-cycle-testing.md
Inspecting graduation candidates, snapshot testing, PIPESTATUS in test wrappers	references/dream-cycle-testing.md
Reading and interpreting cron run logs, detecting silent failures	references/logging-patterns.md
Log rotation, log directory structure, phase completion markers in logs	references/logging-patterns.md
last-dream.md stale, missing injection payload, cron log empty	references/logging-patterns.md
Concurrent dream runs, lockfile already held, duplicate cron invocations	references/concurrency.md
MEMORY.md.tmp left behind, partial write recovery, atomic rename failure	references/concurrency.md
database is locked, SQLite WAL mode, busy_timeout, concurrent DB access	references/concurrency.md
local changes would be overwritten, git stash before GRADUATE branch switch	references/concurrency.md