# Memory V2 Production Rollout Runbook Memory V2 is a facade and policy layer. Production rollout must not introduce new goosed, PG session, SSE, or Portal execution paths. ## Release Gates Run the normal repository release checks first: ```bash bash scripts/check-release-ready.sh ``` Run the Memory V2 backend contract gate: ```bash npm run check:memory-v2-contracts ``` Run the Memory V2 health gate: ```bash npm run check:memory-v2 -- --require-enabled --expect-backend legacy ``` Expected phase-one result: - `ok=true` - contract gate reports unique backend names and legacy backend present - `summary.enabled=true` - `summary.backend=legacy` - `summary.selectedBackend=legacy-conversation-memory` - `fail_open` passes - `legacy_write_compact_supported` passes - `write_uses_legacy` passes - `compact_uses_legacy` passes When validating the real pgvector adapter before canary: ```bash npm run check:memory-v2-contracts -- --include-pgvector-adapter ``` Future backend adapter rule: ```bash npm run scaffold:memory-v2-backend -- \ --name qdrant \ --category semantic \ --capability resolve ``` Use the scaffold as the starting point, add focused adapter tests, then pass `npm run check:memory-v2-contracts` before wiring any external SDK into runtime. Do not wire external services directly into `server.mjs`, `tkmind-proxy.mjs`, goosed, SSE, or Portal execution code. ## Qdrant Read-Only Phase The Qdrant adapter may be visible in runtime status before user traffic is canaried. Fallback-only environment: ```bash MEMORY_ENABLED=1 MEMORY_BACKEND=qdrant MEMORY_QDRANT_ENABLED=1 MEMORY_QDRANT_URL='http://qdrant.example:6333' MEMORY_QDRANT_COLLECTION='memind_memory' ``` Expected without embedding module: - `memory.backend=qdrant` - `memory.selectedBackend=legacy-conversation-memory` - `qdrant.available=false` - `qdrant.reason=embedding_module_not_configured` - `write_uses_legacy` still passes - `compact_uses_legacy` still passes Read-only smoke: ```bash MEMORY_QDRANT_URL='http://qdrant.example:6333' \ npm run smoke:memory-v2-qdrant -- \ --collection memind_memory \ --vector 1,0,0 \ --limit 1 ``` The smoke command sends a Qdrant search request only. It must not create collections, write points, or run from app startup. Runtime read-only canary requires: ```bash MEMORY_QDRANT_EMBEDDING_MODULE='./scripts/embed-memory-v2-qdrant.mjs' ``` Even during read-only canary, `write_uses_legacy` and `compact_uses_legacy` must still pass. ## Mem0 Adapter Phase Mem0 is the Memory Extraction backend boundary. The adapter uses HTTP and is dormant unless explicitly configured. Environment: ```bash MEMORY_ENABLED=1 MEMORY_BACKEND=mem0 MEMORY_MEM0_ENABLED=1 MEMORY_MEM0_API_KEY='...' MEMORY_MEM0_PROJECT_ID='memind_project' MEMORY_MEM0_BASE_URL='https://api.mem0.ai' ``` Smoke: ```bash npm run smoke:memory-v2-external -- --backend mem0 --operation write ``` Do not canary Mem0 writes until sampling, dedupe, rollback, and retention rules are accepted. ## Letta Lifecycle Adapter Phase Letta is the Memory Lifecycle Management backend boundary. The adapter uses HTTP and is dormant unless explicitly configured. Environment: ```bash MEMORY_ENABLED=1 MEMORY_BACKEND=letta MEMORY_LETTA_ENABLED=1 MEMORY_LETTA_API_KEY='...' MEMORY_LETTA_PROJECT_ID='memind_project' MEMORY_LETTA_AGENT_ID='agent_1' MEMORY_LETTA_BASE_URL='https://api.letta.com' ``` Smoke: ```bash npm run smoke:memory-v2-external -- --backend letta --operation resolve ``` Do not canary Letta until ownership boundaries, long/short-term promotion rules, conflict resolution with conversation-memory, rollback, and retention are accepted. ## Remaining External Adapter Phase Weaviate, Neo4j, Redis Streams, and LangGraph are wired with runtime clients and remain dormant unless explicitly configured. Environment examples: ```bash MEMORY_WEAVIATE_ENABLED=1 MEMORY_WEAVIATE_URL='https://weaviate.example' MEMORY_WEAVIATE_COLLECTION='MemindMemory' MEMORY_WEAVIATE_EMBEDDING_MODULE='./scripts/embed-memory-v2-weaviate.mjs' MEMORY_NEO4J_ENABLED=1 MEMORY_NEO4J_HTTP_URL='http://localhost:7474' MEMORY_NEO4J_USER='neo4j' MEMORY_NEO4J_PASSWORD='...' MEMORY_REDIS_STREAMS_ENABLED=1 MEMORY_REDIS_STREAMS_URL='redis://localhost:6379' MEMORY_LANGGRAPH_ENABLED=1 MEMORY_LANGGRAPH_URL='https://langgraph.example' MEMORY_LANGGRAPH_POLICY_ID='memory_policy' ``` Smoke: ```bash npm run smoke:memory-v2-external -- --backend weaviate --operation resolve npm run smoke:memory-v2-external -- --backend neo4j --operation write npm run smoke:memory-v2-external -- --backend redis-streams --operation write npm run smoke:memory-v2-external -- --backend langgraph --operation resolve ``` Future canaries must be backend-specific and must define data ownership, dedupe, observability, rollback, and retention before any adapter can own a Memory V2 operation. ## Phase 1: Facade Over Legacy Environment: ```bash MEMORY_ENABLED=1 MEMORY_BACKEND=legacy MEMORY_FAIL_OPEN=1 MEMORY_VECTOR_ENABLED=0 ``` Runtime verification: ```bash curl -sk https:///api/runtime/status ``` Confirm: - `memory.enabled=true` - `memory.backend=legacy` - `memory.selectedBackend=legacy-conversation-memory` - future plugin slots report `available=false` Rollback: ```bash MEMORY_ENABLED=0 ``` ## Phase 2: Empty pgvector Schema This phase creates schema only. It must not change runtime backend selection. Preconditions: - PostgreSQL server is approved for Memory V2 semantic memory. - `pgvector` extension is available. - Backup/rollback decision is recorded by ops. Dry-run: ```bash node scripts/setup-memory-v2-pgvector-schema.mjs \ --table memory_embeddings \ --dimensions 1536 \ --create-extension ``` Apply only through the approved production migration path: ```bash MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \ node scripts/setup-memory-v2-pgvector-schema.mjs \ --apply \ --table memory_embeddings \ --dimensions 1536 \ --create-extension ``` Keep runtime on legacy: ```bash MEMORY_BACKEND=legacy MEMORY_VECTOR_ENABLED=0 ``` Post-migration smoke verification: ```bash MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \ npm run smoke:memory-v2-pgvector -- \ --table memory_embeddings \ --dimensions 1536 ``` The smoke command inserts synthetic `memory-v2-smoke-*` rows, verifies nearest-vector retrieval, and deletes those rows by default. In production it should run only after the schema migration. Do not pass `--create-schema` in production unless the approved migration procedure explicitly requires it. For local-only plumbing checks, use the deterministic hash embedding module: ```bash MEMORY_PGVECTOR_EMBEDDING_MODULE=./scripts/embed-memory-v2-local-hash.mjs MEMORY_V2_LOCAL_EMBEDDING_DIMENSIONS=3 ``` Do not use this local hash embedding module for production retrieval quality. ## Phase 3: Backfill Backfill must be separate from schema creation and application rollout. Dry-run: ```bash MEMORY_BACKFILL_MYSQL_URL='mysql://...' \ node scripts/backfill-memory-v2-pgvector.mjs --limit 100 ``` Apply: ```bash MEMORY_BACKFILL_MYSQL_URL='mysql://...' \ MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \ node scripts/backfill-memory-v2-pgvector.mjs \ --apply \ --limit 100 \ --embedding-module ./scripts/embed-memory-v2-pgvector.mjs ``` Operational rules: - Store `{ updatedAt, id }` checkpoint externally after each successful batch. - Compare active MySQL memory count with pgvector distinct `source_memory_id` count. - Do not change `MEMORY_BACKEND` during backfill. - Do not delete pgvector rows during app rollback without an explicit retention decision. ## Phase 4: pgvector Resolve Canary Only start canary after schema, backfill, and retrieval quality checks pass. Environment: ```bash MEMORY_ENABLED=1 MEMORY_BACKEND=pgvector MEMORY_VECTOR_ENABLED=1 MEMORY_FAIL_OPEN=1 MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' MEMORY_PGVECTOR_EMBEDDING_MODULE='./scripts/embed-memory-v2-pgvector.mjs' ``` Health gate: ```bash npm run check:memory-v2 -- --require-enabled --expect-backend pgvector ``` Required invariant: - `selectedBackend=pgvector` is allowed for `resolve`. - `write_uses_legacy` must still pass. - `compact_uses_legacy` must still pass. - Any pgvector error must fail open to legacy or degraded empty resolve. Rollback: ```bash MEMORY_BACKEND=legacy MEMORY_VECTOR_ENABLED=0 ``` Emergency rollback: ```bash MEMORY_ENABLED=0 ```