Merge Memory V2 runtime facade

2026-07-03 09:46:03 +08:00
parent 7005c2a834
commit 20116b161f
69 changed files with 11869 additions and 27 deletions
@@ -0,0 +1,910 @@
+# Memind Memory V2
+
+Memory V2 is a facade and policy abstraction layer over existing Memind memory systems.
+
+It is not a new memory system. It does not introduce new storage, new execution paths, or a replacement runtime.
+
+## Scope
+
+Memory V2 wraps the existing:
+
+- `conversation-memory.mjs`
+- MySQL memory tables such as `h5_conversation_messages` and `h5_user_memory_items`
+- capability-driven memory hooks such as `memory_store`, `context_memory`, and `chat_recall`
+
+It provides:
+
+- unified memory API: `resolve`, `write`, `compact`
+- feature-flag control plane
+- backend adapter layer, with the legacy conversation memory backend first
+- future optional plugin points for vector memory, extraction, lifecycle, behavior modeling, and policy routing
+
+It guarantees:
+
+- zero change to goosed runtime
+- zero change to PG session behavior
+- zero change to SSE protocol
+- zero change to core Portal execution flow
+- zero blocking on memory backend failure
+- full backward compatibility with the existing `conversation-memory` system
+
+## Current Implementation
+
+The first implementation is intentionally narrow:
+
+- `memory-v2.mjs` defines the facade.
+- `memory-v2-runtime.mjs` wires server runtime backends from explicit environment variables.
+- `memory-v2-plugin-backends.mjs` defines disabled-by-default plugin slots for future backends.
+- `memory-v2-pgvector.mjs` defines a disabled-by-default semantic memory adapter skeleton.
+- `memory-v2-pgvector-schema.mjs` defines explicit, manual pgvector schema setup helpers.
+- `createLegacyMemoryBackend(...)` adapts the existing conversation memory service.
+- `resolveMemoryV2Policy(...)` maps feature flags into a stable policy object.
+- `server.mjs` creates Memory V2 from the existing conversation memory service.
+- `tkmind-proxy.mjs` reads user memories through `memoryV2.resolve(...)` before passing them into the existing session reconcile path.
+- `/user-memory/v1/remember-recent` writes through `memoryV2.write(...)`.
+- `/user-memory/v1/sync` compacts through `memoryV2.compact(...)`.
+- Runtime startup does not create schema.
+- No goosed or SSE behavior is changed.
+
+The pgvector adapter does not create tables or generate embeddings. It only defines the adapter contract for a future semantic memory backend and requires explicit `enabled: true`, an injected PostgreSQL pool, and either an input embedding or an injected `embedQuery(...)` function.
+
+The server runtime uses `createMemoryV2Runtime(...)`. It keeps pgvector dormant unless all of these are true:
+
+- `MEMORY_VECTOR_ENABLED=1`
+- `MEMORY_PGVECTOR_DATABASE_URL` is set
+- `MEMORY_PGVECTOR_EMBEDDING_MODULE` points to a module exporting `embedQuery(query, input)` or a default function
+
+If any requirement is missing, pgvector reports `available=false` and Memory V2 falls back to the legacy conversation-memory backend. The runtime never reads the app's MySQL `DATABASE_URL` for pgvector.
+
+When pgvector is available and `MEMORY_BACKEND=pgvector`, only `resolve(...)` uses pgvector. `write(...)` and `compact(...)` continue to use the legacy backend unless a future backend explicitly supports those operations.
+
+Local development may call `ensurePgvectorMemorySchema(...)` manually to create an empty table. Production must treat the same schema as a controlled migration with backup, approval, and a separate backfill plan.
+
+## API Contract
+
+### `resolve(input)`
+
+Reads memory for the current request.
+
+Expected input:
+
+```json
+{
+  "userId": "h5 user id",
+  "sessionId": "goosed session id",
+  "query": "current user prompt",
+  "limit": 40
+}
+```
+
+Returns a normalized payload:
+
+```json
+{
+  "ok": true,
+  "enabled": true,
+  "skipped": false,
+  "source": "legacy-conversation-memory",
+  "profile": null,
+  "semanticMemories": [],
+  "behaviorSummary": null,
+  "activeGoals": [],
+  "memories": []
+}
+```
+
+### `write(input)`
+
+Writes or queues memory-related evidence. In the legacy adapter this maps to `saveAndAnalyze(...)`.
+
+Expected input:
+
+```json
+{
+  "userId": "h5 user id",
+  "sessionId": "goosed session id",
+  "messages": []
+}
+```
+
+### `compact(input)`
+
+Compacts or analyzes pending user memory. In the legacy adapter this maps to `analyzeUser(...)`.
+
+Expected input:
+
+```json
+{
+  "userId": "h5 user id",
+  "sessionId": "goosed session id"
+}
+```
+
+### `getStatus()`
+
+Returns the facade policy and backend contract status for read-only observability.
+
+Example:
+
+```json
+{
+  "enabled": true,
+  "backend": "legacy",
+  "selectedBackend": "legacy-conversation-memory",
+  "profileEnabled": true,
+  "eventLogEnabled": true,
+  "vectorEnabled": false,
+  "failOpen": true,
+  "backends": [
+    {
+      "name": "legacy-conversation-memory",
+      "available": true,
+      "supports": {
+        "resolve": true,
+        "write": true,
+        "compact": true
+      }
+    }
+  ]
+}
+```
+
+## Backend Adapter Contract
+
+Every backend adapter must be optional and fail-open. A backend may implement any subset of the API, but it must not create a new execution path.
+
+Required field:
+
+- `name`: stable backend name used by `MEMORY_BACKEND`
+
+Optional fields:
+
+- `isAvailable()`: returns `false` when the backend is configured but not usable
+- `resolve(input)`: returns profile, semantic memories, behavior summary, active goals, or legacy memories
+- `write(input)`: records or queues memory evidence
+- `compact(input)`: performs pending extraction, summarization, or lifecycle maintenance
+
+Backend implementations must not:
+
+- call goosed directly
+- mutate PG session state
+- change SSE payloads
+- block chat when unavailable
+- assume they are the only memory backend
+
+## Plugin Registry
+
+Memory V2 exposes future plugin slots through read-only backend status. This is a contract registry, not a service integration.
+
+The default facade includes unavailable placeholders for:
+
+| Backend | Category | Role |
+| --- | --- | --- |
+| `pgvector` | semantic | primary vector store |
+| `qdrant` | semantic | scale-out vector store |
+| `weaviate` | semantic | knowledge graph fusion |
+| `mem0` | extraction | automatic memory generation |
+| `letta` | lifecycle | long/short-term memory OS |
+| `neo4j` | behavior | behavior graph |
+| `redis-streams` | behavior | event tracking |
+| `langgraph` | policy | routing/reasoning policy |
+
+These placeholders:
+
+- report `available=false`
+- expose declared capability support in `/api/runtime/status`
+- never open network connections
+- never create tables or queues
+- never become selected while unavailable
+- allow `MEMORY_BACKEND=<future-backend>` to safely fall back to legacy
+
+A real backend adapter replaces its placeholder only when the required client, credentials, schema, and tests are explicitly provided.
+
+Backend selection is operation-aware:
+
+- `resolve` may use a semantic backend such as pgvector.
+- `write` must fall back to the first available backend that supports writes, usually legacy conversation memory.
+- `compact` must fall back to the first available backend that supports compaction, usually legacy conversation memory.
+
+This allows semantic retrieval to be canaried without replacing existing memory capture or compaction.
+
+## Backend Contract Gate
+
+`memory-v2-backend-contract.mjs` defines the adapter contract gate for current and future backends.
+
+Every backend must:
+
+- have a stable lowercase kebab-case `name`
+- implement at least one of `resolve`, `write`, or `compact`
+- expose `isAvailable()` or default to available
+- expose an unavailable reason when `isAvailable() === false`
+- use a known plugin category when `category` is present
+- keep feature flags in `SCREAMING_SNAKE_CASE` when `flag` is present
+
+The default gate checks legacy plus all disabled plugin slots:
+
+```bash
+npm run check:memory-v2-contracts
+```
+
+End-to-end local validation can now be run from one entrypoint:
+
+```bash
+npm run check:memory-v2-stack -- \
+  --base-url http://127.0.0.1:18081 \
+  --expect-backend pgvector \
+  --expect-selected-backend pgvector
+```
+
+This aggregate check runs:
+
+- backend contract validation
+- synthetic Memory V2 health validation
+- app-level `/api/runtime/status` canary
+- real cookie-backed `register/login -> agent/start -> agent/runs -> SSE -> remember-recent -> sync` session flow
+- backend smoke probes for every configured non-legacy backend
+
+Unconfigured external backends are reported explicitly as `not_configured` instead of failing the whole check.
+
+To see exactly which env vars are still missing for any backend, run:
+
+```bash
+npm run check:memory-v2-config -- --backend qdrant
+npm run check:memory-v2-config -- --backend letta --format shell
+```
+
+`json` mode reports missing keys and readiness. `shell` mode prints a copy-paste export template for the selected backend.
+
+When replacing the pgvector placeholder with the real disabled adapter:
+
+```bash
+npm run check:memory-v2-contracts -- --include-pgvector-adapter
+```
+
+Future adapters for Qdrant, Weaviate, Mem0, Letta, Neo4j, Redis Streams, or LangGraph must pass this contract before being wired into `createMemoryV2Runtime(...)`.
+
+## Current Validation Baseline
+
+As of the current Memory V2 rollout branch, local validation has already proven:
+
+- `memoryV2.resolve(...)` can select `pgvector`
+- `memoryV2.write(...)` and `memoryV2.compact(...)` remain legacy-first
+- `/user-memory/v1/remember-recent` and `/user-memory/v1/sync` still reconcile back into the active session
+- the live Portal path `agent/start -> agent/runs -> SSE Finish -> session detail` continues to work under Memory V2
+
+That baseline is now captured by `npm run check:memory-v2-session` and included in `npm run check:memory-v2-stack`.
+
+## Backend Scaffold
+
+`memory-v2-adapter-scaffold.mjs` renders disabled-by-default backend adapter templates that satisfy the contract gate.
+
+Dry-run a Qdrant adapter:
+
+```bash
+npm run scaffold:memory-v2-backend -- \
+  --name qdrant \
+  --category semantic \
+  --role scale-out-vector-store \
+  --capability resolve
+```
+
+Write a Mem0 adapter scaffold:
+
+```bash
+npm run scaffold:memory-v2-backend -- \
+  --name mem0 \
+  --category extraction \
+  --role automatic-memory-generation \
+  --capability write \
+  --capability compact \
+  --write
+```
+
+The scaffold command refuses to overwrite existing files. After generating a real adapter, run:
+
+```bash
+npm run check:memory-v2-contracts
+```
+
+Then add focused adapter tests before wiring the adapter into `createMemoryV2Runtime(...)`.
+
+## Feature Flags
+
+`MEMORY_ENABLED`
+
+Global Memory V2 switch. If unset, it follows the existing `USER_CONVERSATION_MEMORY_ENABLED` behavior for backward compatibility.
+
+`MEMORY_PROFILE_ENABLED`
+
+Controls whether resolved profile payloads may be returned.
+
+`MEMORY_EVENT_LOG_ENABLED`
+
+Controls memory write/event capture. When disabled, `write(...)` returns a skipped result without touching the backend.
+
+`MEMORY_VECTOR_ENABLED`
+
+Reserved for future vector backends. It is disabled by default.
+
+`MEMORY_BACKEND`
+
+Preferred backend name. The current default is `legacy`.
+
+`MEMORY_FAIL_OPEN`
+
+Defaults to enabled. Backend failures return degraded empty payloads rather than blocking chat.
+
+`MEMORY_PGVECTOR_DATABASE_URL`
+
+Dedicated PostgreSQL connection string for Memory V2 semantic memory. It must not point at the MySQL business database and is ignored unless `MEMORY_VECTOR_ENABLED=1`.
+
+`MEMORY_PGVECTOR_EMBEDDING_MODULE`
+
+Optional module path for pgvector runtime retrieval. The module must export `embedQuery(query, input)` or a default async function. Without this module, runtime pgvector retrieval stays unavailable even when a PostgreSQL URL is present.
+
+`MEMORY_PGVECTOR_TABLE`
+
+Optional table name for pgvector retrieval. Defaults to `memory_embeddings`.
+
+`MEMORY_PGVECTOR_POOL_MAX`
+
+Optional PostgreSQL pool size for Memory V2 pgvector retrieval. Defaults to `5`.
+
+## Current Integration Points
+
+Memory V2 is only allowed to sit above existing memory code.
+
+Read path:
+
+```text
+tkmind-proxy
+  -> memoryV2.resolve
+  -> legacy conversation-memory backend
+  -> existing reconcileAgentSession
+  -> existing buildSessionMemoryEntries
+  -> existing harness remember/bootstrap
+```
+
+Write path:
+
+```text
+/user-memory/v1/remember-recent
+  -> memoryV2.write
+  -> legacy conversation-memory saveAndAnalyze
+```
+
+Compact path:
+
+```text
+/user-memory/v1/sync
+  -> memoryV2.compact
+  -> legacy conversation-memory analyzeUser
+```
+
+If `MEMORY_ENABLED=0`, Memory V2 returns skipped/empty results and does not touch the legacy backend. The session reconcile path still runs for non-memory context such as sandbox guidance and time anchors.
+
+Runtime observability:
+
+```text
+/api/runtime/status
+  -> memory: memoryV2.getStatus()
+```
+
+This is read-only and exists only to make rollout/debugging visible.
+
+Local health gate:
+
+```bash
+npm run check:memory-v2-contracts
+npm run check:memory-v2 -- --require-enabled --expect-backend legacy
+```
+
+Local app canary against a running Portal service:
+
+```bash
+npm run canary:memory-v2-app -- \
+  --base-url http://127.0.0.1:8081 \
+  --require-enabled \
+  --expect-backend legacy
+```
+
+For pgvector canary, run the app with explicit Memory V2 env and assert both configured and selected backends:
+
+```bash
+npm run canary:memory-v2-app -- \
+  --base-url http://127.0.0.1:18081 \
+  --require-enabled \
+  --require-target-healthy \
+  --expect-backend pgvector \
+  --expect-selected-backend pgvector
+```
+
+Production rollout details live in [production-rollout-runbook.md](production-rollout-runbook.md).
+
+## Backend Plugin Slots
+
+These are optional future backends. None is required for Memory V2 phase one.
+
+Semantic memory:
+
+- pgvector as the primary vector option
+- Qdrant as a scale-out option
+- Weaviate for knowledge graph fusion
+
+Memory extraction:
+
+- Mem0 for automatic memory generation
+
+Memory lifecycle management:
+
+- Letta for long-term and short-term memory operating semantics
+
+Behavior and user model:
+
+- Neo4j for behavior graph modeling
+- Redis Streams for event tracking
+
+Memory policy engine:
+
+- LangGraph for routing and reasoning policy
+
+## Integration Rule
+
+Future integration must use existing Memind injection points. The preferred path is:
+
+```text
+tkmind-proxy / agent run
+  -> Memory V2 resolve
+  -> existing session reconcile
+  -> existing buildSessionMemoryEntries
+  -> existing harness remember/bootstrap
+```
+
+Do not add direct goosed changes, PG session changes, SSE protocol changes, or new blocking calls in the reply path.
+
+## Rollout Checklist
+
+Phase 1 rollout:
+
+1. Keep `MEMORY_BACKEND=legacy`.
+2. Enable `MEMORY_ENABLED=1`.
+3. Keep `MEMORY_VECTOR_ENABLED=0`.
+4. Verify `/api/runtime/status` reports `memory.enabled=true` and `selectedBackend=legacy-conversation-memory`.
+5. Run targeted tests before release.
+
+Rollback:
+
+```bash
+MEMORY_ENABLED=0
+```
+
+Rollback must stop Memory V2 reads/writes while preserving non-memory session reconcile behavior.
+
+Future backend rollout:
+
+1. Add the backend adapter behind `MEMORY_BACKEND=<name>`.
+2. Replace the matching unavailable placeholder with the real adapter only inside Memory V2 wiring.
+3. Verify `getStatus().backends[]` reports capability support.
+4. Keep backend disabled or unavailable by default.
+5. Run local retrieval/write tests without changing goosed/SSE.
+6. Enable for a narrow canary only after legacy fallback remains green.
+
+## pgvector Adapter Skeleton
+
+`memory-v2-pgvector.mjs` is the first semantic-memory backend skeleton.
+
+Current constraints:
+
+- Disabled by default.
+- Requires an injected PostgreSQL pool with a `query(sql, params)` method.
+- Requires an explicit embedding via `resolve({ embedding })` or an injected `embedQuery(query, input)` function.
+- Server runtime only creates the PostgreSQL pool when vector retrieval, a PostgreSQL URL, and an embedding module are all configured.
+- Performs read-only vector lookup.
+- Does not create schema automatically.
+- Does not migrate or copy MySQL memory rows.
+- Does not call goosed, SSE, or Portal execution APIs.
+- Falls back through Memory V2 when unavailable.
+
+Manual local schema helper:
+
+```js
+import { ensurePgvectorMemorySchema } from './memory-v2-pgvector-schema.mjs';
+
+await ensurePgvectorMemorySchema(pool, {
+  tableName: 'memory_embeddings',
+  dimensions: 1536,
+  createExtension: true,
+  createVectorIndex: false,
+});
+```
+
+Dry-run CLI:
+
+```bash
+node scripts/setup-memory-v2-pgvector-schema.mjs \
+  --table memory_embeddings \
+  --dimensions 1536 \
+  --create-extension
+```
+
+Apply CLI:
+
+```bash
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \
+node scripts/setup-memory-v2-pgvector-schema.mjs \
+  --apply \
+  --table memory_embeddings \
+  --dimensions 1536 \
+  --create-extension
+```
+
+The CLI defaults to dry-run and never reads the existing MySQL `DATABASE_URL`. `--apply` requires an explicit PostgreSQL URL environment variable.
+
+Expected table shape:
+
+```sql
+CREATE TABLE memory_embeddings (
+  id BIGSERIAL PRIMARY KEY,
+  user_id TEXT NOT NULL,
+  content TEXT NOT NULL,
+  embedding VECTOR(1536) NOT NULL,
+  type TEXT NOT NULL DEFAULT 'fact',
+  source_memory_id TEXT,
+  source_session_id TEXT,
+  source_message_id TEXT,
+  metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
+  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
+);
+```
+
+Rollout rule:
+
+Keep `MEMORY_BACKEND=legacy` until pgvector schema, embedding generation, backfill, and canary verification are designed separately.
+
+Production migration considerations:
+
+1. Confirm pgvector is installed on the target PostgreSQL server.
+2. Run `CREATE EXTENSION IF NOT EXISTS vector` only through the approved DB migration path.
+3. Create the empty `memory_embeddings` table first; do not backfill in the same release.
+4. Keep `MEMORY_BACKEND=legacy` after table creation.
+5. Build a separate backfill job from `h5_user_memory_items` into pgvector with idempotent checkpoints.
+6. Verify row counts, embedding dimensions, and retrieval quality before canarying `MEMORY_BACKEND=pgvector`.
+7. Rollback for the app remains `MEMORY_BACKEND=legacy` or `MEMORY_ENABLED=0`; database rollback should not delete populated memory rows without an explicit retention decision.
+
+## pgvector Backfill Skeleton
+
+`memory-v2-pgvector-backfill.mjs` defines the future MySQL-to-pgvector backfill contract.
+
+Source:
+
+```text
+h5_user_memory_items
+  WHERE status = 'active'
+  ORDER BY updated_at ASC, id ASC
+```
+
+Destination:
+
+```text
+memory_embeddings.source_memory_id = h5_user_memory_items.id
+```
+
+The `source_memory_id` unique index makes backfill idempotent. Re-running the same batch updates content, embedding, type, source pointers, metadata, and `updated_at`.
+
+Current constraints:
+
+- Defaults to dry-run.
+- Dry-run reads MySQL only and does not call embedding or PostgreSQL.
+- Apply requires `pgPool.query(...)`.
+- Apply requires an explicit `embedMemory(memory) => number[]`.
+- Backfill is paginated with `{ updatedAt, id }` checkpoints.
+- Backfill does not change `MEMORY_BACKEND`.
+- Backfill does not create schema; run schema setup separately first.
+
+Dry-run CLI:
+
+```bash
+MEMORY_BACKFILL_MYSQL_URL='mysql://...' \
+node scripts/backfill-memory-v2-pgvector.mjs \
+  --limit 100 \
+  --cursor-updated-at 0 \
+  --cursor-id ''
+```
+
+Apply CLI:
+
+```bash
+MEMORY_BACKFILL_MYSQL_URL='mysql://...' \
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \
+node scripts/backfill-memory-v2-pgvector.mjs \
+  --apply \
+  --limit 100 \
+  --embedding-module ./scripts/embed-memory-v2-pgvector.mjs
+```
+
+The apply command requires the embedding module to export `embedMemory(memory)` or a default function. The CLI intentionally uses `MEMORY_BACKFILL_MYSQL_URL` instead of the app's normal MySQL env so production backfills are explicit and auditable.
+
+Production backfill sequence:
+
+1. Run schema migration and keep `MEMORY_BACKEND=legacy`.
+2. Run dry-run batches and inspect counts/checkpoints.
+3. Run apply batches with a fixed embedding model and recorded dimensions.
+4. Store the last checkpoint externally after each successful batch.
+5. Compare MySQL active memory count against pgvector distinct `source_memory_id` count.
+6. Validate retrieval quality on a canary user.
+7. Only then consider `MEMORY_BACKEND=pgvector` canary.
+
+## pgvector Smoke Test
+
+`memory-v2-pgvector-smoke.mjs` and `scripts/smoke-memory-v2-pgvector.mjs` provide a local pgvector read/write smoke test.
+
+The smoke test:
+
+- uses synthetic `memory-v2-smoke-*` rows only
+- inserts deterministic vectors
+- verifies the nearest vector is returned first through the Memory V2 pgvector adapter
+- cleans synthetic rows by default
+- never runs from app startup
+- never reads the MySQL `DATABASE_URL`
+
+Existing-schema smoke:
+
+```bash
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \
+npm run smoke:memory-v2-pgvector
+```
+
+Local setup smoke that also creates an empty schema:
+
+```bash
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \
+npm run smoke:memory-v2-pgvector -- \
+  --create-schema \
+  --create-extension \
+  --table memory_embeddings \
+  --dimensions 3
+```
+
+Production usage is limited to post-migration verification. Do not use the smoke script as a migration mechanism unless the migration plan explicitly approves `--create-schema`.
+
+## Local Canary Env
+
+`docs/memory-v2/local-canary.env.example` contains copyable environment examples for:
+
+- legacy no-op facade canary
+- pgvector local semantic canary
+- Qdrant and Weaviate read-only canaries
+- Mem0, Letta, Neo4j, Redis Streams, and LangGraph external canaries
+
+For local semantic smoke only, `scripts/embed-memory-v2-local-hash.mjs` exports a deterministic hash embedding function:
+
+```bash
+MEMORY_PGVECTOR_EMBEDDING_MODULE=./scripts/embed-memory-v2-local-hash.mjs
+MEMORY_V2_LOCAL_EMBEDDING_DIMENSIONS=3
+```
+
+This module is intentionally simple and deterministic. It is useful for local plumbing checks, but not for production retrieval quality.
+
+## Qdrant Read-Only Adapter
+
+`memory-v2-qdrant.mjs` is the first external plugin adapter beyond pgvector.
+
+Current constraints:
+
+- No Qdrant SDK is imported.
+- The adapter uses a minimal read-only HTTP client wrapper.
+- It only calls Qdrant search when `MEMORY_QDRANT_ENABLED=1`, `MEMORY_QDRANT_URL`, and `MEMORY_QDRANT_EMBEDDING_MODULE` are configured.
+- It never creates collections.
+- It never writes or upserts points.
+- `write(...)` and `compact(...)` continue to use legacy memory.
+
+Runtime read-only configuration:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=qdrant
+MEMORY_QDRANT_ENABLED=1
+MEMORY_QDRANT_URL='http://127.0.0.1:6333'
+MEMORY_QDRANT_COLLECTION='memind_memory'
+MEMORY_QDRANT_EMBEDDING_MODULE='./scripts/embed-memory-v2-qdrant.mjs'
+```
+
+Read-only smoke:
+
+```bash
+MEMORY_QDRANT_URL='http://127.0.0.1:6333' \
+npm run smoke:memory-v2-qdrant -- \
+  --collection memind_memory \
+  --vector 1,0,0 \
+  --limit 1
+```
+
+Expected without embedding module:
+
+```text
+selectedBackend=legacy-conversation-memory
+qdrant.available=false
+qdrant.reason=embedding_module_not_configured
+```
+
+Expected with embedding module and reachable Qdrant:
+
+```text
+selectedBackend=qdrant
+write_uses_legacy=true
+compact_uses_legacy=true
+```
+
+## Mem0 Extraction Adapter
+
+`memory-v2-mem0.mjs` defines the Memory Extraction backend boundary.
+
+Current constraints:
+
+- No Mem0 SDK is imported.
+- Runtime uses a lightweight HTTP client only when `MEMORY_MEM0_ENABLED=1` and `MEMORY_MEM0_API_KEY` are configured.
+- HTTP paths are configurable with `MEMORY_MEM0_WRITE_PATH` and `MEMORY_MEM0_COMPACT_PATH`.
+- Without a full configuration, status reports `mem0.available=false` with a concrete reason and Memory V2 falls back to legacy.
+- `MEMORY_BACKEND=mem0` does not affect `resolve(...)`.
+- `write(...)` and `compact(...)` can use Mem0 when explicitly selected; legacy remains the default and rollback path.
+
+Runtime configuration:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=mem0
+MEMORY_MEM0_ENABLED=1
+MEMORY_MEM0_API_KEY='...'
+MEMORY_MEM0_PROJECT_ID='memind_project'
+MEMORY_MEM0_BASE_URL='https://api.mem0.ai'
+```
+
+Smoke:
+
+```bash
+npm run smoke:memory-v2-external -- --backend mem0 --operation write
+```
+
+## Letta Lifecycle Adapter
+
+`memory-v2-letta.mjs` defines the Memory Lifecycle Management backend boundary.
+
+Current constraints:
+
+- No Letta SDK is imported.
+- Runtime uses a lightweight HTTP client only when `MEMORY_LETTA_ENABLED=1`, `MEMORY_LETTA_API_KEY`, and `MEMORY_LETTA_AGENT_ID` are configured.
+- HTTP paths are configurable with `MEMORY_LETTA_RESOLVE_PATH`, `MEMORY_LETTA_WRITE_PATH`, and `MEMORY_LETTA_COMPACT_PATH`.
+- Without a full configuration, status reports `letta.available=false` with a concrete reason and Memory V2 falls back to legacy.
+- `resolve(...)`, `write(...)`, and `compact(...)` can use Letta when explicitly selected.
+
+Runtime configuration:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=letta
+MEMORY_LETTA_ENABLED=1
+MEMORY_LETTA_API_KEY='...'
+MEMORY_LETTA_PROJECT_ID='memind_project'
+MEMORY_LETTA_AGENT_ID='agent_1'
+MEMORY_LETTA_BASE_URL='https://api.letta.com'
+```
+
+Smoke:
+
+```bash
+npm run smoke:memory-v2-external -- --backend letta --operation resolve
+```
+
+Do not enable Letta as the selected backend in production until sampling, retention, conflict resolution, and rollback rules are defined for long/short-term lifecycle ownership.
+
+## Remaining External Adapters
+
+The remaining optional backends now have runtime-wired clients:
+
+| Backend | File | Category | Runtime role |
+|---|---|---|---|
+| Weaviate | `memory-v2-weaviate.mjs` | semantic | GraphQL vector search |
+| Neo4j | `memory-v2-neo4j.mjs` | behavior | HTTP transaction behavior graph |
+| Redis Streams | `memory-v2-redis-streams.mjs` | behavior | Redis `XADD` event tracking |
+| LangGraph | `memory-v2-langgraph.mjs` | policy | HTTP policy resolve |
+
+Current constraints:
+
+- Weaviate and LangGraph use lightweight HTTP clients.
+- Neo4j uses the HTTP transactional endpoint; `neo4j-driver` is not introduced.
+- Redis Streams uses the existing `redis` dependency.
+- No default runtime path opens network connections; each adapter is dormant unless its feature flag and required config are present.
+- If configured incompletely, each adapter reports `available=false` with a concrete reason and Memory V2 falls back to legacy.
+- `redis-streams` is write-only and cannot affect `resolve(...)`.
+
+Runtime flags:
+
+```bash
+MEMORY_WEAVIATE_ENABLED=1
+MEMORY_WEAVIATE_URL='https://weaviate.example'
+MEMORY_WEAVIATE_COLLECTION='MemindMemory'
+MEMORY_WEAVIATE_EMBEDDING_MODULE='./scripts/embed-memory-v2-weaviate.mjs'
+
+MEMORY_NEO4J_ENABLED=1
+MEMORY_NEO4J_HTTP_URL='http://localhost:7474'
+MEMORY_NEO4J_USER='neo4j'
+MEMORY_NEO4J_PASSWORD='...'
+MEMORY_NEO4J_DATABASE='neo4j'
+
+MEMORY_REDIS_STREAMS_ENABLED=1
+MEMORY_REDIS_STREAMS_URL='redis://localhost:6379'
+MEMORY_REDIS_STREAMS_STREAM='memind:memory-events'
+
+MEMORY_LANGGRAPH_ENABLED=1
+MEMORY_LANGGRAPH_URL='https://langgraph.example'
+MEMORY_LANGGRAPH_POLICY_ID='memory_policy'
+```
+
+Smoke examples:
+
+```bash
+npm run smoke:memory-v2-external -- --backend weaviate --operation resolve
+npm run smoke:memory-v2-external -- --backend neo4j --operation write
+npm run smoke:memory-v2-external -- --backend redis-streams --operation write
+npm run smoke:memory-v2-external -- --backend langgraph --operation resolve
+```
+
+Future production canaries must define per-backend ownership boundaries before any of these adapters can take over `resolve(...)` or `write(...)`.
+
+## Verification
+
+Targeted regression command:
+
+```bash
+node --test memory-v2.test.mjs conversation-memory.test.mjs user-memory-profile.test.mjs tkmind-proxy.test.mjs
+node --test memory-v2-adapter-scaffold.test.mjs
+node --test memory-v2-backend-contract.test.mjs
+node --test memory-v2-health.test.mjs
+node --test memory-v2-runtime.test.mjs
+node --test memory-v2-plugin-backends.test.mjs
+node --test memory-v2-pgvector.test.mjs
+node --test memory-v2-pgvector-schema.test.mjs
+node --test memory-v2-pgvector-backfill.test.mjs
+node --test memory-v2-pgvector-smoke.test.mjs
+node --test memory-v2-qdrant.test.mjs
+node --test memory-v2-mem0.test.mjs
+node --test memory-v2-letta.test.mjs
+node --test memory-v2-external-adapters.test.mjs
+node --test scripts/check-memory-v2-contracts.test.mjs
+node --test scripts/check-memory-v2-health.test.mjs
+node --test scripts/setup-memory-v2-pgvector-schema.test.mjs
+node --test scripts/backfill-memory-v2-pgvector.test.mjs
+node --test scripts/scaffold-memory-v2-backend.test.mjs
+node --test scripts/smoke-memory-v2-pgvector.test.mjs
+node --test scripts/smoke-memory-v2-qdrant.test.mjs
+node --test scripts/smoke-memory-v2-external.test.mjs
+node --check server.mjs
+```
+
+The targeted tests cover:
+
+- legacy backend adaptation
+- backend adapter scaffold generation
+- backend adapter contract gate
+- local Memory V2 health gate checks
+- runtime pgvector wiring only when explicitly configured
+- configured backend selection and fallback
+- future plugin slot visibility without runtime selection
+- profile suppression via policy
+- fail-open resolve behavior
+- event-log write gating
+- session start memory injection through Memory V2
+- disabled Memory V2 not touching legacy memory or injecting stored memories
+- runtime status exposing Memory V2 policy and backend contract
+- reply path passing the current user prompt into `resolve(...)` while keeping the existing `/reply` execution path
+- pgvector adapter disabled-by-default behavior and parameterized semantic lookup
+- Qdrant read-only HTTP client, smoke CLI, and fallback behavior
+- Mem0 extraction HTTP client and runtime fallback behavior
+- Letta lifecycle HTTP client and runtime fallback behavior
+- Weaviate, Neo4j, Redis Streams, and LangGraph runtime client behavior
+- manual pgvector schema SQL generation and execution ordering
+- pgvector schema CLI dry-run and explicit apply guard
+- pgvector backfill dry-run, checkpoint pagination, and idempotent upsert contract
+- pgvector backfill CLI dry-run and explicit apply guard
+- pgvector local smoke fixture and CLI guards
@@ -0,0 +1,69 @@
+# Memory V2 local canary example.
+# Copy the variables you need into your local shell or .env.
+# Do not use the local hash embedding module for production retrieval quality.
+
+MEMORY_ENABLED=1
+MEMORY_FAIL_OPEN=1
+
+# Legacy-safe default canary.
+MEMORY_BACKEND=legacy
+
+# pgvector local canary.
+# MEMORY_BACKEND=pgvector
+# MEMORY_VECTOR_ENABLED=1
+# Homebrew PostgreSQL local default on John's machine:
+# MEMORY_PGVECTOR_DATABASE_URL=postgresql://john@127.0.0.1:5432/memind_memory
+# Password-based PostgreSQL example:
+# MEMORY_PGVECTOR_DATABASE_URL=postgresql://postgres:postgres@127.0.0.1:5432/memind_memory
+# MEMORY_PGVECTOR_TABLE=memory_embeddings
+# MEMORY_PGVECTOR_EMBEDDING_MODULE=./scripts/embed-memory-v2-local-hash.mjs
+# MEMORY_V2_LOCAL_EMBEDDING_DIMENSIONS=3
+
+# Qdrant read-only canary.
+# MEMORY_BACKEND=qdrant
+# MEMORY_QDRANT_ENABLED=1
+# MEMORY_QDRANT_URL=http://127.0.0.1:6333
+# MEMORY_QDRANT_COLLECTION=memind_memory
+# MEMORY_QDRANT_EMBEDDING_MODULE=./scripts/embed-memory-v2-local-hash.mjs
+
+# Weaviate read-only canary.
+# MEMORY_BACKEND=weaviate
+# MEMORY_WEAVIATE_ENABLED=1
+# MEMORY_WEAVIATE_URL=http://127.0.0.1:8080
+# MEMORY_WEAVIATE_COLLECTION=MemindMemory
+# MEMORY_WEAVIATE_EMBEDDING_MODULE=./scripts/embed-memory-v2-local-hash.mjs
+
+# Mem0 write/compact canary.
+# MEMORY_BACKEND=mem0
+# MEMORY_MEM0_ENABLED=1
+# MEMORY_MEM0_API_KEY=replace-me
+# MEMORY_MEM0_PROJECT_ID=memind_project
+# MEMORY_MEM0_BASE_URL=https://api.mem0.ai
+
+# Letta lifecycle canary.
+# MEMORY_BACKEND=letta
+# MEMORY_LETTA_ENABLED=1
+# MEMORY_LETTA_API_KEY=replace-me
+# MEMORY_LETTA_AGENT_ID=agent_1
+# MEMORY_LETTA_PROJECT_ID=memind_project
+# MEMORY_LETTA_BASE_URL=https://api.letta.com
+
+# Neo4j behavior graph canary.
+# MEMORY_BACKEND=neo4j
+# MEMORY_NEO4J_ENABLED=1
+# MEMORY_NEO4J_HTTP_URL=http://127.0.0.1:7474
+# MEMORY_NEO4J_USER=neo4j
+# MEMORY_NEO4J_PASSWORD=replace-me
+# MEMORY_NEO4J_DATABASE=neo4j
+
+# Redis Streams event canary.
+# MEMORY_BACKEND=redis-streams
+# MEMORY_REDIS_STREAMS_ENABLED=1
+# MEMORY_REDIS_STREAMS_URL=redis://127.0.0.1:6379
+# MEMORY_REDIS_STREAMS_STREAM=memind:memory-events
+
+# LangGraph policy canary.
+# MEMORY_BACKEND=langgraph
+# MEMORY_LANGGRAPH_ENABLED=1
+# MEMORY_LANGGRAPH_URL=http://127.0.0.1:2024
+# MEMORY_LANGGRAPH_POLICY_ID=memory_policy
@@ -0,0 +1,335 @@
+# Memory V2 Production Rollout Runbook
+
+Memory V2 is a facade and policy layer. Production rollout must not introduce new goosed, PG session, SSE, or Portal execution paths.
+
+## Release Gates
+
+Run the normal repository release checks first:
+
+```bash
+bash scripts/check-release-ready.sh
+```
+
+Run the Memory V2 backend contract gate:
+
+```bash
+npm run check:memory-v2-contracts
+```
+
+Run the Memory V2 health gate:
+
+```bash
+npm run check:memory-v2 -- --require-enabled --expect-backend legacy
+```
+
+Expected phase-one result:
+
+- `ok=true`
+- contract gate reports unique backend names and legacy backend present
+- `summary.enabled=true`
+- `summary.backend=legacy`
+- `summary.selectedBackend=legacy-conversation-memory`
+- `fail_open` passes
+- `legacy_write_compact_supported` passes
+- `write_uses_legacy` passes
+- `compact_uses_legacy` passes
+
+When validating the real pgvector adapter before canary:
+
+```bash
+npm run check:memory-v2-contracts -- --include-pgvector-adapter
+```
+
+Future backend adapter rule:
+
+```bash
+npm run scaffold:memory-v2-backend -- \
+  --name qdrant \
+  --category semantic \
+  --capability resolve
+```
+
+Use the scaffold as the starting point, add focused adapter tests, then pass `npm run check:memory-v2-contracts` before wiring any external SDK into runtime. Do not wire external services directly into `server.mjs`, `tkmind-proxy.mjs`, goosed, SSE, or Portal execution code.
+
+## Qdrant Read-Only Phase
+
+The Qdrant adapter may be visible in runtime status before user traffic is canaried.
+
+Fallback-only environment:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=qdrant
+MEMORY_QDRANT_ENABLED=1
+MEMORY_QDRANT_URL='http://qdrant.example:6333'
+MEMORY_QDRANT_COLLECTION='memind_memory'
+```
+
+Expected without embedding module:
+
+- `memory.backend=qdrant`
+- `memory.selectedBackend=legacy-conversation-memory`
+- `qdrant.available=false`
+- `qdrant.reason=embedding_module_not_configured`
+- `write_uses_legacy` still passes
+- `compact_uses_legacy` still passes
+
+Read-only smoke:
+
+```bash
+MEMORY_QDRANT_URL='http://qdrant.example:6333' \
+npm run smoke:memory-v2-qdrant -- \
+  --collection memind_memory \
+  --vector 1,0,0 \
+  --limit 1
+```
+
+The smoke command sends a Qdrant search request only. It must not create collections, write points, or run from app startup.
+
+Runtime read-only canary requires:
+
+```bash
+MEMORY_QDRANT_EMBEDDING_MODULE='./scripts/embed-memory-v2-qdrant.mjs'
+```
+
+Even during read-only canary, `write_uses_legacy` and `compact_uses_legacy` must still pass.
+
+## Mem0 Adapter Phase
+
+Mem0 is the Memory Extraction backend boundary. The adapter uses HTTP and is dormant unless explicitly configured.
+
+Environment:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=mem0
+MEMORY_MEM0_ENABLED=1
+MEMORY_MEM0_API_KEY='...'
+MEMORY_MEM0_PROJECT_ID='memind_project'
+MEMORY_MEM0_BASE_URL='https://api.mem0.ai'
+```
+
+Smoke:
+
+```bash
+npm run smoke:memory-v2-external -- --backend mem0 --operation write
+```
+
+Do not canary Mem0 writes until sampling, dedupe, rollback, and retention rules are accepted.
+
+## Letta Lifecycle Adapter Phase
+
+Letta is the Memory Lifecycle Management backend boundary. The adapter uses HTTP and is dormant unless explicitly configured.
+
+Environment:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=letta
+MEMORY_LETTA_ENABLED=1
+MEMORY_LETTA_API_KEY='...'
+MEMORY_LETTA_PROJECT_ID='memind_project'
+MEMORY_LETTA_AGENT_ID='agent_1'
+MEMORY_LETTA_BASE_URL='https://api.letta.com'
+```
+
+Smoke:
+
+```bash
+npm run smoke:memory-v2-external -- --backend letta --operation resolve
+```
+
+Do not canary Letta until ownership boundaries, long/short-term promotion rules, conflict resolution with conversation-memory, rollback, and retention are accepted.
+
+## Remaining External Adapter Phase
+
+Weaviate, Neo4j, Redis Streams, and LangGraph are wired with runtime clients and remain dormant unless explicitly configured.
+
+Environment examples:
+
+```bash
+MEMORY_WEAVIATE_ENABLED=1
+MEMORY_WEAVIATE_URL='https://weaviate.example'
+MEMORY_WEAVIATE_COLLECTION='MemindMemory'
+MEMORY_WEAVIATE_EMBEDDING_MODULE='./scripts/embed-memory-v2-weaviate.mjs'
+
+MEMORY_NEO4J_ENABLED=1
+MEMORY_NEO4J_HTTP_URL='http://localhost:7474'
+MEMORY_NEO4J_USER='neo4j'
+MEMORY_NEO4J_PASSWORD='...'
+
+MEMORY_REDIS_STREAMS_ENABLED=1
+MEMORY_REDIS_STREAMS_URL='redis://localhost:6379'
+
+MEMORY_LANGGRAPH_ENABLED=1
+MEMORY_LANGGRAPH_URL='https://langgraph.example'
+MEMORY_LANGGRAPH_POLICY_ID='memory_policy'
+```
+
+Smoke:
+
+```bash
+npm run smoke:memory-v2-external -- --backend weaviate --operation resolve
+npm run smoke:memory-v2-external -- --backend neo4j --operation write
+npm run smoke:memory-v2-external -- --backend redis-streams --operation write
+npm run smoke:memory-v2-external -- --backend langgraph --operation resolve
+```
+
+Future canaries must be backend-specific and must define data ownership, dedupe, observability, rollback, and retention before any adapter can own a Memory V2 operation.
+
+## Phase 1: Facade Over Legacy
+
+Environment:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=legacy
+MEMORY_FAIL_OPEN=1
+MEMORY_VECTOR_ENABLED=0
+```
+
+Runtime verification:
+
+```bash
+curl -sk https://<host>/api/runtime/status
+```
+
+Confirm:
+
+- `memory.enabled=true`
+- `memory.backend=legacy`
+- `memory.selectedBackend=legacy-conversation-memory`
+- future plugin slots report `available=false`
+
+Rollback:
+
+```bash
+MEMORY_ENABLED=0
+```
+
+## Phase 2: Empty pgvector Schema
+
+This phase creates schema only. It must not change runtime backend selection.
+
+Preconditions:
+
+- PostgreSQL server is approved for Memory V2 semantic memory.
+- `pgvector` extension is available.
+- Backup/rollback decision is recorded by ops.
+
+Dry-run:
+
+```bash
+node scripts/setup-memory-v2-pgvector-schema.mjs \
+  --table memory_embeddings \
+  --dimensions 1536 \
+  --create-extension
+```
+
+Apply only through the approved production migration path:
+
+```bash
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \
+node scripts/setup-memory-v2-pgvector-schema.mjs \
+  --apply \
+  --table memory_embeddings \
+  --dimensions 1536 \
+  --create-extension
+```
+
+Keep runtime on legacy:
+
+```bash
+MEMORY_BACKEND=legacy
+MEMORY_VECTOR_ENABLED=0
+```
+
+Post-migration smoke verification:
+
+```bash
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \
+npm run smoke:memory-v2-pgvector -- \
+  --table memory_embeddings \
+  --dimensions 1536
+```
+
+The smoke command inserts synthetic `memory-v2-smoke-*` rows, verifies nearest-vector retrieval, and deletes those rows by default. In production it should run only after the schema migration. Do not pass `--create-schema` in production unless the approved migration procedure explicitly requires it.
+
+For local-only plumbing checks, use the deterministic hash embedding module:
+
+```bash
+MEMORY_PGVECTOR_EMBEDDING_MODULE=./scripts/embed-memory-v2-local-hash.mjs
+MEMORY_V2_LOCAL_EMBEDDING_DIMENSIONS=3
+```
+
+Do not use this local hash embedding module for production retrieval quality.
+
+## Phase 3: Backfill
+
+Backfill must be separate from schema creation and application rollout.
+
+Dry-run:
+
+```bash
+MEMORY_BACKFILL_MYSQL_URL='mysql://...' \
+node scripts/backfill-memory-v2-pgvector.mjs --limit 100
+```
+
+Apply:
+
+```bash
+MEMORY_BACKFILL_MYSQL_URL='mysql://...' \
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...' \
+node scripts/backfill-memory-v2-pgvector.mjs \
+  --apply \
+  --limit 100 \
+  --embedding-module ./scripts/embed-memory-v2-pgvector.mjs
+```
+
+Operational rules:
+
+- Store `{ updatedAt, id }` checkpoint externally after each successful batch.
+- Compare active MySQL memory count with pgvector distinct `source_memory_id` count.
+- Do not change `MEMORY_BACKEND` during backfill.
+- Do not delete pgvector rows during app rollback without an explicit retention decision.
+
+## Phase 4: pgvector Resolve Canary
+
+Only start canary after schema, backfill, and retrieval quality checks pass.
+
+Environment:
+
+```bash
+MEMORY_ENABLED=1
+MEMORY_BACKEND=pgvector
+MEMORY_VECTOR_ENABLED=1
+MEMORY_FAIL_OPEN=1
+MEMORY_PGVECTOR_DATABASE_URL='postgresql://...'
+MEMORY_PGVECTOR_EMBEDDING_MODULE='./scripts/embed-memory-v2-pgvector.mjs'
+```
+
+Health gate:
+
+```bash
+npm run check:memory-v2 -- --require-enabled --expect-backend pgvector
+```
+
+Required invariant:
+
+- `selectedBackend=pgvector` is allowed for `resolve`.
+- `write_uses_legacy` must still pass.
+- `compact_uses_legacy` must still pass.
+- Any pgvector error must fail open to legacy or degraded empty resolve.
+
+Rollback:
+
+```bash
+MEMORY_BACKEND=legacy
+MEMORY_VECTOR_ENABLED=0
+```
+
+Emergency rollback:
+
+```bash
+MEMORY_ENABLED=0
+```