memind/services/orchestrator/README.md

Memind Workflow Orchestrator

This directory is the extraction boundary for Memind's durable workflow runtime. It is intentionally source-colocated with Memind while remaining framework-neutral at its public edge.

Dependency rule

Allowed:

Memind -> orchestrator contracts / client
Orchestrator -> contracts / executor adapters
Executor adapters -> versioned internal APIs

Forbidden:

Orchestrator -> server.mjs
Orchestrator -> user-auth.mjs
Orchestrator -> Memind business tables
Orchestrator -> production filesystem paths
Memind -> LangGraph StateGraph / Command / checkpoint types

LangGraph is an implementation of the generic workflow engine contract. The Memind-facing protocol is orchestrator-run-v1 plus orchestrator-event-v1.

Current stage

Phase 1 established:

• framework-neutral run and event contracts;
• a plug-in workflow engine registry;
• a remote workflow engine client;
• versioned admin configuration;
• deterministic Off / Shadow / Canary / Active routing decisions;
• an environment-level emergency kill switch.

Phase 2 adds:

• a real LangGraph StateGraph service on the versioned HTTP boundary;
• a PostgreSQL PostgresSaver checkpoint owner;
• an explicitly non-durable MemorySaver option for tests and local debugging;
• a code-run-v1 observe-only graph with no executor or tool access;
• a Portal fire-and-forget shadow projection;
• live health probing from memindadm;
• a separate Colima/Compose deployment artifact.

Phase 2.5 adds a Memind-owned Shadow observability projection:

• success/failure totals and rates;
• P50/P95 observation latency;
• Native terminal status beside the Shadow result;
• recent failure codes and messages;
• per-run LangGraph checkpoint and node-event inspection through the versioned remote API.

The projection reads h5_agent_run_events from the Memind control plane. LangGraph still has no access to Memind business tables.

Phase 2.6 adds a read-only Canary readiness gate. It evaluates a fixed seven-day window and never changes the runtime mode. The first gate requires:

• at least 20 non-smoke observations across at least five real sessions;
• at least 95% Shadow success, latency coverage, and Native terminal coverage;
• Shadow P95 latency no greater than two seconds;
• a healthy observe-only service backed by a durable checkpoint;
• a fresh, uncapped metric window while the control plane remains in Shadow.

Passing the gate means only manual_canary_review; it does not authorize or activate Canary routing.

The default memindadm mode remains off. Canary and Active routing are not wired to the LangGraph executor in Phase 2. Native Agent Run remains the sole executor, including in Shadow mode.

Admin configuration

The memindadm page is /ops/admin/orchestrator.

Supported modes:

• off: Native Agent Run only.
• shadow: Native executes; the configured engine may observe.
• canary: user allowlist and deterministic rollout percentage.
• active: the primary engine handles workflow-allowlisted tasks.

MEMIND_ORCHESTRATOR_KILL_SWITCH=1 always forces Native selection.

Local process

PostgreSQL is the default and required checkpoint backend:

MEMIND_ORCHESTRATOR_DATABASE_URL='postgresql://...' \
MEMIND_ORCHESTRATOR_SERVICE_TOKEN='...' \
pnpm dev:orchestrator

Memory mode is intentionally opt-in and non-durable:

MEMIND_ORCHESTRATOR_CHECKPOINT_MODE=memory pnpm dev:orchestrator

The service binds to 127.0.0.1:8093 by default. Configure the same URL and service token in Portal, then enable shadow in /ops/admin/orchestrator.

memindadm exposes the projection through:

GET /admin-api/orchestrator/shadow-runs
GET /admin-api/orchestrator/shadow-runs/:runId
GET /admin-api/orchestrator/canary-readiness

Colima deployment

Colima is the recommended first container host on macOS because this service and its PostgreSQL checkpoint database have an isolated Compose lifecycle:

cd deploy/orchestrator
cp .env.example .env
# Replace both example secrets before starting.
docker compose up -d --build
docker compose ps
curl http://127.0.0.1:8093/ready

When Portal or the Shadow smoke process runs outside Compose, export the same service token into the child process environment. Plain source does not export new shell variables by itself:

set -a
source deploy/orchestrator/.env
set +a
export MEMIND_ORCHESTRATOR_URL=http://127.0.0.1:8093
pnpm dev

The Compose project:

• publishes only the Orchestrator port on loopback;
• does not publish PostgreSQL;
• does not mount the Docker socket or a Memind workspace;
• keeps PostgreSQL on a dedicated internal network while the service also joins a separate edge network for its loopback-published port;
• is not part of the Goosed Compose lifecycle.

This local Compose artifact is not a production release path. Production still requires the repository release gates and a separately approved deployment.

Runtime environment

Variable	Purpose	Default
MEMIND_ORCHESTRATOR_HOST	HTTP bind address	127.0.0.1
MEMIND_ORCHESTRATOR_PORT	HTTP port	8093
MEMIND_ORCHESTRATOR_SERVICE_TOKEN	Bearer token for /v1/*	empty
MEMIND_ORCHESTRATOR_CHECKPOINT_MODE	postgres or explicit memory	postgres
MEMIND_ORCHESTRATOR_DATABASE_URL	Dedicated checkpoint PostgreSQL URL	required
MEMIND_ORCHESTRATOR_DATABASE_SCHEMA	Checkpoint schema	memind_orchestrator

Extraction test

The service is ready to move into a separate repository only when:

1. it can run against a mock Memind control plane;
2. it owns its checkpoint database and credentials;
3. task inputs use resource references rather than host paths;
4. executor calls use versioned adapters;
5. moving the service requires only changing ORCHESTRATOR_URL.