docs: finalize orchestrator phase 5 operations
This commit is contained in:
@@ -54,19 +54,19 @@ Native selection.
|
||||
The initial workflow allowlist contains only `code-run-v1`. Ordinary chat and
|
||||
existing Goosed session traffic remain outside the Orchestrator path.
|
||||
|
||||
Phase 2 implements only the `off` and `shadow` execution semantics. Although
|
||||
Canary and Active can be configured and evaluated by the control plane, the
|
||||
Portal does not hand execution ownership to LangGraph yet. This prevents an
|
||||
administrative configuration mistake from creating two task executors.
|
||||
Phase 2 implemented only the `off` and `shadow` execution semantics. Canary and
|
||||
Active were initially planning-only so an administrative configuration mistake
|
||||
could not create two task executors.
|
||||
|
||||
Phase 3 makes that restriction explicit in the contract. Routing returns both
|
||||
`candidateEngine` and `engine`: Canary or Active may nominate LangGraph, while
|
||||
`engine` remains Native. The decision is projected as
|
||||
`workflow_execution_planned` with `dryRun=true` and `handoffAllowed=false`.
|
||||
The execution adapter normalizes authorization, idempotency, timeout,
|
||||
cancellation, and fallback controls, but its non-Native dispatch path is
|
||||
hard-disabled in code. Changing memindadm configuration or setting an
|
||||
environment override cannot transfer execution ownership.
|
||||
cancellation, and fallback controls. Phase 5 can transfer ownership only when
|
||||
the memindadm switch, Portal environment gate, Orchestrator environment gate,
|
||||
adapter enablement, worker health, authorization, and admission policy all
|
||||
agree. Any missing gate keeps Native effective.
|
||||
|
||||
The Dry-run projection records every code-run decision while the configured
|
||||
mode is Shadow, Canary, or Active, including `canary_not_selected`. In Shadow,
|
||||
@@ -96,11 +96,11 @@ authorization and side-effect policy, timeout, cancellation, fallback, and an
|
||||
idempotency key. The store interface requires atomic create-if-absent semantics
|
||||
so retries return the original job and conflicting payloads fail closed.
|
||||
|
||||
Goosed, Aider, and OpenHands currently appear only as disabled
|
||||
`contract-only` adapters. The reference in-memory store is non-durable and is
|
||||
not a deployment backend. The compile-time dispatch capability remains false,
|
||||
so creating a job records `blocked` state without importing an executor SDK,
|
||||
launching a process, or touching a workspace.
|
||||
Goosed, Aider, and OpenHands began as disabled `contract-only` adapters. The
|
||||
reference in-memory store remains non-durable and is not a deployment backend.
|
||||
In the current Phase 5 implementation the dispatch capability exists, but
|
||||
execution and every adapter remain disabled by default. Shadow jobs still record
|
||||
terminal `blocked` state without launching a process or touching a workspace.
|
||||
|
||||
Phase 3.3 persists that state in the Orchestrator-owned PostgreSQL database.
|
||||
LangGraph checkpoint tables and `executor_jobs` share the isolated
|
||||
@@ -132,10 +132,11 @@ GET /v1/executor-jobs/:jobId
|
||||
GET /v1/executor-jobs/:jobId/events?after=<cursor>&limit=<limit>
|
||||
```
|
||||
|
||||
There is no HTTP submit, retry, claim, or executor cancel endpoint in this
|
||||
phase. memindadm resolves the job reference stored in LangGraph plan state and
|
||||
shows the job snapshot and events beside Native and graph state. The payload
|
||||
contains no task instruction, secret, absolute path, or executor SDK object.
|
||||
Phase 3.4 had no HTTP submit, retry, claim, or executor cancel endpoint.
|
||||
Phase 3.5 adds a separately authenticated worker protocol. The control-plane
|
||||
read projection still contains no task instruction, secret, absolute path, lease
|
||||
token, or executor SDK object; only a successful worker claim receives the
|
||||
normalized request and lease token.
|
||||
|
||||
The implemented internal API is:
|
||||
|
||||
@@ -153,16 +154,44 @@ External state changes must use an idempotency key derived from run, graph
|
||||
version, node, and attempt. Graph state stores resource references and decisions,
|
||||
not API keys, large logs, binary artifacts, or absolute production paths.
|
||||
|
||||
The Phase 2 graph has three deterministic nodes:
|
||||
The graph keeps three deterministic control-plane nodes:
|
||||
|
||||
```text
|
||||
validate_run -> build_plan -> finalize_run
|
||||
```
|
||||
|
||||
It requires `policy.executionMode=observe-only` and
|
||||
`policy.sideEffectsAllowed=false`. It projects the Native executor boundary and
|
||||
persists a blocked Executor Job, but cannot dispatch Goosed, Aider, OpenHands,
|
||||
Tool Gateway, or the filesystem.
|
||||
Observe-only runs require `sideEffectsAllowed=false`, project the Native
|
||||
boundary, and persist a blocked Executor Job. Explicit active runs require
|
||||
`sideEffectsAllowed=true` plus every execution gate; they queue a job and enter
|
||||
`waiting` until the worker-owned job reaches a terminal state. LangGraph never
|
||||
mounts a workspace or imports an executor SDK.
|
||||
|
||||
## Phase 5 worker and loop boundary
|
||||
|
||||
The task loop is split across durable owners:
|
||||
|
||||
```text
|
||||
LangGraph run: validate -> queue -> wait -> project terminal
|
||||
Executor Job: queued -> leased -> running -> terminal/retryable
|
||||
Worker: claim -> heartbeat -> adapter -> artifact references
|
||||
```
|
||||
|
||||
PostgreSQL row locks and lease tokens fence duplicate claims and late worker
|
||||
updates. Expired leases recover to `retryable` until `maxAttempts` is exhausted,
|
||||
then become `timed_out`. Worker heartbeats are persisted independently of job
|
||||
heartbeats so readiness and autoscaling signals also work while the queue is
|
||||
empty.
|
||||
|
||||
Goosed is a remote HTTP/SSE adapter. Aider and OpenHands are process adapters
|
||||
intended to run in separate worker containers with workspace aliases, an
|
||||
allowlisted writable root, no shell, bounded environment/output, resource
|
||||
limits, and no Docker socket. All adapter outputs cross the boundary as bounded
|
||||
events and artifact references.
|
||||
|
||||
Admission is defense in depth: tenant/user/workspace allowlists, global and
|
||||
per-subject concurrency, and per-minute limits are evaluated before queueing.
|
||||
The public metrics endpoint exposes queue states, claimable jobs, expired
|
||||
leases, worker health, and execution-gate state without task contents.
|
||||
|
||||
## Failure isolation
|
||||
|
||||
|
||||
Reference in New Issue
Block a user