diff --git a/.claude/skills/portal-release/SKILL.md b/.claude/skills/portal-release/SKILL.md index 66edbfb..b3def00 100644 --- a/.claude/skills/portal-release/SKILL.md +++ b/.claude/skills/portal-release/SKILL.md @@ -74,12 +74,12 @@ curl -k -i https://m.tkmind.cn/auth/login \ 必须是 **401** + `用户名或密码错误`。如果是 `503` + `未配置用户数据库或访问密码`,说明用户系统 bootstrap 失败——不是简单健康检查能发现的问题,必须当场处理,不能算发布成功。 -### 3.2 Portal + 四个 goosed 实例(SSH 103) +### 3.2 Portal + 九个 goosed 实例(SSH 103) ```bash ssh john@58.38.22.103 ' printf "portal="; curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8081/api/status - for p in 18006 18007 18008 18009; do + for p in $(seq 18006 18014); do printf "goosed_${p}=" curl -k -s -o /dev/null -w "%{http_code}\n" https://127.0.0.1:${p}/status done @@ -89,7 +89,7 @@ ssh john@58.38.22.103 ' ' ``` -必须全部是 `200`,五个 env var 全部 `SET`。 +必须全部是 `200`(Portal + 9 个 goosed target),五个 env var 全部 `SET`。 ### 3.3 日志关键词检查(SSH 103) @@ -164,7 +164,7 @@ ssh john@58.38.22.103 ' - persisted backup 路径 - Portal 健康结果(200/非200) - `/auth/login` 结果(401 + 正确消息,还是别的) -- 18006-18009 健康结果 +- `18006`–`18014` 健康结果(共 9 个 target) - 日志关键词检查结果 - 第 3.4 节真实业务路径验证结果(含遗留的测试 page id,交给用户决定是否清理) - 是否发现并处理了问题;如果发布后发现回滚,说明触发原因 diff --git a/docs/103-runtime-topology.md b/docs/103-runtime-topology.md index b3335b9..f66b1f5 100644 --- a/docs/103-runtime-topology.md +++ b/docs/103-runtime-topology.md @@ -1,6 +1,6 @@ # Local and 103 runtime topology -> Last confirmed: 2026-07-03 15:36 CST. +> Last confirmed: 2026-07-07 09:19 CST. > > This is the current topology source of truth for local Memind and 103 production. Prefer this document over older migration notes. Older architecture documents may contain historical paths from before the MindSpace split. @@ -19,7 +19,7 @@ Local Memind connects to native `goosed`, not Colima/Docker `goosed`. Local ports `18007`-`18009` are not the intended local goosed pool. If they are listening locally, treat that as drift unless a specific local test explicitly opened them. -Local Colima may exist for other workloads, but local Memind/goosed is not supposed to route through the 103-style `goosed-prod-1..4` Colima pool. +Local Colima may exist for other workloads, but local Memind/goosed is not supposed to route through the 103-style `goosed-prod-1..9` Colima pool. ## Portal @@ -31,6 +31,7 @@ Local Colima may exist for other workloads, but local Memind/goosed is not suppo | Start script | `/Users/john/Project/Memind/scripts/run-memind-portal-prod.sh` | | Port | `8081` | | Health check | `curl http://127.0.0.1:8081/api/status` | +| Upstream goosed pool | `TKMIND_API_TARGETS` → `https://127.0.0.1:18006` … `18014`(9 targets,2026-07-07 确认) | Portal must be released as a runtime artifact. Do not edit source directly on 103. @@ -70,7 +71,7 @@ ssh john@58.38.22.103 '/opt/homebrew/opt/postgresql@17/bin/pg_isready -h 127.0.0 `cn.tkmind.goosed-monitor`(103)运行 `goosed-prod/scripts/monitor-goosed-containers.mjs`,本地 native goosed 监控用 `scripts/monitor-goosed-fds.mjs`;两者都会在巡检时调用 `ensure-goose-session-postgres.sh`。 -Container layout: +Container layout (host port → container `18006`): | Container | Host port | Container port | |-----------|-----------|----------------| @@ -78,8 +79,13 @@ Container layout: | `goosed-prod-2` | `18007` | `18006` | | `goosed-prod-3` | `18008` | `18006` | | `goosed-prod-4` | `18009` | `18006` | +| `goosed-prod-5` | `18010` | `18006` | +| `goosed-prod-6` | `18011` | `18006` | +| `goosed-prod-7` | `18012` | `18006` | +| `goosed-prod-8` | `18013` | `18006` | +| `goosed-prod-9` | `18014` | `18006` | -All four ports must return `ok` from `/status`. +All nine host ports must return `ok` from `/status`, and must appear in Portal `.env` `TKMIND_API_TARGETS`. ## PostgreSQL rule @@ -119,5 +125,6 @@ ssh john@58.38.22.103 '/opt/homebrew/bin/docker inspect goosed-prod-1 --format " ```bash ssh john@58.38.22.103 'curl -fsS http://127.0.0.1:8081/api/status' ssh john@58.38.22.103 'curl -fsS http://127.0.0.1:8082/health' -ssh john@58.38.22.103 'for p in 18006 18007 18008 18009; do curl -kfsS https://127.0.0.1:$p/status; echo; done' +ssh john@58.38.22.103 'for p in $(seq 18006 18014); do curl -kfsS https://127.0.0.1:$p/status; echo; done' +ssh john@58.38.22.103 'grep ^TKMIND_API_TARGETS= /Users/john/Project/Memind/.env' ``` diff --git a/docs/architecture/memind-2-runtime-execution-assessment-20260702.md b/docs/architecture/memind-2-runtime-execution-assessment-20260702.md index dff69db..21be803 100644 --- a/docs/architecture/memind-2-runtime-execution-assessment-20260702.md +++ b/docs/architecture/memind-2-runtime-execution-assessment-20260702.md @@ -688,7 +688,7 @@ Data boundary: - LaunchAgent 参数包含 `--prune --retention-days 30` - 只清理 `reports/runtime-slo` 顶层过期 `.json/.md` - 最新报告 `reportPrune.errors=[]` -- P5.14 已给四个 goosed worker 增加常驻 sidecar heartbeat: +- P5.14 已给全部 goosed worker(当前 9 个,`18006..18014`)增加常驻 sidecar heartbeat: - `goosed-1`: `statusCode=200`, `latencyMs=62` - `goosed-2`: `statusCode=200`, `latencyMs=46` - `goosed-3`: `statusCode=200`, `latencyMs=46` diff --git a/docs/architecture/memind-2-streaming-agent-runtime-plan.md b/docs/architecture/memind-2-streaming-agent-runtime-plan.md index 3a1c941..ac38ee8 100644 --- a/docs/architecture/memind-2-streaming-agent-runtime-plan.md +++ b/docs/architecture/memind-2-streaming-agent-runtime-plan.md @@ -54,8 +54,8 @@ - Memind Portal: `/Users/john/Project/Memind/server.mjs`,本机 `:8081`。 - H5 public base: `https://mm.tkmind.cn`;`m.tkmind.cn` 暂时不再作为 H5 public base。 -- goosed worker pool: Docker/Colima 内 `goosed-prod-1..4`,宿主机 `18006..18009`,当前健康。 -- H5 上游配置: `TKMIND_API_TARGETS=https://127.0.0.1:18006,...,18009`。 +- goosed worker pool: Docker/Colima 内 `goosed-prod-1..9`,宿主机 `18006..18014`(2026-07-07 确认 9 个 healthy)。 +- H5 上游配置: `TKMIND_API_TARGETS=https://127.0.0.1:18006,...,18014`(须与 compose 容器数一致)。 - 当前已有 session affinity: start 时分配 worker,后续 reply/events 回到同一 worker。 - imgproxy: 原生 `127.0.0.1:20082`,兼容代理 `10.10.0.2:20081`。 - 风险点: SSE 代理尚未完整处理 `flushHeaders`、`X-Accel-Buffering`、客户端断开 abort、写入 backpressure 和统一 pipeline 收尾。 @@ -113,7 +113,7 @@ flowchart LR - `node --check /Users/john/Project/Memind/server.mjs` 通过。 - 本机 `http://127.0.0.1:8081/api/status` 仍返回 `ok`。 -- 四个 `https://127.0.0.1:18006..18009/status` 仍返回 `ok`。 +- 全部 `https://127.0.0.1:18006..18014/status` 仍返回 `ok`。 ### P1: Gateway SSE 配置 @@ -296,7 +296,7 @@ docker exec memind-runtime-redis redis-cli DEL memind:runtime:worker:goosed-3:dr - Portal 新 PID: `94324`,监听 `*:8081`。 - 运行环境确认: `H5_PUBLIC_BASE_URL=https://mm.tkmind.cn`,`VITE_MINDSPACE_BASE=https://mm.tkmind.cn`。 - 本机 `http://127.0.0.1:8081/api/status` 返回 `ok`。 -- 四个 goosed worker `18006..18009` 均返回 `ok`。 +- 九个 goosed worker `18006..18014` 均返回 `ok`。 - 公网 `https://mm.tkmind.cn/api/status` 当前返回 nginx `502 Bad Gateway`,说明域名侧入口尚未接到当前 Portal;该问题纳入 P1 新入口修复,不再回到 105 转发链路处理。 ### 2026-07-02 P1 Gateway SSE 配置 @@ -376,7 +376,7 @@ docker exec memind-runtime-redis redis-cli DEL memind:runtime:worker:goosed-3:dr - `memind:runtime:worker:goosed-3:heartbeat=` - `memind:runtime:stream:20260701_24:status=active` - `https://mm.tkmind.cn/api/status` 返回 `ok`。 -- 四个 goosed worker `18006..18009` 均返回 `ok`。 +- 九个 goosed worker `18006..18014` 均返回 `ok`。 ### 2026-07-02 P3 Observability v1 @@ -400,7 +400,7 @@ docker exec memind-runtime-redis redis-cli DEL memind:runtime:worker:goosed-3:dr - `/opt/homebrew/opt/node@24/bin/node --check /Users/john/Project/Memind/server.mjs` 通过。 - `https://mm.tkmind.cn/api/runtime/status` 返回 `ok: true`。 - 返回中 `router.enabled=true`,namespace 为 `memind:runtime`。 -- 返回中四个目标 `18006..18009` 均 `healthy: true`。 +- 返回中九个目标 `18006..18014` 均 `healthy: true`。 - `https://mm.tkmind.cn/api/status` 返回 `ok`。 - 未登录访问 `https://mm.tkmind.cn/api/sessions/test-session/events` 返回 `401`,且响应头包含 `x-accel-buffering: no`。 @@ -577,7 +577,7 @@ docker exec memind-runtime-redis redis-cli DEL memind:runtime:worker:goosed-3:dr 生产验证: - `https://mm.tkmind.cn/api/status` 返回 `ok`。 -- `https://mm.tkmind.cn/api/runtime/status` 返回 router enabled,四个 goosed target healthy。 +- `https://mm.tkmind.cn/api/runtime/status` 返回 router enabled,全部 goosed target healthy。 - `scripts/check-stream-runtime.mjs` 返回 `ok=true`。 - `scripts/check-tool-runtime.mjs` 返回 `ok=true`,普通 chat 不注入 Aider/OpenHands,code mode 对白名单用户可注入。 - `scripts/runtime-worker-drain.mjs status` 返回 `ok=true`,四个 worker drain=false。 @@ -678,7 +678,7 @@ docker exec memind-runtime-redis redis-cli DEL memind:runtime:worker:goosed-3:dr - `https://mm.tkmind.cn/api/status` 返回 `ok`。 - `/api/runtime/status.toolRuntime.codeRunsEnabled=false`。 -- 四个 goosed target healthy。 +- 全部 goosed target healthy。 - `scripts/check-agent-code-run-entry.mjs` 返回 `ok=true`。 - `scripts/check-stream-runtime.mjs` 返回 `ok=true`。 - `scripts/runtime-worker-drain.mjs status` 返回 `ok=true`,四个 worker drain=false。 @@ -752,7 +752,7 @@ docker exec memind-runtime-redis redis-cli DEL memind:runtime:worker:goosed-3:dr - `status`:采样但不写 Redis。 - `sample`:采样并写入 Redis。 - `--dry-run`:强制只读。 -- 对 Docker worker `goosed-prod-1..4` 采集: +- 对 Docker worker `goosed-prod-1..9` 采集: - `cpuLoad` - `memoryPressure` - `fdPressure` @@ -789,7 +789,7 @@ docker exec memind-runtime-redis redis-cli DEL memind:runtime:worker:goosed-3:dr 生产验证: -- `node scripts/runtime-worker-metrics.mjs sample` 返回 `ok=true`,四个 `goosed-prod-*` 均 `healthy`。 +- `node scripts/runtime-worker-metrics.mjs sample` 返回 `ok=true`,全部 `goosed-prod-*` 均 `healthy`。 - `/api/runtime/status.router.workers[]` 已显示: - `cpuLoad` - `memoryPressure` diff --git a/docs/local-dev.md b/docs/local-dev.md index 1ed9b67..a1c118b 100644 --- a/docs/local-dev.md +++ b/docs/local-dev.md @@ -109,7 +109,7 @@ Plaza 本地开发说明见 [plaza-local.md](./plaza-local.md)。生产发布、 | 项 | 本地(本机 `.env` / `../../.env.local`) | 103 生产(服务器 `/Users/john/Project/Memind/.env`) | |---|---|---| | 配置真相 | 复制 `.env.example`,只填本地联调值 | 仅在 103 维护,**不提交 Git** | -| `TKMIND_API_TARGET` | 通常单实例 `https://127.0.0.1:18006` | 多 goosed 实例,见 [103 runtime topology](./103-runtime-topology.md) | +| `TKMIND_API_TARGET` | 通常单实例 `https://127.0.0.1:18006` | 9 个 goosed target(`18006`–`18014`),见 [103 runtime topology](./103-runtime-topology.md) | | `H5_PUBLIC_BASE_URL` | `http://127.0.0.1:5173` | `https://mm.tkmind.cn` | | `MEMIND_SESSION_BROKER_ENABLED` | Patch 2 起可本地设 `1` 验证 | 灰度窗口由运维在 103 `.env` 单独开启 | | RDS / Redis | 可连本地 MySQL 或留空 | 生产 RDS + Redis,见 103 `.env` | diff --git a/docs/regression-guards/103-release-0706007-rollback.md b/docs/regression-guards/103-release-0706007-rollback.md index f561cef..7787e7b 100644 --- a/docs/regression-guards/103-release-0706007-rollback.md +++ b/docs/regression-guards/103-release-0706007-rollback.md @@ -45,7 +45,7 @@ | 检查项 | 结果 | |--------|------| | Portal `8081/api/status` | 200 | -| goosed 18006–18009 `/status` | 200 | +| goosed `18006`–`18014` `/status` | 200(2026-07-07 起为 9 个 target;0706007 时代为 4 个) | | `https://m.tkmind.cn/auth/login` 错误密码 | 401 + `用户名或密码错误` | | Portal 日志关键词 | 无异常关键词 | | memind_adm API `8085/health` | 200 | diff --git a/docs/发包必看.md b/docs/发包必看.md index aa8ee66..0dc5654 100644 --- a/docs/发包必看.md +++ b/docs/发包必看.md @@ -86,7 +86,7 @@ curl -k -i https://m.tkmind.cn/auth/login \ ```bash ssh john@58.38.22.103 ' printf "portal="; curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8081/api/status - for p in 18006 18007 18008 18009; do + for p in $(seq 18006 18014); do printf "goosed_${p}=" curl -k -s -o /dev/null -w "%{http_code}\n" https://127.0.0.1:${p}/status done @@ -100,10 +100,7 @@ ssh john@58.38.22.103 ' 必须看到: - `portal=200` -- `goosed_18006=200` -- `goosed_18007=200` -- `goosed_18008=200` -- `goosed_18009=200` +- `goosed_18006=200` … `goosed_18014=200`(共 9 个 target) - `DATABASE_URL=SET` - `TKMIND_API_TARGETS=SET` - `GOOSED_MCP_NODE_PATH=SET` @@ -193,10 +190,10 @@ TKMIND_ENABLE_CODE_EXECUTION_EXTENSION=1 Portal 现在使用: ```bash -TKMIND_API_TARGETS=https://127.0.0.1:18006,https://127.0.0.1:18007,https://127.0.0.1:18008,https://127.0.0.1:18009 +TKMIND_API_TARGETS=https://127.0.0.1:18006,https://127.0.0.1:18007,https://127.0.0.1:18008,https://127.0.0.1:18009,https://127.0.0.1:18010,https://127.0.0.1:18011,https://127.0.0.1:18012,https://127.0.0.1:18013,https://127.0.0.1:18014 ``` -发布或重建 goosed 容器后,要确认 18006-18009 都有 provider 配置。否则 Agent 会出现: +发布或重建 goosed 容器后,要确认 `18006`–`18014` 全部 target 都有 provider 配置。否则 Agent 会出现: ```text Provider 'custom_deepseek' is not configured @@ -233,7 +230,7 @@ ssh john@58.38.22.103 ' export PORTAL_RUNTIME_DIR="$portal_runtime_dir" MINDSPACE_ROOT="$mindspace_root" set +a /opt/homebrew/bin/docker compose -f docker-compose.prod.yml ps - for c in goosed-prod-1 goosed-prod-2 goosed-prod-3 goosed-prod-4; do + for c in goosed-prod-{1..9}; do echo "--- $c" /opt/homebrew/bin/docker exec "$c" sh -lc " ls -l /usr/local/bin/node \ @@ -248,9 +245,9 @@ ssh john@58.38.22.103 ' 重建容器后必须重新确认: -- 四个容器都是 `healthy`。 -- 18006-18009 `/status` 都是 200。 -- provider 已同步到四个 target。 +- 九个容器都是 `healthy`。 +- `18006`–`18014` `/status` 都是 200。 +- provider 已同步到全部 target。 - sandbox-fs smoke 能成功 `add extension` 并完成一次 `/reply`。 ## 6. 出问题时先走这条判断链 @@ -259,9 +256,9 @@ ssh john@58.38.22.103 ' 2. `/auth/login` 是否进入 400/401,而不是 503? 3. 103 Portal 进程是否拿到了 `DATABASE_URL`? 4. `~/Library/Logs/memind-portal.log` 是否有 `User auth bootstrap failed`? -5. 18006-18009 是否全 200? +5. `18006`–`18014` 是否全 200? 6. goosed 容器内 `/usr/local/bin/node` 和 `/opt/portal/mindspace-sandbox-mcp.mjs` 是否存在? -7. provider 是否同步到四个 target? +7. provider 是否同步到全部 target? 8. 如果是微信服务号或专属 Agent 报错,必须测真实业务路径,不要只看健康接口。 ## 7. 发布完成后给用户的最小回报格式 @@ -274,7 +271,7 @@ ssh john@58.38.22.103 ' - persisted backup - Portal 健康结果 - `/auth/login` 结果 -- 18006-18009 健康结果 +- `18006`–`18014` 健康结果(共 9 个 target) - 是否发现并处理了日志里的错误关键词 不要只说“已发布成功”。 diff --git a/scripts/release-portal-runtime-prod.sh b/scripts/release-portal-runtime-prod.sh index ed4ab63..1f7d51c 100755 --- a/scripts/release-portal-runtime-prod.sh +++ b/scripts/release-portal-runtime-prod.sh @@ -214,7 +214,17 @@ fi docker_bin="/opt/homebrew/bin/docker" if [[ -x "${docker_bin}" ]] && "${docker_bin}" ps --format '{{.Names}}' 2>/dev/null | grep -q '^goosed-prod-1$'; then - for index in 1 2 3 4; do + goosed_indexes=() + while IFS= read -r name; do + if [[ "${name}" =~ ^goosed-prod-([0-9]+)$ ]]; then + goosed_indexes+=("${BASH_REMATCH[1]}") + fi + done < <("${docker_bin}" ps --format '{{.Names}}' | sort -V) + if ((${#goosed_indexes[@]} == 0)); then + echo "goosed dependency check failed: no goosed-prod-* containers running" >&2 + missing=1 + fi + for index in "${goosed_indexes[@]}"; do container="goosed-prod-${index}" host_port=$((18005 + index)) if ! "${docker_bin}" ps --format '{{.Names}} {{.Ports}} {{.Status}}' | grep -q "^${container} .*0.0.0.0:${host_port}->18006/tcp.*healthy"; then @@ -233,7 +243,7 @@ if [[ -x "${docker_bin}" ]] && "${docker_bin}" ps --format '{{.Names}}' 2>/dev/n echo "goosed dependency check failed: Portal .env must set GOOSED_MCP_NODE_PATH and GOOSED_MCP_SERVER_PATH for Docker goosed" >&2 missing=1 else - for index in 1 2 3 4; do + for index in "${goosed_indexes[@]}"; do container="goosed-prod-${index}" if ! "${docker_bin}" exec "${container}" sh -lc "test -x '${portal_mcp_node}' && test -f '${portal_mcp_server}'" >/dev/null 2>&1; then echo "goosed dependency check failed: ${container} cannot resolve Portal .env MCP paths '${portal_mcp_node}' and '${portal_mcp_server}'" >&2 @@ -241,6 +251,27 @@ if [[ -x "${docker_bin}" ]] && "${docker_bin}" ps --format '{{.Names}}' 2>/dev/n fi done fi + targets_line="$(grep -E '^TKMIND_API_TARGETS=' /Users/john/Project/Memind/.env | tail -1 | cut -d= -f2- || true)" + if [[ -z "${targets_line}" ]]; then + echo "goosed dependency check failed: Portal .env must set TKMIND_API_TARGETS for Docker goosed pool" >&2 + missing=1 + else + IFS=',' read -ra target_urls <<< "${targets_line}" + for target_url in "${target_urls[@]}"; do + trimmed="$(echo "${target_url}" | tr -d ' \"')" + if [[ -z "${trimmed}" ]]; then + continue + fi + status="$(curl -skS -m 5 "${trimmed}/status" 2>/dev/null || true)" + if [[ "${status}" != "ok" ]]; then + echo "goosed dependency check failed: ${trimmed}/status != ok (${status:-empty})" >&2 + missing=1 + fi + done + if ((${#target_urls[@]} != ${#goosed_indexes[@]})); then + echo "goosed dependency check warning: TKMIND_API_TARGETS count (${#target_urls[@]}) != running containers (${#goosed_indexes[@]})" >&2 + fi + fi fi else api_secret="$(grep -E '^TKMIND_SERVER__SECRET_KEY=' /Users/john/Project/Memind/.env 2>/dev/null | tail -1 | cut -d= -f2- || true)" diff --git a/scripts/run-memind-portal-prod.sh b/scripts/run-memind-portal-prod.sh index ef14260..88d27d0 100755 --- a/scripts/run-memind-portal-prod.sh +++ b/scripts/run-memind-portal-prod.sh @@ -14,7 +14,7 @@ fi export NODE_ENV=production export H5_PORT="${H5_PORT:-8081}" export H5_PUBLIC_BASE_URL="${H5_PUBLIC_BASE_URL:-https://m.tkmind.cn}" -export TKMIND_API_TARGETS="${TKMIND_API_TARGETS:-https://127.0.0.1:18006,https://127.0.0.1:18007,https://127.0.0.1:18008,https://127.0.0.1:18009}" +export TKMIND_API_TARGETS="${TKMIND_API_TARGETS:-https://127.0.0.1:18006,https://127.0.0.1:18007,https://127.0.0.1:18008,https://127.0.0.1:18009,https://127.0.0.1:18010,https://127.0.0.1:18011,https://127.0.0.1:18012,https://127.0.0.1:18013,https://127.0.0.1:18014}" export TKMIND_API_TARGET="${TKMIND_API_TARGET:-https://127.0.0.1:18006}" export TKMIND_API_TARGET_1="${TKMIND_API_TARGET_1:-https://127.0.0.1:18007}"