Files
memind/docs/103-runtime-topology.md
john 78e5a6c026
Memind CI / Test, build, and release guards (pull_request) Failing after 18s
feat(ops): add imgproxy native runtime tooling and record 103 deployment
Bundle vendored imgproxy with Mach-O dylibs for 103 launchd, fix install to exec the binary directly with launchctl enable, and document the live native release in the runtime topology.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-31 05:38:14 +08:00

246 lines
11 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Local and 103 runtime topology
> Last confirmed: 2026-07-30 20:20 CST (goosed native migration).
>
> This is the current topology source of truth for local Memind and 103 production. Prefer this document over older migration notes. Older architecture documents may contain historical paths from before the MindSpace split.
## Local Mac
Local Memind connects to native `goosed`, not Colima/Docker `goosed`.
| Item | Value |
|------|-------|
| Runtime type | Native launchd process |
| Service | `com.tkmind.local-goosed` |
| Binary | `/Users/john/Project/tkmind_go/target/release/goosed` |
| Command role | `goosed agent` |
| Port | `18006` only |
| Session store | PostgreSQL `goose_sessions_dev` |
Local ports `18007`-`18009` are not the intended local goosed pool. If they are listening locally, treat that as drift unless a specific local test explicitly opened them.
Local Colima may exist for other workloads, but local Memind/goosed is not supposed to route through the 103-style `goosed-prod-1..9` Colima pool.
## Portal
| Item | Value |
|------|-------|
| Host | `58.38.22.103` |
| Live directory | `/Users/john/Project/Memind` |
| Service | `cn.tkmind.memind-portal` |
| Start script | `/Users/john/Project/Memind/scripts/run-memind-portal-prod.sh` |
| Port | `8081` |
| Health check | `curl http://127.0.0.1:8081/api/status` |
| Upstream goosed pool | `TKMIND_API_TARGETS``https://127.0.0.1:18006``18014`9 targets2026-07-07 确认) |
Portal must be released as a runtime artifact. Do not edit source directly on 103.
### User-level canary (when enabled)
| Item | Value |
|------|-------|
| Stable edge upstream | `105 nginx -> 58.38.22.103:8081` |
| Identity router | `127.0.0.1:18082` |
| Canary edge tunnel | `105 127.0.0.1:19082 -> 103 127.0.0.1:18082` |
| Candidate Portal | `127.0.0.1:18081` |
| Candidate goosed | `127.0.0.1:18015` |
| Release entry | `scripts/release-portal-canary-prod.sh` |
| Rollback entry | `scripts/rollback-portal-canary-prod.sh` |
The stable Portal stays on 8081 and the normal 105 edge path is a direct upstream, not
the retired 19081 tunnel. A canary release starts an isolated 19082 tunnel and changes
the two committed 105 nginx upstreams only after the candidate, router, 103/105 backups,
immutable identity checks, and tunnel checks pass. It then proves stable fallback before
declaring success. Port 18080 is occupied by a separate Memory V2 mock service and must
not be reused.
## MindSpace Service
| Item | Value |
|------|-------|
| Live directory | `/Users/john/MindSpace` |
| Service | `cn.tkmind.mindspace-service` |
| Start script | `/Users/john/MindSpace/scripts/run-mindspace-prod.sh` |
| Port | `8082` |
| Health check | `curl http://127.0.0.1:8082/health` |
| Contract check | `curl http://127.0.0.1:8082/mindspace/v1/contract` |
| Shared Memind root | `MEMIND_SHARED_ROOT=/Users/john/Project/Memind` |
`/Users/john/MindSpace` is the production MindSpace service root. Do not treat `/Users/john/Project/Memind/MindSpace` as the canonical MindSpace service root after the split.
`/health` and `/mindspace/v1/contract` expose the MindSpace service contract
version and build metadata. Production release scripts must reject a service
whose contract version is stale, whose required authority bindings are missing,
or whose `gitSha` does not match the runtime artifact manifest.
## goosed
103 goosed runs as **native launchd** processes (2026-07-30 migrated). Docker/Colima pool is stopped; `goosed-prod-1` is kept as emergency standby.
> 迁移记录与回滚:[goosed-native-103-migration-plan.md](./goosed-native-103-migration-plan.md)**已执行 2026-07-30**)。脚本:`scripts/goosed-native-103-migrate.sh`。
| Item | Value |
|------|-------|
| Runtime type | Native launchd (`GOOSED_RUNTIME=native`) |
| Root | `/Users/john/Project/tkmind_go-native` |
| Binary | `tkmind_go-native/goosed``releases/goosed-*` |
| Launchd labels | `cn.tkmind.goosed-native-18006``18014` |
| Run script | `run-goosed-native.sh <port>` + `.env.<port>` |
| Session store | PostgreSQL `memind_sessions` via `GOOSE_SESSION_DB_URL``127.0.0.1:5432` |
| FD monitor | `cn.tkmind.goosed-monitor``scripts/monitor-goosed-fds.mjs` |
| Docker standby | `goosed-prod-1` stopped, `restart=no`(紧急回滚 18006 |
| DeepSeek no-think | `cn.tkmind.memind-deepseek-no-think``:18036`native 必需,`MEMIND_DEEPSEEK_PROXY_ENTRYPOINT=1` |
**Critical:** Native goosed uses host PostgreSQL on `127.0.0.1:5432`. If PostgreSQL stops, H5 `/agent/start` hangs ~60s. Recovery:
```bash
ssh john@58.38.22.103 'bash /Users/john/Project/Memind/scripts/ensure-goose-session-postgres.sh'
ssh john@58.38.22.103 '/opt/homebrew/opt/postgresql@17/bin/pg_isready -h 127.0.0.1 -p 5432'
```
Port layout (native listens directly on host port):
| Port | Launchd label |
|------|---------------|
| `18006` | `cn.tkmind.goosed-native-18006` |
| `18007` | `cn.tkmind.goosed-native-18007` |
| … | … |
| `18014` | `cn.tkmind.goosed-native-18014` |
All nine host ports must return `ok` from `/status`, and must appear in Portal `.env` `TKMIND_API_TARGETS`.
### Legacy Docker pool (standby only)
| Item | Value |
|------|-------|
| Compose directory | `/Users/john/Project/goosed-prod` |
| Image pattern | `tkmind/goosed:prod-${GOOSED_TAG}` |
| Containers | `goosed-prod-1..9`**stopped**, `docker update --restart=no` |
Do **not** leave `cn.tkmind.goosed-monitor` on `monitor-goosed-containers.mjs` with `GOOSED_RESTART_UNHEALTHY=1`; it will restart Docker and fight native ports.
```bash
# Emergency: single Docker instance on 18006 only
launchctl bootout gui/$(id -u)/cn.tkmind.goosed-native-18006
docker update --restart=unless-stopped goosed-prod-1 && docker start goosed-prod-1
```
## PostgreSQL rule
Both local native goosed and 103 native goosed persist sessions in PostgreSQL:
- local native goosed: PostgreSQL database `goose_sessions_dev`
- 103 native goosed: PostgreSQL database `memind_sessions`
Do not assume a local SQLite session store when debugging current Memind/goosed behavior.
## Important path rule
The canonical production MindSpace service root is:
```text
/Users/john/MindSpace
```
The old monolith path:
```text
/Users/john/Project/Memind/MindSpace
```
is legacy compatibility/storage context only. If Portal or goosed config still points business MindSpace service behavior at the old path, treat that as configuration drift to fix, not as the desired topology.
In particular, check these before debugging missing folders/pages:
```bash
ssh john@58.38.22.103 'grep -E "^(MINDSPACE|MINDSPACE_ROOT|MINDSPACE_SERVICE|MEMIND_SHARED_ROOT)=" /Users/john/Project/Memind/.env /Users/john/Project/goosed-prod/.env 2>/dev/null'
ssh john@58.38.22.103 'launchctl print gui/$(id -u)/cn.tkmind.mindspace-service | grep -E "program =|working directory|MINDSPACE|MEMIND_SHARED"'
ssh john@58.38.22.103 '/opt/homebrew/bin/docker inspect goosed-prod-1 --format "{{json .Mounts}}"'
```
## Current verification commands
```bash
ssh john@58.38.22.103 'curl -fsS http://127.0.0.1:8081/api/status'
ssh john@58.38.22.103 'curl -fsS http://127.0.0.1:8082/health'
ssh john@58.38.22.103 'for p in $(seq 18006 18014); do curl -kfsS https://127.0.0.1:$p/status; echo; done'
ssh john@58.38.22.103 'grep ^TKMIND_API_TARGETS= /Users/john/Project/Memind/.env'
```
## imgproxy
103 imgproxy runs as a **native vendor runtime** (no brew on 103; Docker `memind-imgproxy` retired).
| Item | Value |
|------|-------|
| Runtime base | `/Users/john/Project/imgproxy-runtime` |
| Current release (2026-07-31) | `20260731-052342-imgproxy-native` (**native launchd**, imgproxy 4.0.7) |
| Legacy Docker release | `20260715-3819ccd5` (retired) |
| Vendor binary | `releases/<id>/vendor/imgproxy-runtime/bin/imgproxy` (+ bundled dylibs) |
| LaunchAgent | `cn.tkmind.imgproxy` → direct binary exec → `127.0.0.1:20082` |
| Compat proxy | `cn.tkmind.imgproxy-compat``10.10.0.2:20081` |
| Public entry | `https://img.tkmind.cn` |
| Signing config | Portal `.env``IMGPROXY_SIGNING_KEY` / `IMGPROXY_SIGNING_SALT` |
| Storage root | `MINDSPACE_STORAGE_ROOT=/Users/john/MindSpace/data/mindspace` |
Build locally, release to 103:
```bash
npm run build:imgproxy-runtime-native
bash scripts/release-imgproxy-native-prod.sh --dry-run # from release/* on synced main
bash scripts/release-imgproxy-native-prod.sh --yes
```
Install on 103 reads Portal `.env` for signing keys and storage root; launchd plist **must exec the vendored binary directly** (not `/bin/bash` wrapper). If bootstrap fails with `error 5`, run `launchctl enable gui/$(id -u)/cn.tkmind.imgproxy` before re-bootstrap.
Verify:
```bash
ssh john@58.38.22.103 'curl -fsS http://127.0.0.1:20082/health'
ssh john@58.38.22.103 'curl -fsS http://10.10.0.2:20081/health'
curl -fsS https://img.tkmind.cn/health
```
Legacy Docker release remains in `scripts/release-imgproxy-runtime-prod.sh` for rollback only.
Do **not** point launchd at `/Users/john/Project/Memind/vendor/...` without a Portal runtime that actually ships that path; imgproxy is an independent runtime under `imgproxy-runtime/`.
## image_make
`image_make` is an independent service. Portal runtime does **not** bundle it.
| Item | Value |
|------|-------|
| Local source | `/Users/john/Project/image_make` |
| 103 runtime base | `/Users/john/Project/image_make` |
| Current release (2026-07-30) | `20260730-220317-native-ceaff399a073` (**native launchd**) |
| Legacy Docker release | `20260720-standalone-rc1` (retired) |
| Listen | `127.0.0.1:18083` |
| LaunchAgent | `cn.tkmind.image-make` |
| Persistent env | `shared/.env` |
| Persistent data | `shared/data/artifacts` |
| Build (local) | `bash scripts/build-image-make-runtime-native.sh` |
| Release (103) | `bash scripts/release-image-make-runtime-prod.sh --yes` |
| Source fingerprint | `release-metadata.env``IMAGE_MAKE_SOURCE_TREE_SHA256` |
103 runs **darwin/arm64 native** (uv venv in release tar). Docker/Colima is no longer required for image_make.
**ComfyUI is a separate runtime** (not bundled into image_make tar):
| Item | Value |
|------|-------|
| Local build tree | `/Users/john/Project/image_make/.runtime/ComfyUI` + `comfyui-venv` |
| 103 runtime base | `/Users/john/Project/comfyui-runtime` |
| Current release (2026-07-30) | `20260730-222051-native-e87e7a81bb76` (**native launchd**) |
| Publish entry | `bash scripts/release-comfyui-runtime-prod.sh --yes` (from local Mac) |
| LaunchAgent | `cn.tkmind.comfyui``127.0.0.1:8188` |
| Fingerprint | `config/comfyui/runtime.lock` + SD1.5 SHA256 |
103 must **not** run `install_comfyui_runtime.sh` or download models directly; the local machine stages
`/Users/john/Project/comfyui-runtime/releases/<id>/` with production paths, tars it, and scp installs it.
Enable `comfyui` in Portal `h5_image_make_admin_config` only after ComfyUI is running.
Current production: default `aliyun_bailian`, `comfyui` enabled as failover backup.
Memind integration: HTTP client only (`image-make-client.mjs`); config via `IMAGE_MAKE_BASE_URL` + token. Docs: [image-make-integration.md](./image-make-integration.md).