tkmind/memind
1
0
Fork 0
Code Issues Pull Requests 2 Actions 1 Packages Projects Releases Wiki Activity

fix(goose-v149): block unattended real LLM smokes by default

Browse Source 
Prevent phase2/phase3/all reruns from burning DashScope tokens unless
GOOSE_V149_ALLOW_REAL_LLM=1 is explicitly set with human approval.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
john
2026-09-09 22:08:39 +08:00
parent 9569d83e58
commit 67220a14ea
23 changed files with 184 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/goose-v149-canary-runbook.md
+12 -8
View File
@@ -1,5 +1,7 @@
# Goose v1.49 本地 Canary 运行手册
> **2026-09-09 收口**：默认禁止真实 LLM 聚合 smoke（`check-goosed-v149-all` / phase2 / phase3 等）。见 [goose-v149-real-llm-gate.md](./goose-v149-real-llm-gate.md)。仅 `GOOSE_V149_ALLOW_REAL_LLM=1` + 明确批准后可跑。
> 范围：仅本机 loopback（`:18049` vs `:18006`），**禁止**连接 103/105 或修改生产 launchd。
> 决策表：[TKMIND_V1_49_MIGRATION.md](../TKMIND_V1_49_MIGRATION.md)
@@ -20,7 +22,8 @@ Memory 在 canary 期间保持 legacy owner（MySQL `h5_user_memory_items`），
```bash
bash scripts/build-goosed-v149-local.sh
bash scripts/ensure-goose-v149-local-db.sh
bash scripts/run-goosed-v149-local.sh # 另开终端，监听 :18049
# 默认已禁用；须明确批准：
GOOSE_V149_ALLOW_REAL_LLM=1 bash scripts/run-goosed-v149-local.sh # 另开终端，监听 :18049
curl -sk https://127.0.0.1:18049/status # → ok
```
@@ -64,14 +67,15 @@ pnpm dev
## 验证
```bash
# Phase 2 聚合（推荐日常）
node scripts/run-goosed-v149-phase2.mjs
# 以下默认 BLOCKED；须 GOOSE_V149_ALLOW_REAL_LLM=1 且有人值守，禁止 Agent 循环重跑：
# node scripts/run-goosed-v149-phase2.mjs
# node scripts/run-goosed-v149-phase3.mjs
# node scripts/check-goosed-v149-all.mjs
# Phase 3 收口（合并前）
node scripts/run-goosed-v149-phase3.mjs
# 单项 / 全量 smoke
node scripts/check-goosed-v149-all.mjs
# 无 LLM 的对照 / 单测（推荐日常）：
node --test scripts/goose-v149-canary.test.mjs
node scripts/compare-goose-v149-message-sanitize.mjs
node scripts/compare-goose-v149-memory-manifest.mjs
node scripts/check-goosed-v149-memory-loop.mjs
node scripts/check-goosed-v149-portal-smoke.mjs # 主 Portal 重启后
node scripts/check-goosed-v149-canary-portal.mjs # 隔离 :8083
docs/goose-v149-phase3-closeout.md
+2
View File
@@ -1,6 +1,8 @@
# Goose v1.49 Phase 3 收口清单
> **本阶段不授权 103/105 操作。** 仅整理证据、回滚路径与合并前闸门。
>
> **2026-09-09**：本机真实 LLM 聚合 smoke 已默认禁用，见 [goose-v149-real-llm-gate.md](./goose-v149-real-llm-gate.md)。`run-goosed-v149-phase3.mjs` 须 `GOOSE_V149_ALLOW_REAL_LLM=1` 且用户明确批准，禁止 Agent 循环重跑。
> 决策表：[TKMIND_V1_49_MIGRATION.md](../TKMIND_V1_49_MIGRATION.md) · Phase 2 台账：[goose-v149-phase2-evidence.md](./goose-v149-phase2-evidence.md)
## 聚合验证入口
docs/goose-v149-real-llm-gate.md
+50
View File
@@ -0,0 +1,50 @@
# Goose v1.49 真实 LLM 测试闸门（2026-09-09 收口）
## 背景
本机 `:18049` v1.49 回归（`check-goosed-v149-all` / `run-goosed-v149-phase3` 等）在一天内可创建 **1000+** Goose session、消耗 **1 亿+** DashScope token，主要来自：
- Portal smoke / resume（完整 Agent 多轮 + 工具）
- Page E2E（苏州一日游等生页场景）
- Memory loop / chat
**默认禁止**再跑这类会调用真实模型的聚合 smoke；只允许显式 opt-in。
## 默认行为（收口后）
| 动作 | 无 opt-in | `GOOSE_V149_ALLOW_REAL_LLM=1` |
|------|-----------|--------------------------------|
| `bash scripts/run-goosed-v149-local.sh` | **拒绝启动** | 允许 |
| `node scripts/check-goosed-v149-all.mjs` | **立即退出 2** | 允许 |
| `node scripts/run-goosed-v149-phase2.mjs` | **立即退出 2** | 允许 |
| `node scripts/run-goosed-v149-phase3.mjs` | **立即退出 2** | 允许 |
| `node scripts/run-goosed-v149-missing-evidence.mjs` | **立即退出 2** | 允许 |
| 单测 / message sanitize / memory manifest 对照 | 仍允许 | 仍允许 |
Portal / page / resume smoke 在未 opt-in 时还会被强制 `*_SKIP=1`（即使单独跑子脚本）。
## 显式批准后再跑（一次性）
```bash
# 1. 仅在需要时启动 v1.49 goosed
GOOSE_V149_ALLOW_REAL_LLM=1 bash scripts/run-goosed-v149-local.sh
# 2. 另开终端，跑指定检查（勿循环 / 勿 CI 默认开启）
GOOSE_V149_ALLOW_REAL_LLM=1 node scripts/check-goosed-v149-reply-smoke.mjs
```
**禁止**：Agent/Cursor 在未获用户明确批准时设置 `GOOSE_V149_ALLOW_REAL_LLM=1` 或循环重跑 phase2/phase3。
## 推荐替代
- 合并前优先：`node --test scripts/goose-v149-canary.test.mjs`、`compare-goose-v149-message-sanitize.mjs`、`compare-goose-v149-memory-manifest.mjs`
- 必须验证 Portal 集成时：单次、有人值守、先确认 DashScope 配额
## 回滚 Canary
若主 Portal 曾切到 `:18049`：
```bash
bash scripts/switch-goose-v149-canary.sh rollback
# 重启 pnpm dev
```
scripts/check-goosed-v149-all.mjs
+2
View File
@@ -8,12 +8,14 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
import {
gooseV149PortalAgentEnv,
waitForAgentRunWorkerIdle,
} from './goose-v149-worker-idle.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('check-goosed-v149-all.mjs');
const checkEnv = prepareGooseV149CheckEnv({ ...process.env }, root);
const checks = [
scripts/check-goosed-v149-cost-smoke.mjs
+3
View File
@@ -11,6 +11,9 @@ import { fileURLToPath } from 'node:url';
import { normalizeTokenState } from '../billing.mjs';
import { enrichTokenStateForBilling, resolveBillingTokenState } from '../billing-token-state.mjs';
import { collectReplyEvents, createV149Client } from './goose-v149-sse.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
enforceRealLlmGate('check-goosed-v149-cost-smoke.mjs');
const memindRoot = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
scripts/check-goosed-v149-deepseek-tools.mjs
+3
View File
@@ -9,6 +9,9 @@ import { randomUUID } from 'node:crypto';
import { fileURLToPath } from 'node:url';
import { collectReplyEvents, createV149Client } from './goose-v149-sse.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
enforceRealLlmGate('check-goosed-v149-deepseek-tools.mjs');
const memindRoot = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
scripts/check-goosed-v149-harness.mjs
+4
View File
@@ -4,6 +4,10 @@
*/
import { Agent, fetch } from 'undici';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
enforceRealLlmGate('check-goosed-v149-harness.mjs');
const port = process.env.GOOSE_V149_PORT || '18049';
const host = process.env.GOOSE_V149_HOST || '127.0.0.1';
const secret = process.env.GOOSE_SERVER__SECRET_KEY || 'local-v149-dev-secret';
scripts/check-goosed-v149-memory-chat.mjs
+2
View File
@@ -10,6 +10,7 @@ import { createConversationMemoryService } from '../conversation-memory.mjs';
import { createMemoryV2 } from '../memory-v2.mjs';
import { createDbPool } from '../db.mjs';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
import {
createReporter,
loginViaApi,
@@ -19,6 +20,7 @@ import {
} from './scenario-test-lib.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('check-goosed-v149-memory-chat.mjs');
prepareGooseV149CheckEnv(process.env, root);
const baseUrl = resolvePortalBase(Number(process.env.H5_PORT ?? 8081));
scripts/check-goosed-v149-memory-loop.mjs
+3
View File
@@ -8,6 +8,9 @@ import mysql from 'mysql2/promise';
import { createConversationMemoryService } from '../conversation-memory.mjs';
import { createMemoryV2 } from '../memory-v2.mjs';
import { loadMemindEnvFiles } from './memind-runtime-profile.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
enforceRealLlmGate('check-goosed-v149-memory-loop.mjs');
const BLOCKED_DB_HOSTS = ['58.38.22.103', '120.26.184.105', 'rds.aliyuncs.com'];
const MARKER = `goose-v149-memory-loop-${Date.now()}`;
scripts/check-goosed-v149-memory-verbal-recall.mjs
+2
View File
@@ -9,6 +9,7 @@ import { fileURLToPath } from 'node:url';
import { createDbPool } from '../db.mjs';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
import {
createReporter,
extractAssistantTexts,
@@ -20,6 +21,7 @@ import {
} from './scenario-test-lib.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('check-goosed-v149-memory-verbal-recall.mjs');
prepareGooseV149CheckEnv(process.env, root);
const baseUrl = resolvePortalBase(Number(process.env.H5_PORT ?? 8081));
scripts/check-goosed-v149-multiturn-provider.mjs
+4
View File
@@ -6,6 +6,10 @@ import { randomUUID } from 'node:crypto';
import { Readable } from 'node:stream';
import { Agent, fetch } from 'undici';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
enforceRealLlmGate('check-goosed-v149-multiturn-provider.mjs');
const port = process.env.GOOSE_V149_PORT || '18049';
const host = process.env.GOOSE_V149_HOST || '127.0.0.1';
const secret = process.env.GOOSE_SERVER__SECRET_KEY || 'local-v149-dev-secret';
scripts/check-goosed-v149-page-e2e.mjs
+2
View File
@@ -8,6 +8,7 @@ import { fileURLToPath } from 'node:url';
import { buildChatSkillPrompt } from '../chat-skills.mjs';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
import {
createReporter,
extractAssistantTexts,
@@ -20,6 +21,7 @@ import {
} from './scenario-test-lib.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('check-goosed-v149-page-e2e.mjs');
prepareGooseV149CheckEnv(process.env, root);
const baseUrl = resolvePortalBase(Number(process.env.H5_PORT ?? 8081));
scripts/check-goosed-v149-portal-resume.mjs
+2
View File
@@ -7,6 +7,7 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
import { waitForAgentRunWorkerIdle } from './goose-v149-worker-idle.mjs';
import {
createReporter,
@@ -16,6 +17,7 @@ import {
} from './scenario-test-lib.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('check-goosed-v149-portal-resume.mjs');
prepareGooseV149CheckEnv(process.env, root);
const baseUrl = resolvePortalBase(Number(process.env.H5_PORT ?? 8081));
scripts/check-goosed-v149-portal-smoke.mjs
+2
View File
@@ -9,10 +9,12 @@ import { execFileSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import { applyGooseV149CanaryBlockEnv, describeGooseCanaryConfig } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
import { loadMemindEnvFiles } from './memind-runtime-profile.mjs';
import { waitForRunTerminal } from './scenario-test-lib.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('check-goosed-v149-portal-smoke.mjs');
loadMemindEnvFiles(root, process.env);
applyGooseV149CanaryBlockEnv(process.env, root);
scripts/check-goosed-v149-provider.mjs
+4
View File
@@ -12,6 +12,10 @@ import { fileURLToPath } from 'node:url';
import { Readable } from 'node:stream';
import { Agent, fetch } from 'undici';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
enforceRealLlmGate('check-goosed-v149-provider.mjs');
const memindRoot = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
function loadEnvFile(filePath) {
scripts/check-goosed-v149-reply-smoke.mjs
+4
View File
@@ -7,6 +7,10 @@ import { randomUUID } from 'node:crypto';
import { Readable } from 'node:stream';
import { Agent, fetch } from 'undici';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
enforceRealLlmGate('check-goosed-v149-reply-smoke.mjs');
const port = process.env.GOOSE_V149_PORT || '18049';
const host = process.env.GOOSE_V149_HOST || '127.0.0.1';
const secret = process.env.GOOSE_SERVER__SECRET_KEY || 'local-v149-dev-secret';
scripts/goose-v149-canary.mjs
+2
View File
@@ -7,6 +7,7 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { createCanaryPolicy, resolveCanaryTarget } from '../release-gate/canary-routing.mjs';
import { applyRealLlmSkipDefaults } from './goose-v149-real-llm-gate.mjs';
import { loadMemindEnvFiles } from './memind-runtime-profile.mjs';
const memindRoot = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
@@ -78,6 +79,7 @@ export function prepareGooseV149CheckEnv(env = process.env, rootDir = memindRoot
loadMemindEnvFiles(rootDir, env);
applyGooseV149CanaryBlockEnv(env, rootDir);
applyLocalScenarioCredentials(env);
applyRealLlmSkipDefaults(env);
const canary = resolveGooseApiTargetsFromEnv(env);
if (canary?.mode && canary.mode !== 'off' && env.GOOSE_V149_MEMORY_DRIFT_REFRESH_AFTER_SMOKE == null) {
env.GOOSE_V149_MEMORY_DRIFT_REFRESH_AFTER_SMOKE = '1';
scripts/goose-v149-real-llm-gate.mjs
+40
View File
@@ -0,0 +1,40 @@
/**
* Hard gate: block unattended Goose v1.49 smoke that calls real LLM providers.
* Bulk portal/page/memory smokes can consume 100M+ tokens/day on DashScope.
*
* Opt-in only:
* GOOSE_V149_ALLOW_REAL_LLM=1 node scripts/check-goosed-v149-all.mjs
*
* See docs/goose-v149-real-llm-gate.md
*/
export const REAL_LLM_GATE_ENV = 'GOOSE_V149_ALLOW_REAL_LLM';
export function isRealLlmAllowed(env = process.env) {
return String(env[REAL_LLM_GATE_ENV] ?? '') === '1';
}
export function realLlmGateMessage(source) {
return [
`GOOSE_V149_REAL_LLM_BLOCKED: ${source}`,
'Unattended v1.49 smokes that call real LLM providers are disabled by default.',
'They previously consumed ~150M tokens/day on local DashScope during phase3 reruns.',
`To run once with explicit approval: ${REAL_LLM_GATE_ENV}=1 <command>`,
'See docs/goose-v149-real-llm-gate.md',
].join('\n');
}
export function enforceRealLlmGate(source, env = process.env) {
if (isRealLlmAllowed(env)) return;
console.error(realLlmGateMessage(source));
process.exit(2);
}
/** Force skip flags for the heaviest portal/page smokes unless explicitly allowed. */
export function applyRealLlmSkipDefaults(env = process.env) {
if (isRealLlmAllowed(env)) return env;
env.GOOSE_V149_PORTAL_SMOKE_SKIP = '1';
env.GOOSE_V149_PAGE_E2E_SKIP = '1';
env.GOOSE_V149_PORTAL_RESUME_SKIP = env.GOOSE_V149_PORTAL_RESUME_SKIP ?? '1';
return env;
}
scripts/goose-v149-real-llm-gate.test.mjs
+29
View File
@@ -0,0 +1,29 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {
applyRealLlmSkipDefaults,
isRealLlmAllowed,
realLlmGateMessage,
} from './goose-v149-real-llm-gate.mjs';
test('real LLM smokes blocked unless GOOSE_V149_ALLOW_REAL_LLM=1', () => {
assert.equal(isRealLlmAllowed({}), false);
assert.equal(isRealLlmAllowed({ GOOSE_V149_ALLOW_REAL_LLM: '1' }), true);
});
test('applyRealLlmSkipDefaults forces portal/page/resume skip when blocked', () => {
const env = applyRealLlmSkipDefaults({});
assert.equal(env.GOOSE_V149_PORTAL_SMOKE_SKIP, '1');
assert.equal(env.GOOSE_V149_PAGE_E2E_SKIP, '1');
assert.equal(env.GOOSE_V149_PORTAL_RESUME_SKIP, '1');
});
test('applyRealLlmSkipDefaults leaves skip flags unset when allowed', () => {
const env = applyRealLlmSkipDefaults({ GOOSE_V149_ALLOW_REAL_LLM: '1' });
assert.equal(env.GOOSE_V149_PORTAL_SMOKE_SKIP, undefined);
});
test('gate message mentions opt-in env', () => {
assert.match(realLlmGateMessage('test'), /GOOSE_V149_ALLOW_REAL_LLM=1/);
});
scripts/run-goosed-v149-local.sh
+6
View File
@@ -3,6 +3,12 @@
# Does not touch 103 production ports 18006-18014 or Portal 8081.
set -euo pipefail
if [[ "${GOOSE_V149_ALLOW_REAL_LLM:-}" != "1" ]]; then
echo "[goose-v149] REAL_LLM_BLOCKED: run-goosed-v149-local.sh disabled by default (see docs/goose-v149-real-llm-gate.md)" >&2
echo "[goose-v149] To start once with explicit approval: GOOSE_V149_ALLOW_REAL_LLM=1 bash scripts/run-goosed-v149-local.sh" >&2
exit 2
fi
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
GOOSED_ROOT="${GOOSED_V149_ROOT:-/Users/john/Project/tkmind_go-v149}"
PORT="${GOOSE_V149_PORT:-18049}"
scripts/run-goosed-v149-missing-evidence.mjs
+2
View File
@@ -9,8 +9,10 @@ import { fileURLToPath } from 'node:url';
import { loadMemindEnvFiles } from './memind-runtime-profile.mjs';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('run-goosed-v149-missing-evidence.mjs');
const checkEnv = prepareGooseV149CheckEnv({ ...process.env }, root);
if (!String(checkEnv.GOOSE_V149_MEMORY_VERBAL_FORCE_DEEP ?? '').trim()) {
checkEnv.GOOSE_V149_MEMORY_VERBAL_FORCE_DEEP = '0';
scripts/run-goosed-v149-phase2.mjs
+2
View File
@@ -9,8 +9,10 @@ import { fileURLToPath } from 'node:url';
import { loadMemindEnvFiles } from './memind-runtime-profile.mjs';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('run-goosed-v149-phase2.mjs');
const checkEnv = prepareGooseV149CheckEnv({ ...process.env }, root);
const checks = [
scripts/run-goosed-v149-phase3.mjs
+2
View File
@@ -9,12 +9,14 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { prepareGooseV149CheckEnv } from './goose-v149-canary.mjs';
import { enforceRealLlmGate } from './goose-v149-real-llm-gate.mjs';
import {
gooseV149PortalAgentEnv,
waitForAgentRunWorkerIdle,
} from './goose-v149-worker-idle.mjs';
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
enforceRealLlmGate('run-goosed-v149-phase3.mjs');
const checkEnv = prepareGooseV149CheckEnv({ ...process.env }, root);
const checks = [