Compare commits

..

13 Commits

Author SHA1 Message Date
john 2f4366e882 merge: enable WeChat media for all users
Memind CI / Test, build, and release guards (push) Successful in 2m18s
2026-07-18 21:21:13 +08:00
john 2ce1527edd feat: enable WeChat media for all users 2026-07-18 21:20:56 +08:00
john 1161292482 merge: fix WeChat multi-image gray flow
Memind CI / Test, build, and release guards (push) Successful in 2m28s
2026-07-18 19:53:21 +08:00
john 711a7d2061 fix: aggregate WeChat multi-image batches 2026-07-18 19:53:09 +08:00
john 341cf5ae89 merge: fix WeChat media gray pipeline
Memind CI / Test, build, and release guards (push) Successful in 1m55s
2026-07-18 19:10:13 +08:00
john ae3e1ba3fa fix: recover stale WeChat active requests 2026-07-18 19:05:33 +08:00
john 8eaf4d23f3 fix: route WeChat media through shared vision pipeline 2026-07-18 18:15:22 +08:00
john 8ccaf3b39d merge: add WeChat media analysis gray rollout
Memind CI / Test, build, and release guards (push) Successful in 2m35s
2026-07-18 17:50:06 +08:00
john 94f347398d feat: gate WeChat media analysis by user 2026-07-18 17:46:20 +08:00
john fd3904fdee feat: add WeChat media analysis adapters 2026-07-18 17:41:32 +08:00
tkmind 055d53c58b Merge pull request 'fix(chat): translate session replay cursors' (#14)
Memind CI / Test, build, and release guards (push) Successful in 4m33s
Preserve Portal/Goose SSE cursor boundaries and recover Finish after reconnect.
2026-07-17 08:55:36 +00:00
john cce56a4c6a fix(chat): translate session replay cursors
Memind CI / Test, build, and release guards (pull_request) Successful in 2m55s
2026-07-17 16:49:31 +08:00
tkmind c9591bc4cf Merge pull request 'fix(analytics): identify Memind users before pageviews' (#13) from codex/fix-umami-users-identify into main
Memind CI / Test, build, and release guards (push) Successful in 4m25s
2026-07-17 06:17:00 +00:00
20 changed files with 1388 additions and 53 deletions
+3
View File
@@ -29,6 +29,7 @@ bash scripts/check-release-ready.sh
| MindSpace 公开页 `edit_file` 落盘 + 聊天 Finish 不丢消息 | [docs/regression-guards/mindspace-publish-and-chat-finish.md](docs/regression-guards/mindspace-publish-and-chat-finish.md) |
| MindSpace remote 页面 sync + storage 缺失缩略图 fallback | [docs/regression-guards/mindspace-remote-page-sync-and-thumbnail.md](docs/regression-guards/mindspace-remote-page-sync-and-thumbnail.md) |
| Page Data 数据集注册、绑定与交付验收 | [docs/regression-guards/page-data-delivery-contract.md](docs/regression-guards/page-data-delivery-contract.md) |
| H5 SSE 断线续播、Portal/Goose 游标映射与 Finish 终态恢复 | [docs/regression-guards/h5-session-stream-replay.md](docs/regression-guards/h5-session-stream-replay.md) |
索引:[docs/regression-guards/README.md](docs/regression-guards/README.md)
@@ -38,6 +39,7 @@ bash scripts/check-release-ready.sh
npm run verify:mindspace-publish-guards
npm run verify:mindspace-publish-guards:full
npm run verify:mindspace-page-sync-guards
npm run verify:h5-session-patches
```
发版脚本(`scripts/release-portal-runtime-prod.sh`)在未 `--skip-tests` 时也会执行相关 verify。
@@ -51,6 +53,7 @@ npm run verify:mindspace-page-sync-guards
- `server.mjs` - session snapshot 需 `hint_mc``hint_ua` 才走缓存
- `mindspace-pages.mjs` - storage 缺失时 HTML 页回退读 workspace `relative_path`
- `mindspace-page-sync-service.mjs` + `server.mjs` - remote 模式也必须 sync public HTML
- `session-stream.mjs` + `session-stream-store.mjs` + `tkmind-proxy.mjs` - Portal replay ID 不得直接作为 Goose `Last-Event-ID`
代码内搜索 `REGRESSION GUARD` 可定位所有内联说明。
+2
View File
@@ -9,12 +9,14 @@
| [mindspace-publish-and-chat-finish.md](./mindspace-publish-and-chat-finish.md) | ① `edit_file` 覆盖 `public/*.html` ② Finish 后聊天不清空、不暴露 agent 内部前缀 |
| [mindspace-remote-page-sync-and-thumbnail.md](./mindspace-remote-page-sync-and-thumbnail.md) | ① remote 模式 public HTML 入库 sync ② storage 缺失时缩略图/读页回退 workspace HTML |
| [page-data-delivery-contract.md](./page-data-delivery-contract.md) | 数据集注册、绑定、真实 page UUID 与交付验收 |
| [h5-session-stream-replay.md](./h5-session-stream-replay.md) | Portal/Goose SSE 游标映射、断线续播与 Finish 终态恢复 |
## 自动化
```bash
npm run verify:mindspace-publish-guards # 推荐:改相关代码后
npm run verify:mindspace-publish-guards:full # 发 Portal runtime 后
npm run verify:h5-session-patches # H5 会话、Agent Run 与 SSE 续播
```
## 新增守卫时
@@ -0,0 +1,40 @@
# H5 会话 SSE 断线续播守卫
## 症状
后台 `h5_agent_runs` 已经是 `succeeded`、Goose 也已发送 `Finish`,但 H5 输入区仍显示“正在执行任务…”。常见触发条件是手机切后台、锁屏或网络切换发生在 `ActiveRequests``Finish` 之间。
## 根因
Portal 对 SSE 帧使用本地持久化游标。该游标可能是 Portal 生成的 UUID,并不一定是 Goose 能识别的上游事件 ID。若重连时把 Portal UUID 原样作为 `Last-Event-ID` 发送给 Goose,Goose 无法续播,浏览器就收不到遗漏的 `Finish`
## 必须保留的行为
1. Portal 游标只用于查询 `h5_session_stream_events`,不得原样传给 Goose。
2. 连接 Goose 时,只能使用已持久化的 `upstream_event_id`
3. 当前游标或重放尾部没有可用 `upstream_event_id` 时,必须省略 `Last-Event-ID`,让 Goose 从权威事件流重新回放。
4. 本地重放最后一条已经是 `Finish``Error` 时,不再连接 Goose。
5. 重放不得破坏现有消息 ID 合并、Finish 幂等计费和 MindSpace Finish 同步守卫。
## 关键路径
- `session-stream.mjs`
- `session-stream-store.mjs`
- `tkmind-proxy.mjs`
- `src/hooks/useTKMindChat.ts`
## 回归测试
```bash
node --test session-stream.test.mjs session-stream-store.test.mjs chat-agent-run-gate.test.mjs
node --test --test-name-pattern='proxySessionEvents' tkmind-proxy.test.mjs
npm run verify:h5-session-patches
node --test billing-session-concurrency.test.mjs
```
必须覆盖:
- Portal 游标存在但 `upstream_event_id` 为空时,不向 Goose 发送错误游标,并能收到 `Finish`
- 有映射时,Portal 游标正确转换成 Goose 游标。
- 多轮会话中选择最新可映射的上游游标,不被上一轮 Finish 截断。
- Finish 重放不重复扣费,消息合并与页面同步守卫继续通过。
+2
View File
@@ -820,6 +820,8 @@ async function bootstrapUserAuth() {
const target = await tkmindProxy.resolveTarget(sessionId);
return tkmindProxy.apiFetchTo(target, pathname, init);
},
submitSessionReply: ({ userId, sessionId, requestId, userMessage }) =>
tkmindProxy.submitSessionReplyForUser(userId, sessionId, requestId, userMessage),
scheduleService: process.env.H5_SCHEDULE_ENABLED === '1' ? scheduleService : null,
wechatScheduleLlmConfigService,
llmProviderService,
+18 -16
View File
@@ -8,6 +8,17 @@ function safeJsonParse(value, fallback = null) {
}
}
function mapEventRow(row) {
if (!row) return null;
return {
id: row.id,
eventType: row.event_type,
payload: safeJsonParse(row.payload_json, null),
upstreamEventId: row.upstream_event_id ?? null,
createdAt: Number(row.created_at),
};
}
export function createSessionStreamStore({ pool }) {
if (!pool) {
throw new Error('createSessionStreamStore requires pool');
@@ -52,9 +63,10 @@ export function createSessionStreamStore({ pool }) {
let afterCreatedAt = null;
let cursorMiss = false;
let cursorEvent = null;
if (afterEventId) {
const [cursorRows] = await pool.query(
`SELECT created_at
`SELECT id, event_type, payload_json, upstream_event_id, created_at
FROM h5_session_stream_events
WHERE id = ? AND agent_session_id = ? AND user_id = ?
LIMIT 1`,
@@ -63,7 +75,8 @@ export function createSessionStreamStore({ pool }) {
if (!cursorRows[0]) {
cursorMiss = true;
} else {
afterCreatedAt = Number(cursorRows[0].created_at);
cursorEvent = mapEventRow(cursorRows[0]);
afterCreatedAt = cursorEvent.createdAt;
}
}
@@ -85,14 +98,9 @@ export function createSessionStreamStore({ pool }) {
);
return {
events: rows.map((row) => ({
id: row.id,
eventType: row.event_type,
payload: safeJsonParse(row.payload_json, null),
upstreamEventId: row.upstream_event_id ?? null,
createdAt: Number(row.created_at),
})),
events: rows.map(mapEventRow),
cursorMiss,
cursorEvent,
};
}
@@ -107,13 +115,7 @@ export function createSessionStreamStore({ pool }) {
);
const row = rows[0];
if (!row) return null;
return {
id: row.id,
eventType: row.event_type,
payload: safeJsonParse(row.payload_json, null),
upstreamEventId: row.upstream_event_id ?? null,
createdAt: Number(row.created_at),
};
return mapEventRow(row);
}
return {
+6 -1
View File
@@ -23,7 +23,7 @@ function createMockPool() {
}
if (sql.includes('WHERE id = ? AND agent_session_id = ?')) {
const row = events.find((item) => item.id === params[0]);
return [[row ? { created_at: row.created_at } : undefined].filter(Boolean)];
return [[row].filter(Boolean)];
}
if (sql.includes('created_at > ?')) {
const after = Number(params[2]);
@@ -67,12 +67,14 @@ test('session stream store append and replay with cursor', async () => {
sessionId: 'sess-1',
payload: { type: 'Message', message: { role: 'assistant', content: [] } },
id: 'evt-1',
upstreamEventId: 'upstream-1',
});
await store.appendEvent({
userId: 'user-1',
sessionId: 'sess-1',
payload: { type: 'Finish' },
id: 'evt-2',
upstreamEventId: 'upstream-2',
});
const head = await store.listEventsForUser('user-1', 'sess-1');
@@ -82,7 +84,10 @@ test('session stream store append and replay with cursor', async () => {
assert.equal(tail.events.length, 1);
assert.equal(tail.events[0].payload.type, 'Finish');
assert.equal(tail.cursorMiss, false);
assert.equal(tail.cursorEvent.id, first.id);
assert.equal(tail.cursorEvent.upstreamEventId, 'upstream-1');
const miss = await store.listEventsForUser('user-1', 'sess-1', { afterEventId: 'missing' });
assert.equal(miss.cursorMiss, true);
assert.equal(miss.cursorEvent, null);
});
+20
View File
@@ -15,6 +15,26 @@ export function parseSessionStreamLastEventId(value) {
return id || null;
}
/**
* Portal replay ids are local database cursors and are not necessarily valid
* Goose SSE ids. Resume Goose from the newest persisted upstream cursor we
* actually know about. If none exists, return null so the caller omits
* Last-Event-ID and lets Goose replay the authoritative stream from the start.
*
* @param {Array<{ upstreamEventId?: string | null }>} events
* @param {{ upstreamEventId?: string | null } | null} cursorEvent
* @returns {string | null}
*/
export function resolveUpstreamResumeEventId(events, cursorEvent = null) {
const candidates = Array.isArray(events) ? [...events].reverse() : [];
if (cursorEvent) candidates.push(cursorEvent);
for (const event of candidates) {
const id = String(event?.upstreamEventId ?? '').trim();
if (id) return id;
}
return null;
}
export function isTerminalSessionEvent(event) {
const type = String(event?.type ?? '').trim();
return SESSION_STREAM_TERMINAL_TYPES.has(type);
+32
View File
@@ -6,6 +6,7 @@ import {
isTerminalSessionEvent,
parseSessionSseBlock,
parseSessionStreamLastEventId,
resolveUpstreamResumeEventId,
shouldPersistSessionStreamEvent,
shouldSkipUpstreamAfterSessionReplay,
} from './session-stream.mjs';
@@ -44,3 +45,34 @@ test('terminal detection and upstream skip policy', () => {
assert.equal(isSessionStreamReplayEnabled({ MEMIND_SESSION_STREAM_REPLAY: '1' }), true);
assert.equal(parseSessionStreamLastEventId(' abc '), 'abc');
});
test('upstream resume cursor never falls back to a Portal replay id', () => {
assert.equal(
resolveUpstreamResumeEventId([], {
id: 'portal-uuid',
upstreamEventId: null,
}),
null,
);
assert.equal(
resolveUpstreamResumeEventId([], {
id: 'portal-uuid',
upstreamEventId: 'upstream-7',
}),
'upstream-7',
);
});
test('upstream resume cursor uses the newest mappable replay event across turns', () => {
assert.equal(
resolveUpstreamResumeEventId(
[
{ upstreamEventId: 'upstream-old-finish' },
{ upstreamEventId: 'upstream-new-message' },
{ upstreamEventId: null },
],
{ upstreamEventId: 'upstream-cursor' },
),
'upstream-new-message',
);
});
+36
View File
@@ -83,6 +83,42 @@ test('buildVisionPayload marks one billable image analysis when vision succeeds'
);
});
test('buildVisionPayload sends all current-turn images to Qwen in order', async () => {
const imageUrls = [
'https://m.tkmind.cn/MindSpace/user-1/public/wechat-mp/first.jpg',
'https://m.tkmind.cn/MindSpace/user-1/public/wechat-mp/second.jpg',
];
let analyzedImages = [];
const result = await buildVisionPayload({
userId: 'user-1',
publishLayout: { publicUrl: 'https://m.tkmind.cn/MindSpace/user-1' },
userMessage: {
content: [{ type: 'text', text: '请结合两张图片生成页面' }],
metadata: { imageUrls },
},
fetchImpl: async () =>
new Response(Buffer.from('fake-image'), {
status: 200,
headers: { 'Content-Type': 'image/jpeg' },
}),
llmProviderService: {
analyzeImagesWithVision: async (images) => {
analyzedImages = images;
return '第一张是宝塔,第二张是晚霞。';
},
},
});
assert.equal(analyzedImages.length, 2);
assert.deepEqual(analyzedImages.map((item) => item.rawUrl), imageUrls);
assert.deepEqual(result?.userMessage?.metadata?.imageUrls, [
'/MindSpace/user-1/public/wechat-mp/first.jpg',
'/MindSpace/user-1/public/wechat-mp/second.jpg',
]);
assert.match(result?.userMessage?.content?.[0]?.text ?? '', /本轮用户仅上传 2 张图片/);
assert.match(result?.userMessage?.content?.[0]?.text ?? '', /图片2/);
});
test('buildVisionPayload does not mark billable usage when vision analysis fails', async () => {
const result = await buildVisionPayload({
userId: 'user-1',
+11 -4
View File
@@ -44,6 +44,7 @@ import {
isSessionStreamReplayEnabled,
parseSessionSseBlock,
parseSessionStreamLastEventId,
resolveUpstreamResumeEventId,
shouldPersistSessionStreamEvent,
shouldSkipUpstreamAfterSessionReplay,
} from './session-stream.mjs';
@@ -2154,6 +2155,7 @@ export function createTkmindProxy({
let clientClosed = false;
const replayEnabled = isSessionStreamReplayEnabled() && sessionStreamStore;
const initialLastEventId = parseSessionStreamLastEventId(req.get('last-event-id'));
let upstreamLastEventId = replayEnabled ? null : initialLastEventId;
const abortUpstream = () => {
clientClosed = true;
upstreamAbort.abort();
@@ -2210,17 +2212,22 @@ export function createTkmindProxy({
if (!res.writableEnded) res.end();
return;
}
upstreamLastEventId = resolveUpstreamResumeEventId(
batch?.events ?? [],
batch?.cursorEvent ?? null,
);
}
const pathname = `/sessions/${sessionId}/events`;
const sessionTarget = await resolveTarget(sessionId);
const upstreamHeaders = { Accept: 'text/event-stream' };
if (upstreamLastEventId) {
upstreamHeaders['Last-Event-ID'] = upstreamLastEventId;
}
const upstream = await apiFetch(sessionTarget, apiSecret, pathname, {
method: 'GET',
signal: upstreamAbort.signal,
headers: {
Accept: 'text/event-stream',
'Last-Event-ID': req.get('last-event-id') ?? '',
},
headers: upstreamHeaders,
});
if (clientClosed) return;
+233
View File
@@ -533,6 +533,178 @@ test('proxySessionEvents attaches session taxonomy when flag enabled', async ()
}
});
test('proxySessionEvents omits an unmapped Portal cursor so Goose can replay Finish', async () => {
const previousReplay = process.env.MEMIND_SESSION_STREAM_REPLAY;
process.env.MEMIND_SESSION_STREAM_REPLAY = '1';
let upstream;
let receivedLastEventId = 'not-requested';
try {
upstream = createServer((req, res) => {
if (req.method === 'GET' && req.url === '/sessions/session-1/events') {
receivedLastEventId = req.headers['last-event-id'] ?? null;
res.writeHead(200, { 'Content-Type': 'text/event-stream' });
res.end(
'id: upstream-finish\n' +
'data: {"type":"Finish","reason":"stop","request_id":"req-1"}\n\n',
);
return;
}
res.writeHead(404, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ message: `unexpected ${req.method} ${req.url}` }));
});
const upstreamPort = await listen(upstream);
const persisted = [];
const proxy = createTkmindProxy({
apiTarget: `http://127.0.0.1:${upstreamPort}`,
apiSecret: 'test-secret',
userAuth: createMemoryTestUserAuth(process.cwd()),
sessionStreamStore: {
async listEventsForUser(_userId, _sessionId, { afterEventId } = {}) {
assert.equal(afterEventId, 'portal-active-request-uuid');
return {
cursorMiss: false,
cursorEvent: {
id: 'portal-active-request-uuid',
payload: { type: 'ActiveRequests', request_ids: ['req-1'] },
upstreamEventId: null,
createdAt: 1,
},
events: [],
};
},
async appendEvent(event) {
persisted.push(event);
},
},
});
const req = new EventEmitter();
req.currentUser = { id: 'user-1', username: 'john' };
req.get = (name) =>
name.toLowerCase() === 'last-event-id' ? 'portal-active-request-uuid' : '';
req.once = req.once.bind(req);
req.off = req.off.bind(req);
const chunks = [];
const res = new EventEmitter();
res.headersSent = false;
res.writableEnded = false;
res.statusCode = 200;
res.status = (code) => {
res.statusCode = code;
return res;
};
res.setHeader = () => {};
res.flushHeaders = () => {
res.headersSent = true;
};
res.write = (chunk) => {
res.headersSent = true;
chunks.push(String(chunk));
return true;
};
res.end = () => {
res.writableEnded = true;
};
res.json = () => {
throw new Error('json must not be called after SSE started');
};
await proxy.proxySessionEvents(req, res, 'session-1');
assert.equal(receivedLastEventId, null);
assert.match(chunks.join(''), /"type":"Finish"/);
await new Promise((resolve) => setImmediate(resolve));
assert.equal(persisted.at(-1)?.upstreamEventId, 'upstream-finish');
} finally {
if (previousReplay == null) delete process.env.MEMIND_SESSION_STREAM_REPLAY;
else process.env.MEMIND_SESSION_STREAM_REPLAY = previousReplay;
await closeServer(upstream);
}
});
test('proxySessionEvents translates a Portal replay cursor to its Goose cursor', async () => {
const previousReplay = process.env.MEMIND_SESSION_STREAM_REPLAY;
process.env.MEMIND_SESSION_STREAM_REPLAY = '1';
let upstream;
let receivedLastEventId = null;
try {
upstream = createServer((req, res) => {
if (req.method === 'GET' && req.url === '/sessions/session-1/events') {
receivedLastEventId = req.headers['last-event-id'] ?? null;
res.writeHead(200, { 'Content-Type': 'text/event-stream' });
res.end('id: upstream-finish\ndata: {"type":"Finish","reason":"stop"}\n\n');
return;
}
res.writeHead(404, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ message: `unexpected ${req.method} ${req.url}` }));
});
const upstreamPort = await listen(upstream);
const proxy = createTkmindProxy({
apiTarget: `http://127.0.0.1:${upstreamPort}`,
apiSecret: 'test-secret',
userAuth: createMemoryTestUserAuth(process.cwd()),
sessionStreamStore: {
async listEventsForUser() {
return {
cursorMiss: false,
cursorEvent: {
id: 'portal-message-uuid',
payload: { type: 'Message' },
upstreamEventId: 'upstream-message-17',
createdAt: 1,
},
events: [],
};
},
async appendEvent() {},
},
});
const req = new EventEmitter();
req.currentUser = { id: 'user-1', username: 'john' };
req.get = (name) => name.toLowerCase() === 'last-event-id' ? 'portal-message-uuid' : '';
req.once = req.once.bind(req);
req.off = req.off.bind(req);
const chunks = [];
const res = new EventEmitter();
res.headersSent = false;
res.writableEnded = false;
res.statusCode = 200;
res.status = (code) => {
res.statusCode = code;
return res;
};
res.setHeader = () => {};
res.flushHeaders = () => {
res.headersSent = true;
};
res.write = (chunk) => {
res.headersSent = true;
chunks.push(String(chunk));
return true;
};
res.end = () => {
res.writableEnded = true;
};
res.json = () => {
throw new Error('json must not be called after SSE started');
};
await proxy.proxySessionEvents(req, res, 'session-1');
assert.equal(receivedLastEventId, 'upstream-message-17');
assert.match(chunks.join(''), /"type":"Finish"/);
} finally {
if (previousReplay == null) delete process.env.MEMIND_SESSION_STREAM_REPLAY;
else process.env.MEMIND_SESSION_STREAM_REPLAY = previousReplay;
await closeServer(upstream);
}
});
test('startSessionForUser resolves memories through Memory V2 facade', async () => {
let resolveInput = null;
await withFakeGoosedSession(async ({ apiTarget, workingDir, harnessEntries }) => {
@@ -726,6 +898,67 @@ test('submitSessionReplyForUser adds goose metadata visibility flags before repl
});
});
test('submitSessionReplyForUser applies the shared Qwen vision preprocessing path', async () => {
await withFakeGoosedSession(async ({ apiTarget, workingDir, replyBodies }) => {
const proxy = createTkmindProxy({
apiTarget,
apiSecret: 'test-secret',
userAuth: {
...createMemoryTestUserAuth(workingDir),
async ownsSession() {
return true;
},
async canUseChat() {
return { ok: true };
},
async getUserById() {
return { id: 'user-1' };
},
async getUserPublishLayout() {
return { publicUrl: 'https://example.com/MindSpace/user-1' };
},
async resolveUserPolicies() {
return { unrestricted: true, policies: {} };
},
},
localFetchAsset: async () => ({
buffer: Buffer.from('fake-image'),
mimeType: 'image/jpeg',
}),
llmProviderService: {
async applyBestProviderForSession() {
return { ok: true };
},
async hasVisionKey() {
return true;
},
async analyzeImagesWithVision() {
return '一件蓝色产品,白色背景,竖版构图。';
},
},
});
await proxy.submitSessionReplyForUser(
'user-1',
'session-1',
'request-qwen-vision',
{
id: 'message-qwen-vision',
role: 'user',
content: [{ type: 'text', text: '请分析这张图片' }],
metadata: {
imageUrls: ['/api/mindspace/v1/assets/asset-1/download?inline=1'],
displayText: '请分析这张图片',
},
},
);
const forwardedText = replyBodies[0]?.user_message?.content?.[0]?.text ?? '';
assert.match(forwardedText, /Qwen VL 图片描述/);
assert.match(forwardedText, /一件蓝色产品/);
});
});
test('submitSessionReplyForUser passes current prompt to Memory V2 resolve before existing reply path', async () => {
let resolveInput = null;
await withFakeGoosedSession(async ({ apiTarget, workingDir }) => {
+88
View File
@@ -9,12 +9,19 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
const DEFAULT_WECHAT_MEDIA_URL = 'https://api.weixin.qq.com/cgi-bin/media/get';
const DEFAULT_MAX_IMAGE_BYTES = 10 * 1024 * 1024;
const DEFAULT_MAX_ATTACHMENT_BYTES = 30 * 1024 * 1024;
const ALLOWED_IMAGE_MIME_TYPES = new Map([
['image/jpeg', 'jpg'],
['image/png', 'png'],
['image/webp', 'webp'],
['image/gif', 'gif'],
]);
const ALLOWED_ATTACHMENT_EXTENSIONS = new Map([
['.doc', 'application/msword'],
['.docx', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
['.xls', 'application/vnd.ms-excel'],
['.xlsx', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'],
]);
function resolveImageExtension(contentType = '', fallbackUrl = '') {
const normalized = String(contentType ?? '')
@@ -44,6 +51,32 @@ function ensureImageWithinLimit(buffer, maxBytes) {
}
}
function sanitizeAttachmentFilename(filename = '') {
const basename = path.basename(String(filename ?? '').trim()).replace(/[\u0000-\u001f\u007f]/g, '');
if (!basename || basename === '.' || basename === '..') {
throw new Error('微信文件缺少有效文件名');
}
const extension = path.extname(basename).toLowerCase();
const mimeType = ALLOWED_ATTACHMENT_EXTENSIONS.get(extension);
if (!mimeType) {
throw new Error('当前服务号文件仅支持 Worddoc/docx)和 Excelxls/xlsx');
}
return {
filename: basename.slice(0, 160),
extension,
mimeType,
};
}
function ensureAttachmentWithinLimit(buffer, maxBytes) {
if (!Buffer.isBuffer(buffer) || buffer.length === 0) {
throw new Error('微信文件内容为空');
}
if (buffer.length > maxBytes) {
throw new Error(`文件超过大小限制(${maxBytes} bytes`);
}
}
export async function downloadTemporaryMedia(accessToken, mediaId, { wechatFetch = undiciFetch } = {}) {
if (!accessToken) throw new Error('缺少微信 access_token');
if (!mediaId) throw new Error('缺少微信 mediaId');
@@ -154,3 +187,58 @@ export async function persistWechatImage(
source,
};
}
export async function persistWechatAttachment(
{
userId,
appId,
openid,
msgId,
mediaId,
filename,
publicBaseUrl,
maxFileBytes = DEFAULT_MAX_ATTACHMENT_BYTES,
},
{
wechatFetch = undiciFetch,
accessToken,
h5Root = __dirname,
} = {},
) {
if (!userId) throw new Error('缺少 userId');
const resolved = sanitizeAttachmentFilename(filename);
const downloaded = await downloadTemporaryMedia(accessToken, mediaId, { wechatFetch });
ensureAttachmentWithinLimit(downloaded.buffer, maxFileBytes);
const publishDir = path.join(h5Root, PUBLISH_ROOT_DIR, String(userId), PUBLIC_ZONE_DIR, 'wechat-mp');
fs.mkdirSync(publishDir, { recursive: true });
const timestamp = Date.now();
const hash = crypto.createHash('sha1').update(downloaded.buffer).digest('hex').slice(0, 12);
const originalStem = path.basename(resolved.filename, resolved.extension)
.replace(/[^\p{L}\p{N}._-]+/gu, '_')
.replace(/^_+|_+$/g, '')
.slice(0, 80) || 'attachment';
const identity = [appId || 'wx', openid || 'openid', msgId || timestamp, mediaId || hash]
.filter(Boolean)
.join('-')
.replace(/[^a-zA-Z0-9._-]+/g, '_')
.slice(0, 100);
const publicFilename = `${originalStem}-${identity}-${hash}${resolved.extension}`;
const absolutePath = path.join(publishDir, publicFilename);
fs.writeFileSync(absolutePath, downloaded.buffer);
return {
absolutePath,
bytes: downloaded.buffer.length,
contentType: resolved.mimeType,
filename: resolved.filename,
publicFilename,
publicUrl: buildWechatImagePublicUrl({
publicBaseUrl,
publishKey: String(userId),
filename: publicFilename,
}),
source: 'wechat_media',
};
}
+10
View File
@@ -23,6 +23,13 @@ function deriveWechatEndpointFromUrl(baseUrl, suffix) {
}
}
function parseCsvList(value = '') {
return String(value ?? '')
.split(',')
.map((item) => item.trim())
.filter(Boolean);
}
export function loadWechatMpConfig(env = process.env) {
const appId = env.H5_WECHAT_MP_APP_ID?.trim() ?? env.H5_WECHAT_APP_ID?.trim() ?? '';
const appSecret =
@@ -67,10 +74,13 @@ export function loadWechatMpConfig(env = process.env) {
mediaPublicBaseUrl:
env.H5_WECHAT_MP_MEDIA_PUBLIC_BASE_URL?.trim()?.replace(/\/$/, '') || publicBaseUrl,
maxImageBytes: Math.max(1, Number(env.H5_WECHAT_MP_MAX_IMAGE_BYTES ?? 10 * 1024 * 1024)),
maxFileBytes: Math.max(1, Number(env.H5_WECHAT_MP_MAX_FILE_BYTES ?? 30 * 1024 * 1024)),
acceptVoice: env.H5_WECHAT_MP_ACCEPT_VOICE !== '0',
acceptImage: env.H5_WECHAT_MP_ACCEPT_IMAGE !== '0',
acceptFile: env.H5_WECHAT_MP_ACCEPT_FILE !== '0',
acceptLocation: env.H5_WECHAT_MP_ACCEPT_LOCATION !== '0',
acceptLink: env.H5_WECHAT_MP_ACCEPT_LINK !== '0',
mediaAnalysisGrayUsers: parseCsvList(env.H5_WECHAT_MP_MEDIA_GRAY_USERS),
encodingAesKey: env.H5_WECHAT_MP_ENCODING_AES_KEY?.trim() ?? '',
};
}
+305 -30
View File
@@ -9,7 +9,11 @@ import { resolveSessionAccess } from './session-broker.mjs';
import { isStubPublicHtmlContent, materializeMissingPublicHtmlWrites } from './mindspace-public-finish-sync.mjs';
import { loadWechatMpConfig } from './wechat-mp-config.mjs';
import { buildPublicUrl, PUBLISH_ROOT_DIR } from './user-publish.mjs';
import { downloadTemporaryMedia, persistWechatImage } from './wechat-media.mjs';
import {
downloadTemporaryMedia,
persistWechatAttachment,
persistWechatImage,
} from './wechat-media.mjs';
import { normalizeWechatName, resolveWechatAddressName } from './wechat/user/display-name.mjs';
import { buildAckText } from './wechat/ack/ack-provider.mjs';
import { guardScheduleConfirmationReply } from './wechat/handlers/schedule-guard.mjs';
@@ -46,6 +50,8 @@ const DEFAULT_WECHAT_CUSTOMER_SERVICE_URL =
'https://api.weixin.qq.com/cgi-bin/message/custom/send';
const DEFAULT_WECHAT_JSAPI_TICKET_URL = 'https://api.weixin.qq.com/cgi-bin/ticket/getticket';
const DEFAULT_ASR_TARGET = process.env.H5_ASR_TARGET ?? 'https://asr.tkmind.cn';
const WECHAT_RECENT_MEDIA_TTL_MS = 15 * 60 * 1000;
const WECHAT_RECENT_IMAGE_MAX_COUNT = 10;
export { loadWechatMpConfig };
const PUBLIC_HTML_LINK_PATTERN =
/https?:\/\/[^\s<>"')\]]+\/MindSpace\/([0-9a-f-]{36}|[a-z0-9._-]+)\/public\/([^\s<>"')\]]+\.html)/gi;
@@ -100,6 +106,8 @@ function parseWechatMessage(xml) {
description: parseXmlField(xml, 'Description'),
url: parseXmlField(xml, 'Url'),
thumbMediaId: parseXmlField(xml, 'ThumbMediaId'),
fileName: parseXmlField(xml, 'FileName') || parseXmlField(xml, 'Filename'),
fileSize: parseXmlField(xml, 'FileSize'),
event: parseXmlField(xml, 'Event').toLowerCase(),
eventKey: parseXmlField(xml, 'EventKey'),
latitude: parseXmlField(xml, 'Latitude'),
@@ -204,7 +212,14 @@ function pushMessage(messages, incoming) {
return [...messages, incoming];
}
async function executeSessionReply(apiFetch, sessionId, requestId, prompt, metadata = {}) {
async function executeSessionReply(
apiFetch,
sessionId,
requestId,
prompt,
metadata = {},
{ submitReply = null } = {},
) {
const eventsResponse = await apiFetch(`/sessions/${sessionId}/events`, {
method: 'GET',
headers: { Accept: 'text/event-stream' },
@@ -214,18 +229,23 @@ async function executeSessionReply(apiFetch, sessionId, requestId, prompt, metad
throw new Error(text || '无法建立公众号消息事件流');
}
const replyResponse = await apiFetch(`/sessions/${sessionId}/reply`, {
method: 'POST',
body: JSON.stringify({
request_id: requestId,
user_message: createUserMessage(prompt, metadata),
}),
});
if (!replyResponse.ok) {
const text = await replyResponse.text().catch(() => '');
throw new Error(text || 'Agent reply 请求失败');
const userMessage = createUserMessage(prompt, metadata);
if (submitReply) {
await submitReply({ sessionId, requestId, userMessage });
} else {
const replyResponse = await apiFetch(`/sessions/${sessionId}/reply`, {
method: 'POST',
body: JSON.stringify({
request_id: requestId,
user_message: userMessage,
}),
});
if (!replyResponse.ok) {
const text = await replyResponse.text().catch(() => '');
throw new Error(text || 'Agent reply 请求失败');
}
replyResponse.body?.cancel().catch?.(() => {});
}
replyResponse.body?.cancel().catch?.(() => {});
const reader = eventsResponse.body.getReader();
const decoder = new TextDecoder();
@@ -994,6 +1014,8 @@ export function isRecoverableWechatAgentSessionError(message) {
if (/stale_session_poisoned_completion/i.test(normalized)) return true;
if (/403|404|not found|无权访问/i.test(normalized)) return true;
if (/tool_calls|tool_call_id|insufficient tool messages/i.test(normalized)) return true;
if (/session already has an active request|active request.*cancel/i.test(normalized)) return true;
if (/wechat_agent_incomplete_reply/i.test(normalized)) return true;
return false;
}
@@ -1024,6 +1046,13 @@ export function findRecoverableWechatAgentErrorInReply(reply) {
export function assertWechatAgentReplyIsSendable(reply) {
const recoverable = findRecoverableWechatAgentErrorInReply(reply);
if (recoverable) throw new Error(recoverable);
const text = String(reply?.text ?? '').trim();
if (
/^(?:let me|i(?:'ll| will))\s+(?:first\s+)?(?:look|check|inspect|analy[sz]e)(?:\s+at)?\s+(?:the\s+)?image(?:\s+first)?[.!]?$/iu.test(text) ||
/^(?:让我|我先)(?:先)?(?:看|查看|检查|分析)(?:一下)?(?:这张|该张|这个)?图片[。!!]?$/u.test(text)
) {
throw new Error('wechat_agent_incomplete_reply');
}
}
export function isWechatAgentApiErrorText(message) {
@@ -1164,6 +1193,24 @@ function normalizeNumber(value) {
return Number.isFinite(num) ? num : null;
}
function isWechatMediaGrayUser(user, configuredUsers = []) {
const allowlist = Array.isArray(configuredUsers)
? configuredUsers.map((value) => String(value ?? '').trim().toLowerCase()).filter(Boolean)
: [];
if (allowlist.length === 0) return false;
if (allowlist.includes('*')) return true;
const identities = [
user?.userId,
user?.username,
user?.slug,
user?.displayName,
user?.nickname,
]
.map((value) => String(value ?? '').trim().toLowerCase())
.filter(Boolean);
return identities.some((identity) => allowlist.includes(identity));
}
function normalizeWechatInboundIntent(inbound) {
const msgType = String(inbound?.msgType ?? '').toLowerCase();
const base = {
@@ -1210,6 +1257,22 @@ function normalizeWechatInboundIntent(inbound) {
};
}
if (msgType === 'file') {
const filename = String(inbound?.fileName ?? '').trim();
return {
...base,
displayText: filename ? `收到文件:${filename}` : '收到文件',
agentText: '',
media: {
mediaId: inbound?.mediaId || '',
},
attachment: {
filename,
sizeBytes: normalizeNumber(inbound?.fileSize),
},
};
}
if (msgType === 'location') {
const latitude = normalizeNumber(inbound?.locationX);
const longitude = normalizeNumber(inbound?.locationY);
@@ -1409,6 +1472,7 @@ export function createWechatMpService({
apiFetch,
startAgentSession = null,
sessionApiFetch = null,
submitSessionReply = null,
scheduleService = null,
wechatScheduleLlmConfigService = null,
llmProviderService = null,
@@ -1434,10 +1498,15 @@ export function createWechatMpService({
jsapiTicketUrl: config.jsapiTicketUrl || DEFAULT_WECHAT_JSAPI_TICKET_URL,
mediaPublicBaseUrl: config.mediaPublicBaseUrl || config.publicBaseUrl,
maxImageBytes: Math.max(1, Number(config.maxImageBytes ?? 10 * 1024 * 1024)),
maxFileBytes: Math.max(1, Number(config.maxFileBytes ?? 30 * 1024 * 1024)),
acceptVoice: config.acceptVoice !== false,
acceptImage: config.acceptImage !== false,
acceptFile: config.acceptFile !== false,
acceptLocation: config.acceptLocation !== false,
acceptLink: config.acceptLink !== false,
mediaAnalysisGrayUsers: Array.isArray(config.mediaAnalysisGrayUsers)
? config.mediaAnalysisGrayUsers
: [],
asrTarget: config.asrTarget || DEFAULT_ASR_TARGET,
};
@@ -1446,6 +1515,8 @@ export function createWechatMpService({
expiresAt: 0,
};
const rememberedWechatContexts = new Map();
const messageTasksByOpenid = new Map();
const recentMediaByOpenid = new Map();
let jsapiTicketCache = {
ticket: null,
expiresAt: 0,
@@ -1454,6 +1525,86 @@ export function createWechatMpService({
const fetchForSession = (sessionId, pathname, init) =>
sessionApiFetch ? sessionApiFetch(sessionId, pathname, init) : apiFetch(pathname, init);
const enqueueMessageTask = (openid, taskFactory) => {
const key = String(openid ?? '').trim();
const previous = messageTasksByOpenid.get(key) ?? Promise.resolve();
const task = previous.catch(() => undefined).then(taskFactory);
messageTasksByOpenid.set(key, task);
void task
.finally(() => {
if (messageTasksByOpenid.get(key) === task) messageTasksByOpenid.delete(key);
})
.catch(() => {});
return task;
};
const rememberRecentMedia = (openid, intent) => {
const publicUrl = String(intent?.media?.publicUrl ?? '').trim();
if (!publicUrl) return;
const key = String(openid ?? '').trim();
const now = Date.now();
const item = {
media: { ...(intent.media ?? {}) },
attachment: intent.attachment ? { ...intent.attachment } : null,
};
const current = recentMediaByOpenid.get(key);
const canAppendImage =
intent?.msgType === 'image' &&
current &&
!current.claimed &&
now - current.rememberedAt <= WECHAT_RECENT_MEDIA_TTL_MS &&
current.items.every((recentItem) => !recentItem.attachment);
const candidates = canAppendImage ? [...current.items, item] : [item];
const deduped = candidates.filter(
(candidate, index, values) =>
values.findIndex(
(value) => String(value?.media?.publicUrl ?? '') === String(candidate?.media?.publicUrl ?? ''),
) === index,
);
recentMediaByOpenid.set(key, {
items: deduped.slice(-WECHAT_RECENT_IMAGE_MAX_COUNT),
rememberedAt: now,
batchId: crypto.randomUUID(),
claimed: false,
});
};
const attachRecentMediaForFollowup = (openid, intent, mediaAnalysisEnabled) => {
if (!mediaAnalysisEnabled || intent?.msgType !== 'text' || intent?.media?.publicUrl) return;
const text = String(intent?.agentText ?? '');
if (!/(?:刚才|之前|上一|这张|这份|图片|图像|照片|文件|文档|表格|excel|word)/iu.test(text)) return;
const key = String(openid ?? '').trim();
const recent = recentMediaByOpenid.get(key);
if (
!recent ||
recent.claimed ||
Date.now() - recent.rememberedAt > WECHAT_RECENT_MEDIA_TTL_MS ||
recent.items.length === 0
) {
return;
}
const items = recent.items.map((item) => ({
media: { ...(item.media ?? {}), source: 'wechat_recent_media' },
attachment: item.attachment ? { ...item.attachment } : null,
}));
const primary = items.at(-1);
intent.media = { ...(primary?.media ?? {}) };
if (primary?.attachment) intent.attachment = { ...primary.attachment };
intent.recentMediaItems = items;
intent.recentMediaBatchId = recent.batchId;
recent.claimed = true;
};
const settleRecentMediaBatch = (openid, intent, { succeeded }) => {
const batchId = String(intent?.recentMediaBatchId ?? '').trim();
if (!batchId) return;
const key = String(openid ?? '').trim();
const recent = recentMediaByOpenid.get(key);
if (!recent || recent.batchId !== batchId) return;
if (succeeded) recentMediaByOpenid.delete(key);
else recent.claimed = false;
};
const resolveWechatBillingTokenState = (sessionId, tokenState) =>
resolveBillingTokenState(tokenState, {
sessionId,
@@ -1872,16 +2023,40 @@ export function createWechatMpService({
}
};
const buildIntentMetadata = (intent) => ({
source: 'wechat_mp',
msgType: intent.msgType,
originalMsgId: intent.msgId || null,
displayText: intent.displayText || '',
mediaPublicUrl: intent.media?.publicUrl || null,
recognition: intent.msgType === 'voice' ? intent.agentText || null : null,
location: intent.location || null,
link: intent.link || null,
});
const buildIntentMetadata = (intent, { mediaAnalysisEnabled = false } = {}) => {
const mediaPublicUrl = intent.media?.publicUrl || null;
const mediaItems = Array.isArray(intent.recentMediaItems) && intent.recentMediaItems.length > 0
? intent.recentMediaItems
: mediaPublicUrl
? [{ media: intent.media, attachment: intent.attachment ?? null }]
: [];
const imageUrls = mediaItems
.filter((item) => !item.attachment)
.map((item) => String(item?.media?.publicUrl ?? '').trim())
.filter((url, index, values) => url && values.indexOf(url) === index);
const fileAttachments = mediaItems
.filter((item) => item.attachment?.filename && item.media?.publicUrl)
.map((item) => ({
assetId: '',
downloadUrl: item.media.publicUrl,
filename: item.attachment.filename,
mimeType: item.attachment.mimeType || 'application/octet-stream',
}));
return {
source: 'wechat_mp',
msgType: intent.msgType,
originalMsgId: intent.msgId || null,
displayText: intent.displayText || '',
mediaPublicUrl,
...(mediaAnalysisEnabled && imageUrls.length > 0
? { imageUrls }
: {}),
...(mediaAnalysisEnabled && fileAttachments.length > 0 ? { fileAttachments } : {}),
recognition: intent.msgType === 'voice' ? intent.agentText || null : null,
location: intent.location || null,
link: intent.link || null,
};
};
const persistIntentDetail = async ({ intent, userId = null, rawXmlHash = '' }) => {
if (typeof userAuth.insertWechatMpMessageDetail !== 'function') return;
@@ -1921,6 +2096,7 @@ export function createWechatMpService({
const runIntentMessage = async ({ inbound, intent, user }) => {
const wechatIntent = classifyWechatIntent(intent);
const mediaAnalysisEnabled = isWechatMediaGrayUser(user, config.mediaAnalysisGrayUsers);
const resetCandidate =
intent.msgType === 'text' || intent.msgType === 'voice' ? intent.agentText : '';
// Page Data delivery owns persistent files, datasets and two publication
@@ -1970,7 +2146,18 @@ export function createWechatMpService({
sessionId,
requestId,
agentPrompt,
buildIntentMetadata(intent),
buildIntentMetadata(intent, { mediaAnalysisEnabled }),
{
submitReply: submitSessionReply
? ({ requestId: replyRequestId, userMessage }) =>
submitSessionReply({
userId: user.userId,
sessionId,
requestId: replyRequestId,
userMessage,
})
: null,
},
);
const workingDir = publishLayout?.publishDir ?? (await userAuth.resolveWorkingDir(user.userId));
const linkExistsForRequest = resolveLinkExistsForWorkingDir(workingDir, linkExists);
@@ -2129,7 +2316,18 @@ export function createWechatMpService({
sessionId,
retryId,
retryPrompt,
buildIntentMetadata(intent),
buildIntentMetadata(intent, { mediaAnalysisEnabled }),
{
submitReply: submitSessionReply
? ({ requestId: replyRequestId, userMessage }) =>
submitSessionReply({
userId: user.userId,
sessionId,
requestId: replyRequestId,
userMessage,
})
: null,
},
);
const workingDir = publishLayout?.publishDir ?? (await userAuth.resolveWorkingDir(user.userId));
const linkExistsForRequest = resolveLinkExistsForWorkingDir(workingDir, linkExists);
@@ -2302,6 +2500,7 @@ export function createWechatMpService({
const supportedByConfig =
(intent.msgType === 'voice' && config.acceptVoice) ||
(intent.msgType === 'image' && config.acceptImage) ||
(intent.msgType === 'file' && config.acceptFile) ||
(intent.msgType === 'location' && config.acceptLocation) ||
(intent.msgType === 'link' && config.acceptLink) ||
intent.msgType === 'text' ||
@@ -2350,6 +2549,26 @@ export function createWechatMpService({
};
}
const mediaAnalysisEnabled = isWechatMediaGrayUser(
boundUser,
config.mediaAnalysisGrayUsers,
);
attachRecentMediaForFollowup(inbound.fromUserName, intent, mediaAnalysisEnabled);
if (intent.msgType === 'file' && !mediaAnalysisEnabled) {
await persistIntentDetail({ intent, userId: boundUser.userId, rawXmlHash });
return {
ok: true,
status: 200,
contentType: 'application/xml; charset=utf-8',
body: buildWechatTextReply({
toUserName: inbound.fromUserName,
fromUserName: inbound.toUserName,
content: '当前账号尚未开启服务号文件分析灰度,请先通过 H5 上传文件。',
}),
};
}
if (intent.msgType === 'voice' && !intent.agentText.trim() && intent.media?.mediaId) {
try {
const fallbackText = await transcribeWechatVoiceMedia(intent.media.mediaId, intent.media.format);
@@ -2418,6 +2637,7 @@ export function createWechatMpService({
source: persisted.source,
};
intent.agentText = `[图片1]: ${persisted.publicUrl}`;
rememberRecentMedia(inbound.fromUserName, intent);
} catch (error) {
await persistIntentDetail({ intent, userId: boundUser.userId, rawXmlHash });
return {
@@ -2433,6 +2653,57 @@ export function createWechatMpService({
}
}
if (intent.msgType === 'file') {
try {
const accessToken = await getStableAccessToken();
const persisted = await persistWechatAttachment(
{
userId: boundUser.userId,
appId: config.appId,
openid: inbound.fromUserName,
msgId: inbound.msgId,
mediaId: inbound.mediaId,
filename: inbound.fileName,
publicBaseUrl: config.mediaPublicBaseUrl,
maxFileBytes: config.maxFileBytes,
},
{
wechatFetch,
accessToken,
},
);
intent.media = {
...intent.media,
mediaId: inbound.mediaId || intent.media?.mediaId || '',
publicUrl: persisted.publicUrl,
format: persisted.contentType,
source: persisted.source,
};
intent.attachment = {
...intent.attachment,
filename: persisted.filename,
publicUrl: persisted.publicUrl,
mimeType: persisted.contentType,
sizeBytes: persisted.bytes,
};
intent.agentText = `[文件1: ${persisted.filename}]: ${persisted.publicUrl}`;
intent.displayText = `文件:${persisted.filename}`;
rememberRecentMedia(inbound.fromUserName, intent);
} catch (error) {
await persistIntentDetail({ intent, userId: boundUser.userId, rawXmlHash });
return {
ok: true,
status: 200,
contentType: 'application/xml; charset=utf-8',
body: buildWechatTextReply({
toUserName: inbound.fromUserName,
fromUserName: inbound.toUserName,
content: error instanceof Error ? error.message : '文件处理失败,请稍后重试。',
}),
};
}
}
if (
intent.msgType === 'location' &&
(intent.location?.latitude == null || intent.location?.longitude == null)
@@ -2555,12 +2826,15 @@ export function createWechatMpService({
};
}
const task = runIntentMessage({
inbound,
intent,
user: boundUser,
})
const task = enqueueMessageTask(inbound.fromUserName, () =>
runIntentMessage({
inbound,
intent,
user: boundUser,
}),
)
.then(async ({ sessionId } = {}) => {
settleRecentMediaBatch(inbound.fromUserName, intent, { succeeded: true });
if (inbound.msgId && typeof userAuth.finishWechatMpMessage === 'function') {
await userAuth.finishWechatMpMessage({
appId: config.appId,
@@ -2572,6 +2846,7 @@ export function createWechatMpService({
}
})
.catch(async (err) => {
settleRecentMediaBatch(inbound.fromUserName, intent, { succeeded: false });
if (inbound.msgId && typeof userAuth.finishWechatMpMessage === 'function') {
await userAuth.finishWechatMpMessage({
appId: config.appId,
+554 -1
View File
@@ -79,6 +79,7 @@ function createBoundWechatService({
config = {},
scheduleService = null,
applySessionLlmProvider = null,
submitSessionReply = null,
}) {
return createWechatMpService({
config: {
@@ -128,6 +129,7 @@ function createBoundWechatService({
},
startAgentSession,
sessionApiFetch,
submitSessionReply,
scheduleService,
applySessionLlmProvider,
wechatFetch,
@@ -2712,6 +2714,10 @@ test('isRecoverableWechatAgentSessionError detects poisoned tool_calls history',
);
assert.equal(isRecoverableWechatAgentSessionError('stale_session_poisoned_completion'), true);
assert.equal(isRecoverableWechatAgentSessionError('无权访问该会话'), true);
assert.equal(
isRecoverableWechatAgentSessionError('Session already has an active request. Cancel it first.'),
true,
);
assert.equal(isRecoverableWechatAgentSessionError('network timeout'), false);
});
@@ -2752,6 +2758,17 @@ test('sanitizeWechatAgentOutboundText replaces raw api errors with friendly text
);
});
test('assertWechatAgentReplyIsSendable rejects image inspection placeholders', () => {
assert.throws(
() => assertWechatAgentReplyIsSendable({ text: 'Let me look at the image first.' }),
/wechat_agent_incomplete_reply/,
);
assert.throws(
() => assertWechatAgentReplyIsSendable({ text: '我先看一下这张图片。' }),
/wechat_agent_incomplete_reply/,
);
});
test('wechat mp service recreates dedicated session when tool_calls error arrives via Finish assistant text', async () => {
const token = 'token';
const timestamp = '1710000000';
@@ -3680,12 +3697,13 @@ test('wechat mp service falls back to ASR when voice recognition is empty', asyn
}
});
test('wechat mp service persists image and routes image url into agent prompt', async () => {
test('wechat mp wildcard media access persists image and routes image url into agent prompt', async () => {
const token = 'token';
const timestamp = '1710000000';
const nonce = 'nonce';
const testUserId = 'test-user-image';
const prompts = [];
const metadataCalls = [];
const detailCalls = [];
const service = createWechatMpService({
config: {
@@ -3701,6 +3719,7 @@ test('wechat mp service persists image and routes image url into agent prompt',
unboundTextPrefix: '请先绑定',
progressDelayMs: 0,
maxImageBytes: 1024 * 1024,
mediaAnalysisGrayUsers: ['*'],
},
userAuth: {
async findWechatUserByOpenid() {
@@ -3747,6 +3766,7 @@ test('wechat mp service persists image and routes image url into agent prompt',
if (pathname === '/sessions/session-1/reply') {
const body = JSON.parse(init.body);
prompts.push(body.user_message.content[0].text);
metadataCalls.push(body.user_message.metadata);
return new Response('{}', { status: 200, headers: { 'Content-Type': 'application/json' } });
}
if (pathname === '/agent/harness_remember' || pathname === '/agent/harness_bootstrap') {
@@ -3809,10 +3829,543 @@ test('wechat mp service persists image and routes image url into agent prompt',
prompts[0],
/\[图片1\]: https:\/\/example\.com\/MindSpace\/test-user-image\/public\/wechat-mp\//,
);
assert.equal(metadataCalls.length, 1);
assert.equal(metadataCalls[0].source, 'wechat_mp');
assert.equal(metadataCalls[0].msgType, 'image');
assert.equal(metadataCalls[0].imageUrls.length, 1);
assert.match(metadataCalls[0].imageUrls[0], /\/public\/wechat-mp\//);
assert.equal(detailCalls.length, 1);
assert.match(detailCalls[0].mediaPublicUrl, /\/wechat-mp\//);
});
test('wechat mp image submission reuses the H5 prepared reply path', async () => {
const token = 'token';
const timestamp = '1710000000';
const nonce = 'nonce';
const testUserId = 'test-user-image-prepared';
const submitCalls = [];
const service = createBoundWechatService({
token,
config: {
mediaAnalysisGrayUsers: [testUserId],
},
userAuth: {
async findWechatUserByOpenid() {
return { userId: testUserId, status: 'active', nickname: '唐' };
},
},
sessionApiFetch: async (_sessionId, pathname) => {
if (pathname === '/sessions/session-1/events') {
return new Response(
[
'data: {"type":"Message","message":{"id":"assistant-1","role":"assistant","metadata":{"userVisible":true},"content":[{"type":"text","text":"图片内容已识别。"}]}}\n\n',
'data: {"type":"Finish"}\n\n',
].join(''),
{ status: 200, headers: { 'Content-Type': 'text/event-stream' } },
);
}
if (pathname === '/agent/harness_remember' || pathname === '/agent/harness_bootstrap') {
return new Response('{}', { status: 200, headers: { 'Content-Type': 'application/json' } });
}
throw new Error(`unexpected api path: ${pathname}`);
},
submitSessionReply: async (input) => {
submitCalls.push(input);
return { ok: true };
},
wechatFetch: async (url) => {
if (String(url).includes('/cgi-bin/stable_token')) {
return new Response(JSON.stringify({ access_token: 'access-1', expires_in: 7200 }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
if (String(url).includes('/cgi-bin/media/get')) {
return new Response(Buffer.from([0x89, 0x50, 0x4e, 0x47]), {
status: 200,
headers: { 'Content-Type': 'image/png' },
});
}
if (String(url).includes('/cgi-bin/message/custom/send')) {
return new Response(JSON.stringify({ errcode: 0, errmsg: 'ok' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
throw new Error(`unexpected wechat url: ${url}`);
},
});
try {
const result = await service.handleInboundMessage(
inboundXml({
msgType: 'image',
content: '',
extraFields: { MediaId: 'media-prepared', PicUrl: 'https://wx.example.com/image.png' },
}),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
await result.task;
} finally {
fs.rmSync(path.join(process.cwd(), 'MindSpace', testUserId), { recursive: true, force: true });
}
assert.equal(submitCalls.length, 1);
assert.equal(submitCalls[0].userId, testUserId);
assert.equal(submitCalls[0].sessionId, 'session-1');
assert.equal(submitCalls[0].userMessage.metadata.imageUrls.length, 1);
assert.match(submitCalls[0].userMessage.metadata.imageUrls[0], /\/public\/wechat-mp\//);
});
test('wechat mp serializes image and follow-up text and reattaches recent image', async () => {
const token = 'token';
const timestamp = '1710000000';
const nonce = 'nonce';
const testUserId = 'test-user-image-followup';
const submitCalls = [];
let eventCall = 0;
let releaseFirst = null;
const service = createBoundWechatService({
token,
config: {
mediaAnalysisGrayUsers: [testUserId],
},
userAuth: {
async findWechatUserByOpenid() {
return { userId: testUserId, status: 'active', nickname: '唐' };
},
},
sessionApiFetch: async (_sessionId, pathname) => {
if (pathname === '/sessions/session-1/events') {
eventCall += 1;
if (eventCall === 1) {
return new Response(
new ReadableStream({
start(controller) {
releaseFirst = () => {
controller.enqueue(
new TextEncoder().encode(
'data: {"type":"Message","message":{"id":"assistant-image","role":"assistant","metadata":{"userVisible":true},"content":[{"type":"text","text":"图片已识别。"}]}}\n\n' +
'data: {"type":"Finish"}\n\n',
),
);
controller.close();
};
},
}),
{ status: 200, headers: { 'Content-Type': 'text/event-stream' } },
);
}
return new Response(
[
'data: {"type":"Message","message":{"id":"assistant-followup","role":"assistant","metadata":{"userVisible":true},"content":[{"type":"text","text":"已结合刚才图片分析主题。"}]}}\n\n',
'data: {"type":"Finish"}\n\n',
].join(''),
{ status: 200, headers: { 'Content-Type': 'text/event-stream' } },
);
}
if (pathname === '/agent/harness_remember' || pathname === '/agent/harness_bootstrap') {
return new Response('{}', { status: 200, headers: { 'Content-Type': 'application/json' } });
}
throw new Error(`unexpected api path: ${pathname}`);
},
submitSessionReply: async (input) => {
submitCalls.push(input);
return { ok: true };
},
wechatFetch: async (url) => {
if (String(url).includes('/cgi-bin/stable_token')) {
return new Response(JSON.stringify({ access_token: 'access-1', expires_in: 7200 }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
if (String(url).includes('/cgi-bin/media/get')) {
return new Response(Buffer.from([0x89, 0x50, 0x4e, 0x47]), {
status: 200,
headers: { 'Content-Type': 'image/png' },
});
}
if (String(url).includes('/cgi-bin/message/custom/send')) {
return new Response(JSON.stringify({ errcode: 0, errmsg: 'ok' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
throw new Error(`unexpected wechat url: ${url}`);
},
});
try {
const imageResult = await service.handleInboundMessage(
inboundXml({
msgType: 'image',
content: '',
extraFields: { MediaId: 'media-followup', PicUrl: 'https://wx.example.com/image.png' },
}),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
while (submitCalls.length === 0) await new Promise((resolve) => setTimeout(resolve, 0));
const followupResult = await service.handleInboundMessage(
inboundXml({ msgType: 'text', content: '请根据刚才图片分析主题' }),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(submitCalls.length, 1);
releaseFirst();
await imageResult.task;
await followupResult.task;
} finally {
fs.rmSync(path.join(process.cwd(), 'MindSpace', testUserId), { recursive: true, force: true });
}
assert.equal(submitCalls.length, 2);
assert.deepEqual(
submitCalls[1].userMessage.metadata.imageUrls,
submitCalls[0].userMessage.metadata.imageUrls,
);
assert.equal(submitCalls[1].userMessage.metadata.msgType, 'text');
});
test('wechat mp groups consecutive images for one follow-up and clears the consumed batch', async () => {
const token = 'token';
const timestamp = '1710000000';
const nonce = 'nonce';
const testUserId = 'test-user-multi-image-followup';
const submitCalls = [];
let eventCall = 0;
let releaseFirst = null;
const service = createBoundWechatService({
token,
config: {
mediaAnalysisGrayUsers: [testUserId],
},
userAuth: {
async findWechatUserByOpenid() {
return { userId: testUserId, status: 'active', nickname: '唐' };
},
},
sessionApiFetch: async (_sessionId, pathname) => {
if (pathname === '/sessions/session-1/events') {
eventCall += 1;
if (eventCall === 1) {
return new Response(
new ReadableStream({
start(controller) {
releaseFirst = () => {
controller.enqueue(
new TextEncoder().encode(
'data: {"type":"Message","message":{"id":"assistant-first","role":"assistant","metadata":{"userVisible":true},"content":[{"type":"text","text":"第一张处理完成。"}]}}\n\n' +
'data: {"type":"Finish"}\n\n',
),
);
controller.close();
};
},
}),
{ status: 200, headers: { 'Content-Type': 'text/event-stream' } },
);
}
return new Response(
[
'data: {"type":"Message","message":{"id":"assistant-ok","role":"assistant","metadata":{"userVisible":true},"content":[{"type":"text","text":"处理完成。"}]}}\n\n',
'data: {"type":"Finish"}\n\n',
].join(''),
{ status: 200, headers: { 'Content-Type': 'text/event-stream' } },
);
}
if (pathname === '/agent/harness_remember' || pathname === '/agent/harness_bootstrap') {
return new Response('{}', { status: 200, headers: { 'Content-Type': 'application/json' } });
}
throw new Error(`unexpected api path: ${pathname}`);
},
submitSessionReply: async (input) => {
submitCalls.push(input);
return { ok: true };
},
wechatFetch: async (url) => {
if (String(url).includes('/cgi-bin/stable_token')) {
return new Response(JSON.stringify({ access_token: 'access-1', expires_in: 7200 }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
if (String(url).includes('/cgi-bin/media/get')) {
return new Response(Buffer.from([0x89, 0x50, 0x4e, 0x47]), {
status: 200,
headers: { 'Content-Type': 'image/png' },
});
}
if (String(url).includes('/cgi-bin/message/custom/send')) {
return new Response(JSON.stringify({ errcode: 0, errmsg: 'ok' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
throw new Error(`unexpected wechat url: ${url}`);
},
});
try {
const firstImageResult = await service.handleInboundMessage(
inboundXml({
msgType: 'image',
content: '',
extraFields: {
MsgId: '10011',
MediaId: 'media-first',
PicUrl: 'https://wx.example.com/media-first.png',
},
}),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
while (submitCalls.length === 0) await new Promise((resolve) => setTimeout(resolve, 0));
const secondImageResult = await service.handleInboundMessage(
inboundXml({
msgType: 'image',
content: '',
extraFields: {
MsgId: '10012',
MediaId: 'media-second',
PicUrl: 'https://wx.example.com/media-second.png',
},
}),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
const followupResult = await service.handleInboundMessage(
inboundXml({
content: '请结合刚才两张图片分析主题',
extraFields: { MsgId: '10013' },
}),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(submitCalls.length, 1);
releaseFirst();
await firstImageResult.task;
await secondImageResult.task;
await followupResult.task;
const laterResult = await service.handleInboundMessage(
inboundXml({
content: '请再次分析刚才图片',
extraFields: { MsgId: '10014' },
}),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
await laterResult.task;
} finally {
fs.rmSync(path.join(process.cwd(), 'MindSpace', testUserId), { recursive: true, force: true });
}
assert.equal(submitCalls.length, 4);
assert.equal(submitCalls[0].userMessage.metadata.imageUrls.length, 1);
assert.equal(submitCalls[1].userMessage.metadata.imageUrls.length, 1);
assert.equal(submitCalls[2].userMessage.metadata.imageUrls.length, 2);
assert.match(submitCalls[2].userMessage.metadata.imageUrls[0], /media-first/);
assert.match(submitCalls[2].userMessage.metadata.imageUrls[1], /media-second/);
assert.equal(submitCalls[2].userMessage.metadata.msgType, 'text');
assert.equal(submitCalls[3].userMessage.metadata.imageUrls, undefined);
});
test('wechat mp service persists Word and Excel files in user public area and reuses H5 attachment metadata', async () => {
const token = 'token';
const timestamp = '1710000000';
const nonce = 'nonce';
const testUserId = 'test-user-wechat-file';
const userMessages = [];
const service = createBoundWechatService({
token,
config: {
maxFileBytes: 1024 * 1024,
acceptFile: true,
mediaAnalysisGrayUsers: [testUserId],
},
userAuth: {
async findWechatUserByOpenid() {
return { userId: testUserId, status: 'active', nickname: '毕升' };
},
},
sessionApiFetch: async (_sessionId, pathname, init = {}) => {
if (pathname === '/sessions/session-1/events') {
return new Response(
[
'data: {"type":"Message","request_id":"req-file","message":{"id":"assistant-1","role":"assistant","metadata":{"userVisible":true},"content":[{"type":"text","text":"文件已解析。"}]}}\n\n',
'data: {"type":"Finish","request_id":"req-file","token_state":{"inputTokens":1,"outputTokens":2}}\n\n',
].join(''),
{ status: 200, headers: { 'Content-Type': 'text/event-stream' } },
);
}
if (pathname === '/sessions/session-1/reply') {
const body = JSON.parse(init.body);
userMessages.push(body.user_message);
return new Response('{}', { status: 200, headers: { 'Content-Type': 'application/json' } });
}
if (pathname === '/agent/harness_remember' || pathname === '/agent/harness_bootstrap') {
return new Response('{}', { status: 200, headers: { 'Content-Type': 'application/json' } });
}
throw new Error(`unexpected api path: ${pathname}`);
},
wechatFetch: async (url) => {
if (String(url).includes('/cgi-bin/stable_token')) {
return new Response(JSON.stringify({ access_token: 'access-1', expires_in: 7200 }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
if (String(url).includes('/cgi-bin/media/get')) {
return new Response(Buffer.from('fake-docx-content'), {
status: 200,
headers: { 'Content-Type': 'application/octet-stream' },
});
}
if (String(url).includes('/cgi-bin/message/custom/send')) {
return new Response(JSON.stringify({ errcode: 0, errmsg: 'ok' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
throw new Error(`unexpected wechat url: ${url}`);
},
});
const originalRandomUuid = crypto.randomUUID;
crypto.randomUUID = () => 'req-file';
try {
for (const [index, filename] of ['季度分析.docx', '销售数据.xlsx'].entries()) {
const result = await service.handleInboundMessage(
inboundXml({
msgType: 'file',
content: '',
extraFields: {
MediaId: `file-media-${index + 1}`,
FileName: filename,
FileSize: 17,
},
}),
{ timestamp, nonce, signature: signatureFor(token, timestamp, nonce) },
);
assert.equal(result.status, 200);
await result.task;
const attachment = userMessages.at(-1)?.metadata?.fileAttachments?.[0];
const publicFilename = decodeURIComponent(new URL(attachment.downloadUrl).pathname.split('/').at(-1));
assert.equal(
fs.existsSync(
path.join(process.cwd(), 'MindSpace', testUserId, 'public', 'wechat-mp', publicFilename),
),
true,
);
}
} finally {
crypto.randomUUID = originalRandomUuid;
fs.rmSync(path.join(process.cwd(), 'MindSpace', testUserId), {
recursive: true,
force: true,
});
}
assert.equal(userMessages.length, 2);
assert.match(userMessages[0].content[0].text, /【微信服务号文件消息】/);
assert.match(userMessages[0].content[0].text, /\[文件1: 季度分析\.docx\]:/);
assert.equal(userMessages[0].metadata.msgType, 'file');
assert.equal(userMessages[0].metadata.fileAttachments.length, 1);
assert.equal(userMessages[0].metadata.fileAttachments[0].filename, '季度分析.docx');
assert.equal(
userMessages[0].metadata.fileAttachments[0].mimeType,
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
);
assert.match(userMessages[0].metadata.fileAttachments[0].downloadUrl, /\/public\/wechat-mp\//);
assert.match(userMessages[1].content[0].text, /\[文件1: 销售数据\.xlsx\]:/);
assert.equal(userMessages[1].metadata.fileAttachments[0].filename, '销售数据.xlsx');
assert.equal(
userMessages[1].metadata.fileAttachments[0].mimeType,
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
);
assert.match(userMessages[1].metadata.fileAttachments[0].downloadUrl, /\/public\/wechat-mp\//);
});
test('wechat mp service rejects unsupported public file types without entering the agent flow', async () => {
let mediaDownloads = 0;
let sessionCalls = 0;
const service = createBoundWechatService({
config: { acceptFile: true, mediaAnalysisGrayUsers: ['user-1'] },
sessionApiFetch: async () => {
sessionCalls += 1;
throw new Error('session should not be called');
},
wechatFetch: async (url) => {
if (String(url).includes('/cgi-bin/stable_token')) {
return new Response(JSON.stringify({ access_token: 'access-1', expires_in: 7200 }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
if (String(url).includes('/cgi-bin/media/get')) mediaDownloads += 1;
throw new Error(`unexpected wechat url: ${url}`);
},
});
const result = await service.handleInboundMessage(
inboundXml({
msgType: 'file',
content: '',
extraFields: { MediaId: 'file-media-2', FileName: 'payload.exe' },
}),
{
timestamp: '1710000000',
nonce: 'nonce',
signature: signatureFor('token', '1710000000', 'nonce'),
},
);
assert.equal(result.status, 200);
assert.match(result.body, /仅支持 Word/);
assert.equal(mediaDownloads, 0);
assert.equal(sessionCalls, 0);
});
test('wechat mp service keeps file analysis disabled outside the media gray allowlist', async () => {
let mediaDownloads = 0;
let sessionCalls = 0;
const service = createBoundWechatService({
config: {
acceptFile: true,
mediaAnalysisGrayUsers: ['唐'],
},
sessionApiFetch: async () => {
sessionCalls += 1;
throw new Error('session should not be called');
},
wechatFetch: async (url) => {
if (String(url).includes('/cgi-bin/media/get')) mediaDownloads += 1;
throw new Error(`unexpected wechat url: ${url}`);
},
});
const result = await service.handleInboundMessage(
inboundXml({
msgType: 'file',
content: '',
extraFields: { MediaId: 'file-media-gray', FileName: '测试.docx' },
}),
{
timestamp: '1710000000',
nonce: 'nonce',
signature: signatureFor('token', '1710000000', 'nonce'),
},
);
assert.equal(result.status, 200);
assert.match(result.body, /尚未开启服务号文件分析灰度/);
assert.equal(mediaDownloads, 0);
assert.equal(sessionCalls, 0);
});
test('wechat mp service accepts full voice xml payload and routes recognition text into agent', async () => {
const token = 'token';
const timestamp = '1710000000';
+7 -1
View File
@@ -21,6 +21,12 @@ const ImageBuilder = {
build: (ctx) => buildText(TEMPLATES.image, ctx),
};
const FileBuilder = {
priority: 80,
support: (ctx) => ctx.msgType === 'file',
build: (ctx) => buildText(TEMPLATES.file, ctx),
};
const VoiceBuilder = {
priority: 80,
support: (ctx) => ctx.msgType === 'voice',
@@ -54,7 +60,7 @@ const DefaultBuilder = {
build: (ctx) => buildText(TEMPLATES.default, ctx),
};
const BUILDERS = [ImageBuilder, VoiceBuilder, LocationBuilder, LinkBuilder, IntentBuilder, DefaultBuilder]
const BUILDERS = [ImageBuilder, FileBuilder, VoiceBuilder, LocationBuilder, LinkBuilder, IntentBuilder, DefaultBuilder]
.sort((a, b) => b.priority - a.priority);
export function selectBuilder(ctx) {
+1
View File
@@ -13,6 +13,7 @@ const INTENT_RULES = [
export function resolveIntent(msgType, text) {
if (msgType === 'image') return { task: 'image_analysis' };
if (msgType === 'file') return { task: 'file_analysis' };
if (msgType === 'voice') return { task: 'voice_analysis' };
if (msgType === 'location') return { task: 'location' };
if (msgType === 'link') return { task: 'link' };
+5
View File
@@ -26,6 +26,11 @@ describe('buildAckText', () => {
assert.equal(result, '图片收到,我先看看。');
});
it('file → file template', () => {
const result = buildAckText({ intent: intent('file'), nickname: '', config: cfg, fallbackText: '' });
assert.equal(result, '文件收到,我先读取分析。');
});
it('voice → voice template', () => {
const result = buildAckText({ intent: intent('voice'), nickname: '', config: cfg, fallbackText: '' });
assert.equal(result, '收到语音,我先听一下。');
+1
View File
@@ -12,6 +12,7 @@ export const TEMPLATES = {
'让我看看这张图片。',
'图片已收到,我来处理。',
],
file: ['文件收到,我先读取分析。'],
voice: [
'收到语音,我先听一下。',
'听到啦,我马上处理。',
+14
View File
@@ -81,6 +81,20 @@ export function buildWechatAgentPrompt(intent, { grantedSkills = [] } = {}) {
.filter(Boolean)
.join('\n');
}
if (msgType === 'file') {
const attachment = intent?.attachment ?? {};
return [
currentTimeHint,
'【微信服务号文件消息】用户发送了需要解析的 Office 文件。',
attachment.filename ? `文件名:${attachment.filename}` : '',
attachment.publicUrl ? `文件链接:${attachment.publicUrl}` : '',
'请复用 H5 附件分析结果回答;Excel 在专用工具可用时必须优先使用 Excel 工具读取完整工作簿。',
'',
String(agentText).trim(),
]
.filter(Boolean)
.join('\n');
}
if (msgType === 'location') {
const location = intent?.location ?? {};
return [