fix(vision): stop read_image from poisoning text-provider sessions

Image turns already get a vision-model description injected into the prompt,
but the agent kept calling read_image to "confirm" the pictures. Those tool
results carry base64 image parts that Goose persists, so every later turn
against the text-only chat provider failed with `unknown variant image_url`
before the agent could write the page. WeChat page requests therefore fell
through to the fail-closed delivery message.

Drop read_image for the turn whenever a vision model handles the images, say
so explicitly in the injected prompt, and teach the poison scan to recognise
tool image parts so already-polluted sessions rotate instead of failing again.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
john
2026-08-22 10:40:12 +08:00
parent 4d12ea438b
commit ef4ce12bbf
9 changed files with 457 additions and 6 deletions
+16 -1
View File
@@ -966,6 +966,12 @@ export async function buildVisionPayload({
(visionDescription
? `Qwen VL 图片描述:\n${visionDescription}\n\n`
: '') +
'视觉检查已由图片模型完成,本轮不会再提供读图工具。'
+ '禁止调用 read_image 或任何读图工具去「确认图片内容」:当前聊天模型不接受图片工具结果,'
+ '强行读图会让本轮及后续请求全部失败。'
+ (visionDescription
? '直接依据上面的描述写文案。\n'
: '本轮没有拿到图片描述时,不要臆造画面细节,按用户文字要求组织内容,并把下面的图片路径原样嵌进页面。\n') +
'写作约束:不得改写图片里人物的年龄、性别、人数或主体关系;如果用户明确要求儿童语气或童趣风格,也只能调整表达方式,不能把图片主体改写成儿童场景。\n' +
`图片 HTML 嵌入路径(直接写入 <img> 标签;以下都是无需 cookie 的公开压缩标准图片,禁止使用 local://、/users/ 私有路径、原图地址或需要登录态的下载链接):\n${pathList}\n` +
'执行要求:必须先调用 load_skill → static-page-publish(每次生成页面都要调用,不可省略),' +
@@ -1979,11 +1985,20 @@ export function createTkmindProxy({
throw err;
}
// REGRESSION GUARD: vision-turn-read-image-isolation — the vision provider
// already describes this turn's images in the injected prompt. Leaving
// read_image available lets the agent re-read the originals and persist
// base64 image parts that the text-only chat provider then rejects for
// every later turn, so drop the tool before the turn starts.
const visionHandlesThisTurn = Boolean(llmProviderService)
&& messageHasImages(userMessage)
&& await llmProviderService.hasVisionKey().catch(() => false);
await reconcileSessionPolicyForUser(userId, sessionId, {
toolMode,
query: firstUserText(userMessage),
forceDeepReasoning,
disableImageReading,
disableImageReading: disableImageReading || visionHandlesThisTurn,
});
await applySessionLlmProvider(sessionId);
await repairSessionToolHistory(sessionId);