fix(vision): stop read_image from poisoning text-provider sessions
Image turns already get a vision-model description injected into the prompt, but the agent kept calling read_image to "confirm" the pictures. Those tool results carry base64 image parts that Goose persists, so every later turn against the text-only chat provider failed with `unknown variant image_url` before the agent could write the page. WeChat page requests therefore fell through to the fail-closed delivery message. Drop read_image for the turn whenever a vision model handles the images, say so explicitly in the injected prompt, and teach the poison scan to recognise tool image parts so already-polluted sessions rotate instead of failing again. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
+16
-1
@@ -966,6 +966,12 @@ export async function buildVisionPayload({
|
||||
(visionDescription
|
||||
? `Qwen VL 图片描述:\n${visionDescription}\n\n`
|
||||
: '') +
|
||||
'视觉检查已由图片模型完成,本轮不会再提供读图工具。'
|
||||
+ '禁止调用 read_image 或任何读图工具去「确认图片内容」:当前聊天模型不接受图片工具结果,'
|
||||
+ '强行读图会让本轮及后续请求全部失败。'
|
||||
+ (visionDescription
|
||||
? '直接依据上面的描述写文案。\n'
|
||||
: '本轮没有拿到图片描述时,不要臆造画面细节,按用户文字要求组织内容,并把下面的图片路径原样嵌进页面。\n') +
|
||||
'写作约束:不得改写图片里人物的年龄、性别、人数或主体关系;如果用户明确要求儿童语气或童趣风格,也只能调整表达方式,不能把图片主体改写成儿童场景。\n' +
|
||||
`图片 HTML 嵌入路径(直接写入 <img> 标签;以下都是无需 cookie 的公开压缩标准图片,禁止使用 local://、/users/ 私有路径、原图地址或需要登录态的下载链接):\n${pathList}\n` +
|
||||
'执行要求:必须先调用 load_skill → static-page-publish(每次生成页面都要调用,不可省略),' +
|
||||
@@ -1979,11 +1985,20 @@ export function createTkmindProxy({
|
||||
throw err;
|
||||
}
|
||||
|
||||
// REGRESSION GUARD: vision-turn-read-image-isolation — the vision provider
|
||||
// already describes this turn's images in the injected prompt. Leaving
|
||||
// read_image available lets the agent re-read the originals and persist
|
||||
// base64 image parts that the text-only chat provider then rejects for
|
||||
// every later turn, so drop the tool before the turn starts.
|
||||
const visionHandlesThisTurn = Boolean(llmProviderService)
|
||||
&& messageHasImages(userMessage)
|
||||
&& await llmProviderService.hasVisionKey().catch(() => false);
|
||||
|
||||
await reconcileSessionPolicyForUser(userId, sessionId, {
|
||||
toolMode,
|
||||
query: firstUserText(userMessage),
|
||||
forceDeepReasoning,
|
||||
disableImageReading,
|
||||
disableImageReading: disableImageReading || visionHandlesThisTurn,
|
||||
});
|
||||
await applySessionLlmProvider(sessionId);
|
||||
await repairSessionToolHistory(sessionId);
|
||||
|
||||
Reference in New Issue
Block a user