Daily Tech Briefing
AI 科技速览
每天 5 分钟内学习 AI。获取最新的人工智能新闻,理解其重要性,并学习如何将其应用于您的工作。
arXiv Machine Learning · 2026/7/31 04:00:00
Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models
AI 中文解读
核心亮点:一项新研究揭示,医疗AI在回答问题前“自言自语”的推理过程,其摆放位置比声称的来源更能左右最终判断,这为开发更可靠的AI医生提供了新思路。
通俗解读:研究人员给医疗视觉AI出了1000道医学影像题,并设计了巧妙的实验:一种是重新提示AI,另一种是强迫它接着自己的推理往下说。结果发现,后者让AI更“听话”地跟推理走。更关键的是,他们给同样的推理贴上不同“标签”——说是医生写的或AI自己想的——AI根本不在意来源,真正影响判断的是这段推理出现在回答的哪个位置。另外,如果拿走影像证据,AI反而更依赖推理;而像“肿瘤在左边还是右边”这类左右信息,AI最不容易忠实跟踪。
实际影响:这项发现提醒我们,医疗AI的“思考”可能并非真心实意,而是被展示顺序牵着走。未来开发诊断系统时,不能只看它说得头头是道,更要检验推理是否真正驱动了结论。这对提升AI在医学场景的透明度和安全性有重要意义,也能让医生和患者更放心地参考AI的建议,避免被“花言巧语”误导。
arXiv:2607.27304v1 Announce Type: new
Abstract: Medical vision-language models (VLMs) generate chain-of-thought (CoT) reasoning before answering clinical questions, but whether this reasoning causally influences predictions remains unclear. We present CoT-Mediate, a behavioral framework that perturbs a single clinically meaningful attribute within a model's own generated reasoning and measures whether the resulting prediction follows the edited reasoning. Our framework combines a dual-arm protocol comparing re-prompted evidence with prefix-forced continuation, together with a provenance-controlled intervention that varies only the attributed source of identical reasoning to disentangle reasoning mediation from sycophancy. We evaluate LLaVA-Med and MedGemma on 1,000 VQA-RAD samples each. Prefix-forced continuation consistently yields higher mediation faithfulness than re-prompting, while the provenance analysis reveals distinct model-specific deference behaviors. Across both models, removing visual evidence increases reliance on injected reasoning, whereas laterality is the least faithfully tracked clinical attribute. These results show that the mechanism used to inject reasoning substantially affects measured faithfulness and that contextual position, rather than stated provenance, is the primary determinant of whether medical VLMs use their generated reasoning.
分享
阅读原文 ↗