Daily Tech Briefing
AI 科技速览
每天 5 分钟内学习 AI。获取最新的人工智能新闻,理解其重要性,并学习如何将其应用于您的工作。
arXiv Machine Learning · 2026/7/31 14:49:18
Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?
AI 中文解读
干预数据一直被视为教AI学会因果推理的“金标准”,但最新研究发现,这个常识并不总是成立。在两类信号方向相反的“辛普森悖论”环境中,训练数据里加入再多的干预样本,AI最终学到手的因果方向依然是错的——错误并非来自训练本身,而是藏在推理时的上下文里。同样一个训练好的模型,上下文里存在观察数据时,50种设定中就有29种出现系统的方向反转;一旦把这些干扰信息清空,被压抑的因果判断力立刻恢复。研究人员还定位出关键开关位于模型中间层,并通过一种“证据平均”协议把因果方向判断的错误率从26%压到9%。业内人士认为,这提醒我们AI的因果推理能力并不稳定,输入提示词的细微变化就可能翻转它的判断。未来那些依赖AI做因果决策的医疗、经济分析,不能只看它给出的结论,更得留意它看到的上下文,才能避免被表面的相关性误导。
Interventional data is widely regarded as the gold standard for teaching models causal reasoning. We test this assumption in a fully controlled synthetic environment pitting observational correlation against causal effect, and find it fails instructively. In Simpson's-paradox worlds, where the two have systematically opposite signs, increasing the fraction of interventional samples in pretraining does not improve causal direction: the magnitude of the model's do()-response grows monotonically, yet its sign is copied from the observational context. What governs whether interventional evidence is used is not the training mixture but the evidence type present in the context at inference time. Under an identical training recipe, a purely observational context induces systematic sign reversal in 29/50 worlds, a mixed context in 19/50, while aligned interventional probes alone yield 41/50 correct. Erasing observational evidence from the context immediately releases the suppressed causal interpolation ability (ratio_true = +0.56); a four-state content manipulation shows the switch is content-mediated and graded. The suppression is stable across training seeds (11/11 strong reversals persist on a matched-protocol second seed) and robust as a rate at 0.93B parameters (31.8% vs. 6% reversals in the matched probe-only arm), even as absolute gains shrink four-fold. An external audit on CLadder exposes a learned positive-effect prior with a two-layer structure: sign-randomized retraining removes it in-distribution but not out-of-distribution. We summarize: the capability lives in the weights; the switch lives in the context, and activation patching localizes the switch to the middle layers' observational rows. We further quantify the sampling noise floor of probe-based causal evaluation and an evidence-averaging protocol that cuts sign errors from 26% to 9%.
分享
阅读原文 ↗