The Information Shadow: Measuring Structural Limits on What Language Models Can Learn

AI 中文解读

1. 核心亮点：研究发现语言模型存在“知识天花板”——无论训练数据再多、模型规模再大，有些东西它永远学不会，这不是因为资料不够，而是学习方式本身的结构性限制。 2. 通俗解读：就像你永远无法通过看菜谱学会“荔枝味”是什么感觉，语言模型也面临类似的困境。科研人员把这个限制区域叫做“信息阴影”，并把它分成三类：第一类，有些现象根本没法用文字准确描述（比如气味、触感）；第二类，数据里藏着两种相反的解释，模型只能靠瞎猜来决定信哪一个；第三类，即使理论上模型有能力学会某个规律，但正常的训练过程就是找不到通往这个规律的路径，就像在迷宫里明明有出口，但所有路都通向死胡同。研究人员还设计了巧妙的测试来证明这些限制确实存在，比如让模型在两种矛盾的规则下训练，结果模型的表现完全取决于它最初的“偏见”。 3. 实际影响：这意味着普通人不能指望AI成为“全知全能”的智能体——它可能会在看似简单的问题上犯下结构性错误，而这种错误不是多喂数据就能解决的。对于开发者而言，在设计AI评测标准时，需要警惕那些隐藏在“信息阴影”里的盲区，避免把模型的无能为力误判为训练不足。未来AI的改进方向可能不再是单纯堆数据和算力，而是考虑如何绕过这些结构性限制。

arXiv:2607.18305v1 Announce Type: new Abstract: Some limits on what language models know are not gaps in data coverage but structural properties of learning from text. We introduce the information shadow: the region of phenomena that a text-trained learner cannot acquire regardless of scale, comprising (I) structures language cannot express, (II) functions that are statistically non-identifiable from the training distribution, and (III) functions that are representable but unreachable by gradient-based training. We give each type a probe that is decisive because the premise of the shadow is, in that setting, provable. For Type I, Language Compression Residuals compare a text learner, which sees only a lossy text-like encoding of the signal, against a full-signal learner, which sees the underlying signal directly. The text learner sits at a computable expressibility ceiling while the full-signal learner pulls away by a gap that stays flat across 300x more data, so the deficit is a property of the channel, not of training. For Type II, the Counterfactual Distinction Test trains models on data exactly consistent with two incompatible rules. Across a provable string task and a language-like agreement task, behavior on counterfactuals is set by the model's inductive bias, while 5% disambiguating data steers the learned rule bidirectionally to either target (r = +/-1.0, p < 1e-10). For Type III, Basin Escape Mapping exhibits a function that is representable at 100% (by hand construction) yet reached 0% of the time by standard training and instantly from a nearby initialization, with width scaling providing no help (p = 1.6 x 10^-14). Each effect is isolated by a control that rules out a capacity or modality artifact. We release the probe suite and discuss implications for benchmark design, capability auditing, and shadow-aware uncertainty.