The Art of Not Forgetting A Local Learning Architecture for Continual Learning

AI 中文解读

一种让AI在学习新知识时“不忘旧事”的新架构在学界引起关注。这篇论文提出的CMP（认知记忆原型）方法，让AI用类似人类记忆的方式存储信息：把输入转化成稀疏的关系编码，放入一个双层竞争记忆库，通过局部更新来学习，而不是依赖传统的全网络反向传播。实验表明，在让AI逐渐学会多门编程语言的任务中，采用CMP的模型“忘掉”旧知识的程度只有传统Transformer加记忆保护方法的一半左右，遗忘行为非常稳定。这个成果的意义在于，当前AI大模型普遍存在“学新忘旧”的痛点——比如你教会它写Python代码，它可能就把以前学的Java忘干净了。CMP通过类似人脑的“记忆空间”设计，让AI能持续积累知识而不丢失，这为打造真正能终身学习的智能助手铺平了道路。未来，你的智能音箱学会新方言后依然能流利讲普通话，或者办公AI学完新软件操作后仍熟练使用旧功能，都将不再是难题。

arXiv:2607.26523v1 Announce Type: new Abstract: We introduce CMP (Cognitive Memory Primitive), a continual-learning architecture that repre?sents inputs as sparse relational codes, stores them in a two-tier competitive memory, and learns through local updates without end-to-end backpropagation through its feature-generating system. We investigate whether combining sparse representations, local learning, and persistent memory can reduce catastrophic forgetting relative to conventional backpropagation-based continual?learning approaches. On a controlled domain-incremental byte-level language modeling protocol, CMP demonstrates substantially lower backward transfer than a parameter-matched Trans?former trained with online Elastic Weight Consolidation (EWC). Across a three-seed replicated 15-domain experiment, CMP exhibits stable forgetting behavior, while separate head-to-head comparisons and domain-order analyses show consistently lower forgetting than the evaluated Transformer baseline under the reported experimental settings. We report these findings alongside a substantial single-domain accuracy gap relative to the Transformer, a null result on a vision benchmark, and a documented failure to combine CMP with an independent accuracy-improving mechanism, reflecting our commitment to reporting both positive and negative outcomes. These results suggest that the combination of sparse representations, local learning, and persistent memory is a promising direction for continual learning, while motivating further investigation into the respective roles of learning rules, representations, and architectural design in mitigating catastrophic forgetting.