ReToken: One Token to Improve Vision-Language Models for Visual Retrieval

AI 中文解读

核心亮点：给AI装上一个“智能筛选器”，让它看再多杂乱图片视频也能迅速找到你要的东西，性能大幅提升。通俗解读：以往AI理解长视频或密集图片时，就像在堆满杂物的房间里找钥匙，东西越多越容易蒙，还会卡到“内存不足”。新方法ReToken相当于教AI先看一眼问题，再只挑出与问题相关的几个画面重点分析，其余忽略。这套方法只需用少量图片问题训练，就能让AI在图片海和长视频中精准定位目标，分数提升超两成，而且一台普通H100显卡就能完成，几乎没有额外硬件门槛。实际影响：普通人使用AI搜索时，以后再也不用担心视频太长、图片太多AI“看不过来”或反应迟钝。无论是让AI在回放录像里找宠物在哪，还是在数百张资料图中找出某张设计图，都能更快更准。这也意味着AI视频问答、智能相册检索、影视分析等应用会变得更实用，未来手机和电脑上的AI助手处理海量视觉信息会像翻自己相册一样轻松自然。

Long visual context poses a challenge for vision-language models: performance degrades as the number of distractors grows, and processing all tokens at once is computationally infeasible under GPU memory constraints. We present ReToken, a single learnable embedding trained as an explicit retrieval target that selects a sparse set of query-relevant visual tokens from a pre-filled visual KV cache. Trained on only a small image-QA dataset, ReToken yields consistent gains across image and video benchmarks: on Visual Haystacks it improves Qwen3VL-8B by 13.4 points and InternVL3.5 by 12.4 points (>20% relative), and on LVBench it transfers zero-shot to long video for an 8.0-point gain with Qwen3VL-8B. Thanks to its lightweight design, both training and long-video inference fit on a single H100. Code is available at: https://github.com/avaxiao/ReToken