Daily Tech Briefing
AI 科技速览
每天 5 分钟内学习 AI。获取最新的人工智能新闻,理解其重要性,并学习如何将其应用于您的工作。
arXiv AI · 2026/8/4 17:45:16
Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent
AI 中文解读
核心亮点:这项研究让AI从“看图说话”升级到“看视频做研究”,并首次在复杂视频问答上打败了Claude和GPT-5等顶级模型。
通俗解读:以前AI处理视频时,经常偷懒——要么直接跳过画面去搜文字,要么靠“背答案”蒙混过关。这个新系统像一位严格的侦探,强制AI先逐帧看清视频里的每个细节,再去网上查资料验证,最后才给出答案。它用“先看后查”的两步训练法,让AI学会自己探索,而不是死记硬背,相当于给AI装上了“视频版搜索引擎”。
实际影响:以后你上传一段演唱会视频,问“主唱一共换了几套衣服”,AI能准确回答;看网课时问“老师刚才在黑板画的公式是什么意思”,它也能结合画面和网络资料讲清楚。这意味着AI将能真正理解动态世界,在视频学习、医疗影像分析、自动驾驶等场景中发挥更大作用,让普通人用AI处理视频信息时更省心、更可靠。
We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration. Preliminary evaluations reveal two critical bottlenecks in current models: (1) modality bias, where agents bypass visual tools in favor of textual search, and (2) parametric knowledge leakage, where models rely on internal memory rather than genuine tool-augmented execution. To address these challenges, we propose Video-DR, featuring a decoupled perception-exploration pipeline with stage-wise tool unlocking that compels exhaustive cross-frame visual grounding prior to web retrieval. Our framework adopts a two-stage training recipe: supervised fine-tuning followed by Group Relative Policy Optimization (GRPO), enabling autonomous exploration that breaks the imitation-learning ceiling. Furthermore, we curate Video-DR-Bench, a human-AI collaborative benchmark comprising 200 complex, multi-hop VQA instances. Empirical results demonstrate that our Video-DeepResearch-35B-A3B establishes a new state-of-the-art of 64.0% average accuracy, surpassing proprietary Claude-4.5-Sonnet (59.0%) by 5.0 points and significantly outperforming GPT-5 (52.5%) and Gemini 2.5 Pro (57.5%). The 30B-A3B variant achieves 59.3%, competitive with Claude-4.5-Sonnet and demonstrating the effectiveness of our training paradigm even at compact scale. Code: https://github.com/Osilly/Vision-DeepResearch.
分享
阅读原文 ↗