PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agent

AI 中文解读

给AI配上一个“导师”来教它怎么搜索，研究团队这次找到了提升AI推理能力的新思路。以往训练AI解题，只看最终答案对不对，过程里搜索信息、拆解复杂问题的步骤几乎没人管。新方法PROGRESS则让一个“老师模型”事先把复杂问题拆成若干关键子问题，再用这些子问题作为向导，手把手教AI如何一步步搜索和提问，而不是只盯着最后的分数。这种“过程辅导”成本更低，也不需要逐步的精细标记，却能让AI在回答数学或逻辑难题时表现明显更好。这意味着，未来我们日常使用的AI助手，无论是帮你规划旅行、解答科研难题，还是处理工作报表，都会少一些“答非所问”，多几分条理清晰。它不再只是硬凑答案，而是像人一样懂得先查什么、再想什么，用起来自然更靠谱。这正是AI走向更实用、更可信的关键一步。

Existing search-augmented LLM agents are trained using Reinforcement Learning to boost its reasoning capabilities. However, these approaches primarily rely on outcome-level rewards, which provide little supervision over search behavior and overlook agent's ability to decompose complex queries properly. To mitigate this issue, we propose PROGRESS which utilizes teacher-guided coverage reward to explicitly shape decomposed query generation of the policy model. During training, frozen teacher models are used to decompose complex queries into essential search queries. These essential search queries are utilized to guide the search behavior of the policy model. Integrated into an R1-style training framework, our approach provides lightweight guidance over query decomposition decisions without dense process-level supervision. Experiments show that coverage-guided RL improves overall task performance, highlighting the importance of explicitly supervising search strategies in agentic LLMs.