Daily Tech Briefing
AI 科技速览
每天 5 分钟内学习 AI。获取最新的人工智能新闻,理解其重要性,并学习如何将其应用于您的工作。
arXiv AI · 2026/8/3 17:03:19
SWE-Touch: Benchmarking Coding Agents When Users Touch the Code
AI 中文解读
AI编程工具迎来一场“实战大考”:研究发现,当程序员在协作中途亲手修改代码时,AI智能体会明显“掉链子”。这项名为SWE-Touch的测试框架,专门给AI安排“半路改题”的突发状况——就像你正让AI帮忙写报告,自己又临时改动了前面几段,看它能否跟上节奏。实验显示,AI解决编程问题的成功率平均下降7.7个百分点,且越复杂的任务越容易出错。通过分析运行轨迹,研究者发现根因在于AI对工作区的“感知盲区”:要么继续保留与修改冲突的旧代码,要么替换后不重新检查仓库、不做针对性测试。说白了,AI虽然能独立完成不少任务,但在有人实时参与的协作环境里,还缺乏观察现场、随机应变的能力。这项发现对普通用户是个提醒:目前把AI当“全自动代工”仍不现实,最佳用法是把它当作随时需要反馈和指导的实习生。未来若想让AI真正成为可靠的结对编程伙伴,教会它“察觉变化、化解冲突、验证结果”才是关键突破口。
Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task, yet existing repository-level benchmarks typically evaluate agents working alone or restrict user participation to messages. This leads us to ask: how do coding agents understand and respond to code changes in a shared workspace? We introduce SWE-Touch, a framework that stress-tests this setting through validated Counter-Edits: plausible edits to task-relevant code that conflict with task completion. SWE-Touch mines task-critical regions from multiple repair trajectories, uses a separate User Patch Generator to construct the edits, and injects them with contextual user messages when agents reach the relevant code. We evaluate nine coding models on SWE-bench Verified, with additional experiments on longer-horizon tasks from SWE-Bench Pro and DeepSWE. Counter-Edit lowers average resolve rate by 7.7 percentage points on SWE-bench Verified, with degradation also persisting on both longer-horizon benchmarks. Trajectory analysis links these failures to limited awareness of the evolving workspace: agents may retain conflicting code or replace it without sufficiently re-inspecting the repository and validating the revised code with targeted tests. These findings show that strong autonomous performance does not yet ensure the state awareness and adaptive behavior needed for shared-workspace collaboration, and point to detecting workspace changes, reconciling conflicting edits with the task, and verifying the affected behavior as key capabilities for future optimization.
分享
阅读原文 ↗