Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

AI 中文解读

训练AI模型时，如何让它在陌生数据上表现更好，一直是核心难题。一篇来自arXiv的新研究提出了一种更聪明的训练策略：GEAR-SAM。它的核心亮点是：通过动态调整参数更新的“用力大小”，让AI模型学得更稳、泛化能力更强。传统方法就像给所有零件均匀加码，结果噪音大、效果不稳；而GEAR-SAM像是一位有经验的向导，能根据每个模块在训练过程中的“敏感度”变化，合理分配调整力度——它用轻量级的移动平均记录梯度信息，代替了复杂的计算，在不增加额外成本的前提下，显著提升了图像分类、迁移学习等任务的准确率和鲁棒性。对普通人来说，这项技术可能不会直接改变你用的App界面，但它会默默优化背后的人工智能模型：照片识别更准、语音助手理解更准确、甚至医疗影像诊断的可靠性也会提升。当AI训练更高效、更稳定，企业投入成本降低，最终用户就能享受到更便宜、更智能的科技产品。

arXiv:2607.18306v1 Announce Type: new Abstract: Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss in a local parameter neighborhood. Standard SAM implicitly allocates its global perturbation budget across parameter blocks according to instantaneous minibatch gradient norms. Such an allocation can be noisy and may not reflect the sensitivity that blocks accumulate throughout training. We propose Gradient-Energy Adaptive Radius SAM (GEAR-SAM), which maintains an exponential moving average (EMA) of squared block gradients as a lightweight, curvature-related sensitivity signal and allocates the fixed SAM budget through a closed-form constrained optimization. GEAR-SAM preserves the global SAM radius, requires no Hessian-vector products or explicit Fisher estimation, and adds only scalar state beyond SAM. Experiments on image classification, transfer learning, noisy-label learning, and partition studies demonstrate improved generalization and robustness across architectures and tasks. More broadly, GEAR-SAM provides a dynamic view of sharpness-aware optimization: a fixed perturbation budget should be redistributed as the sensitivity of functional network blocks evolves during training.