Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods

AI 中文解读

在机器学习领域，模型评估的准确性很大程度上取决于如何将数据分成训练集和测试集。传统随机拆分默认两者分布一致，但面对数据不平衡、自然聚类等问题时往往失效。一篇最新研究提出了一种名为“优化分布法”的新策略，直接以训练集与测试集的统计相似性为优化目标，将平均相似度提升至89%，远超其他方法。简单来说，就像给学生出考题时，要确保平时练习和考试题目难度、范围完全一致，否则学生的真实水平会被高估或低估。以前的随机切分就像随意抽题，而新方法则像精心挑选题目，让测试集与训练集在数据分布上高度匹配，从而让AI模型的表现评估更客观可靠。这项技术的影响很直接：未来AI应用的稳定性将显著提升。例如，在医疗影像诊断中，模型对不同人群的识别结果会更一致；在金融风控中，模型对新客户的判断也会更准确。简单说，以后你用的AI产品——无论是智能客服还是推荐算法——都会因为训练数据切得更合理而变得“更懂你”，错误率更低。

arXiv:2607.26625v1 Announce Type: new Abstract: Accurate model evaluation in machine learning depends critically on how datasets are split into training and testing subsets. Standard random splitting assumes that both partitions share the same underlying distribution, an assumption often violated in datasets with class imbalance, natural clustering, or spatial autocorrelation. This paper investigates the role of statistical similarity in train-test splitting and its consequences for AutoML model evaluation. Five established strategies are compared across fifteen UCI benchmark datasets: random splitting, stratified sampling, Kennard-Stone, Duplex, and SPXY. Similarity is assessed using chi-square, Kolmogorov-Smirnov, and Maximum Mean Discrepancy (MMD) tests. Geometry-based methods consistently produce near-zero MMD scores, introducing instability into downstream performance estimates. The proposed Optimised-Distribution method treats similarity as an explicit optimisation objective and achieves the highest mean MMD similarity, 89.0%, across all strategies evaluated.