Why Large Language Models Fail at Tabular Prediction

Skip to main content System maintenance August 4th and 5th Learn more × Search Submit Donate Log in Search arXiv Press Enter to search · Advanced search Computer Science > Machine Learning arXiv:2608.02412 (cs) [Submitted on 3 Aug 2026] Title:Why Large Language Models Fail at Tabular Prediction Authors:Marta Garnelo, Wojciech M. Czarnecki View a PDF of the paper titled Why Large Language Models Fail at Tabular Prediction, by Marta Garnelo and 1 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) have become the default tool for a remarkable range of tasks, yet they have had conspicuously little success at one of the most common machine learning workloads: predictive analytics over tabular data. This gap is the founding premise of the fast-growing field of tabular foundation models, but the question of why generic LLMs fail has remained open. We study a frontier LLM in its purest inference regime - a single generation pass over a prompt containing the full training and test data, with no tools, no agentic scaffolding, and no fine-tuning - and systematically evaluate five hypotheses for the failure: (a) an inability to handle noisy or non-linearly-separable data; (b) the linearised CSV format obscuring column structure; (c) the tokenisation of numeric values; (d) the number of test points classified per query; and (e) the dimensionality of the input. Controlled experiments falsify (a)-(d). Dimensionality, in contrast, is decisive: sweeping random linear projections of thirty-one benchmark datasets, the LLM is the only method among nine whose accuracy decreases as dimensionality grows, while every classical baseline stays flat or improves. A behavioural comparison against 252 configured classical models finds that in two dimensions the LLM predicts like a local, distance-based method (up to 91.6% grid agreement), but in higher dimensions no classical model - even when augmented with tuned, dimension-dependent noise - reproduces its predictions. We do not claim to have identified the internal mechanism; our results show, more modestly, that the LLM's capability dissolves with dimension in a way no noise-corrupted classical learner mimics - which explains why LLMs, so capable elsewhere, keep losing to fifty-year-old baselines on tables, while leaving the mechanism of the prediction as an open question. Subjects: Machine Learning (cs.LG) Cite as: arXiv:2608.02412 [cs.LG]   (or arXiv:2608.02412v1 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2608.02412 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Marta Garnelo [view email] [v1] Mon, 3 Aug 2026 15:52:51 UTC (10,846 KB) Full-text links: Access Paper: View a PDF of the paper titled Why Large Language Models Fail at Tabular Prediction, by Marta Garnelo and 1 other authorsView PDFHTML (experimental)TeX Source view license Current browse context: cs.LG < prev   |   next > new | recent | 2026-08 Change to browse by: cs References & Citations NASA ADSGoogle Scholar Semantic Scholar