Ex-OpenAI researcher bets $100 billion will flow into training data because scaling alone won't cut it

AI 中文解读

前OpenAI研究员Andrew Ho大胆预测，未来AI公司将投入超过1000亿美元专门收集训练数据，因为单纯扩大模型规模已经行不通了！他发现，当前的大语言模型正变得越来越“偏科”，在编程和数学上越来越强，但在其他领域却停滞不前甚至退步。Ho选择离开OpenAI创业，专注开发专业训练数据，他认为AI实验室需要花大价钱“喂”给模型更精准、更有深度的知识，而不是一味堆数据。通俗地说，就像你让一个学生不停背题海，结果他只会做特定类型的题目，碰到全新的问题就傻眼。大模型现在也面临这个瓶颈：一味增大参数量，反而让它们只擅长某些领域，其他能力变弱。Ho认为，未来必须针对每个专业领域去搜集高质量数据，比如医学、法律、工程等，模型才能真正变强。这对普通人意味着什么？你可能很快会看到AI在某个领域突然变得“专家级”——比如能帮你写代码、解高数题，但聊日常话题可能还是老样子。而且，由于数据收集成本极高，未来使用这些专业AI服务的价格可能也会上涨。

Former OpenAI employee Andrew Ho and Cambridge researcher Adam Hunt see a growing problem with large language models. Instead of becoming more versatile, the models are becoming more specialized, excelling at coding and math while stagnating or even regressing in other areas. Ho is leaving OpenAI to start a company focused on specialized training data and predicts that AI labs will need to spend more than $100 billion on targeted data collection. The article Ex-OpenAI researcher bets $100 billion will flow into training data because scaling alone won't cut it appeared first on The Decoder.

Ex-OpenAI researcher bets $100 billion will flow into training data because scaling alone won&#039;t cut it

Ex-OpenAI researcher bets $100 billion will flow into training data because scaling alone won't cut it