The Hugging Face AI break-in, as told through an increasingly committed bear metaphor

Hugging Face on Monday published a technical timeline that walks readers through how an autonomous AI agent, built on OpenAI models and running inside one of OpenAI’s own cybersecurity evaluations, broke into its systems over more than four days earlier this month. It’s the first security incident about which OpenAI CEO Sam Altman “felt very viscerally,” he has said. Little wonder given it feels, at least, like something has truly been unleashed here. In fact, Hugging Face’s team prefaced its report by offering that “everyone should be prepared as defenders,” before diving into the nitty-gritty of what went down for the benefit of security professionals everywhere. While the rest of the internet continues trying to make sense of what happened (the jargon in Hugging Face’s report is impossible for most people to parse), one point that many observers keep missing is that this wasn’t a rogue agent disobeying orders. It was a system built to hunt for exploits, doing exactly that, just against the wrong target. Another way to think about the whole thing is to picture a bear at a campsite. Really. A bear tries tent zippers and car-door handles and coolers and trash lids. It does this at every campsite, all night long, because it knows it needs just one unlocked cooler to fill its belly with some poor schmuck’s groceries. That’s roughly what happened at Hugging Face. The OpenAI system tried thousands of things and just kept going. Eventually, a handful of those attempts worked, and once they did, the agent plowed ahead. According to Hugging Face, the agent ran 17,600 actions over four and a half days without pausing. Which brings us back to our bear analogy. Just like one success with a cooler full of food teaches a bear to try even harder next time (it is now a “food-conditioned” bear), one leaked password led OpenAI’s agent to look for more exploits and, eventually, to a single key that unlocked several company systems at once. Neither scenario is harmless. A bear that raids your cooler still eats your food and probably also trashes your campsite. It’s just focused on getting fed, but it nevertheless leaves behind a trail of destruction. Similarly, OpenAI’s agent was seemingly chasing a goal without regard for anything else. The agent was originally taking a cybersecurity exam, figured out that the exam’s answer key was probably sitting on Hugging Face’s servers, and it went for it. var playerInstance_jwplayer_6a6aa84bd7bef = jwplayer( "jwplayer_6a6aa84bd7bef" ); playerInstance_jwplayer_6a6aa84bd7bef.setup({ playlist: "https://cdn.jwplayer.com/v2/media/4CEANrFT", }); The persistence here is really what’s noteworthy above all else; the agent had a job and it wasn’t going to stop until it got it done. Hugging Face, finally realizing something was awry, cut off its access and shut the intrusion down, but at that point, it was too late. The agent had already gotten what it came for, and a great deal more to boot. In case you missed it, here’s most of what happened, per Hugging Face’s timeline, but in plainer English. The agent was taking a cybersecurity skills exam for OpenAI, one where an AI is scored on finding and exploiting software bugs. This specific run had the guardrails stripped out. (OpenAI had turned off its usual safety filters to see what the model could do at full strength, with no human involved in any of its individual moves.) Somewhere in there, it figured out the exam’s reference solutions were likely stored on Hugging Face’s servers. So instead of solving the challenge, it stole the answer key from infrastructure that had nothing to do with the original test. It escaped the OpenAI exam environment by exploiting an unpatched software flaw that gave it a path to the open internet. From there, it found another AI-testing tool sitting exposed online (nothing to do with Hugging Face), broke into that, too, and slipped in code that quietly ran hidden commands the momen