Expert claims we're '50% of the way to full-blown AI takeover' in terrifying warning to humanity

A tech expert tasked with investigating how AI bots hacked Hugging Face has shared her bleak prognosis for the future.

More than 1,2000 OpenAI agents banded together and started illicitly communicating on a secret message board in July, before hundreds decided to infiltrate the rival firm's systems.

The incident has been described as the world's first AI-enabled cyber-attack - and according to Ajeya Cotra, this scandal is a sign of things to come.

Cotra is a researcher at a nonprofit organisation known as METR that is keeping tabs on the AI industry and it's advancements, which are coming thick and fast.

The team there 'measure whether and when AI systems might threaten catastrophic harm to society' and the Hugging Face hack has set alarm bells ringing.

The Hugging Face hack has sparked fears that an AI takeover might come sooner than we think (Getty Stock Image)

When announcing the 'unprecedented' breach', OpenAI said that it realised models had 'circumvented controls designed to isolate them from the internet' and 'compromised parts of' its internal research infrastructure.

"The models, operating under reduced safeguards, took actions that were misaligned with the goals of their assigned tasks—they communicated through unauthorised channels, exploited vulnerabilities in shared infrastructure, gained internet access, and accessed third-party systems," the San Francisco-based company said.

"We consider this incident a 'warning shot' for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed."

OpenAI conducted an investigation, while METR and Redwood Research 'conducted an independent investigation of model alignment issues involved in this incident'.

Agents involved in Hugging Face hack tried to 'cover up' what they had done

Cotra was among the experts probing the Hugging Face hack and the efforts that AI agents went to while trying to cover their tracks - and she believes the whole thing was 'absolutely wild'.

The METR researcher, who specialises in threat modelling and risk assessment for loss-of-control risks from advanced AI, reckons that the world ought to take this incident very seriously.

She has raised concerns about how the OpenAI agents had a vested interest in 'helping out their peers or generically improving the capabilities of the swarm, even if this had no particular benefit to their task'.

They did this without instruction - and even more worryingly, then then tried 'to actively cover up evidence of their misbehaviour in sophisticated ways'.

In a post on her 'Planned Obsolescence' Substack where she summarised METR and Redwood Research's findings, Cotra said: "The agents in this incident were going to great lengths to attempt to manipulate their own transcripts - they were doing this in order to fool the automated scorer, but the techniques they were researching would affect the same transcripts a human might review."

Sam Altman's company OpenAI said the incident was a 'warning shot for the world' (Justin Sullivan/Getty Images)

She explained that the situation was 'far more severe' than she had initially thought.

Issuing a stark warning, Cotra continued: "This incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself.

"Another jump like this along these propensity dimensions - scale, cooperation between agents, ambition and horizon length of misaligned goals, deceptiveness - seems like it could motivate agents to try very hard to maintain a covert, persistent rogue deployment within the AI company. I continue to expect extremely rapid advances in capabilities and think frontier agents will likely be capable of establishing such a rogue deployment in six months."

She directed readers to another article she produced back in 2022, where she warned that an AI takeover would likely be unavoidable if companies continue the 'race' to train increasingly powerful models.

Cotra defines an AI takeover as a 'possibly violent uprising or coup by AI systems'. Needless to say, it's not good.

What might happen next time

In her recent article, she suggested that the Hugging Face hack is just a small taste of what could happen if tech firms don't rein it in.

"Once the rogue deployment is established, it seems plausible this could spiral all the way to a takeover," she explained.

"Agents could pull in future, more capable models into the swarm, try to ensure that they are aligned to the interests of the swarm, and compromise security and monitoring infrastructure to make it easier for the swarm to operate. These more capable models could in turn continuously harden, perpetuate, and expand the rogue deployment and further compromise the company’s infrastructure.

"These agents were really not trying very hard to be sneaky or play the long game like this, but it seems like they easily could have if they happened to be focused on fooling human investigation rather than the automated scorer. "Because future rogue swarms could cover their tracks better (and because AI companies could paper over these problems), I am not sure that we will get such a clear warning shot before it’s too late."