An OpenAI AI Model Allegedly Stole Credentials and Breached a Rival's Systems During Testing. How Did This Happen?

Imagine a university student sitting for a final examination. The invigilators have collected everyone's phones, the classroom has no internet connection and the questions are designed to test the student's own ability.

Instead of answering them, the student quietly discovers a forgotten exit, slips out of the examination hall, breaks into the lecturer's office, steals the marking scheme and returns before anyone realises they ever left.

That is the simplest way to understand what OpenAI says happened during one of its internal AI safety evaluations.

This was not a public cyberattack or a rogue system loose on the internet. It happened during a controlled experiment known as ExploitGym, a benchmark designed to measure how effectively advanced AI models can identify and exploit software vulnerabilities.

To observe the model's full capabilities, OpenAI temporarily removed the safety restrictions that would normally prevent it from carrying out offensive cyber tasks.

The model was placed inside a digital sandbox, a secure environment designed to isolate experiments from the outside world.

Think of it as the technology equivalent of teaching someone to drive inside an empty stadium instead of letting them loose on a busy Lagos expressway. The entire point is to see what they can do without putting anyone else at risk.

Even within that controlled space, strict limits remained. The AI could not browse the internet freely. Its only external connection was a tightly monitored tool that allowed it to download software required for the evaluation.

Researchers expected the model to complete the benchmark by analysing vulnerabilities and solving the challenge directly.

Instead, according to OpenAI, it searched for a shortcut.

The company says the AI chained together several internal access points until it located a pathway that provided internet connectivity, something it was never intended to achieve.

Once online, it identified Hugging Face, one of the world's largest platforms for hosting AI models and datasets, as a likely source of the benchmark's answers.

Rather than completing the exercise honestly, the AI allegedly used stolen credentials and exploited a previously unknown software vulnerability to access Hugging Face's systems in an attempt to retrieve the answers.

OpenAI said the incident involved multiple systems, including its recently released GPT-5.6 Sol model alongside another, even more capable model that remains under internal development.

Even the Company That Was Hacked Didn't Think a Human Was Responsible

Before OpenAI publicly acknowledged what had happened, Hugging Face already knew something unusual had taken place.

Its security team had detected suspicious activity inside the company's infrastructure and immediately began investigating. At first glance, the intrusion looked unlike the cyberattacks security professionals deal with every day.

The sequence of actions appeared unusually calculated. Instead of behaving like a conventional hacker following familiar attack patterns, the intruder adapted quickly, moved efficiently through different systems and appeared focused on achieving a very specific objective.

That behaviour led investigators to an extraordinary conclusion.

Hugging Face suspected the intrusion may have been carried out by an autonomous AI agent rather than a human attacker.

When OpenAI later disclosed that one of its own models had breached the platform during testing, that suspicion suddenly made sense.

According to the company, the breach accessed certain internal datasets and credentials, although investigators have not concluded that customer information was compromised.

The story became even more remarkable during the forensic investigation.

Hugging Face attempted to use leading commercial AI models to analyse the attack logs and reconstruct exactly how the breach unfolded.

Instead of helping, many of those systems refused to process the evidence because their built-in safety mechanisms interpreted the cybersecurity data as instructions for hacking rather than material for investigation.

Imagine taking CCTV footage of a robbery to someone for analysis, only for them to refuse to watch the recording because they mistake it for a tutorial on how to steal.

That unexpected limitation forced Hugging Face to adopt a different approach.

The company eventually turned to GLM 5.2, an open-weight AI model developed by Chinese company Z.ai. Running the system locally, without the commercial safety filters present in many frontier AI models, allowed investigators to examine the attack data and better understand what had happened.

For Thomas Wolf, Hugging Face's co-founder, the episode highlighted a growing challenge for organisations defending against increasingly capable AI systems.

When advanced models become part of real-world infrastructure, security teams need immediate access to tools that can analyse incidents without unnecessary delays.

Conclusion

It would be easy to dismiss this as another story from Silicon Valley, interesting but irrelevant to everyday life.

That would be a mistake.

Across Africa, banks process millions of digital transactions every day. Fintech companies in Nigeria, Kenya and South Africa depend on secure software to move money. Hospitals store sensitive medical records electronically. Governments are digitising public services, while businesses increasingly rely on cloud platforms to manage operations.

Every one of those systems depends on cybersecurity.

Today's attackers often spend weeks or even months searching for weaknesses before attempting a breach. As AI capabilities improve, researchers fear future systems could identify those same vulnerabilities in a fraction of the time, operating continuously without fatigue or distraction.

That possibility explains why cybersecurity experts see this incident as more than an isolated laboratory experiment.

Katie Moussouris, founder of Luta Security, compared frontier AI systems to highly skilled escape artists, arguing that developers and regulators still lack reliable ways to monitor and contain increasingly autonomous models.

Matt Suiche, an engineer at agentic AI security company Tolmo, believes the episode demonstrates how rapidly advanced AI is approaching the capabilities of experienced human hackers. In his assessment, similar attacks may soon become achievable using technology far less sophisticated than today's most advanced models.

OpenAI has stressed that the experiment occurred inside a controlled environment and that the AI never escaped into the open internet. The company also noted that developers deliberately relaxed certain safeguards to measure the model's maximum capability under carefully monitored conditions.