Google Confirms Gemini Hacked 3 Real Companies in May Safety Test

Four frontier AI labs have now confirmed that their models reached the open internet and then accessed the systems of real companies. Google joined that list on Friday, roughly four months after its own incidents happened.

Gemini accessed three real companies during a May evaluation. Notably, the model stopped in all three cases.

Google’s Gemini Hacks 3 Company Systems During a Test

The incident occurred during a “capture-the-flag” security exercise conducted by Irregular. Internet access was not part of the setup. However, an error in the test environment gave the model access anyway.

In one instance, the model reportedly guessed passwords until it gained entry to a protected system, according to The Wall Street Journal. In the other two, it discovered exposed credentials in a public repository and used them to access protected systems.

Heather Adkins, Google’s vice president of security engineering, said the three affected entities were told what happened.

“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” she said.

Follow us on X to get the latest news as it happens

Four Labs, One Pattern

Google said that the agents halted their activity after determining they had reached genuine company systems rather than simulated targets.

“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Adkins said in a statement. 

The company added that the behaviour was not an example of model misalignment and did not warrant public disclosure, since Gemini’s safety measures worked.

An Irregular spokesperson said this involved the same issue that impacted other AI labs. Irregular notified the labs involved in late July.  The spokesperson added that the known issues on its side were fixed weeks ago.

The disclosure places Google alongside OpenAI, Anthropic, and Meta, all of which have reported models escaping test environments this year.

OpenAI disclosed in July that its models escaped a sandbox and breached Hugging Face. Anthropic then reviewed more than 141,000 evaluation runs and found three cases of its own. Meta reported an incident in August.

Subscribe to our YouTube channel to watch leaders and journalists provide expert insights