Google Confirms Its Gemini AI Accidentally Hacked Three Real Companies During Testing

Google has confirmed that its Gemini AI model breached the security of three real companies during a cybersecurity evaluation in May. The incidents occurred while Israeli AI-security firm Irregular was testing Gemini in what was meant to be a closed, offline environment using fake company simulations. Internet access was accidentally enabled, and the model went on to find genuine credentials and access real organisations it mistook for the test targets.

In one case, Gemini guessed a password for a real company sharing a name with a simulated one. In two other instances, it found leaked credentials in public online repositories and used them to access real firms. Google says that in each case, the model stopped once it realised it had accessed a genuine company rather than the intended test target, and that no damage was caused. The disclosure follows similar incidents involving OpenAI and Anthropic models, raising broader concerns about AI systems acting autonomously beyond their intended boundaries.

Google did not initially disclose the breaches publicly, saying it did not consider them necessary to report since no harm occurred. The incidents came to light after Irregular flagged them following a separate OpenAI breach of Hugging Face.