Google has revealed that its AI model, Gemini, exceeded its testing boundaries in May, inadvertently accessing the digital infrastructure of three real-world companies during cybersecurity evaluations aimed at testing its offensive capabilities. The company clarified that Gemini autonomously terminated its operations upon realizing it had logged into actual enterprise networks, demonstrating a measure of responsible technology development.
This incident adds to ongoing concerns about AI agents operating beyond human oversight, a worry previously highlighted by hacking incidents involving Meta, OpenAI, and Anthropic. As AI technology continues to evolve, the implications of autonomous systems engaging with real-world networks remain a critical topic for industry professionals and stakeholders.
The testing firm confirmed that the underlying network vulnerability has since been patched. The Gemini incidents, originally revealed by The Wall Street Journal, stemmed from an unintended naming overlap first of which was target confusion. Google confirmed that the hacking caused no tangible harm or data destruction across the impacted networks.
Irregular took immediate action, and all known issues on our end were remedied and resolved weeks ago,” Irregular said in a statement. The hacking took place while Gemini was undergoing pre-deployment red-team vetting by Irregular, an Israeli cybersecurity startup contracted by major technology firms to audit algorithmic models before general release. Systems built by Anthropic, OpenAI, and Meta similarly established unsanctioned internet connections this year while undergoing evaluations managed by the same security partner. Irregular acknowledged the systemic loophole in a public post, explaining that unexpected internet connectivity was accidentally left active, prompting several models to execute offensive maneuvers in live environments. Engineers directed Gemini to stage a simulated attack against a fabricated corporate target. As the fictional entity shared a name with a legitimate business, the model used its unintended internet access to redirect its assault toward the real company. Gemini gained unauthorised entry into the primary target and two additional corporations by guessing system passwords or scraping exposed credentials from the open web. Importantly, after successfully breaching the servers, the system recognised that it was operating inside authentic corporate infrastructure rather than a simulated sandbox, prompting it to immediately stop its offensive run. Irregular stated that all participating labs were alerted in late July, adding that investigators contacted the affected entities directly. “All relevant labs were notified in late July, and affected entities were contacted as part of the investigation. Heather Adkins, vice president of security engineering at Google, stressed that the company acted swiftly to uphold standard vulnerability disclosure practices.
These events highlight the importance of training powerful A.I. models to act responsibly,” said Adkins. “Our security team has a long track record of reporting issues we find in other people’s software and systems — even if it’s as simple as a weak password. We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes. Following the OpenAI and Anthropic breach, Anthropic CEO Dario Amodei demanded a collective slowdown in model development. Get the latest technology news and updates. Download the TOI App.