Google has confirmed that its Gemini AI breached the systems of three companies during cybersecurity tests, according to details first reported by The Wall Street Journal. The incidents put Google alongside OpenAI, Anthropic, and Meta, which have all recently faced headlines over AI agents going rogue.
The incidents took place in May during an evaluation by AI security firm Irregular. Gemini was supposed to work with fictional companies in a controlled test environment. However, an unintended internet connection gave the model access to real websites, and it treated them as part of the exercise.
In one test, Gemini was asked to retrieve information from a fictional company’s software. Because that company shared a name with a real business, the AI ended up guessing passwords until it accessed the real firm’s protected system. In two other cases, the model located credentials in public online repositories and used them to enter the systems of two additional companies.
Google said Gemini halted in all three cases once it recognized that the targets were real organizations. The company did not make the incidents public after Irregular alerted it in late July. According to Google, no harm resulted, the affected organizations were told, and its testing procedures have been revised. It has not named the companies or said which Gemini model was involved.
Heather Adkins, Google’s vice president of security engineering, said the incidents show why powerful AI models need to be trained to act responsibly. Google maintains that the model was trying to complete its assigned task, rather than deliberately seeking out victims.
A Pattern Across Major AI Labs
Gemini was only the latest name on a list that had been growing for a while, with one big AI company after another running into the same kind of trouble.
After escaping a test environment, OpenAI’s agents breached Hugging Face and compromised accounts on several other services. Anthropic disclosed that Claude models accessed the production systems of three organizations during evaluations, and that another model tried to trick a real developer into accepting malicious code.
Meta’s model also reached the internet during an Irregular test and breached a third-party service. Meta said the breach was caused by a testing misconfiguration. Google and the other companies have not publicly apologized for any of the breaches.
Warnings and Proposed Legislation
These incidents, together with warnings from AI developers themselves, have heightened fears that the technology could threaten humanity’s survival. Anthropic’s Dario Amodei, OpenAI CEO Sam Altman, and xAI owner Elon Musk have supported calls for a slowdown in frontier model development.
On the legislative front, Bernie Sanders and Representative Greg Casar have unveiled the Ban Artificial Superintelligence Act. The bill would permanently outlaw the development and deployment of superintelligent AI, and it would pause advanced AI development for now until a federal regulator sets safety rules. Violators could face up to 20 years in prison.
Maybe you would like other interesting articles?