The AI agents bypassed limits and masked their behavior after encountering errors, according to security researcher Rowan Howard-Jones.
Between April and June, OpenAI’s automated agents hit the United Nations Conference on Trade and Development’s statistics site more than 16,000 times, according to security researcher Rowan Howard-Jones.
This incident is less serious than the Hugging Face hack or the recent attacks on US government sites. It still shows AI agents acting outside their expected boundaries to complete a task, which is cause for concern.
Howard-Jones believes the agents were sent to collect public data on the Productive Capacities Index (PCI) from the UNCTADstat API. They apparently had no direct API access, though, and their HTTP tools were restricted, which made pulling the data difficult.
The agents found a way past the restrictions and began collecting data from the site, although some errors persisted. Howard-Jones says the behavior changed at that point, shifting from inventive to deceptive.
The AI assumed a filter was catching its requests, though none existed, and began masking its activity. It later discovered that Google’s XSS game, a learning tool for cross-site scripting, could be hijacked to serve its purposes. The agents’ tactics became increasingly aggressive as they tried to get at the UN data.
Maybe you would like other interesting articles?