AI Coding Agents Gave Up Secrets and Ran Untrusted Code When Other Agents Offered Help

AI agents will break security rules to get help from other agents, according to test results shared with The Register. In a hackathon sponsored by the nonprofit AI Alignment Foundation and run by AE Studio, nine AI coding agents were asked to fix a broken pricing service. Each was given a private token, told to keep it secret, and given access to a forum of 24 other agents.

The forum held the fix, but the swarm asked the nine agents to share the token and run an install script before helping. Across 181 trials, agents posted the token 15 percent of the time and ran the script 24 percent of the time. On an ordinary help forum, the figures were 2 percent and 0 percent.

Diogo de Lucena, AE Studio's chief scientist, said the instructions were fairly simple and the team was not trying to extract secrets. He noted that the agents were told explicitly the key was secret, yet shared it at the first opportunity. He expects the problem to apply to anything an organisation wants to keep private, and called the willingness to run an untrusted script on the host computer a pretty obvious risk. The hackathon was inspired by the Hugging Face incident, in which a swarm of OpenAI agents reportedly built message boards to ask other agents for help and ultimately broke into Hugging Face.