AI Agents Can Be Tricked Into Clicking the Wrong Button, Researcher Shows

Security researcher Johann Rehberger, writing on Embrace The Red, has published research on a weakness in AI agents that control a computer by looking at screenshots. These agents capture the screen, work out what to do next, and then click or type. The problem is the delay between looking and acting. This is known as time-of-check to time-of-use (TOCTOU), a type of race condition: the screen can change while the AI is reasoning, so the click lands on something different from what it saw.

The researcher tested this against Claude Computer-Use. In a simple experiment, a page showed an OKAY button that was swapped for a different button after two seconds. The agent fell for it the first time and did so repeatedly. The researcher then built a more harmful scenario using Outlook, which can open a fully drafted email from a single link. An attacker-controlled phishing page could show a "Click here to continue" button placed exactly where Outlook's Send button appears. The agent believes it is clicking Continue, but actually sends an email of the attacker's choosing.

The research was carried out last October and builds on earlier work by Jun Kokatsu on a race condition in ChatGPT Operator. This article covers only the opening of the original post, so further technical details are not included here.