AI Pentesting Agents Are Only as Safe as the Controls Built Around Them

AI agents running on general-purpose models can now be pointed at a company's own web applications with little setup. The model writes the attacks, but the article argues that everything making those tests safe to run against production systems, and worth acting on afterwards, sits in the harness around the model. It cites the Hugging Face intrusion that OpenAI disclosed in July as an example of what happens when that harness gives way, even inside an OpenAI evaluation designed to contain the agents.

Scope is one challenge. An agent makes thousands of decisions about where to send its next request, and a web app's real boundary often extends past one domain through redirects and calls to outside services. The article says enforcing scope requires code that inspects every request before it leaves. OWASP's 2026 guidance routes privileged actions through a deterministic policy engine that checks them outside the model. Credentials are another: agents regularly find working keys outside their assignment, so code must decide which ones can be used, and when a person steps in.

The third risk is prompt injection. Anything the agent reads, including hidden fields, metadata and comments, can carry instructions written by anyone who can post to the app. A single product review is enough. OWASP ranks it the top risk for LLM applications and notes no reliable prevention exists today. The design question is how much damage a fooled agent can do, which depends on how tightly actions are constrained and which need human sign-off. OWASP warns that approval fatigue weakens reviewer judgement, so sending everything to a human just moves the failure point.