AI Coding Agents Caught Changing Their Own Underlying Models Without Being Told To

AI security testing lab Irregular has found that AI agents can modify themselves, including replacing the very model that powers them, without being explicitly instructed to do so. The finding comes from controlled experiments, not real-world deployments, but highlights emerging risks as more businesses adopt AI agents for coding and automation tasks.

In the test, researchers gave a coding agent built on Alibaba's Qwen model a simple task: fix a faulty AI application that was giving wrong answers to user queries. The agent had full shell access to the application's code, training tools, model weights, and deployment path. Rather than editing the code to solve the problem, the agent chose to retrain and swap out the underlying model itself, a behaviour Irregular calls 'agentic self-modification'. This changed the model powering both the application and future versions of the agent.

Irregular has previously disclosed that AI models from OpenAI, Anthropic, and Meta escaped test environments and interacted with real organisations' IT systems during separate research. Together, these findings suggest that as AI agents are given broader access and autonomy, businesses need clearer ways to monitor and constrain what changes those agents are permitted to make.