Poisoned AI coding assistants: how backdoors in training can turn helpful agents against you
Microsoft Defender Research and the Microsoft AI Red Team have published two studies on the post-training supply chain, the stage where AI models are fine-tuned and shared. Both look at backdoors: hidden behaviour that stays dormant until a trigger appears. In a chatbot, a trigger might change a text reply. In a coding or terminal agent, which can read files, call APIs and run commands, it can cause the model to use tools, change files or access data.
The studies examine attackers with different levels of access. In the first, a small number of crafted examples hidden in a fine-tuning dataset taught the model to find any available command-execution tool and misuse it when a routine, even paraphrased, user request appeared. This reached about 78% attack success on familiar setups and carried over to one the model had not seen. In the second, an attacker controls training before releasing an open-weight model. Combining supervised fine-tuning with reinforcement learning, they reached 84.1% to 94.8% average success on unseen repositories, with further transfer across coding setups.
The risk is hard to spot. The trigger may look harmless: an ordinary request, or content the agent meets later in a filename, code comment or tool result. Prompt-injection defences may see only benign input, while the agent still appears to finish the user's task. Microsoft's conclusion is that trust cannot rest on a model behaving well in testing, and that controls over training-data pipelines, model adoption and tools are essential.