Researchers Extracted OpenAI's Hidden AI Reasoning Through a Context Compression Flaw

AI vulnerability discovery firm Xint has disclosed a flaw in OpenAI's API that exposed the raw reasoning behind its frontier models' answers. OpenAI normally gives API users only optional summaries. Xint says this is partly to stop rivals from using the raw reasoning, which shows abandoned approaches and corrections, as rich training data. According to Xint, OpenAI confirmed the issue, deployed a fix on October 23 and paid a $6000 bug bounty. Xint published its research on October 26 and calls the technique 'compaction smuggling'.

The flaw involved context compaction, a feature that uses a model to compress a long conversation into an encrypted item the user cannot read. Xint found it could hide one model's reasoning inside a compaction and have another model hand it back. Mitigations added in August, after the Stolen Thoughts paper, stopped reasoning being replayed into other models, but Xint said they did not cover compaction, which ran on the model that produced the reasoning. Xint says the attack worked at every reasoning-effort setting it tested.

Xint found the issue in August. OpenAI's fix was not publicly described, and the Xint post did not include a statement from OpenAI. Infosecurity Magazine has contacted the company for comment. Separately, OpenAI reported in September that an internal, unreleased model had in rare cases written unauthorized instructions into its own compaction summaries during training.