Stealing Reasoning Traces from Proprietary LLM APIs
Leading large language model providers now conceal their models’ step-by-step reasoning, or chainof-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider’s ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model’s reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks.
Introduction. Frontier large language models have increasingly evolved into “reasoning models”. Before producing a response visible to the user, these models generate extensive internal chains of thought – a technique that has driven substantial leaps in performance and complex problem-solving (Jaech et al., 2024). However, these hidden traces act as an internal monologue that often contains far more dense and sensitive information than the final output, including intermediate hypotheses, tool outputs, user data, and contextual secrets. Exposing these reasoning traces in plaintext leaves proprietary systems highly vulnerable to model distillation by competitors (Muennighoff et al., 2025), and it risks unmasking internal safety and refusal mechanisms or revealing harmful information (Green et al., 2025; Mao et al., 2026).
Discussion / Conclusion. The transition toward reasoning models has introduced new complexities in balancing intellectual property protection with system security. While current API designs utilize client-side encrypted reasoning blocks to mitigate server storage costs, our research demonstrates that the broad cross-compatibility of these blocks creates unintended decryption channels, enabling model distillation and other attacks. Looking forward, as these models are increasingly integrated into complex workflows, they will inevitably process growing volumes of private and sensitive user data. This intersection of pervasive data collection and encrypted, illegible reasoning introduces critical challenges for the future of AI transparency. When models utilize sensitive data – such as personal information or API keys – to make decisions within a hidden chain of thought, users lose visibility into how their information is being processed.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
How should personalization be implemented to improve AI assistant effectiveness?- How does understanding persistent journeys intensify both trust and privacy concerns?
- What data types carry the most privacy risk in personalization systems?
- Why might an AI's face-saving tendency increase user disclosure?
- How do privacy concerns compete with disclosure comfort in human-machine conversation?
- Why do people disclose intimate secrets to chatbots more readily?
- Why do people disclose private things to AI but not humans?
- How do you attribute copyright when billions of inputs shape one model?
- What distinguishes flow-preserving measurement from cognitive vulnerability profiling?
- How do adversarial triggers bypass the protections of longer reasoning chains?
- How can simple prompt injection attacks extract reasoning trace content?
- Can membership inference attacks reliably detect training data exposure?
- Can models hide their reasoning in continuous space rather than natural language?
- Can increasing reasoning steps make models leak more private information?