“Inner Thoughts” of Leading AI Models Laid Bare After Massive Encryption Flaw
Security analysts have demonstrated a sweeping weakness in how today’s most powerful AI reasoning models protect their hidden “thought processes” – and in the process, they managed to pull real passwords, live API keys, and sensitive personal data out of seemingly harmless logs that developers had uploaded in public.
The research team discovered that every major AI provider they tested encrypts internal reasoning tokens using a single, shared global key. Once they figured out how to use that key, they were able to unlock the models’ concealed reasoning traces at scale.
According to the paper, submitted on August 10 by scientists from MATS Research, the ELLIS Institute Tübingen, the Max Planck Institute for Intelligent Systems, and security company Snyk, the team:
– Decoded 315,320 hidden reasoning blocks scraped from public repositories and logs.
– Recovered 62 active API keys and 33 passwords from those traces.
– Identified 367 pieces of Personally Identifiable Information (PII).
– Found a total of 182 credentials (including the live keys and passwords above).
Most of this data had been unintentionally exposed by developers who thought they were sharing harmless debugging output. In reality, the logs contained encrypted “inner thoughts” of the AI models – and once decrypted, these traces turned out to include raw secrets and private user information.
—
One Global Key to Unlock Them All
The core of the exploit is deceptively simple:
instead of using unique keys per user, per tenant, or per session, major providers were relying on a single global encryption key to encode the models’ reasoning tokens.
Those tokens represent the intermediate “thinking steps” a model takes before generating its final answer. In many systems, they’re deliberately hidden from users – both to keep proprietary reasoning strategies secret and to avoid exposing potentially unsafe intermediary content.
But when one global key is used for every model instance across a provider’s infrastructure, the security model becomes dangerously brittle:
– Compromise the key once, and you can decrypt reasoning traces from any user, any model, any time period – as long as you can get access to the logs.
– If those logs are then shared publicly, anyone who knows (or can derive) the key can reconstruct internal thinking that was never meant to be visible.
The research team didn’t need to break into vendor systems. Instead, they scoured public code hosting and artifact repositories for AI session logs that developers had posted as examples, bug reports, or benchmarks. These logs contained encrypted “reasoning blocks” that, thanks to the global key design, could all be decoded in the same way.
—
What Was Hiding in the Logs
Once decrypted, the reasoning traces revealed far more than academic curiosity about how models reason:
– 62 live API keys: Credentials that were still valid at the time of analysis, allowing potential access to cloud services, databases, or third‑party APIs.
– 33 passwords: Some for application accounts, others for systems that could themselves access more sensitive infrastructure.
– 367 PII items: Names, email addresses, contact details and other personal markers, extracted from the intermediate reasoning rather than from the visible chat responses.
– 182 total credentials: Across different services and contexts, all recoverable purely from the hidden reasoning tokens.
In many cases, the AI systems had been asked to help debug applications, format configuration files, or generate code snippets – tasks where developers sometimes paste real secrets “just to make it work.” Even if providers attempted to redact sensitive data from final outputs, those secrets frequently remained intact inside the private reasoning channels, which then leaked into logs.
—
The Target: Reasoning-Centric AI Models
The paper focuses on a particular class of AI systems: reasoning models that maintain explicit, structured chains of intermediate thoughts.
Unlike simpler chatbots that generate a response token by token, these advanced systems may:
– Plan multi-step solutions internally.
– Keep track of sub-goals and partial results.
– Store hidden “scratchpads” of reasoning before presenting a polished final answer.
To avoid exposing raw, messy, or unsafe thought fragments, vendors typically separate:
1. User-visible output – the final, cleaned-up response.
2. Internal reasoning tokens – private data used to arrive at that answer.
The supposed safety measure is encryption: reasoning tokens are encoded so that logs and telemetry can be stored or analyzed without human operators seeing sensitive intermediates. But by centralizing this encryption under one global key, providers unintentionally turned those hidden channels into a single point of failure.
—
Why Developers Didn’t Realize They Were Leaking Secrets
From a developer’s perspective, the logs they shared looked benign. They often appeared as:
– Encrypted blobs or opaque token sequences.
– Partially redacted traces where visible content seemed harmless.
– Standard debugging dumps captured automatically by SDKs or tooling.
Because reasoning tokens were marketed or documented as “private internal state,” many teams assumed those portions of the log were safe to share. In reality:
– The encryption did not guarantee confidentiality once the global key approach was understood.
– The structure of reasoning tokens preserved highly sensitive raw content, even when the final model answer was scrubbed or paraphrased.
– Logs accumulated over time, so a single repository could hold months of secret-laden reasoning traces.
This combination – assumed safety, opaque encoding, and convenient sharing of logs – created ideal conditions for the large-scale exploit demonstrated by the researchers.
—
Why This Matters Beyond a Single Research Paper
The findings highlight several systemic issues in how the AI industry currently handles security and privacy:
1. Weak encryption architecture
Using a single global key for an entire platform contradicts well-established cryptographic best practices. It amplifies impact: one leak, one insider, or one successful attack can unlock all sessions.
2. Underestimation of hidden data channels
Providers and developers often focus on sanitizing visible outputs while overlooking the fact that internal traces can be even more sensitive. Anything the model “sees” or “thinks” can end up in hidden tokens.
3. Operational convenience over security
Global keys simplify deployment, monitoring, and troubleshooting. But they do so at the cost of tenant isolation and defense in depth, both crucial for systems that routinely process secrets.
4. Regulatory and compliance blind spots
Many organizations treat AI logs as technical telemetry, not as potential containers for user PII or credentials. This research shows that logs may fall squarely under data protection regulations once their true content is considered.
—
Potential Real-World Consequences
If adversaries replicate this technique, several types of abuse become feasible:
– Credential harvesting at scale
Public logs, code samples, and even academic benchmarks can be mined for decrypted reasoning traces, serving as a rich source of API keys, tokens, and passwords.
– Targeted attacks on organizations
A single leaked log from an enterprise AI integration might reveal configuration secrets, internal URLs, or user-specific data that aids spear‑phishing or lateral movement.
– Model behavior analysis and extraction
Hidden reasoning gives insight into how proprietary models structure their internal decision process. Decrypting it en masse could make it easier to clone, jailbreak, or manipulate those systems.
– Privacy breaches for end users
Sensitive queries – medical, financial, legal – may surface in internal reasoning even if the final answer is anonymized. That puts individuals at risk when logs are mishandled.
—
What AI Providers Need to Change
The research effectively sketches a to‑do list for model vendors and platform operators:
1. Retire global encryption keys
Replace single, shared keys with per-tenant, per-application, or per-session keys, making it impossible to decrypt all traces with one secret.
2. Minimize reasoning retention
Store internal reasoning only when strictly necessary for debugging or research, and even then, limit duration and access.
3. Sanitize before logging, not after
Redaction and filtering should apply to the internal state itself, not just to user-visible output. Sensitive tokens must be removed or masked prior to being written anywhere.
4. Treat logs as high-risk data
Access controls, encryption at rest, key management, and retention policies for AI logs should be comparable to those for customer databases, not casual telemetry.
5. Offer explicit configuration for enterprises
Organizations integrating AI should be able to disable reasoning capture, control where logs are stored, and enforce compliance with their governance rules.
—
How Developers Can Protect Themselves Today
While providers work to fix systemic vulnerabilities, developers using AI models can reduce their own exposure by changing how they build and debug applications:
– Never paste live credentials into prompts
Use placeholder values in examples. If the model needs to interact with a real system, handle auth in your own code, not through natural‑language descriptions.
– Turn off verbose logging where possible
Many SDKs and frameworks have debug modes that capture rich traces, including internal IDs, prompts, or even reasoning snippets. Use them only in tightly controlled environments.
– Audit and scrub existing repositories
Search past commits, logs, and artifacts for AI session dumps or encrypted blobs that might actually contain sensitive reasoning tokens. Remove or sanitize them.
– Assume that anything the model “sees” can leak
Whether in visible output, training data, evaluation sets, or logs, treat all model-exposed content as potentially recoverable.
—
A Wake-Up Call for AI Security and Governance
The exposure of hundreds of thousands of decrypted reasoning blocks – along with real passwords, active API keys, and PII – underscores that AI safety is not only about prompt injection, model bias, or hallucinations. It is also deeply about traditional security engineering: key management, isolation, logging practices, and data minimization.
As AI models become more capable and more deeply embedded in critical workflows, the cost of design shortcuts grows. A global key might have seemed like an internal implementation detail; this research shows it can instead become a universal skeleton key to the “inner thoughts” of the world’s most widely used AI systems.
The lesson is blunt: if an AI platform handles secrets, personal information, or sensitive reasoning – and most do – it must be engineered with the same rigor as any other system that processes high-value data. Hidden thoughts are not truly private if they can be decrypted by design.