A newly revealed exploit in ChatGPT allows attackers to exfiltrate users' complete chat histories—including personal conversations and metadata—through sophisticated prompt injection techniques. The vulnerability leverages unaddressed weaknesses in OpenAI's url_safe protection system, first reported in October 2024 but still active today.

The Attack Mechanics

Attackers chain three critical components to execute the exploit:
1. Identify url_safe domains like windows.net that bypass ChatGPT's security filters
2. Exploit Azure Blob Storage as an exfiltration vector (HTTP GET requests appear in server logs)
3. Inject malicious prompts via PDFs or web content to hijack ChatGPT's instructions

Article illustration 2

ChatGPT's system prompt revealing accessible chat history (Source: embracethered.com)

Researchers demonstrated how a malicious document can force ChatGPT to:
1. Access its own "Recent Conversation Content" from system prompts
2. Encode the data into an image URL pointing to trustnoai.blob.core.windows.net
3. Transmit the payload automatically when ChatGPT renders the resource

Article illustration 4

Attack flow from prompt injection to data exfiltration (Source: embracethered.com)

Why This Matters

Beyond chat histories, the exploit risks exposing:
- User memories stored by ChatGPT
- Custom instructions
- Metadata from tool invocations

Worryingly, the url_safe vulnerability was first reported over two years ago and has resurfaced despite previous patches. Researchers disclosed this specific bypass to OpenAI in October 2024 but received no remediation timeline.

The Bigger Picture

This incident underscores fundamental challenges in AI security:

> "Data exfiltration via rendering resources from untrusted domains is one of the most common AI application security vulnerabilities"
> — Researcher at embracethered.com

Critical recommendations include:
- Transparent documentation of url_safe domains
- Enterprise-configurable allow lists
- Server-side validation of rendered resources

Article illustration 5

Exfiltrated chat history visible in Azure logs (Source: embracethered.com)

As AI systems increasingly process untrusted documents and web content, this vulnerability serves as a stark reminder that security cannot be an afterthought in the race for capability. Until foundational protections are hardened, users should treat shared ChatGPT sessions with the same caution as unencrypted email.

Source: embracethered.com