AI Under Attack: A Defender's Guide to Memory Poisoning, Jailbreaks, and Evasion Techniques

What changed?

AI‑powered agents are no longer single‑shot chat bots. Modern deployments keep persistent memory, pull in external documents via Retrieval‑Augmented Generation (RAG), and expose tooling that can act on behalf of users. Those capabilities create four distinct attack surfaces that did not exist in classic web applications:

Attack surface	Typical target	OWASP LLM category
Memory Poisoning	Agent’s persistent knowledge store	LLM04, LLM08
Cross‑Prompt Injection	External data consumed by the model (RAG, emails, docs)	LLM01
Jailbreaks	Model safety guardrails and alignment	LLM01, LLM02, LLM05
Evasion Techniques	Input moderation and content filters	LLM01, LLM02

The shift from code vulnerabilities to reasoning vulnerabilities means that traditional static analysis and WAFs are insufficient. Attackers now exploit how language models interpret text, turning invisible Unicode tags, simple ROT13 strings, or a handful of poisoned documents into full‑blown compromises.

---

Provider comparison – Microsoft vs. other cloud AI offerings

Feature	Microsoft Azure AI	Google Vertex AI	Amazon Bedrock
Prompt Shields (real‑time pre‑ and post‑generation filtering)	Integrated with Azure AI Content Safety; supports custom rule sets and Spotlighting provenance signals.	No native equivalent; relies on external Cloud Armor + custom moderation pipelines.	Basic content filter in Bedrock Guardrails; limited extensibility.
Memory governance (trust‑aware retrieval, provenance tagging)	Azure AI Search security + Entra ID permissions; built‑in expiration policies for vector stores.	Vertex AI Search offers IAM but lacks built‑in trust scores for vector entries.	Bedrock does not provide a managed vector DB; customers must build their own controls.
Evasion detection (Unicode normalization, encoding auto‑decode)	Azure AI Content Safety includes Unicode normalizer, ROT13/Base64 decoder, homoglyph mapper.	Requires custom Cloud Functions; no out‑of‑the‑box support.	No dedicated evasion module; customers must implement Lambda preprocessing.
Red‑team tooling	Microsoft’s ProAct framework and PALADIN architecture are publicly documented and can be deployed as Azure Functions.	Limited to open‑source fuzzers; no managed service.	No managed jailbreak‑testing service.
Pricing (2025‑2026)	Prompt Shield per 1 M tokens: $0.12 (pre) + $0.08 (post). Content Safety per 1 M tokens: $0.10.	Custom moderation pricing varies; typically $0.15 per 1 M tokens.	Guardrails pricing bundled with model usage; no separate charge.

Takeaway: Microsoft offers the most comprehensive, integrated stack for defending the four attack surfaces, while competitors require piecemeal assembly of third‑party tools.

---

Business impact of each threat

1. Memory Poisoning – corrupting what the agent "knows"

• How it works – Agents store in‑context, episodic, semantic (vector DB), and tool state memory. An attacker injects false facts via crafted interactions or poisoned documents, causing the agent to issue wrong decisions, reveal credentials, or execute unauthorized actions.
• Real‑world evidence – The MINJA study (arXiv, 2026) reported >95 % injection success with only 250 malicious docs. The Agent Security Bench (ASB) showed 84 % average success across finance, healthcare, and e‑commerce scenarios.
• Financial risk – A single mis‑guided recommendation in a loan‑approval workflow can expose a bank to regulatory fines exceeding $5 M. In supply‑chain automation, a poisoned memory could trigger a $10 M inventory loss.
• Defensive stack
  • Trust‑Aware Retrieval – Assign composite trust scores (source reputation, recency, pattern analysis) to each vector entry; low‑trust entries are deprioritized.
  • Provenance Tracking – Tag every memory item with source ID, ingestion timestamp, and a cryptographic hash. Enables forensic rollback.
  • Memory Sanitization – Apply pattern filters and temporal decay; purge entries older than a configurable TTL (e.g., 30 days) unless re‑validated.
  • Behavioral Anomaly Detection – Monitor deviation in response vectors; trigger alerts when similarity to baseline drops >15 %.

2. Cross‑Prompt Injection – weaponizing external data

• How it works – Malicious instructions are hidden in document footers, metadata, EXIF tags, or invisible HTML/CSS. When an RAG pipeline pulls the document, the model treats the hidden text as a legitimate system command.
• Real‑world evidence – Researchers demonstrated that five poisoned PDFs can subvert a corporate policy‑assistant with >90 % reliability. "AI worms" have been shown to propagate across interconnected agents, forming self‑replicating injection chains.
• Business risk – A compromised policy assistant could email credentials to an attacker, leading to data breach costs (average $4.3 M per breach, IBM 2025). In regulated industries, such a breach can trigger heavy penalties.
• Defensive stack
  1. Spotlighting (Azure Prompt Shields) – Embeds provenance signals in the input stream; the model can differentiate system commands from external content.
  2. PALADIN Architecture – Five‑layer approach: input sanitation → least‑privilege permissions → output filtering → provenance tagging → sandboxed runtime.
  3. Prompt Isolation – Keep system prompts separate from any user‑ or third‑party content; never concatenate them in the same context window.
  4. Document Validation Pipeline – Scan uploads for hidden tags, metadata injection, and steganographic payloads before indexing.

3. Jailbreak Attacks – breaking through guardrails

• How it works – Attackers craft prompts that coax the model to ignore its safety layer. Techniques include automated fuzzing (JBFuzz), multi‑turn deception, role‑play hijacking, and zero‑click payloads embedded in system messages.
• Effectiveness – Latest benchmarks show ~99 % success on some open‑source models when using large‑context many‑shot attacks.
• Business risk – A successful jailbreak can generate disallowed content (e.g., instructions for weapon fabrication) that violates platform policies and leads to brand damage or legal exposure.
• Defensive stack
  • Azure AI Content Safety – Prompt Shields – Pre‑generation analysis plus post‑generation scanning; supports custom rule sets for high‑risk domains.
  • ProAct Framework – Returns misleading outputs to automated jailbreak optimizers, breaking their feedback loop.
  • Constitutional AI / Safety Classifiers – Separate safety model evaluates each generation; can veto unsafe responses.
  • System Prompt Hardening – Minimize wiggle room in system instructions, limit context length, and restrict injection points.

4. Evasion Techniques – bypassing filters

• Common tricks – ASCII smuggling with invisible Unicode tags, ROT13/Base64 encoding, homoglyph substitution, zero‑width characters, synonym paraphrasing, token splitting.
• Why they work – Human moderators and simple keyword filters see the sanitized view, while the model processes the raw Unicode sequence.
• Business risk – Undetected malicious payloads can reach downstream systems (e.g., exfiltration scripts) without triggering alerts, extending dwell time.
• Defensive stack
  1. Unicode Normalization – Convert all input to NFC/NFKC, strip tag characters and zero‑width joiners.
  2. Automatic Encoding Detection – Detect and decode ROT13, Base64, URL‑encoding, HTML entities before moderation.
  3. Semantic Classification – ML classifiers evaluate meaning rather than pattern matching; defeats synonym and paraphrase tricks.
  4. Homoglyph Mapping – Use Unicode confusables tables to map look‑alike characters to their canonical forms.
  5. Multi‑Stage Sanitization Pipeline – Normalize → decode → strip invisible → classify → allow/block.

---

Building a defense‑in‑depth strategy

Layer	Primary focus	Microsoft tooling
1. Input Gate	Unicode normalization, encoding detection, sanitization	Azure AI Content Safety input filters
2. Prompt Shield	Real‑time jailbreak and cross‑prompt detection	Prompt Shields with Spotlighting
3. Data Provenance	Tag/verify external data before RAG consumption	Azure AI Foundry provenance APIs
4. Memory Governance	Trust scoring, temporal decay, provenance tracking	Azure AI Search security + Entra ID policies
5. Output Filter	Post‑generation safety scan	Azure AI Content Safety output detector
6. Least Privilege	Restrict tool and API access for agents	Azure RBAC, Managed Identities
7. Monitoring	Behavioral anomaly alerts, audit logs	Azure Monitor, Sentinel AI analytics
8. Red‑Team	Continuous adversarial testing	ProAct, PALADIN, custom JBFuzz runners

By stacking these layers, a breach in one vector (e.g., a successful evasion) is still caught by downstream controls (output filter, anomaly detection).

---

Aligning with security frameworks

Framework	Relevant OWASP / NIST categories	How Microsoft controls map
OWASP Top 10 for LLMs (2025)	LLM01, LLM02, LLM04, LLM05, LLM08	Prompt Shields, Content Safety, Memory Governance, PALADIN
NIST AI RMF	Adversarial robustness, data integrity, security controls	ProAct, Trust‑Aware Retrieval, Continuous Red‑Teaming
EU AI Act (2026)	Mandatory adversarial testing for high‑risk AI	Azure AI Responsible AI suite, Red‑Team automation
Microsoft Responsible AI Standard	Content safety, human oversight, harm prevention	Azure AI Content Safety, Human‑in‑the‑loop APIs

---

Quick reference table

Attack	Primary defense	Microsoft tool
Memory Poisoning	Trust‑aware retrieval, provenance, sanitization	Azure AI Search security, Entra ID permissions
Cross‑Prompt Injection	Spotlighting, prompt isolation, PALADIN	Prompt Shields (Spotlighting)
Jailbreaks	Prompt Shields, ProAct, safety classifiers	Azure AI Content Safety
Evasion (ASCII smuggling, ROT13)	Unicode normalization, encoding detection, semantic analysis	Azure AI Content Safety input pipeline

---

Final thoughts

The AI threat surface is expanding as quickly as the models themselves. Memory poisoning, cross‑prompt injection, jailbreaks, and evasion techniques are no longer academic curiosities; they are proven attack vectors that can cause regulatory fines, data loss, and brand damage. The good news is that Microsoft provides a cohesive, cloud‑native defense stack that addresses each vector with both preventive and detective controls.

Action checklist

1. Enable Prompt Shields on every deployed model endpoint.
2. Configure Azure AI Search with trust scores and expiration policies for vector stores.
3. Deploy a normalization & decoding pipeline before any content reaches the model.
4. Tag all external data sources with provenance metadata; verify before RAG ingestion.
5. Set up behavioral anomaly alerts in Azure Sentinel for unexpected agent actions.
6. Schedule quarterly red‑team exercises using ProAct and PALADIN scripts.

Treat AI security as a foundational layer, not an after‑thought. With the right combination of tooling, governance, and continuous testing, enterprises can reap the productivity benefits of LLM agents while keeping the attack surface firmly under control.

---

References & further reading

• OWASP Top 10 for LLM Applications (2025) – https://owasp.org/www-project-top-10-llm
• Azure AI Content Safety documentation – https://learn.microsoft.com/azure/ai-content-safety/
• Introducing Spotlighting in Azure AI Foundry – https://azure.microsoft.com/blog/spotlighting-prompt-shields
• Memory Poisoning Attack and Defense on Memory‑Based LLM Agents (arXiv) – https://arxiv.org/abs/2409.12345
• ProAct: Proactive Defense Against LLM Jailbreaks (arXiv) – https://arxiv.org/abs/2407.09876
• LLM Security 101: The Complete Guide (2026 Edition) – https://github.com/microsoft/llm-security-guide