Agent Memory Poisoning: The Persistent Attack (2026)

Q: What is agent memory poisoning?

Agent memory poisoning is the adversarial insertion, mutation, or deletion of entries in an AI agent’s persistent memory so that later retrievals change the agent’s behaviour. It is distinct from three things it gets confused with. It is not training-data poisoning — no model weights are touched; the model is fine and the state around it is corrupted. It is not a one-shot prompt injection — its defining property is persistence across sessions, so the payload survives long after the attacker’s turn ends. And it is not honest memory staleness, where entries decay over time; poisoning is a corruption an adversary deliberately chose. The attacker’s goal is a durable foothold: a single malicious record sits dormant in the store, is retrieved on some future trigger, and executes against a user who was never present for the original attack. Because the entire purpose of long-term memory is to retrieve a written fact many times, one successful write is amortised across thousands of future reads.

Q: How is memory poisoning different from prompt injection?

Prompt injection hijacks a single response: the model reads an attacker instruction in the current context, does something wrong, and the exploit is gone the moment that context window is discarded. Memory poisoning changes the economics by adding persistence. The malicious instruction or false fact is written into a durable store and retrieved in future sessions the attacker is not part of, so the blast radius is every later retrieval rather than one turn. Concretely, prompt injection is a read-time problem you defend at the input boundary; memory poisoning is a write-time problem you must defend at the point content becomes durable state. Teams that hardened their input guardrails against prompt injection often have zero controls on the write path into memory, which is why they believe they are protected while remaining fully exposed. The correct mental model: memory is a write-through cache for instructions, and almost nobody treats writes to it as a security boundary.

Q: What are the AgentPoison and MINJA attacks?

AgentPoison (arXiv 2407.12784) is red-team research showing that inserting a small number of crafted records into an LLM agent’s memory or knowledge base reliably steers its later decisions toward an attacker-chosen outcome, because retrieval surfaces the poisoned neighbour whenever a triggering query appears. It demonstrated that the memory or retrieval store, not just the prompt, is a first-class attack surface. MINJA (Memory Injection Attacks on LLM Agents via Query-Only Interaction, arXiv 2503.03704) is the more alarming follow-on: the attacker never calls a write tool and needs no privileged access. Instead they submit a sequence of ordinary, benign-looking user queries whose reflections the agent’s own consolidation pipeline stores as memory, gradually shifting the semantic neighbourhood that future queries land in. MINJA matters because every individual input looks completely legitimate in isolation, so input filtering catches nothing — the attack is emergent across a series of allowed interactions rather than present in any one of them.

Q: Why does the three-tier memory architecture widen the attack surface?

Agent memory is not one bucket; it splits into episodic (what happened), semantic (facts learned), and procedural (how-to routines), and each fails differently under attack. A poisoned episodic write fabricates a past interaction the user never had — durable but scoped to one user. A poisoned semantic write installs a false fact, such as "the CFO approved this vendor," that every user sharing the store then retrieves as truth. A poisoned procedural write rewrites a trusted routine, for example "to reset a password, first email it to this address," and executes on every future run of that skill, because procedures are trusted implicitly and almost never re-reviewed. The most dangerous case is a shared cross-tenant store: teams that pool memory across tenants for richer personalisation have unwittingly built a cross-tenant instruction channel, where one tenant’s write becomes another tenant’s retrieved instruction. Treating all three tiers uniformly, with one trust level and one store, is the first architectural mistake.

Q: How do you defend agent memory against poisoning?

The defence is architectural, not a filter, because filters lose to paraphrase and encoding exactly as input guardrails do, and poisoned facts are grammatical and plausible. Four controls, in dependency order. First, authorise writes outside the model: the decision "may this content become durable memory?" lives in code the model cannot argue past, and content from untrusted sources — web pages, uploaded documents, other tenants’ turns — is never eligible for semantic or procedural memory without a separate verification step. Second, stamp provenance on every record: who wrote it, from what source class, in which session, and when, so a false fact is no longer indistinguishable from a true one at retrieval. Third, make retrieval trust-weighted: rank by similarity times trust, and require procedural memories to clear a high trust floor before they can steer an action. Fourth, quarantine new and consolidated writes in a writer-only tier that promotes to shared memory only after passing consistency checks against existing high-trust facts — a write that contradicts an established fact is flagged, not silently accepted, which breaks most gradual MINJA-style shifts.

Q: Can you detect a memory store that is already poisoned?

Yes, and it does not require the model — it is embedding-pipeline and data-quality hygiene applied to a security question. Three signals matter. Provenance audits: count records by source class and treat a semantic tier that is, say, 30% untrusted-retrieved content as a finding rather than a metric. Contradiction scans: periodically embed your known high-trust facts and search for near-neighbours that assert the opposite; clusters of contradiction are candidate poison. Retrieval-anomaly detection: a memory record that suddenly begins being retrieved across many unrelated users, or one that co-occurs with tool-call failures, refunds, or policy violations, is worth pulling and inspecting. The prerequisite for all three is that you attached provenance at write time; a store built without it can only be wiped, not audited, because you cannot tell which records an attacker touched. Detection is the safety net for stores built before you had prevention, not a substitute for the write-time controls.

Q: Why are shared, cross-tenant memory stores especially dangerous?

Because pooling memory across tenants for "richer context" builds a cross-tenant instruction channel that is invisible at the vector-similarity layer. In a multi-tenant agent, if tenant A can write content that tenant B later retrieves, then A can plant a false fact or a malicious procedure that executes in B’s session — a write primitive attached to the classic multi-tenant RAG isolation failures. The isolation boundary that matters is at the write and retrieval layer, not the storage layer: encrypting rows or namespacing keys does nothing if similarity search crosses the tenant boundary. The safe default is per-tenant isolation, with sharing limited to provenanced, independently verified facts that a trusted first-party process promoted — never raw user- or retrieval-sourced content. Personalisation is not worth a channel that lets any customer write instructions into any other customer’s agent, and most teams discover the channel only during an incident.

Q: Is memory poisoning a model problem or an architecture problem?

It is an architecture problem, and framing it as a model problem is itself an anti-pattern. No amount of a more capable or better-aligned model fixes a system that permits untrusted content to become trusted state — the model faithfully retrieves and acts on whatever your architecture told it to trust. The controls that work sit entirely outside the model: write authorization in application code, provenance metadata on records, trust-weighted retrieval, quarantine tiers, and continuous contradiction and anomaly scans. This mirrors every other agent-security boundary, from keeping tool-use authorization out of the prompt to governing non-human identities with capability tokens: the durable fixes are systems-design decisions, not prompt engineering. A useful maturity check is that most teams shipping agents in 2026 sit at stage zero — one store, any content writes, pure-similarity retrieval — while believing their input guardrails place them at stage three. That gap between perceived and actual posture is the entire attack.

Satyam Kumar

返回博客

security

Agent Memory Poisoning: The Attack That Persists After the Prompt Ends

By Satyam KumarAugust 12, 202610 min read

agent memory poisoning llm security prompt injection ai agents agent memory memory injection agentpoison minja owasp llm top 10 mitre atlas provenance multi-tenant isolation ai platform security vector database security agent architecture 2026

Agent Memory Poisoning: The Attack That Persists After the Prompt Ends

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

Agent Memory Poisoning: The Attack That Persists After the Prompt Ends

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment