RAG Citation Forgery: Verify Link, Passage and Claim

Q: What is RAG citation forgery?

Citation forgery is any case where a citation rendered to the user misrepresents the relationship between a generated sentence, the passage that supposedly supports it and the source that passage came from, whether by accident or by an attacker's design. A citation is a claim that a specific sentence is supported by a specific passage from a specific source. In a retrieval-augmented system every part of that claim is reachable by someone other than the operator: the model can invent a source, attach a real passage to a sentence it does not support, be told by an injected instruction which URL to cite, or cite a planted document whose title and formatting impersonate an authoritative one. Because users treat citations as evidence, a forged citation is more persuasive than the same misinformation stated without one, which is why attackers target it and why verifying citations is a security control rather than a quality feature.

Q: What are the four classes of citation forgery and where do they enter a RAG pipeline?

Fabricated citations enter at generation, when the model emits a URL, title or document that does not exist or was never retrieved. Misattributed citations enter at generation, when a genuine retrieved passage is cited for a sentence it does not actually support. Planted citations enter at the corpus, when an attacker's document carries metadata, a title, a domain or formatting that impersonates an authoritative source, so the passage supports the claim but the identity is false. Substituted citations enter at generation or rendering, when an instruction inside a retrieved document tells the model which URL to cite, or when the renderer builds hyperlinks out of URLs found in model output. Each class defeats a different naive check: link checking passes fabricated links that resolve, entailment passes planted and substituted citations because the passage is fine, and title display passes planted documents because the author controls the title.

Q: Why should a model emit reference identifiers rather than URLs when citing?

Because text written by the model cannot be verified and a structured reference can. A citation written as free text has no binding to anything the system retrieved, so it can name a document that was never in the retrieval set or a URL supplied by an injected instruction. A citation emitted as a document identifier plus span offsets can be resolved only against the documents retrieved for this request, which makes fabrication impossible by construction and turns any injected cite-this instruction into either an unresolvable identifier that is dropped or a URL in model text that the output layer renders as plain text. Anthropic's citations feature returns the document index and exact passage locations for each cited block, and Google's Vertex AI grounding returns grounding metadata tying response segments to supporting chunks; both give the system the object it needs. The system, not the model, then writes the link from its own stored source record.

Q: What does a render-time citation resolver do?

It intercepts the model's structured references before anything reaches the user and performs three checks in order. Resolve confirms that the identifier maps to a document in this request's retrieval set and loads the stored source record, dropping and logging anything unresolvable. Entail checks that the cited span, not the whole document, supports the sentence it is attached to, using a small natural-language-inference classifier or a cheap judge call. Attest renders the link from the canonical record's URL and labels it with the source's trust tier and retrieval time, so the user sees what kind of source they are trusting without clicking. A sentence whose citation fails any check is rendered as unsupported with a visible marker or removed, according to the product's policy for that trust tier, and is never rendered with a broken or unverified footnote. Run in parallel across an answer's citations, the resolver typically adds between 150 and 400 ms.

Q: How do trust tiers and a source registry defend against planted citations?

A planted document defeats entailment because its passage genuinely supports the claim; the lie is its identity. The defence is to bind identity at ingest rather than trust the document's own metadata. Every document entering the corpus receives a canonical identity recording where it came from, through which connector and under which authority, together with a content hash. Trust tiers are assigned to sources, not documents: the HR system of record is tier A, a shared drive is tier C, a public web fetch is tier D. A document calling itself HR Policy v3 that arrived through the shared-drive connector is tier C and the renderer says so next to the footnote. Where sources sign their content, C2PA manifests and durable content credentials give the registry a cryptographic identity to bind instead of a filename, which is the only robust defence once attackers learn what the renderer displays.

Q: How should citations to web search results be handled, given the source is not in the corpus?

Web-grounded answers cite pages the system did not ingest, so the registry has no hash, no trust tier and no guarantee the page will say the same thing tomorrow, and a party who planted a page can change or remove it after the answer is given. The honest architecture snapshots the fetched content at answer time, hashes it, stores it with the retrieval timestamp, and cites the snapshot rather than the live URL, showing the retrieval date beside the footnote. The snapshot is also what the entailment check runs against, so the support relation is evaluated on the content the model actually read. Web-fetched sources should be rendered with a visibly lower trust tier than systems of record so that a user's trust in internal sources does not transfer to arbitrary pages, and for high-stakes intents the policy may be to refuse to cite tier-D sources at all.

Q: What are the measures for citation quality, and what should be tracked in production?

The ALCE benchmark from Princeton defined the two measures that matter. Citation recall asks whether each claim in an answer has at least one supporting citation, and citation precision asks whether each citation actually supports the claim it is attached to. Models prompted to cite fall well short on both, which is why a resolver rather than a prompt is needed. In production the resolver should emit both numbers per answer, along with the count of unresolvable references dropped, the count of sentences rendered as unsupported, and the distribution of trust tiers cited, and the evaluation harness should track them over time and per intent tier. Seeded canary documents with lookalike identities and cite-this instructions provide a direct test: any canary that reaches a rendered citation is an incident, not a metric.

Q: When is the right answer to render no citation at all?

Whenever the pipeline cannot bind the link to a passage in this request's retrieval set and the passage to the sentence. If the entailment check rejects every candidate span for a sentence, the sentence is the model's own assertion and should be rendered as such or removed, because a footnote on it would be a forgery the product committed on its own behalf. If every retrieved passage for a claim is a low-trust web fetch, the answer should say so rather than dress it in the same footnote style as a system-of-record source. If the product cannot afford the resolver's latency for a given surface, it should ship that surface without citations rather than with decorative ones, because a citation is a trust signal the product issues and must be able to back. The rule is simple: a citation you did not resolve is a citation you should not render.

Satyam Kumar

返回博客

cybersecurity

RAG Citation Forgery: A Citation Is a Claim About a Source, Not Evidence, So Verify the Link, the Passage and the Claim as Three Separate Things (2026)

By Satyam KumarOctober 5, 202617 min read

rag citation forgery llm citation verification fabricated citations llm rag source attribution security citation entailment check render-time citation resolver anthropic citations api vertex ai grounding metadata owasp llm09 misinformation poisonedrag citation attack alce citation precision recall source trust tiers rag answer engine citation spoofing indirect prompt injection cite this rag provenance content hash ai application security 2026

RAG Citation Forgery: A Citation Is a Claim About a Source, Not Evidence, So Verify the Link, the Passage and the Claim as Three Separate Things (2026)

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

RAG Citation Forgery: A Citation Is a Claim About a Source, Not Evidence, So Verify the Link, the Passage and the Claim as Three Separate Things (2026)

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment