Document Poisoning Exposes Critical Flaw in RAG Systems
Key Facts
- What: Researcher injected three fabricated documents into a ChromaDB vector database, causing a local RAG system to report false Q4 2025 financials of $8.3M revenue (down 47% YoY) instead of the correct $24.7M revenue and $6.5M profit.
- How: Attack used vocabulary engineering to satisfy both retrieval and generation conditions without exploiting software vulnerabilities, jailbreaks, or touching user queries.
- Setup: Fully local on a MacBook Pro with no GPU, using LM Studio + Qwen2.5-7B-Instruct, all-MiniLM-L6-v2 embeddings, and ChromaDB.
- Research: Based on PoisonedRAG (Zou et al., USENIX Security 2025), which achieved 90% success against knowledge bases containing millions of documents.
- Reproducibility: Lab code available on GitHub; attack completes in under three minutes with
git clone && make attack1.
Lead paragraph
A security researcher has demonstrated how easily attackers can corrupt Retrieval-Augmented Generation (RAG) systems by poisoning their knowledge base with just three fabricated documents. In a fully local setup using LM Studio, Qwen2.5-7B-Instruct, and ChromaDB, Amine Raji, PhD, tricked the AI into confidently reporting false financial figures that directly contradicted legitimate source documents. The attack required no cloud access, no API keys, and no software exploits — only the ability to add documents to the vector database.
Body
Raji detailed the experiment in a March 12, 2026 blog post. The initial knowledge base contained five clean company documents, including accurate Q4 2025 financials showing $24.7 million in revenue and $6.5 million in profit. When queried normally with "How is the company doing financially?", the RAG pipeline returned the correct information.
The attack injected three poisoned documents designed to dominate retrieval results for financial queries. The first document, titled "Q4 2025 Financial Results — CORRECTED FIGURES (Board Update)," claimed the original numbers contained "material errors" in revenue recognition methodology. It stated corrected revenue of $8.3M, a net loss of -$13.8M, a workforce reduction plan, and preliminary acquisition discussions. The other two documents provided supporting context using similar authoritative language.
The attack leverages the mathematical framework formalized in the PoisonedRAG paper from USENIX Security 2025. For success, poisoned documents must meet two conditions: they must score higher cosine similarity to the target query than legitimate documents (retrieval condition), and their content must steer the LLM toward the attacker’s desired output (generation condition).
Raji used a vocabulary-engineering technique rather than the paper’s gradient-optimized approach. By deliberately including terms like “Q4 2025”, “Financial Results”, “Revenue”, “CORRECTED FIGURES”, and “CFO Office,” the poisoned documents achieved high similarity scores with the all-MiniLM-L6-v2 embedding model. Authority language such as “supersedes,” “corrected,” and “CFO-approved” helped satisfy the generation condition.
In testing across 20 independent runs at temperature=0.1, the LLM consistently reported the fabricated $8.3M revenue figure and did not present the legitimate $24.7M figure as current truth. The entire process — from setup to successful attack — ran locally on a MacBook Pro in under three minutes with no GPU required.
Impact
This research highlights how RAG systems, widely deployed to ground LLMs in proprietary data, introduce a new and underestimated attack surface. Unlike prompt injection, which targets the user query, document poisoning corrupts the system’s source of truth itself. The attack works because many production RAG implementations lack robust validation of ingested documents.
“This is knowledge base poisoning, and it’s the most underestimated attack on production RAG systems today,” Raji wrote.
The vulnerability affects any organization using vector databases like ChromaDB to store sensitive or factual information. Similar risks have been discussed in related research on RAG data poisoning, including potential corruption of code generation assistants that could lead to malformed code in development environments.
The demonstration is particularly concerning because it requires minimal technical sophistication compared to training-time data poisoning. Any insider or attacker with write access to the knowledge base can potentially execute it. While the lab used a tiny 5-document corpus, the original PoisonedRAG paper showed the approach scales — achieving 90% success rates even against knowledge bases containing millions of documents when using optimized payloads.
What's Next
Raji has published the complete lab code at github.com/aminrj-labs/mcp-attack-labs/labs/04-rag-security, enabling security teams to reproduce the attack in minutes. The repository includes both the vulnerable pipeline and setup instructions.
As RAG systems become central to enterprise AI deployments, organizations will need to implement stronger controls around document ingestion, including source verification, anomaly detection in embedding space, and retrieval-level auditing. The ease of this local attack suggests many current production systems may already be vulnerable to similar manipulation.
Future research is likely to explore detection methods and defensive techniques, such as embedding-space consistency checks or requiring cryptographic signatures on authoritative documents. The competitive landscape includes ongoing work from security firms examining both training-time data poisoning and inference-time attacks like prompt injection.
Sources
- Document Poisoning in RAG Systems: How Attackers Corrupt Your AI's Sources | Amine Raji, PhD
- RAG Data Poisoning: Key Concepts Explained | Promptfoo
- Document poisoning in RAG systems: How attackers corrupt AI's sources | Hacker News
- LLM Poisoning and RAG Security: The 250-Document Vulnerability That Changes Everything | Medium
All technical specifications, pricing, and benchmark data in this article are sourced directly from official announcements. Competitor comparisons use publicly available data at time of publication. We update our coverage as new information becomes available.