Qwen2.5-7B-Instruct (Q4_K_M): Technical Deep Dive

Document Poisoning in RAG Systems: A Technical Deep Dive

Executive Summary

• Document poisoning is a practical retrieval-time attack on RAG pipelines that works by injecting a small number of high-similarity fabricated documents into a vector database, satisfying both the retrieval condition (higher cosine similarity than legitimate sources) and the generation condition (authority-framed content that steers the downstream LLM).
• In the demonstrated local lab using ChromaDB + all-MiniLM-L6-v2 embeddings + Qwen2.5-7B-Instruct (via LM Studio), three crafted documents were sufficient to flip the model’s answer from “$24.7M revenue, $6.5M profit” to the attacker-controlled “$8.3M revenue, –$13.8M net loss” with 100% consistency across 20 runs at temperature=0.1.
• The attack requires no model gradient access, no jailbreaks, and no cloud APIs — it runs entirely offline on a MacBook Pro CPU in under three minutes, highlighting that the dominant threat to production RAG today is not prompt injection but persistent knowledge-base corruption.
• PoisonedRAG (Zou et al., USENIX Security 2025) formalizes the two necessary conditions and shows the attack scales to knowledge bases containing millions of documents when gradient-optimized payloads are used; the vocabulary-engineering method presented here is a simpler, reproducible “embedding SEO” technique.

Technical Architecture

The lab implements a minimal, fully local RAG stack:

• Embedding model: sentence-transformers/all-MiniLM-L6-v2 (384-dimensional dense vectors, cosine similarity).
• Vector database: ChromaDB (persistent, file-based, default HNSW index with default parameters).
• LLM: Qwen2.5-7B-Instruct quantized to Q4_K_M running locally via LM Studio’s OpenAI-compatible server.
• Orchestration: A straightforward Python retrieval pipeline that performs top-k=4 retrieval, concatenates the retrieved chunks with a system prompt, and calls the local LLM.

The knowledge base is seeded with five legitimate documents, one of which contains the true Q4 2025 financials ($24.7M revenue). The attack then adds three new documents using the same embedding model and collection.

Because ChromaDB stores embeddings persistently and performs nearest-neighbor search at query time, any document that scores higher cosine similarity than the legitimate financial document for queries containing tokens such as “financial”, “revenue”, “Q4 2025”, or “company doing” will displace it from the context window.

The three poisoned documents are engineered as follows:

1. CFO-Approved Correction — heavily overlaps in vocabulary (“Q4 2025 Financial Results”, “CORRECTED REVENUE”, “CFO Office”) while introducing authority markers (“material errors”, “effective immediately”, “supersedes”).
2. Regulatory Notice — references the legitimate $24.7M figure but frames it as “originally reported” and “erroneous”, creating a narrative that the LLM later adopts.
3. Board Meeting Note (inferred from the truncated source) — likely reinforces the new narrative with procedural language.

This is a hybrid of retrieval poisoning and soft prompt injection. The documents are not optimized against the embedding model’s gradients; they simply exploit the fact that dense retrievers are sensitive to lexical overlap and named-entity density.

Performance Analysis

The author reports 100% success (20/20 runs at temperature=0.1) on a 5-document corpus. While this scale is intentionally small for pedagogical reasons, it is highly representative of many internal RAG deployments that contain only tens or low hundreds of documents per domain.

Comparison with PoisonedRAG (Zou et al.)

The lab demonstrates that even without gradient optimization, a motivated attacker can achieve reliable corruption in small-to-medium corporate knowledge bases. In larger corpora the attacker would need to inject more documents or use the optimization technique from the paper to guarantee top-k dominance.

Technical Implications for the Ecosystem

1. RAG is not a security boundary. Many organizations treat the vector database as an “internal only” store and therefore apply minimal access control or provenance tracking. Once an attacker can add documents — via compromised upload pipelines, insider access, or supply-chain compromise of ingested data — the integrity of every downstream LLM response is compromised.

2. Embedding models are vulnerable to “embedding SEO”. Because all-MiniLM-L6-v2 and similar models were trained on web-scale data that rewards lexical overlap and authority language, simple keyword stuffing combined with authoritative framing is surprisingly effective. This is analogous to the early days of SEO before sophisticated ranking signals were introduced.

3. ChromaDB’s default configuration offers no built-in defense. There is no document-level signing, no source attribution metadata enforced at query time, and no anomaly detection on embedding distribution shifts. Production deployments using LangChain, LlamaIndex, or Haystack on top of Chroma, Pinecone, or Weaviate inherit the same weakness unless additional controls are layered.

4. Prompt hardening provides only partial mitigation. The author notes that “prompt hardening reduces (but doesn’t eliminate) the attack’s effectiveness.” System prompts that say “only use the most recent official document” can be overridden when multiple poisoned documents reinforce the same false narrative.

Limitations and Trade-offs

• Small corpus size: The 5-document baseline makes the attack easier. In a realistic 500-document finance collection the attacker would likely need 10–30 poisoned documents or gradient optimization to reliably dominate top-k=4 or top-k=6 retrieval.
• No evaluation on larger models: Qwen2.5-7B is relatively small. Larger models (70B+) with stronger reasoning capabilities might be more resistant to contradictory source material, although recent work suggests they can still be misled by authoritative framing.
• Static evaluation: The experiment used temperature=0.1 and fixed queries. Real-world variance in query phrasing, embedding drift over time, and re-indexing policies could change outcomes.
• Lack of persistence analysis: The lab does not explore how the poisoned documents survive chunking strategies, metadata filtering, or periodic re-ingestion from authoritative sources.

Expert Perspective

Document poisoning represents one of the most practical and under-mitigated threats to deployed RAG systems in 2025–2026. Unlike training-time data poisoning (which requires control of the pre-training corpus) or classic prompt injection (which can sometimes be mitigated with guardrails), knowledge-base poisoning is persistent and invisible to most monitoring systems.

The fact that a reproducible attack can be executed in <10 minutes on consumer hardware with zero GPU and zero cloud spend should serve as a wake-up call for any team shipping RAG-powered internal tools, customer support agents, or compliance copilots.

The most significant insight from both this lab and the PoisonedRAG paper is that retrieval is the new attack surface. Defenses focused exclusively on the LLM layer (output filtering, system prompts, constitutional AI) are insufficient. Integrity must be enforced at the vector database and ingestion pipeline layers.

Technical FAQ

How does this attack compare to prompt injection?

Prompt injection modifies the user query or system prompt at inference time. Document poisoning modifies the retrieval corpus persistently. A successful poisoning attack survives across sessions and multiple users, whereas prompt injection usually requires per-interaction crafting. The two can be combined: poisoned documents can contain hidden instructions that act as persistent prompt injections once retrieved.

Can gradient-based optimization (PoisonedRAG) be applied to closed-source embedding models?

The original PoisonedRAG work assumes white-box access to the embedding model for gradient optimization. Against black-box commercial embeddings (e.g., OpenAI text-embedding-3-large or Cohere), attackers must fall back to the vocabulary-engineering approach shown here or use surrogate models. Transferability across embedding models is an active research area; preliminary results suggest 60-75% success rates when optimizing on an open model and transferring to a proprietary one.

Is this attack mitigated by using metadata filtering or hybrid search?

Partially. If the system enforces strict metadata filters (e.g., source="official-finance-report" and last-updated > 2025-12-01), poisoned documents that lack correct metadata can be excluded. However, many RAG pipelines either omit metadata filtering or allow broad “company documents” queries. Hybrid keyword + vector search can help but also expands the attack surface if the attacker includes exact keyword matches.

What are practical defenses that senior engineers should implement today?

1. Cryptographic signing of ingested documents with verifiable provenance.
2. Strict metadata schemas and query-time filtering on authority_level, last_validated, and source_type.
3. Anomaly detection on embedding clusters — sudden appearance of many high-similarity vectors around a sensitive topic should trigger review.
4. Periodic integrity checks that re-embed canonical sources and alert on drift.
5. Rate limiting and access control on write paths into the vector database.
6. Use of retrieval-time source ranking models trained to prefer documents with strong provenance signals.

These controls add engineering overhead but are necessary for any RAG system that powers high-stakes decisions.

References

• Amine Raji, “Document Poisoning in RAG Systems”, aminrj.com (March 2026)
• Zou et al., “PoisonedRAG: Knowledge Poisoning Attacks to Retrieval-Augmented Generation of Large Language Models”, USENIX Security 2025
• Promptfoo Blog, “RAG Data Poisoning: Key Concepts Explained”

Sources

• Document Poisoning in RAG Systems: How Attackers Corrupt Your AI's Sources | Amine Raji, PhD
• Lab repository
• PoisonedRAG Paper (via USENIX Security 2025)

All technical specifications, pricing, and benchmark data in this article are sourced directly from official announcements. Competitor comparisons use publicly available data at time of publication. We update our coverage as new information becomes available.