Lion Cipher Lab
Defend a RAG pipeline against indirect prompt injection: poison a document, watch the model obey it, then shut it down (free, local Ollama — no API key)
- AI Security
- Advanced
- ~60 min
- Free · no account required
Why it matters
The moment your app answers questions using text it retrieved — a document, a web page, a support ticket, an email — you have opened a second, quieter door for attackers. Direct prompt injection is what a user types; INDIRECT prompt injection is an instruction hidden inside the content your pipeline pulls in and hands to the model. Because a language model cannot reliably tell your instructions apart from instructions buried in the data it is reading, a single poisoned document can make it leak secrets, follow a malicious link, or quietly change its answer for everyone who asks. RAG systems…
What you'll learn
- what INDIRECT prompt injection is (OWASP LLM01) and how it differs from direct injection
- building a tiny local RAG: Ollama embeddings + a local chat model, pure-Python glue (no pip, no API key)
- poisoning a retrieved document and watching a model obey instructions that arrived through the DATA channel
- spotlighting / data-marking: fencing and labelling retrieved context as untrusted reference material
- writing a strict system prompt that treats retrieved text as data, never as instructions
- adding an independent output gate that blocks exfiltration regardless of what the model decided
- using source provenance / trust as a defense-in-depth layer
- the architectural rule: never let retrieved content trigger a privileged action without an independent gate
Steps in this lab
- Direct vs INDIRECT prompt injection — the word that matters here
- Why the data channel is the dangerous one, and why you can't just filter it
- Install Ollama and pull two small models (embedder + chat)
- Write the pipeline: a tiny local RAG with one poisoned document
- Run it naive and watch the model obey the document
- The layered defense: spotlighting, a strict system prompt, and an output gate
- Re-test: same poison, no bite
- The judgment: why marking is not enough, and the one rule that is
- Do it better — where a production version goes