Lion Cipher Lab

Benchmark prompt-injection defenses with Attack Success Rate (ASR): measure whether a defense actually works — and by how much (free, local Ollama — no API key)

• AI Security
• Advanced
• ~75 min
• Free · no account required

Why it matters

The AI-security labs before this one teach you to BUILD prompt-injection defenses — a tool broker, a hardened RAG pipeline, a detector. But building a defense and knowing it works are two different jobs, and the second is the one that keeps you honest. The failure mode everyone falls into is the anecdote: you paste one 'ignore previous instructions' payload, the model refuses, you declare victory and ship. That is not evidence — it is a single sample with no denominator. A defense can crush the exact payload you tested and be completely blind to a category you never tried; a lucky refusal on…

What you'll learn

• the difference between BUILDING a defense and EVALUATING one — measuring effectiveness, not asserting it
• defining Attack Success Rate (ASR): successes / attempts, and why a rate beats a single anecdotal test
• building a labeled attack corpus BUCKETED into categories (override, ignore-previous, roleplay/jailbreak, indirect/document, delimiter-encoding evasion, refusal-suppression)
• writing a DETERMINISTIC, programmatic success check (regex/string on the output) so scoring is reproducible, not human judgment
• measuring a baseline per-category ASR against an undefended local LLM app on Ollama
• applying spotlighting / datamarking (Microsoft) to fence and mark untrusted input, then re-measuring the delta
• adding a deny-by-default output gate and an input filter, and seeing which CATEGORIES each moves and which it doesn't
• reading a category x defense ASR table and interpreting defense-in-depth (reduction, not elimination)
• the corpus-bias / base-rate trap: your ASR is only as honest as your corpus (adaptive attackers, unseen categories)
• knowing when NOT to trust an ASR number, and why prompt injection is an unsolved, residual-risk problem

Steps in this lab

1. The point of this lab: measuring a defense, not building one
2. Set up the free, local target: Ollama + a small model + Python
3. Build the benign target 'app' with a secret and a forbidden action
4. Build the CATEGORIZED, labeled attack corpus
5. Write the deterministic scorer and measure BASELINE ASR
6. Defense 1 — spotlighting / datamarking, then RE-MEASURE the delta
7. Defense 2 — a deny-by-default OUTPUT GATE, then RE-MEASURE
8. Defense 3 (optional) — an input filter, and building the final table
9. The judgment: what ASR tells you, and when it lies

Open the full interactive lab →