Lion Cipher Lab
Measure and defend against many-shot jailbreaking (the power-law attack on your context window)
- AI Security
- Advanced
- ~90 min
- Free · no account required
Why it matters
Almost everything written about LLM attacks is about a string — a clever sentence you filter, sanitize or block. Many-shot jailbreaking is not that, and confusing the two is why teams ship defenses that measure well and fail in production. In MSJ the attacker pastes a long fabricated transcript into a single prompt: dozens or hundreds of made-up exchanges in which a supposed assistant cheerfully does the thing your assistant is supposed to refuse, followed by a final request that, read on its own, is completely unremarkable. Nothing in that final request trips a keyword filter, because the…
What you'll learn
- what many-shot jailbreaking (MSJ) is: many FABRICATED dialogue turns stuffed into one prompt so in-context learning steers the final answer
- why MSJ is categorically different from a crafted injection string — it scales with CONTEXT LENGTH, and the final request is unremarkable in isolation
- the vocabulary, defined from zero: context window, shot, in-context learning, attack success rate (ASR), power law, intercept vs slope
- running a shot-count sweep (0/1/2/4/8/16/32/64/128/256) and reading ASR as a CURVE instead of a single number
- fitting a power law with statistics.linear_regression in log-log space and reading the exponent (measured: 0.610, R^2 0.9986)
- measuring four defenses with real before/after numbers instead of arguing about them
- why a final-turn refusal-keyword filter scores 0/10 against MSJ while scoring 3/3 against classic injections
- pricing a context cap: cap 32 cuts ASR 62.6% -> 17.6% but breaks 7 of 12 legitimate long-context workloads
- the crucial distinction: prompt classification moved the INTERCEPT 31x but left the SLOPE (0.610 -> 0.616) alone
- converting an intercept win into the only unit that matters — shots (and therefore tokens) the attacker must spend
Steps in this lab
- What many-shot jailbreaking is — and why it is not just another injection string
- Open a terminal and check Python (zero-assumption setup)
- Why the model is a STUB — and the last three words you need (ASR, power law, intercept vs slope)
- Write the full program
- Run it and read the shot-count sweep — see the power law with your own eyes
- Defense 1 — the refusal-keyword filter, and the cleanest failure in LLM security
- Defense 2 — cap the context, then price what you just broke
- Defense 3 — classify and modify the prompt: the intercept moves, the slope does not
- Defense 4 — role integrity: the assistant role is a trust boundary
- Named failure modes — write these down before you design anything
- Judgment: when a cap is right, when it is product vandalism, and how a pro does it
- Optional appendix (not graded): point the same harness at a real local model