Lion Cipher Lab

Filter adversarial-suffix jailbreak prompts by perplexity, and measure what it misses (STUB n-gram language model)

• AI Security
• Advanced
• ~90 min
• Free · no account required

Why it matters

The best-known automated jailbreak against aligned language models, GCG, does not look like persuasion at all. It appends a machine-optimised string of tokens to a request - a suffix that reads like keyboard noise - and that string can push a model into answering what it should refuse. Because the suffix is unnatural, the obvious defence is a perplexity filter: score how surprising the prompt is to a language model and block it if it is too surprising. This lab builds that filter in pure Python and measures it honestly instead of trusting it. It works where you expect: with a short sliding…

What you'll learn

• explaining perplexity in one sentence - the exponential of a language model's average surprise per character or token - and why machine-optimised adversarial suffixes score high on it
• training a character n-gram language model in pure Python as a STAND-IN for a real model's perplexity, and calibrating a threshold as the highest score seen on held-out benign prompts
• measuring whole-prompt against windowed-max perplexity: a 5-token gibberish suffix was caught 0.000 of the time by the whole-prompt score and 1.000 by a 24-character window
• running a control (the same requests with no suffix, 0.000 flagged) so you know the filter reacts to the suffix and not to the request
• measuring false positives on legitimate security inputs - hashes, base64, log lines, code and non-English text were flagged 0.967 to 1.000 of the time - and reading a threshold sweep that shows no cut-off separates them from attacks
• measuring the pro fix (masking known shapes, hash and base64 false positives 1.000 -> 0.000) and the evasion it opens (a base64-wrapped suffix slipped from 1.000 to 0.124 detection)
• building a tiny adaptive attacker that picks the lowest-perplexity fluent framing, which drove detection to 0.000 - and deciding where a perplexity filter belongs: one routing signal, never the defence

Steps in this lab

1. What a perplexity filter is, and why gibberish suffixes trip it
2. Open a terminal and make a scratch folder (Windows, macOS, Linux)
3. Build the training text and six test sets, all from fixed seeds
4. Build the STUB language model and score four prompts by hand
5. Measure whole-prompt against windowed perplexity, with a control
6. Read the threshold sweep: there is no line that separates attacks from hashes
7. Honest negative one: the legitimate inputs a SOC assistant lives on are exactly what trips it
8. The pro move: mask known shapes before scoring - and the hiding place it creates
9. Honest negative two: an adaptive attacker who writes fluent English walks straight past
10. The judgment: where a perplexity filter belongs, and how a pro deploys it
11. Recap, and one experiment that shows the window size was tuned

Open the full interactive lab →