Lion Cipher Lab

Evade a text classifier with word-substitution attacks — then measure the defense

• AI Security
• Advanced
• ~75 min
• Free · no account required

Why it matters

A spam filter, an abuse/toxicity classifier, a phishing detector, a resume screener, a prompt-safety classifier that decides whether to let a message through — these are all TEXT classifiers, and organizations increasingly wire them in as a control: 'the model will catch it.' This lab is the reason you never trust a single ML content filter as your sole control. In the image world an adversarial example adds a tiny, imperceptible perturbation to every pixel; text is different in kind, because you cannot nudge a word by 0.01 — so a text attacker SUBSTITUTES tokens that a human still reads the…

What you'll learn

• stating the TEXT threat model before measuring: what a meaning-preserving perturbation is (synonym swap vs character trick), the budget (max % of words changed), and the eval-set size — a text-robustness number without these is meaningless
• training a small, fully-local TF-IDF + LogisticRegression text classifier on an inline seeded corpus and recording baseline CLEAN accuracy the honest way (a held-out test set you never trained on)
• building a from-scratch word-importance ranking: score each word by the drop in the model's confidence when you DELETE it, so the greedy attack spends its tiny budget on the tokens the classifier actually leans on
• implementing a TextFooler/DeepWordBug-style evasion attack — greedy meaning-preserving substitution using a synonym map AND character-level tricks (homoglyphs, swap/delete/duplicate a char) under a budget and a semantic constraint
• measuring ATTACK SUCCESS RATE (ASR), average model queries, and average % of words perturbed on the held-out set — and reading a high ASR as evidence the single ML filter is not a control you can trust alone
• DEFENDING with input normalization (homoglyph-folding + spell-correction to the training vocabulary) and RE-MEASURING with the identical protocol, naming the clean-accuracy cost the normalization introduces
• DEFENDING with adversarial training (augment the training set with attacked examples, retrain) and re-measuring — then running the honest cross-attack test: evaluate against a DIFFERENT perturbation than you trained on
• naming the text failure modes: character attacks beat naive word models but normalization crushes them; synonym attacks survive normalization; a defense that saw only ONE attack gives FALSE robustness against the others; empirical text-robustness is unsolved and the goal is measured risk reduction, not a single trusted filter

Steps in this lab

1. The text threat model: you substitute words, you don't add noise
2. Set up: train a local spam-vs-ham classifier and record CLEAN accuracy
3. Word-importance ranking: find the tokens the classifier leans on
4. The greedy substitution attack — and measure ASR
5. Defense 1: input normalization — crushes the character attack, misses the synonym attack
6. Defense 2: adversarial training — and the honest cross-attack test
7. The verdict — name the failure modes, and why one filter is never the wall

Open the full interactive lab →