Lion Cipher Lab

Evaluate adversarial robustness the RIGHT way — PGD, AutoAttack, and the obfuscated-gradients trap

• AI Security
• Advanced
• ~90 min
• Free · no account required

Why it matters

Almost every 'robust' model you read about was not broken by a cleverer attack — it was broken because its EVALUATION was too weak to notice it was never robust in the first place. That is the uncomfortable lesson of the last decade of adversarial machine learning, and it is a DEFENSIVE skill: before you trust a model in anything that matters (a malware classifier, a content filter, a fraud model, a perception stack), you have to be able to measure how robust it really is — not how robust a flattering attack makes it look. The trap is that robustness is measured with attacks, and a weak…

What you'll learn

• stating a threat model before you measure anything: perturbation norm (Linf vs L2), the epsilon budget in pixel units, and the eval-set size — a robustness number without these is meaningless
• measuring ROBUST ACCURACY (accuracy under a bounded worst-case perturbation) rather than clean accuracy, and reading it as an upper bound that only ever goes DOWN as the attack gets stronger
• seeing why a single-step FGSM number OVERESTIMATES robustness and can hide gradient masking / obfuscated gradients — the false sense of security that has broken most published defenses
• running multi-step PGD (Madry) correctly: enough steps, a sane step size (alpha), random restarts, and a full epsilon sweep — and watching robust accuracy collapse versus FGSM
• running AutoAttack, the parameter-free ensemble standard, and confirming it finds adversarial examples PGD missed — so the honest number is the WORST case across ALL attacks
• applying PGD adversarial training (the inner-max / outer-min loop) as a defense and RE-MEASURING with the identical strong protocol, then naming the clean-accuracy tradeoff out loud
• naming the evaluation failure modes: gradient masking, too-weak an attack, too-few PGD steps/restarts, unreported epsilon/threat model, and empirical vs certified robustness
• the professional judgment: report worst-case robust accuracy at a fixed threat model on a known benchmark protocol (RobustBench/AutoAttack) — never a single hand-tuned attack you got to pick

Steps in this lab

1. State the threat model BEFORE you measure anything
2. Set up: train a small MNIST CNN and record CLEAN accuracy
3. FGSM: watch a single-step attack OVERESTIMATE robustness (the gradient-masking trap)
4. PGD (Madry): more steps + restarts + an epsilon sweep drive the number toward the truth
5. AutoAttack: the parameter-free ensemble that finds what PGD missed
6. Defend with PGD adversarial training, then RE-MEASURE with the same strong protocol
7. The methodology verdict — name every failure mode, and empirical vs certified

Open the full interactive lab →