Lion Cipher Lab
Plant and detect a data-poisoning backdoor: train a classifier that passes every clean test yet flips on a hidden trigger (free, local, CPU-only)
- AI Security
- Advanced
- ~60 min
- Free · no account required
Why it matters
Modern machine learning is assembled from other people's parts: datasets scraped from the public web, labels bought from crowd workers, and pretrained weights pulled from a model hub. Any of those can be quietly poisoned before you ever touch them. A data-poisoning backdoor is the sharpest version of that risk — an attacker teaches the model to behave perfectly on normal inputs (so it sails through your accuracy checks) but to flip to an attacker-chosen answer the instant a secret trigger appears. That is a supply-chain compromise wearing a passing test suite. If you build, buy, or deploy ML…
What you'll learn
- planting a BadNets-style backdoor with a fixed trigger patch and a relabel
- poisoning only a small fraction of a training set and still owning the model
- why high clean accuracy hides a backdoor (it passes normal validation)
- measuring Attack Success Rate (ASR) as a separate metric from accuracy
- understanding the ML supply-chain risk: public datasets, crowd labels, pretrained weights
- detecting poison with Activation Clustering (Chen et al.) on hidden-layer activations
- knowing where Neural Cleanse, spectral signatures, and fine-pruning fit — and their limits
- reasoning about defense-in-depth and the honest limits of each backdoor defense
Steps in this lab
- The words you need: backdoor, trigger, target class, clean accuracy, ASR
- Why this is the supply-chain risk of ML — and why clean accuracy can't see it
- Set up the machine and define the trigger
- Plant the backdoor and train the model
- Run it and watch clean accuracy lie to you
- Detect it: Activation Clustering, and where the other defenses fit
- The honest limits, and doing it better