Lion Cipher Lab
Detect a model-extraction attack against your own classifier from its query stream: build a PRADA-style monitor, measure detection vs false positives, and see why a patient attacker still slips through (free, local scikit-learn)
- AI Security
- Advanced
- ~90 min
- Free · no account required
Why it matters
Model extraction is usually taught from the attacker's chair: query a deployed model, harvest its answers, train a clone. But if you RUN a model behind an API, your job is the other side of that transaction, and you cannot un-expose a model you are paid to serve. So the practical question becomes: can you SEE a cloning attempt in the traffic while it is still happening? Juuti et al.'s PRADA work observed that extraction queries, which are engineered to probe the decision boundary, cluster unnaturally: the distribution of distances between a client's successive queries stops looking like the…
What you'll learn
- the defender's view of model extraction: catching a cloning attempt from the query stream, not just stopping it at the model
- what a query stream is and why extraction traffic looks different from natural user traffic
- implementing a PRADA-style detector: per-client growing-minimum distances plus a Shapiro-Wilk normality test
- simulating a benign user stream and a boundary-probing extraction stream against your own classifier
- measuring detection rate (true positives) against false-positive rate, and sweeping the threshold as an ROC-style tradeoff
- choosing an operating point: the largest threshold that still raises zero false alarms on benign power users
- the honest negative: a patient attacker querying slowly with in-distribution inputs evades distance-based detection entirely
- why this detector must be paired with per-client rate limits, query budgets, and output rounding/perturbation to mean anything
Steps in this lab
- The words you need: victim, query stream, and why extraction traffic is shaped differently
- When this detector earns its keep, and when it is security theater
- Set up: install the libraries and build the victim phishing classifier
- Build the PRADA-style detector: growing-minimum distances plus a normality test
- Simulate the two query streams: benign users vs a boundary-probing extractor
- Run it and read the ROC-style tradeoff: detection rate vs false positives
- The honest negative: the patient attacker walks straight through
- Honest limits, and doing it better