Lion Cipher Lab
Steal a model with a model-extraction attack: train a victim classifier, query it, and clone it into a surrogate (free, local scikit-learn — no API key)
- AI Security
- Advanced
- ~60 min
- Free · no account required
Why it matters
The moment you expose a trained model behind a prediction API — a public MLaaS endpoint, an internal microservice, a mobile app that phones home — you are handing out samples of the exact function that model computes. An attacker who never sees your weights or your training data can query that API, keep the input-output pairs, and train their own SURROGATE that copies your decision boundary. That lets a competitor clone a model you spent money and private data to build, and it gives an attacker a free local copy to craft further attacks against without tripping your monitoring. Model…
What you'll learn
- what a model-extraction (model-stealing) attack is and when it is a real threat
- training a small victim classifier and exposing it behind a prediction API
- playing the black-box attacker: querying, harvesting labels, and training a surrogate
- measuring FIDELITY (surrogate-vs-victim agreement) versus plain accuracy
- seeing why an exposed prediction API leaks the function it computes
- the utility-vs-leakage tradeoff of returning confidence scores instead of top-1 labels
- defenses: rate-limiting, top-1 labels, output rounding/perturbation, query monitoring, watermarking
- the honest limits: a surrogate is an approximation and high fidelity costs many queries
Steps in this lab
- The words you need: victim, surrogate, black-box, fidelity
- When extraction is a real threat, and why it works
- Set up: install scikit-learn and split the data three ways
- Train the victim and stand it up behind a local API
- Be the attacker: harvest predictions and train a surrogate
- Run it, then watch fidelity grow with the query budget
- Defend: shrink the leak, raise the cost, and watch for extractors
- Honest limits, and doing it better