Lion Cipher Lab
Scan an ML model for malicious pickle code (before it runs code on you)
- AI Security
- Advanced
- ~75 min
- Free · no account required
Why it matters
The single most common way to ship a model — a PyTorch .bin/.pt or a scikit-learn .pkl — is Python pickle, and loading a pickle runs whatever code the file's author put in it. Public hubs let anyone upload a model with any name; a poisoned 'llama-2-7b.bin' can steal your cloud credentials the moment a data scientist runs torch.load on it. Knowing how to prove a model is malicious WITHOUT trusting it, and how to move your pipeline onto a format that can't execute, is the difference between a model supply chain you can audit and one you just hope is clean.
What you'll learn
- why a pickle-based model file (.bin/.pt/.pkl) is arbitrary code execution the moment you load it
- authoring a benign proof-of-concept malicious model via a __reduce__ payload, safely
- statically scanning an untrusted model with picklescan and fickling WITHOUT executing it
- converting weights to safetensors so they physically cannot carry code, and using torch weights_only
- gating model provenance in CI: scan-before-load, prefer safetensors, treat every downloaded model as untrusted input
Steps in this lab
- The one fact that makes this a security problem
- Set up a disposable workspace
- Craft a benign 'malicious' model
- Watch a naive load execute it
- Scan it WITHOUT running it — picklescan
- Read the actual opcodes — fickling
- Move to a format that can't execute — safetensors
- Gate it in CI — and name what still gets past you
- Clean up