Lion Cipher Lab
Unlearn a data point from a trained model (SISA) and prove it
- AI Security
- Advanced
- ~60 min
- Free · no account required
Why it matters
When someone exercises the GDPR Article 17 'right to erasure' -- the legal right to have their personal data deleted -- most engineers reach for a DELETE statement and consider the job done. It is not. If that person's record was in the training set of a machine-learning model, their influence is baked into the model's weights, and the trained model can still LEAK them: a membership-inference attack (feeding the model an example and reading how confidently it responds) can often tell whether a specific record was in the training data, purely from the model's behavior. Deleting the row from…
What you'll learn
- what machine unlearning IS and why deleting a row from your database does NOT remove that row's influence on already-trained model weights
- building the SISA architecture from scratch: partition data into N shards, train one isolated sub-model per shard, and AGGREGATE their predictions by averaged probabilities (soft voting)
- the key SISA insight -- to forget a point you retrain ONLY the one shard that contained it (1/N of the data), not the whole model -- and measuring that cost saving
- EXACT-unlearning verification: proving the re-aggregated ensemble is byte-for-byte identical (equal sha256 of the pickled models) to a fresh ensemble trained without the record
- a from-scratch membership-inference / confidence probe: showing the forgotten point's confidence collapses from a memorized 1.0000 to an ordinary unseen-point value
- the judgment of exact vs approximate unlearning, and when NOT to unlearn at all: if you can retrain cheaply, a full retrain is the ground truth and beats any shortcut
Steps in this lab
- The problem: deleting the row does not remove its influence
- The vocabulary you need (define every term once)
- Set up a clean, local Python environment
- Write the SISA library and train a BASELINE model
- Build the SISA ensemble and see what sharding costs you
- Unlearn one record -- retrain ONLY its shard
- VERIFY (a): exact unlearning -- byte-for-byte equal to a full retrain
- VERIFY (b): membership probe -- the point now behaves unseen
- The judgment: exact vs approximate, and when NOT to unlearn