Lion Cipher Lab
Leak training data from gradients in federated learning — and defend it
- AI Security
- Advanced
- ~75 min
- Free · no account required
Why it matters
Federated learning is sold on one promise: 'your data never leaves your device — we only share gradients.' Phone keyboards, hospital consortia, and bank fraud models all lean on it. This lab breaks that promise and then repairs it, which is exactly the arc a privacy engineer has to be able to reason about. The uncomfortable fact, established by Zhu, Liu & Han's 'Deep Leakage from Gradients' and sharpened by iDLG and Geiping et al.'s 'Inverting Gradients', is that a gradient is NOT an anonymous summary of your data — it is a deterministic function of the exact input that produced it, and for a…
What you'll learn
- standing up a minimal federated-learning setting in PyTorch: one small model, one client's private example, and the gradient that client would upload to the server
- seeing WHY a shared gradient is not anonymous — it is a function of the exact input that produced it, so it can be inverted
- running the Deep Leakage from Gradients (DLG) attack: optimize a dummy input+label from random noise until its gradient matches the shared gradient, reconstructing the private image pixel by pixel
- the iDLG improvement: recovering the ground-truth label analytically from the sign of the last-layer gradient, which makes the reconstruction far more stable
- quantifying reconstruction quality with pixel MSE and cosine similarity to the original, instead of eyeballing it
- watching leakage degrade as the batch size grows — why single-example leakage is the strong case and large batches are a natural defense
- defending with the DP-SGD mechanism (per-update gradient clipping to an L2 bound, then calibrated Gaussian noise) and watching reconstruction collapse
- naming the privacy/utility tradeoff out loud: a larger noise multiplier buys a stronger worst-case guarantee but costs model accuracy — and where secure aggregation fits
Steps in this lab
- Why a shared gradient is not anonymous
- Build a tiny federated setting: one model, one private image, one shared gradient
- Run the DLG attack: optimize noise until its gradient matches
- iDLG: recover the label for free, and measure the reconstruction
- Find the edge: leakage degrades as the batch grows
- Defend with the DP-SGD mechanism: clip, then add Gaussian noise
- Name the cost: the privacy/utility tradeoff, and where secure aggregation fits
- The judgment: 'we only share gradients' is not a privacy control