Lion Cipher Lab

Quantize a security classifier by hand and measure what each bit actually costs

• AI Security
• Advanced
• ~90 min
• Free · no account required

Why it matters

A defender who wants AI without sending logs to someone else's cloud runs the model locally, often on whatever hardware is spare - and that means QUANTISATION: storing each weight in 8, 4 or even 2 bits instead of 32 so the model fits and runs fast. The usual advice is 'int8 is free, int4 is nearly free', judged by one accuracy number. This lab measures what that number hides on a small security classifier you train and quantise yourself. At int8 accuracy moved by 0.06 points, yet the verdicts that did change landed eight times more often on a rare attacker technique (certutil pulling a file…

What you'll learn

• writing post-training quantisation yourself: symmetric and asymmetric (zero-point) int8, int4 and int2, with one scale per tensor, per row or per block of 32 weights, and percentile clipping
• counting what a quantised model really weighs: 34,956 bytes in float32 against 8,801 at int8 and 4,440 at int4 per-tensor, scales and zero-points included
• measuring more than accuracy: recall at a frozen 1% false-positive threshold, false-positive rate, expected calibration error written by hand, and verdict FLIPS against the float model
• finding the slice an aggregate number hides: at int8 overall flips were 0.33% but the rare certutil-download slice flipped at 2.70%, and the reason is its small margin
• diagnosing an outlier weight: one weight of 12.0 set the per-tensor int4 step to 1.714, rounded 99.6% of the other weights to zero and caused 607 flips; per-row scales cut that to 36, block-32 to 15
• seeing confidence break while accuracy holds: int2 per-row with clipping kept accuracy at 0.9267 but doubled the malicious lines a probability-based router auto-closed (21 to 40)
• mapping the hand-built pieces onto real formats: GGUF Q8_0 and K-quants, GPTQ, AWQ and LLM.int8(), and choosing a bit-width with a per-slice acceptance test instead of a vibe

Steps in this lab

1. The words first: quantisation, scales, zero-points, granularity, clipping, outliers - and the yardsticks that catch what accuracy hides
2. Open a terminal and make a scratch folder (Windows, macOS, Linux)
3. Build 12,000 answer-keyed command lines, with one rare attacker technique planted as the slice to watch
4. Train the float model, then plant ONE outlier weight in plain view
5. Write the quantiser and the yardsticks yourself: scales, zero-points, granularity, clipping, ECE and flips
6. Sweep every bit-width, scheme and granularity: size, accuracy, recall, FPR, slice recall, ECE and flips
7. Honest negative one: the rare slice flips far more often than the aggregate suggests
8. Honest negative two: one outlier weight wrecks per-tensor int4 - and what per-row scales, blocks and clipping each buy
9. Honest negative three: confidence breaks while accuracy holds, and every probability-based rule downstream breaks with it
10. Honest negative four: int2 collapses - and ECE can call a dead model well calibrated
11. How the pros do it: what GGUF Q8_0 and K-quants, GPTQ, AWQ and LLM.int8() do with the same problems
12. Why this, not that: quantise, distil, prune, or buy a smaller model
13. When NOT to quantise (or not this far)
14. The way a pro does it differently, and the acceptance test to ship

Open the full interactive lab →