Lion Cipher Lab

Write your own softmax, temperature, top-k, top-p and seeded sampler in plain Python, point them at an alert-severity pipeline whose answers you already know, and find out that the log-probability of the label it emitted is a real confidence number while the "confidence: 0.90" it writes about itself is a generated string

• AI Security
• Intermediate
• ~100 min
• Free · no account required

Why it matters

Every team that puts a language model into a detection pipeline meets the same two problems in the same order, and almost nobody meets them on purpose. The first is that the pipeline gives a different answer on Tuesday than it gave on Monday, on a log line that did not change - which means the daily diff is noise, the on-call analyst stops reading it, and the audit trail says nothing about why an alert was closed. The second is that somebody needs a confidence number to route work by, so the prompt is amended to end with 'and state your confidence from 0 to 1', and from that day forward a…

What you'll learn

• writing SOFTMAX, LOG-SOFTMAX, temperature scaling, top-k, top-p (nucleus) and a seeded inverse-CDF sampler from scratch in one short file of pure standard-library Python, with the two bugs that silently destroy a hand-rolled decoder called out and avoided: dropping the token that crosses the nucleus threshold, and reading the logprob off the FILTERED distribution instead of the raw one
• quantifying a detection pipeline's NON-DETERMINISM instead of complaining about it: 2.693 distinct verdicts per alert across 16 repeats at temperature 1.0, 337 of 600 rows changing between two runs of identical code, and 78 of those crossing the escalate / do-not-escalate line an on-call analyst actually reads
• knowing the difference between REPRODUCIBLE and CORRECT - a fixed per-alert seed takes the run-to-run diff from 337 rows to 0 while leaving accuracy at 0.4633 and the malformed rate at 29.8%, which is reproducible garbage and worth naming as such
• using top-k and top-p as SCHEMA GUARDS rather than as creativity knobs: the unparseable-verdict rate falls 30.5% -> 11.7% -> 0.2% as top-p goes 1.0 -> 0.9 -> 0.8, and to 0.0% at greedy, with the whole sweep printed
• finding top-p's actual failure mode with a measurement - split the same alerts by first-token entropy and the nucleus at p = 0.95 holds 9.72 tokens on the peaked third and 13.84 on the flat third, with the malformed rate going 17.5% -> 35.0%, so the guard is weakest exactly on the alerts the model is least sure about
• turning a LOG-PROBABILITY into a confidence and grading it against a planted key with a Brier score, a 10-bucket expected calibration error and a discrimination score, then fitting ONE temperature on a held-out 300-alert calibration split (T = 2.00) that takes ECE from 0.1252 to 0.0253 and Brier from 0.1987 to 0.1770 without moving a single label
• proving that a self-reported confidence is a GENERATED STRING and not a measurement: 3 distinct values across 600 alerts, Brier 0.2188 which is worse than printing the base accuracy to everything (0.2080), and 23 wrong verdicts hiding in the 100 it was most confident about against the logprob's 2
• the distinction that survives the stub - CALIBRATION fixes the level, DISCRIMINATION is the information. Histogram-binning the self-report on held-out data takes its ECE from 0.1485 to 0.0663 and leaves its discrimination at 0.2667, exactly where it started, against the logprob's 0.4333
• spotting the LENGTH BIAS in a multi-token label: each decoding position leaks 12-15% of its mass off-schema, so a 3-token label pays that toll three times and a 2-token label twice, and the raw sequence score chose the shorter label in 7 of 7 disagreements while the popular fix (dividing by token count) over-corrected and chose the longer one in 16 of 16
• being able to say why SAMPLE-AND-VOTE cannot beat greedy on a fixed distribution - and then building the version that does, where a 15-call greedy vote reaches 0.7183, the exact ceiling of a noise-free read, while 15 sampled calls reach 0.7067

Steps in this lab

1. The words first: logit, softmax, temperature, top-k, top-p, greedy, seed, log-probability, Brier score, calibration error - and the fact that the model here is a stub
2. Open a terminal and make a scratch folder (all three operating systems)
3. The corpus: 900 alerts whose correct severity you already know
4. Write the decoder from scratch: softmax, temperature, top-k, top-p, greedy and a seeded sampler
5. The model is a STUB, and here is its entire error model in writing
6. The pipeline: one alert in, one structured verdict out - and three ways to score it without sampling
7. Decode one alert with every intermediate number on screen
8. The measurement that gets a pipeline distrusted: 337 of 600 rows changed between two identical runs
9. What top-k and top-p actually buy, and the place top-p is weakest
10. The punchline: a logprob is a measurement, a self-reported confidence is a string
11. The bias nobody mentions: an un-normalised sequence logprob quietly prefers short labels
12. Testing the folklore: does sampling several times and voting beat one greedy decode?
13. Temperature 0 is not a promise of a bit-identical answer, and here is how far from one it actually is
14. When to reach for each of these, when not to, and the way a pro sets it up
15. Ship it: the decoder configuration to run in a detection pipeline, and what to write down

Open the full interactive lab →