Lion Cipher Lab
Test whether a phishing click rate really fell
- GRC / Compliance
- Intermediate
- ~35 min
- Free · no account required
Why it matters
Phishing simulations produce one headline number, the click rate, and every quarter someone reports whether it went down. NIST SP 800-50 Rev. 1 expects a learning program to measure whether it works, and a falling click rate is the usual evidence. A drop of a couple of percentage points across campaigns of a few hundred people is often just chance, and treating it as proof leads to bad decisions: celebrating a program that did nothing, or cutting one that works because a quarter looked worse. A second trap is that click rates depend heavily on how convincing the lure was, so an easier email…
What you'll learn
- the two-proportion z-test, step by step
- reading a p-value without overclaiming
- why small campaigns rarely prove a change
- how pooling more data changes the answer
- spotting a lure-difficulty confound
- reporting awareness metrics honestly
Steps in this lab
- A click rate is a sample proportion
- Four campaigns
- Write the test
- Run it: not proven, then proven but confounded
- Report it so it means something