Lion Cipher Lab
Plant intuitive lure alerts and measure when System-2 reasoning gets anchored too (and fix it with consider-the-opposite)
- AI Security
- Intermediate
- ~75 min
- Free · no account required
Why it matters
Some alerts are built to fool a fast reader, and some just happen to. A Sysinternals tool with a scary name runs on a domain controller, but it was launched under a signed change ticket by the on-call admin. A login looks perfectly quiet, but the session token belongs to a laptop that was reported stolen and revoked yesterday. A domain looks like a typosquat of your vendor, but it is the vendor's real content delivery network. In each case the thing you notice first points one way and the evidence that actually decides the case points the other. Psychologists have a test built from exactly…
What you'll learn
- building an answer-keyed alert set that mixes 500 normal alerts with 100 planted LURES, alerts whose eye-catching surface feature points confidently the wrong way while the decisive evidence points the right way - the shape of a Cognitive Reflection Test item, rebuilt for security triage
- writing one plainly-labelled stub reasoner with three modes whose behaviour is written down as constants: fast (System 1), anchored deliberation that starts from the fast answer, and deliberation with a consider-the-opposite step
- measuring accuracy on normal and lure alerts separately, plus the confidence of wrong answers and the token cost per mode, instead of quoting one overall number
- keeping the honest negative that deliberation which starts from the intuitive answer mostly justifies it: anchored reasoning recovered 18 of 100 lures at 5.5x the tokens of the fast answer
- pricing the debiasing step on the alerts that did not need it: consider-the-opposite lifted lure accuracy to 0.900 but dropped normal accuracy from 0.994 to 0.984 and spent 67,500 extra tokens on normal alerts
- showing with numbers why a confidence gate cannot find lures (System 1 is confidently wrong on them) and replacing it with a cheap conflict detector that sends 104 alerts and scores 0.940 overall for about a third of the all-opposite token bill
Steps in this lab
- The words first: lure, Cognitive Reflection Test, System 1 and System 2, anchoring, consider-the-opposite - and the fact that the reasoner is a stub
- Open a terminal, check Python, make a scratch folder (Windows, macOS, Linux)
- One-step recap: what the race lab and the self-consistency lab already showed, and the blind spot they share
- Build the keyed alert set: 500 normal alerts and 100 planted lures
- Write the stub reasoner: fast, anchored and consider-the-opposite
- The reasoner is a STUB, and here is its whole behaviour in writing
- Measure: accuracy on normal vs lure alerts, confidence on wrong answers, and cost
- Read it honestly: deliberation that starts from the intuitive answer mostly justifies it
- Gate it: why the confidence gate fails on lures, and a cheap lure detector that works better
- When, why, and how a pro does it differently
- Ship it: the policy, what to log, and how to see what the stub is doing