Lion Cipher Lab
Sweep a reasoning budget on alert triage, find the knee, and catch where overthinking starts (deterministic stub reasoner)
- AI Security
- Intermediate
- ~75 min
- Free · no account required
Why it matters
Reasoning models let you set how long they think before answering, and the tempting setting is 'as long as possible' - more thinking must be safer for security work. This lab measures that belief instead of trusting it. On 800 answer-keyed alerts and a deterministic stub reasoner whose behaviour is written down, accuracy climbed fast at first (+84 points per 1,000 tokens), flattened by 512 tokens, and then FELL: always thinking for 2048 tokens scored 0.833 on held-out alerts against 0.907 for 512, at almost four times the cost. The worst damage was on the easiest alerts, where the snap answer…
What you'll learn
- building an 800-alert answer key in four difficulty bands, with two different kinds of difficulty kept apart: evidence that is hard to READ (thinking helps) and evidence that is MISSING (thinking cannot help)
- sweeping a thinking-token budget from 0 to 2048 and reading the result as a curve per band: ALL bands went 0.748 -> 0.907 at 512 -> 0.843 at 2048
- finding the knee with an explicit rule (keep climbing only while a step earns at least 10 accuracy points per 1,000 tokens) instead of eyeballing a chart
- catching overthinking in your own numbers: easy alerts fell from 1.000 to 0.895 at 2048, below the 0.995 snap answer, with 21 correct snap answers lost
- fitting a budget-per-band policy on a calibration half and scoring it on a held-out test half, with the cost of the extra call charged honestly
- telling a confidence signal from a difficulty signal: routing by snap margin kept 0.907 for 14% fewer tokens, routing by channel spread saved more and lost 0.7 points
- deciding when a fixed budget at the knee is good enough, when per-alert routing is worth its machinery, and why a hard budget ceiling matters more than either
Steps in this lab
- The words first: System 1 and System 2, thinking tokens, the budget dial, the knee, overthinking - and the fact that the reasoner here is a stub
- Open a terminal and make a scratch folder (Windows, macOS, Linux)
- Build 800 answer-keyed alerts in four bands, with two different kinds of difficulty
- Write the STUB reasoner: a snap answer, one more look per 32 thinking tokens, and a written-down habit of second-guessing
- Sweep the budget: accuracy per band at 0, 64, 128, 256, 512, 1024 and 2048 thinking tokens
- Read the curve: fast gains, diminishing returns, and the knee at 512
- The honest negative: past the knee, accuracy falls - and it falls on the easy alerts the snap already had right
- Route the budget per alert: fit a policy on one half, score it on the other
- Read the policy honestly: routing saved tokens, not accuracy, and the better-looking signal routed worse
- Why this, not that - and when NOT to raise the budget
- Ship it: the policy, what to log, and one experiment that proves where the decline came from