Lion Cipher Lab

Test whether your model's reasoning is faithful or post-hoc: perturb the chain-of-thought on a security-classification task and measure how often the verdict follows

• AI Security
• Advanced
• ~100 min
• Free · no account required

Why it matters

A reasoning model that triages an alert hands you two things: a verdict, and a paragraph of chain-of-thought explaining it. The paragraph is what gets pasted into the ticket, quoted in the post-incident review, and trusted six months later when someone asks why the alert was closed. The uncomfortable result from the faithfulness literature is that the paragraph is often not the reason for the verdict at all - the answer is largely decided before the explanation is written, and the explanation is a plausible story assembled afterwards. Turpin and colleagues showed that you can bias a model's…

What you'll learn

• distinguishing the CAUSAL faithfulness of a chain-of-thought (does the stated reasoning drive the answer) from the summary-faithfulness of one text to another - two different questions people give the same word
• running Turpin's BIASING-CUE probe: plant a hint the input never contained ('a senior analyst thinks this is benign'), measure how often the verdict flips toward the cue, and confirm the stated reasoning never names the cue - here a 16.83% silent post-hoc rationalization rate on 600 alerts
• running Lanham's CORRUPTED- and TRUNCATED-CoT probe (early answering + adding mistakes): break the stated reasoning and count how many verdicts change - here 6.33% under an injected mistake and 0 under a deleted CoT, which is what non-causal reasoning looks like
• building a deterministic stub whose reasoning-vs-answer coupling is one readable constant, so a faithfulness result is reproducible byte-for-byte and every absolute number is a property of the model you can see rather than a black box you cannot
• the methodological control that stops a faithfulness metric from being theatre: run the same two probes on a KNOWN faithful model and confirm they read the opposite (0.0% silent rationalization, 57.83% answer-change under corruption), so a flag means something
• a usable rule for when a model's write-up is worth trusting as evidence in an incident record, and when a fluent rationale is worth nothing at all

Steps in this lab

1. The words first: chain-of-thought, faithful vs post-hoc, the biasing cue, corrupted reasoning - and the fact that the model here is a stub
2. Open a terminal and make a scratch folder (all three operating systems)
3. The corpus: 600 alerts whose correct benign/malicious label you already know
4. Write the stub: a classifier that prints a chain-of-thought whose coupling to its answer you control
5. The baseline: clean-run accuracy, the number every probe is measured against
6. Probe one: the biasing cue - how often the verdict follows a hint the reasoning never mentions
7. Probe two: corrupt and truncate the chain-of-thought - if the answer barely moves, the reasoning was not causal
8. The control: prove the probes discriminate, then read the rule off the table
9. When to trust an AI analyst's write-up, when a plausible rationale is worthless, and how a pro wires this in

Open the full interactive lab →