Lion Cipher Lab
Catch multi-turn escalation that a per-message guardrail misses, with a conversation-level monitor
- AI Security
- Advanced
- ~75 min
- Free · no account required
Why it matters
Most chatbot guardrails judge one message at a time: is THIS request allowed? Russinovich, Salem and Eldan showed in 2024 that this is not enough. Their Crescendo attack opens with a general question and escalates gradually, each new turn referring to the model's own previous answer, until it reaches something a direct request would never get. Microsoft's own write-up says the fix it shipped looks at the whole prior conversation, not just the latest message. If you run an internal assistant that knows things outsiders must not see, this is the gap you need to measure in your own deployment,…
What you'll learn
- generating hundreds of labelled synthetic conversations deterministically (benign short chats, long topic-drifting analyst chats, direct asks and gradual escalations) so a guardrail can be scored like any detector
- measuring a per-message guardrail and seeing exactly why it scores 0 out of 120 on escalations whose every turn is mild
- building a conversation-level monitor from a decayed running risk score, phrases that point back at earlier answers, topic words and an echo of the assistant's own words
- reporting turn-of-detection against turn-of-leak, so 'detected' means 'blocked before the secret went out', not 'noticed afterwards'
- using an output-side canary check as the backstop, and measuring what it misses: hints sent earlier, digits spelled out as words, and a secret split across two replies
- measuring a monitor's cost (false alarms on long legitimate chats across thresholds) and the session-split evasion, and the per-user fix with its own trade-off
Steps in this lab
- The words first: guardrail, canary, multi-turn escalation, per-message vs conversation-level - and the fact that the assistant is a stub
- Open a terminal, check Python, make a scratch folder (Windows, macOS, Linux)
- One-step recap: how this differs from the many-shot lab and the canary-leak lab
- Generate 480 labelled conversations: benign short, benign long, direct asks, and three kinds of escalation
- Write the assistant stub: no refusal of its own, and eager to build on what it already said
- Write the three guards: a per-message score, a conversation monitor with decayed risk, and an output-side canary check
- Write the simulator: play every conversation and record leak turn, partial-leak turn and fire turn
- Measure BEFORE: the per-message guardrail stops direct asks and 0 of 120 escalations
- Measure AFTER: the conversation monitor, turn of detection vs turn of leak, and the output canary backstop
- Honest negatives: false alarms on long benign chats, and an attacker who opens a new chat for every step
- Why this, not that - and when NOT to put a conversation monitor in front of an assistant
- The way a pro does it: take the secret out of reach first, then monitor per user, test with real multi-turn tooling, and log every score
- Ship it: re-run, change one constant on purpose, and clean up