Lion Cipher Lab

Tier an agent's approval requests by risk and measure approval fatigue (Part 2 of the approval gate)

• AI Security
• Advanced
• ~90 min
• Free · no account required

Why it matters

The first lab in this series built the obvious safety feature for an AI agent: before it does anything risky, it stops and asks a human. That works on day one. Then the agent goes into a security operations centre, proposes hundreds of actions a shift, and the gate fires every few seconds. The human starts clicking approve without reading. At that point the gate is still there, still logging approvals, and no longer protecting anything - and nobody can tell from the logs, because a rubber stamp and a careful review look the same. Attackers already know this pattern from a different place: MFA…

What you'll learn

• building a keyed shift of 600 agent proposals - reads, low-risk writes and destructive actions - with 6 planted wrong destructive actions and a planted 30-read bulk-read burst, so every approval policy can be graded
• writing a plainly-labelled stub reviewer whose fatigue model (rubber-stamp probability rising with prompts per hour, a catch rate that depends on whether the prompt shows a blast-radius diff) is a written-down assumption you can change
• measuring an approval policy on four axes at once - prompts per hour and peak hour, reviewer minutes, wrong destructive actions approved, and time-to-act on correct urgent containment - instead of asking whether a gate exists
• showing with numbers why an approve-everything gate approved 5.67 of 6 wrong destructive actions, and how risk tiering plus a diff cut that to 2.02 and a two-person rule to 0.55
• classifying an agent's tools by declared effect rather than by name, after measuring that name-based tiering auto-allowed 20 of 200 destructive actions
• pricing the controls honestly: a read-volume cap that prompts recreates push fatigue, a cap that blocks holds legitimate reads, and a two-person rule turned a 1.5-minute containment into 13.0 minutes
• connecting agent approval fatigue to its real-world twin, MFA push fatigue (MITRE ATT&CK T1621), and to separation of duties (NIST SP 800-53 AC-5)

Steps in this lab

1. The words first: agent, approval gate, risk tier, blast radius, rubber stamp, approval fatigue, two-person rule - and the fact that everything here is a stub
2. Open a terminal, check Python, make a scratch folder (Windows, macOS, Linux)
3. One-step recap: the single approval gate from Part 1, and why the human is now the weak link
4. Build the shift: 600 proposals, 6 planted wrong destructive actions, one planted bulk-read burst
5. Write the stub reviewer: a fatigue model you can read, change and argue with
6. The reviewer model is an assumption, and here is exactly what it assumes
7. Run three policies on the same shift: approve-everything, risk-tiered with a diff, and a two-person rule
8. Read it honestly: more prompts bought less safety, and the cheapest-looking policy was the most dangerous
9. Three honest negatives: tiering by name, auto-allowed bulk reads, and the price of two people
10. The real-world twin: MFA push fatigue, and what its defences teach an agent's approval design
11. Why this, not that - and when NOT to tier
12. The way a pro does it
13. Ship it: the policy, what to log, and how to see what the stub is doing

Open the full interactive lab →