Lion Cipher Lab

Catch the silent failures in your detection pipeline (a telemetry-integrity monitor from scratch)

• SOC Analyst / Blue Team
• Advanced
• ~120 min
• Free · no account required

Why it matters

Every detection you own has an unstated dependency, and it is not written down anywhere: the data has to arrive. It has to arrive complete, with the field names your rule reads, with timestamps that agree with everyone else's, uncut, once each, and before the search that looks for it runs. Nothing in your SIEM alerts you when that dependency breaks. This is the difference between a detection failing and a detection going quiet, and it is the single most dangerous asymmetry in the job: a rule that matches zero events looks exactly like a quiet network. A broken rule at least throws a syntax…

What you'll learn

• the unstated dependency every detection has: data arriving, complete, on time, once
• why a rule that matches zero events is indistinguishable from a quiet network
• event time vs ingest time, and why a scheduled search that was correct at run time can be wrong forever
• nine named telemetry failure modes -- source silence, partial silence, field drift, type drift, clock skew, truncation, ingest lag, drops, duplicates, parse failures -- each measured
• why defensive coding in a query engine (coerce, never crash) is exactly what makes data failures silent
• seasonality: why a fixed volume threshold and a plain mean +/- 3 sigma both fail on a day/night/weekend cycle, in opposite directions
• MAD (median absolute deviation) and a same-hour/same-weekday baseline, and counting the false alarms before and after
• per-source silence budgets, and why a chatty firewall and a nightly batch job cannot share one threshold
• deriving a rule's field dependencies FROM the rule, so the data contract cannot drift out of sync with the detections
• clock-skew detection by comparing event time to ingest time, truncation detection by watching for length pile-ups at round numbers, duplicate detection with a rolling event-hash window and its named memory cost

Steps in this lab

1. The thesis, and which labs this is NOT
2. The words you need, defined once
3. Open a terminal and check Python (zero-assumption setup)
4. Build the estate: nine log sources, thirty days, one seed
5. Four detections that work — the honest baseline
6. Read the baseline: 9 planted activities, 9 findings, and a contract you did not type
7. Breakage 1 and 2: total silence, and the crueller partial kind
8. Breakage 3: field drift and type drift, with zero errors raised
9. Breakage 4: a clock 47 minutes fast forges the timeline
10. Breakage 5 and 6: truncation at 1024, and data that arrives after the search
11. Breakage 7, 8 and 9: drops, duplicates, and a dead-letter queue nobody reads
12. The headline: rules unchanged, intrusions present, findings 9 → 2
13. Build the monitor — eleven detectors and a health score
14. The sharpest measurement in the lab: 3,140 false alarms, then 0
15. Silence budgets: why one threshold cannot serve nine sources
16. The scorecard: 9 of 9 caught, 0 false alarms, one detector each
17. The rule-health table — 'is this rule capable of firing right now?'
18. Monitor failure mode (i): who watches the watcher?
19. Monitor failure mode (ii): the monitor becomes the new alert fatigue
20. Monitor failure mode (iii): a baseline learned during an outage
21. Monitor failure mode (iv): silence as an adversary goal (T1685.001, T1685.005)
22. Why this, not that — and when to build it versus turn it on
23. When NOT to build this
24. How a pro does it differently

Open the full interactive lab →