Lion Cipher Lab
Write a network detection from a CVE advisory with an LLM — then prove it against a real packet
- AI Security
- Intermediate
- ~70 min
- Free · no account required
Why it matters
A CVE advisory is written for a human: a paragraph of prose explaining what is broken, why it is dangerous, and which versions are affected. A detection is the opposite — a precise, unforgiving machine artifact that either matches the exact bytes on the wire or it does not. The whole job of detection engineering lives in the gap between those two things, and most of that gap is judgment: the majority of any advisory is simply not visible on the network, and the one part that is must be pinned down to a byte pattern the engine can test. This is exactly where a language model is both useful and…
What you'll learn
- reading a CVE advisory and judging what part of it is actually detectable on the network
- using a local LLM (Ollama) to draft a Suricata rule from an advisory
- validating that a rule loads with suricata -T -S before trusting it
- crafting a benign pcap that carries a signature's byte pattern — no exploit, no target
- replaying a pcap through Suricata offline with suricata -r and reading the alert in eve.json
- stress-testing a rule for false positives against benign traffic
- fixing what an LLM got wrong: invented keywords, missing nocase, wrong sticky buffer, an over-broad content match
- judging why string IOCs are brittle (Pyramid of Pain) and when to prefer behaviour-based detection
Steps in this lab
- First: nothing here is attacked
- The gap: an advisory is prose, a detection is a machine artifact
- Install the two tools: Suricata and Ollama
- How a Suricata rule is built — just enough to judge one
- Let the model draft a rule from the advisory
- Validate (1): does the rule even load? — suricata -T -S
- Craft a benign pcap that carries the exact pattern
- Validate (2): replay the pcap and read eve.json
- Stress it: does it false-positive on benign traffic?
- Measure the model — the three numbers that matter
- Why even the fixed rule is brittle — the Pyramid of Pain
- What you learned, and the judgment to carry out of here