Lion Cipher Lab
Triage a phishing email with a local LLM — then verify every word it says
- AI Security
- Intermediate
- ~40 min
- Free · no account required
Why it matters
Analysts are being handed AI assistants to speed up triage, and the tempting failure is to let the model's verdict BE the answer. A language model is a confident guesser: it is genuinely useful for a fast first read and a plain-language summary, and it will also invent an IOC, misread a header, or get fooled by text hidden in the email body — all while sounding certain. The skill that separates a professional from a liability is trust calibration: use the model to go fast, then verify every load-bearing claim against deterministic ground truth before it reaches a ticket. This lab teaches that…
What you'll learn
- running a local LLM offline with Ollama (no data leaves your machine)
- prompting a model for first-pass phishing triage
- extracting indicators (IOCs) and mapping to MITRE ATT&CK T1566
- reading real email headers to establish ground truth
- verifying LLM claims and catching a confident hallucination
- measuring a model's hit/accuracy rate on a task
- trust calibration: when to trust an LLM and when not to
- recognizing prompt injection (OWASP LLM01) in an untrusted email body
Steps in this lab
- The words you need, in plain terms
- Why local, and why we never paste a live sample into a hosted AI
- Confirm your local model is actually running
- Create a synthetic, benign phishing sample
- Ask the local model for a first-pass triage
- Establish ground truth: read the real headers by hand
- Grade the model: score every claim and catch the hallucination
- The body is untrusted input — watch for prompt injection
- Calibrate: when to trust the model, and when not