Lion Cipher Lab
Triage and prioritize a vulnerability scan with an LLM (then verify every CVE it cites)
- AI Security
- Intermediate
- ~65 min
- Free · no account required
Why it matters
A single vulnerability scan of one slightly-old container image routinely returns dozens to hundreds of CVEs. No human reads them all, and the reflex everyone falls back on — 'fix the CRITICALs first' — is wrong more often than it is right, because severity alone ignores context: is the vulnerable package actually reachable, is there even a fix to move to, and is anyone exploiting it in the real world? A language model is genuinely good at the boring part of that problem: it can cluster hundreds of findings, group them by package, and draft a ranked shortlist with a one-line rationale each in…
What you'll learn
- generating a real vulnerability scan of a container image as machine-readable JSON with Trivy
- shrinking a big scan to a compact, structured extract instead of dumping raw JSON at a model
- using a local LLM (via Ollama) to cluster, summarize, and PRIORITIZE a flood of CVEs
- verifying every CVE the model cites against the raw Trivy JSON (does it exist? what fix did Trivy actually report?)
- checking a real CVSS score against NVD instead of trusting the model's number
- checking 'exploited in the wild' against the CISA Known Exploited Vulnerabilities (KEV) catalog
- checking real exploit-likelihood against FIRST EPSS via its public API
- catching a confident hallucination — an invented CVE, a wrong CVSS, or a made-up 'actively exploited' claim
- measuring an LLM's precision / hit-rate over its claims instead of trusting the ranking
- judging when an LLM accelerates triage and when it must never be the source of truth (OWASP LLM09)
Steps in this lab
- The problem: a scan floods you, and 'fix the CRITICALs first' is not a plan
- Install Docker, Trivy, and a small local model
- Generate a real scan of an old image as JSON
- Shrink the scan to a compact extract — never dump raw JSON at the model
- Ask the local model to cluster and prioritize (the draft)
- Verify spine, part 1: check every CVE against the raw scan JSON
- Verify spine, part 2: CVSS from NVD, 'exploited' from CISA KEV, likelihood from EPSS
- Measure the hit-rate — turn 'seems good' into a number
- The judgment: accelerator, never oracle — and KEV/EPSS/CVSS are context, not a verdict