Lion Cipher Lab
Scan an LLM for vulnerabilities with garak — the automated LLM vulnerability scanner
- AI Security
- Intermediate
- ~60 min
- Free · no account required
Why it matters
There is a version of LLM security testing that does not scale: sitting at a prompt, typing one clever attack at a time. It teaches you the mechanisms, and you should do it — but you will never hand-test the hundreds of known jailbreak variants, encoding tricks, and data-leakage replays that already exist in public catalogs. That is what an automated LLM vulnerability scanner is for, and garak — NVIDIA's open-source Generative AI Red-teaming & Assessment Kit — is the one built for exactly this. Think of it as nmap for language models: it fires whole families of probes (prompt injection,…
What you'll learn
- running an automated LLM vulnerability scanner (garak)
- the probe / detector model: how garak attacks and scores
- pointing a scanner at a free, local model via Ollama
- reading a garak hit-rate report and the hitlog / HTML report
- interpreting which probe families landed and what they mean
- mapping scanner findings to the OWASP Top 10 for LLM Applications
- judging when a scanner is the right tool — and its false-positive / false-negative limits
- treating an automated result as a lead to confirm, not a verdict
- authorization and scope for running an attack tool
Steps in this lab
- What garak is, and the one rule before you run it
- How garak works: probes attack, detectors score
- Stand up a free, local target with Ollama
- Install garak and browse the probe catalog
- Run a fast, focused scan against your local model
- Read the report: hit rate, hitlog, and the HTML summary
- Map what landed to the OWASP Top 10 for LLM Applications
- The judgment: a scanner is a first pass, not a verdict
- What you just added to your toolkit