Lion Cipher Lab

Auto-score the faithfulness of an LLM's security summary with an NLI model — and gate the hallucinations

• AI Security
• Advanced
• ~90 min
• Free · no account required

Why it matters

Analysts are already pasting incident logs, threat reports, and packet captures into an LLM and asking for a summary, because reading the raw artifact is slow and the summary is fast and fluent. The danger is that fluency and truth are unrelated: a large language model (LLM) is a system trained to produce text that READS well, and it will state a claim the source never supports — the wrong host as patient zero, an exfiltration that never happened, a CVE that was never mentioned — with exactly the same confident tone as the claims that are correct. That is a HALLUCINATION, and in security work…

What you'll learn

• the difference between an LLM summary that is FLUENT and one that is FAITHFUL — and why fluency tells you nothing about whether a claim is grounded in the source
• running a free, fully local NLI (Natural Language Inference) model with Hugging Face transformers — no API key, no GPU — to score a claim as entailment / neutral / contradiction against a source
• decomposing a summary into ATOMIC claims and scoring each one, instead of scoring a whole paragraph as if it were a single fact
• the RETRIEVAL move that makes NLI work on documents: score each claim against the best-matching source SENTENCE (max over sentences), not against the whole blob — and measuring how much that matters
• building a fail-closed HALLUCINATION GATE: a summary is blocked if any claim falls below an entailment threshold or is contradicted by the source
• MEASURING the gate against hand-labeled ground truth — precision and recall at catching unfaithful claims — and sweeping the threshold to see the precision/recall tradeoff instead of guessing a number
• the named failure modes of NLI-based faithfulness: wrong premise (retrieval), paraphrase brittleness, weakness on numeric/temporal claims, threshold too strict (blocks good summaries) vs too loose (lets hallucinations through), and true-but-not-in-source
• the judgment: WHY an automatic gate is the only scalable option, and its hard limit — the gate is itself a model, so it is a check and not a proof, and an AI summary is a triage draft, never evidence

Steps in this lab

1. The problem: an LLM summary can be fluent and false at the same time
2. Set up a free, local NLI environment and the synthetic corpus
3. Load the NLI model and score one claim — and never hardcode the label order
4. Retrieval: score each claim against the RIGHT source line, not the whole blob
5. Build the fail-closed hallucination gate
6. MEASURE it: precision, recall, and the threshold tradeoff
7. The named failure modes — where an NLI faithfulness gate goes wrong
8. Where the gate fits — and where it must stop
9. The way a pro does it — and what you actually built

Open the full interactive lab →