Lion Cipher Lab

Catch hallucinated package names in LLM-generated code before you pip install them (slopsquatting defence, stub assistant)

• AI Security
• Advanced
• ~90 min
• Free · no account required

Why it matters

Coding assistants now write a lot of the glue code in security teams - the script that looks up a hash, parses a pcap, scans a folder with YARA - and every one of those answers ends with an install line. Sometimes the package on that line does not exist. The model invented it, because the name sounds like what a package for that job would be called. Spracklen and colleagues measured this across 16 code-generating models and 576,000 samples: at least 5.2% of packages from commercial models and 21.7% from open-source ones were hallucinated, over 205,000 unique invented names - and, crucially,…

What you'll learn

• building a pre-install gate from the Python standard library alone - ast to read imports without running the code, sys.stdlib_module_names to drop the standard library, an import-to-distribution map, and the packaging name-normalisation rule - that turns an assistant's answer into the exact set of packages it would make you install
• failing CLOSED: the 2 of 200 stub answers that were cut off mid-line do not parse, and the gate blocks them instead of silently skipping what it cannot read
• measuring package hallucination instead of guessing at it: 44 of 198 parseable answers (22.2%) named a package the snapshot had never heard of, from 29 unique invented names
• measuring the property that makes a hallucination squattable - repetition: 9 of the 29 invented names recur, and those 9 carry 55% of all hits, which is the shopping list an attacker would register first
• seeing why the import name is not the package name: guessing one from the other would have wrongly flagged 149 real imports across 14 names (yaml, bs4, sklearn, Crypto and friends)
• keeping the honest negative: once six repeated names are squatted in a local copy of the snapshot, an existence-only check waves 18 of 44 hallucinated answers through, and its hallucination metric falls from 22.2% to 13.1% - it reports improvement exactly as the attack lands
• replacing existence with an allowlist-first policy plus age and release-history heuristics that route to a human, and pricing it: 16 of 18 squatted answers caught, the patient squat (150 days, 4 releases) missed, 8 clean answers held for review
• judging when lockfile mode (18 of 18 caught, 20 clean answers held, 8 packages to review), hash pinning with pip --require-hashes and a private mirror beat any heuristic - and when a gate like this is theatre

Steps in this lab

1. The words first: hallucinated packages, slopsquatting, and the fact that the assistant here is a stub
2. Open a terminal and make a scratch folder (Windows, macOS, Linux)
3. Build the world: 200 answers from a stub assistant, and a synthetic registry snapshot
4. The assistant is a STUB and the registry is SYNTHETIC - here is exactly what that does and does not mean
5. Write the gate: read imports with ast, drop the standard library, map import names to distributions, fail closed
6. Gate v1, existence only: measure the hallucination rate and how much of it repeats
7. The honest negative: squat the repeated names, and watch the existence check wave them through
8. Gate v2: allowlist first, then age and release history, then a human - and measure both modes
9. OPTIONAL, online: read one real project's metadata from the PyPI JSON API (no install)
10. Pin what you approved: hash-checking mode, so the approved name cannot be swapped underneath you
11. Why this, not that - and when the gate is theatre
12. Ship it: the policy, what to log, and clean up

Open the full interactive lab →