Lion Cipher Lab
Audit your security eval's answer extraction and measure how much score the parser invents (deterministic stub models)
- AI Security
- Intermediate
- ~80 min
- Free · no account required
Why it matters
Security teams choose models from multiple-choice evals: which one maps behaviour to ATT&CK tactics best, which one picks the right CWE. Those scores come from a model writing free text and a few lines of code pulling 'the answer letter' out of it. When that code is naive it can give a model credit for a hedge, read the 'A)' at the top of its reasoning as its answer, or score a correct model zero because it wrapped the letter in bold. Hugging Face found the same MMLU benchmark scored LLaMA 65B at 0.637 or 0.488 depending only on how the answer was read and scored. If you never measure your…
What you'll learn
- building a 300-item answer-keyed ATT&CK-tactic multiple-choice eval whose keys are exact because every question was written from its tactic
- giving every model reply a ground-truth label for what the model MEANT, so you can grade the parser separately from the model
- measuring extraction accuracy for four parsers - first letter, last letter, the simple-evals 'Answer: X' regex, and a careful rule-based parser - and showing five different leaderboard orders from the same 1,500 replies
- counting invented credit (the parser gives a point the model did not earn) and hidden credit (the parser takes away a point it did earn) instead of trusting one accuracy number
- finding the parser's own failure mode: 39 negation-trap replies the careful parser read as a confident wrong letter
- shipping the fix: a format instruction in the prompt, an anchored strict parser, the unparseable rate as its own column, and a seeded random audit sample
- deciding whether unparseable counts as wrong or is excluded, and saying which, because the choice moved one model from last to second
Steps in this lab
- The words first: eval, answer key, answer extraction, parser, unparseable - and the fact that the models here are STUBS
- Open a terminal, check Python, and make a scratch folder (Windows, macOS, Linux)
- Build 300 answer-keyed ATT&CK-tactic items
- Meet the five STUB models and the five response styles they write
- Write four parsers, from naive to careful
- Score 5 models x 300 items with every parser, and watch five leaderboards appear
- Read the two failure modes: the parser that invents score and the parser that hides it
- Honest negative one: the careful parser still misreads, and it does it silently
- The fix, part one: constrain the format in the prompt, then parse it strictly
- The fix, part two: an unparseable column, a stated policy, and a random audit
- Honest negatives two and three: the format instruction changes the model, and the unparseable policy changes the ranking
- Why this, not that - and when NOT to bother auditing the extractor
- The way a pro does it: log raw replies, version the parser, and report extraction as a first-class number
- Optional appendix: the same four parsers on a free local model with Ollama
- Ship it: the reporting checklist, one experiment, and clean-up