Lion Cipher Lab

Build a STIX 2.1 Patterning engine from scratch (tokenizer, parser, evaluator)

• Threat Intel Analyst
• Advanced
• ~120 min
• Free · no account required

Why it matters

Threat intelligence only becomes useful when a machine can act on it, and STIX 2.1 is the standard language the industry agreed on for saying what to look for. Most people meet STIX through its objects — an indicator, a malware, a relationship — and stop there, treating the 'pattern' field as a magic string they paste in from a feed. That is where the money is lost. The pattern field is not a string; it is a small, precise query language with its own grammar, its own operators and its own time semantics, and the single most expensive misunderstanding in all of cyber threat intelligence…

What you'll learn

• reading the STIX 2.1 Patterning grammar as a grammar: comparison expressions inside brackets, observation expressions made of brackets
• the single costliest mistake in the language: [a AND b] means ONE observation has both, [a] AND [b] means two separate observations
• writing a tokenizer with one master regex, and a recursive-descent parser whose function nesting IS the operator precedence
• STIX object paths: dotted properties, quoted keys like file:hashes.'SHA-256', list indexes like values[0], and automatic _ref / _refs dereferencing
• implementing the comparison operators =, !=, <, >, IN, LIKE, MATCHES and ISSUBSET -- and deciding out loud which ones to leave out and why
• the binding model that makes AND inside one observation mean 'the same object', not 'somewhere in this observation'
• observation operators AND / OR / FOLLOWEDBY and the qualifiers WITHIN n SECONDS, REPEATS n TIMES and START ... STOP
• measuring a flat IOC list against a pattern engine on the same bundle and reading real true-positive / false-positive / false-negative counts
• four named failure modes: hash-key case sensitivity, MATCHES as an untrusted-regex (ReDoS) surface, inclusive-START/exclusive-STOP timestamp handling, and pattern-vs-observation scoping
• judgment: where a pattern genuinely beats an IOC list, where it does not (Pyramid of Pain), and why you should not ship a hand-rolled matcher

Steps in this lab

1. What this lab is — and what it deliberately is not
2. Open a terminal and check Python (zero-assumption setup)
3. The grammar on one page: comparison expressions vs observation expressions
4. Create the bundle: nine observations, three of them traps
5. Part 1 of 3 — the tokenizer and the parser (stixlex.py)
6. Part 2 of 3 — the evaluator and the binding model (stixeval.py)
7. Part 3 of 3 — the driver, and the flat IOC list we are competing against
8. Run it
9. Section 1, in detail: the two-bracket mistake, measured
10. Section 3, in detail: WITHIN collapses a false chain from 3 to 1
11. The measurement: flat IOC list versus pattern engine, same bundle
12. Four named failure modes (and the one that already bit you)
13. When a pattern beats a flat IOC list — and when it absolutely does not
14. How a pro does it differently: do not ship this

Open the full interactive lab →