Lion Cipher Lab

Trace one attack across three services with correlation IDs: measure the timestamp join, then make it exact with W3C traceparent

• Cyber / Security Engineer
• Intermediate
• ~75 min
• Free · no account required

Why it matters

When an incident crosses more than one service, the investigator's first job is to stitch together the pieces of one request: the gateway saw the login, auth approved it, orders exported the data, a background worker shipped it. If the logs share nothing but a timestamp and a username, that stitching is guesswork, and guesswork in an incident means either blaming a real customer for the attacker's export or missing the export entirely. This lab makes you measure that guesswork instead of assuming it: you reconstruct one attacker's chains from plain logs, grade them against an answer key, and…

What you'll learn

• Building three cooperating HTTP services (gateway, auth, orders) plus an async queue worker on 127.0.0.1 with only Python's standard library, so you own every log line they write
• Measuring, with a graded answer key instead of eyeballing, how well an investigator can rebuild one attacker's request chains from plain-text logs joined on user and timestamp, and seeing why no time window gets both precision and recall
• Generating a W3C Trace Context traceparent (version-traceid-parentid-flags) at the edge with the secrets module, propagating it hop to hop with a fresh span id per service, and writing structured JSON logs that carry trace_id, span_id and parent_span_id
• Rebuilding the same attack chains by joining on trace_id and proving the result is exact (precision 1.00, recall 1.00, every chain whole)
• Proving that context does NOT survive an async queue hop unless you put it in the message, by measuring the orphaned lines and the recall you lose
• Showing why the public edge must re-root untrusted trace context: measuring the collision when a client-supplied traceparent is continued, validating the W3C format, and keeping the client's claim as data rather than as the join key
• Reproducing log forging (CWE-117) with an obs-folded CRLF in a request-ID header through a real http.server, and comparing a text logger, a JSON logger and a validating edge
• Knowing when NOT to hand-roll this: what OpenTelemetry gives you, and what sampling takes away from an investigation

Steps in this lab

1. Read this first: the scenario, and why it all stays on your machine
2. The problem: one request, four log files, and nothing they share
3. Set up: a terminal, Python, and a folder
4. Build the three services and the traffic driver (stack.py)
5. BEFORE: run it with plain-text logs and look at what an investigator gets
6. Build the investigator and grade the timestamp join (investigate.py)
7. AFTER: mint a traceparent at the edge, log JSON, and join on trace_id
8. Why it worked: born at the edge, random, and carried on every hop
9. Break it 1: the async hop that drops context
10. Break it 2: an edge that trusts the client's traceparent
11. Break it 3: a request ID that forges a log line (CWE-117)
12. When NOT to hand-roll this: OpenTelemetry, and what sampling costs an investigation
13. What you proved, and the checklist to take back

Open the full interactive lab →