Lion Cipher Lab
Ship tamper-evident logs off-box over mutual-TLS syslog, then prove a deletion is detectable
- Cyber / Security Engineer
- Advanced
- ~90 min
- Free · no account required
Why it matters
Almost every logging tutorial ends at 'the logs are arriving at the collector,' and treats that as the finish line. It is not even half the problem. Shipping logs off-box answers one threat -- an attacker who deletes the LOCAL copy on the machine they compromised -- but it silently assumes two things that are not free: that nobody can read or forge the records on the wire, and that nobody can quietly rewrite history ON THE COLLECTOR once the records land. Real incidents turn on exactly those assumptions. An attacker who reaches the log store, or an insider with access to it, can delete the…
What you'll learn
- Standing up a two-component secure-telemetry pipeline -- a syslog sender and a syslog collector -- entirely on your own machine, and reasoning about it as a pipeline rather than a single tool
- Minting a private CA with openssl and issuing a SERVER certificate (the collector) and a CLIENT certificate (the sender) from it, so each side can prove who it is
- Configuring MUTUAL TLS in Python's ssl module (load_cert_chain to present your cert, load_verify_locations + verify_mode=CERT_REQUIRED to demand and check the peer's) -- the RFC 5425 syslog-over-TLS transport, where BOTH endpoints authenticate
- Framing syslog per RFC 5425's octet-counting method and formatting messages per RFC 5424, including a custom STRUCTURED-DATA element (SD-ID + name="value" params) to carry integrity metadata inline
- Building a per-record hash chain h_i = SHA-256(h_(i-1) || record_i) anchored at a genesis value -- the RFC 5848 idea that each record's hash covers the one before it, so a break localizes tampering
- Writing a verifier that recomputes the chain from the stored records and pinpoints the FIRST record where integrity fails -- distinguishing an edit (hash mismatch) from an interior deletion (broken prev-link + sequence gap)
- Proving tail-truncation is only catchable with an external ANCHOR (record the final chain head out-of-band, or periodically sign it), and stating honestly what a hash chain does NOT give you -- existence, availability, or immutability of the store itself
- Naming where the real-world versions live (rsyslog/syslog-ng RFC 5425 TLS transport, systemd journald's Forward Secure Sealing, transparency-log-style anchoring) so the toy maps onto production controls
Steps in this lab
- Read this first: your own machine, your own files, authorized-only
- The threat model: two different problems, two different mechanisms
- Build your own CA, then issue a collector (server) cert and a sender (client) cert
- Design the record: an RFC 5424 message that carries its own integrity metadata
- The hash chain, exactly: what it catches, what it doesn't, and why
- Build the collector: a mutual-TLS syslog server that appends what it receives
- Build the sender: a mutual-TLS syslog client that chains as it ships
- Run it end to end -- and prove the mutual part by getting rejected without a client cert
- Build the verifier: recompute the chain and pinpoint the first break
- The attack: edit, delete, and truncate -- and watch the verifier name the spot
- When it fails, and how to do it better -- the honest limits and the production versions