Lion Cipher Lab
Property-fuzz your own API with Schemathesis (find the 500s your example tests miss)
- AppSec / DevSecOps
- Advanced
- ~70 min
- Free · no account required
Why it matters
Almost every team tests its API the same way: an engineer writes a handful of EXAMPLE tests -- 'send this user id, expect this user back' -- using the inputs they thought of. The problem is that attackers and reality do not stay inside the inputs you thought of. They send a zero where you expected a positive number, a 40-kilobyte string where you expected a name, a negative id, a Unicode surrogate, an empty body. Your example tests are green and your API still 500s in production on the first weird request, because a server error (HTTP 500) is not just an availability bug -- a stack trace can…
What you'll learn
- what an API and an OpenAPI schema are, and why a machine-readable schema is a test-generation goldmine
- the difference between example-based testing (you write the inputs) and property-based testing (the tool derives them)
- standing up a tiny FastAPI app you OWN with an auto-generated OpenAPI schema and a deliberately planted bug
- running Schemathesis against a local schema and reading its four built-in checks (not_a_server_error, status_code_conformance, response_schema_conformance, content_type_conformance)
- reading a MINIMAL failing reproducer that Hypothesis shrinks for you, then fixing the unhandled edge case that returned HTTP 500
- catching a response that violates its OWN declared schema (a contract lie your example tests happily passed)
- adding STATEFUL testing that follows OpenAPI links so create-then-read sequences get exercised
- pinning a seed for deterministic, reproducible CI runs and gating a GitHub Actions job on Schemathesis's exit code
- the judgment of the ORACLE problem: naming exactly what a schema fuzzer can and CANNOT find, and where it sits vs SAST and a manual pentest
Steps in this lab
- The idea: example tests vs a fuzzer that reads your schema
- Install the tools (one-time, from PyPI)
- Build a tiny API you OWN -- with two bugs planted on purpose
- Point Schemathesis at your schema and watch it break your API
- Read the MINIMAL reproducer, then fix the crash
- Re-run green -- but only after documenting the 400
- Go stateful: test sequences, not just single calls
- Gate CI: make the fuzzer fail the build, deterministically
- The judgment: what a schema fuzzer can and CANNOT find