Lion Cipher Lab
Run a security agent fully offline with Ollama — and measure when a small local model is actually trustworthy
- AI Security
- Intermediate
- ~60 min
- Free · no account required
Why it matters
"Just run a local model" is easy to say and easy to get wrong. Small models that fit on a laptop can absolutely do agentic work — loop, pick a tool, read the result, decide the next step — but they are markedly less reliable at it than a frontier model, and the failures are specific and repeatable: they emit JSON wrapped in prose, they invent tool names that don't exist, and they loop without ever deciding they're done. If you are doing incident response on an air-gapped network, or you simply cannot let sensitive data leave the building, running fully offline is not a compromise — it is the…
What you'll learn
- building a minimal ReAct loop (think, call a tool, observe, repeat) against a local model
- running an agent fully offline with Ollama — no internet, no API key
- instrumenting the real failure modes of small local tool-calling models
- measuring a baseline agentic failure rate instead of assuming reliability
- hardening with structured output, a tool allowlist + argument validator, and a step budget
- re-measuring to prove a guardrail actually helped
- judging when offline + local beats a frontier API, and when it is the wrong call
Steps in this lab
- The reliability gap — and why offline is sometimes the RIGHT call
- Open a terminal and install Ollama and a small model
- Build the offline world: synthetic data and three read-only tools
- Two ways to ask the model: naive free text vs constrained JSON
- The instrumented ReAct loop — count every failure mode
- Run it, read the BEFORE/AFTER, and make the judgment
- Make it better — and know when a small local model is the wrong tool