Lion Cipher Lab
Serve a local LLM with FastAPI
- AI Security
- Intermediate
- ~60 min
- Free · no account required
Why it matters
A model you can only talk to from one terminal is a toy; a model behind an API is infrastructure other software can build on. Every real AI feature -- an app, an agent, an internal tool -- reaches its model over an inference API, and standing one up yourself is the moment 'AI engineering' stops being a buzzword and becomes something you can do. Doing it locally keeps the model and every prompt on your hardware, a genuine privacy and cost win over renting a hosted endpoint. And it teaches the security lesson that no tutorial-grade wrapper mentions: the instant you expose a /generate endpoint…
What you'll learn
- wrapping a local LLM in an HTTP inference API
- designing request/response schemas with Pydantic
- input validation and a max-tokens / rate guard
- running an ASGI app with uvicorn
- testing an endpoint with curl
- spotting production concerns: auth, concurrency, timeouts, prompt injection at the API boundary
Steps in this lab
- Why a running model needs an API in front of it
- Get a local model reachable from Python
- Define the request and response shape with Pydantic
- Build the /generate endpoint
- Add validation depth and a max-tokens / rate guard
- Run it with uvicorn and hit it with curl
- Production concerns -- and where security actually lives
- What you actually built