Lion Cipher Lab
Enforce zero-trust east-west traffic with Istio: STRICT mTLS + default-deny authorization (prove 'RBAC: access denied')
- Cloud Security
- Advanced
- ~90 min
- Free · no account required
Why it matters
Perimeter firewalls assume the inside of your network is trustworthy. Zero trust throws that assumption out: every call must prove WHO it is and be explicitly authorized, even between two pods in the same cluster. Inside Kubernetes, pod IPs are ephemeral and reused, so IP-based rules age badly, and by default any pod can talk to any other pod in plaintext. A service mesh answers this by giving every workload a cryptographic IDENTITY (a SPIFFE ID minted by the mesh CA and carried in an X.509 cert), wrapping every east-west call in mutual TLS, and letting you write authorization in terms of…
What you'll learn
- what a service mesh actually enforces: a sidecar (Envoy) proxy per pod that terminates mutual TLS and applies L7 authorization on every east-west call, using a cryptographic WORKLOAD IDENTITY (SPIFFE ID: spiffe://<trust-domain>/ns/<namespace>/sa/<service-account>) issued by Istio's CA
- the PERMISSIVE-vs-STRICT distinction: the mesh ships PERMISSIVE (a port accepts BOTH mTLS and plaintext) so adoption breaks nothing — a false sense of security — and why you must explicitly set PeerAuthentication STRICT and then prove plaintext is refused
- writing a PeerAuthentication resource and scoping it correctly: mesh-wide (in the root namespace istio-system), per-namespace, or per-workload with a selector
- verifying mTLS is truly in force with istioctl — `istioctl x describe pod`, `istioctl proxy-config secret`, and reading the Envoy config to confirm the transport socket is TLS, not a claim you take on faith
- the DEFAULT-ALLOW trap: if NO AuthorizationPolicy selects a workload, every AUTHENTICATED caller is allowed — encryption in transit is NOT authorization; an empty-spec `{}` policy means deny-all
- building a DEFAULT-DENY posture (a deny-all AuthorizationPolicy) plus an explicit identity-based ALLOW keyed on the caller's SPIFFE principal / service account, and proving a disallowed caller gets 'RBAC: access denied'
- the SIDECAR-BYPASS failure mode: a pod in a namespace without the istio-injection label (or otherwise missing its Envoy proxy) is NOT in the mesh, carries no identity, and silently escapes PeerAuthentication and AuthorizationPolicy — a real hole
- authorization evaluation ORDER (CUSTOM -> DENY -> ALLOW) and how a too-broad ALLOW rule re-opens a hole you thought a deny closed
- reading the proofs like an operator: 'RBAC: access denied' vs connection-reset-on-plaintext vs a clean 200, and where each comes from in the request path
- the judgment call: mesh mTLS (L7, identity-aware) vs Kubernetes NetworkPolicy (L3/L4, IP/label) vs app-layer mTLS — layer them, don't pick one — plus Istio ambient (no sidecars) vs sidecar mode, and when a full mesh is OVERKILL (a handful of services -> NetworkPolicy + app mTLS is lighter)
Steps in this lab
- The mental model: sidecars, workload identity (SPIFFE), and the two guarantees you'll enforce
- Stand up a free local cluster and install Istio (demo profile)
- Deploy the two workloads and confirm each pod has a sidecar (or deliberately doesn't)
- Expose the PERMISSIVE trap: plaintext from an out-of-mesh client still works
- Flip to STRICT mTLS and prove plaintext is now refused
- The mTLS-is-not-authorization trap, and the empty-spec `{}` = deny-all gotcha
- Default-deny + an identity-scoped ALLOW: prove the wrong identity gets 'RBAC: access denied'
- The named failure modes: scope, evaluation order, the too-broad ALLOW, and the sidecar bypass
- The judgment: mesh mTLS vs NetworkPolicy vs app mTLS, ambient vs sidecar, and when a mesh is overkill