Lion Cipher Lab
Encrypt Kubernetes Secrets at rest, part 1: prove the leak and close it
- Cloud Security
- Intermediate
- ~50 min
- Free · no account required
Why it matters
A Kubernetes Secret looks protected because its values are base64 blobs, but base64 is a reversible encoding, and by default the API server stores those values UNENCRYPTED in etcd — the key-value datastore that is the cluster's source of truth. That means anyone who can read etcd (a compromised control-plane node, a stolen etcd snapshot, an unencrypted backup tape, a misconfigured etcd port) reads every password, token and TLS key in the whole cluster with no cryptography to stop them. Encryption at rest is the control that turns those raw etcd bytes into ciphertext, and it is an explicit…
What you'll learn
- proving a Kubernetes Secret is base64, not encrypted
- reading a Secret straight out of etcd with etcdctl
- writing an EncryptionConfiguration with an aescbc/secretbox provider
- understanding read-vs-write provider ordering (identity must be last)
- wiring encryption-provider-config into the apiserver on a kind cluster
- re-encrypting already-stored secrets with kubectl replace
- knowing when the built-in provider is enough and when you need a KMS
Steps in this lab
- The problem, in one breath: your Secrets sit in etcd as cleartext
- Build a throwaway cluster with kind
- Create a Secret — and prove it is only base64
- Read the Secret straight out of etcd — watch the cleartext fall out
- Write an EncryptionConfiguration (aescbc/secretbox + identity last)
- Wire it into the apiserver by recreating the kind cluster
- Re-encrypt existing secrets, then verify in etcd
- The judgment: static key vs KMS, and where this sits among the other fixes