💾
Database restore drill (quarterly)
Prove backups work — restore to staging before prod depends on it.
Reviewed: ·Tested on: Kubernetes 1.29, Terraform 1.8, Ubuntu 22.04
Symptoms
- Never tested restore — only backups exist
- Go-live checklist backup item unchecked
- Need RTO/RPO numbers for audit
11. Pick backup source
One system only: pg_dump file, RDS snapshot, Velero backup, or S3 sync.
22. Restore to isolated target
Empty DB, new RDS instance, or Velero restore with namespace mapping — never first try on prod.
# PostgreSQL example
createdb restore_drill
pg_restore -d restore_drill -c latest.dump
psql restore_drill -c "SELECT count(*) FROM critical_table;"33. Smoke test application
Point staging app at restored data or run read-only queries matching backup timestamp.
curl -I https://staging.example.com/health
# compare row counts / checksums to backup time44. Record RTO and gaps
Log restore duration, who ran it, and fix retention/cron if backup was missing or corrupt.