Backup & restore

21 commands — backup, restore, verify. Backups you never restore are wishful thinking.

Reviewed: ·Tested on: Kubernetes 1.29, Terraform 1.8, Ubuntu 22.04

Quarterly restore drill

0/5

By system

🔐

etcd (control plane)

Snapshot etcd before control plane upgrades — cluster brain.

Before every K8s upgrade; store snapshot off control plane node.

📦

Files / configs to S3

Sync /etc, nginx configs, or app uploads to durable object storage.

Cron with --delete only after verify; versioned bucket.

All backup snippets

PostgreSQL logical backup (pg_dump)

backup

pg_dump -h <host> -U <user> -d <db> -Fc -f backup-$(date +%F).dump
# custom format (-Fc) supports parallel restore
aws s3 cp backup-$(date +%F).dump s3://my-backups/postgres/
⚠️ DestructiveOverwrites target database. Restore to empty DB or staging first.

PostgreSQL restore from pg_dump

backup

createdb -h <host> -U <user> restore_test
pg_restore -h <host> -U <user> -d restore_test -c backup-2026-06-21.dump
psql -h <host> -U <user> -d restore_test -c "SELECT count(*) FROM users;"

MySQL mysqldump backup

backup

mysqldump -h <host> -u <user> -p --single-transaction --routines <db> > backup-$(date +%F).sql
gzip backup-$(date +%F).sql
aws s3 cp backup-$(date +%F).sql.gz s3://my-backups/mysql/
⚠️ DestructiveDrops and recreates data in target DB. Use staging first.

MySQL restore from dump

backup

gunzip -c backup-2026-06-21.sql.gz | mysql -h <host> -u <user> -p <db>
mysql -h <host> -u <user> -p -e "SELECT COUNT(*) FROM users;" <db>

MongoDB mongodump archive

backup

mongodump --uri="mongodb://user:pass@host:27017/app" --archive=backup-$(date +%F).gz --gzip
aws s3 cp backup-$(date +%F).gz s3://my-backups/mongo/
⚠️ DestructiveRestores into existing collections. Test on empty DB first.

MongoDB restore from archive

backup

mongorestore --uri="mongodb://host:27017" --archive=backup-2026-06-21.gz --gzip --drop
mongosh --eval "db.users.countDocuments()"

Redis RDB snapshot (BGSAVE)

backup · redis

redis-cli -h <host> BGSAVE
redis-cli -h <host> LASTSAVE
# copy /var/lib/redis/dump.rdb to backup storage
scp redis-host:/var/lib/redis/dump.rdb ./backup-$(date +%F).rdb
⚠️ DestructiveStop Redis, replace dump.rdb, restart — data loss if wrong file.

Redis restore RDB file

backup · redis

sudo systemctl stop redis
sudo cp backup-2026-06-21.rdb /var/lib/redis/dump.rdb
sudo chown redis:redis /var/lib/redis/dump.rdb
sudo systemctl start redis
redis-cli ping

Velero backup namespace

backup · k8s · aws

velero backup create apps-$(date +%F) --include-namespaces apps --wait
velero backup describe apps-$(date +%F)
velero backup logs apps-$(date +%F)
⚠️ DestructiveRestores resources into cluster — use --namespace-mappings to staging NS first.

Velero restore from backup

backup · k8s

velero restore create restore-$(date +%F) --from-backup apps-2026-06-21 --wait
velero restore describe restore-$(date +%F)
kubectl get all -n apps

etcd snapshot (control plane)

backup · k8s

ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-$(date +%F).db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key
etcdctl snapshot status /backup/etcd-$(date +%F).db
⚠️ DestructiveCluster-wide disaster recovery only — follow vendor runbook; wrong step bricks cluster.

etcd restore snapshot

backup · k8s

# STOP API server + etcd on all nodes first — follow your K8s distro docs
ETCDCTL_API=3 etcdctl snapshot restore /backup/etcd-2026-06-21.db \
  --data-dir=/var/lib/etcd-restore
# then reconfigure etcd to use restored data dir

Terraform state backup (pull local copy)

backup · terraform

terraform state pull > terraform.tfstate.backup-$(date +%F).json
# remote backend: enable S3 versioning on state bucket
aws s3api list-object-versions --bucket my-tf-state --prefix prod/terraform.tfstate
⚠️ DestructiveWrong state version can destroy infra on next apply. Coordinate with team.

Restore Terraform state from S3 version

backup · terraform · aws

aws s3api list-object-versions --bucket my-tf-state --prefix prod/terraform.tfstate
aws s3api get-object --bucket my-tf-state --key prod/terraform.tfstate --version-id <VERSION_ID> terraform.tfstate
terraform state push terraform.tfstate  # only if intentional

AWS RDS manual snapshot

backup · aws

aws rds create-db-snapshot --db-instance-identifier mydb --db-snapshot-identifier mydb-pre-upgrade-$(date +%F)
aws rds wait db-snapshot-available --db-snapshot-identifier mydb-pre-upgrade-$(date +%F)
aws rds describe-db-snapshots --db-instance-identifier mydb

AWS RDS restore snapshot to new instance

backup · aws

aws rds restore-db-instance-from-db-snapshot \
  --db-instance-identifier mydb-restore-test \
  --db-snapshot-identifier mydb-pre-upgrade-2026-06-21
# verify app against restore-test endpoint before cutover

Sync directory to S3 (config backup)

backup · aws

aws s3 sync /etc/nginx s3://my-backups/nginx/$(hostname)/ --exclude "*.tmp"
aws s3 sync s3://my-backups/nginx/$(hostname)/ ./restore-test/ --dryrun
⚠️ DestructiveOverwrites local files. Dry-run first; restore to temp dir.

Restore files from S3 sync

backup · aws

aws s3 sync s3://my-backups/nginx/myhost/ /tmp/nginx-restore/ --dryrun
aws s3 sync s3://my-backups/nginx/myhost/ /tmp/nginx-restore/
diff -r /etc/nginx /tmp/nginx-restore/

Check TLS cert expiry from CLI

debug · nginx · backup

echo | openssl s_client -connect app.example.com:443 -servername app.example.com 2>/dev/null | openssl x509 -noout -dates -subject
# days left:
echo | openssl s_client -connect app.example.com:443 2>/dev/null | openssl x509 -noout -checkend 604800

Cron backup script template

backup

#!/bin/bash
# /usr/local/bin/backup.sh — install via cron: 0 2 * * * /usr/local/bin/backup.sh
set -euo pipefail
BACKUP_DIR=/var/backups
DATE=$(date +%F)
pg_dump -U app appdb | gzip > "$BACKUP_DIR/appdb-$DATE.sql.gz"
find "$BACKUP_DIR" -name 'appdb-*.sql.gz' -mtime +14 -delete
aws s3 cp "$BACKUP_DIR/appdb-$DATE.sql.gz" s3://my-backups/cron/

PostgreSQL rotate app user password

k8s · backup

# 1. Create parallel user
CREATE USER app_v2 WITH PASSWORD 'new-secret';
GRANT ALL ON DATABASE appdb TO app_v2;
GRANT ALL ON ALL TABLES IN SCHEMA public TO app_v2;
# 2. Update K8s secret + rollout restart
kubectl create secret generic db-creds --from-literal=password=new-secret -n apps --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout restart deploy/myapp -n apps

When restore fails