- Published on
How to Backup and Restore an etcd Snapshot on a Kubernetes Control Plane Node
2 min read
- Authors
- Name
- Vijaykumar Rajendran
- @karan_6864
Table of Contents
Introduction
This blog post details how to backup and restore the etcd database, which is critical for maintaining the state and operation of your Kubernetes cluster.
Backing up etcd
To create a backup of the etcd database, use the following command:
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
snapshot save /opt/snapshot-pre-boot.
This command saves a snapshot of the etcd database to /opt/snapshot-pre-boot.db, ensuring you have a point-in-time copy of your data.
Restoring from a ETCD
To restore the etcd database from the snapshot you've taken, execute:
ETCDCTL_API=3 etcdctl --data-dir /var/lib/etcd-from-backup snapshot restore /opt/snapshot-pre-boot.db
This command restores the snapshot to a new directory /var/lib/etcd-from-backup, allowing you to recover the etcd state to the point when the snapshot was taken.
Updating Kubernetes to Use the Restored Data Next, you need to update the etcd.yaml file located in /etc/kubernetes/manifests to use the newly restored data directory:
volumes:
- hostPath:
path: /var/lib/etcd-from-backup
type: DirectoryOrCreate
name: etcd-data
This configuration change directs the etcd container to use the restored data at /var/lib/etcd-from-backup.
Monitoring and Troubleshooting Monitor the etcd pod's status to ensure it restarts correctly and troubleshoot if necessary:
watch "crictl ps | grep etcd"
If issues arise, such as the pod not reaching the Ready state (1/1), you may need to manually restart it:
kubectl delete pod -n kube-system etcd-
Wait about a minute after executing this command.