In the face of multi cluster enterprise level complex architecture, it is very important to prepare a set of backup cluster. The backup cluster usually does not participate in reading and writing operations, but always contains all the data of the business cluster. When there is a business cluster that cannot be used, you can directly switch elastic search to complete the disaster recovery effect. This article is about cross cluster data migration and data disaster recovery
1, Snapshot And Restore description
A snapshot is a backup taken from a running Elasticsearch cluster. You can take a snapshot of a single index or an entire cluster and store it in a repository on a shared file system.
Snapshots can be restored to the running cluster through the restore API.
2, First of all, you need a warehouse
vim conf/elasticsearch.yml path.repo /data/backupNote: this directory must be a network shared disk, such as Nfs,Samba, etc
3, Tell Elasticsearch where the warehouse is
PUT /_snapshot/my_backup #Use_ snapshot API specifies the warehouse name { "type": "fs", "settings": { "location": "my_backup_location" #Specify warehouse location "compress": true #Compress Mappings and settings, not data } }
GET /_snapshot/my_backup #Get warehouse location
4, Backup when you're ready
PUT /_snapshot/my_backup/kuaizhao1?wait_for_completion=true
#wait_for_completion wait for the snapshot to complete before returning
1. Return the following string as success
{"snapshot":{"snapshot":"kuaizhao1,"uuid":"x3AXRHavTySEe2BpBhU_FQ","version_id":5060199,"version":"5.6.1","indices":["test-aop-2014.04","test-xtx-2020.05","test2-newlis-2020.05","test-xtx-2014.04"],"state":" SUCCESS","start_time":"2020-05-28T06:58:09.818Z","start_time_in_millis":1590649089818,"end_time":"2020-05-28T06:58:10.258Z", "end_time_in_millis":1590649090258,"duration_in_millis":440,"failures":[],"shards":{"total":8,"failed":0,"successful":8}}}
2. Generate the following files in the backup warehouse / data/backup
-rw-rw-r-- 1 elk elk 29 May 28 11:09 incompatible-snapshots -rw-rw-r-- 1 elk elk 275 May 28 11:03 index-0 -rw-rw-r-- 1 elk elk 8 May 28 11:03 index.latest drwxrwxr-x 6 elk elk 4096 May 28 11:03 indices -rw-rw-r-- 1 elk elk 698 May 28 11:03 meta-RUiiWfeISgWFNrVfQ80sGw.dat -rw-rw-r-- 1 elk elk 232 May 28 11:03 snap-RUiiWfeISgWFNrVfQ80sGw.dat
5, Incremental backup is to create another snapshot in the warehouse
PUT /_snapshot/my_backup/kuaizhao2?wait_for_completion=true
#Automatically identify other snapshot information in the warehouse, only snapshot new or updated indexes
6, Attach the network disk containing snapshot data to any node of another set of cluster
Start restoring snapshot
1. This is a new set of clusters. There are no other indexes or duplicate indexes on it
POST _snapshot/my_backup/kuaizhao1/_restore
2. The snapshot has been recovered before, or there is a duplicate index on it
POST test-xtx-2020.05/_close #Close the index to be restored first POST _snapshot/my_backup/kuaizhao2/_restore GET _cat/shards #Automatically open index after restore
7, Troubleshooting
RepositoryVerificationException[[backup5.28] a file written by master to the store [/data/backup] cannot be accessed on the node []. This might indicate that the store [/data/backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node]
Snapshot warehouse cannot be a local directory, only a network shared disk
location [/data/backup] doesn't match any of the locations specified by path.repo because this setting is empty
First in elasticsearch.yml The configuration warehouse in cannot be resumed until it is restarted