Elastic search uses snapshot data migration and disaster recovery

In the face of multi cluster enterprise level complex architecture, it is very important to prepare a set of backup cluster. The backup cluster usually does not participate in reading and writing operations, but always contains all the data of the business cluster. When there is a business cluster that cannot be used, you can directly switch elastic search to complete the disaster recovery effect. This article is about cross cluster data migration and data disaster recovery


1, Snapshot And Restore description

A snapshot is a backup taken from a running Elasticsearch cluster. You can take a snapshot of a single index or an entire cluster and store it in a repository on a shared file system.

Snapshots can be restored to the running cluster through the restore API.


2, First of all, you need a warehouse

vim conf/elasticsearch.yml
path.repo /data/backup

Note: this directory must be a network shared disk, such as Nfs,Samba, etc


3, Tell Elasticsearch where the warehouse is

PUT /_snapshot/my_backup #Use_ snapshot API specifies the warehouse name
{
  "type": "fs",
  "settings": {
        "location": "my_backup_location" #Specify warehouse location
        "compress": true #Compress Mappings and settings, not data
  }
}
GET /_snapshot/my_backup #Get warehouse location


4, Backup when you're ready

PUT /_snapshot/my_backup/kuaizhao1?wait_for_completion=true

#wait_for_completion wait for the snapshot to complete before returning

1. Return the following string as success

{"snapshot":{"snapshot":"kuaizhao1,"uuid":"x3AXRHavTySEe2BpBhU_FQ","version_id":5060199,"version":"5.6.1","indices":["test-aop-2014.04","test-xtx-2020.05","test2-newlis-2020.05","test-xtx-2014.04"],"state":"
SUCCESS","start_time":"2020-05-28T06:58:09.818Z","start_time_in_millis":1590649089818,"end_time":"2020-05-28T06:58:10.258Z",
"end_time_in_millis":1590649090258,"duration_in_millis":440,"failures":[],"shards":{"total":8,"failed":0,"successful":8}}}


2. Generate the following files in the backup warehouse / data/backup

-rw-rw-r-- 1 elk  elk   29 May 28 11:09 incompatible-snapshots
-rw-rw-r-- 1 elk  elk   275 May 28 11:03 index-0
-rw-rw-r-- 1 elk  elk   8 May 28 11:03 index.latest
drwxrwxr-x 6 elk  elk    4096 May 28 11:03 indices
-rw-rw-r-- 1 elk  elk   698 May 28 11:03 meta-RUiiWfeISgWFNrVfQ80sGw.dat
-rw-rw-r-- 1 elk  elk   232 May 28 11:03 snap-RUiiWfeISgWFNrVfQ80sGw.dat


5, Incremental backup is to create another snapshot in the warehouse

PUT /_snapshot/my_backup/kuaizhao2?wait_for_completion=true

#Automatically identify other snapshot information in the warehouse, only snapshot new or updated indexes


6, Attach the network disk containing snapshot data to any node of another set of cluster

Start restoring snapshot

1. This is a new set of clusters. There are no other indexes or duplicate indexes on it

POST _snapshot/my_backup/kuaizhao1/_restore

2. The snapshot has been recovered before, or there is a duplicate index on it

POST test-xtx-2020.05/_close #Close the index to be restored first
POST _snapshot/my_backup/kuaizhao2/_restore
GET _cat/shards #Automatically open index after restore


7, Troubleshooting

RepositoryVerificationException[[backup5.28] a file written by master to the store [/data/backup] cannot be accessed on the node [{test1-2}{OZKGLppLRgy1cFdhiVrVGw}{5CgnmLyaS_yGLnxFx1Wrxg}{10.1.5.13}{10.1.5.13:9300}]. This might indicate that the store [/data/backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node]

Snapshot warehouse cannot be a local directory, only a network shared disk

location [/data/backup] doesn't match any of the locations specified by path.repo because this setting is empty

First in elasticsearch.yml The configuration warehouse in cannot be resumed until it is restarted

Tags: Linux snapshot ElasticSearch network vim

Posted on Thu, 28 May 2020 07:15:27 -0400 by ayzee