备份恢复

备份恢复的总体流程

1. elasticsearch.yaml中配置的备份路径，拷贝到新的机器上。

   # 备份路径
   path.repo: ["/data/elasticsearch/snapshots/snapshot01","/data/elasticsearch/snapshots/snapshot02"]
   

2. 将备份路径通过nfs挂载到新的集群实例的每个节点上

3. 创建snapshot repo

   • 创建snapshot repo

     esname=elastic
     espassword=xxxxxx
     curl -X PUT -H 'Content-Type: application/json' --user ${esname}:${espassword} -d @snapshot-repo.json  "http://localhost:9200/_snapshot/recover_repo" 
     

   • snapshot-repo.json

     如下所示, location为备份路径

     {
         "type": "fs",
         "settings": {
             "compress": true,
             "location": "/data/els_backup_new"
         }
     }
     

4. 从snapshot repo恢复数据

   esname=elastic
   espassword=xxxxxx
   
   # close duplicated index
   # curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword}  "http://localhost:9200/.security-7/_close"
   
   # snapshot的名字
   snapshot=snapshot_2024-11-05_01-00
   
   curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword} -d @recover-settings.json  "http://localhost:9200/_snapshot/recover_repo/$snapshot/_restore?wait_for_completion=true" | python -m json.tool
   
   

   其中recover-settings.json:

   {
   "indices": "*,-.*"
   }
   

   snapshot的名称可以通过以下接口获取：

   esname=elastic
   espassword=xxxxx
   curl -X GET -H 'Content-Type: application/json' --user ${esname}:${espassword}  "http://localhost:9200/_snapshot/recover_repo" 
   

5. 查看迁移进度

   esname=elastic
   espassword=xxxx
   
   curl -X GET --user ${esname}:${espassword}   "http://localhost:9200/_recovery/" 
   

   响应输出格式为json，可以通过grep -v '100.0%'来查看未恢复完成的索引。

备份恢复成功输出

[portal@es-node1 tmp]$ bash recover-es-from-snapshot.sh 
{"acknowledged":true,"shards_acknowledged":false,"indices":{}}  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27914  100 27914    0     0     61      0  0:07:37  0:07:30  0:00:07  7721
{
    "snapshot": {
        "indices": [
            // 恢复的索引
        ],
        "shards": {
            "failed": 0,
            "successful": 1714,
            "total": 1714
        },
        "snapshot": "snapshot_2024-11-05_01-00"
    }
}

备份恢复的常见问题：

索引冲突

[portal@es-node1 tmp]$ bash recover-es-from-snapshot.sh 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   743  100   743    0     0    354      0  0:00:02  0:00:02 --:--:--   354
{
    "error": {
        "reason": "[recover_repo:snapshot_2024-11-05_01-00/l_5Y6dxoSvGAhAHH5zjwGQ] cannot restore index [.security-7] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name",
        "root_cause": [
            {
                "reason": "[recover_repo:snapshot_2024-11-05_01-00/l_5Y6dxoSvGAhAHH5zjwGQ] cannot restore index [.security-7] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name",
                "type": "snapshot_restore_exception"
            }
        ],
        "type": "snapshot_restore_exception"
    },
    "status": 500
}

出现这种情况，需要把索引关闭：

esname=elastic
espassword=xxx
# 关闭索引
curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword}  "http://localhost:9200/.security-7/_close"

其中.security-7是索引的名称

4.11.06-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [.ds-.logs-deprecation.elasticsearch-default-2024.11.06-000001] is empty or not defined
	at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:89) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:226) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:418) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:349) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:186) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:220) [x-pack-core-7.17.6.jar:7.17.6]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
[2024-11-06T17:00:44,102][ERROR][o.e.x.i.IndexLifecycleRunner] [node-1] policy [ilm-history-ilm-policy] for index [.ds-ilm-history-5-2024.11.06-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [.ds-ilm-history-5-2024.11.06-000001] is empty or not defined
	at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:89) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:226) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:418) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:349) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:186) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:220) [x-pack-core-7.17.6.jar:7.17.6]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]