Elasticsearch Enable Shard Rebalance and Allocation

Elasticsearch Enable Shard Rebalance and Allocation

Opster Team

March 2021


In addition to reading this guide, we recommend you run the Elasticsearch Health Check-Up. It will detect issues and improve your Elasticsearch performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and more.

The Elasticsearch Check-Up is free and requires no installation.

In addition to reading this guide and learning how to enable Elasticsearch shard rebalance and allocation, we recommend you run the Elasticsearch Health Check-Up. It will detect issues and improve your Elasticsearch performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and more.

The Elasticsearch Check-Up is free and requires no installation.

Run the Elasticsearch check-up to receive recommendations like this:

checklist Run Check-Up
error

The following configuration error was detected on node 123...

error-img

Description

This error can have a severe impact on your system. It's important to understand that it was caused by...

error-img

Recommendation

In order to resolve this issue and prevent it from occurring again, we recommend that you begin by changing the configuration to...

1

X-PUT curl -H "Content-Type: application/json" [customized recommendation]

Overview

Cluster shard rebalancing and allocation are often confused with each other.

Cluster shard allocation

This refers to the process by which any shard including new, recovered or rebalanced shards are allocated to Elasticsearch nodes. Cluster shard allocation may be temporarily disabled during maintenance in order to avoid shards from being relocated to nodes that are being restarted and may temporarily leave the cluster.

If cluster shard allocation is NOT enabled, then Elasticsearch will not be able to create new indices, recover corrupted indices or rebalance. This situation is very likely to cause data loss if not corrected quickly.

Cluster shard rebalancing

This refers to the process by which an Elasticsearch cluster may rebalance shards from nodes with high disk utilization (above the value of cluster.routing.allocation.disk.watermark.high) to nodes with lower disk utilisation, or redistribute shards from pre-existing nodes to a new node which has just been added to the cluster.  

If cluster shard rebalancing is not enabled, then Elasticsearch will not be able to optimize disk and cpu utilization across the cluster by rebalancing shards from heavily used nodes to underused nodes. 

How to resolve it

The default values are the recommended ones for both of these settings. To restore the default value run the following command:

PUT _cluster/settings
{
  "transient": {
     "cluster.routing.allocation.enable": "all",

     "cluster.routing.rebalance.enable": "all"
  }
}


Run the Check-Up to get a customized report like this:

Analyze your cluster