High disk watermark exceeded on shards will be relocated away from this node – How to solve this Elasticsearch error

Opster Team

Aug-23, Version: 6.8-7.15

Before you dig into reading this guide, have you tried asking OpsGPT what this log means? You’ll receive a customized analysis of your log.

Try OpsGPT now for step-by-step guidance and tailored insights into your Elasticsearch operation.

Briefly, this error occurs when the disk usage on a node in an Elasticsearch cluster exceeds the high watermark threshold. This is a protective measure to prevent the node from running out of disk space. To resolve this issue, you can either increase the disk space available on the node, delete unnecessary data, or adjust the watermark thresholds. However, be cautious when adjusting the thresholds as it could lead to disk space issues if not managed properly. You can also consider adding more nodes to the cluster to distribute the data and reduce the disk usage on individual nodes.

For a complete solution to your to your search operation, try for free AutoOps for Elasticsearch & OpenSearch . With AutoOps and Opster’s proactive support, you don’t have to worry about your search operation – we take charge of it. Get improved performance & stability with less hardware.

This guide will help you check for common problems that cause the log ” high disk watermark [{}] exceeded on {}; shards will be relocated away from this node; ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: cluster, shards, routing, allocation.

Log Context

Log “high disk watermark [{}] exceeded on {}; shards will be relocated away from this node; ” classname is DiskThresholdMonitor.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :

                    if (usageIncludingRelocations.getFreeBytes() 

 

How helpful was this guide?

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?