Before you begin reading this guide, we recommend you run Elasticsearch Error Check-Up which can resolve issues that cause many errors.
This guide will help you check for common problems that cause the log ” Unexpected failure during ; current state:n ” to appear. It’s important to understand the issues related to the log, so to get started, read the general overview on common issues and tips related to the Elasticsearch concepts: cluster and routing.
Advanced users might want to skip right to the common problems section in each concept or try running the Check-Up which analyses ES to pinpoint the cause of many errors and provides suitable actionable recommendations how to resolve them (free tool that requires no installation).
Overview
In Elasticsearch, routing refers to document routing. When you index a document, Elasticsearch will determine which shard the document should be routed to for indexing.
The shard is selected based on the following formula:
shard = hash(_routing) % number_of_primary_shards
Where the default value of _routing is _id.
It is important to know which shard the document is routed to, because Elasticsearch will need to determine where to find that document later on for document retrieval requests.
Examples
In twitter index with 2 primary shards, the document with _id equal to “440” gets routed to the shard number:
shard = hash( 440 ) % 2
PUT twitter/_doc/440
{
...
}Notes and good things to know
- In order to improve search speed, you can create custom routing. For example, you can enable custom routing that will ensure that only a single shard will be queried (the shard that contains your data).
- To create custom routing in Elasticsearch, you will need to configure and define that not all routing will be completed by default settings. ( v <= 5.0)
PUT my_index/customer/_mapping
{
"order":{
"_routing":{
"required":true
}
}
}- This will ensure that every document in the “customer” type must specify a custom routing. For Elasticsearch version 6 or above you will need to update the same mapping as:
PUT my_index/_mapping
{
"order":{
"_routing":{
"required":true
}
}
}Log Context
Log “Unexpected failure during [{}]; current state:n{}” classname is RoutingService.java
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
Override
public void onFailure(String source; Exception e) {
rerouting.set(false);
ClusterState state = clusterService.state();
if (logger.isTraceEnabled()) {
logger.error(() -> new ParameterizedMessage("unexpected failure during [{}]; current state:\n{}";
source; state); e);
} else {
logger.error(() -> new ParameterizedMessage("unexpected failure during [{}]; current state version [{}]";
source; state.version()); e);
}
Run the Check-Up to get customized recommendations like this:
Run Check-Up
The high disk watermark threshold is about to be reached in specific nodes
Description
There are various “watermark” thresholds on each Elasticsearch cluster. When the high disk watermark threshold has been exceeded, it means disk space is running out. The node will…
Recommendations
Based on your specific ES deployment you should…
X-PUT curl -H [a customized code snippet to resolve the issue]