To understand why Elasticsearch is unable to parse the response body, we recommend you run the AutoOps for Elasticsearch. It will analyse your cluster and help you resolve and prevent this error from occuring again.The tool is free and require no installation.
This guide will explain how to resolve this issue and also review common problems related to shards and searches that might cause shard failure.
Overview
This error is typically related to the REST High Level Client and it occurs whenever the client cannot parse the response received by Elasticsearch’s low-level client.
What it means
The REST High Level Client acts as a wrapper around the low-level client. The latter is the one that will ultimately perform the HTTP request to the cluster. If for any reason the response returned to the High Level Client is broken or does not comply with the schema it is expecting, then it will throw the “Unable to parse response body” exception.
Why it occurs
Below are a few scenarios where users reported receiving this error, and the likely causes:
Version mismatch between the client and the cluster
Elastic does not guarantee that it will maintain the compatibility between different major versions. If you’re running a 6.x cluster, but your application uses a 7.x version of the High Level Client, this could be what is causing the issue, due to differences in the response schema.
You can learn more about compatibility for the High Level REST client here.
The cluster is behind a reverse proxy with a path prefix
If your cluster is behind a reverse proxy and you have set a path prefix to access it, you need to make sure to correctly configure the High Level Client so it reaches and gets a response from the cluster itself, as opposed to from the reverse proxy. If your client is getting the response from the reverse proxy service this could be what is causing the “Unable to parse response body” exception.
Suppose you have a Nginx reverse proxy receiving connections at mycompany.com:80 and you have set the /elasticsearch path prefix that will proxy connections to a cluster running inside your infrastructure. In this case you should make sure the path prefix is properly configured in the configuration file of the client you’re using to access the cluster, not only the host (mycompany.com in this case).
If you have an app that uses the High Level Client to access the cluster your can use the setPathPrefix() to set the path prefix, like so:
new RestHighLevelClient(
RestClient.builder(
new HttpHost("mycompany.com", 80, DEFAULT_SCHEME_NAME)).setPathPrefix("/elasticsearch"));This post and this post from Elastic’s forum might be helpful.
You are reaching the HTTP size limit
Some users have reported the “Unable to parse response body” error when bulk indexing a huge volume of data. This happens because Elasticsearch by default has a maximum size of HTTP request body of 100Mb (you can change the http.max_content_length setting in the Elasticsearch configuration file) and you could be reaching that limit.
This post on Elastic’s forum is an example of such a case and could be helpful if you’re facing a similar situation.
The cluster is running on Kubernetes and its entrypoint is through an Ingress Controller
If you have your cluster running inside a Kubernetes cluster and the entrypoint for the Elasticsearch service is through an Ingress Controller, then you should double check your Ingress configuration, since it is a moving piece between your clients and your cluster.
Check out this post on Elastic’s forum to see an example of misconfiguration in the Ingress Controller related to redirecting incoming connections through SSL, which was not actually configured and therefore was the root cause of the problem that ultimately caused the “Unable to parse response body” exception.
How to resolve it
Here is a checklist to go through when trying to solve the “Unable to parse response body” error:
- Are the client (your App, a third-party tool, even an Elastic tool like Logstash) and the cluster version compatible (same major version)?
- Are you bulk indexing a huge amount of data? Try to increase the http.max_content_length setting in the elasticsearch.yml configuration file.
- Is your cluster behind a reverse proxy?
- Make sure your client is accessing the exact path configured in your proxy that will redirect the request to the cluster.
- Could any other configuration in the reverse proxy be causing the request to fail? Maybe you have a request rate limit or any other request policy in place in your reverse proxy service and this could be causing the request not to reach the cluster. In this case the client will receive a response from the reverse proxy itself and it will not be able to parse the response body (as it would definitely not comply with the expected schema).
- Running Elasticsearch on Kubernetes with an Ingress Controller acting as the entrypoint? Double check your Ingress configurations and make sure you can definitely reach the cluster.
Overview
Rest-high-level is built on top of low-level rest-client and is a method of communicating with Elasticsearch based on HTTP REST endpoints. This concept is majorly popular in the context of a Java-based Elasticsearch client. From day one, Elasticsearch supports transport clients for Java to communicate with Elasticsearch. In version 5.0, a low-level rest-client was released with lots of advantages over the existing transport client such as version independencies, increased stability, and lightweight JAR file libraries.
What it is used for
It is used for communicating with Elasticsearch HTTP REST endpoints in which marshalling and unmarshalling of response objects are handled by the Elasticsearch server itself.
Overview
Any application that interfaces with Elasticsearch to index, update or search data, or to monitor and maintain Elasticsearch using various APIs can be considered a client
It is very important to configure clients properly in order to ensure optimum use of Elasticsearch resources.
Examples
There are many open-source client applications for monitoring, alerting and visualization, such as ElasticHQ, Elastalerts, and Grafana to name a few. On top of Elastic client applications such as filebeat, metricbeat, logstash and kibana that have all been designed to integrate with Elasticsearch.
However it is frequently necessary to create your own client application to interface with Elasticsearch. Below is a simple example of the python client (taken from the client documentation):
from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()
doc = {
'author': 'Testing',
'text': 'Elasticsearch: cool. bonsai cool.',
'timestamp': datetime.now(),
}
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)
print(res['result'])
res = es.get(index="test-index", doc_type='tweet', id=1)
print(res['_source'])
es.indices.refresh(index="test-index")
res = es.search(index="test-index", body={"query": {"match_all": {}}})
print("Got %d Hits:" % res['hits']['total']['value'])
for hit in res['hits']['hits']:
print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])
All of the official Elasticsearch clients follow a similar structure, working as light wrappers around the Elasticsearch rest API, so if you are familiar with Elasticsearch query structure they are usually quite straightforward to implement.
Notes and Good Things to Know
Use official Elasticsearch libraries.
Although it is possible to connect with Elasticsearch using any HTTP method, such as a curl request, the official Elasticsearch libraries have been designed to properly implement connection pooling and keep-alives.
Official Elasticsearch clients are available for java, javascript, Perl, PHP, python, ruby and .NET. Many other programming languages are supported by community versions.
Keep your Elasticsearch version and client versions in sync.
To avoid surprises, always keep your client versions in line with the Elasticsearch version you are using. Always test clients with Elasticsearch since even minor version upgrades can cause issues due to dependencies or a need for code changes.
Load balance across appropriate nodes.
Make sure that the client properly load balances across all of the appropriate nodes in the cluster. In small clusters this will normally mean only across data nodes (never master nodes), or in larger clusters, all dedicated coordinating nodes (if implemented) .
Ensure that the Elasticsearch application properly handles exceptions.
In the case of Elasticsearch being unable to cope with the volume of requests, designing a client application to handle this gracefully (such as through some sort of queueing mechanism) will be better than simply inundating a struggling cluster with repeated requests.
Log Context
Log “Unable to parse response body”classname is RestHighLevelClient.java We extracted the following from Elasticsearch source code for those seeking an in-depth context :
} else {
try {
elasticsearchException = parseEntity(entity; BytesRestResponse::errorFromXContent);
elasticsearchException.addSuppressed(responseException);
} catch (Exception e) {
elasticsearchException = new ElasticsearchStatusException("Unable to parse response body"; restStatus; responseException);
elasticsearchException.addSuppressed(e);
}
}
return elasticsearchException;
}
