Elasticsearch Elasticsearch – Many Index Get Requests with Missing Documents

Average Read Time

1 Mins

Elasticsearch Elasticsearch – Many Index Get Requests with Missing Documents

Opster Team

June-2022

Average Read Time

1 Mins

Opster Team

October 2021

Average Read Time

1 Mins


In addition to reading this guide, we recommend you run the Elasticsearch Health Check-Up. It will detect issues and improve your Elasticsearch performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and more.

The Elasticsearch Check-Up is free and requires no installation.

Run the Elasticsearch check-up to receive recommendations like this:

checklist Run Check-Up
error

An indexing burst is affecting the performance of the following nodes

error-img

Description

The node is unable to keep up with indexing requests, and as a result indexing requests are being queued. If the write queue reaches full capacity, index requests will be rejected, which may cause data loss if...

error-img

Recommendation

In order to resolve your indexing bursts, based on your specific ES deployment, we recommend that you...

1

X-PUT curl -H "Content-Type: application/json" [customized recommendation]

This article is related to Opster’s Check Up tool. The Check-Up is free, does not require any installation and helps users improve performance in Elasticsearch and OpenSearch. Read more about it here, and run the tool here.

What it means

When you try to retrieve an Elasticsearch document by ID, Elasticsearch will count the number of times that a search or index operation searches for an ID which does not exist in the index. The most common cause for this is when an update or bulk update operation is trying to update a record with an ID which does not exist.

How to resolve

When you have a high index get missing total count, it is possible that your application architecture uses “upsert” as a strategy, i.e. updating a document if it exists and creating a new document if it doesn’t exist. In this case it is quite normal to have a high index get missing total count, and so this is expected behavior and there is no need to change anything.

If the amount of get misses is not too high, meaning in the perfectly normal range, it should not be considered an issue. However, when there is a high, consistent rate of get misses it might indicate that there is some application logic that should be optimized and reviewed. A high rate of index misses can cause indexing performance degradation.

As an edge case, if your system architecture expects the document being updated to always exist, then a high number here indicates that clients are trying to update documents that haven’t been indexed previously, so it is possible that indexing operations have failed or not run at all.  In this case, it is worthwhile investigating the cause and taking action to capture failed indexing commands and have a system to retry indexing failures.



Run the Check-Up to get a customized report like this:

Analyze your cluster
Skip to content