Elasticsearch Elasticsearch High Indexing Throttle Time

Average Read Time

2 Mins

Elasticsearch Elasticsearch High Indexing Throttle Time

Opster Team

May-2022

Average Read Time

2 Mins

Opster Team

October 2021

Average Read Time

2 Mins


In addition to reading this guide, we recommend you run the Elasticsearch Health Check-Up. It will detect issues and improve your Elasticsearch performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and more.

The Elasticsearch Check-Up is free and requires no installation.

In addition to reading this guide about Elasticsearch high index throttle time, we recommend you run the Elasticsearch Health Check-Up. It will detect issues and improve your Elasticsearch performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and more.

The Elasticsearch Check-Up is free and requires no installation.

Run the Elasticsearch check-up to receive recommendations like this:

checklist Run Check-Up
error

An indexing burst is affecting the performance of the following nodes

error-img

Description

The node is unable to keep up with indexing requests, and as a result indexing requests are being queued. If the write queue reaches full capacity, index requests will be rejected, which may cause data loss if...

error-img

Recommendation

In order to resolve your indexing bursts, based on your specific ES deployment, we recommend that you...

1

X-PUT curl -H "Content-Type: application/json" [customized recommendation]

This article is related to Opster’s Check Up tool. The Check-Up is free, does not require any installation and helps users improve performance in Elasticsearch and OpenSearch. Read more about it here, and run the tool here.

What it means

During indexing, Elasticsearch accumulates documents in memory and then writes documents to disk to create a new lucene segment. The creation of a large number of segments is inefficient, so there is a separate merge process which merges the small segments created at index time into larger segments.

However, this process takes up a lot of resources, particularly disk i/o resources. When Elasticsearch detects that the merge process cannot keep up with the rate of indexing, then it will start to throttle indexing and this will be indicated by the high value of index throttle time.

When this happens, it is likely that indexing operations will get queued and eventually indexing requests will be rejected. In the best case scenario, data will not be completely up to date, and in the worst case scenario, data may be lost if the application fails to retry the throttled write requests.

Retrying throttled requests can also put an extra burden on processors upstream since they have to hold data in a queue (if possible) and use resources retrying requests.

How to resolve

When index throttling occurs, you should try to optimize indexing using some of the following actions:

  • Reduce the index refresh rate:
PUT /my-index-000001/_settings
{
  "index" : {
    "refresh_interval" : "30s"
  }
}
  • Disable index replicas:
PUT /my-index-000001/_settings
{
  "index" : {
    "number_of_replicas" : 0
  }
}
  • Use bulk indexing rather than individual indexing
  • Upgrade rotating disks to SSD or NVMe types
  • Optimize mappings to reduce unnecessary fields

Also note that default mappings for strings create both keyword and text fields, which is wasteful if you don’t need both.

Notes and good things to know

If you see that index throttling only occurs on some nodes and not all data nodes, this may be due to having an insufficient number of primary shards in the index to spread the indexing activity across all the nodes. This could happen if you have, for instance, one “busy” index with just one shard, which would lead to all the indexing activity becoming concentrated on the node which has the shard for this index. In this case, the solution would be to increase the number of shards in the index settings and create or roll over the index.

For additional recommendations on how to improve your indexing rate, see the full list of recommendations here.   



Run the Check-Up to get a customized report like this:

Analyze your cluster
Skip to content