Now throttling indexing numMergesInFlight= maxNumMerges= – How to solve this Elasticsearch error

Opster Team

Aug-23, Version: 6.8-8.9

Before you dig into reading this guide, have you tried asking OpsGPT what this log means? You’ll receive a customized analysis of your log.

Try OpsGPT now for step-by-step guidance and tailored insights into your Elasticsearch operation.

Briefly, this error occurs when Elasticsearch is experiencing high indexing load and the number of merges in flight exceeds the maximum limit. This is a built-in mechanism to prevent the system from overloading. To resolve this issue, you can consider the following: 1) Increase the system resources (CPU, RAM, Disk I/O) if possible. 2) Optimize your indexing process by using bulk requests, reducing the index refresh rate, or increasing the index buffer size. 3) Review your shard size and number, as large or numerous shards can cause high merge pressure.

For a complete solution to your to your search operation, try for free AutoOps for Elasticsearch & OpenSearch . With AutoOps and Opster’s proactive support, you don’t have to worry about your search operation – we take charge of it. Get improved performance & stability with less hardware.

This guide will help you check for common problems that cause the log ” now throttling indexing: numMergesInFlight={}; maxNumMerges={} ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: index.

Log Context

Log “now throttling indexing: numMergesInFlight={}; maxNumMerges={}” classname is InternalEngine.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :

        @Override
        public synchronized void beforeMerge(OnGoingMerge merge) {
            int maxNumMerges = mergeScheduler.getMaxMergeCount();
            if (numMergesInFlight.incrementAndGet() > maxNumMerges) {
                if (isThrottling.getAndSet(true) == false) {
                    logger.info("now throttling indexing: numMergesInFlight={}; maxNumMerges={}"; numMergesInFlight; maxNumMerges);
                    activateThrottling();
                }
            }
        }

 

How helpful was this guide?

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?