Gc duration collections total memory -> all pools – How to solve this Elasticsearch error

Opster Team

Aug-23, Version: 2.3-2.3

Briefly, this error occurs when Elasticsearch’s Garbage Collection (GC) process takes a long time to complete, indicating potential memory issues. This could be due to insufficient heap size, inefficient GC settings, or high indexing/searching load. To resolve this, you can increase the heap size, optimize GC settings, or balance the load by adding more nodes to the cluster. Additionally, consider optimizing your queries and indexing process to reduce the load on Elasticsearch.

This guide will help you check for common problems that cause the log ” [gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {} ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: memory and monitor.

Log Context

Log “[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}” classname is JvmMonitorService.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :

 
                if (avgCollectionTime > gcThreshold.warnThreshold) {
                    logger.warn("[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}";
                            gc.getName(); seq; gc.getCollectionCount(); TimeValue.timeValueMillis(collectionTime); collections; TimeValue.timeValueMillis(currentJvmStats.getTimestamp() - lastJvmStats.getTimestamp()); TimeValue.timeValueMillis(collectionTime); gc.getCollectionTime(); lastJvmStats.getMem().getHeapUsed(); currentJvmStats.getMem().getHeapUsed(); JvmInfo.jvmInfo().getMem().getHeapMax(); buildPools(lastJvmStats; currentJvmStats));
                } else if (avgCollectionTime > gcThreshold.infoThreshold) {
                    logger.info("[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}";
                            gc.getName(); seq; gc.getCollectionCount(); TimeValue.timeValueMillis(collectionTime); collections; TimeValue.timeValueMillis(currentJvmStats.getTimestamp() - lastJvmStats.getTimestamp()); TimeValue.timeValueMillis(collectionTime); gc.getCollectionTime(); lastJvmStats.getMem().getHeapUsed(); currentJvmStats.getMem().getHeapUsed(); JvmInfo.jvmInfo().getMem().getHeapMax(); buildPools(lastJvmStats; currentJvmStats));
                } else if (avgCollectionTime > gcThreshold.debugThreshold && logger.isDebugEnabled()) {
                    logger.debug("[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}";
                            gc.getName(); seq; gc.getCollectionCount(); TimeValue.timeValueMillis(collectionTime); collections; TimeValue.timeValueMillis(currentJvmStats.getTimestamp() - lastJvmStats.getTimestamp()); TimeValue.timeValueMillis(collectionTime); gc.getCollectionTime(); lastJvmStats.getMem().getHeapUsed(); currentJvmStats.getMem().getHeapUsed(); JvmInfo.jvmInfo().getMem().getHeapMax(); buildPools(lastJvmStats; currentJvmStats));
                }




 

How helpful was this guide?

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?