Log gc duration ; collections /; total /; memory>/; all-pools – How To Solve Related Issues

Log gc duration ; collections /; total /; memory>/; all-pools – How To Solve Related Issues

Updated: Feb-20

Elasticsearch Version: 1.7-8.0

Background

To resolve issues causing many log errors you can try our Elasticsearch Check-Up it analyses ES configuration to provide actionable recommendations (no installation required) 


To troubleshoot log “gc duration ; collections /; total /; memory>/; all-pools” it’s important to understand a few problems related to Elasticsearch concepts memory, monitor. See bellow important tips and explanations on these concepts

Log Context

Log”[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}” classname is JvmMonitorService.java
We extracted the following from Elasticsearch source code for those seeking an in-depth context :

 
                if (avgCollectionTime > gcThreshold.warnThreshold) {
                    logger.warn("[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}";
                            gc.getName(); seq; gc.getCollectionCount(); TimeValue.timeValueMillis(collectionTime); collections; TimeValue.timeValueMillis(currentJvmStats.getTimestamp() - lastJvmStats.getTimestamp()); TimeValue.timeValueMillis(collectionTime); gc.getCollectionTime(); lastJvmStats.getMem().getHeapUsed(); currentJvmStats.getMem().getHeapUsed(); JvmInfo.jvmInfo().getMem().getHeapMax(); buildPools(lastJvmStats; currentJvmStats));
                } else if (avgCollectionTime > gcThreshold.infoThreshold) {
                    logger.info("[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}";
                            gc.getName(); seq; gc.getCollectionCount(); TimeValue.timeValueMillis(collectionTime); collections; TimeValue.timeValueMillis(currentJvmStats.getTimestamp() - lastJvmStats.getTimestamp()); TimeValue.timeValueMillis(collectionTime); gc.getCollectionTime(); lastJvmStats.getMem().getHeapUsed(); currentJvmStats.getMem().getHeapUsed(); JvmInfo.jvmInfo().getMem().getHeapMax(); buildPools(lastJvmStats; currentJvmStats));
                } else if (avgCollectionTime > gcThreshold.debugThreshold && logger.isDebugEnabled()) {
                    logger.debug("[gc][{}][{}][{}] duration [{}]; collections [{}]/[{}]; total [{}]/[{}]; memory [{}]->[{}]/[{}]; all_pools {}";
                            gc.getName(); seq; gc.getCollectionCount(); TimeValue.timeValueMillis(collectionTime); collections; TimeValue.timeValueMillis(currentJvmStats.getTimestamp() - lastJvmStats.getTimestamp()); TimeValue.timeValueMillis(collectionTime); gc.getCollectionTime(); lastJvmStats.getMem().getHeapUsed(); currentJvmStats.getMem().getHeapUsed(); JvmInfo.jvmInfo().getMem().getHeapMax(); buildPools(lastJvmStats; currentJvmStats));
                }




Related issues to this log

We have gathered selected Q&A from the community and issues from Github, that can help fix related issues please review the following for further information :

1 Garbage Collection Log Messages Mon  

Gc Takes A Lot Of Time  

 

About Opster

Opster detects, resolves, optimizes, automates and prevents incidents in Elasticsearch. Opster’s line of products delivers a fundamentally more effective Elasticsearch operation and backs it up with superb production support and consulting.

Find Configuration Errors

Analyze Now