Elasticsearch Bootstrap Checks

Avoid the Next Incident

Check if your ES issues are caused from misconfigured settings

2-min process

Stop Elasticsearch settings related incidents :  Fix My Settings

Last Update: February 2020

Bootstrap Checks in Elasticsearch


Overview

Elasticsearch has many settings that can cause significant production issues if not set correctly. To prevent these in production, as well as to start Elasticsearch quickly, Elasticsearch has the concepts of development and production modes.

In development modes, the settings are not mandatory, and even if missed, Elasticsearch will log the settings as warnings. In production mode, Elasticsearch won’t start and will throw the settings as an error.

There are multiple ways Elasticsearch can enforce production checks (including implicit, as well as explicit, mention of them). One of the most common implicit ways to force them is by using the non-loopback address. Loopback addresses are 127.0.0.1 or localhost, and they are meant for local development, but when running it in production or in the cloud (ex AWS) it’s prevalent to have below setting in elasticsearch.yml

network.host: 0.0.0.0

Most of the time, users don’t know that this will implicitly force the production mode in Elasticsearch. Hence all other settings are hard-checked, and messing up on any one of them causes Elasticsearch startup errors, which often renders users frustrated and unsure of how to continue.

Quick fix with an important disclaimer

When the log below appears during Elasticsearch startup:

bound or publishing to a non-loopback address, enforcing bootstrap checks

There could be a variety of bootstrap checks failures, which are self-explanatory and must be resolved to start Elasticsearch. Below is the list of all the Bootstrap check classes from Elasticsearch source code:

Heap Size Check
OsX File Descriptor Check
File Descriptor Check
Mlock all Check
Max Number Of Threads Check
Max Size Virtual Memory Check
Max FileSize Check
Max Map Count Check
Client Jvm Check
UseS erial GC Check
System Call Filter Check
Might Fork Check
On Error Check
On Out Of Memory Error Check
Early Access Check
G1GC Check
All Permission Check
Discovery Configured Check

Users can disable these bootstrap checks (using the below configs) in the development environment but must know that it can cause serious issues like less memory allocated, discovery settings not suitable, no checks on recommended GC, permissions, JVM, heap, etc. in production:

discovery.type: single-node

Or:

es.enforce.bootstrap.checks=false

For faster debugging, you can set the below config (explicit check for bootstrap) to true in development, which would force al the bootstrap checks to pass and after everything passes, go to the non-development environment:

es.enforce.bootstrap.checks=true
Elasticsearch source code snippet:

org.elasticsearch.bootstrap.BootstrapChecks which does all above checks in below code:

if (enforceLimits) {
    logger.info("bound or publishing to a non-loopback address, enforcing bootstrap checks");
} else if (enforceBootstrapChecks) {
    logger.info("explicitly enforcing bootstrap checks");
}

for (final BootstrapCheck check : checks) {
    final BootstrapCheck.BootstrapCheckResult result = check.check(context);
    if (result.isFailure()) {
        if (!(enforceLimits || enforceBootstrapChecks) && !check.alwaysEnforce()) {
            ignoredErrors.add(result.getMessage());
        } else {
            errors.add(result.getMessage());
        }
    }
}

About Opster

Incorporating deep knowledge and broad history of Elasticsearch issues. Opster solution identifies and predicts root causes of Elasticsearch problems, provides recommendations and can automatically perform various actions to manage, troubleshoot and prevent issues

Learn more: Glossary | Blog| Troubleshooting guides

Need help with any Elasticsearch issue ? Contact Opster

Avoid the next incident use our settings check-up : Prevent Issues


Click below to learn how to fix common problems related to these concepts
« Back to Index