Elasticsearch Elasticsearch.yml

By Opster Team

Updated: Jun 22, 2023

| 2 min read

Mastering Elasticsearch.yml Configuration

Elasticsearch.yml is the primary configuration file for Elasticsearch clusters. It allows you to fine-tune various settings, such as node roles, discovery settings, and cluster-level configurations. In this article, we will discuss some advanced configurations and best practices for optimizing your Elasticsearch cluster.

Configurations and best practices

1. Node Roles Configuration

By default, Elasticsearch nodes have all roles (master, data, and ingest). However, you can assign specific roles to nodes to optimize resource allocation and improve cluster performance. In elasticsearch.yml, use the following settings to assign roles:

node.roles: [master, data, ingest]

You can remove or add roles as needed. For example, to create a dedicated master node, use:

node.roles: [master]

2. Cluster and Node Naming

Assigning meaningful names to your cluster and nodes can help you manage and monitor your Elasticsearch environment more effectively. Use the following settings in elasticsearch.yml:

cluster.name: my-cluster
node.name: my-node

3. Network and Discovery Settings

To ensure proper communication between nodes in a multi-node cluster, configure the network settings in elasticsearch.yml. Set the network host and publish host as follows:

network.host: _site_
http.publish_host: _site_

For node discovery, configure the following settings:

discovery.seed_hosts: ["host1", "host2"]
cluster.initial_master_nodes: ["master-node-1", "master-node-2"]

4. Heap Size Configuration

Elasticsearch runs on the Java Virtual Machine (JVM), and the heap size determines the amount of memory allocated to the JVM. Proper heap size configuration is crucial for performance. Set the heap size in a custom JVM options file located in the `config/jvm.options.d/` folder:

-Xms1g
-Xmx1g

It is recommended to set the heap size between 25% and 50% of the available RAM, not exceeding 32GB.

5. Index and Shard Management

To optimize index and shard management, configure the following settings in elasticsearch.yml:

index.number_of_shards: 3
index.number_of_replicas: 1

Adjust the number of shards and replicas based on your cluster size and use case.

6. Cluster-level Settings

Cluster-level settings can be updated dynamically using the Cluster Update Settings API. However, you can also set default values in elasticsearch.yml:

cluster.routing.allocation.enable: all
cluster.routing.allocation.node_concurrent_recoveries: 2

7. Security Configuration

To enable security features, such as authentication and encryption, configure the following settings in elasticsearch.yml:

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

Conclusion

In conclusion, mastering Elasticsearch.yml configuration is essential for optimizing your Elasticsearch cluster’s performance and security. By following the best practices outlined in this article, you can ensure a stable and efficient Elasticsearch environment.

How helpful was this guide?

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?