Elasticsearch

Elasticsearch keeps the original JSON document in a field called _source. The source field serves special purposes such as...

The aggregations framework is a tool built in every Elasticsearch deployment. The different aggregation types: Bucket, Metric & Pipeline...

In Elasticsearch, an alias is a secondary name to refer to one or more indices. Aliases can be created and deleted dynamically using...

Elasticsearch scripts can place heavy loads on clusters if they are not written carefully. It is a best practice to limit the type of..

There are various methods for retrieving fields in Elasticsearch, including: _source, stored_fields, fields & docvalue_fields. To retrieve...

There are various approaches for autocomplete in Elasticsearch. Here are some tips & examples for choosing the approach best suited to your...

Elasticsearch carries out "bootstrap checks" to ensure that important settings have been set correctly. If any of these fail, ES won't start.

Elasticsearch can be configured to prevent memory swapping on its host machine by adding bootstrap memory_lock true. If bootstrap checks...

Elasticsearch bulk makes it possible to perform many write operations in a single API call, which increases indexing speed. Using bulk API...

Elasticsearch uses 3 types of caches to improve the efficiency of operation: node requests, shards and field data cache. It is possible to...

If the ratio of memory to number of shards in the cluster is low, it suggests that you have insufficient memory compared to the volume...

Elasticsearch has circuit breakers to deal with OutOfMemory errors that cause nodes to crash. Each breaker is used to...

Official Elasticsearch clients are available for java, javascript, Perl, PHP, python, ruby and .NET. To avoid surprises, keep your client....

An Elasticsearch cluster consists of a number of servers (nodes) working together as one to store data and respond to requests. It enables...

A read-only delete block can be applied automatically by the cluster because of a disk space issue. It can also be applied manually by...

The cluster concurrent rebalance setting determines the maximum number of shards the cluster can move to rebalance the distribution of...

In Elasticsearch, the combined_fields query allows you to search several text fields as though their indexed values have been indexed into...

Cross-cluster search enables users to execute a query across multiple Elasticsearch or OpenSearch clusters. To perform cross-cluster search...

Cluster name and data path are default settings that could be destructive for proper Elasticsearch function if handled incorrectly. If you...

Many clusters use coordinating or ingest nodes, while others leave the ingest and coordination functions to the data nodes. In order to...

Once an Elasticsearch cluster reaches a certain size, it's recommended to create 3 dedicated master nodes. Here is how you can create...

DELETE is an Elasticsearch API which removes a document from a specific index. It requires an index name and _id document in order to...

Elasticsearch delete by query is an API, which provides functionality to delete all documents based on the matching query. If you don't...

To find out which functions have been deprecated in Elasticsearch, you can use deprecation logs, deprecation API, read breaking pages...

Discovery occurs when an Elasticsearch node starts, restarts or loses contact with the master node. In those cases the node needs to...

There are various watermark thresholds on an Elasticsearch cluster. As the disk fills up on a node, the 1st threshold to be crossed is...

Elasticsearch uses several parameters to enable it to manage hard disk storage across the cluster, such as...

Each Elasticsearch document is a JSON structure, which is ultimately considered to be a series of key:value pairs. An example for creating...

The different categories in the Elastic Pricing Calculator can impact your final cost. Here's how to efficiently use the pricing calculator.

When you try to retrieve a document by ID, Elasticsearch will count the number of times that it searches for an ID which doesn't exist...

Elasticsearch cluster allocation explain API helps identify why a shard is unassigned or not relocated as expected. Outputs returned by it...

Elasticsearch data frame analytics allows you to supercharge your data with extra insights. There are 3 types of data frame analytic jobs...

The Elasticsearch async search API retrieves many data in a stream fashion instead of a single request. To limit the maximum response size...

There are 4 types of Elasticsearch boolean clauses: filter, must, should & must_not. A single bool query can contain a mix of them. To use...

Elasticsearch boosting query is used to return only documents that match a positive query while minimizing the score of documents that...

The Elasticsearch aggregation 'categorize_text' helps analyze & extract categories from semi-structured text. To use this aggregation...

Circuit breaker exceptions are thrown to alert us that something needs to be fixed in Elasticsearch in order to reduce memory usage. To fix...

Elasticsearch clusters need to maintain the cluster state in memory on each and every nodes, which requires a large amount of resources...

This guide discusses the Elasticsearch collapse feature, including use cases, how to implement it, and best practices for optimizing...

An Elasticsearch composite aggregation allows to paginate every bucket from a multi-level aggregation effectively. An example of....

Elasticsearch provides various compression techniques to optimize storage and network usage. Here's how to compress an index, source and...

In Elasticsearch, the constant score query wraps other queries by executing them in a filter context. To implement constant_score query...

A coordinating node is a node that handles HTTP(S) requests for the cluster, especially indexing & search requests. A coordinating only...

Here are 12 tips to reduce and Optimize your Elasticsearch costs. First, plan data retention: carefully adjust your...

This guide will focus on how to use cURL to perform delete operations in Elasticsearch. It covers best practices, common issues & examples.

The Elasticsearch data stream is an abstraction layer between the names used by applications to facilitate ingestion and search operations...

In this guide, we will discuss the process of deleting documents in Elasticsearch as well as best practices for deleting documents.

This guide discusses the default document size limit in Elasticsearch, the reasons behind it, and how to handle larger documents effectively.

The exists query is used for returning the documents that have an indexed value for a specific field, which means it returns the documents...

The Elasticsearch Explain API is very useful for trying to understand why any particular document got a specific score. Examples of...

The 3 main methods in Elasticsearch to calculate the storage size of specific fields in an index are: using the _disk_usage API, creating...

Elasticsearch filtered aliases can help you filter data more efficiently. To create a filtered alias, you need to define the criteria for...

In this guide, we'll discuss various methods to find documents in Elasticsearch by field value and give examples for each method.

Elasticsearch fuzzy queries offer a powerful way to handle imprecise search terms. To fine-tune the behavior of fuzzy queries...

Terms aggregations rely on an internal data structure known as global ordinals. The eager_global_ordinals parameter is used to...

A common method to perform an Elasticsearch health check is by using cUR. Here's how to use cURL to check the health of your cluster.

When Elasticsearch detects that the merge process cannot keep up with the rate of indexing, then it will start to throttle indexing...

The Elasticsearch High Level REST Client has been deprecated since v7.7.0. Here is why it was deprecated, the alternatives available & more.

"Hotspots" refers to a situation in which a cluster with multiple nodes is not balanced - some nodes are handling more load than others...

Efficient management of Elasticsearch indices is crucial for maintaining optimal performance. The best practices to manage your index list...

Elasticsearch index patterns allow you to define how to match & interact with multiple indices. The best practices for index pattern usage...

When looking at Shard View for the index, it was clear that the index in question wasn’t carrying out the highest indexing rate and wasn’t...

This guide will discuss best practices & performance optimization techniques for inserting documents into Elasticsearch. First, use the API...

Elasticsearch Intervals query provides control over the words & their positions in a text that is required for a document to match a...

Elasticsearch keyword vs. text vs. wildcard vs. text field types. All have different features and are ideal for different use cases

When cluster state becomes too large it poses many challenges. In order to determine the size of your cluster state and reduce it, you...

Elasticsearch will continue to work even if your license expires, but the paid features will be blocked. To buy a new license and update it...

The new match_only_text feature in Elasticsearch can save up to 10% of disk space on logging datasets. This field type will set a flat...

Match, Multi-Match & Match Phrase are all types of Elasticsearch queries, used to search for matching documents in an index. To use them...

One way to evaluate whether your resources are cost efficient it check the ratio of disk usage to the memory allocated...

Here's a detailed guideline on the minimum requirements for Elasticsearch, including hardware specifications, JVM settings...

The `minimum_should_match` parameter in Elasticsearch plays a crucial role in fine-tuning the relevance of search results. An Example...

Here's how to set up Elasticsearch monitoring using Prometheus and Grafana. To monitor Elasticsearch with Prometheus you first need to...

_Msearch in Elasticsearch allows you to send multiple search requests within a single HTTP request. The Multi-Search API...

Multi-index queries in Elasticsearch allow users to search for documents across several indices simultaneously. To query multiple indices...

This article will discuss best practices and advanced techniques for performing multi-index searches in Elasticsearch.

In Elasticsearch’s multi-tier architecture, the tiers are named hot, warm, cold & frozen. This Elasticsearch architecture allows better...

This guide discusses best practices and performance optimization techniques when working with multiple indexes in Elasticsearch.

Elasticsearch nested aggregation is a powerful technique for analyzing complex data structures that contain nested documents. To use...

An Elasticsearch nested query is used to search for documents containing specific criteria within these nested objects. To implement...

This guide focuses on how to handle not indexed fields in Elasticsearch, including use cases, how to configure them, and examples.

Here's how to create not null queries in Elasticsearch, which will help you find documents with existing or missing fields.

Elasticsearch currently provides 3 different techniques for fetching many results: Pagination, Search-After and Scroll. To learn how to...

An Elasticsearch PARTIAL snapshot error indicates that one or more index shard snapshots could not be taken. To resolve this...

The Elasticsearch percolate query is a unique and valuable feature that allows users to perform reverse searches. To implement it...

In this guide, we'll walk through an example of using the Elasticsearch Python client, Elasticsearch-py, to index & search documents.

This guide explores best practices for constructing Elasticsearch queries, focusing on the Query DSL (Domain Specific Language). To create...

This guide explains how to construct and optimize queries for searching text in specific fields using Elasticsearch. To use

The Elasticsearch match_all query allows users to retrieve all documents within an index or multiple indices. To use Match_all...

Here are various techniques to achieve partial matching in Elasticsearch. First, use wildcards as they are...

In this article, we will discuss Elasticsearch releases. including versioning system, compatibility considerations, and upgrade strategies.

Role & user mapping is the process that links users from an external system to roles in Elasticsearch. The role mapping API can be...

By executing Elasticsearch rolling restarts with the help of the API, you can maintain high cluster availability & avoid downtime. To do..

Rollup jobs in Elasticsearch reduce old data storage costs by storing summaries of data for a given time period. Rollup examples include...

An Elasticsearch runtime field is a field evaluated at query time instead of indexing time, which allows to modify our schema at the...

Elasticsearch runtime fields with a type of lookup can retrieve field values from the associated indices using the fields parameter on...

Elasticsearch offers three types of suggesters: term suggesters, phrase suggesters & completion suggesters (autocomplete). Suggesters work...

Elasticsearch searchable snapshots allow data exploration. They can be controlled with ILM Policies or be manually mounted. To implement...

Shingles, also known as word N-grams, are a useful technique for improving the relevance of search results in Elasticsearch. Examples...

By using the Elasticsearch Split Index API, an existing index can be split to create a new index with extra primary shards. To do this...

The Elasticsearch sum aggregation allows you to calculate the sum of a numeric field for a set of documents. To implement...

The synthetic _source mode in Elasticsearch is used to configure an index so that it saves storage space & doesn't duplicate data. To setup...

The Elasticsearch task management API helps you manage long-running tasks. To create, monitor, cancel & retrieve the results of tasks...

The text analysis process is tasked with two functions: tokenization and normalization and is carried out by employing analyzers. When you...

A tokenizer decides how Elasticsearch will take a set of words and divide it into separated terms called “tokens”. To work with synonyms...

In this article, we will discuss various techniques and best practices for truncating fields in Elasticsearch. First...

Upsert in Elasticsearch allows you to perform both update and insert actions in a single request. To perform an upsert operation...

When upgrading to a new Elasticsearch version, you can use the feature migration APIs to avoid deprecation issues. These APIs simplify...

Wildcard queries in Elasticsearch allow users to search for documents containing specific patterns in their text fields

The Elasticsearch _disk_usage API helps get information about disk usage for each analyzed field of indices and/or data streams. To use it...

Adaptive replica selection is a process that prevents a distressed Elasticsearch node from delaying the response to queries. To enable it...

Cluster shard rebalancing and allocation are often confused with each other. If cluster shard rebalancing isn't enabled, then...

In Elasticsearch the term Fielddata is relevant when performing sorting and aggregations on text field. To set fielddata=true, you...

File descriptors are required to keep track of all the files Elasticsearch has open at any given time, as well as all network...

Elasticsearch Filters apply conditions inside the query to narrow down the matching results. A filter clause can be used used in...

When the “disk flood stage” threshold is exceeded on an Elasticsearch cluster, it will start to block core actions. To resolve this issue...

In Elasticsearch, flush is the process of permanently storing data onto the disk for all of the operations that have been stored in memory.

A high heap size in Elasticsearch will give your node more memory for indexing and search operations. However, your node also requires...

Heavy merges use CPU, memory and disk resources, which can slow down the cluster’s response speed. In order to fix...

Elasticsearch cluster pending tasks are updates to the cluster state that were initiated by a user or the cluster. To resolve, list the...

High CPU is often a symptom of other underlying issues. It should be fixed because a distressed node will slow query response time and...

High disk watermark is one of the various thresholds on your Elasticsearch cluster. Passing this threshold is a warning and you should not...

A high number of tasks in management queue can cause Elasticsearch cluster instability which could result in data loss. To resolve...

By analyzing your slow logs, you can understand why searches are slow and how to optimize them. To enable slow logging in Elasticsearch...

Finding the right number of shards for your Elasticsearch indices, and the right size for each shard depends on many factors, including...

Follow these steps to configure all Elasticsearch node role types (master, data, coordinating, ingest, machine learning, remote eligible...

Mappings are the core element of index creation in Elasticsearch. Defining them correctly can improve performance. Mapping types include...

Analyzing search slow logs can provide users with advanced insights like the number of costly queries, reasons why queries were costly, so...

When facing recurring red status events in Elasticsearch, like "high cluster pending tasks", you need to investigate the cause & resolve...

There are multiple ways to improve your Elasticsearch aggregation performance. First, you should limit the scope by filtering documents...

By optimizing and maintaining Elasticsearch search speed, you can improve your product’s user experience. Here's how to speed up search...

There are 2 methods to increase the primary shard count in Elasticsearch: _reindex API & the _split API. Before using either method, you…

Ingest pipelines sit within the Elasticsearch node and will perform a set of alterations on your data that you...

To move data between Elasticsearch versions & clusters, you can: reindex from a remote cluster, use snapshots or Logstash. Transferring...

When migrating from ECK to OpenSearch Operator, you need to consider the hardware specifications & the YAML files. To easily migrate...

The join data type field allows users to establish parent-child relationships between documents in Elasticsearch. To use it, you need to...

Elasticsearch has many methods for defining relationships between documents, such as nested documents. To use the nested field type...

Object types in Elasticsearch can be used to define relationships between documents. Here's how to use the object field type for that purpose.

If you don’t have enough disk space available, Elasticsearch will stop allocating shards to the node. This will eventually prevent you from...

One of the most difficult issues to manage and resolve in Elasticsearch is poor search performance. This blog goes through clear steps to...

Follow these steps to list and restore dangling indices in Elasticsearch: (1) Run the dangling indices API & copy the...

When you have too many shards in your Elasticsearch cluster, there are a few steps you can take in order to reduce the number of shards...

Securing an Elasticsearch cluster and creating TLS certificates will require some downtime on your cluster. Here's how to create...

It's tricky to upgrade Elasticsearch without losing data & facing issues. Here's how to easily upgrade Elasticsearch from version 6 to 7.

Follow the steps listed in this guide to easily upgrade Elasticsearch from version 7 to version 8. First, prepare to upgrade your nodes by...

The 2 approaches for upgrading Elasticsearch versions are full cluster restarts & rolling restarts. Before making an Elasticsearch upgrade...

Follow the steps listed in this guide to easily upgrade your Elasticsearch from version 5 to version 6. First, check the...

How to create an Elasticsearch Index & what it is with a general overview - an index (plural: indices) contains a schema and can have

Index lifecycle management helps automate the creation, management & removal of an Elasticsearch index. Define the index lifecycle policy...

Once an indexing queue exceeds the maximum size, the Elasticsearch node will start rejecting index requests. To resolve this, check the...

Elasticsearch index templates allow us to create indices with user defined configuration. An index can pull the configuration from these...

Indexing is the process of adding or updating new documents to an Elasticsearch index. In its simplest form, you can index a document by...

The causes for indexing failure in Elasticsearch can be broken into 2 areas: index-related & node-related failures. To resolve...

This error occurs when the Elasticsearch cluster doesn't have a quorum of nodes with voting rights to elect a new master node. To resolve...

A saturated coordinating node could cause an increase in search or indexing response latency. This can be fixed by putting a load balancer...

Sometimes you can observe that the CPU and load on some of your data nodes is higher than on others. This can occasionally be caused by...

An overloaded master node may cause instability in the cluster. There are 3 ways to fix loaded master nodes: (1) Checking for...

Low disk watermark is one of the various thresholds on your Elasticsearch cluster. Here are possible actions you can take to resolve...

Elasticsearch Lucene or Apache Lucene is an open-source Java library used as a search engine. Elasticsearch is built on top of Lucene...

Mapping contains the properties of each field in the index. A common issue in Elasticsearch is an incorrectly defined mapping. Examples of...

An Elasticsearch cluster requires a master node to be identified in the cluster. Reasons why a master node is not discovered yet include...

If the max of shards per node is exceeded in Elasticsearch, shards can't be allocated. It is crucial to check if the limit is set at a...

The Elasticsearch process is very memory intensive. Here are the memory requirements and some tips to reduce your Elasticsearch memory usage.

Elasticsearch metadata refers to additional information stored for each document using metadata fields. Metadata fields can be customized...

It's possible to reduce the risk of accidental deletion of indices by preventing the use of wildcard for destructive operations. To check...

In this guide, we'll discuss the key Elasticsearch monitoring metrics that you should keep an eye on to ensure smooth operations.

Named queries allow you to label your queries with a name. Named queries can be utilized in a variety of use cases such as...

The node concurrent recoveries setting determines the max number of shards that can be recovered at once from each node. It's important to...

An Elasticsearch node can disconnect from a cluster for several reasons, including: excessive garbage collection from JVM, configuration...

There are different types of nodes in Elasticsearch. Each has its own role and purpose. Master, coordinating and data nodes differ...

Master nodes are responsible for actions such as creating or deleting indices. If you don't have enough master nodes, it could lead to...

Nested is a special object type that is indexed as a separate document. To demonstrate the use of Elasticsearch nested VS. object fields...

A large number of shards on an Elasticsearch cluster requires extra resources. Learn key ways to avoid and correct oversharding...

In Elasticsearch, Persistent refers to cluster settings that persist across cluster restarts. This setting is used in Cluster Update API...

Plugins in Elasticsearch are used to extend the functionality of Elasticsearch. An Elasticsearch plugin is installed and removed using the...

Queues in Elasticsearch exist in the context of Thread Pools. Queues are used to hold the pending requests for thread pools instead of...

Cluster rebalancing is the process by which an Elasticsearch cluster distributes data across the nodes. To force rebalance manually...

In Elasticsearch, recovery refers to the process of recovering an index or shard when something goes wrong. You can recover data by using...

Elasticsearch red status indicates not only that the primary shard has been lost, but also that a replica has not been promoted...

Elasticsearch requires a refresh operation to make indexed information available for search. You can set the refresh interval by...

To create & restore snapshots, you need to register a snapshot repository with every Elasticsearch node in the cluster. Here are the steps...

Reindex in Elasticsearch refers to copying existing data from a source index to a destination index. In some scenarios, the reindex API is...

There are a number of reasons why a search request can be rejected by the Elasticsearch cluster. To resolve the issue, you need to...

In Elasticsearch there are two types of shards: the primary shard & the replica copy. Each replica is located on a different node to ensure...

Elasticsearch replication refers to storing a redundant copy of the data. Elasticsearch creates 1 primary shard with a replication factor...

An Elasticsearch repository needs to be registered using the _snapshot endpoint. The supported repository types are: S3, HDFS, Azure...

Rest-high-level is built on top of low-level rest-client and is a method of communicating with Elasticsearch based on HTTP REST endpoints...

In Elasticsearch, restore refers to a snapshot restore mechanism. To restore a cluster from the snapshot, an index, or selected indices...

In Elasticsearch, routing refers to document routing. When you index a document, Elasticsearch will determine which shard will be used...

When scaling down Elasticsearch resources, it's important to consider several factors. Here's how to effectively plan to scale down.

Script regex is disabled in Elasticsearch by default, but you can decide to enable it. Regex must be used with care in painless scripts...

The Elasticsearch scroll API is useful when a search returns a large set of results. Large search results are exhaustive for the system...

To search in Elasticsearch, send a GET request to the _search endpoint in the search API. In the query phase and the fetch phase there are...

There are a number of possible causes for slow searches on particular nodes. To correct the issue and improve search performance, you...

This guide explores how to reduce Elasticsearch search latency based on a key study. The first lesson is to always...

An Elasticsearch cluster can start to reject search requests for several reasons. To resolve this, check the state of the thread pool and..

Setting up zone awareness for shard allocation ensures high availability in the case of several servers going down. Here's how to...

Elasticsearch settings can be configured on the cluster-level, node-level and index-level. Here's how to set up and optimize your settings...

Shard allocation is an algorithm by which Elasticsearch decides which unallocated shards should go on which nodes. To resolve unbalanced...

The number of shards is set when an index is created, and cannot be changed without reindexing. To handle unassigned Elasticsearch shards...

It is a best practice that Elasticsearch shard size should not go above 50GB for a single shard. If you go above this limit...

Here are the similarities and differences between Elasticsearch Snapshot Lifecycle Management (SLM) and OpenSearch Snapshot Management (SM).

If the indexing queue is high/causes timeouts, it hints that Elasticsearch nodes can't keep up with the indexing rate. To fix slow indexing...

There are several potential reasons for a slow query in Elasticsearch. Slow logs can be used to detect & troubleshoot slow queries issues...

An Elasticsearch snapshot is a backup of an index taken from a running cluster. It's better to use snapshots instead of disk backups due...

Elasticsearch split brain occurs when there is more than one master in the cluster. By setting the quorum of minimum master nodes...

A task is equivalent to an Elasticsearch operation, any request performed on an Elasticsearch cluster. The following commands are used...

An Elasticsearch template falls into one of these categories: index templates or search templates. Examples of index templates include...

In Elasticsearch, the Terms enum API looks for similarities in the index based on partial matches. To use the terms_enum API...

Elasticsearch threadpools are used to manage how requests are processed and to optimize the use of resources. The write threadpool...

The Elasticsearch Transform APIs can be used to turn existing indices into summarized indices. To create a transform API...

An Elasticsearch upgrade of an existing cluster can be done in 2 ways: through a rolling upgrade or a full cluster restart. To upgrade...

A version corresponds to the Elasticsearch built-in tracking system that tracks the changes in each document. By using _version...

There are at least three use cases where you should consider using transforms instead of aggregations in Elasticsearch. First, when the...

The popularity of Elasticsearch has made it a target for hackers. It's important to protect your cluster by enabling X-Pack Security...

Yellow status indicates that one or more of the replica shards on the Elasticsearch cluster are not allocated to a node. This could occur...

Zen discovery settings for cluster formation were deprecated in Elasticsearch V.7 and should be removed from version 7 and above due to...