Elasticsearch Filter


Elasticsearch Filter

Opster Team

Nov 2020


In addition to reading this guide, run the Elasticsearch Health Check-Up. Detect problems and improve performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and many more.
Free tool that requires no installation with +1000 users.

Filter in Elasticsearch

A filter in Elasticsearch is all about applying some conditions inside the query that are used to narrow down the matching result set.

What it is used for:

When a query is executed, Elasticsearch by default calculates the relevance score of the matching documents. But in some conditions it does not require scores to be calculated, for instance if a document falls in the range of two given timestamps. For all these Yes/No criteria, a filter clause is used.

Examples:

Return all the results of a given index that falls between a date range:

GET my_index/_search
{
  "query": {
    "bool": {
      "filter": {
        "range": {
          "created_at": {
            "gte": "2020-01-01",
            "lte": "2020-01-10"
          }
        }
      }
    }
  }
}

Notes:

  • Queries are used to find out how relevant a document is to a particular query by calculating a score for each document, whereas filters are used to match certain criteria and are cacheable to enable faster execution.
  • Filters do not contribute to scoring and thus are faster to execute.
  • There are major changes introduced in Elasticsearch version 2.x onward related to how query and filters are written and performed internally.

Common problems:

  • The most common problem with filters is incorrect use inside the query. If filters are not used correctly, query performance can be significantly affected. So filters must be used wherever there is scope of not calculating the score. 
  • Another problem often arises when using date range filters, if “now” is used to represent the current time. It has to be noted that “now” is continuously changing the timestamp and thus Elasticsearch cannot use caching of the response since the data set will keep changing.

Related log errors to this ES concept


Cannot install system call filter because JNA is not available
Unable to link C library. native methods privset will be disabled.
Unable to link C library. native methods seatbelt will be disabled.
Unable to link C library. native methods seccomp will be disabled.
Blocking operation due to expired license. Cluster health; cluster stats and indices stats n
Skipping ip filter rules for profile since the profile is not bound to any addresses
Reducing requested filter cache size of to the maximum allowed size of
Failed to add alias . filter
Failed to execute pipeline



Improve Elasticsearch Performance

Run The Analysis