Elasticsearch Fielddata

Elasticsearch Fielddata

Opster Team

Nov 2020


In addition to reading this guide, run the Elasticsearch Health Check-Up. Detect problems and improve performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and many more.
Free tool that requires no installation with +1000 users.

Run the Elasticsearch check-up to receive recommendations like this:

checklist Run Check-Up
error

The following configuration error was detected on node 123...

error-img

Description

This error can have a severe impact on your system. It's important to understand that it was caused by...

error-img

Recommendation

In order to resolve this issue and prevent it from occurring again, we recommend that you begin by changing the configuration to...

1

X-PUT curl -H "Content-Type: application/json" [customized recommendation]

Overview

In Elasticsearch the term fielddata is relevant when sorting and doing aggregations (similar to SQL GROUP BY COUNT and AVERAGE functions) on text fields.  

For performance reasons, there are some rules as to the kinds of fields that can be aggregated. You can group by any numeric field but for text fields, which have to be of keyword type or have fielddata=true since they don’t support doc_values (Doc values are the on-disk inverted index data structure, built at document indexing time, which makes aggregations possible). 

Fielddata is an in-memory data structure used by text fields for the same purpose. Since it uses a lot of heap size it is disabled by default.

Examples

The following PUT mapping API call will enable Fielddata on my_field text field.

PUT my_index/_mapping{"properties":{"my_field":{"type":"text","fielddata":true}}}

Notes

  • As field-data is disabled by default on text fields, in case of an attempt to aggregate on a text field with field-data disabled, you would get the following error message:
    “Fielddata is disabled on text fields by default. Set `fielddata=true` on [`your_field_name`] in order to load field data in memory by uninverting the inverted index. Note that this can however, use “significant memory.” – if this happens you can either enable the field-data on that text field, or choose another way to query the data (again, because field-data consumes a lot of memory and is not recommended).

Run the Elasticsearch check-up to receive recommendations like this:

checklist Run Check-Up
error

The following configuration error was detected on node 123...

error-img

Description

This error can have a severe impact on your system. It's important to understand that it was caused by...

error-img

Recommendation

In order to resolve this issue and prevent it from occurring again, we recommend that you begin by changing the configuration to...

1

X-PUT curl -H "Content-Type: application/json" [customized recommendation]


Related log errors to this ES concept


Failed to call listener on field data cache unloading
Unable to estimate memory overhead
Failed to call listener on atomic field data loading
Failed to call listener on global ordinals loading
Failed to find format
Reducing requested field data cache size of to the maximum allowed size of



Improve Elasticsearch Performance

Run The Analysis