Elasticsearch not indexed field

Introduction

In Elasticsearch, fields can be indexed or not indexed depending on the use case and requirements. Not indexed fields are those fields that are present in the documents but are not searchable or aggregatable. This article will focus on handling not indexed fields in Elasticsearch, including use cases, how to configure them, and how to query documents containing not indexed fields.

Use Cases for Not Indexed Fields

There are several reasons why you might want to use not indexed fields in Elasticsearch:

1. Storage optimization: Not indexing fields can save storage space, as Elasticsearch does not need to store the inverted index for these fields.
2. Performance improvement: By not indexing certain fields, you can reduce the indexing time and improve the overall performance of your Elasticsearch cluster.
3. Sensitive data: You might have fields containing sensitive data that should not be searchable or aggregatable.

Configuring Not Indexed Fields

To configure a field as not indexed, you need to set the “index” property of the field mapping to “false”. Here’s an example of how to create an index with a not indexed field:

PUT /my_index
{
  "mappings": {
    "properties": {
      "title": {
        "type": "text"
      },
      "description": {
        "type": "text"
      },
      "not_indexed_field": {
        "type": "keyword",
        "index": false
      }
    }
  }
}

In this example, the “not_indexed_field” is of type “keyword” and is not indexed.

Querying Documents with Not Indexed Fields

Since not indexed fields are not searchable, you cannot use them in query clauses like “match” or “term”. However, you can still retrieve these fields when fetching documents using the “_source” parameter. Here’s an example of how to fetch a document with a not indexed field:

GET /my_index/_search
{
  "_source": ["title", "description", "not_indexed_field"],
  "query": {
    "match": {
      "title": "example"
    }
  }
}

In this example, the search query matches documents based on the “title” field, and the response includes the not indexed field in the “_source” object.

Updating Not Indexed Fields

You can update not indexed fields using the Update API or the Bulk API. Here’s an example of how to update a not indexed field using the Update API:

POST /my_index/_update/1
{
  "doc": {
    "not_indexed_field": "new_value"
  }
}

In this example, the “not_indexed_field” of the document with ID “1” is updated to “new_value”.

Conclusion

Not indexed fields in Elasticsearch can be useful in certain scenarios, such as optimizing storage and performance or handling sensitive data. By configuring the field mapping and using the appropriate APIs, you can effectively manage and retrieve not indexed fields in your Elasticsearch cluster.

Opster AutoOps and the Opster Support Team can assist you in resolving issues related to Elasticsearch fields not being indexed. By analyzing your setup and configurations, they can identify the root cause and provide tailored solutions to ensure proper indexing of your fields.