Elasticsearch Mastering Timestamps in Elasticsearch

By Opster Team

Updated: Jul 23, 2023

| 2 min read

Introduction 

Timestamps are a crucial component in Elasticsearch, serving as a cornerstone for many operations such as time-based data analysis, log event management, and document versioning. This article delves into the advanced aspects of handling timestamps in Elasticsearch, including indexing, querying, and formatting.

Indexing Timestamps

To index a document with a custom timestamp, you can simply include a date field in your document. Elasticsearch will automatically recognize and map ISO 8601 date-time formats. For example:

`json
PUT /my_index/_doc/1
{
  "@timestamp": "2023-01-01T00:00:00Z",
  "message": "New year, new beginnings!"
}

Querying Timestamps

Elasticsearch provides a range of query types for dealing with date fields. The `range` query is particularly useful for timestamp-based searches. For instance, to find all documents indexed in June 2023, you could use:

json
GET /my_index/_search
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "2023-06-01T00:00:00Z",
        "lt": "2023-07-01T00:00:00Z"
      }
    }
  }
}

Date Math and Rounding

Elasticsearch supports date math expressions, which can be used to calculate relative dates. For example, to find all documents indexed in the last 7 days, you could use:

json
GET /my_index/_search
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-7d/d",
        "lt": "now/d"
      }
    }
  }
}

In this query, `now-7d/d` translates to “7 days ago, rounded down to the start of the day”, and `now/d` translates to “now, rounded down to the start of the day”.

Formatting Timestamps

Elasticsearch allows you to customize the format of date fields using the `date` mapping parameter. For instance, to index a document with a timestamp in Unix time format, you could define a mapping like this:

json
PUT /my_index
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date",
        "format": "epoch_second"
      }
    }
  }
}

Then, you could index a document with a Unix timestamp like this:

json
PUT /my_index/_doc/1
{
  "@timestamp": 1640995200,
  "message": "Happy New Year!"
}

Conclusion 

In conclusion, mastering timestamps in Elasticsearch involves understanding how to index, query, and format date fields. By leveraging these capabilities, you can perform powerful time-based data analysis and event management operations.