Elasticsearch Flush, Translog and Refresh

Elasticsearch Flush, Translog and Refresh

Opster Team

March 2021


In addition to reading this guide, we recommend you run the Elasticsearch Health Check-Up. It will detect issues and improve your Elasticsearch performance by analyzing your shard sizes, threadpools, memory, snapshots, disk watermarks and more.

The Elasticsearch Check-Up is free and requires no installation.

Run the Elasticsearch check-up to receive recommendations like this:

checklist Run Check-Up
error

The following configuration error was detected on node 123...

error-img

Description

This error can have a severe impact on your system. It's important to understand that it was caused by...

error-img

Recommendation

In order to resolve this issue and prevent it from occurring again, we recommend that you begin by changing the configuration to...

1

X-PUT curl -H "Content-Type: application/json" [customized recommendation]

What is an Elasticsearch flush?

In Elasticsearch, flushing is the process of permanently storing data onto the disk for all of the operations that have temporarily been stored in memory. This is also known as a Lucene commit.

How are Elasticsearch documents indexed?

To understand the relevance of flushing, it is necessary to understand how Elasticsearch indexes documents.

As new documents are indexed, the operations are recorded on disk in the translog and stored in memory in a buffer. When the index refreshes, the documents in the buffer are added to a new Lucene segment which is also held in memory.  

Flushing is the process that stores the in-memory segments onto disk, and simultaneously closes the translog generation and starts with a new blank translog generation.

Flushing is a process which is carried out behind the scenes by Elasticsearch using heuristics to optimise the tradeoffs between memory usage and disk writes.  andFlushing normally requires no intervention by users, although a flush api is available.

POST /my-index-000001/_flush

In the event of a node crashing or restarting, then Elasticsearch will retrieve and flush any operations that were stored in the translog prior to the crash in order to ensure that data is not lost.

Run the Elasticsearch check-up to receive recommendations like this:

checklist Run Check-Up
error

The following configuration error was detected on node 123...

error-img

Description

This error can have a severe impact on your system. It's important to understand that it was caused by...

error-img

Recommendation

In order to resolve this issue and prevent it from occurring again, we recommend that you begin by changing the configuration to...

1

X-PUT curl -H "Content-Type: application/json" [customized recommendation]



Improve Elasticsearch Performance

Run The Analysis