Elasticsearch How to Convert a Watch to an ElastAlert

By Opster Team

Updated: Aug 13, 2023

| 4 min read

Quick links:


In this guide, we will explain what Watcher and ElastAlert are and give the pros and cons of each. We will then show what it looks like to convert a watch to an ElastAlert. This guide gives a very basic example to illustrate that process; follow-up pieces will explore more complicated transitions.

Intro to Watcher alerting

In its simplest form, Watcher is able to search Elasticsearch indices to check for certain conditions and alert us through a variety of mediums if those conditions have been met. Let’s look at the basic anatomy of a watch before turning to the pros and cons of Watcher.

The basic anatomy of a watch

  • Trigger: when the watch should run
  • Input: which data to look at, including, but not limited to, Webhooks and Elasticsearch indices
  • Condition: should the watch send an alert
  • Transform: changes the data to a more readable/understandable format
  • Actions: what should Watcher do with the alert (e.g., it can be written to the logs or sent to a number of systems, including, but not limited to, Webhooks, email, and Slack).

The complexity of a watch varies from checking if a field is above a certain value to being as complicated as you can imagine. However, Watcher is usually used for checking static values, meaning that users will find predictive models very difficult to implement. 


  • Built into Elasticsearch
  • UI component in Kibana
  • You will most likely design in Kibana, then automate through the APIs


  • If you are using just this Platinum/Enterprise feature, then the license can become restrictively expensive when sending external alerts for large clusters
  • For more complicated conditions and transforms, you will need knowledge of the Painless scripting language

Intro to ElastAlert

ElastAlert is one of the many alternatives to Watcher. Just like Watcher, in its simplest form, it can search Elasticsearch indices to check for certain conditions and alert us through a variety of mediums if those conditions have been met. Now, let’s look at the basic anatomy of an ElastAlert before turning to its pros and cons. 

The basic anatomy of an ElastAlert

  • config.yml: some of the things in the config are the Elasticsearch URL, the rules folder, how often to run the rules, and the writeback_index, which has the status of ElastAlert
  • Rules folder: contains one or more rules to run
  • Rules: set out what to check for and, if the condition is met, where to send the event
  • Optional files: such as auth to send data.

There are a number of prebuilt rule types (`any`, `change`, `frequency`, `flatline`, etc.) that limit the flexibility of ElastAlert, but you do have the ability to create custom rules. 


  • Free
  • Default rules are shipped with ElastAlert and these are simple to configure


  • Knowledge of Python is needed for the more complicated custom rules
  • ElastAlert must be run separately since it is not built into Elasticsearch or Kibana.

Reasons you may want to convert a Watch to an ElastAlert

There are several reasons why you might want to convert from Watcher to ElastAlert. The four main ones are listed below:

  1. Cost reduction because ElastAlert is open source and free
  2. Painless is a restricted set of Java APIs (with an annoying learning curve), meaning you cannot do anything you want with it
  3. With Python you can do anything you want in ElastAlert, with no limitations; the rules can become very complicated, and you have access to any packages and methods you may want to use to make your life easier
  4. Finally, you might want to convert to ElastAlert as a preparation step before migrating to OpenSearch.


There is a lot of documentation about setting up watches and ElastAlerts. So, here, we will focus on a specific example of converting a watch to an ElastAlert.

Sending threshold alerts to Gmail


PUT _watcher/watch/logs-threshold
  "trigger": {
    "schedule": {
      "interval": "5m"
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
        "rest_total_hits_as_int": true,
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "{{ctx.trigger.scheduled_time}}||-5m",
                    "lte": "{{ctx.trigger.scheduled_time}}",
                    "format": "strict_date_optional_time||epoch_millis"
  "condition": {
    "script": {
      "source": "if (ctx.payload.hits.total > params.threshold) { return true; } return false;",
      "lang": "painless",
      "params": {
        "threshold": 1000
  "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "to": [
        "subject": "Watch logs-threshold has exceeded the threshold",
        "body": {
          "text": "logs index has received more than 1000 documents over the last 5 minutes"

To send the email, you will also need to set up an email account.

Add the following to your elasticsearch.yml file on every node:

        profile: gmail
            auth: true
            starttls.enable: true
            host: smtp.gmail.com
            port: 587
            user: person@example.com

And your password will be stored in the Elasticsearch keystore on every node.

bin/elasticsearch-keystore add xpack.notification.email.account.gmail_account.smtp.secure_password

The instructions are slightly different for other email providers. For Gmail, you will need to generate an app password to store in your keystore. Instructions on how to do that can be found here. These same instructions are needed for ElastAlert.


Feel free to install this via pip and run it locally. However, we recommend using a docker image because all dependencies are already installed there and there are usually fewer issues.

The directory structure is as follows:


 ├── rules/

 │        └── frequency.yml

 ├── config.yml

 ├── docker-compose.yml

 └── google-auth.yml


name: "logs-threshold"
type: "frequency"
index: "logs"
is_enabled: true
num_events: 1000
    minutes: 5
timestamp_field: "@timestamp"
alert_subject: "Watch logs-threshold has exceeded the threshold"
alert_text: "logs index has received more than 1000 documents over the last 5 minutes"
    - query:
          match_all: {}

    - "email"
    - "person@example.com"
smtp_host: "smtp.gmail.com"
smtp_port: 587
smtp_auth_file: "/opt/elastalert/google-auth.yml"
rules_folder: /opt/elastalert/rules
  minutes: 1
  minutes: 15
es_host: host.docker.internal
es_port: 9200
writeback_index: elastalert_status

`docker-compose.yml` can easily be converted to a run command:

version: '3.3'
    container_name: elastalert
    restart: always
      - '$PWD/config.yaml:/opt/elastalert/config.yaml'
      - '$PWD/rules:/opt/elastalert/rules'
      - '$PWD/google-auth.yml:/opt/elastalert/google-auth.yml'
    image: jertel/elastalert2


user: "person@example.com"
password: "****************"


It is fairly easy to convert to ElastAlert, and with some basic knowledge of Python, you can implement a lot of the simple use cases. The basic use cases are limited though, so any complicated watches will take time to convert and could be difficult. But if Watcher is the only feature you are using, as is the case with many users, then there really should be no reason why you should not switch over to ElastAlert to reduce your licensing costs.

How helpful was this guide?

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?