Log Security index is unavailable. short circuiting retrieval of user – How To Solve Related Issues


Log Security index is unavailable. short circuiting retrieval of user – How To Solve Related Issues

Opster Team

April-20, Version: 1.7-8.0

 

This log could be avoided if detected earlier. Before you read this guide, we recommend you run the Elasticsearch Error Check-Up which detects issues in ES that cause log errors. The Check-Up includes checks that would help you prevent log “Security index is unavailable. short circuiting retrieval of user”. It’s a free tool that requires no installation and takes 2 minutes to complete. You can run the Check-Up here.

What the Security index is:

From Elasticsearch version 6.8 and onwards, the Security feature is available for free. This means you can secure your cluster by creating multiple users and roles, and all of this information is stored in a unique index called .security<es-major-version>.

Please note the dot ‘.’ at the beginning of the index name.

What this error means:

Elasticsearch index can have several states, and sometimes due to several factors, it can become unavailable, for instance because of missing primary shards, an Elasticsearch cluster running out of disk space and so on. When Elasticsearch needs to read the user information for a request, several steps occur internally. For example, the request “get user API”, which looks like this:

 GET /_security/user/ 

Note that `_security` is the index name used for the security API call. This is an API that would require Elasticsearch to find the information stored in the security index. The following things happen internally to figure out the user information (its id, role, permission etc):

  1. Elasticsearch freezes the security index, so others can’t update the security index when it’s reading the sensitive (security) information.
  2. Elasticsearch checks if the security index is available or not.
  3. If the index isn’t available, then there is no point of querying the security index and short circuiting the query part, and it logs this as an error message as below:
 security index is unavailable. short circuiting retrieval of user. 

Quick troubleshooting steps:

  1. Check if the `.security` index exists or not, by using below _cat/indices?v API and if the index exists, the output of this API would look like:
     health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open .security-7 9blPln4uSKScEzWtMfJXNA 1 0 7 0 24.3kb 24.3kb 

  2. Check if the security index is available, because there is no direct API that can show this. Still, as mentioned earlier, cluster RED state or disk space can cause an index to become unavailable, and checking and fixing these issues will help make the index available.

This Opster Guide can help identify and fix issues caused by low disk space.

Log context

Log “Security index is unavailable. short circuiting retrieval of user [{}]” classname is NativeUsersStore.java
We extracted the following from Elasticsearch source code for those seeking an in-depth context :

         final SecurityIndexManager frozenSecurityIndex = securityIndex.freeze();
        if (frozenSecurityIndex.isAvailable() == false) {
            if (frozenSecurityIndex.indexExists()) {
                logger.trace("could not retrieve user [{}] because security index does not exist"; user);
            } else {
                logger.error("security index is unavailable. short circuiting retrieval of user [{}]"; user);
            }
            listener.onResponse(null);
        } else {
            securityIndex.checkIndexVersionThenExecute(listener::onFailure; () ->
                    executeAsyncWithOrigin(client.threadPool().getThreadContext(); SECURITY_ORIGIN;




 

Optimize Elasticsearch Performance

Try The Tool