Briefly, this error occurs when Elasticsearch fails to generate a service account token for the elastic/kibana service. This could be due to insufficient permissions, incorrect configuration, or an unavailable Elasticsearch cluster. To resolve this issue, you can: 1) Verify the user has the necessary permissions to create a service account token. 2) Check the Elasticsearch configuration for any errors. 3) Ensure the Elasticsearch cluster is up and running properly. 4) If the problem persists, consider restarting the Elasticsearch service.
Before you dig into reading this guide, have you tried asking OpsGPT what this log means? You’ll receive a customized analysis of your log.
Try OpsGPT now for step-by-step guidance and tailored insights into your Elasticsearch/OpenSearch operation.
Briefly, this error message can occur when attempting to authenticate with Elasticsearch using a Kibana service account. The cause of this error may be due to an incorrect username or password, or the account may not have sufficient permissions to perform the desired action. To resolve this issue, check that the credentials are correct and that the service account has the appropriate permissions.
For a complete solution to your to your search operation, try for free AutoOps for Elasticsearch & OpenSearch . With AutoOps and Opster’s proactive support, you don’t have to worry about your search operation – we take charge of it. Get improved performance & stability with less hardware.
This guide will help you check for common problems that cause the log ” Failed to create token for the elastic/kibana service account ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: plugin.
Introduction
Integrating external services with Elasticsearch is now easier with Elasticsearch Service Accounts and Tokens. Users can utilize this feature to connect and integrate relevant applications with Elasticsearch. Here, the main rule in Elasticsearch Service Accounts is to create credentials/tokens.
Useful background information
How to get a service account
The following API returns a list of all available service accounts:Â
Get _security/serviceÂ
Default Service AccountsÂ
There are some default Elasticsearch service accounts that users can use for any integration process:
- Elastic/kibana
- Elastic/fleet-server
- Elastic/enterprise-search-server
Note:
Each token must have a unique name and include access credentials.
Service Account Concept
Each service account can have a number of individual service tokens associated with it, so multiple instances of each service can each have their own token without having to share tokens.
What is the Difference Between Service Accounts and API Keys in Elasticsearch?
An API key sends requests on behalf of a user with restricted privileges or expiration time, while a Service Account uses service tokens and maintains a fixed set of privileges. Service accounts cannot be authenticated without service tokens.
Creating Service Tokens
Users can create a service token in two ways:
1- Elasticsearch Service Token (CLI)
2- Service Account Token API
Both methods create a token that’s 22 characters long. It is also important to note that service tokens must be at least 10 characters in length.
Creating a Service Account Token Using CLIÂ
./elasticsearch-service-tokens create elastic/kibana my-token1
Note:Â
If tokens are being generated with CLI tools, tokens can only be used on the node where they are generated. Users would need to copy the security file to other nodes to enable authentication throughout the cluster. However, if users utilize the API to create a token, the token will be stored in the .security index and can be used throughout the cluster without any issues.
Creating a Service Account Token Using API
Creating a token with a random name:Â
POST /_security/service/elastic/kibana/credential/token/
Creating a token with a specific name:
POST /_security/service/elastic/kibana/credential/token/token-name
Show all created token/credentials:
GET _security/service/elastic/kibana/credential
How to Delete a Service Account TokenÂ
DELETE /_security/service/elastic/kibana/credential/token/token_name
Possible Issues When Creating Tokens & Solutions
Users may encounter errors, which are explored below:
Failed to Create Token: Failed to Load Service_token
Failed to load service_tokens file [/etc/elasticsearch/service_tokens]
.AccessDeniedException: /etc/elasticsearch/service_tokens
After creating a token in CLI, it’s possible users may face this issue while accessing the service token file.
Solution: Use this command to give the file the required access. However, always be mindful when giving access and review periodically to ensure that this is ideal for you and your system.
Chmod 777 /etc/elasticsearch/security_tokensÂ
OrÂ
Chmod 744 /etc/elasticsearch/service_tokens
Failed to Create Token: Unknown Service Account PrincipalÂ
./elasticsearch-service-tokens create /elastic/kibana myt1
ERROR: Unknown service account principal: [/elastic/kibana]. Must be one of [elastic/fleet-server,elastic/kibana]
Since permissions for service tokens are hard-coded, users can only use these for a predefined set of service accounts. If this particular issue is encountered, check the name of the service account used. For example, here instead of elastic/kibana /elastic/kibana was used, which is not available in Elasticsearch Service Accounts.
Failed to Create Token: Invalid Service Token Name
Service account token names must have at least 1 character and at most 256 alphanumeric (A-Z, a-z, 0-9), hyphen (-), or underscore (_) characters. They must not begin with an underscore (_).
Failed to Create Token: Version_conflict_engine_exception
If users try to create a token with an existing token name, a version conflict engine exception will be raised.
Log Context
Log “Failed to create token for the [elastic/kibana] service account” class name is TransportKibanaEnrollmentAction.java. We extracted the following from Elasticsearch source code for those seeking an in-depth context :
logger.debug(
"Successfully created token [{}] for the [elastic/kibana] service account during kibana enrollment";
response.getName()
);
listener.onResponse(new KibanaEnrollmentResponse(response.getName(); response.getValue(); httpCa));
}; e -> listener.onFailure(new ElasticsearchException("Failed to create token for the [elastic/kibana] service account"; e))));
}
} protected static String getTokenName() {
final ZonedDateTime enrollTime = ZonedDateTime.now(ZoneOffset.UTC);
