Before you begin reading this guide, we recommend you try running the Elasticsearch Error Check-Up which can resolve issues that cause many errors.
This guide will help you check for common problems that cause the log ” Invalid alias filter ” to appear. It’s important to understand the issues related to the log, so to get started, read the general overview on common issues and tips related to the Elasticsearch concepts: filter, search and alias.
Advanced users might want to skip right to the common problems section in each concept or try running the Check-Up which analyses ES to pinpoint the cause of many errors and provides suitable actionable recommendations how to resolve them (free tool that requires no installation).
Filter in Elasticsearch
A filter in Elasticsearch is all about applying some conditions inside the query that are used to narrow down the matching result set.
What it is used for
When a query is executed, Elasticsearch by default calculates the relevance score of the matching documents. But in some conditions it does not require scores to be calculated, for instance if a document falls in the range of two given timestamps. For all these Yes/No criteria, a filter clause is used.
Examples
Return all the results of a given index that falls between a date range:
GET my_index/_search
{
"query": {
"bool": {
"filter": {
"range": {
"created_at": {
"gte": "2020-01-01",
"lte": "2020-01-10"
}
}
}
}
}
}
Notes
- Queries are used to find out how relevant a document is to a particular query by calculating a score for each document, whereas filters are used to match certain criteria and are cacheable to enable faster execution.
- Filters do not contribute to scoring and thus are faster to execute.
- There are major changes introduced in Elasticsearch version 2.x onward related to how query and filters are written and performed internally.
Common Problems
- The most common problem with filters is incorrect use inside the query. If filters are not used correctly, query performance can be significantly affected. So filters must be used wherever there is scope of not calculating the score.
- Another problem often arises when using date range filters, if “now” is used to represent the current time. It has to be noted that “now” is continuously changing the timestamp and thus Elasticsearch cannot use caching of the response since the data set will keep changing.
Search in Elasticsearch
What it is:
Search refers to the searching of documents in an index or multiple indices. The simple search is just a GET request to the _search endpoint. The search query can either be provided in query string or through a request body.
Examples:
When looking for any documents in this index, if search parameters are not provided, every document is a hit and by default 10 hits will be returned.
GET my_documents/_search
A JSON object is returned in response to a search query. A 200 response code will mean the request completed successfully.
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.0,
"hits" : [
...
]
}
}Notes:
- Distributed search is challenging and every shard of the index needs to be searched for hits, and then those hits are combined into a single sorted list as a final result.
- There are two phases of search: the query phase and the fetch phase.
- In the query phase, the query is executed on each shard locally and top hits are returned to the coordinating node. The coordinating node merges the results and creates a global sorted list.
- In the fetch phase, the coordinating node brings the actual documents for those hit IDs and returns them to the requesting client.
- A coordinating node needs enough memory and CPU in order to handle the fetch phase.
Alias in Elasticsearch
In Elasticsearch, an alias is a secondary name to refer to one or more indices. Aliases can be created and removed dynamically using _aliases REST endpoint.
What it is used for
Aliases are used for multiple purposes such as to search across more than one index with a single name, perform the reindexing process with zero downtime and query data based on predefined filters.
Examples
Creating an alias on a single index:
POST /_aliases?pretty
{
"actions": [
{
"add": {
"index": "index_1",
"alias": "alias_1"
}
}
]
}
Creating the same alias on more than one index:
POST /_aliases?pretty
{
"actions": [
{
"add": {
"index": "index_1",
"alias": "alias_1"
}
},
{
"add": {
"index": "index_2",
"alias": "alias_1"
}
}
]
}
Creating a filter-based alias:
POST /_aliases?pretty
{
"actions": [
{
"add": {
"index": "index_1",
"alias": "alias_2",
"filter": {
"term": {
"organization": "opster"
}
}
}
}
]
}
Listing out all of the available aliases in an Elasticsearch cluster:
GET _cat/aliases
Removing an alias:
POST /_aliases?pretty
{
"actions": [
{
"remove": {
"index": "index_2",
"alias": "alias_1"
}
}
]
}
Notes
- An Alias cannot be used for the indexing process if it points to more than one index. If attempted, Elasticsearch will throw an exception.
- Deleting an alias does not delete the actual index.
Common Problems
- Since a single alias can be attached to multiple indices, it is important to keep in mind that querying an alias will send the query against all the associated indices and this can impact the search performance and high resource utilization.
Log Context
Log”Invalid alias filter”classname is ShardSearchRequest.java We extracted the following from Elasticsearch source code for those seeking an in-depth context :
return null;
}
try {
return filterParser.apply(alias.filter().uncompressed());
} catch (IOException ex) {
throw new AliasFilterParsingException(index; alias.getAlias(); "Invalid alias filter"; ex);
}
};
if (aliasNames.length == 1) {
AliasMetadata alias = aliases.get(aliasNames[0]);
if (alias == null) {
