Before you dig into reading this guide, have you tried asking OpsGPT what this log means? You’ll receive a customized analysis of your log.
Try OpsGPT now for step-by-step guidance and tailored insights into your OpenSearch operation.
Briefly, this error occurs when the metadata blob for a repository in OpenSearch exceeds the maximum limit of 5MB. This could be due to a large number of snapshots or indices in the repository. To resolve this, you can create a new repository and start taking snapshots there. Alternatively, you can delete old or unnecessary snapshots from the current repository to reduce the size of the metadata blob. Remember to always maintain a backup of your data to prevent any loss during this process.
For a complete solution to your to your search operation, try for free AutoOps for Elasticsearch & OpenSearch . With AutoOps and Opster’s proactive support, you don’t have to worry about your search operation – we take charge of it. Get improved performance & stability with less hardware.
This guide will help you check for common problems that cause the log ” Your repository metadata blob for repository [{}] is larger than 5MB. Consider moving to a fresh ” to appear. To understand the issues related to this log, read the explanation below about the following OpenSearch concepts: blobstore, repositories, metadata, repository.
Overview
Metadata in OpenSearch refers to additional information stored for each document. This is achieved using the specific metadata fields available in OpenSearch. The default behavior of some of these metadata fields can be customized during mapping creation.
Examples
Using _meta meta-field for storing application-specific information with the mapping:
PUT /my_index?pretty
{
"mappings": {
"_meta": {
"domain": "security",
"release_information": {
"date": "18-01-2020",
"version": "7.5"
}
}
}
}Notes
- Meta fields available include: _index, _type, _id, _source, _size, and _field_name.
Log Context
Log “Your repository metadata blob for repository [{}] is larger than 5MB. Consider moving to a fresh” classname is BlobStoreRepository.java.
We extracted the following from OpenSearch source code for those seeking an in-depth context :
final int len = serialized.length();
if (len > ByteSizeUnit.KB.toBytes(500)) {
logger.debug("Not caching repository data of size [{}] for repository [{}] because it is larger than 500KB in" +
" serialized size"; len; metadata.name());
if (len > ByteSizeUnit.MB.toBytes(5)) {
logger.warn("Your repository metadata blob for repository [{}] is larger than 5MB. Consider moving to a fresh" +
" repository for new snapshots or deleting unneeded snapshots from your repository to ensure stable" +
" repository behavior going forward."; metadata.name());
}
// Set empty repository data to not waste heap for an outdated cached value
latestKnownRepositoryData.set(null);
