How To Solve Issues Related to Log – Unknown blob in the repository:

Prevent Your Next ELK Incident

Try our free Check Up to test if your ES issues are caused from misconfigured settings

Prevent Issue

Updated: Jan-20

In-Page Navigation (click to jump) :

Opster Offer’s World-Class Elasticsearch Expertise In One Powerful Product
Try Our Free ES Check-Up   Prevent Incident

Troubleshooting background

To troubleshoot Elasticsearch log “Unknown blob in the repository:” it’s important to understand common problems related to Elasticsearch concepts: blobstore, repositories, repository-azure. See detailed explanations below complete with common problems, examples and useful tips.

Repository in Elasticsearch

What it is

An Elasticsearch snapshot provides a backup mechanism that takes the current state and data in the cluster and saves it to a repository (read the Glossary term Snapshot for more information). The backup process requires a repository to be created first. The repository needs to be registered using the _snapshot endpoint, and multiple repositories per cluster can be created. The following repository types are supported. 

Repository Types:
Repository TypeConfiguration Type
Shared file systemType: “fs”
S3Type : “s3”
HDFSType :“hdfs”
AzureType: “azure”
Google Cloud StorageType : “gcs”
Examples

To register a repository of type fs:

PUT _snapshot/my_repo_01
{
"type": "fs",
"settings": {
"location": "/mnt/my_repo_dir"
  }
}
Notes and common problems
  • S3, HDFS , Azure and Google Cloud requires a relevant plugin to be installed before it can be used for a snapshot.
  • The setting, path.repo: /mnt/my_repo_dir needs to be added to elasticsearch.yml on all the nodes in case you are planning to use repo type of file system otherwise it will fail
  • In case of using remote repositories , the network bandwidth and repository storage throughput should be high enough to complete the snapshot operations normally , otherwise you will end up in partial snapshots.


To help troubleshoot related issues we have gathered selected Q&A from the community and issues from Github , please review the following for further information :

1 Can’t register a snapshot repository in Elasticsearch 3.41 K 2

2Cannot Register Snapshot Repository  


Log Context

Log ”[{}] Unknown blob in the repository: {}” classname is BlobStoreRepository.java
We have extracted the following from Elasticsearch source code to get an in-depth context :

                 final long curr = Long.parseLong(blobName.substring(INDEX_FILE_PREFIX.length()));
                latest = Math.max(latest; curr);
            } catch (NumberFormatException nfe) {
                // the index- blob wasn't of the format index-N where N is a number;
                // no idea what this blob is but it doesn't belong in the repository!
                logger.warn("[{}] Unknown blob in the repository: {}"; metadata.name(); blobName);
            }
        }
        return latest;
    }







About Opster

Opster identifies and predicts root causes of Elasticsearch problems, provides recommendations and can automatically perform various actions to prevent issues, optimize performance and save resources.

Learn more: Glossary | Blog| Troubleshooting guides | Error Repository

Need help with any Elasticsearch issue ? Contact Opster