Managing Elasticsearch with Elastic Cloud – What Does Elastic Provide

Managing Elasticsearch with Elastic Cloud – What Does Elastic Provide

Managed Elasticsearch – What does Elastic provide?

In this blog we will be covering what Elastic’s managed service covers and what it doesn’t. If you’re wondering whether to run self-hosted or look for a different solution, you can read about different hosting considerations in this blog

What does Elastic cover?

Elastic ECK or Elastic Cloud offer a convenient way to manage your Elasticsearch infrastructure. In addition, you get access to features on top of the managed service that can be used for specific use cases.

Many companies who turn to Elastic Cloud to manage their Elasticsearch deployment have high expectations of the service, hoping to transfer complete ownership of their deployment over to Elastic. However, Elastic themselves have a clear breakdown on their website regarding which elements of operations they cover, and which they don’t. They call this “shared responsibility”, meaning that some responsibilities remain solely on the customer.

Image from elastic.co

Elastic Cloud and ECK cover the infrastructure, orchestration and platform & networking aspects of deployments. Everything else is up to the customer: running operations, managing data and configuring security controls. Each company needs to make their own decisions regarding configuration and ascertain that their deployment can successfully meet their business needs.

Among the responsibilities that Elastic says they leave to the customer are some complicated and often confusing aspects of operations. Many companies end up struggling to successfully manage these elements on their own, especially as operations scale.

Expectations VS. Reality of managing Elasticsearch

Users often start out self-managed before moving to managed services like Elastic Cloud.

Managed services, and in this case specifically Elastic, structure their business in such a way that they purposely do not take care of the application layer. The application layer includes: taking care of ingesting data, configuring and maintaining data structure, optimizing shards, avoiding latency, preventing incidents, improving performance and more.

Managed services don’t provide proactive support, and they take no responsibility for the way you configure your data. This last part makes perfect sense, seeing as it would be out of scope for a managed service to be responsible for the data you want to put in your system. Their agenda is not to intervene with the way you decide to structure, configure and manage your data.

Elastic’s support can be helpful, especially if you have platinum support, as they will try to help with other sections from the application layer. However, this type of support is limited to “break-fix”, meaning they will only help you in a crisis when something isn’t working properly, as opposed to actively trying to optimize and improve.

You can also buy consulting packages that can be very helpful, but they are on an ad-hoc basis and can get very pricey.

Self-Managed – Expectations
Self-Managed – Reality
Ensure that mission-critical applications are running at peak performance and stable
Generic monitoring tools are not enough to operate successfully
Keep costs low – by avoiding payments to outside providers for support and services
Skilled DevOps engineers are hard to find. They then need to develop & maintain internal expertise tooling for multiple technologies. Hardware & employee costs pile up
Ensure that mission-critical applications are running at peak performance and stable
Keep costs low – by avoiding payments to outside providers for support and services
Ensure that mission-critical applications are running at peak performance and stable
Keep costs low – by avoiding payments to outside providers for support and services

Then again, users often have unrealistic expectations of what they’ll get from managed services as well.

Managed Services – Expectations
Managed Services – Reality
Easy to start with & operate
Stability & performance are the responsibility of internal teams
Move ownership to the managed service and enjoy peak performance for the application
Support is limited to infrastructure and doesn’t cover application needs
Cost efficiency
Costs continually increase when trying to solve performance by adding hardware
Features added by the managed service will add great value
Most features go unused, as they are irrelevant to many use casesMost features go unused, as they are irrelevant to many use cases
Easy to start with & operate
Move ownership to the managed service and enjoy peak performance for the application
Cost efficiency
Features added by the managed service will add great value
Easy to start with & operate
Move ownership to the managed service and enjoy peak performance for the application
Cost efficiency
Features added by the managed service will add great value

The majority of managed services, including Elastic, work with a ticket-based support system, which often entails long wait times and an inability to keep up with the dynamic demands of the system, not to mention being reliant on them when there are production crises. 

What does Opster offer?

Opster’s products and support services cover both the infrastructure layer and the data layer of managing Elasticsearch & OpenSearch, with a focus on the data layer that users have always been left alone to handle. Opster can be used either alone, while running self-hosted, or in addition to a managed service like Elastic Cloud to cover the gaps in service and optimize performance.

  • Automatic incident resolution and prevention.
  • Support powered by the AutoOps platform.
  • Optimized and improved performance.
  • Resource utilization and reduction of infrastructure costs.

Opster’s AutoOps detects issues, provides automatic resolution paths and optimizes resource utilization. The technology and tools perform automatic optimizations with the AutoOps operator, an on-prem service, which also enables Opster’s support team to interact with the system in real-time. 

Summary

Elastic support might be a good fit for you as a standalone service to run your search operation, especially if your deployment is not mission-critical. However, if that’s not the case, you’ll likely find that the “shared responsibility” that falls on you takes a heavy toll on your team’s time and abilities.

No matter how you’re hosting your Elasticsearch, you can benefit from Opster’s complete solutions. With an Opster AutoOps subscription, your database administration will be taken care of from start to finish. You’ll benefit from complete resolution of issues in your infrastructure & data layers, end-to-end support and constant optimization of your clusters. To learn more about Opster AutoOps, click here.