It’s never been easier to release features fast and ensure that software is agile and scalable. Cloud-native development is ubiquitous, and Kubernetes is the industry standard for container orchestration, but how do you store and keep tabs on all the data these modern apps depend on to run smoothly?
Here, we look closer at why more and more organisations are turning to cloud-native storage for their Kubernetes-based apps.
What is cloud-native storage?
‘Cloud native’ is a term used to describe the way most applications are now built, delivered, and operated. Unlike traditional monolithic applications, which are deployed as a single unit, apps with cloud-native architectures are broken up into different ‘microservices’, helping to speed up software delivery, manage complexity, and make the process more agile and scalable.
For almost every app to work properly, it needs reliable and resilient storage for a number of use cases, such as databases, files, and objects. These must be secure, resilient, scalable, and highly available—also known as persistent storage. On-site storage is far from ideal—you need to pay for hosting and maintain the hosting and its security. Plus, scaling is high-latency due to the time it takes to order and install new hardware, which adds an additional security burden to your organisation.
Cloud-native storage is a storage technology designed specifically to be used in such an environment, providing data management and other storage solutions for these types of applications. They are similar to other cloud-native tools in that they are scalable, highly available, secure, and resilient and allow for declarative deployment and API-based automation. While many cloud-native offerings are proprietary, you can typically choose a vendor to suit your needs.
What does cloud-native storage have to do with Kubernetes?
Containers play a big role in cloud-native development. They make it easy to move chunks of your app from one cloud to another—using the same processes or tools—and destroy or create new containers when needed. As we explain in our ebook, Steering the Ship: keep on top of containerisation with Kubernetes, Kubernetes is the industry standard for container orchestration and hosting microservice processes.
Cloud-native storage embraces the same ideas behind containerisation, where everything that’s required to manage the storage is encapsulated in containers, which you run from Kubernetes. Like any other part of the overall application landscape, cloud-native storage can be easily provisioned by developers, delivered by code, and moved. Hardware is still required, but it can be abstracted away, either by the cluster or the infrastructure platform.
What does that look like in action?
Stateful applications often need databases like MySQL, PostgreSQL, or MongoDB to persist data. For databases to be viable in a cloud-native setting, you need a cloud-native storage solution.
Cloud-native storage solutions often come in the form of a Kubernetes operator or controller. These add custom resource definitions (CRDs) to the Kubernetes API for your manifest to use. Because everything is done through the Kubernetes API, you can scale the storage as you would your application. Cloud-native storage also includes managed solutions from your cloud provider, such as S3 and RDS on AWS. Again, there are controllers and operators on Kubernetes that can make these accessible via CRDs.
Platform teams can create CRDs that further abstract the above-mentioned CRDs to meet corporate and compliance standards or maximise the value without increasing the cognitive load on engineering teams.
The types of storage covered by these operators and controllers include:
- Block devices—volumes that can be mounted by pods.
- Shared file systems—similar to network file systems NFG/CIFS.
- Object storage—blob-based storage using HTTP, similar to Amazon S3.
Key characteristics of cloud-native storage
Now let’s explore some of the clear advantages that cloud-native storage offers organisations and their Kubernetes-managed applications and what you should look out for when choosing the right technology.
Scalability: Whatever your application's purpose, you'll be accruing more and more data daily, so you need flexible storage to suit your needs. With your storage management taken care of by cloud-native storage, you can focus on the application instead by offloading operational concerns to the cluster and its operating team.
Reliability: You need your storage to perform as you expect. Cloud-native storage enables replication, allowing your storage to scale concurrent read operations, improving the performance and reliability of your application.
Availability: Though scalability and reliability feed into this subject, storage can be decentralised, putting data at the closest location to the user. This improves application responsiveness for the user and can also limit the blast radius of an outage to a location.
Consistency: With cloud-native storage, you can easily observe and capture application states. Providing you with the consistent results you need to manage and distribute data, it ensures a return to operations as usual after any updates and provides almost no delays in data operations.
Flexibility: You are not limited to a single solution or opinion when adopting a Kubernetes storage solution. Engineering teams can break out of the platform and use third-party providers if/when it makes sense to do so. In fact, we recommend doing this when starting out, as your first Kubernetes clusters will not be fully featured. This places some of the burden on your IaaS (infrastructure-as-a-service) platform, such as AWS. A good example of this is RDS for managed database instances.
Durability: When we talk about durable storage, we're referring to the layers of protection in place, the ability to protect itself against data loss by detecting corruption and restoring failures, and how long the data can stay in the system. Cloud-native storage is designed with this in mind to make sure data can stay safe and stored for the long term with suitable recovery mechanisms.
Steering the ship: keep on top of containerisation with Kubernetes
In this eBook, we’re going to provide a quick recap to put Kubernetes in context; explore the key challenges it helps your business solve, including how it can support you as you scale; look at the benefits containerisation can bring; and also ask whether it’s the right solution for your organisation.
What different cloud-native storage solutions are available?
There’s not one specific method of storage you have to stick to when it comes to cloud-native. From the big players like AWS, who we’re proud to partner with, to newer vendors, there is a wide choice of options to choose from, all offering real-time data management capabilities.
There are currently, at the time of writing, three projects listed on the Cloud Native Computing Foundation’s website that are graduated or at least incubating. These are Rook (the one that’s graduated), CubeFS, and Longhorn. If you’re starting on your Kubernetes journey, we strongly recommend using the most stable of the three: Rook. However, all three of these projects offer block, shared, and object storage capabilities. These will often leverage your IaaS platform to back them.
You can also use the IaaS platform directly for your Kubernetes cluster. Though it may feel more familiar to your infrastructure engineers to take this approach to storage, issues may arise, such as availability zone placement on AWS, forcing you to pin your Kubernetes services and pods to a fixed zone, hurting your operational targets.
Clouded judgement?
Unsure about what step to take with your Kubernetes-based apps? Need cloud-native storage but aren’t sure where to turn? We can help you get started on your cloud-native storage journey today.
