Docker and Containerization Storage Buying Guide

Posted on February 25, 2016 By Drew Robb

RssImageAltText

Much has been written of late about Docker and other container technologies. They promise to reduce the development and application integration process in a similar way as physical shipping containers revolutionized the business of transporting goods. Regardless of whether goods are going by train, plane or boat, the same standardized container can be used to minimize time in unloading it from one mode of transportation and loading it into another. Similarly, in IT, containers provide core functionality onto which customization and additional applications can be easily integrated. This saves time in creating apps and in setting up IT building blocks. Popular container technologies include Docker, Kubernetes, Mesos and DH2i’s DxEnterprise for Microsoft Windows Server applications and workloads.

But while containers offer many advantages, they also come with their own issues, particularly on the storage side.

“Docker container storage is ephemeral, so when the container is removed, the storage is deleted,” said Eric Sites, CTO at Meros, a company that provides management, monitoring, logs and alerts for Docker. “You can get around that by using storage from the host that the container is running on, but then you can't easily move the container to a new host. You would need external methods outside of Docker to move the host storage to a new host.”

This issue of Docker storage has become particularly urgent with Docker now moving from being merely the subject of industry conversation and the early adoption into large-scale customer deployment.

“As containers are being used beyond test, development and research and are finding themselves being moved into production roles, the technology also needs persistent storage,” said Greg Schulz, an analyst with StorageIO Group. “However, the ephemeral nature of container storage can be relatively easily addressed by accessing persistent local or network storage, not to mention various Docker storage management tools.”

Let’s then take a closer look at the subject of container storage. Here are some of the existing tools and products that deal with the issues of container storage and container storage management.

Docker

The first place to look is, of course, Docker itself. The company has been taking steps to make storage management and integration much easier. In the beginning days, there was no simple way to move storage to a new host from a Docker container. Fortunately, that has changed.

“Docker recently released a new API to make it easier to move container storage around so that it can be done in a standard way,” said Sites.

The company has also been active in building out its partner ecosystem to get Docker integrated with a wider set of platforms and applications. This includes Amazon CloudWatch from Amazon Web Services (AWS), Elastic, Graylog, Rapid7, Loggly, Papertrail, Sematext Logsene, Sumo Logic and Treasure Data.

Flocker

Flocker from ClusterHQ is an open source container data volume manager for Dockerized applications. In general, Docker data volumes are tied to a single server, whereas a Flocker data volume (known as a dataset) is portable and can be used with any container in a cluster. As a result, volumes follow containers when they are migrated between different hosts. Flocker supports block-based shared storage such as Amazon Elastic Block Stores (EBS) and OpenStack Cinder. That makes it relatively straightforward to choose the storage back-end that best suits a particular application.

Blockbridge

Blockbridge Networks has a Docker-based tool called Elastic Programmable Storage that enables enterprises and service providers to build API-driven storage infrastructure using software-defined storage. The goal is to maximize the value of applications through efficient management, orchestration and operation of infrastructure, said Jonathan Nicklin, founder and CEO of Blockbridge.

“From a container perspective, we provide multi-host aware persistent volumes, the ability for applications to define per-volume storage resource requirements, multi-platform data portability (Docker, OpenStack, bare metal), multi-tenant per-volume encryption, secure deletion, QoS, data protection services, hardware independence and multi-tenant auditing, metering and monitoring,” said Nicklin.

EMC REX-Ray

Over the last couple of years, EMC has been talking up its support for open source software. One result is a company initiative known as EMC {code} which has developed REX-Ray within the open source community. REX-Ray provides the orchestration between a storage provider (EMC or not) and the container scheduler like Mesos or Docker.

REX-Ray is a storage abstraction engine that interacts between the container schedulers and storage providers (like EMC’s ScaleIO, VMAX, Isilon and XtremIO as well as AWS, Google Computer Engine (GCE) and others). When a container is scheduled onto a particular node, its storage requirements are passed to REX-Ray, which is then responsible for delivering the volume to the node chosen by the scheduler. When the container is rescheduled or moved to another node, REX-Ray takes care of moving the volume to the new node.

“Prior to REX-Ray, there wasn’t a mechanism to tie the storage to the container schedulers, which left the community either to manually manage the storage or to introduce less efficient constraints into the scheduling engine,” said Josh Bernstein, vice president of technology in the emerging technologies division at EMC.

EMC also offers two products that take a software-defined approach to container storage. ScaleIO provides block services, and ECS provides object and file storage.

Additional Open Source Solutions

“There are some other open source projects mapping Docker volumes to NFS and Amazon Elastic Block Stores (EBS) using the new Docker volume APIs,” said Schulz. One example is Convoy, which is a Docker volume plugin for managing persistent container volumes. Other initiatives include OpenStack Cinder and GCE Blocks.

Meros

Meros provides management, monitoring, logs and alerts for Docker. Without the need to rely on command line tools, you can use Meros to gather information about Docker hosts, containers and images on one web console. This makes it possible to monitor events, CPU, memory, network, and block I/O real-time statistics. In addition, you can visualize all containers running in a cluster, and control (start, stop, pause and resume) containers with ease.

While this product ventures beyond the realm of storage, it encompasses a couple of areas that are handy for the storage manager to have in his or her toolkit when it comes to monitoring storage capacity.

“We monitor all Docker volumes for disk usage and activity,” said Sites.

Photo courtesy of Shutterstock.


Comment and Contribute
(Maximum characters: 1200). You have
characters left.