The Cloud Storage Initiative (CSI) and Cloud Data Management Interface (CDMI) standard promise to lift the concerns and confusion around cloud storage.
By Val Bercovici
-- Cloud computing dominated IT headlines in 2009, crossing the chasm from early adopter deployments into the mainstream. This focused new attention on cloud computing and cloud storage, and spurred action by IT departments in most public and private sector organizations. The cloud entered 2010 on a strong wind.
According to IDC, cloud storage services are projected to outgrow all other IT cloud services through 2013. Over the next four years, cloud storage will grow from 9% of a $17.4 billion market to 14% of a $44.2 billion market, or nearly $6.2 billion (see Figure).
In an effort to prevent end-user confusion, industry fragmentation and related loss of market momentum, the Storage Networking Industry Association (SNIA) launched a Technical Working Group (TWG) to explore standards for this rapidly developing market.
Cloud storage standards
Businesses, governments, non-profit organizations and individual consumers are all facing growing challenges in storing, managing, protecting and mining the explosion of data being generated in an increasingly digital world. Cloud storage standards can help these groups address the accessibility, security, portability and cost issues associated with the relentlessly growing pools of data.
Cloud storage standards can also help define roles and responsibilities for data ownership, archiving, discovery, retrieval and shredding/retirement. Service level agreements (SLAs) around data storage assessments, assurance and auditing also must be defined in a consistent manner.
In April 2009, more than 140 companies formed the SNIA Cloud Storage Technical Working Group (TWG), and a Google News Group with more than 280 participants was launched. In June 2009, the TWG published its first working documents in the form of Cloud Storage Use Cases and a Reference Model.
Based on the Use Cases and Reference Model, the SNIA Cloud Storage TWG worked on a proposed Cloud Data Management Interface (CDMI) standard throughout the summer. In September 2009, the TWG released draft version 0.8 of the CDMI specification for public review and comment at SNIA's Storage Developer Conference. SNIA Cloud Storage TWG member companies, including Bycast, Cisco, Hitachi Data Systems, NetApp, Olocity, QLogic, Sun and Xyratex were significant contributors to the draft release CDMI. The 1.0 spec is now a draft cloud storage standard released for public review.
The CDMI specification is focused on simplifying all aspects of cloud storage for subscribers (users), service providers, developers and IT hardware and software vendors. More specifically, CDMI defines the functional interface that applications will use to create, retrieve, update and delete data elements from the cloud. Clients will be able to discover the capabilities of the cloud storage offering and use CDMI to manage containers and the data in them. In addition, metadata can be set on containers and their data elements through the CDMI interface.
CDMI can also be used by administrative and management applications to manage containers, accounts, security access and monitoring/billing information, even for storage that is accessible by other familiar protocols such as SAN, NAS, FTP, WebDAV and HTTP/REST. The capabilities of the underlying storage and data services are exposed so that clients can understand the cloud offering.
The CDMI spec is at version 1.0g and is progressing towards SNIA-approved status, which is expected by April at the Storage Networking World (SNW) conference in Orlando. The SNIA Cloud Storage TWG members are also working on a CDMI Reference Implementation to spur cloud storage deployments based on the specification.
What CDMI means to you
Four key groups can benefit from the CDMI standard:
Cloud storage subscribers (users): Service-level expectations for cloud storage security, portability, protection, performance and other criteria among different cloud storage services are best queried and compared over a standard interface. CDMI provides cloud storage subscribers with a simple, common interface to help them discover the appropriate set of compatible cloud storage service providers for their specific requirements.
Cloud storage service providers: Publishing cloud storage service capabilities via a standard interface helps ensure broad market coverage for service providers. CDMI provides a common interface for cloud storage service providers to advertise their specific capabilities and help subscribers discover them. CDMI helps service providers advertise as many or as few capabilities as required to match their targeted subscriber bases. CDMI also provides unique, non-standard extensions for service providers that want to differentiate without sacrificing broad market addressability.
Cloud storage service developers: Operating systems such as Windows, Solaris, Linux and Apple's iPhone have proven the value of standard interfaces for application developers. The success of the cloud will therefore depend on standard interfaces for computing, networking and storage. CDMI provides the only multi-vendor, industry-standard development interface for application developers that want to store data in the cloud. CDMI also ensures a broad infrastructure of compatible service providers for application developers, thereby creating the broadest possible market of potential subscribers to cloud application developers.
Cloud storage service brokers: As subscribers entrust more important data to cloud storage providers, the need to "de-risk" the relationship between subscribers and providers becomes paramount. Enterprises or government entities may also have complex cloud storage requirements that exceed the capabilities of any individual cloud storage provider. In that case, a suite of federated cloud storage services may be required. Cloud storage service brokers can step in and offer "middle-man" services to subscribers. For example, brokers could offer "cloud insurance" via CDMI by combining a primary and secondary set of cloud storage providers to the broker's customers (subscribers). If the primary cloud storage service provider has an outage or terminates the service altogether, the broker-assigned secondary cloud storage service can take over according to the SLAs. Similarly, cloud storage brokers can use the discovery interfaces of CDMI to assemble a custom suite of services. That custom "cloud suite" would be a federation of several distinct cloud storage service providers, presented as a single cloud storage service by the broker to the subscriber.
Soon after the initial draft CDMI specification was released, SNIA members formed the Cloud Storage Initiative (CSI) in October 2009. The primary goal of the CSI is to become the authority for cloud storage standards and evangelism. The CSI is working towards that goal by educating the vendor, developer and user communities about cloud storage.
As part of the larger cloud ecosystem, the CSI is also coordinating with other key standards bodies and industry associations. Examples of partner organizations include the Open Grid Forum's (OGF) Open Cloud Computing Interface (OCCI), the Desktop Management Task Force's (DMTF) Cloud Incubator, and the Cloud-Standards.org industry wiki. The CSI is also working with SNIA Regional Affiliates around the world.
For more information about SNIA and cloud storage, visit the SNIA CSI website.
Val Bercovici is chair of SNIA's Cloud Storage Initiative (CSI), and a senior director in the office of the CTO at NetApp.
CSI founding members:
Hitachi Data Systems
Chair: Val Bercovici (NetApp)
Vice-chair: Mark Carlson (Oracle)
Treasurer: Wayne Adams (EMC)
Business development: Vincent Franceschini (HDS)