“Software-defined storage is…a marketing buzzword for promoting computer data storage technologies.” – Wikipedia.
The promise of software defined storage (SDS) is lofty. This emerging technology offers to turn the fragmented world of data storage – often a disparate patchwork of networked silos – into one seamless pool of easily monitored storage that practically runs itself. Aided by a robust SDS solution, database admins aren’t so focused on, say, that rack of aging servers. Instead they look at a single-pane monitoring screen that displays metrics like total storage available and relative access speeds of storage medium.
In theory, SDS decouples the many chores of data storage from the many limitations of hardware. Tasks like deduplication and replication and provisioning are performed on a “global” basis, without regard to borderlines between various vendors' gear and different OSes.
Not only are the lives of storage admins less stressful, as they set service schedules without worrying about which piece of hardware holds which chunk of data. The enterprise, too, is happy, because SDS allows cheaper commodity hardware – and more efficient use of these lower cost boxes.
However, whether software defined storage actually delivers what it promises is an open question. Or, more accurately, what exactly constitutes SDS is open to interpretation. Everyone agrees that the new technology offers scads of advantages, but the storage industry is far from a clear definition.
Some of the large vendors tout their storage virtualizations products, which have been around for several years, as SDS, which is a newer concept. They’re not giving a good “‘Here’s how SDS is different from storage virtualization,’” says Stuart Miniman, an analyst with Wikibon. These vendors “haven’t differentiated for me, here’s why it’s new and different. It’s just like, ‘Oh no, we’ve been doing that forever.’”
On the other hand, some of the well established storage virtualization solutions from blue chip vendors have matured to the point where they rival a new SDS engineered from the ground up to include all the latest orchestration bells and whistles. So what separates SDS from existing storage virtualization may be a matter of who’s defining it. But for storage vendors, having a product that can somehow be labeled as SDS has become essential.
The Many Definitions of Software Defined Storage
The term “software defined” gained big credence in the summer of 2012 when virtualization leader VMware acquired Nicira, a startup focused on software defined networking. It’s no surprise that VMware is pushing the “software defined” market: For many in the IT sector, software defined is used synonymously with virtualization. That VMware paid $1.26 billion for Nicira meant “software defined” was a buzzword on its way up, however it’s defined.
A core problem with referring to data storage as ‘software defined’ is that software has always been used to administer storage hardware. Even a task as simple of moving files across physical locations is performed via software. Every last server box has an operating system: say, Windows Server 2012 or Red Hat Enterprise Linux Server. Software runs the show in storage.
And software obviously underpins storage virtualization, a well-established technology that hums along in most data centers. As VMware's Chuck Hollis wrote on his blog. “We live in a world where there are more virtualized workloads than physical ones.” The commonly known names in storage virtualization include HP’s StoreVirtual, IBM’s SCV, VMware’s vSphere Storage Appliance and NetApp’s Data ONTAP, among others. A bevy of storage virtualization start-ups offer their own take on the idea.
To what extent a storage virtualization solution can be touted as a full-fledged SDS product is, again, a matter of whose definition you want to use.
Mark Bowker, an analyst with Enterprise Strategy Group, notes that “software defined storage is ultimately doing the same thing that VMware did for the server. So it’s taking that pool of hardware and turning it into one big logical unit.”
You move closer to the world of SDS when you use a virtual storage appliance, VSA (VSA is also an acronym for virtual SAN appliance.). A VSA, which runs on a virtual machine under the guidance of a hypervisor, is liberated from the cold confines of hardware. A VSA pools all the storage on the hardware that’s direct attached, abstracting the storage capacity from the physical volumes that offer it. Its ability to virtually pool and share data helps admins shuffle data between disparate arrays, and VSA's ability to offer shared storage means they allow redundancy, which enables greater security.
Industry observers disagree, but the factor that separates a VSA from the full functionality of SDS is this: an SDS deployment is a true “global operator,” while a VSA is not. Sure, a VSA will replicate between multiple instances but you won’t look to a VSA to turn your entire storage infrastructure into a single unified pool. SDS is much more than federated storage, a technology that enables a common a management system to govern autonomous storage resources – which has been around for years.
By the strictest definition, SDS is hardware agnostic. Its software controller must not rely on any proprietary hardware. It can orchestrate and provision any vendor’s hardware, or be just as efficient with commodity hardware. Furthermore, not a single element in an SDS product should be factory defined. Virtual hardware can be provisioned dynamically, as needed.
An SDS solution can almost said to be “alive.” In contrast to traditional hardware-based storage, an SDS environment is flexible and intelligent. It works in the background as it works in the foreground. It reacts dynamically to workload, shifting critical processing closer to compute cores, shuttling infrequently accessed data toward slower storage media. It can pre-fetch data, schedule dedupe at irregular intervals, archive older snapshots.
All kidding aside, a full-featured SDS solution is such an independent operator that it’s akin to the HAL 9000 computer in 2001: A Space Odyssey. ("I'm sorry Dave, I'm afraid I can't do that.”) Its many tools allow it a kind of mind of its own. It does so much that veteran storage admins might wonder why they’re needed, but software this complex certainly needs human oversight. Although hopefully those human overseers will have better luck than poor Dave.
Large Vendors and Start Ups
In the SDS sector, established blue chip vendors compete with freshly minted start-ups.
An example of a large vendor moving aggressively in the SDS sector is HP, which sells its LeftHand and StoreVirtual solutions. HP’s Dale Degen, Worldwide Category Manager, HP Software Defined Storage, talked about the flexibility and open ended nature of HP’s approach to SDS.