The future of storage is in the operating system

A storage-specific operating system could solve many enterprise storage management problems.


It's 2001, the year HAL-with a wealth of artificial intelligence-would issue that menacing invitation for a game of chess. But in 2001, we're not faced with computers that rule humankind; we're faced with managing the staggering amount of information we create-and rely on.

Bill Miller - StorageNetworks
Click here to enlarge image

In the 1970s and 1980s, data centers were primarily mainframe and minicomputer environments controlled by central IT organizations. During the late 1980s and the early 1990s, the rise of distributed computing reduced centralized control, increased duplication of data, and fostered a new generation of storage-hungry applications. As a result, the amount of storage needed by a typical enterprise dramatically increased, while the ability of IT departments to allocate and manage that storage greatly diminished. Storage networks emerged in the late 1990s from a need to use, share, and manage storage resources more efficiently.

Network storage enables information to be shared concurrently by multiple users and reduces the ratio of servers to storage. Storage area networks (SANs), which communicate at the block level, and network-attached storage (NAS), which communicates at the file level, can increase scalability, enhance performance of the storage and computing environments, help facilitate centralized management, and improve data and application availability.

iSCSI, Fibre Channel over IP, and InfiniBand are some of the emerging technologies that will increase available bandwidth and provide better storage connectivity, but interoperability within today's complex infrastructures and the complexity of managing heterogeneous infrastructures loom large as major storage management issues.

The cost per gigabyte of storage is falling and will continue to fall, while the cost of managing a gigabyte of storage has risen and will continue to do so.

Disk density is doubling approximately every 12 months, enabling us to store more data at a lower cost. Advanced media development shows that the superparamagnetic limit, or the limit of density on magnetic media, will not be reached as soon as was predicted previously.

That means disk density has the capability to grow at the current rate for at least the next several years. Even when the density of magnetic media finally maxes out, other storage technologies like near-field magneto-optical recording and holographic storage will likely take over the trend of increasing capacity.

The capability to store more on the same amount of disk will continue to drive down the cost of storage and commoditize storage resources, but how we solve the problem of managing the behemoth-capacity devices-and all the other distributed components that reside at the edge of the network-is the issue at the heart of the future of storage.

Edging into the core

Storage networks today consist of intelligent components distributed at the edge of the network: sophisticated subsystems, complex servers, and data management applications. Manufacturers develop management tools for their products; how ever, with the absence of a standard way to perform storage and data management, the result is a plethora of freestanding tools needed to manage a storage infrastructure-greatly increasing the cost and complexity of storage management.

With the growth in size of storage networks, management of all the intelligent resources at the network edges becomes an insurmountable challenge. Moving that distributed intelligence into the network core to enhance manageability and reduce complexity is part of the solution. That concept is known as storage virtualization.

There are a number of ap-proaches to virtualizing storage (see InfoStor, Special Report, "SANs rely on storage virtualization," January 2001, p. 20). One approach consists of intelligent switches in the core of the network that perform storage and data management functions.

While storage virtualization affords simplicity, there still needs to be an entity that ties it all together-switches, devices, servers, and hosts-and allows applications that access storage to integrate seamlessly with the infrastructure. That entity is a storage-specific operating system-the future of data storage management.

Storage-specific operating system

Just as a computer operating system makes a computer usable, a storage-specific operating system will make storage devices and their associated networks operate in a usable manner. Powerful storage networking switches may provide the platform for the storage-specific operating system.

Moving intelligence away from servers and storage systems into interconnected switches in the core of the network could solve interoperability problems that are inherent in heterogeneous enterprise environments and administration tools and could allow complex infrastructures to be managed as a single, scalable entity.

Once the basic kernel of the storage operating system is implemented, it will expand to support all data management applications and introduce the ability for these applications to be inter-application-aware.

Today, many IT organizations act as internal service providers, providing their "consumers" with data management applications such as

  • Block-level (SAN) storage for high-end database applications and clustered servers;
  • File-level (NAS) storage for applications that require sharing files among multiple nodes;
  • Backup of disk storage;
  • Local and remote replication of disk storage;
  • Disaster-recovery plans and processes;
  • Archival applications and data migration; and
  • Content distribution.

To provide such services, IT organizations typically deploy hardware and software point solutions that lead to interoperability issues and extremely complex management.

A storage operating system will enable IT organizations to not only perform data management functions, but also to benefit from enhanced storage management functionality such as

  • The ability to obtain accurate and consistent information across a heterogeneous and distributed infrastructure;
  • Integration with storage services management applications (if the organization receives storage management services);
  • Integration with business applications to form a scalable storage and data management environment;
  • Standardized and automated "best practices" for enterprise storage management; and
  • Lower cost of providing or obtaining storage services.

Enterprise storage management re-quires a common object model for all data management applications. A storage operating system will provide a common model that can unify storage resources through defined interfaces for information exchange, control, and management between the storage environment and the enterprise resource environment.

Integrating CRM tools

As a result, corporate resource management tools can be integrated with the storage operating system and resource management information base. Corporate billing applications, for example, could obtain information from the operating system to create accurate bills, and core financial tools could obtain information to monitor usage, return on investment, and depreciation reports.

Not only could the storage operating system enable seamless integration between the storage and computing environments, but it could introduce a significant increase in functionality, ease of use, and productivity, while lowering the cost of resource management.

The more we extend the limits of automation and centralize intelligence and functionality within the core of the network, the more we increase ease of management and control of an extremely complex IT challenge.

Intelligent switch-based storage networks-with the intelligence in an operating system and the management applications that run on it-will drive higher levels of scalability, efficiency, and productivity.

Bill Miller is co-founder, executive vice president, and CTO of StorageNetworks Inc. (www.storagenetworks.com) in Waltham, MA.

This article was originally published on June 01, 2001