By Thomas M. Ruwart
As the density of data storage devices continues to increase, it is becoming more practical and economically feasible to build multi-terabyte and even petabyte-level storage subsystems. However, it is difficult for today's software to scale to these capacities, in part, because many existing operating systems and file systems are based on legacy technologies that are decades old.
At the same time that storage capacities are growing, storage devices are becoming "commoditized" to the point where there is little room for differentiation between vendors. On one hand, commoditization makes a petabyte storage subsystem economically feasible. But on the other hand, the commoditization of disk storage products is making it increasingly difficult for storage vendors to differentiate their products.
One potential solution—object-based storage devices (OSDs)—is well on the way to becoming reality. This article presents some of the work being done in the OSD community—including the Storage Networking Industry Association (SNIA)—to generate more interest in this emerging technology.
The concept of object-based storage is not new. Researchers at the Parallel Data Lab at Carnegie Mellon University (www.pdl.cmu.edu), for example, have been working on the concept for more than a decade. However, OSD is a relatively new protocol that can have a significant effect on everything in the storage hierarchy—from applications down to storage devices. With OSDs, applications could realize far greater quality of service (QoS), and operating systems could access significantly larger amounts of data through shared file systems that, in turn, could more easily manage petabyte-scale storage subsystems. Furthermore, object-based devices can be tailored to meet the requirements of the data they store and bring a broader range of differentiation opportunities to storage vendors (i.e., OSDs could come in different "flavors" such as streaming OSDs, medical image OSDs, etc.).
To make object-based storage a reality, an OSD Technical Working Group (TWG) was formed within SNIA to define and develop a protocol that would address capacity issues as well as bring a new dimension of capabilities to storage devices. Since its inception in 1998, the OSD TWG has grown to about 30 member companies (see box). A draft specification of the OSD protocol as an extension to the SCSI command specification can be found at the SNIA OSD TWG Website (www.snia.org).
OSD TWG work also includes technology demonstrations that provide hands-on experience. To date, the OSD TWG has demonstrated an early version of the OSD protocol using an iSCSI transport. An updated version of this demonstration was presented at the SNIA symposium this month. This demonstration included some of the security features of OSD as well as the concept of "capabilities."
Another interesting project, funded by the Accelerated Strategic Computing Initiative (ASCI) Path Forward program through the Department of Energy, is focused on developing a global file system that can scale in capacity, bandwidth, and transaction rates well beyond what is currently possible with existing file-system technology. OSD is a key piece of this project. The goal of the pilot project, called Lustre, is to develop a file system that can run on a 1,024-processor cluster of computers and efficiently manage access to100TB of storage. While this may seem large by today's standards, it will be commonplace by the end of this decade.
In addition to the government, a number of vendors are involved in the Lustre project and are looking at how OSD and associated file-system technologies can be exploited. Experimental versions of the Lustre file system, and the associated code needed to make an OSD, are available on the Lustre Website (www.lustre.org).
The need for object-based storage is real. The problems of building and managing large storage systems have clearly demonstrated that these systems have outgrown existing technologies. Development of an alternative approach, OSD, is nearing adoption by storage vendors and users.
Thomas M. Ruwart is with the University of Minnesota's Digital Technology Center and can be contacted at email@example.com.
SNIA OSD TWG member companies
Fujitsu Software Technology
IBM Europe and IBM USA
Johns Hopkins University
Storage Tek Europe and Storage Tek USA
University of Minnesota