SPC tackles storage benchmarking
By Zachary Shes
With storage no longer an afterthought in IT environments, a new consortium has recently formed to address issues such as storage optimization, benchmarking, and standards. After several months of planning, the Storage Performance Council (SPC) recently finalized its charter. SPC administrators hope to build a diverse membership base within the storage industry, including the corporate end-user community as well as storage manufacturers, resellers, and integrators.
Some of the SPC founders, as well as current administrator Jack Stephens, were members of the Transaction Processing Performance Council (TPC), a body that provides benchmarks and performance data for OLTP and database environments. They recognized that storage technologies were accounting for an increasing portion of overall system performance, as well as costs. For example, storage costs are now 40% to 60% of a total system costs.
According to Stephens, the impetus to form the SPC came when they realized the need for a body that could provide a similar level of focus and comparative benefits in the storage industry as the TPC did in the database and system areas. "We`re trying to follow a successful model and provide more focus on storage issues," Stephens says. "While the TPC has been very good in characterizing system-level performance, we didn`t think it was providing the focused data needed to help the storage industry."
With the bylaws recently completed, SPC members are turning their attention toward implementing their benchmark program. "So far, we`ve defined the framework for our benchmark specifications and started the process of how to gather and quantify trace data and encapsulate workloads, but it`s in a fairly early stage," says Stephens.
While benchmarks such as LATTIS are often used to gauge the performance of some types of storage products, such as network-attached devices, the SPC plans to offer evaluations with a broader perspective. The consortium hopes to differentiate itself by focusing its analyses on how storage products can best be used in consort with other products to provide optimum solutions. Stephens expects SPC`s benchmarks "to provide targeted (simulated) workloads that would allow you to quantify meaningfully how well the storage system operates under a given environment."
The SPC is not following the "start small and work your way up" school of thought, but Stephens believes the storage industry--including users, resellers, and manufacturers--will be responsive to their approach. "It would certainly be simpler if we started looking at just SCSI spindles, for example. But, rather than looking at it from a component level, we`re trying to take a more application-oriented approach," Stephens says. "It`s an aggressive charter, but we`re hoping that the end result will be to get more meaningful data to users."
In addition to offering benchmarks, the SPC plans to help its membership by disseminating information and acting as a storage clearinghouse through its Web site and meetings held every other month. The organization also plans to provide arbitration services for disputes and other issues within the vendor community.
The formation of the SPC marks the second storage-focused organization to be formed in the last year, following the Storage Networking Industry Association. Stephens believes the two groups are complementary.
"The storage industry needs new benchmarks and the SPC is the organization to do it," says Michael Peterson, president of the SNIA. "In launching new storage and SAN [storage area network] platforms, users require performance standards. The SNIA heartily endorses the SPC`s efforts and is lending full support."
The SPC plans to have a broad representation. As of last month, Stephens reports the group already has solid membership among component and subsystem vendors. Stephens hopes to also attract industry analysts, large corporate users, and systems integrators.