SPC releases I/O benchmark

By Zachary Shess

The Storage Performance Council (SPC), an independent consortium of storage hardware and software vendors, recently released for industry review a hard disk-based subsystem benchmark.

Known as SPC-1, the benchmark simulates an arbitrary random-access environment without focusing on specific hardware components or capacity requirements. SPC-1 primarily measures I/O transactions per second, and charts results in comparison to command response times, to illustrate how a subsystem handles a particular load. Configuration pricing information will also be included to gauge cost versus performance.

"Our hope is that the benchmark can be applied to everything from a single desktop spindle to a SAN fabric of nodes tied together with Fibre Channel," says Jack Stephens, administrator for the SPC, in Red-wood City, CA. Initially, Stephens anticipates the benchmark will be used to measure the performance of high-end RAID array controllers directly attached to hosts.

In addition to garnering input from nearly every leading subsystem vendor, SPC members are confident SPC-1 will be embraced by end users, because an increasing number of users are implementing multi-vendor storage environments without a standard to measure performance.

Further impetus driving benchmark development comes from users who want to see more thorough head-to-head comparative product information, Stephens says. Previously, vendors designed and implemented their own internal benchmark code that, while often an accurate measurement, can be tuned for the strengths of a specific subsystem.

"An industry-wide benchmark leads you to an implementation of a workload generator that's more robust, portable and better documented. It also takes that workload and morphs it into a benchmark by adding consistent rules for disclosure, implementation, audit and verification," Stephens explains.

The benchmark's sole focus on performance has caused at least one defection. For example, EMC in June pulled out of the SPC because officials believed it was not a complete enough gauge of system performance, and did not take system availability and reliability into consideration. Stephens agrees that the benchmark proposal is performance-centric, but says that has been the SPC's focus all along and EMC knew this as a member for the past two years.

SPC also plans to develop a standard centered on sequential processing that characterizes applications such as backup and restore and video-on-demand. And an SPC working group has begun development of a benchmark focused on network-attached storage (NAS) subsystems.

Benchmark kits and tools are expected to be available to the public (both SPC members and non-members) by the end of 2000 or the first quarter of 2001. In the meantime, the SPC will consider feedback and implement suggestions and other revisions. Pricing structures have not been finalized. For more information, visit www.storageperformance.org.

This article was originally published on August 01, 2000