The Need for Realistic Benchmarking Protocols

Posted on March 05, 2014 By Henry Newman

RssImageAltText

Many of us today are accessing storage via appliances and NFS or CIFS, yet neither of these protocols have seen significant changes over the last few decades. pNFS has been a promise for almost a decade and, while there have been some implementations, it has not caught on in terms of market share for a whole bunch of reason, not the least of which is the lack of parallel file system support and development in this area, except for one vendor.

The most recent benchmarks data I have seen for the SPECNFS (and other similar benchmarks) has been done using all flash systems, which we all know is unrealistic. I have a number of customers that copy their files from the NFS storage to their local disk and then operate on the data and then copy the file back to the NFS storage. They do this because the interactive operations they are doing on the information in the file is just too slow over NFS and even worse over CIFS, and from a productivity point of view doing this is a requirement.

It is impossible for these customers to do their work and still run over NFS or CIFS, and meet the organization’s productivity objectives. I think it is time for vendors and the industry to create realistic benchmark on realistic storage configurations that customers will buy for storage that is sold where connectivity is over NFS or CIFS. 

The customer in question cannot afford 500+TB of usable flash storage, and disk-based benchmarks using these protocols are not being done. NFS and CIFS protocols need to be fixed to address the small block random problem. And without realistic benchmarks with real world problems and data all we we are going to get is hype. I am sure that the customer I have is not alone with this type of data analysis problem, but industry seems to be moving away from this, not embracing the real requirements.

Photo courtesy of Shutterstock.


Comment and Contribute
(Maximum characters: 1200). You have
characters left.

InfoStor Article Categories:

SAN - Storage Area Network   Disk Arrays
NAS - Network Attached Storage   Storage Blogs
Storage Management   Archived Issues
Backup and Recovery   Data Storage Archives