The Need for Realistic Benchmarking Protocols

Many of us today are accessing storage via appliances and NFS or CIFS, yet neither of these protocols have seen significant changes over the last few decades. pNFS has been a promise for almost a decade and, while there have been some implementations, it has not caught on in terms of market share for a whole bunch of reason, not the least of which is the lack of parallel file system support and development in this area, except for one vendor.

The most recent benchmarks data I have seen for the SPECNFS (and other similar benchmarks) has been done using all flash systems, which we all know is unrealistic. I have a number of customers that copy their files from the NFS storage to their local disk and then operate on the data and then copy the file back to the NFS storage. They do this because the interactive operations they are doing on the information in the file is just too slow over NFS and even worse over CIFS, and from a productivity point of view doing this is a requirement.

It is impossible for these customers to do their work and still run over NFS or CIFS, and meet the organization’s productivity objectives. I think it is time for vendors and the industry to create realistic benchmark on realistic storage configurations that customers will buy for storage that is sold where connectivity is over NFS or CIFS. 

The customer in question cannot afford 500+TB of usable flash storage, and disk-based benchmarks using these protocols are not being done. NFS and CIFS protocols need to be fixed to address the small block random problem. And without realistic benchmarks with real world problems and data all we we are going to get is hype. I am sure that the customer I have is not alone with this type of data analysis problem, but industry seems to be moving away from this, not embracing the real requirements.

Photo courtesy of Shutterstock.

Labels: NFC,benchmarks,data storage

posted by: Henry Newman

Henry Newman, InfoStor Blogger
by Henry Newman
InfoStor Blogger

Henry Newman is CEO and CTO of Instrumental Inc. and has worked in HPC and large storage environments for 29 years. The outspoken Mr. Newman initially went to school to become a diplomat, but was firmly told during his first year that he might be better suited for a career that didn't require diplomatic skills. Diplomacy's loss was HPC's gain.

Previous Posts