Henry Newman's Storage Blog Archives for May 2012

The usual fix to poor applications design is to buy hardware to solve the problem. What I mean by poor applications design is:

• Applications that make small sequential requests when they could be making fewer larger requests.
• Applications that make large requests and do not use direct I/O (yes, it might be a bit faster for the user, but it is far more work for the kernel)
• Applications doing random I/O when they could be doing sequential I/O.
• Applications doing I/O not aligned on disk and/or RAID sector boundaries.

I am sure we all could provide other examples of poor application design choices. The problem is that the file system and operating system most often have no choice but to do whatever dumb request the application tells it to do, as there is little to no communication along the data path. The usual solution to the problem is the hardware approach -- in other words, throw hardware at the problem. The hardware solution works to a point, and with advent of SSDs, the hardware solution works for a point farther down the path, given the lower latency and often higher bandwidth of SSDs and the ability of SSDs to handle more IOPS.

The question then becomes, is using SSDs to solve an application design problem the right solution? The hardware vendors, of course, will tell you yes, and in the short run they might be correct. Buying a few SSDs might be less expensive than paying to redesign applications in the short run, but in the long run just throwing hardware at the problem is going to have limitations. When that happens, you will have no choice but to re-write your applications.

Then, you will realize all of the money you have spent over the years throwing hardware at the problem, only to still be faced with the cost of the re-write, and you will not be happy.

You might have seen that EMC purchased XtremIO this week. As I said in previous predictions, the SSD market will continue to consolidate. I believe the trend will continue to accelerate for the following reasons:

1. There is a limited market: The market for SSDs whether they are storage controllers with SSDs, individual SSDs or PCIe cards is limited compared to the size of the storage market.
2. It costs money to have marketing and sales staff: Every company must make a profit to stay in business. A limited market and the overhead of having sales and marketing staff all trying to sell into the same limited market is a problem. The cost of sales for non-commodity products is high. Flying people around, hotels, meals, and salaries and benefits require significant profit, and selling non-commodity products is often a long sales cycle.
3. The cost of R&D is expensive: You have many vendors basically making the same products. Yes, you might have some product differentiation with quality, reliability and so on, but how many PCIe SSD vendors does the world need? How many controller SSD vendors does the world need? The answer seems to be fewer than we have today, as the market is consolidating.

Last, but certainly not least, is the fact that SSDs are more expensive than disk storage and cannot be used for everything because of their cost. Most of the people I have talked to are looking for some type of tiered storage, and I think you will continue to see storage controller companies buying SSD companies so they can have tiered storage. The cost of SSDs is still not tractable for large bulk data storage, and it is tractable only for high IOPS environments. The question is what happens after the SSDs and disk tier? That is the topic for my next entry.