Storage capacity is growing at unprecedented rates. So Aaron Ogus, Partner Development Manager at Microsoft Azure, posed an interesting question at this month’s 7th annual Global IT Executive Summit hosted by Fujifilm in Los Angeles: how do you store a zettabyte (ZB) of data?
This is more than an academic exercise for his company. He deals with cloud storage and currently stores exabytes (EB) of data on millions of hard disk drives (HDDs) for his cloud storage. When he began in 2007, the company used four 750 GB HDDs inside 1u servers.
“We have reduced our costs by 98% in that time with 80% of that gain coming from storage design improvement,” said Ogus.
The logic behind the four-server preference is standardization and cost reduction. However, Ogus said this ties your ratio of compute and storage together and that led him to wonder if it wouldn’t be better to have more drives per server. The result of that idea was the creation of EB-scale archival storage.
Known as Pelican rack-scale cold data storage by Microsoft Design, this design takes into account the cost and efficiency of power, cooling, mechanical, electrical, storage and software together. It consists of 1,152 SATA disks in a 52U rack containing only two servers with IO stretched rack-wide. Its power consumption is only be about 3.5 kW per rack as no more than 8% of the disks would be spun up at any one time.
“Pelican is provisioned for cold data workloads so it is optimized for density and dollars per GB ($/GB),” said Ogus. “It uses SATA as a data archive as it has low power, low cost, low performance and high capacity.”
But finding a solution for EB-scale wasn’t enough, the next research project was to work out cost effective ZB-scale storage. Part of the challenge was to bring the storage infrastructure up to the efficiency level of the compute side.
“Moore’s Law hides inefficiency,” said Ogus. “I set a goal of 20% year over year improvement in design.”
Looking over the last 25 years, the pattern is steady gains in memory, CPU and HDD performance and capacity. But would these gains continue? Certainly, CPU, RAM, flash and networking have roadmaps stretching out many years. But it’s a different story for disk.
The current technology used in HDDs is called Perpendicular Magnetic Recording (PMR), which is reaching its areal density limits. Shingled Magnetic Recording (SMR) is one possible answer, which should raise disk density by as much as 25%. Another development on the horizon is heat assisted magnetic recording (HAMR), which uses a laser to write more data in less space.
“The HDD is hitting the limits of PMR and SMR will only be a one-time bump,” said Ogus. “HAMR appears to be the main hope for the future.”
Some anticipate HAMR and other developments taking the cost of disk down as low as 1 or 2 cents per GB by 2020. Currently it is about 4 or 5 cents per GB. How about flash? Microsoft Research numbers put it at about 50 cents per GB right now and predicts it will reach 10 cents per GB at best by 2020 if 3D flash provides the expected gains.
Tape, on the other hand, has 10 TB cartridges shipping and has demonstrated potential densities as high as 220 TB based on existing breakthroughs and tests. Data rates for tape are currently at 320 MB/s and heading towards 1 GB/s. Yet access times of 90 seconds prevent its usage in higher storage tiers. But looking comparatively, 30 optical disks currently equate to 1 tape. By 2020, that will rise to 100 optical disks per tape.
“One of big advantages of tape and optical disk is that you can remove media from the reader,” said Ogus. “But tape has a more credible roadmap compared to optical.”
So let’s look at what this would mean in terms of storing a ZB, a problem that will become all the more apparent by 2020 when about 5 billion smartphones will be in use, each churning out endless quantities of texts, emails, photos and videos.
A ZB on flash, according to Ogus, would cost $1.15 trillion per year to store at the moment, perhaps dropping to as little as 238 billion per year by 2020. Apart from the fact that its cost would be much too high, there is also the issue that flash vendors are not yet capable of producing 200 EB of flash each year.
For HDDs, the price tag for each year of storage would be $41 billion in 2015 and $17 billion per year by 2020. Optical disk would be cheaper at about $27 billion per year in 2015 and dropping perhaps as low as $4 billion per year by 2020. But this is based on the assumption that there would only be one rewrite every 3.5 years, which limits its potential as an archive. And a big issue here is the vast number of optical disks that would be required. That one factor alone precludes its validity as a medium for the storage of a ZB.
Tape would be much cheaper at $8 billion per year in 2015 and $1.9 billion by 2020. The assumption built into this is that tape will be able to run on Ethernet by that time as opposed to Fibre Channel (FC).
“Storing a ZB by 2020 will be feasible in the cloud,” said Ogus. “But some cold archiving strategy must be part of any strategy to cost effectively store a ZB.”
That means tape will form the lower tier for inactive data in an archive with disk and flash holding a much smaller subset of overall data. That’s why Ogus sees tiering as a key to cost control. Therefore, he thinks investment in ways to move data between tiers smoothly will be a critical area of cloud development over the next decade.
“All cloud vendors will be using tape and will be using it at a level never seen before,” said Ogus.