Henry Newman's Storage Blog Archives for July 2014

Needed: Easy to Use Storage Encryption

All enterprise disk drives today support some type of encryption at the drive level, at least those from the big 3 vendors. But I’m not sure about some of the enterprise SSD vendors, though that’s not the point.  I know of very few people using the full disk encryption for enterprise disk and SSD drives.  So I asked myself the question why not? 

After doing some research I think there is plenty of blame to go around.  First, the big storage enclosure vendors do not seemingly want to add this feature.  I am not sure why, but it seems that the interface to the hundreds or thousands of drives to manage is going to be difficult, especially if there are multiple drive vendors in the mix. 

The second reason is that key management is a complex and difficult problem. If you lose the keys – or the key management system fails and is not backed up properly – you can have a really bad mess, like loss all of your data or not be able to add drives.  Neither of which is good and likely the storage enclosure vendors do not want to have to deal with this. And rightly so, as it is complex and has the potential for some really bad consequences. 

We have all read about used disk drives being sold on the open market with data like your SSN number or medical records still on the drive.  This problem could be a thing of the past with encrypted drives, but it is not as easy as it might be for, say, your home computer or laptop. The world needs standard easy to use and easy to backup and restore key management solutions for disk drives – and it needs them integrated into storage controllers and it needs them now.

Photo courtesy of Shutterstock.

Labels: data storage, encryption

posted by: Henry Newman

Optical Storage for Archive?

We all saw the Facebook news article on using Blu-Ray for archival storage.  This got me thinking and doing some research on Blu-Ray technology to see if it is really suitable for archiving things for long periods of time.  Facebook seems to think it is a good idea, but I’m not sure, and here’s why – at least using standard technology:


The interface for Blu-Ray is either SATA or USB.  USB is an extremely poor interface in terms of the high data reliability of the channel.  The channel was never designed for data high reliability and has a much higher potential for silent data corruption than the SATA channel, which has a much higher potential for silent data corruption than SAS or Fibre Channel.  So if you really care about your data you would be using the SAS or Fibre channel interface. And as Blu-Ray does not support these interfaces, Blu-Ray would be off my list for archival devices.


The Blu-Ray standard does not provide any means of doing end-to-end data protection.  With SAS devices on disk you have the ANSI T10 PI/DIF standard, which is supported by all disk vendors. And I’m pretty sure all HBA vendors support it and a number of operating systems including Linux, AIX and Solaris for sure.  Tape, which is the most used cold storage archival technology, has its own ANSI end-to-end data protection that ensures that each block is written correctly.

Hard Error Rates

Enterprise disk hard error rate is 1 sector in 10E16 bits; nearline drives with SAS interface (4TB drives today) have a hard error rate of 1 sector in 10E15 bits; LTO tape is 10E17th bits; and enterprise tape is even a few orders of magnitude higher.  All of this data is published and well known. 

What little I could find on Blu-Ray was 10E12, which means that assume the data is in 512 byte sectors, which is unclear from the document; that mean that every 9 TiB, I get a hard error =10000000000000/(1024*1024*1024*1024).  (It should be noted that this was on the Panasonic web site but was removed. This is all I could find.)

how to chose a hard drive

If the optical industry wants to move toward the archive market they need a lot more reliability, end to end data protection and transparent easy to find information  – no matter what Facebook thinks.


Labels: data storage, Facebook, optical storage

posted by: Henry Newman

HP and "The Machine"

A few weeks ago HP announced a converged infrastructure solutions system with a new architecture and – what I think is equally interesting – a new operating system. 

I think the hardware architecture, though new in combination, is not new. The idea of silicon photonics is not new – Intel talked about them at their developer conference. Memristor has been discussed by HP for a long, long time and though we seemingly get a bit closer we never seem to get to production.

The new area that I am very glad is being discussed is the operating system. I have felt and written about the need for major changes in the operating system to deal with non-volatile memory.  The last thing that is needed is for users to use this memory similarly to what is done with flash and make an in-memory file system. What is needed is a complete re-think of the problem with the ability to allocate the non-volatile memory space from an application and maybe let the faster memory space be controlled by the hardware as it is today. 

The non-volatile memory space needs to be directly addressable from the CPU no differently than memory is today.  The one thing that is missing is resource management, which HP might or might not be dealing with.  This could include non-volatile memory as a resource that controls it, that decides what applications and users can use it, and what applications and users cannot use it today, tomorrow this minute, hour or second. 

If we are going to have hierarchies of performance within a system we are going to finally need to address resource management to control allocations of non-volatile, regular memory such as large pages, shared memory, CPU access and performance, network access and performance, I/O and storage access and performance, just to name a few.  As systems get more complex, we are going to need to readdress the whole issue of resource management, control and reporting, as has been done on the IBM mainframe MVS for the last 40 years.

Photo courtesy of Shutterstock.

Labels: data storage, HP, Converged Infrastructure

posted by: Henry Newman