Henry Newman's Storage Blog Archives for August 2013

New Non-Volatile Data Storage Memory

A number of vendors have come out with SSD DIMMS, which in my mind begs the question: when are the OS changes that will provide memory allocation in non-volatile memory? Of course the question is not just simply providing allocation space, there is going to have to be some additions to other parts of the operating system – specifically in the area of resource management, even if this just used for as a RAMDISK.

As non-volatile memory will be a scarce resource, there will have to be something like quotas to manage allocation. If there is not some user management framework for this new type of memory, there will be limits to how it can be used in a general way. For example, how will it be used in a VM environment? What happens if the memory space is not available for the important application?

I suspect that we will soon be dedicating machines that run a single application, but that is not what the industry wants as it defeats the purpose of the shared server approach that we have been moving toward for the last few years. We are going to have to change operating systems so they can directly address non-volatile memory and deal with it the same way they deal with the path from DDR memory to the L1 cache.

These changes are going to be needed in the long run for this new technology. And there are lots of other non-volatile technologies on the horizon, many of which are byte addressable. This is not the case for NAND flash, which is limited when you’re talking about operating system changes to make this look just like DDR memory, but also control who can use it.

Any new technology faces challenge for acceptance. My feeling is that there has to be some major changes in the way operating systems and user resource allocation works before non-volatile memory technology is widely accepted.

Labels: SSD, Storage

posted by: Henry Newman

Fibre Channel in More Trouble

We have all heard about the problems that QLogic is having and fibre channel’s decline. I even posted something on this back in May.

Now the latest is that Emulex is looking for a buyer. This is not good for those of us that have large fibre channel installations.

Let’s say Emulex gets sold to someone, and the likely competitor is Brocade. That means we are down to two vendors in the fibre channel market place. And if QLogic tries to buy Emulex, we will be down to a single HBA vendor.

Having just a few vendors in a market is not good for the long term, especially in the communications market. Whether it was FDDI, HIPPI, name your interconnect, they all failed to gain enough market share over Ethernet to survive. Fibre channel was the performance leader over Ethernet from about the time it came out until about last year (yes, 10 Gbit/sec Ethernet was out while FC was only 8 Gbit/sec but that is just a small difference). Now 40 Gbit/sec Ethernet cards are hitting the market. FC is still at 16 Gbit/sec, and the cost per Gbit/sec is not looking good for fibre channel

Here's a quick look on the internet for some basic pricing for fibre channel HBA vs. and Ethernet NIC:



Performance in Gbit/sec


Cost per Gbit/sec

Dual Port Emulex




Mellanox 40 Gb Ethernet







Because of volume, Ethernet prices are going to come down a fair amount more than will fibre channel prices. Of course, you have the cost of fibre channel switch ports, which are less than Ethernet today but just like what has happened with 10 GbE switch ports over the last eighteen months, the same will happen for 40 GbE.

It is all about volume. And with storage appliances moving to 40 GbE and RDMA stacks for Ethernet available, fibre channel has no significant advantages. It will not be able to compete given the pricing.

My opinion of course.

Labels: Brocade, Emulex, QLogic, fibre channel, EtherNet

posted by: Henry Newman

SSDs and SMART Data

One of our customers wanted to know if the SSDs they purchased were using going to fail. They were concerned that the SSDs had used up their write budget (all NAND SSDs are over-previsioned as cells fail after a certain number of writes).

I asked someone to collect the SMART data for the SSDs in question, only to find that the information could not be interpreted. So we contacted the “major” hardware vendor who sold the systems.

They had no idea how to interpret the SMART information on the SSD they sold us. To add insult to injury, the senior technical people from the vendor told us that we had to buy SLC SSDs, as they do not wear out like MLC SSDs.

I could not believe my ears and told them they were just wrong. They held their ground, and then I realized why. This vendor only sold enterprise SSDs that were SLC. Please note that SLC NAND is falling out of favor by the vendors that make the NAND, which is one of the reasons I told this vendor they were wrong.

So what did I learn from this little encounter? The first thing I learned is that at least this hardware vendor’s senior technical staff—even the very senior ones—are not well versed on technologies other than the ones they sell. I am not sure if this applies to all hardware vendors or not, but it sounds to me like the company line is “if we do not sell it we do not have to learn about it.”

The second thing I learned is that SMART data collection for SSDs it not obvious. The information passed back is not easily interpreted. At least for the SSDs this customer had, the hardware vendor had no information. We went to the SSD vendor website and the documentation was non-existent—likely because it has NDA information in it. This is totally different than disk drives as you can get everything you want from Seagate’s website.

Labels: SSD, MLC, SLC, Storage

posted by: Henry Newman

NTFS Is a Really Bad File System

I really have limited choices for my home PC for my wife. She uses Photoshop, likes Word and Outlook, and loves our Logitech Squeeze (now a defunct product), so I have a Windows PC.

This weekend our Squeeze box started to skip songs. I had an eerie feeling that it was broken, but figured it was time for a bit of debugging. I, of course, powered it on and off, powered off the routers and checked router performance to make sure the router was not dying—all to no avail.

I was then getting really worried that we would need a new music system. So I tried the Logitech boom box we had, and it had the same problem, which meant it was the PC. I looked at the error logs and found nothing. So I powered everything off and on, thinking it might be the WiFi card.

I then remembered that the last time this happened I needed to upgrade memory as the song and album database had gotten too big for the system. So I checked the memory size and paging performance. That was not the problem, but the disk performance was slow.

Was the disk going bad? It had no errors. It had always cleanly shutdown and booted, so I decided to scan the disk for file system and disk errors, using an MS tool. It took over two hours to run this check, and it found nothing. But when I booted up everything was working properly again.

Clearly there was a file system or disk problem that had been fixed, but the logs showed that nothing was found. Why? This never happens on a Linux system nor on any other file system I have worked on. Windows reported nothing even after the file system check, and the system has been working properly for now three days.

I have complained publicly about NTFS design for years in terms of performance and reliability, but after wasting three hours of my time, I ask the question: Is NTFS the world’s worst file system currently on the market?

I know what my answer is.

Labels: Linux, Windows, file system, ntfs, disk drive

posted by: Henry Newman

Storage Security

Not long ago, all that you needed to protect systems was network security. In reality, we still have little more than that. For storage, we have user permissions, group permissions and ACL, but not much more than that for Windows, Linux or any operating system that I know of, except for SELinux. Add in network communication to a large distributed file system and you get a null set.

The issues become more significant as more and more data is stored online. Think of the mandate that is coming due to online medical records. This presents huge opportunities and huge challenges.

For example, I would very much like my records to be used by researchers to gain a better understand of long-term health issues and trends. I would love an explanation for why at age 37 my cholesterol went up a huge amount and could not be controlled with diet or exercise even though I had no lifestyle changes and no other physical changes. Maybe if they had similar populations that had the same issue, they could figure out the problem. My doctor had never seen this dramatic a change.

On the other hand, I do not want nor would most want their name associated with the actual data for an anonymous researcher. But I would very much like my doctor to understand and be able to see others in the population. This type of data access is going to take a concerted effort in applications, operating systems, file systems, networks and object interfaces. The whole stack is going to require changes and work.

The problem, as I see it, is that there is no longer a standards body or set of bodies to work on the these more intrinsic and difficult issues. People just throw their hands up and say it is too hard. What is too hard is dealing with all of the groups and vendors.

Labels: network security, Storage, Standards Bodies

posted by: Henry Newman