I have been thinking about the state of file systems and big block storage (aka midrange and large RAID arrays), and I am wondering if block storage has a future. My issue is that block storage depends on file systems for the most part, except that some databases can manage their own storage. To be sure, many large storage arrays are broken up into much smaller LUNs, and file systems are made using these smaller LUNs. But what if the file systems are not scaling?
How old, for example, is NTFS? Any guesses? It started in 1993. Do you think we have had some storage changes in 18 years? Does the metadata scale? Block storage and file systems have a symbiotic relationship. Block storage mostly depends on file systems for access, and file systems depend on block storage to provide LUNs. Both need to scale for the relationship to work.
The problem as I see it is that file systems are not scaling, and since enterprises generally are the ones buying block storage for reliability and management, and enterprises generally have larger requirements, the symbiotic relationship it at risk. This in my opinion is why appliance-based storage is gaining more and more traction. More vendors are embedding file systems into storage and ensuring that they scale.
I suspect the most common file system on the planet in terms of instantiations is NTFS, and we all know that it does not scale well. Without changes from the file system community to provide high-performance access, more and more users will be moving to appliance storage. This should not surprise anyone.
Labels: Storage,storage appliance,file system,ntfs
posted by: Henry Newman