POSIX -- What Are We Going to Do?

By Henry Newman

I recently attended a HEC FSIO Workshop and Conference. HEC FSIO is high-end computing file systems and I/O that is funded research by a number of U.S. government agencies.

If you have been reading my column or blogs, you know I have long complained that POSIX I/O is too limiting, and it is not up to the task. It was interesting and music to my ears to hear much of the research community say that POSIX limitations are a significant problem for I/O performance and scaling. What was even more interesting was that many speakers also said that other communities, such as those using Hadoop and other implementations of the MapReduce algorithm, are ignoring the POSIX consistency requirements.

It should not be a surprise that this is happening. Name a single standard that has not changed in about 20 years, and I will bet that people are working around the standards. As I have said time and time again, the vendor community sadly controls POSIX, and vendors do not want to make any changes in the area of I/O, as it will cost them time and money.

What they do not seem to understand is that just saying "NO" like that David Spade commercial for Capital One will cause people to find a way to solve their problem and make the POSIX I/O consistency model a dinosaur. Every single standard I can think of has changed in the past 5 years, much less 20. What planet are the people that control the POSIX I/O consistency modeling living on? It is not the same planet I live on, or maybe it is, as we seem to be in a world where if the facts are denied enough, it makes the denial the truth.

This article was originally published on August 15, 2011