Fibre Channel, Infiniband and 10 Gb Ethernet

By Henry Newman

I have been thinking for a number of years that slowly but surely Fibre Channel (FC) will be taken over by 10 GbE and that even longer term Inifiniband (IB) will prevail.

I really mis-predicted what the market would do back in my 2008 predictions. I had no idea the economy would tank so far so fast. FCoE did not take over the market, as I predicted, and cost of Ethernet ports did not go down nearly as quickly as I had expected.

But that was then, and this is now. I recently looked quickly at pricing on Froogle for some small FC, 10 GbE and IB switches. Here are the prices. Your mileage may vary, of course.

  • 48 port 10 Gb E switch is ~$7,599 ($158 per port)
  • 32 port FC switch ~$17,999.00 ($562 per port)
  • 36 port QLogic ~$5000 ($138 per port)

This is just switch costs without cables. IB and FC adaptors are far more than 10 GbE already.

My understanding from talking with some people in the industry is that large enterprise Ethernet switches cost per port is far less than large IB switches per port. Also remember that Ethernet has far more functionality than IB, but IB does have lower latency. Does latency really matter for big storage? The latency in the datapath to the drive is orders of magnitude more than the latency difference between IB and Ethernet. Ethernet has been and, at least for the foreseeable future, will be a commodity. Neither FC nor IB ever became a commodity and never, I believe, will be. Hence, the price, no matter which way you look at it, will always be higher. As of today, there are only two IB vendors in business. How can IB become a commodity technology and compete with Ethernet in the long term on price? I think we all know the answer.

This article was originally published on October 10, 2011