Fibre Channel, Infiniband and 10 Gb Ethernet

I have been thinking for a number of years that slowly but surely Fibre Channel (FC) will be taken over by 10 GbE and that even longer term Inifiniband (IB) will prevail.

I really mis-predicted what the market would do back in my 2008 predictions. I had no idea the economy would tank so far so fast. FCoE did not take over the market, as I predicted, and cost of Ethernet ports did not go down nearly as quickly as I had expected.

But that was then, and this is now. I recently looked quickly at pricing on Froogle for some small FC, 10 GbE and IB switches. Here are the prices. Your mileage may vary, of course.

  • 48 port 10 Gb E switch is ~$7,599 ($158 per port)
  • 32 port FC switch ~$17,999.00 ($562 per port)
  • 36 port QLogic ~$5000 ($138 per port)

This is just switch costs without cables. IB and FC adaptors are far more than 10 GbE already.

My understanding from talking with some people in the industry is that large enterprise Ethernet switches cost per port is far less than large IB switches per port. Also remember that Ethernet has far more functionality than IB, but IB does have lower latency. Does latency really matter for big storage? The latency in the datapath to the drive is orders of magnitude more than the latency difference between IB and Ethernet. Ethernet has been and, at least for the foreseeable future, will be a commodity. Neither FC nor IB ever became a commodity and never, I believe, will be. Hence, the price, no matter which way you look at it, will always be higher. As of today, there are only two IB vendors in business. How can IB become a commodity technology and compete with Ethernet in the long term on price? I think we all know the answer.




Labels: network protocol,Fibre Channel over Ethernet,InfiniBand,fibre channel,EtherNet,10 GbE

posted by: Henry Newman

Henry Newman, InfoStor Blogger
by Henry Newman
InfoStor Blogger

Henry Newman is CEO and CTO of Instrumental Inc. and has worked in HPC and large storage environments for 29 years. The outspoken Mr. Newman initially went to school to become a diplomat, but was firmly told during his first year that he might be better suited for a career that didn't require diplomatic skills. Diplomacy's loss was HPC's gain.

Previous Posts