IP vs. Fibre Channel SANs

By Alan R. Earls

Although iSCSI and Fibre Channel SANs may be complementary in some situations, if you have to choose, it basically comes down to price and performance evaluations.

The advent of iSCSI-based storage area networks (IP SANs) is generating some controversy relative to established Fibre Channel SANs as some early adopters have finally climbed onboard and observers are placing bets on the outcome.

At issue are questions about cost, performance, interoperability, and reliability. Is iSCSI really going to be cheaper than Fibre Channel, and if so, by how much? Can Gigabit Ethernet really serve up anything close to the throughput associated with

Fibre Channel? Is interoperability in heterogeneous iSCSI SANs really a slam dunk? And can you trust your data to something with a bargain-basement aura? So far, answers have been less than clear-cut. But some clarity—and maybe even a degree of consensus—is emerging. Besides the inevitable FUD that surrounds any new technology there's also a recognition that it's not a question of Fibre Channel or iSCSI-based SANs: Both may be needed to fulfill the growing storage demands at many companies.

This year, the iSCSI market reached a number of milestones on the road to end-user adoption. For example, the standard was ratified early in the year. "Everything before that was all hype and guesses," says Bob Passmore, an analyst with the Gartner consulting firm. And, he notes, some had predicted the spec would only take nine months to develop instead of the three years actually required.

Still, 2003 was not "the year of iSCSI" that some vendors had predicted, and only a handful of companies actually deployed production SANs based on iSCSI—mostly in testing and non-critical environments. But Gartner predicts that 2004 will be a barn-burner year for iSCSI with significant new product announcements in the first half of the year. "I'm guessing both the tire kickers and the risk takers will jump on iSCSI and by mid-2004, we will have enough proof points and reference sites that people will begin to move to iSCSI, and the market will take off," says Passmore.

Of course, notwithstanding the success predicted for iSCSI, it's still not for everyone. For one thing, notes Passmore, there are substantial performance differences between IP and Fibre Channel SANs. First, there is the fact that Fibre Channel is at 2Gbps while most iSCSI SANs run on 1Gbps (Gigabit) Ethernet. However, Passmore notes that the real performance differences have just as much to do with silicon, host adapters, accelerators, and software stacks as they do with wire speed.

It took two or three generations of technology for the storage industry to "get it right" with Fibre Channel, says Passmore, with the result that you can now run virtually any I/O load through a Fibre Channel adapter with little impact on CPU utilization.

By contrast, although everyone has been oohing and aahing over how inexpensive iSCSI is, the reality—lots of expensive add-ons are required to achieve robust performance—is somewhat different. For example, heavy I/O loads may require host adapters with TCP/IP and iSCSI offload engines and accelerators, which can add considerable cost to an iSCSI SAN. On the other hand, some applications may run well using only iSCSI software drivers (which are free) and inexpensive Ethernet network interface cards (NICs).

Again, it can be a question of how iSCSI is deployed and with what expectations. Passmore says under light loads, Fibre Channel offers only slightly better latency than iSCSI, but if the burden on the application server increases, iSCSI performance can degrade sharply.

Of course, none of these issues present insurmountable problems. For instance, a number of TCP offload engines (TOEs) are available from vendors such as Adaptec, Alacritech, Intel, LSI, Xiran and, soon, QLogic and Emulex. However, even with accelerator cards, Passmore notes that iSCSI performance does not quite match the performance of Fibre Channel host bus adapters (HBAs), and the cost differential between iSCSI and Fibre Channel host adapters is shrinking as Fibre Channel HBA manufacturers drop prices.

However, while Fibre Channel costs have been inching lower, it hasn't happened as quickly as many had expected. "Even though manufacturers were seeing cost savings they weren't being passed on to end users," Passmore explains. Why? He says component vendors tended to blame integrators and vice versa. More recently, though, Fibre Channel prices have begun to fall more significantly, perhaps because of the threat of iSCSI.

Passmore predicts that trend will continue with Fibre Channel switches eventually being priced at less than $500 per port and Fibre Channel HBAs at $500. Furthermore, the advent of chip-level Fibre Channel support could cut the cost of HBAs, too. "At the end of the day, iSCSI's price advantage will remain but it will be much smaller," says Passmore.

At least for now, he sees iSCSI confined to the low-end of the market. "There will always be a gray area between the two technologies, but I think iSCSI will be limited primarily to Wintel environments that [haven't implemented Fibre Channel SANs]," Passmore predicts.

Outlook for iSCSI SANs

Other analysts are more upbeat about iSCSI SANs. For example, Steve Duplessie, founder and analyst at the Enterprise Storage Group, predicts success for iSCSI. "Using Ethernet for block-level storage makes way too much sense for it to fail," he says. "There's a lot of economic sensibility behind iSCSI and a lot of big guns behind it."

Duplessie says if there were any doubt about the future of iSCSI SANs, they were settled in Redmond. "I give Microsoft credit for making this market," he says. "They recognized that their world revolves around keeping things simple. By making iSCSI a standard part of their operating systems, Microsoft took the fear out of the equation.

"iSCSI will dramatically alter the landscape for storage networking, and Microsoft is the real driver," continues Duplessie. "And when companies like Dell start to push it, look out."

"IP SANs are generally being adopted the way we anticipated, at mid-sized companies and organizations that have Fibre Channel but don't want to make further Fibre Channel investments for their remote offices," says Arun Taneja, founder and consulting analyst with The Taneja Group. "But IP SAN adoption is still slow." Like Gartner's Passmore, Taneja doesn't expect the iSCSI adoption curve to pick up significantly for another quarter or two.

Taneja expresses some concerns regarding iSCSI interoperability: "Despite all the hoopla about everything being automatically interoperable just because it's IP, that may be hooey." Furthermore, he notes, "there is a big performance difference and a genuine price difference so that puts an automatic demarcation between the two environments."

Application most important

When it comes to comparing Fibre Channel and IP SAN speed, Taneja points out that the application is the most important performance determinant. "So long as the application isn't trying to stream files or as long as hundreds of users aren't putting excessive burdens on the system, I think IP SANs will perform fine," says Taneja. "However, for performance-intensive applications, all bets are off.

"But there are plenty of smaller business environments that aren't performance-sensitive," he adds. "As long as expectations are set correctly and users aren't demanding wire speed, there should be no problem."

Still, some analysts are more dubious about the outlook for iSCSI. For example, John Webster, founder and analyst at the Data Mobility Group consulting firm, says that based on his discussions with VARs there is relatively little evidence of demand for iSCSI. "If this is supposed to be about enterprises extending their SANs out from the core, it doesn't seem to be materializing."

Webster says his suspicions about iSCSI are heightened when he hears vendors excuse their lack of iSCSI products by saying they are waiting for target devices. "There are iSCSI target devices, so what's really hanging this thing up?"

Webster suggests that the problem may be that, despite the maturity of the underlying IP protocol, iSCSI is decidedly less mature than Fibre Channel, which means that enterprises are only willing to experiment with it; and they aren't going to sign on for major deployments. Still, he admits that there is long-term market interest in iSCSI.

Furthermore, Webster does not see any security issues with iSCSI (although others have raised questions about issues such as bi-directional cryptographic authentication, packet authentication, etc.). And while he, too, notes the decline in Fibre Channel prices, he points out that iSCSI, with its potential for much larger volume, may still prove to be a winner as it comes to market and prices drop rapidly due to economies of scale.

Vendors—at least those with a stake in both Fibre Channel and IP SANs—share analysts' mixed opinions on the prospects for iSCSI SANs.

Steve Klotz, co-founder and principal engineer at Medusa Labs (which was acquired by Finisar), a testing and training services company, says he has spent the past four years analyzing the protocols and their relative performance. He predicts that with iSCSI on Gigabit Ethernet it will generally be difficult to get to 30% to 40% of potential wire speed "without a lot of add-ons." However, he admits that with TOEs iSCSI performance improves sharply.

Klotz also points to reliability differences that may frighten some users away from iSCSI. "Fibre Channel data is completely protected—it's designed to be 100% reliable," he says. By comparison, iSCSI—or IP—is designed for expected failures and quick recoveries. What's more, says Klotz, "in the iSCSI standard, cyclic redundancy checking—a data-protection method—is optional, leaving a risk of potential data corruption."

On the negative side of the ledger for Fibre Channel, Klotz notes that Fibre Channel is harder to monitor "because it is all designed in ASICs so there is no way to snoop or tap into it for debugging."

Human side of the equation

On the human side of the equation, Klotz points out that many potential iSCSI users continue to believe that iSCSI will be almost an administrative free ride, since it is built on the ubiquitous IP protocol. However, he notes, "there are lots of storage- and security-specific worries that come with the territory." And Klotz reiterates that iSCSI without add-ons like TOEs is "cheap," but in a more full-featured configuration it's much closer to the cost of Fibre Channel.

Terry Hunt, marketing manager at Finisar, a maker of fiber-optic subsystems and test equipment, agrees, adding that IT people often don't understand storage issues well enough. "We did a presentation at a trade show about iSCSI and storage management, and when you mention that you have to allocate drives it catches some users off-guard," he says.

Bob Preston, marketing vice president at StoneFly Networks, a vendor of iSCSI-based storage-provisioning appliances, says the iSCSI space gets more compelling as more and more users move toward clusters such as Exchange servers.

"That drives you toward a SAN of some flavor, and IP is a value-priced solution for that class of application," says Preston.

In the end, it may boil down to the old adage: You get what you pay for. If you want sports-car performance, you'll have to pay—for FC or a souped-up iSCSI implementation. But if not, IP may good enough.

David Dale, a member of the Storage Networking Industry Association's IP Storage Forum and an "industry evangelist" at Network Appliance, says that since the iSCSI protocol was approved early this year there has been a flurry of announcements from vendors as well as iSCSI initiatives from both Microsoft and Novell, "not to mention iSCSI storage systems from at least a half-dozen vendors and a number of bridge products to link iSCSI to Fibre Channel." (Atto Technology, for example, sells iSCSI bridges.) Dale also notes that a high level of interoperability between iSCSI products from different vendors has been demonstrated at various "plugfests."

Dale says iSCSI offers an appealing alternative for users who might otherwise be considering additional investments in direct-attached storage. That means users like Chris Warlick, IT director at INTEC Engineering. Never a fan of the complexities posed by Fibre Channel, and frustrated by the limitations of DAS and NAS, Warlick is an iSCSI convert. And, aside from the fact that he gets to "re-use" his existing network infrastructure, he also cites direct benefits: a 30% to 40% performance increase with an IP SAN compared to a NAS configuration. (For details about INTEC's—and other companies'—iSCSI SANs, see "Low cost lures users to IP SANs," InfoStor, October 2003, p. 1.)

For Warlick and many other users, that will be more than enough to make iSCSI SANs a viable choice.

Alan R. Earls is a freelance writer in Franklin, MA.

This article was originally published on December 01, 2003