And you thought it was just SCSI over IP? Not that simple. There are at least four IP storage “standards” (mFCP is submitted as a reference only) that are vying for your attention. Whether you are already in Fibre Channel or waiting for “SCSI over IP,” you need to understand the alternatives and how they may impact your storage networking strategy.
Fibre Channel
Before defining “SCSI over IP,” let’s recap the stack elements that make up the Fibre Channel protocol. The lowest layer, FC0, deals with the physical media (fiber or copper), signal strength, distance supported, etc. FC1 deals with 8b/10b encoding of data, FC2 deals with Class of Service and Flow Control, FC3 is reserved for future use, and FC4 is responsible for layering the protocols supported by Fibre Channel. Currently, these are SCSI-3 (serialized SCSI standard), IP, and VI. The most common implementation is 1Gbps (100MBps) Fibre Channel, with 2Gbps products just arriving on the market. The SCSI-3 implementation for Fibre Channel is called FCP.
iSCSI
First proposed by IBM and Cisco, and subsequently submitted to the Internet Engineering Task Force (IETF) for standardization, iSCSI is now endorsed by a majority of storage vendors, including the Fibre Channel community. Simply put, iSCSI describes porting SCSI-3 over TCP/IP so that block-level data can be carried by any network designed to carry TCP/IP, most notably Ethernet.
The huge benefit of this approach is the use of existing, widely used IP equipment. There is no need to learn a new protocol (Fibre Channel); no requirement for a new, separate network; use of common management tools; and use of more readily available IT networking talent.
A key point to note here is that implementation of SCSI-3 in iSCSI is not identical to SCSI-3 described for Fibre Channel (FCP). This is the basis for controversy, and we discuss it in detail below.
iFCP
Initially proposed by Nishan Systems and now endorsed by a variety of vendors (notable exception: Cisco) and also recently submitted to the IETF for consideration as a standard, iFCP describes implementing FCP over TCP/IP. The primary rationale is that there are hundreds of storage applications for Fibre Channel that have been written to FCP. So, while going to TCP/IP as a transport mechanism is worthwhile, why introduce a different SCSI that would require existing applications to be modified? This makes using existing Fibre Channel equipment (e.g., disk arrays and tape libraries) a lot easier as well.
The thing to note is that the FCP protocol is truly mapped to IP, much like SCSI-3 is mapped to IP in iSCSI.
mFCP
Another protocol proposed by Nishan Systems and submitted to the IETF as a reference document (not required to go through full standardization process), mFCP is identical to iFCP, except that TCP is replaced by UDP. Since UDP is connectionless, it is less reliable than TCP but significantly more efficient. The concept is to use mFCP within the data center and for short-haul networks, but iFCP for longer distances and WANs. This would balance performance and reliability.
FCIP
FCIP is a tunneling protocol and a distinct alternative to iFCP. Whereas the FCP protocol is mapped to IP in iFCP and therefore can be routed just like any other IP packet, FCIP proposes to embed the entire Fibre Channel frame inside an IP packet and send it across in a point-to-point connection across any IP-capable equipment. It is primarily designed to extend a Fibre Channel storage area network (SAN) over remote distances. Cisco, Lucent, and others have submitted FCIP to the IETF for standardization. A number of products are already in the market based on pre-standard versions of FCIP. Vendor examples include Entrada, Computer Network Technology (CNT), SANcastle, and SAN Valley.
iSNS
To understand iSNS, another protocol submitted by Nishan Systems to the IETF, it’s necessary to first understand the Simple Name Server (SNS) as defined for Fibre Channel. SNS is a protocol used by an initiator (e.g. host bus adapter [HBA]) to discover what other storage devices are available in the SAN fabric. Think of a small database resident inside a Fibre Channel switch, where data about each device connected to it is maintained-data such as device type (e.g., disk and tape), SCSI, or Fibre Channel, RAID, or JBOD, etc.
When a requesting device such as a host wants to know all the SCSI RAID devices available to it, a quick query into the database, using the SNS protocol, furnishes it. Switches in more-complex SAN environments share information with other switches so that the database becomes a repository of information on SAN devices. Zoning information is also kept here so that only relevant information is furnished to the initiator. SNS queries are significantly more efficient than polling all devices in the SAN.
In the world of IP, the prevalent methodology for device discovery is Domain Name Server (DNS). For instance, this is where a system goes to translate a URL name to its IP address.
iSNS defines a common protocol that leverages SNS and DNS to create a common method of discovery of storage devices in a complex SAN that includes Fibre Channel and IP. In addition to a 24-bit Fibre Channel address, an SNS entry also includes a 32-bit IP address. iSNS databases can be stored on an IP storage switch, an Ethernet switch, or on a separate iSNS server.
iSNS is designed to be agnostic to iSCSI, iFCP, mFCP, FCIP, or other future protocols. It is, therefore, likely to become a standard.
Controversy: iFCP/mFCP vs. iSCSI
If we were starting over in storage networking with all iSCSI-capable devices (e.g., HBAs, disks, and tapes), or assuming that Fibre Channel didn’t exist, there would be no controversy. The issue is how one deals with existing Fibre Channel SANs and devices while moving toward IP-based SANs.
Assume that for the foreseeable future, disk and tape interfaces will remain SCSI or Fibre Channel. iFCP recognizes this, and while IP SANs may someday replace them all (we think this is highly unlikely), Fibre Channel SANs will continue to be installed at-arguably-a slower pace. This is why iFCP ports FCP to IP, so that existing Fibre Channel equipment is smoothly integrated and existing storage applications need no modifications.
iFCP attempts to replace Fibre Channel switches but keep all Fibre Channel and SCSI devices.
iSCSI, on the other hand, requires existing applications to be modified for use with the new SCSI-3 implemented in iSCSI and relies on iSCSI-only installations for growth. Fibre Channel devices are accommodated via gateways such as the Cisco 5420, which has one Fibre Channel and one Gigabit Ethernet port. The pro-iSCSI argument is: “Only 5% to 10% of storage is implemented in Fibre Channel SANs, so who cares? 90% to 95% of the storage universe is still wide open, and that’s where iSCSI should be targeted. Simply accommodate the few Fibre Channel SANs via storage routers like the Cisco 5420. Next-generation HBAs, RAID systems, and tape libraries will all be iSCSI-capable.”
iFCP vs. FCIP
The iFCP vs. FCIP controversy relates to the interconnection of Fibre Channel SANs over long-haul distances. Since iFCP maps Fibre Channel addresses to IP addresses, it can route traffic over IP networks without any restrictions. Devices can talk to remote devices (N_port to N_port) using an individual TCP connection. Routing is inherent. If a path fails, only that TCP session is affected, and other N_port to N_port conversations can continue. Alternate paths are found for the affected session, and the conversation is reinstated. Also, iFCP doesn’t care if the Fibre Channel SANs are replaced with or supplemented with iFCP or iSCSI SANs: The long-haul process remains unchanged.
FCIP, on the other hand, assumes Fibre Channel SANs exist on both ends and does not convert Fibre Channel frames to IP packets; rather, it tunnels the entire Fibre Channel frame inside an IP packet and disassembles it on the other end. Therefore, it creates a point-to-point connection between Fibre Channel SANs. All nodes talk through this link, and a link disruption means all sessions are impacted.
Another aspect of iFCP for SAN extension is that it maintains autonomous regions for each SAN (i.e., it does not create a single SAN like FCIP does). Failures on one SAN do not propagate across the network with iFCP, as they would with FCIP.
So What Do We Think?
At a pure technology level, we prefer the iFCP approach to FCIP. Tunneling has its limitations, as indicated above. iFCP also does a better job of recognizing the existence of Fibre Channel SANs/devices. We also like the fact that existing applications that are FCP-aware require no changes. Had this technology been presented to the IETF with Cisco, IBM, and others, there would probably have been a much stronger recognition of migration requirements (Fibre Channel to IP) in the iSCSI spec. In that case, we would not have needed the FCIP protocol at all. But it is never that simple when conflicting interests are involved.
On the surface, the battle between FCIP and iFCP appears to be between Cisco and Nishan. In reality, we think it’s a battle between Brocade and Cisco. FCIP is a Fibre Channel perpetuation strategy and, therefore, Brocade and other Fibre Channel stalwarts love it. iFCP (or iSCSI) is an IP strategy and, therefore, Cisco and Nishan love it. These latter two companies, while often at odds with each other, are both trying to move the world to an IP-only state. Too bad they didn’t work together in the beginning to produce one spec to get us there.
We think Cisco will continue to provide lip service to FCIP and might even do an FCIP-capable blade for its Catalyst switch, but this will only be a stepping stone toward an all-iSCSI universe. In that world, anytime someone needs to connect to the “legacy” world of Fibre Channel, he or she would use a storage router such as Cisco’s 5420 storage router.
We are impressed with the approach Nishan has taken with its products. Each port can be set to Fibre Channel, iFCP, or iSCSI. Therefore, end users can attach Fibre Channel disks and tapes today, start to enjoy the commonality with other IP equipment, interconnect SANs using iFCP or iSCSI, and still be fully prepared for all-iSCSI SANs, if so desired.
Cisco’s 5420 router is designed to create simple SANs today using existing Fibre Channel devices and iSCSI drivers installed on standard Ethernet network interface cards. It is highly unlikely that one would build a large-scale SAN using this product. But once products with multiple Fibre Channel and Gigabit Ethernet ports become available and TCP/IP hardware acceleration is integrated on the NICs/HBAs, you can start building larger mixed (Fibre Channel and iSCSI) SANs. Products with multiple ports are just beginning to ship from other vendors. TCP/IP-accelerated versions of NICs are expected to be available by year-end. iSCSI RAID arrays are now shipping from IBM and 3ware, allowing you to build simple iSCSI-only SANs, albeit at the expense of performance.
We think iSCSI and iSNS are sure winners. Also, we believe both FCIP and iFCP will have a place in the market and will get endorsed as standards.
Bottom Line
The good news is that no matter which path you take, you will be well covered. We recommend you make your decision recognizing that most enterprise environments will have a mix of Fibre Channel, iSCSI, FCIP, and, possibly, iFCP protocols. Your investment is likely to be protected if you buy equipment that can deal with these multiple protocols.
If you have already invested heavily in Fibre Channel, have finally stabilized your environment, and are starting to enjoy the fruits of your labor, keep buying Fibre Channel equipment.
But while you do this, consider a pilot with a mixed environment (Fibre Channel, iFCP, and iSCSI), learn the “gotchas” now, and be prepared, when necessary, to move to IP SANs over the next few years. Certainly test out an all-iSCSI pilot, keeping in mind the statement we have made on performance and applications availability.
On the other hand, if you have not yet begun Fibre Channel implementations and are falling behind in storage management, jump into Fibre Channel now, preferably in conjunction with a good third-party expert. Fibre Channel is the only fully baked SAN technology that’s ready for mission-critical applications today. You can certainly consider switches (e.g., Nishan’s) that can be all-Fibre Channel today but are capable of morphing into iFCP and/or iSCSI, or Fibre Channel and iSCSI, in the future.
If, however, you are not feeling the pinch, and you have time on your side, you have two paths to consider: iFCP or full iSCSI. Whichever path you take, make sure the applications you need are available and robust. We are ambivalent in this scenario between the Nishan and Cisco approach. Both will eventually take you to iSCSI.
Keep in mind that storage devices are Fibre Channel or SCSI today, with a few entries in the iSCSI category. Don’t expect native iSCSI interfaces on disk drives or other storage devices anytime soon. iSCSI interfaces will be presented via front-end controllers.
While we like the technical elegance and the market savvy of iFCP, we are concerned that Nishan will not get adequate traction from the industry to make iFCP a commercial success. This fact has not gone unnoticed by Nishan, which is now supporting both iFCP and iSCSI on each port. This means that as long as you are choosing a product that lets you build a mixed Fibre Channel and iFCP SAN, or iSCSI SAN, today, with the promise to smoothly migrate to an iSCSI-only SAN in the future, you are likely to do just fine.
Either way, sitting on the fence much longer is not a recommended course of action. Based upon the information provided and your specific IT strategy, get into the game now. You’ll be glad you did.