Next-generation FC SANs: 2Gbps fabrics

Fibre Channel switches running at 2Gbps are just coming to market, paving the way for a transition to 2Gbps storage area networks.


Storage area networks (SANs) continue to evolve as their capabilities expand. The original Fibre Channel-based SANs were simple loop configurations based on the Fibre Channel-Arbitrated Loop (FC-AL) standard. As SANs got more sophisticated and required more scalability and functions, end users began moving to fabric-based switching.

Fabric switching solved many problems, but it also introduced new ones. Immature standards and different implementations among vendors resulted in interoperability issues and forced users into homogeneous, proprietary, and expensive SANs.

The latest movement is toward 2Gbps fabric switches based on interoperability standards and run at twice the speed of previous SANs. Switches based on standards provide heterogeneous capability that will reduce not just the management costs of running a SAN, but also the price of the components.

Benefits of open fabric

The new open fabric standard-Fibre Channel Switch Fabric 2 (FC-SW-2)-will be the foundation of new 2Gbps products. FC-SW-2 offers a number of features that will facilitate the use of 2Gbps Fibre Channel fabrics. For example,

FC-SW-2 defines a distributed name service that specifies the way in which switches communicate distributed name server information. FC-SW-2 also defines a distributed management server that allows users to manage fabric- connected devices from outside the fabric. Users can determine the topology of the network, for example, or find redundant links for fail-over use.

In addition, FC-SW-2 defines mechanisms for exchanging zoning information, which limits a device's view of the fabric to a subset of other devices. A group of drives might be visible only to Solaris servers, for instance, while NT servers see only another group of drives.

One feature not included in FC-SW-2 is a definition for implementing open trunking over multiple interswitch links (although vendor-specific methods are available). FC-SW-2 does include standard protocols for a single interswitch link via E_Ports; however, these protocols specify how E_Ports are to exchange fabric parameters such as the amount of buffer-to-buffer credit available from each side, the relevant time-out values, and the class of service supported. Users who want to take advantage of these FC-SW-2 protocols must use a single interswitch link. This requirement makes the 2Gbps speed especially important for maximizing throughput on the interswitch link and avoiding proprietary trunking schemes.

Ease of integration

By taking advantage of the Fibre Channel Industry Association's SANmark qualification and trademark licensing program, vendors of 2Gbps products will be able to ensure interoperability. The SANmark program is designed to ensure conformance to common network operations that promote heterogeneous Fibre Channel networks. Products that successfully complete the qualification program will carry the "SANmark" label.

At 2Gbps, Fibre Channel also continues the reliability and ease-of-use features that have helped make SANs viable across multiple cost/performance levels-from small systems to supercomputers. Specifically, Fibre Channel provides network features such as protocol multiplexing, as well as traditional channel features for simplicity, repeatable performance, and guaranteed delivery. The 2Gbps products will continue to use all of the existing Fibre Channel protocols to allow a seamless migration. No changes to drivers or Upper Layer Protocols (ULPs) are needed to take advantage of the improved performance of 2Gbps Fibre Channel.

Benefits of 2Gbps

2Gbps Fibre Channel promises a straightforward way to double SAN speeds, just as Gigabit Ethernet provided an easy transition to higher LAN speeds. This Fibre Channel upgrade also enables greater flexibility in configuring SANs for a wide range of applications and will be especially useful for managing 1.5Gbps high-definition video data.

In addition, with enhanced Fibre Channel standards on the horizon and third-party testing such as the SANmark program, interoperability is ensured for 1Gbps and 2Gbps products. These efforts support a Fibre Channel open fabric that offers investment protection for users upgrading to 2Gbps products. Users also have the opportunity to take advantage of a new modular connector that significantly increases port densities.

The benefits of 2Gbps Fibre Channel SANs and associated standards include

  • The ability to stay ahead of SAN bandwidth requirements in the face of faster server buses, faster network connections (both LAN and WAN), and increasing demands for data backup, data warehousing, and online transaction processing (OLTP);
  • The ability to stream uncompressed HDTV-type video;
  • The ability to implement SAN backbones with particular benefits for creating interswitch links (using new standards-based E_Ports) and ensuring in-order delivery of video packets (compared to using two 1Gbps links);
  • Greater flexibility and lower costs from reducing the number of ports needed to achieve a desired throughput;
  • Greater port density due to the use of smaller adapters and connectors;
  • Better assurance of interoperability for products from vendors who participate in SANmark and "plugfest" programs;
  • Greater SAN versatility from the use of new standards-based features such as distributed name server, management server, and zoning; and
  • A straightforward transition to 2Gbps through backward-compatibility with 1Gbps products and auto-negotiation to determine a link's appropriate speed.

Any enterprise that needs more raw-data throughput can benefit from the upgrade to 2Gbps Fibre Channel. Examples include e-commerce companies that must minimize transaction times and storage service providers that must deliver block data to users at the highest possible speeds.

Organizations involved in the production, post-production, and broadcasting of 1.5Gbps high-definition video (as in HDTV) will find the 2Gbps channels indispensable. Now a single fiber can carry a full high-definition video stream without having to cache, buffer, or compress the data.

Enabling backbones

2Gbps Fibre Channel provides the first high-speed backbone capability for Fibre Channel networks. A 2Gbps channel can be used to interconnect two SAN switches, for example. This arrangement boosts overall data throughput across the SAN even if the disk subsystems, servers, etc. continue to operate via 1Gbps channels.

Similarly, a 2Gbps interswitch channel can serve as a simple interconnect for SAN islands (see Figure 1). These islands might represent storage facilities in different buildings and/or special-purpose facilities such as tape backup islands.

Figure 1: 2Gbps links can interconnect multiple SAN islands and devices into a hierarchical fabric.
Click here to enlarge image

The use of a single interswitch link (one 2Gbps link rather than two 1Gbps links) eliminates the synchronization issues that arise when packets can travel over multiple paths. If one path is slightly faster than another, packets might arrive out of order, or excessive latency may occur while the switch delays frames to preserve the original order. Fibre Channel devices provide the best error-detection when packets are delivered in order, so using a single 2Gbps interswitch link helps ensure robust performance.

2Gbps Fibre Channel reduces the number of ports needed to achieve a desired throughput and gives SAN architects greater flexibility. As SANs expand beyond simple hub and mesh topologies, 2Gbps links will be crucial for interconnecting multiple SAN islands and devices into a hierarchical fabric. A simple rule of thumb is to deploy a 2Gbps fabric backbone connecting 1Gbps clusters or 1Gbps fanout configurations. This approach will tend to give the highest price/performance for the functions required.

Higher port density

The new Small Form-factor Pluggable (SFP) modular connector for Fibre Channel is dramatically improving port densities in new products such as 2Gbps equipment. While not strictly necessary for 2Gbps operation, the SFP connector was developed with higher speeds in mind.

The SFP connector has a 0.65-inch minimum horizontal pitch, in contrast to the 1.5-inch space required for the traditional GBIC. SFP easily doubles the number of ports that a standard equipment rack can support.

As with the GBIC, the SFP module acts as an adapter that allows use of either fiber or copper cables by accommodating the appropriate interface circuitry and cable connector. For fiber, the SFP uses the new dual-LC connector, a half-size version of the existing dual-SC connector. As with the GBIC, the SFP module contains all the circuitry needed to support an optical interface. For copper cables, the SFP uses the new HSSDC-2 connector, a half-size version of the existing HSSDC.

The transition to 2Gbps

The key to making an easy transition to 2Gbps throughput is the use of Fibre Channel host bus adapters (HBAs) and other network interfaces that run at either 1Gbps or 2Gbps and auto-negotiate their speed (see Figure 2). A device such as a RAID subsystem that has a 2Gbps/1Gbps Fibre Channel interface automatically detects whether the attachment to the network is running at 1Gbps or 2Gbps and operates at the appropriate speed.

Figure 2: Fibre Channel host bus adapters (HBAs) can run at 1Gbps or 2Gbps and can auto-negotiate the appropriate speed.
Click here to enlarge image

Mixed-speed environments are a part of the transition from 1Gbps to 2Gbps. In addition, when the infrastructure includes a 2Gbps-capable switch, a 2Gbps device can communicate through the switch to a 1Gbps device. The switch manages the difference in speed through the use of buffers and flow-control mechanisms. This capability allows for a mix of devices running at 1Gbps or 2Gbps at the same time.

2Gbps/1Gbps HBAs offer the same opportunity for SANs. The speed-negotiation features of the Fibre Channel Framing and Signaling (FC-FS) standard specify how vendors are to implement the capability and help ensure interoperability among different vendors' products.

How speed negotiation works

A Fibre Channel device begins speed negotiation at its highest speed to see whether a valid connection can be made with another device. If a connection is not possible at the highest speed, the device reduces its speed until an acceptable speed is found. Starting at the highest speed and stepping downward results in the quickest possible convergence to the highest speed supported by both devices.

Fibre Channel speed negotiation is a point-to-point process. Thus, in an environment where a hub provides connectivity, the hub must operate at a fixed rate (as opposed to devices negotiating around the entire loop). Because data is passed around the segments of the loop, the hub must determine the speed of each link in the loop and force all attached devices to the highest speed supported by all the devices. Using this model, any single device operating at 1Gbps will force the entire loop to operate at the lower speed. Using a switch (rather than a hub) to provide the interconnect offers the advantage that 2Gbps devices can operate at the higher speed even though 1Gbps devices are also connected to the switch.

Another speed-negotiation consideration is that JBOD configurations must auto-negotiate as a single entity. This requirement is necessary because speed negotiation must take place within a limited time window. But multiple disks cannot participate in speed negotiation without violating the timing window. Thus, JBODs must operate at a fixed rate or have a single controller (such as a SCSI enclosure service controller, or SES) to speed negotiate for the entire JBOD.

Standards in place

The standards underlying 2Gbps Fibre Channel are either finalized or close to completion. The FC-SW-2 standard is nearly complete. In the final stage of the approval process, the draft standard is sent out for public review, which typically generates only minor revisions. FC-SW-2 is built upon experience with first-generation FC-SW implementations and ensures backward-compatibility with first-generation fabrics.

The 2Gbps/1Gbps speed-negotiation protocol has been accepted into the Framing and Signaling (FC-FS) draft standard. This draft is expected to be approved around October.

The Fibre Channel Physical Interface (FC-PI) standard, which details the physical characteristics of the 2Gbps link and other enhancements to the FC-PH standard, has been forwarded by NCITS Technical Committee T11 for further processing, including first public review.

Two other draft standards that are of interest for 2Gbps Fibre Channel are Methodologies for Interconnects (FC-MI) and Generic Services 3 (FC-GS-3). FC-MI specifies an interoperability profile for infrastructure and edge devices and was expected to reach the letter ballot stage last month. FC-GS-3 covers generic ser vices, including name server and management server. This draft standard has completed the public review process and should be published soon.

The components needed to implement 2Gbps Fibre Channel SANs are available today. These components include disk subsystems; HBAs; optics; GBICs that adapt subsystems to different media (e.g., singlemode or multimode fiber or copper); SFP modular connectors for improving port density; chips for converting between parallel and serial data (SERDES); switches; hubs; and cabling. And computer manufacturers are integrating 2Gbps technology. The improved throughput, flexibility, and interoperability of 2Gbps open fabric Fibre Channel represent the next generation of SANs.

Erik Ottem is senior director of product marketing at Gadzoox Networks (www.gadzoox.com) in San Jose, CA.

This article was originally published on July 01, 2001