Making the move to 2Gbps Fibre Channel

Fibre Channel devices operating at 2Gbps are shipping, and adoption is expected next year.

By Mike Smith

The tremendous growth in data storage is well documented, driven by a number of factors, including the Internet and e-business. To keep pace with this growth, Fibre Channel storage area networks (SANs) need ever-increasing bandwidth for user access, application processing, and data management. Of the various methods that can be used to provide increased SAN bandwidth, the most cost-effective and seamless is to double the link data rate to 2Gbps. Systems that are "2Gbps-ready" provide investment protection by offering compatibility with today's 1Gbps SANs, while giving users an upgrade path to 2Gbps performance as more devices become available.

2Gbps Fibre Channel technology is an extension of the current 1Gbps standard, operating at 2.125GHz and providing link bandwidth of 200MBps half-duplex, or 400MBps running full-duplex. Fibre Channel at 2Gbps supports the same topologies and protocols as 1Gbps Fibre Channel and is backward compatible with existing 1Gbps Fibre Channel equipment.

Figure 1. In a network with both 1Gbps and 2Gbps devices, the devices automatically negotiate down to 1Gbps. To achieve maximum data rates from 2Gbps devices, they must be used on a "pure" 2Gbps link-either a 2Gbps-only environment or a mixed-speed environment with the 1Gbps devices isolated from 2Gbps devices.
Click here to enlarge image

The major benefit of 2Gbps Fibre Channel is a higher-speed "pipe" to connect performance-critical devices. It also provides more effective bandwidth matching between the PCI bus (532MBps) and the link (400MBps). This eliminates performance bottlenecks and enables more efficient use of system resources. 2Gps Fibre Channel also provides a more effective way of using the emerging PCI-X bus, which promises to deliver 1Gbps bandwidth.

How it works

One goal of the Fibre Channel industry in defining the 2Gbps standard was to provide the maximum speed common to connected devices. However, the Fibre Channel standard requires that all connected devices operate at a common speed. In a mixed environment with both 1Gbps and 2Gbps devices, those devices will operate at the speed of the slowest device attached to the link. This is true of both loop and point-to-point connections. For example, if a 1Gbps device and a 2Gbps device are connected to the same link, the link must operate at 1Gbps. Likewise, if both 1Gbps and 2Gbps devices are attached to the same arbitrated loop, the loop must operate at 1Gbps.

Figure 2. Devices and systems with mismatched transfer rates will communicate at 1Gbps.
Click here to enlarge image

To achieve maximum data rate from 2Gbps devices, they must be used on a "pure" 2Gbps link. This can be achieved in a 2Gbps-only environment or in a mixed-speed environment in which the 1Gbps devices are isolated from the 2Gbps devices (see Figure 1). Isolation can be achieved by using a switch that accommodates both 1Gbps devices and 2Gbps devices on separate ports or by separating arbitrated loops with a segmented hub or multiple hubs.

Automatic speed negotiation

The Fibre Channel standard does not require devices to automatically determine the appropriate operating speed. The speed may be fixed at 1Gbps or 2Gbps, or it may be configurable using jumpers, switches, or software utilities. However, devices that feature automatic speed negotiation provide a significant advantage, facilitating interoperability, ease of use, and compatibility within existing 1Gbps infrastructures. Automatic speed negotiation enables a device to detect the speed of another device on the connection and set the appropriate operating speed automatically.

In the case of a point-to-point connection, the speed negotiation process starts after power-up, with each Fibre Channel device sending a high-transition density pattern onto the link. This signal serves as a synchronization signal for the receiving device's phase-locked loop (PLL). The device also monitors an incoming signal. If the receiving device recognizes and acknowledges the synchronization signal and it matches the speed of its outgoing synchronization signal, the two devices are set to communicate. If no acknowledgment is received, or if the incoming signal is at a data rate different than that sent out, the devices cannot communicate. One of the devices must be set to the speed of the other device, either manually or automatically.

In this scenario, the host adapter sends a 2Gbps synchronization pattern to the storage device. The storage system, which is a 1Gbps device, can't recognize the incoming pattern and responds with a 1Gbps data stream. After a period of time, the host adapter re-sends a 1Gbps synchronization pattern, which the storage system recognizes and acknowledges. Both devices are now ready to communicate at 1Gbps data rate (see Figure 2).

Figure 3. The most significant performance difference between 1Gbps and 2Gbps Fibre Channel occurs with block sizes between 4KB and 8KB.
Click here to enlarge image

In the case of an arbitrated loop, each device is essentially connected to, and communicating with, the hub. The attached device issues a synchronization pattern (LIPs and IDLE characters), which is received by the hub. If the hub recognizes the characters, it responds and a connection is established. If the hub cannot recognize the characters, it will not establish a connection with the device. The device must then re-issue the synchronization pattern at the alternate data rate to establish a connection. If the hub determines that any of the connected devices are 1Gbps devices, the entire loop will be forced to operate at 1Gbps.

Each connected device must use the same scheme for synchronization. For example, each device should start at the same frequency (1Gbps or 2Gbps) and allow a pre-determined period of time for synchronization and acknowledgement. There are two proposals currently under review by the industry standards bodies, one of which appears to have gained the majority of support. The proposals define the same basic mechanism for sensing and, ultimately, selecting the required data rate. The proposals differ in the specifics, such as initial synchronization speed, transmit vs. receive speed, and how much time should be allocated for receiving an acknowledgement.

Status report

A wide range of 2Gbps Fibre Channel products are currently available or in development, including host bus adapters (HBAs), disk drives, hubs, switches, connectors, enclosures, and test equipment. And a number of manufacturers have partnered to develop and demonstrate interoperable 2Gbps products (as demonstrated at this month's Comdex show).

Not all 2Gbps products in development support automatic speed negotiation. Some may be fixed at 2Gbps, while others may have jumper or configuration option settings.

Performance testing of 2Gbps products shows significant improvements over the 1Gbps data rate. Current testing shows bandwidth of 195MBps using a single loop and host adapter, running in half-duplex mode. In addition to the obvious bandwidth improvements, I/O processing capability is improved significantly at the 2Gbps data rate, due to reduced data phase time, which leads to faster command execution.

Figure 3 shows the performance test results of a 2Gbps HBA with eight Seagate X15 disk drives running at 1Gbps and 2Gbps, using Intel's iometer benchmark running under Windows NT, to test host adapter performance.

While the top-end results at the smallest block size show little difference between 1Gbps and 2Gbps operation, the results show a significant difference at 4KB and 8KB block sizes, which are commonly used by popular applications and in TPC-C benchmark testing. The 2Gbps data rate yields a 30% to 50% improvement over 1Gbps in I/O performance, using the same host adapter and disk drives.

Physical interface

2Gbps Fibre Channel operates over the same multi- and single-mode fiber-optic cables used for 1Gbps Fibre Channel. Because of the increased challenge with emissions and cross-talk when operating at 2GHz frequencies, the industry focus has been on deploying optical solutions. Distances supported for multimode links will be less at 2Gbps than that supported by 1Gbps Fibre Channel (see Figure 4).

The connectors represent a significant change using 2Gbps Fibre Channel products. 2Gbps products will use a new high-density, small-form-factor (SFF) transceiver, rather than GBICs or 1x9 transceivers used in 1Gbps products. The SFF transceiver, which had previously been adopted by the Ethernet community, is expected to provide lower cost over the long term, due to higher aggregate volumes across all markets. Because of its much smaller size, the SFF transceiver also enables higher port density on hubs and switches, as well as multi-channel host adapters.

Removable media will be provided via the small-form-factor pluggable (SFP) transceiver. The LC-style connector, which supports both short- and long-wave optics, is the industry's choice.

Deploying 2Gbps

Products capable of 2Gbps performance, including host adapters, hubs, SFF optics, and connectors will begin shipping in production volumes this quarter. Widespread adoption and deployment of 2Gbps Fibre Channel products is expected to begin next year. Initially, 2Gbps Fibre Channel will be shipped as part of "2Gbps-ready" systems. A small number of these systems will be deployed in pure 2Gbps environments and fully use the higher data rate. More typically, the initial systems will be installed in 1Gbps environments and will operate at a 1Gbps data rate. As additional equipment such as switches and storage subsystems become available, the systems will be able to automatically "upshift" to operate at the 2Gbps data rate. This capability provides significant investment protection for users, because the installed base will not need to be upgraded.

Figure 4. In a short-wave optical environment, 2Gbps Fibre Channel is limited to 300 meters.
Click here to enlarge image

As 2Gbps peripherals are added to predominantly 1Gbps SANs, it is prudent to segment the SAN into 1Gbps links and 2Gbps links. This will enable the 2Gbps components to operate at maximum speed, while maintaining full connectivity to the existing 1Gbps infrastructure. As described earlier, this can be accomplished using a switch that can accommodate both 1Gbps and 2Gbps ports or by using separate arbitrated loops. With 200MBps bandwidth available on a half-duplex loop, it may be beneficial to attach multiple peripherals to a switch port to aggregate bandwidth and lower system cost.

The most cost-effective and seamless way to improve SAN bandwidth is to increase the Fibre Channel link data rate to 2Gbps. 2Gbps Fibre Channel has the capability to deliver significant performance improvements over today's 1Gbps Fibre Channel products.

As the above performance data shows, I/O performance can be improved significantly, as well as bandwidth performance. Automatic speed negotiation provides the best opportunity to deploy interoperable solutions and facilitate ease of use. This allows the deployment of "2Gbps-ready" systems that can operate in an established environment at 1Gbps, while providing investment protection by up-shifting to 2Gbps as new equipment is added to the SAN.

Mike Smith is vice president of worldwide marketing at Emulex Corp. (www.emulex.com), in Costa Mesa, CA.

This article was originally published on November 01, 2000