By Kevin Komiega and Dave Simpson
Unlike the market for, say, Fibre Channel disk arrays, the markets for Fibre Channel switches and host bus adapters (HBAs) are not crowded. In both cases, two vendors essentially comprise a duopoly in terms of market share: Brocade and Cisco in switches, and Emulex and QLogic in HBAs.
Despite, or perhaps because of, the paucity of players, each vendor goes to great pains to differentiate its products from its competitors' offerings. For this article, we asked representatives from the leading Fibre Channel switch and HBA vendors to briefly outline what they think are their key product differentiators, and what the benefits are to end users.
The Fibre Channel switch market has evolved well beyond speeds, feeds, and port counts. Among Brocade, Cisco, and QLogic, customers can buy fabric switches with as few as eight ports and up to as many as 256 ports and more in director-class products.
Each of the switch vendors supports 8Gbps Fibre Channel, but the mantra seems to be “migrate at your own pace.” Most 8Gbps support can be activated for a license fee so users can buy the latest and greatest switches now, but do not have to commit to 8Gbps premiums, which are slightly higher than the cost of today’s 4Gbps technologies, until the need arises.
Some of the key advancements in Fibre Channel switching have been in the areas of management, support for virtual servers, and fabric-based services.
Now that it has completed the integration of hardware and software platforms obtained through the acquisition of McData, Brocade is focusing on Adaptive Networking services, a set of fabric-based applications to help improve server and storage utilization, optimize network and application performance, and increase security.
Marty Lans, senior director of data-center marketing at Brocade, says the physical network takes a backseat to network management technologies that impact flow control, quality of service (QoS), and application mobility as explosive adoption of virtual servers continues to stress the fabric.
Adaptive Networking services comprise a suite of tools for optimizing fabric behavior and ensuring bandwidth availability for applications. These tools include QoS, Ingress Rate Limiting, Traffic Isolation, and Top Talkers. Another feature, dubbed Integrated Routing, allows users to connect switches in different fabrics.
Brocade’s switches feature native support for expansion ports (E_Ports), which are used to connect to another Fibre Channel switch or bridge device through an inter-switch link. E_Port interoperability across Brocade Fabric OS (FOS) and M-Enterprise OS (M-EOS) devices and fabrics allow Brocade switches to connect to other E-port-capable devices to support SAN scaling.
The company’s switches use a non-blocking architecture with as many as 40 ports concurrently active at 8Gbps (full duplex) with no over-subscription, providing an overall bandwidth of 320Gbps. In addition, enhanced Brocade ISL Trunking enables a single logical link at up to 64Gbps of data throughput.
Brocade also touts power efficiency as a key differentiator. For example, the Brocade 5100 switch consumes less than 2.5 watts of power per port and features consolidated power and fan assemblies to improve environmental performance and reduce overall ownership costs.
Brocade’s Data Center Fabric Manager (DCFM) software can be used to manage switches, HBAs, and fabrics.
Most recently, Brocade began shipping Data Center Fabric Manager (DCFM) 10.0 software, which is a unified storage network management application that manages the SAN from ports on networked storage systems to HBAs attached to physical or virtualized servers (see figure). DCFM combines McData Enterprise Fabric Connectivity Manager (EFCM) and Brocade Fabric Manager into a single fabric management application.
The company recently began shipping a pair of Fibre Channel encryption products: the stand-alone Brocade Encryption Switch and the FS8-18 Encryption Blade for the company’s DCX Backbone chassis (see “Brocade ships encryption switches,” p. 8). The stand-alone encryption switch is a 32-port, auto-sensing 8Gbps Fibre Channel switch, while the FS8-18 offers the same speeds and encryption services in a 16-port blade. Both models provide up to 96Gbps of encryption processing power and work with key management systems from several vendors to secure data “at-rest.”
Cisco’s MDS 9000 Storage Networking platform has been in place since 2002. While the features and functions of the MDS family have changed dramatically through plug-in modules, speed upgrades and software enhancements, the hardware platform itself has remained intact. Platform consistency is a key differentiator, according to Rajeev Bhardwaj, senior director of product management for Cisco’s Data Center and Switching Technology Group (DCSTG).
Bhardwaj says the MDS family has not changed, it has adapted—an important factor in preventing costly forklift upgrades in order to take advantage of new technologies such as 8Gbps connectivity.
Cisco claims advantages in the areas of support for QoS and security for virtual server environments and the inherent data-protection capabilities of the MDS family, including replication, snapshot technologies, and support for serverless backups—all in the fabric.
Bhardwaj also says Cisco was first out of the gate with end-to-end SAN virtualization because of its Virtual SAN (VSAN) technology, which was named by the American National Standard Institute (ANSI) as the industry standard for deploying Virtual Fabrics. Cisco VSANs segment a single, physical SAN fabric into many logical, completely independent SANs (see figure ).
Cisco’s VSAN technology segments physical SAN fabrics into independent logical SANs.
The company’s ability to put storage applications into the network fabric spawns from the Cisco Service-Oriented Network Architecture (SONA), a three-tier architecture that includes network infrastructure, management services, and an application layer. In its entirety, the service-oriented approach is aimed at making it easier to deploy heterogeneous storage applications in the fabric.
Bhardwaj also claims Cisco was first to enable fabric-based encryption on the MDS platform with Storage Media Encryption (SME). The SME feature enables management of data encryption across multiple types of heterogeneous SAN-attached storage devices, including disk arrays, tape devices, and virtual tape libraries (VTLs). Bhardwaj claims that encrypting data in the fabric helps companies secure data on storage devices that lack native encryption capabilities.
Some other Cisco features include an on-demand port activation license, integrated IBM Fibre Connection (FICON) connectivity support, and a Quick Configuration Wizard that quickly creates a SAN environment for SMB applications. The wizard allows server access to storage in a single step.
In recent months, Cisco has also consolidated its network management platforms. The company changed the name of its SAN-OS operating system to NX-OS to accommodate the company’s new line of Nexus switches, designed to bring unified networking based on 10Gbps Ethernet (10GbE) to the data center. Cisco’s ultimate goal is to develop a single operating system that will support a unified fabric combining both LANs and SANs. In addition to support for 8Gbps performance, NX-OS for the MDS platform also includes policy, visibility, and diagnostic tools for managing both physical and virtual server environments. The NX-OS is the foundation for Cisco’s SAN services such as QoS and security for virtual machines.
The third player in the Fibre Channel switch market, QLogic, hangs its hat on value. Marty Holmes, senior product manager for switches with QLogic’s Network Storage Solutions Group, says each of the company’s SANbox 5800V switches ships with all four 10Gbps stacking ports active by default. This cost-saving feature means basic inter-switch link (ISL) is now included with the product.
Every time a switch is added to a fabric, it results in a diminishing return because it must be connected to every other switch on the SAN. The more ports used for ISLs, the fewer available for storage devices.
Holmes says competitors’ non-stacking edge switches force users to reserve ports for future ISLs or risk disruption during expansion as new switches must “steal” device ports from existing switches. With the 5800V, ISL bandwidth provisioning is automatic, enabling the addition of new switches to a stack at any time without affecting the SAN. Up to six switches—120 device ports—can be stacked and managed as a single unit using QLogic’s switches.
QLogic has also increased the speed of its stacking ports from 10Gbps to 20Gbps—without additional cost. The included stacking ports add an extra 50Gbps bandwidth per switch, which is the equivalent of six extra 8Gbps Fibre Channel switch ports. When upgraded to 20Gbps, the same ports provide more than 100Gbps of extra bandwidth, or the equivalent of 12 extra 8Gbps ports per switch.
Given the consensus that 2Gbps and 4Gbps performance is good enough for now, QLogic is offering a phased “pay as you grow” migration to 8Gbps. Holmes says the 5800V helps users defer migration costs by allowing them to install a future-ready hardware platform, without paying for bandwidth before they need it.
The company’s ISLs are auto-trunking by default, which is another key feature, according to Holmes. Bandwidth capacity from multiple physical ISLs is pooled in a single high-speed pipeline. Instead of becoming queued and trapped within individual pathways, traffic is interleaved and load-balanced to fill the entire pipeline, resulting in faster storage access and application performance for users.
QLogic’s trunking implementation occurs automatically, with no need for manual configuration. Trunked ports do not need to be sequential, and up to 128 ISLs can be included in a single trunk. Trunks will even occur automatically across multiple switches. As long as the paths are the same length, traffic will be evenly distributed for maximum throughput.
QLogic’s cost savings go all the way down to the cabling. The company’s switches will work with any vendor’s SFP transceivers, while Holmes says some competitors require customers to buy their own, branded SFPs.
Support for virtual machines is on the horizon. QLogic plans to announce vFabric, a technology designed to simplify the administration and control of growing SANs while maintaining the cost benefits of shared resources. All of QLogic’s 8Gbps products will support the new vFabric virtualization platform, which will become available to customers as a future software release. —Kevin Komiega
Host bus adapters
On the HBA front, the latest advancement is support for 8Gbps Fibre Channel, and most of the differentiation emphasis centers on ease of use/configuration and, more importantly these days, features that optimize I/O in virtualized environments.
Emulex, for example, is quick to extol its role in the development of the N_Port ID Virtualization (NPIV) standard and its breadth of platform support for NPIV, which includes virtually all virtual-server operating systems (including Microsoft’s Hyper-V), as well as conventional operating systems, including Solaris. The company also claims that it supports more virtual ports per HBA than its competitors.
For end users, the key benefit of NPIV, according to Scott McIntyre, Emulex’s vice president of product marketing, is that it enables administrators to maintain their SAN management best practices and tools as they move from physical to virtual environments. NPIV gives a virtual machine a virtual World Wide Name (WWN) so that users can, for example, zone virtual machines separate from other virtual machines on the same physical server. NPIV also enables LUN masking and mapping on virtual machines, as well as other functions.
Emulex also touts the scalability and ease-of-use of its HBA management tools, exemplified by its HBAnyware software, which enables administrators to manage all Emulex HBAs across an enterprise from a single pane of glass, including automating tasks such as driver updates and firmware downloads (see screenshot).
Emulex’s HBAnyware software enables administrators to manage adapters across an enterprise from a single console.
In addition, Emulex claims users can install and configure its Fibre Channel HBAs in only seven steps, in fewer than 15 minutes.
Emulex also highlights its common driver and firmware architecture, which spans multiple generations of HBAs—and converged network adapters (CNAs) based on the Fibre Channel over Ethernet (FCoE) standard—and the ability to manage all devices from a single console.
The company also points to the CPU efficiency, or CPU effectiveness, of its HBAs, a measure of how much of the I/O load an HBA can handle as a percentage of CPU use. For end users, the benefit of CPU efficiency is that it enables them to run smaller, or fewer, servers and/or run more virtual machines per CPU core.
Most recently, Emulex announced the first implementation of data integrity features that came out of the Data Integrity Initiative, a joint initiative initially involving Emulex, LSI, Oracle, and Seagate. The Data Integrity Initiative defines methods for achieving end-to-end (application to disk drive) data integrity via checking mechanisms designed to prevent data corruption from occurring.
Archrival QLogic emphasizes the power efficiency of its 8Gbps HBAs, which do not include heat sinks or require cooling fans on the server, according to Satish Lakshmanan, director of marketing in QLogic’s host solutions group. For example, the company’s single-port QLE2560 adapter consumes 5.5W, and the dual-port QLE2562 consumes 6.2W.
In terms of performance per watt, QLogic claims 50,000 I/Os per second (IOPS) per watt on its single-port adapters and 63,000 IOPS per watt on its dual-port adapters using the company’s Dynamic Power Management technology. (Performance testing included a quad-core Intel Xeon server, Windows Server 2003 R2, and the IOmeter benchmark with 4KB read/write block sizes.)
QLogic also stresses performance in real-life applications such as MS-Exchange and Oracle databases. In Exchange tests using the IOmeter benchmark, QLogic’s 8Gbps QLE2560 topped 185,000 IOPS and 1,300MBps of throughput. (Full test configuration data and results for both MS-Exchange and Oracle testing are available at www.qlogic.com.)
Lakshmanan also points to ASIC-based hardware features that are optimized for virtual server environments, such as support for up to 256 NPIVs per port, and up to 256 queue pairs per port. For end users, the virtualization enhancements translate into benefits such as better utilization of I/O resources at the server level.
In the RAS (reliability, availability, serviceability) space, QLogic officials emphasize their adapters’ mean time between failure (MTBF) rating of 14 million hours, as well as a five-year warranty (versus three years for most other HBAs).
Also on the RAS front, Lakshmanan highlights two features:
- Hardware Assist Firmware Tracing (eHAFT) extends the trace buffer to host memory, which increases the potential trace buffer memory from 8KB of on-chip memory to 4GB in RAM; and
- Overlapping Protection Domains (OPD) is a reliability feature that protects data as it moves between the HBA and the Fibre Channel interface (see figure, below). With the introduction of its 8Gbps adapters this year, QLogic enhanced OPD to protect both the data path and the control path.
QLogic’s Overlapping Protection Domains (OPD) technology protects the data and address paths as data moves between the host bus interface and the Fibre Channel interface.
Brocade is the newcomer in the Fibre Channel HBA space, having entered the market in June with its own 8Gbps adapters. (Brocade previously re-sold 4Gbps HBAs from LSI.)
Not surprisingly, Brocade emphasizes the synergy between its HBAs and switches, particularly in the areas of centralized management and ease-of-use in Brocade SAN fabrics. Centralized management comes courtesy of the company’s Data Center Fabric Manager (DCFM) software. In addition, Brocade’s contends that its “one-stop shop” for SAN gear gives it a competitive edge.
Brocade has brought a number of technologies, or concepts, from its switch equipment to the HBA space, including Adaptive Networking. In the context of HBAs, features under the Adaptive Networking umbrella include QoS and functions such as “top talkers” (which provides a listing of the most “chatty” Fibre Channel ports, virtual machines, etc.) and “I/O histories” (which, for example, track I/O port connections across virtual server fabrics).
Brocade also touts the performance of its 8Gbps Fibre Channel HBAs, claiming up to 500,000 IOPS per port, according to Harry Petty, director of product marketing in Brocade’s server connectivity division.
Tests conducted by Finisar Medusa Labs Test Services measured almost 420,000 IOPS in a Windows environment. (The test configuration consisted of a 50/50 read/write ratio; a queue depth of 640; block sizes of 512, 1,024, and 2,048 bytes; a Super Micro server with a quad-core 3.167GHz CPU and 4GB of RAM; a Third I/O Iris disk array with 10 8Gbps ports, each with four LUNs; the Medusa Labs Test Tool Suite; a Brocade 300 switch; and a BR-815-0100 HBA.)
A throughput (as opposed to IOPS) benchmark conducted by Finisar Medusa recorded 1,561MBps throughput with a 50/50 read/write test pattern, a queue depth of 160, and a block size of 262,144 bytes.
As do other HBA vendors, Brocade boasts a number of features designed for virtual server environments, including support for NPIV, a Virtual Channels feature that enables parallel processing of I/O requests across virtual machines, and V_Port profiles, which allows each port to be assigned a specific profile with its own QoS level (see figure).
Brocade enables provisioning and extending QoS from virtual machines to storage devices.
Brocade enables provisioning and extending QoS from virtual machines to storage devices.
One performance boost expected around the middle of next year, according to Petty, is an N_Port trunking feature that will allow two ports on an HBA to communicate to two ports on a switch with frame-level interleaving of the two streams, which essentially creates a 16Gbps (2× logical) pipe to the fabric.
For security-conscious users, Brocade’s HBAs include built-in encryption (AES 256) for data “at-rest.” Next year, the company plans to add other security features, such as key management and security functions for data “in-flight,” according to Petty.
Atto Technology’s HBA differentiators stem largely from the company’s focus on specific vertical markets and applications, such as media/entertainment, digital video, streaming tape backup, and other applications that require high-bandwidth streaming data, such as video-on-demand. For those applications, Atto touts its “smooth” streaming functionality (i.e., no wide swings between best-case and worst-case bandwidth performance), which is advantageous in applications such as video.
The company achieves its streaming performance via technologies such as buffering, look-ahead, error processing, and PCI-bus management techniques.
Another feature that plays into Atto’s target markets is the proprietary Advanced Data Streaming (ADS) technology, which provides controlled acceleration of data transfers.
ADS is a combination of technologies, including a third-generation RISC processor, bus mastering (which eliminates the need for communicating with the host CPU by transferring data directly between the HBA and system memory), tagged command queuing (or TCQ, which allows for multiple I/O commands to be processed in any order), disconnect/reconnect (eliminates wait time between an adapter and other devices), and special algorithms.
Unlike some other HBA manufacturers, Atto does not make its own chips. In the case of its Fibre Channel adapters, the company’s primary chip partner is PMC-Sierra.
Atto also touts the ease-of-use of its adapters and software, which are often used by non-storage technologists such as video editors and imaging professionals. (Atto is the leading supplier of Fibre Channel adapters for the Mac market, which comprises 30% to 40% of the company’s overall HBA shipments, according to Tim Klein, Atto’s president.)
The company focuses largely on Tier-2 OEMs (examples include Autodesk, Avid, Harris, and Unisys) and provides customization of its HBAs.
Most recently, Atto introduced the Celerity FC-84EN line of quad-channel 8Gbps Fibre Channel adapters. The company claims an aggregate throughput rate of up to 6,400MBps in full-duplex mode (1,600MBps per channel) on the adapters, which include x8 PCI-Express 2.0 bus interconnects. —Dave Simpson