Fibre Channel Charges into Network Storage

Fibre Channel Charges into Network Storage

Welcome to the new world of networked storage based on switches and hubs.

By Vance McCarthy

The worlds of storage and networking are beginning to merge as never before. After more than two years in the lab, and a few corporate beta sites, a handful of Fibre Channel vendors are preparing to roll out high-speed networking technology designed to link multiple storage devices into "virtual storage reservoirs" that scale to tens of terabytes--hundreds of times larger than today`s SCSI-based arrays.

And because 1Gbps Fibre Channel networking technology will be so storage savvy, these huge data centers won`t just be big. They`ll be smart and fast. As storage and networking converge, you`ll be able to hook up your servers and storage devices to central hubs and/or switches, a concept variously--and sometimes incorrectly--referred to as network storage, storage networks, server-storage networks, or storage-area networks. (Look for the Cover Report on storage-area networks in the February issue of InfoStor.) The exact terminology often depends on which side of the field you come from: networking or storage.

The result will be storage networks that rival any current PC server network. The main benefit is a storage configuration that grows with your needs and is still intelligent enough to give users no-wait access to data-intensive applications. Bottom line: New high-speed, intelligent storage networks are poised to break the capacity and performance limitations of existing SCSI-based arrays.

The key to building storage networks is a more scalable and more intelligent family of storage-aware networking tools. Fibre Channel, now supported by more than 40 server and networking vendors, is about to offer a family of 1Gbps intelligent storage adapters and back-end hubs and/or switches to tie high-speed storage devices together. Products have been in development for more than two years and are just beginning to roll out in volume.

At October`s Networld+Interop in Atlanta, more than a dozen Fibre Channel vendors demonstrated a sample Fibre Channel network, built from a variety of vendors` equipment. "Users want to see the pieces work, but they also want to know they`ll have a large number of suppliers to choose from," says Brenda Christensen, vice president of marketing at Brocade Communications and a spokesperson for the Fibre Channel Association (FCA), a group of more than 100 vendors that make Fibre Channel server, adapter, and network components.

Many leading server vendors are already shipping Fibre Channel-based storage, including Compaq, Hewlett-Packard, Sequent, Sun, and Unisys. But October`s demonstration was the most ambitious "proof of concept" yet for networking Fibre Channel storage devices together. Now, attention has turned to lesser known vendors such as Ancor Communications, Arcxel, Brocade, Jaycor Networks, McData, and Vixel.

But just how ready are these new Fibre Channel technologies for your business? More importantly, how ready are you for them? The best way to get the answer is to understand the problems that Fibre Channel can solve and how it works under the covers.

Why Migrate to Fibre?

In case you think that networked Fibre Channel is Buck Rogers stuff, you may already have the first building blocks in your server farms: high-speed Fibre Channel controllers bundled in newer servers and RAID subsystems. (For more information about the advantages of Fibre Channel, see the Cover Story in the October 1997 issue of InfoStor.)

Fibre Channel improves response time over SCSI in LAN-attached RAID by more than 2.5x. While most corporate shops have used SCSI to expand their RAID arrays, many have found that SCSI hits a wall once they get beyond several hundred gigabytes of storage.

"One of the keys to increased interest in Fibre Channel is the limitations of SCSI`s parallel architecture," says William Bedford, vice president of marketing at Raidtec.

"With SCSI, you can saturate at three or four devices, so it`s difficult to build large arrays with a couple hundred gigabytes," says Bedford. "It`s not that people don`t want to do networked storage, but with parallel SCSI they`ve had some unattractive tradeoffs, like slower throughput speeds, higher protocol overhead, and/or lack of guaranteed connections."

Fibre Channel breaks through SCSI limitations in four ways.

First, Fibre Channel offers 1.06Gbps throughput, more than twice the speed of standard SCSI.

Second, Fibre Channel creates a serial port connection between storage devices. In contrast, parallel SCSI requires handshaking between connections.

Third, beyond supporting multiple connections, Fibre Channel supports multiple initiators and multiple target devices. While SCSI arrays support multiple targets, a single initiator (or initiator work group) is typically the limit for each array. To achieve any-to-any connections, a complicated mesh of SCSI arrays is required.

Fourth, Fibre Channel can scale beyond SCSI because its topology features an intelligent storage-based adapter that can communicate to hub or switch ports. This communication between the adapter and central networking devices provides a high level of guaranteed data delivery because data is only sent after the receiving device communicates that it is ready to receive it, and at what speed.

"Unlike SCSI, Fibre Channel is credit based, which means the storage device makes sure that before it sends data you have the capability to receive it, and that the network will make allowances for the transmission of that data," says Charles Bazaar, vice president of marketing at Jaycor Networks Inc. "So, this gives a Fibre Channel network two advantages over any flavor or speed of Ethernet. One, you can use the full bandwidth of Fibre Channel, unlike Ethernet, which is about 40% of the speed advertised. Two, unlike in Ethernet, which always has the potential of packet collisions, there will never be collisions in Fibre Channel because the path has been paved before the data is sent."

These differences are already paying benefits. But Fibre Channel proponents say the real payoff will come when corporations begin thinking about how they can benefit from linking their high-speed computer networks to high-speed storage networks comprised of discrete storage devices linked in an intelligent network.

Because Fibre Channel networked storage moves the basic storage model away from a mere cluster of storage devices to a managed network of devices--thanks to the use of the central hub and/or switch--you can begin to assign throughput priorities by dedicating serial ports to particular work groups and even individual workstations. So, as you add more devices, you don`t suffer the same "hit-the-wall" limits in performance or intelligence associated with traditional SCSI-based arrays.

Three Flavors of Fibre

Even though Fibre Channel is an ANSI standard supported by a wide range of vendors, not all manufacturers see eye to eye on how corporations should install Fibre Channel networks. Generally, there are three main methods:

- Point-to-point is a dedicated connection that, while it is really not networked storage at all, provides allocated bandwidth and sufficient quality of service between two devices. Most Fibre Channel sites are using point-to-point connections to link high-performance devices such as superservers and RAID arrays. Point-to-point configurations typically use a Fibre Channel interface controller to link systems, but the drives may be SCSI or a mix of SCSI and Fibre Channel.

- Arbitrated loop is a notch up the network ladder. Rather than linking point-to-point, arbitrated loop configurations use a central multi-port hub (with shared bandwidth across the entire backplane) or a hybrid hub and switch design (with dedicated bandwidth for each port).

These hubs and switches support up to 126 devices, but the total band-width of the loop does not exceed 1Gbps.

Arbitrated loop was developed with peripheral connectivity in mind. It natively maps SCSI, making it suitable for high-speed I/O connectivity. Native Fibre Channel Arbitrated Loop (FC-AL) disk drives will allow storage applications to take full advantage of Fibre Channel`s gigabit bandwidth, passing SCSI data directly onto the channel with access to multiple servers or nodes. Initially, users are eyeing this architecture for on-line transaction processing, small and midsize data block transmissions, and even archival storage.

- Switch fabric is a switch-based infrastructure that provides an end-to-end, non-shared network storage environment. The Fibre Channel fabric was designed as a generic interface between each node and the physical layer. By adhering to this interface, any Fibre Channel node can communicate over the "fabric" without even knowing whether the device it`s attached to is a crossbar switch, a ring, a frame switch, a circuit switch, or a hub.

With a switch fabric at the core of network storage, users can attach arbitrated loop or point-to-point connections as end-nodes to the central switched fabric, further improving granularity for scalability and quality of service deliveries throughout the network.

It also provides a framework for yet-to-come management software suites that will enable administrators from a central console to reassign storage throughput levels of priority access on a project-by-project basis.

Hubs vs. Switches

Once you move past point-to-point connections, the issue is whether to implement a hub or a switched storage network.

Initially, Fibre Channel was envisioned as purely a switched architecture, but during the specification and building phases of the technology last year, many vendors became concerned that switched port devices would be too expensive. Less than a year ago, switches were expected to be priced at more than $3,000 per port (with accompanying server-side adapters priced even higher). These high prices and a limited market caused many Fibre Channel network vendors to pull back and offer a less expensive (and less scalable) hub--or shared backplane--approach.

Prices are coming down, and that should make switches more appealing. Brocade Communications, for example, ships its Silkworm Fibre Channel switch at $1,875 per port. The switch includes the ability to connect Fibre Channel arbitrated loops, other switches, and even point-to-point devices in the same configuration. The switch also includes an intelligent auto-configuration feature to more easily manage multiple topologies.

Meanwhile, hub vendors claim that to get true enterprise-wide scalability and performance out of a switched fabric, the industry still needs to put crucial middleware in place. FCA members are aware of the need for this software, and there are several initiatives under way with database giants like Oracle and major operating system developers such as Microsoft and Sun.

Meanwhile, as the fine print sorts itself out, some other Fibre Channel vendors are looking at interim approaches to the "unintelligent hub" or "smart switch" features of recent product introductions.

For example, EMC expects to enhance its line of FiberStar hubs and adapters with the launch of a managed hub early next year. At $2,750, the nine-port hub will offer some level of fault detection and SNMP monitoring.

Ancor Communications recently launched a hybrid hub/switch device as an upgrade to its GigWorks MKII line of Fibre Channel products. It offers 16 Fibre Channel ports, 8 of which can be dedicated to a shared arbitrated loop and the others switched. It will cost just under $2,500 per port (including the optics module required for each port).

One thing to keep in mind about price is that unlike in the world of LANs, the adapter--not the hub--is the expensive item because the intelligence is embedded in the adapter. So, don`t expect Fibre Channel storage adapters to be in the ballpark of your run-of-the-mill $100 Ethernet adapters. Fibre Channel adapters can run upwards of $3,500 a pop.

Think Networks for Storage

Despite all the niggling details yet to be flushed out by early users of Fibre Channel, there are a couple of basic facts to keep in mind.

Within the next 12 months, you`ll be hearing a lot about how better networking of your storage will be as important as putting in high-speed Ethernet or ATM at the front end of your enterprise. You may find that even with high-speed links for your servers and workstations, you`re still suffering from latencies in the time it takes to access storage or deliver quality-of- service guarantees to your users.

It isn`t necessary to get bogged down in the details of Fibre Channel before experimenting with the concept. The building blocks are available to enable you to put Fibre Channel in place to see if the approach has any merit for your business. The question of upgrading from SCSI is more one of when--and not why--for most large corporate IT staffs these days.

Click here to enlarge image

Click here to enlarge image

At Networld+Interop, a number of vendors demonstrated a switch-based Fibre Channel network using IP and SCSI protocols.

Click here to enlarge image

Comparing Hubs and Switches


- Per port costs generally range from $400 to $1,400 (excluding adapter), according to emf Associates, a consulting firm in Half Moon Bay, CA. In some instances, prices may exceed $2,000 per port.

- Hub connectivity is limited to 126 devices with a single loop (1Gbps shared across entire loop). In practice, however, loops are usually limited to 40 to 50 devices to avoid congestion.

- Shared media means single communication at a time. All devices share the same bandwidth. If there are N devices and all are equally active, then the average bandwidth per device is 1.062Gbps/N.

- Hot-swappable storage between devices for fault tolerance. However, swapping forces a loop initialization process (LIP) and trashes any transfer in process at the time of device insertion.

- Half duplex. This is a restriction imposed by current-generation drives. Architecturally, the loop is full duplex. You can transmit and receive at the same time, if the devices support full duplex.


- Per port costs generally range between $2,000 and $4,000 (excluding adapter). emf Associates expects the price differences between hubs and switches to decline over time and be small by the year 2001.

- Connectivity scalable to unlimited devices with multiple switches (up to 1Gbps per switched connection). In theory, up to 16 million connections are possible; in reality, 128 to 256 connections with current-generation switches.

- Non-shared media enables multiple concurrent communications. With switches, you can always use the full Fibre Channel bandwidth. Also possible with FC-AL, but you have an arbitration every time you want to transmit, and you have to wait for other devices according to a "fairness" algorithm.

- Hot-swappable storage between devices, and higher dedicated throughput for high availability and fault tolerance. You can hot-swap a drive without disturbing other communications, and no data is lost by other devices. Access to and from the inserted drive is immediately available.

- Full duplex, but it depends on device and host bus adapter capabilities.

- Isolation between a single device and other fabric devices provided by point-to-point link through dedicated switch port. In addition, you can "zone" devices, and there are no security issues with data going to non-source destination ports.

Sources: Vendors, emf Associates (www.emfassoc.com)

Vance McCarthy is a freelance writer in Foster City, CA.

This article was originally published on December 01, 1997