Diverse sites embrace Fibre Channel SANs

User case studies reveal the potential advantages of storage area networks in a wide range of application environments.

By John Haystead

Click here to enlarge image

Reaching out from its initial base of high-end, bleeding-edge implementations, Fibre Channel technology is now moving into mainstream business applications. The path to this broader market penetration is through storage area networks (SANs). No longer a build-it-yourself technology, SANs now offer companies of all sizes and budgets an opportunity to implement cost-effective, scalable storage.

Storage administrators can now comparison shop among a host of low-cost Fibre Channel SAN solutions, complete with disk arrays, hubs, switches, host bus adapters (HBAs), and increasingly powerful management software-all pre-integrated and pre-tested.

As demand for storage capacity continues to grow exponentially, more and more companies are finding that the host-attached storage implementations that served them well in the past are now bringing them to the brink of disaster.

Meanwhile, as Windows NT/2000 gains ground in the low-end to mid-range server markets, Fibre Channel SAN storage is proving to be a frequent companion. The two trends are in fact related, because host-attached storage is particularly cumbersome in NT environments where application requirements quickly outgrow local storage capacity. The extensive legroom of a Fibre Channel SAN makes this a relative non-issue.

Mazda test drives a SAN

In sharp contrast to the situation just a year ago, more and more IT organizations no longer see Fibre Channel as a luxury item, but a practical, relatively risk-free investment path to meet long-term storage needs. For example, Mazda Corporation, in Irvine, CA, is using a Fibre Channel SAN for the tape backup operations of its corporate servers.

Kai Sookwongse, Mazda's systems manager for LAN support, notes that "in terms of overall cost-effectiveness, Fibre Channel is a pretty pricey solution just for backup." In fact, Sookwongse says that given the experience and knowledge they have today, they probably wouldn't implement Fibre Channel just for backup. However, he notes that "when you look at disk storage down the road, as we are, it's much more justifiable. And Fibre Channel prices are constantly dropping."

Bayer Corp.'s storage area network is set up as a linked pair of FC-AL loops, with six servers in each loop.
Click here to enlarge image

Provided by Cranel Inc., a systems integrator in Columbus, OH, Mazda's SAN currently consists of eight servers connected via a Gadzoox Capellix 3000 Fibre Channel switch to a StorageTek Timber

Wolf 9714 tape library capable of up to 3.5TB of DLT tape storage. Originally, Mazda had attempted to implement its SAN using a Fibre Channel hub, but soon found that approach to be inadequate. "If any one device on the hub was shut down, the entire network had to be restarted," recalls Mark Williams, a LAN engineer at Mazda. "Since moving to the switch, however, all of the issues we had relative to Fibre Channel have gone away."

So far, the company has hooked up only its high-availability servers to the SAN, a mix of one/two/four-processor ProLiants running Windows NT. All of the servers are connected to the Gadzoox switch via Emulex HBAs.

Mazda is in the process of migrating from Token Ring to a Gigabit Ethernet LAN. Servers that are not on the Fibre Channel network are connected to backup storage via the Gigabit Ethernet LAN. "We've created a totally isolated backup network, with the lighter-usage servers backed up over Gigabit Ethernet and the high-availability systems over Fibre Channel," says Sookwongse. "As the migration continues, we may add Fibre Channel cards to some of the other servers as well."

Mazda is using Veritas NetBackup 3.2 software, which includes a shared-storage option that makes backups appear to be local by allowing clients to map the 9714 drives to local servers. Williams says the Veritas tools are particularly useful for centralized backup and management.

Mazda has 10 regional offices throughout the US and Canada, each with an HP 718 autoloader and secondary backup server linked to the primary corporate backup server. "Our primary server is managing all of our corporate servers, as well as all the regional offices. If we want to initiate a backup or restore, we can do it from one console, anywhere in the enterprise," Williams explains.

Williams says their backup operations have benefited significantly from the Fibre Channel SAN implementation: "We're simultaneously initiating backups of about 60 servers, and all that traffic can be overwhelming to the primary network. Offloading backup processing from the LAN onto Fibre Channel frees up the server network and helps relieve the bottleneck." Compared to their 16Mbps Token Ring, Williams says their backup times have gone from 13 hours to 3 hours.

Although Mazda is currently using its Fibre Channel SAN exclusively for backup, with all of its on-line storage still handled by local SCSI drives and arrays, Williams says they will begin testing the feasibility of switching to a central Fibre Channel-based storage array this year.

SAN alleviates headaches

Bayer Corporation has also cut costs by shifting from host-attached to shared Fibre Channel SAN storage. Bayer's business operations include a number of large Oracle databases running on RS/6000 servers. Prior to implementing the SAN, each of these servers was backed up to its own dedicated tape drive. Now the company is able to backup all its databases to two StorageTek 9740 tape libraries, each housing eight 9840 tape drives. Currently, six servers share this pool of storage via two STK StorageNet access hubs, each of which can support up to 32 FC-AL connections.

The Bayer storage network is set up as a linked pair of FC-AL loops with eight drives and six servers in each loop. Each server has connections to both hubs via Cambex HBAs. As explained by Tom Mours, Bayer's senior systems analyst, because the company has a mix of Microchannel and PCI-based RS/6000 machines, two different types of HBAs are required. "At the time of our implementation, Cambex was the only vendor that could provide both interfaces," Mours explains.

To manage its servers and storage resources, Bayer implements IBM/Tivoli Systems' Adstar Distributed Storage Manager (ADSM) software. Gresham Software helped with this implementation, together with StorageTek. Gresham's Enterprise DistribuTAPE product works with StorageTek's Automated Cartridge System Library Software (ACSLS) to allow multiple ADSM servers to share tape drives and a single tape library.

Although the current configuration of Bayer's SAN has allowed them to efficiently share storage on common resources, Mours says they will consider a switched fabric topology in their next upgrade. "We've realized considerable cost savings from our SAN implementation, but when we first implemented it, switched networks weren't really available. We do still run into a few network rebooting problems that a switched network would solve."

In general, Mours advises new SAN implementers to be sure to get the latest code from vendors. "Because IBM didn't originally have the necessary HBAs, we had to go to a third party whose microcode wasn't the most stable initially."

Although Mours says these glitches have been worked out, he is nevertheless concerned about the new server he will be adding to his SAN later this year, which will incorporate IBM's new Fibre Channel HBA. "We'll be crossing our fingers that it will work. We've asked the vendors about testing in advance, but each points the finger to the other guy."

Service providers leverage SANs

Many companies aren't interested in making any more of an investment in storage technology than they absolutely have to. However, that doesn't mean that they don't recognize the critical importance of instantaneous, reliable access to-and protection of-their information resources. They simply prefer to procure these capabilities as services rather than products.

In fact, a growing number of companies are now positioning themselves to meet the needs of these firms, many of whom are in turn finding Fibre Channel SANs to be the most flexible, cost-effective approach to meeting both rapid growth and diverse customer requirements.

InterLan Technologies Inc. provides application hosting and Internet connectivity services to small to mid-size companies. Those companies "recognize that they need a Web presence, but either they aren't technology-savvy or they don't want to deal with investing in a data center and technical staff," explains Mark Nilsson, InterLan's chief technology officer.

Mark Nilsson - Chief technology officer, InterLan
Click here to enlarge image

InterLan provides this capability for them, including data center, servers, storage, backup, technical support, connectivity, and application development. Nilsson says 70% to 80% of InterLan's revenue base is currently customers looking for Web hosting and application management, as opposed to pure connectivity. Starting out in a 500-sq.-ft. facility, InterLan's data center now occupies more than 30,000 square feet.

InterLan began operations with Sun StorEdge A1000/D1000 12-disk RAID arrays. However, it quickly became clear that this solution would not be practical for the large amount of database storage they were bringing online. Says Nilsson, "We found a lot of companies were looking for relatively small amounts of RAID storage (100GB or less), and these small arrays took up a lot of shelf space and also used different management systems."

Another problem associated with database storage requirements is the distribution and segregation of arrays, with some disks used for caching, some for volumes, some for data. "This really used up a lot of disks quickly, and you could end up using an entire array for a 9GB database," says Terry Rowbotham, InterLan's vice president of engineering. "Then, each time a customer's needs grew, it would be necessary to shut down the system to add capacity."

Nilsson says they soon recognized they would need to move to on-demand storage. "We wanted to not only meet our customers' current storage needs, but also provide the capacity to increase their storage on demand without having to bring their pipe down or swap hardware."

InterLan's Fibre Channel SAN consists of seven MTI Vivant V20 Fibre Channel storage arrays, servicing both Sun Unix and HP Windows NT servers. Each Vivant V20 array is configured with thirty-six 36GB Seagate Cheetah Fibre Channel drives, for a total of 1.3TB per unit. MTI certifies all of its SAN components, including HBAs. HP HBAs are used in the HP servers, while the Sun servers use Emulex HBAs for PCI connections and JNI HBAs for S-bus connections.

Currently, InterLan has both single and dual-hosting capability over a 25-port switched fabric. Rowbotham says they will add more switching capability as capacity requirements increase. MTI ships V20 arrays with 8-port and 16-port Ancor Communications switches, but each of InterLan's V20s currently has two 16-port switches for a total of 196 switched connections. "Hopefully, Fibre Channel will be going to 32-port and 64-port switches in the very near future," says Rowbotham. [Editor's note: Ancor and Inrange Technologies last month began shipping 64-port Fibre Channel switches.]

Rowbotham strongly recommends that SAN implementers stay with open-system designs. "There isn't any standard for SANs yet, and until one develops, you need to be confident that your vendors will comply with standards when they do come out, and not lock you into a proprietary environment."

Although InterLan hasn't yet benchmarked transfer rates, Rowbotham says they haven't experienced any bottlenecks. "The majority of our customers have large databases that are extremely read/write intensive, and we specifically went with Fibre Channel because of its performance and expandability."

Another primary requirement for Inter-Lan's storage solution was to have LUN mapping (or masking) technology within the storage arrays to isolate each customer's assigned storage. MTI's Data Shield volume-mapping firmware provides this capability by mapping the worldwide name of each server's HBA directly to its assigned storage.

Data Shield has also helped minimize any problems from multiple operating systems on the same SAN. Says Rowbotham, "NT will try to take as much of the disk space as possible, but by isolating the LUNs we can manage it."

SAN bridges stranded storage islands

Electric Lightwave is another service company offering Fibre Channel-based SAN technology to a second tier of storage users. The company provides local-exchange telecommunications services to corporate users in a nine-state region in the Western US. Focusing on providing Internet backbone resources to application service providers (ASPs), Electric Lightwave currently has a mix of 47 local servers, as well as a number of remote systems.

According to Wayne Hall, Electric Lightwave's director of IT, the company decided to move to a SAN about a year ago to deal with the problem of "stranded storage." At the time, Electric Lightwave had only two servers with fiber connectivity, both connected to an EMC Symmetrix 3700. The rest of its servers were using direct-attached HP AutoRAID storage arrays. As noted by Hall, "Although fully configured we could get close to 500GB per array, only one server could use it. In other words, it was stranded storage and we needed to fix that problem."

Electric Lightwave's SAN now accesses almost 4TB of data on multiple re-sources, including 1TB on the Symmetrix 3700, a total of 1.8TB of shared storage on 13 HP AutoRAID arrays, and an additional 1TB of storage on a new HP XP256 disk array.

All of the storage is networked over a fabric of four Brocade SilkWorm 2800 switches, and managed using HP's SAN Manager software. The first servers to go on the SAN were a mix of Sun Enterprise systems and HP 9000 servers. All of the servers use HP/Agilent HBAs, which are based on Agilent's Tachyon chip.

Hall says they've experienced no interoperability problems between the EMC and HP storage arrays. The transition from host-attached to SAN storage involved multiple transfers between devices. The first step was to mirror all of the data from the HP AutoRAID arrays onto the Symmetrix 3700, while production continued on the HP XP256 array. "We then severed the link to the RAID arrays, rolled over to the SAN, and then reconnected the RAID arrays to the SAN." Hall says the transition went fairly smoothly, with the additional benefit of a speed increase of 10% to 20% with the SAN vs. direct-attached SCSI storage.

Having completed the initial phase of its implementation, Electric Lightwave plans to add additional servers to its SAN. Next month, the company will move two VAX servers to the network, followed by 27 NetWare servers, all of which currently have direct-attached storage. After that, Hall says they'll look at bringing their NT platforms onto the SAN. As with other NT users, Hall says he's still "concerned about NT's tendency to find any empty storage and grab it," but believes this is being worked on.

Down the road, Electric Lightwave also plans to add its remote servers to the SAN. "Right now, however," says Hall, "we're doing emulated private loop [EPL] and we don't want to do EPL over a very large WAN. We'll wait until we get the full fabric login capability in the operating system."

In general, Hall recommends that all new SAN implementers start out with a homogeneous server environment. "In six months or so, this may not be necessary, but the software and the protocols haven't been standardized yet."

SAN 101

Ohio State University's Fisher College of Business offers undergraduate through doctoral degree programs to more than 4,500 students. To support these programs, the school has implemented a Fibre Channel SAN that, in addition to supporting its NetWare database and user directories, also hosts storage for its Web applications and e-mail servers. Currently, this accounts for 500GB of storage, having grown more than 150% in less than a year.

One of the interesting things about Fisher College's SAN, however, is that it was delivered as a single integrated package. XIOtech's Magnitude Fibre Channel SAN array comes with a 32-drive array, RAID controller, eight-port Fibre Channel switch, and logical volume management software. Magnitude supports both point-to-point and FC-AL topologies, with up to eight Windows NT or NetWare server connections in a point-to-point configuration. (Seagate Technology recently acquired XIOtech.)

The Fisher College SAN is configured with point-to-point connections to four Dell PowerEdge 4200/4300 NetWare servers and an FC-AL connection to an external Vixel Rapport 2006 six-port FC-AL hub, which links two additional Sun Enterprise 250 e-mail and Web servers. XIOtech's REDI Zone software limits each individual server's access to specific data volumes.

According to Brian Wilson, a network engineer/developer at Fisher College, they first began thinking about implementing a SAN in mid-1998, and spent the following 8 to 10 months researching the possibilities.

One alternative was the use of network-attached storage (NAS) on their existing Gigabit Ethernet LAN backbone. "What we found, however," says Wilson, "is that regardless of the number of ports being served, the bottleneck would always be that one connection to the network. Instead, with Fibre Channel, we get fiber speed to each individual server, with no bottlenecks."

The Magnitude system was also able to handle the school's multiple operating system environments, including NetWare, Windows NT, Solaris, and IRIX.

Working with XIOtech personnel, Wilson says their initial installation went extremely smoothly, with the network up and running within a matter of hours. "It was much like adding a new controller card to our system, and we began adding storage to our network later that day," says Wilson.

Based on his experience, Wilson recommends that all potential SAN users define a very clear set of goals before they start looking for solutions, and that they then "look at as many vendors offerings as they can for the perfect match. Don't settle for anything less."

John Haystead is a freelance writer in Bucksport, Maine, and a regular contributor to InfoStor.

This article was originally published on May 01, 2000