Early adopters shake out Fibre Channel SANs

Early adopters shake out Fibre Channel SANs

Going beyond video applications, a variety of companies are building Fibre Channel storage area networks for a wide range of applications and reasons.

John Haystead

While most companies continue to wait out the impending Y2K risk before moving ahead with next-generation enterprise storage solutions, a few early adopters are already up and running Fibre Channel storage area networks (SANs), and the preliminary results appear positive. With careful up-front planning and vendor selection, many of the feared obstacles associated with the technology can be easily overcome.

However, like any new technology, additional development work is required before Fibre Channel SANs can become all things to all people. In addition to technology considerations, factors such as application type, system complexity, budget, time constraints, and available resources must be taken into account. As always, one of the benefits of being a follower, rather than a leader, is the opportunity to learn from the experiences of others.

For example, Polk Company is a demographic and marketing-information systems company serving the automobile and other industries. The company`s Oracle database contains data on more than 100 million people, as well as customer satisfaction and other statistics on more than 200 million vehicles. On average, the company generates 12GB of information per day.

Already a huge storage user, Polk is now in the process of re-engineering its information systems to shorten order-to-delivery times and to make its resources available to a greater universe of customers via the Internet. A major part of this effort is the implementation of a Fibre Channel SAN that will allow a mix of Sun, SGI, and Windows NT servers to back up to and share the same tape libraries, enterprise-wide. "The SAN is the centerpiece of our re-engineering initiative, improving our information delivery capabilities and enhancing our competitive advantage," says John Manera, Polk`s director of infrastructure information technology.

Built around four McData Corp. ED-5000 Fibre Channel directors, Polk`s SAN supports 17 terabytes of EMC Sym-metrix disk arrays. The 32-port directors provide "any-to-any" Fibre Channel switching with 100MBps per port. The RAID disk arrays provide some 3,200 addressable virtual drive images, each redundantly accessible by assigned servers. Four StorageTek TimberWolf 9710 tape libraries are also attached to the SAN through McData`s EB-1200 SCSI-to-Fibre Channel bridges.

The initial driver for the project, which began roughly a year ago, was the company`s need for higher-speed access to an existing Sun server, as well as redundant failover to distributed on-line storage and tape backup silos. Ultimately, however, Polk`s SAN will encompass multiple Sun servers as well as SGI Unix and Dell NT systems. The mix of servers placed additional demands on the SAN design because "whatever solution we went with had to fully support a heterogeneous environment," Manera explains. "We needed to create a pool of tape drives that could be used with any domain or any server across the enterprise at any time, and which allowed all servers attached to it to see every device."

Working in concert with Veritas Volume Manager and NetBackup software, the four McData directors are managed from a central console via the company`s Java-based Enterprise Fabric Connectivity (EFC) management system. The EFC system allows centralized management of up to 36 ED-5000 directors and ES-2500 switches.

On the hardware side, Polk uses multiple host bus adapters (HBAs), including JNI`s S-bus-to-Fibre Channel adapters for the Sun servers and Emulex HBAs on the NT servers. McData OEMs the HBAs and private labels the bridge (McData Fabric Gate) used on the StorageTek libraries.

While the Sun side of the SAN implementation has been in operation for some time, with a second Solaris server coming on-line this month, the NT and SGI portions of the SAN are still being implemented. The Fibre Channel directors make it possible for any port to talk to any other port, but there are other issues involved with bringing heterogeneous operating systems onto the same fabric. "For example, if NT can see all the storage that Sun can see, they will battle over who gets what," says Richard Lyford, product marketing manager at McData.

To deal with different operating systems, McData`s directors can be configured through "port zoning," which segments the switch to maintain storage integrity and prevent one operating system from seeing the storage assigned to another. Polk`s SGI and NT servers are currently attached to the SAN in this way to isolate their traffic. Therefore, although the Sun, Dell, and SGI servers all coexist on the same fabric, they don`t share the same storage ports.

File systems are still managed by host-based tools at the operating system level, independent of the Fibre Channel fabric. "We can set up any storage port as a target, and map in any LUN behind that port to any host," says Mark Stratton, systems integration lab manager at McData. "We can even make multiple hosts see the same storage, but it`s only some of the more advanced software sharing tools, like Interphase`s FibreView, that are beginning to provide coherent NFS-type file and data-sharing capabilities."

Polk`s Manera expects interoperability of multiple file systems to remain a problem for some time, but says this will be a continuing goal over the next 18 to 24 months. "In particular, the inability of non-Veritas file systems to participate in a clustered system is a way off, and this will be critical to meeting our failover and business-continuance requirements across the enterprise. Until we can move toward a clustered file system, however, the zoning approach is about the only way you can go."

Meanwhile, McData`s Stratton says they`ve also been working closely with adapter, bridge, and array vendors to provide a capability called "persistent binding," which allows ports to be dynamically allocated and tracked to zones according to worldwide names. "For example," Stratton explains, "on an NT server we can plug in a new RAID box, map it into the WWNs of the storage ports in a zone, and without rebooting the NT server, bring new storage on-line." However, he adds, "There`s still a wealth of opportunity for software vendors to provide things like file-system attributes."

Overall, Polk`s Manera says the SAN implementation process went smoother than they anticipated. "We recognized that we were early adopters of some of this technology, particularly with regard to management tools, and we expected to hit more bumps in the road than we actually encountered." For example, Manera expected to have problems getting NT onto the SAN, but in fact "it turned out to be the simplest implementation we had. Our management tools and the zoning capabilities in the switch helped to contain and overcome operating systems."

Manera says the next immediate objective is to more fully exploit backup capabilities by merging NetBackup with Veritas` shared storage option. "We`re not fully deployed across all our servers yet, but that`s the current project."

Polk`s Manera recommends that anyone considering a SAN implementation start with a pilot project to gain experience. "As opposed to allocating storage resources on a server-by-server basis, SANs force designers to think of storage as a pool of total capacity that may or may not be shared by various applications. It forces open-systems people to think with a mainframe mind. That`s the biggest struggle."

"We knew we were in the middle of a core business transformation and we recognized that we had to radically redesign our infrastructure, and to spend some money to get there," says Manera. On the other hand, he`s quick to point out that they didn`t come to the table with an open checkbook. "We worked hard with our partners and vendors to be as cost-effective as possible in our implementation and are now working toward economies of scale." Eventually, Manera expects to see a significant payback on the SAN investment. "Twenty-four months down the road, when our transformation is fully in place, we`ll reap the full benefits."

New South Communications

The reasons for implementing a Fibre Channel SAN often center on increased throughput, connectivity distance and device counts. However, as at the Polk Company, clustering is increasingly becoming a driving force behind SAN deployment. That`s the case at New South Communications, a telecommunications and ISP provider in Greenville, SC.

New South is a relative startup and, unlike most companies, was able to start from the ground up. "We were looking to go to a clustered environment down the road, and we wanted to have development and production Unix servers in the cluster," explains Robert Briscoe, systems administrator at New South. "With clusters, it`s hard to scale SCSI. Fibre Channel provides a much cleaner implementation." Briscoe says its decision to implement a SAN "had nothing to do with speed, capacity, or connection distances."

New South`s SAN is built around two eight-port SilkWorm switches from Brocade Communica-tions. Five Sun Solaris servers are connected in a switched fabric, while HP-UX servers are connected in an arbitrated loop configuration via the same switches. The SAN includes two StorageTek 9145 full Fibre Channel arrays (which STK OEMs from Clariion), with one dedicated to the Sun servers and one to the HP-UX servers. New South uses JNI HBAs on the Sun servers and HP adapters on the HP systems. On the software side, New South manages its RAID arrays with Clariion`s Navisphere package and uses Brocade`s management software to manage the switches.

As for incompatibility problems associated with Fibre Channel, Briscoe says it had none. The company relied on third-party systems integrator--Cranel (Columbus, OH)--to handle most of the product selection and integration, a recommendation voiced by many early adopters of SAN technology. "We brought in outside expertise to help us get it implemented quickly," says Rob Writnauer, director of IT at New South. "Cranel had already tested all the components." It took Cranel and New South about five weeks to implement the entire SAN.

Western Heights

Though Fibre Channel SANs are beginning to make inroads in mainstream applications, video is still the primary application benefiting from the technology. For example, Western Heights School District, in Oklahoma City, is using its new Fibre Channel SAN to deliver on-line video instruction to students` PCs, both in the classroom and at home. The system combines high-quality MPEG video presentations and text and graphics to teach complete courses anywhere across the school district`s 12 campuses.

With each lesson requiring close to 500MB of storage, and each course composed of 40 to 50 lessons, a total of 20GB to 25GB is required per course. As pointed out by Joe Kitchens, Western Heights` superintendent of schools, "This places a heavy demand on servers and storage systems, and we needed a flexible platform to deliver it to both homes and schools." In addition, the district uses the system to store encoded VHS instructional tapes and downloaded satellite streams.

Western Heights is using a Windows NT-based SAN from Dell. The storage network connects 1.8TB of PowerVault 650 Fibre Channel disk storage to four PowerEdge 6350 servers. The network also has a four-drive PowerVault 130T tape backup system connected via a Fibre Channel-to-SCSI bridge. The SAN`s redundant PowerEdge 50F 8-port Fibre Channel switches allow both the disk arrays and tape library to be connected to the same switches with no single point of failure, says Bruce Kornfeld, senior manager of storage product marketing at Dell.

Partnering with Brocade on the switches, Qlogic on the HBAs, StorageTek for the tape library, and Crossroads Systems for the bridge, Dell has been shipping its integrated NT Fibre Channel SAN system since February. "The benefit of an integrated system," says Kornfeld, "is that users are guaranteed up-front that the SAN is fully tested and certified."

Dell has also partnered with Microsoft on an NT-based SAN software management tool--called Open Managed Storage Consolidation Software--which manages all LUNs and volumes on the SAN, ensuring that each server knows which pieces it owns and controls. The technology could eventually be part of Microsoft`s Windows 2000.

The school district, which started planning its SAN implementation about a year ago, installed the system at the end of the school year. Kitchens recalls they had no problems with the installation and initial configuration, which was supported on-site by a team of Dell engineers. "We`ve had up to 25 high-quality full-motion video streams running simultaneously and haven`t had any downtime whatsoever," Kitchens reports.

Western Heights is considering adding another terabyte of capacity to the system in response to requests from other school districts to serve as a data repository for their on-line course work. The district has a DS3 connection to its ISP that will allow them to push data around the state.

Despite the success of its own implementation, Kitchens still strongly recommends that anyone considering implementing a SAN have a full-time person in place to manage and maintain the system. "An in-house technologist, proficient with both the servers and switches, is a fundamental requirement," he advises. "We`ve attempted to get by with people with less ability and experience, and the network just overpowered them."

Patent and Trade Office

Though the number of success stories is growing, planning and implementing a Fibre Channel SAN is far from a no-brainer. The up-front work done by the US Government Patent and Trade Office (PTO) provides valuable insight into some of the issues that have to be addressed.

The PTO stores about 40TB of patent-related text and image data, all of which must be available on-line to patent examiners, attorneys, and others. Currently, all of this data is stored on 31 EMC Fibre Channel and SCSI RAID arrays distributed in separate storage pools throughout the PTO`s Crystal City, VA, data center. Attached are more than 80 HP Unix servers, as well as NT systems. All of the data is archived in several tape formats using EMC`s Data Manager software and Legato Systems backup and recovery software.

Today, all of the server-to-storage connectivity is via point-to-point Fibre Channel connections with dedicated HBAs at the server end and RAID controllers at the storage device, as well as some SCSI-to- Fibre Channel mixes for tape backup operations. However, the PTO is now looking to Fibre SAN technology to take it to the next level of operation.

The PTO`s interest in SANs is driven by a wide range of requirements. In addition to improving its basic productivity and ensuring efficient revenue generation and tracking, the PTO is looking to SAN technology to provide failover between storage systems, disaster recovery, and dynamic data sharing. It is also looking ahead to multiple, geographically dispersed data centers and remote network connections. And, like Polk Company, the PTO plans to integrate its mix of Unix and NT servers into a single, heterogeneous mass-storage environment.

To explore the possibilities of Fibre Channel SANs, about a year ago the PTO tapped the expertise of Litton PRC, a large systems integration firm in McLean, VA, to do some initial feasibility testing. PRC is the Systems Integrator Technical Assistance (SITA) contractor for PTO, responsible for helping guide its direction in new technology implementation.

To accomplish the task, PRC tested and evaluated a wide array of Fibre Channel switches, hubs, host bus adapters, and disk arrays. Presenting its results in the form of a report and recommendations to the PTO, PRC basically concluded that Fibre Channel is still a long way from plug-and-play interoperability, and unless the PTO was willing to go with a single vendor`s suite of systems and software, major problems would arise.

"Our job was to address our customer`s total enterprise requirements, not just a single piece of the pie, and there wasn`t a specific solid solution we could recommend at this point that could address all of the PTO`s application and cost requirements," says John Loveless, principal network engineer at PRC. So our recommendation was to wait until Fibre Channel technology has matured more." Though Loveless agrees that a single vendor "such as HP or EMC could eventually glue everything together and make it work," the downside would be a very expensive solution, and one that would lock them into a single vendor.

Looking ahead to a SAN environment where both Unix and NT servers would be accessing the same set of data and file systems, PRC tested HBAs and switches designed for both platforms. "We found a lot of interoperability issues, either on the server end, storage devices, or both," says Loveless. For example, he notes that while certain HBAs could handle point-to-point and FC-AL loop configurations, they didn`t support switches. "If they did support a switch, they weren`t the ones we had in house, or a certain switch wouldn`t support a particular HBA or storage device. Vendors just aren`t building their drivers around the same standard."

Fibre Channel product interoperability wasn`t the only issue to consider, however. PRC also had to consider the PTO`s extensive investment in existing equipment and vendor relationships. For example, Loveless says StorageTek, which resells Brocade switches, is already in-house with its backup products. And the PTO has a lot of EMC storage products, and EMC has its own enterprise SAN solution. In addition, the PTO has a long-standing relationship with HP.

However, Loveless predicts that hardware interoperability problems will be solved before problems with operating system software. "Right now there`s no way that you can have a Unix file system melded with an NT file system, and though Veritas and others are working on it, we`re still a long way from having different server types accessing the same file system, which is what you want."

Though the PTO will heed the "wait-and-see" lessons learned from its advance testing, it also recognizes that it will almost certainly need to have a Fibre Channel SAN in place before it can bring up its dual data centers, which is planned for the 2001-2002 timeframe. As noted by Loveless, those requirements will demand moving from point-to-point configurations with loops and hubs to a switched environment.

Larry Cogut, director of the PTO`s System Architecture and Engineering Office, is optimistic: "Right now, the hold-up is a lack of interoperability, but I expect these issues to be resolved in the not-too-distant future." In the meantime, PRC has a systems engineer assigned to keeping abreast of Fibre Channel developments.

Click here to enlarge image

Fibre Channel directors provide 32-port "any-to-any" Fibre Channel switching, supporting 17TB.

Click here to enlarge image

John Haystead is a freelance writer in Bucksport, Maine, and a frequent contributor to InfoStor.

This article was originally published on October 01, 1999