Now that the hype has faded, how are organizations successfully deploying storage area networks to lower total cost of ownership and maximize return on investment?
By Zachary Shess
Much like a can't-miss athletic prospect or a young musical prodigy, storage area networks (SANs) were originally touted as The Next Great Thing. Vendors sang the praises, and analysts and the media followed with extensive coverage. But end users looked at the price tag and the complexities associated with untested and incompatible components and collectively said, "Not right now."
In 2002, the din of SAN hype has dissipated, and potential customers are exercising patience in deciphering how SANs solve problems and justifying total cost of ownership (TCO) while offering a solid return on investment (ROI).
For this report, InfoStor interviewed four organizations in the courier/logistics, entertainment, healthcare, and financial services industries that have executed a SAN installation and found the architecture was able to solve many of their storage problems. While approaches and expectations differed, each realized cost benefits, improved and centralized management, and optimized capacity utilization.
For these organizations, measuring TCO and ROI from SANs is an inexact science. Improved management and capacity utilization, for example, can be translated into cost savings. However, the IT managers interviewed also noted that the benefits of storage networking trickled down into improvements in areas that either they didn't realize would be so significant or just didn't expect at all. Some were surprised, others weren't, but all agree that a SAN has helped them pass along cost savings to their customers, pay for fewer software licenses, and even improve the lifestyle of administrators who no longer have to work evenings or weekends.
Last summer, Dallas, TX-based Dynamex, a nationwide, same-day delivery/courier service company, completed a major, two-year IT consolidation as a result of acquiring more than 20 companies in the past couple of years. The IT staff faced the daunting task of consolidating 38 different computing systems running 12 different operating systems, many of which were proprietary. Additionally, the staff wanted to protect the project by establishing a redundant data center, including a SAN, at a collocation site in nearby Richardson.
"Our systems are very time-critical, in that deliveries are executed in the same day and often in as little as two hours," says Dynamex CIO Jim Wicker. "System performance and reliability are absolutely critical to our business, so it was important for us to establish redundancy."
The company first consolidated about 35 servers dispersed around the country and moved to centralized Linux platforms. The environment consisted of three Dell PowerEdge servers backed by a redundant Fibre Channel SAN with two PowerVault 530F and one PowerVault 650 disk arrays, four Fibre Channel switches (OEMed from Brocade), and one 4-drive, 28-slot tape library. Dynamex uses storage virtualization software on the 530Fs to help create virtual partitions in order to mirror LUN blocks over IP to the off-site SAN for backups, without affecting normal operations of the core servers.
With the SAN in place, Wicker reports performance and reliability improvements and substantial ROI and TCO benefits from server/storage consolidation.
In addition, with fewer servers to manage, the number of IT personnel could be reduced from 52 to 34 within six months. It also transformed the culture of the staff from mostly "putting out fires" to working on software development for initiatives such as custom Web-based order-entry tracking and tracing tools. The result of freeing up IT staff to improve the company's Web capabilities has led to more orders being initiated over the Web-which now account for 10% of the company's total orders. And, according to Wicker, a Web order is 60% less expensive to process than a phone order.
Based in midtown Manhattan, C5 Editorial provides post-production sound editing to the film industry, with notable credits, including Crouching Tiger, Hidden Dragon and Silence of the Lambs. The digitization of sound and audio files created huge data volumes that C5 struggled to manage effectively.
C5's biggest struggle occurred when film editors needed to share these large files that were constantly being worked on and altered. Because of the unrelenting complexity and deadlines associated with their projects, several sound editors work simultaneously on a film, each needing the appropriate digital files. Without a storage network, the files had to be duplicated across local disks attached to editors' workstations.
Tired of dealing with those operational inefficiencies, C5 started looking into storage networking. When bids from storage integrators were deemed far too expensive, Lewis Goldstein, C5's director of technology, began growing his own SAN. Admittedly, Goldstein was a film editor by trade with only trial-by-error IT training. Not surprisingly, the initial SAN was somewhat primitive-and cheap. C5 bought Fibre Channel disk drives and hubs off eBay and borrowed 8-bay server enclosures for JBOD racks of mostly 73GB Seagate Cheetah drives.
After about 12 Macintosh workstations were connected, C5 first looked to beef up the SAN by (unsuccessfully) trying to use SCSI-to-Fibre Channel bridges to link the legacy SCSI drives into the SAN. As Fibre Channel drive prices fell and his SCSI drives continued to fail, Goldstein made the decision to go all-Fibre Channel. He installed two racks of Seagate Cheetah drives holding 200GB, added two Gadzoox Capellix Fibre Channel switches, and installed ATTO Technology's ExpressPCI host bus adapters (HBAs) and AccelWare management software.
Once the SAN was installed, C5 saw immediate benefits and an ROI that, while not delineated on the balance sheet, was very tangible to the staff. For example, when film and sound revisions are made, only a single duplication is created and placed on the network for all the editors to access, ensuring that each editor has the most recent version. Management is simplified because the hundreds of drives that were floating around on editor workstations are now consolidated onto a single network.
Goldstein also reports significant capacity utilization improvements, which also helps the bottom line because he doesn't have to buy as many drives. The cost savings have been invested into the C5 SAN, which now has about 35 clients and 2TB of data. The company is preparing to link the SAN to a remote image library SAN.
"The benefit that our SAN provided through savings in manpower, and how we've been able to work more efficiently, has been very significant and beyond any kind of measurement. It's vastly changed how we've been able to work," Goldstein says.
Blue Cross Blue Shield
Blue Cross Blue Shield of Tennessee (BCBST) is a 4,300-person healthcare organization that insures-and stores the records of-approximately 35% of the state's residents. BCBST represents a growing number of organizations that are considering SAN deployment now that components such as switches, directors, and HBAs are providing improved interoperability.
Bob Venable, manager of enterprise systems at BCBST, did not feel comfortable deploying a SAN in a Fibre Channel-Arbitrated Loop configuration a few years ago because he thought arbitration issues would cause inefficient network operation. When presented with the option to deploy a switched-fabric SAN more recently, Venable and his IT team bet that fabric-based SAN connectivity could centralize their storage management, but they first wanted some proof of concept.
As Venable's group continued to get their feet wet assembling and testing equipment and software, they eventually implemented a SAN based on two 32-port McData 3000 Series directors.
The environment grew rapidly. "We thought the 32-port directors were going to last awhile, but before we could turn around, we needed 256 ports," Venable says.
Once assembled, the initial BCBST SAN consisted of
- Two 64-port McData ED-6064 directors, two 32-port ES-3032 fabric switches, and two ES-1000 loop switches;
- Four IBM Enterprise Storage Servers;
- 200 Windows NT servers; and
- 25 IBM AIX servers.
The BCBST IT staff currently enjoys more flexibility resulting from the SAN deployment. One example is the group's ability to logically, instead of physically, allocate storage resources. Previously, when a server grew or shrank in capacity, administrators would have to come in after hours or over the weekend and spend hours taking down servers and reconfiguring them. With the SAN in place, that task is now performed in minutes during the week, which frees up administrators to perform other tasks.
While difficult to quantify, improved ROI was derived from additional time and cost savings from being able to better diagnose and repair problems without taking up a lot of administrator time.
Centralizing and consolidating data have long been touted as core benefits of SAN deployment. Many large organizations store terabytes of data, often split among multiple data centers. This not only leads to inefficient storage management, but in the case of Manulife Financial, a Toronto-based financial services corporation, it results in poor capacity utilization.
Manulife's server and storage architecture were hamstrung by inefficient capacity and port utilization, because dozens of Wintel servers, each of which ran a single application, averaged just 35GB of capacity. The servers attached directly to seven EMC SCSI disk arrays, which at the time they were purchased, had a maximum of 32 ports out the back.
"We would run out of SCSI connections long before we would be able to utilize the entire disk capacity of the EMC disk subsystems," recalls Sheldon Meingarten, director of common storage infrastructure at Manulife. "We had to change the way we connected servers to the disk subsystems in order to fully exploit the potential disk capacity."
Meingarten leads a storage and server consolidation project as part of a SAN deployment expected to be completed this spring. Preliminary results indicate a significant increase in capacity utilization of the EMC arrays, equating to lower costs. Previously, Meingarten's group determined that because each Wintel server had only about 35GB, and that the older arrays were limited to 32-port, they had to buy a new disk subsystem every time total capacity exceeded 1.1TB.
Larger-capacity servers didn't solve the problem because the Wintel servers were ineffective running multiple applications. A Fibre Channel SAN and larger Unix servers proved to be the answer.
The Manulife SAN includes 13 IBM Magstar tape libraries with Fibre Channel attachments; 12 Brocade SilkWorm 2800 edge switches linked to HP-UX, AIX, Windows NT, and Windows 2000 servers; and two Brocade SilkWorm 6400 core switches linked to both EMC and Hitachi Data Systems disk arrays. The original seven EMC SCSI arrays have been reduced to four.
Considering its data-center capacities were almost doubling annually, and that legacy SCSI presented connectivity limitations, Manulife saved hundreds of thousands of dollars by not having to buy a new array (about $400,000 at the time) every 1.1TB.
"With the SAN, we were also able to increase the number of servers attached to any one disk subsystem, well beyond 32 up to as many as 80 servers. And we were able to take that $400,000 and amortize it over more servers," Meingarten explains.
Zachary Shess has covered the storage industry since 1995 as an analyst and former senior editor of InfoStor. He can be reached at email@example.com.
BAA soars with SAN-based backup/recovery
By Dan Murphy
British Airports Authority (BAA) owns and operates seven airports in the UK, which serve more than 120 million passengers a year. BAA also manages commercial facilities such as retail, catering, car parking, and property at these airports. Information security is critical at BAA, but a watertight storage strategy had to be developed for BAA to reach its goals.
BAA's information must be continually available. However, because of the fragmented nature of BAA as a government-run organization prior to its commercialization, most of the IT solutions lacked a strategic coherence. A centrally managed storage environment was needed to provide continuous access to data.
In addition, BAA was implementing a new Oracle Financials application. "When we standardized on Oracle Financials, we identified the need for a secure, high-performance, disaster-recovery strategy to ensure the uninterrupted flow of this information," says Simon Gale, systems service manager at BAA. The Oracle database application runs on a Sun E10000 server.
"We had a very tight time frame to implement a disaster-recovery strategy by centralizing the storage," Gale explains. "A SAN solution [from MTI Technology] offered flexibility, complete data protection, and low cost of ownership."
A storage area network (SAN) can be ideal for storage sharing and high-performance, low-latency applications such as databases. In the case of BAA, a SAN brought coherence to the company's storage infrastructure, allowed better access to the database, and provided a disaster-recovery strategy. In addition, the SAN frees up primary network bandwidth through LAN-free and serverless backup operations.
BAA's business continuity program relies on a disaster-recovery SAN environment based on MTI's Vivant V20 storage systems and a DLT tape library at Gatwick.
All the disk arrays incorporate Fibre Channel switch technology, and each array can support up to eight servers.
Serverless, LAN-free backup
Before the SAN installation, BAA's IT infrastructure had been compromised by a fragmented and manually intensive backup-and-recovery strategy. "We were using three different library backup platforms, no one had ownership of the backup plan, and most of the libraries were full. Also, the total cost of ownership [TCO] was unacceptably high," says Gale. "Furthermore, each site was left to develop its own backup schedule, and minimal training was conducted. Ultimately, this expensive strategy failed to ensure our mission-critical information was totally secure."
Gale and his team identified the objectives for a radically re-engineered backup-and-recovery strategy. "Our goal was to create a cost-effective means of efficiently securing systems and business data across all of BAA's UK sites," he says.
First, to reduce the TCO, the company standardized on a single backup hardware platform and a single storage management software vendor (Legato). The new backup environment had to be able to scale quickly and easily to cope with the fast-changing demands of the business, and then BAA developed a consistent backup schedule.
Having achieved their first goal of standardization, a backup-and-recovery infrastructure was formulated. The solution was a backup/recovery system consisting of DLT tape libraries deployed at each of the seven airports. Legato NetWorker storage management software is used to centrally manage the backups.
Heathrow, BAA's busiest airport, required the installation of two large DLT libraries, but at Gatwick a single DLT library was sufficient to handle data backups. The other sites, including airports and BAA offices, operate a range of libraries. BAA's 24x7 System Monitoring Center (SMC) provides monitoring of the backup environment and problem alerting to local administration staff as required.
Once BAA's first two objectives were completed, a consistent backup schedule was easily implemented. Backups are now conducted every night, in line with pre-determined schedules and rotation policies, and tapes are stored off-site for disaster recovery. Using this approach, the company's Unix and NT servers can be restored to a point as close as possible to the original point of failure.
"With the tape libraries, all our remote site information is now totally protected, available to anyone in the organization, and effectively managed. The solution allowed BAA to reduce the backup window by half, saving considerable time, money, and resources," according to Gale.
The SAN infrastructure combined with a centralized backup-and-recovery strategy has helped BAA maintain its required level of data protection, and at the same time, has helped reduce the costs associated with information storage.
Dan Murphy is a freelance storage industry writer with Alliance Communications, in Los Angeles, CA.