Although most deployments are in relatively small configurations, end users report immediate benefits—and reduced costs—from IP SANs.
By Alan R. Earls
About a year ago, Atlanta-based HomeBanc's direct-attached storage infrastructure was getting too difficult to maintain, so the company installed a Fibre Channel storage area network (SAN) based primarily on EMC CLARiiON disk arrays. However, when the company re-organized its corporate data center, which serves more than 1,500 employees, it once again faced the need for more storage. One option was to extend the Fibre Channel SAN. However, after looking at the cost of expanding its existing SAN, HomeBanc opted for an iSCSI SAN from start-up EqualLogic based on the cost, management, and scalability benefits it offered.
Akil Woolfolk, HomeBanc's network operations team leader, originally planned to expand the Fibre Channel SAN, but cost considerations and application requirements precluded that approach. At first, he was hesitant about deploying iSCSI because the technology was new and the standards had only recently been ratified. But opportunity knocked when the company decided to move its data center from north Atlanta to the downtown area, making it a good time to implement something new. HomeBanc opted for EqualLogic's iSCSI-based PeerStorage architecture.
Woolfolk is not alone. Although it would be an exaggeration to say that the floodgates have opened for iSCSI-based SANs, it does seem that with standards in place and a bevy of vendors offering products, users are beginning to see the appeal of cut-rate SANs built on the IP protocol. In fact, there is broad agreement among early adopters that iSCSI is cost-effective and offers reasonable performance. However, early adopters vary widely as to how and why they're using iSCSI.
Some of those distinctions can be seen in the experience of Kevin King, an IT engineer at Minneapolis-based NRG-energy Inc., which owns and operates a diverse portfolio of power-generation facilities. In late 2001, the company initiated a disaster-recovery and storage consolidation project. At that time, all of the storage for more than 140 Windows servers was direct-attached. When the company began the project, total storage capacity was approximately 3.5TB. This expanded to about 5.5TB within the first year, with most of the growth coming from the demand for backup capacity. Other drivers for implementing a storage network included disaster-recovery requirements, backup window constraints, and the fact that file-level restores were taking too long.
The initial plan was to use a variety of technologies to achieve a recovery plan. For example, file sharing, which accounted for approximately 1TB of data, was to be hosted on network-attached storage (NAS) servers from Procom (NetForce 3000 Duet systems in a NAS-SAN configuration). And NRGenergy was going to use Cisco's iSCSI routers connected to its application servers (Microsoft Exchange, SQL Server, and Oracle) for routing data to the Duet systems. "We were going to use Procom's replication technology for the NAS storage, and Veritas' replication software for host-based replication of the application servers," says King.
The initial configuration consisted of two NAS-SAN "islands," each with redundant NAS filers, redundant Fibre Channel switches from Brocade, and clustered Cisco 5420 Storage Routers. However, during the first few months of the project, the company decided to migrate the NAS shares to iSCSI-attached Windows 2000 Server-based file shares and to re-allocate the NAS storage to SAN storage, according to King.
NRGenergy needed snapshot technology to reduce backup times and to provide near-instantaneous file-level recovery. The company also wanted to replace its host-based replication software, which consumed CPU resources.
To solve those problems, NRGenergy implemented FalconStor Software's IPStor for replication, storage virtualization, and "time marking," in conjunction with Cisco's iSCSI initiators. The company also upgraded from Cisco's 5420s to its 5428 Storage Routers, which support iSCSI and Fibre Channel.
StoneFly's Storage Concentrators provide provisioned logical volumes in an iSCSI-based IP SAN configuration.
Later in 2003, NRGenergy added StorageTek D220 disk subsystems to increase SAN capacity to 11TB, all of which is virtualized through FalconStor's IPStor software. Servers on the IP SAN run iSCSI drivers from Cisco.
The IP SAN includes Windows 2003 file shares; Windows 2000/2003 servers running SQL Server 2000 and Oracle databases; Exchange 2000 servers, and Windows 2000 backup servers running Veritas Backup Exec to back up all direct-attached storage to disk nightly, then to tape once a week for off-site disaster recovery and long-term retention. IPStor time marks are also auto-mounted and shared out nightly, so end users can perform their own file-level restores on the file shares.
Before settling on the iSCSI approach, NRGenergy considered adding Fibre Channel SANs, but decided against that option in part because the iSCSI solution provided lower cost per host and because the company did not have sufficient Fibre Channel expertise and/or infrastructure.
Instead of using accelerator cards on the iSCSI-attached hosts, NRGenergy uses Cisco's iSCSI software drivers and standard Gigabit Ethernet network interface cards (NICs) from Compaq. Although King evaluated iSCSI offload/accelerator cards from Intel, the performance results did not justify purchasing those cards. King reports that the company is achieving sufficient performance for all of its applications using just the iSCSI software drivers and NICs. "The cost of offload cards negates the immediate cost savings per host," says King.
Despite his overall enthusiasm for the iSCSI SAN, though, King offers some cautions. For instance, he says, in using iSCSI to do a remote boot, there are hardware limitations "above and beyond" what he had run into with Fibre Channel host bus adapters (HBAs), which made it difficult to develop a comprehensive disaster-recovery solution using iSCSI.
Despite a few limitations, King says that "iSCSI has proven to be very reliable and easy to manage and deploy." He says that companies can realize significant cost savings with iSCSI SANs versus Fibre Channel SANs, particularly if they do not already have Fibre Channel expertise and existing infrastructure.
Keep it simple
John DeNardo, director of IT for Eagle County, CO, says his organization's move to an IP SAN was driven primarily by disaster-recovery requirements and the need for simplified storage expansion and backup. Eagle County's IT department evaluated a variety of Fibre Channel SAN solutions from vendors such as EMC and Hewlett-Packard, but in the end opted for an IP SAN based on products from LeftHand Networks because it was "scalable, reliable, and easy to manage," according to DeNardo, who adds that the Fibre Channel alternatives would have consumed his entire annual training budget.
The LeftHand IP SAN supports about 75 different applications related to every aspect of county government services, ranging from tax and assessment applications to law enforcement, office automation applications, and Web services. Applications running on the IP SAN include Oracle, MS SQL Server, and Exchange. The IT department also uses the IP SAN for snapshots, remote copying to a disaster-recovery site, and to create clustered servers for high availability.
DeNardo says that the biggest advantages of the IP SAN—compared to direct-attached storage, NAS, or a Fibre Channel SAN—are ease of management, scalability, and the ability to do remote copies over IP. He also credits the IP SAN with increasing staff productivity and allowing them to tackle new projects. And, he adds, backups have become easier and faster. DeNardo maintains a scripted disk image of all the servers on the IP SAN and replicates to the disaster-recovery site every day over T1 lines using asynchronous IP remote copies. DeNardo plans to implement a Gigabit Ethernet circuit that will permit direct synchronous replication.
The IP SAN also eliminated the need for a dedicated backup application, according to DeNardo. The IT department now relies on snapshots, backups, and remote copies for 24x7 data availability.
DeNardo estimates that the IP SAN cost about half of what a Fibre Channel SAN would have cost.
Meanwhile, T.B. Penick and Sons Inc., a San Diego-based general contractor, faced its own set of storage challenges. IT manager Ken Marsh explains that the move to an iSCSI SAN was necessitated in part by rapid capacity growth.
Originally, T.B. Penick's storage infrastructure was completely direct-attached, but Marsh says that DAS produced configuration and backup problems as capacities escalated. The company implemented a 2.5TB iSCSI SAN, over Gigabit Ethernet, to solve those problems. The IP SAN is based on "Storage Concentrators" from StoneFly Networks and disk arrays from Nexsan Technologies. Additional components include Gigabit Ethernet switches and Windows 2003 servers from Dell (see figure on p. 22). The company does incremental backups daily, with full backups on a weekly basis.
"What's really nice with the SAN is that if I need any part of the storage pie I just go grab it," says Marsh. Formerly, with hard drives installed in each server, some of which were used for mirroring, "there was a lot that could go wrong."
Cost is key
3Com is another example of a company that turned to an iSCSI SAN to solve pressing storage problems. But for Rick Nottingham, 3Com's director of IT architecture, the primary reason for deploying an iSCSI SAN, from Intransa, was based on simple economics.
3Com's IT organization supports more than 3,000 employees with mission-critical applications such as ERP and CRM, which run on a Fibre Channel SAN. But the biggest consumer of disk space was turning out to be e-mail and e-mail attachments. With e-mail stored primarily on direct-
attached storage, costs were high and flexibility was low, according to Nottingham, necessitating an alternative storage architecture. A quick look at the potential cost of extending the Fibre Channel SAN ruled out that possibility. Nottingham says that Intransa's iSCSI SAN was appealing because it enabled the IT organization to make use of existing network infrastructure and administrator skills. To boost performance, 3Com added iSCSI accelerator cards to the hosts attached to the IP SAN.
The IP SAN also made it relatively easy for 3Com to consolidate storage and servers. Nottingham expects to reduce the number of servers from 30 to four by the time the 6TB IP SAN implementation is completed.
iSCSI for backup
For some early adopters of iSCSI SANs, the primary goal is to solve a specific problem such as backup. That was the case for Jerome Quiroga, IT manager at Credit Protection Association, a subscriber retention and recovery firm. "We needed a faster way of creating full backups of our mission-critical servers within an allotted time frame," he says.
The company has more than 2TB of data that it was trying to move to tape each night, within a shrinking backup window. After considering a number of different options, Quiroga opted for Overland Data's iSCSI-based REO R2000 disk-to-disk-to-tape backup-and-recovery "accelerator." Overland officials claim that the device can significantly reduce backup-and-restore times, in some cases providing up to 300% improvement in performance. In addition, they contend that iSCSI provides Fibre Channel-class performance without the expense, complexity, and administrative overhead.
At Credit Protection Association, the iSCSI configuration is used to keep application and data files available for near-instantaneous, disk-speed restores from the REO appliance. In addition, tape archives are created directly from the appliance without increasing the backup window. And centralized management eliminates the need for remote tape media handling. The company also uses disk-to-disk backup/restore as part of its overall disaster-recovery plan.
Quiroga installed the R2000 in his backup network, which consists of two backup servers running Computer Associates' BrightStor ARCserve Backup for Windows, a NAS appliance, and an Overland NEO 4000 tape library. Quiroga says that full backups of the servers and NAS appliance are accomplished in less than eight hours.
Data is immediately copied to tape for permanent archival. Quiroga now performs full backups every night without running into time constraints.
He says that he is not ready to embrace iSCSI for non-backup applications, but that it solved the backup problems the company was experiencing—at an affordable price.
Arun Taneja, consulting analyst and founder of The Taneja Group, says that most of the initial traction for iSCSI is coming from mid-sized companies or departments within larger enterprises that don't have the requirements for Fibre Channel, and in applications where latency requirements are not stringent. He also says that most of the early iSCSI implementations do not include accelerator cards, such as TCP/IP offload engines. "If TOEs are required the company will probably consider a Fibre Channel SAN as well."
Alan R. Earls is a freelance writer in Franklin, MA.
IP SAN solves consolidation, backup challenges
By Christine DiJulia
At Pervasive Software, in Austin, TX, direct-attached storage was leading to severe problems with storage expansion and backups.
"We were storing millions of files on direct-attached storage devices," says Pat Downing, director of IT. "When we ran low on space, we would simply attach another storage device to the server, which only temporarily solved our problems."
Not only was this approach costly, but it also significantly increased the time required for backups. "We had so much direct-attached disk space that we were outgrowing our backup capacity. Some of our servers grew beyond our ability to back them up over a 48-hour period," Downing explains.
IP SAN + tiered storage
Downing's goal was to consolidate storage using a hierarchical storage infrastructure, which is a cost-effective means of storing data and making it accessible according to its importance and frequency of use. Another immediate goal was to alleviate backup challenges. The company turned to SANZ, a consulting and integration firm, and LeftHand Networks for an IP storage area network (SAN) solution.
Pervasive evaluated both network-attached storage (NAS) and an IP SAN, but decided that an IP SAN would provide better price/performance. The IP SAN consists of six Network Storage Modules (NSMs) and Distributed Storage Matrix (DSM) software from LeftHand, which enabled Pervasive to resolve its backup issues by taking snapshots of data on the IP SAN. Snapshots provide a method of performing backups on a live system by creating a copy of a volume at a single instant of time, without having to take that volume offline.
While the IP SAN solved some immediate backup issues, Downing wanted to re-define his entire network infrastructure to prepare for future growth.
Pervasive relied on SANZ's Tiered Storage Method, which cost-classifies data and assigns it to appropriate storage tiers, ranging from expensive, high-performance, high-reliability storage devices to inexpensive, slower devices. Pervasive uses three tiers: the IP SAN, ATA-based NAS appliances, and tape libraries in a hierarchical storage infrastructure.
The company moves its least-frequently accessed data off tier 1 storage devices to less-expensive tape libraries for archiving. Pervasive uses a software tool from Legato to look at aged files to determine how much disk space can be recovered by moving files.
"Moving data that's not accessed often off tier 1 storage reduces backup time," Downing says. "The data still has value, but it's not active and therefore doesn't have to be backed up as often." Downing says that additional benefits of the IP SAN and tiered storage infrastructure include a reduction in the number of physical tapes required to back up the environment, and improved data reliability and availability.
"Storage consolidation solves a lot of IT business problems related to backup, disaster recovery, and restoration," Downing says.
Christine DiJulia is a senior writer at Voveo Marketing Group, a marketing services company based in Malvern, PA.
iSCSI-based backup for SMBs
Earlier this month, StoneFly Networks introduced a turnkey system that provides disk-to-disk and disk-to-disk-to-tape backup and restore in iSCSI- based IP storage area networks (SAN) environments.
One configuration of the "Backup Advantage" bundle includes a StoneFly Storage Concentrator i3000 storage provisioning appliance, backup-and-recovery software from CommVault, and 1TB of disk capacity priced from $29,795.
The software is based on Comm- Vault's Galaxy Express, a lower-cost version of CommVault's QiNetix Galaxy backup-and-recovery software. StoneFly is targeting small and medium-sized businesses (SMBs) with 5 to 12 servers in an iSCSI-based IP SAN. Backup-and-restore operations are offloaded from the LAN onto a dedicated IP SAN, an operation sometimes referred to as "LAN-free" backup.
Storage Concentrators provide logical volume management and storage provisioning, in addition to routing/ bridging functionality. Users can also use StoneFly's Replicator mirroring software to remotely replicate backups.