Who's using IP Storage, and why?

SNIA's IP Storage Forum explains the potential benefits of IP Storage and presents case studies of early adopters.

By David Dale

This past year has been an exciting time for IP Storage a the industry moved from new protocols to standards-based interoperable products to end-user deployments in production environments.

A big impetus to deployment came in mid-2003 when Microsoft released iSCSI support for its Windows 2000 client and server versions, Windows XP, and Windows Server 2003 platforms. Other operating system vendors are also on board. For example, iSCSI host software drivers are now available for Linux (RedHat and SuSE), Novell NetWare, and HP/UX. Microsoft's publication of qualified iSCSI storage targets in its Solutions Catalog added to the momentum.

But IP Storage momentum was also driven by end-user requirements for cost-effective storage consolidation and data-protection and disaster-recovery solutions that leverage existing investments in Ethernet infrastructure and expertise.

This article explains where IP Storage fits and describes how a number of IT organizations are using IP Storage solutions in production environments today.

IP Storage defined

IP Storage refers to block storage over Ethernet and consists of three basic protocols: iSCSI (Internet Small Computer System Interface), iFCP (Internet Fibre Channel Protocol), and FCIP (Fibre Channel over IP).

iSCSI is a means of transporting SCSI packets over TCP/IP, providing for Ethernet-based storage area networks (SANs). iFCP and FCIP are bridging technologies that enable interconnection of Fibre Channel SANs through gateways connected to TCP/IP networks.

iSCSI solutions consist of iSCSI initiators (software drivers or host adapters) in servers connected to iSCSI targets (storage systems) by means of standard Gigabit Ethernet switches and cables. iSCSI is particularly interesting for storage consolidation in environments where simplicity, flexibility, and price/performance are critical IT decision factors, as well as for cost-effective backup and disaster-recovery solutions.

iFCP is a TCP/IP-based protocol for interconnecting Fibre Channel storage devices or SANs using an IP infrastructure. iFCP solutions consist of Fibre Channel end-points connected to an IP network by means of iFCP gateways. iFCP is particularly well-suited to providing reliable transport of storage data between SAN domains via TCP/IP LANs, MANs, or WANs.

Click here to enlarge image

FCIP is a TCP/IP-based tunneling protocol that provides point-to-point connections between geographically distributed Fibre Channel SANs using FCIP gateways connected to an IP network. FCIP is well-suited to providing connectivity to remote SANs for backup and restore, or remote data replication applications.

The benefits shared by all of these solutions derive from the cost, flexibility, manageability, distance, and familiarity advantages associated with Ethernet networking technology.

To understand why end users are deploying IP Storage solutions, it's best to start with a view of the overall IT infrastructure of a typical organization.

The figure on p. 20 illustrates the typical IT infrastructure of a large organization. It consists of a core data center, usually running the company's mission-critical applications. Up to 20% of the company's data assets are likely to be managed here. The other 80% of the data is distributed among the company's regional and departmental data centers and across remote offices around the globe.

The core data center usually houses a company's largest servers, is divided into production and test/development environments, and is usually considered home base for corporate IT. The penetration of networked storage, as opposed to direct-attached storage (DAS), is likely to be up to 70% of the storage in the core data center—the vast majority being in Fibre Channel SAN environments. Core data centers typically have Fibre Channel specialists, as well as Ethernet-savvy server and network administrators.

The core data center may be duplicated at a remote facility for disaster-recovery purposes, but the more likely scenario is that the disaster-recovery facility is one of the company's regional or departmental data centers.

In regional and departmental data centers, servers are usually smaller and much more numerous. The applications running in these environments may not be mission-critical, but they are business-critical and are typically characterized by significant data growth. And that data growth causes a constant staffing and asset management problem.

Regional and departmental data centers can benefit significantly from networked storage, but the penetration of networked storage is typically less than 30% in these environments—often with a fairly even mix of network-attached storage (NAS) and small SANs. This type of data center is unlikely to have a dedicated storage staff or Fibre Channel expertise.

Outside of regional and departmental data centers, many companies have a significant number of remote offices. IT support for remote offices poses challenges, and data availability can be a problem. Most companies are struggling with the issues of integrating these offices into their corporate data-protection and management environment.

Where IP Storage fits

IP Storage solutions broaden the options available to IT organizations to address the cost, availability, performance, and manageability issues caused by continual data growth, as well as to accelerate a transition from DAS to networked storage. The figure below illustrates where IP Storage is being deployed today in the context of the IT infrastructure previously described.

IP Storage bridging technologies link existing Fibre Channel SAN environments, primarily for long-distance replication and disaster recovery. The use of IP networks overcomes the distance limitations of Fibre Channel and is less expensive than using proprietary protocols over private fiber links. Several different applications are explained below.

iSCSI provides the basis for SAN storage solutions (primary storage) in environments where Fibre Channel is not cost-effective. Examples of this are in departmental data centers where there are a lot of lower-end servers—where Fibre Channel host adapter and switch costs can be significant compared to the cost of the servers. These environments typically run midrange enterprise applications (often Microsoft applications), which are an excellent price/performance fit for iSCSI over Gigabit Ethernet.

iSCSI is also a potential solution for secondary storage in this environment (e.g., disk-based storage for regulated or archive data). In these environments, disk storage is often based on ATA or Serial ATA disk drives and offers a cost per megabyte that is comparable to small tape libraries, with the advantage of much faster data restore speeds.

iSCSI is also a good candidate for SANs in remote offices, providing the performance and availability benefits of centralized storage. These solutions can also support file storage in converged SAN-NAS configurations. If these remote offices are linked back to primary or secondary storage in a regional data center, the combination of remote replication and point-in time copies (snapshots) not only eliminates the backup window and need for tape in remote offices, but also delivers high data availability.

Deployment examples

The following end-user case studies illustrate the benefits of IP Storage. Diagrams for each of these case studies can be found in the IP Storage Forum customer deployments presentation at www.ipstorage.org.

Disaster recovery for Fibre Channel SANs

Telus – Mcdata

TELUS is a Canadian ISP/SSP that uses IP Storage to provide disaster-recovery services to customers between Calgary and Toronto. Deploying a disaster-recovery solution over 1,600 miles would typically be very expensive, but TELUS offers this service at a fraction of the normal cost.

TELUS is using an iFCP-based solution from McDATA in combination with bandwidth provisioning to enable customers to interconnect geographically dispersed Fibre Channel SANs over a WAN. This approach provides fault isolation for each of the Fibre Channel fabrics in the network, thus making sure that customers' networks are insulated and protected from each other.

Click here to enlarge image

TELUS can use the same infrastructure to carry data over long distances. McDATA's vendor-specific enhancements such as Fast Write and Compression mitigate the latency of the long-distance link and maximize utilization of the customer's slice of the pipe.

Ohio Savings Bank – CNT

Ohio Savings Bank (OSB), one of the largest mortgage lenders in the US, needed to implement a cost-effective disaster-recovery solution to maintain 24x7 access to customer data for its Website, call center, and mortgage applications, with extremely high availability and zero risk of data loss.

The solution OSB chose mirrors data in synchronous mode (real-time) between two EMC Symmetrix storage arrays 11 miles apart over a Gigabit Ethernet link. Also included in the configuration are four CNT UltraNet Edge Storage Routers, which provide the remote connectivity for reliable transfer of SRDF-mirrored data. The solution is fully redundant with no single points of failure. A wireless network, to provide another redundancy layer, also backs up the network.

OSB chose an IP Storage solution because it didn't want the expense of dedicated lines. The company wanted to share a Gigabit Ethernet link with other network traffic, thereby saving bandwidth costs. The results of this approach are reduced business risk from any sort of outage, reduced telecom costs (by using IP), and achieving replication with no impact on performance.

Distance replication for Fibre Channel environments

Skadden, Arps, Slate, Meagher & Flom LLP – FalconStor

To maximize business continuity and meet recovery-time objectives in the event of a data-center disaster, the law firm of Skadden, Arps, Slate, Meagher & Flom LLP's IT department was tasked to replicate several terabytes of data from the firm's document management and messaging systems located in London and Tokyo to its US-based disaster-recovery site.

Skadden selected FalconStor's business continuity and disaster-recovery infrastructure. The FalconStor solution, based on FCIP gateways in combination with IPStor software, provides the ability to replicate live data with reliability (auto reconnect), efficiency (data compression), security (end-to-end encryption) and transactional integrity (database snapshot agent).

The ability to maintain up to 255 versions of each replica in the disaster-recovery center allowed the IT department at Skadden to access any disk image for quick recovery and to guard against "rolling disasters." Snapshot Agents for GroupWise running on NetWare were implemented to guarantee the replica's data integrity, while eliminating the need to perform time-consuming database consistency checks before mounting the replica at the firm's disaster-recovery site.

Fibre Channel SAN extension

PBS – StoneFly Networks

The Public Broadcasting System (PBS) became an early adopter of iSCSI technology last year by capitalizing on IP Storage to leverage capacity in its existing Fibre Channel SAN. The organization's IP SAN is anchored by StoneFly Networks' Storage Concentrators.

PBS's storage challenges included

  • A number of servers that weren't attached to the Fibre Channel SAN;
  • Escalating storage consolidation demands that drove the need for rapid deployment of a scalable, tiered storage model;
  • The need to extend centralized storage to blade servers;
  • Limited IT resources dictated no additional staffing; and
  • Flat budgets required more efficient storage provisioning.

To address these problems, PBS implemented two separate IP SANs supporting streaming media on the PBS Website, intranet Web content, FTP, and SQL databases and Web log processing applications. The iSCSI environment encompasses six production servers (Windows 2000 and Linux) and three development servers.

By connecting this environment to its existing Fibre Channel SAN, PBS implemented a tiered storage model delivering streamlined management and cost-effective storage for all levels of server data as well as blade servers.

Affordable SAN for distributed environment

Eagle County – LeftHand Networks

The IT department for Eagle County, CO, began a project last year to update its aging IT infrastructure. The original DAS model was difficult to manage and scale, and backups were not completing within the backup window. In addition, the county did not have a viable disaster-recovery plan.

Eagle County consolidated its application servers and moved from the cumbersome DAS model to shared storage. The IT department purchased a 10TB iSCSI SAN from LeftHand Networks to support Microsoft Exchange 2000, SQL Server, and a variety of custom applications. About 5TB of primary storage is located in the main facility, and data on the primary storage is replicated to a disaster-recovery facility that houses an additional 5TB of storage. The disaster-recovery facility is located approximately seven miles from the primary site and is connected via a partial T1 link.

Acquisition cost of the IP Storage SAN was 45% less than alternative approaches, and the county's ability to manage the SAN infrastructure with existing staff has also resulted in considerable savings.

Baker Hill – Intransa

Indianapolis-based Baker Hill, a developer of software applications for financial institutions, was running into a typical IT problem: sprawling servers and increasing storage demands. As the number of servers grew, so did the expense and management required. To combat these problems, the company undertook a storage consolidation project.

To consolidate the storage resources for Windows Server 2000 and 2003 servers, Baker Hill implemented an iSCSI SAN with an Intransa IP5000 IP storage system. The 2TB SAN used existing network equipment, including Gigabit Ethernet switches from Extreme, and required minimal administrator training.

The initial deployment at Baker Hall uses the Intransa IP SAN for MKS Integrity Server, Microsoft SharePoint, and disk-to-disk backup. The company uses Intransa's StorControl Snapshot option to reduce backup windows and provide fast recovery times. And provisioning, re-allocating capacity, and accommodating future growth is far easier than with the company's previous DAS architecture.

Affordable storage consolidation

Solucient – EqualLogic

Based in Evanston, IL, Solucient provides market research, consulting services, and applications for the healthcare industry. The company maintains one of the world's largest healthcare databases, with more than 20 million records from hospitals, insurance companies, and pharmaceutical manufacturers.

Solucient's data-storage requirements were increasing by 5TB to 8TB per year as a result of adding clients and new applications. When Solucient recently launched a next-generation application that provides comparative databases and statistical indexes for its hospital and pharmaceutical clients, it needed a networked storage solution for all of the development, configuration management, and testing efforts associated with the project.

After determining the high cost and network integration requirements of expanding its existing Fibre Channel SAN, Solucient decided on an iSCSI SAN from EqualLogic because of its affordability, ease of implementation, and scalability. As a result, Solucient was able to address its immediate storage needs while making a strategic investment in a scalable system that can expand easily as its needs grow—at approximately one-third the cost of a Fibre Channel solution.

Cross Country Healthcare – Adaptec

Cross Country Healthcare Inc. is one of the largest healthcare staffing service providers in the country. The company is extending its online self-help offering for both its healthcare provider customers and the healthcare professionals that serve them—but these online offerings demand significant storage.

Recently, the company's legacy DAS architecture began showing signs of limitations. Storage utilization was low (approximately 65%). This wasted space forced the company to continually buy large-capacity servers and external storage arrays. Cross Country Healthcare determined that an Adaptec iSCSI SAN was a viable option due to lower administration and cost.

Cross Country Healthcare noticed an immediate improvement in storage utilization with the iSCSI SAN. The amount of wasted disk space was down by as much as 90%. And by maximizing disk usage, the company was able to better leverage its existing storage assets—which translated to cost savings.

Centralized management of remote facilities

Navitaire – Network Appliance

Navitaire Inc., an application services company, provides outsourced airline reservations and accounting systems to more than 40 airlines. A wholly owned subsidiary of Accenture, Navitaire is headquartered in Minneapolis, with offices in Salt Lake City, Austin, Sydney, Manila, and Prague, and data centers in Minneapolis and Sydney.

The company deployed a global IP Storage infrastructure with data mirrored from regional offices to a regional data center, which is then backed up to secondary storage at the data centers in Sydney and Minneapolis.

To support the expansion of its international operations, the company needed to deploy networked storage for additional Microsoft Exchange and SQL Server applications in London, Minneapolis, and Sydney. It proved to be cost-prohibitive to build local Fibre Channel SANs and to hire on-site IT resources. Instead, Navitaire deployed Network Appliance's FAS200 storage systems running the iSCSI protocol in London, Sydney, and Minneapolis. Setup of hosts and storage was simple, and now the new applications are tied into the company's global data-management and data-protection infrastructure.


As these case studies illustrate, the emergence of IP SAN solutions means that IT professionals now have SAN options for a broader range of environments at lower prices. Storage consolidation, simplified data protection, affordable disaster recovery, and improved data management are just some of the benefits of IP Storage.

David Dale is chair of the Storage Networking Industry Association's IP Storage Forum (www.ipstorage.org).


This article is based on case studies of customer deployments of IP Storage solutions supplied by the following SNIA IP Storage Forum member companies: Adaptec, CNT, EqualLogic, FalconStor Software, Intransa, LeftHand Networks, McDATA, Network Appliance, and StoneFly Networks.

This article was originally published on May 01, 2004