IP Storage: State-of-the-market report

The Storage Networking Industry Association’s IP Storage Forum provides a market overview, with a focus on iSCSI-based IP SANs.

By David Dale

It is now more than two-and-a-half years since the release of the first standards-compliant IP Storage products. Since then, a rapidly growing market for the technology has developed. Products are available from a broad range of vendors. And this year saw the release of second-generation products with new features and capabilities that will expand the applicability of IP Storage to a broader range of environments.

This article provides a brief overview of the state of the market, with some pointers as to what the IT industry might expect over the next year. However, first we start by defining terms and clarifying how IP Storage relates to other storage networking technologies.

IP Storage definitions

In SNIA terminology, IP Storage generally refers to a SAN solution that uses Ethernet connectivity in some way. These solutions fall into one of two categories: those that use an Ethernet link to interconnect Fibre Channel SANs via gateways, and those in which a SAN is built using Gigabit Ethernet infrastructure instead of Fibre Channel.

The potential benefits of IP Storage derive from the cost, flexibility, reliability, distance, ease-of-management, and familiarity advantages associated with standard Ethernet networking.

Two IP Storage bridging protocols, both of which are IETF standards, are available to interconnect existing Fibre Channel SAN environments: FCIP and iFCP.

FCIP is a TCP/IP-based tunnelling protocol designed to transparently provide point-to-point connections between geographically distributed Fibre Channel SANs via gateways connected to an IP WAN. FCIP is well-suited to providing connectivity to remote SANs for backup and restore, or remote data replication applications.

iFCP is a TCP/IP-based protocol for interconnecting storage devices or Fibre Channel SANs using an IP infrastructure. iFCP solutions consist of Fibre Channel end-points connected to an IP network by means of iFCP gateways. iFCP is particularly well-suited to providing the reliable transport of storage data between SAN domains via TCP/IP LAN, MAN, or WAN infrastructures.

Although these are important IP Storage protocols, the market for them is the existing Fibre Channel SAN market-which is well-understood at this point. Therefore, the rest of this article will focus primarily on iSCSI.

In contrast to FCIP and iFCP, the iSCSI protocol enables the creation of complete SANs based on Ethernet instead of Fibre Channel. iSCSI is the combination of two mature and well-understood technologies: SCSI block storage commands running over a TCP/IP transport. iSCSI SANs (often called IP SANs) consist of iSCSI initiators (software driver or adapter) in the application servers connected to one or more iSCSI storage systems by means of standard Ethernet switches and cables.

IP SANs are particularly interesting as a SAN alternative to direct-attached storage (DAS) in environments where simplicity, flexibility, and price/performance are critical IT decision factors.


Since they both transmit storage traffic over Ethernet, there has been some confusion about the difference between iSCSI and NAS. Some people even position them as competing technologies, while in fact, they are quite different.

iSCSI is a standard storage protocol, just like SCSI and FCP (the protocols that host operating systems use to communicate with disk storage devices-sometimes called block storage protocols). NAS, on the other hand, uses file-access protocols such as NFS and CIFS, which were originally designed for file-sharing applications (where servers or users access network-based shared files).

From an application perspective, iSCSI-based SAN storage looks just like DAS and so it works transparently with all applications. NAS will usually work fine, but since it is a different paradigm it generally needs to be supported by the operating system vendor and qualified by the application vendor. Some popular business applications simply do not support NAS. Where the water gets cloudy is with NAS-qualified applications, where the more highly virtualized NAS storage environment can deliver important ease-of-management benefits.

For a tutorial on this subject, visit www.snia.org.

An interesting development in first-generation iSCSI storage was the availability of storage systems that can support both iSCSI and NFS/CIFS. Since many companies need both SAN and NAS, this development has become popular with users who need to support both access types with a single storage system, without having to buy separate boxes.


Since iSCSI is a SAN protocol, the market for iSCSI was initially seen simply as an adjunct to the market for Fibre Channel SANs. It was usually characterized as a less-capable technology that would connect stranded servers into the corporate SAN, or provide entry-level SANs for small to medium-sized businesses (SMBs). On the other side of this argument were iSCSI proponents who viewed iSCSI as a disruptive technology that would render Fibre Channel obsolete as a SAN interconnect. The reality lies somewhere between these extreme perspectives.

To date, iSCSI and FCP (the Fibre Channel SAN protocol) have often turned out to be complementary technologies-each having a distinct place in the IT infrastructure as a SAN alternative to DAS. Fibre Channel generally provides high performance and high availability for mission-critical applications, usually in the corporate data center. In contrast, iSCSI has generally been used to provide SANs for business applications in smaller regional or departmental data centers.

iSCSI SANs typically replace DAS in environments that Fibre Channel has been unable to penetrate for reasons relating to cost, price/performance, or complexity, and in data centers with no Fibre Channel SAN expertise.

Fibre Channel SANs provide higher performance than iSCSI SANs based on Gigabit Ethernet. However, iSCSI provides adequate performance for the majority of business application workloads.

The important fact on performance is that both are faster than DAS when benchmarked with real-world workloads. A number of performance analysis reports from market research firms and independent laboratories verify this. See, for example, www.enterprisestrategygroup.com and www.veritest.com.

State of the iSCSI market

International Data Corp. (IDC) estimates that the iSCSI market will be about $300 million in 2005, growing to $1.5 billion within two years and $2.5 billion within three years. Most analysts view the technology as having entered the mainstream, and today almost all storage vendors offer native iSCSI products.

More than 6,000 enterprises worldwide are estimated to be using iSCSI-based SANs in IT production environments, typically in environments dominated by midrange and low-end servers. The vast majority of iSCSI deployments today are on Intel-architecture servers running Windows. NetWare and Linux environments are also becoming popular for iSCSI, and solutions for small RISC servers running Unix are just starting to emerge.

The applications running in these environments tend to be business-critical enterprise ones at the departmental level, although there are some mission-critical applications for smaller enterprises on these servers. Interestingly, the classic SMB market has yet to kick in.

The most popular applications in these environments are e-mail, small database applications, and business applications running on smaller databases. Messaging, Web, e-commerce, and some technical applications are also proving to be a good fit for iSCSI. Finally, companies with homegrown applications running on smaller servers with internal storage or DAS are finding iSCSI to be a good storage consolidation solution, since the migration to an IP SAN is affordable and relatively painless.

The driving force behind a transition from DAS to an IP SAN is often high data growth and the need for operations efficiency. In these circumstances the legacy DAS environment becomes increasingly complex; backup/restore operations become increasingly unreliable; and the storage environment is unable to support the demands of the business.

On top of all that, regulatory compliance pressures, the need to integrate geographically dispersed data assets, and the availability of effective information life-cycle management (ILM) solutions are also drivers behind the move to IP SANs.

Another significant driver behind IP SANs is the availability of IT staff. Most smaller, regional, and departmental data centers have to operate with limited staff. And often, the decision to move to a SAN environment hinges on whether the existing staff can handle it. In these cases an IP SAN solution becomes a viable alternative, since it can almost always be managed by existing staff and skill sets.

Best practices

Now that we have real-world iSCSI deployments, a number of “best practices” are becoming evident.

One early assumption was that iSCSI SANs would simply be cheap SANs with limited functionality. This seemed like a reasonable assumption since entry-level Fibre Channel arrays historically have lacked the sophisticated data management capabilities of their midrange and high-end counterparts (e.g., snapshots, clones, remote copy, and mirroring). However, most first-generation iSCSI arrays provided sophisticated data protection and even disaster-recovery capabilities, including snapshots, mirroring, and replication. Two significant “best practices”-elimination of backup windows and affordable disaster recovery-resulted from this.

Eliminating backup windows

Since snapshot functionality is so common with iSCSI, most deployments back up from a recent point-in-time copy, which eliminates the traditional backup window. Data recovery is also extremely fast from disk-based point-in-time copies. The savings in administrative overhead and end-user downtime from these features alone often justify an IP SAN investment.

In addition, the wide availability of remote copy technology is resulting in the use of disk-to-disk backup, with tape becoming an archive medium in a centralized facility. This makes it possible to automate the backup of remote data assets in satellite data centers and remote offices.

Another consequence of this is that many medium/large companies can now consider tiered storage, matching the importance and availability characteristics of their data with the most appropriate storage medium-fast disks for primary storage, low-cost disks for secondary storage, and off-site tape for archive.

Affordable disaster recovery

The broad availability of long-distance replication and mirroring technology with iSCSI storage systems has enabled even SMBs to implement disaster-recovery plans. This usually involves taking snapshots at regular intervals during the day and using remote copy and/or asynchronous mirroring to copy the changes to a remote system. Since everything is over IP, there is no need for the expense of gateways and protocol conversions, and the solution can be designed to deliver the required service level of the applications. Lower-cost SATA-based disk arrays are often the remote target, making the disaster-recovery solution affordable even for modest budgets.


Two IP SAN topologies have emerged, each with its own security best practices.

The most common topology is similar to the setup for a Fibre Channel SAN. The IP SAN has its own Gigabit Ethernet switch(es), and the SAN is a private network that is separate from the data communications network in a secure data center. Standard iSCSI authentication takes care of access permissions between servers and LUNs.

In the other topology, the IP SAN uses existing Gigabit Ethernet infrastructure. In this case, a subnet is defined, storage access is to defined-function ports, and VPNs are often used to separate and protect traffic between any particular application server and its accessible storage. This topology is most popular in medium-sized enterprises where there is no separate network operations organization.

What’s next?

Most first-generation iSCSI products had feature sets targeted at shared storage for modest numbers of smaller servers, delivering the performance and availability requirements of midrange business applications. Data protection and data management technologies made the solutions feature-rich and provided differentiators.

Second-generation iSCSI solutions are now expanding these capabilities. For example, second-generation iSCSI hardware initiators add support for active-active multi-pathing configurations, boot-from-SAN capability, and improved CPU offload. In addition, second-generation iSCSI software initiators improve the support of high-availability environments with multi-pathing, fail-over, and host cluster services support.

The iSCSI specification defines a number of optional functions that leverage the built-in recovery capabilities of TCP/IP environments. These capabilities are in second-generation iSCSI software initiators and include features such as multi-connection sessions that enable software-based multi-pathing for performance and availability.

Second-generation iSCSI storage systems range from entry-level arrays to high-end enterprise arrays. Options include Fibre Channel or SATA drives for primary storage applications, ATA drives for secondary storage applications, hardware targets for high-performance environments, support for multiple protocols, and support for optional iSCSI features such as multi-connection sessions and error recovery.

Host operating system front

On the host operating system front, over the next year, iSCSI deployments on Windows servers will continue to accelerate, further driving the transition from DAS to networked storage for Windows environments. iSCSI on Linux will become much more prevalent (particularly for blade server environments), and iSCSI on Unix will start to expand beyond the early adopter stage.

After some initial confusion, the market for IP Storage has become well-established. Early predictions that the technology would lead to a significant acceleration away from DAS and toward SANs are now being validated by end-user deployments.

This coming year, the market will continue to grow and expand as IP Storage solutions with second-generation functionality address the needs of an increasing percentage of the enterprise application portfolio.

David Dale is chair of the SNIA IP Storage Forum and an industry evangelist at Network Appliance.

This article was originally published on October 01, 2005