Early users report positive results, albeit in limited application and performance environments.
By Dave Simpson
Few storage technologies have shown as much promise—and inflated expectations—as iSCSI, which enables end users to build purportedly low-cost storage area networks (SANs) that run over standard Ethernet TCP/IP networks. Assuming that they live up to their low-cost promises, iSCSI SANs (sometimes referred to as IP SANs, or iSANs) offer an alternative to Fibre Channel SANs, direct-attached storage, and network-attached storage (NAS) although iSCSI can be used in conjunction with NAS servers to provide both file-level and block-level I/O in the same system.
But it's no secret that end-user adoption of iSCSI is one to two years behind original expectations. This has been due to a number of factors, including
- The weak economy, which put the clampdown on vendors' R&D expenditures as well as end users' IT budgets;
- Lack of support and compatible products from major vendors (although heavyweights such as Cisco, IBM, and Intel have been onboard from the beginning); and
- Lack of a finalized standard.
However, that situation is changing—rapidly. In February, the Internet Engineering Task Force (IETF) ratified the iSCSI standard, paving the way for vendors to develop products that are compatible with a stable specification. A finalized standard is also expected to give potential users the warm and fuzzy feeling that standards generally engender.
More importantly, perhaps, Microsoft this month announced that it will provide free iSCSI software drivers in the June time frame. (iSCSI drivers are also available from Cisco, IBM, and Intel.) Microsoft's iSCSI drivers will be available via Web download for Windows 2000 client and server versions, Windows XP client, and Windows Server 2003, which was introduced this month.
The drivers will include support for data encryption (including the IPSec security standard), iSNS management, management via Windows Management Instrumentation (WMI), and an architecture that aggregates different hardware initiators into a common framework. (For more information: www.microsoft.com/storage.)
"Microsoft's iSCSI support is absolutely key to end-user adoption, particularly because of the client support and support for iSNS [Internet Storage Name Service]," says John Hufferd, a senior technical staff member at IBM and author of iSCSI: The Universal Storage Connection (see "Recently published books on IT.storage networking," p. 22).
According to Microsoft officials, key applications for iSCSI will include low-cost SANs, data replication, server consolidation, backup, and SAN-to-SAN connectivity.
More than 60 vendors are developing Windows-based iSCSI software and/or hardware, and Microsoft created an iSCSI Designed for Windows Logo Program for hardware vendors to certify their devices. In further testimony to Microsoft's iSCSI seriousness, the company is hosting "plugfests" at which vendors can test interoperability of iSCSI hardware and software.
(In addition to Microsoft's plugfests, the University of New Hampshire conducts iSCSI interoperability testing at the protocol conformance level, and the Storage Networking Industry Association [SNIA] conducts iSCSI interoperability testing at the application and systems level.)
Also, a number of other recent product announcements indicate imminent traction for iSCSI. For example, Hewlett-Packard took a step toward supporting iSCSI when it recently announced that it will resell a low-end, multi-protocol (Fibre Channel and iSCSI) storage router manufactured by Cisco. HP's StorageWorks SR2122 has two Gigabit Ethernet ports and two 2Gbps Fibre Channel ports and is priced at $9,995 ($2,500 per port).
At this month's Storage Networking World, LSI Logic demonstrated an iSCSI controller designed for OEMs building iSCSI RAID arrays and NAS systems. (The controller can also be used in host bus adapters [HBAs] for PCI host systems.) LSI will begin shipments to OEMs next month. iSCSI RAID arrays from a variety of OEMs are expected late in the third quarter, according to Ken Zarrabi, senior research marketing manager at LSI. (A sampling of LSI's current OEMs includes Dell, Gateway, HP, Hitachi, NEC, Toshiba, and Unisys.)
What about end users?
Despite the recent flurry of iSCSI product announcements, most end users are curious about how—and why—early adopters are using iSCSI. (For a look at how very early adopters are taking advantage of iSCSI, see "iSCSI emerges as a viable SAN option," InfoStor, December 2002, p. 30.)
At most iSCSI SAN sites, low cost was one of the key reasons for adoption. That was in part the case at Southern Insurance Underwriters Inc., in Alpharetta, GA. Previously, the insurance company was using direct-attached storage, primarily Dell PowerVault systems. But server-attached storage was proving too costly to manage as the company's primary applications (document imaging, Exchange e-mail, and SQL Server 2000 database) grew rapidly, according to Robert Filipovich, IT manager at Southern Insurance.
Southern Insurance's iSCSI SAN is based on an iSCSI Storage Concentrator from StoneFly Networks.
Filipovich first evaluated NAS, which looked good for the document imaging application, but not for the Exchange and SQL applications. "As we expanded our database environment, we needed application-aware storage, and NAS wasn't the best fit for that," says Filipovich. "And for those applications, NAS is generally much slower than block-level SANs, whether it's Fibre Channel or iSCSI."
Southern Insurance also considered a Fibre Channel SAN, but that proved too expensive.
The solution? Southern Insurance opted for an iSCSI SAN based on StoneFly Networks' Storage Concentrator iSCSI appliance and an ATA RAID subsystem from Nexsan. The starter configuration consists of three Windows servers, an Ethernet backbone switch, one Storage Concentrator, and a 2.2TB disk array (see figure above). About 220 users are connected to the iSAN.
Filipovich says that the iSCSI solution provided about a 400% cost savings versus a similarly configured NAS solution. Furthermore, the company did not have to hire or train additional staff, which would have been necessary with a Fibre Channel SAN.
According to Filipovich, the StoneFly appliance is the "brains" of the SAN, handling iSCSI and Ethernet I/O. But he says that StoneFly's software provides the real advantage. The company uses StoneFly software to provision volumes on the disk array, to maintain volumes and connections between volumes and hosts, and for status reporting. In the future, the company hopes to use iSCSI and StoneFly's appliance software for remote mirroring and business continuity applications.
One knock against iSCSI (which typically runs over 1Gbps Ethernet) is that it doesn't deliver the performance of 2Gbps Fibre Channel. Although that's true, in many cases iSCSI provides sufficient performance. Southern Insurance, for example, reports no performance degradation vs. its older direct-attached configuration, even in its heaviest I/O applications.
Southern Insurance plans to move all of its data to the iSCSI SAN and to plug its existing direct-attached disk subsystems into the Storage Concentrator. Theoretically, each appliance can handle 45 devices (15 per port), depending on I/O load.
iSCSI solution for ICA
As with Southern Insurance, low cost was a key reason why the Institute of Contemporary Art (ICA), in London, chose an iSCSI solution. The ICA contracted with Street Vision Ltd., a design company in London, to develop a system to handle archived video files that previously had been stored in analog tape format.
The content included high-resolution graphics (1,280+ pixels at 60-frames-per-second rates). The ICA wanted to digitize and manage the files on hard disk arrays and needed to handle more than 10TB spread across facilities in London, Karlsruhe, and Vienna.
Street Vision implemented a Linux-based software system and Eurologic Systems' 1TB Elantra iCS2100 disk arrays (with Serial and Parallel ATA drives), which were among the first native iSCSI disk subsystems available. The ICA also uses Eurologic's virtualization software to logically divide storage into "pools."
"We were looking for a storage architecture that was scalable and future-proof," says Peter Fuller, managing director at Street Vision. Although the company considered a Fibre Channel SAN, "Fibre Channel is costly, perceived to have a not-very-exciting performance ceiling, and it's difficult to make a multi-vendor case," says Fuller.
Fuller says that the ICA is getting adequate performance without using specialized host adapter cards that offload TCP/IP and iSCSI protocol processing. The ICA uses only standard Ethernet network interface cards (NICs) from Intel.
"We're getting enough performance out of iSCSI to drive high-resolution image generation—around 100MBps reads—and we get up to 75MBps writes for digitization," says Fuller. In addition to the Eurologic arrays, the ICA iSCSI SAN uses standard Ethernet switches from Netgear. In the future, the ICA plans to use iSCSI to do mirroring over Ethernet.
iSCSI and disaster recovery
For Read Branch, senior vice president of information services at Davenport Securities in Richmond, VA, cost was the key selling feature of Okapi Software's iSCSI-based ipXcelerator disk-to-disk-to-tape backup appliance.
"At $20,000, it gave us a lot of bang for the buck," says Branch, who purchased the appliance for both disaster recovery and backup-and-recovery purposes. "It's the cornerstone of our local disaster-recovery plan and it serves as an alternative backup-and-recovery site."
iSCSI is a transport protocol that carries SCSI commands from host systems (initiators) to target subsystem (e.g., disk arrays or tape libraries). SCSI commands are carried in command description blocks (CDBs).
Davenport installed ipXcelerator at an off-site document storage facility three miles from the company's Richmond offices. Data is relayed from the Davenport site to the document storage facility via a Gigabit Ethernet microwave relay system mounted on each site's roof. The same data is also backed up daily to an on-site AIT-1 tape library.
"We're still doing our normal backup to tape daily, but we're using ipXcelerator as an alternative backup resource," he explains. "If we have to recover data, our preferred recovery path would be from the appliance [because of its speed]."
Okapi officials claim that users can simultaneously back up four to eight Windows-based servers over Gigabit Ethernet at speeds of more than 90MBps to 100MBps. Because backup is done to disk (Serial ATA drives), restore is near instantaneous.
Branch says he isn't having any problems completing the daily backup to tape within his allotted backup window, but he knows that that may change. When it does, he says he will consider moving the tape library to an off-site location, relegating it to secondary backup status.
Branch is also considering mirroring data from one ipXcelerator to a second ipXcelerator for added security (i.e., redundancy).
As for installation, Branch says it was time-consuming, but straightforward. "The fact that it is an IP-attached storage device means I can put it anywhere in my network. The network simply sees it as an attached drive."
Branch did run into some problems configuring his Windows NT servers but attributes the issues to problems with NT, not the appliance. He ultimately got around the issue by hopping his NT servers to 2000 and then onto the ipXcelerator.
ipXcelerator is housed in a 2U rack-mount chassis equipped with eight 200GB Serial ATA drives (for a capacity to 1.6TB) and two GbE and one 10/100 Ethernet connection. The unit uses IBM iSCSI software drivers for Windows 2000, XP, and NT. ipXcelerator works with backup applications from vendors such as BakBone Software, CommVault, Computer Associates, Legato, and Veritas.
iSCSI and virtualization
When the University of Michigan's Information Technology Communications Services (ITCom) department set out to upgrade its storage infrastructure, it had a number of requirements, including a storage architecture that allowed volume sharing for clustering, fault tolerance for 24x7 services, and simple management services. In addition, it wanted to use its existing Ethernet network and knowledge base, and it wanted to avoid expanding or training staff.
ITCom teamed with YaGUSA Inc., a storage and systems integrator, to build an iSCSI SAN with in-band storage virtualization for volume management, which is provided by SANRAD's Virtualization Switch and software. The V Switch 3000 hardware supports iSCSI, SCSI, and Fibre Channel. Applications include volume mirroring, striping, remote copy, drive concatenation, LUN mapping, multi-pathing, etc.
Yariv Glazer, president of YaGUSA, explains that "by using iSCSI drivers on campus computers, and having the iSCSI V Switch in the data centers, any system connected to the Ethernet network can read and write data to Fibre Channel or SCSI storage systems." In addition to access and protocol conversion, SANRAD's V Switch provides IT administrators with volume management using in-band, block-level virtualization.
The university used iSCSI to build a storage services provider (SSP) model, in a new messaging system, and for network expansion to a remote university.
Although early users report positive results with iSCSI, hurdles remain. For one, there is a paucity of native iSCSI target devices such as RAID arrays and tape libraries. Although iSCSI disk arrays are available from a handful of vendors—such as Eurologic, NEC, and, within a couple months, EqualLogic—leading RAID vendors are not expected to ship iSCSI arrays until at least the third quarter (although several iSCSI RAID arrays were announced at this month's Storage Networking World conference.) One exception is Network Appliance, which recently announced iSCSI support for its F800 and FAS900 filers (see "Network Appliance becomes early iSCSI adopter," InfoStor, March 2003, p. 1).
In the absence of native iSCSI target systems, users have to install multi-protocol routers or bridges, which are available from vendors such as Atto Technology, Cisco, Crossroads, Nishan Systems, SANRAD, and StoneFly.
"End users would rather have disk arrays that can connect directly to Ethernet and iSCSI NICs or HBAs without having to use routers," says Ahmad Zamer, iSCSI subgroup chair of the SNIA IP Storage Forum and a senior product line marketing manager in Intel's Network Storage Products division.
On the tape library front, Spectra Logic this month became the first manufacturer to ship libraries with iSCSI (and Fibre Channel) support. The company claims 60MBps performance on a Gigabit Ethernet quad interface processor card.
Also in short supply are specialized host cards that offload TCP/IP and iSCSI protocol processing from host systems, which use TCP/IP offload engines (TOEs). Alacritech and Intel have been shipping TOE cards for some time; QLogic and Adaptec are shipping TOE-based iSCSI HBAs to OEMs; and Emulex and other vendors are expected to enter the market later this year.
Although it is possible to build iSCSI SANs without specialized cards (by using iSCSI software drivers in conjunction with standard NICs), TOE cards will be necessary for higher performance in I/O-intensive applications. On the down side, these adapters will boost overall costs for iSCSI SANs. Intel's iSCSI card is priced just under $500, while Alacritech's board is just under $1,000. Although less expensive than Fibre Channel HBAs, these cards are considerably more expensive than standard Ethernet NICs.
In the near future, the focus of iSCSI vendors will be primarily on interoperability and management. "Our number-one priority within the SNIA IP Storage Forum has been interoperability," says Bill Lynn, chair of the SNIA IP Storage Forum, and a senior manager in Adaptec's fabric architecture group.
"The next step is to develop management tools, such as the iSCSI MIBs [management information bases] and APIs [application programming interfaces]."
The focus on management underscores another one of iSCSI's perceived advantages: The ability to use existing IP management tools in iSCSI SAN environments. "Generally, enterprises would prefer to use the management tools they already have in place," says Zamer.
Which end-user companies are most likely to be early adopters of iSCSI? "Conventional wisdom said that iSCSI will initially be implemented at the low end where users have not already implemented Fibre Channel SANs," says Lynn. "However," he adds, "it appears that more of the initial iSCSI deployments are actually at sites that have already implemented Fibre Channel SANs."
"Most of the companies that are using iSCSI now are companies that have used SANs and see the value of SANs. Larger companies with extensive experience with TCP/IP are not afraid to buy iSCSI," says Zamer.
Recently published books on IP storage networking
Designing Storage Area Networks¿Second Edition,
by Tom Clark
Released this month, Designing Storage Area Networks —Second Edition is designed to serve as a practical reference for users, integrators, and vendors designing or implementing Fibre Channel and/or IP SANs. The book is endorsed by the Storage Networking Industry Association (SNIA).
"With the introduction of IP SANs, storage virtualization and enhanced Fibre Channel fabric offerings, users now have a wide—and sometimes confusing—variety of new shared storage options available to them," says Tom Clark, the book's author. "This book helps explain the different capabilities of the technologies and how they can be productively deployed to meet the storage requirements of enterprises today."
The 600-page book starts with general information and then delves deep into both IP and Fibre Channel SANs. It presumes that users—armed with this information—will be able to make a decision about which technology is best for their particular environment.
The first two chapters set the stage and define some basic storage and networking concepts. Chapters 3-5 look at Fibre Channel internals (e.g., physical layer options, classes of service, flow control, etc.), SAN topologies (e.g., point-to-point, arbitrated loop, loop addressing/initialization, etc.), and various Fibre Channel products (e.g., HBAs, RAID arrays, switches, etc.). Chapters 6 and 7 are devoted specifically to IP storage.
The chapters that follow tackle a variety of topics, including SAN software (clustering, backup, etc.), problem isolation, SAN management, storage virtualization, and application studies.
Seven appendices serve as desktop references for users looking for information about various SAN resources (e.g., standards and proposals), vendors, the standardization process, SNIA, and the SNIA Shared Storage Model. The publication also includes a SNIA dictionary and supplementary SAN essays.
Tom Clark is the director of technical marketing at Nishan Systems and is a a SNIA board member and co-chair of the Customer Focus Committee (the liaison to the CEC and CAC customer initiatives).
He is also the author of the book IP SANs: A Guide to iSCSI, iFCP, and FCIP Protocols for Storage Area Networks: A Practical Reference for Implementing Fibre Channel SANs (Addison-Wesley, 2001) and Designing Storage Networks: A Practical Reference for Implementing Fibre Channel SANs (Addison-Wesley, 1999).
iSCSI: The Universal Storage Connection,
by John L. Hufferd
This book tells you all you need to know about iSCSI in an easy-to-read format that includes bulleted chapter summaries, alphabetized iSCSI protocol data units (PDUs), and an exhaustive list of definitions and acronyms.
"There's been a real disconnect between marketing managers and engineers [when it comes to providing useful technology information]," says John Hufferd, author of the recently published book. "This book is intended to serve both audiences."
Hufferd says he tried to provide enough—but not too much—background so that readers could segue easily into the more technical sections and develop an understanding of how all the iSCSI pieces fit together.
That said, for those of you who are less technical, Hufferd recommends focusing on the first three or four chapters, leaving the guts of chapters 5 to 11 to sleepless nights. For you engineer types, though, these pages contain all the nitty-gritty technical stuff you thrive on.
Chapter 2: The Value and Position of iSCSI is a must-read, given iSCSI's recent entry into the storage arena. Hufferd shares his view of the developing market and where it makes sense to implement the technology and where it does not.
He explores the iSCSI/Serial ATA connection, addresses the iSCSI/NAS "dual dialect," and discusses the potential "killer app": backup.
Chapter 3 tells how the iSCSI protocol evolved, including its journey through the standardization labyrinth.
Chapter 4 takes a technical look at the SCSI protocol, PDU structure, naming and addressing schemes, etc.
Chapters 5-13 address the topics of session establishment, text commands and keyword processing, session management, command and data ordering and flow, iSCSI structure, task management, error handling, and synchronization and steering.
Chapter 14 wraps it all up. Hufferd shares his expectations for this market and discusses how to deploy the technology and where it will go from here. John Hufferd is a senior technical staff member at IBM's systems group in San Jose, CA. He has worked for IBM for 36 years.