Block-level storage over IP

STEVE DUPLESSIE -- Enterprise Storage Group
Click here to enlarge image

The following Q&A-an interview with storage analyst Steve Duplessie, with a focus on iSCSI-was excerpted from an online chat session hosted by www.searchstorage.com, a Web-based search engine site for storage-related articles and information (searchstorage.com). Questions were posted by end users.

What is iSCSI?

iSCSI is a proposed standard to do block data calls over Internet Protocol-based networks, specifically Ethernet. It was started by Cisco and IBM and has received the most support among IP storage proponents. In my opinion, iSCSI will be the adopted standard, with some modifications and additions. But there is still a lot of work to do.

Who are some of the key players in the IP-based storage market?

Nishan, NuSpeed (which was acquired by Cisco), SAN Valley, Computer Network Technology (CNT), Pirus, 3ware, and SANcastle.

Isn't IP too slow for storage?

IP isn't the problem. TCP is. It kills hosts and slows down everything. But vendors such as Alacritech are trying to fix this by hardware-assisted TCP NICs (network interface cards) and VI-based NICs from such vendors as Giganet and Troika. I've seen some awesome performance-in the labs-out of a Gigabit Ethernet network running block and file data-99% of wire speed at less than 7% CPU utilization. If you could run block data over your existing infrastructure at least as fast as on a separate Fibre Channel network, why go with the separate network?

What is SoIP?

SoIP is Nishan Systems' "Storage over IP" protocol (which the company has trademarked). Nishan is trying to get its protocol attached to the iSCSI spec, and it believes it has the complete story on solving the problems associated with things like ordered messaging and dropped packets-the things with which TCP deals.

Why use IP, and not Fibre Channel, for storage?

IP has lots of standards. IP is necessary as a primary network, and Fibre Channel isn't. If using your standard network for storage had no performance limitations, why wouldn't you?

What is the point of storage over IP?

The reason we have all this noise about storage over IP is basically because, in theory, if we could do block data calls over existing networks, we wouldn't need to create complex, secondary storage networks such as Fibre Channel storage area networks (SANs).

Will iSCSI replace Fibre Channel, or are they complementary technologies?

One can make a legitimate case that, assuming all the technical challenges are worked out with iSCSI, why would you want a separate Fibre Channel network? Don't get me wrong: Some users have to use Fibre Channel today to solve real-world problems, but if all the hype about iSCSI turns into reality, we could very well come to a point where Fibre Channel becomes irrelevant, although that would be years away.

What will iSCSI offer over network-attached storage (NAS)?

That's apples and oranges. iSCSI is for block data, and NAS is for file-level data, although they both run over an IP network. It's a block- versus-file debate.

Are there any SAN companies currently shipping iSCSI products?

No, because the spec isn't finished. But 3ware is shipping an "iSCSI-ready" disk array, and most of the large array vendors are onboard. The only vendors dead against it at this point are the Fibre Channel purists.

What are the advantages of using IP for SANs?

In theory, the main advantage is one unified network. You wouldn't need a secondary, Fibre Channel-based storage network. One network is easier to deal with than two networks. IP is everywhere; compatibility and interoperability aren't issues; standards abound; and all the problems have been worked out. An independent storage network requires new and different hardware and software, and that costs a lot of money.

How will storage-over-IP protocols maintain data packet sequence order?

That's the problem. It still hasn't been determined, but I think proponents will use TCP to provide sequence ordering.

How does iSCSI differ from NAS supporting NFS and CIFS?

Again, iSCSI is for block data (the raw mechanism for SCSI drivers accessing a disk). NAS doesn't deal with blocks at the front end; NAS deals with files. NFS, CIFS, and FTP are all standards that are based on file-level data. Files are easier to manage than blocks, but many applications have to talk blocks. The lines will get fuzzier as companies like Network Appliance apply file-level NAS advantages to block-oriented applications.

Could you compare iSCSI with InfiniBand and Fibre Channel?

There's really no comparison to be made. Fibre Channel and InfiniBand are examples of transports. iSCSI is a protocol that can run over those as well as Ethernet wires. SCSI is both a transport and protocol. With Fibre Channel, you speak about the SCSI protocol over a Fibre Channel transport.

Is iSCSI a feasible solution?

It will be, but it's way too early in the game for it to be used in any sort of mission-critical applications. I expect Cisco, via its NuSpeed acquisition, to offer first-generation products in the first quarter, but that will be for extended distance usage over WANs-not to replace data center SANs.

What did NuSpeed have that Cisco wanted?

NuSpeed proved that a disk drive could be viewed and used by a server as a local SCSI device, even though it's really connected via Ethernet. The ramifications are obvious: More traffic over IP is good for Cisco, and storage represents a huge potential upside for them.

Why are we seeing storage over IP emerging? Isn't Fibre Channel good enough?

It's a question of what's "good enough." Fibre Channel is difficult, and SANs can be brutal to implement and manage as well as expensive.

What are the emerging technologies that allow connecting SANs over WANs and MANs?

The first real benefit of IP-based storage is tying together disparate Fibre Channel SANs via IP. That's basically what CNT, SAN Valley, Nishan, Entrada, Cisco/ NuSpeed, Lucent/Vixel, and Cisco/ Brocade are working on. CNT has been delivering product for some time. And there will be many more to come.

Will most companies put IP storage on a separate back-end LAN, or will they put it on their regular IP network?

I'd separate it, but with switching technology it may become a moot point.

Will iSCSI be secure?

That's certainly one of the big questions. Security will still be based at the host level, so that won't change. If someone can hack your system today and get to block data, the same will be possible in an iSCSI environment.

Are there any optimizations or changes required to TCP in order for iSCSI to be feasible?

TCP is a pig, so hardware-assisted TCP is mandatory in my view. Alacritech is doing this (see InfoStor, November 2000, p. 12), and many other vendors are in development. Hardware-assisted TCP will become the norm, regardless of iSCSI.

Is iSCSI a concept or a working prototype? What's the current status?

iSCSI is in an Internet Engineering Task Force working group. There are working prototypes, but I think it will be another year or so before it's done.

In an earlier answer, you mentioned technical challenges with iSCSI. Could you elaborate on the top challenges?

Ordered messaging and TCP are the biggest challenges. IP can deal with dropped packets or wrong sequences, but storage can't.

You mentioned "file" vs. "block" in regard to NAS and iSCSI. Can you contrast performance between the two? Shouldn't block transfers theoretically be faster?

Block transfers are always faster, but it's getting closer and closer. With VI (Virtual Interface architecture), file transfers can be faster than block transfers.

How will storage controllers deal with the latencies associated with IP storage? What are the performance implications over long- distance IP networks or WANs?

WAN latencies will be a problem whether it's iSCSI or Fibre Channel.

What are some of the early applications that will use iSCSI?

All of them. Applications won't be aware of iSCSI unless there is significant latency. The application still thinks it's talking to a SCSI disk.

Do you foresee a time when a NAS device is just another client connected to a SAN, rather than having its own dedicated storage?

Arguably, that's already done. Network Appliance uses Fibre Channel on the back end and will have a switched SAN sooner or later. EMC plugs its Celerra NAS server into its Symmetrix arrays in a SAN. So the question is: Will the SAN be Fibre Channel or Ethernet? ATL recently introduced an Ethernet tape library, whereby users connect via standard Ethernet and the DLT library has an NDMP server on it.

How will iSCSI fit into backup solutions?

I'm not sure it will. ATL already has an Ethernet-connected tape library, so users can back up directly from a NAS server to tape over Ethernet.

Much of the argument for using IP networks as a transport for storage is that users already have an infrastructure built around IP, and they wouldn't have to build a separate SAN infrastructure. I think this ignores the fact that many IP networks are overloaded right now. How many users really have a dedicated Gigabit or other private network that would be suitable for storage traffic?

True, but users will have to upgrade their IP networks, but do they have to install a separate Fibre Channel network?

How does the Cisco/NuSpeed solution address performance and response times over a WAN?

It doesn't. They're working with other vendors to mitigate performance issues associated with IP WANs.

With disks being seen as IP devices, are there going to be any distance limitations or latency issues?

Yes. Applications that require certain response times will not be able to use remote iSCSI devices (at least as they are now) as primary storage.

Are the blocks that you mention pieces of a database, or blocks on a disk drive? And in either case, wouldn't applications vendors such as Oracle have to be involved to make iSCSI work?

It could be either. Oracle won't have to do anything for iSCSI specifically, because all applications will see iSCSI as SCSI, so they still deal with raw blocks.

You say that SANs are difficult to implement and manage. But what about storage virtualization products? Don't they address the "stuff doesn't work together" issue?

They do, and I wish they existed two years ago. We wouldn't be in this mess. I think it's important for the whole Fibre Channel community to get behind the storage virtualization vendors. Storage virtualization is brand new, so many end users don't even know about it. It wasn't around when users were ripping their hair out last year.

Steve Duplessie is a senior analyst with Enterprise Storage Group (www.enterprisestoragegroup.com).

This article was originally published on January 01, 2001