The first article in a two-part series examines the potential benefits of running storage applications in fabric devices.
By Ashish Nadkarni
Fabric-based intelligence is not new technology, but it is surrounded by a lot of hype. In this two-part series we will help readers answer the question, “Does fabric intelligence make sense in my operation?”
In this article we discuss what fabric intelligence is and why end-user adoption has been slow. Benefits of the technology and business factors that will drive its acceptance are also covered. In part II we will examine products from vendors such as Brocade, Cisco, EMC, HDS, IBM, and Maranti.
The main objective behind augmenting your existing SAN with intelligent fabrics is to be able to make the most of your investment in storage hardware. The biggest advantage that intelligent fabrics offer is the ability to seamlessly and transparently move data between different storage tiers. Before we discuss how intelligent fabrics will play out in your business, let’s first look at what intelligent fabrics do and how they differ from basic fabric switches.
What is fabric intelligence?
Consider an ordinary fabric switch. When a host bus adapter (initiator) is plugged in, it performs a fabric login and discovery process. A similar process occurs when a storage port (target) is plugged in. The discovery process performed by the initiator allows it to access devices (or LUNs) presented by the target. In this scenario the switch acts like a passive device: It functions as a virtual (SCSI) channel between the host port and the storage port. The switch fabric serves as the medium for both the control and data paths from the host to the storage device. In doing so, however, the fabric does not interpret or intercept communication; it is unaware of host-side functions, such as volume management and multi-pathing, and array-side functions such as RAID and data replication.
Now imagine a scenario where the fabric not only intercepts the host-to-storage communication, but also presents a layer of abstraction between the host and storage that allows some or all of the host- and storage-side functions to be “moved up” to the network or fabric. These “intelligent” fabric switches have the ability to present “virtual” or “proxy” initiators to storage ports and targets to initiators. When a storage port is plugged into an intelligent switch port, the proxy initiator configured on that port performs the task of logging in and discovering devices on that storage port. The array doesn’t care; it thinks a real initiator has logged in and presents its devices to this initiator (LUN masking has to be configured for it to do so).
Similarly, when a host bus adapter (HBA) is plugged into such a switch, a virtual target can be presented that the HBA accepts as a real target, logging in and discovering LUNs presented to it. The fabric switch is now fully in control of all communication between the real initiator (HBA) and target (storage port). The “intelligence” in this fabric switch can now provide all host-side functions such as volume management and I/O multi-pathing, and storage-side functions such as RAID, data replication, and memory management, independent of the type of host and storage array it’s connected to. Since the HBA is unaware of the type of array connected to the fabric (since it is logging in and discovering a virtual target presented to it), the administrator can create a volume that spans two different types of arrays and present it to the host. Similarly, it becomes possible to replicate data from one array to another without the need for them to be similar. All this intelligence now resides in the fabric.
From an administrator’s perspective the benefits are clear. Today the unwritten rule is that all hosts plugged into a SAN have some form of host-based volume management, which means every time a change has to be made-either to the software or to the way in which data is accessed (growing or shrinking the file system, moving data from one storage device to another, etc.)-the administrator has to script it or manually perform tasks on every host. If all volume management functions reside in the fabric, changes can be managed centrally with little or no host-side intervention.
Another benefit is improved data mobility. The easiest way to move data between arrays from different vendors today is to use host-based tools or features within volume management. In a large environment, with hundreds or even thousands of hosts connected to a SAN, fabric intelligence can reduce this administrative overhead significantly.
The most important benefit of intelligent fabrics is device and data virtualization. Intelligent fabrics offer two types of virtualization-symmetric and asymmetric.
There are also two types of symmetric virtualization: embedded and appliance-based. In symmetric virtualization, control and data paths are the same, while in asymmetric virtualization control and data paths are separate. Most control path processing is performed by an appliance or a metadata server.
In embedded fabrics, intelligence is ported to the ASIC that controls the switch port or set of ports. In this case, the control and data paths flow through the switch port, eliminating the need for separate control and data paths. In a Brocade AP7420 router, for example, each port has its own ASIC that can, in theory, be programmed with a unique identity. This identity is then responsible for the specific flavor of virtualization. Some of the identities or software flavors that can be programmed are disk and tape virtualization and network-based volume management.
Appliance-based virtualization devices are designed to enable all communication to pass through a single appliance plugged into a SAN fabric. The appliance is responsible for all virtualization and data migration/mobility functions, with no dependencies on the existing fabric. IBM’s SAN Volume Controller (SVC) is an example of appliance-based virtualization. Virtual tape libraries also fall into this category.
In semi-embedded fabrics, the control path is either via a separate out-of-band network or in-band through IP over Fibre Channel. The data path in this case is via the switch port(s). In this situation, the control path is almost always controlled via an embedded blade or an external service processor that communicates with the switch via an API. For example, the SSM module in a Cisco MDS-series switch is a blade that is responsible for providing fabric virtualization. It can be programmed in two ways-either using an embedded application such as Veritas Volume Manager (ported to run on the blade natively) or using its API so it only serves as a data path processor. This type of virtualization is known as asymmetric virtualization.
Interestingly enough, first-generation multi-protocol fabrics are based on the same platforms that are capable of providing fabric intelligence. For example, virtual iSCSI support can be provided using switches connected to storage arrays that do not support iSCSI natively. Even when native iSCSI support is available, it may be cheaper to use a switch. In this case, the virtual initiator in the switch discovers designated devices for iSCSI and then maps them as iSCSI targets onto a Gigabit Ethernet port that iSCSI requests from initiators, which are handled by the switch and mapped onto Fibre Channel (FC) or any other protocols in use. This allows any type of storage array, regardless of vendor or model, to be presented as a virtual iSCSI target.
The same argument applies to FCIP, iFCP, and FC-FC routing: Switches can map native devices onto virtual targets and facilitate the implementation of business continuity, disaster recovery, or data replication at a fraction of the cost it may take to implement native array replication.
How will intelligent fabrics steer storage strategies?
There is no doubt that intelligent fabric technology will mature and eventually become commonplace. Most importantly, perhaps, this technology will enable network convergence-something that Fibre Channel has failed to do. The question then is, “How will it change the way in which companies purchase and provision storage?” It remains to be seen how systems and storage administrators will view the fact that all host- and array-based functions can be moved to the network, leaving very little to be done on hosts or arrays other than pure vanilla provisioning.
It can be argued that a similar shift occurred in the IP industry between layer 2 and 3 devices. It used to be that switches were capable of only providing layer 2 services (i.e., function like switches). All layer 3 services required a router and for application-aware services, a server. Today, most enterprise-class switches are used to provide a fair amount of layer 3 (and higher-layer) services, including routing, content switching, load balancing, and SSL acceleration. These services can be implemented in a single chassis by adding the appropriate blades or line cards and enabling relevant features in the switch operating system. If history is to provide an indication, it will be that the embedded and semi-embedded models become the most popular. In fact, Cisco is already headed in that direction with its MDS and Catalyst switches. Most enterprise models share the same chassis; at some point in the future, storage and IP technologies will converge, allowing users to purchase a single chassis and populate it with line cards necessary to provide all storage services.
Benefits that can be achieved by implementing intelligent fabrics include the following:
Improved ROI through consolidation-If your storage environment consists of several small islands with pockets of unused space, or if it is heterogeneous, you can achieve better ROI by consolidating storage under a single virtual umbrella. This will allow systems and storage administrators to make the most out of existing hardware, transparently and seamlessly distributing data across all existing arrays and minimizing the risk of under- or over-utilized arrays.
Provisioning-Today, a typical provisioning cycle traverses three different layers-storage, network, and host. With intelligent fabrics, most of these tasks can now be handled in the network; once storage has been mapped out for the host, there is very little that needs to be done on the host. Volume management tasks can be handled from the network.
Licensing costs-Typical host-based volume and path management licensing costs are high. Not only is the per-server license dependent on the type of host and the number of processors it has, but for a large environment the sheer number of hosts requiring third-party volume manager and path management licenses swells up the total IT budget. In an ideal world, all operating system vendors would bundle volume and path management software, eliminating the need to purchase additional products. But in reality that is not the case. Porting volume management and path management to the network minimizes per-host licensing costs.
Patches and upgrades-Consider a typical patch or upgrade scenario, such as the following: A volume (or path) management vendor releases a newer version of its product, which means all the hosts in your environment have to be upgraded. Even with enterprise management tools, there is still a fair amount of human interaction involved in this upgrade. With this functionality ported to the network, upgrades are performed in the network with little interaction on the host side. Of course, the downside is that if the upgrade does not go well it affects all the hosts in the environment. This risk can be minimized by connecting each host to two or more switches, a fairly commonplace practice in SANs today.
Data mobility and ILM-It can be argued that this functionality exists today with host-based volume managers. However, the limitation of host-based volume managers is that they have to be addressed one host at a time. For example, it can quickly become a challenging task if data spread across multiple hosts has to be collectively moved across a storage tier. With this functionality in the network, the information life-cycle management (ILM) or data management policy can be centrally applied (and enforced) with little or no intervention on the host side.
Intelligent fabrics offer many benefits, but they are not without challenges. Here are some of the challenges and how to minimize the risks they present:
Where does your data live? Today, with the right set of tools and software, it is relatively simple to get an end-to-end view of your storage utilization. With an additional layer of virtualization the task of making sure that this end-to-end view is maintained becomes much more challenging. While the technology for intelligent fabrics has advanced, the tools and software available may be inadequate to provide that consistent view. This results in the use of native tools, thereby impacting your department’s existing storage management strategy.
Can you effectively leverage your existing storage? In an ideal world, intelligent fabrics should support all kinds of storage devices, including ones that are few years old and all devices from the same vendor. For example, don’t assume that EMC’s “Storage Router” will support all of EMC’s devices. If you have older Fibre Channel Clariion arrays, ask if they are supported. This requirement shouldn’t be underestimated when you are evaluating vendors. Make a list of all the devices you need to support and make sure the vendor either supports them or has plans to do so.
What is the impact to your existing SAN and its management? While vendors are sensitive to customers’ requirements to have devices in their environment be plug-and-play, there is no guarantee that interoperability issues can be avoided. It may be worth the effort to have the prospective vendor provide you with its interoperability test results. It also may be prudent to create a separate VSAN for new devices and introduce them to the network in a phased fashion. Don’t overlook other factors such as security, host support, and management tools.
What is the vendor’s road map? There is peace of mind in knowing that the investment you are making has a future. Although all vendors like to overplay their road maps, do research to make sure that you are purchasing a product that will be backed by the vendor. Ask about the frequency of product and software refreshes, and if there is an upgrade path for the product being purchased.
Are you heading for vendor lock-in? We mentioned that the main objective of purchasing intelligent devices should be to protect your existing investments. However, is this also the objective of the vendor you are purchasing the new product from? Or is it something that the vendor is using to push its own hardware? How committed will a vendor be to ensuring that all competitors’ arrays are supported? Will you eventually be forced by the vendor to buy more storage hardware from them, or will you be able to make an independent and unbiased decision and still be able to make the most of existing investments?
How much up-front investment is required? Investment in intelligent fabrics should not cost you an arm and a leg. In addition to the cost of the devices, research other up-front investment costs. For example, professional services and implementation costs should be factored in when considering a solution. The product may appear to be inexpensive, but it may cost a fortune to have the vendor come in and install it-and you may not have a choice, since the vendor may not support a customer-installed device.
Fabric intelligence sounds utopian in its benefits. So where is the dilemma? While both switch and storage vendors agree that fabric intelligence is a necessity, they have adopted different approaches.
From a switch vendor’s perspective, fabric-based intelligence transforms the fabric from a passive entity to one that plays a central role in all host-to-storage communication. No longer do customers need to be tied into a single storage or host software vendor; they can virtualize all host and storage functions via the fabric.
From a storage vendor’s perspective, intelligent fabrics are fine as long as array functions aren’t duplicated. So vendors such as EMC, HDS, and Network Appliance have taken different approaches to this technology.
HDS, for example, provides virtualization via its TagmaStore array. EMC takes the approach that “intelligence belongs in a storage network closest to what it controls,” according to a white paper on EMC’s Website. This implies that the company’s upcoming “Storage Router” product will not attempt to duplicate the features and functions that are found in its Clariion and Symmetrix arrays. EMC’s claim is that moving storage intelligence to the network attempts to solve problems that have already been solved or in most cases fail to leverage existing technology. In other words, don’t expect EMC to port SRDF or MirrorView to its fabric-based “Storage Router” anytime soon. The technology may exist, but there is a conflict of interest. This is true for most, if not all, storage vendors.
The technology to provide intelligence in the fabric isn’t new. Switch vendors such as Brocade, Cisco, MaXXan, and Maranti have presented road maps and demonstrated intelligent switches, but only recently have vendors begun to push the concept.
Brocade, for example, demonstrated its multi-protocol router with embedded applications from Veritas, Incipient, and Alacritus at its User Conference in spring 2003. At that time the company had bold plans for its launch, but unfortunately the product never made it out in the manner it was envisioned. What went wrong? Was it the technology or the way the big guys push their agenda? The answer lies in the way in which Brocade, McData, and some other switch vendors sell their products-they are resold by vendors such as EMC, HDS, and IBM. Why would EMC sell an intelligent switch running an application from a competitor such as Veritas? Therein lies the other problem: Storage vendors see intelligent fabrics as a threat to proprietary replication and data-protection technologies in their storage arrays. So if a fabric can transparently provide all the facilities available in an expensive storage array, why would someone purchase that array?
Standards may offer a path to ensure that users realize the benefits of fabric intelligence. For example, the Fabric Application Interface Standard (FAIS) is a multi-vendor initiative that defined relationships between infrastructure devices and storage applications. The idea behind this standard is to have all vendors program to a standard set of APIs to allow interoperability and tighter coupling between their products. Look for standards to continue to develop in this area, particularly as more organizations adopt the fabric-based intelligence.
The question is, “Is this technology for you?” If you evaluate all the potential benefits and determine that it makes a solid business case, you should take the next step to evaluate a few products, which will be addressed in part II.
Ashish Nadkarni is a senior consultant with GlassHouse Technologies Inc. (www.glasshouse.com) in Framingham, MA. He can be reached at firstname.lastname@example.org.