Intelligence in the fabric: Myth or reality?

Despite lack of adoption, an end-user survey shows that storage professionals would prefer to run applications in the fabric vs. hosts or disk arrays.


Although much discussion has centered on moving intelligent storage services into the fabric, there has been relatively little adoption of fabric-based services to date. The Enterprise Strategy Group (ESG) set out to determine the real reasons behind this apparent lack of user interest. Are end users actually satisfied with how they manage their current storage environments, or is there just a lack of understanding about the benefits of moving certain services (applications) into the fabric? Or is the real reason more basic than that? Perhaps users aren't aware that fabric-based applications can help them reduce hardware, software, and administrative costs.

In an extensive end-user survey, ESG found that while the majority of end users are interested in adopting fabric-based intelligence, few of them believe they would realize significant cost savings. Yet those that have already adopted fabric-based services have far exceeded their savings expectations and have realized a number of other benefits that they had not expected.

ESG's report on the survey includes input from more than 300 end users, in addition to results from user interviews. The respondents were from a variety of vertical industries, including financial sevices, manufacturing, telecommunications, government, education, and healthcare. All of the respondents were involved in some way in storage purchasing decisions and were responsible for storage budgets that ranged from $250,000 to more than $20 million.

Despite the variety of respondents, we found surprising consistencies among users that had already adopted or were willing to adopt fabric-based services. Also surprising, users with the most capacity were not necessarily the ones looking to adopt fabric-based intelligence.

What is fabric-based intelligence?

For the survey, ESG defined "intelligence in the fabric" as "solutions in which storage services such as virtualization, volume management, and data replication execute directly on network-resident devices such as SAN switches or purpose-built appliances, as opposed to hosts or storage subsystems."

Click here to enlarge image

Our goal was to answer questions such as the following:

  • Are users interested in network-based storage -intelligence?
  • Which market segments have deployed or will deploy intelligent storage network solutions?
  • What is driving the need for fabric-based applications?
  • On what platforms do users want applications to be delivered?
  • Which storage services will move to the network?
  • From which type of vendors (large/established or small/start-up) will users buy intelligent network solutions?

What came across loud and clear is that users absolutely want to run storage services in the fabric, with services such as volume management, provisioning, virtualization, and serverless backup being ranked high on all of the users' lists. ESG ran two independent surveys and found that of the 210 respondents to the first survey, 25% had adopted some form of intelligence in the network, and of the 100 respondents to the second survey, 34% had adopted some level of network-based services.

Among respondents that have implemented fabric-based applications, most had plans to move more services into the network over the course of the next 18 months. What was perhaps more impressive is that 93% of all the "non-adopters" were interested in or open to implementing services in the fabric. It is very rare that we find that little resistance to adoption of new technologies, specifically when adoption will impact existing processes. However, with that much interest, shouldn't there be a lot more traction for fabric-based intelligence?

We attribute slow adoption to lack of

  • Understanding of exactly what "intelligence in the fabric" means;
  • Awareness (and proof points) regarding the benefits of implementing intelligence in the fabric;
  • Availability of products from the large vendors; and
  • Awareness regarding products that actually are available today.

In short, vendors need to educate end users. Now that users are realizing tangible benefits, it's up to the industry to get the word out and to clear up the confusion. One end user from a major financial institution stated:

"Every time a vendor comes in to talk about intelligence in the fabric, I stop them and make them tell me what exactly they mean by 'intelligence.' I know what I think it is, but I need to make sure we are on the same page."

Not only will clearly articulating the benefits of fabric-based intelligence help vendors sell their products, but also it is now becoming increasingly important in order to prevent losing sales. For example, one user we spoke with actually migrated a 1,600-port SAN from an existing vendor to Cisco because Cisco could more clearly articulate a virtualization strategy and had services offerings that aligned with the user's long-term storage management goals.

Testimonials from early adopters clearly indicate that there are significant benefits to running intelligent storage services in the fabric. While the majority of early adopters stated they initially moved to fabric-based services in order to "simplify the management of our storage resources," more than 80% of those users also realized significant hardware cost savings_with a mean average of almost 20% cost savings per year. This is in contrast to users who have yet to implement fabric-based applications, who overwhelmingly downplay the potential cost savings they believe they could realize. The figure on the left shows the differences between expected cost savings and actual cost savings. The concept of cost avoidance also seems to play heavily into users' desire to move to network-based services. The same Cisco customer quoted above stated:

"If (capacity) continued growing at the current rate, we would need 3x the staff [to manage the storage]. With intelligence in the fabric we believe we can maintain with our current staff."

Who wants it, and why?

Unlike many new technologies that tend to gain initial acceptance in particular vertical markets, the ESG survey shows that adoption and interest in fabric-based applications is based more on the complexity of the storage environment than on the type of business or industry. Early adopters and those with current buying plans had multiple SAN fabrics, with large port counts, often located at multiple sites. In fact, about 70% of the early adopters had more than 10 SAN fabrics. Although these organizations also tended to have larger amounts of capacity than non-adopters, that factor alone was not the primary reason for adoption. The survey results show it's not necessarily how much capacity an organization has, but how that capacity is networked and how many fabrics the organization has that drives users to adopt network-based applications.

End users cited "simplifying storage resource management" as the number-one objective of moving to fabric-based intelligence, followed by a variety of other objectives (see figure, right).

According to one user: "We decided to go with an intelligent fabric solution for a few reasons. First, we had poor storage utilization rates-around 42%. We wanted to get that space back without purchasing additional capacity. Second, we wanted a central way to manage all storage, especially provisioning. Last, we wanted the ability to do snapshots between heterogeneous arrays."

It is important to note that not all early adopters have large-scale SAN implementations. For example, approximately 30% of the survey respondents had one to five fabrics in their organization. However, the survey results consistently show that organizations with large-scale, multi-fabric SANs are more apt to be interested in adopting storage services in the fabric. Of those users that have not yet adopted, 40% of those with multiple SAN fabrics plan to implement fabric-based storage services, and 100% of those that are "very interested" in deploying these services have multiple fabrics.

According to another user: "To me, intelligence in the fabric revolves around virtualizing our storage environment so that we can automate everyday storage tasks like provisioning. I know that we're not getting any additional headcount so we need to move toward true policy-based storage management."

What, when, and how?

Regardless of the size of an organization's SAN implementation, users were very consistent about which storage services they would like to reside in the network as opposed to hosts or disk arrays. Volume management topped the charts for both early adopters and non-adopters (see figure, top right). It seems that users are tired of implementing volume manager licenses at the host level; administrative issues and licensing costs were the two key reasons for moving volume management into the fabric.

Virtualization and provisioning capacity were the next most popular reasons; beyond those reasons, early adopters differed a bit in their preferences for serverless backup vs. non-adopters' desire for quality of service (QoS) being implemented at the fabric level. Surprisingly, some of the services that we expected to rank higher (e.g., snapshots, replication, and data migration) did not rank high on users' priority lists, even though these services have been available in fabrics for some time. For example, these services are available from vendors such as Cisco, Candera, FalconStor, Hewlett-Packard, IBM, Maranti, MaXXan, StoreAge, Troika, Veritas, and others.

Overall, early adopters were satisfied with the results of implementing fabric-based services, and virtually all of them intend to move more services into the fabric.

Click here to enlarge image

Although the survey indicated that many users want to implement services on a switch, there were mixed opinions on whether services should be processed on each port on the switch (known as distributed mode) vs. running services to each port from a single "centralized" processor blade (centralized mode). Of course, there are not a lot of options available today. For example, only Maranti is shipping a distributed mode switch implementation, with MaXXan and CNT soon to follow (perhaps by the time this article is published). And only Cisco and MaXXan have blade-based solutions based on a centralized mode model. Many of the users in the ESG survey were not aware of these products. Market leaders such as Brocade and McData aren't expected to ship platforms with intelligent storage services until next year.

Interestingly, many users we interviewed expect to implement services on both switches and appliances, depending on the type of service. One user stated:

"We don't want a single point of failure, so we would want to distribute services across both switches and appliances. Services that move data, such as replication and migration, should be in the switch, while other services will most likely be implemented on appliances."

Other users plan to adopt appliance-based solutions for more practical reasons; for example, appliances are easier to implement in existing fabrics, and they are currently available. Candera, Hewlett-Packard, IBM, Troika, FalconStor (with Network Engines), and MaXXan (with FalconStor) all ship purpose-built appliances (the hardware was built specifically to run fabric-based storage services).

We do not expect users to continue to deploy storage services software on standard PC servers. Although a number of early adopters have deployed this configuration, most would prefer to run applications on purpose-built hardware. Lucky for them, software-only vendors such as FalconStor, Incipient, Softek, StoreAge, and Veritas, as well as EMC, have ported or are planning to port their software to purpose-built appliances or switches.

End users will have a plethora of choices for fabric-based applications, and it will not be easy to determine the right platform for each environment. We expect to see a mix of switch- and appliance-based implementations at first, with users eventually migrating primarily to switch-based solutions. One user who has already adopted an intelligent switch with a distributed (port by port) architecture stated:

"We have to buy new ports anyway as we expand; to us, the only additional cost is turning on the intelligent services on each port, so why pay for a separate implementation?"

The big question is when widespread adoption of fabric-based applications will begin. Of course, some users have already implemented intelligence in fabrics. Vendors such as DataCore, FalconStor, and StoreAge have been shipping fabric-based applications for some time and have achieved varying degrees of success in converting users to fabric-based applications. However, this market won't really take off until products are available from larger vendors. That is not necessarily because those products will be better than what is available today, but because the larger vendors have the marketing budgets and clout to educate end users about the benefits of implementing storage services in the network.

Click here to enlarge image

IBM has already had some success with its Storage Volume Controller, which is available in a stand-alone appliance as well as on a blade that can be integrated into Cisco's MDS switches. Veritas has also ported some of its software to Cisco's blades. Furthermore, we expect product announcements from vendors such as Brocade, EMC, and McData soon, which should bring some momentum to the market. In fact, a number of users we surveyed are waiting for the larger vendors to release products before they consider implementation.

Smaller vendors will benefit from the momentum generated by the large vendors. For example, early adopters and non-adopters alike stated that hardware performance and reliability were key purchasing criteria, with vendor reputation and existing vendor relationships ranking much lower on users' lists. This indicates that innovation can prevail, and because cost is one of the biggest hurdles to adoption smaller vendors may in fact have competitive advantages.

However, ESG predicts that it will be 12 to 18 months before we start seeing a significant ramp in end-user adoption of fabric-based intelligence.

The bottom line

Our survey goal was to determine if there is a market for fabric-based intelligent storage services, and based on the survey results the answer is a resounding "yes." Obviously, most of the large vendors have realized this, too, as they are finally (and in some cases, perhaps reluctantly) offering fabric-based services that will enable users to more effectively manage heterogeneous networks.

With network-based services, users no longer have to purchase mirroring, replication, and migration solutions that only work with one vendor's arrays. Users will no longer have to administer volume management agents on every host and can implement serverless backup across SANs. Users can implement these solutions today, but it is up to the vendor community to educate customers about the benefits of running storage services in the fabric.

For further information about the ESG research report, "The Future of Network-Based Storage Intelligence," visit www.enterprisestrategygroup.com.

Nancy Hurley is a senior analyst with the Enterprise Strategy Group in Milford, MA.

This article was originally published on October 01, 2004