For technology veterans, the old adage “The more things change, the more they stay the same” rings true. The idea of a “converged network” has been proposed many times, with several efforts in just the last twenty years. ATM was proposed as a converged transport for everything in the late 1980s, followed by Fibre Channel in 1996, and then InfiniBand in 2002. Ultimately, all of these converged transports found a role within data centers, but failed to become the sole transport for moving data.
Now Ethernet is being promoted as the latest converged data transport, with vendors touting the benefits of a new technology —Fibre Channel over Ethernet (FCoE). Some see this as an attempt to save Fibre Channel by moving it to Ethernet. However, FCoE is not about standardizing storage on Fibre Channel; it is about standardizing on Ethernet as a data center network transport.
Some data center managers are skeptical about transitioning from the proven dual network strategy to a combined infrastructure, and for good reason. In order to be successful, the costs and benefits of FCoE will have to surpass that of existing technologies. As a result, savvy IT professionals are asking themselves two questions regarding FCoE:
- Will FCoE replace my existing Fibre Channel infrastructure, or is it an optional alternative?
- When should I begin testing and deploying FCoE?
First we will look into the technology behind FCoE, review the vendors supplying equipment, and then provide some guidelines for helping companies determine if and when to investigate FCoE.
FCoE technology overview
An important aspect to understanding when FCoE will be adopted is understanding how data center managers make decisions. Change requires capital spending for equipment and budget for operational expenses for implementation, management and other issues. Typically, equipment upgrades are approved in two situations: The first is when service levels are not being met, and the second is when operational costs of old technology outweigh the cost of new equipment. Thus, adoption of new technology only occurs when the opportunity costs and/or management costs of using old technology are greater than the cost of moving to new technology.
For this reason, the vast majority of technology adoptions occur far slower than initially projected. Pragmatic users and budgets outweigh optimistic projections made by pundits. As a reference, the projections for some past technologies are a good indicator. InfiniBand, iSCSI and even 10Gbps Ethernet shipments have all taken longer to reach significant penetration rates than initially believed. Some technologies may never reach 50% penetration; iSCSI and InfiniBand have yet to do so after more than six years and 10GbE is also at less than 50% penetration after a seven-year timeframe.
Adoption rates of technologies also vary significantly by company size and industry. For companies spending more than $1 million per year on storage, adoption rates were much higher for technologies that delivered high performance. For IT organizations spending less than $100,000 per year on storage, ease of use was the driving factor leading to higher use of iSCSI and NAS. In SMB organizations, Fibre Channel is still at less than 50% penetration.
There are examples of technologies taking off rapidly, but even those seen as immediately successful, such as Fibre Channel, took several years to become the predominate connection technology. The rapid growth of Fibre Channel occurred because the pain of existing storage networking technologies surpassed the costs and problems associated with deploying new technology. Even under these ideal circumstances, Fibre Channel took more than four years to become the predominate technology.
A look at the options available for storage networking is shown in Table 1.
FCoE relies on two components. One is a new standard that maps Fibre Channel frames onto Ethernet frames, which is an IEEE T11 standard approved in June 2009. The other component required is a new standard to provide the reliable Ethernet transport mandated by the FCoE specification. This part of the equation is still in progress, although it is advancing rapidly towards a standard.
FCoE requires the use of so-called Converged and Enhanced Ethernet (CEE). The earlier term, Data-Center Ethernet (DCE), is a proprietary Cisco term. The new 10Gbps CEE removes the need for TCP to guarantee delivery of packets, instead placing this requirement into the Ethernet data link layer, much as Fibre Channel’s data link layer guarantees delivery.
To some, the thought of using converged Ethernet means moving storage protocols to a LAN, but this is not the case. The new CEE networks have all the characteristics of storage networks, including virtual channels, priority and quality services, guaranteed delivery, and redundant connections with transparent failover.
Another potential concern is that Ethernet means using TCP/IP, and hence security issues. Again, this is not the case with CEE, as Ethernet remains a non-routable data transport layer. With converged network adapters (CNAs) and CEE, SAN and LAN connectivity will share the same physical pipe, but they will remain logically segregated and protected with quality of service.
Even though they may share the same transport, iSCSI and FCoE have many differences. iSCSI’s value proposition has always been about low cost and ease of use. iSCSI does not provide the same level of performance as Fibre Channel due to several factors, including the high overhead of processing the TCP protocol.
When to deploy FCoE
Data center IT managers are wondering if FCoE will become important. The answer is almost certainly “Yes.” The more difficult question to answer is, “When will FCoE overtake standard Fibre Channel, and will it replace Fibre Channel and iSCSI storage?” The answer to these questions depends on several factors, including how large your company and IT budget are.
As depicted in Figure 1, Fibre Channel-connected storage will likely be gradually phased out and replaced by both FCoE and iSCSI.
Initially, FCoE will be most appropriate for larger organizations that have a need for 10Gbps speeds, and are able to afford continual infrastructure investments and have the IT expertise necessary to address deployment issues. Currently, iSCSI-connected storage is used by smaller organizations, or for specific deployments that don’t require the highest levels of storage performance. Over time, FCoE will move down market as costs fall and deployment issues are refined. With the help of enhanced Ethernet and 10Gbps speeds, iSCSI will overcome performance issues and find a place in larger enterprise environments.
Ultimately, the deployment of CEE will enable FCoE, while enhancing the performance of iSCSI and NAS-connected storage performance as well. Thus, CEE will become the predominate transport for all LAN and SAN protocols in the near future.
In the case of FCoE, there is a change to equipment required, along with an increase in performance to 10Gbps. Companies that are in the process of upgrading to 8Gbps Fibre Channel equipment are unlikely candidates to move quickly to FCoE. However, environments using 4Gbps or even 2Gbps equipment are more likely to make the move soon.
There were many skeptics when FCoE was first announced, because the last thing most IT environments needed was another set of transports and protocols. There are already two well-established networks: Fibre Channel SANs for storage and Ethernet LANs for everything, including storage. Despite recent interest, InfiniBand is unlikely to overtake Ethernet as the common data transport for LAN and SAN traffic.
Where to deploy FCoE
Due to the non-existence of native FCoE storage systems, and the current limitation of the size of networks built using native FCoE, the most likely place for early FCoE deployments are for server I/O consolidation. Practically, this involves using CNAs in place of HBAs, NICs and TOEs on servers, with FCoE transported over CEE edge switches.
The vast majority of products shipping today support this deployment model. As CEE equipment is enhanced and issues with transporting FCoE and other protocols over converged networks are addressed, the opportunity will arise for end-to-end FCoE deployments. However, it will likely be several years before complete reliance on FCoE is warranted.
The FCoE market
Vendors are rapidly adding support for CEE and FCoE, although the transition to FCoE will not be as easy as the transition from 100Mbps to1Gbps Ethernet, or from 4Gbps to 8Gbps Fibre Channel. The convergence of SAN and LAN traffic onto a common infrastructure will require an upgrade of existing Ethernet infrastructure to new, low-latency Ethernet equipment.
The issue that will help push the adoption of CNA cards past HBAs is the addition of a virtual I/O layer. Virtualization is a key enabler for dynamic IT, information lifecycle management, cloud computing, and many other efforts. However, the existing LAN and SAN protocols in use do not lend themselves to port virtualization. The addition of virtual I/O will help the argument for using a CEE data transport and help drive converged host adapters for virtual I/O.
As shown in Table 2, many vendors are shipping, or planning to ship, products supporting CEE and FCoE (although there are no native FCoE storage targets yet). And vendors such as JDS Uniphase have for some time been shipping FCoE test equipment. (JDS Uniphase recently acquired the Network Tools division of Finisar.)
One reason why FCoE will be successful is the number of vendors entering the market. Traditional Fibre Channel vendors are bringing out FCoE products, as are Ethernet vendors and even the existing InfiniBand vendors are offering CEE equipment. InfiniBand vendors Voltaire and Mellanox, for example, have both announced 10Gbps CEE equipment, following products from Cisco and Brocade. Current pricing for CEE equipment is running around $1,000 per port, which is nearly double the cost of standard 10Gbps Ethernet port costs.
Who wins with FCoE
FCoE has benefits for both large enterprises and SMBs. Large IT organizations will benefit by having a way to consolidate server I/O traffic onto a flexible, virtual data transport with less space, heat and cabling. Additionally, these organizations will be moving to a higher speed connection in most cases, providing increased bandwidth. All of these benefits will occur with a high degree of backward compatibility with existing Fibre Channel networks, and the ability to transport other storage protocols over the new high-speed network.
For smaller business, the benefit of FCoE is that companies won’t have to choose which transport technology they will need in the future. The answer will always be Ethernet. That is, SMB organizations can deploy NAS and iSCSI storage over standard Ethernet today. In the future, if the need for Fibre Channel-connected storage arises, they will still be able to utilize their NAS and iSCSI storage, along with FCoE-connected storage by simply upgrading the Ethernet infrastructure.
Many storage vendors are behind FCoE because they see an opportunity to converge data traffic and leverage their investment in I/O technology. Vendors are looking to transition the physical and data link layers from Fibre Channel to Ethernet with minimal costs. Initially, prices for FCoE-capable CNAs and CEE switches will be as high or higher than 8Gbps Fibre Channel. However, as shipments ramp up, increased competition should result in lower prices for end users.
Thus, nearly everyone wins with FCoE. IT organizations with investment in Fibre Channel can continue to utilize their equipment (with bridges). And those without an investment in Fibre Channel will now have the opportunity to utilize Fibre Channel storage and other protocols, such as iSCSI, NFS and CIFS. By standardizing on CEE Ethernet as a transport, companies both large and small will be able to “future proof” their data infrastructure.
Recommendations
Evaluator Group recommends that companies of all sizes create a storage infrastructure strategy and roadmap that includes the new CEE Ethernet. Depending on storage products in use, companies should also consider the use of FCoE at some point.
Only companies with multi-million dollar IT budgets are advised to investigate CEE and FCoE technology in 2009. New investment in scale-out server technology utilizing virtualization and high I/O density are the best candidates for early CEE and FCoE deployments. Wide-scale infrastructure deployments are not advised until 2012 or later.
Recommended deployment for large enterprises:
- Scale-out server virtualization projects with high I/O requirements (CEE and FCoE at edge only): 2009 – 2011
- Additional server I/O consolidation projects, leveraging converged CEE for additional storage: 2010 – 2013
- Migration of existing Fibre Channel storage networking infrastructure to CEE and FCoE: possibly after 2012
SMBs that do not require significant infrastructure improvements may delay investigation into FCoE and CEE equipment until 2011. For SMBs that already have a significant investment in Fibre Channel equipment, new storage deployments should continue to utilize Fibre Channel connectivity. Additional enhancements to Fibre Channel infrastructure, including upgrading to 8Gbps Fibre Channel equipment, is recommended over the next one to two years.
For SMBs that predominantly utilize Ethernet-connected storage, such as NAS or iSCSI, Evaluator Group recommends continuing to invest in standard Ethernet infrastructure. New CEE Ethernet infrastructure investments are unlikely to provide quantifiable benefits until CEE prices begin to approach pricing parity with standard Ethernet in two to three years.
The future is coming, and the answer is Ethernet.
