EchoStar Communications builds large-scale SAN

Deployment of a storage area network posed challenges and required third-party help from CNT, but the benefits are expected to outweigh the costs.


Two years ago, satellite media company EchoStar Communications began the planning phase for a storage area network (SAN) that would replace its server-attached configurations. Quickly realizing that the project was beyond the expertise of even the most skilled IT departments, EchoStar called upon Computer Network Technology (CNT) to provide not only SAN infrastructure equipment, but also consulting and integration services.

EchoStar is a direct satellite television service provider best-known for its DISH Network (www.dishnetwork.com), which has more than five million customers. DISH Network provides more than 500 channels of digital video and CD-quality audio programming.

Before embarking on the SAN project, EchoStar's servers (mostly HP 9000 systems running HP-UX) were attached via SCSI to EMC Symmetrix arrays and dedicated tape libraries (mostly StorageTek 9840 and DLT libraries). The servers were connected over a 100Mbps Ethernet LAN.

According to Rick Nelson, senior IT architect at EchoStar, the main problems with the original configuration were inefficient tape resource utilization, single points of failure, limited scalability, and difficult and time-consuming management issues.

Nelson says the main benefits he hopes to gain from the SAN are increased flexibility in managing and using I/O ports, better backup operations, and improved disaster recovery and data movement.

"We're an EMC shop, and ports are expensive," says Nelson. "The SAN gives us the flexibility to maximize port usage." Dedicating servers to Symmetrix ports caused hassles in managing I/O and required recabling when new servers were added. Those problems were compounded by the fact that EchoStar's storage-capacity requirements are growing at about 480% per year.

EchoStar is moving from direct-attached storage configurations (top) to a Fibre Channel SAN fabric linked over an IP network to a remote disaster-recovery site.
Click here to enlarge image

"Previously, it was difficult to access the various pockets of storage," Nelson explains. "Now, all the EMC arrays are attached to switches, so I don't have to change cabling. We've reduced [the time it takes to solve problems] from weeks or months into a day or less." He also says that, with the SAN, it's much easier to add servers.

Another goal was to reduce backup times and eliminate the hit on production servers and networks caused by backup operations. EchoStar is using EMC's TimeFinder to offload backup from the Hewlett-Packard servers and Ethernet LAN. Backup now takes place from the Symmetrix arrays to Brocade switches to dedicated backup servers-a process somewhat misleadingly referred to as "server-less backup." According to internal tests, EchoStar expects a 500% improvement in backup throughput with the SAN in place.

Closely related to EchoStar's SAN-based backup goals was an improvement in disaster recovery via remote mirroring and vaulting. Symmetrix arrays at the company's headquarters in Littleton, CO, are linked over an Internet Protocol-based WAN to Symmetrix arrays at a disaster- recovery site in Cheyenne, WY. The links between the two Fibre Channel SANs over the IP network are accomplished with CNT's UltraNet Open Systems Directors.

Nelson says the SAN configuration has also improved data movement and that the company may move from tape-based backup to disk-to-disk backup and refreshing using EMC's TimeFinder over the Fibre Channel fabric.

EchoStar is pursuing a four-phase approach to its SAN:

Phase 1: Local tape backup SAN, providing "zero-impact" backup operations.
Phase 2: Remote disk mirroring over IP.
Phase 3: Remote tape vaulting over IP. (Phases 2 and 3 will be based on CNT's UltraNet Open Systems Directors, which convert Fibre Channel data frames to IP packets.)
Phase 4: Local Fibre Channel SAN fabric with server-less backup.

CNT involvement

EMC recommended CNT as a consultant and potential integrator in the EchoStar SAN project. CNT initially provided services such as assessment, strategic planning, and design analysis. About a year ago, CNT's Professional Services organization got involved with proof-of-concept testing and systems integration, working in conjunction with the EchoStar IT staff. In addition, EchoStar buys most of its SAN-related hardware (such as the Brocade switches) and software (mostly Veritas) from CNT.

Like most SAN vendors, CNT is expanding into the services part of the storage network market. The company's SAN services include assessment (e.g., auditing and analysis), integration, data migration, electronic tape vaulting, training, and support.

Despite having third-party help in designing and implementing its SAN, EchoStar did run into problems, most of which centered on the HP server host bus adapter (HBA) cards and lack of interoperability with non-HP equipment. "We had to wait for replacement cards, which helped but introduced more problems with patches, etc.," says Nelson, adding that most of the problems were taken care of by CNT working with HP.

To manage the SAN, EchoStar uses a combination of HP OpenView and Brocade's management software and is also looking at SAN management software from Veritas.

Echoing the comments of many early adopters, Nelson says that "SANs aren't for the faint of heart. If you don't have a lot of technical expertise, you'll need some outside help."

Despite the implementation hassles-and costs-associated with building the SAN, Nelson expects the effort to be worth it because of all the benefits. "We see savings in manpower, the amount of hours spent moving and shuffling data, and the amount of productive time recovered," he says. "It's not a matter of ROI [return on investment] in terms of fixed assets; it's ROI in terms of time and manpower savings, as well as time to production."

This article was originally published on December 01, 2000