Disk System Eliminates Network Delays in Entertainment Industry
Staging files to a high-capacity storage server eases network bottlenecks.
By Wyici Fables and Jore Park
SeaSeer, Kuala, Hawaii
Converting 3D digital images into film, video, or CD-ROM format is a computationally-intensive process, in some cases requiring several hundred gigaflops (or one billion floating points per second) of total computational power. With data demands like this, network bottlenecks are common.
A disk storage system--based on a MAXSTRAT Gen5XL array and an HSM architecture--at one of the 20 fastest computer centers in the world has eliminated network delays for one entertainment industry application. SeaSeer, a data- rendering firm for the entertainment industry, in Kula, Hawaii, uses the computing power of the Maui High Performance Computer Center (MHPCC) to perform its services.
One of the Department of Defense`s largest computing resources and one of the first research organizations to acquire an IBM RS/6000 Scalable SP system, the MHPCC is a major testbed for scalable parallel technologies and touts the world`s third-largest installation of RS/6000 servers. MHPCC currently offers a total of 539 RS/6000 SP nodes, with 64MB to 1,012MB RAM per node in a shared-nothing configuration with ATM and HIPPI networks.
A key focus at MHPCC has been the development of advanced image processing and visualization technologies, which explains why in 1994 SeaSeer chose this site to pioneer its remote rendering services. SeaSeer`s rendering services bring leading-edge image processing hardware and software to digital studios of all sizes. Large studios, in particular, use SeaSeer`s services extensively for very high peak-processing loads. SeaSeer uses a variety of rendering programs, including Mental Ray 2.0 parallel version, RenderMan, Alias Power Trace/Caster, and Wavefront IPR and TAV20.
Studios can deliver data to SeaSeer in several ways. The usual approach is sending control files over the Internet using encrypted Telnet and FTP sessions. Files are sent and accessed via a T1 connection. Using multiple connections, files can be sent at speeds of up to 45MBps. Studios can also spool their control files on magnetic tape and then send them by courier. Or, SeaSeer can bring Alias, Wavefront, and Softimage animation software up on demand to spawn control files.
The animation process begins when the modeler, using the storyboard as a guide, builds wireframe geometry of the basic story elements. Texture and lights are then applied to the wireframe geometry and integrated into the overall scene. The rendering process then calculates the path of light onto the various objects in the scene. This is a computationally-intensive process. A single movie frame typically consists of 40MB of data; each bit has to be recalculated based on the texture map and lighting for the scene.
The RS/6000 SP system dramatically speeds up the rendering process. In most digital studios, a 40MB film frame takes about and hour to render; a 1MB to 10MB video frame, about 5 to 15 minutes. Since there are 24 frames per second in film and 30 frames per second in video, 10 minutes of film can take up to 24 hours to render. With parallel processing technology, SeaSeer can render the same 10 minutes of film in as little as 90 minutes if every node is applied to the task.
The increase in processing power, however, presents data storage and transfer challenges. As the rendering software processes each frame, it has to call up all of the texture maps in each frame. Typically, each frame has 20 texture maps, each consisting of 4MB TIFF or RGB files. The data management task is further complicated when moving textured map are used in animation sequences--a very common trend in today`s animated films. If moving texture maps are used, the processor requires a completely new texture map for each frame.
If 100 nodes are rendering frames in parallel, 80GB of data is required during each rendering cycle. The texture maps are stored either in MHPCC`s large tape-storage system or in a variety of local disk drives. Each texture map must reach the processor doing the rendering in time to render the frame. If they don`t, the frame is rendered with a black space in place of the missing map, and the rendering job must be repeated.
When SeaSeer first brought hierarchical storage on-line, its storage system could not keep up with the data demand, particularly that of large rendering jobs such as moving texturized maps. As a result, the company frequently had to re-run job at reduced rates to avoid stressing the data storage system. In effort to solve the problem, SeaSeer began staging the files to a 188GB storage server (in this case, the MaxStrat Gen5XL) at the MHPCC.
SeaSeer engineers created a "smart memory path" from MHPCC`s hierarchical storage management system. The path immediately searches and calls up texture maps from the Gen5XL. SeaSeer is experiencing data transfer rates of 41MBps for read operations and 27MBps for write operations, over HIPPI and Fibre Channel connections. The MAXSTRAT Gen5XL system is designed to transfer data at rates as high as 250MBps. Since the Gen5XL storage server is used to simultaneously transfer data over two HIPPI connections, the total sustained transfer rate to and from the SP2 is double that. Since the new staging method was implemented, no rendering job has failed due to missing texture maps.
The Gen5XL frees up the SP2 by off-loading a large portion of the data transfer management overhead. The storage server dramatically enhances performance by allowing the SP2 to sift through its massive databases at realtime speeds. Other features include:
- Battery-backed "fast write" and "read-ahead" caching, which improves short request performance and allows the host to spend less time waiting for command completion responses.
- 24 SCSI-2 Fast/Wide paths and two 200MBps independent data buses for up to 1.7TB of data in a single frame.
- 7 x 24 data availability when the server is configured with multiple I/O channels, dual processors, N+1 power, and UPS.
- Consistent and predictable performance, thanks to selectable RAID-0, -1, -3, and -5 modes.
- The ability to group and partition storage space for multiple concurrent RAID modes and for spreading accesses evenly over many disks.
This kitchen scene with a rich variety of texture maps mounted on the cupboard doors was used to set up a storage-server stress test. The data demand imposed by the continuous stream of complex images is handled by the MAXSTRAT Gen5XL storage server.
The MAXSTRAT Gen5XL storage server supports up to four independent HIPPI host connections, has a 250MBps aggregate data transfer rate, and provides 1.7TB of user data.