InfoStor : Storage's star rises in the entertainment market, April 2007 Page 1

By Thomas Coughlin

April 20, 2007—On the heels of this week's NAB show in Las Vegas, here's an overview of trends in the entertainment market from a storage perspective.

Data storage is a key element in the digital transformation of content creation, editing, distribution, and reception. Digitalization of content enables that content to be made readily available in the various formats required for traditional distribution markets, as well as for developing markets such as IPTV. Increases in storage capacity and performance, form factor changes, lower prices, and growing familiarity with digital editing and distribution are key components in the continued growth and development of entertainment. Because of the large file sizes required for high-resolution images, there is increasing demand for high-capacity storage devices. The entire value chain of content creation, editing, archiving, and distribution, as well as consumer electronics content reception devices, create the driving forces that fuel data storage requirements for all entertainment content applications.

Content acquisition and editing
Many changes are afoot in the acquisition and post-processing of digital content. Digital cameras are gaining ground over traditional film or tape systems, especially in the broadcast industry. For feature films, many producers still prefer to use film for acquisition, with subsequent scanning to digitize the content for editing and other postproduction processes. As the feature film industry moves to 4K capture and editing, and as digital editors learn to reproduce the look and feel of analog film, the next generation of producers will increasingly choose digital acquisition with as high a resolution as the available technology allows. This will drive the requirements for storage capacity, as well as the performance of digital storage devices. Up to 1,600TB may be required for a complete digital movie production at 4K resolution.

Although digital tape still dominates in professional digital camcorders, it is rapidly being displaced by new digital storage technologies such as blue-laser optical media, flash memory modules, and hard disk drives. These technologies allow random access to data in a file format, making field editing much easier.

Non-linear editing is generally done with uncompressed, or at most, slightly compressed, content since heavy compression increases the overhead of editing and can cause timing problems. In the past, managing this uncompressed content limited the growth of storage networking in these applications, but as the I/O performance of storage networks and systems improves there will be a significant move from dependence on direct-attached storage (DAS) to distributed networked storage for editing and other workflow processes.

As an example of current performance requirements during non-linear editing and other postproduction processes, the "file-open" latency of any online media source generally should not be greater than 300 milliseconds (msec), regardless of its storage location. The typical response time is 100msec. Internal ring buffering allotment in RAM is typically about 1 second per stream.

Postproduction processes will be the single biggest driver of online (or high-performance) networked storage demand in the entertainment industry. Likewise, they will also be the biggest driver of nearline (lower performance but higher-capacity and lower-cost) networked storage as the amount of content available to postproduction practitioners mushrooms.

Keeping your masterpieces
For many archiving and distribution applications where content is relatively static, low- cost and high-capacity Serial ATA (SATA) disk arrays, holographic optical disks, and tape-based libraries will dominate. New optical technologies, such as advanced blue- laser DVD technology—as well as holography—will be used for archiving as well as content distribution.

There will be a sharp increase in digital storage capacity used for digital conversion and preservation as demand for historical content grows, as costs for digital conversion are reduced by the availability of service providers that can do the conversion of bulk material at an attractive price, and as the overall cost of storing digital archives is reduced.

In fact, archiving and digital conversion and preservation will be the biggest driver of digital storage capacity growth in the entertainment industry. About 54% of total entertainment market storage capacity was used for content archiving and preservation in 2006. We believe that this will increase to 72% of total capacity by 2012.

Reaching out to the masses
Broadcast, video-on-demand (VOD), digital cinema, and other forms of digital content distribution use compressed formats to make the best use of limited bandwidth resources and to match the content to the display device. Distribution methods will require larger amounts of storage as the digital content resolution increases.

Although traditional distribution of content using broadcast, cable, satellite, and theatres still dominates, distribution using Internet technology such as IPTV will become increasingly important as time goes on. VOD streaming shows the biggest overall storage capacity needs. As IPTV and other new distribution technologies mature, they will require similar quantities of digital storage.

Video-on-demand
As digital distribution increases, cable, satellite and other content-delivery operators are expanding their offerings for video-on-demand. This and the increasing resolution drive digital storage demand. The Table below shows VOD capacity and bandwidth requirements for standard-definition (SD) and high-definition (HD) digital content. VOD requires that content be ingested as well as played or streamed out. Content is ingested at a rate required to refresh the content in the cache that serves the VOD delivery.

Eliminating distribution bottlenecks
Although there are several potential bottlenecks for digital content delivery, there are many ways to boost performance. Following are ways in which media content delivery systems can be improved:

Over-provisioning
—Continue to use the same approach to boost server performance; add more DAS servers to share the load. Individual servers have dedicated storage devices attached to them, providing direct data access to built-in storage. As a result, content may need to be replicated and exchange of content may require file transfer or removable media delivery.
Speed Ethernet and Fibre Channel I/O connectivity within the existing data center
—Increase the speed of each connection and add TCP/IP offload engines (TOEs) to all network interface cards (NICs) in existing servers. This will solve the server network I/O problem without addressing the storage I/O problem.
Replace existing server-to-server and server-to-storage connections with new technology
—As new interconnect technologies such as InfiniBand emerge, consider a wholesale replacement of existing interconnect technologies.
Utilize emerging server platforms optimized for media delivery
—Deploy specialized "next-generation" media-delivery servers that integrate server blades using fast internal connections and unified architectures;
—Maintain existing Ethernet and Fibre Channel networks;
—Mix and match I/O options, including 1Gbps and 10Gbps Ethernet, 2Gbps and 4Gbps Fibre Channel, and 4x and 12x InfiniBand as server network I/O connectivity requirements grow; and
—Preserve investment in management software and staff training.

Digital cinema
Digital cinema offers considerable savings to content creators by significantly reducing distribution costs. Distribution of digital theatre content is through optical disks or hard disks drives. Distribution via satellite or Internet feeds may eventually replace actual physical media. Encryption of these storage devices is common to prevent piracy of high-resolution content.

Even with the projected savings for digital distribution there has been considerable resistance from theatre owners to install the expensive digital projection systems (costs today are about $100,000 for a typical installation, including approximately $17,000 for the server with storage and $40,000 per projector), and theatre owners may have to replace them in a few years. (In contrast, film projectors can last for decades.)

However, equipment costs are steadily declining, DCI standards have been completed, and content distributors are providing partial subsidies. In addition, theatre owners are realizing that digital delivery gives them options to differentiate themselves from increasingly sophisticated home entertainment systems.

In particular, digital cinema makes digital 3D possible, which is less costly and more user-friendly than older 3D technology. We expect that digital cinema will dominate public theatres within a few years. Note that for most digital cinema installations, networked storage dominates over DAS, and most of this will be online storage since theatres will generally not keep a lot of older non-viewed content on-site. A 20X increase in storage capacity requirements for digital cinema are expected from 2006 through 2012 due to digital theatre conversion and increasing digital theatre resolution requirements.

Use of digital storage
From 2005 to 2012 we expect a 10x increase in the required total digital storage capacity and more than an 8x growth in storage capacity shipments per year (from 259,406TB to 2,287,547TB).

About 54% of the total storage capacity in the entertainment industry was used for content archiving and preservation in 2006. This is expected to increase to 72% by 2012. And more than 6 exabytes of digital storage will be used for digital archiving and content conversion and preservation by 2012!

Changes in storage media requirements
Density requirements for storage systems used for content creation, editing, archiving, and distribution drives the use of storage devices in smaller form factors in the entertainment market. It may also open up additional requirements for the high-end of the storage hierarchy. This is reflected in the projections for the growth of various types of digital storage media. In particular, digital tape for content acquisition will give way to flash memory, optical disks, and hard disk drives. Optical storage and hard disk drive arrays will also make some inroads in tape market share for archiving and digital preservation.

The 2006 storage media breakdown for all the digital entertainment content segments was 72% tape, 15% optical disk, 10% hard disk drives, and 3.4% flash memory. By 2012, this should change to 28% tape, 30% optical, 35% hard disk drives, and 7.4% flash memory.

For more information
This article is based on the 2007 Entertainment Content Creation and Digital Storage Report, the fourth annual report on data storage and the entertainment market. The report analyzes requirements and trends in worldwide data storage for content acquisition, editing, archiving, and digital preservation, as well as digital cinema, broadcast, satellite, cable, network, and VOD distribution. For more information, go to the tech papers section of www.tomcoughlin.com.

Tom Coughlin is president of Coughlin Associates (www.tomcoughlin.com). He is also the founder and organizer of the annual Storage Visions Conference. Coughlin Associates provides market and technology analysis, as well as data storage technical consulting services.

Page 1 of 1