InfoStor : SNIA survey spotlights archiving crisis, August 2007 Page 1

By Kevin Komiega

August 23, 2007—The Storage Networking Industry Association (SNIA) claims that the industry is on the verge of a crisis. According to a recent SNIA survey of archive practitioners, long-term data-retention technologies and practices are not up to snuff, and digital information is in serious jeopardy of being lost over time.

Long-term data retention needs are undeniable. The survey, dubbed the SNIA 100 Year Archive Requirements Survey Report, revealed that 80% of respondents have information they must keep for more than 50 years, and 68% of respondents say they must keep some types of data for more than 100 years.

Overall, those surveyed felt that current practices are too manual, prone to error, and costly, and lack adequate coordination across the organization. In fact, 70% of the respondents said they are "highly dissatisfied" with their ability to read their retained information in 50 years.

SNIA surveyed IT professionals from the records information management (RIM), archiving, IT, security, legal, and business communities of 276 organizations in an effort to define the current challenges and operational needs related to long-term digital information retention.

"SNIA started looking into the state of archiving last year as part of our work on information lifecycle management. We began to examine the major technological barriers to long-term data preservation, and that's what drove us to conduct this survey," says Jeff Porter, chair of the SNIA Data Management Forum. "The results confirmed our view that there is a pending crisis in archiving."

Porter says business requirements for long-term retention are still murky, and he hopes the conclusions of the survey will serve as a jumping off point for defining archiving needs.

"The state of archiving is very poor today. The industry has done a lot more to address physical data migration issues rather than the logical piece of the puzzle," says Porter. "If we're going to be able to avoid this crisis we have to create long-term methods for not only preserving information, but also for making it available for analysis in the future."

According to SNIA, long-term data retention is currently defined as 10 to 15 years and is limited by technology refreshes, including logical and physical data migrations. Only 30% of those surveyed say they migrate information in regular intervals.

Porter says the more you move data, the less confidence you have that it will be retrievable in the future. "After the second or third migration you begin to lose confidence in the integrity of the data. You can retrieve the bits and bytes, but you can't necessarily interpret data from older applications," he says.

The survey captured the operating practices, requirements, and issues facing organizations managing large amounts of information and aims to provide a better understanding of market requirements so that the SNIA can frame definitions for best practices and technology solutions.

In response, the SNIA formed The 100 Year Archive Task Force and already has several initiatives under way aimed at addressing the archiving problem.

First, the group plans to produce a reference model for best practices for long-term digital information retention that covers requirements not addressed by existing archive standards and best practices such as ISO 14721:2002, Open Archival Information Systems (OAIS), or the Sedona Conference.

Second, the SNIA claims it will integrate ILM-based practices into the long-term digital information retention process in an effort to automate IT infrastructure in support of business and information requirements.

And finally, the Task Force is attempting to define reference models and possible technical standards that provide for scalable physical and logical migration.

John Webster, principal IT advisor at Illuminata, says, "We rarely think of saving our digitized thoughts for the sake of posterity. But for the sake of historians, law makers, sociologists, and scientists yet to be born, we should. Otherwise, future historians may look back at this time in history as the digital dark ages."

The SNIA's 100 Year Archive Task Force is operated by the SNIA's Data Management Forum as a global, multi-agency group working to define best practices and storage standards for long-term digital information retention.

Page 1 of 1