Strategic governance issues for e-mail archiving

This article identifies some of the higher-level issues in governance and control that can impact the parameters and attributes used in selecting the appropriate e-mail archiving technology.

By Dick Benton

—E-mail archiving is a relatively new area and as yet "best practice" appears to be simply a general recognition that e-mail should in fact be archived, and that formal policies on retention, retrieval, security, and immutability should be in place.

Although some legislation is very specific on retention, much of the legislation is more descriptive than prescriptive. There are currently few precedents outside the financial industry on which to base legal opinions. Many industries and organizations are reduced to the old standby of the "prudent-person test," a somewhat subjective test of what a responsible manager would do in a particular circumstance.

There is an emerging view that an absolute minimum standard of compliance requires a written policy and procedure along with evidence that (a) the procedure is followed, and (b) the policy is under regular review. Whatever the standard, there is increasing awareness and agreement that e-mails are, in fact, regulated documents. Increasingly, e-mails are being treated as an organizational record and require the same records management protection given to traditional records, typically under DOD Records Management standard 5015.2

What all this means is that the prudent IT organization will develop overarching strategic policies directing the protection and retention of e-mail, preferably under central management. This provides a formalized capture, retention, and retrieval policy along with formalized requirements for encryption, immutability, and purge criteria. Business units and IT will implement their own tactical policies in support of these overarching organizational guidelines. This may mean an end to a user's capability to archive e-mail outside of the corporate containers.

Today, there's an increase in the number of employees using e-mail as a knowledge management tool and one that is often considered essential to daily productivity. Indeed, many users might claim that their productivity gains from access to historical e-mail far outweighs any perceived savings that might occur through a formal purge policy of older e-mail. This philosophy can lead to a requirement for a retention policy of "forever." We believe that key performance indicators are an essential component of e-mail governance. Without such metrics there is no way to apply a reasonability and frequency test to what are often highly personalized business needs.

The discovery trap
A key risk of not having a formal e-mail archiving policy is the resulting expense that will be incurred under a court-ordered discovery. Such discoveries are invariably at the cost of the organization on which the order has been served. Should significant investments be required to retrieve data and render the information usable, then this investment is expected by the court. Such impositions on an unprepared organization can result in so much cost and risk that the organization may be forced into premature settlement.

Worse, if an organization has an undisciplined (or even disciplined) approach to e-mail deletion, the courts may assume any "missing" e-mail would be considered damaging to the organization's position. This indicates that purged or deleted e-mail may at worst be damaging to the organization's legal position, and even at best, may leave the organization's e-mail out of context and thus open to interpretation. As such, there may be more downside than upside to e-mail deletion.

This is exacerbated by the fact that e-mail cannot be deleted without extraordinary effort. E-mail exists on desktops, laptops, file servers, backup tapes, recycled backup tapes, disaster recovery media, etc. E-mail hides in every corner of the organization and is extremely difficult to eradicate. Those who breach retention policies by keeping e-mail on their desktop longer than corporate policy dictates may expose the organization to the possibility that this user unique additional retention may be considered a de facto policy.

E-mail archiving solutions should be transparent to end users. Users should feel confident that their e-mail is protected and available when they need it. Unless this confidence level can be established, "underground" archiving will likely take place along with consequent difficulties when the organization is subjected to discovery requirements.

Data growth is a general problem in the storage industry, with projections of 40% to 60% per year from various research firms. As more and more organizations achieve success in filtering spam in a process that front-ends the organization's e-mail, it seems that e-mail growth is somewhat stabilized but at a per-user level. Evidence seems to indicate that e-mail that makes its way through corporate spam filters and then through junk mail filters remains reasonably constant in arrival rate. Growth in the number of employees or corporate events creating spikes appear to be the major factors in e-mail growth. A carefully thought out growth model should be a key component of any e-mail archiving plan—and in particular, one that can model best-case and worst-case scenarios.

Keep it forever?
It is possible that growth rates in non-spam e-mail are unlikely to exceed 20% per year in a stable organization. This growth rate approximates the projections on cost-per-gigabyte reduction in storage. It may, in fact, be possible to retain e-mail "forever" at no more than the current storage technology investment. However, it may be more practical to look at a forecast horizon of perhaps only three to five years. After that, emerging technology trends may move in directions that could substantially impact current e-mail volumes. These trends might include a transition to messaging as well as the increasing use of interactive Websites replacing traditional e-mail transaction chains in business-to-business communications.

Because of the pressures brought by the need for compliance and the day-to-day IT operational drivers, the journey to e-mail archiving often starts without a clear understanding of the associated costs. Implementing policies or directives that originate in legal, risk, or corporate management at the 11th hour can have a disastrous effect on IT budgets and on IT storage and may impact business units as their applications and initiatives have sudden and unexpected constraints imposed by massive storage requirements for the new e-mail archives. Therefore, it is essential to construct a cost model that determines and projects the cost per gigabyte or cost per e-mail for each of the designated retention policies, as well as understanding cost of e-mail archive migrations as aged e-mail is moved to lower tiers.

Inevitably, e-mail archiving will require significant behavior changes in the user community, IT community, and senior management community as the impact and implications of archiving are realized. Rather than position IT as compliance police issuing edicts to the business units on what may and may not be done (e.g., archival, retrieval, etc.), we recommend a self-correcting model based on the service provider principle. Under this approach, IT provides the services required by the business units; however, such services are linked to a cost of service. Thus the business units and senior management can make fact-based decisions on appropriate e-mail archiving services. High-cost services will generally force a more intense examination by management. Use of special services instead of standard services can also be contained in this way. But even where the business unit elects to use a high-cost service, the IT group is capable of providing this level of service, and the cost of that service becomes an integral component of the business unit budgeting process.

Dick Benton is a principal consultant with GlassHouse Technologies (www.glasshouse.com).

For more information on e-mail archiving, view the Webcast, Policy Driven Reference Architecture for E-mail Archiving, at www.infostor.com (click on the Webcasts button). Presented by GlassHouse Technologies, the Webcast reviews a policy-driven approach to an e-mail archiving architecture. Using a sample reference architecture, GlassHouse consultant Dick Benton walks through the generic archive process flow and discusses key issues in e-mail archiving, including what to do in the absence of legal guidance.

This article was originally published on September 11, 2006