It is often said that IT administrators, and storage administrators in particular, are highly risk-averse, which is true. Look at any of the surveys that query storage managers about their most pressing issues. These surveys point to data protection, system reliability and availability, and responsiveness to regulatory agencies and corporate governance as top priorities. Hot products these days are disk-to-disk-to-tape (D2D2T) and e-mail archiving solutions. Managing risk is a prime mover in the storage industry.
I recently explored the well-established practice of fiduciary risk management to see if that endeavor had anything to offer the practice of storage management. Practitioners of fiduciary risk management seek to quantify risk, as will future practitioners of operational risk. IT is included in operational risk.
There are two ways to look at risk. One is hindsight: You concentrate on protecting what you already have. The other is foresight: You are constantly on the lookout for new opportunities because having little or no foresight is as risky as having no hindsight. Without the foresight to see coming marketplace changes and new opportunities, you’re at risk of being run out by a competitor that does have that foresight.
The sad state of storage management-and IT management in general-is that hindsight often takes priority over foresight. These days, improved backup and e-mail archiving projects often take priority over speeding new applications to users’ desktops.
Data is both a blessing and a curse. You have to have it to stay in business, but then the government tells you to save it for years on end.
This summer, my family went to a summer resort outside the US. All of us had one suitcase and one carry-on as we boarded the plane. We watched as other families struggled to get multiple SUV-sized bags across international borders. To draw an analogy, data is essential to business survival and to forward motion. You gotta have it, but it can also be luggage with a capital LUG.
Here’s a modest proposal: Take steps to radically reduce the amount of data you have to lug around. The fewer bits you schlep over system boundaries, the fewer bits you have at risk. Shaving off 10% isn’t going to get you there. Think big-50% or more. Recently I’ve seen some new ways to accomplish this.
Data Coalescence
Rocksoft is a small Australian company that has held the basic patents on data coalescence for more than 10 years. The company’s “Blocklets” data-steamlining technology detects and then eliminates all duplicate data elements from a data stream at the sub-block level (“blocklets”). The process stores only the unique data elements plus metadata for use when the data is retrieved. Further streamlining takes place via compression, leading to radical reductions in the capacity required to store documents. For example, Word documents can be reduced to 5% of their original size, according to Rocksoft.
The company is in the process of developing partnerships with major vendors that will bring the technology to market early next year embedded in other solutions. (For more information, see www.rocksoft.com.)
Parse Data Streams
Traditional data-mining and data-mart applications grind through masses of data already sitting in storage systems and extract specific bits of information from that pre-existing data vault. StreamBase Systems’ StreamSQL starts with nothing more than a data stream and parses that stream for information in real-time-no terabytes of pre-existing storage or massive amounts of hardware required. The process can run on a PC. And the technology is available now. (For more information, see www.streambase.com.)
Shred It
It’s become fashionable in storage circles to talk about automated, policy-based management. However, corporate policies are the most difficult to translate to storage environments because corporate policies are outside the control of the IT department, whereas the other policies remain under IT control.
Corporate policymakers now include attorneys who want e-mails saved in case of litigation, and regulatory compliance managers who want many years of audit trails and transaction logs saved for possible governmental review. Storage administrators should not and cannot make corporate policy, yet they must somehow make things happen. And in the absence of established corporate policy with regard to shredding data, the default position for storage administrators is to save everything.
When corporate policy is clearly understood, the job of translating policy to storage management practices gets easier. However, when corporate policy is unclear-or nonexistent-IT must somehow force clarity from corporate executives. A “no-shredding” policy results in huge amounts of excess data baggage.
We all talk about burgeoning data volumes and data growth rates in excess of 70% per year. The standard response is a call for more-efficient storage management and solutions that enable administrators to do more with less. Rarely do we ask whether we really have to save all the data.
There is little financial incentive for storage vendors to ask this question. And it’s become difficult for storage administrators to pose this question to senior corporate executives who fear the worst from litigious lawyers and government regulators. It’s easier just to save everything
Yet “do more with less” is not the only answer. There are ways to move the capacity growth meter away from the red zone: Travel light.