The year ahead from 30,000 feet
2004 promises to be another active and transformative year in the information storage industry. We expect more mergers and acquisitons, exciting new products and partnerships, and hopefully economic growth for all.
Ultimately the winners are the IT and business professionals who depend on storage systems and increasingly the information they contain, to push their businesses forward, extract incremental revenue, and competitive advantage.
This brief will explore several major user requirements which ESG expects to be important during 2004 as they relate to disk-based storage systems. We present just a few of the high level industry trends and discuss how storage technology vendors are responding to these user requirements. For 2004 to be a year of recovery, growth, and innovation, these vendors must deliver products and solutions that are both cost-effective and highly refined.
Though not meant to be a formal or complete taxonomy of this technology segment this brief will focus on: :
Disk Storage Systems - technologies:
2004 market drivers
Tiering of storage systems and data sets
The "Information Economy" is rolling ahead at full steam as the economic stagnantion of the past 2 years finally begins to loosen, We are seeing indications across every major industry that spending on storage systems, software, and integrated data protection solutions will increase throughout 2004. The rapid pace of information creation continues, perhaps faster than any user had anticipated or any analyst had forecasted.
2003 was the year that "tiered storage infrastructure" leapt off of vendor's powerpoint slides and took hold within the minds and data centers of end users. It is becoming increasingly common to characterize storage systems as residing within one "tier" of a storage infrastructure, so it is appropriate to also consider this methodology when defining coverage:
Note: Costs reflect "MSRP" or list prices for usable capacity.
Many enterprises continue to measure exponentional growth in their primary data stores even while they struggle to reign in operational costs and inject some "intelligence" and consistent policy into their storage management procedures. Many users we have spoken to over the past year focused on improving the utilization of their existing storage system assets, that is, burning through the excess capacity they purchased (or were sold), during the twilight of the Internet bubble.
These same users head into 2004 challenged to first identify and then to better understand the different classes of information within their organizations. We are finding widespead acceptance of and increasing deployment of a tiered storage systems model. ESG expects the trend toward a more definitive tiered systems model to accelerate in 2004 fueled by robust adoption of lower-cost ATA-based systems available in the general maketplace.
Tiering of storage systems and information assets is now resonating with IT and business professionals who think beyond the "capacity under management" metric of how much storage they have within their data centers or remote offices. Users are now considering how to best match the value of their information assets to the appropriate storage system in terms of availablitly, cost, performance, and other criteria.
EMC/Dell, Hitachi Data Systems, HP, IBM, Network Appliance,
LSI Logic, Sun, Xiotech
Ones to Watch:
Compellant, Exagrid, Pillar Data, Storage Matrix,
Year of the Data Protection rookies
One of the most exciting conclusions we are able to draw from the recently completed ESG Research report: "The Evolution of Enterprise Data Protection" is that users are very interested in, and willing to spend budget on, new alternatives to the traditional disk-to-tape data protection paradigm. As economic conditions improve and IT spending ticks upward, 2004 is a critical year for startup vendors with data protection solutions that are now generally available or just coming to market.
Backup and recovery continue to be among the biggest challenges facing IT managers. That said, as IT organizations are required to backup ever increasing amounts of data, they will turn to new disk-based data protection technoliges to help alleviate some of this burden.
Both Enterprise and Mid-tier users are increasingly willing to deploy disk-based solutions such as virtual tape, disk-based backup systems and continuous or incremental capture devices to complement, not replace, exisitng investments in tape.
While daily incremental and weekly full backups to tape routines continue to be the norm, the combined threat of business interruption from failed backup or recovery attempts, unrelenting information growth in primary data stores, and new challenges such as addressing regulatory requirements are squeezing users further and further out of their data protection comfort zones. We expect users to vote with their IT budgets this year and reward some incumbent, and many emerging vendors with purchase orders for new solutions across all segments. We expect robust adoption of next-generation systems at sub $10/GB price points. IT & business professionals recognize the value and importance of protecting their information assets and are willing to invest in solutions from new vendors that hold the promise of increased resiliency.
ADIC, Network Appliance, Quantum,
Avamar, Data Domain, Overland Storage, Revivio, Sepaton,
Ones to Watch:
Object-based storage makes a big splash
Increased consideration is being given to the applications that run the business and create the terabytes of valuable information that keep backup administrators running in circles. Users want to search for and retireve information such as archived files or emails as business objects in their original form and not in terms of rows, columns, LUNs, or volumes. Object-based storage (OBS) systems make ths possible and the flexibility and resiliency of Linux clusters coupled with the availability of ATA disks and commodity hardware will make these systems affordable.
ESG Research has done extensive work in this area, having published the "Reference Information: The Next Wave" report in 2002 and "Compliance: The effect on information management and the storage industry" in 2003. In these reports, we discussed the emerging requirement to address the enormous growth of reference information, also known as fixed content or unstructured data within enterprise and mid-tier organizations. We have also explored the risks associated with being unable to confidently address regulatory compliance and other mandates that effect how business records are managed. 2004 will continue the shift in purchasing by users away from monolithic block- and file-based solutions to purpose-built, modular systems that enable data to be stored as objects or unique pieces of content. Reference information such as email archives and attachments, audio and video files, digitial medical images and photos consume orders of magnitude more storage capacity than structured or flat file data found within OLTP systems or relational databases.
These valuable information assets also have unique properties and requirements and often need to be retained for long periods of time, and retrieved quickly for regulatory or other purposes. IT professionals have had mainly two storage options: disk arrays and tape systems. While block (SAN) and file (NAS) disk systems built around SCSI and Fibre Channel interconnects offer high performance, scalable capacity, and robust data protection via proven RAID technology, these systems fall short in addressing users needs for their reference information. Tape systems continue to be a critical component of user's environments offering low-priced bulk capacity, and reliable long-term archival properties, but due to their serial nature, suffer from slow read and write performance. The pace of business is quickly eclipsing traditional magnetic tape's ability to completely meet the demands of the "always on" economy.
Object-based storage systems promise both the performance of traditional block-based SAN-attached disk and the simplicity and ease of management of file-based network-attached disk or NAS. Additionally, because OBS systems store data as objects identifiable with meta-data, and not files or blocks, the system does not require knowledge of the physical location of the objects in order to retrieve them. This alleviates the block, file, LUN, volume indentification model, which is the source of much frustration among IT managers struggling to protect larger and larger amounts of information. Many OBS systems currently available or in development are built around a node architecture, sometimes referred to as RAIN or random array of independent nodes. The truth is, these nodes each contain processing and communication capabilities, storage capacity, and are aware of one another in the system. Remember the "Borg" from Star Trek -- The Next Generation?
OBS systems also may present an API that enable integration of 3rd party ISV applications to leverage these storage systems as an intelligent repository for content. Other features of these systems may include novel data protection schemes, indexing and search capabilities, in addition to content addressing and commonality factoring that provide each object with a unique identifier, thus eliminating the retention of duplicate objects. Think about this in terms of an email and powerpoint attachment sent to 10 colleagues; the OBS system would have the ability to index the content of each email, but would only store a single version of the powerpoint presentation. SNIA, the Storage Networking Industry Association, has established an Object-based Storage Device working group to develop and evangelize OBS systems and standards, so expect 2004 to be a year in which OBS systems continue to make headlines -- change is afoot!
OBS has the power to transform how users think about and manage their reference information and valuable content and also to change the way technology vendors build storage systems. Storage vendors and the users they serve are becoming more aware of the role that reference information plays in both driving demand for additional storage capacity and in delivering unique value to businesses. The blurring of the lines between data storage and content management will continue throughout 2004 as new OBS systems come to market,. ESG will continue our work in tracking and reporting on what is sure to be one of the most significant developments in the entire storage industry over the next 5 years.
Avamar, HP (Persist), Panassas, Permabit,
Ones to Watch:
Abrevity, Isilon, Reference Information Systems, StorageTek,
Author: Peter A. Gerr