Solving the RAID compatibility puzzle

SNIA’s Common RAID Disk Data Format (DDF) promises to alleviate compatibility issues.

By Matthew Brisse and Bill Dawkins

End users and storage integrators consistently acknowledge the challenges presented by the lack of interoperability in RAID arrays. RAID has been a critical component of server and storage systems for many years, and vendors have developed many ways to implement them, often in a slightly different manner. For example, RAID 5 is typically implemented in a somewhat standardized fashion (e.g., as a disk set with distributed parity), but the data format that is written to disk is essentially proprietary with each vendor using a specific data layout.

This is a result of the way configuration information is stored on each disk within a RAID subsystem. This configuration information is in the form of a data structure and is sometimes called a Configuration On Disk (COD) or metadata for the disk. A COD typically includes, but is not limited to, assignment of physical disks to a RAID group, RAID levels of RAID groups, data mapping on RAID groups, hot spare assignments, etc.

Currently, each vendor implements its own proprietary COD format, and in some cases a single vendor will implement different CODs for the different RAID solution types within the same product portfolio. Because of the different COD structures, it is not possible to physically move the disks participating in a RAID group from one solution to another with the data in place.

Almost all end users purchase disk arrays from a number of different server and RAID vendors. Therefore, the typical data center has many different RAID solutions, from internal software-based RAID implementations that come free with an operating system to off-the-shelf RAID products.

Another factor contributing to the complexity is the difficulty end users face when they’re trying to remove serviceable RAID disks from retired or obsolete hardware and reuse them in a different vendor’s subsystem or newer configurations. These challenges make it difficult and expensive to scale smaller solutions to larger, more-flexible ones, which negatively impacts ROI.

In many cases, data can be lost or corrupted during the migration process. For example, if an administrator migrates disks from one RAID vendor to another, the format of the disks will most likely be unrecognized by the new controller and possibly perceived as blank. Disk signatures may be written to disk, destroying the data. Administrators may inadvertently create a new RAID array over the existing data, resulting in loss of data.

The lack of RAID interoperability is an industry-wide challenge for end users and vendors alike. To address this concern and bring flexibility, choice, and decreased complexity to end users, the Storage Networking Industry Association’s Common RAID Disk Data Format (DDF) Technical Working Group (TWG) has developed the Common RAID DDF specification.

A number of companies and organizations have collaborated on the development of the DDF specification, which standardizes the RAID configuration data structure and describes how data is formatted across disks in a RAID group. This specification will allow for a basic level of interoperability between different suppliers of RAID technologies. The DDF data structure describes how data is distributed across the disks in a RAID array. The figure shows the scope of the DDF data structure in the context of the SNIA Shared Storage Model. The DDF scope is limited to the interface between the block aggregation implementation and storage devices as defined in the SNIA Shared Storage Model. It doesn’t standardize the OS’s RAID controller interface or create a single driver.

The Disk Data Format (DDF) specification is limited to the interface between the block aggregation implementation and storage devices in SNIA?s Shared Storage Model.
Click here to enlarge image

The DDF structure is focused on server-­resident RAID solutions (e.g., PCI RAID cards, RAID on motherboard, or RAID-on-the-motherboard [ROMB], etc.). The DDF TWG is currently focused on DAS; the scope of the specification is not on high-end external RAID solutions.

Part of the DDF specification defines how data is distributed for many basic RAID levels. The specification uses mathematical formulas to document how data is formatted for RAID levels defined by the DDF structure to ensure there is no misinterpretation. The DDF data structure also allows RAID groups with vendor-unique RAID formats. When a RAID group containing a unique format is moved to a different RAID system that doesn’t support the format, the new system will still be able to read the DDF structure stored on each disk. The new system will be able to detect the non-compatible RAID formats. Potential data loss is prevented because the new RAID system will not overwrite the data without administrator confirmation. The DDF structure resides on every disk behind a RAID controller, providing redundancy and preventing random disks from being mixed into a RAID set from a different configuration. Collectively, the DDF structure on the disks defines how data is distributed in a RAID array.

The DDF structure describes features such as the controller data, physical disk data, virtual disk data, sparing assignments, bad block management, and vendor-specific logs. The DDF structure also describes virtual disk configurations such as RAID levels, participating disks, and stripe sizes, cache policies, etc.

The DDF standard will be particularly beneficial for ROMB server implementations, allowing administrators to move disks from one server to another, and in applications such as disk-based backup and disaster recovery, as well as for technology transitions. (For more, go to www.snia.org/tech_activities/ddftwg.)

Matthew Brisse is the SNIA board of directors vice chair, and Bill Dawkins is the chair of the SNIA’s Disk Data Forum Technical Working Group.

This article was originally published on January 01, 2005