Software that allows users to share file systems in storage area network environments may reduce total cost of ownership.
By Chris Stakutis
Storage area networks (SANs) have promised many improvements in managing and utilizing data. Certainly, co-locating and centralizing storage at the physical level helps with overall cost and management. And Logical Unit Number (LUN) masking can be used to partition storage into logical pieces, assigned to different servers. However, LUN masking is only a first step in realizing the full potential of
SANs. Additional benefits are possible by sharing file systems and data among simultaneously connected servers.
There are a number of products in the SAN market that offer the ability to share file systems. The main challenge the industry currently faces is not the technical implementation of shared file systems but, rather, educating the end-user community about these solutions and how they add value to SAN implementations.
Figure 1. Shared file systems combine the advantages of other sharing techniques.
Technology advances have allowed a continuous progression of sharing in a SAN environment (see Figure 1). Until recently, the diagram on the left in Fig. 1 represents how most enterprise managers looked at SANs. Now, with the realization of the true value of a shared SAN file system, the diagram on the right is more accurate. The fundamental difference is in realizing how and when to deploy a shared SAN file system, and for what benefit.
Shared file system SANs have been in existence for several years. Historically, users have concentrated on the top part of the shared-access pyramid; that is, the typical user has a pressing need for every computer to share the exact same data files at nearly the same time. Examples include movie and video production, digital graphics and pre-press, and other vertical markets such as medical and geo-imaging. However, there is a wider set of opportunities for shared SAN file systems in enterprise applications.
The primary new ground for shared SAN file systems is essentially any site that is currently using LUN masking. LUN masking is a SAN "disk sharing" technique (sharing the physical cabinet and resources) that allows users to derive value from aggregating their data into more central cabinets. However, this approach does not provide the value of actually sharing the data, and it tends to incur additional costs related to maintaining all the various file systems.
File systems are the basic unit of management in an enterprise environment. Backups are performed on a per-file-system basis, security is configured and monitored per file system, and overall quota tracking and general administration is per file system. The more file systems, typically the more total cost to run the environment.
Shared SAN file systems can be used as a substitution for LUN masking. In its most LUN masking-like configuration, each system can be configured as its own "metadata owner" of the file systems it deals with the most (even exclusively). Such a configuration allows the same performance of a pure LUN-masking approach, yet opens the door for sharing those file systems. At the other extreme, users can reduce the total number of file systems down to one (or a few) and have the remaining servers "share" those file systems. Such an approach leads to lower total cost of ownership (TCO) through simplified and more centralized management.
Scaling Terminal Server
In the Windows NT space, "Terminal Server" deployments are growing rapidly. This is largely due to the trend towards re-centralizing computing and data resources. Instead of having to update 1,000 field and mobile users with new versions of client-side software, a Terminal Server allows a single (or set) of servers to host the applications and remotely display to the desktops. Wide area network (WAN) and dial-up bandwidth is sufficient to make this not only practical, but in some cases indispensable.
With re-centralization comes the inherent need for larger and/or more scalable servers. Windows NT is especially ripe for scaling via multiple systems, due to the generally smaller server size and lower cost. Mixed with front-end load balancing software, multiple independent systems offer IT managers higher uptime.
SANs are a natural fit in such situations, particularly where several systems need to serve data from the same data pool. With a shared SAN file system, each server can be configured to display the exact same view of the total storage. Thus, any incoming connection can be assigned to any of the systems arbitrarily. This is not possible with a LUN masking approach. The closest similar approach would be to have the servers share a common back-end file system, as in network-attached storage (NAS) configurations. This brings with it the risks associated with a single box, plus the reliance on all-LAN traffic for the data, in contrast to the direct-connect transfers offered by a SAN.
Today, it is still typical for a database server to be directly attached to the disks that hold the database (usually physically embedded in the server box). With external storage now running faster than internal storage (Fibre Channel arrays versus internal SCSI disks, for example) there is a compelling reason to use external storage for databases. Furthermore, mission-critical databases need the protection offered by a RAID array. Although a single database server might not need a half terabyte of capacity, a storage subsystem for other servers' data can lend further economy.
One approach is LUN masking. The downside is that the size of the partition pieces is not easily adjusted, and typically involves a complete data off-load/re-load operation. NAS vendors have been quick to point out that attaching the data store to a NAS box offers many benefits. Examples include more flexible allocations (network-mounted file systems have no bounded size), centrally administered data (both physically and by file system), and high-availability RAID storage.
The drawback of a NAS file server for database storage is the throughput and overhead of a LAN. LANs have fundamental bandwidth limitations, and may steal more than 50% of the CPU cycles on both ends for processing fast streams. Database applications will miss the performance and responsiveness of the direct-attached storage model.
Shared SAN file systems may provide a compelling alternative. They offer all of the benefits of NAS from the file system perspective, including network-mounted file system abstraction. Yet, shared file systems also allow users to choose their own storage and provide for direct access to that storage.
Data mining and data warehousing were once used only by the largest retailers in the world. It was clear to them that any possible inference they could make from their data regarding customers' buying patterns would help them immensely with distribution and marketing, and would create a more efficient operation.
Now, data mining is something that most companies are already doing or will be doing soon. It is no longer relegated to multi-terabyte purchase transaction data. Rather, the most common data to be analyzed, even in small companies, are Web site hits.
The Web server tends to be extremely busy doing its job, serving pages and running SQL queries. It is barely able to keep up with the continuous log notations of every hit. Yet the data is too valuable to skip this step. Typically, data files are processed daily and an attempt is made to process them on systems not involved in the Web serving. However, the only choice thus far has been to introduce another LAN stream to pull this data off to another location. Certainly, serving such a large file with its associated 50% impact on the CPU is not an ideal solution.
Shared SAN file systems can help with this challenge by allowing any other server in the SAN to reach over and start processing the data, even as it is accumulating. The "owning" host is not involved at all, and can continue to perform its primary functions.
Similar examples include sifting large medical databases and files and cross-correlating customers with product inventory. SANs allow many systems, even dissimilar in architecture, to rapidly gain access to the company's largest files. The processes of analyzing and serving these files are now separated, providing the best possible performance.
SAN-based backup has long been discussed as a simple way to improve overall performance by reducing LAN traffic and decreasing backup windows. Although it is not necessary to have a shared SAN file system in order to achieve value from a SAN-based backup system, there are additional capabilities that come with sharing file systems. In particular, the ability to have a "hot staging" area that operates at true SAN speeds is possible.
Historically, most backup solutions result in the data moving directly to some sort of external tape-based storage. When disks were more expensive, this approach was cost effective. However, it results in extremely slow recovery time and the lost productivity quickly erases any cost savings. Furthermore, tape devices are much slower than disk devices, which means that the general backup process takes far longer than expected. Previously, the majority of data was first transferred over a LAN, which can be even slower than the tape devices.
Shared SAN file systems allow users to backup any and all servers to a common global pool or file system. Realizing that the majority of the retrieval requests are for data less than two days old, this technique results in nearly instantaneous recovery times. Sharing a file system for all or many servers means that the total space allocated for this hot staging area can be overly generous, as each system can vie for the exact amount it needs at backup time, as opposed to the hard-partitioning of a LUN masking approach. Furthermore, it means that any other SAN server can recover the data.
In practice, this concept is not new. What is new is that now it can be done at pure SAN speeds, which finally makes this a deployable solution: Previously, the overhead of the LAN and the impact and dependency on a backup server offset the other benefits.
Uptime and availability
Shared SAN file systems offer increased flexibility to companies that are concerned about continuous data availability. In the past, the only choice was to deploy a cluster. If a server of a particular application went down, another server on the same storage bus could start up the same service and continue processing. This is rapidly becoming a requirement in many environments
The down side of a cluster approach is that clusters can only scale to certain levels, typically two or four nodes in the case of Windows NT. Similar to clustering, yet different, a shared SAN file system inherently means that all systems potentially have access to the same data. If any one system is unavailable, the data is still available via another system.
SANs are often deployed with dozens of servers. The biggest difference between a shared file system and a cluster is that with a shared file system many systems can simultaneously access and share the same files and file systems. Clusters do not provide such a capability; there is strictly one machine per file system.
File system approaches
There are a number of SAN file systems available today, and there are two prevailing architectures. One approach is to use a distributed lock manager and essentially treat the SAN nodes as a tight cluster, similar to a VAX Cluster. Global File System (GFS) is a popular choice in that category, due to the fact that it is freely available on Linux. However, the cluster approach tends to be proprietary and homogeneous.
An alternative is a split data-metadata architecture. This architecture uses a single server per logical file system as the coordinator of all the metadata traffic (see Figure 2). File system metadata contains information such as a file's name, allocation information, security and ownership specifics, and other information about the file. This information is typically very small and easily communicated over a LAN between peers. Using a LAN brings the benefit of presenting the storage as a network volume, so that most heterogeneous operability issues are eliminated. The file data, including the actual contents, is accessed directly from each SAN node, once the rights to the allocation table have been arbitrated.
Figure 2. In a split data-metadata architecture, a single server per logical file system acts as the coordinator of all metadata traffic.
This split data-metadata architecture is currently available in products from ADIC (CVFS), DataDirect Networks (CDNA), SGI (CXFS), and Tivoli Systems (SANergy). Furthermore, the Storage Networking Industry Association (SNIA) File System Working Group has developed a similar architecture. Although this architecture has some drawbacks for very small files (because the metadata still involves the LAN), and it requires added coordination of the metadata, it is an efficient solution for large file transfers.
The other differentiator among SAN file systems is whether they exploit existing file systems or introduce new file systems. Most approaches use new file systems. The idea is that a completely redesigned file system, with SANs in mind from the beginning, can allow for better overall performance, coherency, and utility. The counter argument is that new file systems typically take a relatively long time to be fully tested and accepted. Will the new file system work with existing backup, hierarchical storage management (HSM), and quota management software? And, more importantly, will it work reliably with existing applications, such as databases?
Implementing a split data-metadata SAN file system can take considerable work. However, the vast majority of that work is the same regardless of the underlying file system. Users choose operating systems, platforms, and file systems based on their specific business needs. Thus, it seems to make sense to look for a SAN file system that allows the greatest flexibility and exploits the most existing components.
The true value of SANs is realized once multiple servers are allowed to access and share the same file systems. Deploying SAN file systems will result in reduced total cost of ownership, improved overall performance, and more flexibility than is possible with LUN masking alone.
Chris Stakutis is the chief technology officer in the SANergy product line at Tivoli Systems, in San Jose, CA. www.tivoli.com.