VDI-optimized storage: Unleashing the potential of virtual desktops

By Jeff Byrne

-- Client computing – in all its many forms – is ripe for virtualization.  Just as virtualization technology has increased the utilization, manageability and cost effectiveness of server and storage infrastructure, it can also provide benefits for user desktops and laptops.

In fact, one could argue that with client computing, the stakes – and potential benefits – are even higher.  The annual cost of procuring and managing physical client machines in a medium to large organization routinely runs in the thousands of dollars per user.  Beyond just cost issues, employees' laptops and other mobile devices in some companies are a security breach waiting to happen.  The loss of unsecured data over the past few years from stolen or misplaced laptops has cost some firms dearly.

To address these issues, virtual desktop infrastructures (VDIs) play host to virtual user desktops and laptops residing behind corporate firewalls in the data center.  Local and mobile users gain access to their desktop environments via remote desktop protocols, and run the applications as though they were hosted on their own physical computers, which can now be barebones thin clients or mobile devices.  This approach makes IT managers happy: business applications and user data are protected from theft or loss, while the virtual desktops themselves are centrally managed, saving on ongoing administrative costs.  Just as importantly, users gain the flexibility to run the set of applications they need and to access their personalized desktops from just about anywhere. 

VDI challenges: Back to reality
Sound too good to be true?  Unfortunately, in many instances, it is.  Some early adopters of VDIs have found that they can suffer from a variety of maladies.  For example, VDIs can consume excessive amounts of storage capacity – often into the multiple terabytes, even for a medium-sized installation of several hundred desktops.  Provisioning and updates of desktop images can also be an issue, requiring a cumbersome and time-consuming process.  These two often unforeseen problems can plague IT administrators as they scale up a VDI installation.

End users can also encounter some nasty surprises.  Entire groups of users will occasionally find that their desktops take a long time to boot.  End users may also learn that their desktop images are backed up only infrequently.  Worse yet, in the event of a crash, the recovery of desktop images may be painfully slow.

Oddly enough, this whole collection of IT and end user issues will likely occur almost simultaneously, all stemming from the same culprit: the use of traditional, networked storage.  Most standard SANs do not provide the functionality required for administrators to effectively provision and manage storage in a virtual desktop infrastructure.  This type of VDI storage environment is not merely inconvenient; it can be downright costly.  In fact, the cost of procuring and managing sub-optimal storage often makes up more than half of the total cost of a VDI.

To see what lies behind these problems, let's take a closer look at traditional SAN technologies at work in a VDI environment.

The main culprit: Inefficient use of storage
The first issue that arises when employing traditional SAN storage in a VDI is the use of excessive capacity.  Traditional storage utilizes fat volumes to host virtual desktops.  In a traditional storage system, a single virtual desktop – including its configuration files, guest operating system, applications and data – will consume on the order of 8GB to 10GB of space.  When a user requests a new virtual desktop, the boot image for that desktop is created either by taking a traditional snapshot of a golden boot image, or by employing a full-copy approach.  Though the traditional snapshot approach is more economical than the full copy method, few of these snapshots are writable and none are thin.  In a full copy approach, traditional SAN products pre-allocate space for all newly provisioned virtual desktop volumes, and don't allow for common data to be shared across boot volumes.  In the case of snapshots, hefty reservations of 30% are required for each copy of virtual desktop volumes.

The result: a highly inefficient use of networked storage.  These approaches store a large percentage of common data multiple times, failing to take advantage of the fact that desktop boot images are generally 98+% the same.  To illustrate the impact on storage, let's assume a full copy approach is used in a mid-sized VDI consisting of 250 desktops.  If the master volume is sized at 10GB, then the virtual desktop images would consume 250 x 10GB, or approximately 2.5TB of capacity.  In addition, for performance reasons this 2.5TB of boot images may well be deployed on several times that amount of capacity.  This amount of highly available, high performance storage will be very expensive, and will likely eliminate the cost advantages that a VDI would otherwise provide.

Painful provisioning
Unfortunately, the excessive use of VDI storage capacity is just the first in a series of issues stemming from the inefficiencies of traditional SAN technologies.  A second significant pain point arises in the provisioning and updating of boot images.  In the case of a VMware VDI environment with conventional SAN storage, an administrator must first create a virtual machine, either from scratch or a physical-to-virtual (P2V) conversion; next install a guest operating system and VMware Tools; and then convert the VM to a template, which may be customized as needed using tools such as Microsoft SysPrep.  Once the various templates are in place, administrators fulfill requests for a new virtual desktop by identifying the appropriate golden boot template and manually cloning the boot image to a production virtual desktop within their ESX cluster. 

This series of tasks can be difficult and time consuming, even for experienced administrators.  One VMware VDI customer reported that the provisioning of each new virtual desktop takes them 15 minutes.  While this may not sound like a lot, in a 100-seat VDI this translates to 1,500 minutes – or 25 hours – of manual provisioning effort.  This represents roughly three full work-days in which an administrator is not available for higher-value tasks.  Patching and updates are nearly as cumbersome, as they also require a manual cloning process. 

The first two challenges we've discussed primarily affect IT staff and budgets.  But VDI end users must also pay the price for the inefficiencies of traditional storage approaches.  Take, for example, boot performance.  It is not uncommon in some VDI environments for multiple users to demand access to their virtual desktops at roughly the same time.  This happens routinely in a large call center, for instance, as workers arrive for their 8-hour shift.  The need also occurs in unplanned situations; for example, as the result of power outages and other widespread disruptions.

What happens to boot performance in a traditional, shared storage environment when a SAN serving hundreds of virtual desktops is called upon to concurrently boot dozens of images?  Unfortunately, in most instances, boot performance will slow to a crawl.  Snapshot technologies on most traditional storage systems do not share cache for data that is common across different volumes.  So, in a simultaneous booting situation, all full-sized images must be drawn into cache, even though the images may each be 98+% the same.

Given a standard cache size of 32GB in mid-range storage arrays from suppliers such as EMC and Network Appliance, this means that no more than three snapshot volumes of 10GB each can be brought into cache at the same time, assuming nothing else is occurring on the array.  This leads to a heavy load which greatly diminishes cache effectiveness and in turn significantly increases the sequential load on back-end disks.  The result in a 100-client deployment: a 30-minute wait-time for some user desktops to boot.

Limited backups, prolonged recovery
A fourth and final storage-related challenge in VDIs – which impacts both users and IT administrators – is backup and recovery.  Traditional snapshots require large space reservations, which consume large chunks of storage and limit the number of snapshots that can be taken.  In fact, due to the inefficient, full-copy and reservation-based snapshot approaches employed by traditional copy technology, storage capacity needs will often grow linearly with the number of backups taken.

The other half of this challenge is the difficulty and complexity of recovering desktop images.  If the organization employs a third-party backup server, then recovery of a given desktop image will typically require a three-way conversation between storage administrator, backup administrator and the affected user, which can be both cumbersome and time consuming.  
Relief, at last – from two different quarters
Taken together, this set of storage-induced challenges can severely hamper a VDI and even make it cost prohibitive.  Fortunately, during the last few months, the industry has responded, bringing to market solutions that address these issues from two different perspectives: the storage side and the server side.  Let's see how these newly emerging VDI-optimized storage solutions address the challenges.

Storage-centric solutions
For the storage administrator, several vendors – including 3PAR and Network Appliance – have recently introduced offerings that economize on the use of VDI storage.  These solutions employ efficient snapshot technology, which enables boot disk clones to refer to existing data in a master boot disk rather than duplicate it.  That said, 3PAR is the only company today to provide a block-level VDI solution which provides both higher performance and avoids the relatively low limits with which NFS-based storage must contend. 

3PAR employs a reservation-less snapshot that provisions capacity autonomically, on-demand, as a user writes.  The desktop clones, which are writeable snapshots, are effectively linked to a master image through a series of pointers, forming what amounts to a snapshot tree.  This approach enables desktop images to effectively share virtual disks with a master image, significantly conserving disk space.  In the case of 3PAR's thin copy technology, administrators can effectively leverage the written physical capacity of a single golden desktop – in this case, 10GB – across hundreds of physical desktops, and consume no more than 12GB to 20GB in total.  With thin copy technology, storage administrators can reduce VDI capacity requirements by more than 90%, thereby making VDI storage affordable.

While these capacity savings alone are impressive, the benefits of thin technologies don't stop there.  Thin storage also streamlines the provisioning process.  In the case of 3PAR Thin Copy Desktop, for example, a pre-defined, customizable script allows administrators to automate the entire process.  To create new users in a VDI deployment:

• Create a Thin Provisioned volume
• Within that volume, build your golden image (including OS, apps and sysprep) of each VM in a VMware template.
• The Thin Copy Desktop script then discovers this template and creates as many writeable snapshots as there are users
• Thin Copy Desktop then presents the snapshots back to the VMware ESX Cluster, which then discovers the new VDI VMs.
• The desktops are then ready for assignment for end-user access.

Newly provisioned virtual desktops are ready in seconds, without the burden of manual, host-based cloning.  User data is stored separately from the desktop image, so it can be administered independently.  Virtual desktops can subsequently be individually patched or updated by any number of automated update utilities without impacting users' settings, data or applications.  For greater control of the desktop where individual customizations are discouraged, the golden image can also be updated and re-provisioned with the steps outlined above instead of using an automated 1-to-1 utility.

Using this automated provisioning capability, each new virtual desktop in a VMware VDI can be provisioned in just 15-20 seconds – versus the 15 minutes the VMware customer in our earlier example required using a traditional storage solution.  In fact, the provisioning of all 100 desktops would require on the order of one-half hour, saving three full work-days of time that the administrator could now spend more productively.

VDI-tailored storage solutions also eliminate the inefficiencies associated with "boot storms," in which a large number of users attempt to bring up their virtual desktops concurrently.  To accelerate boot performance, storage vendors such as 3PAR rely on two fundamental technologies: intelligent snapshots and adaptive caching.  Adaptive caching recognizes that the overwhelming majority of data is common between boot copies, and therefore, only a single copy of shared data is required in cache.  The success of adaptive caching, in turn, relies on intelligent snapshots, which are capable of sharing cache for data that is common across different storage volumes. 

So how does this work in practice?  The first desktop to boot brings its data into cache, and because of the high degree of commonality among boot images – which is now sharable data in cache – subsequent boot requests will find their boot image already in cache.  This greatly increases the cache hit rate, thereby minimizing back-end accesses to physical disk.  This approach enables a high degree of responsiveness to user boot requests and the near-simultaneous booting of hundreds of virtual desktops.  Adaptive caching also allows boot performance to scale with a growing number of desktops, making VDIs more cost effective for large user environments.

Finally, storage-centric solutions succeed in solving the last of our VDI challenges: cumbersome and time-consuming backup and recovery.  These solutions significantly reduce backup capacity needs by employing thin, copy-on-write snapshots that reside in the array.  The thin snapshots do not duplicate data; instead, they refer to existing, block-level data in previous snapshots, and only new or changed blocks are actually written.  Because each point-in-time snapshot consumes only a small amount of incremental storage, administrators can afford to take and retain many more snapshots than they could with traditional storage.

Recovery, too, becomes much more efficient.  Storage administrators can quickly identify the appropriate point-in-time snapshot and recover the user's virtual desktop.  Unlike in a traditional storage environment, there's no need to mount a volume from a backup server and hunt through multiple images for the correct one, because the point-in-time snapshots reside in the array.

In shopping for VDI storage, IT buyers must be careful to focus on enterprise-class solutions with the scalability, performance and availability characteristics required in a rapidly growing virtual desktop infrastructure.  To be safe, buyers should focus on VDI storage solutions from suppliers such as 3PAR and Network Appliance, which have already been proven in a broad array of VDI environments.

Server-centric solutions
Another recently announced solution, VMware View, enables server administrators to overcome VDI storage challenges.  VMware View 3 represents an upgrade to the company's existing VDI offerings, and includes some enhancements that allow users to better tailor their storage to virtual desktop infrastructures.  View Composer, one of these new capabilities, uses VMware Linked Clones technology to enable server administrators to rapidly create desktop images that share virtual disks with a master image, significantly reducing storage needs.  Administrators identify a gold image from which a desktop pool will be created, and then specify which LUN each of the VMs will reside on.  Composer then creates a replica of each of the LUNs and builds the thin, linked clones.

VMware View Manager, working in concert with View Composer, helps administrators to streamline the provisioning, management and deployment of virtual desktops.  Using linked clones, a newly provisioned VM contains pointers to the original, "gold image" VM for all common data.  When a patch or upgrade is required, administrators simply update the image at the top of the snapshot tree and refresh the links in all downstream desktops to the new version of the image.  Administrators can isolate users' personal settings so that they persist through this update process.  Using this approach, administrators can effectively patch or upgrade hundreds or even thousands of virtual desktop images from a single console, at a fraction of the cost required in traditional storage environments.

While VMware is the first virtualization platform provider to introduce VDI-tailored storage, we fully expect the other leading desktop virtualization players to follow the company's lead during the coming year.

Server vs. storage:  How do I choose?
Given recent industry innovations, IT managers now have solid ways of addressing VDI storage challenges from both a storage and server perspective.  The question is, how should IT organizations choose between the two?

In general, if a storage administrator is driving the VDI storage deployment, then it makes sense to pursue a storage-centric solution to optimize VDI storage.  These solutions embrace processes such as "boot from SAN" as well as provisioning and management practices that a storage administrator will be comfortable with, and which can largely be automated in a VDI environment by using a product such as 3PAR Thin Copy Desktop.  3PAR Utility Storage arrays meet another important criterion: the need for a robust, enterprise-grade storage solution.  Though mid-market storage solutions may include much of the needed functionality, they likely won't have the performance and availability characteristics required in a rapidly growing VDI.  Most storage-centric VDI solutions – including those from 3PAR and Network Appliance – are available for no additional charge to customers that have already invested in the vendors' enterprise storage products, making these solutions particularly cost effective.

If, on the other hand, a server administrator is managing the entire VDI deployment, then a server-centric solution makes more sense.  VMware View, for example, is integrated with vCenter, providing virtual server administrators with a familiar interface for provisioning and managing virtual storage.  Among other capabilities, VMware View facilitates and automates large-scale VDI patches and updates, so if the desire is to patch one-to-many while preserving individual customizations, View provides a nice facility.  One downside: as an add-on option, View Composer storage provisioning and management functionality comes at a price, costing on the order of $100-150 per user.


1-to-1 Updates
(via automated facility)

1-to-Many Updates
(via golden image)
Clean Image Storage or Server VDI SolutionsStorage or Server VDI Solutions
Preserve CustomizationsStorage or Server VDI SolutionsServer VDI Solution Only

Unlocking VDI potential
VDI storage has come a long way in recent months.  Users that have experienced storage growing pains in a virtual desktop infrastructure – or are investing in a VDI for the first time – will delight in the range of VDI-tailored storage solutions now on the market.  These offerings, which approach storage issues from either a storage or server standpoint, enable administrators to overcome the thorny capacity, provisioning and performance issues that plague VDIs configured with traditional networked storage. 

VDI-optimized storage solutions allow organizations to fully exploit the potential of centrally stored and managed virtual desktops.

Jeff Byrne is a senior analyst at the Taneja Group, an analysis and consulting firm focused on storage and storage-centric server technologies with a concentration in the developing area of virtualization. He can be contacted at jeff.byrne@tanejagroup.com.
Related articles:
Analyst View: VDI and snapshots: A winning combination
Storage vendors target virtual servers
Webcast: Virtual servers and the impact on storage

This article was originally published on June 12, 2009