Quantum Speeds Big Data Storage with StorNext Update

By Pedro Hernandez

Big Data looms large over Quantum's latest version of StorNext, the software that runs the company's StorNext file sharing and archiving appliances, and it is now available as a stand-alone offering.

For StorNext 4.3, Quantum "added new features that help people manage the volume, velocity and value of Big Data," said Janet LaFleur, senior product marketing manager for StorNext. While the term has come to mean different things to different IT vendors, Quantum generally categorizes Big Data as lots of massive files.

Quantum's customers are swimming in huge files, typically medical images, high definition video, oil exploration data, satellite imagery and other files that can easily stretch into the multi-gigabyte range. Even run-of-the mill, data-driven enterprises are feeling the crunch, said LaFleur.

These days, "video is creeping everywhere," she noted. "It's so easy to do video," she added, pointing to inexpensive hardware and software that have put once pricey and exotic capabilities like video conference recording within reach of all types of businesses. As expected, expanded storage capacity and management requirements soon follow.

Traditional storage infrastructures are ill-equipped to share this type of Big Data and archive it efficiently, said LaFleur. Essentially, "Big Data gets so large that traditional infrastructure fails."

StorNext 4.3's first step in addressing this challenge is to cast a wider net. LaFleur noted that thanks to a "major database restructure" -- MySQL-based for the curious -- StorNext 4.3 now scales to 1 billion files, a dramatic increase over the roughly 100 million files that previous versions could effectively manage.

Another feature aimed at managing the 'volume' and cost of Big Data is the company's Active Vault technology. Active Vault lowers storage costs by establishing a managed vault in a tape library that is segregated from active, robot-accessible shelves. It makes policy-based recommendations on which tapes to 'vault' while tracking their locations. As a result, infrequently accessed tapes are transitioned to lower-cost tiers of tape storage.

StorNext's 'velocity' improvements come courtesy of its new database. Inherently quicker than past versions, Quantum's file system is now capable of "archive on ingest" rates that are 1.5 times faster (at 1 million files). On their way to a tape library from a disk system, files are immediately archived or truncated. Ingest performance remains steady despite the move to tape because only a small pointer to the file is kept on disk, said Quantum.

Retrieving data gets faster, too. Quantum claims that its StorNext Distributed LAN Client (DLC) protocol beats CIFS or NFS by providing 50 percent faster file access. According to the company, StorNext DLC Windows clients can achieve up to a 360 percent boost in maximum throughput over 10 GbE LAN connections and up to 60 percent for 1 GbE.

Lastly, when it comes to 'variety' StorNext is being a little more accommodating to small files. The software boasts up to 40 percent faster writes for files that smaller than 1 MB.  New directory quota options allow administrators to segment and manage storage on a per-project basis -- particularly handy in video production houses where video files, scripts and related documentation can be centralized into one pool.

Additionally, a new utility re-aligns files within multi-file formats -- like DPX that employ a series of small files to form a larger one -- to optimize data retrieval performance. The software also cuts file creation times of multi-file formats by up to 50 percent.

StorNext 4.3 is available on August 1.

Pedro Hernandez is a contributing editor at InternetNews.com, the news service of the IT Business Edge Network, the network for technology professionals. Follow him on Twitter @ecoINSITE.

Follow InfoStor on Twitter

This article was originally published on July 12, 2012