It's no secret that digital storage needs are increasing rapidly. Experts at IDC predict that by 2020, there will be 40 zettabytes (ZB) of data in the digital universe, or 5,247 GB per person alive at that time. That's the equivalent of 57 times the number of all the grains of sand on all the beaches on earth.
The open source community has developed a number of tools to help people store, secure and manage all that data. This month, we're spotlighting 58 of the best of these projects for home users, businesses and enterprises. They include software for setting up a NAS or SAN appliance on standard hardware, backup and synchronization applications, RAID tools and more.
If you know of additional open source storage projects that you feel should be on the list, please make note in the comments section below.
Ceph describes itself as a "distributed object store and file system designed to provide excellent performance, reliability and scalability." It's built on the Reliable Autonomic Distributed Object Store (RADOS) and allows enterprises to build their own economical storage devices using commodity hardware. Operating System: Linux
The FreeNAS website touts itself as "the most potent and rock-solid open source NAS software," and it counts the United Nations, The Salvation Army, The University of Florida, the Department of Homeland Security, Dr. Phil, Reuters, Michigan State University and Disney among its users. You can use it to turn standard hardware into a BSD-based NAS device, or you can purchase supported, pre-configured TrueNAS appliances based on the same software. Operating System: FreeBSD
Red Hat-owned Gluster is a distributed file system that can handle big data—up to 72 brontobytes. It promises high availability and performance, an elastic hash algortithm, an elastic volume manager and more. Operating System: Linux
Now owned by Xyratex, Lustre is a highly-scalable file system that provides fast performance. It can handle "tens of thousands of nodes and petabytes of storage." Note that although the name is similar to "Gluster," the two are completely independent projects. Operating System: Linux
Like FreeNAS, NAS4Free makes it easy to create your own BSD-based storage solution from standard hardware. It promises a low-cost, powerful network storage appliance that users can customize to their own needs. Operating System: FreeBSD
Openfiler aims to combine NAS features (NFS, CIFS and HTTP) with SAN features (iSCSI, FC) in a very easy-to-setup storage appliance. It's been downloaded more than a million times, and its users include Motorola, Pratt & Whitney, BillMeLater and the London Metropolitan Police. Paid support is available. Operating System: Linux
Similar to Openfile, OpenSMT can be used to create storage devices with some SAN and some NAS features. It uses the ZFS file system. Operating System: OpenSolaris
This NAS solution boasts really easy-to-use storage management, fast setup, uninterruptible power supply support and statistics reporting. It's based on Debian Linux and offers a number of plug-ins to extend its capabilities. Operating System: Linux
Turnkey offers more than 100 different software appliances based on open source software. Among them is a file server that offers simple network attached storage. Operating System: Linux
This popular file system is incorporated into many other open source storage projects. It offers excellent scalability and data integrity, and it's available for most Linux distributions. Operating System: Solaris, OpenSolaris, Linux, OS X, FreeBSD
DRBD provides high availability for storage by applying RAID-1 to the network. Support and training are available through the project owner, LinBit. Operating System: Linux
This piece of the Linux kernel makes it possible to set up and manage your own RAID array using standard hardware. It also provides reporting capabilities. Operating System: Linux
Raider applies RAID 1, 4, 5, 6 or 10 to hard drives. It relies heavily on Linux's mdadm. Operating System: Linux
Monitor your OS X-based RAID arrays with this helpful tool. It runs in the background, and if it detects an error or failure, it will send you an email. Operating System: OS X
This project aims to make it easy for anyone to use RAID on their Linux systems. The website explains the project name by noting, "Salamanders are the only vertebrates that can regenerate limbs. In the same way, a system installed with Salamander can regenerate after a hard-drive failure." Operating System: Linux
SnapRAID backs up disk arrays, primarily those used to store home media collections. It allows recovery even if two separate disks in the array fail. Operating System: Windows, Linux, OS X
Backup and Synchronization
The "Advanced Maryland Automatic Network Disk Archiver," a.k.a. AMANDA, backs up networks to tape or hard drives. Paid support and a cloud-based version are available through Zmanda, which is owned by Carbonite. Operating System: Windows, Linux, OS X.
18. Areca Backup
A good option for backing up a single system, Areca aims to be simple and versatile. Key features include compression, encryption, filters and support for delta backup. Operating System: Windows, Linux
Designed for enterprise users, BackupPC claims to be "highly configurable and easy to install and maintain." It backs up to disk only (not tape) and offers features that reduce the amount of storage space necessary, as well as the number of I/Os. Operating System: Windows, Linux
Another enterprise-grade open source back solution, Bacula offers a number of advanced features for backup and recovery, as well as a fairly easy-to-use interface. Commercial support, training and services are available throughBacula Systems. Operating System: Windows, Linux, OS X
Conceived as a replacement for True Image or Norton Ghost, Clonezilla is a disk imaging application that can do system deployments, as well as bare metal backup and recovery. Operating System: Linux
Create Synchronicity's claim to fame is its lightweight size—just 220KB. It's also very fast, and it offers an intuitive interface for backing up standalone systems. Operating System: Windows