Is there a need for open source storage applications? Clearly.
As the volume of digital data continues to grow, storage becomes a bigger and bigger problem for both home users and organizations. According to the latest Digital Universe study by IDC, 4.4 zettabytes (4.4 billion terrabytes) of digital information existed in 2013, and the amount of data is doubling every two years. That means by the end of 2015, there will be 8.8 zettabytes in our world, and by 2010, there will be 44 zettabytes, enough to fill a stack of tablets stretching to the moon and back six times.
To cope with all this storage, people need digital storage space – and plenty of it. More and more frequently, they are turning to open source applications to help them set up and manage their storage. Some are attracted by the low cost of do-it-yourself storage, while others like the security and privacy or other features offered by open source projects.
Recently, there's been something of a surge in open source cloud storage and software-defined storage projects, so we've updated our previous list of open source storage solutions with quite a few new additions. If you'd like to call our attention to any other open source storage projects we may have missed, feel free to make note in the comments section below.
Ceph offers distributed block and object storage capabilities, as well as a POSIX-compliant network file system. It boasts "high performance, large data storage, and maximum compatibility with legacy applications." Operating System: Linux
Used by the United Nations, the Salvation Army, Reuters, Michigan State University, the Department of Homeland Security, Disney Interactive Media and many other organizations, FreeNAS makes it easy to turn standard hardware into a network attached storage device. Businesses that need paid support can purchase TrueNAS, an appliance that uses the same technology. Operating System: FreeBSD
Similar to FreeNAS, NAS4Free allows users to create their own NAS appliances using standard hardware and the free download. Features include a web-based interface, ZFS v5000 support, ZFS LZ4- compression, disk encryption and software RAID 0, 1, 5 and mix. Operating System: FreeBSD
Used by organizations like Motorola, Pratt & Whitney, BillMeLater and the London Metropolitan Police, Openfiler unites NAS features like NFS, CIFS and HTTP) with SAN features like iSCSI, FC. Commercial support is available. Operating System: Linux
This is another option for building your own NAS/SAN appliance—this time based on OpenSolaris. It uses the ZFS file system. Operating System: OpenSolaris
Aimed at home users or SMBs, Open Media Vault allows users to build a Linux-based NAS appliance. It includes a built-in BitTorrent client and DAAP server, as well as support for SSH, (S)FTP, SMB/CIFS and Rsync. Operating System: Linux
At the Turnkey site, you'll find more than 100 images for building all kinds of open source appliances, including the Turnkey File Server for building a simple network attached storage device. It includes compression support, a Web-based GUI and support for SMB, SFTP and Rsync file transfer. Operating System: Linux
Btrfs is a newer Linux filesystem being developed by Facebook, Fujitsu, Intel, the Linux Foundation, Novell, Oracle, Red Hat and some other organizations. It emphasizes fault tolerance and easy administration, and it supports files as large as 16 EiB. Operating System: Linux
This is the latest version of one of the most popular filesystems for Linux. One of its key benefits is the ability to handle very large amounts of data— 16 TB maximum per file and 1 EB (exabyte, or 1 million terabytes) maximum per filesystem. Operating System: Linux
Owned by RedHat, GlusterFS is a scale-out distributed file system designed to handle petabytes worth of data. Features include high availability, fast performance, global namespace, elastic hash algorithm and an elastic volume manager. Operating System: Linux
Designed for "the world's largest and most complex computing environments," Lustre is a high-performance scale-out file system. It boasts that it can handle tens of thousands of nodes and petabytes of data with very fast throughput. Operating System: Linux
Incorporated into many other open source projects, including many Linux distributions, ZFS is a popular filesystem for large data stores. Features include storage pooling, transactional copy-on-write semantics, checksums, snapshots, simplified administration and excellent scalability. Operating System: Solaris, OpenSolaris, Linux, OS X, FreeBSD
Owned by LinBit, DRBD allows users to build a high availability storage cluster using replication that is very similar to RAID-1. It offers fully synchronous, memory synchronous or asynchronous operation. Operating System: Linux
Part of the Linux kernel, Mdadm enables the management of Linux-based RAID arrays. It includes monitoring and reporting capabilities, as well as the ability to move data between arrays. Operating System: Linux
Raider is a bash shell script for building software RAID 1, 4, 5, 6, or 10 arrays. It's optimized for many popular Linux distributions, including Ubuntu, Debian, Ubuntu, Debian, Arch, Mandriva, Mageia, openSuSE, Fedora, Centos, PCLinuxOS, Linux Mint, Scientific Linux, Gentoo, Slackware, Bodhi Linux and others. Operating System: Linux
This app works with the built-in RAID capabilities of OS X to monitor storage arrays. It sends email and sound alerts when it detects failures. Operating System: OS X
Salamander aims to simplify the process of installing Linux on a RAID array. The name comes from the fact that Salamander arrays can recover after a hard-drive failure, just like living salamanders can regrow limbs after one has been cut off. Operating System: Linux
Designed for home media centers and similar storage solutions, SnapRAID provides data duplication such that your array will be able to recover from the loss of up to six disks. Operating System: Windows, Linux, OS X
Backup and Synchronization
Short for "Advanced Maryland Automatic Network Disk Archiver," AMANDA backs up multiple systems on a network to tape, hard disk or optical media. Zmanda, which sponsors the project, offers commercial products and cloud-based storage that uses the same technology. Operating System: Windows, Linux, OS X.
20. Areca Backup
If you need to back up a standalone system, Areca offers versatility and simple setup. Features include compression, encryption, archive merge, filters and support for incremental, differential, full and delta backup. Operating System: Windows, Linux
This enterprise-grade solution uses a unique pooling and compression scheme to minimize storage space requirements and I/Os. For example, backing up 95 laptops for three weeks may require 1200 GB of space using other backup methods, but with BackupPC, it would require only 150 MB. Operating System: Windows, Linux
Also designed for enterprises, Bacula offers network-based backup, recovery and data verification capabilities. Commercial support, training and services are available throughBacula Systems. Operating System: Windows, Linux, OS X
23. Back In Time
Similar to FlyBack (see below), Back in Time offers a very easy-to-configure snapshot backup solution. GUIs are available for both Gnome and KDE (4.1 or greater). Operating System: Linux
Short for "Backup Archiving Recovery Open Sourced," Bareos is a fork of Bacula that offers features like LTO hardware encryption, efficient bandwidth usage and practical console commands. A commercially supported version of the same software is available through Bareos.com. Operating System: Windows, Linux, OS X
25. Box Backup
Box Backup describes itself as "an open source, completely automatic, online backup system." It creates backups continuously and can support RAID. Operating System: Windows, Linux