S3QL is a file system that stores all its data online. It supports Amazon S3, Google Storage, and OpenStack and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Additional features include compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. The design favors simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection, error handling, and extensive automated test cases are provided.
Libchop is a set of utilities and library for data backup and distributed storage. Its main application is chop-backup, an encrypted backup program that supports data integrity checks, versioning at little cost, distribution among several sites, selective sharing of stored data, adaptive compression, and more. The library itself, which chop-backup builds upon, implements storage techniques such as content-based addressing, content hash keys, Merkle trees, similarity detection, and lossless compression. It makes it easy to combine them in different ways. The ‘chop-archiver’ and ‘chop-block-server’ tools, illustrated in the manual, provide direct access to these facilities from the command line. It is written in C and has Guile (Scheme) bindings.