3 projects tagged "de-duplication"

No download Website Updated 30 Mar 2014 S3QL

Screenshot
Pop 733.07
Vit 30.93

S3QL is a file system that stores all its data online. It supports Amazon S3, Google Storage, and OpenStack and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Additional features include compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. The design favors simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection, error handling, and extensive automated test cases are provided.

Download Website Updated 26 Feb 2014 cyphertite

Screenshot
Pop 456.15
Vit 25.42

Cyphertite is a tar-like secure remote archiver. It deduplicates, compresses, and encrypts data prior to transmission, providing total privacy while reducing unnecessary wire traffic. It seamlessly supports IPv6 and IPv4 on a variety of platforms.

No download No website Updated 31 May 2012 Backshift

Screenshot
Pop 60.33
Vit 1.20

Backshift is a deduplicating (variable-sized, content-based blocks), compressing (xz or bz2) backup program. Full saves and incrementals are pretty indistinct other than the amount of data transmitted, somewhat like with "rsync --link-dest" but without the huge number of hardlinks. It also de-duplicates large file content at a granularity of about 2 megabytes on average; there tends to be a unique copy of each file with size less than around 2 megabytes on average.

Screenshot

Project Spotlight

pg_repack

A tool which reorganizes tables in PostgreSQL databases with minimal locks.

Screenshot

Project Spotlight

Wmconfig

A menu generation tool for various X window managers.