RSS 8 projects tagged "Deduplication"

No download Website Updated 07 Apr 2014 Attic

Screenshot
Pop 395.89
Vit 13.82

Attic is a deduplicating backup program. The main goal of attic is to provide an efficient and secure way to back up data. The data deduplication technique used makes Attic suitable for daily backups since only actual changes are stored. Main features: space efficient storage, optional data encryption, and off-site backups.

No download Website Updated 26 Sep 2011 BlackHole

Screenshot
Pop 92.75
Vit 1.43

BlackHole is an data de-duplicating network block device that also supports mirroring, snapshots, and support for multiple LUNs using the same data store. It is filesystem agnostic and has been tested with ext2/3/4, NTFS, ReiserFS, and the Oracle Cluster File System (OCFS2). It supports encryption, compression, and multiple storage backends. The hashing scheme used is user configurable. The program exports an NBD device which can be mounted in Linux and GNU/Hurd.

No download No website Updated 16 Feb 2014 Duke

Screenshot
Pop 236.37
Vit 12.20

Duke is a fast and flexible record linkage engine. It does not use the traditional blocking (sort by key) approach, but instead relies on Lucene. This makes it high-performance (able to process 1,000,000 records in ~10 minutes). Duke can be run from the command line, but also has an API allowing incremental linking applications to be built easily. It supports reading data from CSV, JDBC, SPARQL, and NTriples, and also supports a number of string comparators and string normalizers.

Download No website Updated 22 Mar 2014 Fileaxy

Screenshot
Pop 78.08
Vit 13.04

Fileaxy is a file de-duplication, organization, and bulk previewing tool which utilizes a new user interface for local file management.

Download Website Updated 10 Apr 2014 Skylable SX

Screenshot
Pop 68.72
Vit 1.03

Skylable SX is a reliable, powerful, fully distributed cluster solution for your data storage needs. It can aggregate the disk space available on multiple servers and merge it into a single storage system. The cluster makes sure that your data is always replicated over multiple nodes (the exact number of copies is defined by the sysadmin) and synchronized. It has built-in support for deduplication, client-side encryption, on-the-fly compression, and much more.

No download Website Updated 28 Aug 2013 WD Arkeia Network Backup

Screenshot
Pop 388.99
Vit 22.80

Arkeia Network Backup is designed for organizations that require fast, easy-to-use, and affordable data protection. It backs up critical data to disk, tape, and cloud storage. Arkeia protects all major virtual platforms including VMware, Hyper-V, XenServer, and more than 200 physical platforms including Windows, Mac OS X, Linux, Netware, most UNIX flavors, and BSDs. The company’s source-side Progressive Deduplication technology helps users realize better performance at a lower cost by reducing data volumes. Arkeia’s deduplication is crucial to accelerating replication of on-premise backups to private or public clouds.

No download No website Updated 12 Feb 2012 image-deduplication-tool

Screenshot
Pop 40.62
Vit 28.28

image-deduplication-tool is a script designed to scan specified paths and calculate the DCT hashes of all the images there. It compares the hashes to find closest-looking image pairs, despite various alternations (such as crop, rotation, gamma/color correction, noise, etc.), optionally presenting them in a feh image viewer for the operator to easily compare and remove one of the versions. It uses libpHash to produce and compare perceptual hashes.

Download Website Updated 22 Aug 2010 lessfs

Screenshot
Pop 168.44
Vit 4.78

Lessfs is a high performance inline data deduplicating file system for Linux. Lessfs complies to the POSIX standard and is very useful for backup purposes as well as providing storage for virtual machine images. Although lessfs is a file system that is implemented in user space with FUSE, it offers decent performance. Lessfs is capable of handling data rates up to 350MB/sec. It supports filesystem encryption.

Screenshot

Project Spotlight

ToPIA

A persistence and application distribution framework.

Screenshot

Project Spotlight

Collax Business Server

An all-in-one Linux server for small- and medium-sized businesses.