Projects / Opendedup

Opendedup

Opendedup is a project that develops a deduplication based filesystem for Windows and Linux called SDFS. SDFS is designed to support the unique needs of virtual environments and supports enhanced functionality for VMWare, Xen, and KVM. It can deduplicate a petabyte or more of data. It supports over 3TB per gigabyte of memory at a 128k chunk size. It can perform deduplication/reduplication at a line speed of 250 MB/s or more. VMware support: it works with vms, and can deduplicate at 4k block sizes. This is required to deduplicate Virtual Machines effectively Deduplicated data can be stored locally, on the network across multiple nodes, or in the cloud. The filesystem can deduplicate inline or periodically based on needs. This can be changed on the fly. There is support for file or folder level snapshots.

Tags
Licenses
Operating Systems

RSS Last announcement

New SDFS Performance Metrics Released 26 May 2011

New SDFS performance numbers are out and have greatly improved in 1.0.5. SDFS now provides over 10 Gb/s for most work loads. Check it at http://ope...

RSS Recent releases

  •  25 Mar 2013 20:53

Release Notes: This release fixes unique block deletion for filebasedchunkstore, volume freezing due to thread deadlock, and Azure bucket name not being honored. It enhances read performance by 40% under NFS through better locking techniques and file caching, and better file level locking under concurrent I/O should allow for lower latency multi write to the same file. It adds block caching for the S3, Azure, and FileChunkStore and defaults to 10MB. This should improve performance for multiple writes. This option can be configured during volume creation using the --chunk-store-read-cache option.

  •  22 Jan 2013 01:19

Release Notes: This version includes an updated Windows port, better garbage collection, and better performance. It also fixes replication, garbage collection, and threading bugs.

  •  28 Aug 2012 22:17

Release Notes: This release adds major enhancements to volume replication.

  •  23 Jul 2012 21:21

Release Notes: This release has many major enhancements including Azure deduplicated storage backend and better volume replication. It also includes murmur3 hashing, which can improve performance by up to 40%.

  •  07 Jun 2012 06:24

Release Notes: Fixes in this release focused on replication and snapshots. The ability to compact a volume offline was added.

Screenshot

Project Spotlight

Antivirus Live CD

A 4MLinux fork including the ClamAV scanner.

Screenshot

Project Spotlight

Crayonizer

Console purtyfication.