Opendedup is a project that develops a deduplication based filesystem for called SDFS. SDFS is a distributed and expandable filesystem designed to provide inline deduplication and flexiblity for applications. It benefits services such as backup, archiving, NAS storage, and Virtual Machine primary and secondary storage, and can be deployed in both a stand alone and distributed, multi node configuration. Standalone, it provides inline deduplication, replication, and unlimited snapshot capabilities. In a multi-node configuration, it adds global intra-volume deduplication, block storage redundancy, and block storage expandability. and will store and share unique data blocks with other volumes within the cluster. These volumes can also specify a level of redundancy for data stored in the cluster.
S3QL is a file system that stores all its data online. It supports Amazon S3, Google Storage, and OpenStack and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Additional features include compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. The design favors simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection, error handling, and extensive automated test cases are provided.
EJDB is an embedded JSON database engine. It aims to be a fast MongoDB-like NoSQL library that can be embedded into C/C++/Nodejs/Python3/Lua applications. It features collection-level write locking, collection level transactions, string token matching queries, and a Node.js binding.
UnQLite is a in-process software library which implements a self-contained, serverless, zero-configuration, transactional NoSQL database engine.It is a document store database similar to MongoDB, Redis, CouchDB, etc. as well a standard key/value store similar to BerkeleyDB, LevelDB, etc. It reads and writes directly to ordinary disk files. A complete database with multiple collections is contained in a single disk file. The database file format is cross-platform, and you can freely copy a database between 32-bit and 64-bit systems or between big-endian and little-endian architectures.
TomP2P is a P2P-based high performance key-value pair storage library. Each peer has a table (either disk-based or memory-based) to store its values. A single value can be queried or updated with a secondary key. The underlying communication framework uses Java NIO to handle many concurrent connections.
Zfswatcher is ZFS storage pool monitoring and notification daemon. It periodically inspects the zpool status and sends configurable notifications on status changes such as disk failures. It also controls the disk enclosure LEDs. There is an embedded Web interface for displaying status and logs.
Sirix is a versioned tree-based storage system. It provides Snapshot Isolation through MVCC and is best suited for flash-based storage. The tree encoding is suited for heavy update operations. It provides diffing algorithms either to import diffs between currently shredded (imported) XML documents to update a tree structure or to get the differences between versions of a tree structure. A GUI provides several visualizations for comparing imported tree structures.
Kvpm is a GUI front end for Linux LVM and GNU parted. LVM2 groups and volumes can be created, removed, and manipulated using most of the options supported by the standard LVM2 tools. Some support for creating and operating on partitions is provided. It also handles creating and mounting filesystems. LVM version 2.02.88 or newer must be installed with the lvm2app library. Only the KDE 4 libraries need to be installed, not the entire desktop, so it my be run from within Gnome, for instance.