S3QL
S3QL is a file system that stores all its data online. It supports Amazon S3 as well as arbitrary SFTP servers and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. S3QL is designed to favor simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection and error handling have been included from the very first line, and S3QL comes with extensive automated test cases for all its components.
| Tags | FUSE Compression File system backup storage encrypted backup encryption de-duplication |
|---|---|
| Licenses | GPLv3 |
| Operating Systems | Linux |
| Implementation | C Python |
Recent releases
- 22 Jan 2012 02:16
Release Notes: This release makes file times no longer change when changing the system time zone, fixes and improves contrib/benchmark.p, fixes the "Too many chained redirections" bug, retries on 5xx server errors, and adds experimental native support for OpenStack/SWIFT servers and experimental proxy support.
- 06 Dec 2011 03:29
Release Notes: A "ValueError: Can't dump NULL values" crash when file system metadata was uploaded during heavy file system activity was fixed. s3qlcp and directory listing performance for very large directories were improved. An rm performance regression was fixed. --blocksize was renamed to --max-obj-size. Some crashes that happened under heavily concurrent file system usage were fixed. python-llfuse 0.37 is now used.
- 27 Nov 2011 22:22
Release Notes: Metadata is now stored in a custom, delta-encoded binary format, and inodes are assigned sequentially and not reused. This makes S3QL fully NFS-compatible. s3qlcp now also copies extended attributes and no longer generates incorrect block reference counts when copying a file which has identical blocks. This release eliminates a race condition in s3qlcp. When copying a file with s3qlcp immediately after it was modified or created, it was possible that s3qlcp would copy the new file attributes but the old data blocks.
- 21 Nov 2011 03:41
Release Notes: fsck will gracefully recover if temporary indices already exist (e.g. if the previous fsck was interrupted). This release will not fail with a "BadDigest" error when objects are only partially retrieved from the backend.
- 20 Oct 2011 21:00
Release Notes: This release fixes parsing of storage URLs: s3s:// no longer generates bogus error messages. Support has been fixed for prefixes in storage URLs. Retries on timeout when transmitting data to a remote server. Does not free a mount point when terminating due to an unhandled exception in the background thread.
Nikratio
24 Jul 2010 16:37
Heartbeat
- Popularity Score
- 493.91
- Vitality Score
- 25.79
- Subscriptions
- 38
