103 projects tagged "Compression"
zlib is designed to be a free, general-purpose, legally unencumbered, lossless data-compression library for use on virtually any computer hardware and operating system. The zlib data format is itself portable across platforms. Unlike the LZW compression method used in Unix compress(1), the compression method currently used in zlib essentially never expands the data. (LZW can double or triple the file size in extreme cases.) zlib's memory footprint is also independent of the input data and can be reduced, if necessary, at some cost in compression.
bzip2 is a portable, lossless data compressor based on the Burrows-Wheeler transform. It achieves good compression and runs on practically every (32/64-bit) platform in the known universe. The compression engine is also available as a library (libbz2), should you desire a programmatic interface.
pngcrush is an excellent batch-mode compression utility for PNG images. Depending on the application that created the original PNGs, it can improve the file size anywhere from a few percent to 40% or more (completely losslessly). The utility also allows specified PNG chunks (e.g. text comments) to be inserted or deleted, and it can fix incorrect gamma info written by Photoshop 5.0 as well as the erroneous iCCP chunk written by Photoshop 5.5.
S3QL is a file system that stores all its data online. It supports Amazon S3, Google Storage, and OpenStack and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Additional features include compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. The design favors simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection, error handling, and extensive automated test cases are provided.
Opendedup is a project that develops a deduplication based filesystem for Windows and Linux called SDFS. SDFS is designed to support the unique needs of virtual environments and supports enhanced functionality for VMWare, Xen, and KVM. It can deduplicate a petabyte or more of data. It supports over 3TB per gigabyte of memory at a 128k chunk size. It can perform deduplication/reduplication at a line speed of 250 MB/s or more. VMware support: it works with vms, and can deduplicate at 4k block sizes. This is required to deduplicate Virtual Machines effectively Deduplicated data can be stored locally, on the network across multiple nodes, or in the cloud. The filesystem can deduplicate inline or periodically based on needs. This can be changed on the fly. There is support for file or folder level snapshots.
Ziproxy is a high-performance forwarding (non-caching) HTTP proxy that gzips text and HTML files, and reduces the size of images by converting them to lower quality JPEGs or JPEG 2000. It is intended to increase the speed for low-speed Internet connections (mobile, dial-up, other). It's suitable for both home and professional usage. Ziproxy is fully configurable and also features transparent proxy mode, HTML/JS/CSS optimization, operation in daemon mode, a detailed access log with compression statistics, basic authentication, and more.
hdup is used to back up a filesystem. Features include encryption of the archive (mcrypt/GnuPG), compression of the archive (bzip2/gzip/lzop/none), the ability to transfer the archive to a remote host or restoring from a remote host (with SSH), the ability to split up archives, and no obscure archive format (it is a normal compressed tar file).