digup is a console tool to update md5sum or shasum digest files. It will read existing digest files, check the current directory for new, updated, modified, renamed, or deleted files, and query the user with a summary of changes. After reviewing the updates, they can be written back to the digest file. This makes digup very useful to update and verify incremental archives like chronological data storages, which are commonly stored and backed up on hard disks. Using a full file digest scan, even slowly creeping bad blocks on old hard disks can be detected. By using a crontab entry, this check can be performed unattended and routinely.
Deduplicator is a simple and efficient data deduplicator that works by hard linking files that have the same content. It is ideal for reducing the size of backups. It can save and restore intermediate results, so you can run it in a few short intervals, and allows you to review changes before they are committed to disk.