fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 19 different types of I/O engines (sync, mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio, and more), I/O priorities (for newer Linux kernels), rate I/O, forked or threaded jobs, and much more. It can work on block devices as well as files. fio accepts job descriptions in a simple-to-understand text format. Several example job files are included. fio displays all sorts of I/O performance information, including complete IO latencies and percentiles. Fio is in wide use in many places, for both benchmarking, QA, and verification purposes. It supports Linux, FreeBSD, NetBSD, OpenBSD, OS X, OpenSolaris, AIX, HP-UX, Android, and Windows.
Lynis is an auditing and hardening tool for Unix derivatives like Linux/BSD/Solaris. It scans systems to detect software and security issues. Besides security-related information, it will also scan for general system information, installed packages, and possible configuration mistakes. The software is aimed at assisting automated auditing, software patch management, and vulnerability and malware scanning of Unix-based systems.
Monit is a utility for managing and monitoring processes, programs, files, directories, and devices on a Unix system. It conducts automatic maintenance and repair and can execute meaningful causal actions in error situations. It can be used to monitor files, directories, and devices for changes, such as timestamps changes, checksum changes, or size changes. It is controlled via an easy to configure control file based on a free-format, token-oriented syntax. It logs to syslog or to its own log file and notifies users about error conditions via customizable alert messages. It can perform various TCP/IP network checks, protocol checks, and can utilize SSL for such checks. It provides an HTTP(S) interface for access.
Berkeley DB (libdb) is a programmatic toolkit that provides embedded database support for both traditional and client/server applications. It includes b+tree, queue, extended linear hashing, fixed, and variable-length record access methods, transactions, locking, logging, shared memory caching, database recovery, and replication for highly available systems. DB supports C, C++, C#, Java, PHP, and Perl APIs. It supports key-value pair (NoSQL), SQL, and Java Object formatted data. It is available for a wide variety of Unix platforms as well as QNX, Android, Mac OS X, and several varieties of Windows.
Socat is a relay for bidirectional data transfer between two independent data channels. Each of these data channels may be a file, pipe, device (terminal or modem, etc.), socket (Unix, IP4, IP6 - raw, UDP, TCP), SSL, a client for SOCKS4, or proxy CONNECT. It supports broadcasts and multicasts, abstract Unix sockets, Linux tun/tap, GNU readline, and PTYs. It provides forking, logging, and dumping and different modes for interprocess communication. Many options are available for tuning socat and its channels. Socat can be used, for example, as a TCP relay (one-shot or daemon), as a daemon-based socksifier, as a shell interface to Unix sockets, as an IP6 relay, or for redirecting TCP-oriented programs to a serial line.
GNU parallel is a shell tool for executing jobs in parallel locally or using remote computers. A job is typically a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. If you use xargs today you will find GNU parallel very easy to use, as GNU parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU parallel as input for other programs.
syslog-ng is a syslogd replacement for a wide variety of UNIX systems that supports IPv6 and is capable of transferring log messages reliably using TCP and SSL and filtering the content of messages using regular expressions. Both RFC3164 and RFC5424 style messages are handled, but more esoteric formats like BSD process accounting logs are supported too. Apart from regular text files, it supports storing messages into SQL and MongoDB databases, and forward messages to local processes via pipes or UNIX domain sockets. This makes syslog-ng ideal as an integration platform. syslog-ng supports extracting structured information from the traditionally text based syslog via csv-parser(), db-parser(), and patterndb. Tag based classification, rewriting messages, and outputting messages in JSON is also possible. This makes syslog-ng ideal for preprocessing events for further analysis, be that home-grown scripts or SIEM systems. syslog-ng scales well on today's multi processor and multi-core systems: reaching 1,000,000 messages per second is a reality for the simplest use cases.
rsync is a replacement for rcp (and scp) that has many more features. It uses the "rsync algorithm" which provides a very fast method for remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand.