GNU parallel is a shell tool for executing jobs in parallel locally or using remote computers. A job is typically a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. If you use xargs today you will find GNU parallel very easy to use, as GNU parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU parallel as input for other programs.
Drqueue is a distributed rendering queue. It helps manage jobs on a render farm. It works under Linux, Mac OS X, FreeBSD, IRIX, and Windows, supports Maya, Blender, Lightwave, Mental Ray, Bmrt (Renderman), 3Delight, and Pixie out of the box, and can support any other renderer or computing task that can be handled via scripting.
distcc is a parallel build system that distributes compilation of C/C++/ObjC code across machines on a network. It can be set up in just a few minutes and makes builds up to ten times faster. It does not require machines to share a filesystem or have the same libraries or header files, and installation does not need superuser privileges.
Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and grids. It is based on a hierarchical design targeted at federations of clusters. Ganglia is currently in use on over 500 clusters around the world and has scaled to handle clusters with 2000 nodes.
radmind is a suite of Unix command-line tools and a server designed to remotely administer the file systems of multiple Unix machines. At its core, radmind operates as a tripwire. It is able to detect changes to any managed filesystem object, e.g. files, directories, links, etc. However, radmind goes further than just integrity checking: once a change is detected, radmind can optionally reverse the change. Each managed machine may have its own loadset composed of multiple, layered overloads. This allows, for example, the operating system to be described separately from applications. Loadsets are stored on a remote server. By updating a loadset on the server, changes can be pushed to managed machines.
Heartbeat is a full-function high-availability system for Linux and other POSIX-like OSes. It monitors services and restarts them on errors. When managing a cluster (more than 1 machine), it will also monitor the members of the cluster and begin recovery of lost services in less than a second. It runs over serial ports and UDP broadcast/multicast, as well as OpenAIS multicast. It is easily adapted to different interconnect media and protocols. When used in a cluster, it can operate using shared disks, data replication, or no data sharing. Versions starting with 2.0 are comparable to any commercial HA package, providing resource monitoring, larger clusters, and detailed dependency information.
smartmontools contains utilities that control and monitor storage devices using the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) system built into ATA and SCSI hard drives. This is used to check the reliability of the hard drive and to predict drive failures. Version 5.x is designed to comply with the ATA/ATAPI-5 specification. Future releases will comply with the ATA/ATAPI-6 and ATA/ATAPI-7 specifications. It is meant to be an up-to-date replacement for the ucsc-smartsuite and smartsuite packages, and is derived from that code.
PVM (Parallel Virtual Machine) is a portable message-passing programming system, designed to link separate host machines to form a ``virtual machine'' which is a single, manageable computing resource. The virtual machine can be composed of hosts of varying types, in physically remote locations. PVM applications can be composed of any number of separate processes, or components, written in a mixture of C, C++ and Fortran. The system is portable to a wide variety of architectures, including workstations, multiprocessors, supercomputers and PCs.