GNU parallel is a shell tool for executing jobs in parallel locally or using remote computers. A job is typically a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. If you use xargs today you will find GNU parallel very easy to use, as GNU parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU parallel as input for other programs.
Collax V-Cube+ is a HA cluster management suite based on a 64bit Linux system and KVM to provide server virtualization. It offers solutions for single virtualization hosts, as well as high availability management on two or more nodes, allowing embedded HA storage using DRBD and iSCSI. By using live snapshots, automatic live migration, and incremental backups, the availability of virtual machines is increased tremendously in case of hardware and software maintenance or even hardware failures. Virtual network switches and the protocols GVRP, LLDP, and RSTP help to set up a virtual DMZ.
The Linux Replicated High Availability Manager (Linuxha.net) allows the creation of clusters for application high availability through data replication. Currently, clusters are limited to two nodes, but multiple applications can be hosted and failed-over between the nodes. The software uses DRBD to provide the data replication facillity.
Warewulf is an operating system management toolkit designed to facilitate large scale deployments of homogeneous and heterogeneous systems on physical, virtual and cloud based infrastructures. Originally, the Warewulf project pioneered the concept of stateless computing in HPC, setting the standard for large-scale cluster provisioning. It provided two functions, provisioning and monitoring but the two functions did not communicate within Warewulf itself, nor was it possible to hook other functions directly into Warewulf itself. Today, Warewulf is more than just a basic provisioning and monitoring solution as it now implements an abstract, object-oriented data store and a modular interface that facilitates a highly extensible, customizable feature set. Current and planned modules include monitoring (operating system, services, filesystems, etc.), provisioning, power management, user management, configuration management, event/trigger handling and notification, scheduler integration, cloud services (both local and remote), etc.
FAI (fully automatic installation) is a non-interactive system to install a Debian GNU/Linux operating system on a group of PCs or a Linux cluster. After installation, the systems are fully configured and ready to run. It is a scalable method for performing unattended installation and updating. Changes to the configuration files of the operating system are made by cfengine, shell, and Perl scripts.
The Bioinformatics Benchmark System is an attempt to build a reasonable testing framework, tests, and data, to enable end users and vendors to probe the performance of their systems. It is not trying to be the last word in informatics benchmarking, as there are simply too many codes, tests, data sets, and databases. The goal is to create a core of tests that all may download and use to probe specific elements of system performance. The end goal is to enable a pluggable set of tests, including the core tests, so that performance data may be gathered.
Whatsup is a cluster node up/down detection utility. It can quickly calculate and output the up and down nodes of a cluster through one of several possible backend tools and several optional cluster node databases. The currently supported backend tools are Ganglia, Cerebro, and Pingd (which can be downloaded as a subpackage of Whatsup). The curently supported cluster databases are genders, gendersllnl (a locally supported LLNL version of genders), and hostsfile (a textfile list of host names). C and Perl libraries for detecting up/down nodes are also included in the package.
Genders is a static cluster configuration database used for cluster configuration management. It is used by a variety of tools and scripts for management of large clusters. The genders database is accessed by every node in a cluster, either through a networked file system or by replicating the database on every node of the cluster. The database describes the layout and configuration of the cluster so that tools and scripts can sense the variations of cluster nodes. By abstracting this information into a plain text file, it becomes possible to change the configuration of a cluster by modifying only one file.
Gled is a C++ framework for rapid development of applications requiring a GUI, 3D graphics, or support for distributed computing. It extends the ROOT framework (a standard data-analysis tool in high-energy physics) with mechanisms for object collection management and serialization, multi-threaded execution, GUI auto-generation (object browser and editor), and dynamic visualization (OpenGL). The distributed computing model of Gled is a hierarchical tree of nodes connected via TCP/IP sockets. Gled provides authentication and access control, data exchange, proxying of object collections, and remote method-call propagation and execution. It can be dynamically extended with library sets. Their creation is facilitated by a set of scripts for creation of user-code stubs. Simple tasks and application configuration can be done efficiently via the interactive C++ interpreter (CINT). Gled is used for development of programs in high energy physics and as a research tool in distributed and grid computing.