Version 3.5.0 of Makeflow

Release Notes: Numerous improvements have been made to WorkQueue to improve support for various batch job submission platforms. WorkQueue now allows extensive configuration and automation of worker creation. Makeflow now supports selecting an arbitrary port in a range for a WorkQueue master.

Other releases

  •  29 Jul 2013 22:58

    Release Notes: Support for workers hierarchies, with a master-foremen-workers paradigm. Additionally, a worker now is able to handle more than one task at a time. Support has been added for task categories. A rule can be labeled with a category, and required computational resources (disk, memory, and CPU) can be specified per category. Makeflow then automatically communicates these requirements to the work queue or condor.

    •  19 Feb 2013 12:30

    Release Notes: Work Queue now allows you to specify chunks (pieces) of an input file to be used as input for worker tasks. Makeflow now supports a -i switch which outputs pre-execution analysis of the Makeflow DAG. Work Queue and Makeflow support submitting tasks to the PBS batch submission platform. The Makeflow makeflow_log_parser now ignores comments in Makeflow logs. Work Queue supports querying workers and tasks at run-time. Makeflow Input file (dependency) renaming is supported with a new "->" operator. Work Queue Tasks are now killed using SIGKILL. Work Queue Protocol based keep-alives were added to workers.

    •  12 Feb 2013 13:46

    Release Notes: In WorkQueue, memory errors that lead to a SEGFAULT, race conditions in signal handling for workers, and incorrect handling of the -C option where a worker would not use the same catalog server as work_queue_pool were corrected. Tasks now have a unique process group to properly kill all task children on abort. Escape codes such as \n and \t are now properly interpreted in Makeflow files.

    •  03 Nov 2012 07:25

    Release Notes: A bug in WorkQueue that resulted in a cancelled task becoming a zombie was fixed. Various corrections were made to the manual and HTML documentation. Makeflow -I and -O options now correctly the output file list to stdout. WorkQueue now correctly removes directories during cleanup.

    Release Notes: WorkQueue now has a new API for logging, improved python binding support and documentation, and various other bugfixes and minor features. Makeflow now supports garbage collection of intermediate files, lexical scoping of makeflow variables, and a new MAKEFLOW keyword for recursive makeflows.

    Screenshot

    Project Spotlight

    TheSSS (The Smallest Server Suite)

    A lightweight server suite for LANs.

    Screenshot

    Project Spotlight

    Fileaxy

    A file de-duplication, organization, and bulk previewing tool.