dispy is a Python framework for parallel execution of computations by distributing them across multiple processors in a single machine (SMP), or among many machines in a cluster or grid. The computations can be standalone programs or Python functions. dispy is well suited for the data parallel (SIMD) paradigm where a computation is evaluated with different (large) datasets independently (similar to Hadoop, MapReduce, Parallel Python). dispy features include automatic distribution of dependencies (files, Python functions, classes, modules), client-side and server-side fault recovery, scheduling of computations to specific nodes, encryption for security, sharing of computation resources if desired, and more.
|Tags||Python Distributed Computing Parallel Computing mapreduce hadoop|
|Operating Systems||Linux Windows Mac OS X|
Release Notes: The asyncoro module (an independent Python framework for asynchronous, concurrent, distributed network programming) has been updated to the latest release, which improves performance of message passing with coroutines.
Release Notes: This release changes the license to the MIT license and updates asyncoro to the latest release.
Release Notes: This release fixes occasional deadlock/potential crash issues during dispy shutdown and crashes with dispyscheduler (shared scheduler).
Release Notes: This release updates asyncoro with support for distributed, fault-tolerant coroutines, and fixes an issue with parsing node selection in dispyscheduler.