Projects / dup


The DUP System is a language for productive, parallel, and distributed stream processing on POSIX systems. Programming with DUP is similar to writing shell scripts with pipes except that filters can have multiple inputs and outputs. Furthermore, the computation can be spread across multiple computers. A distinguishing characteristic of DUP compared to other streaming languages is that filters can be written in almost any programming language. The DUP System distribution includes the runtime system and a collection of over a dozen multi-stream filters.

Operating Systems

Recent releases

  •  14 Jun 2010 09:01

    Release Notes: This release adds support for authentication via SSH. DUP will now also secure the establishment of data streams in the system. This release also adds some minor improvements and various bugfixes. Most importantly, IPv6 support was improved, making the code more robust on dual-stack machines.

    •  27 Nov 2009 11:30

      Release Notes: In addition to fixing various bugs, this release improves the performance of most filters, especially for long lines of input. It also adds support for writing a PID file to dupd, increases the number of file descriptors to 65536 (overriding FD_SETSIZE), fixed various performance issues in dup to dupd communication, and also adds caching of DNS lookups to dupd. The language and network protocol are entirely backwards-compatible. The main changes for this release either improve performance or fix (largely minor) bugs.


      Project Spotlight


      A Fluent OpenStack client API for Java.


      Project Spotlight

      TurnKey TWiki Appliance

      A TWiki appliance that is easy to use and lightweight.