Projects / pysync

pysync

Pysync has both a demonstration implementation of the rsync and related algorithms in pure Python, and a high speed librsync Python extension. The pure Python is not fast and is not optimized, however it does work and provides a simple implementation of the algorithm for reference and experimentation. It includes a combination of ideas taken from librsync, xdelta, and rsync. The librsync Python extension is less flexible and harder to understand, but is very fast.

Tags
Licenses
Operating Systems
Implementation

Recent releases

  •  17 Oct 2003 18:00

    Release Notes: Updated to use librsync 0.9.6. TODO and NEWS files have been added. Changed to use psyco if available, giving a 33% speedup. Changed to using a faster md4sum implementation based on the librsync implementation, and modified to use the RSA API. Rollin/rollout support has been added to historic adler32.py. There are minor cleanups to the rollsum code, and minor tweaks to the handling of block fragment matching.

    •  24 Jun 2002 05:28

      Release Notes: This version adds a fast rollsum extension module, and changes pysync.py to use it, and the README docs were updated.

      •  02 May 2002 18:06

        Release Notes: Be warned that the API and data format have been changed in ways that are not backwards compatible. A high speed librsync swig extension module was added. An MD4 extension module was added. The API was changed to be more like librsync/rdiff. distutils is now used to build releases for Linux and Windows.

        •  12 Mar 2001 09:43

          Release Notes: This release includes support for xdelta style deltas (optimized deltas calculated directly from the sourcefile, not a signature), simplified and optimized code to take advantage of python buffer() copy-less string slicing, and command-line delta calculation now reads files incrementaly.

          •  01 Mar 2001 18:01

            Release Notes: pysync.py was changed to use the new zlib-like API. A serious adler32 bug affecting rolling checksums was fixed. pysync-test.py was added for testing pysync.py.

            Recent comments

            09 Dec 2013 22:14 alexykot

            Last release ten years ago. That's really old school. Is it still working at all?

            23 Jun 2011 20:16 apetit518

            I just downloaded and installed the rpm: pysync-2.24-1.i386.rpm. I'm getting the following error. Any ideas?

            [root@jackson ~]# python
            Python 2.4.3 (#1, Apr 14 2011, 20:41:59)
            [GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
            Type "help", "copyright", "credits" or "license" for more information.
            >>> import pysync
            Traceback (most recent call last):
            File "<stdin>", line 1, in ?
            File "/usr/lib/python2.4/site-packages/pysync.py", line 118, in ?
            import md4,rollsum
            ImportError: No module named md4

            06 Jun 2005 19:50 abo

            Re: Handles large archives?


            > search on google.com for "python rsync".

            > almost right at the top is someone

            > implementing rsync in python, but he

            > hasn't got round to doing the bits that

            > this guy has.

            I think you are refering to rsync.py. (http://www.vdesmedt.com/~vds2212/rsync.html) I just looked at it. It doesn't implement the rsync algorithm or any network transport. It only copies the walk/filter/copy functionality of rsync.

            Another interesting project is zsync. (http://zsync.moria.org.uk/) This implements an inverse rsync algorithm and uses a normal http server for network transport. It has no walk/filter/copy functionality.

            For those who want to add walk/filter functionality to pysync, I have bits and pieces that might be useful;

            efnmatch.py (http://minkirri.apana.org.au/~abo/projects/pybackup/efnmatch.py) rsync style extended fnmatch.

            dirscan.py (http://minkirri.apana.org.au/~abo/projects/pybackup/dirscan.py) rsync style include/exclude pattern directory scanning.

            ddiffutils.py (http://minkirri.apana.org.au/~abo/projects/utils/ddiffutils.py) efficient directory comparison walk generators.

            I haven't yet tied these together into a useful combination, but I should some day :-)

            06 Jun 2005 15:39 lkcl

            Re: Handles large archives?
            search on google.com for "python rsync". almost right at the top is someone implementing rsync in python, but he hasn't got round to doing the bits that this guy has.

            combine the two projects and you have a _complete_ implementation of rsync in python.

            i aim to investigate this project because i want to be able to offer different files "merged" into one single repository, depending on who connects to the rsync server :)

            i.e. i can back up several machines, but the config files will be different.... cool, huh? :)

            09 Jan 2004 00:59 abo

            Re: Handles large archives?

            > I'm curious as to whether this tool has
            > the same architectural limitation as
            > rsync, that it must build a complete
            > archive listing before it begins to
            > transfer files.
            ...


            pysync only implements the delta calculation and patch application, it does not include any directory walk or network transport stuff. So pysync doesn't have those limitations because it doesn't include that kind of functionality. Pysync could be used to implement something that does what you want without those limitations.

            Have a look at librsync, rdiff-backup, unison etc for possible other alternatives that might be closer to what you want.

            Screenshot

            Project Spotlight

            OpenStack4j

            A Fluent OpenStack client API for Java.

            Screenshot

            Project Spotlight

            TurnKey TWiki Appliance

            A TWiki appliance that is easy to use and lightweight.