I just downloaded and installed the rpm: pysync-2.24-1.i386.rpm. I'm getting the following error. Any ideas?
[root@jackson ~]# python
Python 2.4.3 (#1, Apr 14 2011, 20:41:59)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysync
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.4/site-packages/pysync.py", line 118, in ?
ImportError: No module named md4
Re: Handles large archives?
> search on google.com for "python rsync".
> almost right at the top is someone
> implementing rsync in python, but he
> hasn't got round to doing the bits that
> this guy has.
I think you are refering to rsync.py. (www.vdesmedt.com/~vds2...) I just looked at it. It doesn't implement the rsync algorithm or any network transport. It only copies the walk/filter/copy functionality of rsync.
Another interesting project is zsync. (zsync.moria.org.uk/) This implements an inverse rsync algorithm and uses a normal http server for network transport. It has no walk/filter/copy functionality.
For those who want to add walk/filter functionality to pysync, I have bits and pieces that might be useful;
efnmatch.py (minkirri.apana.org.au/...) rsync style extended fnmatch.
dirscan.py (minkirri.apana.org.au/...) rsync style include/exclude pattern directory scanning.
ddiffutils.py (minkirri.apana.org.au/...) efficient directory comparison walk generators.
I haven't yet tied these together into a useful combination, but I should some day :-)
Re: Handles large archives?
search on google.com for "python rsync". almost right at the top is someone implementing rsync in python, but he hasn't got round to doing the bits that this guy has.
combine the two projects and you have a _complete_ implementation of rsync in python.
i aim to investigate this project because i want to be able to offer different files "merged" into one single repository, depending on who connects to the rsync server :)
i.e. i can back up several machines, but the config files will be different.... cool, huh? :)
> I'm curious as to whether this tool has
> the same architectural limitation as
> rsync, that it must build a complete
> archive listing before it begins to
> transfer files.
pysync only implements the delta calculation and patch application, it does not include any directory walk or network transport stuff. So pysync doesn't have those limitations because it doesn't include that kind of functionality. Pysync could be used to implement something that does what you want without those limitations.
Have a look at librsync, rdiff-backup, unison etc for possible other alternatives that might be closer to what you want.
Handles large archives?
I'm curious as to whether this tool has the same architectural limitation as rsync, that it must build a complete archive listing before it begins to transfer files. I have a system with about 6-7 million files, and even though the vast majority of them are --exclude'd out, and even though the system has 2GB of RAM, rsync runs out of memory. Does this program work the same way?
Version 2.24 release
Version 2.24 has been released to update pysync for the new librsync 0.9.6. Also includes some minor tweaks, including psyco support which gives a 33% speedup.
If anyone wants windows binaries, let me know and I'll build them.
Version 2.16 was a quick release to include the rollsum extension module. I am currently working on improving librsync and wanted to release pysync with the work I had completed on it thus far before leaving it for a little while.
It does not include inverse delta support yet, and the librsync extension is unchanged. This means the librsync incremental API and memory leak problems are still present. After I finish improving librsync I will address these problems.
Windows installer also available.
Now that the release is out, I see they didn't like me putting the windows .exe installer as an "OS X" package :-).
For those who want it, a windows installer for python 2.1 is available at the ftp site. (minkirri.apana.org.au/...).
For those who are intensly curious, there is a development diary in the Software Working File (minkirri.apana.org.au/...) also publicly visible.
Release of version 2.7
This release is a major milestone, including both an md4 sum extension module and a swig librsync extension.
Note that the API has changed a little for pysync to bring it more in line with rdiff. Both pysync.py and librsync.py can be used as drop-in replacements for rdiff, with the exception they use "rdelta" instead of "delta" as an option. This is to distinguish from the pysync.py alternative of "xdelta". Note that the file parameters have changed order!
The librsync wrapper supports the higer level file api, but the low-level API is currently faulty. I hope to have this fixed in the next release.
The other major change is use of distutils to build releases. This allows me to produce rpm's and windows installers. The source distribution comes as a unix tar.bz2, or a windows zip. Because windows does not usualy have support for autoconf and swig, the zip includes a pre-configured and swigged librsync. The tar.bz2 does not include librsync, so you will need to get it and the patch from the Sourceforge rproxy (sourceforge.net/projec...) project.
Comming soon: librsync wrapper sponsored by Accellion.
I am currently working on making a Python extension for librsync (part of the rproxy project on sourceforge) to add to pysync. This work has been sponsored by Accellion (www.accellion.com/).
This should be finished before the end of this week (2002-04-26). Those interested in tracking this development can do so in the pysync Software Working File. (minkirri.apana.org.au/...)
A tool to convert Slackware PACKAGES.TXT to various databases formats.
An advanced, high-performance, and secure recursing nameserver.