SpiderBot crawls the Web, retrieves content, and performs actions on the content. It is an effort to design and develop a truly pipelined distributed Web crawler.
Methanol is a modular, customizable Web crawling system with crawlers optimized for speed. It is designed to allow the administrator to set up any kind of filetype handling, parsing, and indexing rules.
Smart Cache Loader is a very configurable Web
grabber with special Smart Cache support.
Software for handling mailing lists, with a nice user-oriented Web interface.
A flexible one time password authentication system with pluggable OTP algorithms and userstorages.