OpenSearchServer is a powerful, enterprise-class, search engine program. Using its Web user interface, crawlers (Web, file, database, etc.), and REST/RESTFul API, you can integrate advanced full-text search capabilities into your application.
Smart Cache Loader is a very configurable Web
grabber with special Smart Cache support.
SpiderBot crawls the Web, retrieves content, and performs actions on the content. It is an effort to design and develop a truly pipelined distributed Web crawler.
A Perl/XML log analyzer, alerter, and reporter.
A code coverage tester.