RSS 3 projects tagged "web crawler"

Download No website Updated 18 Nov 2012 SpiderBot

Screenshot
Pop 16.49
Vit 22.78

SpiderBot crawls the Web, retrieves content, and performs actions on the content. It is an effort to design and develop a truly pipelined distributed Web crawler.

Download Website Updated 06 Apr 2014 OpenSearchServer

Screenshot
Pop 580.70
Vit 79.09

OpenSearchServer is a powerful, enterprise-class, search engine program. Using its Web user interface, crawlers (Web, file, database, etc.), and REST/RESTFul API, you can integrate advanced full-text search capabilities into your application.

No download No website Updated 25 Jun 2009 Methanol

Screenshot
Pop 28.98
Vit 1.01

Methanol is a modular, customizable Web crawling system with crawlers optimized for speed. It is designed to allow the administrator to set up any kind of filetype handling, parsing, and indexing rules.

Screenshot

Project Spotlight

Sculptor

A DSL and code generator for Java enterprise applications.

Screenshot

Project Spotlight

Clonezilla

A partition or disk cloning tool similar to Symantec Ghost.