RSS 5 projects tagged "crawler"

Download Website Updated 31 May 2010 ItSucks

Screenshot
Pop 86.95
Vit 4.60

ItSucks is a Web spider with the ability to download (and resume) files. It is also highly customizable with regular expressions and download templates. All backend functionality is also available in a separate library.

Download Website Updated 06 Apr 2014 OpenSearchServer

Screenshot
Pop 569.58
Vit 89.63

OpenSearchServer is a powerful, enterprise-class, search engine program. Using its Web user interface, crawlers (Web, file, database, etc.), and REST/RESTFul API, you can integrate advanced full-text search capabilities into your application.

No download No website Updated 21 Nov 2010 skipfish

Screenshot
Pop 64.25
Vit 1.55

skipfish is a high-performance, easy, and sophisticated Web application security testing tool. It features a single-threaded multiplexing HTTP stack, heuristic detection of obscure Web frameworks, and advanced, differential security checks capable of detecting blind injection vulnerabilities, stored XSS, and so forth.

Download Website Updated 30 Dec 2010 Ebot

Screenshot
Pop 101.53
Vit 3.28

Ebot is a scalable and distribuited Web crawler. The URLs are saved to a NOSQL database (which supports map/reduce queries) that you can query via RESTful HTTP requests or using your preferred programming languages. The URLs that need to be analyzed are sent to AMQP queues. In this way, it is possible to run several crawlers in parallel and stop and start them without losing URLs.

No download No website Updated 02 Apr 2012 mycelium

Screenshot
Pop 15.65
Vit 27.29

mycelium is an information retrieval system. It aspires to be an alternative to Nutch / Lucene. It uses MongoDB as a storage engine.

Screenshot

Project Spotlight

PCBook

A reservation system for PCs.

Screenshot

Project Spotlight

Paranoid TelnetD

A telnet server with chrooting, whitelisting, and other security features.