1 project tagged "Crawl Library"

Download Website Updated 13 Jul 2011 Niocchi

Screenshot
Pop 27.13
Vit 41.00

Niocchi is a Java crawler library implementing synchronous I/O multiplexing. This specific type of implementation allows crawling tens of thousands of hosts in parallel on a single low end server. Niocchi has been designed for big search engines that need to crawl massive amount of data, but can also be used to write no-frills crawlers.

Screenshot

Project Spotlight

SmartGit/Hg

A GUI client for Git, Mercurial, and SVN.

Screenshot

Project Spotlight

libconfigduo

A C/C++ configuration file manipulation library.