Niocchi is a Java crawler library implementing synchronous I/O multiplexing. This specific type of implementation allows crawling tens of thousands of hosts in parallel on a single low end server. Niocchi has been designed for big search engines that need to crawl massive amount of data, but can also be used to write no-frills crawlers.
|Tags||Software Development Libraries Java library Crawl Library Crawl|
|Operating Systems||OS Independent|