Release Notes: This version improves crawl stability and has been used in a page crawl of 1/3 billion pages. The indexing plugin API was improved to allow plugins to have configure screens. A new example Word Filter plugin has been added. Yioop can now crawl Tor networks. A Manage Groups pane has now been added to Yioop.
Release Notes: The crawler now has its own DNS caching mechanism independent of cURL's. Yioop now has a detection mechanism for when websites are becoming congested. The user can also set a quota on the number of URLs downloaded per hour from sites. A webcrawl statistics page can now be generated for a crawl. Bugs in robots.txt handling and in archive handling, which were introduced in 0.82, have been fixed. The demo site now features an example crawl of 100 million pages crawled with the previous version of the software.