1 project tagged "web crawler"
Updated 25 Jun 2009
Methanol
| Pop | 34.86 |
|---|---|
| Vit | 1.02 |
Methanol is a modular, customizable Web crawling system with crawlers optimized for speed. It is designed to allow the administrator to set up any kind of filetype handling, parsing, and indexing rules.
