RSS 2 projects tagged "crawler"

Download Website Updated 31 May 2010 ItSucks

Screenshot
Pop 87.29
Vit 4.59

ItSucks is a Web spider with the ability to download (and resume) files. It is also highly customizable with regular expressions and download templates. All backend functionality is also available in a separate library.

No download Website Updated 11 Jun 2010 Ex-Crawler

Screenshot
Pop 23.58
Vit 1.03

The Ex-Crawler Project is divided into three subprojects. The main part is the Ex-Crawler daemon server, a highly configurable and flexible Web crawler written in Java. It comes with its own socket server, with which you can manage the server, users, distributed grid/volunteer computing, and much more. Crawled information is stored in a database (Currently MySQL, PostgreSQL, and MSSQL are supported). The second part is a graphical (Java Swing) distributed grid/volunteer computing client, including user computer state detection, based on JADIF Project. The Web search engine is written in PHP. It comes with a Content Management System, user language detection and multi-language support, and templates using Smarty, including an application framework that is partly forked from Joomla 1.5, so that Joomla components can be adapted quickly.

Screenshot

Project Spotlight

rpmorphan

A tool that finds "orphaned" RPM packages.

Screenshot

Project Spotlight

GNU ddrescue

A data recovery tool.