Larbin is a Web crawler intended to fetch a large number of Web pages to fill the database of a search engine. With a network fast enough, it should be able to fetch more than 100 millions pages on a standard PC. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin.
|Tags||Database Information Management Metadata/Semantic Models Internet Web Dynamic Content|
|Implementation||C++ Perl PHP SQL Unix Shell bash|
No changes have been submitted for this release.