This is a tool to collect information from web servers and to spider the web sites. This was written for the Open Source Security Testing Methodology (OSSTM) located on http://www.ideahamster.org/osstmm- description.htm. The spider is a multi-threaded resusable module that can be used in other projects.
|Tags||Internet Web Indexing/Search Site Management Software Development Libraries Java Libraries|
|Operating Systems||BeOS Mac OS X Windows OS Independent|
Release Notes: Support was added for a custom login handler, keyword search, Flash content, File I/O site info, newer HTML tags, dynamic site element handling, multiple spiders in parallel, adding files or folders in elements to spiders, email searches within a specific domain, redirects within a domain, and detection of domain users found in the site. Many bugs were fixed.
Release Notes: Support for socket timeout, a fix for a problem occuring on locked socket condition, a new socket_timeout option in the webglobal.properties file, and updating Jakarta commons-httpclient with its CVS source (13-04-2002).