WebVal is a system that will scan documents for fully-qualified HTTP URLs, keeping its database fresh with newly-seen URLs. It can then be requested to validate the URLs, whereby it will attempt to access each URL via an HTTP request and record the response code; it maintains a list of the most recent codes that have been retrieved. Response codes are classified as "good" (URL is correct and a valid page is there) and "bad" (URL is invalid or outdated). By default any code other than a 2xx code is considered bad, but this can be changed (e.g. to ignore 3xx redirection codes). WebVal can then be used in report mode where it will scan documents for URLs as before, but will report invalid URLs (that is, URLs in the database which have a number of "bad" codes exceeding a certain threshold). These are then printed to stderr in a format that shows the file and line number the URLs were seen in so that they can be corrected.
Khttrack is a easy-to-use offline browser utility with a KDE Wizard Interface. It is a frontend to Httrack, used as a library, for KDE. It allows you to download a WWW site from the Internet to a local directory, recursively building all directories, getting HTML, images, and other files from the server to your computer. It arranges the original site's relative link-structure. It can also update an existing mirrored site, and resume interrupted downloads. It is fully configurable, and has an integrated help system.
Furies is a Java-based Web site testing tool which relies on many Jakarta components. Its goal is to ensure a Web page is "correct" by running one or more conditions on it. The Furies distribution supplies some conditions and it is very easy to program your own. For accessing Web sites, Furies relies on HttpClient and configuration may be done programmatically or via Digester.
phpLinks is a free PHP script which allows you to run a very powerful link farm or search engine simulation. Features include full search capabilities, recursive multilevel site categorization, full referrer tracking, site reviews, site ratings, link validation, related categories, category searching, and search term tracking.
PureTest is primarily targeted for function testing in the earlier phases of a development project. Even though it supports testing a variety of applications it is especially useful for debugging and snooping of Web applications. It includes a HTTP Recorder and Web Crawler which makes it useful for generic verification of HTTP requests and Web content checking.
Hammerhead is a stress testing tool for Web sites. It initiates connections from multiple IP aliases and simulates a user from each alias. It is fully configurable, and there are numerous other options for creating problems with a site. Extensive data collection is also available.
PureLoad is a load testing tool that simulates thousands of users executing requests against server based applications. It can be used to verify that your application will meet the expected performance criteria. PureLoad reports quality and performance problems, as well as detailed statistics gathered during a load test. PureLoad also includes extensive support for easy recording and testing of Web-based applications. The HTTP Recorder ease the process of capturing HTTP and HTTPS requests into PureLoad scenarios. The Web Crawler is used to scan a static Web site for general errors, and it can also be used to generate PureLoad scenarios to be used in a load test.