CCTE is a small utility to provide a way to run automated tests on Web applications. It acts like a Web browser GETing form data to different URLs, and logging output. It has two ways to understand if a CGI did OK: HTTP status, and user-definable strings for both (success and error).
Dead Link Check (DLC) is a Perl script designed to find information on validity of HTTP references. The script may use/generate a cache file for avoiding redoing network requests if the user wants to check added entries. The script works by reading entries from a file (or a list of links from the command line) and output results in file(s) (or STDOUT). DLC was created as an extension to Public Bookmark Generator (PBM), but can be used on its own.
GNU Wget is a utility for noninteractive download of files from the Web. It supports HTTP and FTP protocols, as well as retrieval through HTTP proxies. It can follow HTML links, download many pages, and convert the links for local viewing. It can also mirror FTP hierarchies or only those files that have changed. Wget has been designed for robustness over slow network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved.
Hammerhead is a stress testing tool for Web sites. It initiates connections from multiple IP aliases and simulates a user from each alias. It is fully configurable, and there are numerous other options for creating problems with a site. Extensive data collection is also available.
ht://Check is a link checker derived from ht://Dig. It can retrieve information through HTTP/1.1 and store it in a MySQL database so that after a "crawl", ht://Check can return broken links, anchors not found, content-types, and HTTP status codes summaries. ht://Check also performs accessibility checks in accordance with the principles of the University of Toronto's Open Accessibility Checks (OAC) project, allowing users to discover site-wide barriers like images without proper alternatives, missing titles, etc. A PHP interface lets the user query and view the results directly via the Web.
HTTrack is an easy-to-use offline browser utility. It allows you to download a Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer. HTTrack arranges the original site's relative link-structure. Simply open a page of the mirrored Web site in your browser, and you can browse the site from link to link, as if you were viewing it online. HTTrack can also update an existing mirrored site, and resume interrupted downloads. WebHTTrack is a Web-based GUI for HTTrack.
With LinkChecker, you can check HTML documents and Web sites for broken links. It features recursion, robots.txt exclusion protocol support, HTTP proxy support, i18n support, multithreading, regular expression filtering rules for links, and user/password checking for authorized pages. Output can be colored or normal text, HTML, SQL, CSV, or a sitemap graph in DOT, GML, or XML format. Supported link types are HTTP/1.1 and 1.0, HTTPS, FTP, mailto:, news:, nntp:, Telnet, and local files.
Load is a utility for Web application and SOAP-based Web services performance and scalability testing. It features a scripting language and a library of test objects for the creation of intelligent agents that drive the Web application or SOAP-based Web service. Running hundreds of agents concurrently shows how your software performs in production environments. Load is a Java application that runs on Linux, Solaris, NT, Win2000, and Macintosh. While Load continues to be maintained, the next generation is the TestMaker program.