webcheck is a Web site checking tool for Web masters. It crawls a given Web site and generates a number of reports. The whole system is pluggable, allowing extra reports and checks to be added easily. It supports retrieving Web sites over HTTP, file, and FTP protocols and produces reports on site structure, broken links, old Web pages, overviews of external links, and more. The links that webcheck considers external are configurable through regular expressions, and webcheck honors robots.txt.
Web-Analiser is a script to gather and analyze statistics of Website traffic. It provides the whole gamut of every possible statistics and reports, which analyze visitors, pages, and Website traffic. It is designed to operate both one and several domains (Websites) at once. The statistics are maintained, in which each Website dealt with separately and together. It includes a great number of statistical reports, making it possible to have a complete idea of your audience.
Websitary is a script that monitors Web pages, RSS feeds, and podcasts and reports what's new. For many tasks, it reuses other programs (such as w3m, diff, and webdiff) to do the actual work. By default, it works on an ASCII basis, i.e. with the output of text-based Web browsers. With the help of some friends, it can also work with HTML.
phoneutria is a Web crawler that is multi-threaded, scalable, high performance, extensible, and polite. It can be used to crawl, index, load-test, or even download any Web or enterprise domain and is configurable through a XML configuration file. Phoneutria can be used for either checking the links of a Web site or for load-testing purposes (i.e. the level of politeness can be configured). It provides a plug-in mechanism for further extensions.
SWEC is a program that automates testing of dynamic Web sites. It parses each HTML file it finds for links, and if those links are within the site specified, it will check that page as well. In addition to parsing and locating links, it will also parse the pages looking for known errors and report those. It will report if a page cannot be read (by either returning a 404, 500, or similar).
ht://Check is a link checker derived from ht://Dig. It can retrieve information through HTTP/1.1 and store it in a MySQL database so that after a "crawl", ht://Check can return broken links, anchors not found, content-types, and HTTP status codes summaries. ht://Check also performs accessibility checks in accordance with the principles of the University of Toronto's Open Accessibility Checks (OAC) project, allowing users to discover site-wide barriers like images without proper alternatives, missing titles, etc. A PHP interface lets the user query and view the results directly via the Web.
Active Business Intelligence Portal is software that provides an integrated solution for designing and deploying reports and Web forms over the Internet and intranets. Active BI Portal consists of two parts: Active BI Portal Manager, which only works on Windows, and the server part, which is a collection of PHP scripts. Active BI Portal works with all major databases. It also features an integrated HTML editor, an integrated SQL builder, comprehensive security management, user-definable menus, and more.
smupcheck, which stands for Smart Update Checker, checks Web sites for updates automatically, even if they don't offer an RSS feed. It is a very basic tool, and does not offer advanced features such as checking password-protected Web sites, highlighting changes, or filtering results.