phoneutria is a Web crawler that is multi-threaded, scalable, high performance, extensible, and polite. It can be used to crawl, index, load-test, or even download any Web or enterprise domain and is configurable through a XML configuration file. Phoneutria can be used for either checking the links of a Web site or for load-testing purposes (i.e. the level of politeness can be configured). It provides a plug-in mechanism for further extensions.
SEO SpyGlass Enterprise is a feature-rich backlink analysis software for professional SEOs and webmasters. The tools provide the deepest possible insight into your competitors' link building strategies. It lets you milk your competitors of their most SEO-productive backlink sources and use these sources in your own SEO campaign. You'll check all backlinks for Google PageRank and Alexa Rank, scrutinize the used anchor texts and URLs, discover how many backlinks come from forums and blogs, from homepages, from DMOZ-listed sites and much more. The software generates 5 types of reports that can be further printed out, sent to your clients by e-mail or made available on a Web site. You can brand your reports with a company logo and set their color schemes and data layout. This software is also available within SEO PowerSuite, which includes WebSite Auditor, LinkAssistant, and Rank Tracker as well.
SWEC is a program that automates testing of dynamic Web sites. It parses each HTML file it finds for links, and if those links are within the site specified, it will check that page as well. In addition to parsing and locating links, it will also parse the pages looking for known errors and report those. It will report if a page cannot be read (by either returning a 404, 500, or similar).
Active Business Intelligence Portal is software that provides an integrated solution for designing and deploying reports and Web forms over the Internet and intranets. Active BI Portal consists of two parts: Active BI Portal Manager, which only works on Windows, and the server part, which is a collection of PHP scripts. Active BI Portal works with all major databases. It also features an integrated HTML editor, an integrated SQL builder, comprehensive security management, user-definable menus, and more.
smupcheck, which stands for Smart Update Checker, checks Web sites for updates automatically, even if they don't offer an RSS feed. It is a very basic tool, and does not offer advanced features such as checking password-protected Web sites, highlighting changes, or filtering results.
urlwatch is a script intended to help you watch URLs and get notified (via email) of any changes. The change notification will include the URL that has changed and a unified diff of what has changed. The script works out of a single directory, so there is no need to install anything. State files are kept in the same folder. The script supports stripping parts of a page that are always changing through the use of a filter hook function. It is typically run as a cronjob.
PottyMouth transforms completely unstructured and untrusted text to valid, nice-looking, completely safe XHTML. PottyMouth is designed to handle input text from non-technical, potentially careless, or malicious users. It produces HTML that is completely safe, programmatically and visually, to include on any Web page. You don't need to make your users read any instructions before they start typing. They don't even need to know that PottyMouth is being used.
Site Checker can be used to find broken links in Web pages. First, it retrieves the list of all pages of the Web site, either static pages or dynamic pages generated from data stored in a database. Then it checks the links in each of the pages to verify if they are on the list. Links to external sites can also be verified by performing HTTP requests to the remote Web servers to check whether the pages still exist.
Websitary is a script that monitors Web pages, RSS feeds, and podcasts and reports what's new. For many tasks, it reuses other programs (such as w3m, diff, and webdiff) to do the actual work. By default, it works on an ASCII basis, i.e. with the output of text-based Web browsers. With the help of some friends, it can also work with HTML.