urlmon

urlmon makes a connection to a web site and records the last_modified time for that url. Upon subsequent calls, it will check the url again, this time comparing the information to the previously recorded times. Since the last_modified data is not required to be given by HTTP (it's optional) and is non-existent for ftp, urlmon will then take an MD5 checksum. Filtering is possible, so that URLs whose content is always changing (due to server-side parsing or some equivalent, as often used in rotating adverstisements) can accurately be monitored.