Release Notes: The exploration will now stop after or before loading an URL only - touch to force loading even if the content of the URL is in the database. Fixed index updating bug that removed documents from the index when they are found Not Modified by the crawler. Upgraded md5 code to GPL, and made other small utilities fixes.
Release Notes: -version now shows the version number. An allocation error when updating the full text index and a name server timeout condition handling optimization have been fixed. /etc/my.cnf, ~/.my.cnf, and datadir/my.cnf are now used instead of ~/.my.cnf alone.
Release Notes: Implementation of dynamic updating of the fulltext index, and fixes for a last modified time update bug, a mysql-3.23.19a-gamma namespace conflict, and a bug that left the start point in virgin state artificially.
Release Notes: A -crawlers option to run simultaneous crawlers and a signal handling function for graceful interuption of the crawlers, and the ability for url, url_complete, and url_content tables to grow over 4GB. The hook library is dynamically loadable with the -hook option so that specific full indexing strategies can be implemented as plugins. The -where_url option is taken in account when rebuilding the full text index with -rebuild. Extensions and MIME types have been added to the list of known MIME types. The auth field of the start table was removed because it was not used.
Release Notes: The crawler manual page was completely reviewed for correctness. Bug fixes were made in the mifluz interface. The -agent option was implemented. The -show option family was added to display all URL information from an exploration starting point. The configuration script was improved. Major leaks and concurency problems were fixed in the langrec interface. The scope of the allow/disallow comparison was widened to include CGI parameters. Code to use .my.cnf files (if any) was restored.
Release Notes: Language recognition for French, English, Italian, Spanish, and German was added. URL filtering with regexp is now possible. The documentation was improved.
Release Notes: This release is compatible with the mifluz-0.15 inverted index. Comments have been added to the code and a few bugs have been fixed.
Release Notes: This release features an upgrade to mifluz-0.13.0 and checks the syntax of URLs given as arguments.
Release Notes: Partial conversion to C++, and handling of options for mifluz (use mifluz-0.12 with dynamic key description).
Release Notes: Changing to C++, fixes for memory leaks, integration with mifluz, better detection of zlib and apache, and indexing of tests.