perl-esmtpd is an antispam, antivirus, and authenticating SMTP daemon. It integrates with qmail, sendmail, stunnel, etc. It's simple Perl, so you can hack it to suit your needs. It supports RBLs, mx on from, bounce evil phrases, virus scanning, etc. It offers: RBL-sytle black-hole DNS checking when not relaying, verification of valid MX records for the MAIL FROM (when not relayin, the ability to use Stunnel for smtps connections and still do RBL checks etc., simple authentication (i.e. PLAIN or LOGIN) for offsite users, and the ability to reject email with certain key phrases. Many other useful features are included.
The Anti-Spam SMTP Proxy (ASSP) Server project aims to create an open source platform independent SMTP Proxy server which implements whitelists, Bayesian, and virus filtering to help stop unsolicited commercial email (UCE). Anti-spam tools should be adaptive to new spam and customized for each site’s email patterns. This easy to use tool works with any mail transport and achieves these goals requiring no operator intervention after the initial setup phase.