curl and libcurl is a tool for transferring files using URL syntax. It supports HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP, POP3, IMAP, SMTP, RTSP, RTMP, and FILE, as well as HTTP-post, HTTP-put, cookies, FTP upload, resumed transfers, passwords, port numbers, SSL certificates, Kerberos, and proxies. It is powered by libcurl, the client-side URL transfer library. There are bindings to libcurl for about 40 languages and environments.
DansGuardian is a Web content filtering proxy that uses Squid to do all the fetching. It filters using multiple methods including, but not limited to, phrase matching, file extension matching, MIME type matching, PICS filtering, and URL/domain blocking. It has the ability to switch off filtering by certain criteria including username, domain name, source IP, etc. The configurable logging produces a log in an easy to read format. It has the option to only log text-based pages, thus significantly reducing redundant information (such as every image on a page).
DeleGate is a multi-purpose application level gateway or proxy server that mediates communication of various protocols, applying cache and conversion for mediated data, controlling access from clients, and routing toward servers. It translates protocols between clients and servers, converting between IPv4 and IPv6, applying SSL (TLS) to arbitrary protocols, merging several servers into a single server view with aliasing and filtering. It can be used as a simple origin server for some protocols (HTTP, FTP, and NNTP).
Gyach is a GTK+-based Yahoo! Chat client which can use your existing Yahoo! logins for chatting in Yahoo! Chat. It supports colors, emotes/aliases, PMs, set status, remote user status display, ignore lists (local, non-Yahoo! based), a regex list of triggers to cause users to be automatically ignored, command-line recall via arrow keys, and tab completion of usernames.
Harvest is a system to collect information and make it searchable using a Web interface. It can collect information using HTTP, FTP, NNTP, and local files. Supported formats include HTML, DVI, PS, fulltext, mail, man pages, news, troff, WordPerfect, C sources, and many more. Adding support for new formats is easy due to Harvest's modular design.
HTMLDOC converts HTML files and Web pages into indexed HTML, PostScript, and PDF files suitable for online viewing and printing. It can be used as a standalone GUI application, in a batch document processing environment, as a Web-based report generation application, or in embedded environments to support printing of HTML content. It runs on all Unix platforms as well as Mac OS X and Windows 2000 and higher.
httpMonitor is a Python program which lets you monitor the HTTP traffic between a browser and a Web server. You configure your browser to use httpMonitor as its proxy to ensure that all HTTP traffic between browser and server goes via httpMonitor. In a XML-configuration file you can specify a Python-function as processor for the HTTP-messages you are interested in (requests or responses, which match certain criteria). You may have as many processors as you want, without modifying the actual program. The only thing you need to do is configure the httpMonitor and write your processors.
HTTrack is an easy-to-use offline browser utility. It allows you to download a Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer. HTTrack arranges the original site's relative link-structure. Simply open a page of the mirrored Web site in your browser, and you can browse the site from link to link, as if you were viewing it online. HTTrack can also update an existing mirrored site, and resume interrupted downloads. WebHTTrack is a Web-based GUI for HTTrack.
jftpgw is a proxy server for the FTP protocol. It is highly configurable, allowing you to use an XML-like configuration file to define almost any combination of the client's IP, the server's IP, the user name, the destination port, and so on. jftpgw supports inetd/daemon mode, transparent proxying, forwards, caching, logging, data IP and port specification (port ranges), limiting of the number of simultaeous sessions, and several proxy login styles for clients that support FTP proxies.