This is a tool to collect information from web servers and to spider the web sites. This was written for the Open Source Security Testing Methodology (OSSTM) located on http://www.ideahamster.org/osstmm- description.htm. The spider is a multi-threaded resusable module that can be used in other projects.
pkt is a universal XML packet archiver. It can manipulate files for backing up documents, transferring them across networks, emailing them, and sharing them by other means (p2p etc). It uses plugin filters to do the job and handles virtually any format available if there is a plugin for it. It handles encryption, compression, conversion, parity checking, decryption, decoding, repairing, rebuilding, and other tasks using a universal file format (XML-based). By default, all pkt output is in 7-bit ASCII which is UTF encoded and is compatible with XML parsers. pkt files can be disassembled, reassembled, and repaired by hand if necessary. pkt also supports the use of 8-bit compressed XML packets for its archives using one of several compression algorithms. In general, pkt will try to use the best algorithms available.