nxlog is a modular, multi-threaded, high-performance log management solution with multi-platform support. In concept, it is similar to syslog-ng or rsyslog, but is not limited to Unix/syslog only. It can collect logs from files in various formats, receive logs from the network remotely over UDP, TCP, or TLS/SSL on all supported platforms. It supports platform-specific sources such as the Windows Eventlog, Linux kernel logs, Android device logs, local syslog, etc. Writing and reading logs to/from databases is also supported for many database servers. The collected logs can be stored into files, databases, or forwarded to a remote log server using various protocols. The old BSD Syslog and the newer IETF syslog standard (RFC 3164 and RFC 5424-5426) are fully supported by nxlog in addition to XML, JSON, CSV, GELF, and other custom formats. A key concept in nxlog is to be able to handle and preserve structured logs so there is no need to convert everything to syslog and then parse these logs again at the other side. It has powerful message filtering, log rewrite, and conversion capabilities. Using a lightweight, modular, and multi-threaded architecture which can scale, nxlog can process hundreds of thousands of events per second.
The Logfile Navigator, lnav for short, is a curses-based tool for viewing and analyzing log files. The value added by lnav over text viewers or editors is that it takes advantage of any semantic information that can be gleaned from the log file, such as timestamps and log levels. Using this extra semantic information, lnav can do things like interleaving messages from different files, generate histograms of messages over time, and provide hotkeys for navigating through the file. These features are meant to allow the user to quickly and efficiently focus on problems.
superseriousstats is a small and efficient program for creating a Web page with statistics from various types of IRC logs. It keeps track of its parse history and only processes new activity before storing any accumulated data in a SQLite or MySQL database. It is suitable for high volume IRC channels and large log archives, and is relatively easy to integrate with IRC services (e.g. bots) that interact with the database and provide last seen information and many other statistics directly in your channel.
Fido is a multi-threaded file watcher which searches files in real time for user-defined patterns. When it locates a match, it runs a user-defined program. It is useful for monitoring log files for issues and responding to them. It was designed to recognize log file rotation and start monitoring from the beginning of the new file.
timeplotters is a collection of command line tools for visualizing temporal data. It is especially useful for visualizing data from ad-hoc program logs, helping you to spot patterns and anomalies that you would not otherwise see by just watching how the program works or by looking at the logs with the naked eye. Its input format is tailored to event types typically seen in program logs, and the visualization methods are tailored to the questions typically asked about program performance (e.g. distribution of activity durations).
Malheur is a tool for the automatic analysis of malware behavior (program behavior recorded from malicious software in a sandbox environment). It is designed to support the regular analysis of malicious software and the development of detection and defense measures. It allows for identifying novel classes of malware with similar behavior and assigning unknown malware to discovered classes. It can be applied to recorded program behavior of various formats as long as monitored events are separated by delimiter symbols, e.g. as in reports generated by the popular malware sandboxes CWSandbox, Anubis, Norman Sandbox, and Joebox.