366 projects tagged "Log Analysis"
nxlog is a modular, multi-threaded, high-performance log management solution with multi-platform support. In concept, it is similar to syslog-ng or rsyslog, but is not limited to Unix/syslog only. It can collect logs from files in various formats, receive logs from the network remotely over UDP, TCP, or TLS/SSL on all supported platforms. It supports platform-specific sources such as the Windows Eventlog, Linux kernel logs, Android device logs, local syslog, etc. Writing and reading logs to/from databases is also supported for many database servers. The collected logs can be stored into files, databases, or forwarded to a remote log server using various protocols. The old BSD Syslog and the newer IETF syslog standard (RFC 3164 and RFC 5424-5426) are fully supported by nxlog in addition to XML, JSON, CSV, GELF, and other custom formats. A key concept in nxlog is to be able to handle and preserve structured logs so there is no need to convert everything to syslog and then parse these logs again at the other side. It has powerful message filtering, log rewrite, and conversion capabilities. Using a lightweight, modular, and multi-threaded architecture which can scale, nxlog can process hundreds of thousands of events per second.
The Logfile Navigator, lnav for short, is a curses-based tool for viewing and analyzing log files. The value added by lnav over text viewers or editors is that it takes advantage of any semantic information that can be gleaned from the log file, such as timestamps and log levels. Using this extra semantic information, lnav can do things like interleaving messages from different files, generate histograms of messages over time, and provide hotkeys for navigating through the file. These features are meant to allow the user to quickly and efficiently focus on problems.
360-FAAR (Firewall Analysis Audit and Repair) is an offline, command line, Perl firewall policy manipulation tool to filter, compare to logs, merge, translate, and output firewall commands for new policies, in Checkpoint dbedit, Cisco ASA, or ScreenOS commands. It is all contained in one file. It can read policy and logs for: Checkpoint FW1 (in odumper.csv / logexport format), Netscreen ScreenOS (in get config / syslog format), and Cisco ASA (show run / syslog format). It uses both inclusive and exclusive CIDR and text filters, permitting you to split large policies into smaller ones for virutalization at the same time as removing unused connectivity. It supports policy to log association, object translation, rulebase reordering and simplification, rule moves, and duplicate matching automatically. It allows you to seamlessly move rules to where you need them. 'print' mode creates a spreadsheet for your audit needs with one command.
superseriousstats is a small and efficient program for creating a Web page with statistics from various types of IRC logs. It keeps track of its parse history and only processes new activity before storing any accumulated data in a SQLite or MySQL database. It is suitable for high volume IRC channels and large log archives, and is relatively easy to integrate with IRC services (e.g. bots) that interact with the database and provide last seen information and many other statistics directly in your channel.
Fido is a multi-threaded file watcher which searches files in real time for user-defined patterns. When it locates a match, it runs a user-defined program. It is useful for monitoring log files for issues and responding to them. It was designed to recognize log file rotation and start monitoring from the beginning of the new file.
pmacct is a small set of passive network monitoring tools to account, filter, classify, aggregate, and export IPv4 and IPv6 traffic. A pluggable and flexible architecture allows storing collected network data in memory tables, RDBMSs (MySQL, SQLite, PostgreSQL, BDB), and flat files, and also export via IPFIX, NetFlow, or sFlow protocols to remote collectors. pmacct features fully customizable historical data breakdown, sampling, BGP correlation, tagging, and triggers. Libpcap, ULOG, sFlow v2/v4/v5, NetFlow v1/v5/v7/v8/v9, and IPFIX are supported data capturing methods.
Squid Analyzer parses the native access log format of the Squid proxy and reports general statistics about hits, bytes, users, networks, top URLs, and top second level domains. Statistic reports are oriented toward user and bandwidth control; this is not a pure cache statistics generator.
A Perl base class that allows you attach event callbacks to an object.