Projects / mergelog

mergelog

mergelog is a small and fast C program, which merges HTTP log files by date in 'Common Log Format' (Apache default log format) from Web servers, behind round-robin DNS. It has been designed to easily process huge logs from highly stressed servers, and can manage gzipped files.

Tags
Licenses
Operating Systems
Implementation

Recent releases

  •  11 Apr 2001 13:44

    Release Notes: The corrupted log lines problem which stopped the mergelog process has been solved.

    •  21 Mar 2001 17:35

      Release Notes: A major bug which could break the date sorting of the first log line entry of each log file has been fixed.

      •  05 Mar 2001 11:45

        Release Notes: This release fixes a potential segmentation fault with bad formatted log lines, and packaging has been slightly improved.

        •  31 Jan 2001 09:34

          Release Notes: The project has switched to switched to the autoconf standard.

          •  07 Jan 2001 18:28

            Release Notes: Fixes for all known bugs, and optimization of the new date comparison system introduced in 4.0.

            Recent comments

            13 Jan 2006 19:30 roger_byucougar

            Re: no dots in ip


            > Works well -- seems to have a small bug

            > if the hostname/ipaddress does not

            > contain a period [calls it an error but

            > still outputs it, so not a big

            > problem].

            Seems that the problem is not that hostname's lack a period, but rather if they are of size 3 or less then mergelog mis-parses them. To fix change SCAN_OFFSET to 7.

            05 Jan 2006 13:04 roger_byucougar

            no dots in ip
            Works well -- seems to have a small bug if the hostname/ipaddress does not contain a period [calls it an error but still outputs it, so not a big problem].

            Thanks.

            -Roger

            03 Aug 2004 16:45 smdas

            Very handy!
            mergelog should come with apache. The compiled binary is under 40k (I know, I know, platform, toolchain, etc... but still!).

            How about a mergelog for error_log's? These are trickier because many parsers don't output in apache error log format, but chunks of contiguous aberrant errors could be treated as a single line, and it's date/time could be flagged "after" the first properly formatted log line above it, but before any possible mergeable line. Perhaps an option to strip any malformed (consider corrupted?) lines.

            29 Jun 2002 08:56 cybert

            Re: Damn fast

            > Ok, I'm new to this whole thing. I
            > downloaded it and compiled it. I have 3
            > log files on 3 different servers. nce I
            > pull them down to a local machine, I
            > want to merge them. How do I do this
            > using mergelog?


            You can try the following command:
            mergelog logfile1 logfile2 logfile2 > merge

            You can also try this:
            man mergelog

            ;-)

            26 Jun 2002 13:45 jrthor2

            Re: Damn fast
            Ok, I'm new to this whole thing. I downloaded it and compiled it. I have 3 log files on 3 different servers. nce I pull them down to a local machine, I want to merge them. How do I do this using mergelog?


            > Ok. I`m perl zealot, but this time i
            > should admit C speed against perl
            > analogue merge_apache_logs.
            > It saved me. I`ve successfully merged 3
            > log files with total of 1.5GB like in 2
            > minutes. Webalizer parsed output without
            > problems.
            > Its simple, fast and it does what it
            > should.


            Screenshot

            Project Spotlight

            OpenStack4j

            A Fluent OpenStack client API for Java.

            Screenshot

            Project Spotlight

            TurnKey TWiki Appliance

            A TWiki appliance that is easy to use and lightweight.