[Bro-Dev] Python script/module to parse new log files

Gilbert Clark gc355804 at ohio.edu
Thu Aug 11 23:05:12 PDT 2011


I've got some code for this, but it's a work in progress; you're welcome 
to take a look at what's there, though, and do whatever you'd like with it.

*** Note that this code demands a slightly modified ASCII header (which 
includes the bro path name and the separator char).

Anyway, branch is:

topic/gilbert/log-util

and check: aux/log-util

I think the usage is pretty straightforward:

manager = BroLogManager()
manager.load('/path/to/logs')
#If this is a directory, *all files from all subdirectories* of this 
path will be loaded.
# Otherwise, only the referenced files will be loaded.
# .log, .log.gz, and .log.bz2 are treated as new-style bro logs

print manager['conn'].get_stats('orig_bytes')
# This prints some interesting statistics about the orig_bytes column 
(min / max / mean / std_dev).
# Note that the above will actually load *all* relevant log files and 
perform the calculation, then cache
# the results; the first get_stats() will be very slow, but the rest 
should work pretty quickly.

for e in manager['conn'].entries():
     print e['ts']    # Note that 'e.ts' will also work, but that this 
doesn't work for field names that don't map nicely to Python.

See 'bro-logtool' for a toy script I've been using to play with the library.

--Gilbert

On 8/12/2011 1:11 AM, Gregor Maier wrote:
> Hi,
>
> has anybody already written a python script or module to easily parse
> the new Bro ASCII log files?
>
> cu
> Gregor



More information about the bro-dev mailing list