[Bro] Feature Request, up to 50% done?

Jesse Bowling jessebowling at gmail.com
Mon Feb 11 13:17:07 PST 2013


Hi,

When I need to search my bro log files, I find myself using 'zfgrep' quite
a bit...Out of the box this works fine, however I lose the ability to
translate epoch time to 'human readable' time. The examples I've seen
suggest "zcat $file | bro-cut -d" however this means that every line must
be run through awk, which greatly increases search times. for an example
search of a one hour http file, my 'bcut' method runs in 6.9 seconds, while
using bro-cut properly(? "zcat file | bro-cut -d | fgrep string") takes 5
minutes, 1.2 seconds. Doing my zfgrep before the bro-cut causes only blank
lines to be printed (I assume because there are no description fields to be
read).

The workaround I've been implementing is to:

# alias bcut
alias bcut='awk -f /usr/local/bin/epoch_to_human.gawk'

# cat /usr/local/bin/epoch_to_human.gawk
#!/bin/gawk
{
val=strftime("%Y-%m-%dT%H:%M:%S%z", $1, 0)
$1=val
print $0
}

# zfgrep '10.10.10.10' /usr/local/bro/logs/some_log.txt.gz | bcut

This works well enough. Tt would be nice if there were a switch to bro-cut
that would implement this functionality, however I'm unsure of how to
integrate it myself (most of my awk programs are one line throwaways). The
key would be make it clear that you cannot specify field selections with
bro-cut, that this would only attempt to translate the first field into a
"human readable" format. Some error checking is likely in order as well...

So, I suppose I'm requesting that someone with more gawk chops than myself
give a shot at integrating this into bro-cut, or give me some pointers and
I might be able to stumble through creating a patch for submission...Or,
perhaps I'm approaching this problem in the wrong way, and could use a
pointer on a better way to go about it (aside from "ship the logs elsewhere
that indexes searches"; we'll deal with that scenario later) :)

Cheers,

Jesse

-- 
Jesse Bowling
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130211/a5280d95/attachment.html 


More information about the Bro mailing list