[Bro] Bro + Log rotation (solr ?)

Joe Blow blackhole.em at gmail.com
Mon Sep 15 15:38:59 PDT 2014


Hey all,

I'm using Bro + rsyslog filereader in order to pump Bro into our big data
solution (Apache SOLR).  I'm using custom python scripts to parse the
incoming bro messages, batch them into appropriate sizes, and then POST
them to the SOLR cluster we have setup.  The main problem i'm running into
is that rsyslog does not seem to 'follow' the files once they have gone
through a Bro logrotate.  Is there a way to completely disable logrotate?
Has anyone had any luck with the Bro logrotate and not 'losing' file
handles?

I'd love some help in this matter.  Also - i know that Bro supports elastic
search POSTing (via libcurl).  Is there any reason why an apache SOLR
module can't be written/adapted?  I don't see a need to write to a file and
worry about file handles, when it's almost exactly the same to POST to SOLR
as it is to ES.  Since it's all libcurl (and JSON) under the hood, i'd be
glad to post/share the SOLR schemas i've created for the Bro data.

Thank in advance.

Cheers,

JB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140915/5cbe9c92/attachment.html 


More information about the Bro mailing list