[Bro] Bro + Log rotation (solr ?)

Kurt Grutzmacher grutz at jingojango.net
Mon Sep 15 16:40:35 PDT 2014


Hey Joe,

It is possible with the current setup to write your own logging utility to
pipe events directly to your system of choice. Since SOLR is REST-based
just copy over the ElasticSearch module and do some code tweaking.

Be aware that the devs are working on a new modular method for extending
Bro that will include logging. Should hopefully be a less-painful migration.


--
 Kurt Grutzmacher -=- grutz at jingojango.net

On Mon, Sep 15, 2014 at 3:38 PM, Joe Blow <blackhole.em at gmail.com> wrote:

> Hey all,
>
> I'm using Bro + rsyslog filereader in order to pump Bro into our big data
> solution (Apache SOLR).  I'm using custom python scripts to parse the
> incoming bro messages, batch them into appropriate sizes, and then POST
> them to the SOLR cluster we have setup.  The main problem i'm running into
> is that rsyslog does not seem to 'follow' the files once they have gone
> through a Bro logrotate.  Is there a way to completely disable logrotate?
> Has anyone had any luck with the Bro logrotate and not 'losing' file
> handles?
>
> I'd love some help in this matter.  Also - i know that Bro supports
> elastic search POSTing (via libcurl).  Is there any reason why an apache
> SOLR module can't be written/adapted?  I don't see a need to write to a
> file and worry about file handles, when it's almost exactly the same to
> POST to SOLR as it is to ES.  Since it's all libcurl (and JSON) under the
> hood, i'd be glad to post/share the SOLR schemas i've created for the Bro
> data.
>
> Thank in advance.
>
> Cheers,
>
> JB
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140915/adce2f9b/attachment.html 


More information about the Bro mailing list