[Bro] Bro + Log rotation (solr ?)

[James Lay]

On 2014-09-16 07:58, Joe Blow wrote:
> Hey James,
>
> How exactly are you completely disabling the bro file rotation?  This
> is why i tried in broctl.conf:
>
> SitePolicyStandalone = local.bro
> CfgDir = /usr/local/bro/etc
> SpoolDir = /usr/local/bro/spool
> LogDir = /usr/local/bro/logs
> LogRotationInterval = 0
> MinDiskSpace = 5
>
> But i still see gz files being created.  Am i missing something to
> completely disable?
>
> Cheers,
>
> Justin
>
> On Mon, Sep 15, 2014 at 6:56 PM, James Lay <jlay at slave-tothe-box.net
> [3]> wrote:
>
>> On 2014-09-15 16:38, Joe Blow wrote:
>> > Hey all,
>> >
>> > Im using Bro + rsyslog filereader in order to pump Bro into our
>> big
>> > data solution (Apache SOLR).  Im using custom python scripts to
>> parse
>> > the incoming bro messages, batch them into appropriate sizes, and
>> > then
>> > POST them to the SOLR cluster we have setup.  The main problem
>> im
>> > running into is that rsyslog does not seem to follow the files
>> once
>> > they have gone through a Bro logrotate.  Is there a way to
>> completely
>> > disable logrotate?  Has anyone had any luck with the Bro
>> logrotate
>> > and not losing file handles?
>> >
>> > Id love some help in this matter.  Also - i know that Bro
>> supports
>> > elastic search POSTing (via libcurl).  Is there any reason why
>> an
>> > apache SOLR module cant be written/adapted?  I dont see a need
>> to
>> > write to a file and worry about file handles, when its almost
>> exactly
>> > the same to POST to SOLR as it is to ES.  Since its all libcurl
>> (and
>> > JSON) under the hood, id be glad to post/share the SOLR schemas
>> ive
>> > created for the Bro data.
>> >
>> > Thank in advance.
>> >
>> > Cheers,
>> >
>> > JB
>>
>> I experienced the same thing, but since I rotate the files
>> manually, I
>> restart the syslog service after rotating and thats done the trick
>> for
>> me.
>>
>> James

I don't use broctl, I use bro command line only.  Something like:

/usr/local/bro/bin/bro --no-checksums -i eth0 local "Site::local_nets 
+= { 192.168.1.0/24 }"

James