[Bro] Bro 2.5 and log rotation

[James Lay]

Thanks Seth,
Interestingly, this is on my home network...the largest compressed file
in looking at past logs was tcprecovery at 7.8 megs.  On a hunch, after
this issue came up again on Christmas day, I disabled TCPRS and have
had no issues since.
James
On Tue, 2017-01-03 at 22:00 -0500, Seth Hall wrote:
> I've seen this before when people are generating really huge logs and
> IO on their system goes crazy because the previous logs are still
> being compressed which runs into a downward spiral that it never
> recovers from.  For those logs that you have which haven't been
> rotated as you expected, was there a gzip process running in the
> background?  I suspect that you have a lot of gzip processes running
> and a very high system load.
> 
>   .Seth
> 
> 
> > 
> > On Dec 22, 2016, at 8:49 AM, James Lay <jlay at slave-tothe-box.net>
> > wrote:
> > 
> > I guess I'm in this boat as well.  Since my upgrade, bro will stop
> > rotating logs at some point.  I'm not running bro via
> > broctl.  Here's my process for log rotation:
> > 
> > local.bro:
> >         redef Log::default_rotation_interval = 86400 secs;
> >         redef Log::default_rotation_postprocessor_cmd = "archive-
> > log";
> > 
> > broctl.cfg:
> >         LogRotationInterval = 86400
> > 
> > sudo /usr/local/bro/bin/broctl install
> > 
> > sudo ln -s /usr/local/bro/share/broctl/scripts/archive-log
> > /usr/local/bin/
> > sudo ln -s /usr/local/bro/share/broctl/scripts/broctl-config.sh
> > /usr/local/bin/
> > sudo ln -s /usr/local/bro/share/broctl/scripts/make-archive-name
> > /usr/local/bin/
> > sudo ln -s /usr/local/bro/share/broctl/scripts/expire-logs
> > /usr/local/bin/
> > sudo ln -s /usr/local/bro/share/broctl/scripts/delete-log
> > /usr/local/bin/
> > sudo ln -s /usr/local/bro/share/broctl/scripts/cflow-stats
> > /usr/local/bin/
> > sudo ln -s /usr/local/bro/share/broctl/scripts/stats-to-csv
> > /usr/local/bin/
> > 
> > This will work for a while.  But at some point it stops:
> > <Screenshot from 2016-12-22 05-58-45.png>
> > 
> > at the core I believe it's because bro, after sometime, won't
> > respond to a "normal" kill command.  A "sudo killall bro" will do
> > nothing.  Usually I'll "sudo killall bro", wait a minute, and then
> > my spool directory will be empty, I'll have an email with stats,
> > and I'll have my new archive directory.  I'll have to -9 it in
> > order to get it to stop,  I've restarted this morning and will see
> > how many days it will go.  Thank you.
> > 
> > James
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170104/aee9af72/attachment.html