[Bro] Worker System Memory Exhaustion
Greg Grasmehr
greg.grasmehr at caltech.edu
Fri Apr 6 11:03:04 PDT 2018
Thanks Carl, will consider this avenue too.
Greg
On 04/06/18 17:02:02, Pearson, Carl (cpearson at uidaho.edu) wrote:
> I endorse commenting out "@load misc/scan", that really stabilized memory usage in our environment. We also commented out "@load misc/detect-traceroute" and that seemed to help as well.
>
> From your description I don't know if this applies to your situation, but for what it's worth we use PF_RING and experienced high resource utilization initially due to PF_RING not loading correctly. The issue tracker for it is here: https://bro-tracker.atlassian.net/browse/BIT-1864 - TL;DR, change pfringclusterid in broctl.cfg to 21; if pfringclusterid is set to 0 PF_RING doesn't actually do anything even though everything will indicate it's running. Again, don't know if it applies but thought I'd throw it out there. :)
>
> Thanks,
> Carl
> University of Idaho
>
> -----Original Message-----
> From: bro-bounces at bro.org <bro-bounces at bro.org> On Behalf Of Azoff, Justin S
> Sent: Friday, April 6, 2018 05:25
> To: Greg Grasmehr <greg.grasmehr at caltech.edu>
> Cc: bro at bro.org
> Subject: Re: [Bro] Worker System Memory Exhaustion
>
>
> > On Apr 5, 2018, at 7:36 PM, Greg Grasmehr <greg.grasmehr at caltech.edu> wrote:
> >
> > Hello List,
> >
> > We're running Bro version 2.5-467 and shunting connections in an
> > Arista
> > 5100 via the excellent Dumbno. The system is essentially a
> > Bro-in-a-box with 251G RAM and a Myricom SNF+ card. Bro spawns 32
> > workers to monitor a 10G link that averages roughly 5G and spikes to 7
> > on occasion. Logs are rotated every 1/2 hour.
>
> Cool :-)
>
> > The current configuration has been running for about 2 weeks and thus
> > far the only problem I have encountered is that given enough time, the
> > workers will eventually exhaust all of the available system memory
> > including scratch.
>
> Less cool :-(
>
> > Currently I am running a watcher process which slowly restarts the
> > workers once available memory is < 4G and this has solved the problem,
> > but this seems an imperfect solution thus I am wondering if anyone
> > knows the source of the apparent memory leak? Loaded scripts are
> > attached and thanks in advance for any informative illumination on this issue.
>
> try commenting out
>
> @load misc/scan
>
> from local.bro.
>
> If you have a lot of address space and bro is before any firewall as opposed to after it, this is likely the source of the problems.
>
> If that fixes it there are other scan detection implementations that are a bit more efficient.
>
> —
> Justin Azoff
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
More information about the Bro
mailing list