[Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion

Tritium Cat tritium.cat at gmail.com
Tue Jul 31 16:21:10 PDT 2012


I've came to the conclusion that Bro cannot currently scale up to 100
workers for a single manager, threaded or not.

For now I've reverted back to the 2.0-stable build and have segmented the
cluster into five parts, one per server with 20 workers on each.  That has
been stable, minus the occasional process stuck at 100% CPU.  I'm going to
try switching back to the dev track again with this segmented approach and
see how that works.

--TC


On Tue, Jul 31, 2012 at 4:59 PM, Vlad Grigorescu <vladg at cmu.edu> wrote:

> I've been running 2.0-905 for ~25-26 hours. The manager's memory usage has
> slowly crept up to 13 GB.
>
> One thing of note - I'm using the ElasticSearch log writer. I see 3
> possible scenarios for this memleak:
>
> 1) There is indeed a leak in master, potentially only triggered by
> specific traffic,
> 2) There is a leak in the ElasticSearch log writer,
> 3) My ElasticSearch server can't keep up with the load, and the manager is
> receiving logs faster than it can send them to the writer, so they just
> queue up.
>
> Has anyone else tried the current code over an extended period on live
> traffic? Also, if anyone has any ideas to try to figure out where this
> leak is occurring, please let me know. I'm going to switch back to ASCII
> logs for a bit, and see what that's like.
>
>   --Vlad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120731/c1c9e879/attachment.html 


More information about the bro-dev mailing list