[Bro] Version: 2.0-907 -- Bro manager memory exhaustion

[Tritium Cat]

On Thu, Aug 2, 2012 at 3:54 PM, Robin Sommer <robin at icir.org> wrote:

>
> On Wed, Aug 01, 2012 at 23:48 +0000, you wrote:
>
> > The process is constantly in a uwait status which I believe indicates a
> > problem with threading.
>
> How does it look when you enable displaying individual threads in top?
> Some threads will be pretty much idle most of them time, while others
> should be quite busy. If it really seems to hang somewhere, attaching
> a gdb should show where exactly the threads stall (in particular the
> main one). But seeing the high aggregate CPU load, I'm not sure that's
> what's happening.
>
>
Considering 2.1-beta...

I discovered that last night; displaying by threads showed exactly what
you've described.

I'm not an expert with threads but I do know they are challenging to
program, not to mention debug.  I'm assuming I/O is not the problem here as
the disk latency is 0.2ms at max.

When running 5 clusters instead of one I still have the problem with the
manager consuming all memory on each server, it just takes longer for it to
happen.  The last configuration I tried was 16 workers on each cluster.  I
wasn't able to tell via top and GDB what was going on, so I guess it's time
to litter the code with debugging blocks and figure it out.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120802/ddeb5bd2/attachment.html